You gotta love Perft... just not too much!

MartinBryant · Post by **MartinBryant** » Sat Jul 27, 2019 10:23 am

Whilst re-writing Colossus to use bitboards I obviously used Perft to try to verify the correctness of the new move generators etc, mainly using the positions and results given here...
https://www.chessprogramming.org/Perft_Results

I have automated the testing now and I re-run them every so often just to get a warm, cosy glow

In fact, I was so warm and cosy that I would have bet a large sum of money that the move-generators were now bug free!

However, last night I stumbled across this remarkable mate in 89!
viewtopic.php?f=2&t=70695&p=798285&hili ... in#p798285

Just for fun I put it into Colossus... but was shocked to see it announce a mate in 8!

I stepped through it's PV to get to here...

[d]8/ppp3p1/8/8/3p4/5Q2/1ppp2K1/brk4n w - - 11 7

...which Colossus thought was a mate in 1 (Qf1+)! OOPS!!

So, it turns out that there WAS still a bug in one of my 'generate-moves-out-of-check' routines but it only occurred when a BLACK PROMOTING pawn (it would have worked for white in a mirrored position or shifted one rank north) was blocking the rayed check... boy oh boy... glad I didn't make any bets! Humbling to accept that searching hundreds of billions of positions doesn't necessarily drive your code through all paths!

Perhaps someone with editing permissions would like to add this to the wiki page if they consider it worthy of note?

Uri Blass · Post by **Uri Blass** » Sat Jul 27, 2019 10:51 am

I suggest that you add perft pgn command to verify your move generator.

The idea is that you take a big pgn and calculate the sum of perft(n) for all the positions in the pgn and find if the sum is correct(I am sure that part of the positions should have a promotion of a pawn in reply to check because it happens in chess games.

Movei support this command.
suppose that you have in the same folder the movei engine and also the pgn file that you want to calculate sum of perft(n) for all the positions in the pgn for some n that you choose n>=1
Suppose the name of the pgn is event.pgn

You only need to run movei in console mode and type
perftpgn event.pgn 2 and the program will give you sum of perft(2) for all positions in the pgn.

If you type
perfpgn event.pgn 3 then you will get the sum of perft(3)

Note that I stopped to work on movei many years ago and
I am not sure if movei has no bugs in calculating perft or in reading pgn.

elcabesa · Post by **elcabesa** » Sat Jul 27, 2019 11:10 am

if you like you can find a lot of perft position and result in Vajolet test directory:

https://github.com/elcabesa/vajolet/blo ... ischer.txt
https://github.com/elcabesa/vajolet/blo ... /perft.txt

each line of the file contains a position and the perft value from depth 1 to depth 6 separed by ","

the positions are randomly sample from engine games

MartinBryant · Post by **MartinBryant** » Sat Jul 27, 2019 11:27 am

@Uri
Oh I see... so instead of searching billions of positions from a single root position you search thousands of root positions to a much shallower depth and sum them... great idea. Of course you need at least one other engine that can do this to compare the results against

So thank you for the suggestion and for Movei!

Going off at a tangent, I have a text file containing many 'curious' positions that I've collected over the years... do you remember this one you posted years ago to highlight a bug in Glaurung?

[d]8/6kR/8/8/8/bq6/1rqqqqqq/K1nqnbrq b - - 0 1

Love these weird positions. I think it's amazing that the best engines today (well Stockfish at least) just immediately see this as a draw!

@elcabesa
Thank you so much for that resource! I'm feeling warmer and cosier already

mar · Post by **mar** » Sat Jul 27, 2019 1:03 pm

I believe in a small set of artificial positions that test all the gotchas with perft runs to very shallow depth.
It will run in a couple of milliseconds and the results will be way more valuable than running zillions of positions and waiting forever.
Especially useful when you need to re-test after bugfixes.
It has been discussed here in the past but you'd have to search the forum.
Jon has these in Arasan IIRC (among others).

Also don't forget that in FRC castling can be prevented due to backrank self-check.

Ajedrecista · Post by **Ajedrecista** » Sat Jul 27, 2019 2:52 pm

Hello Martin:

FWIW, here are JetChess perft counter results:

------------------------

First position:

[d]8/ppp3p1/8/8/3p4/5Q2/1ppp2K1/brk4n w - - 11 7