Get MoveGen errorfree

[Saturnus23]

How does one aproach the challenge of finding the bugs in movegen?
I find that my engine-to-be yields a perft 5 of 4,865,351 that should (appearantly) be 4,865,609.
So I am missing a worrysome 258 positions. Very open to suggestions
Or is it considered close enough? (cannot imagine that)

[kranium]

https://www.chessprogramming.org/Perft_Results
See the table entries for depth 5...
258 is the # of enpassant positions.
Check your enpassant detection routine, or add it if you don't have it.

Good luck

[Saturnus23]

Thanks. Very useful link!

[hgm]

To hunt down bugs in the move generator people use 'split perft', which does not just print the total value of perft(N), but also the perft(N-1) after each of the possible moves. Of course you should make your engine to that too, for comparison. Then you can immediately see which move is responsible for the wrong count, and then repeat the process (with a 1-lower perft) from the position after that. That quickly brings you to the (or a) position where it counts the wrong number of moves, and usually you can determine 'by hand' which move is missing (or extra) in that position.

You could use Qperft for generating a split perft from any position. Use the following command in a command-line window:

perft2 N -2 "FEN"

where N is the depth you want, and FEN the position. (Which is optional; if omitted it uses the FIDE start position.) The -2 parameter controls the depth at which it splits, and is also optional (for no split).

Note that the procedure does not always works: I have seen cases where the split perft indicated an error in a certain move. But perfting from the position after that move gave the correct total. The reason was that the engine did not pass e.p. rights correctly, but did read them correctly from a FEN. So the error occurred only in non-root positions.

I can add that perft never has been very useful to me; I almost never have a bug in my move generators. That is of course because I use mostly trivial move generators. And often they do not strictly follow FIDE rules either (e.g. no under-promotion, or just Q and N) so that the perft would not be correct by intention. And finally most of my engines play variants for which the correct perft numbers are not known to begin with. I have had plenty of move bugs in my engines, but not of the kind that would show up in perft. More things like losing moves during sorting, overwriting the killer slots too early, using hash moves without enough checking for key collisions... When I make a complex, error-prone move generator (like in The Mailbox Trials or Inferno) I always start writing a trivial one (triply-nested loop for cycling through board, directions and distances taken from a table), and add some code to compare the move list generated by the complex generator with the one generated by the trivial one, for use in the test phase.

[gflohr]

I've used Stockfish as a reference ("go perft N"). Don't look at the total number of nodes but at the number of nodes that Stockfish reports for each move. Say that you see that from the start position with perft 6, you have the wrong number of nodes for "e2e4". The next try is then:

Code: Select all

position startpos moves e2e4
go perft 5

That will give you the number of nodes for each move after "e2e4", and you can narrow down the error once again. At one point, you reach a position where Stockfish finds a different number of moves than your engine, and then you debug why. Of course, you don't need to start from the start position, but from wherever you want. If you want to check the correctness of your promotion code, then the start position of chess is probably not the optimal one.

The last bug that I had found and fixed was also related to en passant. I forgot that capturing en passant can discover a check, for example, in "8/8/8/K7/1R3p1k/8/6P1/8 w - - 0 1" after "g2g4".

[benvining]

I recommend perft tests and the test cases here: https://github.com/schnitzi/rampart

[Saturnus23]

Thanks all. Very helpful indeed!