Best way to debug perft?
Posted: Mon Jan 25, 2016 2:37 pm
An anecdote, and a question.
I had a bug in my move generating function. It was supposed to handle castling by examining a king move of two horizontal spaces (only if king never moved, not in check, and the square stepped over neither blocked nor threatened); then it was supposed to register it as a castle if the square is vacant, as are all the following squares up to a virgin rook.
But, due to incorrect branching logic, it skipped over the second half of the test if there is an opponent piece in the landing square. This allowed the king to capture two spaces horizontally in certain conditions.
This happens rarely, but it still created one extraneous line in perft(6) from starting position: 1. g3 d5 2. Bh3 Qd6 3. Bxc8 Kxc8
(I think it's interesting how only a single line creates this situation in depth 6. The black queen must move, but not block the white bishop; the black d pawn must move, but not block the queen; the white g pawn must move to release the bishop, but not block it later.)
Anyway, for my real question - the process from getting a wrong perft(6) result to finding the offending line was quite cumbersome for me. I used hgm's qperft with the command "perft 6 -2" to figure out the perft for each initial move, and compared it to my results (which required fiddling with a text editor and Excel to compare the numbers). Found that g3 has a discrepancy, and ran "perft 5 -2 FEN" with FEN being the code for the resulting board. Found the discrepancy and so on. By the time I got to searching for the bad 5th move, "perft 2 -2 FEN" no longer gave me a list of results per move, so I had to run perft on each possible move separately until I found the incorrect one.
So all in all doable, but I think it could have been much faster (and more fun) with a better tool. So, is there a better way to debug perft discrepancies?
I was thinking of a simple website or program with a graphical interface that allows you to either paste a FEN or choose from a few built-in test cases (like the ones on CPW). You also choose a depth and then it gives you a list of moves and their perft. You can also choose to have it sorted by perft for easy comparison. If you click on one of the moves (presumably, one that has a different perft from yours), you get the perft starting from it, and so on. This way you can quickly home in on the offending line(s).
Maybe it could also allow you to paste at every stage your own perft result, and it will automatically find the discrepancy.
Does such a tool exist? If not, is there demand for creating one?
I had a bug in my move generating function. It was supposed to handle castling by examining a king move of two horizontal spaces (only if king never moved, not in check, and the square stepped over neither blocked nor threatened); then it was supposed to register it as a castle if the square is vacant, as are all the following squares up to a virgin rook.
But, due to incorrect branching logic, it skipped over the second half of the test if there is an opponent piece in the landing square. This allowed the king to capture two spaces horizontally in certain conditions.
This happens rarely, but it still created one extraneous line in perft(6) from starting position: 1. g3 d5 2. Bh3 Qd6 3. Bxc8 Kxc8
(I think it's interesting how only a single line creates this situation in depth 6. The black queen must move, but not block the white bishop; the black d pawn must move, but not block the queen; the white g pawn must move to release the bishop, but not block it later.)
Anyway, for my real question - the process from getting a wrong perft(6) result to finding the offending line was quite cumbersome for me. I used hgm's qperft with the command "perft 6 -2" to figure out the perft for each initial move, and compared it to my results (which required fiddling with a text editor and Excel to compare the numbers). Found that g3 has a discrepancy, and ran "perft 5 -2 FEN" with FEN being the code for the resulting board. Found the discrepancy and so on. By the time I got to searching for the bad 5th move, "perft 2 -2 FEN" no longer gave me a list of results per move, so I had to run perft on each possible move separately until I found the incorrect one.
So all in all doable, but I think it could have been much faster (and more fun) with a better tool. So, is there a better way to debug perft discrepancies?
I was thinking of a simple website or program with a graphical interface that allows you to either paste a FEN or choose from a few built-in test cases (like the ones on CPW). You also choose a depth and then it gives you a list of moves and their perft. You can also choose to have it sorted by perft for easy comparison. If you click on one of the moves (presumably, one that has a different perft from yours), you get the perft starting from it, and so on. This way you can quickly home in on the offending line(s).
Maybe it could also allow you to paste at every stage your own perft result, and it will automatically find the discrepancy.
Does such a tool exist? If not, is there demand for creating one?