A method guaranteed to localize the toughest perft() bugs
Posted: Thu Sep 18, 2014 5:52 am
A method guaranteed to localize the toughest perft() bugs:
The following is an idea which I may implement in Oscar so that all those coders with tough perft() bugs might benefit.
With this idea you might need lots of CPU time and a boatload of disk space.
Oscar keeps track of the current variation during perft(). By setting options to disable transposition assistance and to also disable bulk counting, Oscar will visit each and every node in the perft() tree. It would be easy to add an option to the CPU version (not the OpenCL version) of Oscar to dump the current variation, one per line, to a file. When the calculation is complete, the result is a BIG file with every variation seen and none which aren't seen. (An assumption here is that Oscar's perft() is bug-free.)
With this file in hand, the coder with a buggy perft() adds similar variation tracking and dump code to their program, then runs the same perft() from the same position.
As it is unlikely that the two programs traverse the tree in the same order, the two files should first be sorted. Also, both programs must use the same move notation which for Oscar is SAN.
Now, the two files can be compared with the Unix diff utility, and within a few seconds any missing and any extraneous variations will appear. This will show precisely which moves and positions were mishandled.
The following is an idea which I may implement in Oscar so that all those coders with tough perft() bugs might benefit.
With this idea you might need lots of CPU time and a boatload of disk space.
Oscar keeps track of the current variation during perft(). By setting options to disable transposition assistance and to also disable bulk counting, Oscar will visit each and every node in the perft() tree. It would be easy to add an option to the CPU version (not the OpenCL version) of Oscar to dump the current variation, one per line, to a file. When the calculation is complete, the result is a BIG file with every variation seen and none which aren't seen. (An assumption here is that Oscar's perft() is bug-free.)
With this file in hand, the coder with a buggy perft() adds similar variation tracking and dump code to their program, then runs the same perft() from the same position.
As it is unlikely that the two programs traverse the tree in the same order, the two files should first be sorted. Also, both programs must use the same move notation which for Oscar is SAN.
Now, the two files can be compared with the Unix diff utility, and within a few seconds any missing and any extraneous variations will appear. This will show precisely which moves and positions were mishandled.