My Perft Results

[Mike Sherwin]

I was curious enough yesterday to update my slider move/attack algorithm to use magic bitboards instead of just classical, but branchless bitboard methods. I found any performance improvement to be very small (on the order of 1%-2%). I suppose it helps to future enable my engine to run under future CPUs with larger caches.

Thank you for that report. I'm curious about tt size and how it affects the performance of magic vs classical.

[JoAnnP38]

Everything checks out to 7 ply!

Code: Select all

Running Perft and collecting details...
+-------+--------+--------------+------------+---------+---------+------------+------------+
| Depth |  Mnps  |     Nodes    |  Captures  |   E.p.  | Castles |   Checks   | Checkmates |
+-------+--------+--------------+------------+---------+---------+------------+------------+
|   1   |   0.02 |           20 |          0 |       0 |       0 |          0 |          0 |
+-------+--------+--------------+------------+---------+---------+------------+------------+
|   2   |   1.78 |          400 |          0 |       0 |       0 |          0 |          0 |
+-------+--------+--------------+------------+---------+---------+------------+------------+
|   3   |   2.35 |        8,902 |         34 |       0 |       0 |         12 |          0 |
+-------+--------+--------------+------------+---------+---------+------------+------------+
|   4   |   2.27 |      197,281 |      1,576 |       0 |       0 |        469 |          8 |
+-------+--------+--------------+------------+---------+---------+------------+------------+
|   5   |  11.11 |    4,865,609 |     82,719 |     258 |       0 |     27,351 |        347 |
+-------+--------+--------------+------------+---------+---------+------------+------------+
|   6   |  11.70 |  119,060,324 |  2,812,008 |   5,248 |       0 |    809,099 |     10,828 |
+-------+--------+--------------+------------+---------+---------+------------+------------+
|   7   |  11.43 |3,195,901,860 |108,329,926 | 319,617 | 883,453 | 33,103,848 |    435,767 |
+-------+--------+--------------+------------+---------+---------+------------+------------+

And of course when I'm not checking for check and checkmate at the leaf nodes things run a lot snappier:
Single threaded results:
1: Elapsed = 00:00:00.0011450, Mnps: 0.02, nodes = 20
2: Elapsed = 00:00:00.0003211, Mnps: 1.25, nodes = 400
3: Elapsed = 00:00:00.0027505, Mnps: 3.24, nodes = 8902
4: Elapsed = 00:00:00.0625392, Mnps: 3.15, nodes = 197281
5: Elapsed = 00:00:00.3371379, Mnps: 14.43, nodes = 4865609
6: Elapsed = 00:00:06.9856145, Mnps: 17.04, nodes = 119060324
7: Elapsed = 00:03:11.5586089, Mnps: 16.68, nodes = 3195901860

Perft has served its purpose for now. I'm sure I'll use it for regression testing later on, but it looks like my move generator is accurate. Hopefully, I don't end up with too many issues later on where the problem only occurs at 10+ ply down the search tree.

[JVMerlino]

Since I don't see it earlier in the thread, I will bring up that there are many "tricky" positions that can expose bugs in move generation and make/unmake. Here are the ones I use:

r3k2r/8/8/8/3pPp2/8/8/R3K1RR b KQkq e3 0 1 // perft(6) = 485,647,607
r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1 // perft(6) = 706,045,033
8/7p/p5pb/4k3/P1pPn3/8/P5PP/1rB2RK1 b - d3 0 28 // perft(6) = 38,633,283
8/3K4/2p5/p2b2r1/5k2/8/8/1q6 b - - 1 67 // perft(7) = 493,407,574
rnbqkb1r/ppppp1pp/7n/4Pp2/8/8/PPPP1PPP/RNBQKBNR w KQkq f6 0 3 // perft(6) = 244,063,299
r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - // perft(5) = 193,690,690
8/p7/8/1P6/K1k3p1/6P1/7P/8 w - - // perft(8) = 8,103,790
n1n5/PPPk4/8/8/8/8/4Kppp/5N1N b - - // perft(6) = 71,179,139
r3k2r/p6p/8/B7/1pp1p3/3b4/P6P/R3K2R w KQkq - // perft(6) = 77,054,993
8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - // perft(7) = 178,633,661
8/5p2/8/2k3P1/p3K3/8/1P6/8 b - - // perft(8) = 64,451,405
r3k2r/pb3p2/5npp/n2p4/1p1PPB2/6P1/P2N1PBP/R3K2R w KQkq - // perft(5) = 29,179,893

jm

[JoAnnP38]

Since I don't see it earlier in the thread, I will bring up that there are many "tricky" positions that can expose bugs in move generation and make/unmake. Here are the ones I use:

r3k2r/8/8/8/3pPp2/8/8/R3K1RR b KQkq e3 0 1 // perft(6) = 485,647,607
r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1 // perft(6) = 706,045,033
8/7p/p5pb/4k3/P1pPn3/8/P5PP/1rB2RK1 b - d3 0 28 // perft(6) = 38,633,283
8/3K4/2p5/p2b2r1/5k2/8/8/1q6 b - - 1 67 // perft(7) = 493,407,574
rnbqkb1r/ppppp1pp/7n/4Pp2/8/8/PPPP1PPP/RNBQKBNR w KQkq f6 0 3 // perft(6) = 244,063,299
r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - // perft(5) = 193,690,690
8/p7/8/1P6/K1k3p1/6P1/7P/8 w - - // perft(8) = 8,103,790
n1n5/PPPk4/8/8/8/8/4Kppp/5N1N b - - // perft(6) = 71,179,139
r3k2r/p6p/8/B7/1pp1p3/3b4/P6P/R3K2R w KQkq - // perft(6) = 77,054,993
8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - // perft(7) = 178,633,661
8/5p2/8/2k3P1/p3K3/8/1P6/8 b - - // perft(8) = 64,451,405
r3k2r/pb3p2/5npp/n2p4/1p1PPB2/6P1/P2N1PBP/R3K2R w KQkq - // perft(5) = 29,179,893

jm

THANK YOU! I added these cases to my unit tests, and it turns out I failed every case but one. Now I'll have to dig into it to see what is going on. Hopefully it is something trivial like how I'm parsing the fen position, but probably nothing so easy. Thanks again. This helps a lot!

[JVMerlino]

THANK YOU! I added these cases to my unit tests, and it turns out I failed every case but one. Now I'll have to dig into it to see what is going on. Hopefully it is something trivial like how I'm parsing the fen position, but probably nothing so easy. Thanks again. This helps a lot!

Usually incorrect perft results are not due to errors in FEN parsing, but more likely underpromotions, en passant, incorrect castling, etc. But hopefully you can resolve all of your issues quickly. If you haven't yet implemented a "divide" command, you will want to do so right away.

[JoAnnP38]

THANK YOU! I added these cases to my unit tests, and it turns out I failed every case but one. Now I'll have to dig into it to see what is going on. Hopefully it is something trivial like how I'm parsing the fen position, but probably nothing so easy. Thanks again. This helps a lot!

Usually incorrect perft results are not due to errors in FEN parsing, but more likely underpromotions, en passant, incorrect castling, etc. But hopefully you can resolve all of your issues quickly. If you haven't yet implemented a "divide" command, you will want to do so right away.

I'm slowly but surely working my way through this. However, wrt underpromotions... how are they counted? Are they not counted? Looks like all the problems I am encountering so far are associated with promotions.

[Mergi]

However, wrt underpromotions... how are they counted? Are they not counted?

Each type of promotion is it's own move and is counted separately.

[JVMerlino]

Usually incorrect perft results are not due to errors in FEN parsing, but more likely underpromotions, en passant, incorrect castling, etc. But hopefully you can resolve all of your issues quickly. If you haven't yet implemented a "divide" command, you will want to do so right away.

I'm slowly but surely working my way through this. However, wrt underpromotions... how are they counted? Are they not counted? Looks like all the problems I am encountering so far are associated with promotions.

All underpromotions are definitely counted because each one is a unique move. So, for this position:
[fen]4k3/1PP5/8/8/8/8/1pp1K3/8 w - - 0 1[/fen]

perft 1 = 15 (four different promotions for each of the two pawns, and seven moves for the king)
perft 2 = 142
perft 3 = 1,926
perft 4 = 24,912
perft 5 = 371,410
perft 6 = 5,791,523

[JoAnnP38]

All underpromotions are definitely counted because each one is a unique move. So, for this position:
[fen]4k3/1PP5/8/8/8/8/1pp1K3/8 w - - 0 1[/fen]

perft 1 = 15 (four different promotions for each of the two pawns, and seven moves for the king)
perft 2 = 142
perft 3 = 1,926
perft 4 = 24,912
perft 5 = 371,410
perft 6 = 5,791,523

Then I was at least counting the promotions correctly! LOL It turns out that I had two bugs in the move generation with regard to promotions. Once both were fixed everything is being counted correctly. Whew! I was worried it would be something dreadful to find. Luckily, I was able to track them down pretty quickly.

Pertaining to this "Divide" function are you talking about -- are you talk about a method to classify counts into different buckets (i.e. captures, castling, e.p., promotions, etc?) I just added that functionality yesterday, but I didn't really need it here since it was a move generation bug and not a counting bug.

[JVMerlino]

All underpromotions are definitely counted because each one is a unique move. So, for this position:
[fen]4k3/1PP5/8/8/8/8/1pp1K3/8 w - - 0 1[/fen]

perft 1 = 15 (four different promotions for each of the two pawns, and seven moves for the king)
perft 2 = 142
perft 3 = 1,926
perft 4 = 24,912
perft 5 = 371,410
perft 6 = 5,791,523

Then I was at least counting the promotions correctly! LOL It turns out that I had two bugs in the move generation with regard to promotions. Once both were fixed everything is being counted correctly. Whew! I was worried it would be something dreadful to find. Luckily, I was able to track them down pretty quickly.

Pertaining to this "Divide" function are you talking about -- are you talk about a method to classify counts into different buckets (i.e. captures, castling, e.p., promotions, etc?) I just added that functionality yesterday, but I didn't really need it here since it was a move generation bug and not a counting bug.

Great! Glad you got it fixed.
No, "divide" is an enhancement to perft that helps you find the moves that are causing problems when perft results are not correct. https://www.chessprogramming.org/Perft#Divide