Comparison check on engine performance

[Chessnut1071]

Just finished my bitboard, first build, and I need to compare its performance. Below, is an 8-move checkmate puzzle with only 1 solution.
FEN[207] = "1rr3k1/2q4p/2p2p2/1bQpNp2/p2P4/P3P3/5P1P/R1R4K w - - 1 ";
My engine finished with the correct solution in 3 min: 58.04652 seconds w/ 31,200.301 legal engine calls, or, 131,093 calls/sec
Chip specs: 2.5GHz I7 w 16GB RAM 64-OS Win 10
Methods: alpha/beta, Zobirst Hash
Evaluation Priority:
1-double check 2: discovered check 3: single check
4: capture piece type 1-P 2-N 3-B 4-R 5-Q
5: History
6: Pawn Promotion
7: number of enemy king moves
8: ent passant
Data capture per move:
1 piece 2 move 3: capture 4: direction 5: check type 6: from 7: to 8: check direction from e king
9: skip bit 10:Pawn promotion 11: ent passant 12: 2-step pawn move

The last time I tried this, 2 years ago, it took my engine over 68 hours to solve this puzzle, using Visual Basic and integer arrays. The main difference, C#, bitboards and sorting-QuickSort- a single 64-bit field with the above priority packed into the upper 13 bits.
Thanks for all the suggestions; however, I need to get to 1,000,000 engine calls per second on this computer, if that’s possible. Any suggestions on where to go next? Magic bitboards? I really don’t understand that concept yet, but, is it that much faster than regular bitboards?

Solution: // 8-move mate unknown author 1.Rg1+ Kh8 2.Qd6 Rg8 3.QxQ Rg4 4.Nf7+ Kg8 5.RxR+ PxR 6.Nh6+ Kh8 7.Qf7 Pg3 8/Qxp++[fen]1rr3k1/2q4p/2p2p2/1bQpNp2/p2P4/P3P3/5P1P/R1R4K w - - 1 [/fen]

[mvanthoor]

The FEN you posted has an error, and doesn't match the diagram shown.

I've used this FEN from below the diagram, with an extra 0 added to make it a legal FEN-string: 1rr3k1/2q4p/2p2p2/1bQpNp2/p2P4/P3P3/5P1P/R1R4K w - - 0 1

My engine takes 6.2 seconds to go to depth 10 on a 6700K. It will take a LONG time to solve this, and it searches at ~6 to 6.5 million NPS. If you're targeting 1M NPS, it will take almost 8x longer than a long time... assuming your engine has nothing else but alpha/beta and move sorting. (Mine doesn't yet.)

[yeni_sekme]

If you want to test the performance of your bitboard implementation, why not use perft test? Why are you using some random mate puzzle to measure the performance?
Strong engines find this kind of mates quickly not because of their movegen speed but it is because their search and evaluation are insanely good. Movegen speed has very little impact on engine strength. I don't know what does engine call mean but if you mean nps then even a not-well written mailbox engine like mine can reach 1mnps.

[Chessnut1071]

If you want to test the performance of your bitboard implementation, why not use perft test? Why are you using some random mate puzzle to measure the performance?
Strong engines find this kind of mates quickly not because of their movegen speed but it is because their search and evaluation are insanely good. Movegen speed has very little impact on engine strength. I don't know what does engine call mean but if you mean nps then even a not-well written mailbox engine like mine can reach 1mnps.

I assume you mean nodes per second. I'm using engine calls with approximately 40 nodes each. Using your metric 40 x 131.200,30`5,242,520 nodes per second. Also, I'm not sure what perft is. Looking for a description.

[mvanthoor]

If you want to test the performance of your bitboard implementation, why not use perft test? Why are you using some random mate puzzle to measure the performance?
Strong engines find this kind of mates quickly not because of their movegen speed but it is because their search and evaluation are insanely good. Movegen speed has very little impact on engine strength. I don't know what does engine call mean but if you mean nps then even a not-well written mailbox engine like mine can reach 1mnps.

I assume you mean nodes per second. I'm using engine calls with approximately 40 nodes each. Using your metric 40 x 131.200,30`5,242,520 nodes per second. Also, I'm not sure what perft is. Looking for a description.

Perft

[yeni_sekme]

5.2Mnps is quite good and definitely enough to write a very strong engine. Instead of wasting time on move generation, Good move ordering, Transposition tables, Null Move Ordering, Futility Pruning, Reverse Futility Pruning, Late Move Pruning, Late Move Reduction, and most importantly very good evaluation would be much more helpful to your engine strength.

[hgm]

Fruit 2.1 (a mailbox engine) takes 1.03 sec to solve this on my 3.2GHz i7:

Code: Select all

 10	+99.85 	2.74M  	0:01.03	Rg1+ Kh8 Qd6 Rg8 Qxc7 Rg4 Nf7+ Kg8 Rxg4+ fxg4 Nh6+ Kh8 Qf7 Rg8 Qxg8#
  9	+21.18 	329939	0:00.16	Rg1+ Kh8 Qd6 Rg8 Qxc7 Rg4 Nf7+ Kg8 Nh6+ Kh8 Qxb8+ Kg7 Nxf5+ Kf7 Rxg4
  8	+17.70 	225070	0:00.11	Rg1+ Kh8 Qd6 Rg8 Qxc7 Rg4 Qxb8+ Kg7 Nxg4 fxg4 Qc7+ Kg6 Rxg4+ Kh5
  7	+12.69 	69602  	0:00.05	Rg1+ Kh8 Qd6 Qg7 Rxg7 Rf8 Nf7+ Rxf7 Rxf7 Kg8 Qxf6
  6	+9.41 	35923  	0:00.03	Rg1+ Kh8 Qd6 Qg7 Rxg7 fxe5 Qxe5 h6 Re7+ Kg8 Qxf5
  5	+6.99 	12254  	0:00.02	Rg1+ Kh8 Qd6 Rb7 Qxf6+ Qg7 Rxg7 Rxg7 Nf7+ Kg8 Qxf5
  4	+2.72 	3837    	0:00.00	Rg1+ Kh8 Qd6 h6 Qxf6+ Kh7 Qg6+ Kh8 Qxh6+ Qh7
  3	+2.67 	1637    	0:00.00	Rg1+ Kh8 Qd6 h6 Qxf6+ Kh7 Qxf5+ Kh8
  2	+0.21 	547      	0:00.00	Rg1+ Kh8 Nf3
  1	+0.54 	30        	0:00.00	Rg1+ Kh8

Spartacus even does it in 0.64 sec:

Code: Select all

 14	 -319.92 	1.02M  	0:00.64	c1g1 g8h8 c5d6 c8g8 d6c7 g8g4 e5f7 h8g8 f7h6 g8h8 g1g4 f5g4 c7f7 c6c5 f7f6
 13	 -16.70 	458012	0:00.35	c1g1 g8h8 c5d6 c8g8 d6c7 g8g4 e5f7 h8g8 f7h6 g8h8 c7b8 h8g7 h6f5 g7g6 g1g4 g6f5 g4g7
 12	 -7.34 	310424	0:00.28	c1g1 g8h8 c5d6 b8b7 d6f6 c7g7 g1g7 b7g7 e5f7
 11	 -10.95 	110621	0:00.15	c1g1 g8h8 c5d6 c8g8 d6c7 b8f8 e5f7 f8f7 c7f7 g8g1 a1g1
 10	 -6.24 	54352  	0:00.09	c1g1 g8h8 c5d6 b8b7 d6f6 c7g7 g1g7 b7g7 f6f5 c8e8 f2f4
  9	 -5.92 	36775  	0:00.07	c1g1 g8h8 c5d6 c7g7 g1g7 h8g7 a1g1 g7h6 d6f6 h6h5 f6f5 h5h6
  8	 -1.81 	18096  	0:00.03	c1g1 g8h8 c5d6 h7h6 d6f6 h8h7 f6f5 h7h8 f2f4 c6c5 d4c5 c7c5
  7	 -1.79 	9939    	0:00.03	c1g1 g8h8 c5d6 h7h5 d6f6 h8h7 f6f5 h7h6 f2f4 c6c5 d4c5 c7c5
  6	 -1.84 	7132    	0:00.01	c1g1 g8h8 c5d6 h7h5 d6f6 h8h7 f6f5 h7h6 f2f4
  5	+0.41 	3160    	0:00.01	e5f3 g8h8 c5c2 c7d7 a1a2
  4	+0.41 	840      	0:00.00	e5f3 g8h8 c5b4 b5e2
  3	+0.41 	228      	0:00.00	e5f3 g8h8 a1a2
  2	+0.40 	172      	0:00.00	c1g1 g8h8 e5f3
  1	 -0.08 	86        	0:00.00	e5f3

[Ras]

My engine finished with the correct solution in 3 min: 58.04652 seconds w/ 31,200.301 legal engine calls, or, 131,093 calls/sec

My mailbox engine on a 3.7GHz 3400G takes about a second and 1.54M nodes:

Code: Select all

info depth 2 seldepth 3 score cp -60 time 47 nodes 698 nps 14851 hashfull 0 tbhits 0 pv c1g1 g8h8 e5f3
info depth 3 seldepth 4 score cp -61 time 48 nodes 1098 nps 22875 hashfull 0 tbhits 0 pv c1g1 g8h8 e5f3 b5c4
info depth 4 seldepth 4 score cp -89 time 49 nodes 2141 nps 43693 hashfull 0 tbhits 0 pv e5f3 g8h8 f3h4 b5d3
info depth 5 seldepth 5 score cp -70 time 51 nodes 5159 nps 101156 hashfull 0 tbhits 0 pv e5f3 g8h8 c5c2 f5f4 c2f5
info depth 6 seldepth 8 score cp 542 time 53 nodes 9444 nps 178188 hashfull 0 tbhits 0 pv c1g1 g8h8 c5d6 b8b7 d6f6 c7g7 g1g7 b7g7
info depth 7 seldepth 8 score cp 542 time 54 nodes 9900 nps 183333 hashfull 0 tbhits 0 pv c1g1 g8h8 c5d6 b8b7 d6f6 c7g7 g1g7 b7g7
info depth 8 seldepth 9 score cp 554 time 55 nodes 12846 nps 233563 hashfull 0 tbhits 0 pv c1g1 g8h8 c5d6 b8b7 d6f6 c7g7 g1g7 b7g7 f6f5
info depth 9 seldepth 12 score cp 747 time 65 nodes 26018 nps 400276 hashfull 0 tbhits 0 pv c1g1 c7g7 g1g7 g8g7 c5e7 g7h6 e7f6 h6h5 f6f5 h5h6 e5d7 h6g7
info depth 10 seldepth 12 score cp 569 time 73 nodes 40052 nps 548657 hashfull 0 tbhits 0 pv c1g1 g8h8 c5d6 b8b7 d6f6 c7g7 g1g7 b7g7 f6f5 g7g8 f5f6 g8g7
info depth 11 seldepth 12 score cp 577 time 77 nodes 46296 nps 601246 hashfull 0 tbhits 0 pv c1g1 g8h8 c5d6 b8b7 d6f6 c7g7 g1g7 b7g7 f6f5 g7g8 f5f6 g8g7
info depth 12 seldepth 18 score cp 1905 time 131 nodes 127966 nps 976839 hashfull 1 tbhits 0 pv c1g1 g8h8 c5d6 c8g8 d6c7 g8g6 e5g6 h7g6 c7b8 h8g7 b8c7 g7g8 g1g6 g8f8 g6f6 f8g8 a1g1 g8h8
info depth 13 seldepth 18 score cp 1905 time 139 nodes 139961 nps 1006913 hashfull 1 tbhits 0 pv c1g1 g8h8 c5d6 c8g8 d6c7 g8g6 e5g6 h7g6 c7b8 h8g7 b8c7 g7g8 g1g6 g8f8 g6f6 f8g8 a1g1 g8h8
info depth 14 seldepth 18 score cp 2006 time 170 nodes 187785 nps 1104617 hashfull 1 tbhits 0 pv c1g1 g8h8 c5d6 c8g8 d6c7 g8g4 e5f7 h8g8 f7h6 g8h8 c7b8 h8g7 h6f5 g7f7 g1g4 f7e6 b8c8 e6f7
info depth 15 seldepth 19 score mate 10 time 221 nodes 258317 nps 1168855 hashfull 2 tbhits 0 pv c1g1 g8h8 c5d6 c8g8 d6c7 g8g4 c7b8 h8g7 e5g4 f5g4 g1g4 g7h6 b8f4 h6h5 f4f5 h5h6 f5f6 h6h5 f6g5
info depth 16 seldepth 19 score mate 10 time 261 nodes 320556 nps 1228183 hashfull 2 tbhits 0 pv c1g1 g8h8 c5d6 c8g8 d6c7 g8g4 c7b8 h8g7 e5g4 f5g4 g1g4 g7h6 b8f4 h6h5 f4f5 h5h6 f5f6 h6h5 f6g5
info depth 17 seldepth 19 score mate 10 time 306 nodes 394062 nps 1287784 hashfull 3 tbhits 0 pv c1g1 g8h8 c5d6 c8g8 d6c7 g8g4 c7b8 h8g7 e5g4 f5g4 g1g4 g7h6 b8f4 h6h5 f4f5 h5h6 f5f6 h6h5 f6g5
info depth 18 seldepth 19 score mate 10 time 389 nodes 533975 nps 1372686 hashfull 4 tbhits 0 pv c1g1 g8h8 c5d6 c8g8 d6c7 g8g4 c7b8 h8g7 e5g4 f5g4 g1g4 g7h6 b8f4 h6h5 f4f5 h5h6 f5f6 h6h5 f6g5
info depth 19 seldepth 19 score mate 10 time 514 nodes 734014 nps 1428042 hashfull 6 tbhits 0 pv c1g1 g8h8 c5d6 c8g8 d6c7 g8g4 c7b8 h8g7 e5g4 f5g4 g1g4 g7f7 b8g8 f7e7 g4g7 e7d6 g8d8 d6e6 d8d7
info depth 20 seldepth 20 score mate 10 time 711 nodes 1064739 nps 1497523 hashfull 9 tbhits 0 pv c1g1 g8h8 c5d6 c8g8 d6c7 g8g4 c7b8 h8g7 e5g4 f5g4 g1g4 g7f7 b8g8 f7e7 g4g7 e7d6 g8d8 d6e6 d8d7
info time 1001 nodes 1512499 nps 1510988 hashfull 13 tbhits 0 currmove c1b1 currmovenumber 20
info depth 21 seldepth 21 score mate 8 time 1016 nodes 1540629 nps 1516367 hashfull 13 tbhits 0 pv c1g1 g8h8 c5d6 c8g8 d6c7 g8g4 e5f7 h8g8 f7h6 g8h8 g1g4 f5g4 c7f7 b8g8 f7g8

[Chessnut1071]

5.2Mnps is quite good and definitely enough to write a very strong engine. Instead of wasting time on move generation, Good move ordering, Transposition tables, Null Move Ordering, Futility Pruning, Reverse Futility Pruning, Late Move Pruning, Late Move Reduction, and most importantly very good evaluation would be much more helpful to your engine strength.

OMG, it took me 2 months to get the bitboards working after correcting all the known issues. This looks like a never ending task. I didn't think 5.2Mnps was any good. I'm reading some clocking 120Mnps. Are those speeds on a personal computer?

[JVMerlino]

My mediocre and rather slow 2500-ish engine, Myrddin, needs 3.6 seconds to announce Mate in 8, and it needs ~9.5M nodes to do it. So about 2.6KNPS.