Candidates for the computationally most efficient engine?

[dangi12012]

What engine would play with the least amount of CPU cycles to achieve a rating of 1800 or more?
Intuition: Stockfish with fixed node count

With cpu cylces I mean actually reading the instruction count of the CPU (so not time based)
https://c9x.me/x86/html/file_module_x86_id_278.html

But it might be another engine you know that does not come with overhead to achieve even higher strenght and is very small overall?
I know rating is an overloaded term but which open source engine on the TCEC board needs the least CPU time to reach 1800+?

[hgm]

Ratings are measured by comparing engines at equal time usage. So wouldn't this simply be the engine with the highest rating?

[Bo Persson]

What engine would play with the least amount of CPU cycles to achieve a rating of 1800 or more?
Intuition: Stockfish with fixed node count

With cpu cylces I mean actually reading the instruction count of the CPU (so not time based)
https://c9x.me/x86/html/file_module_x86_id_278.html

But it might be another engine you know that does not come with overhead to achieve even higher strenght and is very small overall?
I know rating is an overloaded term but which open source engine on the TCEC board needs the least CPU time to reach 1800+?

Note that RDTSC doesn't count instructions, but clock ticks. And modern processors can (sometimes) do 3-4 simple instructions per clock.

Also, RDTSC may or may not be affected by the turbo mode of the processor.

[jdart]

Stockfish eval is very, very good. So it can beat other engines even with a CPU budget handicap, like fewer cores.

If you look at other measures, like NPS, other engines would score higher.

[Alexander Schmidt]

What engine would play with the least amount of CPU cycles to achieve a rating of 1800 or more?

The rating of Leela with one node is around 2000.

This is played on tournament level for the emulations and 1 node for Leela, nets became better meanwhile:

Code: Select all

                                      Nat    Score     Ta Me Me Fi No Le Sa CX Co Le CX Me De Le Ch Sc Percnt
-------------------------------------------------------------------------------------------------------------
 1: Tasc R30 v2.5                     NED  27.5 / 30   XX =1 11 11 11 == 11 =1 11 11 =1 11 11 11 11 11  91.6%  (+25 -0 =5)
 2: Mephisto Risc II                  NED  25.0 / 30   =0 XX == 10 11 11 11 10 11 1= 11 11 11 11 11 11  83.3%  (+23 -3 =4)
 3: Mephisto Vancouver 68020          ENG  24.5 / 30   00 == XX == =1 1= 11 11 11 =1 11 11 11 11 11 11  81.6%  (+21 -2 =7)
 4: Fidelity Elite Mach IV            USA  21.5 / 30   00 01 == XX 11 10 10 11 =1 01 10 11 11 11 11 11  71.6%  (+20 -7 =3)
 5: Novag Super Expert C              USA  19.0 / 30   00 00 =0 00 XX 10 11 11 =0 1= 11 1= 11 11 11 11  63.3%  (+17 -9 =4)
 6: Leela Zero 0.24.1 63108 (320x24)  BEL  17.5 / 30   == 00 0= 01 01 XX 01 01 =1 11 11 01 10 1= =1 =1  58.3%  (+14 -9 =7)
 7: Saitek Maestro D+ 5MHz            ARG  17.0 / 30   00 00 00 01 00 10 XX 10 01 11 1= 11 11 11 =1 11  56.6%  (+16 -12 =2)
 8: CXG Sphinx Dominator v2.05        NED  15.5 / 30   =0 01 00 00 00 10 01 XX =1 11 0= 00 11 11 11 11  51.6%  (+14 -13 =3)
 9: Conchess Plymate Victoria         SWE  12.5 / 30   00 00 00 =0 =1 =0 10 =0 XX =0 1= 11 01 00 =1 11  41.6%  (+9 -14 =7)
10: Leela Zero 0.24.1 42850 (256x20)  BEL  12.5 / 30   00 0= =0 10 0= 00 00 00 =1 XX =1 1= 01 11 =1 01  41.6%  (+9 -14 =7)
11: CXG Sphinx 40                     USA  12.5 / 30   =0 00 00 01 00 00 0= 1= 0= =0 XX 11 10 10 11 11  41.6%  (+10 -15 =5)
12: Mephisto III-S Glasgow            GER  11.5 / 30   00 00 00 00 0= 10 00 11 00 0= 00 XX 11 11 1= 11  38.3%  (+10 -17 =3)
13: Debut-M                           RUS   7.0 / 30   00 00 00 00 00 01 00 00 10 10 01 00 XX 10 01 01  23.3%  (+7 -23 =0)
14: Leela Zero 0.24.1 591226 (128x10) BEL   6.5 / 30   00 00 00 00 00 0= 00 00 11 00 01 00 01 XX 10 10  21.6%  (+6 -23 =1)
15: Chafiz Destiny Prodigy            USA   5.5 / 30   00 00 00 00 00 =0 =0 00 =0 =0 00 0= 10 01 XX 01  18.3%  (+3 -22 =5)
16: SciSys Chess Champion Mark V      USA   4.5 / 30   00 00 00 00 00 =0 00 00 00 10 00 00 10 01 10 XX  15.0%  (+4 -25 =1)

[smatovic]

This would be the engine which uses the CPU vector unit for move generation, move picking, and evaluation I guess. Scalar code + vector code, the more vector the less CPU cycles?

--
Srdja

[dangi12012]

The rating of Leela with one node is around 2000.

Oh wow I was not aware of that. That is insane since all the other governing and search networks will not be needed when only looking at a single node - so this should be much faster to evaluate than the usual 50k nps.

Interesting!

[hgm]

Is that really comparable to a CCRL rating? In the tourney I only see dedicated machines, which are not on the CCRL list.

I am a bit skeptical that with just a neural network you would be immune to tactical errors that even a stupid alpha-beta searcher like micro-Max would not expose, when you have to play from a normal opening book.

[Edit] Interesting. I don't really know how to use Leela (e.g. where to get networks, how to install those, how to make it use the network I want, and how to force it tu use only 1 node). I had downloaded a Leela version some time ago, and now downlaoded a 320x24 network from GitHub, but I don't know if it was using that net. It seems to always think 4 sec at the TC I set (120 moves/3 min), and the Thinking Output said it was using 4-7 nodes.

Under these conditions it crushes Fairy-Max, but then in the end stalemates it !?

[pgn][Event "Computer Chess Game"]
[Site "MAKRO-PC"]
[Date "2022.07.16"]
[Round "-"]
[White "Lc0 v0.25.1+git.69105b4"]
[Black "Fairy-Max 5.0b6"]
[Result "1/2-1/2"]
[TimeControl "120/180"]

1. e4 e5 2. Nf3 Nc6 3. Bb5 a6 4. Ba4 Nf6 5. O-O Be7 6. Re1 b5 7. Bb3 d6 8.
c3 O-O 9. h3 {+0.21/1 4} Bd7 {-0.13/8 1.9} 10. d4 {+0.27/1 4} h5
{-0.10/8 1.3} 11. Nbd2 {+0.83/1 4} Rb8 {-0.05/8 2.8} 12. a4 {+1.02/1 4} Qc8
{-0.03/7 0.9} 13. a5 {+1.22/2 4} Qb7 {-0.07/8 3} 14. Nf1 {+1.44/1 4} Rfe8
{-0.13/7 1.6} 15. Ng3 {+2.21/1 4} g6 {-0.29/7 2.2} 16. Qd2 {+3.74/2 4} b4
{-0.20/7 3} 17. Qh6 {+8.42/1 4} d5 {-1.21/6 1.5} 18. exd5 {+6.67/2 4} Nxa5
{-1.23/7 1.2} 19. Rxa5 {+5.61/1 4} Bf8 {-0.68/7 1.0} 20. Qg5 {+6.99/2 4}
Nh7 {-0.61/7 1.2} 21. Qd2 {+0.00/1 4} bxc3 {-0.44/8 1.5} 22. bxc3
{+3.00/1 4} Qxb3 {-0.36/8 1.9} 23. dxe5 {+2.90/2 4} Qb6 {-0.50/7 1.1} 24.
Ra2 {+5.08/1 4} Rbc8 {-0.46/7 1.9} 25. Ba3 {+7.54/1 4} Bg7 {-0.47/7 1.4}
26. c4 {+8.60/2 4} Qb3 {-0.20/7 1.4} 27. Rc1 {+5.99/2 4} Bh6 {+0.15/8 1.3}
28. Qxh6 {+14.88/1 4} Qxa2 {-0.21/9 4} 29. Bb4 {+14.69/2 4} h4
{+0.05/7 1.6} 30. Ne4 {+21.95/1 4} Qb2 {-0.85/7 2.1} 31. Bc3 {+30.97/2 4}
Qa3 {-2.96/9 1.0} 32. Nf6+ {+40.04/2 4} Nxf6 {-0.05/10 1.4} 33. exf6
{+21.51/2 4} Qf8 {-1.48/11 1.3} 34. Qxh4 {+14.64/2 4} Qa3 {-1.57/11 2.2}
35. Bd2 {+14.86/2 4} Qb2 {-0.69/7 1.2} 36. Qg5 {+13.44/2 4} Qa3 {-0.62/8 3}
37. c5 {+18.76/2 4} Qb2 {-3.79/8 1.1} 38. Bc3 {+43.24/2 4} Qa3
{-4.82/10 1.7} 39. Qh6 {+15.39/2 4} Qxc1+ {-6.51/11 0.9} 40. Qxc1
{+37.39/2 4} Kh7 {-6.99/11 1.0} 41. Ng5+ {+52.27/2 4} Kg8 {-6.93/11 2.5}
42. Qf4 {+54.21/2 3} Bg4 {-8.37/9 1.0} 43. Nxf7 {+0.00/1 3} Kxf7
{-10.38/10 1.1} 44. Qh6 {+52.39/1 3} Re1+ {-20.13/13 2.3} 45. Kh2
{+64.24/3 3} Rf8 {-14.44/13 1.5} 46. Qg7+ {+58.51/2 3} Ke8 {-13.43/11 1.3}
47. Bxe1 {+68.07/1 2.8} Rf7 {-17.43/14 1.3} 48. Qg8+ {+88.01/2 2.7} Rf8
{-19.45/16 1.3} 49. Qxg6+ {+88.70/2 2.5} Rf7 {-16.72/14 0.9} 50. c6
{+0.00/1 2.3} a5 {-1000.05/15 1.3} 51. Bxa5 {+0.00/1 2.1} Bf5
{-23.55/17 1.5} 52. Qxf5 {+0.00/1 2.0} Rxf6 {-14.30/10 1.0} 53. Qxf6
{+0.00/1 1.6}
{Stalemate} 1/2-1/2
[/pgn]

[Edit]
I now made a version of UCI2WB that has an option to specify a nodes limit, and when that is non-zero only uses the command "go nodes" to let the engine think. This way I could limit Leela to 1 node. At this setting it is certainly not invulnarable to Fairy-Max, (although it often is able to explot the totally worthless King Safety of the latter):

[pgn][Event "Computer Chess Game"]
[Site "MAKRO-PC"]
[Date "2022.07.16"]
[Round "-"]
[White "Lc0 v0.25.1+git.69105b4 (UCI2WB)"]
[Black "Fairy-Max 5.0b6"]
[Result "0-1"]
[TimeControl "120/180"]

1. d4 {+0.07/1} Nf6 2. c4 {+0.08/1 0.1} e6 3. g3 {+0.09/1 0.1} d5 4. Bg2
{+0.12/1 0.1} Nbd7 5. Nf3 {+0.13/1 0.1} c6 6. O-O {+0.19/1 0.1} dxc4
{+0.43/9 1.9} 7. a4 {+0.31/1 0.1} Qb6 {+0.51/8 1.8} 8. Nbd2 {+0.74/1 0.1}
Qa6 {+0.74/9 2.2} 9. Qc2 {+0.64/1 0.1} b5 {+0.71/9 6} 10. b3 {+0.72/1 0.1}
cxb3 {+0.77/9 2.0} 11. Nxb3 {+0.62/1 0.1} bxa4 {+0.85/9 2.0} 12. Nc5
{+0.80/1 0.1} Nxc5 {+0.95/9 2.6} 13. dxc5 {+1.47/1 0.1} Nd5 {+0.76/8 1.9}
14. Rxa4 {+2.54/1 0.1} Qb5 {+0.63/8 0.9} 15. Ba3 {+2.00/1 0.1} Qb7
{+0.66/8 1.8} 16. Bb2 {+2.29/1 0.1} f6 {+0.67/7 1.7} 17. Rb1 {+1.41/1 0.1}
Qc7 {+0.73/7 1.0} 18. e4 {+2.33/1 0.1} Ne7 {+0.65/8 0.9} 19. e5
{+3.02/1 0.1} Nd5 {+0.71/8 1.8} 20. exf6 {+3.49/1 0.1} gxf6 {+0.70/7 1.3}
21. Ng5 {+4.12/1 0.1} e5 {+0.94/8 1.1} 22. Re1 {+3.14/1 0.1} Bh6
{+0.94/8 1.2} 23. Ne4 {+2.85/1 0.1} O-O {+0.85/8 1.9} 24. Nd6 {+1.87/1 0.1}
Bd7 {+0.44/7 1.4} 25. Rh4 {+4.45/1 0.1} Qa5 {-1.94/8 1.3} 26. Rd1
{+9.86/1 0.1} Nb4 {-2.07/8 1.4} 27. Qc4+ {+21.13/1 0.1} Kg7 {-2.11/8 1.1}
28. Rxh6 {+17.73/1 0.1} Kxh6 {-2.89/8 1.9} 29. Nb7 {+13.98/1 0.1} Qc7
{-0.68/9 1.4} 30. Qxb4 {+5.29/1 0.1} Kg7 {-0.72/8 0.9} 31. Nd6
{+10.06/1 0.1} Rab8 {-0.47/8 1.9} 32. Qd2 {+12.59/1 0.1} Rb3 {-0.57/7 1.1}
33. Bxe5 {+17.54/1 0.1} fxe5 {-1.10/9 1.7} 34. Qg5+ {+11.93/1 0.1} Kh8
{-3.77/9 1.9} 35. Qxe5+ {+13.60/1 0.1} Kg8 {-2.10/9 2.0} 36. Bf1
{+15.67/1 0.1} Rb4 {-3.48/10 2.0} 37. Bc4+ {+14.91/1 0.1} Rxc4
{-1.23/11 1.2} 38. Qg5+ {+2.37/1 0.1} Kh8 {-1.25/11 1.9} 39. Nxc4
{+10.89/1 0.1} Be6 {-0.57/9 1.1} 40. Qh6 {+5.55/1 0.1} Qf7 {+1.52/10 1.0}
41. f4 {-0.01/1 0.1} Bxc4 {+1.61/10 0.9} 42. Rd7 {-0.47/1 0.1} Qxd7
{+2.18/12 1.3} 43. Qxf8+ {-1.15/1 0.1} Bg8 {+2.10/14 1.9} 44. Qf6+
{-6.18/1 0.1} Qg7 {+2.22/13 1.1} 45. Qxc6 {-3.57/1 0.1} Qd4+ {+8.91/13 1.2}
46. Kf1 {-10.28/1 0.1} Bc4+ {+8.99/13 0.9} 47. Kg2 {-19.29/1 0.1} Bd5+
{+9.05/14 1.4} 48. Qxd5 {-60.94/1 0.1} Qxd5+ {+14.24/17 1.7} 49. Kf2
{-84.78/1 0.1} a5 {+8.81/10 0.9} 50. c6 {-93.41/1 0.1} Qc4 {+13.58/11 1.3}
51. Ke3 {-90.51/1 0.1} a4 {+15.41/12 2.4} 52. c7 {-97.96/1 0.1} a3
{+16.02/12 1.7} 53. c8=R+ {-91.26/1 0.1} Qxc8 {+16.74/12 1.2} 54. Ke4
{-118.17/1 0.1} a2 {+1000.05/12 2.8} 55. g4 {-118.35/1 0.1} a1=Q
{+1000.04/17 0.9} 56. Ke3 {-119.24/1 0.1} Qc5+ {+1000.03/28 0.2} 57. Ke2
{-110.96/1 0.1} Qc2+ {+1000.02/28} 58. Kf3 {-122.89/1 0.1} Qa3#
{+1000.01/28}
{Black mates} 0-1
[/pgn]

[pgn][Event "Computer Chess Game"]
[Site "MAKRO-PC"]
[Date "2022.07.16"]
[Round "1"]
[White "Lc0 v0.25.1+git.69105b4 (UCI2WB)"]
[Black "Fairy-Max 5.0b6"]
[Result "0-1"]
[TimeControl "120/180"]

1. d4 {+0.07/1} f5 2. Nf3 {+0.23/1 0.1} Nf6 3. g3 {+0.26/1 0.1} g6 4. Bg2
{+0.25/1 0.1} Bg7 {-0.12/8 1.7} 5. c4 {+0.28/1 0.1} O-O 6. O-O
{+0.30/1 0.1} d6 7. Nc3 {+0.28/1 0.1} c6 8. d5 {+0.24/1 0.1} cxd5 9. cxd5
{+0.29/1 0.1} Nbd7 {+0.02/8 1.0} 10. Be3 {+0.33/1 0.1} Ng4 {+0.03/8 1.1}
11. Bd4 {+0.51/1 0.1} Bxd4 {+0.07/8 1.0} 12. Nxd4 {+0.51/1 0.1} Qb6
{+0.01/9 4} 13. Qd2 {+0.82/1 0.1} Nde5 {+0.02/8 2.2} 14. b3 {+0.62/1 0.1}
Bd7 {-0.10/8 1.9} 15. a4 {+0.67/1 0.1} Rfc8 {-0.13/7 1.2} 16. a5
{+0.54/1 0.1} Qb4 {-0.06/8 1.0} 17. Rfc1 {+0.26/1 0.1} Rc5 {-0.02/8 1.5}
18. f4 {+0.04/1 0.1} Rac8 {+0.21/8 1.3} 19. fxe5 {-0.22/1 0.1} dxe5
{+0.73/9 1.7} 20. Nc6 {-0.26/1 0.1} bxc6 {+0.90/9 1.8} 21. dxc6
{-0.83/1 0.1} Bxc6 {+1.38/10 3} 22. Bxc6 {-2.40/1 0.1} R5xc6 {+2.75/9 2.1}
23. Qd5+ {-8.41/1 0.1} e6 {+3.33/10 1.6} 24. Na2 {-10.62/1 0.1} Rxc1+
{+10.47/10 1.2} 25. Rxc1 {-3.63/1 0.1} Rxc1+ {+1000.05/15 1.4} 26. Kg2
{-2.77/1 0.1} Rg1+ {+1000.04/23 1.0} 27. Kh3 {-28.37/1 0.1} Nf2#
{+1000.01/28}
{Black mates} 0-1
[/pgn]

[dangi12012]

How good is Fairy max on TCEC?
Also is a node with LC0 the same as one visited position?

Also does somebody have the source code for a single node version?

If it's really 2k it really would be interesting for MCTS as the leaf random rollout engine. Bonus points because a NN lends itself to parallel evaluation and even exotic hardware like a TPU.

[hgm]

Fairy-Max doesn't play on TCEC. It would not be admissible, because it is not multithreaded.