Old code (dedicated comp) vs New code (Stockfish) Tournament

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

cpeters
Posts: 115
Joined: Wed Feb 17, 2021 7:44 pm
Full name: Christian Petersen

Old code (dedicated comp) vs New code (Stockfish) Tournament

Post by cpeters »

Hello!

Because of the lockdown I did use my idle hands to setup a tournament between (emulated) dedicated chess machines and stockfish* (m68k-binary)running on a Macintosh Quadra 800 (qemu-machine) with the help of cutechess(-cli).

Games so far:

[pgn][Event "lockdown"]
[Site "corona"]
[White "Stockfish 12 m68k"]
[Black "Mephisto Amsterdam"]
[Result "1-0"]
[ECO "C69"]
[GameDuration "00:31:15"]
[Opening "Ruy Lopez"]
[TimeControl "40/600+5"]
[Variation "Exchange, Bronstein Variation"]

1. e4 {+0.49/15 44s} e5 {1.5s} 2. Nf3 {+0.79/12 17s} Nc6 {1.7s}
3. Bb5 {+0.40/13 18s} a6 {1.6s} 4. Bxc6 {+0.63/12 6.1s} dxc6 {2.0s}
5. O-O {+0.17/15 44s} Qd6 {3.4s} 6. c3 {+0.03/13 8.3s} Bg4 {32s}
7. d4 {+0.03/15 24s} Bxf3 {19s} 8. Qxf3 {+0.66/12 10s} exd4 {1.9s}
9. Bf4 {+0.83/12 6.4s} Qf6 {8.4s} 10. Bxc7 {+0.68/17 52s} Qxf3 {32s}
11. gxf3 {+0.88/17 21s} dxc3 {1.6s} 12. Nxc3 {+0.78/15 14s} Bb4 {18s}
13. Rfd1 {+1.20/16 61s} Bxc3 {1.7s} 14. bxc3 {+1.53/14 2.2s} Ne7 {49s}
15. a4 {+2.04/14 32s} Rc8 {16s} 16. Bd6 {+1.29/17 59s} Rd8 {19s}
17. c4 {+1.32/13 3.3s} Rd7 {15s} 18. Bc5 {+1.15/15 8.3s} Ng6 {46s}
19. Rab1 {+0.85/16 45s} Rxd1+ {77s} 20. Rxd1 {+1.93/14 8.8s} Ne5 {38s}
21. Rb1 {+1.79/15 11s} b5 {1.8s} 22. cxb5 {+2.95/15 5.6s} axb5 {4.4s}
23. axb5 {+2.56/17 16s} cxb5 {1.7s} 24. f4 {+3.02/17 24s} Nd3 {25s}
25. Rxb5 {+4.08/15 16s} Kd7 {1.8s} 26. Be3 {+4.12/15 6.3s} Kc6 {53s}
27. Rb6+ {+4.32/14 14s} Kd7 {15s} 28. Rb7+ {+5.28/13 14s} Ke6 {27s}
29. f5+ {+4.96/16 17s} Ke5 {1.7s} 30. f3 {+5.02/16 6.2s} Ne1 {54s}
31. Kf2 {+6.17/16 30s} Nc2 {1.9s} 32. Rxf7 {+7.17/15 15s} Nxe3 {57s}
33. Kxe3 {+13.93/14 8.7s} g5 {2.9s} 34. Re7+ {+9.27/17 48s} Kf6 {17s}
35. Ra7 {+8.70/18 4.4s} h5 {29s} 36. Ra6+ {+10.86/15 26s} Ke7 {1.7s}
37. e5 {+9.99/14 23s} Rb8 {2.1s} 38. Ra7+ {+10.02/17 17s} Kf8 {22s}
39. f6 {+10.53/16 0.012s} Rb5 {11s} 40. Ke4 {+13.76/14 5.9s} Rb3 {30s}
41. e6 {+152.58/26 41s} Kg8 {33s} 42. Kf5 {+M11/42 14s} g4 {39s}
43. Kg6 {+M9/49 14s} Rb8 {21s} 44. Rg7+ {+M7/101 0.013s} Kf8 {8.9s} 45. Kh7
Rb7 {1.4s} 46. Rxb7 gxf3 {1.4s} 47. Rb8# {White mates} 1-0[/pgn]

[pgn][Event "lockdown"]
[Site "corona"]
[White "Mephisto Amsterdam"]
[Black "Stockfish 12 m68k"]
[Result "0-1"]
[ECO "B40"]
[GameDuration "00:25:11"]
[Opening "Sicilian"]
[TimeControl "40/600+5"]
[Variation "Anderssen Variation"]

1. e4 {3.8s} c5 {-0.35/15 61s} 2. Nf3 {1.7s} e6 {-0.29/13 2.6s} 3. d4 {1.7s}
cxd4 {-0.11/12 11s} 4. Nxd4 {1.6s} Nf6 {-0.22/12 6.8s} 5. Nc3 {1.7s}
d6 {-0.41/14 38s} 6. Bc4 {1.6s} a6 {-0.03/14 22s} 7. Bb3 {3.2s}
Be7 {-0.31/15 49s} 8. O-O {17s} Nc6 {-0.22/13 24s} 9. Nxc6 {16s}
bxc6 {-0.22/14 0.12s} 10. Bf4 {6.5s} Qc7 {-0.16/12 0.12s} 11. Qd2 {28s}
h6 {-0.23/15 39s} 12. Rad1 {26s} e5 {-0.34/14 13s} 13. Be3 {17s}
O-O {-0.24/15 15s} 14. Bc4 {11s} Ng4 {+0.22/14 21s} 15. a3 {24s}
Nxe3 {+0.67/16 27s} 16. Qxe3 {34s} Kh8 {+0.64/17 0.14s} 17. Rfe1 {31s}
f5 {+1.12/15 36s} 18. exf5 {25s} Bxf5 {+1.63/15 14s} 19. Bd3 {3.9s}
d5 {+1.51/15 30s} 20. Qxe5 {20s} Bd6 {+1.81/15 11s} 21. Qe3 {62s}
Bxh2+ {+2.47/13 13s} 22. Kh1 {1.7s} Bd6 {+2.03/15 27s} 23. Kg1 {22s}
Bd7 {+2.27/15 0.27s} 24. Bg6 {16s} Rf6 {+3.45/15 32s} 25. Bh5 {1.6s}
Raf8 {+3.54/15 12s} 26. Bf3 {19s} Bh2+ {+2.92/17 53s} 27. Kh1 {72s}
Bf4 {+3.27/18 0.17s} 28. Qe2 {16s} Bd6 {+3.93/15 32s} 29. Qxa6 {23s}
Bc5 {+6.32/16 25s} 30. Rd2 {26s} Rxf3 {+8.78/16 28s} 31. gxf3 {9.0s}
Qf4 {+11.05/17 22s} 32. Qe2 {32s} Bd6 {+M11/27 19s} 33. Qe5 {46s}
Bxe5 {+M9/47 0.061s} 34. Rxe5 {20s} Qxf3+ {+M7/69 0.057s} 35. Kg1 {36s} Bh3
36. Rg5 {1.4s} hxg5 37. a4 {1.4s} Qg2# {+M1/245 1.3s, Black mates} 0-1[/pgn]

[pgn][Event "lockdown"]
[Site "corona"]
[White "Stockfish 12 m68k"]
[Black "Tasc R 30 2.2"]
[Result "1-0"]
[ECO "B09"]
[GameDuration "00:19:36"]
[Opening "Pirc"]
[Termination "time forfeit"]
[TimeControl "40/600"]
[Variation "Austrian attack, 6.Be3"]

1. e4 {+0.34/14 33s} d6 {4.0s} 2. d4 {+0.68/13 15s} Nf6 {4.2s}
3. Nc3 {+0.86/11 5.3s} g6 {4.4s} 4. f4 {+0.96/13 11s} Bg7 {4.1s}
5. Nf3 {+0.88/12 3.6s} O-O {4.2s} 6. Be3 {+0.69/13 64s} Nbd7 {4.2s}
7. Qd2 {+0.49/13 28s} Ng4 {35s} 8. Bg1 {+0.75/13 8.7s} Nb6 {23s}
9. h3 {+1.43/15 18s} Nf6 {4.7s} 10. Bd3 {+1.58/14 0.055s} c6 {31s}
11. g4 {+1.77/14 30s} Be6 {29s} 12. b3 {+2.05/15 35s} Qc7 {42s}
13. f5 {+2.01/15 55s} Bd7 {5.1s} 14. a4 {+2.12/12 11s} a5 {31s}
15. Be3 {+2.27/11 5.3s} gxf5 {31s} 16. gxf5 {+4.24/13 10s} Nh5 {24s}
17. Bh6 {+5.18/15 17s} f6 {36s} 18. Rg1 {+5.54/15 4.0s} Rf7 {16s}
19. Ng5 {+5.76/17 17s} fxg5 {31s} 20. Qxg5 {+5.60/17 12s} c5 {42s}
21. Qxh5 {+5.98/13 9.0s} cxd4 {22s} 22. Ne2 {+6.98/16 18s} Qc5 {32s}
23. Nf4 {+7.75/12 8.3s} Qe5 {30s} 24. O-O-O {+8.22/14 0.077s} Rc8 {59s}
25. Kb1 {+8.40/13 6.9s} Rcf8 {27s} 26. Rg5 {+10.25/15 22s, Black loses on time}
1-0[/pgn]



Questions:

-What would be sane time-controls for the dedicated machines? 30sec/move?
-Do they ponder, when invoked through cutechess (cpu-usage indicates it, but I doubt it) even if the settings (like 'Aktivschach' etc.) have not been entered through the chessengine-for-mame-plugin from Sandro Ronco (a hero!)?
-Is it possible to play with opening files (with cutechess the moves get executed too fast I suppose for the lua-based chessengine for mame) - mind you: linux only/no wine etc. solution please?
-Why does cutechess complains if "-wait 2000" or so is entered? Why can't it wait 2 seconds or more bewtween moves, or can it? What does it even mean ('operator's time'?)?

This gets continued nevertheless: TascR30/The King 2.2 vs The Stockfish m68k now (40 moves/40min)..



* it's veryvery slow: 4kN/s at the ausgangsstellung

stockfish -->

bench 16 1 5 default perft

Code: Select all

Host T4500  @ 2.30GHz				Guest  Macintosh Quadra 800 CPU: 68040

Nodes searched: 21520988			Nodes searched: 21520988


======================================================================================
Total time (ms) : 21.677			Total time (ms) : 878.615


Nodes searched  : 1907298977			Nodes searched  : 1907298977
Nodes/second    : 87987220			Nodes/second    : 87987220


greetings!
Vinvin
Posts: 5228
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: Old code (dedicated comp) vs New code (Stockfish) Tournament

Post by Vinvin »

cpeters wrote: Mon Mar 08, 2021 11:20 pm ...
stockfish -->
bench 16 1 5 default perft

Code: Select all

Host T4500  @ 2.30GHz				Guest  Macintosh Quadra 800 CPU: 68040

Nodes searched: 21520988			Nodes searched: 21520988


======================================================================================
Total time (ms) : 21.677			Total time (ms) : 878.615


Nodes searched  : 1907298977			Nodes searched  : 1907298977
Nodes/second    : 87987220			Nodes/second    : 87987220
How to interpret these numbers ?
cpeters
Posts: 115
Joined: Wed Feb 17, 2021 7:44 pm
Full name: Christian Petersen

Re: Old code (dedicated comp) vs New code (Stockfish) Tournament

Post by cpeters »

These numbers get reported from the binary running in the virtual machine (linuxthorvaltsm68k port) and the host of this construct.


Interpretation:

-m68k is factor 40 slower in my setup
-binaries are same/searchwise
cpeters
Posts: 115
Joined: Wed Feb 17, 2021 7:44 pm
Full name: Christian Petersen

Re: Old code (dedicated comp) vs New code (Stockfish) Tournament

Post by cpeters »

cpeters
Posts: 115
Joined: Wed Feb 17, 2021 7:44 pm
Full name: Christian Petersen

Re: Old code (dedicated comp) vs New code (Stockfish) Tournament

Post by cpeters »

-Do they ponder, when invoked through cutechess (cpu-usage indicates it, but I doubt it) even if the settings (like 'Aktivschach' etc.) have not been entered through the chessengine-for-mame-plugin from Sandro Ronco (a hero!)?
Nevermind. I did find it out:
-some options can be changed when invoked with cutechess-cli, as the mame-chessplugin reports them uciwise; others directly in the corresponding *.lua files (for that it seems easier if you're used to the machine/know its manual)...
cpeters
Posts: 115
Joined: Wed Feb 17, 2021 7:44 pm
Full name: Christian Petersen

Re: Old code (dedicated comp) vs New code (Stockfish) Tournament

Post by cpeters »

Phase 1: done.

Establish/get an idea of the playing strength (m68k host runs unlimited; in my setup good enough for ca. 900 MHz whithin the Quadra); let stockfish there use a small/varied (!) book (polyglot make-book -max-ply 10... from Frank Quisinsky's opening vault), 8 mb hash and give it access to syzygy 345 -bases.
Try to correct for limited ressources the host has to offer for mame (Mephisto Amsterdam runs very well; the Tasc is sluggish (speed 65%)- so: correct it with the timecontrol due cutechess -° this is still ongoing.

Code: Select all

--------------------------------------------
 1: Stockfish 12 m68k@900MHz        9.0 /  9   
 2: Tasc R30 V 2.2           2375   0.0 /  9   
--------------------------------------------
9 games: +5 =0 -4


----------------------------------------------
 1: Stockfish 12 m68k@900MHz        12.0 /  12   
 2: Mephisto Amsterdam LVL 4*  1931   0.0 /  12   
----------------------------------------------
12 games: +6 =0 -6
*=30sec/move; mode for the Tasc is 40sec/move

Conclusion:stockfish is too fast and needs to be limited. As a start I did in the host (give the binary 10% cpu ressources) so it use ca. 90MHz. This is running right now... Aim is to get to the orignal speed (around 30MHz).


OS/programs used for this setup so far:

-KaliLinux/x64 with everything
-qemu-quadra800-m68
-Debian Sid port (fully upgraded) for this platform
-mame (emulator) for the dedicated machines
-openssh to talk to the quadra
-cutechess-cli for play

pgn for amster
pgn for tasc
Vinvin
Posts: 5228
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: Old code (dedicated comp) vs New code (Stockfish) Tournament

Post by Vinvin »

cpeters wrote: Tue Mar 09, 2021 2:22 am These numbers get reported from the binary running in the virtual machine (linuxthorvaltsm68k port) and the host of this construct.


Interpretation:

-m68k is factor 40 slower in my setup
-binaries are same/searchwise
That still not 100% clear for me :-/
What does mean the 88 millions nodes/second ?

Code: Select all

Host T4500  @ 2.30GHz				Guest  Macintosh Quadra 800 CPU: 68040

Nodes searched: 21520988			Nodes searched: 21520988

======================================================================================
Total time (ms) : 21.677			Total time (ms) : 878.615

Nodes searched  : 1907298977			Nodes searched  : 1907298977
Nodes/second    : 87987220			Nodes/second    : 87987220
cpeters
Posts: 115
Joined: Wed Feb 17, 2021 7:44 pm
Full name: Christian Petersen

Re: Old code (dedicated comp) vs New code (Stockfish) Tournament

Post by cpeters »

Vinvin wrote: Fri Mar 12, 2021 3:08 pm

That still not 100% clear for me :-/
What does mean the 88 millions nodes/second ?
Sorry for the irritation Vincent, I fucked it up/posted wrong numbers:
Nodes/second for the virtual machine should read: 2170801 (so Linux x86-64 Host vs Linux m68k = 88MN/s vs 2MN/s)


Anyway, I fetched myself the source for stockfish13, compiled it and let it run (no "bench 16 1 5 default perft"), just "bench"

Code: Select all

Host:							Guest @reported 900MHz:

Total time(ms)		:6521				360478
Nodes searched		:3766422			3766422
Nodes/second		:577583				10448


Matches continue. I'll disable syzygy-bases, limit to 45MHz and use the new 13-version.
Vinvin
Posts: 5228
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: Old code (dedicated comp) vs New code (Stockfish) Tournament

Post by Vinvin »

cpeters wrote: Fri Mar 12, 2021 5:19 pm
Vinvin wrote: Fri Mar 12, 2021 3:08 pm

That still not 100% clear for me :-/
What does mean the 88 millions nodes/second ?
Sorry for the irritation Vincent, I fucked it up/posted wrong numbers:
Nodes/second for the virtual machine should read: 2170801 (so Linux x86-64 Host vs Linux m68k = 88MN/s vs 2MN/s)


Anyway, I fetched myself the source for stockfish13, compiled it and let it run (no "bench 16 1 5 default perft"), just "bench"

Code: Select all

Host:							Guest @reported 900MHz:

Total time(ms)		:6521				360478
Nodes searched		:3766422			3766422
Nodes/second		:577583				10448


Matches continue. I'll disable syzygy-bases, limit to 45MHz and use the new 13-version.
Thanks for the clarification !
Vinvin
Posts: 5228
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: Old code (dedicated comp) vs New code (Stockfish) Tournament

Post by Vinvin »

cpeters wrote: Thu Mar 11, 2021 3:20 pm ...

Conclusion:stockfish is too fast and needs to be limited. As a start I did in the host (give the binary 10% cpu ressources) so it use ca. 90MHz. This is running right now... Aim is to get to the orignal speed (around 30MHz).
...
SF (close to V11) at 20 KNodes/seconds plays at strength around 3035 (see here : http://talkchess.com/forum3/viewtopic.p ... 66#p827666 )

Tasc R30 (ARM2 30 MHz) has a strength around 2356 https://www.schach-computer.info/wiki/i ... -Elo-Liste

So, SF13, even at 1% speed, would destroy the Task R30 with a score around 99%-1% ;-)