Stockfish_dev is probably stronger than Sargon 1978 v1.00

[Laskos]

Bill Forster, author of the Tarrasch GUI, has released a UCI version of Sargon 1978 v1.00, the well known program and champion of 1978 from the Spracklens. A thread was posted here with this news http://talkchess.com/forum3/viewtopic.php?f=2&t=74027
Sargon can be found here https://github.com/billforsternz/retro-sargon/releases
Sargon 1978 v1.00 is the oldest UCI engine I know of and is an A/B engine probably without quiescence but using some lookahead techniques. I decided to check whether Stockfish_dev is able to beat that engine from 1978 on equal hardware. It indeed seems to beat it by a bit:


Score of SF_dev vs Sargon_1978: 34 - 27 - 39 [0.535] 100

... SF_dev playing White: 21 - 10 - 19 [0.610] 50
... SF_dev playing Black: 13 - 17 - 20 [0.460] 50
... White vs Black: 38 - 23 - 39 [0.575] 100
Elo difference: 24.4 +/- 53.6, LOS: 81.5 %, DrawRatio: 39.0 %
Finished match


That was depth=5 match. The depth=6 match takes longer and the result is here:


Score of SF_dev vs Sargon_1978: 31 - 23 - 46 [0.540] 100

... SF_dev playing White: 14 - 14 - 22 [0.500] 50
... SF_dev playing Black: 17 - 9 - 24 [0.580] 50
... White vs Black: 23 - 31 - 46 [0.460] 100
Elo difference: 27.9 +/- 50.3, LOS: 86.2 %, DrawRatio: 46.0 %
Finished match

The constancy of the value of a ply across 42 years of computer chess is amazing, one ply kept its Elo value almost constant.

To check its strength at fixed time, I played games at 40 moves in 1 minute time control against another champion, Mephisto Amsterdam of 1985 of Richard Lang, FIDE rated at about 1850 Elo. That too was ported and made UCI compatible.

Code: Select all

Games Completed = 100 of 100 (Avg game length = 125.953 sec)
Settings = Gauntlet/64MB/60000ms in 40 moves/M 9999cp for 1000 moves, D 1000 moves/EPD:C:\LittleBlitzer\3moves_GM_04.epd(817)
Time = 3262 sec elapsed, 0 sec remaining
 1.  Mephisto Amsterdam 1985  	97.0/100	94-0-6  	(L: m=0  t=0 i=0 a=0)	(D: r=4 i=2 f=0 s=0 a=0)	(tpm=1444.8 d=1.00 nps=0)
 2.  Sargon 1978              	 3.0/100	0-94-6  	(L: m=94 t=0 i=0 a=0)	(D: r=4 i=2 f=0 s=0 a=0)	(tpm=1247.2 d=4.79 nps=34016)

About 600 Elo points difference or maybe about 400 human FIDE Elo points difference. Taking into account that the original 1978 Sargon ran on 2 MHz Z80, but Mephisto Amsterdam on 12 MHz 68000 CPU, Sargon 1978 v1.00 is in 1350-1400 FIDE Elo ballpark on the original hardware. That at tournament time control on 1978 Z80 hardware. On modern hardware and Blitz, it is maybe 1900 FIDE Elo, so it will beat me quite consistently. But for many people here it might be fun to play it, choose a nice GUI to enjoy.

What changed the most dramatically with A/B engines in time is the Effective Branching Factor (EBF), keeping the Elo value of a ply constant. Here is some rudimentary computation of EBF from initial starting position.

Code: Select all

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq -

Engine: Sargon 1978 V1.00 (0 MB)
by Dan and Kathe Spracklin, Windows port b

5.00   0:00   +0.75    1.Nc3 Nc6 2.d4 Nf6 3.Bf4 (29.116) 38 

6.00   0:02   0.00     1.Nc3 Nc6 2.d4 e5 3.d5 Nd4 (112.922) 38 

7.00   0:20   +0.75    1.Nc3 Nc6 2.Nf3 Nf6 3.d4 d5 4.Be3 (790.919) 38 

8.00   1:30   0.00     1.Nc3 Nc6 2.Nf3 Nf6 3.d4 d5 4.Bf4 Bf5 (3.430.248) 38 

9.00   10:56  +0.75    1.d4 Nc6 2.Nc3 Nf6 3.Nf3 d5 4.Qd3 Qd6 5.Qb5 (25.187.030) 38 
==================================================================================

EBF = 5.42
ln(EBF) = 1.69

Code: Select all

 
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq -

Engine: Stockfish 260520 64 BMI2 (2048 MB)
by T. Romstad, M. Costalba, J. Kiiski, G.


29/43  0:21   +0.70++  1.d4 (33.393.061) 1535 

29/43  0:23   +0.66    1.d4 d5 2.c4 e6 3.Nf3 Nf6 4.Nc3 Bb4 
                       5.e3 O-O 6.Bd3 c5 7.O-O Nc6 8.dxc5 Bxc5 
                       9.a3 Bd6 10.cxd5 exd5 11.e4 Re8 
                       12.exd5 Ne5 13.Be2 Nxf3+ 14.Bxf3 (35.621.943) 1535 

30/41  0:26   +0.76++  1.d4 (41.006.569) 1537 

30/41  0:29   +0.55--  1.d4 d5 (45.026.940) 1530 

30/48  0:43   +0.54    1.d4 d5 2.c4 e6 3.cxd5 exd5 4.Nc3 c6 
                       5.Bf4 Nf6 6.e3 Bd6 7.Bxd6 Qxd6 8.Bd3 Bg4 
                       9.Nge2 O-O 10.Qc2 Bxe2 11.Bxe2 Nbd7 
                       12.Rd1 Rfe8 13.O-O g6 14.Rc1 (65.845.544) 1524 

31/42  0:46   +0.44--  1.d4 d5 (70.931.174) 1523 

31/43  0:53   +0.54++  1.d4 (80.211.312) 1512 

31/43  0:57   +0.38    1.d4 d5 2.c4 e6 3.Nf3 Nf6 4.Nc3 c6 
                       5.Bg5 dxc4 6.e4 b5 7.e5 h6 8.Bh4 g5 
                       9.Nxg5 hxg5 10.Bxg5 Be7 11.exf6 Bxf6 
                       12.Be3 Bb7 13.Qf3 a6 14.a3 (86.766.585) 1496 

32/50  1:12   +0.48++  1.d4 (109.028.076) 1503 

32/50  1:13   +0.58++  1.d4 (110.753.097) 1503 

32/50  1:14   +0.60    1.d4 d5 2.c4 e6 3.Nc3 c6 4.g3 dxc4 
                       5.Nf3 Nf6 6.Bg2 Nbd7 7.a4 Bb4 8.O-O O-O 
                       9.Qc2 h6 10.Rd1 Qa5 11.Bf4 Nd5 
                       12.Bd2 N5f6 13.e4 Qh5 14.h3 (111.915.509) 1503 

33/39  1:19   +0.50--  1.d4 d5 (118.803.678) 1502 

33/39  1:21   +0.39--  1.d4 d5 (122.761.967) 1503 

33/45  1:24   +0.50++  1.d4 (127.743.235) 1504 

33/49  1:27   +0.70++  1.d4 (131.812.705) 1505 

33/54  1:39   +0.69    1.d4 d5 2.c4 e6 3.Nc3 c5 4.cxd5 exd5 
                       5.Nf3 cxd4 6.Nxd4 Nf6 7.Bg5 Be7 8.e3 O-O 
                       9.Be2 Nc6 10.O-O Be6 11.h3 Rc8 
                       12.Rc1 a6 13.a3 h6 14.Bh4 (150.113.010) 1511 

34/47  1:43   +0.59--  1.d4 d5 (156.996.026) 1511 

34/47  1:46   +0.49--  1.d4 d5 (161.054.290) 1512 

34/51  2:24   +0.59++  1.d4 (219.542.817) 1517 

34/51  2:27   +0.52    1.d4 d5 2.c4 e6 3.cxd5 exd5 4.Nc3 c6 
                       5.Bf4 Bd6 6.Bxd6 Qxd6 7.Nf3 Bf5 
                       8.Nh4 Bg6 9.Qb3 Nd7 10.e3 Rb8 11.Be2 Ne7 
                       12.Nxg6 hxg6 13.h3 O-O 14.O-O (224.419.581) 1517 

35/48  2:35   +0.42--  1.d4 d5 (235.992.051) 1517 
==================================================

EBF = 1.39
ln(EBF) = 0.33

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

ln(EBF_Sargon) / ln(EBF_SF) = 5.1
Stockfish simply searches about 5 times deeper without losing strength at fixed depth. And that gives about 2000 Elo points difference between the two engines (fixed time).

Here are 2 games, side and reversed from the standard starting position between Stockfish and Sargon at 4 minutes + 2 seconds time control:

[pgn]
[Event "SF - Sargon"]
[Site "?"]
[Date "2020.05.29"]
[Round "1.1"]
[White "Stockfish 260520 64 BMI2"]
[Black "Sargon 1978 V1.00"]
[Result "1-0"]
[ECO "C77"]
[Annotator "1.17;0.00"]
[PlyCount "55"]
[EventDate "2020.05.29"]
[EventType "tourn"]
[TimeControl "240+2"]

{Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz 0 MHz W=52.5 ply; 5,537kN/s B=5.6
ply; 17kN/s} 1. e4 {[%eval 117,27] [%emt 0:00:05]} e5 {
[%eval 1,0] [%emt 0:00:00]} 2. Nf3 {[%eval 126,24] [%emt 0:00:03]} Nc6 {
[%eval 0,6] [%emt 0:00:08]} 3. Bb5 {(d4) [%eval 129,25] [%emt 0:00:06]} a6 {
(Nf6) [%eval 37,6] [%emt 0:00:10]} 4. Ba4 {(Bxc6) [%eval 116,25] [%emt 0:00:04]
} Nf6 {[%eval 0,6] [%emt 0:00:09]} 5. d3 {[%eval 113,27] [%emt 0:00:08]} Bb4+ {
(b5) [%eval 25,6] [%emt 0:00:08]} 6. c3 {[%eval 170,23] [%emt 0:00:03]} Bc5 {
(Bd6) [%eval 100,6] [%emt 0:00:12]} 7. Bxc6 {[%eval 189,23] [%emt 0:00:03]}
dxc6 {(bxc6) [%eval 50,6] [%emt 0:00:10]} 8. Nxe5 {
(b4) [%eval 182,26] [%emt 0:00:05]} Qe7 {(0-0) [%eval 0,5] [%emt 0:00:12]} 9.
f4 {(Bf4) [%eval 180,26] [%emt 0:00:11]} O-O {(h5) [%eval -25,5] [%emt 0:00:05]
} 10. Qe2 {(d4) [%eval 195,26] [%emt 0:00:06]} Bd6 {
(Re8) [%eval -75,5] [%emt 0:00:04]} 11. Nf3 {
(0-0) [%eval 230,25] [%emt 0:00:02]} Re8 {(Bc5) [%eval 75,5] [%emt 0:00:03]}
12. O-O {(e5) [%eval 234,25] [%emt 0:00:06]} Bg4 {
(b5) [%eval 25,5] [%emt 0:00:04]} 13. h3 {[%eval 269,24] [%emt 0:00:04]} Bxf3 {
(Bc5+) [%eval 25,5] [%emt 0:00:04]} 14. Qxf3 {
(Rxf3) [%eval 299,27] [%emt 0:00:09]} Bc5+ {[%eval 25,5] [%emt 0:00:02]} 15.
Kh1 {(d4) [%eval 316,25] [%emt 0:00:02]} Qd8 {
(Rad8) [%eval 25,5] [%emt 0:00:04]} 16. e5 {[%eval 351,29] [%emt 0:00:14]} Nd5
{[%eval 25,5] [%emt 0:00:03]} 17. d4 {[%eval 296,28] [%emt 0:00:11]} Bf8 {
[%eval -25,5] [%emt 0:00:04]} 18. Nd2 {(c4) [%eval 326,26] [%emt 0:00:06]} Be7
{(c5) [%eval -12,5] [%emt 0:00:04]} 19. f5 {(Nb3) [%eval 468,25] [%emt 0:00:03]
} Bg5 {[%eval 100,5] [%emt 0:00:03]} 20. Ne4 {[%eval 519,26] [%emt 0:00:05]}
Bxc1 {[%eval 125,5] [%emt 0:00:03]} 21. Raxc1 {[%eval 536,25] [%emt 0:00:04]}
f6 {[%eval 137,5] [%emt 0:00:03]} 22. Rce1 {(e6) [%eval 570,25] [%emt 0:00:05]}
fxe5 {(b5) [%eval 0,5] [%emt 0:00:02]} 23. Qg3 {
(Nc5) [%eval 901,29] [%emt 0:00:05]} h5 {(Nf6) [%eval 125,5] [%emt 0:00:03]}
24. f6 {[%eval 1603,25] [%emt 0:00:05]} Kf7 {(Qd7) [%eval 237,5] [%emt 0:00:11]
} 25. Qxg7+ {(fxg7+) [%eval 32763,245] [%emt 0:00:01]} Ke6 {
[%eval 32764,6] [%emt 0:00:03]} 26. Nc5+ {[%eval 32764,245] [%emt 0:00:00]} Kd6
{[%eval 32765,6] [%emt 0:00:04]} 27. Nxb7+ {[%eval 32765,245] [%emt 0:00:00]}
Ke6 {[%eval 32766,6] [%emt 0:00:03]} 28. Rxe5# {
[%eval 32766,245] [%emt 0:00:00]} 1-0
[/pgn]


[pgn]
[Event "SF - Sargon"]
[Site "?"]
[Date "2020.05.29"]
[Round "2.1"]
[White "Sargon 1978 V1.00"]
[Black "Stockfish 260520 64 BMI2"]
[Result "0-1"]
[ECO "D00"]
[Annotator "0.25;-0.77"]
[PlyCount "38"]
[EventDate "2020.05.29"]
[EventType "tourn"]
[TimeControl "240+2"]

{Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz 0 MHz W=6.0 ply; 28kN/s B=31.2 ply;
6,098kN/s} 1. d4 {[%eval 1,0] [%emt 0:00:00]} Nf6 {
[%eval -77,28] [%emt 0:00:13]} 2. Nc3 {(c4) [%eval 25,6] [%emt 0:00:08]} d5 {
[%eval -99,22] [%emt 0:00:02]} 3. Bf4 {[%eval 0,6] [%emt 0:00:08]} e6 {
(Bf5) [%eval -116,24] [%emt 0:00:02]} 4. Nf3 {(Nb5) [%eval 25,6] [%emt 0:00:10]
} Bb4 {(Nc6) [%eval -138,23] [%emt 0:00:03]} 5. Qd3 {
(e3) [%eval 25,6] [%emt 0:00:07]} Ne4 {(Nc6) [%eval -91,29] [%emt 0:00:21]} 6.
Qe3 {(Nd2) [%eval 75,5] [%emt 0:00:14]} c5 {
(Bxc3+) [%eval -177,21] [%emt 0:00:03]} 7. O-O-O {
(Nd2) [%eval 75,5] [%emt 0:00:03]} Bxc3 {[%eval -562,23] [%emt 0:00:04]} 8.
bxc3 {[%eval 75,5] [%emt 0:00:02]} Qa5 {[%eval -516,32] [%emt 0:00:13]} 9. Kb2
{(a4) [%eval 125,5] [%emt 0:00:02]} Nd7 {(Nc6) [%eval -638,26] [%emt 0:00:03]}
10. Ng5 {(dxc5) [%eval 75,5] [%emt 0:00:05]} Nb6 {
(Qb6+) [%eval -1105,28] [%emt 0:00:03]} 11. Nxe4 {[%eval -475,6] [%emt 0:00:08]
} Nc4+ {[%eval -1130,28] [%emt 0:00:03]} 12. Ka1 {[%eval -350,6] [%emt 0:00:05]
} dxe4 {(Nxe3) [%eval -1133,30] [%emt 0:00:04]} 13. Qg3 {
(Qd2) [%eval -525,6] [%emt 0:00:11]} e3 {[%eval -1197,27] [%emt 0:00:03]} 14.
Qxe3 {[%eval -562,6] [%emt 0:00:11]} Nxe3 {[%eval -1222,28] [%emt 0:00:04]} 15.
fxe3 {(Bxe3) [%eval -675,6] [%emt 0:00:04]} e5 {
(Qxc3+) [%eval -1579,25] [%emt 0:00:03]} 16. Bxe5 {
(Kb2) [%eval -625,6] [%emt 0:00:03]} Qxc3+ {[%eval -1803,27] [%emt 0:00:04]}
17. Kb1 {[%eval -725,6] [%emt 0:00:02]} Bf5 {[%eval -1863,28] [%emt 0:00:03]}
18. Rc1 {(Rd3) [%eval -725,6] [%emt 0:00:06]} Qb4+ {
(f6) [%eval -32757,57] [%emt 0:00:04]} 19. Ka1 {[%eval -725,6] [%emt 0:00:03]}
Qa3 {(f6) [%eval -32758,60] [%emt 0:00:04]} 0-1
[/pgn]

[lkaufman]

I think that Stockfish at fixed depths like 5 or 6 ply is quite awful, because it does PV pruning at low depths which makes nonsense PVs. So if you are getting close results with Sargon it's just a coincidence, the total lack of knowledge in Sargon happens to offset the horrible tactics of SF at these depths. I suspect that if you ran Komodo vs. Sargon at five or six ply it would be a massacre, even though Komodo does LMR and null move pruning and move count pruning at these depths, but not much PV pruning.

[Werewolf]

Do they hand out awards for unintentionally funny post titles?

[Laskos]

I think that Stockfish at fixed depths like 5 or 6 ply is quite awful, because it does PV pruning at low depths which makes nonsense PVs. So if you are getting close results with Sargon it's just a coincidence, the total lack of knowledge in Sargon happens to offset the horrible tactics of SF at these depths. I suspect that if you ran Komodo vs. Sargon at five or six ply it would be a massacre, even though Komodo does LMR and null move pruning and move count pruning at these depths, but not much PV pruning.

Yes, that seems the case for SF, I knew vaguely, but not that it is to such extent compared to Komodo. Komodo 14 beats handily Sargon at depth=5:

Score of Komodo_14 vs Sargon_1978: 78 - 2 - 20 [0.880] 100

... Komodo_14 playing White: 37 - 2 - 11 [0.850] 50
... Komodo_14 playing Black: 41 - 0 - 9 [0.910] 50
... White vs Black: 37 - 43 - 20 [0.470] 100
Elo difference: 346.1 +/- 79.5, LOS: 100.0 %, DrawRatio: 20.0 %
Finished match

To note that Komodo 14 also has a higher Effective Branching Factor (EBF) compared to SF. Again, rudimentarily from the starting position:

Code: Select all

rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq -

Engine: Komodo 14 64-bit (2048 MB)
by Don Dailey, Larry Kaufman, Mark Lefler
 

23.01  0:06   +0.43--  1.e4 e5 (10.160.348) 1484 

23.01  0:07   +0.35--  1.e4 e5 (10.499.191) 1486 

23.01  0:07   +0.24--  1.e4 e5 (11.218.093) 1489 

23.01  0:08   +0.27++  1.e4 e5 2.Nf3 (12.658.800) 1494 

23.01  0:08   +0.28    1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.O-O Bc5 
                       5.d3 d6 6.Bg5 Na5 7.Nc3 Nxc4 8.dxc4 Be6 
                       9.Qd3 O-O 10.Nd5 Bxd5 11.cxd5 h6 
                       12.Bxf6 (13.408.014) 1499 

24.01  0:09   +0.34++  1.e4 e5 2.Nf3 (14.747.355) 1502 

24.01  0:10   +0.32--  1.e4 e5 (15.440.768) 1505 

24.01  0:11   +0.21--  1.e4 e5 (17.493.989) 1508 

24.02  0:13   +0.34++  1.d4 Nf6 2.c4 (20.153.178) 1502 

24.01  0:15   +0.31--  1.d4 Nf6 (23.516.929) 1501 

24.01  0:16   +0.27    1.d4 Nf6 2.c4 e6 3.Nc3 Bb4 4.Nf3 O-O 
                       5.e3 b6 6.Bd2 d5 7.Bd3 Ba6 8.b3 dxc4 
                       9.bxc4 c5 10.O-O Nc6 11.a3 Ba5 
                       12.Nb5 Bb7 13.Kh1 a6 (25.170.377) 1500 

25.01  0:17   +0.21--  1.d4 Nf6 (26.865.467) 1501 

25.01  0:19   +0.23++  1.d4 Nf6 2.c4 (29.003.399) 1502 

25.01  0:19   +0.22    1.d4 Nf6 2.c4 e6 3.Nc3 Bb4 4.Nf3 O-O 
                       5.e3 b6 6.Be2 Bxc3+ 7.bxc3 Bb7 8.O-O Ne4 
                       9.Qc2 f5 10.Nd2 Nxd2 11.Bxd2 d6 
                       12.Bd3 Nd7 13.e4 f4 (29.556.521) 1501 

26.01  0:22   +0.28++  1.d4 Nf6 2.c4 (33.316.981) 1502 

26.01  0:23   +0.36++  1.d4 Nf6 2.c4 (35.103.856) 1503 

26.01  0:24   +0.34--  1.d4 Nf6 (36.641.370) 1502 

26.01  0:27   +0.37++  1.d4 Nf6 2.c4 (41.189.007) 1504 

26.01  0:28   +0.37    1.d4 Nf6 2.c4 e6 3.Nf3 b6 4.Bf4 Bb4+ 
                       5.Nbd2 Bxd2+ 6.Qxd2 Bb7 7.e3 O-O 
                       8.Bd3 d6 9.Rd1 Nbd7 10.O-O Bxf3 
                       11.gxf3 Nh5 12.Bg3 f5 13.Qc2 Rb8 
                       14.Rfe1 (43.138.288) 1503 

27.01  0:31   +0.43++  1.d4 Nf6 2.c4 (47.200.756) 1504 

27.01  0:32   +0.41--  1.d4 Nf6 (48.976.760) 1503 

27.01  0:35   +0.30--  1.d4 d5 (54.041.178) 1503 

27.01  0:39   +0.33++  1.d4 d5 2.c4 (59.621.928) 1504 

27.01  0:41   +0.36    1.d4 d5 2.c4 e6 3.Nc3 Nf6 4.Nf3 c5 
                       5.cxd5 exd5 6.Bg5 Be7 7.dxc5 O-O 
                       8.e3 Bxc5 9.Bd3 Be6 10.Rc1 Nc6 
                       11.O-O h6 12.Bxf6 Qxf6 13.Nxd5 Bxd5 (62.881.954) 1505 

28.01  0:43   +0.30--  1.d4 d5 (65.222.433) 1505 

28.01  0:46   +0.32++  1.d4 d5 2.c4 (69.573.066) 1503 

28.01  0:48   +0.30--  1.d4 d5 (73.483.605) 1504 

28.01  0:58   +0.15--  1.d4 d5 (88.362.560) 1499 

28.02  1:03   +0.33++  1.e4 e5 2.Nf3 (95.837.999) 1498 

28.01  1:05   +0.28--  1.e4 e5 (98.813.632) 1499 

28.01  1:09   +0.35++  1.e4 e5 2.Nf3 (103.759.800) 1497 

28.01  1:12   +0.38    1.e4 e5 2.Nf3 Nc6 3.Nc3 Nf6 4.Be2 Bd6 
                       5.O-O O-O 6.h3 h6 7.Bc4 Bc5 8.d3 d6 
                       9.Nd5 Nxd5 10.Bxd5 Bd7 11.c3 Ne7 
                       12.Bb3 Bb5 13.Re1 Bb6 14.d4 (108.278.468) 1497 

29.01  1:13   +0.32--  1.e4 e5 (110.513.799) 1497 
=================================================

EBF = 1.49
ln(EBF) = 0.40

ln(EBF_Sargon) / ln(EBF_Komodo) = 4.2 compared to 5.1 of Stockfish_dev
Komodo searches about 4 times deeper, gaining at the same time in strength at fixed depth (at low depths at least). Pretty amazingly effective pruning and reducing together with extensions in the current top A/B engines.

[Laskos]

Interesting to note that to depth=5, Sargon 1978 needs 29,116 nodes which SF_dev matches in strength to depth=5 using only 671 nodes and Komodo 14 betters by 300+ Elo points using only 1,825 nodes. It is pretty clear that without a sophisticated evaluation function, that is impossible to achieve.

[jjoshua2]

Also though the title was kind of funny/surprsing. What is the EFB and nodes it takes for lc0 to equal sargon depth 5? I think maybe even a single node of the biggest networks could be close to equal?

[mclane]

Stockfish on 1000 nodes per move is not strong

[Laskos]

Also though the title was kind of funny/surprsing. What is the EFB and nodes it takes for lc0 to equal sargon depth 5? I think maybe even a single node of the biggest networks could be close to equal?

1 node Leela with SV_3200 384x30 net is significantly stronger than Sargon depth 5, by some 300 Elo points.

Code: Select all

Score of Lc0_Net3200 nodes=1 vs Sargon_1978 depth=5: 32 - 4 - 4  [0.850] 40

...      Lc0_Net3200 playing White: 17 - 1 - 2  [0.900] 20
...      Lc0_Net3200 playing Black: 15 - 3 - 2  [0.800] 20
...      White vs Black: 20 - 16 - 4  [0.550] 40
Elo difference: 301.3 +/- 158.2, LOS: 100.0 %, DrawRatio: 10.0 %
Finished match  

Sargon depth=5 needs many dozens of thousands of nodes to reach this depth and was pretty much LTC in 1978. About Effective Branching Factor (EBF), isn't it superfluous for Lc0 MCTS search, with depth some arbitrary logarithm of nodes? For A/B engines we still have the iterative search and depth still means something (although the meaning is blurred even there). Anyway, I computed the EBF for Lc0 v25.1 with a 256x20 net (maybe even one of your free nets):

Code: Select all

info depth 15 seldepth 48 time 4724 nodes 35660 score cp 21 nps 16555 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 b1c3 f8e7 h2h3 f5h4 f1d1 d8e8 f3h4 e7h4 g2g4 a7a5 a2a4 b7b6 c1f4 e8e7 g1g2 h7h5
info depth 15 seldepth 49 time 5487 nodes 51700 score cp 21 nps 17717 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 b1c3 f8e7 h2h3 f5h4 f1d1 d8e8 f3h4 e7h4 g2g4 a7a5 a2a4 b7b6 c1f4 e8e7 g1g2 h7h5 f2f3 e7e6
info depth 16 seldepth 49 time 5861 nodes 59316 score cp 21 nps 18018 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 b1c3 d8e8 h2h3 h7h5 c1g5 c8e6 f1d1 f8e7 d1d2 a8d8 d2d8 e8d8 a1d1 d8e8 b2b3 h5h4
info depth 16 seldepth 50 time 6163 nodes 65329 score cp 21 nps 18177 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 b1c3 d8e8 h2h3 h7h5 c1g5 c8e6 f1d1 f8e7 d1d2 a8d8 d2d8 e8d8 a1d1 d8e8 b2b3 h5h4 g5f4
info depth 16 seldepth 51 time 6254 nodes 67163 score cp 21 nps 18226 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 h2h3 f8e7 b1c3 f5h4 f1d1 d8e8 f3h4 e7h4 g2g4 a7a5 a2a4 b7b6 c1f4 e8e7 g1g2 h7h5 f2f3 e7e6 c3e2 c6c5
info depth 16 seldepth 52 time 7176 nodes 87165 score cp 20 nps 18920 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 h2h3 f8e7 b1c3 f5h4 f1d1 d8e8 f3h4 e7h4 g2g4 a7a5 a2a4 b7b6 c1f4 e8e7 g1g2 h7h5 f2f3 e7e6 c3e2 c6c5
info depth 16 seldepth 53 time 7864 nodes 102503 score cp 20 nps 19358 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 h2h3 f8e7 b1c3 f5h4 f1d1 d8e8 f3h4 e7h4 g2g4 a7a5 a2a4 b7b6 c1f4 e8e7 g1g2 h7h5 f2f3 e7e6 c3e2 c6c5
info depth 17 seldepth 53 time 8008 nodes 105039 score cp 20 nps 19315 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 h2h3 f8e7 b1c3 f5h4 f1d1 d8e8 f3h4 e7h4 g2g4 a7a5 a2a4 b7b6 c1f4 e8e7 g1g2 h7h5 f2f3 e7e6 c3e2 c6c5
info depth 17 seldepth 54 time 8034 nodes 105799 score cp 20 nps 19359 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 h2h3 f8e7 b1c3 f5h4 f1d1 d8e8 f3h4 e7h4 g2g4 a7a5 a2a4 b7b6 c1f4 e8e7 g1g2 h7h5 f2f3 e7e6 c3e2 c6c5
info depth 17 seldepth 55 time 9043 nodes 128845 score cp 20 nps 19901 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 h2h3 f8e7 b1c3 f5h4 f1d1 d8e8 f3h4 e7h4 g2g4 a7a5 a2a4 b7b6 c1f4 e8e7 g1g2 h7h5 f2f3 e7e6 c3e2 c6c5
info depth 18 seldepth 55 time 12388 nodes 210792 score cp 19 nps 21467 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 h2h3 f8e7 b1c3 f5h4 f1d1 d8e8 f3h4 e7h4 g2g4 a7a5 a2a4 b7b6 c1f4 e8e7 g1g2 h7h5 f2f3 e7e6 c3e2 c6c5
info depth 17 seldepth 55 time 12400 nodes 211152 score cp 19 nps 21478 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 h2h3 f8e7 b1c3 f5h4 f1d1 d8e8 f3h4 e7h4 g2g4 a7a5 a2a4 b7b6 c1f4 e8e7 g1g2 h7h5 f2f3 e7e6 c3e2 c6c5
info depth 18 seldepth 55 time 12426 nodes 211656 score cp 19 nps 21472 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 h2h3 f8e7 b1c3 f5h4 f1d1 d8e8 f3h4 e7h4 g2g4 a7a5 a2a4 b7b6 c1f4 e8e7 g1g2 h7h5 f2f3 e7e6 c3e2 c6c5
info depth 18 seldepth 55 time 17446 nodes 333843 score cp 18 nps 22440 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 h2h3 h7h5 b1c3 f8e7 c1f4 f5h4 f3h4 e7h4 a1d1 d8e8 c3e2 h4e7 e2d4 g7g5 f4h2 h8h6 f1e1 c8d7 e5e6 d7e6
info depth 19 seldepth 55 time 19216 nodes 376660 score cp 18 nps 22626 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 h2h3 h7h5 b1c3 f8e7 c1f4 f5h4 f3h4 e7h4 a1d1 d8e8 c3e2 h4e7 e2d4 g7g5 f4h2 g5g4 h3g4 h5g4 f1e1 h8h6 h2f4
info depth 18 seldepth 55 time 19225 nodes 377017 score cp 18 nps 22635 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 h2h3 h7h5 b1c3 f8e7 c1f4 f5h4 f3h4 e7h4 a1d1 d8e8 c3e2 h4e7 e2d4 g7g5 f4h2 g5g4 h3g4 h5g4 f1e1 h8h6 h2f4
info depth 19 seldepth 55 time 19239 nodes 377337 score cp 18 nps 22637 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 d2d4 e4d6 b5c6 d7c6 d4e5 d6f5 d1d8 e8d8 h2h3 h7h5 b1c3 f8e7 c1f4 f5h4 f3h4 e7h4 a1d1 d8e8 c3e2 h4e7 e2d4 g7g5 f4h2 g5g4 h3g4 h5g4 f1e1 h8h6 h2f4
info depth 19 seldepth 55 time 24249 nodes 503588 score cp 18 nps 23228 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 f1e1 e4d6 f3e5 f8e7 b5f1 c6e5 e1e5 e8g8 d2d4 e7f6 e5e1 f8e8 c2c3 e8e1 d1e1 d6e8 c1f4 d7d5 b1d2 c7c6 f1d3 g7g6 d2f3 e8g7 f4h6 g7f5 d3f5 c8f5 e1e3 d8e7 e3f4 a8e8 g2g4 f5e4 a1e1 f6g7 h6g7 g8g7 f3d2
info depth 19 seldepth 55 time 29258 nodes 629619 score cp 18 nps 23591 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 f1e1 e4d6 f3e5 f8e7 b5f1 c6e5 e1e5 e8g8 d2d4 e7f6 e5e1 f8e8 c2c3 e8e1 d1e1 d6e8 c1f4 d7d5 b1d2 c7c6 f1d3 g7g6 d2f3 e8g7 f4h6 g7f5 d3f5 c8f5 e1e3 d8e7 e3f4 a8e8 g2g4 f5e4 a1e1 f6g7 h6g7 g8g7 f3d2 g6g5
info depth 20 seldepth 55 time 30250 nodes 653163 score cp 18 nps 23596 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 f1e1 e4d6 f3e5 f8e7 b5f1 c6e5 e1e5 e8g8 d2d4 e7f6 e5e1 f8e8 c2c3 e8e1 d1e1 d6e8 c1f4 d7d5 b1d2 c7c6 f1d3 g7g6 d2f3 e8g7 f4h6 g7f5 d3f5 c8f5 e1e3 d8e7 e3f4 a8e8 g2g4 f5e4 a1e1 f6g7 h6g7 g8g7 f3d2 g6g5 f4g3
info depth 20 seldepth 56 time 34343 nodes 758602 score cp 17 nps 23874 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 f1e1 e4d6 f3e5 f8e7 b5f1 c6e5 e1e5 e8g8 d2d4 e7f6 e5e1 f8e8 c2c3 e8e1 d1e1 d6e8 c1f4 d7d5 b1d2 c7c6 f1d3 g7g6 d2f3 e8g7 f4h6 g7f5 d3f5 c8f5 e1e3 d8e7 e3f4 a8e8 g2g4 f5e4 a1e1 f6g7 h6g7 g8g7 f3d2 g6g5 f4g3 e7d7
info depth 20 seldepth 57 time 37164 nodes 832068 score cp 17 nps 24051 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 f1e1 e4d6 f3e5 f8e7 b5f1 c6e5 e1e5 e8g8 d2d4 e7f6 e5e1 f8e8 c2c3 e8e1 d1e1 d6e8 c1f4 d7d5 b1d2 c7c6 f1d3 g7g6 d2f3 e8g7 f4h6 g7f5 d3f5 c8f5 e1e3 d8e7 e3f4 a8e8 g2g4 f5e4 a1e1 f6g7 h6g7 g8g7 f3d2 g6g5 f4g3 e7d7
info depth 20 seldepth 57 time 42169 nodes 966118 score cp 17 nps 24396 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 f1e1 e4d6 f3e5 f8e7 b5f1 c6e5 e1e5 e8g8 d2d4 e7f6 e5e1 f8e8 c2c3 e8e1 d1e1 d6e8 c1f4 d7d5 b1d2 c7c6 f1d3 g7g6 d2f3 e8g7 f4h6 g7f5 d3f5 c8f5 e1e3 d8e7 e3f4 a8e8 h2h3 e7d8 f3e5 f6h8 g2g4 f7f6 e5c6 b7c6
info depth 20 seldepth 57 time 47183 nodes 1093282 score cp 17 nps 24505 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 f1e1 e4d6 f3e5 f8e7 b5f1 c6e5 e1e5 e8g8 d2d4 e7f6 e5e1 f8e8 c2c3 e8e1 d1e1 d6e8 c1f4 d7d5 b1d2 c7c6 f1d3 g7g6 d2f3 e8g7 f4h6 g7f5 d3f5 c8f5 e1e3 d8e7 e3f4 a8e8 g2g4 f5e4 a1e1 f6g7 h6g7 g8g7 f3d2 g6g5 f4g3 e7d7
info depth 20 seldepth 58 time 48770 nodes 1133760 score cp 17 nps 24539 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 f1e1 e4d6 f3e5 f8e7 b5f1 c6e5 e1e5 e8g8 d2d4 e7f6 e5e1 f8e8 c2c3 e8e1 d1e1 d6e8 c1f4 d7d5 b1d2 c7c6 f1d3 g7g6 d2f3 e8g7 f4h6 g7f5 d3f5 c8f5 e1e3 d8e7 e3f4 a8e8 g2g4 f5e4 a1e1 f6g7 h6g7 g8g7 f3d2 g6g5 f4g3 e7d7
info depth 21 seldepth 58 time 49302 nodes 1148567 score cp 17 nps 24577 tbhits 0 pv e2e4 e7e5 g1f3 b8c6 f1b5 g8f6 e1g1 f6e4 f1e1 e4d6 f3e5 f8e7 b5f1 c6e5 e1e5 e8g8 d2d4 e7f6 e5e1 f8e8 c2c3 e8e1 d1e1 d6e8 c1f4 d7d5 b1d2 c7c6 f1d3 g7g6 d2f3 e8g7 f4h6 g7f5 d3f5 c8f5 e1e3 d8e7 e3f4 a8e8 g2g4 f5e4 a1e1 f6g7 h6g7 g8g7 f3d2 g6g5 f4g3 e7d7

EBF = 1.78

It is larger than that of SF or Komodo, so the depths are lower. But I think it's anyway an arbitrary quantity for Lc0.

[Uri Blass]

I do not see the point of comparing stockfish with sargon at small fixed depth.

Stockfish is not optimized to play well at small fixed depth and does some strange pruning that no human does.
I do not see the point of using it at fixed depth of 5 plies when it believes that after 1.e4 d5 2.exd5 Qxd5 white wins the queen by 3.Nc3



FEN: rnb1kbnr/ppp1pppp/8/3q4/8/8/PPPP1PPP/RNBQKBNR w KQkq - 0 3

Stockfish_20052607_x64_modern:
1/1 00:00 395 198k +0.92 3.Qe2
2/2 00:00 564 282k +9.08 3.Nc3 c6
3/3 00:00 708 354k +9.10 3.Nc3 e6 4.Nxd5
4/4 00:00 865 433k +9.16 3.Nc3 e6 4.Nxd5 exd5 5.Bb5+ Nc6
5/6 00:00 1k 520k +9.38 3.Nc3 e6 4.Nxd5 exd5
6/7 00:00 2k 842k +9.04 3.Nc3 Be6 4.Nxd5 Bxd5 5.h4

I would like to see a different strong engine that try to search like a human.
Unfortunately the stockfish team care only about elo and not about trying to use some smart algorithm that humans can learn from it useful things.

If humans try to look at stockfish's tree to learn how to search and see lines like 3,Nc3 e6 they can get the conclusion that stockfish is a very weak and useless engine.

[Dann Corbit]

I don't think the weak 6 ply searches matter at all.
You can get to 15 plies in an eyeblink.
If you like ultra-hyper-bullet game in one tenth of a second, then use Komodo.
I am not sure how the NN programs would do at ludicrously short time control.
It would be interesting to see an LC0/Komodo match at very high speed.

For reasonable time control on both play and analysis, Stockfish does very well indeed.