THE SSDF RATING LIST 2022-01-06 - In memory of Guy Haworth

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

Lars Sandin
Posts: 2032
Joined: Sun Nov 18, 2007 5:17 pm
Location: Sundsvall, Sweden

THE SSDF RATING LIST 2022-01-06 - In memory of Guy Haworth

Post by Lars Sandin »

Code: Select all

THE SSDF RATING LIST 2022-01-06    158725 games played by  418 computers							
                                           Rating   +	  -  Games   Won  Oppo
                                           ------  ---   --- -----   ---  ----
   1 Stockfish 13 x64 1800X 3.6 GHz          3578   42   -38   360   71%  3424
   2 Lc0 0.26.3 Cuda(67362) 3060Ti           3573   29   -27   680   68%  3445
   3 Dragon Komodo 2.51 x64 1800X 3.6 GHz    3569   49   -45   240   66%  3436
   4 Stockfish 12 NNUE x64 1800X 3.6 GHz     3558   30   -29   560   62%  3471
   5 Stockfish 14 x64 1800X 3.6 GHz          3554   51   -48   202   59%  3502
   6 Dragon by Komodo x64 1800X 3.6 GHz      3547   34   -32   460   63%  3456
   7 Stockfish 11 x64 1800X 3.6 GHz          3540   36   -34   450   70%  3398
   8 Stockfish 10 x64 1800X 3.6 GHz          3516   25   -24   880   68%  3383
   9 Dragon Komodo 2 MCTS x64 1800X 3.6 GHz  3482   45   -44   240   55%  3453
  10 Stockfish 9 x64 1800X 3.6 GHz           3477   26   -24   882   70%  3333
  11 Komodo 13.1 x64 1800X 3.6 GHz           3465   30   -29   560   62%  3383
  12 Komodo 14 x64 1800X 3.6 GHz             3464   33   -32   440   55%  3434
  13 Komodo 13.02 x64 1800X 3.6 GHz          3458   30   -29   600   65%  3349
  14 Komodo 12.3 x64 1800X 3.6 GHz           3449   27   -26   760   66%  3332
  15 Stockfish 9 x64 Q6600 2.4 GHz           3441   32   -31   480   56%  3398
  16 Arasan 23.01 x64 1800X 3.6 GHz          3437   45   -44   240   53%  3428
  17 Booot 6.5 x64 1800X 3.6 GHz             3432   48   -50   200   44%  3486
  18 Pedone 3.1 x64 1800X 3.6 GHz            3423   46   -47   220   48%  3420
  19 Dragon Komodo MCTS x64 1800X 3.6 GHz    3415   41   -41   280   52%  3404
  20 Pedone 3 x64 1800X 3.6 GHz              3386   44   -45   240   43%  3436
  21 Wasp 5 x64 1800X 3.6 GHz                3378   63   -73   120   32%  3526
  22 Pedone 3.1 x64 Q6600 2.4 GHZ            3376   45   -43   260   62%  3285
  23 Booot 6.4 x64 1800X 3.6 GHz             3368   36   -36   360   51%  3364
  24 Arasan 22.3 x64 1800X 3.6 GHz           3363   45   -49   240   35%  3467
  25 Booot 6.5 x64 Q6600 2.4 GHz             3359   48   -44   240   63%  3267
  26 Deep Shredder 13 x64 1800X 3.6 GHz      3355   24   -24   880   64%  3255
  27 Marvin 5.1.0 x64 1800X 3.6 GHz          3324   44   -47   240   37%  3416
  28 Pedone 3 x64 Q6600 2.4 GHz              3315   45   -43   247   57%  3263
  29 Booot 6.3.1 x64 Q6600 2.4 GHz           3291   30   -30   520   53%  3268
  30 Vajolet2 2.8 x64 1800X 3.6 GHz          3291   27   -28   650   38%  3376
  31 Arasan 21.2 x64 1800X 3.6 GHz           3285   25   -26   800   36%  3387
  32 Wasp 4.5 x64 1800X 3.6 GHz              3264   64   -83   140   21%  3512
  33 Wasp 4 x64 1800X 3.6 GHz                3255   42   -45   280   33%  3377
  34 Marvin 5.1.0 x64 Q6600 2.4 GHz          3225   56   -58   147   47%  3254
  35 Deep Hiarcs 14 1800X 3.6 GHz            3213   23   -24   880   38%  3298
  36 Deep Rybka 4 x64 Q6600 2.4 GHz          3195   19   -18  1488   63%  3103
  37 Revelation 2 Hiarcs 14.1 PXA320 800 MHz 2925   47   -45   228   56%  2882
  38 Chessmaster King 3.5 x64 Q6600 2.4 GHz  2859   24   -25   932   30%  3008
  39 Revelation Hiarcs 13.3 PXA255 500 MHz   2772   57   -52   177   66%  2660
  40 Revelation Shredder 12 PXA255 500 MHz   2705   60   -58   140   56%  2665
  41 Revelation Rybka 2.2 PXA255 500 MHz     2634   45   -43   260   63%  2546
  42 Revelation Deep Sjeng 3 PXA255 500 MHz  2600   62   -67   120   41%  2664
  43 Millennium The King Exclusive 300 MHz   2535   52   -51   180   56%  2495
  44 Revelation Ruffian 2.1 PXA255 500 MHz   2346   68   -71   100   45%  2384
  45 Millennium ChessGenius Excl. M7 300 MHz 2247   54   -51   180   59%  2183
  46 Mephisto London 68030 33 MHz            2194   31   -31   482   50%  2191
  47 Millennium ChessGenius Pro M4 120 MHz   2162   59   -54   160   63%  2066
  48 Mephisto London 68020 12 MHz            2094   67   -60   131   65%  1987
  49 Millennium ChessGenius ARM M4 48 MHz    2069   45   -43   251   58%  2015
  50 Mephisto London 68000 12 MHz            2004   59   -58   140   53%  1981

 3 Dragon Komodo 2.51 x64 1800X 3.6 GHz, 3569											
LC0 3060Ti      19,5-20,5  Stoc14 1800X      20-20    Arasa23 1800X   25,5-14,5
Boo65 1800X       28-12    Wasp5 1800X       31-9     Wasp45 1800X    35,5-4,5 

 5 Stockfish 14 x64 1800X 3.6 GHz, 3554											
LC0 3060Ti        19-21    Dra251 1800X      20-20    SF12NU 1800X    21,5-18,5
Ped31 1800X       26-14    Komo11 1800X       2-0     Arasa22 1800X     31-9 

 16 Arasan 23.01 x64 1800X 3.6 GHz, 3437											
LC0 3060Ti      14,5-25,5  Dra251 1800X    14,5-25,5  Ped31 1800X       19-21
Arasa22 1800X   23,5-16,5  Mar51 1800X     24,5-15,5  Ped21 1800X       31-9 

 17 Booot 6.5 x64 1800X 3.6 GHz, 3432											
Stoc13 1800X    10,5-29,5  LC0 3060Ti        13-27    Dra251 1800X      12-28
Arasa22 1800X     25-15    Mar51 1800X     26,5-13,5  

 18 Pedone 3.1 x64 1800X 3.6 GHz, 3423											
LC0 3060Ti         9-31    Stoc14 1800X      14-26    Arasa23 1800X     21-19
Wasp5 1800X       21-19    Wasp45 1800X      41-19    

 21 Wasp 5 x64 1800X 3.6 GHz, 3378											
LC0 3060Ti        10-30    Dra251 1800X       9-31    Ped31 1800X       19-21

 22 Pedone 3.1 x64 Q6600 2.4 GHZ, 3376											
Komo12 Q6600      18-22    Boo65 Q6600       22-18    DShre13 Q6600     22-18
Arasa21 Q6600   26,5-13,5  Mar51 Q6600     45,5-14,5  DRybka3 Q6600     28-12

 24 Arasan 22.3 x64 1800X 3.6 GHz, 3363											
LC0 3060Ti         8-32    Stoc14 1800X       9-31    Dra2MC 1800X    11,5-28,5
Arasa23 1800X   16,5-23,5  Boo65 1800X       15-25    Mar51 1800X       23-17

 25 Booot 6.5 x64 Q6600 2.4 GHz, 3359											
Ped31 Q6600       18-22    Ped3 Q6600      22,5-17,5  DShre13 Q6600     25-15
Arasa21 Q6600     26-14    Chiro3 Q6600    60,5-19,5  

 27 Marvin 5.1.0 x64 1800X 3.6 GHz, 3324											
LC0 3060Ti         6-34    Ko14 1800X         9-31    Arasa23 1800X   15,5-24,5
Boo65 1800X     13,5-26,5  Arasa22 1800X     17-23    DHiar14 1800X     28-12

 33 Wasp 4.5 x64 1800X 3.6 GHz, 3264											
LC0 3060Ti       5,5-34,5  Dra251 1800X     4,5-35,5  Ped31 1800X       19-41

 35 Marvin 5.1.0 x64 Q6600 2.4 GHz, 3225											
Ped31 Q6600     14,5-45,5  Ped3 Q6600       2,5-4,5   DRybka4 Q6600     24-16
Spike14 Q6600   27,5-12,5  

This last list of 2021 and first list of 2022, will be dedicated in honor of the
late Guy Haworth.
 
On this new years list, we can present seven new programs on our two hardware
levels. 

First one out is Mark Lefler and Larry Kaufman's latest version of Dragon  
Komodo, named 2.51. We have initially tested the non-MCTS version and after 
the first 240 games it has reached a rating of 3569, that is 22 points stronger
than the last non-MCTS Dragon Komodo we tested. It is now just four points 
behind the leader from the last rating list - Lc0. It is also nine points below
Stockfish 13 - which now has taken over the leader spot in the rating list. 
Dragon Komodo 2.51 is now more than 100 points stronger than the best non-NNUE 
version of Komodo we have tested! As before we have used the opening book 
"out10-35.bin" of Erdogan Gunes for the testing of Dragon Komodo.  

Next one out is the latest creation from team Stockfish, namely Stockfish 14. 
After the first 202 games it has reached a rating of 3554, which is 24 points
behind Stockfish 13 in first place, and 4 points behind Stockfish 12.
For the testing of Stockfish 14 we have used the opening book by Fauzi Dabat 
named "Aggressive 5.0 by Fauzi.abk". More games will probably be needed to
stabilise the rating against the other engines in the top, lower the error 
bars and see if it will be able to supersede the older version as more games
are played.

We have also tested two new engines from Jon Dart, named Arasan 22.3 and the 
neural network architecture (NNUE) version named 23.01. For the testing of 
these two engines, we have used Arasan's own opening book. Arasan 22.3 has 
reached the rating of 3363 after the 240 games. This is 78 points stronger 
than the last Arasan 21.2 version we have tested. The NNUE version: 
Arasan 23.01, has then gained a further 74 points on that, resulting in the
rating of 3437 after the first 240 games.

We also have the latest (and last) free Pedone-version in this rating list. 
It is Pedone 3.1 by Fabio Gobbato. Like the 3.0-version of the same program,
it uses NNUE in the search. We have tested it on both our hardware levels. 
On the 1800X - Pedone 3.1 has reached a rating of 3423 after 220 games 
played. On the Q6600 hardware, Pedone 3.1 has achieved a rating of 3376 after
260 games. The difference between the two hardware is 47 points, and 
Pedone 3.1 on the 1800X is 37 points stronger than Pedone 3.0 1800X. 
On the Q6600, the difference between Pedone 3.1 and 3.0 is 53 points. 
We have used Pedone's own opening book for the testing of Pedone 3.1.    

We can also present a rating of Alex Morozov's latest Booot program, named 
Booot 6.5. We have tested this program on two of our hardware levels. On
our 1800X hardware, Booot 6.5 has received a rating of 3432 after 200 games. 
On our Q6600 hardware, Booot 6.5 has reached a rating of 3359 after 240 
games. The difference between the two different hardware is 73 points. 
The 6.5-version is 64 points ahead of the formerly tested Booot 6.4 
on the 1800X and on the Q6600, the difference between Booot 6.5 and 
Booot 6.3.1 is 68 points. We have used Sedat Canbaz's "Perfect2021.abk"
for the testing of Booot 6.5.

We are also able to welcome a Swedish newcomer in this rating list! 
Probably the first Swedish program in the list after Per Ola Valfridsson's - 
Ruffian, I guess?  
It is Martin Danielsson's engine: Marvin 5.1.0, which we have tested on both 
our hardware levels. We have used Marvin's own opening book for the testing. 
Marvin is from the 5.0.0-version a neural network architecture (NNUE)-program.
On our 1800X hardware, Marvin 5.1.0 has reached a rating of 3324 after 
240 games. On the Q6600 hardware, Marvin 5.1.0 has gotten a rating of 
3225 after the first 147 games. The difference between the two hardware is 
just shy of 100 points. 

Last, but not least, is the two Wasp-programs from John Stanback. The first one 
is Wasp 4.5, which uses ordinary search, and has received a rating of 3264 
after the first 140 games. This is nine points ahead of the formerly tested 
Wasp 4. Wasp 5.0 has introduced NNUE in its search. This has proven to be 
valuable for Wasp (as for other NNUE engines), and Wasp 5.0 1800X has 
received a rating of 3378 after the first 120 games. Albeit it's early in 
the testing, the difference between the two versions is now 114 points at 
least!

Lars Sandin        
Lars Sandin, SSDF
Jouni
Posts: 3298
Joined: Wed Mar 08, 2006 8:15 pm

Re: THE SSDF RATING LIST 2022-01-06 - In memory of Guy Haworth

Post by Jouni »

8 cores Stockfish loss against Lc0 is weird. In others tests 1 core SF beats Lc0. I looked at games http://bozze.hopto.org/ssdf/games/. There was 38 draws and 2 losses. White loss has weird ?? blunder. And black lose was from +1 opening.
[pgn]
[Event "SSDF-match"]
[Site "Sundsvall"]
[Date "2021.09.26"]
[Round "10"]
[White "Stockfish 14 x64 1800X"]
[Black "Lc0 0.26.3 Cuda(67362) 3060Ti"]
[Result "0-1"]
[ECO "B80"]
[BlackElo "3581"]
[Annotator "Sandin,Lars"]
[PlyCount "168"]
[EventDate "2021.09.23"]
[EventType "match"]
[EventRounds "40"]
[EventCountry "SWE"]
[TimeControl "40/7200:20/3600:20/3600"]

1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 a6 6. Be3 e6 7. f3 b5 8. Qd2
Nbd7 {-0.27/22 138} 9. a3 h5 {-0.13/15 135} 10. O-O-O Bb7 {-0.14/19 0} 11. Rg1
Rc8 {-0.12/20 147} 12. Kb1 Be7 {-0.11/19 60} 13. Be2 Ne5 {-0.11/19 12} 14. Bg5
Ng6 {-0.08/23 382} 15. Rge1 O-O {-0.06/25 312} 16. f4 Nxe4 {-0.06/27 142} 17.
Nxe4 Bxe4 {-0.06/25 62} 18. Bxh5 Bxg5 {-0.03/24 503} 19. fxg5 d5 {-0.03/23 49}
20. Bf3 Qc7 {-0.02/24 1} 21. Bxe4 dxe4 {-0.02/21 227} 22. Rxe4 Rfd8 {0.01/19
463} 23. Qe2 Qxh2 {0.00/17 132} 24. Rf1 Qd6 {0.00/16 137} 25. Rh1 Qd5 {0.00/16
479} 26. Nf3 Qf5 {0.00/16 262} 27. g3 Qd5 {0.02/14 309} 28. Nh4 Qxg5 {0.02/21
351} 29. Nxg6 Qxg6 {0.00/28 1} 30. Reh4 Kf8 {0.00/26 0} 31. Qg2 Rc7 {0.00/25 0}
32. Rh8+ Ke7 {-0.01/1 0} 33. Rxd8 Kxd8 {-0.01/21 1} 34. Rd1+ Rd7 {-0.01/20 0}
35. Qa8+ Ke7 {-0.01/18 0} 36. Rxd7+ Kxd7 {-0.02/17 0} 37. Qxa6 Qf5 {-0.01/15
316} 38. Qa7+ Kc8 {-0.01/13 721} 39. b3 g5 {0.00/13 498} 40. Qa6+ Kc7 {0.00/12
857} 41. a4 bxa4 {0.00/12 314} 42. Qa7+ Kd6 {0.00/11 687} 43. Qb6+ Kd7 {0.00/
11 46} 44. bxa4 Qe5 {0.01/12 201} 45. a5 Qxg3 {0.03/16 371} 46. Qb7+ Qc7 {0.04/
12 165} 47. Qb5+ Qc6 {0.02/11 325} 48. Qxg5 f5 {0.02/10 0} 49. Qd2+ Kc7 {0.01/
9 315} 50. Qb4 Qd6 {0.01/8 275} 51. Qc4+ Kb8 {0.02/10 154} 52. Kc1 f4 {0.05/10
129} 53. Qb3+ Ka7 {0.05/10 100} 54. Qf3 e5 {0.05/10 152} 55. c4 Qd4 {0.04/13
130} 56. Qd5 Qc3+ {0.04/11 115} 57. Kd1 f3 {0.04/10 0} 58. Qd7+ Kb8 {0.02/8 167
} 59. Qd8+ Kb7 {0.02/9 209} 60. Qd7+ Ka8 {0.02/11 192} 61. Qc8+ Ka7 {0.01/1 0}
62. Qc7+ Ka8 {0.00/5 374} 63. Qc8+ Ka7 {0.05/1 0} 64. Qc7+ Ka8 {0.00/3 394} 65.
Qd8+ Kb7 {0.01/6 628} 66. Qd5+ Kc8 {0.03/13 407} 67. Qe6+ Kd8 {0.03/10 370} 68.
Qd6+ Ke8 {0.03/10 198} 69. Qe6+ Kd8 {0.03/12 163} 70. Qd6+ Ke8 {0.03/11 141}
71. Qg6+ Kd7 {0.03/11 110} 72. Qf7+ Kc8 {0.02/13 60} 73. Qe8+ Kc7 {0.03/10 122}
74. Qe7+ Kc6 {0.03/8 130} 75. Qe6+ Kb7 {0.02/13 0} 76. Qb6+ Kc8 {0.02/7 105}
77. Qc6+ Kb8 {0.01/6 88} 78. Qd6+ Kb7 {0.02/8 138} 79. Qd7+ Ka6 {0.01/5 83} 80.
c5 Qd4+ {#118/1 0} 81. Qxd4 exd4 {#117/1 0} 82. Ke1 d3 {#114/1 0} 83. c6 Ka7 {
#115/2 0} 84. Kf2 d2 {White resigns} 0-1

[/pgn]

[pgn]
[Event "SSDF-match"]
[Site "Sundsvall"]
[Date "2021.09.30"]
[Round "19"]
[White "Lc0 0.26.3 Cuda(67362) 3060Ti"]
[Black "Stockfish 14 x64 1800X"]
[Result "1-0"]
[ECO "E97"]
[WhiteElo "3581"]
[Annotator "Sandin,Lars"]
[PlyCount "145"]
[EventDate "2021.09.23"]
[EventType "match"]
[EventRounds "40"]
[EventCountry "SWE"]
[TimeControl "40/7200:20/3600:20/3600"]

1. d4 Nf6 2. c4 g6 3. Nc3 Bg7 4. e4 d6 5. Nf3 O-O 6. Be2 e5 7. O-O Nc6 8. d5
Ne7 9. b4 {0.34/13 104} Nh5 10. Re1 {0.41/15 153} f5 11. a4 {0.41/17 127} a5
12. bxa5 {0.41/17 0} Rxa5 13. Bd2 {0.47/24 472} Nf6 14. Ng5 {0.48/17 75} b6 15.
exf5 {0.53/17 262} Nxf5 16. Bd3 {0.49/16 217} h6 17. Ne6 {0.64/27 256} Bxe6 18.
dxe6 {0.61/31 77} Qe7 19. Bxf5 {0.65/33 130} gxf5 20. Nd5 {0.67/20 32} Nxd5 21.
Bxa5 {0.65/23 0} Nf4 22. Bc3 {0.65/21 0} Nxe6 23. a5 {0.59/19 1239} bxa5 24.
Rxa5 {0.59/16 0} Kh7 25. Kh1 {0.58/18 0} Qh4 26. Qe2 {0.68/15 330} Bf6 27. Rg1
{0.72/12 361} Rf7 28. f3 {0.70/12 517} Qh5 29. Ra3 {0.69/13 124} Re7 30. Qd3 {
0.68/12 344} Qg6 31. Qd1 {0.67/16 0} Nf4 32. Bd2 {0.73/15 255} Ne6 33. Be3 {0.
75/21 54} f4 34. Bf2 {0.73/20 0} e4 35. fxe4 {0.72/24 12} Qxe4 36. Ra4 {0.71/
22 0} Qf5 37. Rf1 {0.72/23 1} Rf7 38. Qd5 {0.77/20 649} Bg7 39. Bg1 {0.81/25
435} h5 40. g3 {0.81/28 0} Be5 41. Ra2 {0.82/30 297} Qg6 42. gxf4 {0.82/33 1}
Nxf4 43. Qc6 {0.86/27 389} Qd3 44. Rf3 {0.93/27 721} Qb1 45. Raf2 {0.92/30 0}
Qg6 46. Rg3 {0.90/29 4} Qf5 47. Qf3 {0.90/27 0} Rg7 48. Rxg7+ {0.90/24 0} Kxg7
49. Rf1 {0.90/31 2} Qg4 50. Bd4 {0.89/28 654} Qxf3+ 51. Rxf3 {0.88/27 0} Ng6
52. Be3 {1.65/20 606} Nh4 53. Rf2 {1.88/21 170} Kg6 54. Bc1 {1.94/19 1} Bd4 55.
Rf8 {2.09/23 384} Be5 56. h3 {2.09/8 0} Nf5 57. Kg2 {3.33/16 356} Ng7 58. Rh8 {
3.60/16 127} Kf5 59. Kf3 {3.87/16 217} Bf6 60. Be3 {3.92/15 72} Ke5 61. Bf2 {
4.02/16 73} Kf5 62. Be1 {4.21/15 166} Kg6 63. Rg8 {4.70/15 497} Kf5 64. Bh4 {
4.85/16 44} Be5 65. Rf8+ {5.67/18 257} Kg6 66. Ke4 {6.73/14 368} Ne6 67. Rf1 {
7.25/14 35} Ng7 68. c5 {10.68/12 608} Ne6 69. c6 {11.29/12 158} Nc5+ 70. Kd5 {
11.87/12 266} Nd3 71. Ke6 {12.41/12 162} Nc5+ 72. Ke7 {12.12/12 101} d5 73. Rd1
{Black resigns} 1-0

[/pgn]
Jouni
mehmet123
Posts: 671
Joined: Sun Jan 26, 2020 10:38 pm
Location: Turkey
Full name: Mehmet Karaman

Re: THE SSDF RATING LIST 2022-01-06 - In memory of Guy Haworth

Post by mehmet123 »

Jouni wrote: Sun Jan 09, 2022 9:20 am 8 cores Stockfish loss against Lc0 is weird. In others tests 1 core SF beats Lc0. I looked at games http://bozze.hopto.org/ssdf/games/. There was 38 draws and 2 losses. White loss has weird ?? blunder. And black lose was from +1 opening.
The GPU used by Lc0 is a strong GPU (3060 Ti). I don' t think Stockfish at 1 core beat Lc0 with any 30XX GPU (3090, 3080, 3070,3060,3050)