THE SSDF RATING LIST 2019-02-28

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

Lars Sandin
Posts: 2032
Joined: Sun Nov 18, 2007 5:17 pm
Location: Sundsvall, Sweden

THE SSDF RATING LIST 2019-02-28

Post by Lars Sandin »

Code: Select all

THE SSDF RATING LIST 2019-02-28    148673 games played by  377 computers							
                                           Rating   +	  -  Games   Won  Oppo
                                           ------  ---   --- -----   ---  ----
   1 Stockfish 9 x64 1800X 3.6 GHz           3494   32   -30   642   74%  3308
   2 Komodo 12.3 x64 1800X 3.6 GHz           3456   30   -28   640   68%  3321
   3 Stockfish 9 x64 Q6600 2.4 GHz           3446   50   -48   200   57%  3396
   4 Stockfish 8 x64 1800X 3.6 GHz           3432   26   -24  1059   77%  3217
   5 Stockfish 8 x64 Q6600 2.4 GHz           3418   38   -35   440   72%  3251
   6 Komodo 11.01 x64 1800X 3.6 GHz          3397   23   -22  1134   72%  3229
   7 Deep Shredder 13 x64 1800X 3.6 GHz      3360   25   -24   830   66%  3246
   8 Booot 6.3.1 x64 1800X 3.6 GHz           3352   29   -29   560   54%  3319
   9 Komodo 9.1 x64 Q6600 2.4 GHz            3340   21   -20  1435   72%  3175
  10 Komodo 11.01 x64 Q6600 2.4 GHz          3333   32   -31   482   55%  3300
  11 Stockfish 6 x64 Q6600 2.4 GHz           3326   22   -21  1216   68%  3192
  12 Deep Shredder 13 x64 Q6600 2.4 GHz      3296   24   -23   884   64%  3193
  13 Booot 6.3.1 x64 Q6600 2.4 GHz           3280   41   -39   320   64%  3179
  14 Komodo 7 x64 Q6600 2.4 GHz              3270   23   -23   974   65%  3160
  15 Arasan 21.2 x64 1800X 3.6 GHz           3269   61   -68   127   35%  3375
  16 Komodo 5.1 x64 Q6600 2.4 GHz            3245   22   -22  1038   64%  3145
  17 Deep Hiarcs 14 1800X 3.6 GHz            3219   26   -27   680   41%  3280
  18 Wasp 3 x64 1800X 3.6 GHz                3218   27   -28   642   43%  3266
  19 Stockfish 3 x64 Q6600 2.4 GHz           3203   19   -18  1420   61%  3127
  20 Deep Rybka 4 x64 Q6600 2.4 GHz          3201   21   -20  1288   67%  3080
  21 Deep Rybka 3 x64 Q6600 2.4 GHz          3194   22   -21  1371   75%  3003
  22 Deep Hiarcs 14 Q6600 2.4 GHz            3189   19   -18  1450   61%  3112
  23 Chiron 3.01 x64 Q6600 2.4 GHz           3180   27   -27   656   45%  3217
  24 Wasp 2.01 x64 1800X 3.6 GHz             3159   26   -28   726   34%  3276
  25 Naum 4.2 x64 Q6600 2.4 GHz              3146   21   -21  1123   60%  3078
  26 Deep Junior Yokohama x64 Q6600 2.4 GHz  3126   22   -22  1010   42%  3184
  27 Naum 4 x64 Q6600 2.4 GHz                3121   19   -18  1436   61%  3040
  28 Deep Junior 13.3 x64 Q6600 2.4 GHz      3116   19   -19  1290   50%  3119
  29 Spike 1.4 Q6600 2.4 GHz                 3108   15   -15  1951   52%  3092
  30 Deep Shredder 12 x64 Q6600 2.4 GHz      3104   18   -18  1507   62%  3016
  31 Hiarcs 14 Athlon 1.2 GHz                3100   29   -29   560   55%  3065
  32 Deep Fritz 13 Q6600 2.4 GHz             3097   24   -24   826   55%  3064
  33 The Baron 3.43 x64 1800X 3.6 GHz        3095   32   -35   538   27%  3265
  34 Deep Rybka 3 Athlon 1.2 GHz             3071   36   -35   372   53%  3048
  35 Wasp 2.01 x64 Q6600 2.4 GHz             3065   40   -46   404   20%  3302
  36 Crafty 25 x64 Q6600 2.4 GHz             3028   25   -26   804   35%  3134
  37 The Baron 3.43 x64 Q6600 2.4 GHz        3023   33   -33   440   51%  3019
  38 Arasan 17.2 x64 Q6600 2.4 GHz           3000   26   -26   686   46%  3029
  39 Revelation 2 Hiarcs 14.1 PXA320 800 MHz 2924   47   -46   220   55%  2889
  40 Chessmaster King 3.5 x64 Q6600 2.4 GHz  2861   24   -25   932   30%  3009
  41 Revelation Hiarcs 13.3 PXA255 500 MHz   2772   57   -52   177   66%  2661
  42 Revelation Shredder 12 PXA255 500 MHz   2703   60   -58   140   56%  2663
  43 Revelation Rybka 2.2 PXA255 500 MHz     2628   47   -44   240   62%  2545
  44 Revelation Deep Sjeng 3 PXA255 500 MHz  2599   68   -76   100   37%  2692
  45 ChessGenius 3 ZTE Apex3 ARM A53 1.3 GHz 2457   75   -68   100   62%  2376
  46 Revelation Ruffian 2.1 PXA255 500 MHz   2353   68   -71   100   45%  2390
  47 TASC R30 v. 2.5 ARM6 30 MHz             2273   42   -38   343   69%  2136
  48 Millenium ChessGenius Pro M4 120 MHz    2176   67   -59   140   68%  2048
  49 Millenium ChessGenius ARM M4 48 MHz     2078   51   -47   210   62%  1990
  50 Mephisto London 68000 12 MHz            2035   69   -70   100   49%  2043

 2 Komodo 12.3 x64 1800X 3.6 GHz, 3456											
Stock9 1800X    18,5-21,5  Stock9 Q6600      20-20    Stock8 1800X    22,5-17,5
Stock8 Q6600      20-20    Komo11 1800X      22-18    DShre13 1800X     28-12
Booot6 1800X      24-16    Komo9.1 Q6600     27-13    Komo11 Q6600    56,5-23,5
Booot6 Q6600      31-9     Arasa21 1800X   30,5-9,5   DHiar14 1800X   34,5-5,5 
Wasp3 1800X     30,5-9,5   Wasp201 1800X     35-5     Baro343 1800X   36,5-3,5 

 3 Stockfish 9 x64 Q6600 2.4 GHz, 3446											
Stock9 1800X    16,5-23,5  Komo12 1800X      20-20    Komo11 1800X    21,5-18,5
Booot6 1800X    26,5-13,5  Booot6 Q6600    29,5-10,5  

 13 Booot 6.3.1 x64 Q6600 2.4 GHz, 3280											
Komo12 1800X       9-31    Stock9 Q6600    10,5-29,5  Stockf6 Q6600   20,5-19,5
DjunYok Q6600   27,5-12,5  Hiarc14 A1200     33-7     DRybka3 A1200   34,5-5,5 
Shred12 A1200     35-5     Rybk231 A1200   35,5-4,5   

 15 Arasan 21.2 x64 1800X 3.6 GHz, 3269											
Komo12 1800X     9,5-30,5  Stock8 1800X     7,5-31,5  DShre13 1800X   11,5-18,5
Baro343 1800X   15,5-2,5   

 18 Wasp 3 x64 1800X 3.6 GHz, 3218											
Stock9 1800X     6,5-33,5  Komo12 1800X     9,5-30,5  Stock8 1800X     5,5-34,5
Stock8 Q6600     7,5-32,5  Komo11 1800X     6,5-33,5  DShre13 1800X   14,5-25,5
Booot6 1800X      11-29    Komo7.0 Q6600   13,5-26,5  DHiar14 1800X   19,5-20,5
Stockf3 Q6600   17,5-22,5  Chiro3 Q6600      23-17    Wasp201 1800X     25-15
Spike14 Q6600     27-13    Baro343 1800X   29,5-10,5  Zap!Mx2 Q6600   30,5-9,5 
Wasp201 Q6600     29-13    

 50 Mephisto London 68000 12 MHz, 2035											
MCG Pro          6,5-13,5  MCG               10-10    Star Sapphire    6,5-13,5
Milano Pro      12,5-7,5   Nigel Short       13-7     

On our latest February rating list we can present six new Chess Computer 
programs. First out is the winner of all three classes at the ICGA World
Championship that was played in Stockholm/Sweden last summer - namely Komodo.
Komodo is written by Mark Lefler, Larry Kaufman and the late Don Dailey.
The version we have tested is named Komodo 12.3 and it has achieved a very 
strong rating of 3456 after 640 games played on our 1800X hardware level. This
is 59 points stronger than the 11.01 version we tested last time. We have used
Komodo's opening book by Erdogan Guenes for the testing.

The next new version we have on this list is Stockfish 9, which now have 
sufficient amount of games on our Q6600-hardware level. After the 
first 200 games it has reached the rating of 3446. This is 28 points stronger
than Stockfish 8 on the same platform, and is 48 points lower than the same
version on our 1800X hardware. We have of course used the same opening book 
as we previously used on our 1800X hardware, when we tested Stockfish 9; that
is the free "Aggressive 3.5 by Fauzi.abk" opening book by Fauzi Dabat.

We have also got enough games with Booot 6.3.1 on our Q6600 hardware to be
able to introduce it on the list. The rating is 3280 after the first 320 games.
This is 72 points lower than the 1800X version which we introduced on the
last rating list.

Just in time for this rating list, we could also present an early rating of 
the latest Arasan program by Jon Dart. We have tested it on our 1800X hardware,
with it's own included opening book and it has reached a rating of 3269 after 
the first 127 games. More games will be needed to get a more reliable rating, 
but the rating of 3269 is still 269 points stronger than the last Arasan
version we have tested! Even though some of the difference are due to 
the fact that we tested the 17.2 version on our Q6600-hardware, the 
performance gain from improvements made in the software is impressive.

Next out is the Wasp 3.0 program by John Stanback. We have played 642 games
with Wasp 3.0 on our 1800X hardware, and it has reached the rating of 3218. 
This is 59 points stronger than the 2.01 version which we introduced in our
rating list in January 2018. We have used the free opening book by Sedat
Canbaz, called "Perfect2017.abk" for the testing of Wasp 3.0. 

Last out in this rating list is the manually tested "oldie" - the dedicated
Mephisto London 68000 chess computer. This is a computer which we never got
to test back in the days, but now are able to finally introduce in the 
rating list, 23 years after it was introduced back in 1996! 
The Mephisto London is the program of Richard Lang and uses the module 
hardware for the old Modular/Exclusive/München boards. It features an 
Motorola 68000 processor at 12 MHz and uses 512 kB RAM for hash-tables.
The rating after the first 100 games is 2035 and this is 18 points stronger
than the Berlin 68000, which previously was the strongest Lang program on 
the 68000 hardware in our rating list.

Compared to the last rating list, Deep Hiarcs 14 1800X gained 8 points and
Novag Star Sapphire gained 6 points. Meanwhile The Baron 3.43 1800X lost 
20 points, Millenium ChessGenius lost 9 points and Stockfish 9 1800X lost 
8 points. 

For our next list we will focus on getting more games with Komodo 12.3 on 
our Q6600 hardware and also Arasan 21.2 on both the 1800X and Q6600 level. 
We will also begin testing of Stockfish 10, Wasp 3.5 and hopefully also 
the Monte Carlo Tree Search (MCTS) version of Komodo. 
We are also manually testing both the Millenium ChessGenius Exclusive and 
the latest Millenium The King Exclusive now.

Our next rating list will probably be out before summer. 
           
Lars Sandin
Lars Sandin, SSDF
mehmet karaman
Posts: 142
Joined: Tue Jan 28, 2014 8:37 am
Location: TURKEY

Re: THE SSDF RATING LIST 2019-02-28

Post by mehmet karaman »

Thanks for this great rating list.

I hope we can see Stockfish 10 in the next rating list.
Modern Times
Posts: 3550
Joined: Thu Jun 07, 2012 11:02 pm

Re: THE SSDF RATING LIST 2019-02-28

Post by Modern Times »

With lots of different hardware configurations already on this list, there are no complications whatsoever with adding GPU testing. Very easy for them :)
Dann Corbit
Posts: 12541
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: THE SSDF RATING LIST 2019-02-28

Post by Dann Corbit »

Modern Times wrote: Thu Feb 28, 2019 9:26 pm With lots of different hardware configurations already on this list, there are no complications whatsoever with adding GPU testing. Very easy for them :)
The only complication is money.
Who is going to pay for the GPUs?
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Modern Times
Posts: 3550
Joined: Thu Jun 07, 2012 11:02 pm

Re: THE SSDF RATING LIST 2019-02-28

Post by Modern Times »

Money is always an issue with computer testing !!

They don't need to be £500 RTX, you can go lower than that.
Chessqueen
Posts: 5588
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: THE SSDF RATING LIST 2019-02-28

Post by Chessqueen »

Modern Times wrote: Thu Feb 28, 2019 10:56 pm Money is always an issue with computer testing !!

They don't need to be £500 RTX, you can go lower than that.
The problem with going lower is that the other engines will be using superior hardware, which does not make it fair to test LCZero or other NN engines against Stockfish or Komodo, unless somebody can figure out what lower CPU can be compared to a lower GPU, otherwise what is the use of testing GPU engines using inferior GPU ? :roll:

PS: I have a suggestion, why doesn't Mr. Graham set up a donation thru Pay Pal to donate at least a minimum of $5.00 and those whose would like to donate more Like myself can also do it, in order to collect money for a decent GPU to test LCZero and other NN engines versus the rest of the other engines?
Who is 17 years old GM Gukesh 2nd at the Candidate in Toronto?
https://indianexpress.com/article/sport ... t-9281394/
Uri Blass
Posts: 10297
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: THE SSDF RATING LIST 2019-02-28

Post by Uri Blass »

hardware is not equal in the SSDF list so I see nothing unfair.

https://ssdf.bosjo.net/long.txt

old engines in the SSDF list also get inferior hardware.
For example Fritz5.32 got only P200 MHz MMX or K6-2 450 Mhz when Stockfish9 got clearly superior hardware.

1 Stockfish 9 x64 1800X 3.6 GHz 3494 32 -30 642 74% 3308
3 Stockfish 9 x64 Q6600 2.4 GHz 3446 50 -48 200 57% 3396

150 Fritz 5.32 K6-2 450 MHz 2557 20 -20 1194 48% 2574
166 Fritz 5.32 64MB P200 MHz MMX 2500 13 -13 3023 38% 2588
jorose
Posts: 360
Joined: Thu Jan 22, 2015 3:21 pm
Location: Zurich, Switzerland
Full name: Jonathan Rosenthal

Re: THE SSDF RATING LIST 2019-02-28

Post by jorose »

Chessqueen wrote: Fri Mar 01, 2019 4:22 pm
Modern Times wrote: Thu Feb 28, 2019 10:56 pm Money is always an issue with computer testing !!

They don't need to be £500 RTX, you can go lower than that.
The problem with going lower is that the other engines will be using superior hardware, which does not make it fair to test LCZero or other NN engines against Stockfish or Komodo, [...]
I see 2 processors at the top. The Q6600 is from 2008. The Ryzen 1800X is from 2017 and can be purchased new for 260.- USD from Newegg. A GPU machine will also need a CPU.
-Jonathan
Chessqueen
Posts: 5588
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: THE SSDF RATING LIST 2019-02-28

Post by Chessqueen »

jorose wrote: Fri Mar 01, 2019 5:46 pm
Chessqueen wrote: Fri Mar 01, 2019 4:22 pm
Modern Times wrote: Thu Feb 28, 2019 10:56 pm Money is always an issue with computer testing !!

They don't need to be £500 RTX, you can go lower than that.
The problem with going lower is that the other engines will be using superior hardware, which does not make it fair to test LCZero or other NN engines against Stockfish or Komodo, [...]
I see 2 processors at the top. The Q6600 is from 2008. The Ryzen 1800X is from 2017 and can be purchased new for 260.- USD from Newegg. A GPU machine will also need a CPU.
Are you implying to use The Q6600 from 2008 against the Ryzen1800x from 2017 which might give a closer match ?

https://www.newegg.com/Product/Product. ... lsrc=aw.ds
Who is 17 years old GM Gukesh 2nd at the Candidate in Toronto?
https://indianexpress.com/article/sport ... t-9281394/