Discussion of anything and everything relating to chess playing software and machines.
Moderators: hgm , Rebel , chrisw
Adam Hair
Posts: 3226Joined: Wed May 06, 2009 10:31 pmLocation: Fuquay-Varina, North Carolina
Post by Adam Hair Sat Jan 14, 2017 9:20 pm
Here is a chart of Stockfish's month-by-month improvement since the birth of Fishtest, using SF 2.3.1 as the reference and including the official releases (and 2 or 3 versions that predate Fishtest).
CPUs: Xeon L5420
Time Control: 60" + 0.1"
Hash: 128 MB
Threads: 1
Openings: 50 positions 2 moves deep (a subset of the openings used in round 2 of TCEC)
Adjudication: resign @ 450 cp for 4 moves; draw after 120 moves if score is less than 50 cp for 4 moves
UI: cutechess-cli
Total number of games: 159600 (games available upon request)
Note: There is a distortion in the Elo ratings for Stockfish 3 and Stockfish 4 due to 7 time forfeits (all by Stockfish 3 when playing Stockfish 4) that have not been replayed yet.
I will be updating this chart each month at my
my blog .
Laskos
Posts: 10948Joined: Wed Jul 26, 2006 10:21 pmFull name: Kai Laskos
Post by Laskos Sun Jan 15, 2017 8:02 am
Impressive test! I understand that there are 57 engines with 100 games against each other engine, total 5,600 games per engine, right? (57*56/2)*100 = 159,600
Colin-G
Posts: 191Joined: Mon Oct 31, 2016 6:30 pmLocation: England
Post by Colin-G Sun Jan 15, 2017 11:11 am
With your fast Xeon cpu, but short thinking time, (I assume 60" + 0.1" means a total of "1 minute plus 0.1 second per move" for the whole game) I was wondering what the average move depth was in the games.
Laskos
Posts: 10948Joined: Wed Jul 26, 2006 10:21 pmFull name: Kai Laskos
Post by Laskos Sun Jan 15, 2017 11:51 am
Colin-G wrote: With your fast Xeon cpu, but short thinking time, (I assume 60" + 0.1" means a total of "1 minute plus 0.1 second per move" for the whole game) I was wondering what the average move depth was in the games.
The time control is not that short, it's the time control of Stockfish regression tests and LTC of Stockfish framework. Their STC time control is 15''+0.1''.
Ozymandias
Posts: 1532Joined: Sun Oct 25, 2009 2:30 am
Post by Ozymandias Sun Jan 15, 2017 12:53 pm
Lassos wrote: The pace of progress is impressive, probably only matched by Rybka 1 -> Rybka 3 progress
I'd say that Rybka progress was quite different. Take a look at these snapshots from the CCRL 40/40, back in the day:
Code: Select all
1 Rybka 1.2 64-bit 2996 +23 −22 75.6% −181.9 32.7% 739
Rybka 1.1 64-bit 2988 +24 −24 77.5% −193.7 31.8% 663
Rybka 1.1 32-bit 2969 +25 −24 71.7% −150.5 34.5% 577
Rybka 1.2 32-bit 2964 +33 −32 69.1% −133.9 33.3% 330
Rybka 1.0 64-bit 2918 +22 −21 68.9% −127.0 35.8% 745
Rybka 1.0 32-bit 2884 +19 −19 63.9% − 93.4 38.0% 919
2 Shredder 10 2878 +27 −26 61.6% − 79.7 34.1% 469
3 Hiarcs X50 32-bit 2858 +29 −29 49.9% +2.2 41.1% 355
4 Fritz 9 2849 +16 −16 55.5% − 38.1 35.2% 1293
5 Fruit 2.2-2.2.1 2843 +15 −15 55.4% − 35.8 39.8% 1418
Toga II 1.1a 2829 +18 −18 55.6% − 36.2 37.4% 968
As soon as Rybka 1 was properly tuned (version 1.1), no one came between a hundred ELO points. Then we had some "slow" year, to bring SMP around and finally, another revolution with Rybka 3:
Code: Select all
1 Rybka 3 64-bit 4CPU 3229 +24 −24 79.2% −209.2 33.9% 684
Rybka 2.3.2a 64-bit 4CPU 3128 +15 −15 70.1% −133.7 41.4% 1552
Rybka 2.2 64-bit 4CPU 3111 +30 −29 73.3% −157.6 38.6% 404
Rybka 2.1 64-bit 4CPU 3082 +39 −37 72.9% −161.5 36.5% 249
2 Naum 3.1 64-bit 4CPU 3077 +20 −20 61.1% − 71.9 43.7% 819
3 Zappa Mexico II 64b 4CPU 3074 +18 −18 61.4% − 73.5 46.4% 947
What we have nowadays, is cumulative evolution. And that allowed SF to share 1st, somewhere along the road, first with Houdini 4, then with Komodo 8+ and currently, with the two of them. Nothing resembling the kind of dominance, Rybka showed.
duncan
Posts: 12038Joined: Mon Jul 07, 2008 10:50 pm
Post by duncan Sun Jan 15, 2017 1:46 pm
Ozymandias wrote: Lassos wrote: The pace of progress is impressive, probably only matched by Rybka 1 -> Rybka 3 progress
I'd say that Rybka progress was quite different. Take a look at these snapshots from the CCRL 40/40, back in the day:
Code: Select all
1 Rybka 1.2 64-bit 2996 +23 −22 75.6% −181.9 32.7% 739
Rybka 1.1 64-bit 2988 +24 −24 77.5% −193.7 31.8% 663
Rybka 1.1 32-bit 2969 +25 −24 71.7% −150.5 34.5% 577
Rybka 1.2 32-bit 2964 +33 −32 69.1% −133.9 33.3% 330
Rybka 1.0 64-bit 2918 +22 −21 68.9% −127.0 35.8% 745
Rybka 1.0 32-bit 2884 +19 −19 63.9% − 93.4 38.0% 919
2 Shredder 10 2878 +27 −26 61.6% − 79.7 34.1% 469
3 Hiarcs X50 32-bit 2858 +29 −29 49.9% +2.2 41.1% 355
4 Fritz 9 2849 +16 −16 55.5% − 38.1 35.2% 1293
5 Fruit 2.2-2.2.1 2843 +15 −15 55.4% − 35.8 39.8% 1418
Toga II 1.1a 2829 +18 −18 55.6% − 36.2 37.4% 968
As soon as Rybka 1 was properly tuned (version 1.1), no one came between a hundred ELO points. Then we had some "slow" year, to bring SMP around and finally, another revolution with Rybka 3:
Code: Select all
1 Rybka 3 64-bit 4CPU 3229 +24 −24 79.2% −209.2 33.9% 684
Rybka 2.3.2a 64-bit 4CPU 3128 +15 −15 70.1% −133.7 41.4% 1552
Rybka 2.2 64-bit 4CPU 3111 +30 −29 73.3% −157.6 38.6% 404
Rybka 2.1 64-bit 4CPU 3082 +39 −37 72.9% −161.5 36.5% 249
2 Naum 3.1 64-bit 4CPU 3077 +20 −20 61.1% − 71.9 43.7% 819
3 Zappa Mexico II 64b 4CPU 3074 +18 −18 61.4% − 73.5 46.4% 947
What we have nowadays, is cumulative evolution. And that allowed SF to share 1st, somewhere along the road, first with Houdini 4, then with Komodo 8+ and currently, with the two of them. Nothing resembling the kind of dominance, Rybka showed.
what would hapen if stockfish was private like rybka, would there be more dominance ?
Adam Hair
Posts: 3226Joined: Wed May 06, 2009 10:31 pmLocation: Fuquay-Varina, North Carolina
Post by Adam Hair Sun Jan 15, 2017 1:55 pm
Laskos wrote: Impressive test! I understand that there are 57 engines with 100 games against each other engine, total 5,600 games per engine, right? (57*56/2)*100 = 159,600
Thanks! You are right. 5600 games per engine, error ranges from 6 to 8 Elo.
Code: Select all
# PLAYER : RATING ERROR POINTS PLAYED (%) CFS(%) W D L D(%) OppAvg OppErr OppN OppDiv
1 Stockfish 221216 64 : 8 7 4190.0 5600 74.8 93 2947 2486 167 44.4 -212 6 56 56.0
2 Stockfish 251116 64 : 1 7 4149.0 5600 74.1 55 2886 2526 188 45.1 -212 6 56 56.0
3 Stockfish 8 64 : 0 7 4145.5 5600 74.0 57 2904 2483 213 44.3 -212 6 56 56.0
4 Stockfish 220916 64 : -1 7 4140.5 5600 73.9 87 2880 2521 199 45.0 -212 6 56 56.0
5 Stockfish 221016 64 : -7 7 4108.0 5600 73.4 100 2798 2620 182 46.8 -211 6 56 56.0
6 Stockfish 190816 64 : -35 7 3937.0 5600 70.3 82 2605 2664 331 47.6 -211 6 56 56.0
7 Stockfish 210616 64 : -40 7 3910.5 5600 69.8 95 2513 2795 292 49.9 -211 6 56 56.0
8 Stockfish 160716 64 : -47 6 3863.5 5600 69.0 98 2520 2687 393 48.0 -211 6 56 56.0
9 Stockfish 200516 64 : -57 6 3800.5 5600 67.9 99 2349 2903 348 51.8 -211 6 56 56.0
10 Stockfish 180416 64 : -69 6 3725.5 5600 66.5 96 2271 2909 420 51.9 -210 6 56 56.0
11 Stockfish 150316 64 : -77 6 3674.0 5600 65.6 84 2198 2952 450 52.7 -210 6 56 56.0
12 Stockfish 200116 64 : -82 6 3644.5 5600 65.1 52 2147 2995 458 53.5 -210 6 56 56.0
13 Stockfish 210216 64 : -82 6 3643.0 5600 65.1 90 2184 2918 498 52.1 -210 6 56 56.0
14 Stockfish 221215 64 : -88 6 3605.5 5600 64.4 77 2109 2993 498 53.4 -210 6 56 56.0
15 Stockfish 7 64 : -91 6 3584.0 5600 64.0 78 2081 3006 513 53.7 -210 6 56 56.0
16 Stockfish 231015 64 : -94 6 3561.0 5600 63.6 55 2078 2966 556 53.0 -210 6 56 56.0
17 Stockfish 211115 64 : -95 6 3557.5 5600 63.5 100 2069 2977 554 53.2 -210 6 56 56.0
18 Stockfish 190915 64 : -116 6 3414.5 5600 61.0 91 1919 2991 690 53.4 -210 6 56 56.0
19 Stockfish 200815 64 : -122 6 3374.0 5600 60.3 74 1889 2970 741 53.0 -209 6 56 56.0
20 Stockfish 230715 64 : -125 6 3354.5 5600 59.9 66 1878 2953 769 52.7 -209 6 56 56.0
21 Stockfish 210515 64 : -127 6 3342.0 5600 59.7 94 1835 3014 751 53.8 -209 6 56 56.0
22 Stockfish 170615 64 : -134 6 3294.0 5600 58.8 97 1799 2990 811 53.4 -209 6 56 56.0
23 Stockfish 180415 64 : -142 6 3238.5 5600 57.8 96 1715 3047 838 54.4 -209 6 56 56.0
24 Stockfish 210315 64 : -150 6 3186.5 5600 56.9 100 1720 2933 947 52.4 -209 6 56 56.0
25 Stockfish 220215 64 : -162 6 3100.0 5600 55.4 98 1566 3068 966 54.8 -209 6 56 56.0
26 Stockfish 6 64 : -171 6 3034.5 5600 54.2 93 1543 2983 1074 53.3 -209 6 56 56.0
27 Stockfish 6 RC1 64 : -178 6 2989.0 5600 53.4 93 1511 2956 1133 52.8 -208 6 56 56.0
28 Stockfish 221214 64 : -184 6 2943.5 5600 52.6 53 1406 3075 1119 54.9 -208 6 56 56.0
29 Stockfish 211114 64 : -185 6 2941.0 5600 52.5 98 1471 2940 1189 52.5 -208 6 56 56.0
30 Stockfish 151014 64 : -194 6 2878.5 5600 51.4 100 1405 2947 1248 52.6 -208 6 56 56.0
31 Stockfish 100814 64 : -210 6 2765.5 5600 49.4 83 1367 2797 1436 49.9 -208 6 56 56.0
32 Stockfish 240614 64 : -214 6 2736.5 5600 48.9 56 1303 2867 1430 51.2 -208 6 56 56.0
33 Stockfish 210914 64 : -215 6 2731.5 5600 48.8 58 1261 2941 1398 52.5 -208 6 56 56.0
34 Stockfish 200714 64 : -216 6 2725.0 5600 48.7 99 1298 2854 1448 51.0 -208 6 56 56.0
35 Stockfish 190514 64 : -226 6 2654.0 5600 47.4 92 1240 2828 1532 50.5 -208 6 56 56.0
36 Stockfish 210414 64 : -232 6 2611.5 5600 46.6 93 1252 2719 1629 48.6 -207 6 56 56.0
37 Stockfish 5 64 : -238 6 2567.5 5600 45.8 96 1172 2791 1637 49.8 -207 6 56 56.0
38 Stockfish 220214 64 : -246 6 2514.0 5600 44.9 73 1134 2760 1706 49.3 -207 6 56 56.0
39 Stockfish 220314 64 : -249 6 2495.0 5600 44.6 100 1133 2724 1743 48.6 -207 6 56 56.0
40 Stockfish 190114 64 : -263 6 2398.0 5600 42.8 100 1053 2690 1857 48.0 -207 6 56 56.0
41 Stockfish 191213 64 : -288 6 2231.5 5600 39.8 97 950 2563 2087 45.8 -206 6 56 56.0
42 Stockfish 191113 64 : -296 6 2174.0 5600 38.8 51 866 2616 2118 46.7 -206 6 56 56.0
43 Stockfish DD 64 : -296 6 2173.5 5600 38.8 100 872 2603 2125 46.5 -206 6 56 56.0
44 Stockfish 231013 64 : -318 6 2031.0 5600 36.3 96 795 2472 2333 44.1 -206 6 56 56.0
45 Stockfish 190913 64 : -326 6 1977.0 5600 35.3 100 766 2422 2412 43.3 -206 6 56 56.0
46 Stockfish 4 64 : -359 7 1774.5 5600 31.7 98 660 2229 2711 39.8 -205 6 56 56.0
47 Stockfish 230813 64 : -369 7 1712.5 5600 30.6 100 592 2241 2767 40.0 -205 6 56 56.0
48 Stockfish 220713 64 : -390 7 1589.5 5600 28.4 89 505 2169 2926 38.7 -205 6 56 56.0
49 Stockfish 220613 64 : -396 7 1555.5 5600 27.8 99 466 2179 2955 38.9 -205 6 56 56.0
50 Stockfish 19-05-13 64bit : -408 7 1485.5 5600 26.5 100 435 2101 3064 37.5 -204 6 56 56.0
51 Stockfish 3 64bit : -445 7 1287.5 5600 23.0 79 337 1901 3362 33.9 -204 6 56 56.0
52 Stockfish 19-04-13 64bit : -450 7 1265.0 5600 22.6 83 309 1912 3379 34.1 -204 6 56 56.0
53 Stockfish 19-03-13 64bit : -455 7 1239.5 5600 22.1 90 325 1829 3446 32.7 -203 6 56 56.0
54 Stockfish 21-02-13 64bit : -462 7 1205.0 5600 21.5 51 274 1862 3464 33.3 -203 6 56 56.0
55 Stockfish 2.3.1 64bit : -462 8 1204.0 5600 21.5 100 311 1786 3503 31.9 -203 6 56 56.0
56 Stockfish 130127 64bit : -481 8 1114.0 5600 19.9 99 241 1746 3613 31.2 -203 6 56 56.0
57 Stockfish 121228 64bit : -494 8 1052.0 5600 18.8 --- 208 1688 3704 30.1 -203 6 56 56.0
White advantage = 33.87 +/- 0.62
Draw rate (equal opponents) = 64.06 % +/- 0.17Laskos wrote:
The pace of progress is impressive, probably only matched by Rybka 1 -> Rybka 3 progress. We also must note that Komodo's and Houdini's progress was also impressive, to almost match that of Stockfish. The last 5 or so years brought competent testing frameworks based on scientific method and a UI like Cutechess-Cli.
When I first joined CCC, proper and effective testing methods were beginning to be acknowledged by the general population of authors and testers (having been informed by the scientists and mathematicians), but there were few tools available. Lack of tools is no longer a problem. Building a GM-level chess engine has changed from being an amalgam of art and computer science to being an engineering process.
Adam Hair
Posts: 3226Joined: Wed May 06, 2009 10:31 pmLocation: Fuquay-Varina, North Carolina
Post by Adam Hair Sun Jan 15, 2017 2:01 pm
Colin-G wrote: With your fast Xeon cpu, but short thinking time, (I assume 60" + 0.1" means a total of "1 minute plus 0.1 second per move" for the whole game) I was wondering what the average move depth was in the games.
The CPUs are 2.5 GHz and 8 years old, so they are not fast relative to today's processors.
Here is the output from ProTools 1.6a:
Code: Select all
Engine Depth Time Games Moves Average Forfeit Book
Stockfish 19-04-13 6 16.54 94:44:11 5600 337030 1.01 0 11155 (1.99)
Stockfish 121228 64 16.22 92:50:34 5600 330401 1.01 0 11136 (1.99)
Stockfish 130127 64 16.42 92:30:28 5600 330072 1.01 0 11154 (1.99)
Stockfish 19-03-13 6 16.60 94:15:40 5600 336661 1.01 0 11137 (1.99)
Stockfish 3 64bit 16.68 94:52:10 5600 340789 1.00 7 11146 (1.99)
Stockfish 2.3.1 64bi 16.23 92:15:38 5600 334218 0.99 0 11129 (1.99)
Stockfish 21-02-13 6 16.50 92:44:04 5600 336634 0.99 0 11150 (1.99)
Stockfish 220713 64 17.05 94:49:04 5600 347803 0.98 0 11136 (1.99)
Stockfish 19-05-13 6 17.06 95:07:03 5600 349034 0.98 0 11147 (1.99)
Stockfish 230813 64 17.18 94:04:48 5600 345728 0.98 0 11145 (1.99)
Stockfish 220613 64 16.98 95:02:56 5600 350417 0.98 0 11121 (1.99)
Stockfish 4 64 17.22 94:45:36 5600 352257 0.97 0 11125 (1.99)
Stockfish 231013 64 18.28 92:25:04 5600 353525 0.94 0 11132 (1.99)
Stockfish 191113 64 18.60 93:51:07 5600 359855 0.94 0 11136 (1.99)
Stockfish 191213 64 18.75 94:01:55 5600 362028 0.94 0 11139 (1.99)
Stockfish DD 64 18.65 93:51:07 5600 361599 0.93 0 11149 (1.99)
Stockfish 8 64 20.06 86:24:01 5600 340892 0.91 0 12119 (2.16)
Stockfish 190816 64 19.20 87:57:14 5600 348267 0.91 0 12157 (2.17)
Stockfish 220916 64 19.83 86:54:56 5600 344475 0.91 0 12044 (2.15)
Stockfish 210216 64 19.47 90:31:10 5600 358809 0.91 0 12356 (2.21)
Stockfish 221016 64 20.05 86:29:11 5600 343150 0.91 0 12112 (2.16)
Stockfish 251116 64 20.09 85:52:35 5600 340911 0.91 0 12045 (2.15)
Stockfish 160716 64 18.88 87:59:26 5600 349472 0.91 0 12226 (2.18)
Stockfish 190913 64 17.75 89:29:49 5600 355809 0.91 0 11132 (1.99)
Stockfish 220314 64 19.03 94:02:17 5600 374161 0.90 0 12592 (2.25)
Stockfish 221216 64 20.19 86:15:14 5600 343898 0.90 0 12082 (2.16)
Stockfish 6 64 18.73 93:55:30 5600 375026 0.90 0 12468 (2.23)
Stockfish 6 RC1 64 18.62 94:06:45 5600 375875 0.90 0 12471 (2.23)
Stockfish 210616 64 19.57 87:12:27 5600 349617 0.90 0 12329 (2.20)
Stockfish 220214 64 19.22 93:02:47 5600 373030 0.90 0 12598 (2.25)
Stockfish 220215 64 18.78 94:02:08 5600 377274 0.90 0 12485 (2.23)
Stockfish 211114 64 18.47 94:02:51 5600 377480 0.90 0 12418 (2.22)
Stockfish 190114 64 18.98 92:35:54 5600 372178 0.90 0 12679 (2.26)
Stockfish 211115 64 18.68 90:56:13 5600 365589 0.90 0 12345 (2.20)
Stockfish 5 64 19.20 93:52:38 5600 379443 0.89 0 12685 (2.27)
Stockfish 200815 64 18.77 91:29:57 5600 369936 0.89 0 12482 (2.23)
Stockfish 100814 64 18.91 93:38:00 5600 378729 0.89 0 12624 (2.25)
Stockfish 230715 64 18.62 91:41:21 5600 371189 0.89 0 12325 (2.20)
Stockfish 210914 64 18.87 93:09:52 5600 377454 0.89 0 12566 (2.24)
Stockfish 221214 64 18.71 93:53:56 5600 380567 0.89 0 12482 (2.23)
Stockfish 200516 64 19.70 87:33:39 5600 354895 0.89 0 12324 (2.20)
Stockfish 240614 64 18.94 93:21:23 5600 378669 0.89 0 12577 (2.25)
Stockfish 190514 64 19.29 93:22:49 5600 378953 0.89 0 12697 (2.27)
Stockfish 200714 64 19.01 93:34:09 5600 379740 0.89 0 12659 (2.26)
Stockfish 210414 64 19.11 94:01:24 5600 381724 0.89 0 12708 (2.27)
Stockfish 210315 64 18.46 92:08:08 5600 375238 0.88 0 12461 (2.23)
Stockfish 180416 64 19.30 87:52:07 5600 358107 0.88 0 12300 (2.20)
Stockfish 180415 64 19.02 92:43:10 5600 378868 0.88 0 12567 (2.24)
Stockfish 231015 64 18.75 91:05:32 5600 372386 0.88 0 12290 (2.19)
Stockfish 170615 64 19.08 92:26:18 5600 378020 0.88 0 12478 (2.23)
Stockfish 190915 64 18.88 91:36:39 5600 375343 0.88 0 12340 (2.20)
Stockfish 200116 64 19.48 87:37:11 5600 359091 0.88 0 12294 (2.20)
Stockfish 210515 64 19.02 92:18:16 5600 379080 0.88 0 12431 (2.22)
Stockfish 151014 64 19.67 93:53:14 5600 386856 0.87 0 12499 (2.23)
Stockfish 221215 64 18.88 88:21:58 5600 364631 0.87 0 12322 (2.20)
Stockfish 150316 64 19.28 88:13:47 5600 364921 0.87 0 12317 (2.20)
Stockfish 7 64 18.89 88:27:35 5600 365877 0.87 0 12334 (2.20)
Time control comparison between engines
Depth : Average search depth
Time : Total time engine used
Moves : Total moves engine played
Average : Average time per move in centi-seconds
Forfeit : Games engine lost due to time forfeit
List is sorted on Average Time indicating the engine that uses the most time tops.
Ozymandias
Posts: 1532Joined: Sun Oct 25, 2009 2:30 am
Post by Ozymandias Sun Jan 15, 2017 2:34 pm
duncan wrote: what would hapen if stockfish was private like rybka, would there be more dominance ?
As long as they were able to keep so many people involved, and no one leaked the sources... But those are two big conditionals.
Adam Hair
Posts: 3226Joined: Wed May 06, 2009 10:31 pmLocation: Fuquay-Varina, North Carolina
Post by Adam Hair Fri Apr 28, 2017 1:01 am
Updated list (up to April 22):
Code: Select all
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 Stockfish 180317 64 : 19.3 8.9 4459.5 6000 74.3%
2 Stockfish 200417 64 : 11.6 9.0 4412.5 6000 73.5%
3 Stockfish 210117 64 : 11.2 9.1 4410.5 6000 73.5%
4 Stockfish 190217 64 : 8.7 9.0 4395.0 6000 73.2%
5 Stockfish 221216 64 : 7.3 9.2 4386.5 6000 73.1%
6 Stockfish 8 64 : 0.0 ---- 4341.0 6000 72.3%
7 Stockfish 251116 64 : 0.0 9.1 4341.0 6000 72.3%
8 Stockfish 220916 64 : -1.8 8.8 4330.0 6000 72.2%
9 Stockfish 221016 64 : -5.9 8.8 4304.0 6000 71.7%
10 Stockfish 190816 64 : -36.2 8.7 4107.0 6000 68.5%
11 Stockfish 210616 64 : -41.3 9.2 4073.0 6000 67.9%
12 Stockfish 160716 64 : -47.8 8.9 4029.5 6000 67.2%
13 Stockfish 200516 64 : -58.9 8.5 3954.0 6000 65.9%
14 Stockfish 180416 64 : -69.9 8.7 3877.5 6000 64.6%
15 Stockfish 150316 64 : -76.5 8.7 3831.5 6000 63.9%
16 Stockfish 210216 64 : -80.9 8.7 3800.5 6000 63.3%
17 Stockfish 200116 64 : -82.3 8.7 3790.5 6000 63.2%
18 Stockfish 221215 64 : -87.4 8.5 3755.0 6000 62.6%
19 Stockfish 7 64 : -90.0 8.7 3736.0 6000 62.3%
20 Stockfish 211115 64 : -94.2 8.6 3706.0 6000 61.8%
21 Stockfish 231015 64 : -94.7 8.7 3703.0 6000 61.7%
22 Stockfish 190915 64 : -115.8 8.5 3551.0 6000 59.2%
23 Stockfish 200815 64 : -122.4 8.6 3503.0 6000 58.4%
24 Stockfish 230715 64 : -125.2 8.3 3482.0 6000 58.0%
25 Stockfish 210515 64 : -126.9 8.4 3469.5 6000 57.8%
26 Stockfish 170615 64 : -134.2 8.5 3416.0 6000 56.9%
27 Stockfish 180415 64 : -142.4 8.8 3356.0 6000 55.9%
28 Stockfish 210315 64 : -151.7 8.5 3287.5 6000 54.8%
29 Stockfish 220215 64 : -163.4 8.7 3201.0 6000 53.4%
30 Stockfish 6 64 : -171.5 8.5 3141.0 6000 52.4%
31 Stockfish 6 RC1 64 : -178.7 8.6 3087.5 6000 51.5%
32 Stockfish 221214 64 : -183.8 8.6 3049.5 6000 50.8%
33 Stockfish 211114 64 : -184.6 8.4 3044.0 6000 50.7%
34 Stockfish 151014 64 : -193.7 8.7 2976.5 6000 49.6%
35 Stockfish 100814 64 : -208.9 8.7 2864.0 6000 47.7%
36 Stockfish 240614 64 : -215.2 8.9 2817.5 6000 47.0%
37 Stockfish 210914 64 : -215.3 8.8 2817.0 6000 47.0%
38 Stockfish 200714 64 : -217.1 8.6 2803.5 6000 46.7%
39 Stockfish 190514 64 : -226.9 8.4 2731.5 6000 45.5%
40 Stockfish 210414 64 : -232.8 8.4 2688.0 6000 44.8%
41 Stockfish 5 64 : -238.9 8.6 2644.0 6000 44.1%
42 Stockfish 220214 64 : -248.6 8.7 2573.5 6000 42.9%
43 Stockfish 220314 64 : -250.6 8.4 2559.0 6000 42.6%
44 Stockfish 190114 64 : -263.4 9.1 2467.0 6000 41.1%
45 Stockfish 191213 64 : -287.7 8.8 2295.0 6000 38.2%
46 Stockfish DD 64 : -295.7 8.8 2239.0 6000 37.3%
47 Stockfish 191113 64 : -297.3 8.7 2228.0 6000 37.1%
48 Stockfish 231013 64 : -318.3 9.0 2084.5 6000 34.7%
49 Stockfish 190913 64 : -327.1 9.2 2026.0 6000 33.8%
50 Stockfish 4 64 : -360.0 9.1 1812.5 6000 30.2%
51 Stockfish 230813 64 : -369.0 9.1 1756.0 6000 29.3%
52 Stockfish 220713 64 : -389.8 9.2 1629.5 6000 27.2%
53 Stockfish 220613 64 : -395.8 9.1 1593.5 6000 26.6%
54 Stockfish 19-05-13 64bit : -409.5 9.3 1514.0 6000 25.2%
55 Stockfish 3 64bit : -445.8 9.7 1314.5 6000 21.9%
56 Stockfish 19-04-13 64bit : -449.5 9.7 1295.0 6000 21.6%
57 Stockfish 19-03-13 64bit : -455.4 9.5 1264.5 6000 21.1%
58 Stockfish 21-02-13 64bit : -461.7 9.9 1232.0 6000 20.5%
59 Stockfish 2.3.1 64bit : -461.8 9.6 1231.5 6000 20.5%
60 Stockfish 130127 64bit : -481.5 10.0 1134.5 6000 18.9%
61 Stockfish 121228 64bit : -493.7 10.2 1077.0 6000 17.9%
