39 Free Single-CPU engines in CCRL 40/4

[Kirill Kryukov]

Now 39 free single-CPU engines are more or less well-tested in our blitz (counting only strongest versions of each engine). (Well-tested means large number of games with each other, not only overall):

Code: Select all

 Rank                 Engine                  ELO (Inc)   +    -   Score  AvOp  Games
    1 Rybka 1.0 Beta 64-bit                  2950 (-3)   +19  -19  76.2% -194.3  1152
    2 Strelka 1.8 32-bit                     2877 (+6)   +19  -18  61.5%  -84.5  1022
    3 Fruit 2.3 5-men-egbb                   2875 (New)  +22  -22  67.2% -122.4   736
    4 Spike 1.2 Turin                        2827 (-1)    +7   -7  46.3%  +25.6  7755
    5 Naum 2.0 32-bit                        2797 (-1)    +8   -9  48.7%   +3.5  5438
    6 Alaric 707                             2774 (New)  +17  -17  52.9%  -21.1  1223
    7 Glaurung 1.2.1 32-bit                  2758 (-1)    +8   -9  41.6%  +60.0  5429
    8 Scorpio 1.91 (5-men-egbb)              2731 (-1)   +17  -17  49.8%   -0.3  1154
    9 Pro Deo 1.2                            2722 (0)    +12  -12  43.3%  +51.4  2563
   10 Delfi 5.1                              2716 (0)    +12  -12  50.1%   -1.4  2348
11-12 Slow Chess Blitz WV2.1                 2712 (+1)    +9   -9  41.8%  +60.2  4762
11-12 Zappa 1.1 64-bit                       2712 (+1)   +17  -17  46.6%  +24.2  1216
   13 Frenzee 3.0 64-bit                     2709 (0)    +17  -17  49.1%   +6.4  1317
   14 List 5.12                              2698 (0)    +12  -12  49.5%   +3.0  2664
   15 Pharaon 3.5.1                          2697 (-1)   +12  -12  38.2%  +87.8  2664
   16 WildCat 7                              2694 (-2)   +11  -12  46.5%  +26.0  2817
   17 SOS 5.1                                2680 (-3)   +13  -13  46.6%  +24.0  2281
   18 Ruffian 1.0.5                          2676 (+3)   +13  -13  49.8%   +0.4  2114
   19 Pseudo 0.7c                            2675 (+1)   +12  -11  50.7%   -8.4  2805
   20 Aristarch 4.50                         2669 (+1)   +12  -12  45.8%  +31.9  2567
21-22 Booot 4.13.1                           2666 (+14)  +17  -17  49.9%   +0.7  1209
21-22 Petir 4.39                             2666 (-2)   +14  -14  44.3%  +41.9  1786
   23 Colossus 2007a                         2657 (-4)   +16  -16  43.9%  +43.9  1435
   24 The Baron 1.8.1                        2652 (-2)   +18  -18  47.7%  +17.2  1138
   25 Jonny 2.83 32-bit                      2647 (0)    +15  -15  44.4%  +43.1  1693
   26 Fritz 5.32                             2644 (+3)   +17  -17  53.7%  -28.5  1205
27-28 Crafty 21.5 PS 64-bit                  2643 (-6)   +16  -16  43.3%  +49.3  1360
27-28 Smarthink 0.17a                        2643 (-4)   +17  -17  47.8%  +15.7  1117
   29 Anaconda 2.0.1                         2637 (0)    +15  -15  49.1%   +6.7  1523
   30 Movei 0.08.403                         2634 (-1)   +17  -17  51.9%  -15.0  1273
   31 Thinker 4.7a                           2632 (0)    +13  -14  48.2%   +7.6  1992
   32 Ktulu 4.2                              2628 (New)  +21  -21  48.7%   +9.4   766
   33 AnMon 5.60                             2617 (0)    +16  -16  46.4%  +25.5  1456
   34 Little Goliath Evolution 3.12          2616 (+2)   +15  -15  53.4%  -28.3  1722
   35 Ufim 8.02                              2615 (+2)   +13  -13  49.9%   -3.9  2197
   36 Trace 1.37a                            2612 (-5)   +17  -17  43.5%  +48.7  1236
   37 Yace 0.99.87                           2610 (+2)   +21  -21  45.5%  +29.4   768
   38 E.T. Chess 18.11.05                    2603 (New)  +14  -15  54.3%  -39.6  1798
   39 Amyan 1.597                            2564 (New)  +16  -16  46.8%  +21.5  1458

Rating list and cross-tables

(Previous report was posted here)

Comments:

It is not clear which Fruit version is strongest, so I just include Fruit 2.3 which is well-tested with other Free engines. Same for Scorpio.

Alaric 707 substituted 704 with 54 ELO increase.

Pro Deo 1.2 has some gaps (untested pairs) because I am unable to test it on my machines. (It does not run on machine with 4 GB of RAM or more).

Ktulu 4.2 is newly tested, and is ranked #32 now!

Also added are E.T. Chess 18.11.05 and Amyan 1.597 (#38 and #39 rigth now).

There may be engines that I missed and which will rank at #30-#40 when they are tested. However I think it is less likely that any potential top 30 engine is missed. So I think this list already shows reliable blitz rankings for top 30 free single-CPU engines.

Note that this list is just my custom selection, you can construct any other selection on our web-site. Also note that this list does not list all free engines we tested, but only those that I consider to have reliable ranking in free single-CPU context.

Thanks to all testers contributing to this comparison. I'll post if there are any other news.

Best,
Kirill

[Tony Thomas]

I dont see any version of Toga on the list that you posted, was that intentional?

[Kirill Kryukov]

I dont see any version of Toga on the list that you posted, was that intentional?

Yes. Toga and Fruit are too similar to each other, so I take them as versions of the same engine. So only stronger one is listed. I have to add that I don't know which one is really stronger, so I list stronger one out of those that have enough games with other free engines.

Actually I now realized that this is not consistent with including Strelka in the list. I should either remove Strelka or add Toga to the list. I'll think about it..

Once again I'll mention: you can modify the selection and include any engines you want to see there. You can do this using "Alter engine selection" control on custom comparison pages on our web-site.

[Ovyron]

counting only strongest versions of each engine

Code: Select all

 Rank                 Engine                  ELO (Inc)   +    -   Score  AvOp  Games
    9 Pro Deo 1.2                            2722 (0)    +12  -12  43.3%  +51.4  2563

I think latest Pro Deo 1.5 with EOC learning is at least 30 ELO stronger than 1.2, though I also think it hasn't been properly tested (Mainly because Ed claims that it does not contain any engine improvement, but I consider its learning capabilities as part of the engine).

[playjunior]

Kirill, are those 32 bit, 64 bit versions or mixed?

[Kirill Kryukov]

counting only strongest versions of each engine

Code: Select all

 Rank                 Engine                  ELO (Inc)   +    -   Score  AvOp  Games
    9 Pro Deo 1.2                            2722 (0)    +12  -12  43.3%  +51.4  2563

I think latest Pro Deo 1.5 with EOC learning is at least 30 ELO stronger than 1.2, though I also think it hasn't been properly tested (Mainly because Ed claims that it does not contain any engine improvement, but I consider its learning capabilities as part of the engine).

There are two points that stop me from testing 1.5.

1. Versions 1.2 - 1.5 don't run on my machines. It does not work when you have 4 GB of RAM or more. Ed is aware of this problem, and he already fixed it and I already have the fixed version but I'm waiting till it becomes public before I do any testing.

2. We don't use any learning in our tests, because we can't compare or discuss ratings of something that changes along the way. We can use Pro Deo 1.5 with EOC off, but Ed said it should be about same strength with 1.2.

Learning is difficult to use in testing. If I test Pro Deo 1.5 with learning and it gets N ELO - it's not useful to anyone, because someone who just downloaded Pro Deo 1.5 will not have the learning data I have and Pro Deo will play differently for him.

Of course anyone is free to test with learning, I only say that I am not interested to do that.

[Kirill Kryukov]

Kirill, are those 32 bit, 64 bit versions or mixed?

Mixed, but we try to add "32-bit" or "64-bit" in those where confusion is possible (where both 32 and 64-bit compiles exist). Generally if there is no "64-bit" in the name you should assume it's 32-bit.

(Sometimes we forget to add it, or sometimes we don't know the 64-bit one exists, etc).

[gerold]

Thanks Kirill,

Looks like Rybka 1.0 appears to still be the
strongest free engine.

Good luck,

Gerold.

[Kirill Kryukov]

Thanks Kirill,

Looks like Rybka 1.0 appears to still be the
strongest free engine.

Good luck,

Gerold.

Hi Gerold! Apparently Rybka 1.0 continues to be still stronger than any other non-Rybka single-CPU engine (free or commercial), as can be seen at Single-CPU list.

[playjunior]

Thank you Kirill !
I thought that the version of Rybka with 2950 was the 64-bit one.
Btw can I get the books with which you test in CCRL? I have downloaded some books for Arena, one of them seems reasonable to me(none of the engines "complained" about positions out of opening). With this book I run tournaments on my Core2 Duo 2.16, 256 MB Hash, with engines like Toga, Fruit, Glaurung 1.2.1 SMP...
For example now it goes:

Engine Score Fr To Gl Ry S-B
1: Fruit-2-3-1 6.0/9 ··· === 11= 110 23.75
2: TogaII-1.3.1 5.0/9 === ··· 1=0 1== 20.75
3: Glaurung 1.2.1 SMP 4.5/9 00= 0=1 ··· 11= 16.75
4: Rybka v1.0 Beta.w32 2.5/9 001 0== 00= ··· 13.25

I cannot believe this engine is 100 points stronger than the nearest competitor