New LightSpeed-rankinglist (smallest errorbar of all times)

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

User avatar
pohl4711
Posts: 2435
Joined: Sat Sep 03, 2011 7:25 am
Location: Berlin, Germany
Full name: Stefan Pohl

New LightSpeed-rankinglist (smallest errorbar of all times)

Post by pohl4711 »

The LS-rankinglist (LightSpeed-rankinglist):

Intel i7-2630QM (SSE42 support, Windows 7 64bit, Quadcore, FritzMark=20.2), 64 MB Hash, 1 core per engine (Hyperthreading off), no ponder, no endgame-bases, no resign. 500 selected opening-positions (all 8 moves deep, from Frank Q.-database)
Elos calculated with bayeselo (mm 0 1)(fixpoint Robbolito 0.085g3 3000 Elo). LittleBlitzerGUI (gauntlet-mode only, because this GUI chooses opening-positions per random in the round-robin-mode from the PGN-file...)
Time: 20''+250ms Fischerbonus (= 40 seconds per game/engine).

LS-rankinglist with best engine-versions only:

Code: Select all

Rank Name                    Elo    +    - games score oppo. draws 
   1 Houdini 2.0c x64       3120    6    5  9000   64%  3020   40% 
   2 Strelka 5.5 x64        3078    5    5  9000   58%  3024   48% (singlecore)
   3 Critter 1.6a x64       3066    5    5  9000   56%  3026   50% 
   4 Ivanhoe 46h x64        3028    5    5  9000   50%  3030   52% 
   5 Robbolito 0.10 x64s    3025    5    5  9000   49%  3030   53% 
   6 Rybka 4.1 x64s         3012    5    5  9000   47%  3032   43% 
   7 Robbolito 0.085g3 x64  3000    5    5  9000   45%  3033   49% (singlecore)(Ippolit 2009)
   8 Komodo 4 x64s          3000    6    5  9000   46%  3033   39% (singlecore)
   9 Saros 3.0 x64          2995    5    5  9000   45%  3033   45% 
  10 Stockfish 2.2.2 x64s   2973    6    5  9000   41%  3036   40% 
The crosstable for that list:

Code: Select all

                                |     01    |     02    |     03    |     04    |     05    |     06    |     07    |     08    |     09    |     10    |
                                +-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+--------------    			
01) Houdini 2.0c x64            |     **    |548.5-451.5|563.0-437.0|619.0-381.0|642.5-357.5|644.0-356.0|668.5-331.5|685.5-314.5|674.5-325.5|692.5-307.5| 5738.0/9000		
02) Strelka 5.5 x64             |451.5-548.5|     **    |498.5-501.5|586.0-414.0|565.5-434.5|597.0-403.0|614.0-386.0|610.0-390.0|638.0-362.0|635.5-364.5| 5196.0/9000
03) Critter 1.6a x64            |437.0-563.0|501.5-498.5|     **    |566.5-433.5|554.0-446.0|569.5-430.5|584.5-415.5|593.5-406.5|587.0-413.0|630.0-370.0| 5023.5/9000
04) Ivanhoe 46h x64             |381.0-619.0|414.0-586.0|433.5-566.5|     **    |505.0-495.0|527.0-473.0|549.5-450.5|543.0-457.0|550.5-449.5|564.5-435.5| 4468.0/9000
05) Robbolito 0.10 x64s         |357.5-642.5|434.5-565.5|446.0-554.5|495.0-505.0|     **    |507.0-493.0|521.0-479.0|555.0-445.0|543.0-457.0|566.0-434.0| 4425.0/9000
06) Rybka 4.1 x64s              |356.0-644.0|403.0-597.0|430.5-569.5|473.0-527.0|493.0-507.0|     **    |520.5-479.5|502.5-497.5|521.5-478.5|555.5-444.5| 4255.5/9000
07) Komodo 4 x64s               |331.5-668.5|386.0-614.0|415.5-584.5|450.5-549.5|479.0-521.0|479.5-520.5|     **    |514.4-485.5|505.0-495.0|536.5-463.5| 4098.0/9000
08) Robbolito 0.085g3 x64       |314.5-685.5|390.0-610.0|406.5-593.0|457.0-543.0|445.0-555.0|497.5-502.5|485.5-514.4|     **    |521.0-479.0|545.0-455.0| 4062.0/9000
09) Saros 3.0 x64               |325.5-674.5|362.0-638.0|413.0-587.0|449.5-550.5|457.0-543.0|478.5-521.5|495.0-505.0|479.0-521.0|     **    |553.0-447.0| 4012.5/9000
10) Stockfish 2.2.2 x64s        |307.5-692.5|364.5-635.5|370.0-630.0|435.5-564.5|434.0-566.0|444.5-555.5|463.5-536.5|455.0-545.0|447.0-553.0|     **    | 3721.5/9000
                                +-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+-----------+--------------
LS-rankinglist with all engines/versions/settings tested so far:

Code: Select all

Rank Name                    Elo    +    - games score oppo. draws 
   1 Houdini 2.0c x64       3122    5    5 10000   63%  3028   40% 
   2 Houdini 1.5a x64       3087    5    5 10000   58%  3031   42% (best freeware (multicore))
   3 Strelka 5.5 x64        3080    5    5 10000   57%  3032   48% (singlecore)
   4 Critter 1.6a x64       3069    5    5 10000   55%  3033   50% 
   5 Ivanhoe 46h x64        3029    5    5 10000   49%  3037   51% 
   6 Robbolito 0.10 x64s    3026    5    5 10000   48%  3037   52% 
   7 Rybka 4.1 x64s         3013    5    5 10000   46%  3039   42% 
   8 Komodo 4 x64s          3002    5    5 10000   45%  3040   38% (singlecore)
   9 Robbolito 0.085g3 x64  3000    5    5 10000   44%  3040   49% (singlecore)(Ippolit 2009)
  10 Saros 3.0 x64          2997    5    5 10000   44%  3040   45% 
  11 Stockfish 2.2.2 x64s   2974    5    5 10000   41%  3042   39% 
(x64=64bit version, x64s=64bit SSE42-version)

If you want to get the games from the LS-list, send me an eMail-adress per PM. I will send the games (PGN-file) as soon as possible...

Greetings – Stefan
John_F
Posts: 44
Joined: Wed Jan 27, 2010 2:58 pm

Re: New LightSpeed-rankinglist (smallest errorbar of all tim

Post by John_F »

Interesting how on this list, under these conditions, Houdini version 2 comes out clearly superior (+35) to version 1.5a.
User avatar
pohl4711
Posts: 2435
Joined: Sat Sep 03, 2011 7:25 am
Location: Berlin, Germany
Full name: Stefan Pohl

Re: New LightSpeed-rankinglist (smallest errorbar of all tim

Post by pohl4711 »

Yes, in bullet-speed Houdini 2.0 is incredible strong.

Stefan
User avatar
Houdini
Posts: 1471
Joined: Tue Mar 16, 2010 12:00 am

Re: New LightSpeed-rankinglist (smallest errorbar of all tim

Post by Houdini »

In my bullet tests at even faster TC Houdini 2.0 is 25 (+/- 2) Elo stronger than Houdini 1.5a.