Komodo 8 on the rating lists.

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
lkaufman
Posts: 3772
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Komodo 8 on the rating lists.

Post by lkaufman » Sun Sep 14, 2014 3:22 am

Now that we have had more than a week to get Komodo 8 rated on the various rating lists, I did a new compilation of all the non-blitz rating lists that include both Komodo 8 and Komodo 7a, with at least 300 games for each. With Komodo 8 set at zero, here are the ratings for Komodo 7a, Houdini 4, and Stockfish 5.

IPON: -39, -14, -2.
CCRL 40/40 4 threads: -61, -30, -22.
CEGT 40/20 1 thread: -42, -32, -10.
CEGT 5' +3": -41, -6, -4.
FCT1 40/10: -33, x, -35. This list includes a recent SF dev. at -6.

Average ratings relative to Komodo 8: Komodo 7a minus 43, Houdini 4 minus 21, Stockfish 5 minus 15. Note that four of the five lists are single core, but Komodo 8 looks best on the one list running 4 threads. My forecast for gains over Komodo 7a was 40 elo on one thread and 50 on four threads, so my forecast for the above average would be 42 elo, as compared to the actual gain of 43 elo!

So based on these lists Stockfish dev. needs to be plus 15 elo over Stockfish 5 to be equal to Komodo 8.
Komodo rules!

Jouni
Posts: 2032
Joined: Wed Mar 08, 2006 7:15 pm

Re: Komodo 8 on the rating lists.

Post by Jouni » Sun Sep 14, 2014 6:38 am

Congratulations for Komodo team! BTW framework shows now +15,68 for latest SF dev version - next TCEC will be really close.
Jouni

Nay Lin Tun
Posts: 530
Joined: Mon Jan 16, 2012 5:34 am

Re: Komodo 8 on the rating lists.

Post by Nay Lin Tun » Sun Sep 14, 2014 7:28 am

What do you mean? +15 to +68 elo?

Modern Times
Posts: 2422
Joined: Thu Jun 07, 2012 9:02 pm

Re: Komodo 8 on the rating lists.

Post by Modern Times » Sun Sep 14, 2014 7:37 am

I think it is 15.68 Elo

The Europeans use a comma for decimals.

Frank Quisinsky
Posts: 4852
Joined: Wed Nov 18, 2009 6:16 pm
Location: Trier, Germany
Contact:

Re: Komodo 8 on the rating lists.

Post by Frank Quisinsky » Sun Sep 14, 2014 10:27 am

Hi Larry,

I like it so much that K8 is on first position.
If I see that, often I must remembered on Don ...

Leiden in the year 1998 or 1999 (I don't know).

Don sitting on a Notebook (Denksport-Zentrum Leiden) with a network connection and plays with CilkChess. To brace up and go to him and try it in my Frankish. Don know me, laughed - very smart and friendly - and say different things to me but after all I understand (we say in German ... American people sing more if they are speaking ... Texas slang ... means, to fast for me to understand all of it ... unfortunately) Don was with one of the games not really happy. But CilkChess plays very nice games in Leiden and Don was the event here ... I am sure for all of us.

Now one of the programmer from the first time (you too and Mark also) are on first position. That's good for computer chess. Will give the younger programmers a good motivation.

But not very important ... K8 or SF on first position.
Both projects are great and we have two engines with different styles on the first positions. Hope that the development of Komodo will run a long time.

Best
Frank
I like computer chess!

Hai
Posts: 455
Joined: Sun Aug 04, 2013 11:19 am

Re: Komodo 8 on the rating lists.

Post by Hai » Sun Sep 14, 2014 1:10 pm

I miss the: "get involved" button on the Komodo page.

lkaufman
Posts: 3772
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Komodo 8 on the rating lists.

Post by lkaufman » Sun Sep 14, 2014 4:49 pm

Jouni wrote:Congratulations for Komodo team! BTW framework shows now +15,68 for latest SF dev version - next TCEC will be really close.
Thanks. That number agrees well with my own testing, so at this moment K8 and latest SF dev. are equal in strength as far as can be measured. Whether they will still be close in November remains to be seen.

One quick question: Do you use the same parameter settings for dev. testing as the defaults in the official release? In particular, I am asking whether you use safer time control parameters in the official release than in your own testing.
Komodo rules!

lkaufman
Posts: 3772
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Komodo 8 on the rating lists.

Post by lkaufman » Sun Sep 14, 2014 6:27 pm

I should have waited another day for the weekly CEGT list. Now Komodo 8 on 4 threads has enough games at 40/20 to qualify for my list, and it was the best result yet! So the revised ratings on CEGT at 40/20 relative to Komodo 8 at zero are:

Komodo 7 -53, Houdini 4 -59, Stockfish 5 -51.

Revised ratings on one thread for CEGT 40/20 are:

Komodo 7 -40, Houdini 4 -29, Stockfish 5 -8.

Komodo 8 seems to be relatively stronger than the other engines on 4 threads compared to 1.

Revised average ratings on one thread are:
Komodo 7 -38, Houdini 4 -16, Stockfish 5 -12.

Revised ratings on four threads are:
Komodo 7 -57, Houdini 4 -45, Stockfish 5 -37
Komodo rules!

Will Singleton
Posts: 128
Joined: Thu Mar 09, 2006 4:14 pm
Location: Los Angeles, CA

Re: Komodo 8 on the rating lists.

Post by Will Singleton » Wed Sep 17, 2014 5:59 am

Frank,

Nice history, thanks.

Will

Post Reply