Discussion of computer chess matches and engine tournaments.
Moderators: hgm , Rebel , chrisw
Aser Huerga
Posts: 812 Joined: Tue Jun 16, 2009 10:09 am
Location: Spain
Post
by Aser Huerga » Sun Dec 22, 2013 11:16 am
As a Shaun Brewer suggestion, I decided to run my games at different Time Controls to see how the top engines strength change as time increases. Here are the results:
Five
i7-3930K CPUs
4.25 GHz
1 core for all engines
Ponder off
1024 Hash
3-4-5 EGTBs (when available) in
SSDs
Code: Select all
3'+1" Time Control
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 Houdini 4 : 32.1 13.3 340.0 600 56.7%
2 Komodo TCEC : -14.4 13.4 282.0 600 47.0%
3 Stockfish DD : -17.6 13.8 278.0 600 46.3%
9'+3" Time Control
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 Stockfish DD : 16.6 14.1 320.5 600 53.4%
2 Houdini 4 : 8.1 13.5 310.0 600 51.7%
3 Komodo TCEC : -24.7 13.4 269.5 600 44.9%
27'+9" Time Control
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 Stockfish DD : 22.6 14.1 328.0 600 54.7%
2 Houdini 4 : 5.7 13.4 307.0 600 51.2%
3 Komodo TCEC : -28.3 13.4 265.0 600 44.2%
54'+18" Time Control
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 Stockfish DD : 11.4 14.2 314.0 600 52.3%
2 Houdini 4 : 0.4 13.7 300.5 600 50.1%
3 Komodo TCEC : -11.8 13.6 285.5 600 47.6%
90'+30" Time Control
# PLAYER : RATING ERROR POINTS PLAYED (%)
1 Stockfish DD : 10.5 12.9 313.0 600 52.2%
2 Komodo TCEC : 0.8 13.2 301.0 600 50.2%
3 Houdini 4 : -11.3 13.1 286.0 600 47.7%
I want to thanks Adam Hair for his help in the presentation of graph and results.
( All the games can be downloaded here:
TTC_All_Games )
Laskos
Posts: 10948 Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos
Post
by Laskos » Sun Dec 22, 2013 12:04 pm
Thanks, Aser. This confirms that SF doesn't need much to overtake Houdini, Komodo does need some time, and scales well afterwards.
Stefan Schiffermueller
Posts: 12 Joined: Thu Dec 05, 2013 10:48 am
Post
by Stefan Schiffermueller » Sun Dec 22, 2013 2:21 pm
Interesting. What is the value of the contempt-parameter in Houdini 4?
Aser Huerga
Posts: 812 Joined: Tue Jun 16, 2009 10:09 am
Location: Spain
Post
by Aser Huerga » Sun Dec 22, 2013 3:01 pm
Stefan Schiffermueller wrote: Interesting. What is the value of the contempt-parameter in Houdini 4?
Thanks Stefan.
All engines plays at default settings.
PaulieD
Posts: 213 Joined: Tue Jun 25, 2013 8:19 pm
Post
by PaulieD » Sun Dec 22, 2013 3:34 pm
A picture truly is worth a thousand words. That is a wonderful depiction of what a lot of people have been trying to say.
ouachita
Posts: 454 Joined: Tue Jan 15, 2013 4:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson
Post
by ouachita » Sun Dec 22, 2013 3:40 pm
PaulieD wrote: A picture truly is worth a thousand words. That is a wonderful depiction of what a lot of people have been trying to say.
Yep, I've posted numerous time that it looked to me that the crossover was >60+, and I actually worked on the same type of Excel chart. Then I went off on rants saying one cannot accurately extrapolate these 90+ data points out to 40/120, ad infinitum.
SIM, PhD, MBA, PE
ouachita
Posts: 454 Joined: Tue Jan 15, 2013 4:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson
Post
by ouachita » Sun Dec 22, 2013 4:04 pm
However, we can feel safe in saying that:
1. it seems highly unlikely that H's Elo line would ever intersect the other two.
2. the SF and K lines >90 may or may not intersect.
SIM, PhD, MBA, PE
Shaun
Posts: 322 Joined: Wed Mar 08, 2006 9:55 pm
Location: Brighton - UK
Post
by Shaun » Sun Dec 22, 2013 5:31 pm
Thank you!!!
Modern Times
Posts: 3553 Joined: Thu Jun 07, 2012 11:02 pm
Post
by Modern Times » Sun Dec 22, 2013 6:03 pm
Brilliant work Aser, the graph is very enlightening.
Question is, at what point does Komodo level off...
ouachita
Posts: 454 Joined: Tue Jan 15, 2013 4:33 pm
Location: Ritz-Carlton, NYC
Full name: Bobby Johnson
Post
by ouachita » Sun Dec 22, 2013 6:22 pm
Modern Times wrote: Question is, at what point does Komodo level off...
. . . which can only be answered thru LTC (120+) testing. Perhaps a day or day on the cluster would help to answer this question?
SIM, PhD, MBA, PE