ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Elo Increase per Doubling
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions Flat
View previous topic :: View next topic  
Author Message
Adam Hair



Joined: 06 May 2009
Posts: 1657
Location: Fuquay-Varina, North Carolina

PostPost subject: Elo Increase per Doubling    Posted: Mon May 07, 2012 11:40 am Reply to topic Reply with quote

In the past few months, several people (such as Peter Österlund and myself) have measured the increase in Elo per doubling of time control. The increases discovered in each case was, on average, much higher than the often quoted range of 50 to 70 Elo per doubling of speed (which is emulated by doubling the time control). However, one objection that has been raised is that the time controls are too short. While I do not believe the previous results were invalid, I do know that the short base time control added noise to the measurement. So, I decided to test the Elo increase per doubling at a longer base time control.

    I used Gaviota 0.85.1 64-bit as the engine to measure.

    Gaviota played against 10 opponents (Jonny 4.00, SmarThink 1.20 64-bit, Glaurung 2.2 64-bit, Booot 5.1.0, Quazar 0.4 64-bit, Nemo 1.01b 64-bit, Naum 4.2 64-bit, Gull 1.0a 64-bit, Spike 1.4, Hannibal 1.1 64-bit).

    The base time control was 1 minute + 1 second per move.

    I randomly choice 100 positions in epd format to use for all games, with each position played with reversed colors so that there was 200 games played for each match.

    Gaviota played a gauntlet against the opponents at the base time control, and then it played with 2 times, 4 times, and 8 times the base time control


The following ratings were computed with Bayeselo with the commands "mm 0 1" and "covariance":

Code:
Rank Name                       Elo    +    - games score oppo. draws
   1 Naum 4.2 64-bit            163   21   21   800   74%   -24   27%
   2 Gull 1.0a 64-bit           113   20   20   800   68%   -24   28%
   3 Gaviota 0.85.1 64-bit(8)    97   12   12  2000   62%    10   30%
   4 Quazar 0.4 64-bit           39   19   19   800   59%   -24   32%
   5 Hannibal 1.1 64-bit         33   19   19   800   58%   -24   31%
   6 Spike 1.4                   33   19   19   800   58%   -24   31%
   7 Gaviota 0.85.1 64-bit(4)    20   12   12  2000   51%    10   32%
   8 Nemo 1.01 beta 64-bit       -6   19   19   800   52%   -24   32%
   9 Booot 5.1.0                -35   19   19   801   48%   -24   33%
  10 Glaurung 2.2 JA 64-bit     -43   19   19   800   47%   -24   25%
  11 Gaviota 0.85.1 64-bit(2)   -65   12   12  2001   40%    10   28%
  12 SmarThink 1.20 64-bit      -99   20   20   800   40%   -24   25%
  13 Jonny 4.00                -100   20   20   800   40%   -24   26%
  14 Gaviota 0.85.1 64-bit     -148   13   13  2000   29%    10   26%


As it can be seen, the increase in Elo per doubling was 83, 85, and 77 Elo. The increase in winning percentage for each doubling was 11%.

I also performed a RR tournament with Gaviota's opponents. When I include those games, the ratings are as follows:

Code:
Rank Name                       Elo    +    - games score oppo. draws
   1 Naum 4.2 64-bit            165   11   11  2600   73%   -13   28%
   2 Gull 1.0a 64-bit           103   10   10  2600   65%    -8   32%
   3 Gaviota 0.85.1 64-bit(8)    98   12   12  2000   62%    10   30%
   4 Spike 1.4                   49   10   10  2600   57%    -4   32%
   5 Hannibal 1.1 64-bit         40   10   10  2600   56%    -3   34%
   6 Quazar 0.4 64-bit           22   10   10  2600   53%    -2   35%
   7 Gaviota 0.85.1 64-bit(4)    21   12   12  2000   51%    10   32%
   8 Nemo 1.01 beta 64-bit      -10   10   10  2600   48%     1   32%
   9 Booot 5.1.0                -29   10   10  2601   46%     2   34%
  10 Glaurung 2.2 JA 64-bit     -47   10   10  2600   43%     4   29%
  11 Gaviota 0.85.1 64-bit(2)   -65   12   12  2001   40%    10   28%
  12 Jonny 4.00                 -95   11   11  2600   36%     7   27%
  13 SmarThink 1.20 64-bit     -102   11   11  2600   35%     8   26%
  14 Gaviota 0.85.1 64-bit     -149   13   13  2000   29%    10   26%


The increase per doubling is now 84, 86, and 77 Elo.

From my previous test, using a base time control of 6 seconds + 0.1 seconds per move , I found that Gaviota 0.84 gained 104 Elo per doubling (on average). Though the method of measurement was different between the two tests, and two different versions of Gaviota are being compared, it may not be unreasonable to claim that this shows that:

1) At depths higher than those used to determine the estimate of 50 to 70 Elo per doubling of speed originally ("How Computers Play Chess" by David Levy and Monty Newborn ?), the expected increase per doubling for modern engines is greater than the quoted numbers.

2) The expected increase per doubling of speed may decrease at higher time controls. If the base time control was 120 minutes + 90 seconds per move, the measured Elo increase per doubling might be 40 to 70 Elo.

All the games used to compute the ratings can be found here:

http://www.mediafire.com/file/guwf0e3x9gvad5r/Gaviota_Time_Odds.7z
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Subject Author Date/Time
Elo Increase per Doubling Adam Hair Mon May 07, 2012 11:40 am
      Re: Elo Increase per Doubling Julien MARCEL Mon May 07, 2012 12:04 pm
            Re: Elo Increase per Doubling Adam Hair Mon May 07, 2012 4:48 pm
                  Re: Elo Increase per Doubling Julien MARCEL Mon May 07, 2012 4:54 pm
            Re: Elo Increase per Doubling Adam Hair Mon May 07, 2012 8:50 pm
                  Re: Elo Increase per Doubling Julien MARCEL Mon May 07, 2012 9:00 pm
                        Re: Elo Increase per Doubling Mark Pearce Thu May 10, 2012 4:43 pm
      Re: Elo Increase per Doubling H.G.Muller Mon May 07, 2012 12:13 pm
            Re: Elo Increase per Doubling Michel Van den Bergh Mon May 07, 2012 2:01 pm
            Re: Elo Increase per Doubling Adam Hair Mon May 07, 2012 5:12 pm
                  Re: Elo Increase per Doubling Don Dailey Mon May 07, 2012 6:40 pm
                        Re: Elo Increase per Doubling Robert Wagner Campos Tue May 08, 2012 12:38 am
                              Re: Elo Increase per Doubling Don Dailey Tue May 08, 2012 3:32 am
                                    Re: Elo Increase per Doubling Adam Hair Tue May 08, 2012 10:02 am
                                          Re: Elo Increase per Doubling Julien MARCEL Tue May 08, 2012 10:55 am
                                                Re: Elo Increase per Doubling Adam Hair Tue May 08, 2012 8:08 pm
                                          Re: Elo Increase per Doubling Don Dailey Tue May 08, 2012 11:29 am
                                          Re: Elo Increase per Doubling Kai Laskos Tue May 08, 2012 11:59 am
                                                Re: Elo Increase per Doubling Adam Hair Tue May 08, 2012 8:13 pm
                                                      Re: Elo Increase per Doubling Kai Laskos Tue May 08, 2012 10:21 pm
                                                            Re: Elo Increase per Doubling Don Dailey Tue May 08, 2012 10:52 pm
                                                            Re: Elo Increase per Doubling Adam Hair Tue May 08, 2012 11:14 pm
                                                                  Re: Elo Increase per Doubling Kai Laskos Wed May 09, 2012 12:20 am
                                                                        Re: Elo Increase per Doubling Adam Hair Wed May 09, 2012 2:44 am
                                                                              Re: Elo Increase per Doubling H.G.Muller Wed May 09, 2012 8:05 am
                                                                                    Re: Elo Increase per Doubling Adam Hair Wed May 09, 2012 5:12 pm
                                                                                          Re: Elo Increase per Doubling Larry Kaufman Thu May 10, 2012 3:11 am
                                                                                          Re: Elo Increase per Doubling Don Dailey Thu May 10, 2012 5:03 am
                                                                                          Re: Elo Increase per Doubling Larry Kaufman Thu May 10, 2012 5:19 am
                                                                                          Re: Elo Increase per Doubling Don Dailey Thu May 10, 2012 12:43 pm
                                                                                          Re: Elo Increase per Doubling Peter Österlund Thu May 10, 2012 4:59 pm
                                                                                          Re: Elo Increase per Doubling Julien MARCEL Thu May 10, 2012 5:09 pm
                                                                                          Re: Elo Increase per Doubling Don Dailey Thu May 10, 2012 5:41 pm
                                                                                          Re: Elo Increase per Doubling Don Dailey Thu May 10, 2012 5:46 pm
                                          Re: Elo Increase per Doubling Ed Schroder Tue May 08, 2012 12:09 pm
                                                Re: Elo Increase per Doubling Don Dailey Tue May 08, 2012 12:10 pm
                                                      Re: Elo Increase per Doubling Ed Schroder Tue May 08, 2012 12:28 pm
                                                            Re: Elo Increase per Doubling Don Dailey Tue May 08, 2012 12:36 pm
                                                                  Re: Elo Increase per Doubling Ed Schroder Tue May 08, 2012 1:15 pm
                                                                        Re: Elo Increase per Doubling Don Dailey Tue May 08, 2012 2:37 pm
                                                                              Re: Elo Increase per Doubling Julien MARCEL Tue May 08, 2012 3:15 pm
                                                                                    Re: Elo Increase per Doubling Don Dailey Tue May 08, 2012 6:15 pm
                                                                                          Re: Elo Increase per Doubling Peter Österlund Tue May 08, 2012 8:06 pm
                                                                                          Re: Elo Increase per Doubling Don Dailey Tue May 08, 2012 8:25 pm
                                                                                          Re: Elo Increase per Doubling Peter Österlund Tue May 08, 2012 9:00 pm
                                                                                          Re: Elo Increase per Doubling Julien MARCEL Tue May 08, 2012 9:02 pm
                                                                                          Re: Elo Increase per Doubling Don Dailey Tue May 08, 2012 9:06 pm
                                                                                          Re: Elo Increase per Doubling Don Dailey Wed May 09, 2012 4:26 pm
                                                                                          Re: Elo Increase per Doubling Peter Österlund Wed May 09, 2012 4:52 pm
                                                                                          Re: Elo Increase per Doubling Don Dailey Wed May 09, 2012 5:19 pm
                                                                                          Re: Elo Increase per Doubling Don Dailey Thu May 10, 2012 1:41 pm
                                                                                          Re: Elo Increase per Doubling Don Dailey Thu May 10, 2012 4:52 pm
                                                                                          Re: Elo Increase per Doubling Adam Hair Thu May 10, 2012 9:07 pm
                                                                                          Re: Elo Increase per Doubling Don Dailey Thu May 10, 2012 10:55 pm
                                                                                          Re: Elo Increase per Doubling Adam Hair Fri May 11, 2012 12:46 am
                                                                                          Re: Elo Increase per Doubling Adam Hair Fri May 11, 2012 1:06 am
                                                                                          Re: Elo Increase per Doubling Don Dailey Wed May 09, 2012 5:00 pm
                                                                                          Re: Elo Increase per Doubling Adam Hair Tue May 08, 2012 9:05 pm
                                                                                          Re: Elo Increase per Doubling Don Dailey Tue May 08, 2012 9:18 pm
                                                                                          Re: Elo Increase per Doubling Adam Hair Tue May 08, 2012 11:26 pm
                                                                                          Re: Elo Increase per Doubling Don Dailey Wed May 09, 2012 2:50 am
                                                                                          Re: Elo Increase per Doubling Adam Hair Tue May 15, 2012 3:17 am
                                                                                    Re: Elo Increase per Doubling Don Dailey Tue May 08, 2012 6:23 pm
                                          Re: Elo Increase per Doubling Ed Schroder Tue May 08, 2012 12:18 pm
                                                Re: Elo Increase per Doubling Don Dailey Tue May 08, 2012 12:25 pm
                                                      Re: Elo Increase per Doubling Don Dailey Tue May 08, 2012 12:27 pm
                                                            Re: Elo Increase per Doubling Robert Hyatt Tue May 08, 2012 3:36 pm
                                                                  Re: Elo Increase per Doubling Don Dailey Tue May 08, 2012 5:32 pm
      Re: Elo Increase per Doubling Daniel Shawul Fri May 11, 2012 5:48 pm
            Re: Elo Increase per Doubling Don Dailey Fri May 11, 2012 6:30 pm
                  Re: Elo Increase per Doubling Julien MARCEL Fri May 11, 2012 6:52 pm
                        Re: Elo Increase per Doubling Don Dailey Fri May 11, 2012 7:05 pm
                  Re: Elo Increase per Doubling Daniel Shawul Fri May 11, 2012 7:08 pm
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads