Over the years I've seen various estimates for the amount of Elo gained (or lost) for a doubling (or halving) of processor time. Larry Kaufman once said it was 60 points which seems too low; I have seen 72 points and 75 points, and somewhere I recall even 100 points which seems too high.
Also if your opponent thinks on your time (whether the opponent is a human or machine), then giving a processor twice as much more time is not the same as doubling the clock, and should give an Elo rating difference less than doubling the clock.
How many Elo points change for a doubling of uP time?
Moderators: hgm, Rebel, chrisw
-
- Posts: 18
- Joined: Mon Aug 11, 2014 7:36 am
-
- Posts: 18
- Joined: Mon Aug 11, 2014 7:36 am
Re: How many Elo points change for a doubling of uP time?
I found the reference to 100 points: it was by Monty Newborn on p.257 in his 2011 book "Beyond Deep Blue" and he further does not give much if any practical difference between doubling the time and doubling the processor speed.
-
- Posts: 10297
- Joined: Thu Mar 09, 2006 12:37 am
- Location: Tel-Aviv Israel
Re: How many Elo points change for a doubling of uP time?
I think that it is dependent on time control and on software and hardware so no answer.JayRod wrote:Over the years I've seen various estimates for the amount of Elo gained (or lost) for a doubling (or halving) of processor time. Larry Kaufman once said it was 60 points which seems too low; I have seen 72 points and 75 points, and somewhere I recall even 100 points which seems too high.
Also if your opponent thinks on your time (whether the opponent is a human or machine), then giving a processor twice as much more time is not the same as doubling the clock, and should give an Elo rating difference less than doubling the clock.
basically you have the following:
1)better software->more elo points for doubling
2)longer time control or better hardware->less elo points for doubling.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: How many Elo points change for a doubling of uP time?
http://www.talkchess.com/forum/viewtopic.php?t=48733JayRod wrote:Over the years I've seen various estimates for the amount of Elo gained (or lost) for a doubling (or halving) of processor time. Larry Kaufman once said it was 60 points which seems too low; I have seen 72 points and 75 points, and somewhere I recall even 100 points which seems too high.
Also if your opponent thinks on your time (whether the opponent is a human or machine), then giving a processor twice as much more time is not the same as doubling the clock, and should give an Elo rating difference less than doubling the clock.
-
- Posts: 18
- Joined: Mon Aug 11, 2014 7:36 am
Re: How many Elo points change for a doubling of uP time?
Wow! Thanks Kai Lakos! This logarithmic graph was very informative and actually ties into what my next question was, which I hope somebody can answer.
I notice when I play a strong chess engine at progressively more halving of time, there is a non-linear relationship that does not obey the "75 points for every halving of time" rule.
Let me explain:
You take a strong engine, or really any chess engine, and see what the Elo is at 180 seconds a move average. Then you see what the Elo is at half the time, or 90 seconds a move average. Then you half again to 45 seconds, check the Elo, then 22.5, again check Elo, then 11.25. What you find is that the Elo is non-linear, and higher than expected from a simple linear interpolation approximation.
So, in table form:
(Scenario I, what I seem to observe):
Time --> Expected Elo (75 points less per halving of time) --> Actual Elo (I make this number up but it gives a rough idea of what I am talking about)
180 seconds --> 3000 Elo (expected) --> 3000 Elo (actual)
90 seconds --> 2925 Elo --> 2925 Elo
45 seconds --> 2850 Elo --> 2890 Elo
22.5 seconds --> 2775 Elo --> 2860 Elo
11.25 seconds --> 2700 Elo --> 2835 Elo
Anybody else can confirm this? Is this relationship what you have observed too, or does the actual Elo drop faster than expected?
That is, do you observe this instead? (Scenario II, doubtful, speculative):
180 seconds --> 3000 Elo (expected) --> 3000 Elo (actual)
90 seconds --> 2925 Elo --> 2975 Elo
45 seconds --> 2850 Elo --> 2875 Elo
22.5 seconds --> 2775 Elo --> 2700 Elo
11.25 seconds --> 2700 Elo --> 2450 Elo
Or, do you see a linear relationship, Scenario III, with a fixed Elo drop for every halving of time for actual Elo?
Which of the three scenarios do you see? I've not researched this, but my experience seems to be Scenario I.
JayRod
I notice when I play a strong chess engine at progressively more halving of time, there is a non-linear relationship that does not obey the "75 points for every halving of time" rule.
Let me explain:
You take a strong engine, or really any chess engine, and see what the Elo is at 180 seconds a move average. Then you see what the Elo is at half the time, or 90 seconds a move average. Then you half again to 45 seconds, check the Elo, then 22.5, again check Elo, then 11.25. What you find is that the Elo is non-linear, and higher than expected from a simple linear interpolation approximation.
So, in table form:
(Scenario I, what I seem to observe):
Time --> Expected Elo (75 points less per halving of time) --> Actual Elo (I make this number up but it gives a rough idea of what I am talking about)
180 seconds --> 3000 Elo (expected) --> 3000 Elo (actual)
90 seconds --> 2925 Elo --> 2925 Elo
45 seconds --> 2850 Elo --> 2890 Elo
22.5 seconds --> 2775 Elo --> 2860 Elo
11.25 seconds --> 2700 Elo --> 2835 Elo
Anybody else can confirm this? Is this relationship what you have observed too, or does the actual Elo drop faster than expected?
That is, do you observe this instead? (Scenario II, doubtful, speculative):
180 seconds --> 3000 Elo (expected) --> 3000 Elo (actual)
90 seconds --> 2925 Elo --> 2975 Elo
45 seconds --> 2850 Elo --> 2875 Elo
22.5 seconds --> 2775 Elo --> 2700 Elo
11.25 seconds --> 2700 Elo --> 2450 Elo
Or, do you see a linear relationship, Scenario III, with a fixed Elo drop for every halving of time for actual Elo?
Which of the three scenarios do you see? I've not researched this, but my experience seems to be Scenario I.
JayRod
-
- Posts: 900
- Joined: Tue Apr 27, 2010 3:48 pm
Re: How many Elo points change for a doubling of uP time?
I expect that it's dependent on how long the time control is.
If two engines are playing very long games with good hardware, the number of draws will go up and therefore the advantage of playing better chess is smaller.
In very fast games, the quality of the engine shows more as there are fewer draws.
If two engines are playing very long games with good hardware, the number of draws will go up and therefore the advantage of playing better chess is smaller.
In very fast games, the quality of the engine shows more as there are fewer draws.
-
- Posts: 3226
- Joined: Wed May 06, 2009 10:31 pm
- Location: Fuquay-Varina, North Carolina
Re: How many Elo points change for a doubling of uP time?
In this thread you will find multiple results for tests that examine the change in Elo as you double (or halve) the search time or change the number of plies searched (a related subject). Each one is consistent with scenario II. Elo(2x seconds) - Elo(x seconds) decreases as x increases.JayRod wrote:Wow! Thanks Kai Lakos! This logarithmic graph was very informative and actually ties into what my next question was, which I hope somebody can answer.
I notice when I play a strong chess engine at progressively more halving of time, there is a non-linear relationship that does not obey the "75 points for every halving of time" rule.
Let me explain:
You take a strong engine, or really any chess engine, and see what the Elo is at 180 seconds a move average. Then you see what the Elo is at half the time, or 90 seconds a move average. Then you half again to 45 seconds, check the Elo, then 22.5, again check Elo, then 11.25. What you find is that the Elo is non-linear, and higher than expected from a simple linear interpolation approximation.
So, in table form:
(Scenario I, what I seem to observe):
Time --> Expected Elo (75 points less per halving of time) --> Actual Elo (I make this number up but it gives a rough idea of what I am talking about)
180 seconds --> 3000 Elo (expected) --> 3000 Elo (actual)
90 seconds --> 2925 Elo --> 2925 Elo
45 seconds --> 2850 Elo --> 2890 Elo
22.5 seconds --> 2775 Elo --> 2860 Elo
11.25 seconds --> 2700 Elo --> 2835 Elo
Anybody else can confirm this? Is this relationship what you have observed too, or does the actual Elo drop faster than expected?
That is, do you observe this instead? (Scenario II, doubtful, speculative):
180 seconds --> 3000 Elo (expected) --> 3000 Elo (actual)
90 seconds --> 2925 Elo --> 2975 Elo
45 seconds --> 2850 Elo --> 2875 Elo
22.5 seconds --> 2775 Elo --> 2700 Elo
11.25 seconds --> 2700 Elo --> 2450 Elo
Or, do you see a linear relationship, Scenario III, with a fixed Elo drop for every halving of time for actual Elo?
Which of the three scenarios do you see? I've not researched this, but my experience seems to be Scenario I.
JayRod
-
- Posts: 27808
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: How many Elo points change for a doubling of uP time?
Note that this can be an articaft from the increase in the draw rate at high playing level, which makes the conventional Elo definition a poor measure for the quality of play. Chess is a drawish game, so when you exceed a certain quality of play you might be able to draw no matter how much better your opponent is.
-
- Posts: 3226
- Joined: Wed May 06, 2009 10:31 pm
- Location: Fuquay-Varina, North Carolina
Re: How many Elo points change for a doubling of uP time?
Yes, it is an artifact. If you do not include draws in the calculation, the decrease in rating points per doubling is much less as the quality of play increases.hgm wrote:Note that this can be an articaft from the increase in the draw rate at high playing level, which makes the conventional Elo definition a poor measure for the quality of play.
I have been collecting data using nodes in place of a time control. The following data is the measured difference in strength between Gaviota playing at x nodes per move and 2x nodes per move:
With draws
Code: Select all
Nodes (in thousands) rating difference
128 124.4
112 122.5
96 123.1
80 133.1
64 126.2
56 131.5
48 138.6
40 138.5
32 146.4
28 140.1
24 147.3
20 146.1
16 151.5
14 157.6
12 154.9
10 159.1
9 161.1
8 164.6
7 169.4
6 170.3
5 177
4.5 179.9
4 178.4
3 186.6
2 194.4
Code: Select all
Nodes (in thousands) rating difference
128 245.3
112 234.8
96 234
80 252.1
64 234.4
56 243.9
48 251.2
40 247.7
32 254.4
28 241.4
24 253.9
20 249.1
16 252.5
14 264.2
12 254.5
10 256.8
9 255
8 263.1
7 262.1
6 257.3
5 263
4.5 265.5
4 259.5
3 266.5
2 265.4
I think that this is supported by the data from the rating lists. The difference in strength of two engines has a much smaller effect on the draw rate than the average strength.hgm wrote: Chess is a drawish game, so when you exceed a certain quality of play you might be able to draw no matter how much better your opponent is.
-
- Posts: 6401
- Joined: Thu Mar 09, 2006 8:30 pm
- Location: Chicago, Illinois, USA
Re: How many Elo points change for a doubling of uP time?
That is exactly what the problem is. We have been working with Adam on collecting some data, and the basic conclusion is that the _changes_ throughout the scale in draw rate screw the ratings. I have a theory of a new rating system in which the differences in "rating number" won't predict the performance in a match, but the ratio W/L. The semi-absolute number will predict the draw rate (in a way, it is an expansion of the current Ordo model with different "energy" levels). At one point this will become Ordo II or whatever.hgm wrote:Note that this can be an articaft from the increase in the draw rate at high playing level, which makes the conventional Elo definition a poor measure for the quality of play. Chess is a drawish game, so when you exceed a certain quality of play you might be able to draw no matter how much better your opponent is.
The diminishing returns we observe in the curve Strength vs log(time) are in great part a problem that reflects not a real deviation, but that we are not calculating "strength" properly. We have seen this in small scale and when things are corrected, they become shockingly linear. [EDIT: I see that Adam just posted it above while I was writing]
Miguel