How many Elo points change for a doubling of uP time?

Discussion of chess software programming and technical issues.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
JayRod
Posts: 18
Joined: Mon Aug 11, 2014 5:36 am

How many Elo points change for a doubling of uP time?

Post by JayRod » Thu Aug 14, 2014 8:08 am

Over the years I've seen various estimates for the amount of Elo gained (or lost) for a doubling (or halving) of processor time. Larry Kaufman once said it was 60 points which seems too low; I have seen 72 points and 75 points, and somewhere I recall even 100 points which seems too high.

Also if your opponent thinks on your time (whether the opponent is a human or machine), then giving a processor twice as much more time is not the same as doubling the clock, and should give an Elo rating difference less than doubling the clock.

JayRod
Posts: 18
Joined: Mon Aug 11, 2014 5:36 am

Re: How many Elo points change for a doubling of uP time?

Post by JayRod » Thu Aug 14, 2014 8:12 am

I found the reference to 100 points: it was by Monty Newborn on p.257 in his 2011 book "Beyond Deep Blue" and he further does not give much if any practical difference between doubling the time and doubling the processor speed.

Uri Blass
Posts: 8942
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: How many Elo points change for a doubling of uP time?

Post by Uri Blass » Thu Aug 14, 2014 9:13 am

JayRod wrote:Over the years I've seen various estimates for the amount of Elo gained (or lost) for a doubling (or halving) of processor time. Larry Kaufman once said it was 60 points which seems too low; I have seen 72 points and 75 points, and somewhere I recall even 100 points which seems too high.

Also if your opponent thinks on your time (whether the opponent is a human or machine), then giving a processor twice as much more time is not the same as doubling the clock, and should give an Elo rating difference less than doubling the clock.
I think that it is dependent on time control and on software and hardware so no answer.

basically you have the following:
1)better software->more elo points for doubling
2)longer time control or better hardware->less elo points for doubling.

User avatar
Laskos
Posts: 10949
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: How many Elo points change for a doubling of uP time?

Post by Laskos » Thu Aug 14, 2014 10:04 am

JayRod wrote:Over the years I've seen various estimates for the amount of Elo gained (or lost) for a doubling (or halving) of processor time. Larry Kaufman once said it was 60 points which seems too low; I have seen 72 points and 75 points, and somewhere I recall even 100 points which seems too high.

Also if your opponent thinks on your time (whether the opponent is a human or machine), then giving a processor twice as much more time is not the same as doubling the clock, and should give an Elo rating difference less than doubling the clock.
http://www.talkchess.com/forum/viewtopic.php?t=48733

JayRod
Posts: 18
Joined: Mon Aug 11, 2014 5:36 am

Re: How many Elo points change for a doubling of uP time?

Post by JayRod » Sat Aug 30, 2014 7:54 am

Wow! Thanks Kai Lakos! This logarithmic graph was very informative and actually ties into what my next question was, which I hope somebody can answer.

I notice when I play a strong chess engine at progressively more halving of time, there is a non-linear relationship that does not obey the "75 points for every halving of time" rule.

Let me explain:

You take a strong engine, or really any chess engine, and see what the Elo is at 180 seconds a move average. Then you see what the Elo is at half the time, or 90 seconds a move average. Then you half again to 45 seconds, check the Elo, then 22.5, again check Elo, then 11.25. What you find is that the Elo is non-linear, and higher than expected from a simple linear interpolation approximation.

So, in table form:

(Scenario I, what I seem to observe):

Time --> Expected Elo (75 points less per halving of time) --> Actual Elo (I make this number up but it gives a rough idea of what I am talking about)

180 seconds --> 3000 Elo (expected) --> 3000 Elo (actual)

90 seconds --> 2925 Elo --> 2925 Elo

45 seconds --> 2850 Elo --> 2890 Elo

22.5 seconds --> 2775 Elo --> 2860 Elo

11.25 seconds --> 2700 Elo --> 2835 Elo

Anybody else can confirm this? Is this relationship what you have observed too, or does the actual Elo drop faster than expected?

That is, do you observe this instead? (Scenario II, doubtful, speculative):

180 seconds --> 3000 Elo (expected) --> 3000 Elo (actual)

90 seconds --> 2925 Elo --> 2975 Elo

45 seconds --> 2850 Elo --> 2875 Elo

22.5 seconds --> 2775 Elo --> 2700 Elo

11.25 seconds --> 2700 Elo --> 2450 Elo

Or, do you see a linear relationship, Scenario III, with a fixed Elo drop for every halving of time for actual Elo?

Which of the three scenarios do you see? I've not researched this, but my experience seems to be Scenario I.

JayRod

rbarreira
Posts: 900
Joined: Tue Apr 27, 2010 1:48 pm

Re: How many Elo points change for a doubling of uP time?

Post by rbarreira » Sat Aug 30, 2014 9:04 am

I expect that it's dependent on how long the time control is.

If two engines are playing very long games with good hardware, the number of draws will go up and therefore the advantage of playing better chess is smaller.

In very fast games, the quality of the engine shows more as there are fewer draws.

Adam Hair
Posts: 3226
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: How many Elo points change for a doubling of uP time?

Post by Adam Hair » Sat Aug 30, 2014 9:16 am

JayRod wrote:Wow! Thanks Kai Lakos! This logarithmic graph was very informative and actually ties into what my next question was, which I hope somebody can answer.

I notice when I play a strong chess engine at progressively more halving of time, there is a non-linear relationship that does not obey the "75 points for every halving of time" rule.

Let me explain:

You take a strong engine, or really any chess engine, and see what the Elo is at 180 seconds a move average. Then you see what the Elo is at half the time, or 90 seconds a move average. Then you half again to 45 seconds, check the Elo, then 22.5, again check Elo, then 11.25. What you find is that the Elo is non-linear, and higher than expected from a simple linear interpolation approximation.

So, in table form:

(Scenario I, what I seem to observe):

Time --> Expected Elo (75 points less per halving of time) --> Actual Elo (I make this number up but it gives a rough idea of what I am talking about)

180 seconds --> 3000 Elo (expected) --> 3000 Elo (actual)

90 seconds --> 2925 Elo --> 2925 Elo

45 seconds --> 2850 Elo --> 2890 Elo

22.5 seconds --> 2775 Elo --> 2860 Elo

11.25 seconds --> 2700 Elo --> 2835 Elo

Anybody else can confirm this? Is this relationship what you have observed too, or does the actual Elo drop faster than expected?

That is, do you observe this instead? (Scenario II, doubtful, speculative):

180 seconds --> 3000 Elo (expected) --> 3000 Elo (actual)

90 seconds --> 2925 Elo --> 2975 Elo

45 seconds --> 2850 Elo --> 2875 Elo

22.5 seconds --> 2775 Elo --> 2700 Elo

11.25 seconds --> 2700 Elo --> 2450 Elo

Or, do you see a linear relationship, Scenario III, with a fixed Elo drop for every halving of time for actual Elo?

Which of the three scenarios do you see? I've not researched this, but my experience seems to be Scenario I.

JayRod
In this thread you will find multiple results for tests that examine the change in Elo as you double (or halve) the search time or change the number of plies searched (a related subject). Each one is consistent with scenario II. Elo(2x seconds) - Elo(x seconds) decreases as x increases.

User avatar
hgm
Posts: 25841
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: How many Elo points change for a doubling of uP time?

Post by hgm » Sat Aug 30, 2014 10:05 am

Note that this can be an articaft from the increase in the draw rate at high playing level, which makes the conventional Elo definition a poor measure for the quality of play. Chess is a drawish game, so when you exceed a certain quality of play you might be able to draw no matter how much better your opponent is.

Adam Hair
Posts: 3226
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: How many Elo points change for a doubling of uP time?

Post by Adam Hair » Sat Aug 30, 2014 12:07 pm

hgm wrote:Note that this can be an articaft from the increase in the draw rate at high playing level, which makes the conventional Elo definition a poor measure for the quality of play.
Yes, it is an artifact. If you do not include draws in the calculation, the decrease in rating points per doubling is much less as the quality of play increases.

I have been collecting data using nodes in place of a time control. The following data is the measured difference in strength between Gaviota playing at x nodes per move and 2x nodes per move:

With draws

Code: Select all

Nodes (in thousands)	rating difference
128	124.4
112	122.5
96	123.1
80	133.1
64	126.2
56	131.5
48	138.6
40	138.5
32	146.4
28	140.1
24	147.3
20	146.1
16	151.5
14	157.6
12	154.9
10	159.1
9	161.1
8	164.6
7	169.4
6	170.3
5	177
4.5	179.9
4	178.4
3	186.6
2	194.4
Without draws

Code: Select all

Nodes (in thousands)	rating difference
128	245.3
112	234.8
96	234
80	252.1
64	234.4
56	243.9
48	251.2
40	247.7
32	254.4
28	241.4
24	253.9
20	249.1
16	252.5
14	264.2
12	254.5
10	256.8
9	255
8	263.1
7	262.1
6	257.3
5	263
4.5	265.5
4	259.5
3	266.5
2	265.4
hgm wrote: Chess is a drawish game, so when you exceed a certain quality of play you might be able to draw no matter how much better your opponent is.
I think that this is supported by the data from the rating lists. The difference in strength of two engines has a much smaller effect on the draw rate than the average strength.

User avatar
michiguel
Posts: 6401
Joined: Thu Mar 09, 2006 7:30 pm
Location: Chicago, Illinois, USA
Contact:

Re: How many Elo points change for a doubling of uP time?

Post by michiguel » Sat Aug 30, 2014 12:30 pm

hgm wrote:Note that this can be an articaft from the increase in the draw rate at high playing level, which makes the conventional Elo definition a poor measure for the quality of play. Chess is a drawish game, so when you exceed a certain quality of play you might be able to draw no matter how much better your opponent is.
That is exactly what the problem is. We have been working with Adam on collecting some data, and the basic conclusion is that the _changes_ throughout the scale in draw rate screw the ratings. I have a theory of a new rating system in which the differences in "rating number" won't predict the performance in a match, but the ratio W/L. The semi-absolute number will predict the draw rate (in a way, it is an expansion of the current Ordo model with different "energy" levels). At one point this will become Ordo II or whatever.

The diminishing returns we observe in the curve Strength vs log(time) are in great part a problem that reflects not a real deviation, but that we are not calculating "strength" properly. We have seen this in small scale and when things are corrected, they become shockingly linear. [EDIT: I see that Adam just posted it above while I was writing]

Miguel

Post Reply