How much elo is pondering worth?

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Michel
Posts: 2036
Joined: Sun Sep 28, 2008 11:50 pm

How much elo is pondering worth?

Post by Michel » Wed Aug 07, 2013 9:58 am

Has this question been researched?

Adam Hair
Posts: 3201
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: How much elo is pondering worth?

Post by Adam Hair » Wed Aug 07, 2013 10:20 am

I have one scant piece of data related to your question. Playing 750 games each at the CCRL blitz time control against a gauntlet of opponents, Gaviota 0.86 with ponder on measured 66 Elo stronger than ponder off (as measured by Ordo).

Michel
Posts: 2036
Joined: Sun Sep 28, 2008 11:50 pm

Re: How much elo is pondering worth?

Post by Michel » Wed Aug 07, 2013 10:27 am

Thank you!

66 elo is a lot. You would think that pondering at most doubles your
thinking time. So 70 elo should be an upperbound.

It appears that Gaviota reaches this upperbound in your experiment but of course there are error bars to be considered which are quite large for
750 games.

User avatar
Laskos
Posts: 9246
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: How much elo is pondering worth?

Post by Laskos » Wed Aug 07, 2013 10:40 am

Michel wrote:Thank you!

66 elo is a lot. You would think that pondering at most doubles your
thinking time. So 70 elo should be an upperbound.

It appears that Gaviota reaches this upperbound in your experiment but of course there are error bars to be considered which are quite large for
750 games.
If 70 points is the doubling, isn't pondering worth significantly less? The ponder hits are about 60% for unrelated engines, so I would expect ~45 points if 70 is the doubling.

User avatar
lucasart
Posts: 3035
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: How much elo is pondering worth?

Post by lucasart » Wed Aug 07, 2013 10:50 am

Doubling is probably worth more than 70 elo. Although it depends on the time control and the absolute strength of the engine at hand (diminishing returns effect on both variable).
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

User avatar
Laskos
Posts: 9246
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: How much elo is pondering worth?

Post by Laskos » Wed Aug 07, 2013 10:53 am

lucasart wrote:Doubling is probably worth more than 70 elo. Although it depends on the time control and the absolute strength of the engine at hand (diminishing returns effect on both variable).
I measured that, at 40/4' CCRL and CEGT one core doubling is 70 Elo points for Houdini 3. By extrapolation, doubling at 40/40' CCRL and CEGT one core is worth 55 points, and 40/120' 1 core 45 Elo points.
http://talkchess.com/forum/viewtopic.ph ... 3&start=12
Last edited by Laskos on Wed Aug 07, 2013 10:57 am, edited 1 time in total.

Michel
Posts: 2036
Joined: Sun Sep 28, 2008 11:50 pm

Re: How much elo is pondering worth?

Post by Michel » Wed Aug 07, 2013 10:57 am

f 70 points is the doubling, isn't pondering worth significantly less? The ponder hits are about 60% for unrelated engines, so I would expect ~45 points if 70 is the doubling.
70 elo/doubling is a rule of thumb that appears to be generally accepted (it has been reported here that it is not valid for very fast time controls; but Adam's experiment was for CCRL blitz time controls which are certainly not "very fast").

45 elo would be compatible with the error bars in Adam's experiment.

User avatar
Laskos
Posts: 9246
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: How much elo is pondering worth?

Post by Laskos » Wed Aug 07, 2013 11:02 am

Michel wrote:
f 70 points is the doubling, isn't pondering worth significantly less? The ponder hits are about 60% for unrelated engines, so I would expect ~45 points if 70 is the doubling.
70 elo/doubling is a rule of thumb that appears to be generally accepted (it has been reported here that it is not valid for very fast time controls; but Adam's experiment was for CCRL blitz time controls which are certainly not "very fast").

45 elo would be compatible with the error bars in Adam's experiment.
Diminishing returns are very visible, at ultra-short controls doubling can be 150 points, at blitz CCRL and CEGT 70 points, and on 8 core 40/120' 35 points. So, one has to assume 40/4' on a pretty old CCRL and CEGT core to talk of 70 points for doubling.

Adam Hair
Posts: 3201
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: How much elo is pondering worth?

Post by Adam Hair » Wed Aug 07, 2013 12:04 pm

The ponder on/ponder off games used an actual time time control of 40 moves in 155 seconds, which roughly equates games on my (deceased) computer to the CCRL 40/4 reference. From previous time odds experiments, I estimate that doubling the speed at that time control on my computer is worth 70 to 90 Elo.

User avatar
velmarin
Posts: 1598
Joined: Mon Feb 21, 2011 8:48 am

Re: How much elo is pondering worth?

Post by velmarin » Wed Aug 07, 2013 1:10 pm

But it seems very difficult to measure these issues.

If you put "ponder ON" in both engines should not have differences.

For example, Houdini has a strength of 3000 on computer "A" of a core, and will have an strength of 3000 on computer "B"of a 4 cores,
against ther others engines within the same system.

But if it competes "A" versus "B", things change, still the same engine,
"B" crush "A".

The grace of "Ponder ON" the engine is hand made the expected move, which will be already stored in the HASH, but if the move is a surprise, is useless.

Post Reply