Hello:
I do not know if my post is off-topic or not, but Don posted an experiment some months ago:
Computer Chess Scalability Study
Here is the thread where Don exposed his experiment. There are also more studies in the same issue with the participation of Peter Österlund and Adam Hair:
Elo versus speed
Elo Increase per Doubling
I hope these links will be useful for comparison.
Regards from Spain.
Ajedrecista.
Elo points gain from doubling time
Moderators: hgm, Rebel, chrisw
-
- Posts: 1971
- Joined: Wed Jul 13, 2011 9:04 pm
- Location: Madrid, Spain.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Elo points gain from doubling time.
Thanks, so most of it was been done before, although my time controls are in the upper limits of other tests. The results seem consistent with mine.Ajedrecista wrote:Hello:
I do not know if my post is off-topic or not, but Don posted an experiment some months ago:
Computer Chess Scalability Study
Here is the thread where Don exposed his experiment. There are also more studies in the same issue with the participation of Peter Österlund and Adam Hair:
Elo versus speed
Elo Increase per Doubling
I hope these links will be useful for comparison.
Regards from Spain.
Ajedrecista.
-
- Posts: 6995
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Elo points gain from doubling time.
Here is another study about the diminishing return effect.Laskos wrote:Thanks, so most of it was been done before, although my time controls are in the upper limits of other tests. The results seem consistent with mine.Ajedrecista wrote:Hello:
I do not know if my post is off-topic or not, but Don posted an experiment some months ago:
Computer Chess Scalability Study
Here is the thread where Don exposed his experiment. There are also more studies in the same issue with the participation of Peter Österlund and Adam Hair:
Elo versus speed
Elo Increase per Doubling
I hope these links will be useful for comparison.
Regards from Spain.
Ajedrecista.
http://www.top-5000.nl/ply.htm
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Elo points gain from doubling time
A bit off the noise, besides that, if I am lucky, I could fall inside 1SD error margins. Also, 2SD fit still shows a decrease with time in gain (but large uncertainties). I saw other studies shown to me by Jesus Munoz, and it seems that the rule of thumb 107*(time in seconds per move)^(-0.20) fits the data of others pretty well. Yes, the errors are large, but playing more than 2,000 games at 4s/move TC is a pain. I focused specifically at these longer TC, because at ultra-short controls it's easy to see 150-200 points per doubling, but it's irrelevant.hgm wrote:Well, the 'rule of thumb' seems to be 100 Elo per doubling. (Which, btw, is more than I expected. I always assumed 70 Elo per doubling.) The rest of the analysis seems mostly analysing noise, based on a +7 +/- 4.2 @1s and a -19 +/- 6.3 @ 4s.Laskos wrote:These 2,000-4,000 blitz games matches take a lot of time, so I will not throw in different TC Houdinis, as I just wanted to see the rule of thumb law.
-
- Posts: 5106
- Joined: Tue Apr 29, 2008 4:27 pm
Re: Elo points gain from doubling time
I don't think it is - the number of draws goes up pretty steadily with the quality of the players. I think in a few years we are going to see the ratings of the top programs get really compressed due to this. You will play a 100 games match and they will be mostly draws. It will be like checkers is now.hgm wrote:The result is weird anyway: the draw fraction seems to go up enormously at higher depth. I would consider 80% draws a ridiculously high draw fraction, between nearly equal engines.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Elo points gain from doubling time
It was Houdini 3 ply 14 vs ply 15, ultra-bullet, 80% draws are impossible. The highest I seen at very long time controls is 73% or so.Don wrote:I don't think it is - the number of draws goes up pretty steadily with the quality of the players. I think in a few years we are going to see the ratings of the top programs get really compressed due to this. You will play a 100 games match and they will be mostly draws. It will be like checkers is now.hgm wrote:The result is weird anyway: the draw fraction seems to go up enormously at higher depth. I would consider 80% draws a ridiculously high draw fraction, between nearly equal engines.
-
- Posts: 5106
- Joined: Tue Apr 29, 2008 4:27 pm
Re: Elo points gain from doubling time
The elo value per doubling drops off rather sharply with each additional doubling so I don't think there is simple rule of thumb. I have done many experiments that show this. Again, I think it's primarily due the increased number of draws as they tend to go up very steadily with depth too.hgm wrote:Well, the 'rule of thumb' seems to be 100 Elo per doubling. (Which, btw, is more than I expected. I always assumed 70 Elo per doubling.) The rest of the analysis seems mostly analysing noise, based on a +7 +/- 4.2 @1s and a -19 +/- 6.3 @ 4s.Laskos wrote:These 2,000-4,000 blitz games matches take a lot of time, so I will not throw in different TC Houdinis, as I just wanted to see the rule of thumb law.
It would be interesting to study that too. Imagine doing one of these logarithmic studies where you throw out all draws. You might have a much flatter rating curve.
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
-
- Posts: 5106
- Joined: Tue Apr 29, 2008 4:27 pm
Re: Elo points gain from doubling time
I should have qualified my statement - it seems ridiculously high at that time control for sure.Laskos wrote:It was Houdini 3 ply 14 vs ply 15, ultra-bullet, 80% draws are impossible. The highest I seen at very long time controls is 73% or so.Don wrote:I don't think it is - the number of draws goes up pretty steadily with the quality of the players. I think in a few years we are going to see the ratings of the top programs get really compressed due to this. You will play a 100 games match and they will be mostly draws. It will be like checkers is now.hgm wrote:The result is weird anyway: the draw fraction seems to go up enormously at higher depth. I would consider 80% draws a ridiculously high draw fraction, between nearly equal engines.
I'm running a 30 second test right now to test something that looks good at much faster time controls and I'm getting this:
Code: Select all
white wins 908
black wins 628
64.67 percent draws
4,347 games
I think in the not so distant future you will see very few decisive games and almost all of them will be by white.
I happened to have some games played at 0.8 seconds per games + 0.008 increment and I get this:
Code: Select all
white wins 12913
black wins 11499
36.30 percent draws
about 38,000 games
You can see the trend, with increasing strength the draw ratio rises non-trivially and of the remaining games white tends to wins a higher percentage. This is far from comprehensive of course but I have consistently seen it - at least with Komodo.
Another problem I have identified is the contempt factor. I am of the opinion that the relative strength of the opponent should be communicated to the engines and built it to the protocol because this is becoming an important issue too. It can be worth 100 ELO or more if you are playing way up or down by several hundred ELO (I studied this too.) Unless that is handled a perfect player will draw far too many games against weak opponents. In human play it's rare not to have a rough idea of the strength of your opponent. That can be communicated via some user defined contempt factor but I would like to see it built in to the protocol.
Don
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
-
- Posts: 1471
- Joined: Tue Mar 16, 2010 12:00 am
Re: Elo points gain from doubling time
Correct, with Houdini 3 this is nearly impossible - it suggests that the opening positions for the test are not well chosen.Laskos wrote:It was Houdini 3 ply 14 vs ply 15, ultra-bullet, 80% draws are impossible. The highest I seen at very long time controls is 73% or so.
Even at long time control Houdini will have close to 50% decided games. See for example the 90 min+30 sec/move tests I played with the Houdini 3 beta against Houdini 2, Stockfish 2.3.1 and Komodo 5, which over-all was +135 -50 =175.
Robert
-
- Posts: 10299
- Joined: Thu Mar 09, 2006 12:37 am
- Location: Tel-Aviv Israel
Re: Elo points gain from doubling time
Note that it is possible that having houdini3 against houdini3 and not against other programs increase the number of draws.Houdini wrote:Correct, with Houdini 3 this is nearly impossible - it suggests that the opening positions for the test are not well chosen.Laskos wrote:It was Houdini 3 ply 14 vs ply 15, ultra-bullet, 80% draws are impossible. The highest I seen at very long time controls is 73% or so.
Even at long time control Houdini will have close to 50% decided games. See for example the 90 min+30 sec/move tests I played with the Houdini 3 beta against Houdini 2, Stockfish 2.3.1 and Komodo 5, which over-all was +135 -50 =175.
Robert
I still think that 15 plies against 14 plies should not give so many draws unless you have a bad choice of the opening positions(for example using a big book that cause the engines to start to play only at move 20 or move 30 is a bad idea)