your guestimate is 30 moves or so to get to 3700-3800 CCRL ELO ?Laskos wrote:Maybe double of what we are seeing today. Just guesstimate. It won't get to non-losing player from standard opening position, but very close to that.duncan wrote:would you know about how many extra ply is needed to get to 3700-3800 CCRL ELO level with same evaluation. ?Laskos wrote:It says that at this 3700-3800 CCRL ELO level the doubling won't give any gain and draw rate becomes 100% for Komodo in self-play.
Doubling of time control
Moderators: hgm, Rebel, chrisw
-
- Posts: 12038
- Joined: Mon Jul 07, 2008 10:50 pm
Re: Doubling of time control.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Doubling of time control.
Yes, maybe. Shown depth is pretty meaningless with how engines search today, it shows iteration. There are extensions and reductions, and the tree is pretty bushy at low depths and sparse at much higher depths. But the engine with 60-70 plies shown depth (iteration) has a pretty good view on the whole relevant part of the game. Also take into account the improvements in opening and endgame databases. I expect it to be very close to non-losing player from starting position.duncan wrote:your guestimate is 30 moves or so to get to 3700-3800 CCRL ELO ?Laskos wrote:Maybe double of what we are seeing today. Just guesstimate. It won't get to non-losing player from standard opening position, but very close to that.duncan wrote:would you know about how many extra ply is needed to get to 3700-3800 CCRL ELO level with same evaluation. ?Laskos wrote:It says that at this 3700-3800 CCRL ELO level the doubling won't give any gain and draw rate becomes 100% for Komodo in self-play.
-
- Posts: 12038
- Joined: Mon Jul 07, 2008 10:50 pm
Re: Doubling of time control.
and guestimate to get to 30 moves takes how long. 30 years ?Laskos wrote:Yes, maybe. Shown depth is pretty meaningless with how engines search today, it shows iteration. There are extensions and reductions, and the tree is pretty bushy at low depths and sparse at much higher depths. But the engine with 60-70 plies shown depth (iteration) has a pretty good view on the whole relevant part of the game. Also take into account the improvements in opening and endgame databases. I expect it to be very close to non-losing player from starting position.duncan wrote:your guestimate is 30 moves or so to get to 3700-3800 CCRL ELO ?Laskos wrote:Maybe double of what we are seeing today. Just guesstimate. It won't get to non-losing player from standard opening position, but very close to that.duncan wrote:would you know about how many extra ply is needed to get to 3700-3800 CCRL ELO level with same evaluation. ?Laskos wrote:It says that at this 3700-3800 CCRL ELO level the doubling won't give any gain and draw rate becomes 100% for Komodo in self-play.
-
- Posts: 140
- Joined: Wed Jun 03, 2020 6:46 am
- Full name: Kurt Lanc
Re: Doubling of time control.
Thank you very much for posting this, very interesting! Just for the record (as a summary of your post), the Elo gain for time t relative to 10s is given by the approximationAjedrecista wrote: ↑Sat Oct 22, 2016 1:56 pm Hello:
I have been looking for an adjust of this data and I think I have something decent. Here I go:
I looked into a Gompertz function (an example is here) and I came with the following:
1.- I converted accumulated Elo gain (0, 144, 277,...) into score with the Elo model µ = 1/[1 + 10^(-Elo/400)].
2.- I used the TC values of 0, 1, 2 and so on in the horizontal axis, like it is seen in the cited paper just after equation 1.
3.- I used the numbers ln[ln(µ_1/µ_0)], ln[ln(µ_2/µ_1)], ..., ln[ln(µ_8/µ_7)] in the vertical axis. (Equation 4 of the paper).
4.- I did a linear regression with Excel to obtain beta and gamma parameters:
5.- By definition, alpha is the saturation level, so we can expect that max(µ) = 1 = alpha --> horizontal asymptote. If that:Code: Select all
Gompertz fit: Fitted_µ = alpha*exp[-beta*exp(-gamma*TC)] Linear fit of the 8 data points = m*TC + n ~ -0.64087232*TC - 0.51556368 (R² ~ 0.99744438) (Equation 4): gamma = -m ~ 0.64087232 (Equation 4): beta = exp(n)/[exp(gamma) - 1] ~ 0.66489741
6.- Equation 5 of the paper proposes the following:Code: Select all
Fitted_µ ~ exp[-0.66489741*exp(-gamma*0.64087232)] Converting fitted_µ into Elo gain (rounding up to the nearest Elo integer): TC Elo Fitted Elo Elo - (fitted Elo) 1 144 151 -7 2 277 277 0 3 389 396 -9 4 490 512 -22 5 583 625 -42 6 656 738 -82 7 715 850 -135 8 766 961 -195 Average error = -61.5 Elo
I know that it sets the upper bound of 99.31% of score, that is, circa 863.3 Elo gain at most. But the average error has improved a lot.Code: Select all
alpha_TC = exp[ln(µ_TC) + beta*exp(-gamma*TC)] I obtain 8 values of alpha_TC. If I randomly choose alpha = average(alpha_TC) ~ 0.99310185 Fitted_µ ~ 0.99310185*exp[-0.66489741*exp(-gamma*0.64087232)] TC Elo Fitted Elo Elo - (fitted Elo) 1 144 147 -3 2 277 270 -7 3 389 384 +5 4 490 489 +1 5 583 585 -2 6 656 668 -12 7 715 735 -20 8 766 785 -19 Average error ~ -7.1 Elo
Furthermore, I did not take into account error bars.
Bonus: if I continue giving increasing values of TC to fitted_µ ~ 0.99310185*exp[-0.66489741*exp(-gamma*0.64087232)], I get the next estimated Elo gains:
I hope no typos. 818 (+33) should be understood as 818 - 785 = +33 Elo in (10240 + 102.4 vs 5120 + 51.2) and +818 Elo in (10240 + 102.4 vs 10 + 0.1).Code: Select all
Converting fitted_µ into Elo gain (rounding up to the nearest Elo integer): Comparison TC Fitted Elo 5120 + 51.2 vs 2560 + 25.6 8 785 10240 + 102.4 vs 5120 + 51.2 9 818 (+33) 20480 + 204.8 vs 10240 + 102.4 10 838 (+20) 40960 + 409.6 vs 20480 + 204.8 11 849 (+11) 81920 + 819.2 vs 40960 + 409.6 12 856 ( +7)
It might be interesting to fit win ratio, draw ratio and lose ratio in similar ways.
Last but not least: thank you very much, Andreas.
Regards from Spain.
Ajedrecista.
Code: Select all
g(t) = -400*log( 1 /(0.99310185*exp(-0.66489741*(t/10)^(-0.64087232/ln(2)))) -1 ).
Code: Select all
| Time | Elo loss rel to 40/2.5h|
| ------- | ---------------------- |
| 6430+64 | 0 |
| 1800+18 | 64 |
| 900+9 | 122 |
| 300+3 | 249 |
| 180+1.8 | 319 |
| 120+1.2 | 379 |
| 60+0.6 | 488 |
-
- Posts: 5
- Joined: Sat Nov 14, 2020 12:11 pm
- Full name: Martin Luzian
Re: Doubling of time control.
Interesting. This thread is only 3 yrs old. SF12 on 8 CPUs and Blitz TC (2'+1" on i74770K) is at ~3700 CCRL Elo and lost only 13 games out of 1390 games (<1%). With longer TC (40/15) it lost twice out of 566 games, against weaker opponents though.duncan wrote: ↑Sun Jul 30, 2017 2:16 amand guestimate to get to 30 moves takes how long. 30 years ?Laskos wrote:Yes, maybe. Shown depth is pretty meaningless with how engines search today, it shows iteration. There are extensions and reductions, and the tree is pretty bushy at low depths and sparse at much higher depths. But the engine with 60-70 plies shown depth (iteration) has a pretty good view on the whole relevant part of the game. Also take into account the improvements in opening and endgame databases. I expect it to be very close to non-losing player from starting position.duncan wrote:your guestimate is 30 moves or so to get to 3700-3800 CCRL ELO ?Laskos wrote:Maybe double of what we are seeing today. Just guesstimate. It won't get to non-losing player from standard opening position, but very close to that.duncan wrote:would you know about how many extra ply is needed to get to 3700-3800 CCRL ELO level with same evaluation. ?Laskos wrote:It says that at this 3700-3800 CCRL ELO level the doubling won't give any gain and draw rate becomes 100% for Komodo in self-play.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Doubling of time control.
I have to backtrack some of the too general statements I made about relationships between Elo, Elo differences, Elo gain from doubling, draw rate, diminishing returns, limiting Elo, etc. These relationships are defined for very particular play conditions and depend onMars wrote: ↑Wed Nov 25, 2020 1:34 pmInteresting. This thread is only 3 yrs old. SF12 on 8 CPUs and Blitz TC (2'+1" on i74770K) is at ~3700 CCRL Elo and lost only 13 games out of 1390 games (<1%). With longer TC (40/15) it lost twice out of 566 games, against weaker opponents though.duncan wrote: ↑Sun Jul 30, 2017 2:16 amand guestimate to get to 30 moves takes how long. 30 years ?Laskos wrote:Yes, maybe. Shown depth is pretty meaningless with how engines search today, it shows iteration. There are extensions and reductions, and the tree is pretty bushy at low depths and sparse at much higher depths. But the engine with 60-70 plies shown depth (iteration) has a pretty good view on the whole relevant part of the game. Also take into account the improvements in opening and endgame databases. I expect it to be very close to non-losing player from starting position.duncan wrote:your guestimate is 30 moves or so to get to 3700-3800 CCRL ELO ?Laskos wrote:Maybe double of what we are seeing today. Just guesstimate. It won't get to non-losing player from standard opening position, but very close to that.duncan wrote:would you know about how many extra ply is needed to get to 3700-3800 CCRL ELO level with same evaluation. ?Laskos wrote:It says that at this 3700-3800 CCRL ELO level the doubling won't give any gain and draw rate becomes 100% for Komodo in self-play.
- Openings -- balanced versus unbalanced, regular versus weird, early versus late, etc.
- Pool of opponents -- strengths differences, types of engines (classical, NNUE, DCNN, AB, MCTS) etc.
- Elo draw model used to rate the engines
-
-
-
I no longer see a real problem with say Draw Death of chess (just use a well conceived unbalanced openings set, as one solution), diminishing returns are there, but have to be better defined than just Elo gain and depth or Elo gain and time. I will no longer give a general limiting Elo for "the perfect engine", as in a pool of non-perfect engines its limiting Elo can even be unbounded by swindling and trolling every non-perfect engine until it loses. And there can be a "perfect engine" in the form of 32 men TBs which will have an Elo rating in a pool of non-perfect engines worse than many of these non-perfect engines. I will no longer talk generally about "Elo gain from doubling time control (or nodes or speed)", as it depends on too many factors. Et caetera, and many of these are not well defined problems to give a quick and general answer to.
Only in very well defined conditions such questions make sense.
Last edited by Laskos on Thu Nov 26, 2020 5:18 pm, edited 3 times in total.
-
- Posts: 2727
- Joined: Wed May 12, 2010 10:00 pm
Re: Doubling of time control.
BINGO!Laskos wrote: ↑Thu Nov 26, 2020 4:50 pmI have to backtrack some of the statements I made about relationships between Elo, Elo differences, Elo gain from doubling, draw rate, diminishing returns, limitign ELo, etc. These relationships are defined for very particular play conditions and depend onMars wrote: ↑Wed Nov 25, 2020 1:34 pmInteresting. This thread is only 3 yrs old. SF12 on 8 CPUs and Blitz TC (2'+1" on i74770K) is at ~3700 CCRL Elo and lost only 13 games out of 1390 games (<1%). With longer TC (40/15) it lost twice out of 566 games, against weaker opponents though.duncan wrote: ↑Sun Jul 30, 2017 2:16 amand guestimate to get to 30 moves takes how long. 30 years ?Laskos wrote:Yes, maybe. Shown depth is pretty meaningless with how engines search today, it shows iteration. There are extensions and reductions, and the tree is pretty bushy at low depths and sparse at much higher depths. But the engine with 60-70 plies shown depth (iteration) has a pretty good view on the whole relevant part of the game. Also take into account the improvements in opening and endgame databases. I expect it to be very close to non-losing player from starting position.duncan wrote:your guestimate is 30 moves or so to get to 3700-3800 CCRL ELO ?Laskos wrote:Maybe double of what we are seeing today. Just guesstimate. It won't get to non-losing player from standard opening position, but very close to that.duncan wrote:would you know about how many extra ply is needed to get to 3700-3800 CCRL ELO level with same evaluation. ?Laskos wrote:It says that at this 3700-3800 CCRL ELO level the doubling won't give any gain and draw rate becomes 100% for Komodo in self-play.
- Openings -- balanced versus unbalanced, regular versus weird, early versus late, etc.
- Pool of opponents -- strengths differences, types of engines (classical, NNUE, DCNN, AB, MCTS) etc.
- Elo draw model used to rate the engines
-
-
-
I no longer see a real problem with say Draw Death of chess (just use a well conceived unbalanced openings set, as one solution), diminishing returns are there, but have to be better defined than just Elo gain and depth or Elo gain and time. I will no longer give a limiting Elo for "the perfect engine", as in a pool of non-perfect engines its limiting Elo can be unbounded by swindling and trolling every non-perfect engine until it loses. And there can be a "perfect engine" in the form of 32 men TBs which will have an Elo rating in a pool of non-perfect engines worse than many of these non-perfect engines. Et caetera, and many of these are not well defined problems to give a quick and general answer to.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
But my words like silent raindrops fell. And echoed in the wells of silence.
-
- Posts: 5
- Joined: Sat Nov 14, 2020 12:11 pm
- Full name: Martin Luzian
Re: Doubling of time control.
Sorry, this wasn´t meant as picking at you b/c of some old statements. Not at all. I found it just interesting that after a pretty short period of time the deveopment in strength was - maybe - faster than anticipated. And that the number of lost games by SF12 is already pretty low indeed.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Doubling of time control.
No, I was not meaning you picked at me or something. Just that I was often in the past sloppy not elaborating on particular conditions needed for my words to make sense. Bluntly talking too generally of "Elo gain from doubling time" or "Elo of the perfect engine" can give an impression that the things can be described by some general rules of thumb, while the things cannot be stated shortly and have often to be particularized to precise conditions and definitions. For example, there might be in the future a 32 men implementation of TBs as perfect engine not being able as Black to win a single game against SF12 from the standard opening position. While a heuristic non-perfect say SF16 will be able to easily beat from time to time as Black SF12. These quirks are eluded by general statements, and I need not be misunderstood too often.Mars wrote: ↑Thu Nov 26, 2020 8:11 pmSorry, this wasn´t meant as picking at you b/c of some old statements. Not at all. I found it just interesting that after a pretty short period of time the deveopment in strength was - maybe - faster than anticipated. And that the number of lost games by SF12 is already pretty low indeed.