Doubling of time control

[duncan]

It says that at this 3700-3800 CCRL ELO level the doubling won't give any gain and draw rate becomes 100% for Komodo in self-play.

would you know about how many extra ply is needed to get to 3700-3800 CCRL ELO level with same evaluation. ?

Maybe double of what we are seeing today. Just guesstimate. It won't get to non-losing player from standard opening position, but very close to that.

your guestimate is 30 moves or so to get to 3700-3800 CCRL ELO ?

[Laskos]

It says that at this 3700-3800 CCRL ELO level the doubling won't give any gain and draw rate becomes 100% for Komodo in self-play.

would you know about how many extra ply is needed to get to 3700-3800 CCRL ELO level with same evaluation. ?

Maybe double of what we are seeing today. Just guesstimate. It won't get to non-losing player from standard opening position, but very close to that.

your guestimate is 30 moves or so to get to 3700-3800 CCRL ELO ?

Yes, maybe. Shown depth is pretty meaningless with how engines search today, it shows iteration. There are extensions and reductions, and the tree is pretty bushy at low depths and sparse at much higher depths. But the engine with 60-70 plies shown depth (iteration) has a pretty good view on the whole relevant part of the game. Also take into account the improvements in opening and endgame databases. I expect it to be very close to non-losing player from starting position.

[duncan]

It says that at this 3700-3800 CCRL ELO level the doubling won't give any gain and draw rate becomes 100% for Komodo in self-play.

would you know about how many extra ply is needed to get to 3700-3800 CCRL ELO level with same evaluation. ?

Maybe double of what we are seeing today. Just guesstimate. It won't get to non-losing player from standard opening position, but very close to that.

your guestimate is 30 moves or so to get to 3700-3800 CCRL ELO ?

Yes, maybe. Shown depth is pretty meaningless with how engines search today, it shows iteration. There are extensions and reductions, and the tree is pretty bushy at low depths and sparse at much higher depths. But the engine with 60-70 plies shown depth (iteration) has a pretty good view on the whole relevant part of the game. Also take into account the improvements in opening and endgame databases. I expect it to be very close to non-losing player from starting position.

and guestimate to get to 30 moves takes how long. 30 years ?

[KLc]

Hello:

I have been looking for an adjust of this data and I think I have something decent. Here I go:

I looked into a Gompertz function (an example is here) and I came with the following:

1.- I converted accumulated Elo gain (0, 144, 277,...) into score with the Elo model µ = 1/[1 + 10^(-Elo/400)].

2.- I used the TC values of 0, 1, 2 and so on in the horizontal axis, like it is seen in the cited paper just after equation 1.

3.- I used the numbers ln[ln(µ_1/µ_0)], ln[ln(µ_2/µ_1)], ..., ln[ln(µ_8/µ_7)] in the vertical axis. (Equation 4 of the paper).

4.- I did a linear regression with Excel to obtain beta and gamma parameters:

Code: Select all

Gompertz fit:
Fitted_µ = alpha*exp[-beta*exp(-gamma*TC)]

Linear fit of the 8 data points = m*TC + n ~ -0.64087232*TC - 0.51556368 (R² ~ 0.99744438)

(Equation 4): gamma = -m ~ 0.64087232
(Equation 4): beta = exp(n)/[exp(gamma) - 1] ~ 0.66489741

5.- By definition, alpha is the saturation level, so we can expect that max(µ) = 1 = alpha --> horizontal asymptote. If that:

Code: Select all

Fitted_µ ~ exp[-0.66489741*exp(-gamma*0.64087232)]

Converting fitted_µ into Elo gain (rounding up to the nearest Elo integer):

TC  Elo   Fitted Elo   Elo - (fitted Elo)
 1  144       151               -7
 2  277       277                0
 3  389       396               -9
 4  490       512              -22
 5  583       625              -42
 6  656       738              -82
 7  715       850             -135
 8  766       961             -195

Average error = -61.5 Elo

6.- Equation 5 of the paper proposes the following:

Code: Select all

alpha_TC = exp[ln(µ_TC) + beta*exp(-gamma*TC)]

I obtain 8 values of alpha_TC. If I randomly choose alpha = average(alpha_TC) ~ 0.99310185

Fitted_µ ~ 0.99310185*exp[-0.66489741*exp(-gamma*0.64087232)]

TC  Elo   Fitted Elo   Elo - (fitted Elo)
 1  144       147               -3
 2  277       270               -7
 3  389       384               +5
 4  490       489               +1
 5  583       585               -2
 6  656       668              -12
 7  715       735              -20
 8  766       785              -19

Average error ~ -7.1 Elo

I know that it sets the upper bound of 99.31% of score, that is, circa 863.3 Elo gain at most. But the average error has improved a lot.

Furthermore, I did not take into account error bars.

Bonus: if I continue giving increasing values of TC to fitted_µ ~ 0.99310185*exp[-0.66489741*exp(-gamma*0.64087232)], I get the next estimated Elo gains:

Code: Select all

Converting fitted_µ into Elo gain (rounding up to the nearest Elo integer):

          Comparison               TC  Fitted Elo
 5120 +  51.2 vs  2560 +  25.6      8  785
10240 + 102.4 vs  5120 +  51.2      9  818 (+33)
20480 + 204.8 vs 10240 + 102.4     10  838 (+20)
40960 + 409.6 vs 20480 + 204.8     11  849 (+11)
81920 + 819.2 vs 40960 + 409.6     12  856 ( +7)

I hope no typos. 818 (+33) should be understood as 818 - 785 = +33 Elo in (10240 + 102.4 vs 5120 + 51.2) and +818 Elo in (10240 + 102.4 vs 10 + 0.1).

It might be interesting to fit win ratio, draw ratio and lose ratio in similar ways.

Last but not least: thank you very much, Andreas.

Regards from Spain.

Ajedrecista.

Thank you very much for posting this, very interesting! Just for the record (as a summary of your post), the Elo gain for time t relative to 10s is given by the approximation

Code: Select all

g(t) = -400*log( 1 /(0.99310185*exp(-0.66489741*(t/10)^(-0.64087232/ln(2)))) -1 ). 

Also for the record, I was interested in the Elo loss compared to classical (Fischer-Spassky) time control 40/2.5h, which corresponds to 6430+64 assuming 40 moves/game on average. Here are some values:

Code: Select all

| Time    | Elo loss rel to 40/2.5h|
| ------- | ---------------------- |
| 6430+64 | 0                      |
| 1800+18 | 64                     |
| 900+9   | 122                    |
| 300+3   | 249                    |
| 180+1.8 | 319                    |
| 120+1.2 | 379                    |
| 60+0.6  | 488                    |

[Mars]

It says that at this 3700-3800 CCRL ELO level the doubling won't give any gain and draw rate becomes 100% for Komodo in self-play.

would you know about how many extra ply is needed to get to 3700-3800 CCRL ELO level with same evaluation. ?

Maybe double of what we are seeing today. Just guesstimate. It won't get to non-losing player from standard opening position, but very close to that.

your guestimate is 30 moves or so to get to 3700-3800 CCRL ELO ?

Yes, maybe. Shown depth is pretty meaningless with how engines search today, it shows iteration. There are extensions and reductions, and the tree is pretty bushy at low depths and sparse at much higher depths. But the engine with 60-70 plies shown depth (iteration) has a pretty good view on the whole relevant part of the game. Also take into account the improvements in opening and endgame databases. I expect it to be very close to non-losing player from starting position.

and guestimate to get to 30 moves takes how long. 30 years ?

Interesting. This thread is only 3 yrs old. SF12 on 8 CPUs and Blitz TC (2'+1" on i74770K) is at ~3700 CCRL Elo and lost only 13 games out of 1390 games (<1%). With longer TC (40/15) it lost twice out of 566 games, against weaker opponents though.

[Laskos]

It says that at this 3700-3800 CCRL ELO level the doubling won't give any gain and draw rate becomes 100% for Komodo in self-play.

would you know about how many extra ply is needed to get to 3700-3800 CCRL ELO level with same evaluation. ?

Maybe double of what we are seeing today. Just guesstimate. It won't get to non-losing player from standard opening position, but very close to that.

your guestimate is 30 moves or so to get to 3700-3800 CCRL ELO ?

Yes, maybe. Shown depth is pretty meaningless with how engines search today, it shows iteration. There are extensions and reductions, and the tree is pretty bushy at low depths and sparse at much higher depths. But the engine with 60-70 plies shown depth (iteration) has a pretty good view on the whole relevant part of the game. Also take into account the improvements in opening and endgame databases. I expect it to be very close to non-losing player from starting position.

and guestimate to get to 30 moves takes how long. 30 years ?

Interesting. This thread is only 3 yrs old. SF12 on 8 CPUs and Blitz TC (2'+1" on i74770K) is at ~3700 CCRL Elo and lost only 13 games out of 1390 games (<1%). With longer TC (40/15) it lost twice out of 566 games, against weaker opponents though.

I have to backtrack some of the too general statements I made about relationships between Elo, Elo differences, Elo gain from doubling, draw rate, diminishing returns, limiting Elo, etc. These relationships are defined for very particular play conditions and depend on
- Openings -- balanced versus unbalanced, regular versus weird, early versus late, etc.
- Pool of opponents -- strengths differences, types of engines (classical, NNUE, DCNN, AB, MCTS) etc.
- Elo draw model used to rate the engines
-
-
-

I no longer see a real problem with say Draw Death of chess (just use a well conceived unbalanced openings set, as one solution), diminishing returns are there, but have to be better defined than just Elo gain and depth or Elo gain and time. I will no longer give a general limiting Elo for "the perfect engine", as in a pool of non-perfect engines its limiting Elo can even be unbounded by swindling and trolling every non-perfect engine until it loses. And there can be a "perfect engine" in the form of 32 men TBs which will have an Elo rating in a pool of non-perfect engines worse than many of these non-perfect engines. I will no longer talk generally about "Elo gain from doubling time control (or nodes or speed)", as it depends on too many factors. Et caetera, and many of these are not well defined problems to give a quick and general answer to.

Only in very well defined conditions such questions make sense.

[mwyoung]

It says that at this 3700-3800 CCRL ELO level the doubling won't give any gain and draw rate becomes 100% for Komodo in self-play.

would you know about how many extra ply is needed to get to 3700-3800 CCRL ELO level with same evaluation. ?

Maybe double of what we are seeing today. Just guesstimate. It won't get to non-losing player from standard opening position, but very close to that.

your guestimate is 30 moves or so to get to 3700-3800 CCRL ELO ?

Yes, maybe. Shown depth is pretty meaningless with how engines search today, it shows iteration. There are extensions and reductions, and the tree is pretty bushy at low depths and sparse at much higher depths. But the engine with 60-70 plies shown depth (iteration) has a pretty good view on the whole relevant part of the game. Also take into account the improvements in opening and endgame databases. I expect it to be very close to non-losing player from starting position.

and guestimate to get to 30 moves takes how long. 30 years ?

Interesting. This thread is only 3 yrs old. SF12 on 8 CPUs and Blitz TC (2'+1" on i74770K) is at ~3700 CCRL Elo and lost only 13 games out of 1390 games (<1%). With longer TC (40/15) it lost twice out of 566 games, against weaker opponents though.

I have to backtrack some of the statements I made about relationships between Elo, Elo differences, Elo gain from doubling, draw rate, diminishing returns, limitign ELo, etc. These relationships are defined for very particular play conditions and depend on
- Openings -- balanced versus unbalanced, regular versus weird, early versus late, etc.
- Pool of opponents -- strengths differences, types of engines (classical, NNUE, DCNN, AB, MCTS) etc.
- Elo draw model used to rate the engines
-
-
-

I no longer see a real problem with say Draw Death of chess (just use a well conceived unbalanced openings set, as one solution), diminishing returns are there, but have to be better defined than just Elo gain and depth or Elo gain and time. I will no longer give a limiting Elo for "the perfect engine", as in a pool of non-perfect engines its limiting Elo can be unbounded by swindling and trolling every non-perfect engine until it loses. And there can be a "perfect engine" in the form of 32 men TBs which will have an Elo rating in a pool of non-perfect engines worse than many of these non-perfect engines. Et caetera, and many of these are not well defined problems to give a quick and general answer to.

BINGO!

[Mars]

... I will no longer talk generally about ...

Sorry, this wasn´t meant as picking at you b/c of some old statements. Not at all. I found it just interesting that after a pretty short period of time the deveopment in strength was - maybe - faster than anticipated. And that the number of lost games by SF12 is already pretty low indeed.

[Laskos]

... I will no longer talk generally about ...

Sorry, this wasn´t meant as picking at you b/c of some old statements. Not at all. I found it just interesting that after a pretty short period of time the deveopment in strength was - maybe - faster than anticipated. And that the number of lost games by SF12 is already pretty low indeed.

No, I was not meaning you picked at me or something. Just that I was often in the past sloppy not elaborating on particular conditions needed for my words to make sense. Bluntly talking too generally of "Elo gain from doubling time" or "Elo of the perfect engine" can give an impression that the things can be described by some general rules of thumb, while the things cannot be stated shortly and have often to be particularized to precise conditions and definitions. For example, there might be in the future a 32 men implementation of TBs as perfect engine not being able as Black to win a single game against SF12 from the standard opening position. While a heuristic non-perfect say SF16 will be able to easily beat from time to time as Black SF12. These quirks are eluded by general statements, and I need not be misunderstood too often.