Is there any information about the participants of the test matches and the opening book used?
Without these infos the data set published is hanging in the air.
Basically I think that the Elo system has no fundamental issue but the power of a chess engine really and relatively downfalls with the enhancement of move time.
This can be attributed to the pruning and the fact that the number of the possible variations grows with moving time.
Doubling of time control
Moderators: hgm, Rebel, chrisw
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
-
- Posts: 4619
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: Doubling of time control
What test matches do you mean? The original thread and provided datacorres wrote:Is there any information about the participants of the test matches and the opening book used?
Without these infos the data set published is hanging in the air.
...
is about Komodo 9.3 and a set of 1500 openings (this info is in the first post).
-
- Posts: 818
- Joined: Mon Aug 19, 2013 6:57 pm
Re: Doubling of time control
Hello Robert,
perhaps the conditions in my initial post wasn't detailed enough.
Openings:
1500 different opening positions with changing colors = 3000 games for each match
Engine:
Komodo 9.3 against itself
Andreas
perhaps the conditions in my initial post wasn't detailed enough.
Openings:
1500 different opening positions with changing colors = 3000 games for each match
Engine:
Komodo 9.3 against itself
Andreas
-
- Posts: 818
- Joined: Mon Aug 19, 2013 6:57 pm
Re: Doubling of time control
Thanks Kai, i have asked Ferdinand if he can help me with such a tool.
-
- Posts: 818
- Joined: Mon Aug 19, 2013 6:57 pm
Re: Doubling of time control
Ferdinand Mosca helped me. Thanks a lot!
He found a program (Protools) written by Ed Schröder:
http://www.top-5000.nl/dl/protools15.zip
and the latest prodeo.exe
http://www.talkchess.com/forum/viewtopi ... 77&t=61721
With this tool I have created the following additional data:
He found a program (Protools) written by Ed Schröder:
http://www.top-5000.nl/dl/protools15.zip
and the latest prodeo.exe
http://www.talkchess.com/forum/viewtopi ... 77&t=61721
With this tool I have created the following additional data:
Code: Select all
Engine Depth Time Games Moves Average Forfeit Book
Komodo 9.3 T1 20+0.2 17.40 28:34:57 3000 256891 0.40 0 26737 (8.91)
Komodo 9.3 T1 10+0.1 15.44 14:19:45 3000 256342 0.20 0 24694 (8.23)
Engine Depth Time Games Moves Average Forfeit Book
Komodo 9.3 T1 40+0.4 19.05 58:18:17 3000 265755 0.79 0 27193 (9.06)
Komodo 9.3 T1 20+0.2 17.07 29:24:34 3000 265279 0.40 0 25454 (8.48)
Engine Depth Time Games Moves Average Forfeit Book
Komodo 9.3 T1 80+0.8 20.70 116:39:50 3000 267468 1.57 0 27456 (9.15)
Komodo 9.3 T1 40+0.4 18.85 58:53:16 3000 267048 0.79 0 25729 (8.58)
Engine Depth Time Games Moves Average Forfeit Book
Komodo 9.3 T1 160+1.6 22.45 234:51:52 3000 267920 3.16 0 27371 (9.12)
Komodo 9.3 T1 80+0.8 20.50 118:06:24 3000 267555 1.59 0 26095 (8.70)
Engine Depth Time Games Moves Average Forfeit Book
Komodo 9.3 T1 320+3.2 24.30 476:30:53 3000 274164 6.26 0 27771 (9.26)
Komodo 9.3 T1 160+1.6 22.29 239:40:46 3000 273826 3.15 0 26428 (8.81)
Engine Depth Time Games Moves Average Forfeit Book
Komodo 9.3 T1 640+6.4 26.28 950:20:50 3000 272343 12.56 0 27972 (9.32)
Komodo 9.3 T1 320+3.2 24.26 478:20:21 3000 272091 6.33 0 26638 (8.88)
Engine Depth Time Games Moves Average Forfeit Book
Komodo 9.3 T1 1280+12.8 28.09 1908:54:31 3000 276907 24.82 0 28475 (9.49)
Komodo 9.3 T1 640+6.4 26.18 960:08:07 3000 276750 12.49 0 27004 (9.00)
Engine Depth Time Games Moves Average Forfeit Book
Komodo 9.3 T1 2560+25.6 29.92 3806:05:02 3000 275195 49.79 0 28760 (9.59)
Komodo 9.3 T1 1280+12.8 28.01 1914:36:07 3000 275034 25.06 0 27544 (9.18)
Time control comparison between engines
Depth : Average search depth
Time : Total time engine used
Moves : Total moves engine played
Average : Average time per move in centi-seconds
Forfeit : Games engine lost due to time forfeit
List is sorted on Average Time indicating the engine that uses the most time tops.
-
- Posts: 4619
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: Doubling of time control
This is great! I will look closer on it next week and also try to make updated graphs, but probably Kai will be faster ;-)fastgm wrote:Ferdinand Mosca helped me. Thanks a lot!
He found a program (Protools) written by Ed Schröder:
http://www.top-5000.nl/dl/protools15.zip
and the latest prodeo.exe
http://www.talkchess.com/forum/viewtopi ... 77&t=61721
With this tool I have created the following additional data:
Code: Select all
Engine Depth Time Games Moves Average Forfeit Book Komodo 9.3 T1 20+0.2 17.40 28:34:57 3000 256891 0.40 0 26737 (8.91) Komodo 9.3 T1 10+0.1 15.44 14:19:45 3000 256342 0.20 0 24694 (8.23) Engine Depth Time Games Moves Average Forfeit Book Komodo 9.3 T1 40+0.4 19.05 58:18:17 3000 265755 0.79 0 27193 (9.06) Komodo 9.3 T1 20+0.2 17.07 29:24:34 3000 265279 0.40 0 25454 (8.48) Engine Depth Time Games Moves Average Forfeit Book Komodo 9.3 T1 80+0.8 20.70 116:39:50 3000 267468 1.57 0 27456 (9.15) Komodo 9.3 T1 40+0.4 18.85 58:53:16 3000 267048 0.79 0 25729 (8.58) Engine Depth Time Games Moves Average Forfeit Book Komodo 9.3 T1 160+1.6 22.45 234:51:52 3000 267920 3.16 0 27371 (9.12) Komodo 9.3 T1 80+0.8 20.50 118:06:24 3000 267555 1.59 0 26095 (8.70) Engine Depth Time Games Moves Average Forfeit Book Komodo 9.3 T1 320+3.2 24.30 476:30:53 3000 274164 6.26 0 27771 (9.26) Komodo 9.3 T1 160+1.6 22.29 239:40:46 3000 273826 3.15 0 26428 (8.81) Engine Depth Time Games Moves Average Forfeit Book Komodo 9.3 T1 640+6.4 26.28 950:20:50 3000 272343 12.56 0 27972 (9.32) Komodo 9.3 T1 320+3.2 24.26 478:20:21 3000 272091 6.33 0 26638 (8.88) Engine Depth Time Games Moves Average Forfeit Book Komodo 9.3 T1 1280+12.8 28.09 1908:54:31 3000 276907 24.82 0 28475 (9.49) Komodo 9.3 T1 640+6.4 26.18 960:08:07 3000 276750 12.49 0 27004 (9.00) Engine Depth Time Games Moves Average Forfeit Book Komodo 9.3 T1 2560+25.6 29.92 3806:05:02 3000 275195 49.79 0 28760 (9.59) Komodo 9.3 T1 1280+12.8 28.01 1914:36:07 3000 275034 25.06 0 27544 (9.18) Time control comparison between engines Depth : Average search depth Time : Total time engine used Moves : Total moves engine played Average : Average time per move in centi-seconds Forfeit : Games engine lost due to time forfeit List is sorted on Average Time indicating the engine that uses the most time tops.
Note also that we even have 6000 games for an average depth for all entries except the first and the last ones.
Code: Select all
Engine Depth Time Games Moves Average Forfeit Book Avg.D G Avg.D Inc
Komodo 9.3 T1 10+0.1 15,44 14:19:45 3000 256342 0,20 0 24694 8,23 15,440 3000
Komodo 9.3 T1 20+0.2 17,07 29:24:34 3000 265279 0,40 0 25454 8,48 17,235 6000 1,795
Komodo 9.3 T1 20+0.2 17,40 28:34:57 3000 256891 0,40 0 26737 8,91
Komodo 9.3 T1 40+0.4 18,85 58:53:16 3000 267048 0,79 0 25729 8,58 18,950 6000 1,715
Komodo 9.3 T1 40+0.4 19,05 58:18:17 3000 265755 0,79 0 27193 9,06
Komodo 9.3 T1 80+0.8 20,50 118:06:24 3000 267555 1,59 0 26095 8,70 20,600 6000 1,650
Komodo 9.3 T1 80+0.8 20,70 116:39:50 3000 267468 1,57 0 27456 9,15
Komodo 9.3 T1 160+1.6 22,29 239:40:46 3000 273826 3,15 0 26428 8,81 22,370 6000 1,770
Komodo 9.3 T1 160+1.6 22,45 234:51:52 3000 267920 3,16 0 27371 9,12
Komodo 9.3 T1 320+3.2 24,26 478:20:21 3000 272091 6,33 0 26638 8,88 24,280 6000 1,910
Komodo 9.3 T1 320+3.2 24,30 476:30:53 3000 274164 6,26 0 27771 9,26
Komodo 9.3 T1 640+6.4 26,18 960:08:07 3000 276750 12,49 0 27004 9,00 26,230 6000 1,950
Komodo 9.3 T1 640+6.4 26,28 950:20:50 3000 272343 12,56 0 27972 9,32
Komodo 9.3 T1 1280+12.8 28,01 1914:36:07 3000 275034 25,06 0 27544 9,18 28,050 6000 1,715
Komodo 9.3 T1 1280+12.8 28,09 1908:54:31 3000 276907 24,82 0 28475 9,49
Komodo 9.3 T1 2560+25.6 29,92 3806:05:02 3000 275195 49,79 0 28760 9,59 29,920 3000 1,870
-
- Posts: 4619
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: Doubling of time control
BTW what resign and/or adjudication options were in chargfastgm wrote:Hello Robert,
perhaps the conditions in my initial post wasn't detailed enough.
Openings:
1500 different opening positions with changing colors = 3000 games for each match
Engine:
Komodo 9.3 against itself
Andreas
Guenther
-
- Posts: 818
- Joined: Mon Aug 19, 2013 6:57 pm
Re: Doubling of time control
No adjudication (draw, resign) options.
-
- Posts: 1973
- Joined: Wed Jul 13, 2011 9:04 pm
- Location: Madrid, Spain.
Re: Doubling of time control.
Hello:
I have been looking for an adjust of this data and I think I have something decent. Here I go:
I looked into a Gompertz function (an example is here) and I came with the following:
1.- I converted accumulated Elo gain (0, 144, 277,...) into score with the Elo model µ = 1/[1 + 10^(-Elo/400)].
2.- I used the TC values of 0, 1, 2 and so on in the horizontal axis, like it is seen in the cited paper just after equation 1.
3.- I used the numbers ln[ln(µ_1/µ_0)], ln[ln(µ_2/µ_1)], ..., ln[ln(µ_8/µ_7)] in the vertical axis. (Equation 4 of the paper).
4.- I did a linear regression with Excel to obtain beta and gamma parameters:
5.- By definition, alpha is the saturation level, so we can expect that max(µ) = 1 = alpha --> horizontal asymptote. If that:
6.- Equation 5 of the paper proposes the following:
I know that it sets the upper bound of 99.31% of score, that is, circa 863.3 Elo gain at most. But the average error has improved a lot.
Furthermore, I did not take into account error bars.
Bonus: if I continue giving increasing values of TC to fitted_µ ~ 0.99310185*exp[-0.66489741*exp(-gamma*0.64087232)], I get the next estimated Elo gains:
I hope no typos. 818 (+33) should be understood as 818 - 785 = +33 Elo in (10240 + 102.4 vs 5120 + 51.2) and +818 Elo in (10240 + 102.4 vs 10 + 0.1).
It might be interesting to fit win ratio, draw ratio and lose ratio in similar ways.
Last but not least: thank you very much, Andreas.
Regards from Spain.
Ajedrecista.
I have been looking for an adjust of this data and I think I have something decent. Here I go:
I looked into a Gompertz function (an example is here) and I came with the following:
1.- I converted accumulated Elo gain (0, 144, 277,...) into score with the Elo model µ = 1/[1 + 10^(-Elo/400)].
2.- I used the TC values of 0, 1, 2 and so on in the horizontal axis, like it is seen in the cited paper just after equation 1.
3.- I used the numbers ln[ln(µ_1/µ_0)], ln[ln(µ_2/µ_1)], ..., ln[ln(µ_8/µ_7)] in the vertical axis. (Equation 4 of the paper).
4.- I did a linear regression with Excel to obtain beta and gamma parameters:
Code: Select all
Gompertz fit:
Fitted_µ = alpha*exp[-beta*exp(-gamma*TC)]
Linear fit of the 8 data points = m*TC + n ~ -0.64087232*TC - 0.51556368 (R² ~ 0.99744438)
(Equation 4): gamma = -m ~ 0.64087232
(Equation 4): beta = exp(n)/[exp(gamma) - 1] ~ 0.66489741
Code: Select all
Fitted_µ ~ exp[-0.66489741*exp(-gamma*0.64087232)]
Converting fitted_µ into Elo gain (rounding up to the nearest Elo integer):
TC Elo Fitted Elo Elo - (fitted Elo)
1 144 151 -7
2 277 277 0
3 389 396 -9
4 490 512 -22
5 583 625 -42
6 656 738 -82
7 715 850 -135
8 766 961 -195
Average error = -61.5 Elo
Code: Select all
alpha_TC = exp[ln(µ_TC) + beta*exp(-gamma*TC)]
I obtain 8 values of alpha_TC. If I randomly choose alpha = average(alpha_TC) ~ 0.99310185
Fitted_µ ~ 0.99310185*exp[-0.66489741*exp(-gamma*0.64087232)]
TC Elo Fitted Elo Elo - (fitted Elo)
1 144 147 -3
2 277 270 -7
3 389 384 +5
4 490 489 +1
5 583 585 -2
6 656 668 -12
7 715 735 -20
8 766 785 -19
Average error ~ -7.1 Elo
Furthermore, I did not take into account error bars.
Bonus: if I continue giving increasing values of TC to fitted_µ ~ 0.99310185*exp[-0.66489741*exp(-gamma*0.64087232)], I get the next estimated Elo gains:
Code: Select all
Converting fitted_µ into Elo gain (rounding up to the nearest Elo integer):
Comparison TC Fitted Elo
5120 + 51.2 vs 2560 + 25.6 8 785
10240 + 102.4 vs 5120 + 51.2 9 818 (+33)
20480 + 204.8 vs 10240 + 102.4 10 838 (+20)
40960 + 409.6 vs 20480 + 204.8 11 849 (+11)
81920 + 819.2 vs 40960 + 409.6 12 856 ( +7)
It might be interesting to fit win ratio, draw ratio and lose ratio in similar ways.
Last but not least: thank you very much, Andreas.
Regards from Spain.
Ajedrecista.
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: Doubling of time control
Thanks for correction.
But where are the 1500 opening positions?
I should like to know that how you can calculate the Elo number from a self play match. Elo number is always a relative number. But in the case of self play what is the basic point?
I think from your post some facts are missing still.
Robert
But where are the 1500 opening positions?
I should like to know that how you can calculate the Elo number from a self play match. Elo number is always a relative number. But in the case of self play what is the basic point?
I think from your post some facts are missing still.
Robert