FGRL Rating list - 60 seconds + 0.6 seconds
Komodo 11.2 (+34 to Komodo 11.01)
http://www.fastgm.de
FGRL 60 sec + 0.6 sec Rating list - Komodo 11.2
Moderators: hgm, Rebel, chrisw
-
- Posts: 818
- Joined: Mon Aug 19, 2013 6:57 pm
-
- Posts: 1346
- Joined: Sat Apr 19, 2014 1:47 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo 11.2
Nice result so far. Still loosing Stockfish 8 at bullet but now same rating and winning against Houdini 5.
-
- Posts: 818
- Joined: Mon Aug 19, 2013 6:57 pm
FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
Komodo - Progress
.
Code: Select all
Engine : Elo Error Games (%) W D L D(%) CFS
----------------------------------------------------------------------------------------------------
5 Komodo 11.2 64-bit : 3479 9 14000 91.84 11974 1767 259 12.62 100
7 Komodo 11.01 64-bit : 3445 8 15750 90.29 13134 2174 442 13.80 77
8 Komodo 10.4 64-bit : 3441 8 15250 90.47 12718 2157 375 14.14 85
10 Komodo 10.3 64-bit : 3421 8 14750 89.14 12045 2205 500 14.95 66
11 Komodo 10.2 64-bit : 3419 7 17250 86.59 13435 3002 813 17.40 97
13 Komodo 10.1 64-bit : 3401 6 22250 86.10 17037 4239 974 19.05 63
17 Komodo 10 64-bit : 3384 6 21500 84.48 15931 4464 1105 20.76 100
18 Komodo 9.42 64-bit : 3372 6 21750 83.37 15818 4629 1303 21.28 100
19 Komodo 9.42 64-bit Dyn110 : 3361 6 18000 83.75 13176 3797 1027 21.09 55
20 Komodo 9.4 64-bit : 3360 6 19000 82.91 13643 4221 1136 22.22 98
22 Komodo 9.3 64-bit : 3342 5 22250 81.57 15609 5080 1561 22.83 85
23 Komodo 9.2 64-bit : 3338 5 22250 81.31 15623 4938 1689 22.19 100
25 Komodo 9.1 64-bit : 3316 4 35000 81.64 24311 8525 2164 24.36 78
27 Komodo 9 64-bit : 3306 5 22500 78.71 14857 5707 1936 25.36 100
31 Komodo 8 64-bit : 3262 4 32000 75.90 20102 8370 3528 26.16 92
38 Komodo 7a 64-bit : 3204 5 19750 75.53 12252 5332 2166 27.00 100
40 Komodo TCECr 64-bit : 3177 5 19750 73.13 11633 5621 2496 28.46 96
43 Komodo 6 64-bit : 3159 5 19750 71.53 11212 5830 2708 29.52 75
48 Komodo CCT 64-bit : 3141 5 19750 69.83 10900 5783 3067 29.28 98
54 Komodo 5.1 64-bit : 3126 5 19750 68.33 10603 5784 3363 29.29 83
-
- Posts: 1476
- Joined: Mon Jan 28, 2013 2:51 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo 11.2
Please test Sting SF 8.5. Thanks!!
-
- Posts: 6052
- Joined: Tue Jun 12, 2012 12:41 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
there seem to be some idyosyncratic swings at this TC between different versions.fastgm wrote:Komodo - Progress.Code: Select all
Engine : Elo Error Games (%) W D L D(%) CFS ---------------------------------------------------------------------------------------------------- 5 Komodo 11.2 64-bit : 3479 9 14000 91.84 11974 1767 259 12.62 100 7 Komodo 11.01 64-bit : 3445 8 15750 90.29 13134 2174 442 13.80 77 8 Komodo 10.4 64-bit : 3441 8 15250 90.47 12718 2157 375 14.14 85 10 Komodo 10.3 64-bit : 3421 8 14750 89.14 12045 2205 500 14.95 66 11 Komodo 10.2 64-bit : 3419 7 17250 86.59 13435 3002 813 17.40 97 13 Komodo 10.1 64-bit : 3401 6 22250 86.10 17037 4239 974 19.05 63 17 Komodo 10 64-bit : 3384 6 21500 84.48 15931 4464 1105 20.76 100 18 Komodo 9.42 64-bit : 3372 6 21750 83.37 15818 4629 1303 21.28 100 19 Komodo 9.42 64-bit Dyn110 : 3361 6 18000 83.75 13176 3797 1027 21.09 55 20 Komodo 9.4 64-bit : 3360 6 19000 82.91 13643 4221 1136 22.22 98 22 Komodo 9.3 64-bit : 3342 5 22250 81.57 15609 5080 1561 22.83 85 23 Komodo 9.2 64-bit : 3338 5 22250 81.31 15623 4938 1689 22.19 100 25 Komodo 9.1 64-bit : 3316 4 35000 81.64 24311 8525 2164 24.36 78 27 Komodo 9 64-bit : 3306 5 22500 78.71 14857 5707 1936 25.36 100 31 Komodo 8 64-bit : 3262 4 32000 75.90 20102 8370 3528 26.16 92 38 Komodo 7a 64-bit : 3204 5 19750 75.53 12252 5332 2166 27.00 100 40 Komodo TCECr 64-bit : 3177 5 19750 73.13 11633 5621 2496 28.46 96 43 Komodo 6 64-bit : 3159 5 19750 71.53 11212 5830 2708 29.52 75 48 Komodo CCT 64-bit : 3141 5 19750 69.83 10900 5783 3067 29.28 98 54 Komodo 5.1 64-bit : 3126 5 19750 68.33 10603 5784 3363 29.29 83
ranging from 0 elo to 35 elo.
at the same time, they have been reporting steady progress with a larger number of games and possibly a bit longer TC.
so, quite probably, the big deviations are due to either:
1) a bit bigger error bars
2) discrepancy of performance at different TCs for specific versions
-
- Posts: 5960
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
From Komodo 9 (when we went to the roughly quarterly update subscription system) thru Komodo 11.01, ignoring versions which were just bugfixes (only the second digit after the decimal changed), the average gain per version is 14 elo, remarkably close to our 15 elo per version goal. Most of the deviations from this are within sample error, although as you say in some cases we made changes that helped at longer TCs but might have hurt at bullet chess. The gain from 11.01 to 11.2 clearly breaks this pattern; sometimes we find something significant.Lyudmil Tsvetkov wrote:there seem to be some idyosyncratic swings at this TC between different versions.fastgm wrote:Komodo - Progress.Code: Select all
Engine : Elo Error Games (%) W D L D(%) CFS ---------------------------------------------------------------------------------------------------- 5 Komodo 11.2 64-bit : 3479 9 14000 91.84 11974 1767 259 12.62 100 7 Komodo 11.01 64-bit : 3445 8 15750 90.29 13134 2174 442 13.80 77 8 Komodo 10.4 64-bit : 3441 8 15250 90.47 12718 2157 375 14.14 85 10 Komodo 10.3 64-bit : 3421 8 14750 89.14 12045 2205 500 14.95 66 11 Komodo 10.2 64-bit : 3419 7 17250 86.59 13435 3002 813 17.40 97 13 Komodo 10.1 64-bit : 3401 6 22250 86.10 17037 4239 974 19.05 63 17 Komodo 10 64-bit : 3384 6 21500 84.48 15931 4464 1105 20.76 100 18 Komodo 9.42 64-bit : 3372 6 21750 83.37 15818 4629 1303 21.28 100 19 Komodo 9.42 64-bit Dyn110 : 3361 6 18000 83.75 13176 3797 1027 21.09 55 20 Komodo 9.4 64-bit : 3360 6 19000 82.91 13643 4221 1136 22.22 98 22 Komodo 9.3 64-bit : 3342 5 22250 81.57 15609 5080 1561 22.83 85 23 Komodo 9.2 64-bit : 3338 5 22250 81.31 15623 4938 1689 22.19 100 25 Komodo 9.1 64-bit : 3316 4 35000 81.64 24311 8525 2164 24.36 78 27 Komodo 9 64-bit : 3306 5 22500 78.71 14857 5707 1936 25.36 100 31 Komodo 8 64-bit : 3262 4 32000 75.90 20102 8370 3528 26.16 92 38 Komodo 7a 64-bit : 3204 5 19750 75.53 12252 5332 2166 27.00 100 40 Komodo TCECr 64-bit : 3177 5 19750 73.13 11633 5621 2496 28.46 96 43 Komodo 6 64-bit : 3159 5 19750 71.53 11212 5830 2708 29.52 75 48 Komodo CCT 64-bit : 3141 5 19750 69.83 10900 5783 3067 29.28 98 54 Komodo 5.1 64-bit : 3126 5 19750 68.33 10603 5784 3363 29.29 83
ranging from 0 elo to 35 elo.
at the same time, they have been reporting steady progress with a larger number of games and possibly a bit longer TC.
so, quite probably, the big deviations are due to either:
1) a bit bigger error bars
2) discrepancy of performance at different TCs for specific versions
Komodo rules!
-
- Posts: 6052
- Joined: Tue Jun 12, 2012 12:41 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
yeah, I commented only because having 0 elo gain at the one end, and 35 at the other seems a bit strange.lkaufman wrote:From Komodo 9 (when we went to the roughly quarterly update subscription system) thru Komodo 11.01, ignoring versions which were just bugfixes (only the second digit after the decimal changed), the average gain per version is 14 elo, remarkably close to our 15 elo per version goal. Most of the deviations from this are within sample error, although as you say in some cases we made changes that helped at longer TCs but might have hurt at bullet chess. The gain from 11.01 to 11.2 clearly breaks this pattern; sometimes we find something significant.Lyudmil Tsvetkov wrote:there seem to be some idyosyncratic swings at this TC between different versions.fastgm wrote:Komodo - Progress.Code: Select all
Engine : Elo Error Games (%) W D L D(%) CFS ---------------------------------------------------------------------------------------------------- 5 Komodo 11.2 64-bit : 3479 9 14000 91.84 11974 1767 259 12.62 100 7 Komodo 11.01 64-bit : 3445 8 15750 90.29 13134 2174 442 13.80 77 8 Komodo 10.4 64-bit : 3441 8 15250 90.47 12718 2157 375 14.14 85 10 Komodo 10.3 64-bit : 3421 8 14750 89.14 12045 2205 500 14.95 66 11 Komodo 10.2 64-bit : 3419 7 17250 86.59 13435 3002 813 17.40 97 13 Komodo 10.1 64-bit : 3401 6 22250 86.10 17037 4239 974 19.05 63 17 Komodo 10 64-bit : 3384 6 21500 84.48 15931 4464 1105 20.76 100 18 Komodo 9.42 64-bit : 3372 6 21750 83.37 15818 4629 1303 21.28 100 19 Komodo 9.42 64-bit Dyn110 : 3361 6 18000 83.75 13176 3797 1027 21.09 55 20 Komodo 9.4 64-bit : 3360 6 19000 82.91 13643 4221 1136 22.22 98 22 Komodo 9.3 64-bit : 3342 5 22250 81.57 15609 5080 1561 22.83 85 23 Komodo 9.2 64-bit : 3338 5 22250 81.31 15623 4938 1689 22.19 100 25 Komodo 9.1 64-bit : 3316 4 35000 81.64 24311 8525 2164 24.36 78 27 Komodo 9 64-bit : 3306 5 22500 78.71 14857 5707 1936 25.36 100 31 Komodo 8 64-bit : 3262 4 32000 75.90 20102 8370 3528 26.16 92 38 Komodo 7a 64-bit : 3204 5 19750 75.53 12252 5332 2166 27.00 100 40 Komodo TCECr 64-bit : 3177 5 19750 73.13 11633 5621 2496 28.46 96 43 Komodo 6 64-bit : 3159 5 19750 71.53 11212 5830 2708 29.52 75 48 Komodo CCT 64-bit : 3141 5 19750 69.83 10900 5783 3067 29.28 98 54 Komodo 5.1 64-bit : 3126 5 19750 68.33 10603 5784 3363 29.29 83
ranging from 0 elo to 35 elo.
at the same time, they have been reporting steady progress with a larger number of games and possibly a bit longer TC.
so, quite probably, the big deviations are due to either:
1) a bit bigger error bars
2) discrepancy of performance at different TCs for specific versions
you know best what you have implemented, maybe your gain is larger this time, but it could also be possible that a single patch makes Komodo start back-scaling well again.
who knows?
in any case, now you have found the secret that makes Komodo play bullet as well as Houdini, whatever it is.
btw., would this mean that now Komodo would not scale so great from bullet to VLTC? (excuse me, Larry, I always ask questions )
-
- Posts: 1346
- Joined: Sat Apr 19, 2014 1:47 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
I like your questions Lyudmil
-
- Posts: 6052
- Joined: Tue Jun 12, 2012 12:41 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
you make me flush...JJJ wrote:I like your questions Lyudmil
let's see what master Larry will say.
-
- Posts: 1494
- Joined: Thu Mar 30, 2006 2:08 pm
Re: FGRL 60 sec + 0.6 sec Rating list - Komodo - Progress
The changes we made should help elo at all time controls. We do not expect them to be specific to blitz time controls.
We do not know why Komodo does not do as well as some other programs at fast time controls. I think it is due to Komodo's extensive (and larger) evaluation, which slows our node rate down, but seems to help more at longer time controls. But if we can get enough elo gain, we will start looking better at these fast time controls. This is speculation.
I also think Komodo got a little lucky with the K 11.2 results reported here, and a little unlucky with some earlier results. Our reported gains at about 3 min + 1 sec were 24 elo. So I think once enough testing groups get more games we can get a more accurate estimate of elo gain at different time controls.
Mark
We do not know why Komodo does not do as well as some other programs at fast time controls. I think it is due to Komodo's extensive (and larger) evaluation, which slows our node rate down, but seems to help more at longer time controls. But if we can get enough elo gain, we will start looking better at these fast time controls. This is speculation.
I also think Komodo got a little lucky with the K 11.2 results reported here, and a little unlucky with some earlier results. Our reported gains at about 3 min + 1 sec were 24 elo. So I think once enough testing groups get more games we can get a more accurate estimate of elo gain at different time controls.
Mark