FGRL 60 sec + 0.6 sec Rating list - Komodo 11.2

[fastgm]

FGRL Rating list - 60 seconds + 0.6 seconds

Komodo 11.2 (+34 to Komodo 11.01)

http://www.fastgm.de

[JJJ]

Nice result so far. Still loosing Stockfish 8 at bullet but now same rating and winning against Houdini 5.

[fastgm]

Komodo - Progress

Code: Select all

     Engine                       :    Elo  Error   Games    (%)      W       D      L     D(%)   CFS
  ----------------------------------------------------------------------------------------------------
   5 Komodo 11.2 64-bit           :   3479      9   14000   91.84   11974   1767    259   12.62   100
   7 Komodo 11.01 64-bit          :   3445      8   15750   90.29   13134   2174    442   13.80    77
   8 Komodo 10.4 64-bit           :   3441      8   15250   90.47   12718   2157    375   14.14    85
  10 Komodo 10.3 64-bit           :   3421      8   14750   89.14   12045   2205    500   14.95    66
  11 Komodo 10.2 64-bit           :   3419      7   17250   86.59   13435   3002    813   17.40    97
  13 Komodo 10.1 64-bit           :   3401      6   22250   86.10   17037   4239    974   19.05    63
  17 Komodo 10 64-bit             :   3384      6   21500   84.48   15931   4464   1105   20.76   100
  18 Komodo 9.42 64-bit           :   3372      6   21750   83.37   15818   4629   1303   21.28   100
  19 Komodo 9.42 64-bit Dyn110    :   3361      6   18000   83.75   13176   3797   1027   21.09    55
  20 Komodo 9.4 64-bit            :   3360      6   19000   82.91   13643   4221   1136   22.22    98
  22 Komodo 9.3 64-bit            :   3342      5   22250   81.57   15609   5080   1561   22.83    85
  23 Komodo 9.2 64-bit            :   3338      5   22250   81.31   15623   4938   1689   22.19   100
  25 Komodo 9.1 64-bit            :   3316      4   35000   81.64   24311   8525   2164   24.36    78
  27 Komodo 9 64-bit              :   3306      5   22500   78.71   14857   5707   1936   25.36   100
  31 Komodo 8 64-bit              :   3262      4   32000   75.90   20102   8370   3528   26.16    92
  38 Komodo 7a 64-bit             :   3204      5   19750   75.53   12252   5332   2166   27.00   100
  40 Komodo TCECr 64-bit          :   3177      5   19750   73.13   11633   5621   2496   28.46    96
  43 Komodo 6 64-bit              :   3159      5   19750   71.53   11212   5830   2708   29.52    75
  48 Komodo CCT 64-bit            :   3141      5   19750   69.83   10900   5783   3067   29.28    98
  54 Komodo 5.1 64-bit            :   3126      5   19750   68.33   10603   5784   3363   29.29    83

.

[Gusev]

Please test Sting SF 8.5. Thanks!!

[Lyudmil Tsvetkov]

Komodo - Progress

Code: Select all

     Engine                       :    Elo  Error   Games    (%)      W       D      L     D(%)   CFS
  ----------------------------------------------------------------------------------------------------
   5 Komodo 11.2 64-bit           :   3479      9   14000   91.84   11974   1767    259   12.62   100
   7 Komodo 11.01 64-bit          :   3445      8   15750   90.29   13134   2174    442   13.80    77
   8 Komodo 10.4 64-bit           :   3441      8   15250   90.47   12718   2157    375   14.14    85
  10 Komodo 10.3 64-bit           :   3421      8   14750   89.14   12045   2205    500   14.95    66
  11 Komodo 10.2 64-bit           :   3419      7   17250   86.59   13435   3002    813   17.40    97
  13 Komodo 10.1 64-bit           :   3401      6   22250   86.10   17037   4239    974   19.05    63
  17 Komodo 10 64-bit             :   3384      6   21500   84.48   15931   4464   1105   20.76   100
  18 Komodo 9.42 64-bit           :   3372      6   21750   83.37   15818   4629   1303   21.28   100
  19 Komodo 9.42 64-bit Dyn110    :   3361      6   18000   83.75   13176   3797   1027   21.09    55
  20 Komodo 9.4 64-bit            :   3360      6   19000   82.91   13643   4221   1136   22.22    98
  22 Komodo 9.3 64-bit            :   3342      5   22250   81.57   15609   5080   1561   22.83    85
  23 Komodo 9.2 64-bit            :   3338      5   22250   81.31   15623   4938   1689   22.19   100
  25 Komodo 9.1 64-bit            :   3316      4   35000   81.64   24311   8525   2164   24.36    78
  27 Komodo 9 64-bit              :   3306      5   22500   78.71   14857   5707   1936   25.36   100
  31 Komodo 8 64-bit              :   3262      4   32000   75.90   20102   8370   3528   26.16    92
  38 Komodo 7a 64-bit             :   3204      5   19750   75.53   12252   5332   2166   27.00   100
  40 Komodo TCECr 64-bit          :   3177      5   19750   73.13   11633   5621   2496   28.46    96
  43 Komodo 6 64-bit              :   3159      5   19750   71.53   11212   5830   2708   29.52    75
  48 Komodo CCT 64-bit            :   3141      5   19750   69.83   10900   5783   3067   29.28    98
  54 Komodo 5.1 64-bit            :   3126      5   19750   68.33   10603   5784   3363   29.29    83

.

there seem to be some idyosyncratic swings at this TC between different versions.

ranging from 0 elo to 35 elo.

at the same time, they have been reporting steady progress with a larger number of games and possibly a bit longer TC.

so, quite probably, the big deviations are due to either:

1) a bit bigger error bars
2) discrepancy of performance at different TCs for specific versions

[lkaufman]

Komodo - Progress

Code: Select all

     Engine                       :    Elo  Error   Games    (%)      W       D      L     D(%)   CFS
  ----------------------------------------------------------------------------------------------------
   5 Komodo 11.2 64-bit           :   3479      9   14000   91.84   11974   1767    259   12.62   100
   7 Komodo 11.01 64-bit          :   3445      8   15750   90.29   13134   2174    442   13.80    77
   8 Komodo 10.4 64-bit           :   3441      8   15250   90.47   12718   2157    375   14.14    85
  10 Komodo 10.3 64-bit           :   3421      8   14750   89.14   12045   2205    500   14.95    66
  11 Komodo 10.2 64-bit           :   3419      7   17250   86.59   13435   3002    813   17.40    97
  13 Komodo 10.1 64-bit           :   3401      6   22250   86.10   17037   4239    974   19.05    63
  17 Komodo 10 64-bit             :   3384      6   21500   84.48   15931   4464   1105   20.76   100
  18 Komodo 9.42 64-bit           :   3372      6   21750   83.37   15818   4629   1303   21.28   100
  19 Komodo 9.42 64-bit Dyn110    :   3361      6   18000   83.75   13176   3797   1027   21.09    55
  20 Komodo 9.4 64-bit            :   3360      6   19000   82.91   13643   4221   1136   22.22    98
  22 Komodo 9.3 64-bit            :   3342      5   22250   81.57   15609   5080   1561   22.83    85
  23 Komodo 9.2 64-bit            :   3338      5   22250   81.31   15623   4938   1689   22.19   100
  25 Komodo 9.1 64-bit            :   3316      4   35000   81.64   24311   8525   2164   24.36    78
  27 Komodo 9 64-bit              :   3306      5   22500   78.71   14857   5707   1936   25.36   100
  31 Komodo 8 64-bit              :   3262      4   32000   75.90   20102   8370   3528   26.16    92
  38 Komodo 7a 64-bit             :   3204      5   19750   75.53   12252   5332   2166   27.00   100
  40 Komodo TCECr 64-bit          :   3177      5   19750   73.13   11633   5621   2496   28.46    96
  43 Komodo 6 64-bit              :   3159      5   19750   71.53   11212   5830   2708   29.52    75
  48 Komodo CCT 64-bit            :   3141      5   19750   69.83   10900   5783   3067   29.28    98
  54 Komodo 5.1 64-bit            :   3126      5   19750   68.33   10603   5784   3363   29.29    83

.

there seem to be some idyosyncratic swings at this TC between different versions.

ranging from 0 elo to 35 elo.

at the same time, they have been reporting steady progress with a larger number of games and possibly a bit longer TC.

so, quite probably, the big deviations are due to either:

1) a bit bigger error bars
2) discrepancy of performance at different TCs for specific versions

From Komodo 9 (when we went to the roughly quarterly update subscription system) thru Komodo 11.01, ignoring versions which were just bugfixes (only the second digit after the decimal changed), the average gain per version is 14 elo, remarkably close to our 15 elo per version goal. Most of the deviations from this are within sample error, although as you say in some cases we made changes that helped at longer TCs but might have hurt at bullet chess. The gain from 11.01 to 11.2 clearly breaks this pattern; sometimes we find something significant.

[Lyudmil Tsvetkov]

Komodo - Progress

Code: Select all

     Engine                       :    Elo  Error   Games    (%)      W       D      L     D(%)   CFS
  ----------------------------------------------------------------------------------------------------
   5 Komodo 11.2 64-bit           :   3479      9   14000   91.84   11974   1767    259   12.62   100
   7 Komodo 11.01 64-bit          :   3445      8   15750   90.29   13134   2174    442   13.80    77
   8 Komodo 10.4 64-bit           :   3441      8   15250   90.47   12718   2157    375   14.14    85
  10 Komodo 10.3 64-bit           :   3421      8   14750   89.14   12045   2205    500   14.95    66
  11 Komodo 10.2 64-bit           :   3419      7   17250   86.59   13435   3002    813   17.40    97
  13 Komodo 10.1 64-bit           :   3401      6   22250   86.10   17037   4239    974   19.05    63
  17 Komodo 10 64-bit             :   3384      6   21500   84.48   15931   4464   1105   20.76   100
  18 Komodo 9.42 64-bit           :   3372      6   21750   83.37   15818   4629   1303   21.28   100
  19 Komodo 9.42 64-bit Dyn110    :   3361      6   18000   83.75   13176   3797   1027   21.09    55
  20 Komodo 9.4 64-bit            :   3360      6   19000   82.91   13643   4221   1136   22.22    98
  22 Komodo 9.3 64-bit            :   3342      5   22250   81.57   15609   5080   1561   22.83    85
  23 Komodo 9.2 64-bit            :   3338      5   22250   81.31   15623   4938   1689   22.19   100
  25 Komodo 9.1 64-bit            :   3316      4   35000   81.64   24311   8525   2164   24.36    78
  27 Komodo 9 64-bit              :   3306      5   22500   78.71   14857   5707   1936   25.36   100
  31 Komodo 8 64-bit              :   3262      4   32000   75.90   20102   8370   3528   26.16    92
  38 Komodo 7a 64-bit             :   3204      5   19750   75.53   12252   5332   2166   27.00   100
  40 Komodo TCECr 64-bit          :   3177      5   19750   73.13   11633   5621   2496   28.46    96
  43 Komodo 6 64-bit              :   3159      5   19750   71.53   11212   5830   2708   29.52    75
  48 Komodo CCT 64-bit            :   3141      5   19750   69.83   10900   5783   3067   29.28    98
  54 Komodo 5.1 64-bit            :   3126      5   19750   68.33   10603   5784   3363   29.29    83

.

there seem to be some idyosyncratic swings at this TC between different versions.

ranging from 0 elo to 35 elo.

at the same time, they have been reporting steady progress with a larger number of games and possibly a bit longer TC.

so, quite probably, the big deviations are due to either:

1) a bit bigger error bars
2) discrepancy of performance at different TCs for specific versions

From Komodo 9 (when we went to the roughly quarterly update subscription system) thru Komodo 11.01, ignoring versions which were just bugfixes (only the second digit after the decimal changed), the average gain per version is 14 elo, remarkably close to our 15 elo per version goal. Most of the deviations from this are within sample error, although as you say in some cases we made changes that helped at longer TCs but might have hurt at bullet chess. The gain from 11.01 to 11.2 clearly breaks this pattern; sometimes we find something significant.

yeah, I commented only because having 0 elo gain at the one end, and 35 at the other seems a bit strange.

you know best what you have implemented, maybe your gain is larger this time, but it could also be possible that a single patch makes Komodo start back-scaling well again.

who knows?

in any case, now you have found the secret that makes Komodo play bullet as well as Houdini, whatever it is.

btw., would this mean that now Komodo would not scale so great from bullet to VLTC? (excuse me, Larry, I always ask questions )

[JJJ]

I like your questions Lyudmil

[Lyudmil Tsvetkov]

I like your questions Lyudmil

you make me flush...

let's see what master Larry will say.

[mjlef]

The changes we made should help elo at all time controls. We do not expect them to be specific to blitz time controls.

We do not know why Komodo does not do as well as some other programs at fast time controls. I think it is due to Komodo's extensive (and larger) evaluation, which slows our node rate down, but seems to help more at longer time controls. But if we can get enough elo gain, we will start looking better at these fast time controls. This is speculation.

I also think Komodo got a little lucky with the K 11.2 results reported here, and a little unlucky with some earlier results. Our reported gains at about 3 min + 1 sec were 24 elo. So I think once enough testing groups get more games we can get a more accurate estimate of elo gain at different time controls.

Mark