Werewolf wrote: ↑Mon May 28, 2018 10:25 pm
To me it looks like stalling. I wonder if this can be resurrected.
From NN342 to NN352 it indeed seems to be stalling against an A/B engine Arasan 20.5. Here is the results of gauntlet:
Code: Select all
Games Completed = 1600 of 1600 (Avg game length = 13.001 sec)
Settings = Gauntlet/64MB/100ms per move/M 9000cp for 30 moves, D 150 moves/EPD:C:\LittleBlitzer\3moves_GM_04.epd(817)
Time = 29352 sec elapsed, 0 sec remaining
1. Arasan 20.5 1062.5/1600 872-347-381 (L: m=347 t=0 i=0 a=0) (D: r=297 i=55 f=7 s=9 a=13) (tpm=107.6 d=16.21 nps=1682392)
2. Lc0 NN342 269.0/800 172-434-194 (L: m=429 t=0 i=5 a=0) (D: r=149 i=26 f=4 s=6 a=9) (tpm=108.8 d=1.25 nps=1871)
3. Lc0 NN352 268.5/800 175-438-187 (L: m=436 t=0 i=2 a=0) (D: r=148 i=29 f=3 s=3 a=4) (tpm=108.6 d=1.22 nps=1700)
Lc0 is the cuDNN version with default settings, I am tired of fiddling with its parameters, with performances varying at time controls, and it seems anyway there are some serious bugs.
To see the
scaling, I played matches of 800 games each at 0.1s/move, 0.4s/move, 1.6s/move, in total 4 doublings, of Lc0 NN352 on GTX 1060 6GB against Arasan 20.5 on one core (3061 CCRL 40/4' Elo points on one core). In fact the time used per move was taken from what LittleBlitzer reports, and it is not exactly what I set there. The CCRL performance of Lc0 cuDNN ID352 at three time controls are:
0.109 s/move --> 2943 Elo points
0.344 s/move --> 3015 Elo points
1.215 s/move --> 3091 Elo points
The fitted function as CCRL Elo performance as function of time control (scaling) of Lc0 with time per move is
Code: Select all
CCRL Elo of cuDNN Lc0 = 3079.51 + 61.3627 * ln(seconds per move)
on GTX 1060.
I also assumed that Arasan 20.5 scales as a standard A/B engine. The fitted intuitive function of scaling with two parameters correlates 0.99999 with the three data points, and it is not an overfit. Here is the plot:
So, at LTC, cuDNN Lc0 ID352 is above 3300 CCRL Elo points by this extrapolation, which confirms my earlier results, when playing games against Stockfish (but with very weak accumulated statistic). Here the weak point is the extrapolation itself, but I prefer that. For a top GTX 1080 Ti GPU, add some 70 Elo points to these results, maybe more.