2100 elo engine!!!!Graham Banks wrote: ↑Sun May 27, 2018 12:08 am http://www.computerchess.org.uk/ccrl/40 ... 4-bit_w323
Leela Chess 0.10 64-bit w323 #110‑111 (2651 +19 −19)
LCZero: Progress and Scaling. Relation to CCRL Elo
Moderators: hgm, Rebel, chrisw
-
- Posts: 1142
- Joined: Thu Dec 28, 2017 4:06 pm
- Location: Argentina
Re: LCZero: Progress and Scaling. Relation to CCRL Elo
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls
-
- Posts: 3019
- Joined: Wed Mar 08, 2006 9:57 pm
- Location: Rio de Janeiro, Brazil
Re: LCZero: Progress and Scaling. Relation to CCRL Elo
It is. Around 2100. You are not really going to nitpick over a piddly 550 Elo are you? Mind you, I think that was just using the CPU version too.CMCanavessi wrote: ↑Mon May 28, 2018 12:46 am2100 elo engine!!!!Graham Banks wrote: ↑Sun May 27, 2018 12:08 am http://www.computerchess.org.uk/ccrl/40 ... 4-bit_w323
Leela Chess 0.10 64-bit w323 #110‑111 (2651 +19 −19)
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."
-
- Posts: 1797
- Joined: Thu Sep 18, 2008 10:24 pm
Re: LCZero: Progress and Scaling. Relation to CCRL Elo
Has Leela hit a plateau? There doesn't seem to be progress anymore.
Also any news on the bigger network?
Also any news on the bigger network?
-
- Posts: 4190
- Joined: Wed Nov 25, 2009 1:47 am
Re: LCZero: Progress and Scaling. Relation to CCRL Elo
It hit plateau already early with 15x192 network around 237. So far this was mostly regression and very small improvement (since network 320 basically).
20x256 won't bring anything since net gain is smaller than computation slowdown.
-
- Posts: 1766
- Joined: Wed Jun 03, 2009 12:14 am
Re: LCZero: Progress and Scaling. Relation to CCRL Elo
don't know any details, but superficially it looks like self-play elo has plateaued but benchmarks against SF show progress continuing. the latest net looks like the strongest by that criteria & seems like it'd be a good one for thorough testing at longer time controls. think the CCRL rating on a GTX 1060 would surprise a lot of people one way or another if it was tested properly.
https://docs.google.com/spreadsheets/d/ ... li=1#gid=0
https://docs.google.com/spreadsheets/d/ ... edit#gid=0
-
- Posts: 2071
- Joined: Thu May 04, 2006 3:40 am
- Location: Dune
Re: LCZero: Progress and Scaling. Relation to CCRL Elo
According to this chart it seems Leela is now almost as strong as Rodent 3, so it seems to be making progress:
https://docs.google.com/spreadsheets/d/ ... edit#gid=0
https://docs.google.com/spreadsheets/d/ ... edit#gid=0
-
- Posts: 1797
- Joined: Thu Sep 18, 2008 10:24 pm
Re: LCZero: Progress and Scaling. Relation to CCRL Elo
To me it looks like stalling. I wonder if this can be resurrected.
-
- Posts: 1142
- Joined: Thu Dec 28, 2017 4:06 pm
- Location: Argentina
Re: LCZero: Progress and Scaling. Relation to CCRL Elo
How can you "resurrect" something that's not dead?
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls
-
- Posts: 1766
- Joined: Wed Jun 03, 2009 12:14 am
Re: LCZero: Progress and Scaling. Relation to CCRL Elo
yeah idk, maybe look at it harder. it's not gonna be 50 elo/day or whatever, that part of the curve has sailed. it's clearly not stalled. 321-351 is tiny a blip & there's measurable growth.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: LCZero: Progress and Scaling. Relation to CCRL Elo
From NN342 to NN352 it indeed seems to be stalling against an A/B engine Arasan 20.5. Here is the results of gauntlet:
Code: Select all
Games Completed = 1600 of 1600 (Avg game length = 13.001 sec)
Settings = Gauntlet/64MB/100ms per move/M 9000cp for 30 moves, D 150 moves/EPD:C:\LittleBlitzer\3moves_GM_04.epd(817)
Time = 29352 sec elapsed, 0 sec remaining
1. Arasan 20.5 1062.5/1600 872-347-381 (L: m=347 t=0 i=0 a=0) (D: r=297 i=55 f=7 s=9 a=13) (tpm=107.6 d=16.21 nps=1682392)
2. Lc0 NN342 269.0/800 172-434-194 (L: m=429 t=0 i=5 a=0) (D: r=149 i=26 f=4 s=6 a=9) (tpm=108.8 d=1.25 nps=1871)
3. Lc0 NN352 268.5/800 175-438-187 (L: m=436 t=0 i=2 a=0) (D: r=148 i=29 f=3 s=3 a=4) (tpm=108.6 d=1.22 nps=1700)
To see the scaling, I played matches of 800 games each at 0.1s/move, 0.4s/move, 1.6s/move, in total 4 doublings, of Lc0 NN352 on GTX 1060 6GB against Arasan 20.5 on one core (3061 CCRL 40/4' Elo points on one core). In fact the time used per move was taken from what LittleBlitzer reports, and it is not exactly what I set there. The CCRL performance of Lc0 cuDNN ID352 at three time controls are:
0.109 s/move --> 2943 Elo points
0.344 s/move --> 3015 Elo points
1.215 s/move --> 3091 Elo points
The fitted function as CCRL Elo performance as function of time control (scaling) of Lc0 with time per move is
Code: Select all
CCRL Elo of cuDNN Lc0 = 3079.51 + 61.3627 * ln(seconds per move)
on GTX 1060.
So, at LTC, cuDNN Lc0 ID352 is above 3300 CCRL Elo points by this extrapolation, which confirms my earlier results, when playing games against Stockfish (but with very weak accumulated statistic). Here the weak point is the extrapolation itself, but I prefer that. For a top GTX 1080 Ti GPU, add some 70 Elo points to these results, maybe more.