what is your 'PC'?? pre-pentium, for sure...Henk wrote: ↑Fri Jul 13, 2018 3:25 pm One month 50 (real) Elo and 4 million training games later. 1 training game would keep my pc busy whole day. So 4 million days later would be 11000 years. If your pc runs 100 times faster then you only need 110 years to get this fantastic result of 50 Elo points.
Something goes wrong with lc0 since yesterday?
Moderators: hgm, Rebel, chrisw
-
- Posts: 1766
- Joined: Wed Jun 03, 2009 12:14 am
Re: Something goes wrong with lc0 since yesterday?
-
- Posts: 7220
- Joined: Mon May 27, 2013 10:31 am
Re: Something goes wrong with lc0 since yesterday?
I-5 3210M CPU 2.50GHZ. Bought in 2013 or so
When I start my computer I already have to wait half an hour before I can use it. This was not so one year ago. I don't know what happened.
Maybe visual studio 2017 ? Before I used 2015 version.
When I start my computer I already have to wait half an hour before I can use it. This was not so one year ago. I don't know what happened.
Maybe visual studio 2017 ? Before I used 2015 version.
-
- Posts: 1627
- Joined: Thu Mar 09, 2006 12:35 pm
Re: Something goes wrong with lc0 since yesterday?
Despite what you said here, i used 40/2 for Leela also, in the following gaunlet since i find it unfair to offer Leela double time.Laskos wrote: ↑Wed Jul 11, 2018 7:53 pm 40/2' against single core is fine, but the most strictly correct simulation of CCRL conditions with lc0 on GPU would probably be to leave the lc0 at the same 40/4', as lc0 speed on GPU is very weakly dependent on CPU (2 threads), and for AB engines use 40/2' on a reasonable i7 core. Anyway, we get a picture, I use the same 40/2' bench for both lc0 and AB engines as you use. My GPU is Nvidia 1060 6GB.
The following 2 gaunlets are for the latest test net, but the way it evolved last day is AMAZING!
AB engines use 1 CPU i5-6500 3.4 GHz that i tested to be 2.1 times faster than CCRL's hardware so to have equivalence of the CCRL 40/4 Elo i used a 40/2 time control.
Leela is using Lc0 with GTX 1070 Ti with 40/2 time control also. I tested 2 testing nets, 10076 and the 1 day later(where they dropped LR) 10080.
Test10 10076 after 100 games had in the following gaunlet a 2970 ±45.4 CCRL 40/4 Elo performance and
Test10 10080 after 63 games so far has in the same gaunlet a 3243 ±75.1 CCRL 40/4 Elo performance!
Lc0 Test10 10076 after 100 games:
Code: Select all
Program CCRL Elo Error(cl 95%) Games Score
Lc0 Test10 10076 2969.7 ±45.4 100 (+26,=41,-33) 46.5 %
vs. : games ( +, =, -), (%) : Diff, SD, CFS (%)
Laser 1.5 : 20 ( 3, 8, 9), 35.0 : -127.3, 23.2, 0.0
Pedone 1.7 : 20 ( 5, 11, 4), 52.5 : -43.3, 23.2, 3.1
Wasp 3.0 : 20 ( 4, 11, 5), 47.5 : -17.3, 23.2, 22.7
Rybka 2.3.2a 64 bit : 20 ( 5, 4, 11), 35.0 : +6.7, 23.2, 61.3
Spike 1.4 : 20 ( 9, 7, 4), 62.5 : +51.7, 23.2, 98.7
Code: Select all
Program CCRL Elo Error(cl 95%) Games Score
Lc0 Test10 10080 3242.9 ±75.1 63 (+44,=12,-7), 79.4 %
vs. : games ( +, =, -), (%) : Diff, SD, CFS (%)
Laser 1.5 : 13 ( 8, 4, 1), 76.9 : +145.9, 38.3, 100.0
Pedone 1.7 : 12 ( 7, 3, 2), 70.8 : +229.9, 38.3, 100.0
Wasp 3.0 : 13 ( 12, 1, 0), 96.2 : +255.9, 38.3, 100.0
Rybka 2.3.2a 64 bit : 13 ( 8, 2, 3), 69.2 : +279.9, 38.3, 100.0
Spike 1.4 : 12 ( 9, 2, 1), 83.3 : +324.9, 38.3, 100.0
After his son's birth they've asked him:
"Is it a boy or girl?"
YES! He replied.....
"Is it a boy or girl?"
YES! He replied.....
-
- Posts: 177
- Joined: Wed May 23, 2018 9:29 pm
Re: Something goes wrong with lc0 since yesterday?
The learning rate was changed at network 10077 (as planned), so progress should indeed be much faster now.George Tsavdaris wrote: ↑Tue Jul 17, 2018 4:53 pmDespite what you said here, i used 40/2 for Leela also, in the following gaunlet since i find it unfair to offer Leela double time.Laskos wrote: ↑Wed Jul 11, 2018 7:53 pm 40/2' against single core is fine, but the most strictly correct simulation of CCRL conditions with lc0 on GPU would probably be to leave the lc0 at the same 40/4', as lc0 speed on GPU is very weakly dependent on CPU (2 threads), and for AB engines use 40/2' on a reasonable i7 core. Anyway, we get a picture, I use the same 40/2' bench for both lc0 and AB engines as you use. My GPU is Nvidia 1060 6GB.
The following 2 gaunlets are for the latest test net, but the way it evolved last day is AMAZING!
-
- Posts: 1796
- Joined: Thu Sep 18, 2008 10:24 pm
Re: Something goes wrong with lc0 since yesterday?
So the learning rate has been reduced and this speeds up learning? Hmm. One day I might understand it.crem wrote: ↑Tue Jul 17, 2018 5:31 pmThe learning rate was changed at network 10077 (as planned), so progress should indeed be much faster now.George Tsavdaris wrote: ↑Tue Jul 17, 2018 4:53 pmDespite what you said here, i used 40/2 for Leela also, in the following gaunlet since i find it unfair to offer Leela double time.Laskos wrote: ↑Wed Jul 11, 2018 7:53 pm 40/2' against single core is fine, but the most strictly correct simulation of CCRL conditions with lc0 on GPU would probably be to leave the lc0 at the same 40/4', as lc0 speed on GPU is very weakly dependent on CPU (2 threads), and for AB engines use 40/2' on a reasonable i7 core. Anyway, we get a picture, I use the same 40/2' bench for both lc0 and AB engines as you use. My GPU is Nvidia 1060 6GB.
The following 2 gaunlets are for the latest test net, but the way it evolved last day is AMAZING!
-
- Posts: 558
- Joined: Sat Mar 25, 2006 8:27 pm
Re: Something goes wrong with lc0 since yesterday?
Here's how I understand it:
With a high learning rate, you have a good chance of overshooting the nearest "best" values, but hopefully you are generally moving through any locally good but not absolutely best settings, and getting generally close to the absolutely best area.
With a lower learning rate, you will progress directly towards the nearest best settings, but there is a risk that this is only a local minimum, and not the absolute best set of settings. So it will generally show more steady improvement but is more likely to hit a dead-end.
I'm never going to find the highest mountain in America if I start in Iowa and limit my steps to the next highest hill in sight.
-
- Posts: 1796
- Joined: Thu Sep 18, 2008 10:24 pm
Re: Something goes wrong with lc0 since yesterday?
Thanks Robert, that makes sense.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Something goes wrong with lc0 since yesterday?
Good, I haven't followed it recently, thanks for the update.George Tsavdaris wrote: ↑Tue Jul 17, 2018 4:53 pmDespite what you said here, i used 40/2 for Leela also, in the following gaunlet since i find it unfair to offer Leela double time.Laskos wrote: ↑Wed Jul 11, 2018 7:53 pm 40/2' against single core is fine, but the most strictly correct simulation of CCRL conditions with lc0 on GPU would probably be to leave the lc0 at the same 40/4', as lc0 speed on GPU is very weakly dependent on CPU (2 threads), and for AB engines use 40/2' on a reasonable i7 core. Anyway, we get a picture, I use the same 40/2' bench for both lc0 and AB engines as you use. My GPU is Nvidia 1060 6GB.
The following 2 gaunlets are for the latest test net, but the way it evolved last day is AMAZING!
AB engines use 1 CPU i5-6500 3.4 GHz that i tested to be 2.1 times faster than CCRL's hardware so to have equivalence of the CCRL 40/4 Elo i used a 40/2 time control.
Leela is using Lc0 with GTX 1070 Ti with 40/2 time control also. I tested 2 testing nets, 10076 and the 1 day later(where they dropped LR) 10080.
Test10 10076 after 100 games had in the following gaunlet a 2970 ±45.4 CCRL 40/4 Elo performance and
Test10 10080 after 63 games so far has in the same gaunlet a 3243 ±75.1 CCRL 40/4 Elo performance!
Lc0 Test10 10076 after 100 games:Lc0 Test10 10080 after 63 games so far:Code: Select all
Program CCRL Elo Error(cl 95%) Games Score Lc0 Test10 10076 2969.7 ±45.4 100 (+26,=41,-33) 46.5 % vs. : games ( +, =, -), (%) : Diff, SD, CFS (%) Laser 1.5 : 20 ( 3, 8, 9), 35.0 : -127.3, 23.2, 0.0 Pedone 1.7 : 20 ( 5, 11, 4), 52.5 : -43.3, 23.2, 3.1 Wasp 3.0 : 20 ( 4, 11, 5), 47.5 : -17.3, 23.2, 22.7 Rybka 2.3.2a 64 bit : 20 ( 5, 4, 11), 35.0 : +6.7, 23.2, 61.3 Spike 1.4 : 20 ( 9, 7, 4), 62.5 : +51.7, 23.2, 98.7
Code: Select all
Program CCRL Elo Error(cl 95%) Games Score Lc0 Test10 10080 3242.9 ±75.1 63 (+44,=12,-7), 79.4 % vs. : games ( +, =, -), (%) : Diff, SD, CFS (%) Laser 1.5 : 13 ( 8, 4, 1), 76.9 : +145.9, 38.3, 100.0 Pedone 1.7 : 12 ( 7, 3, 2), 70.8 : +229.9, 38.3, 100.0 Wasp 3.0 : 13 ( 12, 1, 0), 96.2 : +255.9, 38.3, 100.0 Rybka 2.3.2a 64 bit : 13 ( 8, 2, 3), 69.2 : +279.9, 38.3, 100.0 Spike 1.4 : 12 ( 9, 2, 1), 83.3 : +324.9, 38.3, 100.0
ID10084 comes in my test at only about 80 Elo points behind ID495 net of the main branch, which seems the best overall as of now, or some 3250 or a bit more CCRL 40/4' Elo points on GTX 1060. On your 1070 Ti it is probably 3300 or so. Huge improvements on the testserver since LR changed!
-
- Posts: 1346
- Joined: Sat Apr 19, 2014 1:47 pm
Re: Something goes wrong with lc0 since yesterday?
Laskos wrote: ↑Tue Jul 17, 2018 11:01 pmGood, I haven't followed it recently, thanks for the update.George Tsavdaris wrote: ↑Tue Jul 17, 2018 4:53 pmDespite what you said here, i used 40/2 for Leela also, in the following gaunlet since i find it unfair to offer Leela double time.Laskos wrote: ↑Wed Jul 11, 2018 7:53 pm 40/2' against single core is fine, but the most strictly correct simulation of CCRL conditions with lc0 on GPU would probably be to leave the lc0 at the same 40/4', as lc0 speed on GPU is very weakly dependent on CPU (2 threads), and for AB engines use 40/2' on a reasonable i7 core. Anyway, we get a picture, I use the same 40/2' bench for both lc0 and AB engines as you use. My GPU is Nvidia 1060 6GB.
The following 2 gaunlets are for the latest test net, but the way it evolved last day is AMAZING!
AB engines use 1 CPU i5-6500 3.4 GHz that i tested to be 2.1 times faster than CCRL's hardware so to have equivalence of the CCRL 40/4 Elo i used a 40/2 time control.
Leela is using Lc0 with GTX 1070 Ti with 40/2 time control also. I tested 2 testing nets, 10076 and the 1 day later(where they dropped LR) 10080.
Test10 10076 after 100 games had in the following gaunlet a 2970 ±45.4 CCRL 40/4 Elo performance and
Test10 10080 after 63 games so far has in the same gaunlet a 3243 ±75.1 CCRL 40/4 Elo performance!
Lc0 Test10 10076 after 100 games:Lc0 Test10 10080 after 63 games so far:Code: Select all
Program CCRL Elo Error(cl 95%) Games Score Lc0 Test10 10076 2969.7 ±45.4 100 (+26,=41,-33) 46.5 % vs. : games ( +, =, -), (%) : Diff, SD, CFS (%) Laser 1.5 : 20 ( 3, 8, 9), 35.0 : -127.3, 23.2, 0.0 Pedone 1.7 : 20 ( 5, 11, 4), 52.5 : -43.3, 23.2, 3.1 Wasp 3.0 : 20 ( 4, 11, 5), 47.5 : -17.3, 23.2, 22.7 Rybka 2.3.2a 64 bit : 20 ( 5, 4, 11), 35.0 : +6.7, 23.2, 61.3 Spike 1.4 : 20 ( 9, 7, 4), 62.5 : +51.7, 23.2, 98.7
Code: Select all
Program CCRL Elo Error(cl 95%) Games Score Lc0 Test10 10080 3242.9 ±75.1 63 (+44,=12,-7), 79.4 % vs. : games ( +, =, -), (%) : Diff, SD, CFS (%) Laser 1.5 : 13 ( 8, 4, 1), 76.9 : +145.9, 38.3, 100.0 Pedone 1.7 : 12 ( 7, 3, 2), 70.8 : +229.9, 38.3, 100.0 Wasp 3.0 : 13 ( 12, 1, 0), 96.2 : +255.9, 38.3, 100.0 Rybka 2.3.2a 64 bit : 13 ( 8, 2, 3), 69.2 : +279.9, 38.3, 100.0 Spike 1.4 : 12 ( 9, 2, 1), 83.3 : +324.9, 38.3, 100.0
ID10084 comes in my test at only about 80 Elo points behind ID495 net of the main branch, which seems the best overall as of now, or some 3250 or a bit more CCRL 40/4' Elo points on GTX 1060. On your 1070 Ti it is probably 3300 or so. Huge improvements on the testserver since LR changed!
That's good news, that's mean at this rate we might see a new strongest id in few days.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: Something goes wrong with lc0 since yesterday?
These days I probed lc0 ID495 (main branch) for behavior on certain openings against Gaviota 1.0 regular engine (on two cores for randomization). I played overall several thousands ultra-fast games from Noomen topical opening suite.
Here I present over- and under- performance (over- with green, under- with red) in Elo compared to general performance for most common openings:
Ruy Lopez: -46
French: -43
Sicilian: -46
Queen's Pawn: +75
King's Indian: -31
Nimzo Indian: +64
Reti: +31
The 2SD error margins are about 25 Elo points.
Here I present over- and under- performance (over- with green, under- with red) in Elo compared to general performance for most common openings:
Ruy Lopez: -46
French: -43
Sicilian: -46
Queen's Pawn: +75
King's Indian: -31
Nimzo Indian: +64
Reti: +31
The 2SD error margins are about 25 Elo points.