Okay, this is my first step in the mysterious nothing world. Got the CPU version to run together with the W323 network as used by CCRL. Did not get the GPU version to work, not even after a driver update, perhaps the NVIDIA Quadro FX 1800 is too old, I am dunno. So I will have to do with the slow CPU version for the moment.
I ran the STS test with MEA and compared with the results of (good old) Fruit 2.1
http://rebel13.nl/lczero.html
Not bad at all for a start, will run it again when a new approved network version becomes available.
Just inform me.
LcZero and STS
Moderators: hgm, Rebel, chrisw
-
- Posts: 6995
- Joined: Thu Aug 18, 2011 12:04 pm
LcZero and STS
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 1142
- Joined: Thu Dec 28, 2017 4:06 pm
- Location: Argentina
Re: LcZero and STS
Interesting results!
Would you run net 390 and/or 395 to see if there's any improvement? They are the strongest networks (in actual game play) so far.
Would you run net 390 and/or 395 to see if there's any improvement? They are the strongest networks (in actual game play) so far.
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls
-
- Posts: 593
- Joined: Sat Aug 20, 2011 9:43 am
Re: LcZero and STS
The FX 1800 isn't supported by tensorflow, which requires a card scoring 3.5 or higher on this page:
https://developer.nvidia.com/cuda-gpus#collapse2
Actually, 3.0 is possible with a custom compile of tensorflow.
Am not certain this is why that card isn't working.
(reference: https://www.tensorflow.org/install/)
-
- Posts: 4190
- Joined: Wed Nov 25, 2009 1:47 am
Re: LcZero and STS
3.0 capability is enough. Problem is Quadro FX 1800 is not even listed as CUDA capable, i.e. NVIDIA simply doesn't provide CUDA support for it.Jesse Gersenson wrote: ↑Thu Jun 14, 2018 2:49 pmThe FX 1800 isn't supported by tensorflow, which requires a card scoring 3.5 or higher on this page:
https://developer.nvidia.com/cuda-gpus#collapse2
Actually, 3.0 is possible with a custom compile of tensorflow.
Am not certain this is why that card isn't working.
(reference: https://www.tensorflow.org/install/)
The reason is probably that FX 1800 is way too poor since it was made 10 years ago in a prehistoric technology. So even if somehow managed to run LC0 I wouldn't expect more than 50nps at best, more probably something around 20-30.
P.S. It has nothing to do with tensorflow. TF is Google's framework. CUDA is NVIDIA's propitiatory library. LC0 is running on CUDA alone (TF framework was dropped quite some time ago).
-
- Posts: 1563
- Joined: Thu Jul 16, 2009 10:47 am
- Location: Almere, The Netherlands
Re: LcZero and STS
Out of curiosity I ran STS with the latest LC0 on my GTX-1080ti, 1 sec. per move only because 10 secs. takes me too long.
I have the impression that LC0 doesn't make full use of movetime 1000 and that it uses less, more like 700 msec. or so, maybe this has something to do with a fixed number of playouts? Certainly something that can be improved.
In 2013 I used STS to tune my own engine, I didn't change anything in the evaluation function since, with Nightmare the result is as follows:
According to STS, LC0 with network 395 on a 1080ti is roughly comparable to my own engine on 10x 4GHz. threads, which is remarkable.
I really don't know what the CCRL rating of Nightmare is because I never cared to have it tested, according to my own tests it is roughly 3000 Elo 40/40.
Code: Select all
A. Processor
Brand : Intel(R) Core(TM) i7-5960X CPU @ 4.00GHz
Arch : X86_64
Count : 16
B. Engine settings
Threads : 2
Hash (mb) : 128
Time(s)/pos : 1.0
C. Test set
Filename : STS.epd
NumPos : 1500
D. Results
Engine : Rating Top1 MaxTop1 Top1Rate Score MaxScore ScoreRate
LC0-090618-395 : 2884 1081 1500 0.721 12324 15000 0.822
In 2013 I used STS to tune my own engine, I didn't change anything in the evaluation function since, with Nightmare the result is as follows:
Code: Select all
A. Processor
Brand : Intel(R) Core(TM) i7-6950X CPU @ 4.00GHz
Arch : X86_64
Count : 20
B. Engine settings
Threads : 10
Hash (mb) : 128
Time(s)/pos : 1.0
C. Test set
Filename : STS.epd
NumPos : 1500
D. Results
Engine : Rating Top1 MaxTop1 Top1Rate Score MaxScore ScoreRate
Nightmare v7.80 : 2884 1078 1500 0.719 12213 15000 0.814
I really don't know what the CCRL rating of Nightmare is because I never cared to have it tested, according to my own tests it is roughly 3000 Elo 40/40.
-
- Posts: 1142
- Joined: Thu Dec 28, 2017 4:06 pm
- Location: Argentina
Re: LcZero and STS
Is your engine private or available somewhere? I'd love to run it through my leagues, if it's ~3000 elo it's good enough to survive and even climb up a couple of leagues.Joost Buijs wrote: ↑Thu Jun 14, 2018 3:12 pm Out of curiosity I ran STS with the latest LC0 on my GTX-1080ti, 1 sec. per move only because 10 secs. takes me too long.
I have the impression that LC0 doesn't make full use of movetime 1000 and that it uses less, more like 700 msec. or so, maybe this has something to do with a fixed number of playouts? Certainly something that can be improved.Code: Select all
A. Processor Brand : Intel(R) Core(TM) i7-5960X CPU @ 4.00GHz Arch : X86_64 Count : 16 B. Engine settings Threads : 2 Hash (mb) : 128 Time(s)/pos : 1.0 C. Test set Filename : STS.epd NumPos : 1500 D. Results Engine : Rating Top1 MaxTop1 Top1Rate Score MaxScore ScoreRate LC0-090618-395 : 2884 1081 1500 0.721 12324 15000 0.822
In 2013 I used STS to tune my own engine, I didn't change anything in the evaluation function since, with Nightmare the result is as follows:
According to STS, LC0 with network 395 on a 1080ti is roughly comparable to my own engine on 10x 4GHz. threads, which is remarkable.Code: Select all
A. Processor Brand : Intel(R) Core(TM) i7-6950X CPU @ 4.00GHz Arch : X86_64 Count : 20 B. Engine settings Threads : 10 Hash (mb) : 128 Time(s)/pos : 1.0 C. Test set Filename : STS.epd NumPos : 1500 D. Results Engine : Rating Top1 MaxTop1 Top1Rate Score MaxScore ScoreRate Nightmare v7.80 : 2884 1078 1500 0.719 12213 15000 0.814
I really don't know what the CCRL rating of Nightmare is because I never cared to have it tested, according to my own tests it is roughly 3000 Elo 40/40.
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls
-
- Posts: 1563
- Joined: Thu Jul 16, 2009 10:47 am
- Location: Almere, The Netherlands
Re: LcZero and STS
It is a private engine, mainly used to run in the monthly HGM tournament and the ACCA tournaments. The problem is that most things are hard coded and that I never took the time to implement parameter setup via UCI, for me no problem, but for others it is unusable.CMCanavessi wrote: ↑Thu Jun 14, 2018 3:20 pm Is your engine private or available somewhere? I'd love to run it through my leagues, if it's ~3000 elo it's good enough to survive and even climb up a couple of leagues.
A couple of years ago I started working on a new version from scratch, the past 2 years I had other obligations, but since a few months I'm working on the engine again. I still have to do the positional evaluation, but for the rest is everything finished including the UCI protocol. When everything is finished and I'm satisfied with it, I will probably release it.
-
- Posts: 1142
- Joined: Thu Dec 28, 2017 4:06 pm
- Location: Argentina
Re: LcZero and STS
Awesome!Joost Buijs wrote: ↑Thu Jun 14, 2018 3:54 pmIt is a private engine, mainly used to run in the monthly HGM tournament and the ACCA tournaments. The problem is that most things are hard coded and that I never took the time to implement parameter setup via UCI, for me no problem, but for others it is unusable.CMCanavessi wrote: ↑Thu Jun 14, 2018 3:20 pm Is your engine private or available somewhere? I'd love to run it through my leagues, if it's ~3000 elo it's good enough to survive and even climb up a couple of leagues.
A couple of years ago I started working on a new version from scratch, the past 2 years I had other obligations, but since a few months I'm working on the engine again. I still have to do the positional evaluation, but for the rest is everything finished including the UCI protocol. When everything is finished and I'm satisfied with it, I will probably release it.
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls
-
- Posts: 6995
- Joined: Thu Aug 18, 2011 12:04 pm
Re: LcZero and STS
390 looks obviously better, see:CMCanavessi wrote: ↑Thu Jun 14, 2018 1:06 pm Interesting results!
Would you run net 390 and/or 395 to see if there's any improvement? They are the strongest networks (in actual game play) so far.
http://rebel13.nl/zero1.html
http://rebel13.nl/zero2.html
http://rebel13.nl/zero3.html
Running 395 now.
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 1142
- Joined: Thu Dec 28, 2017 4:06 pm
- Location: Argentina
Re: LcZero and STS
What do you use to process all the STS epds and get a score? I may try the same with lc0.
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls