Best Nets for Lc0 Page

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Laskos
Posts: 9834
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Best Nets for Lc0 Page

Post by Laskos » Sat Dec 28, 2019 2:33 pm

corres wrote:
Sat Dec 28, 2019 2:21 pm
Laskos wrote:
Sat Dec 28, 2019 2:12 pm
corres wrote:
Sat Dec 28, 2019 1:58 pm
A note
I wrote about jhorthos T40B.4-160 net.
This was stronger on my system than T40B.4-200.
I guess any difference between them is within 5 Elo points. I tested them in 2000 games at fixed 1000 nodes, they are that close, with 4-200 coming a couple of points better than 4-160, but well within error margins.
Tests with fixed nodes are not comparable to tests with fixed time.
As I noted earlier the move/time used by you is too short and disturb the results.
NN engines are non-effective in those short time control.
They need more time to "warm-up".
The nets are almost identical, having less than 0.5% difference in speed. So fixed nodes will differ by no more than 3-4 Elo points from fixed time. Anyway, I will not insist on your bickering here about 5 Elo points potential difference between -160 and -200 nets. For now I am testing Sergio's 384x30b BigNet against the best JHorthos 256x20b net in 2 games from an unbalanced opening position, side and reversed. Time control is similar to TCEC TC, 2 hours + 30 seconds. The opening is unbalanced, because with balanced openings the Draw rate is above 90% at this TC. Maybe I will get an unbalanced result in 2 LTC games side and reversed from this unbalanced opening. Not much in 2 LTC games, but some even vague idea of what's up with Sergio's BigNet to very long time controls on RTX GPU. Now this is the picture, BigNet is White.

Image

corres
Posts: 2023
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Best Nets for Lc0 Page

Post by corres » Sat Dec 28, 2019 3:38 pm

Laskos wrote:
Sat Dec 28, 2019 2:33 pm
...
For now I am testing Sergio's 384x30b BigNet against the best JHorthos 256x20b net in 2 games from an unbalanced opening position, side and reversed. Time control is similar to TCEC TC, 2 hours + 30 seconds. The opening is unbalanced, because with balanced openings the Draw rate is above 90% at this TC. Maybe I will get an unbalanced result in 2 LTC games side and reversed from this unbalanced opening. Not much in 2 LTC games, but some even vague idea of what's up with Sergio's BigNet to very long time controls on RTX GPU.
...
I am curious to see the result.
What are the exact number of Sergio`s 384x30b BigNet and jHorthos 256x20b net?

User avatar
Laskos
Posts: 9834
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Best Nets for Lc0 Page

Post by Laskos » Sat Dec 28, 2019 3:44 pm

corres wrote:
Sat Dec 28, 2019 3:38 pm
Laskos wrote:
Sat Dec 28, 2019 2:33 pm
...
For now I am testing Sergio's 384x30b BigNet against the best JHorthos 256x20b net in 2 games from an unbalanced opening position, side and reversed. Time control is similar to TCEC TC, 2 hours + 30 seconds. The opening is unbalanced, because with balanced openings the Draw rate is above 90% at this TC. Maybe I will get an unbalanced result in 2 LTC games side and reversed from this unbalanced opening. Not much in 2 LTC games, but some even vague idea of what's up with Sergio's BigNet to very long time controls on RTX GPU.
...
I am curious to see the result.
What are the exact number of Sergio`s 384x30b BigNet and jHorthos 256x20b net?
1573 and 160.

User avatar
Laskos
Posts: 9834
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Best Nets for Lc0 Page

Post by Laskos » Sun Dec 29, 2019 7:37 am

Laskos wrote:
Sat Dec 28, 2019 3:44 pm
corres wrote:
Sat Dec 28, 2019 3:38 pm
Laskos wrote:
Sat Dec 28, 2019 2:33 pm
...
For now I am testing Sergio's 384x30b BigNet against the best JHorthos 256x20b net in 2 games from an unbalanced opening position, side and reversed. Time control is similar to TCEC TC, 2 hours + 30 seconds. The opening is unbalanced, because with balanced openings the Draw rate is above 90% at this TC. Maybe I will get an unbalanced result in 2 LTC games side and reversed from this unbalanced opening. Not much in 2 LTC games, but some even vague idea of what's up with Sergio's BigNet to very long time controls on RTX GPU.
...
I am curious to see the result.
What are the exact number of Sergio`s 384x30b BigNet and jHorthos 256x20b net?
1573 and 160.
The first two games ended both in draw, although Sergio BigNet almost won the first one, but stumbled upon a locked position with above +2.00 advantage for a long time. I started another two with a bit more unbalanced position.


pohl4711
Posts: 1162
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: Best Nets for Lc0 Page

Post by pohl4711 » Sun Dec 29, 2019 11:48 am

The best Net is Leelenstein 12.2. No doubt about that, anymore, after finishing my 3000 game testrun:

https://www.sp-cc.de/lc0-testing.htm

Congrats to Josh!
https://www.patreon.com/jjosh/posts

(you have to be a Patreon of Josh (3$ per month (plus taxes)), to download LS 12.2, so it is a commercial net. But it is +70 Elo stronger, than the other commercial and non-zero Net (Fat Fritz) in my testings.)

User avatar
Laskos
Posts: 9834
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Best Nets for Lc0 Page

Post by Laskos » Sun Dec 29, 2019 1:00 pm

pohl4711 wrote:
Sun Dec 29, 2019 11:48 am
The best Net is Leelenstein 12.2. No doubt about that, anymore, after finishing my 3000 game testrun:

https://www.sp-cc.de/lc0-testing.htm

Congrats to Josh!
https://www.patreon.com/jjosh/posts

(you have to be a Patreon of Josh (3$ per month (plus taxes)), to download LS 12.2, so it is a commercial net. But it is +70 Elo stronger, than the other commercial and non-zero Net (Fat Fritz) in my testings.)
I see it 22 Elo points better than Jhorthos B4-160, or a strength very similar to that of Sergio 1400 net. Will soon check Sergio 1425 (or maybe a newer one will appear). You seem to use v23 engine for LS 12 nets and v22 with other nets, hope it doesn't matter. Also, BigNets improve with time control, so it seems hard for me to say what happens to longish time controls (RTX GPU).

shrapnel
Posts: 1261
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: Best Nets for Lc0 Page

Post by shrapnel » Sun Dec 29, 2019 5:33 pm

Hai wrote:
Sat Dec 28, 2019 9:06 am
Bigger nets are stronger, when they have trained enough. And of course the mb get bigger and bigger.
That's completely normal.
The Sergio nets should be already stronger on 1 rtx gpu then other nets, so you can use 2 rtx gpus for even bigger nets.
Sergio nets need an upgrade to a bigger size of 40x512, maybe someone can do this.
7-men tablebases rescaling should be done too.

Leelenstein would have the strongest nets when upgrading to a size of 40x512.

The LC0 project should also do 7-men tablebases rescaling and train from chess960 start positions.
Lc0 on my Machine is already using 2x2080 Tis, see my Signature.
If Sergio could release 40x512 NNs, that would be simply AWESOME and would simply blow away the Competition.
I think such large NNs would finally start utilizing the FULL POWER of two 2080 Tis
I'm not too impressed by either the much-vaunted Leelenstein or Jhorthos NNs, which frankly didn't do much while being used by lc0 on my Machine.
By the way, Master Sergio just released his latest T60 NN today, the 384x30-t60-2-1530.pb.
Enough typing, I'm off to try it on some Unfortunate on Playchess :D
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

User avatar
Laskos
Posts: 9834
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Best Nets for Lc0 Page

Post by Laskos » Sun Dec 29, 2019 5:39 pm

Laskos wrote:
Sun Dec 29, 2019 7:37 am
Laskos wrote:
Sat Dec 28, 2019 3:44 pm
corres wrote:
Sat Dec 28, 2019 3:38 pm
Laskos wrote:
Sat Dec 28, 2019 2:33 pm
...
For now I am testing Sergio's 384x30b BigNet against the best JHorthos 256x20b net in 2 games from an unbalanced opening position, side and reversed. Time control is similar to TCEC TC, 2 hours + 30 seconds. The opening is unbalanced, because with balanced openings the Draw rate is above 90% at this TC. Maybe I will get an unbalanced result in 2 LTC games side and reversed from this unbalanced opening. Not much in 2 LTC games, but some even vague idea of what's up with Sergio's BigNet to very long time controls on RTX GPU.
...
I am curious to see the result.
What are the exact number of Sergio`s 384x30b BigNet and jHorthos 256x20b net?
1573 and 160.
The first two games ended both in draw, although Sergio BigNet almost won the first one, but stumbled upon a locked position with above +2.00 advantage for a long time. I started another two with a bit more unbalanced position.

From the more unbalanced opening, both won as White, and again Sergio net more easily. In 4 games, 2:2 score, one win each and two draws. Cannot really separate them.


User avatar
Laskos
Posts: 9834
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Best Nets for Lc0 Page

Post by Laskos » Mon Dec 30, 2019 1:29 am

Laskos wrote:
Sun Dec 29, 2019 1:00 pm
pohl4711 wrote:
Sun Dec 29, 2019 11:48 am
The best Net is Leelenstein 12.2. No doubt about that, anymore, after finishing my 3000 game testrun:

https://www.sp-cc.de/lc0-testing.htm

Congrats to Josh!
https://www.patreon.com/jjosh/posts

(you have to be a Patreon of Josh (3$ per month (plus taxes)), to download LS 12.2, so it is a commercial net. But it is +70 Elo stronger, than the other commercial and non-zero Net (Fat Fritz) in my testings.)
I see it 22 Elo points better than Jhorthos B4-160, or a strength very similar to that of Sergio 1400 net. Will soon check Sergio 1425 (or maybe a newer one will appear). You seem to use v23 engine for LS 12 nets and v22 with other nets, hope it doesn't matter. Also, BigNets improve with time control, so it seems hard for me to say what happens to longish time controls (RTX GPU).
And the Sergio's 1467 net seems even better than 1400:
800 games at 30'' + 0.3''

Code: Select all

Score of lc0_T40_B4_160 vs lc0_256x20_SV1467: 166 - 224 - 410  [0.464] 800
Elo difference: -25.23 +/- 16.80
Finished match
The games were played from unbalanced openings to reduce the draw rate, and the real error margin (1 standard deviation) using pentanomial variance (side and reversed games) is about 5 Elo points. With the commercial LS 12.2 net, probably the strongest nets around for RTX GPU from Blitz to at least Rapid, if not real LTC (where the Sergio's BigNet might be slightly better).

shrapnel
Posts: 1261
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: Best Nets for Lc0 Page

Post by shrapnel » Tue Dec 31, 2019 4:28 pm

Laskos wrote:
Mon Dec 30, 2019 1:29 am
And the Sergio's 1467 net seems even better than 1400:
So you can just imagine how strong the latest Sergio T60 net 384x30-t60-2-1538.pb is.
Really, it is an insult to Sergio to compare the weak Leelenstein 12.2 andJhorthos NNs to his awesome NNs.
I know this for a FACT, no matter what anyone's 'Tests' say because I wasted 3$ getting hold of the Leelenstein 12.2 NN and tried it on Playchess and could get nothing better than repeated Draws against even players with weak hardware.
Maybe some Testers, whom I will not name, are being paid a Commission to boost Sales of Leelenstein.

Free Sergio NNs are getting stronger with each Release.

ONLY Sergio NNs give Lc0 on my Machine positive results against Stockfish and also against the COMMERCIAL Fat Fritz, as the following 2 games demonstrate....
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

Post Reply