Best Nets for Lc0 Page

[shrapnel]

Just to emphasize the Point, watch FREE Lc0 using FREE Sergio NN THRASH Commercial Fat Fritz with BLACK....
[pgn][Event "Rated game, 7m+3s"]
[Site "Engine Room"]
[Date "2019.12.31"]
[Round "?"]
[White "Pestif"]
[Black "Shrapnel, Lc0 v0.23.1+git.683"]
[Result "0-1"]
[ECO "B86"]
[WhiteElo "2453"]
[BlackElo "2524"]
[Annotator "-1.07;-0.02"]
[PlyCount "146"]
[EventDate "2019.12.31"]
[SourceTitle "playchess.com"]
[TimeControl "420+3"]

{Fat Fritz (in lc0) cuda (2 cores): 17.6 plies; 68kN/s AMD Ryzen 7 2700X
Eight-Core Processor 3700MHz, (8 cores, 16 threads), Perfect2019.ctg,
32768 MB} 1. e4 {B 0} c5 {B 0} 2. Nf3 {B 0} d6 {B 0} 3. d4 {B 0} cxd4 {B 0} 4.
Nxd4 {B 0} Nf6 {B 0} 5. Nc3 {B 0} a6 {B 0} 6. Bc4 {B 0} e6 {B 0} 7. Bb3 {B 0}
Nbd7 {B 0} 8. Bg5 {B 0} Nc5 {-0.02/12 11} 9. f4 {B 0 (0-0)} Be7 {-0.13/13 9}
10. Qf3 {B 0} Qc7 {-0.15/14 9} 11. O-O-O {B 0} b5 {-0.22/15 12} 12. Bxf6 {
B 0 (Rhe1)} Bxf6 {-1.14/14 13} 13. g4 {-1.07/14 8} b4 {-1.13/17 9} 14. Na4 {
-1.12/18 3} O-O {-1.18/17 4} 15. g5 {-1.16/19 5} Be7 {-1.20/16 5} 16. Rhg1 {
-1.18/20 8 (Rhe1)} Bd7 {-1.29/17 17} 17. f5 {-1.14/21 0} Bxa4 {-1.33/18 6} 18.
Bxa4 {-1.14/21 0} Nxa4 {-1.30/18 6} 19. f6 {-1.11/18 0} Rac8 {-1.38/16 13} 20.
fxe7 {-1.12/20 5} Qxe7 {-1.35/15 7} 21. h4 {-1.10/18 11} Qc7 {-1.35/15 1 (Nc5)}
22. Rg2 {-1.04/16 42 (Qg2)} Qa5 {-1.45/14 25} 23. Kb1 {-1.07/21 11 (h5)} Rc4 {
-1.65/13 29} 24. Qb3 {-1.19/19 4 (Qd3)} Rfc8 {-1.46/17 21 (d5)} 25. h5 {
-1.20/20 39} Rc3 {-1.60/22 0} 26. bxc3 {-1.23/27 18} Rxc3 {-1.68/22 1} 27. Ne2
{-1.22/26 8} Rxb3+ {-1.67/19 0} 28. axb3 {-1.17/24 9} Nc5 {-1.69/17 0} 29. Rg4
{-1.16/20 1} g6 {-1.75/18 14 (Qc7)} 30. hxg6 {-1.70/18 79 (Rxd6)} hxg6 {
-1.87/18 1} 31. Rxd6 {-1.69/15 0} Nb7 {-1.87/18 10} 32. Rd7 {-1.80/18 31 (Rd3)}
Qb5 {-1.97/21 43} 33. Rd3 {-1.86/19 0} Nc5 {-1.97/24 11 (Qc5)} 34. Rd8+ {
-1.70/16 14} Kg7 {-1.97/23 1} 35. Nc1 {-1.84/18 27 (Rh4)} Qb6 {-2.22/14 36} 36.
Rd1 {-1.89/18 0} e5 {-2.25/16 12 (Qb7)} 37. Rd5 {-2.02/15 22 (Nd3)} Qe6 {
-2.46/14 25} 38. Rxc5 {-2.14/17 0} Qxg4 {-2.45/14 8} 39. Rxe5 {-2.15/16 0} Qg1
{-2.51/13 8 (Qd1)} 40. Rd5 {-2.25/14 51 (Ra5)} Qe3 {-2.64/13 9} 41. Nd3 {
-2.31/15 10} Qxe4 {-2.69/12 10} 42. Re5 {-2.34/13 8} Qd4 {-2.82/11 22} 43. Rc5
{-2.38/13 2} Qd6 {-2.89/11 10} 44. Ra5 {-2.42/13 12 (Rc4)} Qb6 {-3.14/10 20}
45. Rc5 {-2.45/13 0} f5 {-3.26/10 20 (a5)} 46. gxf6+ {-2.42/13 17} Kxf6 {
-3.23/10 0} 47. Rc4 {-2.44/13 9 (Nf2)} a5 {-3.51/9 15} 48. Rf4+ {-2.46/14 11
(Rc5)} Kg5 {-3.43/9 33 (Kg7)} 49. Rc4 {-2.20/12 19 (Re4)} Kh5 {-3.77/9 4 (Qg1+)
} 50. Nf4+ {-2.15/14 12 (Rc5+)} Kh6 {-4.03/9 6} 51. Nxg6 {-2.17/15 1 (Nd3)}
Kxg6 {-5.29/7 11 (Qg1+)} 52. Rg4+ {-1.36/12 14 (Kb2)} Kf5 {-4.77/7 14 (Kf7)}
53. Rg3 {-1.49/18 2 (Rh4)} Ke4 {-4.69/7 8 (Qf2)} 54. Rd3 {-1.65/13 15 (Rg5)}
Qb5 {-4.84/7 6 (Qf6)} 55. Rd2 {-1.47/11 11 (Rg3)} Ke3 {-5.30/7 6 (Kf3)} 56. Kc1
{-2.13/12 18 (Rd6)} Qe5 {-7.17/7 4 (Qg5)} 57. Rd3+ {-2.54/12 3} Ke2 {-6.16/7 4
(Ke4)} 58. Rd2+ {-2.18/14 5 (Kb1)} Ke1 {-6.40/8 7 (Ke3)} 59. Rd1+ {-2.09/16 5}
Kf2 {-5.14/8 3 (Ke2)} 60. Rd2+ {-2.14/15 5 (Kb1)} Kf1 {-4.88/9 9 (Ke3)} 61.
Rd1+ {-2.09/16 5} Ke2 {-4.41/9 5 (Kf2)} 62. Rd2+ {-2.48/15 5} Ke3 {-4.02/9 4}
63. Rd3+ {-2.74/14 0} Ke4 {-3.93/8 1} 64. Rh3 {-2.85/13 2 (Kb1)} a4 {-7.99/6 5
(Qa1+)} 65. bxa4 {-1.54/13 7 (Kd2)} Qa1+ {-128.00/2 3} 66. Kd2 {-1.57/13 0}
Qxa4 {-128.00/2 6 (Kd5)} 67. Rd3 {-1.21/13 8} Qa7 {-6/1 0 (Qa1)} 68. Kc1 {
-1.09/12 7} Qa1+ {-5/1 0 (Qa2)} 69. Kd2 {-1.10/14 0} Qg1 {-4/1 0 (Qb1)} 70. Rb3
{-0.97/13 6} Qg5+ {-3/1 0 (Qh1)} 71. Kd1 {-128.00/2 6} Qd5+ {-2/1 0 (Qh6)} 72.
Kc1 {-128.00/2 6} Qxb3 {-2/1 0 (Qd4)} 73. cxb3 {-1/0 0} Kd3 {-4/1 0 Pestif,Fat
Fritz (in lc0) c resigns} 0-1[/pgn]

[Robert Flesher]

Sergio 1705 net is out. Strong!!

[Laskos]

Sergio 1705 net is out. Strong!!

I would say that Sergio's 256x20b net 1541 net is the strongest around for my OC-ed RTX 2070 GPU on any reasonable TC, even LTC.

Here is what I get with the BigNets:

Code: Select all

Games at 30'' + 0.3''

SV 384x30b
Rank Name                          Elo     +/-   Games   Score   Draws
   1 1573                           36      19     800   55.1%   38.3%
   2 1538                           26      19     800   53.8%   38.3%
   3 1530                          -62      20     800   41.1%   35.0%
Finished match

And:

Code: Select all

Games at 4 min + 2.4 s

Score of Sergio_384x30_1573 vs Sergio_256x20_1467: 24 - 59 - 117 [0.412]
Elo difference: -61.4 +/- 30.8, LOS: 0.0 %, DrawRatio: 58.5 %

200 of 200 games finished.

In the second one, the time control is already real Blitz. I don't think that the SV BigNet scales so well to catch-up with the 256x20b SV net even to LTC on my GPU, 60 Elo points are too much.

And today came the new Sergio_256x20_1541 net which again improves sensibly on 1467:

Code: Select all

1000 nodes per move:

Score of 1541 vs 1467: 611 - 541 - 848  [0.517] 2000
Elo difference: 12.17 +/- 11.55
Finished match

The real error margins are a factor of 1.4 smaller than the shown ones because a I used unbalanced positions side-and-reversed.

So, all in all, I would take for my RTX GPU Sergio's SV_256x20_1541 at any time control (aside longish analysis, where BigNet 1705 is probably the best).

[Laskos]

Sergio 1705 net is out. Strong!!

I would say that Sergio's 256x20b net 1541 net is the strongest around for my OC-ed RTX 2070 GPU on any reasonable TC, even LTC.

Here is what I get with the BigNets:

Code: Select all

Games at 30'' + 0.3''

SV 384x30b
Rank Name                          Elo     +/-   Games   Score   Draws
   1 1573                           36      19     800   55.1%   38.3%
   2 1538                           26      19     800   53.8%   38.3%
   3 1530                          -62      20     800   41.1%   35.0%
Finished match

And:

Code: Select all

Games at 4 min + 2.4 s

Score of Sergio_384x30_1573 vs Sergio_256x20_1467: 24 - 59 - 117 [0.412]
Elo difference: -61.4 +/- 30.8, LOS: 0.0 %, DrawRatio: 58.5 %

200 of 200 games finished.

In the second one, the time control is already real Blitz. I don't think that the SV BigNet scales so well to catch-up with the 256x20b SV net even to LTC on my GPU, 60 Elo points are too much.

And today came the new Sergio_256x20_1541 net which again improves sensibly on 1467:

Code: Select all

1000 nodes per move:

Score of 1541 vs 1467: 611 - 541 - 848  [0.517] 2000
Elo difference: 12.17 +/- 11.55
Finished match

The real error margins are a factor of 1.4 smaller than the shown ones because a I used unbalanced positions side-and-reversed.

So, all in all, I would take for my RTX GPU Sergio's SV_256x20_1541 at any time control (aside longish analysis, where BigNet 1705 is probably the best).

Just finished the test of SV 1541 against vanilla T40 net 42850, which ruled supreme for some time in the past.

Code: Select all

Games in 30'' + 0.3''

Score of SV_256x20_1541 vs 42850: 257 - 166 - 377  [0.557] 800
Elo difference: 39.69 +/- 17.51
Finished match

Impressive and consistent. JHorthos best 256x20 nets are some 20 Elo points above vanilla 42850, and Sergio's SV 256x20_1541 is another impressive 30 Elo points above the best JHorthos nets. These improvements do not add orthogonally, so one would expect a total of maybe 40-43 Elo points improvement over the vanilla, consistent with the result. Sergio is doing some amazing stuff there, because his training games are those of the same vanilla T40 run. Maybe he can reach even a total of 50 Elo improvement over the vanilla T40, and I have never expected this.

[MikeB]

The Sergio nets are very very good, On an RTX 2060 Super , the latest Sergio net is at least about equal ( maybe better) to the latest Komodo running on the Threadripper with all 64 threads.

[mwyoung]

"There is no single "best net" for Leela but there a few worth recommending for various purposes. The most important consideration in choosing a net is picking the right size for your hardware and time controls (TC). In general, if you have a weak GPU (or no GPU at all) and you want to only spend milliseconds per move, then you want a smaller net that evaluates positions more quickly, i.e. higher NPS (nodes per second). On the other hand, if you have an RTX card (or several) and you want to run an analysis on a single move for hours at a time, then the quality of the evaluation is more important than the speed and a larger (but slower) net is probably going to work best."

https://github.com/LeelaChessZero/lc0/w ... ts-for-Lc0

The one point you did not mention is the configuration of the Lc0 net in the GUI. The default settings are awful and more so for the bigger nets. In my current testing with the T61xxx nets. With a few simple tweaks this net series is testing 50 to 60 elo better then default settings. And doing this at fast time controls against the Best A/B engines.

[shrapnel]

With a few simple tweaks this net series is testing 50 to 60 elo better then default settings.

Share your Settings please.

[corres]

...
The default settings are awful and more so for the bigger nets. In my current testing with the T61xxx nets. With a few simple tweaks this net series is testing 50 to 60 elo better then default settings. And doing this at fast time controls against the Best A/B engines.

There is only one problem: Using very short time control it is very far from the practical games. A test is useful only if the circumstances of the test are the same as in the case of normal games. It is a more important viewpoint than getting a mathematically exact but practically useless Elo. If somebody stick to gain relative exact Elo one should not regret the time for getting a practically useful result.
It is also an important viewpoint the connection between power of GPU and the dimension of net.
If somebody would like to get the real knowledge what is in a big net that one need a GPU with higher power.
Obviously as the dimension of a net is bigger as it needs more time and/or more faster GPU to scan its content.

[mwyoung]

...
The default settings are awful and more so for the bigger nets. In my current testing with the T61xxx nets. With a few simple tweaks this net series is testing 50 to 60 elo better then default settings. And doing this at fast time controls against the Best A/B engines.

There is only one problem: Using very short time control it is very far from the practical games. A test is useful only if the circumstances of the test are the same as in the case of normal games. It is a more important viewpoint than getting a mathematically exact but practically useless Elo. If somebody stick to gain relative exact Elo one should not regret the time for getting a practically useful result.
It is also an important viewpoint the connection between power of GPU and the dimension of net.
If somebody would like to get the real knowledge what is in a big net that one need a GPU with higher power.
Obviously as the dimension of a net is bigger as it needs more time and/or more faster GPU to scan its content.

I agree, and that is why I test realistic time controls that would be useful. From 5s a move to 5m a move.

I will say a higher power gpu is nice. But if you are running a bad configuration you are wasting a lot of that horse power.

[Robert Flesher]

With a few simple tweaks this net series is testing 50 to 60 elo better then default settings.

Share your Settings please.

+1