SF+NNUE 16 cores aganst Stockfish-dev 16 cores and Leela

[Leo]

In my earlier post I reported my test but as I see it escaped the attention.
It is pity but the title of a post can not be modifiable later so I obliged to repeat the part of my test.

Test 2
NNUE-net = SV-2138
HASH = 2GB
Leela ver.0.25.1 Threads = 6, NNCache=20000000, Backend= Multiplexing,
BackendOptions=(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)),(backend=cudnn-fp16,gpu=2)
and using Kiudee params
Leela net = SV-384x30-t60-3010

Result 2
SF+NNUE PO 270720 popc 16 cores : kTRIAD = 14 : 6 (180 draw) 200 games
Elo difference ~25 Elo

+25 ELO points for NNUE, an engine that consume 50 watts of CPU (that's what i mesured on my PC with NNUE engine) vs 200 watts (CPU + GPU) !

And size difference of nets are worse : 20MB NNUE nets vs 160MB SV-384x30-t60-3010 LEELA net. NNUE nets are way more optimized ! NNUE nets are just beginning to learn... 1 year of learning and NNUE nets will be unbeatable.

LEELA (and LC0) are "finished" and out of date.

What a shock to LCO supporters.

[Nay Lin Tun]

OP may consider testing against J92-70 Leela net.


Match: @jhorthos J92-70 vs stockfish 11 in fixed nodes
LC0 version: 0.25.1
LC0 options: cudnn-fp16, 1 thread, default for the rest
SF options: 1 thread, 1GB Hash.
Time control: Leela: 1 kn/move, SF 11: 2 Mn/move
Hardware: RTX 2070, i7-7700
Book: New Chad 6ply book (500 openings)
Tablebases: 6-man TB for both.
Adjudication: 6-man TB, -draw movenumber=50 movecount=5 score=8 -resign movecount=5 score=1000
Software:cutechess-cli
Comments: J92-70: +36 elo, takes the 1st place with safe distance to the 2nd.

Leela: 1 thread, 1Kn/move
SF 11: 1 thread, 2Mn/move

Chad 6-ply book
# PLAYER : RATING ERROR POINTS PLAYED (%) CFS(%) W D L
+ 1 lc0.net.J92-70.1k : 36 15 549.5 1000 55.0 80 288 523 189
2 lc0.net.SV_4585.1k : 27 14 537.5 1000 53.8 56 263 549 188
3 lc0.net.SV_4175.1k : 26 15 535.5 1000 53.5 50 277 517 206
4 lc0.net.SV_4619.1k : 26 15 535.5 1000 53.5 66 275 521 204
...
8 lc0.net.J90-78.1k : 17 14 523.0 1000 52.3 56 243 560 197
9 lc0.net.J91-20.1k : 15 14 521.0 1000 52.1 53 239 564 197
10 lc0.net.J91-80.1k : 14 15 520.0 1000 52.0 56 241 558 201
11 lc0.net.J91-40.1k : 13 14 518.0 1000 51.8 62 233 570 197
12 lc0.net.J90-60.1k : 10 15 513.5 1000 51.4 50 225 577 198
13 lc0.net.J91-150.1k : 10 14 513.5 1000 51.4 56 238 551 211
...
- 17 Stockfish_11.2m : 0 ---- 8140.0 17000 47.9 78 3501 9278 4221
18 lc0.net.SV_3010.1k : -6 16 491.5 1000 49.1 --- 224 535 241

[zullil]

OP may consider testing against J92-70 Leela net.


Match: @jhorthos J92-70 vs stockfish 11 in fixed nodes
LC0 version: 0.25.1
LC0 options: cudnn-fp16, 1 thread, default for the rest
SF options: 1 thread, 1GB Hash.
Time control: Leela: 1 kn/move, SF 11: 2 Mn/move
Hardware: RTX 2070, i7-7700
Book: New Chad 6ply book (500 openings)
Tablebases: 6-man TB for both.
Adjudication: 6-man TB, -draw movenumber=50 movecount=5 score=8 -resign movecount=5 score=1000
Software:cutechess-cli
Comments: J92-70: +36 elo, takes the 1st place with safe distance to the 2nd.

Leela: 1 thread, 1Kn/move
SF 11: 1 thread, 2Mn/move

Chad 6-ply book
# PLAYER : RATING ERROR POINTS PLAYED (%) CFS(%) W D L
+ 1 lc0.net.J92-70.1k : 36 15 549.5 1000 55.0 80 288 523 189
2 lc0.net.SV_4585.1k : 27 14 537.5 1000 53.8 56 263 549 188
3 lc0.net.SV_4175.1k : 26 15 535.5 1000 53.5 50 277 517 206
4 lc0.net.SV_4619.1k : 26 15 535.5 1000 53.5 66 275 521 204
...
8 lc0.net.J90-78.1k : 17 14 523.0 1000 52.3 56 243 560 197
9 lc0.net.J91-20.1k : 15 14 521.0 1000 52.1 53 239 564 197
10 lc0.net.J91-80.1k : 14 15 520.0 1000 52.0 56 241 558 201
11 lc0.net.J91-40.1k : 13 14 518.0 1000 51.8 62 233 570 197
12 lc0.net.J90-60.1k : 10 15 513.5 1000 51.4 50 225 577 198
13 lc0.net.J91-150.1k : 10 14 513.5 1000 51.4 56 238 551 211
...
- 17 Stockfish_11.2m : 0 ---- 8140.0 17000 47.9 78 3501 9278 4221
18 lc0.net.SV_3010.1k : -6 16 491.5 1000 49.1 --- 224 535 241

But Stockfish-dev (with NNUE) is like +100 Elo above Stockfish 11.

[Nay Lin Tun]

Stockfish 11 or 12 is not important, it is about leela network strength comparision. (J92-70 vs sv 3010)
Leela team will always test with Stockfish 11 anyway,( in comparision with known opponent strength).

[zullil]

Stockfish 11 or 12 is not important, it is about leela network strength comparision. (J92-70 vs sv 3010)
Leela team will always test with Stockfish 11 anyway,( in comparision with known opponent strength).

OK, it's a very strong network. But how does it compare to Stockfish-dev? Which is now much stronger (+120 Elo?) than Stockfish 11.

[Nay Lin Tun]

This guy will test in TCEC simulation condition tomorrow! (Known as Alternative TCEC in Leela discord )


SUPREMACY GRAND FINAL

SF-NNUE- Official-Latest v Lc0-JH-92-70-Official Best
These two best engines will fight it out for the title of Computer Chess Supremacy

Stockfish-NNUE-latest binary & net with ARCH-vnni
Lc0-JH-92-70-Strongest Lc0 available

100 game match at Classical long time control 120 minutes + 30 second increments each.
Book: WCEC Dynamic – sequential reverse colors
TBS: Full 3/4/5/6 and selected 7 pieces
Hardware: i9-10980EX 36 threads CPU and 2 x RTX 2070 GPU
A 28 day event

UPGRADES:
Both engines and binaries will be upgraded to the latest available after each 20 games, so as to keep the match relevant with each new improved release.

PURPOSE OF THE EVENT
1. Pure entertainment
2. Scaling of NNUE at Classical very long time control
3. To find out if NNUE is the REAL DEAL

STARTING Wednesday 19th August at 2:00 pm UCT/GMT (10:00 am USA UCT-zone 4)

[Dann Corbit]

RE: "Hardware: i9-10980EX 36 threads CPU and 2 x RTX 2070 GPU"

Not close to this:
Current TCEC CPU server
CPUs: 4 x Intel Xeon 4xE5-4669v4
Cores: 88 physical / 176 threads
RAM: 128 GB DDR4 (available to engines)
RAM: 1 TB (available to 6-pieces Syzygy)
HDD: 7 TB total
OS: CentOS Linux release 7.7.1908 (Core)
Current TCEC GPU server
GPUs: 4x V100
CPU: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz, 32 vcores
RAM: 48GiB (available to engines)
RAM 128GiB (RAM unused by engines is used for caching tablebase files)
SSD: 500GB
6-piece Syzygy and 6-piece Scorpio bitbases
OS: Ubuntu 18.04.4 LTS

[Nay Lin Tun]

He does SF vs Leela speed ratio the same as Tcec. ( or the fair way he think)

[corres]

From where we can download jhorthos J92-70 net?

[MMarco]

From where we can download jhorthos J92-70 net?

From here: https://github.com/jhorthos/lczero-trai ... a-Training