SF+NNUE 16 cores aganst Stockfish-dev 16 cores and Leela

[corres]

In my earlier post I reported my test but as I see it escaped the attention.
It is pity but the title of a post can not be modifiable later so I obliged to repeat the part of my test.

My machine
OS - Windows 10 x64
CPU - Ryzen 9 3950x 16 x 4.4 GHz Fixed, SMT=disabled PB=disabled
RAM -32 GB
GUI - Fritz 14
SYZYGY -6 men
Nalimov -5 men (for GUI only)
kTriad (NVIDIA GPUs) = 1 x RTX 2080 Ti + 2 x RTX 2060 (chess power ~ 2 x RTX 2080 Ti)

Test params
TC = 1 min + 2 sec / move
Opening positions 100 with alternated color (200 games) from my Minibook

Test 1
NNUE net = SV-2138
HASH = 2GB

Result 1
SF+NNUE PO 270720 popc 16 cores : Stockfish200718 popc 16 cores = 22(!) : 2 (176 draw) 200 games
Elo difference ~ 70 Elo


Test 2
NNUE-net = SV-2138
HASH = 2GB
Leela ver.0.25.1 Threads = 6, NNCache=20000000, Backend= Multiplexing,
BackendOptions=(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)),(backend=cudnn-fp16,gpu=2)
and using Kiudee params
Leela net = SV-384x30-t60-3010

Result 2
SF+NNUE PO 270720 popc 16 cores : kTRIAD = 14 : 6 (180 draw) 200 games
Elo difference ~25 Elo

[M ANSARI]

If this score holds then things look like SF will be ahead even on 8 core against 2 x 2080Ti Lc0. Not sure if your network is the strongest network for SF NN but things are progressing so fast that in 2 to 4 weeks there will probably be some fast gains for SF NN and then things will stabilize. By then it should be a good bit stronger and maybe on 8 core or even 4 cores it can go toe to toe with Lc0 at max hardware. I have a 2080Ti card in my system and it is huge! I can't imagine having 2 or 3 cards in my box !!! The thing is that 32 core is becoming more mainstream and a system with 2 x 2080Ti cards can be better compared to a 64 core system. With 64 cores I think Lc0 has no chance against SF NNUE no matter what the hardware ... unless a major improvement in Lc0 takes place.

[corres]

If this score holds then things look like SF will be ahead even on 8 core against 2 x 2080Ti Lc0. Not sure if your network is the strongest network for SF NN but things are progressing so fast that in 2 to 4 weeks there will probably be some fast gains for SF NN and then things will stabilize. By then it should be a good bit stronger and maybe on 8 core or even 4 cores it can go toe to toe with Lc0 at max hardware. I have a 2080Ti card in my system and it is huge! I can't imagine having 2 or 3 cards in my box !!! The thing is that 32 core is becoming more mainstream and a system with 2 x 2080Ti cards can be better compared to a 64 core system. With 64 cores I think Lc0 has no chance against SF NNUE no matter what the hardware ... unless a major improvement in Lc0 takes place.

The NVIDIA RTX 3090 is under development this will be the major improvement for Leela.

[Laskos]

If this score holds then things look like SF will be ahead even on 8 core against 2 x 2080Ti Lc0. Not sure if your network is the strongest network for SF NN but things are progressing so fast that in 2 to 4 weeks there will probably be some fast gains for SF NN and then things will stabilize. By then it should be a good bit stronger and maybe on 8 core or even 4 cores it can go toe to toe with Lc0 at max hardware. I have a 2080Ti card in my system and it is huge! I can't imagine having 2 or 3 cards in my box !!! The thing is that 32 core is becoming more mainstream and a system with 2 x 2080Ti cards can be better compared to a 64 core system. With 64 cores I think Lc0 has no chance against SF NNUE no matter what the hardware ... unless a major improvement in Lc0 takes place.

The NVIDIA RTX 3090 is under development this will be the major improvement for Leela.

Might also benefit from new backend on new CUDA and cuDNN. NNUE are only good or best for such tactical games like Shogi and Chess, but in Go Alpha Zero like NNs on GPU rule. KataGo is unbelievable strong on my RTX2070 GPU, by now I can blunder check any human Go game, Leela Zero Go games and Alpha Go games. Sometimes KataGo on RTX2070 with many playouts (say half a million) seems stronger than Alpha Zero Go.

[Werewolf]

If this score holds then things look like SF will be ahead even on 8 core against 2 x 2080Ti Lc0. Not sure if your network is the strongest network for SF NN but things are progressing so fast that in 2 to 4 weeks there will probably be some fast gains for SF NN and then things will stabilize. By then it should be a good bit stronger and maybe on 8 core or even 4 cores it can go toe to toe with Lc0 at max hardware. I have a 2080Ti card in my system and it is huge! I can't imagine having 2 or 3 cards in my box !!! The thing is that 32 core is becoming more mainstream and a system with 2 x 2080Ti cards can be better compared to a 64 core system. With 64 cores I think Lc0 has no chance against SF NNUE no matter what the hardware ... unless a major improvement in Lc0 takes place.

The NVIDIA RTX 3090 is under development this will be the major improvement for Leela.

But how much? Some estimates for Lc0 are as low as +19% speedup, which won't be enough

[Raphexon]

If this score holds then things look like SF will be ahead even on 8 core against 2 x 2080Ti Lc0. Not sure if your network is the strongest network for SF NN but things are progressing so fast that in 2 to 4 weeks there will probably be some fast gains for SF NN and then things will stabilize. By then it should be a good bit stronger and maybe on 8 core or even 4 cores it can go toe to toe with Lc0 at max hardware. I have a 2080Ti card in my system and it is huge! I can't imagine having 2 or 3 cards in my box !!! The thing is that 32 core is becoming more mainstream and a system with 2 x 2080Ti cards can be better compared to a 64 core system. With 64 cores I think Lc0 has no chance against SF NNUE no matter what the hardware ... unless a major improvement in Lc0 takes place.

The NVIDIA RTX 3090 is under development this will be the major improvement for Leela.

Might also benefit from new backend on new CUDA and cuDNN. NNUE are only good or best for such tactical games like Shogi and Chess, but in Go Alpha Zero like NNs on GPU rule. KataGo is unbelievable strong on my RTX2070 GPU, by now I can blunder check any human Go game, Leela Zero Go games and Alpha Go games. Sometimes KataGo on RTX2070 with many playouts (say half a million) seems stronger than Alpha Zero Go.

Are there any serious Go-engine writers using AB in the past 15 years?
With increased core counts I think AB should be more viable in the past. 9x9 Go should definitely be in the range where AB is sufficiently effective.

The other problem is that somebody would either have to find good, simple heuristics for move ordering or we need a second fast NN to deal with move ordering.
Else the BF of 19x19 is going to kill the chances an AB engine may have.

[Laskos]

If this score holds then things look like SF will be ahead even on 8 core against 2 x 2080Ti Lc0. Not sure if your network is the strongest network for SF NN but things are progressing so fast that in 2 to 4 weeks there will probably be some fast gains for SF NN and then things will stabilize. By then it should be a good bit stronger and maybe on 8 core or even 4 cores it can go toe to toe with Lc0 at max hardware. I have a 2080Ti card in my system and it is huge! I can't imagine having 2 or 3 cards in my box !!! The thing is that 32 core is becoming more mainstream and a system with 2 x 2080Ti cards can be better compared to a 64 core system. With 64 cores I think Lc0 has no chance against SF NNUE no matter what the hardware ... unless a major improvement in Lc0 takes place.

The NVIDIA RTX 3090 is under development this will be the major improvement for Leela.

Might also benefit from new backend on new CUDA and cuDNN. NNUE are only good or best for such tactical games like Shogi and Chess, but in Go Alpha Zero like NNs on GPU rule. KataGo is unbelievable strong on my RTX2070 GPU, by now I can blunder check any human Go game, Leela Zero Go games and Alpha Go games. Sometimes KataGo on RTX2070 with many playouts (say half a million) seems stronger than Alpha Zero Go.

Are there any serious Go-engine writers using AB in the past 15 years?
With increased core counts I think AB should be more viable in the past. 9x9 Go should definitely be in the range where AB is sufficiently effective.

The other problem is that somebody would either have to find good, simple heuristics for move ordering or we need a second fast NN to deal with move ordering.
Else the BF of 19x19 is going to kill the chances an AB engine may have.

I am not aware of strong Go engines of the past 15 years using AB. Until NNs, first without search, then with PUCT MCTS search, there was hardly any solid static eval of position, and move ordering heuristics were rudimentary too. Yes, I think for AB search separate value network and a much smaller policy network are needed. Or a heuristic, but good and simple move ordering heuristics are notoriously difficult for 19x19 Go since long.

[corres]

If this score holds then things look like SF will be ahead even on 8 core against 2 x 2080Ti Lc0. Not sure if your network is the strongest network for SF NN but things are progressing so fast that in 2 to 4 weeks there will probably be some fast gains for SF NN and then things will stabilize. By then it should be a good bit stronger and maybe on 8 core or even 4 cores it can go toe to toe with Lc0 at max hardware. I have a 2080Ti card in my system and it is huge! I can't imagine having 2 or 3 cards in my box !!! The thing is that 32 core is becoming more mainstream and a system with 2 x 2080Ti cards can be better compared to a 64 core system. With 64 cores I think Lc0 has no chance against SF NNUE no matter what the hardware ... unless a major improvement in Lc0 takes place.

The NVIDIA RTX 3090 is under development this will be the major improvement for Leela.

But how much? Some estimates for Lc0 are as low as +19% speedup, which won't be enough

NVIDIA develops not only new GPUs but new software for their GPUs. RTX 2000 series GPUs yield ~30 %
speedup using CUDA11 and cudnn 7.6.4.
The enhancement of Leela`s effectiveness using more than two GPUs is also a possibility.
In general the greater net of Leela give more exact evaluation than the small net of NNUE, I think.

[Werewolf]

In general the greater net of Leela give more exact evaluation than the small net of NNUE, I think.

Yes I was thinking this. And Lc0 may move to a bigger net one day too.

How deeply does the Lc0 MCTS search look ahead? It's not complete games is it?

[mehmet123]

If this score holds then things look like SF will be ahead even on 8 core against 2 x 2080Ti Lc0. Not sure if your network is the strongest network for SF NN but things are progressing so fast that in 2 to 4 weeks there will probably be some fast gains for SF NN and then things will stabilize. By then it should be a good bit stronger and maybe on 8 core or even 4 cores it can go toe to toe with Lc0 at max hardware. I have a 2080Ti card in my system and it is huge! I can't imagine having 2 or 3 cards in my box !!! The thing is that 32 core is becoming more mainstream and a system with 2 x 2080Ti cards can be better compared to a 64 core system. With 64 cores I think Lc0 has no chance against SF NNUE no matter what the hardware ... unless a major improvement in Lc0 takes place.

The NVIDIA RTX 3090 is under development this will be the major improvement for Leela.

Might also benefit from new backend on new CUDA and cuDNN. NNUE are only good or best for such tactical games like Shogi and Chess, but in Go Alpha Zero like NNs on GPU rule. KataGo is unbelievable strong on my RTX2070 GPU, by now I can blunder check any human Go game, Leela Zero Go games and Alpha Go games. Sometimes KataGo on RTX2070 with many playouts (say half a million) seems stronger than Alpha Zero Go.

Is KataGO commercial, private or free program. If it's commercial or private then which is the most strongest free Go program.