TalkChess.com

Best SF NNUE nets (Sergio's of today, for example) with GK binary seem to hold against Lc0 even in my very GPU-skewed conditions.

4 i7 cores
RTX 2070 GPU
Old "Leela Ratio" of about 3.0 --- very GPU favorable.

60s + 0.6s
SF_dev is beaten heavily in these conditions by Lc0, but SF NNUE holds its ground against Lc0,
If SF NNUE holds in these conditions (3.0 Leela Ratio), then in TCEC and CCCC, where the old "Leela Ratio" is most often about ~1, Lc0 will be beaten heavily by SF NNUE. In TCEC 18, even SF_dev managed to beat it.

A fast test between two of the latest sergio's nets, 15+0.1 Same best sergio's net against GK best (that was already better than Stockfish dev), 20 + 0.1: Congratulations Sergio!! Your net is terrible strong!

Laskos wrote: ↑Wed Jul 22, 2020 10:48 am Best SF NNUE nets (Sergio's of today, for example) with GK binary seem to hold against Lc0 even in my very GPU-skewed conditions.

4 i7 cores
RTX 2070 GPU
Old "Leela Ratio" of about 3.0 --- very GPU favorable.

60s + 0.6s

Code: Select all

Score of Sergio_22.07_0944 NNUE vs Lc0 SV_1541: 27 - 28 - 45  [0.495] 100

...      Sergio_22.07_0944 NNUE playing White: 27 - 2 - 21  [0.750] 50
...      Sergio_22.07_0944 NNUE playing Black: 0 - 26 - 24  [0.240] 50
...      White vs Black: 53 - 2 - 45  [0.755] 100
Elo difference: -3.5 +/- 50.8, LOS: 44.6 %, DrawRatio: 45.0 %
Finished match

SF_dev is beaten heavily in these conditions by Lc0, but SF NNUE holds its ground against Lc0,
If SF NNUE holds in these conditions (3.0 Leela Ratio), then in TCEC and CCCC, where the old "Leela Ratio" is most often about ~1, Lc0 will be beaten heavily by SF NNUE. In TCEC 18, even SF_dev managed to beat it.

In CCC, lczero won nicely against Stockfish NNUE with a nice margin. So who knows ?

JJJ wrote: ↑Wed Jul 22, 2020 2:52 pm

Laskos wrote: ↑Wed Jul 22, 2020 10:48 am Best SF NNUE nets (Sergio's of today, for example) with GK binary seem to hold against Lc0 even in my very GPU-skewed conditions.

4 i7 cores
RTX 2070 GPU
Old "Leela Ratio" of about 3.0 --- very GPU favorable.

60s + 0.6s

Code: Select all

Score of Sergio_22.07_0944 NNUE vs Lc0 SV_1541: 27 - 28 - 45  [0.495] 100

...      Sergio_22.07_0944 NNUE playing White: 27 - 2 - 21  [0.750] 50
...      Sergio_22.07_0944 NNUE playing Black: 0 - 26 - 24  [0.240] 50
...      White vs Black: 53 - 2 - 45  [0.755] 100
Elo difference: -3.5 +/- 50.8, LOS: 44.6 %, DrawRatio: 45.0 %
Finished match

SF_dev is beaten heavily in these conditions by Lc0, but SF NNUE holds its ground against Lc0,
If SF NNUE holds in these conditions (3.0 Leela Ratio), then in TCEC and CCCC, where the old "Leela Ratio" is most often about ~1, Lc0 will be beaten heavily by SF NNUE. In TCEC 18, even SF_dev managed to beat it.

In CCC, lczero won nicely against Stockfish NNUE with a nice margin. So who knows ?

Untested net from Jjoshua that's behind a paywall on Patreon.
Maybe the extra slowdown is also an issue inherent to 384x sized net on that hardware.
Maybe VRM is overheating from AVX usage.


Vondele's 250 thread test (although SSS) indicated no problems:
https://github.com/official-stockfish/S ... -660519116

Though I wasn't part of the technical discussions, I can confirm that the Lc0-SFNNUE match had some compile issues- we never got instructions to set a flag, or something. And who knows how fast a jjosh net should be?

So, this is your alert not to use that result.

Right now we're running a three-way, 3|2 match among Stockfish, Stockfish+NNUE, and Lc0 where everyone is running quite nicely. And this second one can be used as strength evidence, if there's still any doubt about SFNNUE.

CCC matches are anyway totally not representative of anything. Lc0 has a ridiculous advantage in hardware (CPU hardware is a total joke, weaker than Ryzen 3970X), plus TC is the best possible one for Lc0 (in shorter and longer TC SF scales better).

Milos wrote: ↑Thu Jul 23, 2020 12:11 am CCC matches are anyway totally not representative of anything. Lc0 has a ridiculous advantage in hardware (CPU hardware is a total joke, weaker than Ryzen 3970X), plus TC is the best possible one for Lc0 (in shorter and longer TC SF scales better).

They're still double Xeon plats...

Also TC should suit Stockfish, every other TC is a joke and a scam.

Raphexon wrote: ↑Thu Jul 23, 2020 12:16 am They're still double Xeon plats...

Yes, they are glorified overpriced pieces of junk. Probably the worst performance/price ratio CPUs in recent history.

Everything is a moving target (all the mentioned engines) and so it is very hard to make predictions.

I guess the ultimate winner will be ...
Something new. At some point anyway.
It quite often turns out that way.

We get surprised and we get surprised and somehow, even after being surprised over and over and over, we get surprised again. It's pretty strange when you think about it.

Watch Olithink become world champion. Of course by then it might be bloated to nearly 100k.

Dann Corbit wrote: ↑Thu Jul 23, 2020 7:44 am Everything is a moving target (all the mentioned engines) and so it is very hard to make predictions.

I guess the ultimate winner will be ...
Something new. At some point anyway.
It quite often turns out that way.

We get surprised and we get surprised and somehow, even after being surprised over and over and over, we get surprised again. It's pretty strange when you think about it.

Watch Olithink become world champion. Of course by then it might be bloated to nearly 100k.

Well it'll be over for quite some period. MCTS and big nets are not the way to go as it has been known for a while. And A/B with handcrafted eval is not using hardware in the best possible way. It's been clear for a while that A/B with fast small NN eval is a way to go. We lacked good implementation of it (we can now see how bad and "noob" was Giraffe NN for example). Now it's there. Once main SF devs embrace it and start tuning search together with eval, sky is the limit. This is a real paradigm shift, not A0 or Lc0.
Plus there is another aspect. New CPUs will be having more and more support for vector INT8 and BFLOAT16 multiplication operations. So running things on GPUs is not gonna make advantage.