Score of Sergio_22.07_0944 NNUE vs Lc0 SV_1541: 27 - 28 - 45 [0.495] 100
... Sergio_22.07_0944 NNUE playing White: 27 - 2 - 21 [0.750] 50
... Sergio_22.07_0944 NNUE playing Black: 0 - 26 - 24 [0.240] 50
... White vs Black: 53 - 2 - 45 [0.755] 100
Elo difference: -3.5 +/- 50.8, LOS: 44.6 %, DrawRatio: 45.0 %
Finished match
SF_dev is beaten heavily in these conditions by Lc0, but SF NNUE holds its ground against Lc0,
If SF NNUE holds in these conditions (3.0 Leela Ratio), then in TCEC and CCCC, where the old "Leela Ratio" is most often about ~1, Lc0 will be beaten heavily by SF NNUE. In TCEC 18, even SF_dev managed to beat it.
Laskos wrote: ↑Wed Jul 22, 2020 10:48 am
Best SF NNUE nets (Sergio's of today, for example) with GK binary seem to hold against Lc0 even in my very GPU-skewed conditions.
4 i7 cores
RTX 2070 GPU
Old "Leela Ratio" of about 3.0 --- very GPU favorable.
Score of Sergio_22.07_0944 NNUE vs Lc0 SV_1541: 27 - 28 - 45 [0.495] 100
... Sergio_22.07_0944 NNUE playing White: 27 - 2 - 21 [0.750] 50
... Sergio_22.07_0944 NNUE playing Black: 0 - 26 - 24 [0.240] 50
... White vs Black: 53 - 2 - 45 [0.755] 100
Elo difference: -3.5 +/- 50.8, LOS: 44.6 %, DrawRatio: 45.0 %
Finished match
SF_dev is beaten heavily in these conditions by Lc0, but SF NNUE holds its ground against Lc0,
If SF NNUE holds in these conditions (3.0 Leela Ratio), then in TCEC and CCCC, where the old "Leela Ratio" is most often about ~1, Lc0 will be beaten heavily by SF NNUE. In TCEC 18, even SF_dev managed to beat it.
In CCC, lczero won nicely against Stockfish NNUE with a nice margin. So who knows ?
Laskos wrote: ↑Wed Jul 22, 2020 10:48 am
Best SF NNUE nets (Sergio's of today, for example) with GK binary seem to hold against Lc0 even in my very GPU-skewed conditions.
4 i7 cores
RTX 2070 GPU
Old "Leela Ratio" of about 3.0 --- very GPU favorable.
Score of Sergio_22.07_0944 NNUE vs Lc0 SV_1541: 27 - 28 - 45 [0.495] 100
... Sergio_22.07_0944 NNUE playing White: 27 - 2 - 21 [0.750] 50
... Sergio_22.07_0944 NNUE playing Black: 0 - 26 - 24 [0.240] 50
... White vs Black: 53 - 2 - 45 [0.755] 100
Elo difference: -3.5 +/- 50.8, LOS: 44.6 %, DrawRatio: 45.0 %
Finished match
SF_dev is beaten heavily in these conditions by Lc0, but SF NNUE holds its ground against Lc0,
If SF NNUE holds in these conditions (3.0 Leela Ratio), then in TCEC and CCCC, where the old "Leela Ratio" is most often about ~1, Lc0 will be beaten heavily by SF NNUE. In TCEC 18, even SF_dev managed to beat it.
In CCC, lczero won nicely against Stockfish NNUE with a nice margin. So who knows ?
Untested net from Jjoshua that's behind a paywall on Patreon.
Maybe the extra slowdown is also an issue inherent to 384x sized net on that hardware.
Maybe VRM is overheating from AVX usage.
Though I wasn't part of the technical discussions, I can confirm that the Lc0-SFNNUE match had some compile issues- we never got instructions to set a flag, or something. And who knows how fast a jjosh net should be?
So, this is your alert not to use that result.
Right now we're running a three-way, 3|2 match among Stockfish, Stockfish+NNUE, and Lc0 where everyone is running quite nicely. And this second one can be used as strength evidence, if there's still any doubt about SFNNUE.
CCC matches are anyway totally not representative of anything. Lc0 has a ridiculous advantage in hardware (CPU hardware is a total joke, weaker than Ryzen 3970X), plus TC is the best possible one for Lc0 (in shorter and longer TC SF scales better).
Milos wrote: ↑Thu Jul 23, 2020 12:11 am
CCC matches are anyway totally not representative of anything. Lc0 has a ridiculous advantage in hardware (CPU hardware is a total joke, weaker than Ryzen 3970X), plus TC is the best possible one for Lc0 (in shorter and longer TC SF scales better).
They're still double Xeon plats...
Also TC should suit Stockfish, every other TC is a joke and a scam.
Everything is a moving target (all the mentioned engines) and so it is very hard to make predictions.
I guess the ultimate winner will be ...
Something new. At some point anyway.
It quite often turns out that way.
We get surprised and we get surprised and somehow, even after being surprised over and over and over, we get surprised again. It's pretty strange when you think about it.
Watch Olithink become world champion. Of course by then it might be bloated to nearly 100k.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Dann Corbit wrote: ↑Thu Jul 23, 2020 7:44 am
Everything is a moving target (all the mentioned engines) and so it is very hard to make predictions.
I guess the ultimate winner will be ...
Something new. At some point anyway.
It quite often turns out that way.
We get surprised and we get surprised and somehow, even after being surprised over and over and over, we get surprised again. It's pretty strange when you think about it.
Watch Olithink become world champion. Of course by then it might be bloated to nearly 100k.
Well it'll be over for quite some period. MCTS and big nets are not the way to go as it has been known for a while. And A/B with handcrafted eval is not using hardware in the best possible way. It's been clear for a while that A/B with fast small NN eval is a way to go. We lacked good implementation of it (we can now see how bad and "noob" was Giraffe NN for example). Now it's there. Once main SF devs embrace it and start tuning search together with eval, sky is the limit. This is a real paradigm shift, not A0 or Lc0.
Plus there is another aspect. New CPUs will be having more and more support for vector INT8 and BFLOAT16 multiplication operations. So running things on GPUs is not gonna make advantage.
