this will be the merge of a lifetime : SF 80 Elo+

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Raphexon
Posts: 386
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by Raphexon » Wed Aug 05, 2020 4:08 pm

syzygy wrote:
Wed Aug 05, 2020 3:00 pm
jdart wrote:
Wed Aug 05, 2020 1:30 pm
Can someone explain this project to me in small words? I gathter it is basically Stockfish with a NN eval, or? And is the neural net part running on CPU, or GPU? Apologies, I have not been following the lengthy threads on this.
Yes, with the NN running on CPU.

Apparently the NN is 20MB but using it in the eval only halves nps, which I find quite remarkable. I haven't tried to understand the code yet, but it seems to rely heavily on vector instructions (unsurprisingly).

So it turns out that NNs are much better than humans at writing evaluation functions (even taking into account speed of execution).
It only updates parts incrementally when the board state changes. That way it doesn't need to recalculate the whole net every time.

And while 20MB, it has a lot of functionally useless features. Can't compare it 1:1 with Leela.
There are also millions of different ways to encode information inside the NN, speed is affected depending on how much it has to recalculate each ply searched.
See below for examples:
https://github.com/tttak/Stockfish/tree ... hitectures

halfkp-mobility_256x2-32-32 doesn't add a lot of size, but it causes an enormous slowdown.
halfkpe4 (which I don't really understand) quadruples size but only introduces a very minor slowdown.

Cornfed
Posts: 191
Joined: Sun Apr 26, 2020 9:40 pm
Full name: Brian D. Smith

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by Cornfed » Thu Aug 06, 2020 12:02 am

Ovyron wrote:
Wed Aug 05, 2020 10:41 am
This is the breakthrough I've been predicting for years! It finally happened, and I'm just glad chess looks very healthy after it (draw rates have gone down! Some lines are proving to be playable again!)
Until everyone uses these?

mwyoung
Posts: 2727
Joined: Wed May 12, 2010 8:00 pm

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by mwyoung » Thu Aug 06, 2020 5:52 am

MikeB wrote:
Tue Aug 04, 2020 5:58 pm
Leto wrote:
Tue Aug 04, 2020 4:17 pm
Is that at LTC?
It scales very well.

OK.. It scales very well...?!

Re: Stockfish NNUE (Sergio 2138) vs Stockfish 170720 (32 Thread Test) (TC = 30m+30s)
Post by mwyoung » Thu Aug 06, 2020 4:31 am

End of Match.

DESKTOP-CORSAIR, Rapid 30.0min+30.0sec 0

1 SF+NNUE PO 290720 x64 popc +17 +7/=95/-2 52.40% 54.5/104
2 Stockfish 170720 64 POPCNT -17 +2/=95/-7 47.60% 49.5/104
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

User avatar
Laskos
Posts: 10949
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by Laskos » Thu Aug 06, 2020 5:57 am

mwyoung wrote:
Thu Aug 06, 2020 5:52 am
MikeB wrote:
Tue Aug 04, 2020 5:58 pm
Leto wrote:
Tue Aug 04, 2020 4:17 pm
Is that at LTC?
It scales very well.

OK.. It scales very well...?!

Re: Stockfish NNUE (Sergio 2138) vs Stockfish 170720 (32 Thread Test) (TC = 30m+30s)
Post by mwyoung » Thu Aug 06, 2020 4:31 am

End of Match.

DESKTOP-CORSAIR, Rapid 30.0min+30.0sec 0

1 SF+NNUE PO 290720 x64 popc +17 +7/=95/-2 52.40% 54.5/104
2 Stockfish 170720 64 POPCNT -17 +2/=95/-7 47.60% 49.5/104
It's your fault having 91%+ draw rate. Chose better openings.

mwyoung
Posts: 2727
Joined: Wed May 12, 2010 8:00 pm

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by mwyoung » Thu Aug 06, 2020 6:02 am

Laskos wrote:
Thu Aug 06, 2020 5:57 am
mwyoung wrote:
Thu Aug 06, 2020 5:52 am
MikeB wrote:
Tue Aug 04, 2020 5:58 pm
Leto wrote:
Tue Aug 04, 2020 4:17 pm
Is that at LTC?
It scales very well.

OK.. It scales very well...?!

Re: Stockfish NNUE (Sergio 2138) vs Stockfish 170720 (32 Thread Test) (TC = 30m+30s)
Post by mwyoung » Thu Aug 06, 2020 4:31 am

End of Match.

DESKTOP-CORSAIR, Rapid 30.0min+30.0sec 0

1 SF+NNUE PO 290720 x64 popc +17 +7/=95/-2 52.40% 54.5/104
2 Stockfish 170720 64 POPCNT -17 +2/=95/-7 47.60% 49.5/104
It's your fault having 91%+ draw rate. Chose better openings.
Book is set to 6 moves. The same settings as when I got a huge win factor over Stockfish also at short time controls. Sorry you need a new theory! And you need to do the leg work before advertising NNUE scales very well!
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

User avatar
Laskos
Posts: 10949
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by Laskos » Thu Aug 06, 2020 6:18 am

mwyoung wrote:
Thu Aug 06, 2020 6:02 am
Laskos wrote:
Thu Aug 06, 2020 5:57 am
mwyoung wrote:
Thu Aug 06, 2020 5:52 am
MikeB wrote:
Tue Aug 04, 2020 5:58 pm
Leto wrote:
Tue Aug 04, 2020 4:17 pm
Is that at LTC?
It scales very well.

OK.. It scales very well...?!

Re: Stockfish NNUE (Sergio 2138) vs Stockfish 170720 (32 Thread Test) (TC = 30m+30s)
Post by mwyoung » Thu Aug 06, 2020 4:31 am

End of Match.

DESKTOP-CORSAIR, Rapid 30.0min+30.0sec 0

1 SF+NNUE PO 290720 x64 popc +17 +7/=95/-2 52.40% 54.5/104
2 Stockfish 170720 64 POPCNT -17 +2/=95/-7 47.60% 49.5/104
It's your fault having 91%+ draw rate. Chose better openings.
Book is set to 6 moves. The same settings as when I got a huge win factor over Stockfish also at short time controls. Sorry you need a new theory! And you need to do the leg work before advertising NNUE scales very well!
Even your crappy test shows +7 -2, an excellent W/L ratio, probably denoting that at 60% draw rate, the difference could easily be 80 Elo points.

mwyoung
Posts: 2727
Joined: Wed May 12, 2010 8:00 pm

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by mwyoung » Thu Aug 06, 2020 6:25 am

Laskos wrote:
Thu Aug 06, 2020 6:18 am
mwyoung wrote:
Thu Aug 06, 2020 6:02 am
Laskos wrote:
Thu Aug 06, 2020 5:57 am
mwyoung wrote:
Thu Aug 06, 2020 5:52 am
MikeB wrote:
Tue Aug 04, 2020 5:58 pm
Leto wrote:
Tue Aug 04, 2020 4:17 pm
Is that at LTC?
It scales very well.

OK.. It scales very well...?!

Re: Stockfish NNUE (Sergio 2138) vs Stockfish 170720 (32 Thread Test) (TC = 30m+30s)
Post by mwyoung » Thu Aug 06, 2020 4:31 am

End of Match.

DESKTOP-CORSAIR, Rapid 30.0min+30.0sec 0

1 SF+NNUE PO 290720 x64 popc +17 +7/=95/-2 52.40% 54.5/104
2 Stockfish 170720 64 POPCNT -17 +2/=95/-7 47.60% 49.5/104
It's your fault having 91%+ draw rate. Chose better openings.
Book is set to 6 moves. The same settings as when I got a huge win factor over Stockfish also at short time controls. Sorry you need a new theory! And you need to do the leg work before advertising NNUE scales very well!
Even your crappy test shows +7 -2, an excellent W/L ratio, probably denoting that at 60% draw rate, the difference could easily be 80 Elo points.
It could be, and if my grandma had wheels she would be a wagon. Stockfish NNUE scales good enough. The point is you never tested at LTC. So you don't know anything as usual. :lol:
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.

Raphexon
Posts: 386
Joined: Sun Mar 17, 2019 11:00 am
Full name: Henk Drost

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by Raphexon » Thu Aug 06, 2020 6:35 am

Laskos wrote:
Thu Aug 06, 2020 6:18 am
mwyoung wrote:
Thu Aug 06, 2020 6:02 am
Laskos wrote:
Thu Aug 06, 2020 5:57 am
mwyoung wrote:
Thu Aug 06, 2020 5:52 am
MikeB wrote:
Tue Aug 04, 2020 5:58 pm
Leto wrote:
Tue Aug 04, 2020 4:17 pm
Is that at LTC?
It scales very well.

OK.. It scales very well...?!

Re: Stockfish NNUE (Sergio 2138) vs Stockfish 170720 (32 Thread Test) (TC = 30m+30s)
Post by mwyoung » Thu Aug 06, 2020 4:31 am

End of Match.

DESKTOP-CORSAIR, Rapid 30.0min+30.0sec 0

1 SF+NNUE PO 290720 x64 popc +17 +7/=95/-2 52.40% 54.5/104
2 Stockfish 170720 64 POPCNT -17 +2/=95/-7 47.60% 49.5/104
It's your fault having 91%+ draw rate. Chose better openings.
Book is set to 6 moves. The same settings as when I got a huge win factor over Stockfish also at short time controls. Sorry you need a new theory! And you need to do the leg work before advertising NNUE scales very well!
Even your crappy test shows +7 -2, an excellent W/L ratio, probably denoting that at 60% draw rate, the difference could easily be 80 Elo points.
NNUE's "lack of scaling" posts can be summarized in just a few words: Coin flipping and elo compression.

User avatar
M ANSARI
Posts: 3502
Joined: Thu Mar 16, 2006 6:10 pm

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by M ANSARI » Thu Aug 06, 2020 7:03 am

Is there anyway to run the NN evaluation on a programmable customized daughter card and get it to run faster or more efficiently? It is funny as everyone had been looking for a way to run Lc0 on GPU while having some sort of safety net to prevent Lc0 tactical weakness ... I guess using an AB engine with a Lc0 evaluation to cover for positional weakness works better! It sure makes using an engine for analysis much more useful as with Lc0 you always needed a sanity check to make sure no cheap tricks were missed. This is probably just the beginning of an arms race and most likely there will be specific NN modules that concentrate on certain openings or certain stages of the game. An engine that can on the fly choose between several smaller targeted nets (depending on what is needed ) will be able to run those nets much faster ... so theoretical should have a strong advantage. Now that I think about it this would be really good for Chess 960 as you can make a custom net for every starting position and then once the game goes along switch to another net.

User avatar
Ovyron
Posts: 4415
Joined: Tue Jul 03, 2007 2:30 am

Re: this will be the merge of a lifetime : SF 80 Elo+

Post by Ovyron » Thu Aug 06, 2020 7:53 am

Cornfed wrote:
Thu Aug 06, 2020 12:02 am
Ovyron wrote:
Wed Aug 05, 2020 10:41 am
This is the breakthrough I've been predicting for years! It finally happened, and I'm just glad chess looks very healthy after it (draw rates have gone down! Some lines are proving to be playable again!)
Until everyone uses these?
Everyone is already using these, when Eman Enigma 1.00 (closed source stockfish clone) appeared NNUE got widespread adoption. It's sad that now the most used engine in the world is an illegal one (you can't have its source), but at this point all NNUE clones are basically the same, what matters mostly is the net you use (the biggest impact.) I've been beating people that use Eman Enigma 1.10 that use much faster hardware than mine, because of their poor choice of net (I assume, since they're outsearching me with their NNUE, yet my eval is superior allowing the win.)

The effect seen has mimicked the world's capitalist economy, the guys with most elo gained more when switching to NNUE and became unreachable, the weakest ones sank more (I'm on this group :mrgreen: ). The ones in the middle sunk, then went back to where they were after fixing their openings (critical, because lines that were very solid before became suicidal now).

I'm an atheist but this bible passage speaks volumes about how the elo of everyone behaved after NNUE:

For whoever has will be given more, and they will have an abundance. Whoever does not have, even what they have will be taken from them. Matthew 25:29

Post Reply