this will be the merge of a lifetime : SF 80 Elo+

[zullil]

And now 6.8. NNUE is merged: Stockfish 060820 by the Stockfish developers (see AUTHORS file)

Code: Select all

$ ./stockfish 
Stockfish 060820 by the Stockfish developers (see AUTHORS file)
bench

Position: 1/47
info string classical evaluation enabled.
info depth 1 seldepth 1 multipv 1 score cp 110 nodes 20 nps 20000 tbhits 0 time 1 pv e2e3
info depth 2 seldepth 2 multipv 1 score cp 122 nodes 56 nps 56000 tbhits 0 time 1 pv e2e3 b7b6
info depth 3 seldepth 3 multipv 1 score cp 116 nodes 150 nps 150000 tbhits 0 time 1 pv e2e3 b7b6 g1e2
info depth 4 seldepth 4 multipv 1 score cp 67 nodes 479 nps 239500 tbhits 0 time 2 pv e2e3 h7h6 b2b3 e7e6
info depth 5 seldepth 5 multipv 1 score cp 63 nodes 989 nps 494500 tbhits 0 time 2 pv e2e3 e7e6 d2d4 d7d5
info depth 6 seldepth 6 multipv 1 score cp 106 nodes 1161 nps 580500 tbhits 0 time 2 pv e2e3 e7e6 d2d4
info depth 7 seldepth 7 multipv 1 score cp 114 nodes 2808 nps 702000 tbhits 0 time 4 pv b1c3 e7e6 g1f3 d7d6 d2d4 d6d5
info depth 8 seldepth 11 multipv 1 score cp 110 nodes 10224 nps 1136000 tbhits 0 time 9 pv e2e4 d7d5 e4d5 g8f6 b1c3 f6d5 d2d4
info depth 9 seldepth 12 multipv 1 score cp 114 nodes 15801 nps 1215461 tbhits 0 time 13 pv e2e4 d7d5 e4d5 d8d5 b1c3 d5a5 g1f3 g8f6 d2d4
info depth 10 seldepth 14 multipv 1 score cp 109 nodes 25231 nps 1261550 tbhits 0 time 20 pv e2e4 d7d5 e4d5 d8d5 b1c3 d5a5 d2d4 e7e5 d4e5 b8c6 g1f3
info depth 11 seldepth 18 multipv 1 score cp 123 nodes 36205 nps 1340925 tbhits 0 time 27 pv e2e4 d7d5 e4d5 d8d5 b1c3 d5a5 d2d4 e7e5 d4e5 a5e5 d1e2 e5e2 g1e2 g8f6 c1f4
info depth 12 seldepth 20 multipv 1 score cp 101 nodes 98957 nps 1434159 tbhits 0 time 69 pv e2e4 e7e6 d2d4 d7d5 e4e5 c7c5 c2c3 c5d4 c3d4 d8a5 c1d2 f8b4 d2b4 a5b4 b1c3
info depth 13 seldepth 21 multipv 1 score cp 67 nodes 159227 nps 1488102 tbhits 0 time 107 pv e2e4 e7e6 d2d4 d7d5 e4d5 e6d5 d1e2 d8e7 b1c3 c7c6 c1g5 e7e2 f1e2 c8f5 e1c1 b8d7 g1f3
bestmove e2e4 ponder e7e6

[snip]

Position: 47/47
info string classical evaluation enabled.
info depth 1 seldepth 1 multipv 1 score cp 94 nodes 36 nps 36000 tbhits 0 time 1 pv d1e3
info depth 2 seldepth 2 multipv 1 score cp 1376 nodes 71 nps 71000 tbhits 0 time 1 pv d1e3 h3d7 e7d7
info depth 3 seldepth 3 multipv 1 score cp 1376 nodes 106 nps 106000 tbhits 0 time 1 pv d1e3 h3d7 e7d7
info depth 4 seldepth 4 multipv 1 score cp 122 nodes 268 nps 268000 tbhits 0 time 1 pv d1e3 d8f7 c2c3 b7b6
info depth 5 seldepth 5 multipv 1 score cp 32 nodes 904 nps 452000 tbhits 0 time 2 pv e1g2 h3f5 h2h4 d8c6 e7e3
info depth 6 seldepth 6 multipv 1 score cp 12 nodes 1633 nps 816500 tbhits 0 time 2 pv b2b3 b7b6 e1g2 d8c6 e7e3 h3d7
info depth 7 seldepth 8 multipv 1 score cp 27 nodes 1969 nps 984500 tbhits 0 time 2 pv b2b3 b7b6 e1g2 h3g4 g2f4 d8f7
info depth 8 seldepth 11 multipv 1 score cp 23 nodes 5548 nps 1109600 tbhits 0 time 5 pv b2b3 b7b6 e7e3 d8e6 c2c3 c7c5 a1b2
info depth 9 seldepth 12 multipv 1 score cp 21 nodes 12822 nps 1424666 tbhits 0 time 9 pv e7e3 b7b6 b2b3 a8b7 c2c3 d8e6 e3d3 c7c5
info depth 10 seldepth 12 multipv 1 score cp 73 nodes 15232 nps 1384727 tbhits 0 time 11 pv e7e3 b7b6 b2b3 f8e8 e3d3 c7c5 d1e3
info depth 11 seldepth 17 multipv 1 score cp 50 nodes 42348 nps 1512428 tbhits 0 time 28 pv e7e3 d8f7 c2c3 h3d7 e1g2 b7b6 e3d3 c7c5 d1e3 c5c4 d3d1
info depth 12 seldepth 19 multipv 1 score cp 5 nodes 81779 nps 1543000 tbhits 0 time 53 pv e7e3 b7b6 b2b4 c7c6 c2c3 a8b7 e1g2 h3d7 e3d3 g7g6 d1e3 g8g7
info depth 13 seldepth 21 multipv 1 score cp 34 nodes 102738 nps 1580584 tbhits 0 time 65 pv e7e3 b7b6 c2c3 c7c6 b2b3 g7g6 e3f3 g8g7 d1e3 a8b7 e1d3 d8f7
bestmove e7e3 ponder b7b6

===========================
Total time (ms) : 2302
Nodes searched  : 4746616
Nodes/second    : 2061953
setoption name Use NNUE value true
bench

Position: 1/47
info string NNUE evaluation using nn-97f742aaefcd.nnue enabled.
info depth 1 seldepth 1 multipv 1 score cp 26 nodes 20 nps 20000 tbhits 0 time 1 pv d2d4
info depth 2 seldepth 2 multipv 1 score cp 52 nodes 47 nps 23500 tbhits 0 time 2 pv d2d4 a7a6
info depth 3 seldepth 3 multipv 1 score cp 41 nodes 186 nps 93000 tbhits 0 time 2 pv e2e4 g7g6 d2d4
info depth 4 seldepth 4 multipv 1 score cp 30 nodes 665 nps 332500 tbhits 0 time 2 pv g1f3 c7c5 e2e4 a7a6
info depth 5 seldepth 5 multipv 1 score cp 30 nodes 981 nps 327000 tbhits 0 time 3 pv g1f3 c7c5 e2e4 a7a6 d2d4 c5d4 d1d4
info depth 6 seldepth 6 multipv 1 score cp 18 nodes 2221 nps 555250 tbhits 0 time 4 pv e2e4 c7c5 d2d4 c5d4 d1d4 e7e6
info depth 7 seldepth 10 multipv 1 score cp 10 nodes 5192 nps 649000 tbhits 0 time 8 pv e2e4 c7c5 b1c3 b8c6 g2g3 e7e6 f1g2
info depth 8 seldepth 10 multipv 1 score cp 11 nodes 9061 nps 755083 tbhits 0 time 12 pv e2e3 d7d5 d2d4 e7e6 c2c4 g8f6 b1c3 c7c5 c4d5
info depth 9 seldepth 9 multipv 1 score cp 16 nodes 13116 nps 771529 tbhits 0 time 17 pv c2c4 e7e6 g1f3 g8f6 g2g3 d7d5 c4d5 e6d5 f1g2
info depth 10 seldepth 14 multipv 1 score cp 28 nodes 19608 nps 784320 tbhits 0 time 25 pv c2c4 e7e6 d2d4 d7d5 b1c3 g8f6 g1f3 d5c4 e2e3 c7c5 f1c4 c5d4
info depth 11 seldepth 17 multipv 1 score cp 26 nodes 33601 nps 819536 tbhits 0 time 41 pv e2e4 c7c5 g1f3 e7e6 d2d4 c5d4 f3d4 b8c6 f1e2 g8f6 b1c3
info depth 12 seldepth 19 multipv 1 score cp 19 nodes 58852 nps 828901 tbhits 0 time 71 pv e2e4 c7c5 g1f3 e7e6 b1c3 b8c6 d2d4 c5d4 f3d4 d7d6 d4c6 b7c6 e4e5 d6e5 d1d8 e8d8
info depth 13 seldepth 23 multipv 1 score cp 16 nodes 97949 nps 851730 tbhits 0 time 115 pv e2e4 c7c5 g1f3 e7e6 f1e2 d7d5 e4d5 e6d5 e1g1 f8e7 d2d4 g8f6 d4c5 e7c5 f1e1 e8g8 b1c3 b8c6
bestmove e2e4 ponder c7c5

[snip]

Position: 47/47
info string NNUE evaluation using nn-97f742aaefcd.nnue enabled.
info depth 1 seldepth 1 multipv 1 score cp 83 nodes 37 nps 37000 tbhits 0 time 1 pv e7e3
info depth 2 seldepth 2 multipv 1 score cp 1063 nodes 73 nps 73000 tbhits 0 time 1 pv e7e3 h3g2 e1g2
info depth 3 seldepth 3 multipv 1 score cp 93 nodes 164 nps 82000 tbhits 0 time 2 pv e7e3 d8c6 c2c3
info depth 4 seldepth 4 multipv 1 score cp 560 nodes 236 nps 118000 tbhits 0 time 2 pv d1c3 d8c6
info depth 5 seldepth 5 multipv 1 score cp 73 nodes 689 nps 344500 tbhits 0 time 2 pv d1c3 h3c8 g1g2 f6f5 e7g5
info depth 6 seldepth 6 multipv 1 score cp 57 nodes 1302 nps 434000 tbhits 0 time 3 pv e7e3 d8f7 e3d3 f8e8 d1c3
info depth 7 seldepth 7 multipv 1 score cp 84 nodes 1558 nps 519333 tbhits 0 time 3 pv e7e3 d8f7 e3d3 f8e8 d1c3
info depth 8 seldepth 9 multipv 1 score cp 153 nodes 2345 nps 586250 tbhits 0 time 4 pv e7e3 d8f7 e3d3 f8e8 e1g2
info depth 9 seldepth 14 multipv 1 score cp 61 nodes 9329 nps 777416 tbhits 0 time 12 pv e7e3 d8f7 e3d3 h3h6 e1g2 g8f8 d1e3 d6e4 f1d1 f7d6 h2h4
info depth 10 seldepth 15 multipv 1 score cp 114 nodes 14356 nps 844470 tbhits 0 time 17 pv e7e3 d8f7 e3d3 h3h6 e1g2 g8f8 d1e3
info depth 11 seldepth 16 multipv 1 score cp 40 nodes 31873 nps 861432 tbhits 0 time 37 pv e7e3 f8e8 e3d3 d8e6 c2c3 g8h8 e1g2 f6f5 f2f3 h3h5 d1f2 c7c5 d4c5
info depth 12 seldepth 20 multipv 1 score cp 41 nodes 60115 nps 871231 tbhits 0 time 69 pv e7e3 f8e8 e3d3 d8e6 e1g2 g8h8 f2f3 f6f5 d1e3 c7c6 g1f1 h3h6 a2a4 f5f4 g2f4 e6f4
info depth 13 seldepth 22 multipv 1 score cp 28 nodes 108152 nps 879284 tbhits 0 time 123 pv e7e3 f8e8 e3f3 c7c6 c2c3 d6c4 f3d3 g8h8 d1e3 c4e3 f2e3 f6f5 b1c2 d8e6 e1g2 c6c5 d4c5
bestmove e7e3 ponder f8e8

===========================
Total time (ms) : 3918
Nodes searched  : 4254913
Nodes/second    : 1085991
quit

So the speed of the NNUE eval is about 53% of the speed of the classical eval. At least on my system (bmi2, avx2).

[Leto]

A new patch has been released, it appears that Stockfish is now a hybrid engine, it plays as Stockfish NNUE during quiet balanced material positions, and as Stockfish during all other positions.

[zullil]

A new patch has been released, it appears that Stockfish is now a hybrid engine, it plays as Stockfish NNUE during quiet balanced material positions, and as Stockfish during all other positions.

Correct. And the user has no way of choosing to use just the NN evaluation (but can choose to use just the classical evaluation).

[Leto]

Seems to me there should be options to enable NNUE, run only Stockfish, and enable the Stockfish + NNUE hybryd.

[zullil]

Seems to me there should be options to enable NNUE, run only Stockfish, and enable the Stockfish + NNUE hybryd.

See comments at https://github.com/official-stockfish/S ... 5aac9e322f

Note: The eval command (static evaluation) strictly respects the Use NNUE setting:

Code: Select all

$ ./stockfish 
Stockfish 060820 by the Stockfish developers (see AUTHORS file)
d

 +---+---+---+---+---+---+---+---+
 | r | n | b | q | k | b | n | r | 8
 +---+---+---+---+---+---+---+---+
 | p | p | p | p | p | p | p | p | 7
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   |   |   | 6
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   |   |   | 5
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   |   |   | 4
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   |   |   | 3
 +---+---+---+---+---+---+---+---+
 | P | P | P | P | P | P | P | P | 2
 +---+---+---+---+---+---+---+---+
 | R | N | B | Q | K | B | N | R | 1
 +---+---+---+---+---+---+---+---+
   a   b   c   d   e   f   g   h

Fen: rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1
Key: 8F8F01D4562F59FB
Checkers: 
eval
info string classical evaluation enabled.

     Term    |    White    |    Black    |    Total   
             |   MG    EG  |   MG    EG  |   MG    EG 
 ------------+-------------+-------------+------------
    Material |  ----  ---- |  ----  ---- |  0.00  0.00
   Imbalance |  ----  ---- |  ----  ---- |  0.00  0.00
       Pawns |  0.54 -0.12 |  0.54 -0.12 |  0.00  0.00
     Knights | -0.02 -0.19 | -0.02 -0.19 |  0.00  0.00
     Bishops | -0.03 -0.37 | -0.03 -0.37 |  0.00  0.00
       Rooks | -0.27 -0.06 | -0.27 -0.06 |  0.00  0.00
      Queens |  0.00  0.00 |  0.00  0.00 |  0.00  0.00
    Mobility | -0.83 -1.18 | -0.83 -1.18 |  0.00  0.00
 King safety |  0.81 -0.05 |  0.81 -0.05 |  0.00  0.00
     Threats |  0.00  0.00 |  0.00  0.00 |  0.00  0.00
      Passed |  0.00  0.00 |  0.00  0.00 |  0.00  0.00
       Space |  0.41  0.00 |  0.41  0.00 |  0.00  0.00
    Winnable |  ----  ---- |  ----  ---- |  0.00  0.00
 ------------+-------------+-------------+------------
       Total |  ----  ---- |  ----  ---- |  0.00  0.00

Final evaluation: 0.14 (white side)

setoption name Use NNUE value true
eval
info string NNUE evaluation using nn-97f742aaefcd.nnue enabled.


Final evaluation: 0.24 (white side)

quit

[gladius]

And now 6.8. NNUE is merged: Stockfish 060820 by the Stockfish developers (see AUTHORS file)

The power of machine learning and neural nets continues to surprise! Finding the right way to apply them to the problem can be extremely non-trivial, but when you do, look out! The NNUE approach is extremely inventive. I can't wait to see how it gets extended in the future as well.

I prototyped a bitboard mobility feature as an input https://github.com/glinscott/Stockfish/ ... bility.cpp, but it was way too expensive to make it worth it. However, if you plug mobility into a much smaller side net, and then have that as input in the fully-connected layers, that could work potentially. Exciting times ahead .

[mwyoung]

And now 6.8. NNUE is merged: Stockfish 060820 by the Stockfish developers (see AUTHORS file)

The power of machine learning and neural nets continues to surprise! Finding the right way to apply them to the problem can be extremely non-trivial, but when you do, look out! The NNUE approach is extremely inventive. I can't wait to see how it gets extended in the future as well.

I prototyped a bitboard mobility feature as an input https://github.com/glinscott/Stockfish/ ... bility.cpp, but it was way too expensive to make it worth it. However, if you plug mobility into a much smaller side net, and then have that as input in the fully-connected layers, that could work potentially. Exciting times ahead .

The new hybrid (Stockfish 060820) is doing well against SF NNUE after 33 games. Up +2 games, and + 20 Elo.

Stockfish NNUE (Sergio 1817) vs Stockfish 060820 (Sergio 1817) (32 Thread Test) (TC = 3m+2s)

[M ANSARI]

This makes sense but I don't think this is the optimal way to go about this. Of course in quiet positions 50% less nps is not as critical as in the endgame, but I believe even in the endgame the NN evaluation can have a better evaluation in many positions. At the moment this seems like a band aid fix to low nps but I feel that optimized networks for different parts of the game that can be changed on the fly is a more productive method.

[Ovyron]

The new hybrid (Stockfish 060820) is doing well against SF NNUE after 33 games. Up +2 games, and + 20 Elo.

Would this translate to +120 against Stockfish 11 because SF NNUE was already at +100?

I said after 1 month +50 elo over Stockfish NNUE as a joke, but if a simple idea like only using NNUE some of the time bridges the gap to 30 elo...

Will Stockfish 12 be 200 elo stronger than Stockfish 11? And if so, when? One month ago I'd have said around Spring 2024. Who's willing to bet 2020???

[Laskos]

https://tests.stockfishchess.org/tests/ ... 64f05e4c58

I think this qualifies for SF 12!!

Using an unbalanced opening book built from 2moves_v1 to decrease the draw rate, I am getting about 150 Elo points advantage of the best SV nets and latest SF compile over SF_dev at 15s + 0.25s time control.

Complete destruction of SF_dev, Win/Loss ratio of more than 3.

Code: Select all

Games Completed = 100 of 100 (Avg game length = 61.968 sec)
Settings = Gauntlet/128MB/15000ms+250ms/M 700cp for 3 moves, D 120 moves/EPD:C:\LittleBlitzer\2moves_80_100.epd(1749)
Time = 1593 sec elapsed, 0 sec remaining
 1.  SFNNUE 0633 SF binary    	70.0/100	57-17-26  	(L: m=0 t=0 i=0 a=17)	(D: r=13 i=4 f=1 s=0 a=8)	(tpm=443.3 d=19.28 nps=1128817)
 2.  SF_dev                   	30.0/100	17-57-26  	(L: m=0 t=0 i=0 a=57)	(D: r=13 i=4 f=1 s=0 a=8)	(tpm=447.8 d=20.01 nps=1616564)