Stockfish NNUE (Sergio 1817) vs Stockfish 060820 (Sergio 1817) (32 Thread Test) (TC = 3m+2s)
Hardware 2950x, RTX 2080 ti
TC=3m+2s.
Ponder off.
32 threads.
16 Gb hash.
6 man TB.
6 move book
Default settings
Stockfish 060820 changes:
"Author: MJZ1977
Date: Thu Aug 6 21:46:31 2020 +0200
Timestamp: 1596743191
NNUE evaluation threshold
The idea is to use NNUE only on quite balanced material positions. This bring a big speedup on research since NNUE eval is slower than classical eval for most of the hardwares and specially on unbalanced positions with LazyEval.
STC: https://tests.stockfishchess.org/tests/ ... cbfee85b61
LLR: 2.95 (-2.94,2.94) {-0.50,1.50}
Total: 3168 W: 560 L: 400 D: 2208 Elo +17.56
Ptnml(0-2): 21, 294, 819, 404, 46
LTC: https://tests.stockfishchess.org/tests/ ... cbfee85b69
LLR: 2.98 (-2.94,2.94) {0.25,1.75}
Total: 3200 W: 287 L: 183 D: 2730 Elo +11.30
Ptnml(0-2): 4, 149, 1191, 251, 5
closes https://github.com/official-stockfish/S ... /pull/2916
Bench 4746616
Author: nodchip
Date: Thu Aug 6 16:37:45 2020 +0200
Timestamp: 1596724665
Add NNUE evaluation
This patch ports the efficiently updatable neural network (NNUE) evaluation to Stockfish.
Both the NNUE and the classical evaluations are available, and can be used to
assign a value to a position that is later used in alpha-beta (PVS) search to find the
best move. The classical evaluation computes this value as a function of various chess
concepts, handcrafted by experts, tested and tuned using fishtest. The NNUE evaluation
computes this value with a neural network based on basic inputs. The network is optimized
and trained on the evalutions of millions of positions at moderate search depth.
The NNUE evaluation was first introduced in shogi, and ported to Stockfish afterward.
It can be evaluated efficiently on CPUs, and exploits the fact that only parts
of the neural network need to be updated after a typical chess move.
[The nodchip repository](https://github.com/nodchip/Stockfish) provides additional
tools to train and develop the NNUE networks.
This patch is the result of contributions of various authors, from various communities,
including: nodchip, ynasu87, yaneurao (initial port and NNUE authors), domschl, FireFather,
rqs, xXH4CKST3RXx, tttak, zz4032, joergoster, mstembera, nguyenpham, erbsenzaehler,
dorzechowski, and vondele.
This new evaluation needed various changes to fishtest and the corresponding infrastructure,
for which tomtor, ppigazzini, noobpwnftw, daylen, and vondele are gratefully acknowledged.
The first networks have been provided by gekkehenker and sergiovieri, with the latter
net (nn-97f742aaefcd.nnue) being the current default.
The evaluation function can be selected at run time with the `Use NNUE` (true/false) UCI option,
provided the `EvalFile` option points the the network file (depending on the GUI, with full path).
The performance of the NNUE evaluation relative to the classical evaluation depends somewhat on
the hardware, and is expected to improve quickly, but is currently on > 80 Elo on fishtest:
60000 @ 10+0.1 th 1
https://tests.stockfishchess.org/tests/ ... 64f05e4c4c
ELO: 92.77 +-2.1 (95%) LOS: 100.0%
Total: 60000 W: 24193 L: 8543 D: 27264 Elo +92.77
Ptnml(0-2): 609, 3850, 9708, 10948, 4885
40000 @ 20+0.2 th 8
https://tests.stockfishchess.org/tests/ ... 64f05e4c58
ELO: 89.47 +-2.0 (95%) LOS: 100.0%
Total: 40000 W: 12756 L: 2677 D: 24567 Elo +89.47
Ptnml(0-2): 74, 1583, 8550, 7776, 2017
At the same time, the impact on the classical evaluation remains minimal, causing no significant
regression:
sprt @ 10+0.1 th 1
https://tests.stockfishchess.org/tests/ ... 64f05e4c5b
LLR: 2.94 (-2.94,2.94) {-6.00,-4.00}
Total: 34936 W: 6502 L: 6825 D: 21609 Elo -3.21
Ptnml(0-2): 571, 4082, 8434, 3861, 520
sprt @ 60+0.6 th 1
https://tests.stockfishchess.org/tests/ ... 64f05e4c5d
LLR: 2.93 (-2.94,2.94) {-6.00,-4.00}
Total: 10088 W: 1232 L: 1265 D: 7591 Elo -1.14
Ptnml(0-2): 49, 914, 3170, 843, 68
The needed networks can be found at https://tests.stockfishchess.org/nns
It is recommended to use the default one as indicated by the `EvalFile` UCI option.
Guidelines for testing new nets can be found at
https://github.com/glinscott/fishtest/w ... -net-tests
Integration has been discussed in various issues:
https://github.com/official-stockfish/S ... ssues/2823
https://github.com/official-stockfish/S ... ssues/2728
The integration branch will be closed after the merge:
https://github.com/official-stockfish/S ... /pull/2825
https://github.com/official-stockfish/S ... player-wip
closes https://github.com/official-stockfish/S ... /pull/2912
This will be an exciting time for computer chess, looking forward to seeing the evolution of
this approach.
Bench: 4746616"
Live Stream:
Stockfish NNUE (Sergio 1817) vs Stockfish 060820 (Sergio 1817) (32 Thread Test) (TC = 3m+2s)
Moderators: hgm, Rebel, chrisw
-
- Posts: 2727
- Joined: Wed May 12, 2010 10:00 pm
Stockfish NNUE (Sergio 1817) vs Stockfish 060820 (Sergio 1817) (32 Thread Test) (TC = 3m+2s)
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
But my words like silent raindrops fell. And echoed in the wells of silence.
-
- Posts: 2727
- Joined: Wed May 12, 2010 10:00 pm
Re: Stockfish NNUE (Sergio 1817) vs Stockfish 060820 (Sergio 1817) (32 Thread Test) (TC = 3m+2s)
The new SF crashed. We will try again.
Live Stream:
Live Stream:
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
But my words like silent raindrops fell. And echoed in the wells of silence.
-
- Posts: 2727
- Joined: Wed May 12, 2010 10:00 pm
Re: Stockfish NNUE (Sergio 1817) vs Stockfish 060820 (Sergio 1817) (32 Thread Test) (TC = 3m+2s)
Update 1
Stockfish 060820 (Sergio 1817) is winning after 32 games. +2 =30 -0 +22 Elo.
Stockfish 060820 (Sergio 1817) is winning after 32 games. +2 =30 -0 +22 Elo.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
But my words like silent raindrops fell. And echoed in the wells of silence.