Stockfish NNUE

[MikeB]

I solved all those problems by making a folder named N:\eval and placing all bin files there. Then in the UCI options I set the N:\eval\binfilename.bin

It's "N:" because I don't have a drive c: on my machine. I assume on most windows machines it would be on drive c:

I haven't tried to to set it up in Cutechess-cli yet. I guess the next step for me. Anyone have a clue how to do this. i.e. an example file....


Regards,

Zenmastur

A shell script to run match between two Stockfish NNUE nets. Install Git Bash for windows and run it under a Gut bash terminal window

name this file run.sh and place it cutechess-cli.exe folder
and then give it permissions
chmod +755 run.sh

Code: Select all

#!/bin/bash

### Allows you to run exe's under Git Bash without the "./" preceding the exe name
export PATH=$PATH:.

## USER OPTIONS
base=6
ROUNDS=5  # total num of games will be ( num of engines * (num of engines -1) * rounds
CONCUR=50   ## reflects num of comcurrent games . THREADS * CONCUR should ALWAYS be less than # of logical CPUS

### ENGINE options
ENG2="1134"
ENGCMD1="Stockfish-XI-NN.exe"
EVFILE1="./eval/20200723-1134.bin"

ENG1="2344"
ENGCMD2="Stockfish-XI-NN.exe"
EVFILE2="./eval/20200724-2344.bin"

###TOURNAMENT OPTIONS
### common options -each
DIR="c:/cluster.mfb/Popcnt-LP"
HASH=128
SYZYGY="option.SyzygyPath=c:/syzygy"
THREADS=1

DB="-debug"
OFILE="C:/cluster.mfb/Popcnt-LP/books/NBSC_30k_5mvs.epd"
ORDER="random"   # other option -> sequential
TOUR="round-robin"
pgn="c:/cluster.mfb/pgn/$DATE$ENG1$ENG2.pgn";



### hack to use floating point(fp) in bash (bc not available/working in Git basb), but it does screw up the Atom auto color, needed for calc of increment : inc= base/100 or if base is 10 seconds , 10/100=0.1 etc.
inc=$(printf %.2f "$(((10**3 * base)/100))e-3"); #echo $inc
## "$(printf %.2f "$((()/1))e-3"
TC="$base+$inc"

cutechess-cli   -repeat -rounds $ROUNDS -games 2 -tournament $TOUR -tb c:/syzygy -tbpieces 6 -resign movecount=1 score=700 twosided=true -draw movenumber=40 movecount=10 score=2 -concurrency $CONCUR -openings file=$OFILE format=epd order=$ORDER plies=16 -engine name=$ENG1 cmd=$ENGCMD1 dir=$DIR  option.EvalFile=$EVFILE1 -engine name=$ENG2 dir=$DIR cmd=$ENGCMD2 option.EvalFile=$EVFILE2 -each tc=$TC proto=uci option.Threads=$THREADS  option.Hash=$HASH $SYZYGY -pgnout $pgn 2>/dev/null

cutechess-cli does have erroneous error reporting - I silenced these erroneous errors by adding "2>/dev/null" at the end - but if you're having issues getting this to run. Run the cutechess-cli with the -debug flag "cutechess-cli -debug -repeat -rounds $ROUNDS ..." and remove the "2>/dev/null" at the end to see why it is not running correctly.

This script works, example output with 6 sec base and 5 rounds - 10 games in total :

Code: Select all

$ ./run.sh
Indexing opening suite...
Started game 4 of 10 (1134 vs 2344)
Started game 10 of 10 (1134 vs 2344)
Started game 2 of 10 (1134 vs 2344)
Started game 3 of 10 (2344 vs 1134)
Started game 9 of 10 (2344 vs 1134)
Started game 7 of 10 (2344 vs 1134)
Started game 1 of 10 (2344 vs 1134)
Started game 6 of 10 (1134 vs 2344)
Started game 8 of 10 (1134 vs 2344)
Started game 5 of 10 (2344 vs 1134)
Finished game 1 (2344 vs 1134): 1/2-1/2 {Draw by adjudication: SyzygyTB}
Score of 2344 vs 1134: 0 - 0 - 1  [0.500] 1
Finished game 6 (1134 vs 2344): 1-0 {White wins by adjudication}
Score of 2344 vs 1134: 0 - 1 - 1  [0.250] 2
Finished game 7 (2344 vs 1134): 1/2-1/2 {Draw by adjudication}
Score of 2344 vs 1134: 0 - 1 - 2  [0.333] 3
Finished game 5 (2344 vs 1134): 1/2-1/2 {Draw by adjudication: SyzygyTB}
Score of 2344 vs 1134: 0 - 1 - 3  [0.375] 4
Finished game 9 (2344 vs 1134): 0-1 {Black wins by adjudication}
Score of 2344 vs 1134: 0 - 2 - 3  [0.300] 5
Finished game 8 (1134 vs 2344): 0-1 {Black wins by adjudication}
Score of 2344 vs 1134: 1 - 2 - 3  [0.417] 6
Finished game 2 (1134 vs 2344): 1/2-1/2 {Draw by adjudication: SyzygyTB}
Score of 2344 vs 1134: 1 - 2 - 4  [0.429] 7
Finished game 10 (1134 vs 2344): 0-1 {Black wins by adjudication}
Score of 2344 vs 1134: 2 - 2 - 4  [0.500] 8
Finished game 3 (2344 vs 1134): 1/2-1/2 {Draw by adjudication}
Score of 2344 vs 1134: 2 - 2 - 5  [0.500] 9
Finished game 4 (1134 vs 2344): 1-0 {White wins by adjudication}
Score of 2344 vs 1134: 2 - 3 - 5  [0.450] 10
...      2344 playing White: 0 - 1 - 4  [0.400] 5
...      2344 playing Black: 2 - 2 - 1  [0.500] 5
...      White vs Black: 2 - 3 - 5  [0.450] 10
Elo difference: -34.9 +/- 163.5, LOS: 32.7 %, DrawRatio: 50.0 %
Finished match

Then if you want to get fancier and run it with additional script to report the results in a more meaningful way on a periodic basis, you can try the cadillac version...

http://talkchess.com/forum3/viewtopic.php?f=2&t=74566

In addition to running the match under cutechess-cli, you will get this type of reporting in a separate terminal window on periodic basis

Code: Select all

pgn file: c:/cluster.mfb/pgn/2007251425-23441134.pgn
tc/base+inc: 30+0.30
games planned: 4000
Threads: 1
Hash: 128

Current date : time (EDST)
Date: 07/25/20 : 16:35:44

Projected-> Time: 2h:8m:0s
Running  -> Time: 2h:10m:11s

4000 game(s) loaded
Rank Name  Rating   Δ     +    -     #     Σ    Σ%     W    L    D   W%    =%   OppR
---------------------------------------------------------------------------------------------------------

   1 2344   3501   0.0    7    7  4000 2013.5  50.3  973  946 2081  24.3  52.0  3499
   2 1134   3499   2.4    7    7  4000 1986.5  49.7  946  973 2081  23.6  52.0  3501
---------------------------------------------------------------------------------------------------------

  Δ = delta from the next higher rated opponent
  # = number of games played
  Σ = total score, 1 point for win, 1/2 point for draw

LOS:
      23 11
2344     74
1134  25

4000 game(s) loaded

loops/scheduled: 114/114

waiting: 64
  ...seconds remaining:   0

done

[Lion]

I tried this, but even if there is no eval.bin file, the engine would still work..... thus I have no clue which eval file it uses.

Rgds

[MikeB]

I tried this, but even if there is no eval.bin file, the engine would still work..... thus I have no clue which eval file it uses.

Rgds

if you turn the debug or logging function on, you will see if the file is loaded or not found at startup.

[Zenmastur]

Hello

Yes but there is no clear full pass such as
C:\Users\David\Documents\ChessEngines\StockfishNull\Stockfish-NNUE\eval

but just something like eval/eval.bin

rgds

If it's .\eval\eval.bin then rename your net file eval.bin and put in that eval folder. That or edit the current path to whatever you prefer.

The problem is that even if I errase all eval.bin files in the folder in question, Stockfish NNUE still works/calculate...... so my assumption is that it is not there => C:\Users\David\Documents\ChessEngines\StockfishNull\Stockfish-NNUE\eval

Yes, so does mine. But it prefers 1. a3 to start the game, for example:

Code: Select all

$ ./stockfish+nnue 
StockfishNNUE 200720 64 BMI2 by StockfishTeam

Compiled by g++ (GNUC) 10.1.0 on Linux
 __VERSION__ macro expands to: 10.1.0

ucinewgame
Error! ./eval/nn.bin not found or wrong format
go infinite
info depth 1 seldepth 1 multipv 1 score cp 0 nodes 20 nps 20000 tbhits 0 time 1 pv a2a3
info depth 2 seldepth 2 multipv 1 score cp 0 nodes 61 nps 30500 tbhits 0 time 2 pv a2a3 a7a6
info depth 3 seldepth 3 multipv 1 score cp 0 nodes 112 nps 56000 tbhits 0 time 2 pv a2a3 a7a6 a3a4
info depth 4 seldepth 4 multipv 1 score cp 0 nodes 220 nps 110000 tbhits 0 time 2 pv a2a3 a7a6 a3a4 a6a5
info depth 5 seldepth 5 multipv 1 score cp 0 nodes 418 nps 209000 tbhits 0 time 2 pv a2a3 a7a6 a3a4 a6a5 a1a3
info depth 6 seldepth 6 multipv 1 score cp 0 nodes 674 nps 337000 tbhits 0 time 2 pv a2a3 a7a6 a3a4 a6a5 a1a3 g8h6
info depth 7 seldepth 7 multipv 1 score cp 0 nodes 1002 nps 334000 tbhits 0 time 3 pv a2a3 a7a6 a3a4 a6a5 a1a3 g8h6 a3b3

a3 as the first move means your .bin file isn't loading for some reason. When I finally got my to load properly the a3 moves stopped showing up!
They should print a message which bin file has load if any and if none has loaded it should give an error message. I think a lot of people are having this problem and don't even know it.

[zullil]

Hello

Yes but there is no clear full pass such as
C:\Users\David\Documents\ChessEngines\StockfishNull\Stockfish-NNUE\eval

but just something like eval/eval.bin

rgds

If it's .\eval\eval.bin then rename your net file eval.bin and put in that eval folder. That or edit the current path to whatever you prefer.

The problem is that even if I errase all eval.bin files in the folder in question, Stockfish NNUE still works/calculate...... so my assumption is that it is not there => C:\Users\David\Documents\ChessEngines\StockfishNull\Stockfish-NNUE\eval

Yes, so does mine. But it prefers 1. a3 to start the game, for example:

Code: Select all

$ ./stockfish+nnue 
StockfishNNUE 200720 64 BMI2 by StockfishTeam

Compiled by g++ (GNUC) 10.1.0 on Linux
 __VERSION__ macro expands to: 10.1.0

ucinewgame
Error! ./eval/nn.bin not found or wrong format
go infinite
info depth 1 seldepth 1 multipv 1 score cp 0 nodes 20 nps 20000 tbhits 0 time 1 pv a2a3
info depth 2 seldepth 2 multipv 1 score cp 0 nodes 61 nps 30500 tbhits 0 time 2 pv a2a3 a7a6
info depth 3 seldepth 3 multipv 1 score cp 0 nodes 112 nps 56000 tbhits 0 time 2 pv a2a3 a7a6 a3a4
info depth 4 seldepth 4 multipv 1 score cp 0 nodes 220 nps 110000 tbhits 0 time 2 pv a2a3 a7a6 a3a4 a6a5
info depth 5 seldepth 5 multipv 1 score cp 0 nodes 418 nps 209000 tbhits 0 time 2 pv a2a3 a7a6 a3a4 a6a5 a1a3
info depth 6 seldepth 6 multipv 1 score cp 0 nodes 674 nps 337000 tbhits 0 time 2 pv a2a3 a7a6 a3a4 a6a5 a1a3 g8h6
info depth 7 seldepth 7 multipv 1 score cp 0 nodes 1002 nps 334000 tbhits 0 time 3 pv a2a3 a7a6 a3a4 a6a5 a1a3 g8h6 a3b3

a3 as the first move means your .bin file isn't loading for some reason. When I finally got my to load properly the a3 moves stopped showing up!
They should print a message which bin file has load if any and if none has loaded it should give an error message. I think a lot of people are having this problem and don't even know it.

Yes, I know this. I was trying to help the OP, and intentionally moved my nn.bin file---so that he could see that, even without such a file, there will be output from the engine.

I've tried my best to help, but it doesn't seem to be reaching him.

EDIT---And there is an error message; it's right there is my pasted output. But I guess GUI's won't show that.

[zullil]

I tried this, but even if there is no eval.bin file, the engine would still work..... thus I have no clue which eval file it uses.

Rgds

Yes, it will still work. Who cares why right now? Does it give you 1. a3 as its move?

Just put your eval.bin in the eval folder and see if you get a sensible first move!

[Dann Corbit]

I don't know about the fritz gui, but in Arena, I just browse to the file.
That way you know for sure it is pointed at the right thing and it found it.
And when I start the engine, it says that it loaded the file on the screen.

[carldaman]

I've got the same problem with ShashChessNNUE in Arena, it won't load the nn.bin file even if the path is correct. It wants to play a3 with a 0.00 eval.

[frankp]

Struggling to use this in Linux.
Downloaded and compiled Stockfish-NNUE-master from git hub.
The download contains nn.bin files, in nnueNets/20MB/eval/nn.bin and nnueNets/30MB/eval/nn.bin
Stockfish does not automatically pick these up.
If I type (in a console) setoption name EvalFile name <pathname>
I get
No such option: EvalFile

Similarly uci does not list EvalFile as an option
//------------------------------------------------------------------------------------------------
uci
id name StockfishNNUE 260720 64 POPCNT
id author StockfishTeam

option name Debug Log File type string default
option name Contempt type spin default 24 min -100 max 100
option name Analysis Contempt type combo default Both var Off var White var Black var Both
option name Threads type spin default 1 min 1 max 512
option name Hash type spin default 16 min 1 max 33554432
option name Clear Hash type button
option name Ponder type check default false
option name MultiPV type spin default 1 min 1 max 500
option name Skill Level type spin default 20 min 0 max 20
option name Move Overhead type spin default 10 min 0 max 5000
option name Slow Mover type spin default 100 min 10 max 1000
option name nodestime type spin default 0 min 0 max 10000
option name UCI_Chess960 type check default false
option name UCI_AnalyseMode type check default false
option name UCI_LimitStrength type check default false
option name UCI_Elo type spin default 1350 min 1350 max 2850
option name UCI_ShowWDL type check default false
option name SyzygyPath type string default <empty>
option name SyzygyProbeDepth type spin default 1 min 1 max 100
option name Syzygy50MoveRule type check default true
option name SyzygyProbeLimit type spin default 7 min 0 max 7
uciok
//--------------------------------------------------------------------------------------------------------

Confused.

[zullil]

Struggling to use this in Linux.
Downloaded and compiled Stockfish-NNUE-master from git hub.
The download contains nn.bin files, in nnueNets/20MB/eval/nn.bin and nnueNets/30MB/eval/nn.bin
Stockfish does not automatically pick these up.
If I type (in a console) setoption name EvalFile name <pathname>
I get
No such option: EvalFile

Similarly uci does not list EvalFile as an option
//------------------------------------------------------------------------------------------------
uci
id name StockfishNNUE 260720 64 POPCNT
id author StockfishTeam

option name Debug Log File type string default
option name Contempt type spin default 24 min -100 max 100
option name Analysis Contempt type combo default Both var Off var White var Black var Both
option name Threads type spin default 1 min 1 max 512
option name Hash type spin default 16 min 1 max 33554432
option name Clear Hash type button
option name Ponder type check default false
option name MultiPV type spin default 1 min 1 max 500
option name Skill Level type spin default 20 min 0 max 20
option name Move Overhead type spin default 10 min 0 max 5000
option name Slow Mover type spin default 100 min 10 max 1000
option name nodestime type spin default 0 min 0 max 10000
option name UCI_Chess960 type check default false
option name UCI_AnalyseMode type check default false
option name UCI_LimitStrength type check default false
option name UCI_Elo type spin default 1350 min 1350 max 2850
option name UCI_ShowWDL type check default false
option name SyzygyPath type string default <empty>
option name SyzygyProbeDepth type spin default 1 min 1 max 100
option name Syzygy50MoveRule type check default true
option name SyzygyProbeLimit type spin default 7 min 0 max 7
uciok
//--------------------------------------------------------------------------------------------------------

Confused.

You likely didn't use the correct Make target. Do 'make clean' and then 'make help'.

Look for a target that contains nnue!

[EDIT] Nevermind. Too many Stockfish-nnue repos already. Make help has not be updated at the repo you went to.

In any case, try

make profile-nnue ARCH=x86-64-[your choice here]