NNUE Research Project

[dkappe]

I am looking for more engines that moved to NNUE.

Anyone?

The data so far is too less for conclusions.

Toga III https://github.com/dkappe/TogaIII/releases

Solly, ignore previous post, real numbers -

SF12 -> RMS=72.63 | SIM=46.22
SF13 -> RMS=72.20 | SIM=45.65

Note that nets trained on non-stockfish data often have to have their scores multiplied by a factor in order to fall into a good range for alpha-beta. Values between 1.5 to 2.5 are common. I use a python script to adjust the final layer for this. The toga net used a 1.55 multiplier, if I recall.

[Madeleine Birchfield]

Note that nets trained on non-stockfish data often have to have their scores multiplied by a factor in order to fall into a good range for alpha-beta. Values between 1.5 to 2.5 are common. I use a python script to adjust the final layer for this. The toga net used a 1.55 multiplier, if I recall.

If eval multipliers affect alpha-beta search, then it would be interesting to test the simex of regular Stockfish 13 against Stockfish 13 with a multiplier of 1.5 or 2.

[dkappe]

Note that nets trained on non-stockfish data often have to have their scores multiplied by a factor in order to fall into a good range for alpha-beta. Values between 1.5 to 2.5 are common. I use a python script to adjust the final layer for this. The toga net used a 1.55 multiplier, if I recall.

If eval multipliers affect alpha-beta search, then it would be interesting to test the simex of regular Stockfish 13 against Stockfish 13 with a multiplier of 1.5 or 2.

1.55 was optimal for the Toga engine based on 1+0.6 play. It might be different for SF. I recall Igel used a pretty hefty factor for night nurse (2.5?).

[Rebel]

I am looking for more engines that moved to NNUE.

Anyone?

The data so far is too less for conclusions.

The latest versions of Pedone, Marvin, Komodo, Scorpio, Halogen, Seer, and Tornado should have NNUE as well.

Pedone 3
SF12 -> RMS=69.88 | SIM=43.26
SF13 -> RMS=70.18 | SIM=42.74

Halogen 10
SF12 -> RMS=70.54 | SIM=49.25
SF13 -> RMS=71.62 | SIM=49.49

Marvin 5
SF12 -> RMS=86.92 | SIM=48.81
SF13 -> RMS=87.44 | SIM=49.76

Seer 1.2.1
SF12 -> RMS=107.47 | SIM=36.27
SF13 -> RMS=108.57 | SIM=36.69

. Tornado supports nnue, but doesn't have a net of its own.
. I saw no Scorpio nnue executable.

[gerhard]

This thread gives help to run Scorpio on NNUE only

http://talkchess.com/forum3/viewtopic.php?f=2&t=76514

Btw: since two days there is a new NNUE Net available on Scorpio Website. Its stored in „nets-nnue.zip“

https://github.com/dshawul/Scorpio/releases - see under Assets

[Madeleine Birchfield]

Pedone 3
SF12 -> RMS=69.88 | SIM=43.26
SF13 -> RMS=70.18 | SIM=42.74

Halogen 10
SF12 -> RMS=70.54 | SIM=49.25
SF13 -> RMS=71.62 | SIM=49.49

Marvin 5
SF12 -> RMS=86.92 | SIM=48.81
SF13 -> RMS=87.44 | SIM=49.76

Seer 1.2.1
SF12 -> RMS=107.47 | SIM=36.27
SF13 -> RMS=108.57 | SIM=36.69

. Tornado supports nnue, but doesn't have a net of its own.
. I saw no Scorpio nnue executable.

To me, this plus the Toga III results imply that if one doesn't use the nodchip architecture in Stockfish and doesn't use any Stockfish data in the training process, then one should get a reasonably unique engine; i.e. the Simex is working for NNUE engines as well.

Whether the architecture or the training process is more important in differentiating engines from each other is still up for debate; another Simex test to try would be Minic 3.0 vs Seer 1.2.1; they share the same architecture but have different nets trained on different data by different training processes.

[jshriver]

If anyone is looking for a source of binpacks, I've been building and releasing them on one of my sites for download.

https://openchessdb.com/nnue.html

Just added 6.5 Billion more positions.

[Ferdy]

If anyone is looking for a source of binpacks, I've been building and releasing them on one of my sites for download.

https://openchessdb.com/nnue.html

Just added 6.5 Billion more positions.

Thanks for sharing. Do you have info what options were used to generate these bins? What stockfish version?

[Dann Corbit]

If anyone is looking for a source of binpacks, I've been building and releasing them on one of my sites for download.

https://openchessdb.com/nnue.html

Just added 6.5 Billion more positions.

What is a binpack, exactly?

[Ferdy]

If anyone is looking for a source of binpacks, I've been building and releasing them on one of my sites for download.

https://openchessdb.com/nnue.html

Just added 6.5 Billion more positions.

What is a binpack, exactly?

A format for training and validation files.