Rubichess NN questions

[jdart]

I am looking at:
https://github.com/Matthies/NN

Re this:

nn-f21733c196-20201028.nnue: Learned from Rubi depth 10 training positions (TB disabled) using the SF learner binary; +95 Elo +/-25 vs. HCE

Doesn't that binary use the Stockfish eval, not the Rubichess eval?

And this:

nn-375bdd2d7f-20210112.nnue: Learned from Rubi depth 8 training using eval of last net, 6-men-TB and disabled pruning using trainer of SV mod branch

What is the "SV mod branch" (was it meant to say: SF mod branch?)

[RubiChess]

Hi Jon.

I am looking at:
https://github.com/Matthies/NN

Re this:

nn-f21733c196-20201028.nnue: Learned from Rubi depth 10 training positions (TB disabled) using the SF learner binary; +95 Elo +/-25 vs. HCE

Doesn't that binary use the Stockfish eval, not the Rubichess eval?

Rubis evaluation is in the training data.
The SF learner binary is used to "create" (train, optimize) a network (starting from scratch or from an earlier Rubi network) that minimizes the difference of 1. an NNUE evaluation with this net and 2. the evaluation/result in the training data. The value of 1. is a standard NNUE evaluation (independent from the engine but given by the network topology) with the exception that a depth 1 search (SF qsearch) is used to get a quiet position. Neither the handcrafted evaluation of Stockfish nor the evaluation of a Stockfish net is used. At least this is what I read from the code and from my debugging.

And this:

nn-375bdd2d7f-20210112.nnue: Learned from Rubi depth 8 training using eval of last net, 6-men-TB and disabled pruning using trainer of SV mod branch

What is the "SV mod branch" (was it meant to say: SF mod branch?)

No. SV stands for Sergio Vieri. Beginning with the nn-375bdd2d7f-20210112.nnue I used his modification of the original Nodchip NNUE learner branch: https://github.com/sergiovieri/Stockfish/tree/mod

Let me add some words about the (for me quite shocking) results of my last version 2.1:
My explanation for the nearly +100 Elo compared to version 2.0 are
1. I fixed some bugs in the training data generation code.
2. I created more and better data with deaper search.
3. I tried parameters for the learner that Sergio used in his Workflow (https://github.com/sergiovieri/Stockfish/tree/mod/nnue)

When I realized how strong the nn-cf8c56d366-20210326.nnue is I asked Ed Schroeder for a similarity test with his NNUE research tools from http://rebel13.nl/home/nnue.html and he kindly tested it immediately and sent me these results which seem to show less SF-similarity than many others:

Score comparison between 'sf12.epd' and 'rubi-nnue.epd'

Code: Select all

dev0    dev1    dev2    dev3    dev4    dev5    RMS     SIM
2202    1767    1408    973     638     1236    74.83   54.95

Score comparison between 'sf13.epd' and 'rubi-nnue.epd'

Code: Select all

dev0    dev1    dev2    dev3    dev4    dev5    RMS     SIM
1919    1652    1388    1028    797     1442    80.31   49.23

Excellent results.

I still have the goal to port a learner into Rubi to be completely independant from SF learner binary but this is not ready for now.

Regards, Andreas

[voffka]

These are exceptionally strong results, I was never able to train a network in Igel that would be more than marginally stronger than the last best Igel network (with-in 25 elo) and in some cases it took me 4 months of training and hundreds of re-tries from scratch just to get my usual -20 elo network I see so often.

Does SV branch uses the Stockfish trainer with Stockfish qsearch as well?

P.S. I am using pytorch trainer for Igel

[RubiChess]

These are exceptionally strong results, I was never able to train a network in Igel that would be more than marginally stronger than the last best Igel network (with-in 25 elo) and in some cases it took me 4 months of training and hundreds of re-tries from scratch just to get my usual -20 elo network I see so often.

Well, don't believe that I just had to press the "+100Elo" button. I also had many failures and it took a lot of CPU time and trial-and-error to find a good net.

Does SV branch uses the Stockfish trainer with Stockfish qsearch as well?

Yes. It is the Nodchip learner with some LR drop logic and some bugs fixed.

[voffka]

Thanks Andreas, I'll checkout the SV mod branch (after I finish experimenting with pytorch), looks interesting indeed.