Games for Training NNUE?

[Rebel]

30 seconds per game is an insane amount of time. You can get away with a small number of fixed nodes per move. My strongest nets are with 40k nodes per move. 80k nodes actually lost elo.

Fascinating! It makes sense because you'll undoubtedly have a lot of crazy positions in that training set.

— Steve

And that should be, the net must learn what is good and bad.

[jstanback]

For Wasp, I collect positions from Wasp's games against itself and other engines typically played at something like Game in 5s with 100ms increment. I guess it would be fine to play these games at faster time controls. Lately I've been using several different opening books, endgame books, and FRC openings to try to increase the variety of positions encountered in the games. I randomly extract about 10% of the positions from each game and then do a depth=4 search of those positions and save the quiescent position at the end of the PV to use for training. These positions are then scored using my most recent Wasp (at the time) with a 35K node fixed depth search. The position, game result, and the search score are saved in a 40 byte per position binary format. So far, I have only about 230M training positions. It's interesting that Andrew found that it was actually worse to score the positions using deeper searches as I have been wondering whether that would help. My network has 1572 input nodes, 1024 hidden layer nodes, and 7 output nodes. One of the 7 output nodes is used, based on the material for a given position. For training, I randomly pick a 32K "minibatch" of positions from the 230M set to backpropogate and update the weights. It takes about 6M of these minibatches to fully train the net. This takes about 48 hours on my Ryzen 7950x.

I'm still making progress by playing more games using recent nets and adding more positions scored using recent nets, but the rate of improvement is slowing. I welcome any ideas that might improve my training methodology. Perhaps I will try some sort of king-piece-square net one of these days...

John

[chrisw]

So you take completely random positions (but legal ones? any other filters?) and then you search them to depth 9 with an existing (probably HCE in the first iteration) evaluation. Under the assumption that the eval after a depth 9 search is more accurate than a static eval looking just at the position you then train your NN on this improved evaluation data and the evaluation to gets better? Then you repeat the process?

Is that the standard that everyone uses or have alternatives been tried?

E.g for my non-NN eval the tuner finds appropriate weights from a set of PGNs of selfplay games. From each game several positions are taken and scored under the assumption, that a game that white wins eventually contains only positions that are winning for white. Seems to work because mislabeled positions tend to cancel each other out and a small imbalance towards the ground truth remains. I wonder if it would also work for training NNUE type evals.

I don’t really know what “everyone else” does, but I think the general process is self-play games off a very large openings test suite, and/or self play games off opening positions generated by random moves N deep off the start position, and/or throw in sone FRC. Connor (apologies if I misread him) worked from the opposite direct and trained a net from EGTB6, then backed up one ply, played forward using the egtb6-nnue to evaluate the 7’s and so on, iteratively backwards. A few months ago SF broke the self-imposed good programmer embargo on not using your own engine to create the test positions and started using LCZ data (they weren’t the first of course), so now, maybe “everyone else” just uses training positions sourced from wherever. I get the impression that at the leading edge there’s a whole witches cauldron of training positions being groomed, filtered, massaged, newts and essence of valerian added and sub-suites generated. There’s sub-suites to start off with and sub-suites to finish with. Etc. The winners of the witches cauldron method will be those with the largest training and test resources, ie SF.

[Witek]

My strongest nets are with 40k nodes per move. 80k nodes actually lost elo.

Interesting. Are we talking about same amount of 80k and 40k games / positions used for training?

[AndrewGrant]

My strongest nets are with 40k nodes per move. 80k nodes actually lost elo.

Interesting. Are we talking about same amount of 80k and 40k games / positions used for training?

Not a mixture of the two, although that failed as well. 80k nodes data is too accurate it seems to be trained on.

[Henk]

If you use a database your hard disk may crash. That was my experience. Maybe better to recalculate.

O wait If you recalculate perhaps fan might get broken earlier I don't know.

[smatovic]

If you use a database your hard disk may crash. That was my experience. Maybe better to recalculate.

O wait If you recalculate perhaps fan might get broken earlier I don't know.

+1

Don't torture your laptop with 24/7 high load computer chess, use it as a client for your rig/workstation.

--
Srdja

[dangi12012]

Prerequisities: pgn-extract

This gets you 1Billion distinct fens out of real games. You can filter to above certain elo too.

Code: Select all

wget -qO- https://database.lichess.org/standard/lichess_db_standard_rated_2022-08.pgn.bz2 | bzip2 -dc | ./pgn-extract -Wfen -s | grep -v "\[" | awk '{ print $1, $2, $3, $4}' | grep "\S" | awk '!x[$0]++' | head -n1000000000 | sort> fens_1G.txt

Task: Rotate to the perspective of the player to move (a network does not need to learn symmetry)
Selfplay these games with the strongest engine currently (A small subset 1E6) and save results for each fen
Do Linear regression into the weights.

Result: A network that can predict outcomes for a position. (can be extended for WDL)


Bonus: Find positions that drastically change evaluation with engine depth from 15 to 20 (so even the best engine does not find a hidden tactic at first glance) - train a network on those.
When the network triggers during actual games more thinking time is needed.