jdart wrote: ↑Sat Mar 27, 2021 7:53 pm
I am doing some experiments with
https://github.com/bmdanielsson/nnue-trainer and have a couple of questions:
1. What's a typical number of positions for training and validation?
2. I am seeing that the objective values seem to go down with increasing training set size. For example, 5 million positions generated with depth 8 and 500k positions for validation produce output like this:
Code: Select all
Epoch 0, 100% (611/611) => 0.04480
Epoch 1, 100% (611/611) => 0.03570
Epoch 2, 100% (611/611) => 0.03236
Epoch 3, 100% (611/611) => 0.02443
...
.
From my limited experience even larger input sets produce very small errors and this may make optimization problematic?
This is what I have on 500m when using
nodchip/stockfish pos generator and trainer.
Code: Select all
csvsql --query "SELECT tn,tpos,tdep,vdep,valcnt,rmpv,rmpvdi,lambda,mgrad,elo,remarks FROM nnue_training_log WHERE (tpos = '500mill') ORDER BY elo DESC" nnue_training_log.csv | csvlook
| tn | tpos | tdep | vdep | valcnt | rmpv | rmpvdi | lambda | mgrad | elo | remarks |
| -- | ------- | ---- | ---- | ------- | ---- | ------ | ------ | ----- | --- | ------------------------------------------------- |
| 30 | 500mill | 5 | 10 | 100,000 | 4 | 200 | 1 | 0.3 | 10 | +10 (500 games tc5s+50ms test) over not using net |
| 20 | 500mill | 5 | 10 | 2,000 | 4 | 100 | 1 | 0.3 | -10 | |
tpos: traning positions
tdep: training depth
vdep: validation depth
valcnt: validation count used in learning
rmpv: random_multi_pv
rmpvdi: random_multi_pv_diff
mgrad: max_grad
The net is tested using deuterium engine with a slower nnue_cpu probing code from daniel shawul.
From your test I think the objective is to minimize the loss, so that is fine.
I will test later this tn 30 data using
pytorch trainer and compare the perf with nodchip/stockfish trainer.
From what I have observed higher tpos, tdep, vdep and valcnt performs better. But there are also other
factors such as rmpv
etc that might affect the perf of the output net. I am working on soon to be released
hyperparameter optimizer to possibly address this optimization challenges.