Choice of loss function to train a neural network evaluation

[Fabio Gobbato]

If you convert the centipawn score to probability what is the best loss function that you have tried to train a neural network evaluation?
I have tried with mean squared error and it gives good results, but the best net I've found so far is from error^2.5.
The best networks generated with the 2 loss function differ of about 2 elo, so there isn't much to gain.
Have you experimented with other loss function? With what results?

[dangi12012]

cp is pure garbage.
Train for WDL metric with a logistic function.
Don't forget that WL is much worse than WDL because you then can discriminate forced draws from drawing positions etc.

If you just mean loss the sum of squared differences works quite well.

[JoAnnP38]

cp is pure garbage.
Train for WDL metric with a logistic function.
Don't forget that WL is much worse than WDL because you then can discriminate forced draws from drawing positions etc.

If you just mean loss the sum of squared differences works quite well.

I have been looking into that over the past couple of days. A logistic regression or logit model seems like a nice stepping-stone to ML without going all the way to NN. I noticed that some engines that use this map the probability function onto cp purely for the purpose of reporting score back via UCI (or XBoard I would assume.) So instead of using it as an intermediary for training a NN, why not use it as the evaluation function itself? Maximizing the probability for a win seems more straightforward than trying to maximize a cp advantage.

[jdart]

As I understand it, Stockfish currently tunes with lambda = 1.0, so only using scores, not WDL.

[Fabio Gobbato]

As I understand it, Stockfish currently tunes with lambda = 1.0, so only using scores, not WDL.

I don't know how stockfish does but from my experience you should convert the score of the search to probability and train the neural network with that. I think that lambda 1.0 means use only the probability from the search score and not the game result of that position. But as I have said I don't know exactly how the stockfish trainer works.