Statistical interpretation of search and eval scores.

[jwes]

My math and stats are pretty rusty, but I seem to recall a good estimate would have an unbiased mean and normally distributed errors. I don't see how scores in centipawns can have these properties, as the true value is win, draw or loss. Can someone help me understand what form of scores would have these properties?

[Robert Pope]

My $0.02;

1. The score isn't an estimate of the game theoretic value (win/draw/loss). We are estimating the expected outcome over the universe of games that would be played that contain that position.

2. You are quite frequently doing some sort of transform before you create your estimator. You hope the errors of the transformed estimators are normally distributed, but that doesn't mean the untransformed errors would be.

[jdart]

The usual way to turn scores into a outcome predictor is to use a sigmoid function to transform the values -MATE .. MATE into a 0..1 range (0 being 100% probability of loss, and 1 being 100% probability of win). A scaling factor may have to be applied in the function to get the probabilities correct. But this seems to be empirically sound, at least. It is also a standard model in the machine learning community (Logistic Regression).

Since there are three possible game outcomes though, for machine learning Ordinal Logistic Regression is possibly a sounder approach.

--Jon

[DustyMonkey]

A static eval score is used to determine an ordering for all positions. While in practice we can loosely convert this value into something differently "meaningful" such as "centipawn advantage" or "win probability", that isnt its actual purpose.