Statistical interpretation of search and eval scores.
Moderators: Harvey Williamson, bob, hgm
Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Statistical interpretation of search and eval scores.
My math and stats are pretty rusty, but I seem to recall a good estimate would have an unbiased mean and normally distributed errors. I don't see how scores in centipawns can have these properties, as the true value is win, draw or loss. Can someone help me understand what form of scores would have these properties?

 Posts: 522
 Joined: Sat Mar 25, 2006 7:27 pm
Re: Statistical interpretation of search and eval scores.
My $0.02;
1. The score isn't an estimate of the game theoretic value (win/draw/loss). We are estimating the expected outcome over the universe of games that would be played that contain that position.
2. You are quite frequently doing some sort of transform before you create your estimator. You hope the errors of the transformed estimators are normally distributed, but that doesn't mean the untransformed errors would be.
1. The score isn't an estimate of the game theoretic value (win/draw/loss). We are estimating the expected outcome over the universe of games that would be played that contain that position.
2. You are quite frequently doing some sort of transform before you create your estimator. You hope the errors of the transformed estimators are normally distributed, but that doesn't mean the untransformed errors would be.

 Posts: 4008
 Joined: Fri Mar 10, 2006 4:23 am
 Location: http://www.arasanchess.org
Re: Statistical interpretation of search and eval scores.
The usual way to turn scores into a outcome predictor is to use a sigmoid function to transform the values MATE .. MATE into a 0..1 range (0 being 100% probability of loss, and 1 being 100% probability of win). A scaling factor may have to be applied in the function to get the probabilities correct. But this seems to be empirically sound, at least. It is also a standard model in the machine learning community (Logistic Regression).
Since there are three possible game outcomes though, for machine learning Ordinal Logistic Regression is possibly a sounder approach.
Jon
Since there are three possible game outcomes though, for machine learning Ordinal Logistic Regression is possibly a sounder approach.
Jon

 Posts: 61
 Joined: Wed Feb 19, 2014 9:11 pm
Re: Statistical interpretation of search and eval scores.
A static eval score is used to determine an ordering for all positions. While in practice we can loosely convert this value into something differently "meaningful" such as "centipawn advantage" or "win probability", that isnt its actual purpose.