TalkChess.com

What I was talking about is apparently related to a standard model selection method called AIC

https://en.wikipedia.org/wiki/Akaike_in ... _criterion

It seems one has to make a correction in case of comparing models with different numbers of estimated parameters.

Here are some slides on AIC

http://myweb.uiowa.edu/cavaaugh/ms_lec_2_ho.pdf

It seems to be based on the premise that cross entropy is the right measure to compare distributions. The AIC is a clever way to estimate the cross entropy, up to a constant, between the unknown(!) true model and the model under test.

The actual formula for the AIC is given on page 14.

It is basically the generalized log likelihood corrected by the number of parameters. So it appears trivial to compute for an elo model, given a database of games.

Actually BayesElo already gives the value of the log likelihood function under elo/mm/likelihood. Of course Ordo knows it too internally for its elo model.

One caveat is that when comparing models one needs the true log likelihood. So one must work with normalized distributions.

Often one is a bit sloppy with normalization constants since they do not affect the MLE. But this is not ok when comparing models.

Michel wrote:What I was talking about is apparently related to a standard model selection method called AIC

https://en.wikipedia.org/wiki/Akaike_in ... _criterion

It seems one has to make a correction in case of comparing models with different numbers of estimated parameters.

AIC is what I use for selecting models in my biochemical research after doing non-linear regression fitting of experimental data.
http://www.sciencedirect.com/science/ar ... 3615000303

I like it.

Miguel

AIC is what I use for selecting models in my biochemical research after doing non-linear regression fitting of experimental data.
http://www.sciencedirect.com/science/ar ... 3615000303

I like it.

Miguel

Wonderful!!

OT: I see that the article is under a CC license. I know it is possible to buy an open access license from Elsevier but it is the first time I see such an article. In math people don't bother since all research is already freely available through the arxiv.

AIC is what I use for selecting models in my biochemical research after doing non-linear regression fitting of experimental data.
http://www.sciencedirect.com/science/ar ... 3615000303

I like it.

Miguel

AIC can also be used to fit an evaluation function to game results (or some other oracle). Since it takes the number of parameters into account (in a theoretically correct way) it guards in a natural way against overfitting.

Perhaps you are already doing this in Gaviota?