What I was talking about is apparently related to a standard model selection method called AIC
https://en.wikipedia.org/wiki/Akaike_in ... _criterion
It seems one has to make a correction in case of comparing models with different numbers of estimated parameters.
Name for elo without draws?
Moderators: hgm, Rebel, chrisw
-
- Posts: 2272
- Joined: Mon Sep 29, 2008 1:50 am
Re: Name for elo without draws?
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
Without ideas there is nothing to simplify.
-
- Posts: 2272
- Joined: Mon Sep 29, 2008 1:50 am
Re: Name for elo without draws?
Here are some slides on AIC
http://myweb.uiowa.edu/cavaaugh/ms_lec_2_ho.pdf
It seems to be based on the premise that cross entropy is the right measure to compare distributions. The AIC is a clever way to estimate the cross entropy, up to a constant, between the unknown(!) true model and the model under test.
The actual formula for the AIC is given on page 14.
It is basically the generalized log likelihood corrected by the number of parameters. So it appears trivial to compute for an elo model, given a database of games.
http://myweb.uiowa.edu/cavaaugh/ms_lec_2_ho.pdf
It seems to be based on the premise that cross entropy is the right measure to compare distributions. The AIC is a clever way to estimate the cross entropy, up to a constant, between the unknown(!) true model and the model under test.
The actual formula for the AIC is given on page 14.
It is basically the generalized log likelihood corrected by the number of parameters. So it appears trivial to compute for an elo model, given a database of games.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
Without ideas there is nothing to simplify.
-
- Posts: 2272
- Joined: Mon Sep 29, 2008 1:50 am
Re: Name for elo without draws?
Actually BayesElo already gives the value of the log likelihood function under elo/mm/likelihood. Of course Ordo knows it too internally for its elo model.
One caveat is that when comparing models one needs the true log likelihood. So one must work with normalized distributions.
Often one is a bit sloppy with normalization constants since they do not affect the MLE. But this is not ok when comparing models.
One caveat is that when comparing models one needs the true log likelihood. So one must work with normalized distributions.
Often one is a bit sloppy with normalization constants since they do not affect the MLE. But this is not ok when comparing models.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
Without ideas there is nothing to simplify.
-
- Posts: 6401
- Joined: Thu Mar 09, 2006 8:30 pm
- Location: Chicago, Illinois, USA
Re: Name for elo without draws?
AIC is what I use for selecting models in my biochemical research after doing non-linear regression fitting of experimental data.Michel wrote:What I was talking about is apparently related to a standard model selection method called AIC
https://en.wikipedia.org/wiki/Akaike_in ... _criterion
It seems one has to make a correction in case of comparing models with different numbers of estimated parameters.
http://www.sciencedirect.com/science/ar ... 3615000303
I like it.
Miguel
-
- Posts: 2272
- Joined: Mon Sep 29, 2008 1:50 am
Re: Name for elo without draws?
Wonderful!!AIC is what I use for selecting models in my biochemical research after doing non-linear regression fitting of experimental data.
http://www.sciencedirect.com/science/ar ... 3615000303
I like it.
Miguel
OT: I see that the article is under a CC license. I know it is possible to buy an open access license from Elsevier but it is the first time I see such an article. In math people don't bother since all research is already freely available through the arxiv.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
Without ideas there is nothing to simplify.
-
- Posts: 2272
- Joined: Mon Sep 29, 2008 1:50 am
Re: Name for elo without draws?
AIC can also be used to fit an evaluation function to game results (or some other oracle). Since it takes the number of parameters into account (in a theoretically correct way) it guards in a natural way against overfitting.AIC is what I use for selecting models in my biochemical research after doing non-linear regression fitting of experimental data.
http://www.sciencedirect.com/science/ar ... 3615000303
I like it.
Miguel
Perhaps you are already doing this in Gaviota?
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
Without ideas there is nothing to simplify.