Name for elo without draws?

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Michel
Posts: 2040
Joined: Sun Sep 28, 2008 11:50 pm

Re: Name for elo without draws?

Post by Michel » Mon Sep 07, 2015 10:59 am

What I was talking about is apparently related to a standard model selection method called AIC

https://en.wikipedia.org/wiki/Akaike_in ... _criterion

It seems one has to make a correction in case of comparing models with different numbers of estimated parameters.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.

Michel
Posts: 2040
Joined: Sun Sep 28, 2008 11:50 pm

Re: Name for elo without draws?

Post by Michel » Mon Sep 07, 2015 6:10 pm

Here are some slides on AIC

http://myweb.uiowa.edu/cavaaugh/ms_lec_2_ho.pdf

It seems to be based on the premise that cross entropy is the right measure to compare distributions. The AIC is a clever way to estimate the cross entropy, up to a constant, between the unknown(!) true model and the model under test.

The actual formula for the AIC is given on page 14.

It is basically the generalized log likelihood corrected by the number of parameters. So it appears trivial to compute for an elo model, given a database of games.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.

Michel
Posts: 2040
Joined: Sun Sep 28, 2008 11:50 pm

Re: Name for elo without draws?

Post by Michel » Tue Sep 08, 2015 4:55 am

Actually BayesElo already gives the value of the log likelihood function under elo/mm/likelihood. Of course Ordo knows it too internally for its elo model.

One caveat is that when comparing models one needs the true log likelihood. So one must work with normalized distributions.

Often one is a bit sloppy with normalization constants since they do not affect the MLE. But this is not ok when comparing models.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.

User avatar
michiguel
Posts: 6388
Joined: Thu Mar 09, 2006 7:30 pm
Location: Chicago, Illinois, USA
Contact:

Re: Name for elo without draws?

Post by michiguel » Tue Sep 08, 2015 5:16 am

Michel wrote:What I was talking about is apparently related to a standard model selection method called AIC

https://en.wikipedia.org/wiki/Akaike_in ... _criterion

It seems one has to make a correction in case of comparing models with different numbers of estimated parameters.
AIC is what I use for selecting models in my biochemical research after doing non-linear regression fitting of experimental data.
http://www.sciencedirect.com/science/ar ... 3615000303

I like it.

Miguel

Michel
Posts: 2040
Joined: Sun Sep 28, 2008 11:50 pm

Re: Name for elo without draws?

Post by Michel » Tue Sep 08, 2015 5:28 am

AIC is what I use for selecting models in my biochemical research after doing non-linear regression fitting of experimental data.
http://www.sciencedirect.com/science/ar ... 3615000303

I like it.

Miguel
Wonderful!!

OT: I see that the article is under a CC license. I know it is possible to buy an open access license from Elsevier but it is the first time I see such an article. In math people don't bother since all research is already freely available through the arxiv.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.

Michel
Posts: 2040
Joined: Sun Sep 28, 2008 11:50 pm

Re: Name for elo without draws?

Post by Michel » Tue Sep 08, 2015 7:34 am

AIC is what I use for selecting models in my biochemical research after doing non-linear regression fitting of experimental data.
http://www.sciencedirect.com/science/ar ... 3615000303

I like it.

Miguel
AIC can also be used to fit an evaluation function to game results (or some other oracle). Since it takes the number of parameters into account (in a theoretically correct way) it guards in a natural way against overfitting.

Perhaps you are already doing this in Gaviota?
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.

Post Reply