Likelihood of superiority

[Rémi Coulom]

How would this compare to using a z-test on a proportion? I always thought that is how it was calculated but when I compare my numbers to yours they look very different.

When the number of games is high, the numbers should match.

For instance, with n1 = 120, n0 = 110, you have

mu = 120/230
sigma² = mu*(1-mu)/229
(mu-0.5)/sigma = 0.658568

Look it up in that table:
http://www.math.unb.ca/~knight/utility/NormTble.htm
-> something between 0.7422 and 0.7454

My program gives 0.7447, so that matches very well.

If you add a few draws, for instance 100

mu = 170/330
sigma² = (120*(1-mu)² + 100*(0.5-mu)² + 110*(0-mu)²)/(329*330)
(mu - 0.5)/sigma = 0.661198

It is almost identical.

Rémi

[Michel]

I always wondered why BayesElo uses Bayesian statistics.

Maximum likelihood also works for traditional frequentist statistics and then one doesn't need
to choose a prior.

This is not criticism. I am just curious to know the reason.

[Rémi Coulom]

I always wondered why BayesElo uses Bayesian statistics.

Maximum likelihood also works for traditional frequentist statistics and then one doesn't need
to choose a prior.

This is not criticism. I am just curious to know the reason.

Pure maximum-likelihood without a prior does not work as well as using a prior. The prior helps to regularize and avoids overfitting.

I measured quality of predictions with or without a prior on the WBEC database, and found that having a prior improved very significantly. This is particularly obvious when a player has a winning rate close to 0 or 1.

Rémi

[Michel]

I measured quality of predictions with or without a prior on the WBEC database, and found that having a prior improved very significantly. This is particularly obvious when a player has a winning rate close to 0 or 1.

Thank you!

I just wanted to know.