about error margins?

[Kempelen]

Could someone explain to me where the current formulas to calculate error margins according to number of games cames from? (i.e. +-10 elo points for 4000 games). I mean, Why those numbers and no others?. What observations has been done to arrive to that conclusions?
thx Fermin

[Daniel Shawul]

Could someone explain to me where the current formulas to calculate error margins according to number of games cames from? (i.e. +-10 elo points for 4000 games). I mean, Why those numbers and no others?. What observations has been done to arrive to that conclusions?
thx Fermin

I think you would need an estimate of the sample standard deviation to give specific values. Otherwise all you can say is that for every doubling of number of games ,the error margin would be halved... For 95% confidence i.e 2 sigma you have , mean +/- 2 * sd / sqrt(n). Then from winning percentage and elo relation you can tell how much the error margin is interms of elo. If you have a record of games, like WLWWWDDLLLWL etc , you can estimate the sample mean and sd from it.

[hgm]

What observations has been done to arrive to that conclusions?

No observations have to be done, because this is simple mathematics. One also doesn't do observations to establish that 5+5=10...

It follows from the definition of standard deviation, confidence interval and the fact that the games are independent.

[Sven]

I think you would need an estimate of the sample standard deviation to give specific values. Otherwise all you can say is that for every doubling of number of games ,the error margin would be halved...

In principle I agree, but the sample standard deviation is already an estimate (of the "real" standard deviation which is unknown) so you don't need to estimate it again. And for every doubling of the number of games the error margin is divided by sqrt(2), or for every multiplying of the number of games by four the error margin is halved.

Sven

[Daniel Shawul]

I think you would need an estimate of the sample standard deviation to give specific values. Otherwise all you can say is that for every doubling of number of games ,the error margin would be halved...

In principle I agree, but the sample standard deviation is already an estimate (of the "real" standard deviation which is unknown) so you don't need to estimate it again. And for every doubling of the number of games the error margin is divided by sqrt(2), or for every multiplying of the number of games by four the error margin is halved.

Sven

Just left that mistake for mr nitpicker to find it Correction people,it is when number of games is _quadrupled_ not doubled you get s.e halved !

But I was not saying he should use the population sd but just the sample sd. Note that he mentioned only number of games and associated error of margin. It is impossilbe to determine error of margin from number of games alone so he needs to estimate the standard deviation from the sample of games. 4000 games is pretty high so sample or population statistic don't matter much even if the later is known. It is a good question though, as I have noticed some say (including myself) that 4000 games is a +/10 elo or so which is not quite right. Probably that is what Fermin meant by observations. Most of the time you would get a +/- 10 elo from bayeselo at around 3-4 thousand games.. For example a 50% winning percentage could be from a WLWLWLWLWL or DDDDDDDDDD. The latter has a big sd ,but the latter has 0 sd!

Code: Select all

sd = sqrt(wins * (1 - m)^2 + losses * (0 - m)^2 + draws * (0.5 - m)^2) / sqrt(n - 1)

If the draw ratio is 0, it is possible to tell the error margin from the number of games and winning percentage alone.

[Laskos]

I think you would need an estimate of the sample standard deviation to give specific values. Otherwise all you can say is that for every doubling of number of games ,the error margin would be halved...

In principle I agree, but the sample standard deviation is already an estimate (of the "real" standard deviation which is unknown) so you don't need to estimate it again. And for every doubling of the number of games the error margin is divided by sqrt(2), or for every multiplying of the number of games by four the error margin is halved.

Sven

Just left that mistake for mr nitpicker to find it Correction people,it is when number of games is _quadrupled_ not doubled you get s.e halved !

But I was not saying he should use the population sd but just the sample sd. Note that he mentioned only number of games and associated error of margin. It is impossilbe to determine error of margin from number of games alone so he needs to estimate the standard deviation from the sample of games. 4000 games is pretty high so sample or population statistic don't matter much even if the later is known. It is a good question though, as I have noticed some say (including myself) that 4000 games is a +/10 elo or so which is not quite right. Probably that is what Fermin meant by observations. Most of the time you would get a +/- 10 elo from bayeselo at around 3-4 thousand games.. For example a 50% winning percentage could be from a WLWLWLWLWL or DDDDDDDDDD. The latter has a big sd ,but the latter has 0 sd!

Code: Select all

sd = sqrt(wins * (1 - m)^2 + losses * (0 - m)^2 + draws * (0.5 - m)^2) / sqrt(n - 1)

If the draw ratio is 0, it is possible to tell the error margin from the number of games and winning percentage alone.

DDDD... gives 0 sd only in the approximation you use.

Kai

[Daniel Shawul]

How? Note that I am calculating sd of winning percentage. When you go to elo calculation with bayeselo, there is ofcourse elodraw and eloadvantage. So a DDDDDD is more variable when you consider those things... Anyway I was just trying to demostrate why one can't tell margin of error of elo from the number of games and winning percentage alone.

[Laskos]

How? Note that I am calculating sd of winning percentage. When you go to elo calculation with bayeselo, there is ofcourse elodraw and eloadvantage. So a DDDDDD is more variable when you consider those things... Anyway I was just trying to demostrate why one can't tell margin of error of elo from the number of games and winning percentage alone.

That thing you wrote is the error (SD) in the normal approximation of the trinomial. You can still use it for 10 draws match, but keep in mind that after that match, probability of W,L = 1/13, for D = 11/13, and input these into your formula.

Kai

[Daniel Shawul]

How? Note that I am calculating sd of winning percentage. When you go to elo calculation with bayeselo, there is ofcourse elodraw and eloadvantage. So a DDDDDD is more variable when you consider those things... Anyway I was just trying to demostrate why one can't tell margin of error of elo from the number of games and winning percentage alone.

That thing you wrote is the error (SD) in the normal approximation of the trinomial. You can still use it for 10 draws match, but keep in mind that after that match, probability of W,L = 1/13, for D = 11/13, and input these into your formula.

Kai

The probability of a W,L would surely be lowered a lot after that odd observation.. That formula is for calculating standard deviation of a given sample that does not assume probablities for WDL. The rewards are fixed at 0,0.5,1 ofcourse. This was just a quick example , but I know it does not directly translate to elo because there you have mix of players with different strength, white elo advantage etc..

[Laskos]

How? Note that I am calculating sd of winning percentage. When you go to elo calculation with bayeselo, there is ofcourse elodraw and eloadvantage. So a DDDDDD is more variable when you consider those things... Anyway I was just trying to demostrate why one can't tell margin of error of elo from the number of games and winning percentage alone.

That thing you wrote is the error (SD) in the normal approximation of the trinomial. You can still use it for 10 draws match, but keep in mind that after that match, probability of W,L = 1/13, for D = 11/13, and input these into your formula.

Kai

The probability of a W,L would surely be lowered a lot after that odd observation.. That formula is for calculating standard deviation of a given sample that does not assume probablities for WDL. The rewards are fixed at 0,0.5,1 ofcourse. This was just a quick example , but I know it does not directly translate to elo because there you have mix of players with different strength, white elo advantage etc..

I think the formula you wrote was derived using normal approximation and probabilities, which were translated into games by multiplying times n. The formula gives the SD as percentages, then you can convert to Elo by using the logistic. That 1/13 was derived from (w+1)/(n+3), for w=0, n=10 (10 draws, n=10, d=10)

Kai