Could someone explain to me where the current formulas to calculate error margins according to number of games cames from? (i.e. +-10 elo points for 4000 games). I mean, Why those numbers and no others?. What observations has been done to arrive to that conclusions?
thx Fermin
about error margins?
Moderators: hgm, Rebel, chrisw
-
- Posts: 620
- Joined: Fri Feb 08, 2008 10:44 am
- Location: Madrid - Spain
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: about error margins?
I think you would need an estimate of the sample standard deviation to give specific values. Otherwise all you can say is that for every doubling of number of games ,the error margin would be halved... For 95% confidence i.e 2 sigma you have , mean +/- 2 * sd / sqrt(n). Then from winning percentage and elo relation you can tell how much the error margin is interms of elo. If you have a record of games, like WLWWWDDLLLWL etc , you can estimate the sample mean and sd from it.Kempelen wrote:Could someone explain to me where the current formulas to calculate error margins according to number of games cames from? (i.e. +-10 elo points for 4000 games). I mean, Why those numbers and no others?. What observations has been done to arrive to that conclusions?
thx Fermin
-
- Posts: 27790
- Joined: Fri Mar 10, 2006 10:06 am
- Location: Amsterdam
- Full name: H G Muller
Re: about error margins?
No observations have to be done, because this is simple mathematics. One also doesn't do observations to establish that 5+5=10...Kempelen wrote:What observations has been done to arrive to that conclusions?
It follows from the definition of standard deviation, confidence interval and the fact that the games are independent.
-
- Posts: 4052
- Joined: Thu May 15, 2008 9:57 pm
- Location: Berlin, Germany
- Full name: Sven Schüle
Re: about error margins?
In principle I agree, but the sample standard deviation is already an estimate (of the "real" standard deviation which is unknown) so you don't need to estimate it again. And for every doubling of the number of games the error margin is divided by sqrt(2), or for every multiplying of the number of games by four the error margin is halved.Daniel Shawul wrote:I think you would need an estimate of the sample standard deviation to give specific values. Otherwise all you can say is that for every doubling of number of games ,the error margin would be halved...
Sven
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: about error margins?
Just left that mistake for mr nitpicker to find it Correction people,it is when number of games is _quadrupled_ not doubled you get s.e halved !Sven Schüle wrote:In principle I agree, but the sample standard deviation is already an estimate (of the "real" standard deviation which is unknown) so you don't need to estimate it again. And for every doubling of the number of games the error margin is divided by sqrt(2), or for every multiplying of the number of games by four the error margin is halved.Daniel Shawul wrote:I think you would need an estimate of the sample standard deviation to give specific values. Otherwise all you can say is that for every doubling of number of games ,the error margin would be halved...
Sven
But I was not saying he should use the population sd but just the sample sd. Note that he mentioned only number of games and associated error of margin. It is impossilbe to determine error of margin from number of games alone so he needs to estimate the standard deviation from the sample of games. 4000 games is pretty high so sample or population statistic don't matter much even if the later is known. It is a good question though, as I have noticed some say (including myself) that 4000 games is a +/10 elo or so which is not quite right. Probably that is what Fermin meant by observations. Most of the time you would get a +/- 10 elo from bayeselo at around 3-4 thousand games.. For example a 50% winning percentage could be from a WLWLWLWLWL or DDDDDDDDDD. The latter has a big sd ,but the latter has 0 sd!
Code: Select all
sd = sqrt(wins * (1 - m)^2 + losses * (0 - m)^2 + draws * (0.5 - m)^2) / sqrt(n - 1)
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: about error margins?
DDDD... gives 0 sd only in the approximation you use.Daniel Shawul wrote:Just left that mistake for mr nitpicker to find it Correction people,it is when number of games is _quadrupled_ not doubled you get s.e halved !Sven Schüle wrote:In principle I agree, but the sample standard deviation is already an estimate (of the "real" standard deviation which is unknown) so you don't need to estimate it again. And for every doubling of the number of games the error margin is divided by sqrt(2), or for every multiplying of the number of games by four the error margin is halved.Daniel Shawul wrote:I think you would need an estimate of the sample standard deviation to give specific values. Otherwise all you can say is that for every doubling of number of games ,the error margin would be halved...
Sven
But I was not saying he should use the population sd but just the sample sd. Note that he mentioned only number of games and associated error of margin. It is impossilbe to determine error of margin from number of games alone so he needs to estimate the standard deviation from the sample of games. 4000 games is pretty high so sample or population statistic don't matter much even if the later is known. It is a good question though, as I have noticed some say (including myself) that 4000 games is a +/10 elo or so which is not quite right. Probably that is what Fermin meant by observations. Most of the time you would get a +/- 10 elo from bayeselo at around 3-4 thousand games.. For example a 50% winning percentage could be from a WLWLWLWLWL or DDDDDDDDDD. The latter has a big sd ,but the latter has 0 sd!If the draw ratio is 0, it is possible to tell the error margin from the number of games and winning percentage alone.Code: Select all
sd = sqrt(wins * (1 - m)^2 + losses * (0 - m)^2 + draws * (0.5 - m)^2) / sqrt(n - 1)
Kai
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: about error margins?
How? Note that I am calculating sd of winning percentage. When you go to elo calculation with bayeselo, there is ofcourse elodraw and eloadvantage. So a DDDDDD is more variable when you consider those things... Anyway I was just trying to demostrate why one can't tell margin of error of elo from the number of games and winning percentage alone.
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: about error margins?
That thing you wrote is the error (SD) in the normal approximation of the trinomial. You can still use it for 10 draws match, but keep in mind that after that match, probability of W,L = 1/13, for D = 11/13, and input these into your formula.Daniel Shawul wrote:How? Note that I am calculating sd of winning percentage. When you go to elo calculation with bayeselo, there is ofcourse elodraw and eloadvantage. So a DDDDDD is more variable when you consider those things... Anyway I was just trying to demostrate why one can't tell margin of error of elo from the number of games and winning percentage alone.
Kai
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: about error margins?
The probability of a W,L would surely be lowered a lot after that odd observation.. That formula is for calculating standard deviation of a given sample that does not assume probablities for WDL. The rewards are fixed at 0,0.5,1 ofcourse. This was just a quick example , but I know it does not directly translate to elo because there you have mix of players with different strength, white elo advantage etc..Laskos wrote:That thing you wrote is the error (SD) in the normal approximation of the trinomial. You can still use it for 10 draws match, but keep in mind that after that match, probability of W,L = 1/13, for D = 11/13, and input these into your formula.Daniel Shawul wrote:How? Note that I am calculating sd of winning percentage. When you go to elo calculation with bayeselo, there is ofcourse elodraw and eloadvantage. So a DDDDDD is more variable when you consider those things... Anyway I was just trying to demostrate why one can't tell margin of error of elo from the number of games and winning percentage alone.
Kai
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: about error margins?
I think the formula you wrote was derived using normal approximation and probabilities, which were translated into games by multiplying times n. The formula gives the SD as percentages, then you can convert to Elo by using the logistic. That 1/13 was derived from (w+1)/(n+3), for w=0, n=10 (10 draws, n=10, d=10)Daniel Shawul wrote:The probability of a W,L would surely be lowered a lot after that odd observation.. That formula is for calculating standard deviation of a given sample that does not assume probablities for WDL. The rewards are fixed at 0,0.5,1 ofcourse. This was just a quick example , but I know it does not directly translate to elo because there you have mix of players with different strength, white elo advantage etc..Laskos wrote:That thing you wrote is the error (SD) in the normal approximation of the trinomial. You can still use it for 10 draws match, but keep in mind that after that match, probability of W,L = 1/13, for D = 11/13, and input these into your formula.Daniel Shawul wrote:How? Note that I am calculating sd of winning percentage. When you go to elo calculation with bayeselo, there is ofcourse elodraw and eloadvantage. So a DDDDDD is more variable when you consider those things... Anyway I was just trying to demostrate why one can't tell margin of error of elo from the number of games and winning percentage alone.
Kai
Kai