## LOS (again)

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Ajedrecista
Posts: 1405
Joined: Wed Jul 13, 2011 7:04 pm
Contact:

### Re: LOS (again).

Hi Ed:

You assign a wrong value for LOS in your code box: I calculate k = |0.5 - score|/sigma and then I calculate the area below the normal distribution density function between -infinity and k; I have to do it approximating the integral with the Simpson's composite rule because I do not know other form to do it in Fortran.

I have other programme for calculate error bars and LOS values:

Code: Select all

``````LOS_and_Elo_uncertainties_calculator, ® 2012.

----------------------------------------------------------------
Calculation of Elo uncertainties in a match between two engines&#58;
----------------------------------------------------------------

&#40;The input and output data is referred to the first engine&#41;.

Maximum number of games supported&#58; 2147483647.

Write down the number of wins &#40;up to 1825361100&#41;&#58;

658

Write down the number of loses &#40;up to 1825361100&#41;&#58;

696

Write down the number of draws &#40;up to 2147482293&#41;&#58;

960

Write down the confidence level &#40;in percentage&#41; between 65% and 99.9% &#40;it will be rounded up to 0.01%)&#58;

95

Write down the clock rate of the CPU &#40;in GHz&#41;, only for timing the elapsed time of the calculations&#58;

3

---------------------------------------
Elo interval for 95.00 % confidence&#58;

Elo rating difference&#58;     -5.71 Elo

Lower rating difference&#58;  -16.54 Elo
Upper rating difference&#58;    5.12 Elo

Lower bound uncertainty&#58;  -10.84 Elo
Upper bound uncertainty&#58;   10.83 Elo
Average error&#58;        +/-  10.83 Elo

K = &#40;average error&#41;*&#91;sqrt&#40;n&#41;&#93; =  521.08

Elo interval&#58; &#93; -16.54,    5.12&#91;
---------------------------------------

Number of games of the match&#58;      2314
Score&#58; 49.18 %
Elo rating difference&#58;   -5.71 Elo
Draw ratio&#58; 41.49 %

*********************************************************
Standard deviation&#58;  1.5580 % of the points of the match.
*********************************************************

Error bars were calculated with two-sided tests; values are rounded up to 0.01 Elo, or 0.01 in the case of K.

-------------------------------------------------------------------
Calculation of likelihood of superiority &#40;LOS&#41; in a one-sided test&#58;
-------------------------------------------------------------------

LOS &#40;taking into account draws&#41; is always calculated, if possible.

LOS &#40;not taking into account draws&#41; is only calculated if wins + loses < 16001.

LOS &#40;average value&#41; is calculated only when LOS &#40;not taking into account draws&#41; is calculated.
______________________________________________

LOS&#58;  15.08 % &#40;taking into account draws&#41;.
LOS&#58;  15.10 % &#40;not taking into account draws&#41;.
LOS&#58;  15.09 % &#40;average value&#41;.
______________________________________________

These values of LOS are rounded up to 0.01%

End of the calculations. Approximated elapsed time&#58;   74 ms.

Thanks for using LOS_and_Elo_uncertainties_calculator. Press Enter to exit.
``````
I get a LOS value of ~ 15.08% using my own method; the other LOS value ~ 15.1% is done in other way proposed by Rémi Coulom in the last equation of this post. As you can see, 15.08% and 15.1% are enough close between them to almost consider both values as the same.

------------------------

I will try to compute LOS tables from 2000 games to 100000 games in steps of 1000 games (I mean: 2000, 3000, 4000, ..., 99000 and 100000 games) fixing the draw ratio to 32%, just as CPW tables. Please do not forget that they are only (good) approximations of the trinomial distribution.

The generation of these 99 tables will take a while (maybe a few hours), so please be patient. I have the intention of uploading the notepads here for everyone.

Regarding your last post, I do not have experience in engine testing, so all my efforts to help you will be useless.
Rebel wrote:Is it possible for you to email those larger tables? I like to put them on my website as a reference.
Thank you very much for your confidence!

Regards from Spain.

Ajedrecista.

Ajedrecista
Posts: 1405
Joined: Wed Jul 13, 2011 7:04 pm
Contact:

### 99 LOS tables.

Hi again:
Ajedrecista wrote:I will try to compute LOS tables from 2000 games to 100000 games in steps of 1000 games (I mean: 2000, 3000, 4000, ..., 99000 and 100000 games) fixing the draw ratio to 32%, just as CPW tables. Please do not forget that they are only (good) approximations of the trinomial distribution.

The generation of these 99 tables will take a while (maybe a few hours), so please be patient. I have the intention of uploading the notepads here for everyone.
I have computed 99 LOS tables from 2000 games up to 100000 games, in steps of 1000 games, as promised. I fixed the draw ratio to 32% (the same as the tables that are hosted at CPW) although very similar LOS values should be obtained with different draw ratios that are not extreme (I mean: not draw ratios of 99% or 100%, for example).

LOS_tables (0.24 MB)

I hope no typos in the notepads; I include a Readme.txt file with little info that must be considered.

Regards from Spain.

Ajedrecista.

Rebel
Posts: 4808
Joined: Thu Aug 18, 2011 10:04 am

### Re: 99 LOS tables.

Thank you, I will put the most important ones on a few web-pages.

For myself I am using Rémi Coulom's formula now.

Posts: 9760
Joined: Wed Jul 26, 2006 8:21 pm

### Re: 99 LOS tables.

Rebel wrote:Thank you, I will put the most important ones on a few web-pages.

For myself I am using Rémi Coulom's formula now.
What is Remi's formula? For LOS in a single match I used, if I am not wrong

LOS = (Sum_over_i [Binomial[wins + losses, i], {i from 0 to wins - 1}] + 0.5*Binomial[wins + losses, wins]) / 2^(wins + losses)

I tried without success to sum it up to a Hypergeometric 2F1 and such.

Kai

Rebel
Posts: 4808
Joined: Thu Aug 18, 2011 10:04 am

### Re: 99 LOS tables.

Rebel
Posts: 4808
Joined: Thu Aug 18, 2011 10:04 am

### Re: 99 LOS tables.

LOS tables from 2000 games and on are up.

http://www.top-5000.nl/los.htm

Ajedrecista
Posts: 1405
Joined: Wed Jul 13, 2011 7:04 pm
Contact:

### Re: 99 LOS tables.

Hi Ed:
Rebel wrote:LOS tables from 2000 games and on are up.

http://www.top-5000.nl/los.htm
Thanks for the upload and the credits!

I must say that columns for 10000 and 25000 games are the same in your web: you wrongly copied the file with 10000 games twice and the true column for 25000 games is not present right now. I hope a quick fix.

I forgot to write in Readme.txt file that the draw ratio was fixed to 32% but fortunately I included that info in each notepad with LOS tables. Once again, thank you very much for your confidence!

Regards from Spain.

Ajedrecista.

Rebel
Posts: 4808
Joined: Thu Aug 18, 2011 10:04 am

### Re: 99 LOS tables.

Corrected the error, tx for pointing out.

Speaking of errors, Bayesian elo and Elostat display an error-margin. Is the formula known and undisputed? The page deserves such a reference table.

Ajedrecista
Posts: 1405
Joined: Wed Jul 13, 2011 7:04 pm
Contact:

Hi again:
Rebel wrote:Speaking of errors, Bayesian elo and Elostat display an error-margin. Is the formula known and undisputed? The page deserves such a reference table.
I am not sure at all, but maybe BayesElo and EloSTAT differ a little; I use a method that gives similar error bars than EloSTAT if I am not wrong. Again I use the same mean and standard deviation of the same normal distribution that I used for generating LOS tables.

Of course error bars depend on the confidence level you want (95%, 99%, etc.; a higher confidence implies a larger error bar for the same match) but I do not feel confident of calculate error bars in this case because the formula may be disputed (I mean: there are probably several approaches to calculate error bars). You should ask to other people to obtain more valuable help.

Regards from Spain.

Ajedrecista.

John_F
Posts: 44
Joined: Wed Jan 27, 2010 1:58 pm

### Re: 99 LOS tables.

Does anyone know of a spreadsheet formula to calculate LOS? (Like for example, to determine the LOS of the winner in a match between two programs)