TalkChess.com

I just put a little chess statistics tool page online. Maybe it is of use to someone.

http://glass.2.ag/tools.php


if you enter the results of a match between two engines (wins, draws, losses) it will output:

• number of games
• score
• score-difference
• draw-ratio
• win-ratio
• elo
• los

regards,
Edmund

Edmund wrote:I just put a little chess statistics tool page online. Maybe it is of use to someone.

http://glass.2.ag/tools.php


if you enter the results of a match between two engines (wins, draws, losses) it will output:

• number of games
• score
• score-difference
• draw-ratio
• win-ratio
• elo
• los

regards,
Edmund

Thanks Edmund,

There is something funny with certain numbers. I do not know whether it's the formula or something else.

For instance, for a w/d/l of 1000/500/1020 you get a negative LOS, which has no physical meaning.
1000/500/1000 gives 50%, but X/Y/Z and Z/Y/X should give numbers that are symmetrical (LOS1 should be the same as 100-LOS2) and that does not happen.

Miguel

Moreover basing LOS just on difference but not including draws is simply wrong.
For example scores 1/10000/0 and 1/0/0 cannot have the same LOS.

Milos wrote:Moreover basing LOS just on difference but not including draws is simply wrong.
For example scores 1/10000/0 and 1/0/0 cannot have the same LOS.

No, LOS doesn't depend on number of draws. Error intervals depend, yes, but not LOS. I can give you a short formula for LOS.

What is wrong here, is only that you have to input the score of the better engine, but it is quickly adjustable.

Kai

Milos wrote:Moreover basing LOS just on difference but not including draws is simply wrong.
For example scores 1/10000/0 and 1/0/0 cannot have the same LOS.

This los calculation uses an normal distribution to approximate the win distribution. The otherwise needed multinominal distribution would take ages to calculate an exact value for your request with > 10000 games.

The downside of this is that the approximation is not too accurate for such extreme examples.
But if you try more realistic examples:
60/50/50 -> 82.98%
30/50/20 -> 92.14%
you will notice that the draw rate is very much considered.

Edmund wrote: you will notice that the draw rate is very much considered.

It shouldn't be considered for LOS.

Kai

michiguel wrote:

Edmund wrote:I just put a little chess statistics tool page online. Maybe it is of use to someone.

http://glass.2.ag/tools.php


if you enter the results of a match between two engines (wins, draws, losses) it will output:

• number of games
• score
• score-difference
• draw-ratio
• win-ratio
• elo
• los

regards,
Edmund

Thanks Edmund,

There is something funny with certain numbers. I do not know whether it's the formula or something else.

For instance, for a w/d/l of 1000/500/1020 you get a negative LOS, which has no physical meaning.
1000/500/1000 gives 50%, but X/Y/Z and Z/Y/X should give numbers that are symmetrical (LOS1 should be the same as 100-LOS2) and that does not happen.

Miguel

Right, thanks for the hint. I fixed it now. However I notice that the output might still be a couple of 0.01 percent of the correct result as it seems PHP makes some rounding errors.

for your 1020/500/1000
if I calculate it with excel I get 67.19957 ...%
and the form prints 67.18%

the inverse 1000/500/1020 outputs now after the fix 32.82%

Edmund wrote:This los calculation uses an normal distribution to approximate the win distribution. The otherwise needed multinominal distribution would take ages to calculate an exact value for your request with > 10000 games.

Of course you use normal distribution approximation. There is nothing wrong in using it per se.
However even there draw ration is implicitly included in variance.
The problem is by calculating LOS from difference tables. That is wrong.
Since you have the actual normal distribution approximation of ELO for both engines (lets call random variables for according PDFs X and Y), you could easily calculate the PDF of random variable Z=X-Y (just a simple convolution). And LOS would be Pr(Z>0).

Laskos wrote:

Edmund wrote: you will notice that the draw rate is very much considered.

It shouldn't be considered for LOS.
(your example is wrong, look at the number of losses).

Seams you have a different definition of LOS. How are you defining it then?

P.S. Table formulas like ones based on just difference in wins/losses are simply wrong.

Milos wrote:

Laskos wrote:

Edmund wrote: you will notice that the draw rate is very much considered.

It shouldn't be considered for LOS.
(your example is wrong, look at the number of losses).

Seams you have a different definition of LOS. How are you defining it then?

P.S. Table formulas like ones based on just difference in wins/losses are simply wrong.

Likelihood of Success that one engine is better than another. It does not depend on the number of draws. Error intervals yes, depend. If you want a precise formula for LOS, I can give it.

Kai