Likelihood of superiority

[Rémi Coulom]

A draw will at the same time make estimated Elo ratings closer to each other, and reduce the width of confidence intervals. It does this in such a way that the LOS does not change.

Wow, this is absolutely a counterintuitive result (at least for me). I already knew statistic could be very counter intuitive sometime, but this really surprised me !

Thanks again for the explanation. I have learnt something this evening

Well, if you try it in bayeselo, you will see that it is not perfectly true. In particular, bayeselo considers that playing first is an advantage, so a draw with black means you are likely to be stronger.

If we don't consider the advantage of playing first, I hope the chess* explanation can convince your intuition.

Rémi

[michiguel]

A draw will at the same time make estimated Elo ratings closer to each other, and reduce the width of confidence intervals. It does this in such a way that the LOS does not change.

Wow, this is absolutely a counterintuitive result (at least for me). I already knew statistic could be very counter intuitive sometime, but this really surprised me !

Thanks again for the explanation. I have learnt something this evening

Because you are analyzing the probability to be better in a head to head competition, not how much better. If you think about it, it makes sense. I do not believe this extrapolates to estimate chances against a third player, though.

Miguel

[michiguel]

A draw will at the same time make estimated Elo ratings closer to each other, and reduce the width of confidence intervals. It does this in such a way that the LOS does not change.

Wow, this is absolutely a counterintuitive result (at least for me). I already knew statistic could be very counter intuitive sometime, but this really surprised me !

Thanks again for the explanation. I have learnt something this evening

Well, if you try it in bayeselo, you will see that it is not perfectly true. In particular, bayeselo considers that playing first is an advantage, so a draw with black means you are likely to be stronger.

If we don't consider the advantage of playing first, I hope the chess* explanation can convince your intuition.

Rémi

What do you mean exactly playing first is an advantage? In a match with 10 victories in a row and ten losses later you get a higher ELO?
Is is a time sensitive procedure?

Miguel

[Rémi Coulom]

What do you mean exactly playing first is an advantage? In a match with 10 victories in a row and ten losses later you get a higher ELO?
Is is a time sensitive procedure?

No. All results are supposed independent. Order of games does not matter. By playing first, I meant playing White, ie moving first in one game.

Maybe I used this more general formulation because I am a Go programmer now. In Go, Black plays first.

Rémi

[Rein Halbersma]

More precisely, here is a mathematical proof. Probabilities of win, draw, and loss are p_1, p_0.5, and p_0. Number of wins, draws, losses are n_1, n_0.5, and n_0. With a uniform prior:

Rémi

Very interesting! Could you give a reference from the paper where this derivation is taken from? I couldn't find anything on your website. And how is this formula related to the piece of C++ code you provided at the beginning of this thread? It wasn't reall obvious what that code did. Some explanation would be appreciated!

Rein

[Rémi Coulom]

More precisely, here is a mathematical proof. Probabilities of win, draw, and loss are p_1, p_0.5, and p_0. Number of wins, draws, losses are n_1, n_0.5, and n_0. With a uniform prior:

Rémi

Very interesting! Could you give a reference from the paper where this derivation is taken from? I couldn't find anything on your website. And how is this formula related to the piece of C++ code you provided at the beginning of this thread? It wasn't reall obvious what that code did. Some explanation would be appreciated!

Rein

I had planned to write a paper about testing statistics some years ago, but gave up. The current draft really contains nothing interesting, I don't plan to release it.

With a little math, you can prove that:
P(p1>p0 | n1,n0) = ( P(p1>p0 | n1-1,n0) + P(p1>p0 | n1,n0-1) ) / 2
with
P(p1>p0 | n1,-1) = 1
P(p1>p0 | -1,n0) = 0

So the probabilities can be computed by a kind of Pascal triangle.

Rémi

[Rein Halbersma]

More precisely, here is a mathematical proof. Probabilities of win, draw, and loss are p_1, p_0.5, and p_0. Number of wins, draws, losses are n_1, n_0.5, and n_0. With a uniform prior:

Rémi

Very interesting! Could you give a reference from the paper where this derivation is taken from? I couldn't find anything on your website. And how is this formula related to the piece of C++ code you provided at the beginning of this thread? It wasn't reall obvious what that code did. Some explanation would be appreciated!

Rein

I had planned to write a paper about testing statistics some years ago, but gave up. The current draft really contains nothing interesting, I don't plan to release it.

With a little math, you can prove that:
P(p1>p0 | n1,n0) = ( P(p1>p0 | n1-1,n0) + P(p1>p0 | n1,n0-1) ) / 2
with
P(p1>p0 | n1,-1) = 1
P(p1>p0 | -1,n0) = 0

So the probabilities can be computed by a kind of Pascal triangle.

Rémi

Interesting. I computed the integral with Mathematica and got an expression in terms of Gamma functions and hypergeometric functions. I did some manipulation but couldn't find a simplified expression. I'll try to see what solving the recurrence relation gives.

Rein

[Rémi Coulom]

Interesting. I computed the integral with Mathematica and got an expression in terms of Gamma functions and hypergeometric functions. I did some manipulation but couldn't find a simplified expression. I'll try to see what solving the recurrence relation gives.

Rein

I did not manage to find an easy way. It took me a lot of effort to prove it.

It looks so simple that there may be an easy way. I did not manage to build an intuition for that formula.

I found it by writing numerical values for p(p1>p0|n1,n0) in a table.

It must be a classic.

Rémi

[Rein Halbersma]

Interesting. I computed the integral with Mathematica and got an expression in terms of Gamma functions and hypergeometric functions. I did some manipulation but couldn't find a simplified expression. I'll try to see what solving the recurrence relation gives.

Rein

I did not manage to find an easy way. It took me a lot of effort to prove it.

It looks so simple that there may be an easy way. I did not manage to build an intuition for that formula.

I found it by writing numerical values for p(p1>p0|n1,n0) in a table.

It must be a classic.

Rémi

Yes it looks simple, very similar to a midpoint rule for solving partial differential equations. Yet, Mathematica is unable to solve the recurrence relation. Tonight, I'll try to see if it can prove the recurrence relation directly on your integral expression.

[Hart]

How would this compare to using a z-test on a proportion? I always thought that is how it was calculated but when I compare my numbers to yours they look very different.