Likelihood of superiority

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Rémi Coulom
Posts: 432
Joined: Mon Apr 24, 2006 6:06 pm
Contact:

Re: Likelihood of superiority

Post by Rémi Coulom » Sun Nov 15, 2009 11:15 pm

mcostalba wrote:
Rémi Coulom wrote: A draw will at the same time make estimated Elo ratings closer to each other, and reduce the width of confidence intervals. It does this in such a way that the LOS does not change.
Wow, this is absolutely a counterintuitive result (at least for me). I already knew statistic could be very counter intuitive sometime, but this really surprised me !

Thanks again for the explanation. I have learnt something this evening :-)
Well, if you try it in bayeselo, you will see that it is not perfectly true. In particular, bayeselo considers that playing first is an advantage, so a draw with black means you are likely to be stronger.

If we don't consider the advantage of playing first, I hope the chess* explanation can convince your intuition.

Rémi

User avatar
michiguel
Posts: 6388
Joined: Thu Mar 09, 2006 7:30 pm
Location: Chicago, Illinois, USA
Contact:

Re: Likelihood of superiority

Post by michiguel » Sun Nov 15, 2009 11:48 pm

mcostalba wrote:
Rémi Coulom wrote: A draw will at the same time make estimated Elo ratings closer to each other, and reduce the width of confidence intervals. It does this in such a way that the LOS does not change.
Wow, this is absolutely a counterintuitive result (at least for me). I already knew statistic could be very counter intuitive sometime, but this really surprised me !

Thanks again for the explanation. I have learnt something this evening :-)
Because you are analyzing the probability to be better in a head to head competition, not how much better. If you think about it, it makes sense. I do not believe this extrapolates to estimate chances against a third player, though.

Miguel

User avatar
michiguel
Posts: 6388
Joined: Thu Mar 09, 2006 7:30 pm
Location: Chicago, Illinois, USA
Contact:

Re: Likelihood of superiority

Post by michiguel » Sun Nov 15, 2009 11:51 pm

Rémi Coulom wrote:
mcostalba wrote:
Rémi Coulom wrote: A draw will at the same time make estimated Elo ratings closer to each other, and reduce the width of confidence intervals. It does this in such a way that the LOS does not change.
Wow, this is absolutely a counterintuitive result (at least for me). I already knew statistic could be very counter intuitive sometime, but this really surprised me !

Thanks again for the explanation. I have learnt something this evening :-)
Well, if you try it in bayeselo, you will see that it is not perfectly true. In particular, bayeselo considers that playing first is an advantage, so a draw with black means you are likely to be stronger.

If we don't consider the advantage of playing first, I hope the chess* explanation can convince your intuition.

Rémi
What do you mean exactly playing first is an advantage? In a match with 10 victories in a row and ten losses later you get a higher ELO?
Is is a time sensitive procedure?

Miguel

Rémi Coulom
Posts: 432
Joined: Mon Apr 24, 2006 6:06 pm
Contact:

Re: Likelihood of superiority

Post by Rémi Coulom » Sun Nov 15, 2009 11:56 pm

michiguel wrote: What do you mean exactly playing first is an advantage? In a match with 10 victories in a row and ten losses later you get a higher ELO?
Is is a time sensitive procedure?
No. All results are supposed independent. Order of games does not matter. By playing first, I meant playing White, ie moving first in one game.

Maybe I used this more general formulation because I am a Go programmer now. In Go, Black plays first.

Rémi

Rein Halbersma
Posts: 685
Joined: Tue May 22, 2007 9:13 am

Re: Likelihood of superiority

Post by Rein Halbersma » Mon Nov 16, 2009 2:03 pm

Rémi Coulom wrote:More precisely, here is a mathematical proof. Probabilities of win, draw, and loss are p_1, p_0.5, and p_0. Number of wins, draws, losses are n_1, n_0.5, and n_0. With a uniform prior:

Rémi
Very interesting! Could you give a reference from the paper where this derivation is taken from? I couldn't find anything on your website. And how is this formula related to the piece of C++ code you provided at the beginning of this thread? It wasn't reall obvious what that code did. Some explanation would be appreciated!

Rein

Rémi Coulom
Posts: 432
Joined: Mon Apr 24, 2006 6:06 pm
Contact:

Re: Likelihood of superiority

Post by Rémi Coulom » Mon Nov 16, 2009 2:33 pm

Rein Halbersma wrote:
Rémi Coulom wrote:More precisely, here is a mathematical proof. Probabilities of win, draw, and loss are p_1, p_0.5, and p_0. Number of wins, draws, losses are n_1, n_0.5, and n_0. With a uniform prior:

Rémi
Very interesting! Could you give a reference from the paper where this derivation is taken from? I couldn't find anything on your website. And how is this formula related to the piece of C++ code you provided at the beginning of this thread? It wasn't reall obvious what that code did. Some explanation would be appreciated!

Rein
I had planned to write a paper about testing statistics some years ago, but gave up. The current draft really contains nothing interesting, I don't plan to release it.

With a little math, you can prove that:
P(p1>p0 | n1,n0) = ( P(p1>p0 | n1-1,n0) + P(p1>p0 | n1,n0-1) ) / 2
with
P(p1>p0 | n1,-1) = 1
P(p1>p0 | -1,n0) = 0

So the probabilities can be computed by a kind of Pascal triangle.

Rémi

Rein Halbersma
Posts: 685
Joined: Tue May 22, 2007 9:13 am

Re: Likelihood of superiority

Post by Rein Halbersma » Mon Nov 16, 2009 2:46 pm

Rémi Coulom wrote:
Rein Halbersma wrote:
Rémi Coulom wrote:More precisely, here is a mathematical proof. Probabilities of win, draw, and loss are p_1, p_0.5, and p_0. Number of wins, draws, losses are n_1, n_0.5, and n_0. With a uniform prior:

Rémi
Very interesting! Could you give a reference from the paper where this derivation is taken from? I couldn't find anything on your website. And how is this formula related to the piece of C++ code you provided at the beginning of this thread? It wasn't reall obvious what that code did. Some explanation would be appreciated!

Rein
I had planned to write a paper about testing statistics some years ago, but gave up. The current draft really contains nothing interesting, I don't plan to release it.

With a little math, you can prove that:
P(p1>p0 | n1,n0) = ( P(p1>p0 | n1-1,n0) + P(p1>p0 | n1,n0-1) ) / 2
with
P(p1>p0 | n1,-1) = 1
P(p1>p0 | -1,n0) = 0

So the probabilities can be computed by a kind of Pascal triangle.

Rémi
Interesting. I computed the integral with Mathematica and got an expression in terms of Gamma functions and hypergeometric functions. I did some manipulation but couldn't find a simplified expression. I'll try to see what solving the recurrence relation gives.

Rein

Rémi Coulom
Posts: 432
Joined: Mon Apr 24, 2006 6:06 pm
Contact:

Re: Likelihood of superiority

Post by Rémi Coulom » Mon Nov 16, 2009 2:59 pm

Rein Halbersma wrote: Interesting. I computed the integral with Mathematica and got an expression in terms of Gamma functions and hypergeometric functions. I did some manipulation but couldn't find a simplified expression. I'll try to see what solving the recurrence relation gives.

Rein
I did not manage to find an easy way. It took me a lot of effort to prove it.

It looks so simple that there may be an easy way. I did not manage to build an intuition for that formula.

I found it by writing numerical values for p(p1>p0|n1,n0) in a table.

It must be a classic.

Rémi

Rein Halbersma
Posts: 685
Joined: Tue May 22, 2007 9:13 am

Re: Likelihood of superiority

Post by Rein Halbersma » Mon Nov 16, 2009 3:07 pm

Rémi Coulom wrote:
Rein Halbersma wrote: Interesting. I computed the integral with Mathematica and got an expression in terms of Gamma functions and hypergeometric functions. I did some manipulation but couldn't find a simplified expression. I'll try to see what solving the recurrence relation gives.

Rein
I did not manage to find an easy way. It took me a lot of effort to prove it.

It looks so simple that there may be an easy way. I did not manage to build an intuition for that formula.

I found it by writing numerical values for p(p1>p0|n1,n0) in a table.

It must be a classic.

Rémi
Yes it looks simple, very similar to a midpoint rule for solving partial differential equations. Yet, Mathematica is unable to solve the recurrence relation. Tonight, I'll try to see if it can prove the recurrence relation directly on your integral expression.

Hart

Re: Likelihood of superiority

Post by Hart » Wed Nov 18, 2009 3:13 am

How would this compare to using a z-test on a proportion? I always thought that is how it was calculated but when I compare my numbers to yours they look very different.

Post Reply