Re: SPRT question
Posted: Fri Aug 21, 2015 2:55 pm
A quick back of the envelope calculation shows that under the following conditionsMichel wrote: It is not clear to me if and when the SPRT will terminate with probabilty one if the number of games of A is kept fixed (it seems like an easy problem, but I have not taken the time to consider it properly). Obviously if eloA is only vaguely known, and the difference between eloA and eloB is small, one will never be able to prove there is a difference no matter how many games B plays (recall that A and B are not playing each other).
(0) The number of games of A is kept constant.
(1) eloA=eloA' (i.e. the measured elo of A, which stays constant, is equal to the true elo).
(2) eloB=eloA+epsilon/2 (i.e. halfway between H0 and H1)
there is a non-zero probability that the SPRT will not terminate.
Even if H0 or H1 is true _and epsilon is below some easily calculated bound_, depending on the number of games played by A, there will be a non-zero probability that the SPRT does not terminate.
So the conclusion is that when using the SPRT in this fashion one should add games for both A and B (although probably not in the same ratio).