In fact, without any Elo, expressing elo0 and elo1 as scores S0 and S1, for example elo0=0 => S0=0.5, elo1=5 =>S1=0.5073, one gets a simpler expression for 3-nomial LLR:Laskos wrote:Thanks for the script! Yes, this Richard's implementation is the most straightforward use of SPRT for 3- and 5-nomials, maybe Cutechess-Cli will follow. I tried to compute 3-nomial expression for LLR in closed form without substitutions, I hoped it will simplify a bit, but the most I get is still very long, and 5-nomial would be even longer. Just for fun, LLR for 3-nomial:Michel wrote:Thanks for the explanation. I put a link for the script computing LLR's here.Laskos wrote:It is probable that it's my fault. This stop (where test in not yet accepted) was my ad-hoc stop, the test was still running (it stops when both 3- and 5-nomial reach desired LLR), but I wanted to see what 3-nomial LLR is when 5-nomial stops. 2 others were stopped regularly (tests were accepted), should be ok to fine details.Michel wrote:There seems to be a puzzling deviation in the reported 3-nomial LLR's. E.g. for the last experiment I get
wdl: w: 231, d: 352, l: 187
LLR=1.02
whereas the reported LLR is 1.1.
The 5-nomial case is ok.
http://hardy.uhasselt.be/Toga/computeLLR.py
Now that there is an implementation in the Amoebe tourney manager perhaps cutechess-cli can follow? The code is really trivial. Much easier than the current code in cutechess-cli.
LLR = (2 N*(N - 1)*(S1 - S0) [draws (1 - S0 - S1) - (losses + wins) (S0 + S1) + 2 wins])/(draws (losses + wins) + 4 losses wins)