comparing opening books

[flok]

I'm experimenting with creating an opening book for Dog.
I've created 3 versions that I'd like to compare.
Using cutechess-cli I can let Dog play against itself using the 3 versions of the book. Using sprt cc-cli says it can decide on the result in 137 games:

Code: Select all

Rank Name                          Elo     +/-   Games   Score    Draw 
   1 v1                             80      53      97   61.3%   42.3% 
   2 v2                            -29      49      97   45.9%   50.5% 
   3 v3                            -61      56      80   41.3%   47.5% 

I'm a bit sceptic about that 137-number.
I use tc=8+0.08 and `-sprt elo0=0 elo1=10 alpha=0.05 beta=0.05'.
If I sprt with -5/5, I get:

Code: Select all

Rank Name                          Elo     +/-   Games   Score    Draw 
   1 1                              41      47      93   55.9%   55.9% 
   2 2                              11      46      93   51.6%   58.1% 
   3 3                             -61      50      80   41.3%   57.5% 

Also in +/- 137 games (suspicious).

Is this the path to walk? Or should I take a different approach?

[flok]

Code: Select all

Rank Name                          Elo     +/-   Games   Score    Draw 
   1 1                              42      12    2008   56.0%   40.3% 
   2 2                             -13      12    2008   48.1%   41.2% 
   3 3                             -29      12    2008   45.9%   40.4% 

This is with tc=1+0. The oracle (chatgpt) also says this is the way to test this, so I guess my new books just suck