Is cutechess-cli's rating interval 1 sigma or 2 sigma?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
fierz
Posts: 62
Joined: Mon Mar 07, 2016 3:41 pm
Location: Zürich, Switzerland
Contact:

Is cutechess-cli's rating interval 1 sigma or 2 sigma?

Post by fierz » Fri May 18, 2018 3:29 pm

I can't see anything on the rating table that cutechess-cli produces in its documentation. Does anyone know if the +- corresponds to a 1 sigma (68%) or 2 sigma (95%) interval?

cheers
Martin

F. Bluemers
Posts: 860
Joined: Thu Mar 09, 2006 10:21 pm
Location: Nederland
Contact:

Re: Is cutechess-cli's rating interval 1 sigma or 2 sigma?

Post by F. Bluemers » Fri May 18, 2018 6:49 pm

cutechess-cli output:
Score of dirty vs dirtyx: 5512 - 4797 - 4638 [0.524] 14947
Elo difference: 16.63 +/- 4.62

in dirty elo tool (WDL):
eb 5512 4638 4797
80% confidence 14 <= 17 <= 20
90% confidence 13 <= 17 <= 21
95% confidence 12 <= 17 <= 21
98% confidence 11 <= 17 <= 22
99% confidence 11 <= 17 <= 23
looks like 95%

fierz
Posts: 62
Joined: Mon Mar 07, 2016 3:41 pm
Location: Zürich, Switzerland
Contact:

Re: Is cutechess-cli's rating interval 1 sigma or 2 sigma?

Post by fierz » Sat May 19, 2018 7:24 am

Thanks! I was worried a bit about the size of my confidence intervals but if they are 95% then it's not as bad as I thought (I usually use 1 sigma by default in anything I do...)

Post Reply