Test suites

[laurietunnicliffe]

Hey,
I want to use a test suite to rate my program.
There seems to be a few that I can download BUT
I cant seem to find how you use the score to calculate an ELO.
Any help ???

[smatovic]

Hi,

you can get an Elo estimation by yourself via STS++

https://talkchess.com/forum3/viewtopic. ... 10#p945710

Have fun.

--
Srdja

You can take some rated engines from CCRL, run STS, and then compare, or alike.

STS Rating v14.1
Engine: Stockfish 15
Hash: 32, Threads: 1, time/pos: 0.100s

Number of positions in ./epd/STS1-STS15_LAN_v4.epd: 1500
Max score = 1500 x 10 = 15000
Test duration: 00h:02m:43s
Expected time to finish: 00h:03m:15s

STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 74 66 76 74 86 70 65 66 64 73 65 69 73 72 59 1052
Score 928 878 909 921 954 950 879 872 838 925 875 903 906 899 893 13530
Score(%) 92.8 87.8 90.9 92.1 95.4 95.0 87.9 87.2 83.8 92.5 87.5 90.3 90.6 89.9 89.3 90.2

[laurietunnicliffe]

That is a automated one.
I would like to know the formula as to how
calculate the ELO after the tests.

[smatovic]

***edit***

There is still

Code: Select all

  print("--getrating, calculate CCRL 40/4 rating estimate for uci engines only")"

in Ferdy's repo.

From the notes in the README:

The program --getrating flag is only applicable for the STS1-STS15_LAN_v3.epd test.

Dunno from which year the CCRL Elo raing will be.

....

There was once, from 2015, an CCRL Elo estimation inlcuded:

https://talkchess.com/forum3/viewtopic. ... ce#p628006
https://talkchess.com/forum3/viewtopic. ... ce#p628023

CCRL Elo is a moving target, CCRL did archive some old games, if you are intersted in 2015 CCRL Elo estimation, you can use for example from my repo:

https://github.com/smatovic/ZetaDva/blo ... 5C2-L135C2

with:
https://github.com/smatovic/ZetaDva/blo ... 15_LAN.EPD


--
Srdja