Page 2 of 2
Re: SF Progression since Fishtest inception
Posted: Fri Apr 28, 2017 1:52 am
by jdart
I am very impressed, especially since SF 2.3.1 was already far from a weak engine.
With the current fishtest, how many cores does a LTC test typically run on?
--Jon
Re: SF Progression since Fishtest inception
Posted: Fri Apr 28, 2017 3:12 am
by MikeB
jdart wrote:I am very impressed, especially since SF 2.3.1 was already far from a weak engine.
With the current fishtest, how many cores does a LTC test typically run on?
--Jon
it generally varies from 400 to 600, click the link below to see the current active cores...
http://tests.stockfishchess.org/tests
Re: SF Progression since Fishtest inception
Posted: Fri Apr 28, 2017 3:30 am
by MikeB
Laskos wrote:Impressive test! I understand that there are 57 engines with 100 games against each other engine, total 5,600 games per engine, right? (57*56/2)*100 = 159,600
And the error margins for 95% confidence are 6-7 ELO points, right? As the monthly difference is on average about 10 ELO points, even successive results are statistically significant, without saying that those separated by several months are highly significant.
The pace of progress is impressive, probably only matched by Rybka 1 -> Rybka 3 progress. We also must note that Komodo's and Houdini's progress was also impressive, to almost match that of Stockfish. The last 5 or so years brought competent testing frameworks based on scientific method and a UI like Cutechess-Cli.
I did not check the math, but I thought it was closer to 80 ELO/12 months, still impressive - but 120 ELO per year sounds high to me.