SF Progression since Fishtest inception

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
jdart
Posts: 3835
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: SF Progression since Fishtest inception

Post by jdart » Thu Apr 27, 2017 11:52 pm

I am very impressed, especially since SF 2.3.1 was already far from a weak engine.

With the current fishtest, how many cores does a LTC test typically run on?

--Jon

MikeB
Posts: 3541
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: SF Progression since Fishtest inception

Post by MikeB » Fri Apr 28, 2017 1:12 am

jdart wrote:I am very impressed, especially since SF 2.3.1 was already far from a weak engine.

With the current fishtest, how many cores does a LTC test typically run on?

--Jon
it generally varies from 400 to 600, click the link below to see the current active cores...

http://tests.stockfishchess.org/tests

MikeB
Posts: 3541
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: SF Progression since Fishtest inception

Post by MikeB » Fri Apr 28, 2017 1:30 am

Laskos wrote:Impressive test! I understand that there are 57 engines with 100 games against each other engine, total 5,600 games per engine, right? (57*56/2)*100 = 159,600
And the error margins for 95% confidence are 6-7 ELO points, right? As the monthly difference is on average about 10 ELO points, even successive results are statistically significant, without saying that those separated by several months are highly significant.

The pace of progress is impressive, probably only matched by Rybka 1 -> Rybka 3 progress. We also must note that Komodo's and Houdini's progress was also impressive, to almost match that of Stockfish. The last 5 or so years brought competent testing frameworks based on scientific method and a UI like Cutechess-Cli.
I did not check the math, but I thought it was closer to 80 ELO/12 months, still impressive - but 120 ELO per year sounds high to me.

Post Reply