Stockfish testing at STC and LTC: one question

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
jdart
Posts: 3924
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Stockfish testing at STC and LTC: one question

Post by jdart » Mon Sep 25, 2017 7:12 pm

Besides which, nobody has the resources to test at tournament time control.
That is true now, but I am not sure it is going to be always true.

If you had 2000 cores available, you could run a match of 50 games on each core and get 100,000 games. 50 games blitz (say 5+3) wouldn't take more than a few hours. 50 games rapid might take a day or so. And a lot of tests wouldn't take 100k games to show a significant result.

Stockfish has a few hundred cores in its testing network now. And the core counts on processors just keep going up.

--Jon

Uri Blass
Posts: 8756
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Stockfish testing at STC and LTC: one question

Post by Uri Blass » Mon Sep 25, 2017 9:05 pm

jdart wrote:
Besides which, nobody has the resources to test at tournament time control.
That is true now, but I am not sure it is going to be always true.

If you had 2000 cores available, you could run a match of 50 games on each core and get 100,000 games. 50 games blitz (say 5+3) wouldn't take more than a few hours. 50 games rapid might take a day or so. And a lot of tests wouldn't take 100k games to show a significant result.

Stockfish has a few hundred cores in its testing network now. And the core counts on processors just keep going up.

--Jon
It is something that I do not understand.

Why so many people choose to give computer time to stockfish when they are not being able even to choose the time control that they test and what they test.

I think that a better model should be to allow people who give computer time to choose the patch that they test if they want to do it when maybe the default option will be not to choose a specific patch.

patches that the team consider as not interesting may be in a special set and only if a person ask to test them they will get computer time.

Dann Corbit
Posts: 10843
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Stockfish testing at STC and LTC: one question

Post by Dann Corbit » Mon Sep 25, 2017 9:44 pm

Uri Blass wrote:
jdart wrote:
Besides which, nobody has the resources to test at tournament time control.
That is true now, but I am not sure it is going to be always true.

If you had 2000 cores available, you could run a match of 50 games on each core and get 100,000 games. 50 games blitz (say 5+3) wouldn't take more than a few hours. 50 games rapid might take a day or so. And a lot of tests wouldn't take 100k games to show a significant result.

Stockfish has a few hundred cores in its testing network now. And the core counts on processors just keep going up.

--Jon
It is something that I do not understand.

Why so many people choose to give computer time to stockfish when they are not being able even to choose the time control that they test and what they test.

I think that a better model should be to allow people who give computer time to choose the patch that they test if they want to do it when maybe the default option will be not to choose a specific patch.

patches that the team consider as not interesting may be in a special set and only if a person ask to test them they will get computer time.
I guess that if you did a test using their own time control multiplied by 10 and showed that it would pass they would accept it.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Post Reply