Stockfish testing at STC and LTC: one question

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

jdart
Posts: 4366
Joined: Fri Mar 10, 2006 5:23 am
Location: http://www.arasanchess.org

Re: Stockfish testing at STC and LTC: one question

Post by jdart »

Besides which, nobody has the resources to test at tournament time control.
That is true now, but I am not sure it is going to be always true.

If you had 2000 cores available, you could run a match of 50 games on each core and get 100,000 games. 50 games blitz (say 5+3) wouldn't take more than a few hours. 50 games rapid might take a day or so. And a lot of tests wouldn't take 100k games to show a significant result.

Stockfish has a few hundred cores in its testing network now. And the core counts on processors just keep going up.

--Jon
Uri Blass
Posts: 10282
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Stockfish testing at STC and LTC: one question

Post by Uri Blass »

jdart wrote:
Besides which, nobody has the resources to test at tournament time control.
That is true now, but I am not sure it is going to be always true.

If you had 2000 cores available, you could run a match of 50 games on each core and get 100,000 games. 50 games blitz (say 5+3) wouldn't take more than a few hours. 50 games rapid might take a day or so. And a lot of tests wouldn't take 100k games to show a significant result.

Stockfish has a few hundred cores in its testing network now. And the core counts on processors just keep going up.

--Jon
It is something that I do not understand.

Why so many people choose to give computer time to stockfish when they are not being able even to choose the time control that they test and what they test.

I think that a better model should be to allow people who give computer time to choose the patch that they test if they want to do it when maybe the default option will be not to choose a specific patch.

patches that the team consider as not interesting may be in a special set and only if a person ask to test them they will get computer time.
Dann Corbit
Posts: 12540
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Stockfish testing at STC and LTC: one question

Post by Dann Corbit »

Uri Blass wrote:
jdart wrote:
Besides which, nobody has the resources to test at tournament time control.
That is true now, but I am not sure it is going to be always true.

If you had 2000 cores available, you could run a match of 50 games on each core and get 100,000 games. 50 games blitz (say 5+3) wouldn't take more than a few hours. 50 games rapid might take a day or so. And a lot of tests wouldn't take 100k games to show a significant result.

Stockfish has a few hundred cores in its testing network now. And the core counts on processors just keep going up.

--Jon
It is something that I do not understand.

Why so many people choose to give computer time to stockfish when they are not being able even to choose the time control that they test and what they test.

I think that a better model should be to allow people who give computer time to choose the patch that they test if they want to do it when maybe the default option will be not to choose a specific patch.

patches that the team consider as not interesting may be in a special set and only if a person ask to test them they will get computer time.
I guess that if you did a test using their own time control multiplied by 10 and showed that it would pass they would accept it.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.