Beta for Stockfish distributed testing

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
gladius
Posts: 538
Joined: Tue Dec 12, 2006 9:10 am

Beta for Stockfish distributed testing

Post by gladius » Tue Mar 05, 2013 12:58 am

If you have at least 4 spare cores, preferably running Linux (but not required) Stockfish would love to get your help! We are testing changes with thousands of games, and the more cpu power, the faster Stockfish will improve :). If you are interested, please e-mail me at glinscott@gmail.com.

The current queue of tests for Stockfish is up at http://54.235.120.254:6543/tests. Please note, operations like start run, delete run, etc. are restricted.

Technical details:
Over the past few weeks, we've put together a distributed testing framework. It is written in Python, has a web interface backed by MongoDb for storing results. Clients get tasks from a REST API, and use cutechess-cli to run the actual games. Source is up on github here: https://github.com/glinscott/fishtest.

Thanks,
Gary

Daniel Shawul
Posts: 3757
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: Beta for Stockfish distributed testing

Post by Daniel Shawul » Tue Mar 05, 2013 2:02 am

Is that like SETI, the search for extraterestrial ELO :) Sorry i couldn't resist.
Great work!

Aaron Becker
Posts: 292
Joined: Tue Jul 07, 2009 2:56 am

Re: Beta for Stockfish distributed testing

Post by Aaron Becker » Tue Mar 05, 2013 7:28 am

This looks really great; I'd like to use it in my testing for Daydreamer. If you like, I'd be happy to contribute any modifications back upstream. Thanks for making this publicly available.

ThomasJMiller
Posts: 96
Joined: Fri Jan 11, 2013 11:22 am

Re: Beta for Stockfish distributed testing

Post by ThomasJMiller » Tue Mar 05, 2013 9:59 am

Very interesting project Gary! So this is what you were talking about some days ago @TCEC. I hope we'll soon have a stronger Stockfish! :)

User avatar
lucasart
Posts: 3040
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: Beta for Stockfish distributed testing

Post by lucasart » Tue Mar 05, 2013 11:08 am

gladius wrote:If you have at least 4 spare cores, preferably running Linux (but not required) Stockfish would love to get your help! We are testing changes with thousands of games, and the more cpu power, the faster Stockfish will improve :). If you are interested, please e-mail me at glinscott@gmail.com.

The current queue of tests for Stockfish is up at http://54.235.120.254:6543/tests. Please note, operations like start run, delete run, etc. are restricted.

Technical details:
Over the past few weeks, we've put together a distributed testing framework. It is written in Python, has a web interface backed by MongoDb for storing results. Clients get tasks from a REST API, and use cutechess-cli to run the actual games. Source is up on github here: https://github.com/glinscott/fishtest.

Thanks,
Gary
Great work. You've really industrialized the process :D

I'll gladly contribute to the testing.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

gladius
Posts: 538
Joined: Tue Dec 12, 2006 9:10 am

Re: Beta for Stockfish distributed testing

Post by gladius » Tue Mar 05, 2013 1:25 pm

Aaron Becker wrote:This looks really great; I'd like to use it in my testing for Daydreamer. If you like, I'd be happy to contribute any modifications back upstream. Thanks for making this publicly available.
Great! Yes, would love to have more folks working on it :).

gladius
Posts: 538
Joined: Tue Dec 12, 2006 9:10 am

Re: Beta for Stockfish distributed testing

Post by gladius » Tue Mar 05, 2013 1:28 pm

lucasart wrote:
gladius wrote:If you have at least 4 spare cores, preferably running Linux (but not required) Stockfish would love to get your help! We are testing changes with thousands of games, and the more cpu power, the faster Stockfish will improve :). If you are interested, please e-mail me at glinscott@gmail.com.

The current queue of tests for Stockfish is up at http://54.235.120.254:6543/tests. Please note, operations like start run, delete run, etc. are restricted.

Technical details:
Over the past few weeks, we've put together a distributed testing framework. It is written in Python, has a web interface backed by MongoDb for storing results. Clients get tasks from a REST API, and use cutechess-cli to run the actual games. Source is up on github here: https://github.com/glinscott/fishtest.

Thanks,
Gary
Great work. You've really industrialized the process :D

I'll gladly contribute to the testing.
Awesome, thanks Lucas! I will need to send over a username/password if you want to run the worker, or run tests.

It's really nice to be able to submit a test and forget about it until the results pop up :).

IGarcia
Posts: 543
Joined: Mon Jul 05, 2010 8:27 pm

Re: Beta for Stockfish distributed testing

Post by IGarcia » Tue Mar 05, 2013 6:30 pm

Has to be 4 cores, mandatory?

I have a core 2 (2 cores) under linux

Ignacio.

Vinvin
Posts: 4377
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: Beta for Stockfish distributed testing

Post by Vinvin » Tue Mar 05, 2013 7:15 pm

I've 2 cores (+2 with HT) too, is that good for SF testing ?

gladius
Posts: 538
Joined: Tue Dec 12, 2006 9:10 am

Re: Beta for Stockfish distributed testing

Post by gladius » Tue Mar 05, 2013 7:19 pm

IGarcia wrote:Has to be 4 cores, mandatory?

I have a core 2 (2 cores) under linux

Ignacio.
For the beta, at least 4 cores, yes. The goal is to find the bugs/problems in the system, so a smaller number of higher core machines is easier.

Thanks though :). I will definitely post when we are scaling things up some more.

Post Reply