Page 2 of 2

Re: Basic automated testing

Posted: Mon Oct 01, 2018 3:37 pm
by mar
AndrewGrant wrote: Mon Oct 01, 2018 12:51 am My instance is running here http://chess.grantnet.us/index/ , if you want to get an idea of the work flow.

You would be able to connect any of your machines, and the results will be collected together.

Time controls will be scaled per CPU, IE in the case that one machine is faster or something of the sort.

In case its not clear, I wrote OpenBench
That looks really nice. How do you supply test binaries? It seems you use branches in git so I assume it just fetches the branches and rebuilds on client?

Re: Basic automated testing

Posted: Mon Oct 01, 2018 8:13 pm
by AndrewGrant
mar wrote: Mon Oct 01, 2018 3:37 pm
AndrewGrant wrote: Mon Oct 01, 2018 12:51 am My instance is running here http://chess.grantnet.us/index/ , if you want to get an idea of the work flow.

You would be able to connect any of your machines, and the results will be collected together.

Time controls will be scaled per CPU, IE in the case that one machine is faster or something of the sort.

In case its not clear, I wrote OpenBench
That looks really nice. How do you supply test binaries? It seems you use branches in git so I assume it just fetches the branches and rebuilds on client?
Correct. OpenBench assumes that gcc/g++ and make are on the system path. It will pull down a zip file from github -- no actual git comments are taking place.

Re: Basic automated testing

Posted: Mon Oct 01, 2018 8:22 pm
by AndrewGrant
cdani wrote: Mon Oct 01, 2018 5:53 am
AndrewGrant wrote: Mon Oct 01, 2018 12:51 am
mar wrote: Sun Sep 30, 2018 10:40 pm
Ratosh wrote: Fri Sep 28, 2018 8:31 pm I'd recommend OpenBench.
Interesting, what does this OpenBench do exactly? It seems like a wrapper on top of cutechess-cli with some client-server functionality.
If it allows me say to test on say two quads at the same time + gather the results then this would cut my testing time in half,
which sounds really interesting.
My instance is running here http://chess.grantnet.us/index/ , if you want to get an idea of the work flow.

You would be able to connect any of your machines, and the results will be collected together.

Time controls will be scaled per CPU, IE in the case that one machine is faster or something of the sort.

In case its not clear, I wrote OpenBench
I knew you have it, but I find is good time to congratulate you for your nice and interesting work! :-)
Thanks. The same can be said for Andscacs :)

Re: Basic automated testing

Posted: Wed Oct 03, 2018 12:27 am
by jdart
What do your cutechess-cli scripts do? This might be what I am looking for (OpenBench looks too advanced for my current needs)
They are pretty simple. They distribute a compiled binary to a set of directories on a set of networked machines. Then they run a bunch of parallel matches on each machine. Then they gather all the completed games into a big game file and run BayesELO on it.

Unfortunately I have not bothered to make them very portable or reusable, so they have some hardcoded machine names, paths, etc.

--Jon