Interesting, what does this OpenBench do exactly? It seems like a wrapper on top of cutechess-cli with some client-server functionality.
If it allows me say to test on say two quads at the same time + gather the results then this would cut my testing time in half,
which sounds really interesting.
You would be able to connect any of your machines, and the results will be collected together.
Time controls will be scaled per CPU, IE in the case that one machine is faster or something of the sort.
In case its not clear, I wrote OpenBench
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra "Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
I have a bunch of unit tests that include basic functionality. One of the components is perft (https://www.chessprogramming.org/Perft), which is quite useful for sanity checking.
This is apart from performance testing (performance in games), which is really another topic. I use cutechess-cli for that, driven by a bunch of shell scripts.
Interesting, what does this OpenBench do exactly? It seems like a wrapper on top of cutechess-cli with some client-server functionality.
If it allows me say to test on say two quads at the same time + gather the results then this would cut my testing time in half,
which sounds really interesting.
jdart wrote: ↑Mon Oct 01, 2018 5:21 am
I have a bunch of unit tests that include basic functionality. One of the components is perft (https://www.chessprogramming.org/Perft), which is quite useful for sanity checking.
This is apart from performance testing (performance in games), which is really another topic. I use cutechess-cli for that, driven by a bunch of shell scripts.
--Jon
What do your cutechess-cli scripts do? This might be what I am looking for (OpenBench looks too advanced for my current needs)
You would be able to connect any of your machines, and the results will be collected together.
Time controls will be scaled per CPU, IE in the case that one machine is faster or something of the sort.
In case its not clear, I wrote OpenBench
That looks really nice. How do you supply test binaries? It seems you use branches in git so I assume it just fetches the branches and rebuilds on client?
You would be able to connect any of your machines, and the results will be collected together.
Time controls will be scaled per CPU, IE in the case that one machine is faster or something of the sort.
In case its not clear, I wrote OpenBench
That looks really nice. How do you supply test binaries? It seems you use branches in git so I assume it just fetches the branches and rebuilds on client?
Correct. OpenBench assumes that gcc/g++ and make are on the system path. It will pull down a zip file from github -- no actual git comments are taking place.
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra "Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
Interesting, what does this OpenBench do exactly? It seems like a wrapper on top of cutechess-cli with some client-server functionality.
If it allows me say to test on say two quads at the same time + gather the results then this would cut my testing time in half,
which sounds really interesting.
You would be able to connect any of your machines, and the results will be collected together.
Time controls will be scaled per CPU, IE in the case that one machine is faster or something of the sort.
In case its not clear, I wrote OpenBench
I knew you have it, but I find is good time to congratulate you for your nice and interesting work!
Thanks. The same can be said for Andscacs
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra "Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
What do your cutechess-cli scripts do? This might be what I am looking for (OpenBench looks too advanced for my current needs)
They are pretty simple. They distribute a compiled binary to a set of directories on a set of networked machines. Then they run a bunch of parallel matches on each machine. Then they gather all the completed games into a big game file and run BayesELO on it.
Unfortunately I have not bothered to make them very portable or reusable, so they have some hardcoded machine names, paths, etc.