Basic automated testing
Moderators: hgm, Dann Corbit, Harvey Williamson
Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Basic automated testing
I have a question for everyone: what is your basic approach to automated testing? What tools do you use? Do you test engine strength only, or do you test for other things as well?
I want to get a workflow set up where if I make a change in the evening, I can click a button before I go to sleep and know by morning if I broke anything critical, and how much of an improvement the change made.
I want to get a workflow set up where if I make a change in the evening, I can click a button before I go to sleep and know by morning if I broke anything critical, and how much of an improvement the change made.
Re: Basic automated testing
Problem with writing many test cases is that you have to maintain them. So if your interfaces change you have to rewrite many test cases as well.
So it is not for lazy developers with changing ideas every other day.
By the way if you write test cases at the end of development when interfaces are stable you are too late.
I usually write the least number of test cases. The less code the better. (*yawn*)
So it is not for lazy developers with changing ideas every other day.
By the way if you write test cases at the end of development when interfaces are stable you are too late.
I usually write the least number of test cases. The less code the better. (*yawn*)
Re: Basic automated testing
I'd recommend OpenBench.
Re: Basic automated testing
Looks interesting. Can you describe its use a little more ?
Re: Basic automated testing
I use tests embedded into the my engine. The tests can be found in the hedgehog source code. But it depends on what you are expecting to test out.
Eugene Kotlov
Hedgehog 2.1 64-bit coming soon...
Hedgehog 2.1 64-bit coming soon...
Re: Basic automated testing
Interesting, what does this OpenBench do exactly? It seems like a wrapper on top of cutechess-cli with some client-server functionality.
If it allows me say to test on say two quads at the same time + gather the results then this would cut my testing time in half,
which sounds really interesting.
Martin Sedlak
-
- Posts: 956
- Joined: Tue Apr 19, 2016 4:08 am
- Location: U.S.A
- Full name: Andrew Grant
- Contact:
Re: Basic automated testing
My instance is running here http://chess.grantnet.us/index/ , if you want to get an idea of the work flow.mar wrote: ↑Sun Sep 30, 2018 8:40 pmInteresting, what does this OpenBench do exactly? It seems like a wrapper on top of cutechess-cli with some client-server functionality.
If it allows me say to test on say two quads at the same time + gather the results then this would cut my testing time in half,
which sounds really interesting.
You would be able to connect any of your machines, and the results will be collected together.
Time controls will be scaled per CPU, IE in the case that one machine is faster or something of the sort.
In case its not clear, I wrote OpenBench
-
- Posts: 4078
- Joined: Fri Mar 10, 2006 4:23 am
- Location: http://www.arasanchess.org
Re: Basic automated testing
I have a bunch of unit tests that include basic functionality. One of the components is perft (https://www.chessprogramming.org/Perft), which is quite useful for sanity checking.
This is apart from performance testing (performance in games), which is really another topic. I use cutechess-cli for that, driven by a bunch of shell scripts.
--Jon
This is apart from performance testing (performance in games), which is really another topic. I use cutechess-cli for that, driven by a bunch of shell scripts.
--Jon
Re: Basic automated testing
I knew you have it, but I find is good time to congratulate you for your nice and interesting work!AndrewGrant wrote: ↑Sun Sep 30, 2018 10:51 pmMy instance is running here http://chess.grantnet.us/index/ , if you want to get an idea of the work flow.mar wrote: ↑Sun Sep 30, 2018 8:40 pmInteresting, what does this OpenBench do exactly? It seems like a wrapper on top of cutechess-cli with some client-server functionality.
If it allows me say to test on say two quads at the same time + gather the results then this would cut my testing time in half,
which sounds really interesting.
You would be able to connect any of your machines, and the results will be collected together.
Time controls will be scaled per CPU, IE in the case that one machine is faster or something of the sort.
In case its not clear, I wrote OpenBench

Daniel José -
http://www.andscacs.com

Re: Basic automated testing
What do your cutechess-cli scripts do? This might be what I am looking for (OpenBench looks too advanced for my current needs)jdart wrote: ↑Mon Oct 01, 2018 3:21 amI have a bunch of unit tests that include basic functionality. One of the components is perft (https://www.chessprogramming.org/Perft), which is quite useful for sanity checking.
This is apart from performance testing (performance in games), which is really another topic. I use cutechess-cli for that, driven by a bunch of shell scripts.
--Jon