I have a question for everyone: what is your basic approach to automated testing? What tools do you use? Do you test engine strength only, or do you test for other things as well?
I want to get a workflow set up where if I make a change in the evening, I can click a button before I go to sleep and know by morning if I broke anything critical, and how much of an improvement the change made.
Basic automated testing
Moderators: hgm, Rebel, chrisw
-
- Posts: 96
- Joined: Fri Jul 06, 2018 1:09 am
- Location: Chicago, IL
- Full name: Josh Odom
-
- Posts: 7218
- Joined: Mon May 27, 2013 10:31 am
Re: Basic automated testing
Problem with writing many test cases is that you have to maintain them. So if your interfaces change you have to rewrite many test cases as well.
So it is not for lazy developers with changing ideas every other day.
By the way if you write test cases at the end of development when interfaces are stable you are too late.
I usually write the least number of test cases. The less code the better. (*yawn*)
So it is not for lazy developers with changing ideas every other day.
By the way if you write test cases at the end of development when interfaces are stable you are too late.
I usually write the least number of test cases. The less code the better. (*yawn*)
-
- Posts: 77
- Joined: Mon Apr 16, 2018 6:56 pm
Re: Basic automated testing
I'd recommend OpenBench.
-
- Posts: 1871
- Joined: Sat Nov 25, 2017 2:28 pm
- Location: France
Re: Basic automated testing
Looks interesting. Can you describe its use a little more ?
-
- Posts: 266
- Joined: Fri Jul 10, 2015 9:23 pm
- Location: Russia
Re: Basic automated testing
I use tests embedded into the my engine. The tests can be found in the hedgehog source code. But it depends on what you are expecting to test out.
Eugene Kotlov
Hedgehog 2.1 64-bit coming soon...
Hedgehog 2.1 64-bit coming soon...
-
- Posts: 2555
- Joined: Fri Nov 26, 2010 2:00 pm
- Location: Czech Republic
- Full name: Martin Sedlak
Re: Basic automated testing
Interesting, what does this OpenBench do exactly? It seems like a wrapper on top of cutechess-cli with some client-server functionality.
If it allows me say to test on say two quads at the same time + gather the results then this would cut my testing time in half,
which sounds really interesting.
Martin Sedlak
-
- Posts: 1754
- Joined: Tue Apr 19, 2016 6:08 am
- Location: U.S.A
- Full name: Andrew Grant
Re: Basic automated testing
My instance is running here http://chess.grantnet.us/index/ , if you want to get an idea of the work flow.mar wrote: ↑Sun Sep 30, 2018 10:40 pmInteresting, what does this OpenBench do exactly? It seems like a wrapper on top of cutechess-cli with some client-server functionality.
If it allows me say to test on say two quads at the same time + gather the results then this would cut my testing time in half,
which sounds really interesting.
You would be able to connect any of your machines, and the results will be collected together.
Time controls will be scaled per CPU, IE in the case that one machine is faster or something of the sort.
In case its not clear, I wrote OpenBench
#WeAreAllDraude #JusticeForDraude #RememberDraude #LeptirBigUltra
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
"Those who can't do, clone instead" - Eduard ( A real life friend, not this forum's Eduard )
-
- Posts: 4366
- Joined: Fri Mar 10, 2006 5:23 am
- Location: http://www.arasanchess.org
Re: Basic automated testing
I have a bunch of unit tests that include basic functionality. One of the components is perft (https://www.chessprogramming.org/Perft), which is quite useful for sanity checking.
This is apart from performance testing (performance in games), which is really another topic. I use cutechess-cli for that, driven by a bunch of shell scripts.
--Jon
This is apart from performance testing (performance in games), which is really another topic. I use cutechess-cli for that, driven by a bunch of shell scripts.
--Jon
-
- Posts: 2204
- Joined: Sat Jan 18, 2014 10:24 am
- Location: Andorra
Re: Basic automated testing
I knew you have it, but I find is good time to congratulate you for your nice and interesting work!AndrewGrant wrote: ↑Mon Oct 01, 2018 12:51 amMy instance is running here http://chess.grantnet.us/index/ , if you want to get an idea of the work flow.mar wrote: ↑Sun Sep 30, 2018 10:40 pmInteresting, what does this OpenBench do exactly? It seems like a wrapper on top of cutechess-cli with some client-server functionality.
If it allows me say to test on say two quads at the same time + gather the results then this would cut my testing time in half,
which sounds really interesting.
You would be able to connect any of your machines, and the results will be collected together.
Time controls will be scaled per CPU, IE in the case that one machine is faster or something of the sort.
In case its not clear, I wrote OpenBench
Daniel José - http://www.andscacs.com
-
- Posts: 96
- Joined: Fri Jul 06, 2018 1:09 am
- Location: Chicago, IL
- Full name: Josh Odom
Re: Basic automated testing
What do your cutechess-cli scripts do? This might be what I am looking for (OpenBench looks too advanced for my current needs)jdart wrote: ↑Mon Oct 01, 2018 5:21 am I have a bunch of unit tests that include basic functionality. One of the components is perft (https://www.chessprogramming.org/Perft), which is quite useful for sanity checking.
This is apart from performance testing (performance in games), which is really another topic. I use cutechess-cli for that, driven by a bunch of shell scripts.
--Jon