Page 14 of 18
Re: Cutechess-cli: A command line tool for engine-engine mat
Posted: Sat Aug 22, 2009 10:12 pm
by mcostalba
ilari wrote:Gian-Carlo Pascutto wrote:Next up the wish-list: SMP support. I know you can launch separate instances, but the tournaments don't necessarily stop at the same time.
I've already planned support for concurrent tournament games, once tournaments have been implemented. If you wish I can add concurrency support for matches between two engines as well.
Hi Ilari,
I would be very interested in concurrency support for matches between two engines.
We would like to start as example 8 threads in parallel so to play 4 games, each one between two versions of the same engine.
Currently we need to start 4 separate instances of cutechess, but this is not an optimal solution because you need to group the results at the end, while if cutechess would be able to handle that you have only one pgn file and one results output. And also you have to create 4 directories with engines, books, etc..
We cannot say is an elegant solution
I have cloned the latest repo but I didn't see this feature. Are you planning to support that ?
Thanks
Marco
Re: Cutechess-cli: A command line tool for engine-engine mat
Posted: Sun Aug 23, 2009 4:17 am
by Edsel Apostol
mcostalba wrote:ilari wrote:Gian-Carlo Pascutto wrote:Next up the wish-list: SMP support. I know you can launch separate instances, but the tournaments don't necessarily stop at the same time.
I've already planned support for concurrent tournament games, once tournaments have been implemented. If you wish I can add concurrency support for matches between two engines as well.
Hi Ilari,
I would be very interested in concurrency support for matches between two engines.
We would like to start as example 8 threads in parallel so to play 4 games, each one between two versions of the same engine.
Currently we need to start 4 separate instances of cutechess, but this is not an optimal solution because you need to group the results at the end, while if cutechess would be able to handle that you have only one pgn file and one results output. And also you have to create 4 directories with engines, books, etc..
We cannot say is an elegant solution
I have cloned the latest repo but I didn't see this feature. Are you planning to support that ?
Thanks
Marco
I am also interested in this feature. In my testing I only used one pgn output for 4 matches running at the same time and in around 1200 games there would be around 1 or 2 games missing as they might be written at the same time by one of the cutechess instances. If concurrency is supported I think that it wouldn't be an issue anymore.
It would also be good to have support for epd as starting positions. I can't seem to find this feature if its already been implemented.
And another feature request is the ability to turn off the computer after a match or a tournament just like in Arena.
Re: Cutechess-cli: A command line tool for engine-engine mat
Posted: Sun Aug 23, 2009 9:31 am
by ilari
mcostalba wrote:ilari wrote:Gian-Carlo Pascutto wrote:Next up the wish-list: SMP support. I know you can launch separate instances, but the tournaments don't necessarily stop at the same time.
I've already planned support for concurrent tournament games, once tournaments have been implemented. If you wish I can add concurrency support for matches between two engines as well.
Hi Ilari,
I would be very interested in concurrency support for matches between two engines.
We would like to start as example 8 threads in parallel so to play 4 games, each one between two versions of the same engine.
Hi,
Something like that is coming. We've both been busy this summer, so development has been somewhat slow. But we have designed a GameManager class that can handle multiple concurrent and/or queued games, and it will be used for both engine-engine matches and tournaments.
If we were just developing an application, Cute Chess 1.0 would probably be finished already. But we're also trying to write a powerful, yet easy to use library that others can use to make their own chess tools, guis, etc. That's taking most of the time, because we have to make it easy to extend the library, provide a very high abstraction level, complete API documentation, maybe even binary compatibility between different versions, etc.
Re: Cutechess-cli: A command line tool for engine-engine mat
Posted: Sun Aug 23, 2009 9:38 am
by ilari
Edsel Apostol wrote:It would also be good to have support for epd as starting positions. I can't seem to find this feature if its already been implemented.
That's on my TODO list, but there are many high-priority tasks that we'll have to do first.
And another feature request is the ability to turn off the computer after a match or a tournament just like in Arena.
That's easy, but also annoyingly platform-specific, so we'll see what we can do.
Re: Cutechess-cli: A command line tool for engine-engine mat
Posted: Sun Aug 23, 2009 11:52 am
by Tord Romstad
Edsel Apostol wrote:It would also be good to have support for epd as starting positions. I can't seem to find this feature if its already been implemented.
Just write a little program which takes an EPD file as input and spits out a PGN file. I use the following Haskell two-liner:
Code: Select all
pgnify = (\x -> "[FEN "" ++ x ++ ""]\n*\n") . unwords . take 4 . words
main = interact $ unlines . map pgnify . lines
Use like this:
Code: Select all
cat silver.epd | epdtopgn > silver.pgn
Re: Cutechess-cli: A command line tool for engine-engine mat
Posted: Sun Aug 23, 2009 12:07 pm
by mcostalba
ilari wrote:
Hi,
Something like that is coming. We've both been busy this summer, so development has been somewhat slow. But we have designed a GameManager class that can handle multiple concurrent and/or queued games, and it will be used for both engine-engine matches and tournaments.
If we were just developing an application, Cute Chess 1.0 would probably be finished already. But we're also trying to write a powerful, yet easy to use library that others can use to make their own chess tools, guis, etc. That's taking most of the time, because we have to make it easy to extend the library, provide a very high abstraction level, complete API documentation, maybe even binary compatibility between different versions, etc.
Thanks !
I have seen from the git log that you have rewritten a lot of stuff lately
Re: Cutechess-cli: A command line tool for engine-engine mat
Posted: Sun Aug 23, 2009 12:37 pm
by ilari
mcostalba wrote:Thanks !
I have seen from the git log that you have rewritten a lot of stuff lately
Yes, and there's always more rewriting to do. For example the rewritten engine configuration classes will probably be rewritten again. But as long as the next iteration is better than the previous one, it's all good.
Cutechess-cli version 0.1.8 released
Posted: Tue Sep 29, 2009 2:21 am
by ilari
It's been a loooong time since the last binary update and version number bump. However, a lot of changes were made between 0.1.7 and 0.1.8, most of them under the hood. These are the visible/interesting ones:
* Draw claims that happen on the opponent's turn are evaluated immediately
* When using PGN games as an opening book, the starting side is now set correctly according to the "FEN" tag (if any)
* Fixed a bug that caused some legal en-passant moves in SAN format to be rejected
* Fixed a bug that sometimes caused a crash when using SAN move format with an Xboard engine (like Crafty)
* Fixed a bug where cutechess-cli would try to terminate an already terminated engine
* Some optimizations, especially in move generation and parsing moves
* SIGINT (Ctrl-C) now terminates the engines forcefully
* Scale move time's decimal precision in PGN comments based on the time
* Support for Polyglot castling moves where the king "captures" a friendly rook
* Detect mate-in-n scores and mark them properly in PGN output. This works with all UCI engines and most Xboard engines.
* Parse Xboard engines' thinking output more flexibly
* Add a timestamp to every engine debug message
* The engine option code has been rewritten. As a result, generic engine options 'cpus', 'egtpath' and 'egbbpath' are no longer available (for now). "option.option_name=option_value" still works for both UCI and Xboard engines.
* The match won't be started if the opening book or PGN input file can't be read
* There's a README file in the package that explains all the options
* THE ABILITY TO RUN CONCURRENT GAMES!
The last feature is probably the most exciting one, and the most requested one. There's a new option, '-concurrency <n>' where <n> is the maximum number of games to run concurrently. The default is of course 1. Concurrency shouldn't cause any side effects in cutechess-cli - games are started in the right order and they're saved in PGN output in the right order. That said, I can't guarantee that the engines also won't have side effects - if there are multiple instances of an engine using the same learning, log, or other output file at the same time, ugly things can happen.
I've tested the concurrency feature with tens of thousands of superfast games on 64-bit Linux, and it seems to work. On Windows and Mac I didn't test much, so let's hope everything is fine on those platforms as well. I look forward to any feedback and bug reports.
As always, download links are in the first message of this thread. Some of the other information in that message may not be true/relevant anymore so I recommend reading the README file that comes with the program.
Re: Cutechess-cli: A command line tool for engine-engine mat
Posted: Tue Sep 29, 2009 2:42 am
by Dann Corbit
It would be very nice if it could support USI also (there is no tournament GUI for USI at all that I know of, though you can play a single engine against a single engine for N games with Shogidokoro):
http://www.glaurungchess.com/shogi/usi.html
Re: Cutechess-cli version 0.1.8 released
Posted: Tue Sep 29, 2009 9:14 am
by mcostalba
Great job !!!!
Thanks a lot, especially for concurrency that is ver very welcomed
