Cutechess-cli: A command line tool for engine-engine matches

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 7:45 pm
Location: Finland

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by ilari »

Zach Wegner wrote:
IanO wrote:Doesn't work for UCI engines on OS X 10.4

$ ./cutechess-cli.sh -fcp cmd=crafty proto=xboard -scp cmd=homer "dir=/Applications/Games/Sigma Chess 6.1/Homer" proto=uci -both tc=60/300 -pgnout crafty-homer.pgn -debug
Warning: Cannot start engine "homer"
Warning: QProcess: Destroyed while process is still running.

I confirmed that I can start UCI engines from within their home directory.
Change "cmd=homer" to "cmd=./homer", and make sure you have execute permissions on the engine. Just figured this out myself about 15 minutes ago. ;)
Correct. If the path to the executable isn't included in the PATH environment variable, "./" has to be inserted before the command to tell the shell that we're executing something that's in the working directory. It's a Unix thing. In Windows you can just use the executable name.

And if the engine is a Windows executable and you're running Linux/OSX, you can use cmd="wine engine.exe", assuming that Wine is installed.
IanO
Posts: 496
Joined: Wed Mar 08, 2006 9:45 pm
Location: Portland, OR

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by IanO »

ilari wrote:
Zach Wegner wrote:
IanO wrote:Doesn't work for UCI engines on OS X 10.4

$ ./cutechess-cli.sh -fcp cmd=crafty proto=xboard -scp cmd=homer "dir=/Applications/Games/Sigma Chess 6.1/Homer" proto=uci -both tc=60/300 -pgnout crafty-homer.pgn -debug
Warning: Cannot start engine "homer"
Warning: QProcess: Destroyed while process is still running.

I confirmed that I can start UCI engines from within their home directory.
Change "cmd=homer" to "cmd=./homer", and make sure you have execute permissions on the engine. Just figured this out myself about 15 minutes ago. ;)
Correct. If the path to the executable isn't included in the PATH environment variable, "./" has to be inserted before the command to tell the shell that we're executing something that's in the working directory. It's a Unix thing. In Windows you can just use the executable name.

And if the engine is a Windows executable and you're running Linux/OSX, you can use cmd="wine engine.exe", assuming that Wine is installed.
"cmd=./homer" also didn't work. I only got it to work by running within the Homer directory (cmd=./homer dir=.)

If I use the absolute path, "cmd=/Applications/Games/Sigma Chess 6.1/Homer/homer", I get:

Warning: Cannot start engine "/Applications/Games/Sigma"

which points out problems in either your tool's or QProcess::start()'s command line parsing. (It is a common bug that tools don't handle spaces in file/directory names correctly.)
User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 7:45 pm
Location: Finland

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by ilari »

IanO wrote:"cmd=./homer" also didn't work. I only got it to work by running within the Homer directory (cmd=./homer dir=.)

If I use the absolute path, "cmd=/Applications/Games/Sigma Chess 6.1/Homer/homer", I get:

Warning: Cannot start engine "/Applications/Games/Sigma"

which points out problems in either your tool's or QProcess::start()'s command line parsing. (It is a common bug that tools don't handle spaces in file/directory names correctly.)
You're right, it's the space. But the problem is not in my program or QProcess, it's in the startup script cutechess-cli.sh which removes quotation marks from the command. You can fix the problem by replacing $* with "$@" on the last line. So it becomes:

Code: Select all

$dirname/$appname "$@"
The quotation marks are important there.

I'm going to release a new version of Cutechess-cli shortly with a couple of fixes and new features.
User avatar
WinPooh
Posts: 267
Joined: Fri Mar 17, 2006 8:01 am
Location: Russia
Full name: Vladimir Medvedev

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by WinPooh »

Thank you for a nice tool!
One question: does it support ultra-fast time controls (like 0.1 sec per whole game)?
User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 7:45 pm
Location: Finland

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by ilari »

WinPooh wrote:Thank you for a nice tool!
One question: does it support ultra-fast time controls (like 0.1 sec per whole game)?
Currently 1 sec per game is the fastest possible. Internally all time handling is done in milliseconds, so in theory it would work if the "tc" command line argument supported milliseconds. But in practice most engines wouldn't be able to handle it, and the inaccuracy of the timers and lag would probably cause big problems. Also, Xboard uses centiseconds for the "time" and "otim" commands, so when there's less than a centisecond left, the game is practically over. Even worse, the Xboard "level" command uses second-precision.

I'll run some tests to see what happens.
IanO
Posts: 496
Joined: Wed Mar 08, 2006 9:45 pm
Location: Portland, OR

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by IanO »

ilari wrote:
IanO wrote:"cmd=./homer" also didn't work. I only got it to work by running within the Homer directory (cmd=./homer dir=.)

If I use the absolute path, "cmd=/Applications/Games/Sigma Chess 6.1/Homer/homer", I get:

Warning: Cannot start engine "/Applications/Games/Sigma"

which points out problems in either your tool's or QProcess::start()'s command line parsing. (It is a common bug that tools don't handle spaces in file/directory names correctly.)
You're right, it's the space. But the problem is not in my program or QProcess, it's in the startup script cutechess-cli.sh which removes quotation marks from the command. You can fix the problem by replacing $* with "$@" on the last line. So it becomes:

Code: Select all

$dirname/$appname "$@"
The quotation marks are important there.

I'm going to release a new version of Cutechess-cli shortly with a couple of fixes and new features.
Thanks, that helped a little. Now the error message is:

Warning: Cannot start engine "/Applications/Games/Sigma Chess 6.1/Homer/homer"

Next problem: how do you pass command line arguments to the programs? I tried "cmd=crafty bookpath=/opt/local/share/crafty" but got back :

Warning: Two engines are needed
User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 7:45 pm
Location: Finland

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by ilari »

IanO wrote:Thanks, that helped a little. Now the error message is:

Warning: Cannot start engine "/Applications/Games/Sigma Chess 6.1/Homer/homer"
Looks like I need to add quotes around the command so that QProcess::start() will accept it. In the meantime, try cmd=./homer dir="/Applications/Games/Sigma Chess 6.1/Homer". That should work now that you've fixed the cutechess-cli.sh script.

Next problem: how do you pass command line arguments to the programs? I tried "cmd=crafty bookpath=/opt/local/share/crafty" but got back :

Warning: Two engines are needed
Hadn't thought of that. I'll add a new option for the engines' command line arguments. I'll probably release a new version during the weekend.
Edsel Apostol
Posts: 803
Joined: Mon Jul 17, 2006 5:53 am
Full name: Edsel Apostol

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by Edsel Apostol »

Hi Ilari,

How about running a match between two UCI engines using fixed nodes, is it possible?
User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 7:45 pm
Location: Finland

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by ilari »

Edsel Apostol wrote:Hi Ilari,

How about running a match between two UCI engines using fixed nodes, is it possible?
Not yet, but I'll add it on my todo list. I'll try to do it for Xboard engines as well, but since hgm's Xboard proposal only has the "nps" command, the solution is going to be a bit hackish.
IanO
Posts: 496
Joined: Wed Mar 08, 2006 9:45 pm
Location: Portland, OR

Re: Cutechess-cli: A command line tool for engine-engine mat

Post by IanO »

Another problem: in a match between Hamsters and Crafty 20.14, several of the games were lost by crafty due to an illegal first move, which turns out to be a legal move in the preceding drawn game (once by 50-move rule, twice by threefold repetition). Looks like a race condition between starting a new game and waiting for the previous program to quit the previous game.