Cutechess-cli: A command line tool for engine-engine matches

[Tord Romstad]

I want to add that Glaurung 2.2 sometimes makes an illegal move around 3 to 5 times in 60 games.

This has nothing to do with cutechess-cli: Glaurung 2.2 can't play fast games.

Tord

[Edsel Apostol]

I want to add that Glaurung 2.2 sometimes makes an illegal move around 3 to 5 times in 60 games.

This has nothing to do with cutechess-cli: Glaurung 2.2 can't play fast games.

Tord

This means less sparring partner for TL in fixed nodes.

Only Spike is perfect for my tests, the latest Bright doesn't do it perfectly as it only do some estimate and is sometimes off by a few kilo nodes and might not be consistent when the same match is being run again.

[Edsel Apostol]

One more request. Is it possible to support the "name" command even if it is a UCI engine? I'm used to testing different settings of my engine and I couldn't differentiate it from the default as the tool saves only the UCI name of the engine in the PGN file.

[ilari]

I've tried the fixed node match and it worked well. Great tool!

By the way, it would be good if you could support enabling/disabling the depth, time and score in the PGN in order to save storage space.

I actually have a "bool minimal" option in PgnGame::write() already, there's just no option for using it in cutechess-cli. Stay tuned...

[Tord Romstad]

This has nothing to do with cutechess-cli: Glaurung 2.2 can't play fast games.

This means less sparring partner for TL in fixed nodes.

Only for very small numbers: If you try something like 50k or 100k nodes, I think it will work fine.

If there will ever be another version, it is likely that it will work at much lower node counts, as a consequence of the work to adapt the program to very slow hardware.

Tord

[ilari]

Hello Ilari,

Another little bug report. I actually noticed this because I was setting the time control to 1+0, which turns out to be 1 second/game. I think that's rather confusing, as it breaks with the xboard protocol (which would specify one second as 0:01).

I didn't even try to do it as it's done in Xboard. If you want 1 minute per game you can use "1:0" or "60".

But anyways, it sends the level command to the engine without 0-padding the time, so the engine receives "level 0 0:1 0". I traced the cause to this function, hoping I could fix it, but I know nothing about Qt.

Code: Select all

static QString msToXboardTime(int ms)
{
    int sec = ms / 1000;

    QString number = QString::number(sec / 60);
    if (sec % 60 != 0)
        number += QString(":%1").arg(sec % 60);

    return number;
}

It didn't occur to me that Xboard would need the 0-padding. Fortunately this is another easy fix.

One other request: could you put an interrupt handler in the program? This is a pretty minor issue, but sometimes I need to kill the program, and the logfile just abruptly ends now. It would be nice to have a clean break there, and maybe save the first part of the game in the pgn file.

I'll see if there's something in the Qt libraries for handling interruptions. I hope I won't have to write platform-specific code.

[ilari]

One more request. Is it possible to support the "name" command even if it is a UCI engine? I'm used to testing different settings of my engine and I couldn't differentiate it from the default as the tool saves only the UCI name of the engine in the PGN file.

The "name" option isn't specific to Xboard. It also works with UCI engines if they don't send the "id name" command. But I can change it so that the "name" option overrides the engine's "id name" option.

[ilari]

I've uploaded Cutechess-cli 0.1.2 with the changes that were suggested to me:

- Fix Xboard's "level" command when time per tc is less than 10 seconds.

- Use infinite maximum book depth by default. The depth is still limited by the opening book or the PGN game's move count.

- Support minimal PGN output where only data that is required by the PGN standard is saved. To use the minimal mode, add "min" as the second parameter to "-pgnout".

- Catch SIGINT and SIGTERM, save the game in progress (if any) and exit.

- Override the name given by the engine with the "name" option.

[ilari]

I suppose I should release my code at some point, but it is really tied to unix. It is what I use on our cluster and will handle using multiple starting positions, play as many games per position as you want, alternating colors (I use 2 here), and creates a PGN file containing all the games with results. I have another program that will take a bunch of scripts that use the above program, and execute as many at once as you want, so that you can use all your CPUs or just 1/2 of them or whatever. But again, all is pure Unix-based, no windows at all, and documentation is <nil>

It would be interesting to see that code, even without documentation. Did you write it all in C?


Btw, Cutechess-cli doesn't let your program resign because Crafty outputs an unprintable control character at the beginning of the command. Like this: "1-0 {Black resigns}"
In Crafty's logs it looks fine and it's also not visible in the console, but in Cutechess-cli's and also Xboard's logfile it shows.

I'm using a 64-bit Linux, and I tested versions 20.14 and 22.9 of Crafty.

[Edsel Apostol]

Is it possible to support milliseconds or centiseconds for the time control? It would be great if we can set the time increment to a fraction of a second for very fast games.