Cutechess-cli 0.3.0 released

[ilari]

After half a year it's time to push out a new version of Cutechess-cli.

Changes include:
* The required Qt version is now 4.6 (was 4.5).
* Redesigned and rewritten concurrency code. The rare deadlocks and segfaults between games (especially with high "-concurrency" values) should be gone now.
* Fix the command line parser in the Windows version. Now command line arguments following the executable name are parsed properly.
* Handle invalid draw claims from Xboard engines properly. Now an invalid draw claim from an engine in "Thinking" mode is recognized properly and does not result in a "Connection stalls" error.
* Check the legality of opening book moves. It's no longer possible to break the game with a faulty book file.
* Option "-srand" for setting the random seed for the book move selector manually.
* The "invertscores" option was renamed to "whitepov".
* Evaluation perspective ("whitepov") and init strings can now be configured with the engines.json config file.
* Option "-recover" for automatically recovering crashed or stalled engines instead of terminating the whole match.
* Display the ELO difference after the match has ended.

We also have a mailing list now. I don't know if it works yet (Arto is administering it), but I hope it does. It should be a good place to announce new releases, report bugs and ask questions.


About the engines.json file:

Instead of using the "cmd", "name", "proto", "dir", "whitepov" and "initstr" arguments it's possible to keep engine configurations in a file and invoke an engine with just "conf=name". The file should be named "engines.json" and stored in the same folder with cutechess-cli. The chosen notation is JSON (JavaScript Object Notation), and the contents should look like this:

Code: Select all

[
   {
      "command":"crafty",
      "name":"Crafty-22.10",
      "protocol":"xboard",
      "whitepov":true
   },
   {
      "command":"wine \"Rybka v2.2n2.mp.w32.exe\"",
      "name":"Rybka-2.2n2",
      "protocol":"uci",
      "workingDirectory":"/home/ilari/chess/Rybka22n2"
   },
   {
      "command":"./sloppy",
      "initStrings" : [
         "init str 1",
         "init str 2"
      ],
      "name":"Sloppy-0.2.2",
      "protocol":"xboard",
      "workingDirectory":"/home/ilari/chess/sloppy",
   },
]

Remember to escape special characters with a backslash. So a double quote is \", a backslash is \\, etc.


Source code (cutechess-lib + cutechess-gui + cutechess-cli) here

Binaries:
* Linux 32-bit
* Linux 64-bit
* Mac OSX (Leopard or newer)
* Windows 32-bit

Have fun!

[Ferdy]

After half a year it's time to push out a new version of Cutechess-cli.

Changes include:
* The required Qt version is now 4.6 (was 4.5).
* Redesigned and rewritten concurrency code. The rare deadlocks and segfaults between games (especially with high "-concurrency" values) should be gone now.
* Fix the command line parser in the Windows version. Now command line arguments following the executable name are parsed properly.
* Handle invalid draw claims from Xboard engines properly. Now an invalid draw claim from an engine in "Thinking" mode is recognized properly and does not result in a "Connection stalls" error.
* Check the legality of opening book moves. It's no longer possible to break the game with a faulty book file.
* Option "-srand" for setting the random seed for the book move selector manually.
* The "invertscores" option was renamed to "whitepov".
* Evaluation perspective ("whitepov") and init strings can now be configured with the engines.json config file.
* Option "-recover" for automatically recovering crashed or stalled engines instead of terminating the whole match.
* Display the ELO difference after the match has ended.

We also have a mailing list now. I don't know if it works yet (Arto is administering it), but I hope it does. It should be a good place to announce new releases, report bugs and ask questions.


About the engines.json file:

Instead of using the "cmd", "name", "proto", "dir", "whitepov" and "initstr" arguments it's possible to keep engine configurations in a file and invoke an engine with just "conf=name". The file should be named "engines.json" and stored in the same folder with cutechess-cli. The chosen notation is JSON (JavaScript Object Notation), and the contents should look like this:

Code: Select all

[
   {
      "command":"crafty",
      "name":"Crafty-22.10",
      "protocol":"xboard",
      "whitepov":true
   },
   {
      "command":"wine "Rybka v2.2n2.mp.w32.exe"",
      "name":"Rybka-2.2n2",
      "protocol":"uci",
      "workingDirectory":"/home/ilari/chess/Rybka22n2"
   },
   {
      "command":"./sloppy",
      "initStrings" : [
         "init str 1",
         "init str 2"
      ],
      "name":"Sloppy-0.2.2",
      "protocol":"xboard",
      "workingDirectory":"/home/ilari/chess/sloppy",
   },
]

Remember to escape special characters with a backslash. So a double quote is ", a backslash is \\, etc.


Source code (cutechess-lib + cutechess-gui + cutechess-cli) here

Binaries:
* Linux 32-bit
* Linux 64-bit
* Mac OSX (Leopard or newer)
* Windows 32-bit

Have fun!

* Option "-recover" for automatically recovering crashed or stalled engines instead of terminating the whole match.

Why not make this as default?
Would this mean that this will be applied only for suspecting engines?

Thank you again for this great tool. My development is very productive because of this.

[Ferdy]

After half a year it's time to push out a new version of Cutechess-cli.

Changes include:
* The required Qt version is now 4.6 (was 4.5).
* Redesigned and rewritten concurrency code. The rare deadlocks and segfaults between games (especially with high "-concurrency" values) should be gone now.
* Fix the command line parser in the Windows version. Now command line arguments following the executable name are parsed properly.
* Handle invalid draw claims from Xboard engines properly. Now an invalid draw claim from an engine in "Thinking" mode is recognized properly and does not result in a "Connection stalls" error.
* Check the legality of opening book moves. It's no longer possible to break the game with a faulty book file.
* Option "-srand" for setting the random seed for the book move selector manually.
* The "invertscores" option was renamed to "whitepov".
* Evaluation perspective ("whitepov") and init strings can now be configured with the engines.json config file.
* Option "-recover" for automatically recovering crashed or stalled engines instead of terminating the whole match.
* Display the ELO difference after the match has ended.

We also have a mailing list now. I don't know if it works yet (Arto is administering it), but I hope it does. It should be a good place to announce new releases, report bugs and ask questions.


About the engines.json file:

Instead of using the "cmd", "name", "proto", "dir", "whitepov" and "initstr" arguments it's possible to keep engine configurations in a file and invoke an engine with just "conf=name". The file should be named "engines.json" and stored in the same folder with cutechess-cli. The chosen notation is JSON (JavaScript Object Notation), and the contents should look like this:

Code: Select all

[
   {
      "command":"crafty",
      "name":"Crafty-22.10",
      "protocol":"xboard",
      "whitepov":true
   },
   {
      "command":"wine "Rybka v2.2n2.mp.w32.exe"",
      "name":"Rybka-2.2n2",
      "protocol":"uci",
      "workingDirectory":"/home/ilari/chess/Rybka22n2"
   },
   {
      "command":"./sloppy",
      "initStrings" : [
         "init str 1",
         "init str 2"
      ],
      "name":"Sloppy-0.2.2",
      "protocol":"xboard",
      "workingDirectory":"/home/ilari/chess/sloppy",
   },
]

Remember to escape special characters with a backslash. So a double quote is ", a backslash is \\, etc.


Source code (cutechess-lib + cutechess-gui + cutechess-cli) here

Binaries:
* Linux 32-bit
* Linux 64-bit
* Mac OSX (Leopard or newer)
* Windows 32-bit

Have fun!

problem with the first sample run with this version.

Started game 1 of 100
Game 1 ended: 0-1 {Black wins by adjudication}
Hamsters 0.7.1 wins the game as black
Score of XboardEngine vs Hamsters 0.7.1: 0 - 1 - 0

First engine was not identified in the first game, other games are ok.
I did not use the option name, concurrency is set to 1.

This probem did not happen on previous verions 0.2.2.

[ilari]

* Option "-recover" for automatically recovering crashed or stalled engines instead of terminating the whole match.

Why not make this as default?
Would this mean that this will be applied only for suspecting engines?

Thank you again for this great tool. My development is very productive because of this.

The -recover option applies to any engine, even remote engines (eg. via SSH). It means that if an engine crashes, it loses the game and is restarted for the next game.

It's not a default because most people using cutechess-cli are testing unstable development versions of their engines, and they often want to halt the test run immediately when a crash happens. But if one just wants to find out which engine is stronger, A or B, then it's recommended to use the -recover option to make sure all games are played.

[ilari]

problem with the first sample run with this version.

Started game 1 of 100
Game 1 ended: 0-1 {Black wins by adjudication}
Hamsters 0.7.1 wins the game as black
Score of XboardEngine vs Hamsters 0.7.1: 0 - 1 - 0

First engine was not identified in the first game, other games are ok.
I did not use the option name, concurrency is set to 1.

This probem did not happen on previous verions 0.2.2.

Thanks for the report. I think I know why this happens now and didn't happen with v0.2.2. Is it possible that your engine sends feature done=1 before myname=engine_name?

[Ferdy]

problem with the first sample run with this version.

Started game 1 of 100
Game 1 ended: 0-1 {Black wins by adjudication}
Hamsters 0.7.1 wins the game as black
Score of XboardEngine vs Hamsters 0.7.1: 0 - 1 - 0

First engine was not identified in the first game, other games are ok.
I did not use the option name, concurrency is set to 1.

This probem did not happen on previous verions 0.2.2.

Thanks for the report. I think I know why this happens now and didn't happen with v0.2.2. Is it possible that your engine sends feature done=1 before myname=engine_name?

No. It is myname first before feature done=1.

But forget about this, sorry only happened if i enable egbb use, both versions actually failed. I initialized egbb before entering xboard mode.
I will check further on this.

[micron]

I tried the new option

Code: Select all

-srand <n>&#58;&#58;
	Set the random seed for the book move selector to <n>.

expecting that it would change the starting positions in a match.
But it has no discernible effect. With -debug I can see that the positions selected from MyGameStarts.pgn are unchanged.

What is -srand meant to do?

Robert P.

[ilari]

I tried the new option

Code: Select all

-srand <n>&#58;&#58;
	Set the random seed for the book move selector to <n>.

expecting that it would change the starting positions in a match.
But it has no discernible effect. With -debug I can see that the positions selected from MyGameStarts.pgn are unchanged.

What is -srand meant to do?

Robert P.

It sets the seed for the random number generator that is used for selecting moves from a Polyglot book file. It has no effect if you use a PGN file as an opening book.

[WinPooh]

When I try to run Aristarch 4.50 under cutechess-cli, I get the message:

Aristarch" doesn't support variant "standard"

This is Aristarch's entry in my engines.json:

Code: Select all

   {
      "command":"Aristarch.exe",
      "name":"Aristarch",
      "protocol":"xboard",
      "workingDirectory":"engines\\Aristarch"
   }

And this is bat-file I use to run the match:

Code: Select all

set FCP="Fruit"
set SCP="Aristarch"
set TC=5+0.1
set GAMES=60

cutechess-cli.exe -games %GAMES% -repeat -fcp conf=%FCP% -scp conf=%SCP% -both tc=%TC% book=varied.bin bookdepth=7 -pgnout test.pgn min

I am pretty sure Aristarch is capable to play standard chess
What do I do wrong?

[WinPooh]

However, when I changed protocol to UCI, it looks ok.