cutechess-cli 0.8.0 beta (with pondering)

[ilari]

Hi,

These last couple of months have been good for cutechess-cli development and now things got even better - pondering is finally supported after a long wait! I think the feature is big enough to warrant a beta release for some brave souls to test. I've provided binaries for the two biggest platforms:

* Linux 64-bit: http://koti.mbnet.fi/~ilaripih/bin/cute ... x64.tar.gz
* Windows 32-bit: http://koti.mbnet.fi/~ilaripih/bin/cute ... -win32.zip

I'm especially interested in hearing how pondering works with UCI engines. On the Xboard side the implementation was so trivial that there's not much risk of a regression. Pondering can be enabled with the new "ponder" engine option which is also documented in the manuals. If you find a bug or anything suspicious, please report your findings here, preferably with cutechess-cli's debug output. You'll probably want to run some games with the -debug option anyway, just to be sure that the engines are actually pondering.

Thanks!

[Robert Pope]

Since you seem to be so active on cutechess-cli recently, I was wondering if you might add a "pause" feature. I'd love to set up a 20,000 game tournament, but I don't use my computer for chess exclusively, so it would be awesome if I could ^C to stop a tournament, do something else, and then just type "cutechess -resume" or something to pick up where it left off.

[royb]

Since you seem to be so active on cutechess-cli recently, I was wondering if you might add a "pause" feature. I'd love to set up a 20,000 game tournament, but I don't use my computer for chess exclusively, so it would be awesome if I could ^C to stop a tournament, do something else, and then just type "cutechess -resume" or something to pick up where it left off.

Absolutely an excellent idea!! I hope that can be added!

[ilari]

Since you seem to be so active on cutechess-cli recently, I was wondering if you might add a "pause" feature. I'd love to set up a 20,000 game tournament, but I don't use my computer for chess exclusively, so it would be awesome if I could ^C to stop a tournament, do something else, and then just type "cutechess -resume" or something to pick up where it left off.

It's on the roadmap as a part of a bigger feature. You should see it in version 1.0.0 some day.

[ilari]

Well, didn't take long for the first bug to surface - when an engine responds to a "ponderhit" with "bestmove" quicker than the opponent responds to "isready", things go out of sync and the game ends with a "X's connection stalls" error. Stay tuned for a fix. Might take a couple of days...

[Ferdy]

Well, didn't take long for the first bug to surface - when an engine responds to a "ponderhit" with "bestmove" quicker than the opponent responds to "isready", things go out of sync and the game ends with a "X's connection stalls" error. Stay tuned for a fix. Might take a couple of days...

I just tried the beta and encountered something similar to this problem in Deuterium. What happened was while pondering, the maximum iteration depth was reached early and Deuterium sent the bestmove without waiting for ponderhit or stop commands. When later it received the ponderhit from cutechess, Deuterium sent bestmove 0000, this indicates that Deuterium is no longer searching since it already sent its bestmove before. Cutechess then flagged Deuterium of illegal move.
This is now fixed in Deuterium development version. I think cutechess-cli is doing fine in this case, the engine should not stop searching or stop searching and send bestmove, in ponder and/or infinite mode until told to do so by ponderhit or stop commands, according to uci.

[ilari]

Well, didn't take long for the first bug to surface - when an engine responds to a "ponderhit" with "bestmove" quicker than the opponent responds to "isready", things go out of sync and the game ends with a "X's connection stalls" error. Stay tuned for a fix. Might take a couple of days...

The fix is in! Here's the new beta version:

* Linux 64-bit: http://koti.mbnet.fi/~ilaripih/bin/cute ... x64.tar.gz
* Windows 32-bit: http://koti.mbnet.fi/~ilaripih/bin/cute ... -win32.zip

I tested it on Windows in some really punishing conditions and seems to work now. Just keep in mind that with pondering enabled you're using twice the CPU time so it's not a great idea to use "-concurrency 4" and ponder on a 4-core machine. Cutechess-cli should handle it just fine but the engines probably won't.

[petero2]

Hi and thanks for the new pondering feature.

I have encountered a small problem related to how games are saved. The PGN output file is now kept open, and it uses internal buffering. This means that if you try to read the file during a match, the most recently finished game is typically truncated. To fix this I suggest the following patch:

Code: Select all

diff --git a/projects/lib/src/pgngame.cpp b/projects/lib/src/pgngame.cpp
index 128e09a..ebd7c9f 100644
--- a/projects/lib/src/pgngame.cpp
+++ b/projects/lib/src/pgngame.cpp
@@ -308,6 +308,8 @@ bool PgnGame::write(QTextStream& out, PgnMode mode) const
        else
                out << " " << str << "\n\n";
 
+       out.flush&#40;);
+
        return true;
 &#125;
 

[ilari]

Hi and thanks for the new pondering feature.

I have encountered a small problem related to how games are saved. The PGN output file is now kept open, and it uses internal buffering. This means that if you try to read the file during a match, the most recently finished game is typically truncated. To fix this I suggest the following patch:

Code: Select all

diff --git a/projects/lib/src/pgngame.cpp b/projects/lib/src/pgngame.cpp
index 128e09a..ebd7c9f 100644
--- a/projects/lib/src/pgngame.cpp
+++ b/projects/lib/src/pgngame.cpp
@@ -308,6 +308,8 @@ bool PgnGame::write(QTextStream& out, PgnMode mode) const
        else
                out << " " << str << "\n\n";
 
+       out.flush&#40;);
+
        return true;
 &#125;
 

Thanks! That sounds very sensible. Did you test this change and verify that it solves the problem?

[petero2]

Hi and thanks for the new pondering feature.

I have encountered a small problem related to how games are saved. The PGN output file is now kept open, and it uses internal buffering. This means that if you try to read the file during a match, the most recently finished game is typically truncated. To fix this I suggest the following patch:

Code: Select all

diff --git a/projects/lib/src/pgngame.cpp b/projects/lib/src/pgngame.cpp
index 128e09a..ebd7c9f 100644
--- a/projects/lib/src/pgngame.cpp
+++ b/projects/lib/src/pgngame.cpp
@@ -308,6 +308,8 @@ bool PgnGame::write(QTextStream& out, PgnMode mode) const
        else
                out << " " << str << "\n\n";
 
+       out.flush&#40;);
+
        return true;
 &#125;
 

Thanks! That sounds very sensible. Did you test this change and verify that it solves the problem?

Yes it does solve my problem. I have only tested in linux because that is what I use to run all my engine tests.