cutechess-cli 0.8.0 beta (with pondering)

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 5:45 pm
Location: Finland
Contact:

cutechess-cli 0.8.0 beta (with pondering)

Post by ilari » Tue Oct 27, 2015 7:04 pm

Hi,

These last couple of months have been good for cutechess-cli development and now things got even better - pondering is finally supported after a long wait! I think the feature is big enough to warrant a beta release for some brave souls to test. I've provided binaries for the two biggest platforms:

* Linux 64-bit: http://koti.mbnet.fi/~ilaripih/bin/cute ... x64.tar.gz
* Windows 32-bit: http://koti.mbnet.fi/~ilaripih/bin/cute ... -win32.zip

I'm especially interested in hearing how pondering works with UCI engines. On the Xboard side the implementation was so trivial that there's not much risk of a regression. Pondering can be enabled with the new "ponder" engine option which is also documented in the manuals. If you find a bug or anything suspicious, please report your findings here, preferably with cutechess-cli's debug output. You'll probably want to run some games with the -debug option anyway, just to be sure that the engines are actually pondering.

Thanks!

Robert Pope
Posts: 510
Joined: Sat Mar 25, 2006 7:27 pm

Re: cutechess-cli 0.8.0 beta (with pondering)

Post by Robert Pope » Tue Oct 27, 2015 8:37 pm

Since you seem to be so active on cutechess-cli recently, I was wondering if you might add a "pause" feature. I'd love to set up a 20,000 game tournament, but I don't use my computer for chess exclusively, so it would be awesome if I could ^C to stop a tournament, do something else, and then just type "cutechess -resume" or something to pick up where it left off. :)

royb
Posts: 518
Joined: Wed Mar 08, 2006 11:53 pm

Re: cutechess-cli 0.8.0 beta (with pondering)

Post by royb » Tue Oct 27, 2015 8:40 pm

Robert Pope wrote:Since you seem to be so active on cutechess-cli recently, I was wondering if you might add a "pause" feature. I'd love to set up a 20,000 game tournament, but I don't use my computer for chess exclusively, so it would be awesome if I could ^C to stop a tournament, do something else, and then just type "cutechess -resume" or something to pick up where it left off. :)
Absolutely an excellent idea!! I hope that can be added!

User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 5:45 pm
Location: Finland
Contact:

Re: cutechess-cli 0.8.0 beta (with pondering)

Post by ilari » Tue Oct 27, 2015 9:03 pm

Robert Pope wrote:Since you seem to be so active on cutechess-cli recently, I was wondering if you might add a "pause" feature. I'd love to set up a 20,000 game tournament, but I don't use my computer for chess exclusively, so it would be awesome if I could ^C to stop a tournament, do something else, and then just type "cutechess -resume" or something to pick up where it left off. :)
It's on the roadmap as a part of a bigger feature. You should see it in version 1.0.0 some day.

User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 5:45 pm
Location: Finland
Contact:

Re: cutechess-cli 0.8.0 beta (with pondering)

Post by ilari » Tue Oct 27, 2015 11:48 pm

Well, didn't take long for the first bug to surface - when an engine responds to a "ponderhit" with "bestmove" quicker than the opponent responds to "isready", things go out of sync and the game ends with a "X's connection stalls" error. Stay tuned for a fix. Might take a couple of days...

Ferdy
Posts: 4080
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: cutechess-cli 0.8.0 beta (with pondering)

Post by Ferdy » Wed Oct 28, 2015 8:31 am

ilari wrote:Well, didn't take long for the first bug to surface - when an engine responds to a "ponderhit" with "bestmove" quicker than the opponent responds to "isready", things go out of sync and the game ends with a "X's connection stalls" error. Stay tuned for a fix. Might take a couple of days...
I just tried the beta and encountered something similar to this problem in Deuterium. What happened was while pondering, the maximum iteration depth was reached early and Deuterium sent the bestmove without waiting for ponderhit or stop commands. When later it received the ponderhit from cutechess, Deuterium sent bestmove 0000, this indicates that Deuterium is no longer searching since it already sent its bestmove before. Cutechess then flagged Deuterium of illegal move.
This is now fixed in Deuterium development version. I think cutechess-cli is doing fine in this case, the engine should not stop searching or stop searching and send bestmove, in ponder and/or infinite mode until told to do so by ponderhit or stop commands, according to uci.

User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 5:45 pm
Location: Finland
Contact:

Re: cutechess-cli 0.8.0 beta (with pondering)

Post by ilari » Thu Oct 29, 2015 5:58 pm

ilari wrote:Well, didn't take long for the first bug to surface - when an engine responds to a "ponderhit" with "bestmove" quicker than the opponent responds to "isready", things go out of sync and the game ends with a "X's connection stalls" error. Stay tuned for a fix. Might take a couple of days...
The fix is in! Here's the new beta version:

* Linux 64-bit: http://koti.mbnet.fi/~ilaripih/bin/cute ... x64.tar.gz
* Windows 32-bit: http://koti.mbnet.fi/~ilaripih/bin/cute ... -win32.zip

I tested it on Windows in some really punishing conditions and seems to work now. Just keep in mind that with pondering enabled you're using twice the CPU time so it's not a great idea to use "-concurrency 4" and ponder on a 4-core machine. Cutechess-cli should handle it just fine but the engines probably won't.

petero2
Posts: 581
Joined: Mon Apr 19, 2010 5:07 pm
Location: Sweden
Contact:

Re: cutechess-cli 0.8.0 beta (with pondering)

Post by petero2 » Sun Nov 01, 2015 8:40 pm

Hi and thanks for the new pondering feature.

I have encountered a small problem related to how games are saved. The PGN output file is now kept open, and it uses internal buffering. This means that if you try to read the file during a match, the most recently finished game is typically truncated. To fix this I suggest the following patch:

Code: Select all

diff --git a/projects/lib/src/pgngame.cpp b/projects/lib/src/pgngame.cpp
index 128e09a..ebd7c9f 100644
--- a/projects/lib/src/pgngame.cpp
+++ b/projects/lib/src/pgngame.cpp
@@ -308,6 +308,8 @@ bool PgnGame::write(QTextStream& out, PgnMode mode) const
        else
                out << " " << str << "\n\n";
 
+       out.flush&#40;);
+
        return true;
 &#125;
 

User avatar
ilari
Posts: 750
Joined: Mon Mar 27, 2006 5:45 pm
Location: Finland
Contact:

Re: cutechess-cli 0.8.0 beta (with pondering)

Post by ilari » Mon Nov 02, 2015 2:32 pm

petero2 wrote:Hi and thanks for the new pondering feature.

I have encountered a small problem related to how games are saved. The PGN output file is now kept open, and it uses internal buffering. This means that if you try to read the file during a match, the most recently finished game is typically truncated. To fix this I suggest the following patch:

Code: Select all

diff --git a/projects/lib/src/pgngame.cpp b/projects/lib/src/pgngame.cpp
index 128e09a..ebd7c9f 100644
--- a/projects/lib/src/pgngame.cpp
+++ b/projects/lib/src/pgngame.cpp
@@ -308,6 +308,8 @@ bool PgnGame&#58;&#58;write&#40;QTextStream& out, PgnMode mode&#41; const
        else
                out << " " << str << "\n\n";
 
+       out.flush&#40;);
+
        return true;
 &#125;
 
Thanks! That sounds very sensible. Did you test this change and verify that it solves the problem?

petero2
Posts: 581
Joined: Mon Apr 19, 2010 5:07 pm
Location: Sweden
Contact:

Re: cutechess-cli 0.8.0 beta (with pondering)

Post by petero2 » Mon Nov 02, 2015 6:29 pm

ilari wrote:
petero2 wrote:Hi and thanks for the new pondering feature.

I have encountered a small problem related to how games are saved. The PGN output file is now kept open, and it uses internal buffering. This means that if you try to read the file during a match, the most recently finished game is typically truncated. To fix this I suggest the following patch:

Code: Select all

diff --git a/projects/lib/src/pgngame.cpp b/projects/lib/src/pgngame.cpp
index 128e09a..ebd7c9f 100644
--- a/projects/lib/src/pgngame.cpp
+++ b/projects/lib/src/pgngame.cpp
@@ -308,6 +308,8 @@ bool PgnGame&#58;&#58;write&#40;QTextStream& out, PgnMode mode&#41; const
        else
                out << " " << str << "\n\n";
 
+       out.flush&#40;);
+
        return true;
 &#125;
 
Thanks! That sounds very sensible. Did you test this change and verify that it solves the problem?
Yes it does solve my problem. I have only tested in linux because that is what I use to run all my engine tests.

Post Reply