Page 15 of 18

Re: Cutechess-cli: A command line tool for engine-engine mat

Posted: Tue Sep 29, 2009 10:34 am
by ilari
Dann Corbit wrote:It would be very nice if it could support USI also (there is no tournament GUI for USI at all that I know of, though you can play a single engine against a single engine for N games with Shogidokoro):
http://www.glaurungchess.com/shogi/usi.html
Implementing a new chess protocol wouldn't be that hard, but implementing a new chess variant like Shogi certainly would. We're going to redesign the chess protocol classes a bit so that 3rd party developers can easily add new protocols just by subclassing the ChessEngine class, without modifying the chesslib source code. I'm thinking of doing the same thing for the chessboard and chess variant code. Then it would be possible to add new variants without too much pain.

I'll think about this. The feature set of Cute Chess 1.0 is pretty much frozen, but who knows what happens after that.

Re: Cutechess-cli: A command line tool for engine-engine mat

Posted: Tue Sep 29, 2009 12:25 pm
by hgm
Dann Corbit wrote:It would be very nice if it could support USI also (there is no tournament GUI for USI at all that I know of, though you can play a single engine against a single engine for N games with Shogidokoro):
http://www.glaurungchess.com/shogi/usi.html
Perhaps it would be easier to adapt Polyglot to USI. Then all USI engines could run under WinBoard+PSWBTM, just like all UCCI engines already run under WinBoard.

Re: Cutechess-cli 0.1.9 released

Posted: Sun Oct 04, 2009 7:29 pm
by ilari
When there are new features there are new bugs. Version 0.1.9 fixes some bugs that were introduced in 0.1.8:

- Crash due to a player resigning on the opponent's turn
- Crash due to a player disconnecting or being terminated
- Crash/freeze due to failure to respond to ping

Thanks for everyone who submitted bug reports. I've updated the binaries.

Re: Cutechess-cli 0.1.9 released

Posted: Tue Oct 20, 2009 8:41 am
by Gian-Carlo Pascutto
Today I got this, first time I ever saw it:

Score of Deep Sjeng 3.2 LE2 vs Deep Sjeng 3.2 BL14: 786 - 885 - 1292
Started game 2967 of 5000
pure virtual method called
terminate called without an active exception
Aborted

Re: Cutechess-cli 0.1.9 released

Posted: Tue Oct 20, 2009 5:11 pm
by ilari
Gian-Carlo Pascutto wrote:Today I got this, first time I ever saw it:

Score of Deep Sjeng 3.2 LE2 vs Deep Sjeng 3.2 BL14: 786 - 885 - 1292
Started game 2967 of 5000
pure virtual method called
terminate called without an active exception
Aborted
Whoa, that's interesting. I hate it when that happens at game 3000 or so, makes it pretty annoying to debug. A few questions:
- What cutechess-cli settings/command did you use? I'm most interested in the time control and concurrency settings.
- Which platform (Windows/Mac/Linux, 32-bit or 64-bit)?
- If you're running Linux, which version of Qt do you have?
- Is it one of my builds or did you compile cutechess-cli yourself?

Re: Cutechess-cli 0.1.9 released

Posted: Tue Oct 20, 2009 5:26 pm
by Gian-Carlo Pascutto
ilari wrote: Whoa, that's interesting. I hate it when that happens at game 3000 or so, makes it pretty annoying to debug. A few questions:
- What cutechess-cli settings/command did you use? I'm most interested in the time control and concurrency settings.
- Which platform (Windows/Mac/Linux, 32-bit or 64-bit)?
- If you're running Linux, which version of Qt do you have?
- Is it one of my builds or did you compile cutechess-cli yourself?
Own compile from your:
commit 178918d6df59ac0c7e7cbae177907187de46e3ae

64 bit AMD64 Linux
gcc (Debian 4.3.4-5) 4.3.4

Package: libqt4-core
Version: 4:4.5.3-2

Command line something like this:
./cutechess-cli -fcp cmd=./sjengns4 dir=~/chess/sjeng/ proto=uci -scp cmd=./sjengbl14 dir=~/chess/sjeng proto=uci -both tc=5+0.1 -book book.bin -draw 80 1 -resign 3 500 -games 5000 -repeat -pgnout ns4bl14.pgn -concurrency 4

Whatever the bug was, it must be very seldom. I've played a few hundred thousand games with this tool already and never hit this before. Some obscure threading bug?

Re: Cutechess-cli 0.1.9 released

Posted: Tue Oct 20, 2009 7:30 pm
by ilari
Gian-Carlo Pascutto wrote: Own compile from your:
commit 178918d6df59ac0c7e7cbae177907187de46e3ae

64 bit AMD64 Linux
gcc (Debian 4.3.4-5) 4.3.4

Package: libqt4-core
Version: 4:4.5.3-2

Command line something like this:
./cutechess-cli -fcp cmd=./sjengns4 dir=~/chess/sjeng/ proto=uci -scp cmd=./sjengbl14 dir=~/chess/sjeng proto=uci -both tc=5+0.1 -book book.bin -draw 80 1 -resign 3 500 -games 5000 -repeat -pgnout ns4bl14.pgn -concurrency 4
Looks fine. The Qt version was the biggest thing I was worried about. We only test Cute Chess with Qt 4.5., so previous versions may not work so well.
Whatever the bug was, it must be very seldom. I've played a few hundred thousand games with this tool already and never hit this before. Some obscure threading bug?
Hard to say. I'm pretty sure we're not calling virtual function from a constructor so this could be a threading + dangling pointer problem. GDB would probably show a nice backtrace if we encountered this error during a debugging run.

Re: Cutechess-cli 0.1.9 released

Posted: Tue Oct 20, 2009 8:48 pm
by Gian-Carlo Pascutto
Maybe it can be reproduced by running with very high concurrencies.

Re: Cutechess-cli 0.1.9 released

Posted: Tue Oct 20, 2009 10:12 pm
by ilari
Gian-Carlo Pascutto wrote:Maybe it can be reproduced by running with very high concurrencies.
Maybe. One bug was discovered in the Windows version when running with "-concurrency 4" while "-concurrency 2" worked fine. What may help even more is running the games really quickly, like 1 sec/game + agressive draw and resign adjudications. This error looks like something that would happen between games rather than during them.

Re: Millispeconds and a new EPD spec

Posted: Wed Oct 21, 2009 12:29 am
by Dann Corbit
sje wrote:
ilari wrote:I've uploaded a new version (0.1.3) with a few changes:

- Support millisecond-precision in time controls
I agree that millisecond resolution is the way to go. Piloted spacecraft use a millisecond period update for critical telemetry; if it's good enough for a spaceship then I suppose it's good enough for a chess clock.

A new EPD specification is coming, and it uses a tighter formalism for a number of things including time values. Example:

Code: Select all

r1bqkbnr/pppp1ppp/2n5/8/3Q4/2N5/PPP1PPPP/R1B1KBNR w KQkq - 1 4 acd 6; acn 1436575; cc 4:55.265 4:54.606; pes +0.020; pv Qd3 h6 Nf3 Nf6 e4 Bd6; te 1.445; tu 1.333;
Note:

1) The half move counter and the full move number now appear explicitly as the fifth and sixth fields of a standard FEN position specification. The earlier EPD version had these omitted or appearing as optional operations (hmvc and fmvn).

2) Most things don't change: acd, acn, am, bm, pv, sv, etc.

3) Some things are deprecated, such as pm (use the first parameter value of pv instead).
I find pm useful (predicted move may not be the same thing as best move for instance). I have code that uses the pm field.
4) All time values share the same the same format of DDD:HH:MM:SS.mmm with optional leading zero suppression. Example: cc (chess clock) has a White and a Black countdown time operand.

5) Instead of acs, te records analysis elapsed (wall) time, and tu represents usage time. Usage may be greater than elapsed time for multicore analysis.

6) The ce operator is deprecated; the replacement pes (pawn equivalent score) operator gives an evaluation in signed floating point pawns with millipawn resolution. There are a few non-decimal values: MateInN (e.g., MateIn3), LoseInN, Even, PosInf, NegInf, and Broken.
Perhaps inconvenient for some, since almost nobody uses floating point for evaluation. But if you are going to make it floating point, and 1.0 is supposed to represent a pawn, then why use milliseconds? Just let it be a double and then they can have an internal precision that they like. In any case, we're going to use sscanf("%lf"... to collect it.
7) A few operations are undecided at the moment, like dm and solve.
I find dm very useful. I do not even know what solve is for because I have never run across that field. People are going to use different scales for checkmate (even though you have 32767 listed as written in stone, lots of people use 9999, 99999 or some other value instead, so dm has a clear purpose here).

8) Symbol operands are never quoted and string operands are always quoted. Examples:

id WAC.241;
I have literally hundreds of thousands of id fields with spaces in them. There was nothing in the original spec to prevent that and I will find it most annoying if I am not allowed to quote id fields. In fact, I can tell you now that I will ignore that new part of the spec because I have not enough time nor the compulsion to correct them.

name "Win at Chess problem 241";