Cutechess-cli: A command line tool for engine-engine matches

[Dann Corbit]

I've uploaded a new version (0.1.3) with a few changes:

- Support millisecond-precision in time controls

I agree that millisecond resolution is the way to go. Piloted spacecraft use a millisecond period update for critical telemetry; if it's good enough for a spaceship then I suppose it's good enough for a chess clock.

A new EPD specification is coming, and it uses a tighter formalism for a number of things including time values. Example:

Code: Select all

r1bqkbnr/pppp1ppp/2n5/8/3Q4/2N5/PPP1PPPP/R1B1KBNR w KQkq - 1 4 acd 6; acn 1436575; cc 4:55.265 4:54.606; pes +0.020; pv Qd3 h6 Nf3 Nf6 e4 Bd6; te 1.445; tu 1.333;

Note:

1) The half move counter and the full move number now appear explicitly as the fifth and sixth fields of a standard FEN position specification. The earlier EPD version had these omitted or appearing as optional operations (hmvc and fmvn).

2) Most things don't change: acd, acn, am, bm, pv, sv, etc.

3) Some things are deprecated, such as pm (use the first parameter value of pv instead).

I find pm useful (predicted move may not be the same thing as best move for instance). I have code that uses the pm field.

4) All time values share the same the same format of DDD:HH:MM:SS.mmm with optional leading zero suppression. Example: cc (chess clock) has a White and a Black countdown time operand.

5) Instead of acs, te records analysis elapsed (wall) time, and tu represents usage time. Usage may be greater than elapsed time for multicore analysis.

6) The ce operator is deprecated; the replacement pes (pawn equivalent score) operator gives an evaluation in signed floating point pawns with millipawn resolution. There are a few non-decimal values: MateInN (e.g., MateIn3), LoseInN, Even, PosInf, NegInf, and Broken.

Perhaps inconvenient for some, since almost nobody uses floating point for evaluation. But if you are going to make it floating point, and 1.0 is supposed to represent a pawn, then why use milliseconds? Just let it be a double and then they can have an internal precision that they like. In any case, we're going to use sscanf("%lf"... to collect it.

7) A few operations are undecided at the moment, like dm and solve.

I find dm very useful. I do not even know what solve is for because I have never run across that field. People are going to use different scales for checkmate (even though you have 32767 listed as written in stone, lots of people use 9999, 99999 or some other value instead, so dm has a clear purpose here).

8) Symbol operands are never quoted and string operands are always quoted. Examples:

id WAC.241;

I have literally hundreds of thousands of id fields with spaces in them. There was nothing in the original spec to prevent that and I will find it most annoying if I am not allowed to quote id fields. In fact, I can tell you now that I will ignore that new part of the spec because I have not enough time nor the compulsion to correct them.

name "Win at Chess problem 241";

One more thing while I am whinging on...

I would like an 'er' field for 'expected response'. This field would be very useful for engine pondering.

[Dann Corbit]

I've uploaded a new version (0.1.3) with a few changes:

- Support millisecond-precision in time controls

I agree that millisecond resolution is the way to go. Piloted spacecraft use a millisecond period update for critical telemetry; if it's good enough for a spaceship then I suppose it's good enough for a chess clock.

A new EPD specification is coming, and it uses a tighter formalism for a number of things including time values. Example:

Code: Select all

r1bqkbnr/pppp1ppp/2n5/8/3Q4/2N5/PPP1PPPP/R1B1KBNR w KQkq - 1 4 acd 6; acn 1436575; cc 4:55.265 4:54.606; pes +0.020; pv Qd3 h6 Nf3 Nf6 e4 Bd6; te 1.445; tu 1.333;

Note:

1) The half move counter and the full move number now appear explicitly as the fifth and sixth fields of a standard FEN position specification. The earlier EPD version had these omitted or appearing as optional operations (hmvc and fmvn).

2) Most things don't change: acd, acn, am, bm, pv, sv, etc.

3) Some things are deprecated, such as pm (use the first parameter value of pv instead).

I find pm useful (predicted move may not be the same thing as best move for instance). I have code that uses the pm field.

4) All time values share the same the same format of DDD:HH:MM:SS.mmm with optional leading zero suppression. Example: cc (chess clock) has a White and a Black countdown time operand.

5) Instead of acs, te records analysis elapsed (wall) time, and tu represents usage time. Usage may be greater than elapsed time for multicore analysis.

6) The ce operator is deprecated; the replacement pes (pawn equivalent score) operator gives an evaluation in signed floating point pawns with millipawn resolution. There are a few non-decimal values: MateInN (e.g., MateIn3), LoseInN, Even, PosInf, NegInf, and Broken.

Perhaps inconvenient for some, since almost nobody uses floating point for evaluation. But if you are going to make it floating point, and 1.0 is supposed to represent a pawn, then why use milliseconds? Just let it be a double and then they can have an internal precision that they like. In any case, we're going to use sscanf("%lf"... to collect it.

7) A few operations are undecided at the moment, like dm and solve.

I find dm very useful. I do not even know what solve is for because I have never run across that field. People are going to use different scales for checkmate (even though you have 32767 listed as written in stone, lots of people use 9999, 99999 or some other value instead, so dm has a clear purpose here).

8) Symbol operands are never quoted and string operands are always quoted. Examples:

id WAC.241;

I have literally hundreds of thousands of id fields with spaces in them. There was nothing in the original spec to prevent that and I will find it most annoying if I am not allowed to quote id fields. In fact, I can tell you now that I will ignore that new part of the spec because I have not enough time nor the compulsion to correct them.

name "Win at Chess problem 241";

One more thing while I am whinging on...

I would like an 'er' field for 'expected response'. This field would be very useful for engine pondering.

Another thing, this *has* to be optional:

1) The half move counter and the full move number now appear explicitly as the fifth and sixth fields of a standard FEN position specification. The earlier EPD version had these omitted or appearing as optional operations (hmvc and fmvn).

I have at least a million EPD records that will be broken if it is a requirement.

So, if missing, the full move number and half move clock must be assumed as initial state.

[ilari]

Mostly bug fixes again.

New in version 0.2.0:
- Stricter parsing of command line arguments
- Fixed some inter-process communication bugs in the Windows version
- Fixed a couple of concurrency bugs
- Better handling of SIGINT (CTRL-C)

This version shouldn't suffer from the random crashes that sometimes (rarely) happened with the Windows version of 0.1.9 when using the concurrency option. Chances are good that the "pure virtual method called" bug is fixed too.

Sorry about using you guys as a free testing team. I really should start writing some unit tests that I could run before releasing experimental stuff like this...

[Anton]

Mostly bug fixes again.

New in version 0.2.0:
- Stricter parsing of command line arguments
- Fixed some inter-process communication bugs in the Windows version
- Fixed a couple of concurrency bugs
- Better handling of SIGINT (CTRL-C)

This version shouldn't suffer from the random crashes that sometimes (rarely) happened with the Windows version of 0.1.9 when using the concurrency option. Chances are good that the "pure virtual method called" bug is fixed too.

Sorry about using you guys as a free testing team. I really should start writing some unit tests that I could run before releasing experimental stuff like this...

This sounds very interesting.

Is there a linux_32 binary available?

[ilari]

This sounds very interesting.

Is there a linux_32 binary available?

Unfortunately no, because I only use 64-bit Linux, and don't have the 32-bit support libraries installed. Maybe someone could compile a 32-bit build (I need a cutechess-cli executable and the libchess library files) and send it to me, then I could host it.

[Anton]

This sounds very interesting.

Is there a linux_32 binary available?

Unfortunately no, because I only use 64-bit Linux, and don't have the 32-bit support libraries installed. Maybe someone could compile a 32-bit build (I need a cutechess-cli executable and the libchess library files) and send it to me, then I could host it.

I took the plunge, and installed the QT libs.

I will send you the cutechess-cli binary + the 4 libchess files for hosting.

[ilari]

This sounds very interesting.

Is there a linux_32 binary available?

Unfortunately no, because I only use 64-bit Linux, and don't have the 32-bit support libraries installed. Maybe someone could compile a 32-bit build (I need a cutechess-cli executable and the libchess library files) and send it to me, then I could host it.

I took the plunge, and installed the QT libs.

I will send you the cutechess-cli binary + the 4 libchess files for hosting.

Thanks a bunch. The 32-bit Linux binaries are now available to everyone here: http://koti.mbnet.fi/~ilaripih/bin/cute ... x32.tar.gz

[bob]

I've uploaded a new version (0.1.3) with a few changes:

- Support millisecond-precision in time controls

I agree that millisecond resolution is the way to go. Piloted spacecraft use a millisecond period update for critical telemetry; if it's good enough for a spaceship then I suppose it's good enough for a chess clock.

A new EPD specification is coming, and it uses a tighter formalism for a number of things including time values. Example:

Code: Select all

r1bqkbnr/pppp1ppp/2n5/8/3Q4/2N5/PPP1PPPP/R1B1KBNR w KQkq - 1 4 acd 6; acn 1436575; cc 4:55.265 4:54.606; pes +0.020; pv Qd3 h6 Nf3 Nf6 e4 Bd6; te 1.445; tu 1.333;

Note:

1) The half move counter and the full move number now appear explicitly as the fifth and sixth fields of a standard FEN position specification. The earlier EPD version had these omitted or appearing as optional operations (hmvc and fmvn).

2) Most things don't change: acd, acn, am, bm, pv, sv, etc.

3) Some things are deprecated, such as pm (use the first parameter value of pv instead).

4) All time values share the same the same format of DDD:HH:MM:SS.mmm with optional leading zero suppression. Example: cc (chess clock) has a White and a Black countdown time operand.

5) Instead of acs, te records analysis elapsed (wall) time, and tu represents usage time. Usage may be greater than elapsed time for multicore analysis.

6) The ce operator is deprecated; the replacement pes (pawn equivalent score) operator gives an evaluation in signed floating point pawns with millipawn resolution. There are a few non-decimal values: MateInN (e.g., MateIn3), LoseInN, Even, PosInf, NegInf, and Broken.

7) A few operations are undecided at the moment, like dm and solve.

8) Symbol operands are never quoted and string operands are always quoted. Examples:

id WAC.241;
name "Win at Chess problem 241";

I'd suggest just one time. "usage" is effectively meaningless. It is sensitive to machine load, parallel search overhead, number of cores, whether or not hyperthreading is enabled, etc. It is effectively a random number and could easily be discarded with no loss of information, since it really carries none.

[bob]

I've uploaded a new version (0.1.3) with a few changes:

- Support millisecond-precision in time controls

I agree that millisecond resolution is the way to go. Piloted spacecraft use a millisecond period update for critical telemetry; if it's good enough for a spaceship then I suppose it's good enough for a chess clock.

A new EPD specification is coming, and it uses a tighter formalism for a number of things including time values. Example:

Code: Select all

r1bqkbnr/pppp1ppp/2n5/8/3Q4/2N5/PPP1PPPP/R1B1KBNR w KQkq - 1 4 acd 6; acn 1436575; cc 4:55.265 4:54.606; pes +0.020; pv Qd3 h6 Nf3 Nf6 e4 Bd6; te 1.445; tu 1.333;

Note:

1) The half move counter and the full move number now appear explicitly as the fifth and sixth fields of a standard FEN position specification. The earlier EPD version had these omitted or appearing as optional operations (hmvc and fmvn).

2) Most things don't change: acd, acn, am, bm, pv, sv, etc.

3) Some things are deprecated, such as pm (use the first parameter value of pv instead).

I find pm useful (predicted move may not be the same thing as best move for instance). I have code that uses the pm field.

4) All time values share the same the same format of DDD:HH:MM:SS.mmm with optional leading zero suppression. Example: cc (chess clock) has a White and a Black countdown time operand.

5) Instead of acs, te records analysis elapsed (wall) time, and tu represents usage time. Usage may be greater than elapsed time for multicore analysis.

6) The ce operator is deprecated; the replacement pes (pawn equivalent score) operator gives an evaluation in signed floating point pawns with millipawn resolution. There are a few non-decimal values: MateInN (e.g., MateIn3), LoseInN, Even, PosInf, NegInf, and Broken.

Perhaps inconvenient for some, since almost nobody uses floating point for evaluation. But if you are going to make it floating point, and 1.0 is supposed to represent a pawn, then why use milliseconds? Just let it be a double and then they can have an internal precision that they like. In any case, we're going to use sscanf("%lf"... to collect it.

7) A few operations are undecided at the moment, like dm and solve.

I find dm very useful. I do not even know what solve is for because I have never run across that field. People are going to use different scales for checkmate (even though you have 32767 listed as written in stone, lots of people use 9999, 99999 or some other value instead, so dm has a clear purpose here).

Under what circumstance(s) would this be true? I can't come up with a single example where a program would have move X as the second move in the PV, and move Y as the predicted move. Makes no sense to me.

8) Symbol operands are never quoted and string operands are always quoted. Examples:

id WAC.241;

I have literally hundreds of thousands of id fields with spaces in them. There was nothing in the original spec to prevent that and I will find it most annoying if I am not allowed to quote id fields. In fact, I can tell you now that I will ignore that new part of the spec because I have not enough time nor the compulsion to correct them.

I actually like that idea myself. Parsing is a science, not an art. embedded spaces makes it more art than science. If you check out vista and windows 7, microsoft finally admitted that crap like "Program Files" was a bad idea without the quotes. It'd be easy to write s tiny awk script to quote the "id" tag since a ; or end of line is the delimiter.

name "Win at Chess problem 241";

[Dann Corbit]

I've uploaded a new version (0.1.3) with a few changes:

- Support millisecond-precision in time controls

I agree that millisecond resolution is the way to go. Piloted spacecraft use a millisecond period update for critical telemetry; if it's good enough for a spaceship then I suppose it's good enough for a chess clock.

A new EPD specification is coming, and it uses a tighter formalism for a number of things including time values. Example:

Code: Select all

r1bqkbnr/pppp1ppp/2n5/8/3Q4/2N5/PPP1PPPP/R1B1KBNR w KQkq - 1 4 acd 6; acn 1436575; cc 4:55.265 4:54.606; pes +0.020; pv Qd3 h6 Nf3 Nf6 e4 Bd6; te 1.445; tu 1.333;

Note:

1) The half move counter and the full move number now appear explicitly as the fifth and sixth fields of a standard FEN position specification. The earlier EPD version had these omitted or appearing as optional operations (hmvc and fmvn).

2) Most things don't change: acd, acn, am, bm, pv, sv, etc.

3) Some things are deprecated, such as pm (use the first parameter value of pv instead).

I find pm useful (predicted move may not be the same thing as best move for instance). I have code that uses the pm field.

4) All time values share the same the same format of DDD:HH:MM:SS.mmm with optional leading zero suppression. Example: cc (chess clock) has a White and a Black countdown time operand.

5) Instead of acs, te records analysis elapsed (wall) time, and tu represents usage time. Usage may be greater than elapsed time for multicore analysis.

6) The ce operator is deprecated; the replacement pes (pawn equivalent score) operator gives an evaluation in signed floating point pawns with millipawn resolution. There are a few non-decimal values: MateInN (e.g., MateIn3), LoseInN, Even, PosInf, NegInf, and Broken.

Perhaps inconvenient for some, since almost nobody uses floating point for evaluation. But if you are going to make it floating point, and 1.0 is supposed to represent a pawn, then why use milliseconds? Just let it be a double and then they can have an internal precision that they like. In any case, we're going to use sscanf("%lf"... to collect it.

7) A few operations are undecided at the moment, like dm and solve.

I find dm very useful. I do not even know what solve is for because I have never run across that field. People are going to use different scales for checkmate (even though you have 32767 listed as written in stone, lots of people use 9999, 99999 or some other value instead, so dm has a clear purpose here).

Under what circumstance(s) would this be true? I can't come up with a single example where a program would have move X as the second move in the PV, and move Y as the predicted move. Makes no sense to me.

Under the circumstances where I have a best move and predicted response but I do not have a stored pv.

8) Symbol operands are never quoted and string operands are always quoted. Examples:

id WAC.241;

I have literally hundreds of thousands of id fields with spaces in them. There was nothing in the original spec to prevent that and I will find it most annoying if I am not allowed to quote id fields. In fact, I can tell you now that I will ignore that new part of the spec because I have not enough time nor the compulsion to correct them.

I actually like that idea myself. Parsing is a science, not an art. embedded spaces makes it more art than science. If you check out vista and windows 7, microsoft finally admitted that crap like "Program Files" was a bad idea without the quotes. It'd be easy to write s tiny awk script to quote the "id" tag since a ; or end of line is the delimiter.

name "Win at Chess problem 241";