"go ponder"/"ponderhit" issue in UCI

[hgm]

Assuming the catalyst for this is the chess.com tournament.

I've told them I won't be supporting pondering, and that they should use real cores and not hyperthreads if they want good chess.

Any particular reason you don't want to support pondering?

And isn't there any way that the engine can detect how many real cores the machine on which it is running has, and refrain from using more than that even if the option setting allows it? E.g. you could make the maximum setting of the Threads option dependent on the machine you run on. Note that 'Threads' is not a standard UCI option. You are free to have it mean whatever you want it to mean.

[jdart]

Do you think this is a bug?

I wasn't sure, which is why I posted this. But now, I think not.

--Jon

[jdart]

And isn't there any way that the engine can detect how many real cores the machine on which it is running has, and refrain from using more than that even if the option setting allows it?

This is surprisingly hard, given the variety of CPU and NUMA configurations possible, although hwloc (https://www.open-mpi.org/projects/hwloc ... /index.php) can help with that.

-Jon

[Ferdy]

Using cutechess-cli in UCI mode with pondering on, I see quite a few instances in the debug log where "go ponder" is immediately followed by a "ponderhit" command, with no intervening commands from cutechess or either engine.

As I understand it, this is really equivalent to a foreground (non-pondering) search because "ponderhit" is informing the engine that the predicted move was actually made, and so it is the engine's turn to move, and if it has not completed the ponder search, it should continue it but only up to a time or depth limit. Is there any reason in this case to not just send a "go" command (w/o ponder) and omit the "ponderhit"?

(I am not sure if UCI programs other than cutechess-cli have this issue).

--Jon

Looking at the log it looks like a bug in cutechess.

Normally the pattern is when cutechess sent position command to engine1 it will follow it with go ponder to same engine1.
Once it receives bestmove from engine2, it will send ponderhit or stop to engine1.

Code: Select all

1657899 >Deuterium v2018.1.35.514(0): position startpos moves g1f3 d7d5 d2d4 g8f6 c2c4 c7c6 e2e3 g7g6 b1c3 f8g7 f1e2 e8g8 e1g1 e7e6 d1c2 b8d7 e3e4 d5e4 c3e4 f6e4 c2e4 e6e5 d4e5 d7e5 f3e5 f8e8 f2f4 f7f6 f1d1 d8e7 g2g4 e8f8 c1e3 f6e5 d1f1 e7f6 a1d1 e5f4 f1f4 f6e6 c4c5 e6e4 f4e4 g7b2 e4e7 g8h8 e3h6 f8g8 d1d6 b2g7 e2c4 g7h6 c4g8 c8g4 e7h7 h8g8 h7h6 g4f5 h6h4 a8e8 h4b4 e8e7 d6d8 g8f7 d8b8 f7e6 b4b7 e7b7 b8b7 a7a6 b7b6 e6d5 b6a6 d5c5 a2a4 f5d3 a6a8 c5b6 g1f2 c6c5 f2e3 d3f5 a8b8 b6c7 b8b5 c7c6 b5b2 c6c7 a4a5 f5g4 a5a6 g4e6 b2b5 e6h3 a6a7 h3g2 h2h4 c5c4 b5b8 c7d6 a7a8q g2a8 b8a8 d6e6 a8c8 e6d5 c8c7 c4c3 c7c3 d5e5 c3d3 e5e6 e3f4 e6f7 d3e3 f7f6 f4g4 f6g7 g4g5 g7f7 e3e2 f7g7 e2f2
1657901 >Deuterium v2018.1.35.514(0): go ponder wtime 120148 btime 38089 winc 3000 binc 3000
1658433 <Arasan 20.4.1(1): bestmove e2f2 ponder g7h7
1658433 >Deuterium v2018.1.35.514(0): ponderhit

After sending ponderhit to engine1 it will then process position and go ponder to engine2.

1658434 >Arasan 20.4.1(1): position startpos moves g1f3 d7d5 d2d4 g8f6 c2c4 c7c6 e2e3 g7g6 b1c3 f8g7 f1e2 e8g8 e1g1 e7e6 d1c2 b8d7 e3e4 d5e4 c3e4 f6e4 c2e4 e6e5 d4e5 d7e5 f3e5 f8e8 f2f4 f7f6 f1d1 d8e7 g2g4 e8f8 c1e3 f6e5 d1f1 e7f6 a1d1 e5f4 f1f4 f6e6 c4c5 e6e4 f4e4 g7b2 e4e7 g8h8 e3h6 f8g8 d1d6 b2g7 e2c4 g7h6 c4g8 c8g4 e7h7 h8g8 h7h6 g4f5 h6h4 a8e8 h4b4 e8e7 d6d8 g8f7 d8b8 f7e6 b4b7 e7b7 b8b7 a7a6 b7b6 e6d5 b6a6 d5c5 a2a4 f5d3 a6a8 c5b6 g1f2 c6c5 f2e3 d3f5 a8b8 b6c7 b8b5 c7c6 b5b2 c6c7 a4a5 f5g4 a5a6 g4e6 b2b5 e6h3 a6a7 h3g2 h2h4 c5c4 b5b8 c7d6 a7a8q g2a8 b8a8 d6e6 a8c8 e6d5 c8c7 c4c3 c7c3 d5e5 c3d3 e5e6 e3f4 e6f7 d3e3 f7f6 f4g4 f6g7 g4g5 g7f7 e3e2 f7g7 e2f2 g7h7
1658435 >Arasan 20.4.1(1): go ponder wtime 122614 btime 38089 winc 3000 binc 3000

Wait for best move of engine1 since ponderhit was already sent to it.

Code: Select all

1660593 <Deuterium v2018.1.35.514(0): bestmove g7h7 ponder f2f7

There is a case that cutechess does not follow the above pattern, see below.

Code: Select all

1661661 >Arasan 20.4.1(1): ponderhit
1661661 >Deuterium v2018.1.35.514(0): position startpos moves g1f3 d7d5 d2d4 g8f6 c2c4 c7c6 e2e3 g7g6 b1c3 f8g7 f1e2 e8g8 e1g1 e7e6 d1c2 b8d7 e3e4 d5e4 c3e4 f6e4 c2e4 e6e5 d4e5 d7e5 f3e5 f8e8 f2f4 f7f6 f1d1 d8e7 g2g4 e8f8 c1e3 f6e5 d1f1 e7f6 a1d1 e5f4 f1f4 f6e6 c4c5 e6e4 f4e4 g7b2 e4e7 g8h8 e3h6 f8g8 d1d6 b2g7 e2c4 g7h6 c4g8 c8g4 e7h7 h8g8 h7h6 g4f5 h6h4 a8e8 h4b4 e8e7 d6d8 g8f7 d8b8 f7e6 b4b7 e7b7 b8b7 a7a6 b7b6 e6d5 b6a6 d5c5 a2a4 f5d3 a6a8 c5b6 g1f2 c6c5 f2e3 d3f5 a8b8 b6c7 b8b5 c7c6 b5b2 c6c7 a4a5 f5g4 a5a6 g4e6 b2b5 e6h3 a6a7 h3g2 h2h4 c5c4 b5b8 c7d6 a7a8q g2a8 b8a8 d6e6 a8c8 e6d5 c8c7 c4c3 c7c3 d5e5 c3d3 e5e6 e3f4 e6f7 d3e3 f7f6 f4g4 f6g7 g4g5 g7f7 e3e2 f7g7 e2f2 g7h7 f2f7 h7h8 g5h6 h8g8 h6g6
1661663 <Arasan 20.4.1(1): bestmove h6g6 ponder g8h8
1661663 >Arasan 20.4.1(1): position startpos moves g1f3 d7d5 d2d4 g8f6 c2c4 c7c6 e2e3 g7g6 b1c3 f8g7 f1e2 e8g8 e1g1 e7e6 d1c2 b8d7 e3e4 d5e4 c3e4 f6e4 c2e4 e6e5 d4e5 d7e5 f3e5 f8e8 f2f4 f7f6 f1d1 d8e7 g2g4 e8f8 c1e3 f6e5 d1f1 e7f6 a1d1 e5f4 f1f4 f6e6 c4c5 e6e4 f4e4 g7b2 e4e7 g8h8 e3h6 f8g8 d1d6 b2g7 e2c4 g7h6 c4g8 c8g4 e7h7 h8g8 h7h6 g4f5 h6h4 a8e8 h4b4 e8e7 d6d8 g8f7 d8b8 f7e6 b4b7 e7b7 b8b7 a7a6 b7b6 e6d5 b6a6 d5c5 a2a4 f5d3 a6a8 c5b6 g1f2 c6c5 f2e3 d3f5 a8b8 b6c7 b8b5 c7c6 b5b2 c6c7 a4a5 f5g4 a5a6 g4e6 b2b5 e6h3 a6a7 h3g2 h2h4 c5c4 b5b8 c7d6 a7a8q g2a8 b8a8 d6e6 a8c8 e6d5 c8c7 c4c3 c7c3 d5e5 c3d3 e5e6 e3f4 e6f7 d3e3 f7f6 f4g4 f6g7 g4g5 g7f7 e3e2 f7g7 e2f2 g7h7 f2f7 h7h8 g5h6 h8g8 h6g6 g8h8
1661664 >Deuterium v2018.1.35.514(0): go ponder wtime 128338 btime 44146 winc 3000 binc 3000
1661665 >Deuterium v2018.1.35.514(0): ponderhit

Deuterium received position command but has not received the go ponder immediately.
It seems cutechess does this if there is successive ponderhit for both engines. But perhaps the pattern,

Code: Select all

> engine1 position ...
> engine1 go ponder ...

should have been maintained.

After further log examination, cutechess will only send go ponder command if the engine will respond to isready with readyok. Also cutechess will always send isready after position command to same engine.

Code: Select all

21524 >Arasan 20.4.1(1): position startpos moves e2e4 c7c5 a2a3 g7g6 g1f3 f8g7 c2c3 d7d5 e4d5 d8d5 d2d4 g8f6 f1e2 e8g8 e1g1 c5d4 c3d4 d5d6 b1c3 f8d8 f3e5 f6d7 c3b5 d6b6 c1e3
21525 >Arasan 20.4.1(1): isready
21525 <Arasan 20.4.1(1): readyok
21525 >Arasan 20.4.1(1): go ponder wtime 55844 btime 50988 winc 1000 binc 1000
22774 <Stockfish 9(0): bestmove c1e3 ponder a7a6
22774 >Arasan 20.4.1(1): ponderhit
22774 >Stockfish 9(0): position startpos moves e2e4 c7c5 a2a3 g7g6 g1f3 f8g7 c2c3 d7d5 e4d5 d8d5 d2d4 g8f6 f1e2 e8g8 e1g1 c5d4 c3d4 d5d6 b1c3 f8d8 f3e5 f6d7 c3b5 d6b6 c1e3 a7a6
22775 >Stockfish 9(0): isready
22775 <Stockfish 9(0): readyok
22775 >Stockfish 9(0): go ponder wtime 55598 btime 50988 winc 1000 binc 1000
24589 <Arasan 20.4.1(1): bestmove a7a6 ponder d1b3
24589 >Stockfish 9(0): ponderhit
24589 >Arasan 20.4.1(1): position startpos moves e2e4 c7c5 a2a3 g7g6 g1f3 f8g7 c2c3 d7d5 e4d5 d8d5 d2d4 g8f6 f1e2 e8g8 e1g1 c5d4 c3d4 d5d6 b1c3 f8d8 f3e5 f6d7 c3b5 d6b6 c1e3 a7a6 d1b3
24590 >Arasan 20.4.1(1): isready
24590 <Arasan 20.4.1(1): readyok
24590 >Arasan 20.4.1(1): go ponder wtime 55598 btime 50170 winc 1000 binc 1000
28883 <Stockfish 9(0): bestmove d1a4 ponder b8c6
28883 >Arasan 20.4.1(1): stop
28883 >Stockfish 9(0): position startpos moves e2e4 c7c5 a2a3 g7g6 g1f3 f8g7 c2c3 d7d5 e4d5 d8d5 d2d4 g8f6 f1e2 e8g8 e1g1 c5d4 c3d4 d5d6 b1c3 f8d8 f3e5 f6d7 c3b5 d6b6 c1e3 a7a6 d1a4 b8c6
28884 >Stockfish 9(0): isready
28884 <Stockfish 9(0): readyok
28884 >Stockfish 9(0): go ponder wtime 52305 btime 50170 winc 1000 binc 1000

If the readyok from the engine is delayed (for whatever reasons ...), so is the go from cutechess. So the unfortunate go ponder/ponderhit is possible.

Code: Select all

39755 >Arasan 20.4.1(1): position startpos moves e2e4 c7c5 a2a3 g7g6 g1f3 f8g7 c2c3 d7d5 e4d5 d8d5 d2d4 g8f6 f1e2 e8g8 e1g1 c5d4 c3d4 d5d6 b1c3 f8d8 f3e5 f6d7 c3b5 d6b6 c1e3 a7a6 d1a4 d8f8 a1c1 b8c6 e5c6
39755 >Arasan 20.4.1(1): isready
39756 <Stockfish 9(0): bestmove e5c6 ponder b7c6
39756 >Stockfish 9(0): position startpos moves e2e4 c7c5 a2a3 g7g6 g1f3 f8g7 c2c3 d7d5 e4d5 d8d5 d2d4 g8f6 f1e2 e8g8 e1g1 c5d4 c3d4 d5d6 b1c3 f8d8 f3e5 f6d7 c3b5 d6b6 c1e3 a7a6 d1a4 d8f8 a1c1 b8c6 e5c6 b7c6
39756 >Stockfish 9(0): isready
39756 <Arasan 20.4.1(1): readyok
39756 >Arasan 20.4.1(1): go ponder wtime 52334 btime 42295 winc 1000 binc 1000
39756 >Arasan 20.4.1(1): ponderhit

Cutechess' use of isready seemed abusive, it is sending isready all the time . Of course it is a design choice to completely control the engines. "If you don't give me the readyok I won't give you a go"

From uci protocol.

* isready
this is used to synchronize the engine with the GUI. When the GUI has sent a command or
multiple commands that can take some time to complete,
this command can be used to wait for the engine to be ready again or
to ping the engine to find out if it is still alive.
E.g. this should be sent after setting the path to the tablebases as this can take some time.
This command is also required once before the engine is asked to do any search
to wait for the engine to finish initializing.
This command must always be answered with "readyok" and can be sent also when the engine is calculating
in which case the engine should also immediately answer with "readyok" without stopping the search.