UCI protocol issue

[Don]

... - and refusing to move is not an option. At least not with UCI.

I think you make that up entirely yourself. The protocol specs don't say that at all.

What they do explicitly say is that the engine can send 0000 as a move, to be understood as a null move. It does NOT say there are any restrictions to its use (like only in the pv info). In fact it does not even specify that moves sent by the engine have to be legal moves anywhere.

If an engine sends "bestmove 0000", with a score info of 0 cp (or -0 mate) , it is totally conforming to the UCI specs. If a GUI cannot handle that, this GUI is not UCI compliant...

Here is what the spec says:

* bestmove <move1> [ ponder <move2> ]
the engine has stopped searching and found the move <move> best in this position.
the engine can send the move it likes to ponder on. The engine must not start pondering automatically.
this command must always be sent if the engine stops searching, also in pondering mode if there is a
"stop" command, so for every "go" command a "bestmove" command is needed!
Directly before that the engine should send a final "info" command with the final search information,
the the GUI has the complete statistics about the last search.


"null" is not a move and it's not in the rules of chess - so this is really quite wrong. The protocol does define 0000 to be a null move sent from the engine - but it is obviously only valid when there is NO legal move to sends - "null move" means "no move", it's not a special kind of move and it's not a pass. With your very wide interpretation the engine should be able to say, "I don't have a move" any time it wants and the GUI should just be able to gracefully handle it. The only possible interpretation of it is that there is legal moves. Otherwise it is just a forfeit. I'm not making this up, try doing this in the next tournament you are playing in, tell the TD you don't wish to play a move (perhaps in a zugzwang position) and see what happens.

I personally think 0000 is another glitch in the protocol, it cannot be used to offer a draw or resign as it has no meaning in the context of chess and if a GUI cannot handle that it's NOT a bug. No GUI should every ask for a search when you are in checkmate or stalemate or the game is over due to repetition.

The other little nastiness in the protocol is that by the rules of chess there is 3-fold repetition and 50 move draw and the possibility to resign or offer a draw. With 3-fold and repetition some player on move must claim it. I implemented 50 move rule in Komodo and saw some games where Komodo threw away the game - giving away pieces to lose when 50 moves had passed due to a buggy GUI. I could make Komodo play as if the draw was farther away but that really seems broken to me. So there SHOULD be a way for a engine to claim the draw. Sending 0000 is not a way to claim a draw because it doesn't explicitly mean anything. It could mean that you resign for example. The only possible meaning it can have by the rules of chess is "there is no legal move to play" and beyond the rules of chess it could only mean "I refuse to play."

If a GUI cannot handle 0000 you say it's not UCI compliant - but I say it's a bug in the UCI spec, not the GUI because what is a GUI supposed to do with meaningless information? Do you mean if it crashes it is not compliant? What behavior would you consider UCI compliant? Here are the broken choices:

1. Treat is as a resignation.
2. Treat it as a draw offer (how is the draw offer accepted or denied?)
3. The GUI exits cleanly (without crashing?)
4. The game is recorded as unfinished? (I'll make Komodo do this every time it is losing)
5. Treat is as a chess variant where any player can pass at will? (That is not chess.)

There is NO spec which says how the GUI is to handle this case and in fact there IS NO sensible way to handle it - so it's a UCI protocol bug or at least a bug in the specification document.

If a chess program is asked to search a position with no moves, I consider that a GUI bug - but there is room for debate on this one. At least sending back 0000 in this (and only this) case is sensible because by definition there is "no move" or "null move." But no serious GUI should ask to search a position that has no legal moves. If a GUI does this it's almost a sure sign it has no legal move generator of it's own - it doesn't even know the rules of the game.

[lucasart]

A GUI should have no need to examine the move

.

That is true but I am not sure all GUIs will ignore it.

IMO this really shows up a fault in the protocol design.

The problem is not the protocol, the GUI is at fault here. It should never send a position where the game is over by chess rules. In this case it should not send the go ponder, recognizing that the position after the ponder move is game over, so there's nothing to ponder.

Not sending game over positions is one of the golden rules of any well designed protocol (I suppose Xboard too). The GUI is the referee, and the engine always assumes valid input from the GUI.

[mcostalba]

1. UCI GUI sends a move list whose terminal position is a repetition draw (last move is the 3rd repetition).

GUI is broken !


GUI should not send, I fully agree with Lucas here.

[hgm]

I'm not making this up, try doing this in the next tournament you are playing in, tell the TD you don't wish to play a move (perhaps in a zugzwang position) and see what happens.

Isn't it obvious what happens?

1) Try it in a position where you are stalemated: TD will declare a draw.
2) Try it in a position where there is 3-fold-repeat: TD will declare a draw.
3) Try it after 50 reversible moves: TD will declare a draw.
4) Try it in other positions: TD will forfeit you.

Now with (2) and (3) things might go more smoothly if you accompany your message that you don't want to play a move with a reason (because the TD might not immediately see that): "I don't want to play a move, because I already have done that 2 times before in the same position", etc. But in UCI the engine is obliged to comment on its moves anyway through a pv info command just before it, and its opinion that refusing to move should result in a 0 score should be a good enough hint to the GUI to check the 50-move or 3-fold-rep condition.

With (4) you forfeit, which has the same effect as resiging.

I personally think 0000 is another glitch in the protocol, it cannot be used to offer a draw or resign as it has no meaning in the context of chess and if a GUI cannot handle that it's NOT a bug.

Arguably the entire UCI protocol is a glitch... But it is what it is, and the specs do define 0000 as a move. And that it cannot be used to resign (or anything else) is not in the specs, but just something made up by you.

No GUI should every ask for a search when you are in checkmate or stalemate or the game is over due to repetition.

But according to FIDE rules, a game is never over due to repetition. It is only over when a player claims. In effect you are saying: "No (UCI) GUI should ever obey FIDE rules"...

Sending 0000 is not a way to claim a draw because it doesn't explicitly mean anything. It could mean that you resign for example.

It explicitly means that you refuse to move. The reason can go in the mandatory pv info that comes with it. In particular, sending a 0 score according to the specs means "I don't want to move, and I think this is a draw after that". If that is not a draw claim, I wouldn't know what is.

The only possible meaning it can have by the rules of chess is "there is no legal move to play" and beyond the rules of chess it could only mean "I refuse to play."

Except that the latter is not "beyond the rules of chess". The rules of chess specify exactly when you can refuse to play: after a 3-fold rep and 50 reversible moves you can refuse to play a move but opt for a draw in stead, and in any other case you can resign if you don't want to play on.

If a GUI cannot handle 0000 you say it's not UCI compliant - but I say it's a bug in the UCI spec,

Yeah, and I think that if a GUI sends 'uci' I think that it is a GUI bug. It should send 'xboard' in stead... But what you or I think is not very relevant. There is no arguing with the specs.

not the GUI because what is a GUI supposed to do with meaningless information? Do you mean if it crashes it is not compliant? What behavior would you consider UCI compliant? Here are the broken choices:

1. Treat is as a resignation.
2. Treat it as a draw offer (how is the draw offer accepted or denied?)
3. The GUI exits cleanly (without crashing?)
4. The game is recorded as unfinished? (I'll make Komodo do this every time it is losing)
5. Treat is as a chess variant where any player can pass at will? (That is not chess.)

Software that crashes is never any good. I would say recognation or draw claim. Just like in real life using FIDE rules. The GUI should be able to figure out if this is a position where a draw claim should be homored, and can declare a loss otherwise.

Whether you want a GUI to exit after a game finishes is a design choice. In general you would not want that. (But in match mode started with the -mm command-line option XBoard exits after the match finishes, and you can ask for matches of 1 game, so it is not unheard of.)

Note that UCI is also used for Chess variants (Chess960, Chinese Chess). Like Chess these variants do not support turn passing, however. But it is not inconceivable someone would want to use UCI for a varinant that does (such as Chu Shogi). In that case there would of course be no reason for the GUI to terminate the game when an engine plays a null move. You could still adopt the convention that reporting a score of -INFINITY (mated in 0) together with the null move is a resign. Games that involve turn passing usually do not have special draw-claim rules. (E.g. in Go the game ends naturally when both sides pass their turn.)

There is NO spec which says how the GUI is to handle this case and in fact there IS NO sensible way to handle it - so it's a UCI protocol bug or at least a bug in the specification document.

Actually the specs don't even stipulate how a GUI should handle it when the engine supplies a legal bestmove. The specs very heavily rely on common sense.

If a chess program is asked to search a position with no moves, I consider that a GUI bug - but there is room for debate on this one. At least sending back 0000 in this (and only this) case is sensible because by definition there is "no move" or "null move." But no serious GUI should ask to search a position that has no legal moves. If a GUI does this it's almost a sure sign it has no legal move generator of it's own - it doesn't even know the rules of the game.

The specs do not explicitly forbid this. You could argue it is common sense to let the GUI handle positions that are 'game-end by FIDE rules'.Unlike with XBoard protocol, having a GUI that does not have rule knowledge is not really feasible with UCI. Unless you would be prepared to interpret 'bestmove 0000' accompanied by 'info score +INFINITY' (or whatever UCI says when it delivers checkmate) as a win claim because of an illegal move. Then the GUI could take the previous move back, and present the 'illegal move' message to the user, just like XBoard takes the latest move back when the engine sends it 'Illegal move'. This surey seems much preferable over the engine just hanging up when there is an illegal move in the position-moves command, as many engines do now.

[Sven]

I have had a bug report that involved the following scenario when running a match with UCI and ponder on:

1. UCI GUI sends a move list whose terminal position is a repetition draw (last move is the 3rd repetition).

2. UCI gives the engine a "go ponder" command.

3. The engine does the search but because it is a draw before any move is made, no move is returned.

4. Other engine sends a move and it is not the predicted one and not one that would cause a repetition draw.

5. UCI sends "stop"

6. Now the UCI protocol requires that a "bestmove" command be sent from the engine. But the engine has no move to send. What is correct behavior in this case? (This could also occur in other situations - for example move being pondered leads to a stalemate or mate).

--Jon

I have read all the replies so far but it may be possible that an important detail of the given problem has been missed.

The key is that we are not in the context of a normal "go" command but the engine has received a "go ponder" command. The spec says this:

* ponder
start searching in pondering mode.
Do not exit the search in ponder mode, even if it's mate!
This means that the last move sent in in the position string is the ponder move.
The engine can do what it wants to do, but after a "ponderhit" command
it should execute the suggested move to ponder on. This means that the ponder move sent by
the GUI can be interpreted as a recommendation about which move to ponder. However, if the
engine decides to ponder on a different move, it should not display any mainlines as they are
likely to be misinterpreted by the GUI because the GUI expects the engine to ponder
on the suggested move.

The last move in the move sequence is the ponder move. The engine shall start its search in the position before the ponder move, and the latter is a "recommendation". Now if the ponder move leads to a rep draw then the engine has the choice to decide whether it wants to ponder on a different move (and try to win), or wait for the "stop" and then send back "bestmove" with the ponder move which will end the game by rep draw.

If the engine behaves like in point 3 described above then it is definitely an engine bug since it starts the search from the wrong position, i.e. it searches after making the ponder move.

Sven

[hgm]

Beware that it is the opponent that will eventually decide if the ponder move is played or not, not the engine. So the latter cannot decide to 'go for a win'. But I agree that if your opponent can salvage the draw with his next move, it makes little sense to ponder on what you are going to do after he plays that. Your time would be much better spent by pondering on what you would reply to one of his other moves, which do not lead to an immediate draw. (When the opponent 'goes for the win'.)

But I think this is a moot point: if you have an engine that is smart enough to understand this, it should of course have passed that other move in the 'bestmove-ponder' command it sent to the GUI before. Because most GUIs would dutifully feed the ponder move back to the engine. So the engine gets no new information on which it can decide what to ponder on. It is always better (although not required by the specs) that the GUI knows which move the engine is pondering on. This helps it to present useful output to the user. So it only makes sense for an engine to use the lattitude the protocol specs grant it if it did not know in advance it would want to switch ponder move (e.g. when during the ponder process there occurs a disastrous fail high at some depth, making it likely you are pondering a blunder).

So unlike what some claim here, receiving a ponder move that leads to game end (mate, insufficient material etc.), it is really a bug of the engine, telling the GUI that it would like to ponder on such a move. If it cannot handle the situation after this move, it really should not have sent that ponder move to the GUI. The specs don't oblige the engine to send a ponder move. Only if the GUI would be sending other moves than the engine suggested for pondering on, it can be hold accountable for any mishaps.

[Sven]

Beware that it is the opponent that will eventually decide if the ponder move is played or not, not the engine. So the latter cannot decide to 'go for a win'. But I agree that if your opponent can salvage the draw with his next move, it makes little sense to ponder on what you are going to do after he plays that. Your time would be much better spent by pondering on what you would reply to one of his other moves, which do not lead to an immediate draw. (When the opponent 'goes for the win'.)

But I think this is a moot point: if you have an engine that is smart enough to understand this, it should of course have passed that other move in the 'bestmove-ponder' command it sent to the GUI before. Because most GUIs would dutifully feed the ponder move back to the engine. So the engine gets no new information on which it can decide what to ponder on. It is always better (although not required by the specs) that the GUI knows which move the engine is pondering on. This helps it to present useful output to the user. So it only makes sense for an engine to use the lattitude the protocol specs grant it if it did not know in advance it would want to switch ponder move (e.g. when during the ponder process there occurs a disastrous fail high at some depth, making it likely you are pondering a blunder).

So unlike what some claim here, receiving a ponder move that leads to game end (mate, insufficient material etc.), it is really a bug of the engine, telling the GUI that it would like to ponder on such a move. If it cannot handle the situation after this move, it really should not have sent that ponder move to the GUI. The specs don't oblige the engine to send a ponder move. Only if the GUI would be sending other moves than the engine suggested for pondering on, it can be hold accountable for any mishaps.

O.k., I almost fully agree (see some lines below for the only exception ), and I was wrong when writing

or wait for the "stop" and then send back "bestmove" with the ponder move

since that is not possible (the ponder move is the opponent's move). What I had in mind was the "try to win" idea by preparing for the case that the opponent does not play the drawing move, but the words I chose did not match that unfortunately. Preparing for a case that can only occur if the GUI misbehaves by not terminating the game on rep draw would indeed be a waste of time.

As you state, the situation we have here will arise if the engine itself has sent "bestmove xxxx ponder yyyy" as its last "bestmove" command where "yyyy" leads to a rep draw. This "yyyy" comes back to us as the last move in the "position ... moves ..." sequence prior to "go ponder". This is the only point where I slightly disagree to your statement: I do not consider this behaviour, i.e. announcing to ponder on the reply "yyyy" that draws, a bug of the engine, I think it is valid if the engine states that it expects "yyyy" as the best reply of the opponent to the move "xxxx", i.e. it does not see a way to win and plays "xxxx" while knowing that the opponent can enforce the draw with "yyyy". Announcing a different expected reply would be too smart in my view, and it might even annoy the user who might believe that the engine does not see that the opponent can enforce the draw.

Apart from this I stick to my proposal to start a ponder search on a different move that does enforce a draw (the question "which one" would now be valid, of course, but that is a different task.) Now there are two possible cases:

a) The GUI sends "ponderhit". This means the opponent has made the ponder move which leads to a rep draw but the GUI does not terminate the game, unlike what everyone would expect. Since UCI does not offer a choice for an engine to claim a draw this is a very bad situation for the engine: the combination of a protocol bug and a GUI bug. The engine can't solve it, and whatever it does is "wrong" in some way. I would simply ignore the repetition draw in this case and continue the normal search after the ponder move since everything else, like "bestmove 0000", would be a protocol violation and might also lose the game. The position that has occurred for the third time is not immediately a draw according to the FIDE rules, only if the draw is claimed. Under UCI the GUI must take this part, and the engine has no way to repair the misbehaviour of a GUI. But I think this is not the original case here.

b) The GUI sends a different move (this the original case of Jon). This is simple in my view since there is no rep draw involved (assuming that only the ponder move leads to a rep draw). Two subcases here:

b1) The opponent's move is actually the move we decided to ponder on, then we behave as if we received "ponderhit".
b2) The opponent's move is another move, then we switch to a normal search after that move.

Sven

[Sven]

I wrote:

Apart from this I stick to my proposal to start a ponder search on a different move that does enforce a draw

and obviously I meant "a different move that does NOT enforce a draw" ...

It is quite hot today here in Berlin ...

Sven

[Maarten]

To me the UCI spec is ambiguous in handling positions send by the GUI which are draw/won by rules. I share Don's point of view: "UCI is way to loosely defined and could be greatly improved by defining what specific behavior is expected under certain situations."
As a practical approach, to handle the 3-fold repetition case I have just disabled the 3-fold repetition check at the root, so the engine always returns a move. For other cases (i.e. the position send by GUI is a checkmate) I let the engine return 0000 as bestmove.

[Don]

1. UCI GUI sends a move list whose terminal position is a repetition draw (last move is the 3rd repetition).

GUI is broken !


GUI should not send, I fully agree with Lucas here.

I third that motion.