UCI protocol issue

[Don]

I'm not making this up, try doing this in the next tournament you are playing in, tell the TD you don't wish to play a move (perhaps in a zugzwang position) and see what happens.

Isn't it obvious what happens?

1) Try it in a position where you are stalemated: TD will declare a draw.
2) Try it in a position where there is 3-fold-repeat: TD will declare a draw.
3) Try it after 50 reversible moves: TD will declare a draw.
4) Try it in other positions: TD will forfeit you.

Now with (2) and (3) things might go more smoothly if you accompany your message that you don't want to play a move with a reason (because the TD might not immediately see that): "I don't want to play a move, because I already have done that 2 times before in the same position", etc. But in UCI the engine is obliged to comment on its moves anyway through a pv info command just before it, and its opinion that refusing to move should result in a 0 score should be a good enough hint to the GUI to check the 50-move or 3-fold-rep condition.

With (4) you forfeit, which has the same effect as resiging.

I personally think 0000 is another glitch in the protocol, it cannot be used to offer a draw or resign as it has no meaning in the context of chess and if a GUI cannot handle that it's NOT a bug.

Arguably the entire UCI protocol is a glitch... But it is what it is, and the specs do define 0000 as a move. And that it cannot be used to resign (or anything else) is not in the specs, but just something made up by you.

I'm not making this up - you cannot reasonably expect 0000 to be interpreted as a resignation! If a GUI does in fact interpret it as a resignation who is making things up, me or the GUI author? It would be an arbitrary decision for a GUI to do so, especially since it is not defined to mean anything other than a "null move."

YOU are making things up because you have come up with an interpretation that to you sounds reasonable - even though it's not defined. Even though I agree that is the most reasonable interpretation it is not defined that way in the protocol. So who is making things up? I think you are making things up because you like the logic behind it, not because it is actually is defined to mean "resign."

[Sven]

1. UCI GUI sends a move list whose terminal position is a repetition draw (last move is the 3rd repetition).

GUI is broken !


GUI should not send, I fully agree with Lucas here.

I third that motion.

Why? The engine had previously sent "bestmove xxxx ponder yyyy" expecting "yyyy" to be the best reply, even if "yyyy" enforces a rep draw, and the GUI which does not know yet whether the opponent will actually play "yyyy" or a different move advices the engine to start its ponder search with "go ponder", leaving open what the engine wants to do in that situation (ponder on "yyyy", on a different move, or not to ponder at all). The only case where I would see a broken GUI would be if the opponent would play "yyyy" and the GUI would send "ponderhit" to the engine. But in the given case the opponent plays a different move and the GUI correctly sends "stop" to stop the engine's ponder search, whatever the engine has done during that time.

My suggestion for the engine was to ponder on a different move than "yyyy" and react to "stop" with a normal "bestmove".

Sven

[hgm]

Announcing a different expected reply would be too smart in my view, and it might even annoy the user who might believe that the engine does not see that the opponent can enforce the draw.

I think you confuse things here: 'bestmove xxxx ponder yyyy' does in no way imply yyyy is the expected (=best) reply. It means yyyy is the move you want to ponder on. Nothing more, nothing less. The expected reply should follow from the info-pv that the engine send. If that would be by definition the same as the ponder move, there would not have been any need whatsoever to send the 'ponder' addition to 'bestmove'. The very design of UCI makes a provision for these moves to be different.

Apart from this I stick to my proposal to start a ponder search on a different move that does enforce a draw (the question "which one" would now be valid, of course, but that is a different task.) Now there are two possible cases:

a) The GUI sends "ponderhit". This means the opponent has made the ponder move which leads to a rep draw but the GUI does not terminate the game, unlike what everyone would expect. Since UCI does not offer a choice for an engine to claim a draw this is a very bad situation for the engine: the combination of a protocol bug and a GUI bug.

Of course your assertion that the engine would not be able to claim is kind of shaky. It could send bestmove 0000 with a 0 cp score... Do the UCI specs say anywhere that an engine should not be able to claim? Seems to me this 'protocol bug' is purely imagined.

The engine can't solve it, and whatever it does is "wrong" in some way. I would simply ignore the repetition draw in this case and continue the normal search after the ponder move since everything else, like "bestmove 0000", would be a protocol violation

Uh? Can you quote the exact rule in the protocol specs that would be violated, then?

and might also lose the game.

Indeed. False claims tend to do that.

[Don]

I'm not making this up, try doing this in the next tournament you are playing in, tell the TD you don't wish to play a move (perhaps in a zugzwang position) and see what happens.

Isn't it obvious what happens?

1) Try it in a position where you are stalemated: TD will declare a draw.
2) Try it in a position where there is 3-fold-repeat: TD will declare a draw.
3) Try it after 50 reversible moves: TD will declare a draw.
4) Try it in other positions: TD will forfeit you.

Now with (2) and (3) things might go more smoothly if you accompany your message that you don't want to play a move with a reason (because the TD might not immediately see that): "I don't want to play a move, because I already have done that 2 times before in the same position", etc. But in UCI the engine is obliged to comment on its moves anyway through a pv info command just before it, and its opinion that refusing to move should result in a 0 score should be a good enough hint to the GUI to check the 50-move or 3-fold-rep condition.

With (4) you forfeit, which has the same effect as resiging.

I personally think 0000 is another glitch in the protocol, it cannot be used to offer a draw or resign as it has no meaning in the context of chess and if a GUI cannot handle that it's NOT a bug.

Arguably the entire UCI protocol is a glitch... But it is what it is, and the specs do define 0000 as a move. And that it cannot be used to resign (or anything else) is not in the specs, but just something made up by you.

No GUI should every ask for a search when you are in checkmate or stalemate or the game is over due to repetition.

But according to FIDE rules, a game is never over due to repetition. It is only over when a player claims. In effect you are saying: "No (UCI) GUI should ever obey FIDE rules"...

I'm a little conflicted on how to answer this because of the implicit nature of UCI, a protocol designed to relegate some of the more mundane decisions to the GUI. Technically however you are exactly right and I concede that point.

However, I would not want it to be any other way that what I said. The reason is that I do not want to add post or pre-move gamesmanship to Komodo and very few programs do this. For example it is possible (although almost silly) to think that a chess program would purposely walk into a draw by repetition in the hopes the opponent will be the one to vary based on some estimation of the opponents limitations. A properly designed engine would never get this wrong, but I like UCI precisely because I don't want to have to worry about this. Komodo doesn't count the number of repetitions but I could easily add that, but I like that I don't have to do that.

So we have a situation where the GUI should be doing this anyway (a GUI that doesn't know if a draw claim is valid is broken) so why should Komodo have to do it?

I admit there is room for debate here, but my vision of an ideal protocol is one that requires as little as possible from the engine author - he should be free to focus on the things that really matter and not the intricacies of GUI design. The engine should not have to "help" the GUI. So anything that a GUI has to do anyway, the engine should not have to do.

In a way we each have totally different prospective's that probably cannot be considered fully objective. You are lazy, I am lazy. I want the GUI to do as much for me as possible and you are exactly the opposite. You want the engine to do all the work because you are primarily a GUI maker.

Part of the issue with you is that you are supporting so many games you are almost forced to lean on the engine. I don't think your GUI even understands what moves are legal in some of those games - is that correct? I'm not being critical, I fully understand what a pain it would be to build a fully legal move checking system for so many games. It's not easy even to build a bug-free legal move generator for chess that understand all the dark corners of the rules.

Sending 0000 is not a way to claim a draw because it doesn't explicitly mean anything. It could mean that you resign for example.

It explicitly means that you refuse to move. The reason can go in the mandatory pv info that comes with it. In particular, sending a 0 score according to the specs means "I don't want to move, and I think this is a draw after that". If that is not a draw claim, I wouldn't know what is.

The only possible meaning it can have by the rules of chess is "there is no legal move to play" and beyond the rules of chess it could only mean "I refuse to play."

Except that the latter is not "beyond the rules of chess". The rules of chess specify exactly when you can refuse to play: after a 3-fold rep and 50 reversible moves you can refuse to play a move but opt for a draw in stead, and in any other case you can resign if you don't want to play on.

If a GUI cannot handle 0000 you say it's not UCI compliant - but I say it's a bug in the UCI spec,

Yeah, and I think that if a GUI sends 'uci' I think that it is a GUI bug. It should send 'xboard' in stead... But what you or I think is not very relevant. There is no arguing with the specs.

not the GUI because what is a GUI supposed to do with meaningless information? Do you mean if it crashes it is not compliant? What behavior would you consider UCI compliant? Here are the broken choices:

1. Treat is as a resignation.
2. Treat it as a draw offer (how is the draw offer accepted or denied?)
3. The GUI exits cleanly (without crashing?)
4. The game is recorded as unfinished? (I'll make Komodo do this every time it is losing)
5. Treat is as a chess variant where any player can pass at will? (That is not chess.)

Software that crashes is never any good. I would say recognation or draw claim. Just like in real life using FIDE rules. The GUI should be able to figure out if this is a position where a draw claim should be homored, and can declare a loss otherwise.

Whether you want a GUI to exit after a game finishes is a design choice. In general you would not want that. (But in match mode started with the -mm command-line option XBoard exits after the match finishes, and you can ask for matches of 1 game, so it is not unheard of.)

Note that UCI is also used for Chess variants (Chess960, Chinese Chess). Like Chess these variants do not support turn passing, however. But it is not inconceivable someone would want to use UCI for a varinant that does (such as Chu Shogi). In that case there would of course be no reason for the GUI to terminate the game when an engine plays a null move. You could still adopt the convention that reporting a score of -INFINITY (mated in 0) together with the null move is a resign. Games that involve turn passing usually do not have special draw-claim rules. (E.g. in Go the game ends naturally when both sides pass their turn.)

There is NO spec which says how the GUI is to handle this case and in fact there IS NO sensible way to handle it - so it's a UCI protocol bug or at least a bug in the specification document.

Actually the specs don't even stipulate how a GUI should handle it when the engine supplies a legal bestmove. The specs very heavily rely on common sense.

If a chess program is asked to search a position with no moves, I consider that a GUI bug - but there is room for debate on this one. At least sending back 0000 in this (and only this) case is sensible because by definition there is "no move" or "null move." But no serious GUI should ask to search a position that has no legal moves. If a GUI does this it's almost a sure sign it has no legal move generator of it's own - it doesn't even know the rules of the game.

The specs do not explicitly forbid this. You could argue it is common sense to let the GUI handle positions that are 'game-end by FIDE rules'.Unlike with XBoard protocol, having a GUI that does not have rule knowledge is not really feasible with UCI. Unless you would be prepared to interpret 'bestmove 0000' accompanied by 'info score +INFINITY' (or whatever UCI says when it delivers checkmate) as a win claim because of an illegal move. Then the GUI could take the previous move back, and present the 'illegal move' message to the user, just like XBoard takes the latest move back when the engine sends it 'Illegal move'. This surey seems much preferable over the engine just hanging up when there is an illegal move in the position-moves command, as many engines do now.

[jdart]

Both UCI and Winboard GUIs need to do legal move checking and end-of-game condition checks because some engines are buggy and will do invalid things.

I personally prefer Winboard as a protocol because it gives more information to the engine. There are many examples. UCI does not provide the opponent rating to the engine, nor whether or not it is a computer. It doesn't send the game result to the engine. It doesn't tell the engine how many moves remain until time control. And so on.

On the other hand UCI has many fewer commands and is generally much simpler to implement, although as this thread shows there are tricky parts to it.

--Jon

[Don]

Beware that it is the opponent that will eventually decide if the ponder move is played or not, not the engine. So the latter cannot decide to 'go for a win'. But I agree that if your opponent can salvage the draw with his next move, it makes little sense to ponder on what you are going to do after he plays that. Your time would be much better spent by pondering on what you would reply to one of his other moves, which do not lead to an immediate draw. (When the opponent 'goes for the win'.)

But I think this is a moot point: if you have an engine that is smart enough to understand this, it should of course have passed that other move in the 'bestmove-ponder' command it sent to the GUI before. Because most GUIs would dutifully feed the ponder move back to the engine. So the engine gets no new information on which it can decide what to ponder on. It is always better (although not required by the specs) that the GUI knows which move the engine is pondering on. This helps it to present useful output to the user. So it only makes sense for an engine to use the lattitude the protocol specs grant it if it did not know in advance it would want to switch ponder move (e.g. when during the ponder process there occurs a disastrous fail high at some depth, making it likely you are pondering a blunder).

So unlike what some claim here, receiving a ponder move that leads to game end (mate, insufficient material etc.), it is really a bug of the engine, telling the GUI that it would like to ponder on such a move. If it cannot handle the situation after this move, it really should not have sent that ponder move to the GUI. The specs don't oblige the engine to send a ponder move. Only if the GUI would be sending other moves than the engine suggested for pondering on, it can be hold accountable for any mishaps.

O.k., I almost fully agree (see some lines below for the only exception ), and I was wrong when writing

or wait for the "stop" and then send back "bestmove" with the ponder move

since that is not possible (the ponder move is the opponent's move). What I had in mind was the "try to win" idea by preparing for the case that the opponent does not play the drawing move, but the words I chose did not match that unfortunately. Preparing for a case that can only occur if the GUI misbehaves by not terminating the game on rep draw would indeed be a waste of time.

As you state, the situation we have here will arise if the engine itself has sent "bestmove xxxx ponder yyyy" as its last "bestmove" command where "yyyy" leads to a rep draw. This "yyyy" comes back to us as the last move in the "position ... moves ..." sequence prior to "go ponder". This is the only point where I slightly disagree to your statement: I do not consider this behaviour, i.e. announcing to ponder on the reply "yyyy" that draws, a bug of the engine, I think it is valid if the engine states that it expects "yyyy" as the best reply of the opponent to the move "xxxx", i.e. it does not see a way to win and plays "xxxx" while knowing that the opponent can enforce the draw with "yyyy". Announcing a different expected reply would be too smart in my view, and it might even annoy the user who might believe that the engine does not see that the opponent can enforce the draw.

Apart from this I stick to my proposal to start a ponder search on a different move that does enforce a draw (the question "which one" would now be valid, of course, but that is a different task.) Now there are two possible cases:

a) The GUI sends "ponderhit". This means the opponent has made the ponder move which leads to a rep draw but the GUI does not terminate the game, unlike what everyone would expect. Since UCI does not offer a choice for an engine to claim a draw this is a very bad situation for the engine: the combination of a protocol bug and a GUI bug. The engine can't solve it, and whatever it does is "wrong" in some way. I would simply ignore the repetition draw in this case and continue the normal search after the ponder move since everything else, like "bestmove 0000", would be a protocol violation and might also lose the game. The position that has occurred for the third time is not immediately a draw according to the FIDE rules, only if the draw is claimed. Under UCI the GUI must take this part, and the engine has no way to repair the misbehaviour of a GUI. But I think this is not the original case here.

b) The GUI sends a different move (this the original case of Jon). This is simple in my view since there is no rep draw involved (assuming that only the ponder move leads to a rep draw). Two subcases here:

b1) The opponent's move is actually the move we decided to ponder on, then we behave as if we received "ponderhit".
b2) The opponent's move is another move, then we switch to a normal search after that move.

Sven

What I'm arguing for is that the spec be made clear - I don't care if it's somewhat broken or sloppy as long as these cases are spelled out in the spec.

My personal preference is that an engine is allowed to be as stupid as possible and the GUI does all the mundane work. On the big unified rating list there are 4,725 engines listed (some in same family of course) so any additional work the GUI can take off the hands of the engine author lowers the barrier to writing a chess engine.

So simply as a matter of preference I would like to see that Komodo is NEVER sent a position to search that has no legal moves and I don't want to deal with extra tedious protocol to send resignation offers, draw offers, etc ...

In fact if the engine sends a ponder move such as a checkmate or stalemate I don't want to have to check that either! Yes, I am lazy but I think this is how it should be. If the engine sends a checkmate as a ponder move the GUI should check this and not ask it to ponder on that move - so that I don't have to do this.

I stress that this is a personal preference based on my own philosophy of how a protocol should be designed. But more importantly all these corner cases should be specified by the protocol. To address something HG said, the GUI should never send a list of moves that contain illegal positions either. The program should NOT handle that gracefully, otherwise you may never know there is a bug in the GUI.

[Don]

1. UCI GUI sends a move list whose terminal position is a repetition draw (last move is the 3rd repetition).

GUI is broken !


GUI should not send, I fully agree with Lucas here.

I third that motion.

Why? The engine had previously sent "bestmove xxxx ponder yyyy" expecting "yyyy" to be the best reply, even if "yyyy" enforces a rep draw, and the GUI which does not know yet whether the opponent will actually play "yyyy" or a different move advices the engine to start its ponder search with "go ponder", leaving open what the engine wants to do in that situation (ponder on "yyyy", on a different move, or not to ponder at all). The only case where I would see a broken GUI would be if the opponent would play "yyyy" and the GUI would send "ponderhit" to the engine. But in the given case the opponent plays a different move and the GUI correctly sends "stop" to stop the engine's ponder search, whatever the engine has done during that time.

My suggestion for the engine was to ponder on a different move than "yyyy" and react to "stop" with a normal "bestmove".

Sven

I actually advocate that the GUI NOT send one of these moves, even if the engine sent it as a ponder move. But of course that is not in the UCI spec.

[Sven]

Announcing a different expected reply would be too smart in my view, and it might even annoy the user who might believe that the engine does not see that the opponent can enforce the draw.

I think you confuse things here: 'bestmove xxxx ponder yyyy' does in no way imply yyyy is the expected (=best) reply. It means yyyy is the move you want to ponder on. Nothing more, nothing less. The expected reply should follow from the info-pv that the engine send. If that would be by definition the same as the ponder move, there would not have been any need whatsoever to send the 'ponder' addition to 'bestmove'. The very design of UCI makes a provision for these moves to be different.

Apart from this I stick to my proposal to start a ponder search on a different move that does enforce a draw (the question "which one" would now be valid, of course, but that is a different task.) Now there are two possible cases:

a) The GUI sends "ponderhit". This means the opponent has made the ponder move which leads to a rep draw but the GUI does not terminate the game, unlike what everyone would expect. Since UCI does not offer a choice for an engine to claim a draw this is a very bad situation for the engine: the combination of a protocol bug and a GUI bug.

Of course your assertion that the engine would not be able to claim is kind of shaky. It could send bestmove 0000 with a 0 cp score... Do the UCI specs say anywhere that an engine should not be able to claim? Seems to me this 'protocol bug' is purely imagined.

Do the UCI specs say anywhere that an engine should not be able to swim through the pacific ocean?

The UCI specs do not provide any means for an engine to claim a draw. That is a fact. If you disagree then show me the rule that specifies the handling of a draw claim.

A position that is a rep draw is not a terminal position where no legal moves exist, that is another fact. So at least "bestmove 0000" must not be sent if its intention would be to state that the engine has no legal moves.

The UCI specs, under "move format", specify indeed that an engine can send a "null move" as "0000", but the specs do not mention what that means. So the GUI would be free to take your "bestmove 0000" as a statement that you refuse to move. Since in the given case of Jon we are talking about a ponder search that is stopped after the opponent played a different move, and the engine did not receive "ponderhit" but "stop", we might assume that our "bestmove" reply will be ignored anyway but we cannot be sure since the semantics of "bestmove 0000" is not fully specified, therefore I would refrain from using it. The logical implication is that the engine should do *some* ponder search at least after "go ponder" so that it has *something* to reply in case of a "stop". Pondering on a different move than the one enforcing the rep draw would be a better use of resources as you already confirmed.

The engine can't solve it, and whatever it does is "wrong" in some way. I would simply ignore the repetition draw in this case and continue the normal search after the ponder move since everything else, like "bestmove 0000", would be a protocol violation

Uh? Can you quote the exact rule in the protocol specs that would be violated, then?

No. So let's call it not a "protocol violation" but a "dangerous behaviour" of the engine, as I described above

and might also lose the game.

Indeed. False claims tend to do that.

I did not mean that "bestmove 0000" is a form of a draw claim. There is no draw claim in UCI.

Sven

[Sven]

Both UCI and Winboard GUIs need to do legal move checking and end-of-game condition checks because some engines are buggy and will do invalid things.

But if the move sequence in the "position command" ends with a ponder move and the next command is "go ponder" then the game is not finished, it would only be finished if the opponent would actually play the move that enforces the draw. That is not the case in your scenario! And the GUI sends exactly the ponder move that your engine provided with its previous "bestmove xxxx ponder yyyy" command. I would understand your trouble if the GUI would send "ponderhit", meaning that it does not terminate the game despite the rep draw, but what it does in your scenario is to abort your ponder search (whatever it did) with a "stop" before telling you the different move of the opponent. And there your engine should be able to have *some* move to reply with as "bestmove", knowing that "stop" only aborts the ponder search.

I personally prefer Winboard as a protocol because it gives more information to the engine. There are many examples. UCI does not provide the opponent rating to the engine, nor whether or not it is a computer. It doesn't send the game result to the engine. It doesn't tell the engine how many moves remain until time control. And so on.

My full support here

Sven

[Sven]

So simply as a matter of preference I would like to see that Komodo is NEVER sent a position to search that has no legal moves

That is certainly o.k. but in the given case the GUI does not do so, it sends the ponder move that your engine provided with its last "bestmove xxxx ponder yyyy" line, and even if "yyyy" leads to a rep draw this is not a situation with no legal moves. When instructed to do a ponder search with "go ponder" you are free to decide on which move to ponder, and if your own proposal "yyyy" that you had previously sent enforces a draw you might prepare for the case that the opponent goes for a win with a different move, and use your resources that way.

Sven