CECP "time" and "otim"

[Ras]

On state diagrams: I wrote software to control machinery and micro-controllers for a LONG time.

Yeah me too - actually, that's the whole point of my project, i.e. the embedded microcontroller version, and the PC/UCI version is just a convenient mockup.

[hgm]

For some reason, I feel that having the states is only going to be needed to discard or accept certain commands, in case the GUI does something stupid. At this point I don't even HAVE states, but the engine does work as expected (at least for the parts I've implemented).

That is correct. As I also mention at the link, a full state diagram is not a useful way to look at CECP. Or in fact at any protocol: if you would make a similar one for UCI it would also look horrible. You would for instance need to run through many states during the reception of a 'go' command, to keep track of whether you have already received wtime / btime / infinite / ponder or which of those is still allowed to come considering what you have already seen, etc. In the practical use of CECP the commands only come in certain groups very similar to how they come in UCI, e.g. "time T1 + otim T2 + go" vs "go wtime T1 btime T2". The only difference then is that in CECP they are separated by linefeeds, and in UCI by spaces.

The real difference between CECP and UCI is that CECP has a state variable that detemines which color the engine is playing for, which it uses to automatically launch searches for thinking, pondering or analyzing. And the state diagram for that variable is hardly interesting, as the commands that alter its value can pretty much come in any of the states, and just define the new value.

[Ras]

Or in fact at any protocol:

Protocol state diagrams are both useful and common. In fact, when the protocol diagram starts to look too convoluted for the task at hand, that's already a warning sign in the early design phase.

if you would make a similar one for UCI it would also look horrible. You would for instance need to run through many states during the reception of a 'go' command

Disagree. This has no place in a protocol state diagram.

The real difference between CECP and UCI is that CECP has a state variable that detemines which color the engine is playing for

The key difference is that CECP transmits the moves incrementally because it was designed for manual human input on the terminal.

[hgm]

Disagree. This has no place in a protocol state diagram.

Then you are making a big deal of whether the keyword separator must be a linefeed or can also be a space. Which makes no sense. Any state diagram can be highly simplified by redefining entire groups of command as a single 'super-command', and eliminating all the states the contributing commands lead you through from the state diagram.

The real difference between CECP and UCI is that CECP has a state variable that detemines which color the engine is playing for

The key difference is that CECP transmits the moves incrementally because it was designed for manual human input on the terminal.
[/quote]

[Ras]

Then you are making a big deal of whether the keyword separator must be a linefeed or can also be a space.

It's not just that CECP has more commands because UCI sums them up in one line. What matters is that UCI has one uniform way of transmitting a game, a position, the game move list, taking back moves, redoing moves. It's not that the individual commands are more complex, it's that you use the same way for multiple things.

The price you pay for that uniformity is that this way is not feasible for direct terminal play unless your terminal buffers input and supports line repetition with arrow-up - which, ironically, only Windows' cmd shell does, but not Linux terminals because they leave that to the application.

You can use CECP quite UCI-like with setboard/force.

However, you can also transmit single moves with no setboard/force. You can take back a ply with undo, but taking back two plies is not two times undo - it's remove instead. Redoing moves is force plus moves. Or, different example, aborting an ongoing search is "?", but aborting analysis is "exit".

[mvanthoor]

Hm... with regard to undoing/removing moves, I haven't even thought of that. I still need to implement this.

Shouldn't be too hard though; I can just back up one or two plies in my history with either one or two unmake() calls. The engine won't automatically start to think in my case; all engine routines basically just sit there until the engine thread sets them going. So, I can just do whatever I need to if a command comes in (remove one ply or two), and then decide if the engine should think or do something else or whatever.

Every operation in the engine is separate from any other operation. If I send SearchControl::Start, the search will just run and keep repeating its progress. If I send SearchControl::Stop, it stops and sends the best move to the engine thread. The engine thread can then decide to send the best move (in case of UCI, or XBoard "?") or not (in case of xboard "exit").

This way of doing things allows me to handle a command by compounding different actions within the engine, without needing to worry that one action will influence another. This is also the reason why UCI and XBoard, in this engine, are just different ways to use the engine's features. Internally, many UCI and XBoard commands are handled by the same routines.

[hgm]

Then you are making a big deal of whether the keyword separator must be a linefeed or can also be a space.

It's not just that CECP has more commands because UCI sums them up in one line. What matters is that UCI has one uniform way of transmitting a game, a position, the game move list, taking back moves, redoing moves. It's not that the individual commands are more complex, it's that you use the same way for multiple things.

The price you pay for that uniformity is that this way is not feasible for direct terminal play unless your terminal buffers input and supports line repetition with arrow-up - which, ironically, only Windows' cmd shell does, but not Linux terminals because they leave that to the application.

You can use CECP quite UCI-like with setboard/force.

However, you can also transmit single moves with no setboard/force. You can take back a ply with undo, but taking back two plies is not two times undo - it's remove instead. Redoing moves is force plus moves. Or, different example, aborting an ongoing search is "?", but aborting analysis is "exit".

I always wondered why they introduced the 'exit' command; it does exactly the same as 'force'. The analyze-mode commands were added by the first implementation of analysis, when analysis was a new thing (was this in Crafty?), and it seems not a whole lot of thought went into designing the protocol. This doesn't add any extra states or transitions to the state diagram, though. It is just that the transition to force mode is sometimes labelled 'exit', and in most other cases 'force'. An annoyance to GUI developers, but engines doesn't even have to be aware of it, and can treat exit and force as synonyms.

That UCI is amnesic w.r.t. game state doesn't affect the protocol state diagram much. In fact it complicates the latter, because you now need an extra state 'game-state undefined', to which you transition after every search. One can imagine a protocol 'UCI+', which is identical to UCI except that it does define the game state after a search, as the one after the 'bestmove' that this search returned has been performed. This would be upward compatible with orthodox UCI, but you could then skip the position/moves command in normal play. You could even define an 'extended game state', which includes the latest ponder move (as returned by bestmove-ponder), and stipulate that go-ponder acts on the current game state with the ponder move applied, and that a go-ponder aborted by 'stop' would leave the extended game state as it was before the command. Then you would also not need any position-moves command in ponder games.

It is definitely true that the UCI commands are more 'hierachically organized' than in CECP. But I consider that mostly a cosmetic difference. I don't really care whether the command to set the engine analyzing is 'go infinite', 'go analyze' or just 'analyze', or that you have to send 'setoption name ponder value true'/'setoption name ponder value false' instead of 'hard'/' easy'.

A funny thing is that I always implement undo/remove in my CECP engines in the 'UCI style': the engines remember the game history and start FEN, and when it receives undo it just sets up the board position anew, and plays the history except for the last move. That is usually easier than storing 'augmented game history' that also contains the information needed to take back the moves (i.e. what was captured and where).

[Ras]

but engines doesn't even have to be aware of it, and can treat exit and force as synonyms.

Designing redundant commands for the same thing causes more transition paths in the state diagram. Or the two way feature negotiation, which I don't see much of a use case for. Sure, the engine can just ignore any accepted/rejected, which is a common implementation, but then it's not only redundant, but even useless commands. With UCI, the engine announces its features one-way, and if the GUI can't deal with some of them, the GUI just doesn't use them.

In fact it complicates the latter, because you now need an extra state 'game-state undefined', to which you transition after every search.

You stay in the same state as before receiving "go". In fact, you can receive the "go" again now, with only hashtables etc. being in a different state. No extra state required. The protocol spec only requires this indirectly in that "go" is supposed to calculate on the position given with the "position" command, which a position after executing the resulting "bestmove" wouldn't be, and neither an invalidated one.

but you could then skip the position/moves command in normal play.

That's exactly the kind of design that would make it less uniform which equals more complicated. It doesn't matter how many moves are transmitted before every move turn - it's not like one pays per byte on stdin. Sure, you would gain the option to play in the terminal, but that's not really a wide-spread use case. Even if it were, an ncurses style pseudo-GUI would do that a lot better, and then with the same user interface for both CECP and UCI engines.

That is usually easier than storing 'augmented game history' that also contains the information needed to take back the moves (i.e. what was captured and where).

Depending on the engine design, you may have that available anyway because the usual make/unmake move in search also need that. That is, if you use the same game stack for game history as for search.

[hgm]

Designing redundant commands for the same thing causes more transition paths in the state diagram.

Not in this case, because the transition paths start from a different state to begin with (analze mode or other), and would have to be there anyway. They just get different labels.

You can argue that it is bad that analyze mode, thinking and pondering are different states in the first place, and should all have been the same state, 'searching'. But that is not really different in UCI: there you also have timed searches (which have an extra spontaneous transition to the 'undefined' state) and untimed searches, and "go infinite" also behaves differently from "go ponder", e.g. on reception of 'ponderhit'.

Or the two way feature negotiation, which I don't see much of a use case for. Sure, the engine can just ignore any accepted/rejected, which is a common implementation, but then it's not only redundant, but even useless commands. With UCI, the engine announces its features one-way, and if the GUI can't deal with some of them, the GUI just doesn't use them.

Well, the engine could use a feature to probe whether the GUI would accept some extended form of engine output. E.g. when feature debug=1 is accepted, it can safely print anything by prefixing the line with #, without running the risk for "undefined behavior".

You stay in the same state as before receiving "go". In fact, you can receive the "go" again now, with only hashtables etc. being in a different state. No extra state required. The protocol spec only requires this indirectly in that "go" is supposed to calculate on the position given with the "position" command, which a position after executing the resulting "bestmove" wouldn't be, and neither an invalidated one.

I don't think so. It is not legal to send another 'go' without a new position-moves. Perhaps many engines implement it that way, but that is just one particular form of undefined behavior they choose to exhibit.

That's exactly the kind of design that would make it less uniform which equals more complicated. It doesn't matter how many moves are transmitted before every move turn - it's not like one pays per byte on stdin. Sure, you would gain the option to play in the terminal, but that's not really a wide-spread use case. Even if it were, an ncurses style pseudo-GUI would do that a lot better, and then with the same user interface for both CECP and UCI engines.

It doesn't complicate anything. Having Search() execute the best move is just as arbitrary as letting it restore the original position (which you claim is demanded by the specs...). In several of my engines Search() automatically returns with the best move made.

Depending on the engine design, you may have that available anyway because the usual make/unmake move in search also need that. That is, if you use the same game stack for game history as for search.

That is true, but it is a bit unusual design. Usually I store the 'undo info' in a local variable, and then this would require you to never completely unwind the recursion. I do something like that in Shokidoki for implementing speculative pondering: I always ponder the position, and above a certain depth the ply=1 node will no longer return, but deepen indefinitely until the search gets aborted. So it starts a shallow search from the root (which presumably picks up the best move of a deep search from the hash), performs it speculatively, and then ponders on the resulting position. On a ponder hit the lowest two ply return before doing UnMake(), on a miss they return normally.

[Ras]

But that is not really different in UCI: there you also have timed searches (which have an extra spontaneous transition to the 'undefined' state) and untimed searches

Which is the same state because you can just implement "infinite" as "many years". That's how I do it. Ponder is indeed different, also in CECP, because it's a different kind of activity with the speculative move.

E.g. when feature debug=1 is accepted, it can safely print anything by prefixing the line with #, without running the risk for "undefined behavior".

That's the wrong way around - debug should be enabled from the GUI, which only GUIs will do that support it in the first place. If the engine doesn't support it, the command will just be discarded.

It is not legal to send another 'go' without a new position-moves.

The protocol spec does not say this.

Perhaps many engines implement it that way, but that is just one particular form of undefined behavior they choose to exhibit.

It's not undefined, and even if it were, it wouldn't actually require an additional state.

In several of my engines Search() automatically returns with the best move made.

Of course, that's the best choice for a CECP engine. The player can enter his answer move right away in the terminal, but then the position has to be the one with the previous engine move already executed.

That is true, but it is a bit unusual design.

It makes scanning for repetitions easier because the scan happens within the same array even for moves before the root position, i.e. from the game history.