CECP "time" and "otim"

[mvanthoor]

[Edit]@mvanthoor - Our messages crossed, but I think we are saying exactly the same thing. The understanding is really an issue of the specs, I think. If it would have been explaine as

Before every command that makes the engine think the GUI will send a 'time' command to say how many centi-seconds the thinking player has left on his clock, and an 'otim' command to say how much the opponent has.

I don't think there would have been any difficulty in that case.

Indeed. It does feel a bit strange though.

time 2000
otim 1000

is similar to UCI's

go wtime 20_000 btime 10_000

but they are only the same if the next move is made by the engine, for white. If you swap the colors when playing in UCI mode and you want the engine to still have the time it had before, the GUI has to swap them.

It feels strange assigning a time to the engine, and a time to the engine's opponent; I'm much more used to thinking about time in "white" and "black" terms, and I think that this is what makes this stuff confusing. Most people (including me) would not expect the engine to keep its own time when colors are switched.

[mvanthoor]

Did you also implement the "?" command? Since you have implemented UCI, the engine already must have a way to deal with input during search for isready and stop, so the same infrastructure could be used for CECP's "?".

Almost forgot about that command.

And yes, "?" can be built on top of the existing infrastructure. It has three separate threads: the search thread, communication thread, and the engine thread itself, which controls everything. The incoming commands for UCI and XBoard are transformed into "UciReports" and "XBoardReports", which are then sent to the engine thread.

The engine does what it has to do (using its internal structs and other stuff) to comply with the command. For example:

XBoard::SetBoard(fen) and Uci::Position(fen) both cause the engine to use the same function (read_fen()) to set up its board. (Actually, "position startpos" is also transformed into "position fen <string>" before its sent to the engine thread.) If a response is required, the engine sends a "CommControl" response. "CommControl::SearchSummary" sends the search stuff to the comm module, which will then output it in either UCI or XBoard format, depending on which of the modules is running. (Maybe I should rename "CommControl" to "CommResponse"; it has turned out to be more accurate of a name.)

In this engine, UCI and XBoard are just two different ways of using the engine's infrastructure. There are no UCI or XBoard commands or responses interwoven anywhere in the code.

Therefore I could also implement the imaginary eXtreme Chess Program Thing (XCPT) in a few steps:

- Add a Comm module for that protocol to send "XCPTReport" to the engine thread.
- Add a function in the engine thread to catch those reports and call the engine's functionality to execute them. (If the engine can't actually do what the protocol demands of it, the engine has to be extended, obviously.)
- Add CommControl responses, if the existing ones don't cover everything already.

It was some work to keep it all separated, and I didn't succeed 100%, because UCI and XBoard are not 100% interchangeable. There are some things XBoard does that UCI doesn't; Ping/Pong, for example. Therefore there's a CommControl::Pong, which will never be used if the UCI module is active. Another example is "CommControl::AnalyzeStat01", which is sent upon receiving the "Dot" command, which is also never used by UCI, as it sends its stats in an ongoing "info" stream.

[mvanthoor]

I don't know, I try to support both protocols, even though I recently fixed another bug in my SAN implementation; SAN is probably fine for PV, but not so great for move commands (it's optional in CECP as far as I remember)

The specs actually say that using SAN is not recommended...

http://hgm.nubati.net/CECP.html
"Requests the GUI to send moves in SAN format rather than coordinate notation. Use of SAN is not recommended."

I like CECP because it's less verbose, but comes with some intricacies related to implementation details. Also pondering is entirely up to the engine - it's more flexible but also slightly harder to implement.

It seems to depend on what you want... UCI has longer commands, and a single command can pack more punch. That makes the implementation longer, but you have to implement less commands. With regard to code needed, they seem to be about equal. (In Rust, at least.)

I do concede that if you're using C, and parsing everything yourself character by character, CECP can be much easier because the commands are shorter, don't have so many parts, and if they have parts, things are sent in a predefined order. In UCI, you mostly see things such as:

go wtime 5000 btime 5000 winc 1000 binc 1000 movestogo 20

That doesn't have to be the case though; this stuff can come in any order, such as:

go movestogo 20 btime 5000 winc 1000 binc 1000 wtime 5000

That's hard, if you need to parse this by hand. In Rust, it's easily solved by just exploding the entire string into a vector and run through the elements, ignoring the ones you don't need (such as "go" in this case), and then parse the rest as token, value, token, value... and return an error as soon as something is amiss.

Handling and parsing strings by hand is something I don't miss since I switched from C. That shit is too easy to get wrong. (Been there, done that...BSOD'd the computer.)

[mvanthoor]

I don't know if this question was directed at me or Marcel, but I treat "?" as the UCI stop command, except that I ignore it when the engine is currently analysing or pondering.

I intend to do the same thing. "?" is basically UCI's stop (stopping the search, sending a best move).

XBoard's "exit" for exiting the analyze mode does not exist in UCI. In UCI, "Stop" is used there as well, and thus the engine sends "bestmove"... which is then promptly ignored by the GUI, if it started the search with "go infinite".

[mar]

yes, well with UCI being verbose I wasn't thinking about sending time, but rather sending the game state, which is sent before each search.
like position startpos moves ..... and a long noodle of say 80+ (half)moves, of course GUIs could use FEN compression from the last irreversible move, so position fen .... moves ..., assuming the engine supports position fen of course - might be a nice option for GUIs to have, actually

in general, I prefer CECP mode on Linux (where I use xboard) and when analysing a position from the console => the xboard output is so much nicer than the clutter sent by UCI and you could also send smileys on fail low/fail high

this doesn't mean I don't like UCI, quite the contrary - my experience is that (assuming you're writing an engine that plays regular chess) UCI is easier to start with (IMO - YMMV)
CECP is more flexible, but also more difficult to get right. as for variants, I think xboard/winboard + CECP is the best bet, even though I believe there're unofficial modified UCI versions that support variants other than 960

[mvanthoor]

yes, well with UCI being verbose I wasn't thinking about sending time, but rather sending the game state, which is sent before each search.
like position startpos moves ..... and a long noodle of say 80+ (half)moves, of course GUIs could use FEN compression from the last irreversible move, so position fen .... moves ..., assuming the engine supports position fen of course - might be a nice option for GUIs to have, actually

I've actually wondered WHY GUI's don't do that. I have actually only implemented handling "position fen <string>"; when I encounter "position startpos", I just send "position fen <startposition_here>" to the engine thread. It's trivial to support both "position startpos" and "position fen".

With regard to verbosity... I don't really mind. everything after "moves" is just packed up into a vector, flung to the engine thread, and there the engine starts executing moves one by one from the position. If it encounters an illegal move (which has only happened in testing, with me putting in an illegal move), it stops and prints an "info string".

in general, I prefer CECP mode on Linux (where I use xboard) and when analysing a position from the console => the xboard output is so much nicer than the clutter sent by UCI and you could also send smileys on fail low/fail high

To mitigate this, I've implemented a --quiet option in my engine:

--quiet 0 => full search output including info updates for time, speed, curr_move, etc...
--quiet 1 => only send info when a depth was finished.
--quiet 2 => send nothing at all. I don't use this myself. I basically 'upgraded" --quiet to take a parameter, so I could use the state caused by "--quiet 2" for XBoards "nopost".

this doesn't mean I don't like UCI, quite the contrary - my experience is that (assuming you're writing an engine that plays regular chess) UCI is easier to start with (IMO - YMMV)
CECP is more flexible, but also more difficult to get right. as for variants, I think xboard/winboard + CECP is the best bet, even though I believe there're unofficial modified UCI versions that support variants other than 960

There's an (unofficial) variant "USI" for shogi, and "UCCI" for XiangQi (Universal Chinese Chess Interface).

Personally, I don't like massive protocols and engines that try to cater to everything under the sun. It makes things confusing.

If I'd ever want to write a Shogi engine, I'd use USI and a proper Shogi GUI. Same for XiangQi.

I'll probably never use Winboard, as the program is not my kind of software. I've also been a UCI-engine user since forever (under the Fritz GUI, and Fritz itself before I actually started to use other engines), so I probably won't even run my own engine in XBoard mode.

I do respect Winboard/XBoard however, and the CECP protocol, and I can understand why people like the program and the protocol; and that's why I implement CECP to make the engine run as a native XBoard engine. That way, people who absolutely loath UCI and don't trust or want to use adapters, can still use the engine in XBoard (or Arena or Banksia for that matter; they both work, at least the current analysis mode).

When I'm done with these last bits in XBoard CECP, the implementations in the engine will be equal. The engine can then do with CECP what it can do in UCI, and I intend to keep them on par... as long as I don't have to go out of my way and suddenly discover that I'd have to rewrite half my engine to support some feature in XBoard. (But I doubt that, to be honest.)

[hgm]

..., assuming
playother typically comes when it's the opponent's turn.

'Playother' is pointless during the opponent's turn. Because the meaning then is that you should play for yourself, which you are already doing (or you would not have had an opponent). There is no guarantee in the specs that you will never get pointless commands, though.

[Harald]

May be this post and picture can help a bit:
Winboard state machine diagram
http://www.talkchess.com/forum3/viewtop ... ne#p796223

But I think this is even a little bit too complicated and the GUI + engine will stay even more
in the force mode and less in the waiting, thinking, pondering, ponder_complete states.

If you (*) are not entirely confused by this diagram then you can probably tell me
how it can be improved or simplified.

(*) anyone

[Ras]

Winboard state machine diagram

Alternative title: "the best argument for UCI".

[mvanthoor]

Winboard state machine diagram

Alternative title: "the best argument for UCI".

LOL. Indeed.

For some reason, I feel that having the states is only going to be needed to discard or accept certain commands, in case the GUI does something stupid. At this point I don't even HAVE states, but the engine does work as expected (at least for the parts I've implemented).

For example "exit" should exit the analyze mode... but in my case, it just stops the search. If the search is not running, the command does nothing. Same for the "." command. It requests search statistics. The search sends those statistics to the engine thread just as it does when in UCI mode. The difference is that in UCI mode, the engine thread immediately sends those statistics out through the Comm module, as an "info" string, but in XBoard mode, it buffers/updates the stats until the "." command comes in. So, if the search is not running, the search stats are empty, so nothing is sent when the "." is sent.

The protocol says that "." is only sent in analyze mode. I wonder why; it's perfectly valid to request ongoing search stats when playing a normal game. Actually... if I don't specifically block the reply to the "." command when playing, the engine WOULD send out valid search stats. Internally, there's no difference between "analyzing" and "searching when playing a game".

Fortunately, because UCI and XBoard are not interwoven in my engine, if I do need or want sates, I can just define an enum and create a "state" variable in my "XBoardSpecifics" part of the engine settings, and have the Comm module ask for it if necessary. Adding states (or whatever else) to the engine for XBoard won't have zero effect on the UCI-functionality.

PS: I don't want these threads to degenerate into a CECP vs. UCI fight. Both protocols have their merits, and their own philosophies. Pick whatever suits your purpose. Or both, as I do, because I'm one of those guys. CPW says most newer engines don't support XBoard, so I'll support it because I can. I specifically wrote the engine in such a way that I can support any protocol I want, so I see no reason not to (except for not wanting to do the extra work).

On state diagrams: I wrote software to control machinery and micro-controllers for a LONG time. State diagrams are the core of what I've been programming for many years. Most of my programs start out as a state diagram. I can see why many people have problems with it, but to me, the XBoard state diagram is clear enough.