UCI to CECP

[Fulvio]

I wrote 6 functions to abstract UCI that are like this:

Code: Select all

handshake:
	send: uci
	parse options
	waitfor: uciok

setOptions:
	stop()
	send: setoption name $name value $value
	...
	send: setoption name $name value $value
	send: isready
	waitfor: readyok

newGame:
	stop()
	send: ucinewgame
	send: isready
	waitfor: readyok
	
go $time:
	stop()
	$is_thinking = true
	send: position startpos moves ...
	send: go $time
	parse info
	waitfor: bestmove
	$is_thinking = false

stop:
	if $is_thinking:
		send: stop
		
quit:
	stop()
	send: quit

And I use them like this:

Code: Select all

Analysis of 1.e4 e5:

handshake
setOptions
newGame
go infinite (startpos moves e2e4)
  wait_for_the_user_to_move_forward
go infinite (startpos moves e2e4 e7e5)
  let_the_user_change_some_options
setOptions
go infinite (startpos moves e2e4 e7e5)
quit

Code: Select all

Play against the user, 5 second per move:

handshake
setOptions
newGame
do
	go movetime 5000 (currpos)
	play_the_move_and_wait_for_the_user_move
while not_endgame_or_user_stop
quit

What is the correct way of implementing those function for CECP engines?

[hgm]

Not all functions have a direct equivalent. In CECP you don't have to command the engine to think on every move; at the start of the game you would command it to play for a certain color, and after that it would think and move fully automatically whenever that color gets on move. And it gets on move by receiving the opponent's move. So during most of a game you just send it (single) moves, usually (but optionally) also sending the remaining time on their clock just before it.

Code: Select all

handshake:
  > xboard
  > protover 2
  < parse features
  < wait for feature done=1
  
set options:
 > memory M
 > cores N
 > option $name=$value
 > ...
 > setoption $name=$value
 > ping N
 < wait for pong N
 
 new game:
   > force
   > new
   > level $time
   > ping N
   < wait for pong N
   if engine must play white:
   > go
   if the engine must play black
   > MOVE
   if just loading moves:
   > force
   > MOVE_1
   > ...
   > MOVE_N
   
 quit:
   > quit

Code: Select all

Analysis of 1.e4 e5:

handshake
set options
> new
> force
> e2e4
> e7e5
> analyze
  wait for user to move forward
> MOVE
  wait for user to retract move
> undo
  let user change some options
> exit
> option $name=$value
> analyze

> exit
> quit

Code: Select all

Play against user

handshake
setoptions

new game
st 5
do
  > time 500
  > MOVE
  wait for the engine to move, and then for the user to move
while not end game

Complication for a GUI developer is that the protocol changed over time, so that not all engines speak the same dialect. Engines that speak a more advanced dialect let this know through 'feature' commands. Of particular importance are:

feature ping=1

An engine that does not say this at startup ('parse features') will not recognize the 'ping' command.

feature usermove=1

All moves sent to the engine must be prefixed by the keyword 'usermove'.

Another problem is that engines do not always terminate a search with printing a move, in particular when the search is aborted. And they also not always immediately respond to an abort. This makes it sometimes difficult to determine whether an incoming move was from an aborted search or from the next one. This is the reason 'ping' was added to the protocol; by following a command with 'ping', the corresponding 'pong' will signal the completion of that command, even when it terminated silently. Note that unlike UCI 'isready', there is no instant response to 'ping' during thinking; it is only processed after the thinking terminates.

[Fulvio]

Note that unlike UCI 'isready', there is no instant response to 'ping' during thinking; it is only processed after the thinking terminates.

Thanks.
It's strange that the meaning of UCI's "isready" is ping and the meaning of CECP's "ping" is isready
Some questions:
- are all the options optional, or "memory" and "cores" must be sent if the engine requested "feature memory=1"?
- "exit" must be sent only once like the UCI's "stop" and only if the engine is analyzing?
- it is necessary to send "exit" before "quit" if the engine is analyzing?
- when analyzing the user can jump to a very different position: it is better to use "setboard FEN"?

I can write these function for CECP:

Code: Select all

setOptions:
	stop()
	send: mandatory_options?
	send: options_set_by_the_user

newAnalysis:
	stop()
	send: new
	send: force
	
analyzePos:
	stop()
	$is_thinking = true
	send: setboard FEN
	send: analyze
	parse info

stop:
	if $is_thinking:
		send: exit
		$is_thinking = false
	send: ping 1
	waitfor: pong 1

quit:
	send: quit

and I can use them in the same way as the UCI's ones:

Code: Select all

handshake
setOptions
newAnalysis
analyzePos (FEN1)
  wait_for_the_user_to_move_to_a_different_position
analyzePos (FEN2)
  let_the_user_change_some_options
setOptions
analyzePos (FEN2)
quit

Is this correct?

[hgm]

- are all the options optional, or "memory" and "cores" must be sent if the engine requested "feature memory=1"?

Strictly speaking the 'cores' command should only be sent if the engine requested "feature smp=1". But fully compliant engines should not get upset if you send them a command they do not understand; they should reply with the "Unknown command" error message. If you don't send these commands, engines would run with their default hash size and number of threads, which they did not announce, and might be very different from what you want. So in practice these commands should be considered mandatory. If they are not supported the engine will likely have some non-compliant way to configure those, like command-line arguments or a config file.

For user-defined options (through 'feature option="$name -$type $value ..." ') you would know the current $value, and there is no need to set them when this is the desired one. Presumably most such options would have the $value that is optimal for the engine as default.

- "exit" must be sent only once like the UCI's "stop" and only if the engine is analyzing?

"exit" in all my engines does exactly the same as "force": switch the engine to a mode where it would not think no matter who is on move. So it seems a redundant command. But the specs describe it as the command needed to stop analysis. So strictly speaking a pedantic engine receiving "exit" while thinking or receeiving "force" while analyzing would be in its rights to refuse those commands with an error message.

- it is necessary to send "exit" before "quit" if the engine is analyzing?

I suppose not. In principle "quit", "force"/"exit", and "?" (= move now) are commands that should be instantly processed during thinking / analyzing. I think WinBoard first stops the thinking before sending "quit", though, and it is always tricky do do what engines have come not to expect. Analyzing would not work if the engines weren't sensitive to moves and "undo" commands, but many engine authors do not care about instant response to commands while an engine is thinking. So there are many engines that ignore all input until after they moved.

- when analyzing the user can jump to a very different position: it is better to use "setboard FEN"?

Well, WinBoard never sends a "setboard" to an analyzing engine, although the protocol specs say it would be allowed. But the user interface is such that the situation never arises. When you enter Edit Position mode it already terminates the analysis. And pasting FENs is implemented as if it was Edit Position followed by a transition to Edit Game, in which the engine is kept in 'force mode'.

Code: Select all

setOptions:
	stop()
	send: mandatory_options?
	send: options_set_by_the_user

newAnalysis:
	stop()
	send: new
	send: force
	
analyzePos:
	stop()
	$is_thinking = true
	send: setboard FEN
	send: analyze
	parse info

stop:
	if $is_thinking:
		send: exit
		$is_thinking = false
	send: ping 1
	waitfor: pong 1

quit:
	send: quit

If 'is_thinking' means 'is_analyzing' this seems OK. By just analyzing positions from a game you would not allow the engine to detect the result of repetitions, though. So it might be better to load the position through a 'new' + all game moves. This could in some engines clear the hash table, though. So if it is important that results from the previous analysis are kept (as in back-propagation of a score along a line), it would be better to just move along the line through the required number of extra moves or 'undo' commands.

Note that 'ping' after 'exit' is not really necessary; exiting from analysis mode will never produce a move, but should also happen instantly. And even if there was a small delay, clocks are not running anyway during analysis. But it never hurts to send extra pings.

Code: Select all

handshake
setOptions
newAnalysis
analyzePos (FEN1)
  wait_for_the_user_to_move_to_a_different_position
analyzePos (FEN2)
  let_the_user_change_some_options
setOptions
analyzePos (FEN2)
quit

[quoute]Is this correct?
[/quote]
I think so.

[jdart]

CECP is quite complex and there are a number of nasty corner cases and possible timing issues.

I read from stdin in a separate thread. That thread places all incoming received commands into a stack. Commands are then processed by reading from the stack, usually in order, but there are exceptions. Periodically in the search, I look at the pending stack. Some commands are processed during the search, some commands like "exit" cause the search to terminate but then must also be processed after the search, and some commands are just deferred until the search ends. You must be especially careful when the game terminates, e.g. not sending a move after game termination.

--Jon

[hgm]

I think the specs say that it is a valid implementation to let any command terminate thinking. So I guess that GUIs are supposed to never send anything to the engine while it is thinking if they do not want to interrupt the game. (That means they should relay draw offers the opponent makes during his own turn only after the engine moves.) It would not be difficult to process simple commands, such as switching on and off pondering or thinking output, as soon as they are received.

In UCI2WB I also use two threads, basically one reading from the GUI, and another from the engine. The one receiving from the GUI queues the commands internally, and the one that reads from the engine reads the queue (possibly waiting for a command to arrive) when nothing can be expected from the engine. (No search in progress.) CECP commands that should terminate a search (force, exit, ?, quit) cause immediate sending of 'stop' to the engine, though, before being put in the queue, so that you can be sure the engine thread will start to look at them. A ponder miss is also treated that way.