Xboard, CECP, how to handle long inits

[smatovic]

My Zeta engine needs up to some seconds to init the gpu once, and the gpu memory
for each new game...so what is the best way to signal xboard that clock should
not run?

--
Srdja

[hgm]

XBoard sends 'ping' after each 'new', and in tournament mode actually waits with starting the clock until both engines have submitted the corresponding 'pong's. Not sure if it also does that in old-fashioned two-player matches, though. (The tournament mode has another, much more complex NextGame function, because this could involve changing engines.) This might depend on the version. But there is nothing that would stop you from playing a tournament with only two participants instead.

[Daniel Shawul]

What I do is to send "feature done=0" before anything else, and send "feature done=1" once you are done with initialization.
According to the protocol, this gives you upto 1 hour to do whatever you need.

Sometime ago i have had serious issues with cutechess-cli who doesn't seem to implement this correctly.
Because my engine was taking upto 40 seconds loading neural networks, and cutechess-cli puts a hard limit of
30 seconds (also not configurable if i am not mistaken) on initializing --thereby overrding the xboard protocol specification.
The reason for the slow loading time was TensoRT optimizations of neural network on the fly.
After I started saving the optmized plan file to disk and loading that, the problem went away, h

Daniel

[hgm]

Note, however, that this is done only at engine startup. Not for every game, if you let the same engine process play multiple games. So if you rely on this method, you have to let the engine print feature reuse=0 . But then it will certainly also work in old-fashioned two-player matches.

[phhnguyen]

XBoard sends 'ping' after each 'new', and in tournament mode actually waits with starting the clock until both engines have submitted the corresponding 'pong's. Not sure if it also does that in old-fashioned two-player matches, though. (The tournament mode has another, much more complex NextGame function, because this could involve changing engines.) This might depend on the version. But there is nothing that would stop you from playing a tournament with only two participants instead.

I am a bit confused here. What do you expect (engine’s responding) between the protocol ‘xboard’ from WB and the first ‘pong’ from engine? Can the engine keep silent all the time? Does WB have a time threshold or does it wait for the first ‘pong’ forever?

[JVMerlino]

What I do is to send "feature done=0" before anything else, and send "feature done=1" once you are done with initialization.
According to the protocol, this gives you upto 1 hour to do whatever you need.

This is what I do as well in Myrddin. For some reason, sometimes initializing the Gaviota TBs takes several seconds, and doing the above solves the problem with the few GUIs I have used.

jm

[hgm]

I am a bit confused here. What do you expect (engine’s responding) between the protocol ‘xboard’ from WB and the first ‘pong’ from engine? Can the engine keep silent all the time? Does WB have a time threshold or does it wait for the first ‘pong’ forever?

After 'xboard' the GUI sends 'protover 2' (or it would be an obsolete GUI supporting only version 1 of the protocol), and the engine should respond to that with 'feature' commands to specify which of the version-2 extensions it supports). If the engine says nothing during a timeout period (in WinBoard this is 10 sec, I think), the GUI should assume that it is an engine that only supports v1 protocol, and none of the extensions. Engines that want to be treated as v2, but which conceivaby take a longer time to initialize than the timeout period should send 'feature done=0' immediately, and then start initializing. This extends the timeout period to 1 hour. They should then always send 'feature done=1' after they are done initializing (and sending features) to prevent that WinBoard would actually wait a full hour, to see if more 'feature' commands are forthcoming. Even if they don't want to use any of the v2 protocol extensions they should at least send 'feature done=1', to cut short the default 10 sec timeout.

After the features the engine can stay silent until it receives 'ping'. (And it will only receive that if it let the GUI know it supports it by sending 'feature ping=1' , as 'ping' is a v2 extension.) Then it has to reply with a corresponding 'pong'.

Originally WinBoard never waited for a 'pong'; the 'ping's were only used to unambiguously assign engine responses to commands sent earlier. In particular, if an engine search gets interrupted by a 'force' command because the GUI wants to terminate the game (e.g. because the opponent suddenly resigns), there is a race condition between the search ending spontaneously and sending a move, or the search aborting in response to the 'force' without a move. If a new game starts after that (with 'new', 'go') you would not know if the next move that came in was the result of the 'go' or still belonged to the previous game. By inserting 'ping' between the 'force' and 'new' you would know that: moves received before the 'pong' would be from the previous game, moves received after it would be from the new game.

Because not all engines terminate their search instantly in response to 'force', but first finish thinking about the move, just sending 'force', ['ping',] 'new', ['ping',] 'go' and starting their clock can corrupt the games from an automatically played sequence by wrong timing, I made WinBoard actually wait for a 'ping' sent after 'new' before it starts the clock.

[phhnguyen]

Thanks HGM for very detail and clear reply!

Just a bit out of topic: for UCI, what are commands similar to the pair "feature done='0'/'1'" of WB? In other words, how to make GUI wait longer for engine initialization?

Thanks

[phhnguyen]

Another off topic issue since I am confused in what is correct reactions between GUI and engines. I am trying to support WB protocol for my app (Banksia the chess tournament manager) and prefer to work with moves in coordinate format which I think it is standard in WB protocol document. When testing with Crafty v23.6, it sent me some features:

Code: Select all

   feature ping=1 setboard=1 san=1 time=1 draw=1

I accepted all but "san" one:

Code: Select all

   accepted setboard
   rejected san
   accepted time
   accepted draw

However Crafty replied me in a un-formal way:

Code: Select all

   ERROR.  feature san rejected by xboard

and then it gave me the first move in san format:

Code: Select all

   move h3

My questions:
- How to control engines to work with coordinate moves? Do GUIs have to "obey" engines in those cases?
- How to deal with unformal anwsers (how to understand them)?
- How to deal with un-expected-format information (such as SAN format moves when I am waiting for a coordinate one)?

Thanks

[pedrox]

Thanks HGM for very detail and clear reply!

Just a bit out of topic: for UCI, what are commands similar to the pair "feature done='0'/'1'" of WB? In other words, how to make GUI wait longer for engine initialization?

Thanks

Code: Select all

* isready
	this is used to synchronize the engine with the GUI. When the GUI has sent a command or
	multiple commands that can take some time to complete,
	this command can be used to wait for the engine to be ready again or
	to ping the engine to find out if it is still alive.
	E.g. this should be sent after setting the path to the tablebases as this can take some time.
	This command is also required once before the engine is asked to do any search
	to wait for the engine to finish initializing.
	This command must always be answered with "readyok" and can be sent also when the engine is calculating
	in which case the engine should also immediately answer with "readyok" without stopping the search.