Ugly UCI

[hgm]

I agree, but then "done=1"??

You mean that this also could have been handled by sending a ping after the protover? But what if the engine in not supporting ping? That would mean you cannot send the ping immediately with the protover, but would have to wait with it until after receiving a ping=1 feature, which in itself might never come.

The real design flaw here is that in v2 ping is still optional. It would have been much better if v2 would not have been half-hearted about the introduction of some new commands, like ping and setboard, and declare them obligatory.

I guess the choice to not do this was motivated by the wish that many engines would quickly 'formally' convert to v2 by answering the protover command with "feature done=1" and nothing else, which would be a trivial change. Note that the introduction of v2 did hurt all v1 engines, by introducing an annoying waiting period after their startup, for features that were not going to come, until the GUI finally timed out, so there would be a real benefit even from such a formal conversion only. Making other changes mandatory to work as v2 might have discouraged engine authors to convert, although implementation of 'ping' seems sufficiently trivial that I don't really see that. (Setboard is of course another matter.) In that light it would be understandable that "feature done=1" was offered as a method for terminating the waiting for features that would work without ping support.

[hgm]

Does that always help? Is the following design ok?
In Elephant the input is tested in the main loop and every 10000(?) nodes in
the search. The input can change some state variables and it can decide to
stop the search, the game or the program. In that case it throws an
exception that is catched in the main loop. But it immediately responds to
ping N with pong N without even telling the ongoing search about it.

No, that is completely broken. Unless you do it only during pondering, which in CECP does not count as processing of any command. When the search results in printing of the move, and you would have responded to the ping before emitting that move, a compliant GUI would not think that move was the result of the current search, but of the next one. E.g.

Code: Select all

new
usermove e2e4

force
ping 13
new
go
  pong 13
  move e7e5
result 0-1 {Forfeit by illegal move: e7e5}

would get you forfeited by illegal move because you play e7e5 as white in the following game.

[stegemma]

[...]
No, that is completely broken. Unless you do it only during pondering, which in CECP does not count as processing of any command. When the search results in printing of the move, and you would have responded to the ping before emitting that move, a compliant GUI would not think that move was the result of the current search, but of the next one. E.g.

Code: Select all

new
usermove e2e4

force
ping 13
new
go
  pong 13
  move e7e5
result 0-1 {Forfeit by illegal move: e7e5}

would get you forfeited by illegal move because you play e7e5 as white in the following game.

That's sounds to me as a broken GUI. If the GUI sends PING 13 is because it needs to know if the program is still running, so why to send NEW before the response from the engine? If the GUI doesn't receive the PONG 13 it should suppose that the engine has crashed or has entered an infinite loop or something other bad thing. Of course that's true only if PONG is mandatory but, if it isn't, it serves to nothing...

[stegemma]

[...]In Elephant the input is tested in the main loop and every 10000(?) nodes in the search.[...]

It is better, IMHO, to use a separate thread to handle input from then GUI. This way you would only have to set a boolean to stop the search and maybe even kill the search thread if needed.

[hgm]

That's sounds to me as a broken GUI. If the GUI sends PING 13 is because it needs to know if the program is still running, so why to send NEW before the response from the engine? If the GUI doesn't receive the PONG 13 it should suppose that the engine has crashed or has entered an infinite loop or something other bad thing. Of course that's true only if PONG is mandatory but, if it isn't, it serves to nothing...

No, that is how things are supposed to work. The GUI doesn't send "ping 13" to know whether the program is still running, but as a 'fence' to be able to determine later whether the search terminated with or without producing a move. If it receives the "pong 13" before receiving any move it knows that the "force" command must be fully processed, and apparently stopped the search without leading to the production of a move. (The latter would always be a possibility, even if CECP explicitly forbade it, (which its doesn't), because the "force" and "move" could cross.)

The problem with what you propose is that you never know how long you have to wait. Engines are allowed to defer processing of any following commands until they finished the search on their own accord, and most engines actually do that. So it could take minutes before the "force" gets even seen by the engine, in a long-TC game.

Delaying the "new" command until after the engine has finished thinking serves no purpose. It just involves extra cycles of engine<->GUI communication that produce overhead in engines that work smoothly. The "new" will simply wait in the pipe until the engine is ready to process it, and doesn't have to be requested separately (by pong).

[hgm]

It is better, IMHO, to use a separate thread to handle input from then GUI. This way you would only have to set a boolean to stop the search and maybe even kill the search thread if needed.

Well, starting threads requires a lot of code overhead, introduces dependence on libraries you otherwise would not need, and is not trivial to make platform independent. It might also confront you with race conditions and thread-synchronization problems. Polling is often much simpler, if you did not need the threads anyway for implementing parallel search.

[stegemma]

It is better, IMHO, to use a separate thread to handle input from then GUI. This way you would only have to set a boolean to stop the search and maybe even kill the search thread if needed.

Well, starting threads requires a lot of code overhead, introduces dependence on libraries you otherwise would not need, and is not trivial to make platform independent. It might also confront you with race conditions and thread-synchronization problems. Polling is often much simpler, if you did not need the threads anyway for implementing parallel search.

Of course, polling is very much simpler. The overhead to starts a thread seems not to be a problem, is faster enough to never notice some delay. Still you could start the input thread and the search thread both at program start-up and never stop them, so the starting time would be virtually zero, between moves.

For portability, I use pthread library and a simple wrapper to C++ (not strictly necessary) and the library works well in Windows/Linux/Mac OS X. It seems to be supported on Android NDK too but I've never tried it this way, at present.

For me, implementing threads has been an interesting part of the chess programming hobby, that I've used in my business applications too. As said, it's not mandatory and the simplest solution of polling would be enough.

[hgm]

Yes, speed would not be a problem. Using threaded input would actually be faster, because there is no polling delay. The input thread wakes up as soon as the input arrives, and you can afford polling the 'abort flag' it sets in every node. The overhead I mentioned was referring to the amount of code you would have to produce to implement this.

[D Sceviour]

It is better, IMHO, to use a separate thread to handle input from then GUI. This way you would only have to set a boolean to stop the search and maybe even kill the search thread if needed.

Well, starting threads requires a lot of code overhead, introduces dependence on libraries you otherwise would not need, and is not trivial to make platform independent. It might also confront you with race conditions and thread-synchronization problems. Polling is often much simpler, if you did not need the threads anyway for implementing parallel search.

At a suggestion from HGM, I created a polling thread. One of the difficulties was synchronizing I/O (at least with windows platform). There are tricks with mutex to get around this, but the simplest method was not to poll or use any I/O functions inside the thread. Instead, the polling thread continuously monitors only two functions: Timer() and PeekNamedPipe(). Then it sets an abort_search flag for time out, or an internal pipe_request flag for the pipe. The only overhead on the engine search is to monitor two internal boolean variables: abort_search and pipe_request.

Upon receiving the pipe_request flag the engine can (1) interrupt the search anywhere, (2) Input() the waiting pipe_request, and (3) set the abort_search flag depending on the conditions. Clean, no mutex, no I/O conflict, no display garbage and easy to debug. And there are no extra demands on WB protocol!

[stegemma]

[...]the polling thread continuously monitors only two functions: Timer() and PeekNamedPipe(). Then it sets an abort_search flag for time out, or an internal pipe_request flag for the pipe. The only overhead on the engine search is to monitor two internal boolean variables: abort_search and pipe_request.[...]

I've separated the engine thread against the interface thread and the input thread, so the interface thread handles protocol commands (reading from input thread with a mutex) and time and the engine itself only have to search for the best move. This way, I need only one boolean flag for the search engine, that say "hey, stop you!". All the input/output stuff are handled by the interface thread, that eventually just stop the engine, for commands that require an immediate answer (or abort and so on). I use Mutex to protect any structure shared between threads, even to get the PV from the engine (only the best move, for now).

For debugging purpose, I use a FIFO stack protected by a mutex where the engine puts its strings (but even the interface and all other objects of the program use it). Using mutex seems to me to be the more flexible way to communicates between threads and you "only" have to take care of dead-locks.

This way, the search engine doesn't know anything about the protocol used, it only starts/stops thinking, eventually sets some options through its functions/parameters. You can change protocol with no changes in the search engine... or don0't use a protocol at all, for stand-alone software.