Ugly UCI

[D Sceviour]

[...]the polling thread continuously monitors only two functions: Timer() and PeekNamedPipe(). Then it sets an abort_search flag for time out, or an internal pipe_request flag for the pipe. The only overhead on the engine search is to monitor two internal boolean variables: abort_search and pipe_request.[...]

I've separated the engine thread against the interface thread and the input thread, so the interface thread handles protocol commands (reading from input thread with a mutex) and time and the engine itself only have to search for the best move. This way, I need only one boolean flag for the search engine, that say "hey, stop you!". All the input/output stuff are handled by the interface thread, that eventually just stop the engine, for commands that require an immediate answer (or abort and so on). I use Mutex to protect any structure shared between threads, even to get the PV from the engine (only the best move, for now).

For debugging purpose, I use a FIFO stack protected by a mutex where the engine puts its strings (but even the interface and all other objects of the program use it). Using mutex seems to me to be the more flexible way to communicates between threads and you "only" have to take care of dead-locks.

This way, the search engine doesn't know anything about the protocol used, it only starts/stops thinking, eventually sets some options through its functions/parameters. You can change protocol with no changes in the search engine... or don0't use a protocol at all, for stand-alone software.

I have tried mutex before with great difficulty. There also seems to be some kind of problem chasing nStdHandle when using I/O inside threads. Only the root core needs to monitor the flags so there is no need for a separate Input() thread. I/O can be done in subroutines. I suppose one could combine the time and pipe calls by using a single variable to save a little time monitoring the flag:

abort_search = STOP_NOW
abort_search = PIPE_REQUEST

All the I/O, mutex and debug problems can be cleaned up by looping the polling thread on something like this:

Code: Select all

Do
;   Wait() or Sleep() thing

   time_used = timer - start
   If (time_used > time_limit) and (analyze_mode=0) then
	abort_search = 1
   end if
   if xboard and (pipe_request=0) then
   	dw=0
   	PeekNamedPipe(nStdHandle, 0, 0, 0, @dw, 0)
   	if dw then pipe_request=1
   end if
Loop until terminate

[stegemma]

My loop is partially similar, but using my objects "objStdIn" as input thread and "sfout" as output thread:

Code: Select all

bool clsEngineSelector::Execute()
{
  PushCommand("new");
  objStdIn.RunJob();
  if (pEngine)
  {
    for (bool bOk = true; bOk;)
    {
      uint64_t TickNow = GetTicks();

      if (pEngine && pEngine->IsRunning())
      {
        if (TickEnd > 0 && TickNow > TickEnd)
        {
          pEngine->Stop();
        } else
        {
          uint64_t sec = (TickNow - TickLast) / CLOCKS_PER_SEC;
          if (sec >= 1)
          {
            TickLast = TickNow;
            SendPV();
          }
        }
      } else
      {
        if (!pEngine) break;
        switch (mode)
        {
          case enModeSet:
          case enModeEnd:
            break;
          case enModePlayWhite:
          case enModePlayBlack:
          case enModeAutoPlay:
          case enModeTestPositions:
            {
              SendPV();
              vtype Value;
              clsString sBestMove = pEngine->UserGetBestMove(true, Value);
              if (sBestMove.Length())
              {
                uint64_t ms = ((TickNow - Tick0) * 1000) / CLOCKS_PER_SEC;
                pEngine->UserMove(sBestMove);
                sfout.Push(clsString("move ") + sBestMove);
              }
              Pong();
            }
            break;
        }
      }
      // this could be lowered down to 1 ms
      Sleep_ms(10);
      clsString s = objStdIn.Pop();
      if (s.Length())
      {
        clsString sCommand = s.TokenLeft(' ');
        clsString sParameters = s.TokenRight(' ');
        switch (XBoardCommand(sCommand, sParameters))
        {
          case intQuit:
            objStdIn.Cancel();
            bOk = false;
            break;
        }
      }
    }
  }
  return true;
}

The commands are handled in XBoardCommand function, not in the engine (pEngine points to a clsEngine object). PushCommand function simply push a command in objStdIn:

Code: Select all

void clsEngineSelector::PushCommand(const clsString &Command)
{
   objStdIn.Push(Command);
}

Maybe I have exaggerated a little with complexity. A command sent by the GUI or response to GUI takes this path:

Code: Select all

GUI -> std::in -> objStdIn (mutex_in) -> clsEngineSelector -> clsEngine

clsEngine -> (loop) clsEngineSelector -> (mutex_out) sfout -> std::out -> GUI

(clsEngineSelector serves also as a selector to various versions of my engine)

More than this, I have a clsEngineInterface object that I use as a base class to connect to external engines in UCI/XBOARD protocols... but this stuff is from the GUI perspective.

I don't like to make things simple...

[D Sceviour]

I don't like to make things simple...

Your timer function is clear enough. Where is the pipe being examined? What is the need for SendPV() inside a polling thread loop?

[hgm]

I typically just put the reading of a command line and the parsing of it as a protocol command in a subroutine, which would suppress the actual reading if the command-line buffer is already filled. (And clears the buffer if it processes the command in it.) The search calls this routine if checking for pipe input (which it does every 1000 nodes or so) shows there is input waiting.

The routine first checks for commands that can/must be handled during the search without disturbing it, and would return without setting the abortFlag after processing those. The remaining commands would only be parsed if the routine was called from the main command loop (as indicated by a parameter passed to it), and would lead to an early return after setting the abortFlag (and without clearing the command-line buffer) when called from the search. The search will then unwind, and the following call from the main command loop will re-encounter the command.

A special case is an input move during pondering, which would have to be judged for being a ponder hit/miss, and lead to search abortion in the latter case, but must change the state from pondering to searching on a hit.

[stegemma]

I don't like to make things simple...

Your timer function is clear enough. Where is the pipe being examined? What is the need for SendPV() inside a polling thread loop?

Here's the objStdIn object:

Code: Select all

clsStdInThread::clsStdInThread(): clsThread("stdin")
{
}

clsStdInThread::~clsStdInThread()
{
	return;
}

bool clsStdInThread::Execute()
{
	while(state==enThreadRunning)
	{
		std::string s;
#ifdef _WIN32
		HANDLE hstdin = GetStdHandle(STD_INPUT_HANDLE);
		if (WaitForSingleObject(hstdin, 10) == WAIT_OBJECT_0)
#else
#endif
		{
			std::getline(std::cin, s);
			if (s.length())	ssx.PushString(s.c_str());
		} else Sleep_ms(10);
		pthread_testcancel();
	}
	return true;
}

clsString clsStdInThread::Pop(bool bWait)
{
	// sfout.Push("# stdin pop");
	clsString sRet;
  do
  {
    if(bWait) Sleep_ms(10);
		sRet = ssx.PopFirstString(); //  cGet(0);
  }while(bWait && sRet.Length()==0);
	return sRet;
}

clsStdInThread &clsStdInThread::Push(const clsString &s)
{
	if(mutInput.Lock())
	{
		ssx.PushString(s);
		mutInput.UnLock();
		// sfout.Push(" done");
	}
	return *this;
}

int clsStdInThread::Count()
{
	return ssx.Count();
}

The object ssx is a "protected string", that takes care of writing/reading with the protection of a mutex.

The SendPV is needed because the search thread never sends output (but for debug). When at root the search engine finds a new best move, it save that move in a protected object and go ahead with search. The main loop periodically check for a PV to send out and then send it to std::out always through the sfout output thread object. This way, the search thread only have to save the best move (protected by a mutex) and then doesn't care about anything related to std::out, just save the string to a string "collection" (a FIFO stack, protected by a mutex).

Of course there are various levels in my objects library and it would be impossible to show all of them. The chess engine is in a chess library while the thread/input/output objects and so on are in a general library, shared between chess and business applications. With this stratifications I can really reuse all the objects that I've wrote... even to get some money from them... but not from chess stuffs!