Polling standard input from C++

[sje]

Polling standard input from C++

Code: Select all

bool IsTerminalInput(void)
{
  return isatty(fileno(stdin));
}

bool WaitUnixFileByIndexMsec(const int index, const Msec msec)
{
  fd_set fdset;
  struct timeval waittval;
  const int fdcount = index + 1;
  int rc;

  FD_ZERO(&fdset);
  FD_SET(index, &fdset);
  TimeValFromUsec((msec * 1000), waittval);

  do
  {
    rc = select(fdcount, &fdset, 0, 0, &waittval);
  } while ((rc < 0) && (errno == EINTR));

  if (rc < 0)
    Die("WaitUnixFileByIndexMsec", "Bad select");
  return FD_ISSET(index, &fdset);
}

bool WaitStandardInputMsec(const Msec msec)
{
  return WaitUnixFileByIndexMsec(fileno(stdin), msec);
}

bool IsStandardInputDataAvalable(void)
{
  return WaitStandardInputMsec(0);
}

The interesting bit of code is the select() call inside the do/while loop in WaitUnixFileByIndexMsec(). It is amazing that the program can run for many days with millions of calls to select() without a problem until a timer signal is somehow sent where it doesn't need to be sent and the select() call returns early with an EINTR error.

[mar]

I solved this problem by simply reading stdin in a loop in main thread (blocking) and having engine (=main search) in separate thread.
Portable plus no need to poll within search.

[sje]

The input watcher does get its own thread. This thread has a loop which blocks on input for 100 msec each time through. And each time through, it handles any input plus it checks for any signals like HUP, ABRT, etc. The output of the watcher is a packet of data sent to the main reader thread (incl. semaphore+queue) which in turn gates it to the command processor thread (also incl. semaphore+queue).

[bob]

Polling standard input from C++

Code: Select all

bool IsTerminalInput(void)
{
  return isatty(fileno(stdin));
}

bool WaitUnixFileByIndexMsec(const int index, const Msec msec)
{
  fd_set fdset;
  struct timeval waittval;
  const int fdcount = index + 1;
  int rc;

  FD_ZERO(&fdset);
  FD_SET(index, &fdset);
  TimeValFromUsec((msec * 1000), waittval);

  do
  {
    rc = select(fdcount, &fdset, 0, 0, &waittval);
  } while ((rc < 0) && (errno == EINTR));

  if (rc < 0)
    Die("WaitUnixFileByIndexMsec", "Bad select");
  return FD_ISSET(index, &fdset);
}

bool WaitStandardInputMsec(const Msec msec)
{
  return WaitUnixFileByIndexMsec(fileno(stdin), msec);
}

bool IsStandardInputDataAvalable(void)
{
  return WaitStandardInputMsec(0);
}

The interesting bit of code is the select() call inside the do/while loop in WaitUnixFileByIndexMsec(). It is amazing that the program can run for many days with millions of calls to select() without a problem until a timer signal is somehow sent where it doesn't need to be sent and the select() call returns early with an EINTR error.

There is a simple rule to follow here.

(a) if you use ANY system call that can block (and select() certainly fits that) And if you use any signal handlers whatsoever, then

(b) after ANY call that can block, you have to check errno to see if it is EINTR. If so, you ignore the return and go right back to the system call.

and for a (c) you MUST verify that the call returns an error indication (-1) before you check errno. If no error is detected, errno is not set to zero, it is left alone.

Even if YOU don't use signals, you have to make certain no underlying software does (posix threads for example) since anything that creates a new process most likely will be interested in SIGCHLD...

[zd3nik]

I solved this problem by simply reading stdin in a loop in main thread (blocking) and having engine (=main search) in separate thread.
Portable plus no need to poll within search.

After trying the blocking approach with many frustrations along the way (mostly in respect to portability - unix and windows have very different frameworks for this sort of thing) I also abandoned that approach in favor of the approach Martin describes.

STC

[michiguel]

I solved this problem by simply reading stdin in a loop in main thread (blocking) and having engine (=main search) in separate thread.
Portable plus no need to poll within search.

+1000

It is the only approach I ever used. In fact, at the end of last century , I learned multithreading programming just to do this (which was not that difficult for something simple like this). Any other approach gave me headaches before starting.

Miguel

[sje]

The idea for using a timed block instead of an indefinitely long block is to allow the thread to be able to terminate itself via a simple return vs an externally supplied pthread_cancel() (or a KILL or TERM signal).

In a Watcher thread, the event loop continues to run until it has its volatile bool variable isrunning set true by an external thread. Until then, it efficiently watches for input and for four different signals.

Code: Select all

void *Watcher::Starter(void *objptr)
{
  ((Watcher *) objptr)->EventLoop();
  return 0;
}

Watcher::Watcher(ReaderPtr ptr):readerptr(ptr)
{
  DIPtr->Log("Watcher created");
  isrunning = true;
  threadptr = new Thread(Starter, this);
}

Watcher::~Watcher(void)
{
  isrunning = false;
  delete threadptr;
  DIPtr->Log("Watcher destroyed");
}

void Watcher::EventLoop(void)
{
  std::string text = "";
  bool sawEOF = false;

  while (isrunning)
  {
    WaitStandardInputMsec(100);
    while (IsStandardInputDataAvalable() && !sawEOF)
    {
      char ch;
      int rc;

      rc = (int) read(fileno(stdin), &ch, 1);
      switch (rc)
      {
        case 0:
          {
            DIPtr->Log("Watcher::EventLoop Saw EOF on input");
            readerptr->PostInEOF();
            sawEOF = true;
          };
          break;

        case 1:
          if (ch != '\n')
            text += ch;
          else
          {
            DIPtr->EchoInputToLog(text);
            readerptr->PostInString(text);
            text = "";
          };
          break;

        default:
          SwitchFault("Watcher::EventLoop");
          break;
      };
    };

    if (isrunning && DIPtr->IsSignalPending())
    {
      if (DIPtr->PendingSIGHUP)
      {
        DIPtr->Log("Watcher::EventLoop Got signal SIGHUP");
        DIPtr->PendingSIGHUP  = false;
        readerptr->PostInSignal((ui) SIGHUP);
      };

      if (DIPtr->PendingSIGINT)
      {
        DIPtr->Log("Watcher::EventLoop Got signal SIGINT");
        DIPtr->PendingSIGINT  = false;
        readerptr->PostInSignal((ui) SIGINT);
      };

      if (DIPtr->PendingSIGQUIT)
      {
        DIPtr->Log("Watcher::EventLoop Got signal SIGQUIT");
        DIPtr->PendingSIGQUIT  = false;
        readerptr->PostInSignal((ui) SIGQUIT);
      };

      if (DIPtr->PendingSIGTERM)
      {
        DIPtr->Log("Watcher::EventLoop Got signal SIGTERM");
        DIPtr->PendingSIGTERM  = false;
        readerptr->PostInSignal((ui) SIGTERM);
      };
    };
  };
}

[lucasart]

I solved this problem by simply reading stdin in a loop in main thread (blocking) and having engine (=main search) in separate thread.
Portable plus no need to poll within search.

+1000

It is the only approach I ever used. In fact, at the end of last century , I learned multithreading programming just to do this (which was not that difficult for something simple like this). Any other approach gave me headaches before starting.

Miguel

Using `select()` is really easy. It's the Windows equivalent that is messy!

But doing it with threads is much more elegant, and with C++11 it is even portable.

[sje]

I've since moved the Unix signal delivery from the semaphore queuing scheme to have it handed without delay to a running task. Example: the random game generator now checks the SIGINT (Control-C) flag set by a signal handler once each time through the game loop; the generation now concludes early but correctly if the signal occurs. The checking requires no system call; only a memory read of a single volatile bool flag.

----

Measurements of Symbolic show that its semaphore queuing and all inter-thread fiddling eats less than one percent of the CPU. There are virtually no calls to unproductive sleeps/waits/polls other than a a very few during program initialization or termination.

The only waiting noticeable by the user is for allocation or clearing of transposition table storage. Allocation runs between one and two GiB per second and the multithreaded clearing runs between ten to twenty GiB per second:

Code: Select all

[2015-05-12 10:59:14.903] Transposition table set allocation begun
[2015-05-12 10:59:14.903]   Allocating full position transposition table
[2015-05-12 10:59:15.398]     ItemSize: 24 B   ItemCount: 33,554,432 (2^25)   TableSize: 768 MiB
[2015-05-12 10:59:15.398]   Allocating material balance transposition table
[2015-05-12 10:59:15.433]     ItemSize: 24 B   ItemCount: 1,048,576 (2^20)   TableSize: 24 MiB
[2015-05-12 10:59:15.433]   Allocating pawn structure transposition table
[2015-05-12 10:59:16.258]     ItemSize: 80 B   ItemCount: 16,777,216 (2^24)   TableSize: 1,280 MiB
[2015-05-12 10:59:16.258] Transposition table set allocation ended

----

My own Thread class is as good and as portable as the C++11 near-equivalent, so I'm not losing anything here by not having the C++11 tool chain on all of the target platforms.

----

If available, the system call poll() is slightly simpler than select() and can handle some of what select() does.

http://www.ipnom.com/FreeBSD-Man-Pages/poll.2.html

For lower level control, the routines to use are kqueue() and kevent().

http://www.ipnom.com/FreeBSD-Man-Pages/kevent.2.html
----

I have no clue as to how Windows does any of this, but sooner or later any functionality which shows up in BSD or Linux will also appear in Windows in some contorted way. You only have to wait.

[Look]

I have no clue as to how Windows does any of this, but sooner or later any functionality which shows up in BSD or Linux will also appear in Windows in some contorted way. You only have to wait.

Please consider this link:

https://msdn.microsoft.com/en-us/librar ... s.85).aspx