Move generator

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

CThinker
Posts: 388
Joined: Wed Mar 08, 2006 10:08 pm

Re: Move generator

Post by CThinker »

Also, clock() isn't required to have any particular resolution. It may be no better than 1 second resolution even if CLOCKS_PER_SEC says something else. It might just multiply the time value by that to give the impression of it being more.
As far as resolution goes, the same can be said for gettimeofday(), which is what I was trying to avoid in favor of clock(). That is, timeval.usec may just always be zero.

If I remember it correctly, most clock() implementations from *nix variants simply make use of gettimeofday().

I have yet to see clock() misbehave on me, from stamp-sized HD16 boards with Hitachi C, to IBM C on big AS/400s. So far, I have been very lucky. So, I guess I have to start watching out, from now on.

Cheers!
Carey
Posts: 313
Joined: Wed Mar 08, 2006 8:18 pm

Re: Move generator

Post by Carey »

hgm wrote:I think it is resonable to expect that the wrap-around time is longer than any time interval that would plausibly have to be measured
Reasonable... Maybe on the PC platform. (Assuming you aren't near the end of the timer period when the program starts.)

But no guarantee.

And you still have to deal with whether or not clock() deals with process time or wall clock time. Do you really want to be using process time when your program is expecting wall time? A good way to loose track of 5, 10 or more seconds and loose the game on time.

And you still have to face the possibility that the clock resolution is only 1 second, regardless of what CLOCKS_PER_SEC says. For example, some old DOS C compilers used to multiply the 18.2 pc clock to make it look like it was much higher. 60 or 100 per second. (That's just an example of a compiler writer lying to the programmer.)

by a program. If CLOCKS_PER_SEC was the CPU clock of 2.4GHz, the compiler writer would most certainly not make clock_t a 32-bit type (making the wrap-around occur in less than 2 sec).
Yes, that's why I *DID* say 64 bit....
Trying to guard against that is paranoia at the same level that an int might be only a single bit.
That wasn't the issue. That was an example of just how unspecified clock() is. And that even if the compiler writer tried to be helpful, by making it the cpu clock cycle counter (the ultimate in timer resolution), you'd still have problems.

clock() is just too unspecified to be useful in generic programming. It's only really useful for specific platforms with specific compilers. In situations where you actually test it and know for sure what is happening with that specific platform.
Carey
Posts: 313
Joined: Wed Mar 08, 2006 8:18 pm

Re: Move generator

Post by Carey »

CThinker wrote:
Also, clock() isn't required to have any particular resolution. It may be no better than 1 second resolution even if CLOCKS_PER_SEC says something else. It might just multiply the time value by that to give the impression of it being more.
As far as resolution goes, the same can be said for gettimeofday(), which is what I was trying to avoid in favor of clock(). That is, timeval.usec may just always be zero.
True.
If I remember it correctly, most clock() implementations from *nix variants simply make use of gettimeofday().
That I don't know, but that is a good example of unspecified, non-standard libraries.

The C standard tried to specify a lot of useful library routines, but they did screw up in some areas such as time & sorting. And about anything that touches the real hardware or the real world.... But it's still better than no standard at all.
I have yet to see clock() misbehave on me, from stamp-sized HD16 boards with Hitachi C, to IBM C on big AS/400s. So far, I have been very lucky. So, I guess I have to start watching out, from now on.
I bet you did and just didn't notice it.... I bet in at least a few of those cases, it was wall-clock time and in others it was process time.

Often they are so close together you don't notice it. I bet the IBM C got it right, though, and used process time.


You could do ftime().

It's not a standard function, but many platforms support it because Unix had it.

There is still no guarnatee that its resolution is any better than 1 second, but it is faster than many of the Windows specific routines that do provide guarantee higher precision.

Meaning it's something that can be used in your search routine and called hundreds or thousands of times per second without putting too much overhead into the search.

Right off the top of my head, I don't know what most Windows programmers use. I actually try not too look too much at other people's programs. (The only programs I care about are the old programs from the 70's & 80's.)


You could even do a 'fall-back' approach. Write your own wrapper that can call one of several routines depending on the accuracy available on that platform and whether you want wall-clock or process-clock.

When you are testing on your own system, it's often better to do process-clock (that way you can have other stuff running in the background), but in games it's better to have wall-clock. So it can be helpful to program them both in and switch between them depending on a command flag you enter.
User avatar
xsadar
Posts: 147
Joined: Wed Jun 06, 2007 10:01 am
Location: United States
Full name: Mike Leany

Re: Move generator

Post by xsadar »

hgm wrote:I think it is resonable to expect that the wrap-around time is longer than any time interval that would plausibly have to be measured by a program.
My man page says:
Note that the time can wrap around. On a 32-bit system where CLOCKS_PER_SEC equals 1000000 this function will return the same value approximately every 72 minutes.
From my (somewhat limited) personal experience these values seem to be the norm for gcc on 32-bit Linux systems. I can think of a lot of things I might have to measure that are longer than 72 minutes, some of them involving chess (deep searches, deep perfts, etc). However, if clock() always gave wallclock time, you could use it in combination with time(). However, it does not, as indicated by this line from my manpage:
The clock() function returns an approximation of processor time used by the program.
And that's the main reason I won't use it for my chess engine.
User avatar
hgm
Posts: 27808
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Move generator

Post by hgm »

On the Linux systems I have seen CLOCKS_PER_SEC was not 1000000, but 1000. So you would have 72000 minutes...
User avatar
Roman Hartmann
Posts: 295
Joined: Wed Mar 08, 2006 8:29 pm

Re: Move generator

Post by Roman Hartmann »

hgm wrote:On the Linux systems I have seen CLOCKS_PER_SEC was not 1000000, but 1000. So you would have 72000 minutes...
I guess it depends on the distro. CLOCKS_PER_SEC is defined to be 10^6 on my Linux box. That gives you only 72 minutes which is not enough (if you're careless and don't use a unsigned variable it will be 36 minutes only). I never had any accuracy issues with clock() though.

clock() is perfectly ok if you stick to windows and don't need portability.

Roman
Carey
Posts: 313
Joined: Wed Mar 08, 2006 8:18 pm

Re: Move generator

Post by Carey »

hgm wrote:On the Linux systems I have seen CLOCKS_PER_SEC was not 1000000, but 1000. So you would have 72000 minutes...
If that's what it is on yours, then that's what it is on yours. Nobody is disagreeing with that.

What we are saying is that various platforms have a wide variety of values and that you can't really depend on it in a portable application.


Do a google search for clocks_per_sec and you can find several values mentioned.

Even a GNU page talking about how clock_t type can actually be floating point, instead of an 'int'. (Which is actually correct. The C standard just says clock_t has to be an arithmetic type.)

I also quickly found a page saying that the XSI extension to POSIX *requires* clocks_per_tick to be set at one million. (I didn't check, but I would certainly hope it also requires clock_t to be 64 bit. But how many people actually use that data type, and instead store it into an 'int'...?)

(And, with the timer being 1,000,000 that opens up the question of what is the granularity of that timer? It may say one million but if it's actually only doing 18.2 ticks per second, then you can be in trouble if your program is actually expecting 100 or even 1000 ticks per second.)

I have no idea how common that extension is in various systems, but it's pretty clear that you can get a range anywhere from 18.2 to one million for CLOCKS_PER_SEC.

So with the wide range in ticks available, plus uncertainty whether it's wall or processor time, it should be pretty obvious that it's not a reliable source for a portable timer count.

All of this could have been avoided if the C standard had just done a bit more work on the library. (Of course, I can't complain too loudly... It's better than some other standardized languages.)
Dann Corbit
Posts: 12541
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Move generator

Post by Dann Corbit »

Carey wrote:
hgm wrote:On the Linux systems I have seen CLOCKS_PER_SEC was not 1000000, but 1000. So you would have 72000 minutes...
If that's what it is on yours, then that's what it is on yours. Nobody is disagreeing with that.

What we are saying is that various platforms have a wide variety of values and that you can't really depend on it in a portable application.


Do a google search for clocks_per_sec and you can find several values mentioned.

Even a GNU page talking about how clock_t type can actually be floating point, instead of an 'int'. (Which is actually correct. The C standard just says clock_t has to be an arithmetic type.)

I also quickly found a page saying that the XSI extension to POSIX *requires* clocks_per_tick to be set at one million. (I didn't check, but I would certainly hope it also requires clock_t to be 64 bit. But how many people actually use that data type, and instead store it into an 'int'...?)

(And, with the timer being 1,000,000 that opens up the question of what is the granularity of that timer? It may say one million but if it's actually only doing 18.2 ticks per second, then you can be in trouble if your program is actually expecting 100 or even 1000 ticks per second.)

I have no idea how common that extension is in various systems, but it's pretty clear that you can get a range anywhere from 18.2 to one million for CLOCKS_PER_SEC.

So with the wide range in ticks available, plus uncertainty whether it's wall or processor time, it should be pretty obvious that it's not a reliable source for a portable timer count.

All of this could have been avoided if the C standard had just done a bit more work on the library. (Of course, I can't complain too loudly... It's better than some other standardized languages.)
The clock() function can return -1 for any call to clock() in a conforming implementation:

"If the processor time used is not available or its value cannot be represented, the function returns the value (clock_t)(-1)."
page: 338 section: Library §7.23.2.2 under the clock() function description.

Since we are concerned in a chess program with the observer's time, clock() is clearly the wrong function to use anyway. Suppose that a database query runs, taking 50% of the CPU's time an you have ten seconds left. You might consume only 5 seconds using clock(), but the observer measures ten seconds consumed and you lose on time. Clearly, that measure will also accumulate error. A chess program needs to use wall time for measurement.
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 7:43 pm

Re: Move generator

Post by sje »

Don't use clock().

Something that's portable to sane systems that you can use:

Code: Select all

#include <sys/time.h>
#include <sys/resource.h>

#include <iomanip>
#include <ostream>

// Some stuff removed, but&#58;

typedef unsigned long long int CTQwrd;
typedef unsigned long long int CTUsec;
typedef unsigned long long int CTMsec;

CTUsec CTCurrentTimeUsec&#40;void&#41;
&#123;
  struct timeval theTimeval;

  gettimeofday&#40;&theTimeval, 0&#41;;
  return
    ((&#40;CTQwrd&#41; theTimeval.tv_sec&#41; * 1000000ULL&#41; +
    (&#40;CTQwrd&#41; theTimeval.tv_usec&#41;;
&#125;

CTUsec CTCurrentUsageUsec&#40;void&#41;
&#123;
  struct rusage theRusage;

  getrusage&#40;RUSAGE_SELF, &theRusage&#41;;

  const CTSec theSec =
    theRusage.ru_utime.tv_sec + theRusage.ru_stime.tv_sec;
  const CTUsec theUsec =
    theRusage.ru_utime.tv_usec + theRusage.ru_stime.tv_usec;

  return &#40;theSec * 1000000ULL&#41; + theUsec;
&#125;

void CTMapUsecToTM&#40;const CTUsec theUsec, struct tm &theTM&#41;
&#123;
  const time_t theTime = theUsec / 1000000ULL;

  gmtime_r&#40;&theTime, &theTM&#41;;
&#125;

void CTEncodeTMYMD&#40;std&#58;&#58;ostream &theOstr, const struct tm &theTM&#41;
&#123;
  theOstr <<
    std&#58;&#58;setfill&#40;'0') << std&#58;&#58;setw&#40;4&#41; << &#40;theTM.tm_year + 1900&#41; << '.' <<
    std&#58;&#58;setw&#40;2&#41; << &#40;theTM.tm_mon + 1&#41; << '.' <<
    std&#58;&#58;setw&#40;2&#41; << theTM.tm_mday <<
    std&#58;&#58;setfill&#40;' ');
&#125;

void CTEncodeTMHMS&#40;std&#58;&#58;ostream &theOstr, const struct tm &theTM&#41;
&#123;
  theOstr <<
    std&#58;&#58;setfill&#40;'0') << std&#58;&#58;setw&#40;2&#41; << theTM.tm_hour << '&#58;' <<
    std&#58;&#58;setw&#40;2&#41; << theTM.tm_min << '&#58;' << std&#58;&#58;setw&#40;2&#41; << theTM.tm_sec <<
    std&#58;&#58;setfill&#40;' ');
&#125;

void CTEncodeTM&#40;std&#58;&#58;ostream &theOstr, const struct tm &theTM&#41;
&#123;
  CTEncodeTMYMD&#40;theOstr, theTM&#41;;
  theOstr << ' ';
  CTEncodeTMHMS&#40;theOstr, theTM&#41;;
&#125;

CTUsec CTDecodeUsec&#40;const char *theStr&#41;
&#123;
  CTUsec theUsec;

  CTLexStrToUsec&#40;theStr, theUsec&#41;;
  return theUsec;
&#125;

void CTEncodeUsec&#40;std&#58;&#58;ostream &theOstr, const CTUsec theUsec&#41;
&#123;
  const CTQwrd theTotalUSec     = theUsec;
  const CTQwrd theTotalSeconds  = theTotalUSec / 1000000ULL;
  const CTQwrd theCentiSeconds  = &#40;theTotalUSec % 1000000ULL&#41; / 10000;
  const CTQwrd theTotalMinutes  = theTotalSeconds / 60;
  const CTQwrd theTotalHours    = theTotalMinutes / 60;
  const CTQwrd theTotalDays     = theTotalHours / 24;

  theOstr <<
    std&#58;&#58;setfill&#40;'0') << std&#58;&#58;setw&#40;3&#41; << theTotalDays <<
    '&#58;' << std&#58;&#58;setw&#40;2&#41; << theTotalHours % 24 <<
    '&#58;' << std&#58;&#58;setw&#40;2&#41; << theTotalMinutes % 60 <<
    '&#58;' << std&#58;&#58;setw&#40;2&#41; << theTotalSeconds % 60 <<
    '.' << std&#58;&#58;setw&#40;2&#41; << theCentiSeconds << std&#58;&#58;setfill&#40;' ');
&#125;

void CTEncodeUsecNLZ&#40;std&#58;&#58;ostream &theOstr, const CTUsec theUsec&#41;
&#123;
  const CTQwrd theTotalUSec     = theUsec;
  const CTQwrd theTotalSeconds  = theTotalUSec / 1000000ULL;
  const CTQwrd theCentiSeconds  = &#40;theTotalUSec % 1000000ULL&#41; / 10000;
  const CTQwrd theTotalMinutes  = theTotalSeconds / 60;
  const CTQwrd theTotalHours    = theTotalMinutes / 60;
  const CTQwrd theTotalDays     = theTotalHours / 24;
  bool isPending = false;

  theOstr << std&#58;&#58;setfill&#40;'0');

  if &#40;isPending || theTotalDays&#41;
  &#123;
    theOstr << std&#58;&#58;setw&#40;3&#41; << theTotalDays << '&#58;';
    isPending = true;
  &#125;;

  if &#40;isPending || theTotalHours&#41;
  &#123;
    theOstr << std&#58;&#58;setw&#40;2&#41; << theTotalHours % 24 << '&#58;';
    isPending = true;
  &#125;;

  if &#40;isPending || theTotalMinutes&#41;
  &#123;
    theOstr << std&#58;&#58;setw&#40;2&#41; << theTotalMinutes % 60 << '&#58;';
    isPending = true;
  &#125;;

  theOstr <<
    std&#58;&#58;setw&#40;2&#41; << theTotalSeconds % 60 << '.' << std&#58;&#58;setw&#40;2&#41; <<
    theCentiSeconds << std&#58;&#58;setfill&#40;' ');
&#125;

void CTEncodeUsecFullNLZ&#40;std&#58;&#58;ostream &theOstr, const CTUsec theUsec&#41;
&#123;
  const CTQwrd theTotalUSec     = theUsec;
  const CTQwrd theTotalSeconds  = theTotalUSec / 1000000ULL;
  const CTQwrd theMicroSeconds  = theTotalUSec % 1000000ULL;
  const CTQwrd theTotalMinutes  = theTotalSeconds / 60;
  const CTQwrd theTotalHours    = theTotalMinutes / 60;
  const CTQwrd theTotalDays     = theTotalHours / 24;
  bool isPending = false;

  theOstr << std&#58;&#58;setfill&#40;'0');

  if &#40;isPending || theTotalDays&#41;
  &#123;
    theOstr << std&#58;&#58;setw&#40;3&#41; << theTotalDays << '&#58;';
    isPending = true;
  &#125;;

  if &#40;isPending || theTotalHours&#41;
  &#123;
    theOstr << std&#58;&#58;setw&#40;2&#41; << theTotalHours % 24 << '&#58;';
    isPending = true;
  &#125;;

  if &#40;isPending || theTotalMinutes&#41;
  &#123;
    theOstr << std&#58;&#58;setw&#40;2&#41; << theTotalMinutes % 60 << '&#58;';
    isPending = true;
  &#125;;

  theOstr <<
    std&#58;&#58;setw&#40;2&#41; << theTotalSeconds % 60 << '.' << std&#58;&#58;setw&#40;6&#41; <<
    theMicroSeconds << std&#58;&#58;setfill&#40;' ');
&#125;

void CTEncodeUsecDate&#40;std&#58;&#58;ostream &theOstr, const CTUsec theUsec&#41;
&#123;
  const time_t theTime = theUsec / 1000000ULL;
  struct tm theTM;

  localtime_r&#40;&theTime, &theTM&#41;;
  CTEncodeTM&#40;theOstr, theTM&#41;;
&#125;

void CTEncodeCurrentDate&#40;std&#58;&#58;ostream &theOstr&#41;
&#123;
  const time_t theTime = time&#40;0&#41;;
  struct tm theTM;

  localtime_r&#40;&theTime, &theTM&#41;;
  CTEncodeTMYMD&#40;theOstr, theTM&#41;;
&#125;

void CTEncodeCurrentTime&#40;std&#58;&#58;ostream &theOstr&#41;
&#123;
  const time_t theTime = time&#40;0&#41;;
  struct tm theTM;

  localtime_r&#40;&theTime, &theTM&#41;;
  CTEncodeTMHMS&#40;theOstr, theTM&#41;;
&#125;

void CTEncodeCurrentDateAndTime&#40;std&#58;&#58;ostream &theOstr&#41;
&#123;
  const time_t theTime = time&#40;0&#41;;
  struct tm theTM;

  localtime_r&#40;&theTime, &theTM&#41;;
  CTEncodeTM&#40;theOstr, theTM&#41;;
&#125;

void CTEncodeCurrentDateAndTimeMsec&#40;std&#58;&#58;ostream &theOstr&#41;
&#123;
  const CTUsec theUsec = CTCurrentTimeUsec&#40;);
  const time_t theTime = &#40;time_t&#41; &#40;theUsec / 1000000ULL&#41;;
  const CTUsec theRemUsec = theUsec % 1000000ULL;
  struct tm theTM;

  localtime_r&#40;&theTime, &theTM&#41;;
  CTEncodeTM&#40;theOstr, theTM&#41;;
  theOstr <<
    '.' << std&#58;&#58;setfill&#40;'0') << std&#58;&#58;setw&#40;3&#41; <<
    &#40;theRemUsec / 1000&#41; << std&#58;&#58;setfill&#40;' ');
&#125;

void CTEncodeCurrentLTS&#40;std&#58;&#58;ostream &theOstr&#41;
&#123;
  const time_t theTime = time&#40;0&#41;;
  char theCVec&#91;CTTextLineCharLen&#93;;

  ctime_r&#40;&theTime, theCVec&#41;;
  theCVec&#91;strlen&#40;theCVec&#41; - 1&#93; = '\0';
  theOstr << theCVec;
&#125;

void CTEncodeCurrentUTC&#40;std&#58;&#58;ostream &theOstr&#41;
&#123;
  const time_t theTime = time&#40;0&#41;;
  struct tm theTM;

  gmtime_r&#40;&theTime, &theTM&#41;;
  CTEncodeTM&#40;theOstr, theTM&#41;;
&#125;

void CTSleepMsec&#40;const CTMsec theMsec&#41;
&#123;
  struct timespec theTimespec;

  theTimespec.tv_sec = theMsec / 1000;
  theTimespec.tv_nsec = &#40;theMsec % 1000&#41; * 1000000ULL;
  nanosleep&#40;&theTimespec, 0&#41;;
&#125;

void CTSleepUsec&#40;const CTUsec theUsec&#41;
&#123;
  struct timespec theTimespec;

  theTimespec.tv_sec = theUsec / 1000000;
  theTimespec.tv_nsec = &#40;theUsec % 1000000ULL&#41; * 1000;
  nanosleep&#40;&theTimespec, 0&#41;;
&#125;
CThinker
Posts: 388
Joined: Wed Mar 08, 2006 10:08 pm

Re: Move generator

Post by CThinker »

"portable"?

gettimeofday() is not portable. Its resolution is also not guaranteed.