Move Tables - explain as if I'm five.

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Repetition detection approach (Re: Move Tables ...)

Post by Sven »

Gerd Isenberg wrote:These approaches do not consider all repetitions of positions, i.e. where two rooks of each side exchange their places.
If a white rook on a1 and a black rook on a8 exchange their places, wouldn't the code above flag that as a (false) repetition?

Code: Select all

Ra1-b1 => ++b[a1], --b[b1] => c++, c++
Ra8-a7 => ++b[a8], --b[a7] => c++, c++
Rb1-b8 => ++b[b1], --b[b8] => c--, c++
Ra7-a2 => ++b[a7], --b[a2] => c--, c++
Rb8-a8 => ++b[b8], --b[a8] => c--, c--
Ra2-a1 => ++b[a2], --b[a1] => c--, c-- => c = 0
The problem is that colors are not considered.

Sven
Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Re: Move Tables - explain as if I'm five.

Post by Sven »

hgm wrote:Just the opposite, right? They do detect false repetitions when you exchange two unequal pieces, as they only consider square occupancy, and thus implicitly consider all pieces equal. This could be fixed by not incrementing the square counters by 1, but by a piece-type-dependent number. This would require storing of the piece type in the move list, however.
That's even worse than what I thought: not only colors but also piece types are not considered.

I think there is still nothing substantially better for repetition detection than the current "state of the art" approach of comparing hash keys for ply-4, ply-6, ... until the 50 moves counter is reached.

Sven
Gerd Isenberg
Posts: 2250
Joined: Wed Mar 08, 2006 8:47 pm
Location: Hattingen, Germany

Re: Move Tables - explain as if I'm five.

Post by Gerd Isenberg »

hgm wrote:Just the opposite, right? They do detect false repetitions when you exchange two unequal pieces, as they only consider square occupancy, and thus implicitly consider all pieces equal. This could be fixed by not incrementing the square counters by 1, but by a piece-type-dependent number. This would require storing of the piece type in the move list, however.
Yes, you are right. So this gnu chess aproach is really buggy.
User avatar
hgm
Posts: 27808
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Move Tables - explain as if I'm five.

Post by hgm »

Indeed, keys are superior. Especially if you already have calculated them for other purposes.

Whether performing a linear search through the list is the optimal method is debatable, though. Organizing them in a small hash table that stores (key, count) pairs (where count is the number of times the position occurred, i.e. an entry with count=0 is empty). Even with only 64 entries in the table the table would almost always be mostly empty, so that your first probe in it would already hit on a position with count=0, proving the current position is not a repetition. If you hit upon a non-zero count you step through the table with the usual linear search until you reach a zero count or a matching key. (And without a match you would use the entry where the search ended to store the key of the new position with count=1.) If you want to discard null-move-spanning repeats, the table would have to record the ply level of the latest occurrence of that position, which you would have to compare to your current level minus irreversible move counter.

Especially in variants where there are no irreversible moves (such as Crazyhouse), this gives a huge gain.
Gerd Isenberg
Posts: 2250
Joined: Wed Mar 08, 2006 8:47 pm
Location: Hattingen, Germany

Re: Repetition detection approach (Re: Move Tables ...)

Post by Gerd Isenberg »

Sven Schüle wrote:
Gerd Isenberg wrote:These approaches do not consider all repetitions of positions, i.e. where two rooks of each side exchange their places.
If a white rook on a1 and a black rook on a8 exchange their places, wouldn't the code above flag that as a (false) repetition?

Code: Select all

Ra1-b1 => ++b[a1], --b[b1] => c++, c++
Ra8-a7 => ++b[a8], --b[a7] => c++, c++
Rb1-b8 => ++b[b1], --b[b8] => c--, c++
Ra7-a2 => ++b[a7], --b[a2] => c--, c++
Rb8-a8 => ++b[b8], --b[a8] => c--, c--
Ra2-a1 => ++b[a2], --b[a1] => c--, c-- => c = 0
The problem is that colors are not considered.

Sven
Yep, therefor some "improvements" in Gnu Chess 4 ...

Code: Select all

inline int repetition ()
/*  Check for draw by threefold repetition.  */
{
  register SHORT i, c, cnt;
  register SHORT f, t;
  SHORT b[64];

  cnt = c = 0;
  /* try to avoid work */
  memset ((CHAR *) b, 0, sizeof (b));
  for (i = GameCnt; i > Game50; i--)
  {
    f = GameList[i].gmove >> 8;
    t = GameList[i].gmove & 0x3f;

    if (b[f]) c--;
    if (b[t]) c--;
    b&#91;f&#93; = &#40;board&#91;f&#93; + &#40;color&#91;f&#93; << 8&#41;)
         - &#40;board&#91;t&#93; + &#40;color&#91;t&#93; << 8&#41;) + b&#91;t&#93;;
    b&#91;t&#93; = &#40;board&#91;t&#93; + &#40;color&#91;t&#93; << 8&#41;) - &#40;no_piece + &#40;neutral << 8&#41;);
    if &#40;b&#91;f&#93;) c++;
    if &#40;b&#91;t&#93;) c++;
    if &#40;c == 0&#41;
      if ((&#40;i ^ GameCnt&#41; & 1&#41;)
        cnt++;
  &#125;
  return &#40;cnt&#41;;
&#125;
Edmund
Posts: 670
Joined: Mon Dec 03, 2007 3:01 pm
Location: Barcelona, Spain

Re: Move Tables - explain as if I'm five.

Post by Edmund »

hgm wrote:Indeed, keys are superior. Especially if you already have calculated them for other purposes.

Whether performing a linear search through the list is the optimal method is debatable, though. Organizing them in a small hash table that stores (key, count) pairs (where count is the number of times the position occurred, i.e. an entry with count=0 is empty). Even with only 64 entries in the table the table would almost always be mostly empty, so that your first probe in it would already hit on a position with count=0, proving the current position is not a repetition. If you hit upon a non-zero count you step through the table with the usual linear search until you reach a zero count or a matching key. (And without a match you would use the entry where the search ended to store the key of the new position with count=1.) If you want to discard null-move-spanning repeats, the table would have to record the ply level of the latest occurrence of that position, which you would have to compare to your current level minus irreversible move counter.

Especially in variants where there are no irreversible moves (such as Crazyhouse), this gives a huge gain.
I think I don't fully comprehend. Am I getting your right?

You have a list of hash values as well as a small (64 entries) transposition table.
When you make a move on the board you try to skip the linear search through the list if the count of the tt entry on slot (hash & 63) is either 0 or 1. 0 meaning no repetition, 1 meaning either repetition or not depending on whether the keys match.
When you make a valid move, you add the new key to the list. If the tt entry slot is count 0 you add the key and make count 1; if count > 0 you increment count.
User avatar
hgm
Posts: 27808
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Move Tables - explain as if I'm five.

Post by hgm »

No, I only have the hash table. (I thought of having one for each side, but they can be combined into one of size 128.)

Code: Select all

// global
struct &#123;
  int64 key;
  int ply;
&#125; repTab&#91;150&#93;;

// In Search&#58;
if&#40;reversibleMoveCnt > 3&#41; &#123;
  for&#40;i=hashKey&127; repTab&#91;i&#93;.ply; i++) &#123;
    if&#40;repTab&#91;i&#93;.key == hashKey&#41; &#123;
      repetition = &#40;repTab&#91;i&#93;.ply >= ply - reversibleMoveCnt&#41;; // occurred after last null move
      break;
    &#125;
  &#125; // leaves i at first empty slot
&#125;

if&#40;repetition&#41; score = 0; else &#123;
  int previousOccurrence = repTab&#91;i&#93;.ply;
  repTab&#91;i&#93;.key = hashKey;
  repTab&#91;i&#93;.ply = ply; // suppose ply counting is such that 0 never occurs

  MakeMove&#40;);
  score = -Search&#40;...);
  UnMake&#40;);

  repTab&#91;i&#93;.ply = previousOccurrence;
&#125;
Note I did not bother to make the linear search through the hash table wrap (i = i+1&127), but just declared a bit larger table.
User avatar
stegemma
Posts: 859
Joined: Mon Aug 10, 2009 10:05 pm
Location: Italy
Full name: Stefano Gemma

Re: Move Tables - explain as if I'm five.

Post by stegemma »

ZirconiumX wrote:Fail - it usually helps to show how it's being probed.

Code: Select all

int gen_moves&#40;int is_rook, uint32_t * stack, int offset, int from&#41;
&#123;
	mt_node node;
	int piece;
	int side;
	int to;
	int x;

	x = offset;
	side = WHITE;

	node = move_head&#91;is_rook&#93;&#91;from&#93;;
	do &#123;
		to = node->to;
		stack&#91;x&#93; = from | to << 6;
		x       += !&#40;board&#91;to&#93; & side&#41;;
		node     = node->next&#91;board&#91;to&#93; != NP&#93;;
	&#125; while &#40;node&#41;;

	return x;
&#125;
Matthew:out
I don't know what should do this code but i would change the do...while loop this way:

Code: Select all

	node = move_head&#91;is_rook&#93;&#91;from&#93;;
	while &#40;node&#41; &#123;
		to = node->to;
		stack&#91;x&#93; = from | to << 6;
		x       += !&#40;board&#91;to&#93; & side&#41;;
		node     = node->next&#91;board&#91;to&#93; != NP&#93;;
	&#125;
Edmund
Posts: 670
Joined: Mon Dec 03, 2007 3:01 pm
Location: Barcelona, Spain

Re: Move Tables - explain as if I'm five.

Post by Edmund »

hgm wrote:No, I only have the hash table. (I thought of having one for each side, but they can be combined into one of size 128.)

Code: Select all

// global
struct &#123;
  int64 key;
  int ply;
&#125; repTab&#91;150&#93;;

// In Search&#58;
if&#40;reversibleMoveCnt > 3&#41; &#123;
  for&#40;i=hashKey&127; repTab&#91;i&#93;.ply; i++) &#123;
    if&#40;repTab&#91;i&#93;.key == hashKey&#41; &#123;
      repetition = &#40;repTab&#91;i&#93;.ply >= ply - reversibleMoveCnt&#41;; // occurred after last null move
      break;
    &#125;
  &#125; // leaves i at first empty slot
&#125;

if&#40;repetition&#41; score = 0; else &#123;
  int previousOccurrence = repTab&#91;i&#93;.ply;
  repTab&#91;i&#93;.key = hashKey;
  repTab&#91;i&#93;.ply = ply; // suppose ply counting is such that 0 never occurs

  MakeMove&#40;);
  score = -Search&#40;...);
  UnMake&#40;);

  repTab&#91;i&#93;.ply = previousOccurrence;
&#125;
Note I did not bother to make the linear search through the hash table wrap (i = i+1&127), but just declared a bit larger table.
Clever idea. I missed the idea that you could save the index to the entry between repetition check and move make/unmake.
This should be faster than any list searches. (even though most engines have to keep their hash values in a list anyway, if they are not incrementally updating hash values on unmake)
The only problem I see remaining is your 150 bound. In chess you can get upto 100 halfmoves before the 50-move rule kicks in. so you can easily run into an out of bound error here.
User avatar
rvida
Posts: 481
Joined: Thu Apr 16, 2009 12:00 pm
Location: Slovakia, EU

Re: Move Tables - explain as if I'm five.

Post by rvida »

Edmund wrote:
hgm wrote:No, I only have the hash table. (I thought of having one for each side, but they can be combined into one of size 128.)

Code: Select all

// global
struct &#123;
  int64 key;
  int ply;
&#125; repTab&#91;150&#93;;

// In Search&#58;
if&#40;reversibleMoveCnt > 3&#41; &#123;
  for&#40;i=hashKey&127; repTab&#91;i&#93;.ply; i++) &#123;
    if&#40;repTab&#91;i&#93;.key == hashKey&#41; &#123;
      repetition = &#40;repTab&#91;i&#93;.ply >= ply - reversibleMoveCnt&#41;; // occurred after last null move
      break;
    &#125;
  &#125; // leaves i at first empty slot
&#125;

if&#40;repetition&#41; score = 0; else &#123;
  int previousOccurrence = repTab&#91;i&#93;.ply;
  repTab&#91;i&#93;.key = hashKey;
  repTab&#91;i&#93;.ply = ply; // suppose ply counting is such that 0 never occurs

  MakeMove&#40;);
  score = -Search&#40;...);
  UnMake&#40;);

  repTab&#91;i&#93;.ply = previousOccurrence;
&#125;
Note I did not bother to make the linear search through the hash table wrap (i = i+1&127), but just declared a bit larger table.
Clever idea. I missed the idea that you could save the index to the entry between repetition check and move make/unmake.
This should be faster than any list searches. (even though most engines have to keep their hash values in a list anyway, if they are not incrementally updating hash values on unmake)
The only problem I see remaining is your 150 bound. In chess you can get upto 100 halfmoves before the 50-move rule kicks in. so you can easily run into an out of bound error here.
What about SMP search? During the split() operation you would need to copy the entire repetition hash structure. It is way more costly than to copy just a few entries from a list of keys (since long chains of reversible moves are rare in the search tree, most of the time the rule50 counter is some small value, say <= 10).