Recommended board representation for a rookie

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

Gerd Isenberg
Posts: 2250
Joined: Wed Mar 08, 2006 8:47 pm
Location: Hattingen, Germany

Re: Recommended board representation for a rookie

Post by Gerd Isenberg »

hgm wrote: Usually my piece encoding uses a WHITE bit or a BLACK bit (e.g. 32 and 64) set on the pieces, and then set both bits on the border guards. When the move generator calculates a target square to = from + step, and examines the board through victim = board[to], it just has to test the bit for the stm (if(victim & stm)...) to know if it hit a friendly piece or strayed off board, which are exactly the cases where the move is invalid.
Very nice, combining own piece and off the board test in one case. I thought for the blocking loop in capture generation it is better to test for a bit which is not set in the sentinel squares - but the boolean not operator is for free of course, only the inverse conditional jump.

Code: Select all

for (tosq = fromsq + increment; board[tosq] == EMPTY; tosq += increment);
if ( ! (board[tosq] & own_piece_bit ) ) // instead of if (board[tosq] & opponent_piece_bit ) 
{
  /* generate capture */
}
User avatar
hgm
Posts: 27794
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Recommended board representation for a rookie

Post by hgm »

Indeed, but this works because the while loop has excluded the possibility that the square would be EMPTY. For leaper captures-only it is more difficult, because you want to test purely for enemy pieces.

If you have bits to spare in the piece encoding, you can duplicate the WHITE and BLACK bits, and only set one of each pair in the border guards. When generating leaper captures you can then test for the enemy bit that is not set in the guards. The disadvantage is that piece lists will be wasting a lot of space this way.

An alternative is to encode the guard as the bit just above the color bits. Say EMPTY = 0, WHITE = 32, BLACK = 64, GUARD = 128. Then you would have:

00000000 empty
001nnnnn white pieces (use 32-62)
010nnnnn black pieces (use 64-94)
10000000 guard (128)

11000000 test mask for rejecting black any-moves
10100000 test mask for rejecting white any-moves
00100000 test mask for accepting black captures-only
01000000 test mask for accepting white captures-only
11100000 test mask for rejecting Pawn non-captures

Usually border guards would not be used to index any piece tables, so you won't care that it is an outlier. Such tables need often to be indexed by empty squares, however. So it wastes about 33% space that codes 1-31 are not in use. This can be ameliorated by interleaving all piece tables with elements of the same data size with a stride of 63, and not using nnnnn = 31 for any piece. Then the EMPTY slot of the next table maps into the hole between the white and black pieces of the one before it. Then only the first table still has some unused elements in it. But you can probably find a table that would never need to be indexed by an empty square (e.g. capture codes), and make that the first one, chopping off the first 32 entries entirely.

For having efficient capture generation in a mailbox engine it would be much more important to eliminate the entire while(board[to] == EMPTY) loop, though. (I guess this should qualify as a non-rookie topic!) This can be done by maintaining an array viewDistance[square][direction] that stores the distance to the nearest obstacle in each of the 8 directions for all occopied board squares and the border guards adjacent to the board. (The latter only for those directions that lead back to the board.) Then you can get the to-square of slider captures in a single branch-free step, to = from + viewDistance[direction]*step[direction].

Update of this viewDistance[][] is surprisingly cheap, and can be done just before you start move generation. For the evacuated square sq you can do

Code: Select all

for&#40;d=0; d<4; d++) &#123;
  int upstream = sq + viewDistance&#91;sq&#93;&#91;d&#93;*step&#91;d&#93;;
  int downstream = sq - viewDistance&#91;sq&#93;&#91;d+4&#93;*step&#91;d&#93;;
  viewDistance&#91;upstream&#93;&#91;d+4&#93; += viewDistance&#91;sq&#93;&#91;d+4&#93;;
  viewDistance&#91;downstream&#93;&#91;d&#93; += viewDistance&#91;sq&#93;&#91;d&#93;;
&#125;
(where directions are numbered such that d+4 is the direction opposite to d), and for the UnMake the same with -= instead of +=. Because the large majority of moves in the tree are captures, an evacuated square is all you have to deal with, as the to-square was already blocked before the move by the capture victim, and will remain so.

For the comparatively rare non-captures you would need to account for the blocking, which is a bit more expensive, and does require a board scan, as the viewDistance did not hold any data on the previously empty square. So you would have to reconstruct the data for that square the hard way. (And save the old content, as it could still be relevant for UnMake of previous plies that evacuated it.) The cost would be comparable to generating captures of only a single Rook or Bishop through ray scans, however, as you would only have to scan in 4 directions to reach an occupied square, after which the viewDistance data found there will point you directly to the first obstacle in the opposite direction.

There is an alternative way for doing this blocking update, which becomes competitive for large boards that are close to empty (so that ray scans would usually continue all the way to the board edge). This would start at a border guard that looks in the direction of the newly occupied square (obtained from a table indexed by that square and the direction to be handled), and use the viewDistance info to hop from occupied square to occupied square, until you overshoot the newly occupied square. (On a nearly empty board the first try would likely already overshoot the target.) The last two squares visited would be the 'end stops' of the newly occupied square, and should get their viewDistance in the direction of the latter updated.

Code: Select all

for&#40;d=0; d<4; d++) &#123;
  int oldHop, v = step&#91;d&#93;;
  int hop = edge&#91;sq&#93;&#91;dir&#93;; // tabulated intersection of rays with border
  do &#123;
    oldHop = hop;
    hop += viewDistance&#91;hop&#93;&#91;d&#93;*v; // hop to next obstacle
  &#125; while&#40;baseStep&#91;sq - hop&#93; == v&#41;; // baseStep flips when we overshoot
  viewDistance&#91;sq&#93;&#91;d&#93; = viewDistance&#91;hop&#93;&#91;d+4&#93; = distance&#91;sq-hop&#93;; // update
  viewDistance&#91;sq&#93;&#91;d+4&#93; = viewDistance&#91;oldHop&#93;&#91;d&#93; = distance&#91;sq-oldHop&#93;;
&#125;
The UnMake is again simple, because then the table itself contains all info needed to do it. It is in fact the same code as given above for evacuating a square on MakeMove(). Followed by restoring the info on the square itself for all its directions. (As the distances easily fit in bytes, you can save and restore that as a single uint64.)
Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Re: Recommended board representation for a rookie

Post by Sven »

Gerd Isenberg wrote:
Sven Schüle wrote:
hgm wrote:So it would just be a 0x88 board with two ranks above and below it, all filled, just as the unused area of the 0x88 board, with border guards that make it obvious the square is off board to anything trying to go there.
With 0x88 I don't fill the unused area with border guards since I would need two tests, one for "0 <= square <= 127" and one for the presence of the border guard. Instead of that I only use the "is valid square" test ((square & 0x88) != 0).
For move generation purposes, you don't need to test for valid indices (beside assert of a debug version), since by design, the surrounded squares ensure a valid array index for all move direction increments starting from a valid on-the board index. While the ((square & 0x88) != 0) looks cheap, it is a somehow redundant, since you have to access the board if the 0x88 test passes a valid index in most cases anyway. So it seems smarter to sacrifice the 0x88 property and to add one additional off-the board case beside piece codes and empty square. See also the old posting 0x88 is not so smart by Christophe Théron. 16x16 is fine, 15x15 has the advantage that lsb of square index determines the square color ;-)
I was talking about original 0x88 using a 16x8 board without border guards. There the 0x88 test for valid square index is necessary. Any larger mailbox representation using border guards does not need that, of course, but that is no longer "0x88" for me. I don't see any real disadvantage of the 16x8 board, though, and it is definitely "fast enough" and simple to code for an engine in a beginner's state.
User avatar
hgm
Posts: 27794
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Recommended board representation for a rookie

Post by hgm »

Well, you need the extra test on the square number. Which from a performance point of view is probably indeed insignificant. But it is still redundant, and still extra code. Why make things unnecessarily complicated by putting in redundant tests, especially for a beginner?

Another reason I don't like the classical 0x88 test on the square number is that it only works by the accidental fact that the board has 8 files. For, say, a 10-wide board (making 20x8) it would suddenly not be a simple AND operation anymore. You would probably have to resort to some table lookup to tell you whether a square number is valid or not, e.g. from a sqr2file table, to see if the file number <= 8. In that case it is especially elegant that you use the board itself for testing if you are on it in the same operation that tells you whether, when you are indeed on it, if you like what occupies the target square.

For me, 0x88 just means you use double the number of files to make square-number differences unambiguous.
Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Re: Recommended board representation for a rookie

Post by Sven »

hgm wrote:Well, you need the extra test on the square number. Which from a performance point of view is probably indeed insignificant. But it is still redundant, and still extra code. Why make things unnecessarily complicated by putting in redundant tests, especially for a beginner?
The "extra code" is almost negligible, and it does not access memory.

Code: Select all

bool isValidSqrId&#40;SqrId s&#41;
&#123;
    return bool&#40;&#40;s & 0x88&#41; == 0&#41;;
&#125;

void generateStraightMoves&#40;Board const & b, SqrId from, MoveList & moveList&#41;
&#123;
    for &#40;uint i = 0; i < 4; i++) &#123;
        int j = straightOffset&#40;i&#41;;
        SqrId to = from + j;
        while &#40;isValidSqrId&#40;to&#41; && b.isEmpty&#40;to&#41;) &#123;
            generateOneMove&#40;moveList, b, from, to&#41;;
            to += j;
        &#125;
        if &#40;isValidSqrId&#40;to&#41; && b.color&#40;to&#41; == opponent&#40;b.side&#40;))) &#123;
            generateOneMove&#40;moveList, b, from, to&#41;;
        &#125;
    &#125;
&#125;

void generateDiagonalMoves&#40;Board const & b, SqrId from, MoveList & moveList&#41;
&#123;
    for &#40;uint i = 0; i < 4; i++) &#123;
        int j = diagonalOffset&#40;i&#41;;
        SqrId to = from + j;
        while &#40;isValidSqrId&#40;to&#41; && b.isEmpty&#40;to&#41;) &#123;
            generateOneMove&#40;moveList, b, from, to&#41;;
            to += j;
        &#125;
        if &#40;isValidSqrId&#40;to&#41; && b.color&#40;to&#41; == opponent&#40;b.side&#40;))) &#123;
            generateOneMove&#40;moveList, b, from, to&#41;;
        &#125;
    &#125;
&#125;
Hitting the border is done without accessing memory. In contrast to that, crossing an empty square needs an additional "0x88" calculation that does not access memory and is therefore cheap. I'm not sure whether this is really significantly slower than the "border guards" approach. Almost the same is true for knight/king/pawn moves where testing for a valid "to" square needs the additional "0x88" test but does not need an additional memory access. Also, for all piece types, the "border guard" approach at some point needs to test whether the target square is either empty or occupied by an enemy piece (which needs two comparisons or some very neat HGM tricks). The two comparisons occur at the end of the loop for a sliding piece and always for other pieces. I test for "valid square number" and "not a friendly piece", so it is one memory access and two comparisons as well.
hgm wrote:Another reason I don't like the classical 0x88 test on the square number is that it only works by the accidental fact that the board has 8 files. For, say, a 10-wide board (making 20x8) it would suddenly not be a simple AND operation anymore. You would probably have to resort to some table lookup to tell you whether a square number is valid or not, e.g. from a sqr2file table, to see if the file number <= 8. In that case it is especially elegant that you use the board itself for testing if you are on it in the same operation that tells you whether, when you are indeed on it, if you like what occupies the target square.
You are perfectly right for an engine that is designed to be aware of other chess variants. But for those engines which aren't, 0x88 is fine.
User avatar
hgm
Posts: 27794
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Recommended board representation for a rookie

Post by hgm »

Code: Select all

void SliderMoves &#40;GameState *b, int *stepList, Square from, MoveList *m&#41;
&#123;
  int step, ownOrGuard = b->sideToMove | GUARD;
  while&#40;&#40;step = *stepList++)) &#123;
    int to = from;
    while&#40;!&#40;b->board&#91;to += step&#93; & ownOrGuard&#41;) &#123;
      GenerateOneMove&#40;m, from, to&#41;;
      if&#40;b->board&#91;to&#93;) break;
    &#125;
  &#125;
&#125;
I usually like to keep captures separated from non-captures, which again adds a tiny bit of complexity:

Code: Select all

void SliderMoves &#40;GameState *b, int *stepList, Square from, MoveList *m&#41;
&#123;
  int step, ownOrGuard = b->sideToMove | GUARD;
  while&#40;&#40;step = *stepList++)) &#123;
    int to = from;
    while&#40;!&#40;b->board&#91;to += step&#93; & ownOrGuard&#41;) &#123;
      if&#40;b->board&#91;to&#93;) &#123;
        GenerateOneCapture&#40;m, from, to&#41;;
        break;
      &#125; else
        GenerateOneNonCapture&#40;m, from, to&#41;;
    &#125;
  &#125;
&#125;
User avatar
hgm
Posts: 27794
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Recommended board representation for a rookie

Post by hgm »

BTW, the logic

Code: Select all

void SliderMoves &#40;GameState *b, int *stepList, Square from, MoveList *m&#41;
&#123;
  int step, ownOrGuard = b->sideToMove | GUARD;
  while&#40;&#40;step = *stepList++)) &#123;
    int to = from;
    while&#40;&#40;b->board&#91;to += step&#93; ==EMPTY&#41; &#123;
      GenerateOneNonCapture&#40;m, from, to&#41;;
    &#125;
    if&#40;!&#40;b->board&#91;to&#93; & ownOrGuard&#41;
      GenerateOneCapture&#40;m, from, to&#41;;
  &#125;
&#125;
that your example uses would also be a fine way to keep captures and non-captures separated. I guess I am a bit biased against it because I often have to deal with limited-range sliders, where even empty squares are no guarantee that you can continue the loop. This also erases the difference between sliders and leapers, as leapers are simply sliders with a maximum range of one step.

Code: Select all

void AnyMoves &#40;GameState *b, int *stepAndRangeList, Square from, MoveList *m&#41;
&#123;
  int step, ownOrGuard = b->sideToMove | GUARD;
  while&#40;&#40;step = *stepAndRangeList++)) &#123;
    int range = *stepAndRangeList++; //piece description stored as &#40;step, range&#41; pairs
    int to = from; // start new ray
    do &#123;
      int victim = b->board&#91;to += step&#93;;
      if&#40;victim & ownOrGuard&#41; break;
      if&#40;victim == EMPTY&#41;
        GenerateOneNonCapture&#40;m, from, to&#41;;
      else &#123;
        GenerateOneCapture&#40;m, from, to&#41;;
        break;
      &#125;
   &#125; while&#40;--range&#41;;
&#125;
mjlef
Posts: 1494
Joined: Thu Mar 30, 2006 2:08 pm

Re: Recommended board representation for a rookie

Post by mjlef »

hgm wrote:
Sven Schüle wrote:With 0x88 I don't fill the unused area with border guards since I would need two tests, one for "0 <= square <= 127" and one for the presence of the border guard.
This is why I put two extra ranks above and below the board. There is no need to test for 0 <= square <= 127, because squares < 0 and > 127 will hit a border guard on those extra ranks. The border has to be wide enough to catch the piece with the largest stride, which in Chess is a Knight. So two ranks suffice.

Usually my piece encoding uses a WHITE bit or a BLACK bit (e.g. 32 and 64) set on the pieces, and then set both bits on the border guards. When the move generator calculates a target square to = from + step, and examines the board through victim = board[to], it just has to test the bit for the stm (if(victim & stm)...) to know if it hit a friendly piece or strayed off board, which are exactly the cases where the move is invalid.

Although a width of 15 is in principle enough to make the square-number difference unambiguous, it is very costly in terms of memory efficiency, as the unused 7 board files are just to small to do anything useful with them. Only the actual board needs boundary guards, all other board-size tables (PST, Zobrist keys) will only be accessed for on-board squares. So with 16x8 you could store a white PST in the left half, and the corresponding black PST in the right half, for perfect memory filling, rather than using only 8/15 of the memory. If you really want to use the lowest bit for easy square shade, it would be better to use 17x8. So that you can use 16/17 of the memory.

Having a unused part as least as large as the used part is also very useful for move encoding; you can use the square numbers in the unused part (the 'shadow board') to code for a move to the corresponding square of the real board together with some special effect (like promotion).
This is just the scheme I used in NOW. Setting border squares to have bits for both colors (I used 8 and 16 instead of 32 and 64, but it is the same idea). Once that is done, move generation is as simple as checking if a to position for a piece is not the same color (does not have this side's bit set). At the time it was faster on an 8088 than doing the test for edge then test for not the same color.
Gerd Isenberg
Posts: 2250
Joined: Wed Mar 08, 2006 8:47 pm
Location: Hattingen, Germany

Re: Recommended board representation for a rookie

Post by Gerd Isenberg »

The mailbox piece coding of Gambiet 8x by Win Rens is interesting as well:

Code: Select all

        7654 3210
king    x110 1111
queen   x000 1001
rook    x000 0101
bishop  x000 0011
knight  x011 0011
pawn    x001 0001
border  1100 0000
empty   0000 0000
only one Z80 asm shift left one instruction and four conditional jumps on zero, parity odd, minus and carry for five cases:

Code: Select all

LDA  A,&#40;SQUARE&#41;    ; get piece code to Accu &#40;A&#41;
SLA  A             ; A &#58;= A << 1
JP   Z, EMPTY      ; if ( A == 0 ) empty square 
JP   PO, OFFBOARD  ; else if ( parity&#40;A&#41; is odd ) off the board
JP   M, KING       ; else if ( A < 0 ) occupied by king
JP   C, BLACK      ; else if ( carry ) occupied by black piece
JP   WHITE         ; else occupied by white piece
I think this trick is no longer recommend for recent processors...