Project help required: Bitboard Fruit 2.1

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

ZirconiumX
Posts: 1334
Joined: Sun Jul 17, 2011 11:14 am

Re: Project help required: Bitboard Fruit 2.1

Post by ZirconiumX »

It would appear that I have buggered the quiet piece move generator, so that only pawn quiet moves, captures and promotions are generated.

But I practically followed DC's implementation to the letter:

DC:

Code: Select all

move_t *gen_piece_moves(const Board *B, uint64_t targets, move_t *mlist, bool king_moves)
/* Generates piece moves, when the board is not in check. Uses targets to filter the tss, eg.
 * targets = ~friends (all moves), empty (quiet moves only), enemies (captures only). */
{
	assert(!king_moves || !board_is_check(B));	// do not use when in check (use gen_evasion)
	const int us = B->turn;
	assert(!(targets & B->all[us]));
	uint64_t fss;

	// Knight Moves
	fss = B->b[us][Knight];
	while (fss) {
		int fsq = next_bit(&fss);
		uint64_t tss = NAttacks[fsq] & targets;
		while (tss) {
			int tsq = next_bit(&tss);
			mlist = serialize_moves(B, fsq, tsq, mlist);
		}
	}

	// Rook Queen moves
	fss = get_RQ(B, us);
	while (fss) {
		int fsq = next_bit(&fss);
		uint64_t tss = targets & rook_attack(fsq, B->st->occ);
		while (tss) {
			int tsq = next_bit(&tss);
			mlist = serialize_moves(B, fsq, tsq, mlist);
		}
	}

	// Bishop Queen moves
	fss = get_BQ(B, us);
	while (fss) {
		int fsq = next_bit(&fss);
		uint64_t tss = targets & bishop_attack(fsq, B->st->occ);
		while (tss) {
			int tsq = next_bit(&tss);
			mlist = serialize_moves(B, fsq, tsq, mlist);
		}
	}

	// King moves (king_moves == false is only used for check escapes)
	if (king_moves) {
		int fsq = B->king_pos[us];
		// here we also filter direct self checks, which shouldn't be sent to serial_moves
		uint64_t tss = KAttacks[fsq] & targets & ~B->st->attacks;
		while (tss) {
			int tsq = next_bit(&tss);
			mlist = serialize_moves(B, fsq, tsq, mlist);
		}
	}

	return mlist;
}
Fruit:

Code: Select all

// add_quiet_moves()

static void add_quiet_moves(list_t * list, const board_t * board) {

   int me;
   const sq_t * ptr;
   int from, to;

   uint64 piece_bb;

   uint64 attacks;

   uint64 me_bb;

   ASSERT(list!=NULL);
   ASSERT(board!=NULL);

   me = board->turn;
   me_bb = COLOUR_IS_WHITE(me) ? WHITE_BB : BLACK_BB;

   // piece moves
   
   // knight

   piece_bb = KNIGHT_BB & me_bb;

   while (piece_bb) {
	   from = next_bit(&piece_bb);
	   attacks = knight_attacks&#40;1ULL << from&#41; & EMPTY_BB;
	   while &#40;attacks&#41; &#123;
		   to = next_bit&#40;&attacks&#41;;
		   LIST_ADD&#40;list,MOVE_MAKE&#40;from,to&#41;);
	   &#125;
   &#125;

   // bishop and queen

   piece_bb = &#40;BISHOP_BB | QUEEN_BB&#41; & me_bb;

   while &#40;piece_bb&#41; &#123;
	   from = next_bit&#40;&piece_bb&#41;;
	   attacks = bishop_attacks&#40;from,OCC_BB&#41; & EMPTY_BB;
	   while &#40;attacks&#41; &#123;
		   to = next_bit&#40;&attacks&#41;;
		   LIST_ADD&#40;list,MOVE_MAKE&#40;from,to&#41;);
	   &#125;
   &#125;

   // rook and queen

   piece_bb = &#40;ROOK_BB | QUEEN_BB&#41; & me_bb;

   while &#40;piece_bb&#41; &#123;
	   from = next_bit&#40;&piece_bb&#41;;
	   attacks = rook_attacks&#40;from,OCC_BB&#41; & EMPTY_BB;
	   while &#40;attacks&#41; &#123;
		   to = next_bit&#40;&attacks&#41;;
		   LIST_ADD&#40;list,MOVE_MAKE&#40;from,to&#41;);
	   &#125;
   &#125;

   // king

   piece_bb = KING_BB & me_bb;

   while &#40;piece_bb&#41; &#123;
	   from = next_bit&#40;&piece_bb&#41;;
	   attacks = king_attacks&#40;1ULL << from&#41; & EMPTY_BB;
	   while &#40;attacks&#41; &#123;
		   to = next_bit&#40;&attacks&#41;;
		   LIST_ADD&#40;list,MOVE_MAKE&#40;from,to&#41;);
	   &#125;
   &#125;

/**** THIS ISN'T MINE - THIS IS FABIEN'S CODE ****/

   // pawn moves

   if &#40;COLOUR_IS_WHITE&#40;me&#41;) &#123;

      for &#40;ptr = &board->pawn&#91;me&#93;&#91;0&#93;; &#40;from=*ptr&#41; != SquareNone; ptr++) &#123;

         // non promotes

         if &#40;SQUARE_RANK&#40;from&#41; != Rank7&#41; &#123;
            to = from + 16;
            if &#40;board->square&#91;to&#93; == Empty&#41; &#123;
               ASSERT&#40;!SQUARE_IS_PROMOTE&#40;to&#41;);
               LIST_ADD&#40;list,MOVE_MAKE&#40;from,to&#41;);
               if &#40;SQUARE_RANK&#40;from&#41; == Rank2&#41; &#123;
                  to = from + 32;
                  if &#40;board->square&#91;to&#93; == Empty&#41; &#123;
                     ASSERT&#40;!SQUARE_IS_PROMOTE&#40;to&#41;);
                     LIST_ADD&#40;list,MOVE_MAKE&#40;from,to&#41;);
                  &#125;
               &#125;
            &#125;
         &#125;
      &#125;

   &#125; else &#123; // black

      for &#40;ptr = &board->pawn&#91;me&#93;&#91;0&#93;; &#40;from=*ptr&#41; != SquareNone; ptr++) &#123;

         // non promotes

         if &#40;SQUARE_RANK&#40;from&#41; != Rank2&#41; &#123;
            to = from - 16;
            if &#40;board->square&#91;to&#93; == Empty&#41; &#123;
               ASSERT&#40;!SQUARE_IS_PROMOTE&#40;to&#41;);
               LIST_ADD&#40;list,MOVE_MAKE&#40;from,to&#41;);
               if &#40;SQUARE_RANK&#40;from&#41; == Rank7&#41; &#123;
                  to = from - 32;
                  if &#40;board->square&#91;to&#93; == Empty&#41; &#123;
                     ASSERT&#40;!SQUARE_IS_PROMOTE&#40;to&#41;);
                     LIST_ADD&#40;list,MOVE_MAKE&#40;from,to&#41;);
                  &#125;
               &#125;
            &#125;
         &#125;
      &#125;
   &#125;
&#125;
So what am I doing wrong?

I ran the test again between Fruit 2.1 and Fruit 2.1 Bitboard. 10- 0= 0+

Matthew:out
Some believe in the almighty dollar.

I believe in the almighty printf statement.
User avatar
hgm
Posts: 27787
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Project help required: Bitboard Fruit 2.1

Post by hgm »

Well, if your perft(2) is 320 in stead of 400, it should not be so difficult to figure out which moves you are missing. Just let it print all moves you generate at the 2-ply level, then you will see which moves are missing.

It would be a lot harder if only perft(6) fell 1 short...
User avatar
lucasart
Posts: 3232
Joined: Mon May 31, 2010 1:29 pm
Full name: lucasart

Re: Project help required: Bitboard Fruit 2.1

Post by lucasart »

ZirconiumX wrote:It would appear that I have buggered the quiet piece move generator, so that only pawn quiet moves, captures and promotions are generated.

But I practically followed DC's implementation to the letter:
Here is the protocol I follow, to give you an idea:

1. non functional modification

1.1. move generation or board playing code, low level bitboard machinery and the likes. run a perft verification on the following positions
http://chessprogramming.wikispaces.com/Perft+Results
all the way to the last depth indicated in the page and all positions. the slightest difference indicates a bug, and must be fixed before attempting anything else. You can use DC to help you debug your perft problems. Let's say for eg that your perft(3) is wrong in the start position, then run DC as follows:

Code: Select all

lucas@lucas-desktop&#58;~/Chess/Engines$ ./doublecheck_3.4.1
uci
id name DoubleCheck 3.4
id author Lucas Braesch
option name Hash type spin default 32 min 1 max 8192
option name Verbose type check default true
uciok
ucinewgame
position startpos
perft 3
Nb1a3	400
Nb1c3	440
Ng1f3	440
Ng1h3	400
a2a3	380
b2b3	420
c2c3	420
d2d3	539
e2e3	599
f2f3	380
g2g3	420
h2h3	380
a2a4	420
b2b4	421
c2c4	441
d2d4	560
e2e4	600
f2f4	401
g2g4	421
h2h4	420
time&#40;ms&#41; = 0
perft&#40;3&#41; = 8902
DC displays perft(N) but also all the root moves with their perft(N-1). If you code the same funtion you can compare line by line and debug recusrively. For eg after c2c4 you have sth different to 441, then repeat the operation on a perft(2) in the fen reached after c2c4 etc. Finding bugs this way is very quick and painless, so this should help you a lot and it's worth writing a proper perft function that displays the perft(N-1).

1.2 other non functional modification

run a ./DC bench and compare the nodes. The time also validates alleged speed improvements. Always use a profiler to guide you before doing useless optimizations that mess up your code for no good reason.

2. other modifications

The mere fact that your modification reaches fater depth 13 in the start position or whatever doesn't mean anything. Even the fact that it scores better in tactical EPD test suites, isn't relevant either (although a version that scores better in thpose and the same in self play should be preferred).

The only way is self testing with lots of games. Typically I run 1000 very fast games. Often the modification preserves the speed in NPS (as verified by the ./DC bench) and so I can run these 1000 games in a fixed node limit (typically 50000 or 100000 nodes).

To summarize:
* write a ./FruitFly bench similar to SF's benchmark function
* write a proper perft function like DC
* follow the protocol to the letter.

I know it sounds painful and tedious, but it's the only way forward, and the time you take doing that now will be reimbursed hundred folds, you'll see.

PS: Also put assert at the entry point of every function to validate the correctness of its arguments, and embed into such assert-checked functions even the most trivial operations
Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Re: Project help required: Bitboard Fruit 2.1

Post by Sven »

ZirconiumX wrote:It would appear that I have buggered the quiet piece move generator, so that only pawn quiet moves, captures and promotions are generated.
[...]
So what am I doing wrong?
Hi Matthew,

after some quick testing I can confirm that no quiet non-pawn moves are generated in the last version you published.

It seems the problem is not in the move generator. Your piece bitboards (at least these) seem to keep a value of 0x0, and I strongly believe that the root of this problem is in your bitboard related additions to board_from_fen() in fen.cpp. Please have a close look to it.

I am also missing any update of the OCC and EMPTY bitboards in board_from_fen(), maybe I have overlooked something.

How did you define the orientation of squares in your bitboards? From the order in board_from_fen() it seems as if A8=0, H8=7, ..., A1=56, H1=63 but I am not sure about the rest of the bitboard code.

Sven
ZirconiumX
Posts: 1334
Joined: Sun Jul 17, 2011 11:14 am

Re: Project help required: Bitboard Fruit 2.1

Post by ZirconiumX »

My main problem is that I don't have much of a choice in the matter...

I can either:
  • Update the bitboards at the same time as the mailbox array is updated - which is what I am doing at the moment.
  • Poll the board twice - which is slower than what I want considering most go commands follow a position command.
  • Something else - which is awkward because the A8-H1 mapping (LSR(?)) is how FEN works...
I have been attempting to use LSF mapping (a1=0, h8=63), but as stated, FEN seems to use LSR (which I think means I XOR by 56 (?))

Matthew:out
Some believe in the almighty dollar.

I believe in the almighty printf statement.
User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: Project help required: Bitboard Fruit 2.1

Post by Evert »

ZirconiumX wrote:Something else - which is awkward because the A8-H1 mapping (LSR(?)) is how FEN works...
I would not base my internal encoding based on how FEN encoding works...

That said, whether you use A1-H8 or A8-H1 mapping (or H8-A1, or something completely different again) should be entirely irrelevant as long as you're consistent...

Personally I use A1-H8, which makes the most intuitive sense to me. The downside is that if you print out the numbers in the most direct way the board is flipped from the way you normally look at it.
ZirconiumX
Posts: 1334
Joined: Sun Jul 17, 2011 11:14 am

Re: Project help required: Bitboard Fruit 2.1

Post by ZirconiumX »

I have done some hacking and changed the FEN to bitboard thingy to this:

Code: Select all

...
   // piece placement

   for &#40;rank = Rank8, rank8 = 8; rank >= Rank1 && rank8 >= 1; rank--, rank8--) &#123; // <---

      for &#40;file = FileA, file8 = 1; file <= FileH && file8 <= 8;) &#123; // <---

         if &#40;c >= '1' && c <= '8') &#123; // empty square&#40;s&#41;

            len = c - '0';

            for &#40;i = 0; i < len; i++) &#123;
               if &#40;file > FileH&#41; my_fatal&#40;"board_from_fen&#40;)&#58; bad FEN &#40;pos=%d&#41;\n",pos&#41;;
               board->square&#91;SQUARE_MAKE&#40;file,rank&#41;&#93; = Empty;
               file++;
	       file8++; // <---
            &#125;

         &#125; else &#123; // piece

            piece = piece_from_char&#40;c&#41;;
            if &#40;piece == PieceNone256&#41; my_fatal&#40;"board_from_fen&#40;)&#58; bad FEN &#40;pos=%d&#41;\n",pos&#41;;

            board->square&#91;SQUARE_MAKE&#40;file,rank&#41;&#93; = piece;
            file++;
	    file8++; // <---

	    board->bitboards&#91;PIECE_COLOUR&#40;piece&#41;&#93; ^= 1 << &#40;rank8 * file8&#41;; // <---
	    board->bitboards&#91;PIECE_BB&#40;PIECE_TO_12&#40;piece&#41;)&#93; ^= 1 << &#40;rank8 * file8&#41;; // <---
         &#125;

         c = fen&#91;++pos&#93;;
      &#125;

      if &#40;rank > Rank1&#41; &#123;
         if &#40;c != '/') my_fatal&#40;"board_from_fen&#40;)&#58; bad FEN &#40;pos=%d&#41;\n",pos&#41;;
         c = fen&#91;++pos&#93;;
      &#125;
   &#125;

   // bitboards // <---
   
   OCC_BB = PAWN_BB | KNIGHT_BB | BISHOP_BB | ROOK_BB | QUEEN_BB | KING_BB; // <---
   EMPTY_BB = ~OCC_BB; // <---
...
This still does not help.

I think I will call on the print_bb() function in magic.cpp...

Matthew:out
Some believe in the almighty dollar.

I believe in the almighty printf statement.
ZirconiumX
Posts: 1334
Joined: Sun Jul 17, 2011 11:14 am

Re: Project help required: Bitboard Fruit 2.1

Post by ZirconiumX »

After some more quick code I get this:

Code: Select all

White
 X X X . . . . .
 . X X X . . . .
 . . . X . . . .
 . . . . . . . .
 . X . X . X X .
 . . . . . . X X
 . . . X . . X X
 . . . . . . . X
Black
 X X X . . . . .
 . X X X . . . .
 . . . X . . . .
 . . . . . . . .
 . X . X . X X .
 . . . . . . X X
 . . . X . . X X
 . . . . . . . X
Pawns
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
Knights
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
Bishops
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
Rooks
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
Queens
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
Kings
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
Empty
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
Occupied
 X X X X X X X .
 X X X X X X X X
 X X X X X X X X
 X X X X X X X X
 X X X X X X X X
 X X X X X X X X
 X X X X X X X X
 X X X X X X X X
Which is seriously wrong.

I'm going to take a break - can't be on the computer forever.

Matthew:out
Some believe in the almighty dollar.

I believe in the almighty printf statement.
Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Re: Project help required: Bitboard Fruit 2.1

Post by Sven »

ZirconiumX wrote:My main problem is that I don't have much of a choice in the matter...

I can either:
  • Update the bitboards at the same time as the mailbox array is updated - which is what I am doing at the moment.
  • Poll the board twice - which is slower than what I want considering most go commands follow a position command.
  • Something else - which is awkward because the A8-H1 mapping (LSR(?)) is how FEN works...
I have been attempting to use LSF mapping (a1=0, h8=63), but as stated, FEN seems to use LSR (which I think means I XOR by 56 (?))

Matthew:out
Think straight forward. The original board_from_fen() function is built as follows, reduced to what is relevant here:

Code: Select all

void board_from_fen&#40;board_t * board, const char fen&#91;&#93;) &#123;
   board_clear&#40;board&#41;;
   // some initialization ...
   for &#40;rank = Rank8; rank >= Rank1; rank--) &#123;
      for &#40;file = FileA; file <= FileH;) &#123;
         if &#40;found digit N&#41; &#123;
            for &#40;i = 0; i < N; i++) &#123;
               set SQUARE_MAKE&#40;file, rank&#41; to EMPTY;
               file++;
            &#125;
         &#125; else &#123;
            add a piece on SQUARE_MAKE&#40;file, rank&#41;;
            file++;
         &#125;
      &#125;
   &#125;
&#125;
I would do exactly the same also with bitboards. Start by fixing the board_clear() implementation, this must leave the board in an empty state. All piece bitboards, colour bitboards, and your "occupied" bitboard must be zero, and your "empty" bitboard must have all bits set to 1. I think this is not the case with your current version - I downloaded yesterday last time. I even think there is a bug in the code initializing your bitboards in board_clear(), it looks like out-of-bound access.

Now, how to update the bitboards in board_from_fen(): you already have the square given by SQUARE_MAKE(file, rank), so I would simply take SQUARE_TO_64 of it as your bit position and perform the proper operation: when adding a piece, set the related bit in the piece bitboard, colour bitboard, and "occupied" bitboard, but clear the bit in the "empty" bitboard. When setting a square to EMPTY, clear the bit everywhere but set the bit in the "empty" bitboard. You HAVE to do it that way, since you use your bitboards like that everywhere else in your program (look at move_do.cpp for instance).

At the moment I mentioned move_do.cpp I realized that you also have to fix your bitboard update code there. You are using the 0..255 square in the "1ULL << square" expressions but you have to use "1ULL << SQUARE_TO_64(square)" instead. The result currently is that none of the moves has any effect on the board since all 0..255 square values are >= 64 (A1=68) so that shifting results in a "0".

Sven
Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Re: Project help required: Bitboard Fruit 2.1

Post by Sven »

ZirconiumX wrote:After some more quick code I get this:

Code: Select all

White
 X X X . . . . .
 . X X X . . . .
 . . . X . . . .
 . . . . . . . .
 . X . X . X X .
 . . . . . . X X
 . . . X . . X X
 . . . . . . . X
Black
 X X X . . . . .
 . X X X . . . .
 . . . X . . . .
 . . . . . . . .
 . X . X . X X .
 . . . . . . X X
 . . . X . . X X
 . . . . . . . X
Pawns
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
Knights
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
Bishops
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
Rooks
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
Queens
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
Kings
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
Empty
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
 . . . . . . . .
Occupied
 X X X X X X X .
 X X X X X X X X
 X X X X X X X X
 X X X X X X X X
 X X X X X X X X
 X X X X X X X X
 X X X X X X X X
 X X X X X X X X
Which is seriously wrong.

I'm going to take a break - can't be on the computer forever.

Matthew:out
Don't do "some more quick code". It is a waste of time. When returning from your break, I suggest that you start over from scratch, in your case from original Fruit 2.1. Now put in the code step by step, following all the advice that has been given here (I especially recommend what Lucas wrote), and test everything until it works, including "perft". Don't add 200 lines when you did not test the first three lines.

For your bitboards in the board data structure, I suggest that you use something like this:

uint64 bb_colour[2];
uint64 bb_piece[6];
uint64 bb_occ;
uint64 bb_empty;

instead of the array of 10 entries. I think it will be easier to debug.

As to your other mail that I only saw after sending my previous post, I don't think this improves your version of board_from_fen().

Sven