Magic Move Generation

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

User avatar
cms271828
Posts: 316
Joined: Wed Apr 12, 2006 10:47 pm

Magic Move Generation

Post by cms271828 »

Hi, I was looking at this pdf..
http://www.prism.gatech.edu/~gtg365v/Bu ... oards.pdf

Essentially at the code on page 6...

Code: Select all

return moveDB[square][((occ & mask[square]) * magic[square]) >> (64-s)]
And the Data for square A1(ROOK), where magic=0x1234567890123456 and bits=10.

So.. with a rook on a1, there are 128 possible occupancies above it, and 128 to the right of it, so 16384 possible occupancies with a rook fixed on A1.

I tested each of each of...

Code: Select all

((occ & mask[square]) * magic[square]) >> (64-s)
for all 16384 occupancies, but there was a clash, on the position with pieces on a1,a8, and the position with rook on a1.

Also I don't understand why the ROOK MASK on page 6, doesn't go to the edge of the board, but stops 1 short, same for BISHOP MASK.

Any ideas? Thanks
Colin
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Magic Move Generation

Post by bob »

cms271828 wrote:Hi, I was looking at this pdf..
http://www.prism.gatech.edu/~gtg365v/Bu ... oards.pdf

Essentially at the code on page 6...

Code: Select all

return moveDB[square][((occ & mask[square]) * magic[square]) >> (64-s)]
And the Data for square A1(ROOK), where magic=0x1234567890123456 and bits=10.

So.. with a rook on a1, there are 128 possible occupancies above it, and 128 to the right of it, so 16384 possible occupancies with a rook fixed on A1.

I tested each of each of...

Code: Select all

((occ & mask[square]) * magic[square]) >> (64-s)
for all 16384 occupancies, but there was a clash, on the position with pieces on a1,a8, and the position with rook on a1.

Also I don't understand why the ROOK MASK on page 6, doesn't go to the edge of the board, but stops 1 short, same for BISHOP MASK.

Any ideas? Thanks
The mask idea came from the rotated bitboard stuff. The end squares do not matter as to whether they are occupied or not. If they are occupied, the rook can capture the piece on the edge square, if they are empty, the rook can just move to that square. So it turns out the edge squares are not needed, and omitting them saves some memory in the table lookups...
Pradu
Posts: 287
Joined: Sat Mar 11, 2006 3:19 am
Location: Atlanta, GA

Re: Magic Move Generation

Post by Pradu »

cms271828 wrote: And the Data for square A1(ROOK), where magic=0x1234567890123456 and bits=10.
I wrote this wrong. 0x1234567890123456 is just a placeholder for a table that never got finished. Try 0x0080001020400080 for A1 bits=12.
User avatar
cms271828
Posts: 316
Joined: Wed Apr 12, 2006 10:47 pm

Re: Magic Move Generation

Post by cms271828 »

Thanks...
I just figured out different occupancies will give the same value below..

Code: Select all

((occ & mask[square]) * magic[square]) >> (64-s)
It really just depends on how far the closest piece to A1 is, on the file and the rank.

I was about to write a message saying the old magic number still doesn't work, but I see you corrected it already, I tried it, and it appears to work, but I haven't tested it thoroughly yet.

Now I just need the other 63 for rook, and 64 for bishop :)

You think Its worth, me trying to work it out myself? I understand how the Magic array works for converting 2^n into n, since when you multiply by 2^n, you just shift the magic number, and its still a debruijn sequence.

But here, you're multiplying by a number made from 1 - 15 bits.
This results in a 64-bit number which isn't a debruijn sequence, but the first 12 bits don't seem to appear anywhere else.

I'm not clear on how it works precisely, it seems significantly harder than converting 2^n into n.

Any suggestions on how I proceed? Thanks
Colin
Gerd Isenberg
Posts: 2250
Joined: Wed Mar 08, 2006 8:47 pm
Location: Hattingen, Germany

Re: Magic Move Generation

Post by Gerd Isenberg »

cms271828 wrote: Any suggestions on how I proceed? Thanks
Search your magics on your own - it is fun and gives you a unique engine ;-)

There are several brute force approaches.

The inner loop (likely inside a boolean routine) tries one magic/shift of one square - it traverses all subsets of the set of all relevant blockers of that ray - and looks if any negative collisions occurs:

Code: Select all

bool testMagicBishops(long magic2Try, int shift2Try, int square)
{
   int count = 0;
   int collision = 0;
   Initialize bookholdingAttacks[maxsize] with -1 
   long oset = conventionalBishopAttacks(square, 0) & 0x007E7E7E7E7E7E00;
   long occ = 0; // first empty subset
   do
   {  // for all subsets
      index   = (magic2Try * b) >> shift2Try; // the magic hashing function
      attacks = conventionalBishopAttacks(square, occ);
      if ( bookholdingAttacks[index] == -1 )
      {  // free slot, store attacks
         bookholdingAttacks[index] = attacks;
         count++;
      }
      else if ( bookholdingAttacks[index] == attack )
      {
         collision++; // different occupancies map same attack, fine
      }
      else
      {
         return false; // negative collision, fails
      }
      occ = (occ-oset) & oset; // next subset
   }
   while( occ != 0);
   // print magic, shift, square, count and collisions
   return true; // ok, magic, shift uniqly maps all relevant occupancies
}
I like snoob, same number of one bits, for magics with 4 to 8 or 56..60 bit sets.
http://chessprogramming.wikispaces.com/ ... 20a%20Set4
You may try random numbers as well or xor them with some constant, like a De Bruijn sequence or whatever else.

Code: Select all

bool tryShiftBishop(int shift2Try, int square, int nBits4magic)
{
   long x, y, first;
   first = &#40;1 << nBits4magic&#41; - 1;
   for &#40;x = first; &#40;y = snoob&#40;x&#41;) > x; x = y&#41;
      if ( testMagicBishops ( x, shift, square&#41; ) return true; 
   return false;
&#125;
Finally you may traverse over shitf amounts, the higher the shift the denser the range per square. For bishops there are index ranges of only four bits.

Code: Select all

bool findMagicBishop&#40;int square, int nBits4magic, int rangeBits&#41;
&#123;
   for &#40;int s = 64 - rangeBits; s > 64-12; s--)
      if ( tryShiftBishop&#40;s, int square, nBits4magic&#41; ) return true;

   return false;
&#125;
41KByte bishops is a matter of seconds, the more denser you like the more time it takes - unless somebody finds a better algorithm.

Gerd
Gerd Isenberg
Posts: 2250
Joined: Wed Mar 08, 2006 8:47 pm
Location: Hattingen, Germany

Re: Magic Move Generation

Post by Gerd Isenberg »

cms271828 wrote: Any suggestions on how I proceed? Thanks
Search your magics on your own - it is fun and gives you a unique engine ;-)

There are several brute force approaches.

The inner loop (likely inside a boolean routine) tries one magic/shift of one square - it traverses all subsets of the set of all relevant blockers of that ray - and looks if any negative collisions occurs:

Code: Select all

bool testMagicBishops&#40;long magic2Try, int shift2Try, int square&#41;
&#123;
   int count = 0;
   int collision = 0;
   Initialize bookholdingAttacks&#91;maxsize&#93; with -1 
   long oset = conventionalBishopAttacks&#40;square, 0&#41; & 0x007E7E7E7E7E7E00;
   long occ = 0; // first empty subset
   do
   &#123;  // for all subsets
      index   = &#40;magic2Try * occ&#41; >> shift2Try; // the magic hashing function
      attacks = conventionalBishopAttacks&#40;square, occ&#41;;
      if ( bookholdingAttacks&#91;index&#93; == -1 )
      &#123;  // free slot, store attacks
         bookholdingAttacks&#91;index&#93; = attacks;
         count++;
      &#125;
      else if ( bookholdingAttacks&#91;index&#93; == attack )
      &#123;
         collision++; // different occupancies map same attack, fine
      &#125;
      else
      &#123;
         return false; // negative collision, fails
      &#125;
      occ = &#40;occ-oset&#41; & oset; // next subset
   &#125;
   while&#40; occ != 0&#41;;
   // print magic, shift, square, count and collisions
   return true; // ok, magic, shift uniqly maps all relevant occupancies
&#125;
I like snoob, same number of one bits, for magics with 4 to 8 or 56..60 bit sets.
http://chessprogramming.wikispaces.com/ ... 20a%20Set4
You may try random numbers as well or xor them with some constant, like a De Bruijn sequence or whatever else.

Code: Select all

bool tryShiftBishop&#40;int shift2Try, int square, int nBits4magic&#41;
&#123;
   long x, y, first;
   first = &#40;1 << nBits4magic&#41; - 1;
   for &#40;x = first; &#40;y = snoob&#40;x&#41;) > x; x = y&#41;
      if ( testMagicBishops ( x, shift, square&#41; ) return true; 
   return false;
&#125;
Finally you may traverse over shitf amounts, the higher the shift the denser the range per square. For bishops there are index ranges of only four bits.

Code: Select all

bool findMagicBishop&#40;int square, int nBits4magic, int rangeBits&#41;
&#123;
   for &#40;int s = 64 - rangeBits; s > 64-12; s--)
      if ( tryShiftBishop&#40;s, int square, nBits4magic&#41; ) return true;
   return false;
&#125;
41KByte bishops is a matter of seconds, the more denser you like the more time it takes - unless somebody finds a better algorithm.

Gerd
User avatar
cms271828
Posts: 316
Joined: Wed Apr 12, 2006 10:47 pm

Re: Magic Move Generation

Post by cms271828 »

Thanks, I read the other paper by Pradyumna, I understand it except for the collision thing.

I want to just look at the rook for now, starting on a1.
I have a magic=0x0080001020400080
with s=12 for that square, which works.

But I want to try and generate my own magic for it.
I guess the mask would be a2-a7,b1-g1, which means (occ & mask) has 2^12 possible values.

But I'm not sure how to go about finding the magic bits by trial and error.
Colin
Pradu
Posts: 287
Joined: Sat Mar 11, 2006 3:19 am
Location: Atlanta, GA

Re: Magic Move Generation

Post by Pradu »

cms271828 wrote:Thanks, I read the other paper by Pradyumna, I understand it except for the collision thing.

I want to just look at the rook for now, starting on a1.
I have a magic=0x0080001020400080
with s=12 for that square, which works.

But I want to try and generate my own magic for it.
I guess the mask would be a2-a7,b1-g1, which means (occ & mask) has 2^12 possible values.
The trial error method in that paper is only useful if you want to find all possible magics for a square or if you want to prove that a magic for a certain number of bits doesn't exist. Practically, good magics can be generated by trying out random numbers and by chance some of them will work.

The trial error method works marvelous for bishops and you should try it out for that first (IIRC magics for Bishop on D4 can be "proven" in less than a second). It takes very long for rooks (unless you find some clever way to reduce the search tree 8-)) but I have done it for 6-bits on A1 in a few minutes before.
But I'm not sure how to go about finding the magic bits by trial and error.
Which part.
User avatar
cms271828
Posts: 316
Joined: Wed Apr 12, 2006 10:47 pm

Re: Magic Move Generation

Post by cms271828 »

I don't quite get any part of how to generate the magics.

For the rook on a1, won't there be 49 possibilities in total(7 x 7).
so thats 6 bits required( Which is optimal )

With 12 bits like I was using before, thats 2^12 = 4096.
And for performance, I would rather have 64 array entries on a1 than 4096.

Is it possible to do every square (rooks + bishops) with 6 bits?

I guess I need to read Listing 8, on page 10.
Does this method generate optimal magic values?

Thanks
Colin
Pradu
Posts: 287
Joined: Sat Mar 11, 2006 3:19 am
Location: Atlanta, GA

Re: Magic Move Generation

Post by Pradu »

cms271828 wrote:I don't quite get any part of how to generate the magics.
That's helpful 8-). Would you like to discuss it by email (praduk @t gmail.com)?
For the rook on a1, won't there be 49 possibilities in total(7 x 7).
so thats 6 bits required( Which is optimal
Yep.
With 12 bits like I was using before, thats 2^12 = 4096.
And for performance, I would rather have 64 array entries on a1 than 4096.
Yes, that would be nice.
Is it possible to do every square (rooks + bishops) with 6 bits?
No. I found for Rook on A1 it is impossible to do it with 6-bits. You can find optimal magics fairly quickly for several bishop squares though. I guess the next step is to try 7 bits and 8 bits for rooks but they will take a bit longer. We'd probably have to come up with some more clever ways to get more "cutoffs" before we can actually start attempting larger number of bits in the index.
I guess I need to read Listing 8, on page 10.
Does this method generate optimal magic values?

Thanks
Listing 8 determines whether indexes can be computed given the bits guessed already. If you can compute an index you can start testing for collisions. If you can't, then you are going to have to guess some more bits. There are probably (hopefully) more clever ways to do it; this one only checks for carry effects on the upper bits.