Scorpio BBs, licenses

[Zach Wegner]

I have some questions. Teemu Pudas has implemented the Scorpio BBs into ZCT, and I'm having some issues. I had to find a way to compile shared objects with OS X, I'll show how if there's interest.

First, it seems there is no error checking. I downloaded the 3+4 egbbs, but not the 5's. If it does not find any files or only some, there is no way of knowing.

Second, the libraries seem to be thread-safe (with no option of turning it off), but not process-safe. I think it could be amended with some light hacking, but I don't know if I'll have the time to do it. I wonder if anyone has run into this problem, or anyone is interested in having someone (me?) fix it.

Third, when running some initial tests, there was a quick problem. I set my code to only probe with <= 4 pieces on the board, and I set up a position with 5 pieces:

Code: Select all

(zct)1. setboard kbb///////KN w - -
(zct)1. analyze
Depth          Time     Score   PV (nodes)
  1/ 1        0.000    +310.73   1. Ka2 (1)
[ 1/ 1]       0.000    +310.73   1. Ka2 (5)
  2/ 3        0.001    -310.73   1. Ka2 Ka7 (26)
[ 2/ 3]       0.001    -310.73   1. Ka2 Ka7 (34)
  3/ 6        0.003    -310.73   1. Ka2 Be6+ 2. Ka1 (140)
[ 3/ 6]       0.003    -310.73   1. Ka2 Be6+ 2. Ka1 (148)
  4/ 6        0.004    -310.73   1. Ka2 Be6+ 2. Ka1 Be5+ 3. Nc3 (253)
[ 4/ 6]       0.004    -310.73   1. Ka2 Be6+ 2. Ka1 Be5+ 3. Nc3 (261)
  5/ 7        0.005     +0.00   1. Ka2 (396)
[ 5/ 7]       0.005     +0.00   1. Ka2 (400)
  6/ 7        0.006     +0.00   1. Ka2 (505)
[ 6/ 7]       0.006     +0.00   1. Ka2 (509)
  7/ 7        0.010     +0.00   1. Ka2 (808)
[ 7/ 7]       0.011     +0.00   1. Ka2 (812)
  8/10        0.032    -310.73   1. Ka2 Be5 2. Nc3 Be6+ 3. Ka1 (2267)
  8/10        0.052    -310.72   1. Kb2 Be5+ (3221)
[ 8/10]       0.058    -310.72   1. Kb2 Be5+ (3567)
  9/12        0.073     +0.00   1. Kb2 (6162)
[ 9/12]       0.073     +0.00   1. Kb2 (6166)
 10/12        0.099    -310.73   1. Kb2 Be5+ 2. Nc3 Be6 3. Ka1 Bh2 4. Kb1 (8311)
[10/12]       0.135    -310.73   1. Kb2 Be5+ 2. Nc3 Be6 3. Ka1 Bh2 4. Kb1 (10848)

At this point, it segfaults. Using GDB I find the problem here:

Code: Select all

[10/12]       0.193    -310.73   1. Kb2 Be5+ 2. Nc3 Be6 3. Ka1 Bh2 4. Kb1 (10848)

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x4496051b
0x01008ff4 in EGBB&#58;&#58;decode &#40;this=0x1280380, in_table=0x4496051b <Address 0x4496051b out of bounds>, out_table=0x12a3ee4 "", size=805960900&#41; at egbbdll.cpp&#58;512
512                     HUFFMAN_DECODE&#40;huffman,v&#41;;
&#40;gdb&#41; up
#1  0x01009eb0 in EGBB&#58;&#58;get_score &#40;this=0x12a3ee4, index=360463, psearcher=0x1291ff0&#41; at egbbdll.cpp&#58;582
582                             block_size = decode&#40;&table&#91;psearcher->info.start_index&#93;,psearcher->info.block,block_size&#41;;
Current language&#58;  auto; currently c++
&#40;gdb&#41; 
#2  0x0100ad14 in SEARCHER&#58;&#58;get_score &#40;this=0x1291ff0, alpha=-1, beta=1, side=0, w_king=0, b_king=40, piece1=10, square1=19399552, piece2=10, square2=58, piece3=0, square3=-1&#41; at egbbdll.cpp&#58;1406
1406                    return pegbb->get_score&#40;pos_index,this&#41;;

I don't really have time to try to find the problem here, but the problem seems to be in the square1 parameter to get_score. Has anyone seen this type of behavior before? Is it just an implementation bug?

And finally, what license are the bitbases released under? I am planning to release my engine under a license similar to but a bit more restrictive than the GPL (I'll start a thread about it soon).

[Harald Johnsen]

The SEARCHER::get_score function will probe the bb if they are available, if you only have one side to move bitbases it will generate the moves and do the probing for the other side (there is a mini alpha beta there).

Now if you have the 2 stm bitbases installed than the parameters we see in the trace are the parameters you gave to the probe_egbb function.

I think that the bb code is public domain since nothing is said in the source files.


HJ.

[Dann Corbit]

The SEARCHER::get_score function will probe the bb if they are available, if you only have one side to move bitbases it will generate the moves and do the probing for the other side (there is a mini alpha beta there).

Now if you have the 2 stm bitbases installed than the parameters we see in the trace are the parameters you gave to the probe_egbb function.

I think that the bb code is public domain since nothing is said in the source files.


HJ.

Lack of copyright notice does not place something in public domain. You still own the copyright unless you explicitly donate it.

I think that Daniel Shawul has clearly designed the bitbase files to be used by other projects, but you should get clarification from him on the license.

[Denis P. Mendoza]

what license are the bitbases released under

I found this license text with the bitbase code.

Copyright (c) 2005, Daniel Shawul
All rights reserved.

Redistribution and use in source and binary forms,
with or without modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* It is illegal to endorse or promote products derived from this software without
specific prior written permission from Daniel Shawul.


THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

And the readme...

Thanks
------
Dann Corbit
Gerhard Schwager
Oliver Deuville
Gunther Simon
Leo Dijksman
Peter Eizenhimmer
Ed Shroeder
and all Winboard forum fellows!

BOOK
----
Salvo Spiltarie is the book author for scorpio.
Thanks also goes to Oliver Deuville for the book which
earlier version use.
Personality
-----------
scorpio supports multi personality for its opening, middle game
and endgame. You can also use one personality for the whole of the
game.
eg. personality mid
or
multi_personality opn mid end
EGBB
----
Scorpio uses its own endgame bitbases upto 5 pieces.
4 piece egbbs are 6.4mb in size and comes along with scorpio
pack. The total 5 piece take 1.6GB. Compression of egbbs is a
work in progress..

Installation
------------
Dont forget to set the egbb_path and egbb_cache_size
in scorpio.ini. Suggested cache size for 4 piece egbbs is
minimum 4mb, and fro 5 piece minimum 16mb.

Backward compatibility
----------------------
The new bitbases also work with engines that supported the old ones.
so you dont need to change anything unless

1. You want to use the 5 piece bitbases

2. You want to probe the bitbases in internal nodes also.
In the old ones this was impossible as most of kxxk types of bitbases
were absent. Now all of them are available so you can cutoff inside tree.
There are many ways to implement this , here i will explain how scorpio uses them.
Since bitbase only contain Win/Draw/Loss informaion we can not stop search when we get
to a won endgame . We have to make sure of progress first. Scorpio doesnot probe in the
first half of the search. This solves the problem of not making progress. We can also probe
the bitbases in the first half of the search when the move is a capture/pawn move.
This moves zero the fifty move count and we can assume that we are making progress.

/*
Interior node recognizer
-> dont probe at the leaves as this will slow down search
For 4/3 pieces probe there also.
-> After captures and pawn moves assume exact score and cutoff tree,
because we are making progress. Note this is not done only for speed.
-> if we are far from root (depth / 2), assume exact score and cutoff tree
*/
if( egbb_is_loaded
&& all_man_c <= 5
&& (all_man_c <= 4 || search_depth - ply >= (search_depth / 3))
&& (ply >= (search_depth / 2)
|| is_cap_prom((pstack - 1)->current_move)
|| PIECE(m_piece((pstack - 1)->current_move)) == pawn
)
&& probe_bitbases(score)
) {
pstack->best_score = score;
return true;
}

How to use egbbdll
==================
First take a look at probe.cpp. It is self explanatory i guess.
I will give some general ideas here.
egbbdll.dll has to be loaded and we have to get the address of
probe_egbb and load_egbb functions. New functions are provided for 5men
egbbs , probe_egbb_5men and load_egbb_5men. So if you want to use 5men
load_egbb = (PLOAD_EGBB) GetProcAddress(hmod,"load_egbb_5men");
probe_egbb = (PPROBE_EGBB) GetProcAddress(hmod,"probe_egbb_5men");
Then we can call these functions like any other standard functions.


typedef int (*PLOAD_EGBB) (char* path,int cache_size,int load_options);

Parameters:
path => it is the directory where the bitbases are located
cache_size => size of cache used in bytes
load_options => options to load some bitbase in to RAM. By default the
4 men are loaded in to RAM.
Returns:
true if loading succeeds false otherwise.

typedef int (*PPROBE_EGBB) (int player, int w_king, int b_king,
int piece1, int square1,
int piece2, int square2,
int piece3, int square3);
Parameters:
player => side to move
w_king => square of white king (remeber to use A1 = 0 ... H8 = 63)
b_king => square of black king
piece1 => type of piece
square1 => square of piece1 etc...
Returns:
0 => draw
>0 => win
<0 => loss
_NOTFOUND => bitbase is not included

Scores are modified according to many factors.
score = WIN_SCORE
- 10 * distance(SQ6488(w_king),SQ6488(b_king))
+ (5 - all_c) * 1000;
This type of scoring helps in making progress.
->3men endings are prefered over 5men.
->Cornering the losing king is a good idea
Special endings like kbnk are also taken care of.

[Dann Corbit]

what license are the bitbases released under

I found this license text with the bitbase code.

Copyright (c) 2005, Daniel Shawul
All rights reserved.

Redistribution and use in source and binary forms,
with or without modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* It is illegal to endorse or promote products derived from this software without
specific prior written permission from Daniel Shawul.


THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

And the readme...

Thanks
------
Dann Corbit
Gerhard Schwager
Oliver Deuville
Gunther Simon
Leo Dijksman
Peter Eizenhimmer
Ed Shroeder
and all Winboard forum fellows!

BOOK
----
Salvo Spiltarie is the book author for scorpio.
Thanks also goes to Oliver Deuville for the book which
earlier version use.
Personality
-----------
scorpio supports multi personality for its opening, middle game
and endgame. You can also use one personality for the whole of the
game.
eg. personality mid
or
multi_personality opn mid end
EGBB
----
Scorpio uses its own endgame bitbases upto 5 pieces.
4 piece egbbs are 6.4mb in size and comes along with scorpio
pack. The total 5 piece take 1.6GB. Compression of egbbs is a
work in progress..

Installation
------------
Dont forget to set the egbb_path and egbb_cache_size
in scorpio.ini. Suggested cache size for 4 piece egbbs is
minimum 4mb, and fro 5 piece minimum 16mb.

Backward compatibility
----------------------
The new bitbases also work with engines that supported the old ones.
so you dont need to change anything unless

1. You want to use the 5 piece bitbases

2. You want to probe the bitbases in internal nodes also.
In the old ones this was impossible as most of kxxk types of bitbases
were absent. Now all of them are available so you can cutoff inside tree.
There are many ways to implement this , here i will explain how scorpio uses them.
Since bitbase only contain Win/Draw/Loss informaion we can not stop search when we get
to a won endgame . We have to make sure of progress first. Scorpio doesnot probe in the
first half of the search. This solves the problem of not making progress. We can also probe
the bitbases in the first half of the search when the move is a capture/pawn move.
This moves zero the fifty move count and we can assume that we are making progress.

/*
Interior node recognizer
-> dont probe at the leaves as this will slow down search
For 4/3 pieces probe there also.
-> After captures and pawn moves assume exact score and cutoff tree,
because we are making progress. Note this is not done only for speed.
-> if we are far from root (depth / 2), assume exact score and cutoff tree
*/
if( egbb_is_loaded
&& all_man_c <= 5
&& (all_man_c <= 4 || search_depth - ply >= (search_depth / 3))
&& (ply >= (search_depth / 2)
|| is_cap_prom((pstack - 1)->current_move)
|| PIECE(m_piece((pstack - 1)->current_move)) == pawn
)
&& probe_bitbases(score)
) {
pstack->best_score = score;
return true;
}

How to use egbbdll
==================
First take a look at probe.cpp. It is self explanatory i guess.
I will give some general ideas here.
egbbdll.dll has to be loaded and we have to get the address of
probe_egbb and load_egbb functions. New functions are provided for 5men
egbbs , probe_egbb_5men and load_egbb_5men. So if you want to use 5men
load_egbb = (PLOAD_EGBB) GetProcAddress(hmod,"load_egbb_5men");
probe_egbb = (PPROBE_EGBB) GetProcAddress(hmod,"probe_egbb_5men");
Then we can call these functions like any other standard functions.


typedef int (*PLOAD_EGBB) (char* path,int cache_size,int load_options);

Parameters:
path => it is the directory where the bitbases are located
cache_size => size of cache used in bytes
load_options => options to load some bitbase in to RAM. By default the
4 men are loaded in to RAM.
Returns:
true if loading succeeds false otherwise.

typedef int (*PPROBE_EGBB) (int player, int w_king, int b_king,
int piece1, int square1,
int piece2, int square2,
int piece3, int square3);
Parameters:
player => side to move
w_king => square of white king (remeber to use A1 = 0 ... H8 = 63)
b_king => square of black king
piece1 => type of piece
square1 => square of piece1 etc...
Returns:
0 => draw
>0 => win
<0 => loss
_NOTFOUND => bitbase is not included

Scores are modified according to many factors.
score = WIN_SCORE
- 10 * distance(SQ6488(w_king),SQ6488(b_king))
+ (5 - all_c) * 1000;
This type of scoring helps in making progress.
->3men endings are prefered over 5men.
->Cornering the losing king is a good idea
Special endings like kbnk are also taken care of.

That is a Berkeley style license.