Using EGTB

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Tommi

Using EGTB

Post by Tommi » Fri May 23, 2008 4:25 pm

I'm considering to use Nalimov tablebases in
my program. Is there any guide that would help
in implementation? Also it would help if there
would be some statistics about the percentages of
won game if specific sort of endgames.

Tommi

Tommi

Re: Using EGTB

Post by Tommi » Fri May 23, 2008 6:35 pm

There was supposed to be an article called probe.txt about this subject on Robert Hyatts home pages, but I didn't find it. Perhaps some one else has this article or could at least post a link.

User avatar
beachknight
Posts: 3533
Joined: Tue Jan 09, 2007 7:33 pm
Location: Antalya, Turkey
Contact:

Re: Using EGTB

Post by beachknight » Fri May 23, 2008 6:41 pm

Do you mean probe.c?

If yes, you can check the source of Crafty 19.20.

Best,
hi, merhaba, hallo HT

Pradu
Posts: 287
Joined: Sat Mar 11, 2006 2:19 am
Location: Atlanta, GA
Contact:

Re: Using EGTB

Post by Pradu » Fri May 23, 2008 6:43 pm

Tommi wrote:There was supposed to be an article called probe.txt about this subject on Robert Hyatts home pages, but I didn't find it. Perhaps some one else has this article or could at least post a link.
I likewise cannot find it. I have a copy of the old 5-men Nalimov files (PM me your email and I can send it to you) but I cannot find the new 6-men ones with the new readmes.

Dann Corbit
Posts: 9846
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Using EGTB

Post by Dann Corbit » Fri May 23, 2008 6:59 pm

Dieter Buerssner wrote a really nice article on how to interface with Nalimov endgame tablebase files.

It's not hard to do that part. The really, really hard part is to get ahold of Eugene Nalimov and get permission to use them. I know people who have tried for years unsuccessfully to get permission.

Pradu
Posts: 287
Joined: Sat Mar 11, 2006 2:19 am
Location: Atlanta, GA
Contact:

Re: Using EGTB

Post by Pradu » Fri May 23, 2008 7:52 pm

Dann Corbit wrote:Dieter Buerssner wrote a really nice article on how to interface with Nalimov endgame tablebase files.

It's not hard to do that part. The really, really hard part is to get ahold of Eugene Nalimov and get permission to use them. I know people who have tried for years unsuccessfully to get permission.
I think he started giving out permission a while ago. I got permission two years after I sent the email. 8-) Do you have a link to the article by Buerssner? Is the article also valid for probing the new 6-men Nalimov tablebases?

Tommi

Re: Using EGTB

Post by Tommi » Sat May 24, 2008 1:36 pm

I have that permission already. If you know where that document is, please post a link.

Dann Corbit
Posts: 9846
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Using EGTB

Post by Dann Corbit » Tue May 27, 2008 6:31 pm


Dann Corbit
Posts: 9846
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Using EGTB

Post by Dann Corbit » Tue May 27, 2008 6:32 pm

Here is also a code sample Dieter sent to me:

Code: Select all

/* Anybody can use this. Originally written by 
   Dieter Buerssner, 2004. */ 
    
#include <stdlib.h> 
#include <stdio.h> 
#include <string.h> 
#include <ctype.h> 

/* We use the same here as Nalimov */ 
enum piece_type &#123;nothing, pawn, knight, bishop, rook, queen, king&#125;; 
enum color &#123;white, black, empty&#125;; 

#define SHORT_CASTLE_WHITE  1 
#define LONG_CASTLE_WHITE   2 
#define SHORT_CASTLE_BLACK  4 
#define LONG_CASTLE_BLACK   8 

#define TO64&#40;r,c&#41; &#40;8*&#40;r&#41;+&#40;c&#41;) 

typedef struct &#123; 
  int board&#91;8&#93;&#91;8&#93;; 
  int cboard&#91;8&#93;&#91;8&#93;; 
  int stm; 
  unsigned castling_state; 
  int eps; 
&#125; POSITION; 

#if 0 
#define MM_DEBUG&#40;string&#41; my_print&#40;DEBUG_INFO, "%s\n", string&#41; 
#else 
#define MM_DEBUG&#40;string&#41; 
#endif 

/* Nasty macro, with side effects and assuming the variable names.  */ 
#define SET_PIECE&#40;c, p&#41; \
   do &#123; \
     if &#40;col > 7&#41; &#123;            \
     printf&#40;"ERROR setboard c = %c, column outside of board\n", c&#41;; \
     return 1; \
     &#125; \
     pos->cboard&#91;row&#93;&#91;col&#93;=c;pos->board&#91;row&#93;&#91;col&#93;=p;col++;&#125; while&#40;0&#41;       


int setboard&#40;const char *str, char **ep, POSITION *pos&#41; 
&#123; 
  int col, row, c, c2, i, j, sawwk, sawbk, board_done; 
    
  for &#40;i=0; i<8; i++) 
    for &#40;j=0; j<8; j++) 
    &#123; 
      pos->board&#91;i&#93;&#91;j&#93; = nothing; 
      pos->cboard&#91;i&#93;&#91;j&#93; = empty; 
    &#125; 
  pos->castling_state = 0; 
  pos->eps = -1; 
  /* trim spaces */ 
  while &#40;isspace&#40;*str&#41;) 
    str++; 

  sawwk = sawbk = 0; /* didn't find kings yet */ 
  col = 0; 
  row = 7; 
  board_done = 0; 
  while (&#40;c = *str&#41; != '\0' && !board_done&#41; 
  &#123; 
    MM_DEBUG&#40;str&#41;; 
    switch &#40;c&#41; 
    &#123; 
      case '0'&#58; case '1'&#58; case '2'&#58; case '3'&#58; 
      case '4'&#58; case '5'&#58; case '6'&#58; case '7'&#58; case '8'&#58; 
   col += c - '0'; break; 
      case '/'&#58; /* We support missing number before / */ 
   if (--row < 0&#41; /* Swallow additional / sometimes found at the end 
                     of wrong FENs */ 
     board_done = 1; /* Actually incorrect FEN */ 
   col = 0; 
   break; 
      case 'p'&#58; SET_PIECE&#40;black,pawn&#41;; break; /* Should check for 1st/8th row */ 
      case 'P'&#58; SET_PIECE&#40;white,pawn&#41;; break; 
      case 'n'&#58; SET_PIECE&#40;black,knight&#41;; break; 
      case 'N'&#58; SET_PIECE&#40;white,knight&#41;; break; 
      case 'b'&#58; SET_PIECE&#40;black,bishop&#41;; break; 
      case 'B'&#58; SET_PIECE&#40;white,bishop&#41;; break; 
      case 'r'&#58; SET_PIECE&#40;black,rook&#41;; break; 
      case 'R'&#58; SET_PIECE&#40;white,rook&#41;; break; 
      case 'q'&#58; SET_PIECE&#40;black,queen&#41;; break; 
      case 'Q'&#58; SET_PIECE&#40;white,queen&#41;; break; 
      case 'k'&#58; SET_PIECE&#40;black,king&#41;; sawbk++; break; 
      case 'K'&#58; SET_PIECE&#40;white,king&#41;; sawwk++; break; 
      case ' '&#58; case '\t'&#58; /* We support all sorts of abbrev. */ 
   board_done = 1; 
   break; 
      default&#58; 
   &#123; 
     printf&#40;"ERROR setboard c = %c, unknown piece letter\n", c&#41;; 
     return 1; 
   &#125;    
    &#125; 
    str++; 
  &#125; 
  MM_DEBUG&#40;"after pos"); 
  if &#40;sawwk != 1 || sawbk != 1&#41; 
  &#123; 
    printf&#40;"ERROR setboard kings are not correct\n"); 
    return 1; 
  &#125; 
  while &#40;isspace&#40;*str&#41;) /* Actually only spaces allowed */ 
    str++; 
  MM_DEBUG&#40;"trimmed spaces"); 
  if (*str == 'w' || *str == 'W') 
  &#123; 
    pos->stm = 0; 
    str++; 
  &#125; 
  else if (*str == 'b' || *str == 'B') 
  &#123; 
    pos->stm = 1; 
    str++; 
  &#125; 
  else 
    return 1; /* report stm missing */ 
  MM_DEBUG&#40;"after color"); 
  /* We support short FENs without castling ep etc. */ 
  /* get castling status */ 
  while &#40;isspace&#40;*str&#41;) /* Actually no spaces allowed */ 
    str++; 
  switch (*str&#41; 
  &#123; 
    case 'K'&#58; pos->castling_state |= SHORT_CASTLE_WHITE; str++; break; 
    case 'Q'&#58; pos->castling_state |= LONG_CASTLE_WHITE; str++; break; 
    case 'k'&#58; pos->castling_state |= SHORT_CASTLE_BLACK; str++; break; 
    case 'q'&#58; pos->castling_state |= LONG_CASTLE_BLACK;; str++; break; 
    case '-'&#58; case 'n'&#58; str++; goto no_castling; 
  &#125; 
  MM_DEBUG&#40;"after castling 1"); 
  while &#40;isspace&#40;*str&#41;) /* Actually no spaces allowed */ 
    str++; 
  switch (*str&#41; 
  &#123; 
    case 'K'&#58; pos->castling_state |= SHORT_CASTLE_WHITE; str++; break; 
    case 'Q'&#58; pos->castling_state |= LONG_CASTLE_WHITE; str++; break; 
    case 'k'&#58; pos->castling_state |= SHORT_CASTLE_BLACK; str++; break; 
    case 'q'&#58; pos->castling_state |= LONG_CASTLE_BLACK;; str++; break; 
  &#125; 
  MM_DEBUG&#40;"after castling 2"); 
  while &#40;isspace&#40;*str&#41;) /* Actually no spaces allowed */ 
    str++; 
  switch (*str&#41; 
  &#123; 
    case 'K'&#58; pos->castling_state |= SHORT_CASTLE_WHITE; str++; break; 
    case 'Q'&#58; pos->castling_state |= LONG_CASTLE_WHITE; str++; break; 
    case 'k'&#58; pos->castling_state |= SHORT_CASTLE_BLACK; str++; break; 
    case 'q'&#58; pos->castling_state |= LONG_CASTLE_BLACK;; str++; break; 
  &#125; 
  MM_DEBUG&#40;"after castling 3"); 
  while &#40;isspace&#40;*str&#41;) /* Actually no spaces allowed */ 
    str++; 
  switch (*str&#41; 
  &#123; 
    case 'K'&#58; pos->castling_state |= SHORT_CASTLE_WHITE; str++; break; 
    case 'Q'&#58; pos->castling_state |= LONG_CASTLE_WHITE; str++; break; 
    case 'k'&#58; pos->castling_state |= SHORT_CASTLE_BLACK; str++; break; 
    case 'q'&#58; pos->castling_state |= LONG_CASTLE_BLACK;; str++; break; 
  &#125; 
  MM_DEBUG&#40;"after castling 4"); 
no_castling&#58; 
  /* Check that castling state is in accordance with placement 
     of kings/rooks. */ 

  /* Caution, Omitted here */ 
  
  /* e.p. field */ 
  while &#40;isspace&#40;*str&#41;) 
    str++; 
  c = *str; 
  MM_DEBUG&#40;"trimmed spaces for ep"); 
  if &#40;c == 'n' || c == '-') /* support bogos "n" in some FENs */ 
    str++; 
  else 
  &#123; 
    if &#40;isupper&#40;c&#41;) /* Some compilers have broken tolower in case of 
                       none uppercase letter */ 
      c = tolower&#40;c&#41;;    
    if &#40;c >= 'a' && c <= 'h') 
    &#123; 
      c2 = *++str; 
      str++; 
      if (&#40;c2 == '3' && pos->stm == black&#41; || &#40;c2 == '6' && pos->stm == white&#41;) 
        pos->eps = &#40;c2 - '1') * 8 + c - 'a'; /* Assumes ASCII */ 
    &#125; 
  &#125; 
  MM_DEBUG&#40;"after ep"); 
  if &#40;isspace&#40;*str&#41;) /* Actually only spaces allowed */ 
  &#123; 
    while &#40;isspace&#40;*str&#41;) /* Actually only spaces allowed */ 
      str++; 
    /* 50 moves counter, ignore it */ 
    while &#40;isdigit&#40;*str&#41;) 
      str++; 
    if &#40;isspace&#40;*str&#41;) /* Actually only spaces allowed */ 
    &#123; 
      while &#40;isspace&#40;*str&#41;) /* Actually only spaces allowed */ 
        str++; 
      /* Move number, ignore it */ 
      while &#40;isdigit&#40;*str&#41;) 
        str++; 
    &#125; 
  &#125; 
  if &#40;ep&#41; /* set end of string */ 
    *ep = &#40;char *&#41;str; 
  MM_DEBUG&#40;"assignement ep"); 
  
  /* Caution, Omitted here */ 

  /* Check, that we have a consistent ep square. All of it omitted. 
     Only one check actually done, and correct FEN assumed otherwise. 
     Note that this check really is needed here, because the Standard 
     for FEN wants a set ep target, even when ep is not possible. 
     Nalimov code however will do strange things sometimes when ep 
     is set, but an epmove is not possible. 
     So, without changes, this code will not correct even with 
     correct FENs in some cases. User should not give the ep-flag, 
     in case ep is not possible. 
     */ 

  /* Do all sort of checks on the position, all omitted here */ 
  /* if &#40;verify_position&#40;pos, &etext&#41; == 0&#41; ... */ 
  
  return 0; 
&#125; 

/* From start of tbindex.cpp. */ 

#define pageL 65536 /* 256 */ 

/* tablebase byte entry semispan length */ 

#define tbbe_ssL (&#40;pageL - 4&#41; / 2&#41; 

/* tablebase signed byte entry values */ 

#define bev_broken  &#40;tbbe_ssL + 1&#41;  /* illegal or busted */ 
#if defined (_MSC_VER&#41; 
#define   TB_FASTCALL   __fastcall 
#else 
#define   TB_FASTCALL 
#endif 

#define  T_INDEX64 
#define  XX  127 
#define  C_PIECES  3    /* Maximum # of pieces of one color OTB */ 

#if defined &#40;T_INDEX64&#41; && defined (_MSC_VER&#41; 
typedef unsigned __int64 INDEX; 
#elif defined &#40;T_INDEX64&#41; 
typedef unsigned long long INDEX; 
#else 
typedef unsigned long INDEX; 
#endif 

typedef unsigned int square; 

typedef INDEX &#40;TB_FASTCALL * PfnCalcIndex&#41; 
      &#40;square*, square*, square, int fInverse&#41;; 

extern int IDescFindFromCounters &#40;int*); 
extern int FRegisteredFun &#40;int, /*color*/int&#41;; 
#define FRegistered FRegisteredFun 
extern PfnCalcIndex PfnIndCalcFun &#40;int, /*color*/int&#41;; 
#define PfnIndCalc PfnIndCalcFun 
extern int TB_FASTCALL L_TbtProbeTable &#40;int, /*color*/int, INDEX&#41;; 
extern int IInitializeTb&#40;char *); 
extern int FTbSetCacheSize&#40;void   *, unsigned long&#41;; 
extern int cbEGTBCompBytes; 


size_t egtb_cache_size = 2L*1024L*1024L; 

int table_pieces=0; 
static void *cache_buf = 0; 

void set_egtb_cache&#40;long siz&#41; 
&#123; 
  egtb_cache_size = siz; 
  if&#40;table_pieces > 0&#41; 
  &#123; 
    if &#40;cache_buf&#41; 
    &#123; 
      free&#40;cache_buf&#41;; 
      cache_buf = 0; 
    &#125; 
    cache_buf = malloc&#40;egtb_cache_size&#41;; 
    if &#40;cache_buf && FTbSetCacheSize&#40;cache_buf, egtb_cache_size&#41;) 
      printf&#40;"Using %.3f MB cache for TBs\n", egtb_cache_size/&#40;1024. * 1024.)); 
    else 
    &#123; 
      table_pieces = 0; 
      if &#40;cache_buf&#41; 
      &#123; 
         free&#40;cache_buf&#41;; 
         cache_buf = 0; 
      &#125; 
    &#125; 
  &#125; 
&#125; 

/* Should use const char *, but Eugene does not ... */ 
void init_egtb&#40;char *path&#41; 
&#123; 
  if &#40;cache_buf&#41; 
  &#123; 
    free&#40;cache_buf&#41;; 
    cache_buf = 0; 
  &#125; 
  table_pieces = IInitializeTb&#40;path&#41;; 
  if&#40;table_pieces != 0&#41; 
  &#123; 
    printf&#40;"%d-men endgame table bases found, %.3f MB used for decompression tables\n", 
          table_pieces, cbEGTBCompBytes/&#40;1024. * 1024.)); 
    set_egtb_cache&#40;egtb_cache_size&#41;; 
  &#125; 
  else 
    printf&#40;"No endgame tables found\n"); 
&#125; 

#define MYPIECE_TO_NALIMOV&#40;p&#41; &#40;p&#41; /* We use the same internal piece type */ 
#define ROWCOL_TO_64&#40;r,c&#41; (&#40;r&#41;*8+&#40;c&#41;) /* Nalimov uses a1=0, b1=1, ... h8=63 */ 

int probe_egtb&#40;POSITION *pos, int *score&#41; 
&#123; 
  int piece_count&#91;10&#93;; 
  square white_squares&#91;5*C_PIECES+1&#93;, black_squares&#91;5*C_PIECES+1&#93;; 
  int idx_tb, side, invert, value, ep, i, j, pc, sq, nwp, nbp; 
  INDEX idx; 
  square *wp, *bp; 

  if &#40;pos->castling_state&#41; 
    return 0; 
  /* Setup arrays white_pieces and black_pieces, so that they will 
     work with original (=Crafty&#41; SqFind...() macros. Set also up 
     piece_counts */ 
  /* Note, that a real chess engine would probably have a piece list, 
     and the following code would be more efficiently done. 
     Also number of white and black pieces will already be known. */ 
  for &#40;i=0; i<10; i++) 
    piece_count&#91;i&#93; = 0; 
  for &#40;i=0; i<=5*C_PIECES; i++) 
    white_squares&#91;i&#93; = black_squares&#91;i&#93; = 0; 
  nwp = nbp = 0; 
  for &#40;i=0; i<8; i++) 
  &#123; 
    for &#40;j=0; j<8; j++) 
    &#123; 
      if &#40;pos->cboard&#91;i&#93;&#91;j&#93; == white&#41; 
      &#123; 
        nwp++; 
        if &#40;nwp > C_PIECES+1&#41; /* Avoid out of bounds index */ 
          return 0; 
        pc = MYPIECE_TO_NALIMOV&#40;pos->board&#91;i&#93;&#91;j&#93;); 
        sq = ROWCOL_TO_64&#40;i,j&#41;; 
        pc = pos->board&#91;i&#93;&#91;j&#93;; 
        if &#40;pc == king&#41; 
          white_squares&#91;5*C_PIECES&#93; = sq; 
        else 
          white_squares&#91;&#40;pc-1&#41; * C_PIECES + piece_count&#91;pc-1&#93;++&#93; = sq; 
      &#125; 
      else if &#40;pos->cboard&#91;i&#93;&#91;j&#93; == black&#41; 
      &#123; 
        nbp++; 
        if &#40;nbp > C_PIECES+1&#41; /* Avoid out of bounds index */ 
          return 0; 
        pc = MYPIECE_TO_NALIMOV&#40;pos->board&#91;i&#93;&#91;j&#93;); 
        sq = ROWCOL_TO_64&#40;i,j&#41;; 
        pc = pos->board&#91;i&#93;&#91;j&#93;; 
        if &#40;pc == king&#41; 
          black_squares&#91;5*C_PIECES&#93; = sq; 
        else 
          black_squares&#91;&#40;pc-1&#41; * C_PIECES + piece_count&#91;pc-1+5&#93;++&#93; = sq; 
      &#125; 
    &#125; 
  &#125;    
  
  if &#40;nwp + nbp > table_pieces&#41; /* We have more pieces on the board, 
                                   than found in any TB */ 
    return 0; 

  /* Everything is setup here, now call the Nalimov functions 
     to do the real work for the actual probing. */ 
  idx_tb = IDescFindFromCounters&#40;piece_count&#41;; 
  if &#40;idx_tb == 0&#41; 
    return 0; 
  else if &#40;idx_tb > 0&#41; 
  &#123; 
    side = pos->stm; 
    invert = 0; 
    wp = white_squares; 
    bp = black_squares; 
  &#125; 
  else 
  &#123; 
    side = pos->stm^1; 
    invert = 1; 
    wp = black_squares; 
    bp = white_squares; 
    idx_tb = -idx_tb; 
  &#125; 

  if&#40;!FRegisteredFun&#40;idx_tb, side&#41;) 
    return 0; 
  ep = &#40;pos->eps > 0&#41; ? pos->eps &#58; XX; /* Careful, see comment in setboard 
                                     Note that pos uses "Nalimov squares" for 
                                     ep already */ 
  idx = PfnIndCalcFun&#40;idx_tb, side&#41;&#40;wp, bp, ep, invert&#41;; 
  value = L_TbtProbeTable&#40;idx_tb, side, idx&#41;; 
  if&#40;value == bev_broken&#41; 
    return 0; 
  *score = value; 
  return 1; 
&#125; 

/* 0 = draw; 1 = mate in one; -1 = mated; -2 = mated in 1; -3 = mated in 2 */ 
#define MATE_FROM_TB_SCORE&#40;score&#41; \
  ((&#40;score&#41; > 0&#41; ? 32767-&#40;score&#41; &#58; (&#40;score&#41; == 0 ? 0 &#58; -(&#40;score&#41;+32766&#41;-1&#41;) 

int main&#40;void&#41; 
&#123; 
  char buf&#91;256&#93;; 
  POSITION pos; 
  int v; 
  
  init_egtb&#40;"c&#58;/tb;c&#58;/6men"); /* Hardcoded ... */ 
  while&#40;fgets&#40;buf, sizeof buf, stdin&#41;) 
  &#123; 
    if &#40;setboard&#40;buf, NULL, &pos&#41; != 0&#41; 
      break; /* for example quit ... */ 
    if &#40;probe_egtb&#40;&pos, &v&#41; == 0&#41; 
      printf&#40;"pos not found in TB\n"); 
    else 
    &#123; 
      printf&#40;"TB returned %d which means ", v&#41;; 
      v = MATE_FROM_TB_SCORE&#40;v&#41;; 
      if &#40;v == 0&#41; 
        printf&#40;"draw\n"); 
      else if &#40;v > 0&#41; 
        printf&#40;"mate in %d\n", v&#41;; 
      else 
        printf&#40;"mated in %d\n", -v-1&#41;; 
    &#125; 
  &#125; 
  return 0; 
&#125; 

Pradu
Posts: 287
Joined: Sat Mar 11, 2006 2:19 am
Location: Atlanta, GA
Contact:

Re: Using EGTB

Post by Pradu » Tue May 27, 2008 7:02 pm

Dann Corbit wrote:Here is Dieter's article:
http://www.open-aurec.com/wbforum/viewt ... 4158&t=931
This is an example of how to use Nalimov's probing code. I'm looking for the original 6-man Nalimov code (I have the 5-man). I'm looking for a set of these original files for 6-man:

Code: Select all

tbcmp.cpp
tbgen.cpp
tbstat.cpp
tbdecode.h
tbindex.cpp
probe.txt
readme.txt
I've emailed Eugene and Andrew for it but they take a while to look at their emails. 8-)

Post Reply