Opening books format

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
mhalstern
Posts: 484
Joined: Wed Nov 18, 2009 12:09 am

Re: Opening books format

Post by mhalstern » Sun Dec 27, 2009 8:40 pm

Thanks for all of the code and links.

Has anybody written a program to convert to and from the chessbase *.ctg format?

If so, pls post and or link.

Stephan Vermeire (Brutus)
Posts: 34
Joined: Sun Oct 12, 2008 4:32 pm

Re: Opening books format

Post by Stephan Vermeire (Brutus) » Mon Dec 28, 2009 4:08 pm

Hi Marc,

CTG-files have a completely different structure than .pgn files. With CTG, each unique position is stored regardless of how we came there. In pgn a specific sequence of moves is stored. That makes it harder to convert thse formats than other formats.

As far as I know, there is no program available to convert ctg. Check this also:
http://www.open-aurec.com/wbforum/viewt ... tg#p190852

Best wishes,
Stephan

User avatar
jshriver
Posts: 1105
Joined: Wed Mar 08, 2006 8:41 pm
Location: Morgantown, WV, USA
Contact:

Re: Opening books format

Post by jshriver » Mon Dec 28, 2009 10:03 pm

I'd be willing to write one pending abk isn't patented or will get me sued :)
Will have to research it later to see what the format looks like.

Any other formats people want dumped to PGN?

-Josh

Michel
Posts: 2057
Joined: Sun Sep 28, 2008 11:50 pm

Re: Opening books format

Post by Michel » Tue Dec 29, 2009 5:55 am

Any other formats people want dumped to PGN?
Dumping to PGN is not the only issue. One also needs to know how Arena assigns probabilities to moves. I asked a while ago here if it was known how Arena does this but there was no reply. So I guess it is not known.

Stephan Vermeire (Brutus)
Posts: 34
Joined: Sun Oct 12, 2008 4:32 pm

Re: Opening books format

Post by Stephan Vermeire (Brutus) » Tue Dec 29, 2009 8:11 am

jshriver wrote:I'd be willing to write one pending abk isn't patented or will get me sued :)
Will have to research it later to see what the format looks like.

-Josh
Hi Joshua,

In case you want to write a ctg->pgn adapter, I can give you some code that is capable of probing ctg-databases. It won't be too hard to alter that into a converter.

I have written an object with the following functions:

Code: Select all

class book{
public:
	book();
	void setBookPath(char* newBookPath);
	int probeBook(char* pFEN, char* pMove);
	void setRandomness(int rnd); //With this you can set: best move only or one of the best moves
                .....
};
When you enter a FEN ("rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"), the move is returned in SAN-format (like e2e4). With this code you can probe specific positions and retrieve all moves and statistics that have been stored in the database. This link is very useful also: http://rybkaforum.net/cgi-bin/rybkaforu ... ?pid=26942

Let me know if you are interested in this code. I haven't published it due too legal reasons (don't want to be sued either :wink:)

Stephan

yoshiharu
Posts: 56
Joined: Sat Nov 11, 2006 10:14 pm

Re: Opening books format

Post by yoshiharu » Thu Dec 31, 2009 8:35 pm

Hi everybody,

I made a few corrections to the source, because of two small bugs.
Now it goes like the following:

Code: Select all

// Author Juri Osipov - mods by Dann Corbit, Lance Perkins - couple of bugs fixed by Mauro Riccardi
#include <windows.h> 
#include <stdio.h> 
#include <stdlib.h> 
#include <io.h> 

static const char prm&#91;&#93; = "RNBQRNBQ"; 

struct BOOK &#123; 
    unsigned char   move_from; 
    unsigned char   move_to; 
    unsigned char   move_promo; 
    unsigned char   priority; 
    unsigned int    games; 
    unsigned int    won_games; 
    unsigned int    lost_games; 
    unsigned int    hz; 
    int             first_child; 
    int             next_sibling; 
&#125;   *book; 

FILE           *ft; 
int             linelength = 0; 

void            print_head&#40;) 
&#123; 
    fprintf&#40;ft, "&#91;Event \"?\"&#93;\n"); 
    fprintf&#40;ft, "&#91;Site \"?\"&#93;\n"); 
    fprintf&#40;ft, "&#91;Date \"????.??.??\"&#93;\n"); 
    fprintf&#40;ft, "&#91;Round \"?\"&#93;\n"); 
    fprintf&#40;ft, "&#91;Result \"*\"&#93;\n"); 
    fprintf&#40;ft, "&#91;White \"?\"&#93;\n"); 
    fprintf&#40;ft, "&#91;Black \"?\"&#93;\n\n"); 
&#125; 


void print_move&#40;unsigned char move_from, unsigned char move_to,char move_promo, int ply&#41; 
&#123; 
    if (&#40;move_from >> 3&#41; + 1 == 32 && &#40;move_to >> 3&#41; + 1 == 32&#41; return; 
    if &#40;linelength > 80&#41; &#123; 
        fprintf&#40;ft, "\n"); 
        linelength = 0; 
    &#125; 
    if (&#40;ply & 1&#41; == 0&#41; &#123; 
        fprintf&#40;ft, "%d.", ply / 2 + 1&#41;; 
        linelength += 3; 
    &#125; 
    fprintf&#40;ft, "%c%d%c%d", 
            &#40;move_from & 7&#41; + 'a', &#40;move_from >> 3&#41; + 1, 
            &#40;move_to & 7&#41; + 'a', &#40;move_to >> 3&#41; + 1&#41;; 

    if &#40;move_promo&#41; &#123;  // from Lance Perkins        
        fprintf&#40;ft,"%c", prm&#91;abs&#40;move_promo&#41;-1&#93;); 
    &#125; 
    fprintf&#40;ft, " "); 
    linelength += 5; 
&#125; 

int main&#40;int argc, char *argv&#91;&#93;) 
&#123; 
    int             i, 
                    ply; 
    size_t          filesize; 
    int             node&#91;1000&#93;; 
    struct MOVE &#123; 
        unsigned char   move_from; 
        unsigned char   move_to; 
        unsigned char   move_promo; 
    &#125;   moves&#91;1000&#93;; 

    if &#40;argc < 2&#41; &#123; 
        printf&#40;"Simple command-line converter from ABK to PGN\n"); 
        printf&#40;"Usage&#58; abk.exe <abk_file>\n"); 
        printf&#40;"Result PGN-file&#58; out.pgn\n"); 
        exit&#40;0&#41;; 
    &#125; 
    ft = fopen&#40;argv&#91;1&#93;, "rb"); 
    if &#40;ft == NULL&#41; &#123; 
        printf&#40;"Can't open file %s\n", argv&#91;1&#93;); 
        exit&#40;0&#41;; 
    &#125; 
    filesize = filelength&#40;fileno&#40;ft&#41;); 
    book = malloc&#40;filesize&#41;; 
    fread&#40;book, filesize / sizeof&#40;struct BOOK&#41;, sizeof&#40;struct BOOK&#41;, ft&#41;; 
    fclose&#40;ft&#41;; 
    ft = fopen&#40;"out.pgn", "w"); 
    print_head&#40;); 
    ply = 0; 
    node&#91;0&#93; = 900;  // offset to first node in abk-file 

    for (;;) &#123; 
        print_move&#40;book&#91;node&#91;ply&#93;&#93;.move_from, book&#91;node&#91;ply&#93;&#93;.move_to, 
                   book&#91;node&#91;ply&#93;&#93;.move_promo, ply&#41;; 
        if &#40;book&#91;node&#91;ply&#93;&#93;.first_child > 0&#41; &#123;  // current game 

            moves&#91;ply&#93;.move_from = book&#91;node&#91;ply&#93;&#93;.move_from; 
            moves&#91;ply&#93;.move_to = book&#91;node&#91;ply&#93;&#93;.move_to; 
            moves&#91;ply&#93;.move_promo = book&#91;node&#91;ply&#93;&#93;.move_promo; 
            node&#91;ply + 1&#93; = book&#91;node&#91;ply&#93;&#93;.first_child; 
            ply++; 
        &#125; else &#123; // new game 
            fprintf&#40;ft, "\n*\n\n"); 
            node&#91;ply&#93; = book&#91;node&#91;ply&#93;&#93;.next_sibling; 
            while &#40;node&#91;ply&#93; < 0&#41; &#123; 
                ply--; 
                if &#40;ply < 0&#41; &#123; 
                    fclose&#40;ft&#41;;                     
                    exit&#40;0&#41;; 
                &#125; 
                node&#91;ply&#93; = book&#91;node&#91;ply&#93;&#93;.next_sibling; 
            &#125; 
            print_head&#40;); 
            linelength = 0; 
            for &#40;i = 0; i < ply; i++) &#123; 
                print_move&#40;moves&#91;i&#93;.move_from, moves&#91;i&#93;.move_to, moves&#91;i&#93;.move_promo, i&#41;; 
            &#125; 
        &#125; 
        if &#40;node&#91;ply&#93; >= filesize / &#40;int&#41; sizeof&#40;struct BOOK&#41;) &#123; 
            printf&#40;"Error&#58; out of file\n"); 
            fclose&#40;ft&#41;; 
            exit&#40;0&#41;; 
        &#125; 
    &#125; 
    return 0; 
&#125; 

Cheers, Mauro

User avatar
jshriver
Posts: 1105
Joined: Wed Mar 08, 2006 8:41 pm
Location: Morgantown, WV, USA
Contact:

Re: Opening books format

Post by jshriver » Thu Dec 31, 2009 8:42 pm

I made some small mods too to get this to work under linux (any unix really).

I've never used #ifdef for win vs unix systems but here are the additions and changes.

Add:
#include <stdlib.h>
#include <sys/stat.h>
Remove windows.h and io.h

struct stat filecheck;

then removed the filesize line and replace with this:

stat(argv[1], &filecheck);
filesize = filecheck.st_size;

-Josh

Dann Corbit
Posts: 10207
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Opening books format

Post by Dann Corbit » Thu Dec 31, 2009 9:21 pm

Maybe:

Code: Select all

// Author Juri Osipov - mods by Dann Corbit, Lance Perkins - couple of bugs fixed by Mauro Riccardi
#ifdef _MSC_VER
#include <windows.h>
#include <io.h>
#else
#include <unistd.h>
#include <sys/stat.h>
static struct stat filecheck;
#endif
#include <stdio.h>
#include <stdlib.h>

static const char prm&#91;&#93; = "RNBQRNBQ";

struct BOOK &#123;
    unsigned char   move_from;
    unsigned char   move_to;
    unsigned char   move_promo;
    unsigned char   priority;
    unsigned int    games;
    unsigned int    won_games;
    unsigned int    lost_games;
    unsigned int    hz;
    int             first_child;
    int             next_sibling;
&#125;   *book;

FILE           *ft;
int             linelength = 0;

void            print_head&#40;)
&#123;
    fprintf&#40;ft, "&#91;Event \"?\"&#93;\n");
    fprintf&#40;ft, "&#91;Site \"?\"&#93;\n");
    fprintf&#40;ft, "&#91;Date \"????.??.??\"&#93;\n");
    fprintf&#40;ft, "&#91;Round \"?\"&#93;\n");
    fprintf&#40;ft, "&#91;Result \"*\"&#93;\n");
    fprintf&#40;ft, "&#91;White \"?\"&#93;\n");
    fprintf&#40;ft, "&#91;Black \"?\"&#93;\n\n");
&#125;


void print_move&#40;unsigned char move_from, unsigned char move_to,char move_promo, int ply&#41;
&#123;
    if (&#40;move_from >> 3&#41; + 1 == 32 && &#40;move_to >> 3&#41; + 1 == 32&#41; return;
    if &#40;linelength > 80&#41; &#123;
        fprintf&#40;ft, "\n");
        linelength = 0;
    &#125;
    if (&#40;ply & 1&#41; == 0&#41; &#123;
        fprintf&#40;ft, "%d.", ply / 2 + 1&#41;;
        linelength += 3;
    &#125;
    fprintf&#40;ft, "%c%d%c%d",
            &#40;move_from & 7&#41; + 'a', &#40;move_from >> 3&#41; + 1,
            &#40;move_to & 7&#41; + 'a', &#40;move_to >> 3&#41; + 1&#41;;

    if &#40;move_promo&#41; &#123;  // from Lance Perkins
        fprintf&#40;ft,"%c", prm&#91;abs&#40;move_promo&#41;-1&#93;);
    &#125;
    fprintf&#40;ft, " ");
    linelength += 5;
&#125;

int main&#40;int argc, char *argv&#91;&#93;)
&#123;
    int             i, ply;
    size_t          filesize;
    int             node&#91;1000&#93;;
    struct MOVE &#123;
        unsigned char   move_from;
        unsigned char   move_to;
        unsigned char   move_promo;
    &#125;   moves&#91;1000&#93;;

    if &#40;argc < 2&#41; &#123;
        printf&#40;"Simple command-line converter from ABK to PGN\n");
        printf&#40;"Usage&#58; abk.exe <abk_file>\n");
        printf&#40;"Result PGN-file&#58; out.pgn\n");
        exit&#40;0&#41;;
    &#125;
    ft = fopen&#40;argv&#91;1&#93;, "rb");
    if &#40;ft == NULL&#41; &#123;
        printf&#40;"Can't open file %s\n", argv&#91;1&#93;);
        exit&#40;0&#41;;
    &#125;
#ifdef _MSC_VER
    filesize = filelength&#40;fileno&#40;ft&#41;);
#else
    stat&#40;argv&#91;1&#93;, &filecheck&#41;;
    filesize = filecheck.st_size;
#endif

    book = malloc&#40;filesize&#41;;
    fread&#40;book, filesize / sizeof&#40;struct BOOK&#41;, sizeof&#40;struct BOOK&#41;, ft&#41;;
    fclose&#40;ft&#41;;
    ft = fopen&#40;"out.pgn", "w");
    print_head&#40;);
    ply = 0;
    node&#91;0&#93; = 900;  // offset to first node in abk-file

    for (;;) &#123;
        print_move&#40;book&#91;node&#91;ply&#93;&#93;.move_from, book&#91;node&#91;ply&#93;&#93;.move_to,
                   book&#91;node&#91;ply&#93;&#93;.move_promo, ply&#41;;
        if &#40;book&#91;node&#91;ply&#93;&#93;.first_child > 0&#41; &#123;  // current game

            moves&#91;ply&#93;.move_from = book&#91;node&#91;ply&#93;&#93;.move_from;
            moves&#91;ply&#93;.move_to = book&#91;node&#91;ply&#93;&#93;.move_to;
            moves&#91;ply&#93;.move_promo = book&#91;node&#91;ply&#93;&#93;.move_promo;
            node&#91;ply + 1&#93; = book&#91;node&#91;ply&#93;&#93;.first_child;
            ply++;
        &#125; else &#123; // new game
            fprintf&#40;ft, "\n*\n\n");
            node&#91;ply&#93; = book&#91;node&#91;ply&#93;&#93;.next_sibling;
            while &#40;node&#91;ply&#93; < 0&#41; &#123;
                ply--;
                if &#40;ply < 0&#41; &#123;
                    fclose&#40;ft&#41;;
                    exit&#40;0&#41;;
                &#125;
                node&#91;ply&#93; = book&#91;node&#91;ply&#93;&#93;.next_sibling;
            &#125;
            print_head&#40;);
            linelength = 0;
            for &#40;i = 0; i < ply; i++) &#123;
                print_move&#40;moves&#91;i&#93;.move_from, moves&#91;i&#93;.move_to, moves&#91;i&#93;.move_promo, i&#41;;
            &#125;
        &#125;
        if &#40;node&#91;ply&#93; >= filesize / &#40;int&#41; sizeof&#40;struct BOOK&#41;) &#123;
            printf&#40;"Error&#58; out of file\n");
            fclose&#40;ft&#41;;
            exit&#40;0&#41;;
        &#125;
    &#125;
    return 0;
&#125;

/*
/home/cnxuser> gcc abk2pgn.c
/home/cnxuser> ./a.out
Simple command-line converter from ABK to PGN
Usage&#58; abk.exe <abk_file>
Result PGN-file&#58; out.pgn

c&#58;\tmp>cl /W1 /Ox abk2pgn.c
Microsoft &#40;R&#41; C/C++ Optimizing Compiler Version 15.00.30729.01 for x64
Copyright &#40;C&#41; Microsoft Corporation.  All rights reserved.

abk2pgn.c
Microsoft &#40;R&#41; Incremental Linker Version 9.00.30729.01
Copyright &#40;C&#41; Microsoft Corporation.  All rights reserved.

/out&#58;abk2pgn.exe
abk2pgn.obj

c&#58;\tmp>abk2pgn
Simple command-line converter from ABK to PGN
Usage&#58; abk.exe <abk_file>
Result PGN-file&#58; out.pgn

*/

User avatar
Graham Banks
Posts: 33253
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Re: Opening books format

Post by Graham Banks » Thu Dec 31, 2009 11:05 pm

mhalstern wrote:Thanks for all of the code and links.

Has anybody written a program to convert to and from the chessbase *.ctg format?

If so, pls post and or link.
My friend, Nigel Pattinson, wrote a utility that will convert the Chessmaster books into ctg format. Would that be useful to you?
http://homepages.paradise.net.nz/npatti ... kPage.html

Cheers,
Graham.
My email addresses:
gbanksnz at gmail.com
gbanksnz at yahoo.co.nz

alpha123
Posts: 660
Joined: Sat Dec 05, 2009 4:13 am
Location: Colorado, USA

Re: Opening books format

Post by alpha123 » Fri Jan 01, 2010 12:07 am

Graham Banks wrote:
mhalstern wrote:Thanks for all of the code and links.

Has anybody written a program to convert to and from the chessbase *.ctg format?

If so, pls post and or link.
My friend, Nigel Pattinson, wrote a utility that will convert the Chessmaster books into ctg format. Would that be useful to you?
http://homepages.paradise.net.nz/npatti ... kPage.html

Cheers,
Graham.
Well, that might be useful to me.... Thanks!
I assume he reverse engineered the .ctg format to do that, so could he write a .ctg to .pgn or .ctg to .abk converter?

Thanks,
Peter

Post Reply