Book format for the new engine

[Dragan]

I am writing a new chess engine. Hope to have something by the end of the year.
My questions is about the book format. Is there any standard book format that I should be implementing or every engine has its own?
Is there any benefit either way?

[Dan Honeycutt]

I am writing a new chess engine. Hope to have something by the end of the year.
My questions is about the book format. Is there any standard book format that I should be implementing or every engine has its own?
Is there any benefit either way?

Lot's of ways to do it - see my recent thread Bookbuilding 101. As I note, I'm working on a book making utility. If you want to coordinate efforts we can have a standard followed by two

Best
Dan H.

[jdart]

If you are a UCI engine then you don't really need to implement opening book logic in the engine .. the GUI can and commonly does handle the book. There may be reasons you'd want to do your own, if you want to do something more elaborate than the GUI can do, which is basically just look up moves at runtime.

polyglot is the main standardized book format many programs are using (http://wbec-ridderkerk.nl/html/details1/PolyGlot.html).

[Dragan]

I need a book if I want to play on ICC. Implementing Polyglot standard would be perfect for this since my engine is UCI.
Are there any good Polyglot books? Do you use your own format in Arasan?

[Dragan]

I have a better idea. I will let you do all the hard work and then use the results
I don't want to spend too much time on the book. So many other things I need to do...

[Don]

I am writing a new chess engine. Hope to have something by the end of the year.
My questions is about the book format. Is there any standard book format that I should be implementing or every engine has its own?
Is there any benefit either way?

Polyglot is probably a good way to go. At least it's an open standard.

If you want flexibility then I would propose using sqlite3 and doing you own thing. Since it would be a sql database it would automatically be open.

The schema I would suggest is:

fen
move
priority

fen should not include en-passant target square unless en-passant is possible on that square.

I may do Komodo's book that way in the future. Since the book is stored on disk there is no real issue with the fact that this format will take a lot more disk space than polyglot - but it's way more flexible as you can add fields at will for your own private processing and it would be 100% cross platform.

Don

[jdart]

Arasan uses its own book format (the code for that is MIT licensed, if you want to use it - but no other program currently uses it besides Arasan).

There are a lot of polyglot books. Arena (http://www.playwitharena.com) comes with some.

--Jon

[jefk]

[quote="Dragan"]I need a book if I want to play on ICC. Implementing Polyglot standard would be perfect for this since my engine is UCI.
Are there any good Polyglot books? Do you use your own format in Arasan?[/quote]

well i haven't looked at polyglot for ages but one standard which is useful in most advanced tools like Aquarium and so on is EPD format, one position, and a score, so then you only need to recognize a position and search up possible next position. EPD is used by Crafty in some old subroutine by Edwards.

And you can later extend it with some statistics by reading in pgn's and attaching some percentages. But personally i don't think they are so reliable,not even talking about the problems with transposition, and jumping from positive to negatie statistics every next move. Your just imitating (grand)masters in such a way. Now why on earth would a chess computer ever would like to do a thing like that i wonder ?

example: checkers is a draw, so any opening is possible.
Now in chess -although almost certainly also a draw- you still like to put maximum pressure on the other side, which might imply some dififcult bookmoves, which might have scored poor by humans, but nevertheless might be very well useable by computers, i.e. strong engines.

but on the other hand in the first 15 moves or so you like to
weed out erroneous or strange computer moves and thus the
statistics of moves played more than a certain number of times
make it muhc more reliable.

good luck,

PS someone wrote that a minimax is costing such a long time,
well in the bookbuilder36 of which the binaries are freely downloadable
on my site superchess.com a posbase with almost twenty million
positions is minimaxed in about twenty minutes. thats not a long time.

and i'm thinking of releasing that sourcecode to someone who i's
willing to do some usefult stuff with it, rather than reinventing
the wheel like they tried in Aquarium and come up with awesome
slow minimax routines.

Another thing i'm thinking of is áttach an option of 'sharpness'
to every position, like the mato-meter in Fritz. In such a way you could
combine the backsolved Epd score with the sharpness of the current position, and depending on the opponent choose for a sharp or positional book line. For example French against e4 if you want to play positional,
or Sicilian if you want to play sharp. and in such a way the engine also has something to do when you;r still in book. I often play games on ICC where the engine is simply idle the first fifteen moves as i'm still in book but
the opponent (eg Crafty) isnt anymore. waste of computer power isnt it:)

well at least it could do a tree search in the book, something like rookie
seems to do, and indeed seems to get a bit better over time although
it will still take Marcel a few years before its GM quality i suspect.

[Dragan]

I will go the Polyglot way, but I will put the book building code for later releases.
I assume that Polyglot can already handle playing book moves for my engine when I play on ICC, so I don't really need to do anything unless I want to improve on the Polyglot book standard.

Thanks everybody for your input.
Dragan

[jefk]

[quote="Dragan"]I will go the Polyglot way, but I will put the book building code for later releases.Dragan[/quote]

ok good luck;

and..: apparently there already exist a little program for
creating Polyglot books:

http://www.chess.com/download/view/crea ... essanalyse

haven't tried it myself but from the description it sounds useful,
as you can set a minimum number of (master)games ,eg. three

jef



I assume that Polyglot can already handle playing book moves for my engine when I play on ICC, so I don't really need to do anything unless I want to improve on the Polyglot book standard.

Thanks everybody for your input.