Opening Books

[cms271828]

Hi, I have built a simple opening book a while ago.
Its just a text file, with lots of different lines of play.

I think its possible to use pgn files to create an opening book.
I found this website with lots of pgn opening files...
http://www.chessopeningspgn.com/chess/Openings.html

So I was thinking of downloading a load of these pgn files, and putting them on my server, then with chess program(java application), I could possibly make a call to a php page on server which can look through the pgn files, and retrieve the best reply.

Is that how it generally works?

[Bill Rogers]

colin
I am also building a file of opening books but I use only winning games from top players or machines. I don't care about draws just wins for either black or white. I am creating two sets of books, one for white and one for black. I may take a long time but I think it will be worth it.
Bill

[cms271828]

Thanks, but how does one precisely find an opening?

Is it just a matter of scanning through pgn files, each time, reducing your possible set of moves, until there are none left, and engine is started?

I could see this being slow, especially with large pgns of several meg in size.

I would probably do this in php, keeping the pgn files on the server.
I'm not sure how quick it would be.

Any thoughts?

[Edmund]

Saving all the lines has a lot of redundancy. It would be easier to convert all the games and only save positions in the opening book. With the position you can then either save a score for the position (depending on the number of times it appeared on board and the percentage of games the stm won) or you save the possible answers.

You may want to have a look at http://64.68.157.89/forum/viewtopic.php?t=21616

It is then up to you how you store the positions. Either using fen or zobrist keys (or something else) - it all depends on whether you want to be able to edit the file afterwards using an editor.

Anyway looking up the next book move won't take long then. Max log2 n tries (where n is the number of entries), as the positions will obviously be sorted in some sort.

Regards Edmund

[nczempin]

Saving all the lines has a lot of redundancy. It would be easier to convert all the games and only save positions in the opening book. With the position you can then either save a score for the position (depending on the number of times it appeared on board and the percentage of games the stm won) or you save the possible answers.

You may want to have a look at http://64.68.157.89/forum/viewtopic.php?t=21616

It is then up to you how you store the positions. Either using fen or zobrist keys (or something else) - it all depends on whether you want to be able to edit the file afterwards using an editor.

Anyway looking up the next book move won't take long then. Max log2 n tries (where n is the number of entries), as the positions will obviously be sorted in some sort.

Regards Edmund

It will take negligible linear time if you just build a hash table, offline. Redundancy etc. is all not a big problem because you can do the work ahead of time.

[nczempin]

Thanks, but how does one precisely find an opening?

Is it just a matter of scanning through pgn files, each time, reducing your possible set of moves, until there are none left, and engine is started?

I could see this being slow, especially with large pgns of several meg in size.

I would probably do this in php, keeping the pgn files on the server.
I'm not sure how quick it would be.

Any thoughts?

Why would you want to add an extra layer of PHP?

Just make a hash table, with multiple alternative moves getting different weights depending on the eventual results.

Or, do it like I did, and just choose one variation at each node.

All this is done before the engine starts, "offline". The engine just reads the hash table from disk in some way. There is no reason why a book move should ever be anything but instantaneous.

[cms271828]

I'm thinking of doing it the easier way, so 1 move for each position.

The entries in my TT have depth/flag/score/best move, and the 2nd half of the key(to make up the 64-bit key for the position)

But I think for an opening book, all I need is the second half of the key, and the move to play, so I could build a large table(but much much smaller than TT), say 2^18 (quarter million) entries, so 46 bits of the entry would be the other half of the key, and 12 bits for the move, leaves 6 bits unused.

So with that approach I would need to initialize the table before play starts. Assuming I have 2^18 entries, thats 2^18 hex Strings each of length 16 characters, or 1,572,864 characters of text.

I'm not sure how long it would take to read this in from a text file, without trying it first, but is this a good way of doing it?

Thanks