Opening books

[cms271828]

Hi,

Is there a basic way to create a simple opening book?

One idea I've had...
When a search is started, get the full hashkey(with castling rights etc) for the position(64 bits)

Then have an array, with say a 1000 entries, each entry contains a 64 bit key together with the corresponding move
Then just loop through the array till you find a matching key, and then play that move.

But for a given opening, say Queens Gambit, there aren't that many positions, maybe in the hundreds if you look at different transpositions of the same opening.

I guess you could also put the entries in the hash table, but I'm not sure what you would do with the depth,flag,score data.

Thanks for any help

[Dann Corbit]

Hi,

Is there a basic way to create a simple opening book?

One idea I've had...
When a search is started, get the full hashkey(with castling rights etc) for the position(64 bits)

Then have an array, with say a 1000 entries, each entry contains a 64 bit key together with the corresponding move
Then just loop through the array till you find a matching key, and then play that move.

But for a given opening, say Queens Gambit, there aren't that many positions, maybe in the hundreds if you look at different transpositions of the same opening.

I guess you could also put the entries in the hash table, but I'm not sure what you would do with the depth,flag,score data.

Thanks for any help

Typically, book positions just get played without any calculations unless you are pondering. If you are pondering, then you can think about your book move on the opponent's time if he is out of book.

If the opening book is large, one strategy that has been tried is to think on book moves for 10% of normal time. The idea was simply to watch out for bad book moves. In automatically built books, it is a common occurence for bad book lines to get inserted (they are even found in professional programs occasionally).

I prefer a database approach using something like FastDB:
http://www.garret.ru/~knizhnik/fastdb.html
and you can store all sorts of statistical stuff with the record along with the EPD data.
In addition, you can store "actually played" pv nodes in the book along with some depth information and then you can use these as self-expanding book lines. The statistical data will also help to weed out bad lines.

Early on, you don't need a sophisticated book. A few hundred positions stored in the program's data should be good enough to begin with.

[Edmund]

Hi!

When writing your own opening book, you should as well think about how you are going to insert the moves. So if you are just thinking about an easy way, you could do something like in tscp, where the book is just a txt file with each line containing an openingline (eg: g1f3 g8f6 c2c4 b7b6 g2g3) This would be easily editable with an editor.
If you want to build up an opening book built up of hash-keys, you have the advantage of transposition dedection. You should just make sure that no collisions can occure, which is quite probable for bigger books, if the hash is too short. Often for each move there is stored (especially for automatically generated opening books), number of times played, % wins for white/black/draws and as well a customisable value, which is could be used as a learn-value or for you to tell the engine which lines to prefer.

Edmund

[bob]

Hi,

Is there a basic way to create a simple opening book?

One idea I've had...
When a search is started, get the full hashkey(with castling rights etc) for the position(64 bits)

Then have an array, with say a 1000 entries, each entry contains a 64 bit key together with the corresponding move
Then just loop through the array till you find a matching key, and then play that move.

But for a given opening, say Queens Gambit, there aren't that many positions, maybe in the hundreds if you look at different transpositions of the same opening.

I guess you could also put the entries in the hash table, but I'm not sure what you would do with the depth,flag,score data.

Thanks for any help

You might or might not need the move. If you have the hash signature before the move is made, just make each move, update the hash signature, probe for that signature, and if you get a hit, that is a book move. This works better because it will catch all possible transpositions in a simple way. If you want a way to choose between the "hits" you can store any information you want, from learning results, to frequency that the position was reached in real games, to anything you choose.

[adams161]

hi,

You are welcome to use a simple text based book, about a 300k document , i dont have it open so not sure exactly, that i give out with pulsar. To be honest i found it 9 years ago on the web , i had been looking for something easy to parse, and no longer have any idea where it came from. I've given it out over the years to people who are looking to make a simple book to get started, and most move on to more sophisticated books after awhile but its an easy file to start making a simple book quick. file is bigbook.txt

I think the lines in teh book are not always teh best computer lines so i made a smaller book, openbk.txt that it looks for a move in first that has my handpicked lines, again hate to say it but i picked these lines years ago and havent looked at it much since, but i found the lines in a collection of eco lines somewhere and decided to copy maybe a hundred or 200 into my book.

hope this helps, hope i havent gotten myself into some kind of copyright trouble, but i suspect the collection of lines i made in the two books in 99 has long been forgotten by whoever orginally compiled them.

Mike

ps the format in bigbook.txt is very simple. e2e4e7e5 etc. not even teh full pgn standard at all. always a 4 character move like b1c3.

[adams161]

sorry download for bigbook.txt and openbk.tx is www.adam16mr.org/puslar.html I think any of the engine releases contain both books.

[adams161]

i still cant type my link correctly. www.adam16mr.org/pulsar.html

[cms271828]

Thanks, I've downloaded both books...

Looking at the bigbook, each line is a sequence of moves,
My original idea was to get a 64 bit zobrist key for each position during opening stage of game, then somehow look into a table to get the best move...

But now I see we can just go through move by move, is this the right idea (below).

Let computer be white, I can play as black.
Choose a good first move for white to play say e2e4.

Then supposing I choose c7c6
Program needs to look up the lines beginning e2e4c7c6 and then basically choose any of those lines, and play the next move...say d2d4

Then I play my next move as black..
If I choose something strange like a7a6, then program will not be able to find e2e4c7c6d2d4a7a6... thus taking it out of the opening book, and into normal search.

2 questions..
Some openings have moves which can be played in different orders, I'm guessing the main transpositions of common openings are in the bigbook file?

The lines seem to be arranged in alphabetical/numerical order as you go down the list, is this correct? I might need this fact when scanning through it.

Thanks for opening books, should be a great asset for my engine.

[Robert Pope]

The lines seem to be arranged in alphabetical/numerical order as you go down the list, is this correct? I might need this fact when scanning through it.

Yes, it's alphabetical so you can do a quick binary search, and with a dozen probes find your move, instead of checking every single line.

[adams161]

I use a node tree to treverse the book. you have a node with from to *up *side. say your 5 half moves deep and looking for move 6. you start at root and go sideways tell you find your move. say e2e4. then you go up say looking for d7d5. if you dont find it in up you go sideways tell you do find d7d5 or your out of book. then you go up from d7d5 and if its not the first move you again move sideways. once you find your last move in line you go up and sideways tell you have collected all the possible moves you can play and make one of them. Doesnt seem to take me that long to do it this way. maybe 150 ms lookup and search time. This method does not catch transpositions but it will give you a basic book.

Mike