is there a tool to generate killer or drawing books?

[Uri Blass]

The idea is the following:
You have Engine X and you want to write an opening book to avoid losing against it(I will call it anti-X book).
You start with the empty book.
You let Stockfish to play with white against X.
In case Stockfish does not lose the game you simply enter the moves of the game to your book and the game is part of the book.

In the next game you let stockfish with the book of stockfish moves from the first game to play against X.
X may be not deterministic program so may play a different move when you may get out of book and the rest of the moves stockfish play against X and after the game if you do not lose you add the moves to the book.
You continue in this way game after game with the idea that even if X is not deterministic it has limited number of choices so after enough games you see that you win or draw every game that you play with white out of book and you have anti-X opening book for white(you can do the same for black).

[phhnguyen]

I have worked with tools to generate/compute opening books for a while and I am quite interested in any new idea. I guess you thought a lot about your method thus I have some questions/wonderings:
- what will you do if games between those engines are quite deterministic, say, there are only 5 distinguished games from several thousand played games?
- what will you do if after entering a game to the book, both engines follow that game forever (become deterministic without any changes)?
- suppose the 1st successful game has 30 moves entered the book. With the new game, SF followed 10 moves of the 1st game then lost. What will you do with data on the book of the 1st game? If you want to ignore that game, we can suppose SF continues losing, say all next 10000 games, after following some moves of the 1st game?
- since that’s the book from computing of only two engines without using any book, I don’t think it has as many variants as a standard book - do you agree? Do you think the book can help (I suppose you may integrate that book inside the engine to use all the time) if we start a tournament between those engines, using a standard book? Say, their games won’t start from move zero/start position but from move 10th? From positions not in that book?
- (similar to above question) what use of that book? Especially when almost all typical tests, tournaments, engine matches use opening books

[mvanthoor]

As far as I'm aware, there is no tool to do this automatically.

You could, however, have Stockfish play a match against engine X, filter out the games won by Stockfish, and then create an opening book based on those games. If you make it a Polyglot book (.bin), you could build the book into the engine, and have it play from there if the position on the board occurs in it.

This is exactly the reason why using such a book is mostly forbidden in engine-engine matches, to avoid creating an engine that is either in its opening book or endgame database for a lot of the game.

[Ferdy]

The idea is the following:
You have Engine X and you want to write an opening book to avoid losing against it(I will call it anti-X book).
You start with the empty book.
You let Stockfish to play with white against X.
In case Stockfish does not lose the game you simply enter the moves of the game to your book and the game is part of the book.

In the next game you let stockfish with the book of stockfish moves from the first game to play against X.
X may be not deterministic program so may play a different move when you may get out of book and the rest of the moves stockfish play against X and after the game if you do not lose you add the moves to the book.
You continue in this way game after game with the idea that even if X is not deterministic it has limited number of choices so after enough games you see that you win or draw every game that you play with white out of book and you have anti-X opening book for white(you can do the same for black).

You can try using polyglot to build a book for the engine. This actually tests how effective the polyglot algorithm is.
Procedure:
1. Run a single game engine match using cutechess-cli for example. One engine is using a polyglot book built in (2) and the other is not.
2. After the match, delete existing book and build a new book using the output pgn in (1).
3. Goto (1).

I tried SF13 vs SF13 and black is using a book. TC=100ms/move. So far black with book is not getting any upper hand.




They only play a single opening and white varies at move 4 with the following stats. Score above 50% favors white.




match.bat

Code: Select all

@echo off
cls

:start
echo starting match

cutechess-cli.exe ^
-tournament gauntlet ^
-games 1 ^
-rounds 1 ^
-noswap ^
-pgnout games.pgn ^
-concurrency 1 ^
-wait 100 ^
-maxmoves 300 ^
-resign movecount=4 score=700 twosided=true ^
-draw movenumber=60 movecount=6 score=0 ^
-each proto=uci option.Threads=1 ^
-engine cmd="sfpoly13.exe" name=nobook st=0.1 timemargin=50 ^
-engine cmd="sfpoly13_book.exe" name=withbook option.Book1=true option."Book1 File"=F:\Project\book_killer\book.bin option."Book1 BestBookMove"=true st=0.1 timemargin=50


if exist book.bin (
    del book.bin
	echo deleting book ...
)

echo creating book ...
call createbook.bat
echo book is created


goto start

createbook.bat

Code: Select all

polyglot64.exe make-book -pgn games.pgn -bin book.bin -max-ply 160 -min-game 4 -min-score 2

I took Stockfish polyglot 13 from massimilianogoi.

[Uri Blass]

I have worked with tools to generate/compute opening books for a while and I am quite interested in any new idea. I guess you thought a lot about your method thus I have some questions/wonderings:
- what will you do if games between those engines are quite deterministic, say, there are only 5 distinguished games from several thousand played games?
- what will you do if after entering a game to the book, both engines follow that game forever (become deterministic without any changes)?
- suppose the 1st successful game has 30 moves entered the book. With the new game, SF followed 10 moves of the 1st game then lost. What will you do with data on the book of the 1st game? If you want to ignore that game, we can suppose SF continues losing, say all next 10000 games, after following some moves of the 1st game?
- since that’s the book from computing of only two engines without using any book, I don’t think it has as many variants as a standard book - do you agree? Do you think the book can help (I suppose you may integrate that book inside the engine to use all the time) if we start a tournament between those engines, using a standard book? Say, their games won’t start from move zero/start position but from move 10th? From positions not in that book?
- (similar to above question) what use of that book? Especially when almost all typical tests, tournaments, engine matches use opening books

1)If I see that the games are deterministic then a short book is enough to be a killer book to get a draw against the engine
but I believe stockfish with many cores is not deterministic.

2)I believe it is very hard to beat stockfish so I do not expect it to lose against X by its own moves again and again.
In the rare case when it happen I may use longer time control to see if I need to correct part of the moves that stockfish played and remove them from the book.

3)I agree that the book does not have many variants and I agree it is not going to help in a tournament when engines use standard book that is not the choice of the engines.

I thought mainly that the idea may test the variety of engines(because there are engines that are more deterministic even with many cores and there are engines that are less deterministic) and it is interesting to know how many games you need to memorize if the target is to beat or draw against an engine with no book only based on memorization of moves.

It is of course harder to beat or draw against engine with some fixed book only based on memorization and only if the first task is easy enough I think that there is a point in trying to get better result in lists like SSDF rating list when all engines use their own book and do not get and external book.