Polyglot extension tool

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Jonathan003
Posts: 239
Joined: Fri Jul 06, 2018 4:23 pm
Full name: Jonathan Cremers

Re: Polyglot extension tool

Post by Jonathan003 »

There is a tool that can convert bin books to pgn called
Cerebellum book CONVERTOR

http://www.chessgod101.com/t3840-cerebe ... -convertor

Of course I don't know if it's done correctly.

I can also import best lines from a bin book (for one color), or al best lines (for one color), to opening lines in Lucas Chess.
And in turn export this opening lines to a pgn file.

In scid it's also possible to load a small bin book and export the bin book to pgn, but that's very limited in size and takes a long time.
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Polyglot extension tool

Post by Ferdy »

Rebel wrote: Sun Jan 06, 2019 4:18 pm Poly version 1.0

The polyglot book format is the most widespread format in use by chess engines. The format contains 4 unused bytes. Have a look at the nice things you can do using those bytes.

http://rebel13.nl/download/books.html
I have a polyglot book created from pgn file. I also have some epd's analyzed out of this pgn file. Can I use poly to just add/update the score in bin book using epd's with the following format?

Code: Select all

r1bqkbnr/pppp1ppp/2n5/4p3/4P3/5N2/PPPP1PPP/RNBQKB1R w KQkq - ce 50;
User avatar
hgm
Posts: 27787
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Polyglot extension tool

Post by hgm »

Jonathan003 wrote: Fri Jan 18, 2019 1:02 am There is a tool that can convert bin books to pgn called
Cerebellum book CONVERTOR
You would have to be careful with that, as converting Polyglot books to PGN is not generally possible. There is no way to convert the position keys in the book back to positions. So you would be dependent on following the opening lines from the start position to see to which position they lead, and then see if that position occurs in the book. This is fine if all the lines in the book are uninterrupted, so that every position in the book is reachable from the start position by a sequence of moves from the book. But even in a book genererated by Polyglot that is not always the case, as moves with zero weight are not stored. So if a move only scored losses, the position after that move (where all moves scored only wins) will be disconnected from the opening tree, and just following the book moves will miss it. You could work around that by generating all moves from leaf positions, to see if they were omitted this way, or even deeper perfts. But there is no guarantee you would find everything in the book.

Also, if the book weights were tuned, it can be impossible for a PGN collection to correctly reflect the weights, so that converting it back to a book would give you a different book.
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: Polyglot extension tool

Post by Rebel »

Ferdy wrote: Fri Jan 18, 2019 12:31 pm
Rebel wrote: Sun Jan 06, 2019 4:18 pm Poly version 1.0

The polyglot book format is the most widespread format in use by chess engines. The format contains 4 unused bytes. Have a look at the nice things you can do using those bytes.

http://rebel13.nl/download/books.html
I have a polyglot book created from pgn file. I also have some epd's analyzed out of this pgn file. Can I use poly to just add/update the score in bin book using epd's with the following format?

Code: Select all

r1bqkbnr/pppp1ppp/2n5/4p3/4P3/5N2/PPPP1PPP/RNBQKB1R w KQkq - ce 50;
As long as the EPD is in the book, sure.

I need to add 2 more options to the tool, import from PGN and import from EPD.

Not much work, stay tuned.
90% of coding is debugging, the other 10% is writing bugs.
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: Polyglot extension tool

Post by Rebel »

Rebel wrote: Fri Jan 18, 2019 5:30 pm
Ferdy wrote: Fri Jan 18, 2019 12:31 pm
Rebel wrote: Sun Jan 06, 2019 4:18 pm Poly version 1.0

The polyglot book format is the most widespread format in use by chess engines. The format contains 4 unused bytes. Have a look at the nice things you can do using those bytes.

http://rebel13.nl/download/books.html
I have a polyglot book created from pgn file. I also have some epd's analyzed out of this pgn file. Can I use poly to just add/update the score in bin book using epd's with the following format?

Code: Select all

r1bqkbnr/pppp1ppp/2n5/4p3/4P3/5N2/PPPP1PPP/RNBQKB1R w KQkq - ce 50;
As long as the EPD is in the book, sure.

I need to add 2 more options to the tool, import from PGN and import from EPD.

Not much work, stay tuned.
That was a bad answer, there has to be a "bm" in the EPD.

Version update tomorrow.
90% of coding is debugging, the other 10% is writing bugs.
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: Polyglot extension tool

Post by Rebel »

UPDATE version 1.2

http://rebel13.nl/download/books.html

Added 4 essential functions which allows the user to immediately make use of the new features without programmer support.

Weights on score
1. Option 'B' (best move) sets the weight of moves on 100% with the highest Stockfish score. The book will only play those 100% moves. Using this function basically emulates the results in table-1 above.

2. Option 'V' (variation) based on the Stockfish scores will calculate new weights.

Weights on learn
1. Option 'B' (best move) sets the weight of moves on 100% with the highest learn value. The book will only play those 100% moves. Using this function basically emulates the results in table-2 above.

2. Option 'V' (variation) based on the learn scores will calculate new weights.

Import Score and Depth from PGN
Import Score & Depth from an annotated (thus with Score and Depth) ENG-ENG match.

Import Score and Depth from EPD
. Required tags: 'bm' (move) and 'ce' (score).
. Preferable (although not obliged) tag: 'acd' (depth)

The advantage of the two last options is that you can add the analysis (score & depth) of any engine to Polyglot books.

Check the new Polyglot weights with ProDeo 2.9 (book.txt) or with Scid.

______________________________________________________________

If there are new wishes now is a good time.
90% of coding is debugging, the other 10% is writing bugs.
User avatar
Ozymandias
Posts: 1532
Joined: Sun Oct 25, 2009 2:30 am

Re: Polyglot extension tool

Post by Ozymandias »

Rebel wrote: Sun Jan 20, 2019 11:13 amWeights on learn
1. Option 'B' (best move) sets the weight of moves on 100% with the highest learn value. The book will only play those 100% moves. Using this function basically emulates the results in table-2 above.

2. Option 'V' (variation) based on the learn scores will calculate new weights.
I have to play with this. Thanks.
Jonathan003
Posts: 239
Joined: Fri Jul 06, 2018 4:23 pm
Full name: Jonathan Cremers

Re: Polyglot extension tool

Post by Jonathan003 »

Hi Ed

I tried the new version of poly 1.2
If I chose for the option 'V' to calculate new weights.
And I openen the new created book in SCID. I can't see the weight of the moves. But just strange characters.

I have included a screenshot in the attachment.

If I try to import this new bin book to opening lines in Lucas Chess. Everything seems to work fine.

I want to be able to see the weights of the moves in SCID to. If I want to change some weights manually.
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: Polyglot extension tool

Post by Rebel »

That's odd, everything okay with ProDeo. I will look into it.
90% of coding is debugging, the other 10% is writing bugs.
User avatar
Rebel
Posts: 6991
Joined: Thu Aug 18, 2011 12:04 pm

Re: Polyglot extension tool

Post by Rebel »

Jonathan003 wrote: Sun Jan 20, 2019 2:38 pm Hi Ed

I tried the new version of poly 1.2
If I chose for the option 'V' to calculate new weights.
And I openen the new created book in SCID. I can't see the weight of the moves. But just strange characters.

I have included a screenshot in the attachment.

If I try to import this new bin book to opening lines in Lucas Chess. Everything seems to work fine.

I want to be able to see the weights of the moves in SCID to. If I want to change some weights manually.
Everything is running fine on my end. But I have an idea why things got wrong on your end. The by default created NEW.BIN is meant to keep your orginal unchanged. However if you start with NEW.BIN as source the destination is the same and this result in a conflict, hence the strange output. I will build in a check that source (NEW.BIN) can never be as the same as the destination (NEW.BIN).

So try again.
90% of coding is debugging, the other 10% is writing bugs.