CLI tool for composing polyglot book from PGN

[likeawizard]

When implementing a polyglot book for my chess engine. I was not able to find tools to parse PGN's to UCI and compose polyglot books.
So I made my own. What it does in its most basic form- parses a PGN containing one or many games into UCI and those moves are composed into a polyglot opening book. (A move made by the winning side adds weight 2, a move that results in a draw adds weight 1 and a move made by the losing side is ignored)

At the moment this is something I use personally and more like a proof of concept. I am not sure what tools are out there and if this could be of any practical use to anyone besides me. If anyone sees any use in it or potential use with added features, let me know. At the moment it has a very long loading time as it built on a cannibalized version of my chess engine which generates magics each time on initialization and that takes on my system ~1s. Those magics could probably be hardcoded to allow faster execution. Also I have not tested it extensively so bugs are more than guaranteed.

https://github.com/likeawizard/polyglot-composer

[hgm]

I don't get it. Polyglot is a CLI tool that can convert PGN to Polyglot books, right? And it doesn't need 'UCI' (whatever that may mean).

[likeawizard]

I don't get it. Polyglot is a CLI tool that can convert PGN to Polyglot books, right? And it doesn't need 'UCI' (whatever that may mean).

Correct. It's a largely irrelevant detail for now. The SAN notation is a bit annoying to interpret in my experience as it requires more context to decode certain moves.
Compare the two positions:
[fen]3k4/8/8/5b2/8/8/2N1N3/3K4 w - - 0 1[/fen]
[fen]3k4/8/8/8/6b1/8/2N1N3/3K4 w - - 0 1[/fen]
If you want to play c2d4 in the first position it is Ncd4 and in the second it is simply Nd4 since the knight on e2 is pinned so no need for disambiguation. So basically I ripped the move generator out of my chess engine and generate all the legal moves and try to match it with SAN moves given in a PGN converting the SAN notation to UCI move notation as an intermediate step to play out games. Calculate the zobrist hashes used for polyglot keys.

Also I want to thank you personally for the nice documentation you put together on the polyglot book specification. It was great help understanding the format and implementing it.

[hgm]

I think Michel van den Bergh wrote most of that documentation. I just extended it for use in chess variants.

OK, I understand that SAN in some respects can be a pain. And in particular this 'in check disambiguation'. I just don't understand what this has to do with Polyglot books. Polyglot doesn't make any errors ever when disambiguating SAN, I would think.

[likeawizard]

Polyglot doesn't make any errors ever when disambiguating SAN, I would think.

Yeah, because polyglot and SAN have nothing to do with each other? Completely different notation / encoding. The target is simple take a list of PGN games contained in a single file and generate a polyglot book based on all the moves within. So it has nothing to do with the UCI format apart from it being a very comfortable intermediate notation as the polyglot move encoding is a straightforward mapping from UCI move format. (With the exception of castling moves e1g1 -> e1h1.)

So yeah, it has nothing to do with the UCI move format directly. I just mentioned it simply as it is part of the internal algorithm. In essence it is SAN -> UCI move format -> play out the variation and use the board state to generate polyglot keys and store the move as needed. Or am I missing something more substantial?

[hgm]

Well, I guess what I don't understand is why you want to program this yourself rather than just use Polyglot.

[likeawizard]

oh... I guess I wasn't aware of it. But yea then this is pretty redundant.

Is this the one you are talking about https://github.com/sshivaji/polyglot ?

[hgm]

Yes, that's it, but a very old version. A more recent one that I used to bundle with WinBoard is in my own on-line repository ( http://hgm.nubati.net/cgi-bin/gitweb.cgi )

[likeawizard]

Thank you I will check it out. I had the idea of coding it so I got to it without doing enough research of what was already available...