huge opening file like 8moves_GM.pgn?

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

Ferdy
Posts: 4846
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: huge opening file like 8moves_GM.pgn?

Post by Ferdy »

Adam Hair wrote:In case anybody wants the results of Lucas' filtering of 8moves_GM in PGN format, here is a link:

http://www.mediafire.com/download/wvvqb ... _LB.pgn.7z

44,624 openings, all 8 moves (16 plies) deep.
In the KID from ff position, 8.d5 Ne7 position is missing (it is nice to see the how the engine will handle white bayonet attack after 9.b4!). Only 8.Be3 and 8.dxe5 is included. I wonder if that is one of those positions that are cut. IMO this is a good position to test engine plays in close position.
[d]r1bq1rk1/ppp2pbp/2np1np1/4p3/2PPP3/2N2N2/PP2BPPP/R1BQ1RK1 w - - 2 8
User avatar
hgm
Posts: 28341
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: huge opening file like 8moves_GM.pgn?

Post by hgm »

What I did in Shokidoki was simply modify its search that for every requested depth >= N the window would be forced open to the range I was interested in (e.g. from -50cP to +50cP), and let it print the path to the current position, with final score, after searching such a node. (N would be the depth to which I wanted the positions verified.) When the hash probe finds a result outside the range at depth >= N, it just returns that result, even if the requested depth was larger. (I.e. I won't expand lines further that are outside the window at the nominal verification depth; these are rejected forever.) Then I just set it searching, and wait.

It doesn't get deep very fast, however. Because it tries to generate all possible opening theory, rather than just a book for itself. So it became much faster when I decide for which side it must make a book in advance, and then only expands the best move for that side, and all moves for the opponent.
Frank Quisinsky
Posts: 6888
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: huge opening file like 8moves_GM.pgn?

Post by Frank Quisinsky »

Hi Sam,

can be donwload on my webpage (www.amateurschach.de) in my over 100Mb download file to SWCR-1.

I am working around one year on the optimation of this database. Optimaze with 160.000 SWCR-1 games and with a long documentation about the building.

I made a random book for Shredder GUI with the database and try here "with random" to get the same results in ECO code A00 - E99 the actual GM database of the latest 20 years has. Postions, engines made fast a draw are deleated. All end positions are checked with Q9550 4Core 1 Minute analyes with Stockfish, Junior and Komodo.

Used at start of this work the modern GM opening theorie from the latest 20 years. But read the documention about this work. One of the main Topics in SWCR-1 (not only the Rating list, more this work for my future of engine testing).

Today I used this Shredder Random book for eng-eng games. Absolutely perfect for testing ... no Problems with fast remis / won games or bad opening for chess engines.

Try it and have fun.

Best
Frank
mvk
Posts: 589
Joined: Tue Jun 04, 2013 10:15 pm

Re: huge opening file like 8moves_GM.pgn?

Post by mvk »

lucasart wrote:Actually, I'm thinking of creating my own opening book, fram scratch,

[... snip ...]

The above could easily be coded in a Python script, so that the same algorithm can be used with any engine supporting multi-PV really.
Welcome to the club :-)
Norm Pollock
Posts: 1070
Joined: Thu Mar 09, 2006 4:15 pm
Location: Long Island, NY, USA

Re: huge opening file like 8moves_GM.pgn?

Post by Norm Pollock »

I put up an "EPD" version It has 44,747 "fair" positions after 8 full moves. I added a few positions.

Download "engineTest" at the bottom of my chess page. (See "www" below for the link to my chess page.)
User avatar
beachknight
Posts: 3533
Joined: Tue Jan 09, 2007 8:33 pm
Location: Antalya, Turkey

Re: huge opening file like 8moves_GM.pgn?

Post by beachknight »

Norm Pollock wrote:I put up an "EPD" version It has 44,747 "fair" positions after 8 full moves. I added a few positions.

Download "engineTest" at the bottom of my chess page. (See "www" below for the link to my chess page.)
Thank you Norm.

I'll test this epd file in my new engine tests.

Best,
hi, merhaba, hallo HT
Norm Pollock
Posts: 1070
Joined: Thu Mar 09, 2006 4:15 pm
Location: Long Island, NY, USA

Re: huge opening file like 8moves_GM.pgn?

Post by Norm Pollock »

It's now at 54,351 epd positions. Each position is unique. I started with my own list and have merged positions from two large pgn books. Those books are:

8moves_GM_LB.pgn
HS-Grand_8-moves-31437.pgn

Thanks for the links.

Then I did some heavy lifting, which was to remove duplicates and near-duplicates (differing ep square or differing castling rights, keeping the epd with most possible moves).