Bookbuilding 101

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
jdart
Posts: 3825
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Bookbuilding 101

Post by jdart » Fri Aug 10, 2012 4:05 pm

For the rest the Crafty book also gets stuck for White after 1.d4 Nf6, and then is going to think for itself it seems. and comes up with 2.Nf3 or c4 after a long time. they should fix that i think.
This is a feature. As I understand it Crafty does a short search among the available book moves and if no moves have an acceptable score, it will do a deeper search rather than use a book move.

--Jon

jdart
Posts: 3825
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Bookbuilding 101

Post by jdart » Fri Aug 10, 2012 4:12 pm

Norm Pollock wrote:
I did realize that. But what I am asking for is a way to capture the games that were excluded. Namely, the games that seem to have an incorrect result based on the evaluation of the final position.

-Norm
I was replying to Juan. Sorry, my utility doesn't do that.

--Jon

User avatar
Ozymandias
Posts: 1132
Joined: Sun Oct 25, 2009 12:30 am

Re: Bookbuilding 101

Post by Ozymandias » Fri Aug 10, 2012 7:05 pm

">", of course, I should have tried that or "->".
Great utility.

User avatar
Ozymandias
Posts: 1132
Joined: Sun Oct 25, 2009 12:30 am

Re: Bookbuilding 101

Post by Ozymandias » Fri Aug 10, 2012 7:31 pm

Thnx for the input, but Jon already answered my question.

jefk
Posts: 394
Joined: Sun Jul 25, 2010 8:07 pm
Location: the Netherlands
Contact:

Re: Bookbuilding 101

Post by jefk » Fri Aug 10, 2012 10:06 pm

[quote="jdart"][quote]For the rest the Crafty book also gets stuck for White after 1.d4 Nf6,
[/quote]This is a feature. As I understand it Crafty does a short search
--Jon[/quote]

feature ? well quite lousy feature then..
:)

as it -after some long thinking- always comes up with 2.Nf3 (g6!)
or 2.e3 (...d5), whereas 2.c4 should be the move.

maybe mr Hyatt should ask a student to look a bit more
to the Crafty opening book features, as otherwise the
engine isnt playing so bad, with its 12 cores or so.
jefk

jwes
Posts: 778
Joined: Sat Jul 01, 2006 5:11 am

Re: Bookbuilding 101

Post by jwes » Sun Aug 12, 2012 9:56 pm

Don wrote:
Richard Allbert wrote:I agree with Harvey, the evaluation of each position is going to make the whole thing last a long time.

I wrote a bookmaker for Jabba - what I did was hunt down all the Playchess Engine room pgn files I could find - as in the engine room there you basically have the top engines playing vs themselves with different books.

I found out here what Bob wrote - the IO is the bottleneck.

I then computed win / loss / no. games, and simply wrote the positions in the book according to thresholds for win and number of games.

Problem is Jabba has larger weaknesses than the book, so I'm not sure if the result was a good book or not :)

Another method could be to take 150 ECO positions (there was an epd file posted here with them) to about move 6 or so, and play very fast matches with good engines from these positions, and build a book from the result.

Initially, I made the mistake of using a massive human games DB, unchecked, assuming that the bad moes would be filtered out by the "add to book if > X games played) and watched in horror as Jabba blundered a piece in the opening - a blunder that had apparently occurred enough times to make it into the book! :) :)

Good luck

Richard
In the book I talked about I was horrified to see that 1. h4 made it into the book. It was played at least 10 times. The TWIC database has mostly games by strong players but it also includes important tournaments that can have very weak players in them such as tournaments for the very young (where the strongest players may not even be masters and and the weakest may be quite weak or even beginners.) I did not filter by ELO.

I manually removed 1. h4 but didn't try beyond that - most the blunders are probably in the first 3 or 4 ply but I'm sure there are a few beyond that which still survive the "at least 10 time" frequency test.
A simple fix would be to add a second condition that a move is played in a given position at least n percent of the time. This could eliminate moves near the starting position that are rarely played.

User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 2:27 pm

Re: Bookbuilding 101

Post by Don » Sun Aug 12, 2012 10:08 pm

jwes wrote:
Don wrote:
Richard Allbert wrote:I agree with Harvey, the evaluation of each position is going to make the whole thing last a long time.

I wrote a bookmaker for Jabba - what I did was hunt down all the Playchess Engine room pgn files I could find - as in the engine room there you basically have the top engines playing vs themselves with different books.

I found out here what Bob wrote - the IO is the bottleneck.

I then computed win / loss / no. games, and simply wrote the positions in the book according to thresholds for win and number of games.

Problem is Jabba has larger weaknesses than the book, so I'm not sure if the result was a good book or not :)

Another method could be to take 150 ECO positions (there was an epd file posted here with them) to about move 6 or so, and play very fast matches with good engines from these positions, and build a book from the result.

Initially, I made the mistake of using a massive human games DB, unchecked, assuming that the bad moes would be filtered out by the "add to book if > X games played) and watched in horror as Jabba blundered a piece in the opening - a blunder that had apparently occurred enough times to make it into the book! :) :)

Good luck

Richard
In the book I talked about I was horrified to see that 1. h4 made it into the book. It was played at least 10 times. The TWIC database has mostly games by strong players but it also includes important tournaments that can have very weak players in them such as tournaments for the very young (where the strongest players may not even be masters and and the weakest may be quite weak or even beginners.) I did not filter by ELO.

I manually removed 1. h4 but didn't try beyond that - most the blunders are probably in the first 3 or 4 ply but I'm sure there are a few beyond that which still survive the "at least 10 time" frequency test.
A simple fix would be to add a second condition that a move is played in a given position at least n percent of the time. This could eliminate moves near the starting position that are rarely played.
That is an excellent idea! Even if the percentage was really low it would eliminate 1. h4.

Don
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.

User avatar
Dan Honeycutt
Posts: 5170
Joined: Mon Feb 27, 2006 3:31 pm
Location: Atlanta, Georgia

Re: Bookbuilding 101

Post by Dan Honeycutt » Tue Aug 14, 2012 12:28 am

Norm Pollock wrote:
jdart wrote:To save the output you need to redirect to a file, for example:

playchess2 input.pgn > output.pgn

--Jon
I did realize that. But what I am asking for is a way to capture the games that were excluded. Namely, the games that seem to have an incorrect result based on the evaluation of the final position.

-Norm
Hi Norm

My book utility is at a rough beta stage at the moment. It has a pgn check function which looks for mismatched braces, conflicting results and the like. It also has my Cupcake engine built in. I don't think it would be too much trouble to make a "suspicious results" check if you want. Do you have criteria you'd like to use?

Best
Dan H.

Norm Pollock
Posts: 1017
Joined: Thu Mar 09, 2006 3:15 pm
Location: Long Island, NY, USA
Contact:

Re: Bookbuilding 101

Post by Norm Pollock » Tue Aug 14, 2012 12:48 am

Dan Honeycutt wrote:
Norm Pollock wrote:
jdart wrote:To save the output you need to redirect to a file, for example:

playchess2 input.pgn > output.pgn

--Jon
I did realize that. But what I am asking for is a way to capture the games that were excluded. Namely, the games that seem to have an incorrect result based on the evaluation of the final position.

-Norm
Hi Norm

My book utility is at a rough beta stage at the moment. It has a pgn check function which looks for mismatched braces, conflicting results and the like. It also has my Cupcake engine built in. I don't think it would be too much trouble to make a "suspicious results" check if you want. Do you have criteria you'd like to use?

Best
Dan H.
Hi Dan,

Thanks for the response. For a start, you could try:

1) If the game ended in a draw, in most cases, the pawn-based score for the final position should be between -1.00 and +1.00 inclusive regardless of the color to move.

2) If the game ended 1-0, in most cases, the pawn-based score for the final position should be greater than +2.00 if White is to move, or less than -2.00 if Black is to move.

3) If the game ended 0-1, in most cases, the pawn-based score for the final position should be less than -2.00 if White is to move, or greater than +2.00 if Black is to move.

If the result is not consistent with the score of the final position using the above criteria, then the game should be extracted for analysis. There could be countless reasons for the mismatch- such as time pressure, blunder, a non-winnable position with unequal material, or a recording error.

-Norm

User avatar
Dan Honeycutt
Posts: 5170
Joined: Mon Feb 27, 2006 3:31 pm
Location: Atlanta, Georgia

Re: Bookbuilding 101

Post by Dan Honeycutt » Tue Aug 14, 2012 2:57 am

Norm Pollock wrote:Hi Dan,

Thanks for the response. For a start, you could try:

1) If the game ended in a draw, in most cases, the pawn-based score for the final position should be between -1.00 and +1.00 inclusive regardless of the color to move.

2) If the game ended 1-0, in most cases, the pawn-based score for the final position should be greater than +2.00 if White is to move, or less than -2.00 if Black is to move.

3) If the game ended 0-1, in most cases, the pawn-based score for the final position should be less than -2.00 if White is to move, or greater than +2.00 if Black is to move.

If the result is not consistent with the score of the final position using the above criteria, then the game should be extracted for analysis. There could be countless reasons for the mismatch- such as time pressure, blunder, a non-winnable position with unequal material, or a recording error.

-Norm
Hi Norm,

Ok, give it a try:

https://dl.dropbox.com/u/80776806/Bookworm.jar

It is a Java .jar file which you should be able to run like a normal .exe as long as your java version is current. There will be a button in the upper left called "Debug" Click that, then navigate to your .pgn file. It will run my pgn file check with your 'suspicious result' code patched in. It will search the final position in each game for 3 seconds. Thus it could take a while and I don't give you any hourglass or progress bar (the pgn check is normally instantaneous or nearly so).

If a suspicious result (or other error) occurs it will stop and give you the tag pairs for the game in question. Output to a file would be better but I'm using the error check mechanics already in place. If this looks like something useful and I add it permanently I'll doctor it up.

I gave it a quick check and it seemed to work ok but let me know if you find bugs.

Best
Dan

Post Reply