Search-Based Opening Book Construction

[Dann Corbit]

Full manuscript here: https://drive.google.com/open?id=0B2pvW ... UpkRE0tem8

One more thing, you are testing against Crafty book, how do you maintain that both engines leave the book at the same time?
If that is not maintained than the engine with the longer book will almost always score better not because its book is better, but because it will have more thinking time!

Your conjecture seems doubtful to me.
It supposed that the quality of the move is always good.
Suppose that some moves are poor choices. They may do more damage than the time saved.
In my experience, all chess opening books have serious defects. It is simply too expensive to test all interior and leaf nodes to 37 plies or more to ensure good play.
There are many fairly standard openings with gaffes in them that simply get played by rote.
In ten years, our opening books will be 200 Elo stronger than today's books, ans some of that will be due to efforts like the author of this thread.

[Milos]

It seems a lot of interesting projects you find disturbing for some reason.
That is OK but I find it puzzling.

That's only your impression and is not based on facts.

Sometimes it is possible to tell that some approach will not work well from first principles.
This project has similarities to Brain fish but there are some important differences.
I don't see anything negative about this work and his publication of the source means that other can repeat or even expand his experiments.

I don't see any advantage over Brainfish. I see many similarities (approach is practically the same except Brainfish uses hash what would be really the only logical way, and this one doesn't and that's practically all the novelty) and I see 2 huge drawbacks:
1) He uses Crafty for leaf analysis which is pathetically weak engine compared to SF.
2) Size is terms of number of positions is extremely small compared to Brainfish (26 times smaller) which guarantees much worse performance.

[Milos]

Your conjecture seems doubtful to me.
It supposed that the quality of the move is always good.
Suppose that some moves are poor choices. They may do more damage than the time saved.
In my experience, all chess opening books have serious defects. It is simply too expensive to test all interior and leaf nodes to 37 plies or more to

It seems you haven't given much thought to this. Since OP book is created using exactly same Crafty engine, in test once Crafty is out of its own book, you will get exactly the same analysis of the given position that happened in the process of creation of the OP book. So unless move time during testing is longer than analysis time used when creating OP book (which is not the case), there is absolutely no way the move is inferior. Unless you claim that Crafty at depth N produces better move than the same Crafty at depth N+M.

Btw. I did proper testing of Brainfish book against same Brainfish without the book (so no compile speed differences) in a way that they start with the same TC, but once Brainfish with book is out of its book, the remaining time is reset for the Brainfish without book. This way the extra time due to book usage is removed as a factor, and you know what, difference was below 30Elo (TC was 60''+0.6'', single core).
And bear in mind Brainfish book is much, much stronger than OP book. So his 8.6% better result is mainly due to longer thinking time of Crafty with OP book.

[Dann Corbit]

It seems a lot of interesting projects you find disturbing for some reason.
That is OK but I find it puzzling.

That's only your impression and is not based on facts.

Sometimes it is possible to tell that some approach will not work well from first principles.
This project has similarities to Brain fish but there are some important differences.
I don't see anything negative about this work and his publication of the source means that other can repeat or even expand his experiments.

I don't see any advantage over Brainfish. I see many similarities (approach is practically the same except Brainfish uses hash what would be really the only logical way, and this one doesn't and that's practically all the novelty) and I see 2 huge drawbacks:
1) He uses Crafty for leaf analysis which is pathetically weak engine compared to SF.

He has already agreed to publish the source. Hence, it can be used with any open source engine.

2) Size is terms of number of positions is extremely small compared to Brainfish (26 times smaller) which guarantees much worse performance.

Brainfish code is not available (only the reader). You or anyone can make a book as large as they like.
I have about a billion analyzed positions. I think they might make a nice input.

[Dann Corbit]

Your conjecture seems doubtful to me.
It supposed that the quality of the move is always good.
Suppose that some moves are poor choices. They may do more damage than the time saved.
In my experience, all chess opening books have serious defects. It is simply too expensive to test all interior and leaf nodes to 37 plies or more to

It seems you haven't given much thought to this. Since OP book is created using exactly same Crafty engine, in test once Crafty is out of its own book, you will get exactly the same analysis of the given position that happened in the process of creation of the OP book. So unless move time during testing is longer than analysis time used when creating OP book (which is not the case), there is absolutely no way the move is inferior. Unless you claim that Crafty at depth N produces better move than the same Crafty at depth N+M.

Btw. I did proper testing of Brainfish book against same Brainfish without the book (so no compile speed differences) in a way that they start with the same TC, but once Brainfish with book is out of its book, the remaining time is reset for the Brainfish without book. This way the extra time due to book usage is removed as a factor, and you know what, difference was below 30Elo (TC was 60''+0.6'', single core).
And bear in mind Brainfish book is much, much stronger than OP book. So his 8.6% better result is mainly due to longer thinking time of Crafty with OP book.

I have not read his paper yet, but you did not address any of the points I raised.

[Milos]

I have not read his paper yet, but you did not address any of the points I raised.

Your points have nothing to do with his testing case (they address the case of general opening book, not books created like OP or brainfish), i.e. they are red herring, so no need to address them really. You particularly made doubt about my claim that if methodology is not proper most of OP book gain is actually gain due to longer thinking time. I explained you why, if you had counter arguments you'd present them instead of your laconi-c (forum software converts laconi-c into concise?!???) answer (and admitting you don't have all the facts).

[Dann Corbit]

I have not read his paper yet, but you did not address any of the points I raised.

Your points have nothing to do with his testing case (they address the case of general opening book, not books created like OP or brainfish), i.e. they are red herring, so no need to address them really. You particularly made doubt about my claim that if methodology is not proper most of OP book gain is actually gain due to longer thinking time. I explained you why, if you had counter arguments you'd present them instead of your laconi-c (forum software converts laconi-c into concise?!???) answer (and admitting you don't have all the facts).

It's OK if you don't understand it very well.
I don't have any problem with that.

[Milos]

It's OK if you don't understand it very well.
I don't have any problem with that.

It's ok if you play for a fool. It's obvious you don't understand the topic well (quite obvious from your lack of any argumentation from your side).

[Dann Corbit]

It's OK if you don't understand it very well.
I don't have any problem with that.

It's ok if you play for a fool. It's obvious you don't understand the topic well (quite obvious from your lack of any argumentation from your side).

You don't understand how the crafty book is generated.
You don't understand how the author's books are generated (there is a large collection, and each is a function of depth of analysis, not just eval)
You don't understand my expertize in understanding book construction and the errors they contain (ask any of the foremost book builders).

Basically, you have no understanding of the problem space, the proposed solution, or my suggestions.

Other than that, you have a great grasp of it.

You complain that there are not enough positions, but you can use the source to produce as many positions as you like.
You complain that it is not as good as Brainfish, but we do not know if that is true, and if it is you can use his source to make an even better book.
Brainfish is closed source and requires money.

This project is open source and can be coupled to any engine and can be used to produce data of any desired quality.

Brainfish you like, this one you don't.

That's just a few of the obvious things. There are several others as well.

[Milos]

You don't understand how the crafty book is generated.
You don't understand how the author's books are generated (there is a large collection, and each is a function of depth of analysis, not just eval)
You don't understand my expertize in understanding book construction and the errors they contain (ask any of the foremost book builders).

Basically, you have no understanding of the problem space, the proposed solution, or my suggestions.

Says a guy who by his own admission didn't even read the paper .
Compared to you, I have read the paper and I do very well understand how OP's book is created. I also have a very good understanding of how Brainfish book is created. You on the contrary don't understand any of those, haven't read the paper, don't understand what I write and try to lead some discussion. Gee. Sorry, but you are totally incompetent.
Once you provide single meaningful argument we can discuss, till then, you can discuss with other trolls, since trolling is the only thing you do on this thread since the very beginning (beside accusing me of constantly spreading hatred with no bases at all?!??)

P.S. I know very well you expertise in computer chess programming, no need to convince me more