How much benefit from opening book?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
jack512
Posts: 19
Joined: Sat Nov 14, 2015 3:29 pm

How much benefit from opening book?

Post by jack512 » Sun Mar 06, 2016 2:24 pm

In today's search engines, how many extra ISO rating points do today's opening books buy you? I.e. the same engine with vs. without the OB. Does the benefit decrease as the search engine itself becomes more powerful, say with a faster processor?

jdart
Posts: 3825
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: How much benefit from opening book?

Post by jdart » Sun Mar 06, 2016 3:16 pm

I think it is significant. But book vs. no book is probably not a good way to measure it. Without books engines frequently depart from known theory early in the game.

The real advantage shows when a poor or shallow book is matched against a very good book. The engine with the weaker book will play into inferior lines and get busted.

Sedat Canbaz ran a series of tournaments just to test opening books fairly recently (https://sites.google.com/site/computers ... ournaments). There were significant differences depending on the book used.

--Jon

jack512
Posts: 19
Joined: Sat Nov 14, 2015 3:29 pm

Re: How much benefit from opening book?

Post by jack512 » Sun Mar 06, 2016 4:25 pm

Thx for the link. In Sedat's tournament https://sites.google.com/site/computerschess/scct-cs-6,
19 different books are compared all using a fixed engine, namely Stockfish 7 beta 1 x64.

One of the books is "No Book", which is the engine playing without a book.
It finished 6th out of 19, with a score of 3421.4. Number 1 was "SCCT Champions at 3438.0. 19th place score was 3339.2.

I am astonished that No Book would perform so well, finishing ahead of 13 out of 19 books. Or am I misinterpreting something?

User avatar
Guenther
Posts: 3053
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: How much benefit from opening book?

Post by Guenther » Sun Mar 06, 2016 4:34 pm

jack512 wrote:Thx for the link. In Sedat's tournament https://sites.google.com/site/computerschess/scct-cs-6,
19 different books are compared all using a fixed engine, namely Stockfish 7 beta 1 x64.

One of the books is "No Book", which is the engine playing without a book.
It finished 6th out of 19, with a score of 3421.4. Number 1 was "SCCT Champions at 3438.0. 19th place score was 3339.2.

I am astonished that No Book would perform so well, finishing ahead of 13 out of 19 books. Or am I misinterpreting something?
The test you refer to was only for books with max. 8 moves.

elcabesa
Posts: 815
Joined: Sun May 23, 2010 11:32 am
Contact:

Re: How much benefit from opening book?

Post by elcabesa » Sun Mar 06, 2016 4:50 pm

some time ago , the italian g-sei site has done a BOOK TOURNAMENT. The used engine was Stockfish and the result of the first tournament was similar to the result achieved by sedat, NO-BOOK is better than a not good book.

full result ( in italian) ca be found at http://www.g-sei.org/il-primo-torneo-de ... aperture2/

reported below the result of the tournament:
01 RpC-Needle 17.0/24
02 Fauzi 3.4 15.0/24
03 None 14.5/24
04 Diffusion 1.0.5A2 12.5/24
05 NathanaelRussell 2014 12.0/24
06 Aaricia 2012 12.0/24
07 Perfect 2014t 12.0/24
08 Performance 2005 11.5/24
09 SS-ForChiron 1.5 11.5/24
10 Superbook 0.2 11.0/24
11 HS-Grandbook 1.8 10.0/24
12 Ref2550 Lite 8.5/24

jack512
Posts: 19
Joined: Sat Nov 14, 2015 3:29 pm

Re: How much benefit from opening book?

Post by jack512 » Sun Mar 06, 2016 5:15 pm

I suppose the explanation must be the mismatch between the goals of the player whose games went into the opening book, versus what the engine is able to see of its own goals when it comes out of the book.

tpoppins
Posts: 919
Joined: Tue Nov 24, 2015 8:11 pm
Location: upstate

Re: How much benefit from opening book?

Post by tpoppins » Mon Mar 07, 2016 4:19 am

jack512 wrote:In Sedat's tournament https://sites.google.com/site/computerschess/scct-cs-6,
19 different books are compared all using a fixed engine, namely Stockfish 7 beta 1 x64.

One of the books is "No Book", which is the engine playing without a book.
It finished 6th out of 19, with a score of 3421.4. Number 1 was "SCCT Champions at 3438.0. 19th place score was 3339.2.

I am astonished that No Book would perform so well, finishing ahead of 13 out of 19 books. Or am I misinterpreting something?
With so few games these numbers don't mean much. E.g., the difference between #1 and #6 is a mere 17 Elo. To have a decently high LOS in this case you'd need somewhere between 1000 and 2000 games (Kai or Ferd would give you an exact minimum); only 50 were played. The Italian tmt had even fewer games and the results there are even harder to take seriously.

These tournaments are mostly done for the sake of entertainment, methinks.

bob
Posts: 20559
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: How much benefit from opening book?

Post by bob » Mon Mar 07, 2016 4:49 am

jack512 wrote:Thx for the link. In Sedat's tournament https://sites.google.com/site/computerschess/scct-cs-6,
19 different books are compared all using a fixed engine, namely Stockfish 7 beta 1 x64.

One of the books is "No Book", which is the engine playing without a book.
It finished 6th out of 19, with a score of 3421.4. Number 1 was "SCCT Champions at 3438.0. 19th place score was 3339.2.

I am astonished that No Book would perform so well, finishing ahead of 13 out of 19 books. Or am I misinterpreting something?
"the number of games..."

Dann Corbit
Posts: 10122
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: How much benefit from opening book?

Post by Dann Corbit » Mon Mar 07, 2016 10:02 am

Ed got over 100 Elo benefit in a test.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Jouni
Posts: 2005
Joined: Wed Mar 08, 2006 7:15 pm

Re: How much benefit from opening book?

Post by Jouni » Mon Mar 07, 2016 10:35 am

In CSS forum there was really stunning result with Perfect2015:

Code: Select all

Stockfish 7 x64 popcnt       - Stockfish 7 x64 popcnt oBuch  : 301,0/600 48-46-506
Stockfish 7 x64 popcnt oBuch - Stockfish 7 x64 popcnt        : 299,0/600 46-48-506

Komodo-8-64bit        - Komodo-8-64bit oBuch  : 278,5/506 95-44-367
Komodo-8-64bit oBuch  - Komodo-8-64bit        : 227,5/506 44-95-367

:shock: :P
Jouni

Post Reply