Low impact of opening phase in engine play?

[Laskos]

The basic aspect of an untuned book is "only" time saving.

You simply do not understand the purpose of this experiment. I am using the 3-mover GM book for variety of openings and reasonable, balanced openings. 2moves_v1.epd does not offer reasonable balanced openings. I am not interested in the value of 20-move tuned book, which might be worth hundreds of Elo points even between equal engines in ultra-fast games. And the experiment measures the (played by engine) contribution of opening phase of the game (first 12 moves) to general play starting with fairly equal positions, preferably as close as possible to the standard opening position. If I wanted to measure the impact of the book to gameplay, I would pit two identical Stockfishes, one with a book, the other without.

[Laskos]

I repeated the test from 3moves_GM.epd file, this time pitting Stockfish dev and Cheng 4.39. I performed the same for moves 12,24,36,48,60,72,84 to see the contribution of each move interval.

Stockfish - Cheng: 272 Elo points difference
Contributions by moves:

1 - 12 ---> 54 Elo points (20%)
13 - 24 ---> 90 Elo points (33%)
25 - 36 ---> 42 Elo points (15%)
37 - 48 ---> 31 Elo points (11%)
49 - 60 ---> 17 Elo points (6.3%)
61 - 72 ---> 13 Elo points (4.8%)
73 - 84 ---> 9 Elo points (3.3%)


It is again visible that the opening phase has lower impact than early middlegame phase for game-play results, with endgames decaying in importance.

Interesting to note that this result should have an impact on time management. From Ferdinand Mosca chart on time used each move by Stockfish:

we see that Stockfish with 1'+1'' and 2'+0'' time controls follow closely my chart, and probably using a small increment is the most efficient time control for Stockfish. But one point is missed: in openings less time should be used. I think everybody assumes that per move, openings are the most important in game-play, but it seems that it is not the case, moves 13-24 seem more important.

[Milos]

You simply do not understand the purpose of this experiment. I am using the 3-mover GM book for variety of openings and reasonable, balanced openings. 2moves_v1.epd does not offer reasonable balanced openings. I am not interested in the value of 20-move tuned book, which might be worth hundreds of Elo points even between equal engines in ultra-fast games. And the experiment measures the (played by engine) contribution of opening phase of the game (first 12 moves) to general play starting with fairly equal positions, preferably as close as possible to the standard opening position. If I wanted to measure the impact of the book to gameplay, I would pit two identical Stockfishes, one with a book, the other without.

It is actually not that far one from the other. If you play Brainfish against SF and limit Brainfish book to first N moves, it would be similar as playing SF against itself where one version has Kx more time for the first N moves. Since you can estimate how much Kx more time brings in terms of Elo you could derive a portion that first N moves bring.

[Ras]

You simply do not understand the purpose of this experiment.

To cut it short: if you design your experiment so that you ignore the main possible strength implications of an opening book, then the strength results, ELO-wise, will not matter that much. Big deal here.

[Laskos]

You simply do not understand the purpose of this experiment. I am using the 3-mover GM book for variety of openings and reasonable, balanced openings. 2moves_v1.epd does not offer reasonable balanced openings. I am not interested in the value of 20-move tuned book, which might be worth hundreds of Elo points even between equal engines in ultra-fast games. And the experiment measures the (played by engine) contribution of opening phase of the game (first 12 moves) to general play starting with fairly equal positions, preferably as close as possible to the standard opening position. If I wanted to measure the impact of the book to gameplay, I would pit two identical Stockfishes, one with a book, the other without.

It is actually not that far one from the other. If you play Brainfish against SF and limit Brainfish book to first N moves, it would be similar as playing SF against itself where one version has Kx more time for the first N moves. Since you can estimate how much Kx more time brings in terms of Elo you could derive a portion that first N moves bring.

Yes, in particular case of Cerebellum book, that will be close. There will be some issues like exiting early the book or gauging the general Elo difference, as Kx is very large, my games are at 200ms/move, Cerebellum has positions analyzed for some 1000 seconds or so, that's 12 doublings, and one cannot just say like "70 Elo points per doubling".