Low impact of opening phase in engine play?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Laskos
Posts: 9417
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Low impact of opening phase in engine play?

Post by Laskos » Thu Apr 20, 2017 12:10 am

Ras wrote:
The basic aspect of an untuned book is "only" time saving.
You simply do not understand the purpose of this experiment. I am using the 3-mover GM book for variety of openings and reasonable, balanced openings. 2moves_v1.epd does not offer reasonable balanced openings. I am not interested in the value of 20-move tuned book, which might be worth hundreds of Elo points even between equal engines in ultra-fast games. And the experiment measures the (played by engine) contribution of opening phase of the game (first 12 moves) to general play starting with fairly equal positions, preferably as close as possible to the standard opening position. If I wanted to measure the impact of the book to gameplay, I would pit two identical Stockfishes, one with a book, the other without.

User avatar
Laskos
Posts: 9417
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Low impact of opening phase in engine play?

Post by Laskos » Thu Apr 20, 2017 9:56 am

I repeated the test from 3moves_GM.epd file, this time pitting Stockfish dev and Cheng 4.39. I performed the same for moves 12,24,36,48,60,72,84 to see the contribution of each move interval.

Stockfish - Cheng: 272 Elo points difference
Contributions by moves:

1 - 12 ---> 54 Elo points (20%)
13 - 24 ---> 90 Elo points (33%)
25 - 36 ---> 42 Elo points (15%)
37 - 48 ---> 31 Elo points (11%)
49 - 60 ---> 17 Elo points (6.3%)
61 - 72 ---> 13 Elo points (4.8%)
73 - 84 ---> 9 Elo points (3.3%)

Image
It is again visible that the opening phase has lower impact than early middlegame phase for game-play results, with endgames decaying in importance.

Interesting to note that this result should have an impact on time management. From Ferdinand Mosca chart on time used each move by Stockfish:
Image
we see that Stockfish with 1'+1'' and 2'+0'' time controls follow closely my chart, and probably using a small increment is the most efficient time control for Stockfish. But one point is missed: in openings less time should be used. I think everybody assumes that per move, openings are the most important in game-play, but it seems that it is not the case, moves 13-24 seem more important.

Milos
Posts: 3387
Joined: Wed Nov 25, 2009 12:47 am

Re: Low impact of opening phase in engine play?

Post by Milos » Thu Apr 20, 2017 12:42 pm

Laskos wrote:You simply do not understand the purpose of this experiment. I am using the 3-mover GM book for variety of openings and reasonable, balanced openings. 2moves_v1.epd does not offer reasonable balanced openings. I am not interested in the value of 20-move tuned book, which might be worth hundreds of Elo points even between equal engines in ultra-fast games. And the experiment measures the (played by engine) contribution of opening phase of the game (first 12 moves) to general play starting with fairly equal positions, preferably as close as possible to the standard opening position. If I wanted to measure the impact of the book to gameplay, I would pit two identical Stockfishes, one with a book, the other without.
It is actually not that far one from the other. If you play Brainfish against SF and limit Brainfish book to first N moves, it would be similar as playing SF against itself where one version has Kx more time for the first N moves. Since you can estimate how much Kx more time brings in terms of Elo you could derive a portion that first N moves bring.

Ras
Posts: 1143
Joined: Tue Aug 30, 2016 6:19 pm
Contact:

Re: Low impact of opening phase in engine play?

Post by Ras » Thu Apr 20, 2017 11:21 pm

Laskos wrote:You simply do not understand the purpose of this experiment.
To cut it short: if you design your experiment so that you ignore the main possible strength implications of an opening book, then the strength results, ELO-wise, will not matter that much. Big deal here.

User avatar
Laskos
Posts: 9417
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Low impact of opening phase in engine play?

Post by Laskos » Fri Apr 21, 2017 3:26 am

Milos wrote:
Laskos wrote:You simply do not understand the purpose of this experiment. I am using the 3-mover GM book for variety of openings and reasonable, balanced openings. 2moves_v1.epd does not offer reasonable balanced openings. I am not interested in the value of 20-move tuned book, which might be worth hundreds of Elo points even between equal engines in ultra-fast games. And the experiment measures the (played by engine) contribution of opening phase of the game (first 12 moves) to general play starting with fairly equal positions, preferably as close as possible to the standard opening position. If I wanted to measure the impact of the book to gameplay, I would pit two identical Stockfishes, one with a book, the other without.
It is actually not that far one from the other. If you play Brainfish against SF and limit Brainfish book to first N moves, it would be similar as playing SF against itself where one version has Kx more time for the first N moves. Since you can estimate how much Kx more time brings in terms of Elo you could derive a portion that first N moves bring.
Yes, in particular case of Cerebellum book, that will be close. There will be some issues like exiting early the book or gauging the general Elo difference, as Kx is very large, my games are at 200ms/move, Cerebellum has positions analyzed for some 1000 seconds or so, that's 12 doublings, and one cannot just say like "70 Elo points per doubling".

Post Reply