mvanthoor wrote: ↑Sat Sep 26, 2020 3:23 am
syzygy wrote: ↑Sat Sep 26, 2020 2:05 am
Aren't most TCEC openings deliberately chosen to give white an advantage?
Why would you want to do that?
Personally, I'd like to see how engines play the openings by themselves; so give them an 8 ply opening book, and then let them go at it on their own.
Remember that TCEC is not a testing site. And TCEC wants to avoid draws as much as possible, TCEC wants views. Wins means more views is their calculus.
To be fair here it is in TCEC own words. And how TCEC explains the choice of openings, for good or ill.
Just remember TCEC is not a testing site, it is a exhibition. And TCEC will play openings that the engines themselves would never play. So this favors the more tactical engines such as Stockfish. As pure positional play has been taken out of the equation, and more so as you get closer to the super final.
TCEC Openings FAQ
Q: Why do TCEC chess games not start from the opening position?
A: The answer is rooted on our site’s general objectives, past experience trying different play formats, and our judgment on what makes an entertaining competition and therefore a successful website.
· Our primary goal is to provide our web audience with fair and entertaining chess competitions.
· We believe that a champion chess program ought to be able to perform well in a wide variety of opening systems, not merely those that it favors. This approach is different from human tournaments by design.
· We see the traditional opening position as merely one position out of many thousands that might present a chess program with a challenge. What distinguishes the opening position in computer chess is that it is relatively balanced, meaning it has a high probability of resulting in a draw when played by top competitors at long time controls on very strong hardware.
· Many chess programs repeatedly play a given position the same way. When two such programs meet, they invariably produce repetitive contests. A 100-game test we ran between leading engines some time ago resulted in 88% of the games concentrating into just five ECOs. Such outcomes frustrate the majority’s desire for entertaining chess.
· The fact that different competitors must cope with an unpredictable and unequal set of openings does add an element of chance to the competition. Mitigating this chance element,
1) competitors play the selected openings from both sides of the board in consecutive games,
2) the length of our season tournaments assures a wide variety of openings, and
3) the openings tend, over time, to balance out in terms of drawishness and bias.
This latter point underscores the principal risk to our approach, namely, that competitors can be saddled with openings that are too draw-prone or too one-sided, resulting in game-pairs that fail to differentiate the competitors on the cross-table. Our openings team is skilled at selecting opening positions that are neither hopelessly draw-prone nor so one-sided that the weaker side has no chance of holding the draw.
Q: Who selects the openings?
A: Nelson Hernandez (“CatoTheYounger_TCEC”) selects openings in all tournament stages except the Superfinal, which is selected by Jeroen Noomen (“Jeroen_TCEC”).
Nelson took the openings reins in 2013, mid-way through Season 5, during founder Martin Thoresen’s period of tournament directorship. Nelson’s is primarily known for his proprietary ‘Catobase’, a database of human and engine games now over 15 years in active development. He was a participant in Anson Williams’ three-time champion freestyle chess team. He has been enthralled by chess programs since 1982, when he bought Sargon II for his Apple II+.
Jeroen took responsibility for the Superfinal at Nelson’s invitation before Season 9. He has been active in computer chess since the early 1980s. He has been associated with the Rebel and Rybka programs in the past as an opening book consultant and over the years has participated in numerous live computer chess tournaments in the Netherlands and elsewhere.
Q: How are the openings structured?
A: Each seasonal tournament, from the Qualifying League through the Superfinal, has a different opening ‘book’ that issues starting positions:
Tournament Author Book Length # Positions Selection Opening Bias Perpetual
Qualifying League Nelson 4 moves 557 random none yes
League 2 Nelson 6 moves 629 random none yes
League 1 Nelson 8 moves 1000 random low/moderate yes
Premier League Nelson 8 moves 1000 random moderate/high yes
Superfinal Jeroen varied 50 sequential high/very high no
# Positions signifies the number of unique positions contained within each book.
Selection indicates how positions contained within the books are selected. (Thus, in League 1 there are 1,000 unique positions in the book which are chosen at random prior to each new game-pair.)
Opening Bias indicates the degree to which the average opening is skewed to favor one color or the other. (An exception to this is that Jeroen sometimes starts the Superfinal with two bookless games.)
Perpetual indicates that the first four leagues get starting positions from the same book from season to season whereas Superfinal books are for one-time use with a new book each season.
Q: How were positions within the books selected?
A: For the Qualifying League and League 2, Nelson has tried to balance three objectives:
1) to deliver the widest variety of opening systems,
2) to keep the overall draw-rate under 80%, and
3) to limit the instances of same-color wins from game-pairs.
To find these positions Nelson did a comprehensive search of his Catobase to look for positions that met various criteria within his empirical and evaluative data. He avoided book-exit positions where the next move was obvious, where pieces already had been (or were about to be) exchanged, or which had happened very rarely in his database. To achieve a subsidiary objective, he constructed these two books in such a way that openings selected tend to reflect human preferences as indicated in his database.
For League 1 and Premier League the caliber of the competition is much higher and longer time controls at these levels further boost play-quality. For this reason opening bias must be introduced to a progressively higher degree otherwise the draw-rate will soar to unacceptable levels.
In response, two 8-move books were created for the top two leagues. In terms of “ECO-diversity” they are rather similar but the most volatile openings within each opening system were assigned to the Premier League. Separately, the preference for human openings seen in the first two leagues of the season is not present in these books; the positions are informed by tens of millions of Catobase games.
Jeroen’s approach to the Superfinal is less statistical and much more chess knowledge-driven. His process is to use leading chess engines to check if a line is obviously drawish or is certain to lead to heavy exchanges. He judges whether a given position offers the prospect of a good fight with multiple playing options for both colors. His opening picks always include a wide variety of systems, pawn structures and themes. There are positional and sharp openings, those with short and long castling, gambits, main lines, antique and topical lines, and occasionally a dubious line. (As long as dubious lines are not hopelessly one-sided we think they are fine.)
Note that both Nelson and Jeroen freely acknowledge that few traditional and neural net engines would find themselves in most TCEC opening positions of their own accord. This is deliberate. The whole point is to force engines to play unexpected positions and demonstrate their strength in all kinds of game situations. This is especially true for neural nets, which in training tend to emphasize favorite lines and neglect what they deem to be inferior ones.
Q: What do you mean when you say you “bias” openings, and why do you do it?
A: Bias is the degree to which a particular opening, or a set of openings, seem to favor one side or the other according to empirical data (historical outcomes) and evaluative data (deep analysis from top engines).
The opening position typically offers white a slightly favorable evaluation of +0.15 to +0.20 on traditional chess programs where the “comtempt” setting is neutralized. Low or zero bias in an opening book makes little difference in a tournament where time controls are shorter and contestants have a higher range of Elos. In that case, there will be ample decisive games.
However, if very strong, closely matched programs are playing at long time controls with no bias that invites a draw-rate approaching 90%. To combat this, we introduce bias in League 1 and to an even greater degree in Premier League and the Superfinal. In the latter two, you will see some openings where one color has no realistic winning chances and the game is all about successfully holding a draw from an almost desperate position.
Q: To what degree do you bias openings?
A: In League 1 Nelson is in his comfort zone when opening evaluations in traditional chess engines (i.e. not neural nets) are between +/-0.40 and +/-0.65. In Premier League, a score of +/-0.50 to +/-0.80 is more common. Sometimes he will deviate from these guidelines to the upside or downside to satisfying opening variety goals.
In each season’s Superfinal Jeroen has a free hand to do whatever he thinks will result in an interesting and varied contest. Sometimes his book-exit evaluations will exceed +/-1.00; sometimes he will offer speculative gambits. His goal is to keep the draw-rate in the 65-80% range each season without a surfeit of one-sided openings.
Q: Why such an aversion to draws? If chess is naturally drawish, why try to distort that reality?
A: The ultimate objective of a tournament is to crown a deserving champion. If you play 100 Superfinal games and 95 of them are draws you will not only drive away much of your audience, you wind up crowning a champion that may have only scored three wins against his opponent’s two. In that event, statistics tell us that the winner may have qualified for the Superfinal based on skill but may have won it entirely on luck. A greater number of decisive game-pairs (i.e. 1.5-0.5, 2-0) reduce the probability of luck being the critical factor as there is a greater chance of one engine pulling away for a clear win.
While we do not object to close contests in the Superfinal—on the contrary—we would prefer as many decisive (i.e. not tied) game-pairs as possible, as this way the strongest competitor will have the greatest chance to demonstrate all-around superiority. Plus, it is more entertaining!
Q: How long can you maintain your targeted 65-80% draw-rate in the Superfinal when engines are continually improving?
A: Draw abatement will definitely be more challenging as the competitors increase in strength in the years ahead. A mental comparison between today’s leading chess programs and those of ten years ago, combined with extrapolation into the future, points to a time where something will have to give. On the other hand, as long as neural nets and the best alpha-beta engines remain competitive with each other their contrasting play-styles may help keep draw-rates lower than they otherwise might have been.
Q: What impact will the advent of neural nets and their very different evaluative frame of reference have on your opening set selections?
A: This is impossible to answer as each executable/neural net pairing is unique and presents a different behavioral profile with respect to strength, resistance to draws, evaluation volatility and reliability, etc. The whole field of AI is undergoing rapid development with innovations happening continually; the competitive landscape changes from season to season in surprising ways.
That said, TCEC’s history is one of gradual evolution in response to emerging trends and practical experience; we expect that to continue.