TalkChess.com

Strong engines playing each other draw more often than weak engines playing each other.

Now, if we draw a graph (elo/draw%), find a best fit curve and extrapolate it, we should be able to estimate the strength of engines playing perfect chess (=100% draws).

Does this seem reasonable? Has it been done?

Yes, I have done some study on CCRL data, by filtering only programs which have an average opposition within 20 elo of their rating.

A better study would be to filter on a game by game basis instead of player by player, but I've had no luck with database programs (CCRL database seems too big for SCID to filter).

Below the results.

Very reasonable result. Between 3800 and 5000 CCRL ELO points was my result with diminishing returns (ELO value of doubling time or nodes).

If engines make 50 ELO in a year then we need to wait ten years before we know 3800 ELO is right estimate for perfect play.

Why would not it be a S-curve ?

I don't think extrapolations are logical when it comes to current chess engines. Consider the concepts that engines do not grasp at all, such as "always/never" (different types of fortresses and dominations) or cyclic improvements (see Blathy studies): engines built around the selective brute force approach cannot grasp these, so will never play perfect. On the other hand, if a new type of engine is built (e.g. neural networks), then the graph related to the "old type of engines" is completely irrelevant

Have fun though

In TCEC superfinal, engines gets 90% of Draw easy even on imbalanced opening. They play with a strenght around 3500, so I don't think the graph here is accurate.

100% draws does not necessarily mean perfect play. It only means that the difference of the imperfections is not large enough to push closely matched players out of the draw zone.

Kornrade wrote:I don't think extrapolations are logical when it comes to current chess engines. Consider the concepts that engines do not grasp at all, such as "always/never" (different types of fortresses and dominations) or cyclic improvements (see Blathy studies): engines built around the selective brute force approach cannot grasp these, so will never play perfect. On the other hand, if a new type of engine is built (e.g. neural networks), then the graph related to the "old type of engines" is completely irrelevant

Have fun though

There is nothing special about chess "with different types of domination" and other fancy things. The odds are it will be weakly solved a-la checkers using this type of engines as today in some decades to come. With neural networks using alpha/beta search and reductions, Giraffe chess engine enjoys pretty same draw rates and gains with time/depth.

megamau wrote:Yes, I have done some study on CCRL data, by filtering only programs which have an average opposition within 20 elo of their rating.

A better study would be to filter on a game by game basis instead of player by player, but I've had no luck with database programs (CCRL database seems too big for SCID to filter).

Use the tool "eloGap" in 40H-PGN tools (see www below for link). It is a command line tool that does game by game filtering based on Elo difference.

From its readme:

"eloGap" extracts games in which the players' Elo gap (absolute
value of their Elo difference) is less than or equal to a
user-specified maximum Elo gap.

For example:

eloGap alpha.pgn 100

extracts all games in alpha.pgn with an Elo gap from 0 to 100.

The output file is "outF.pgn". Games not extracted to "outF.pgn"
are in "excludeF.pgn".

"eloGap" does not extract games missing an Elo tag.

Syntax: eloGap filename.pgn elo_distance

Example: eloGap alpha.pgn 100

Output: outF.pgn, excludeF.pgn