TalkChess.com

From the excellent FastGM Rating List (FGRL), I downloaded the database for 10 minutes + 6 seconds increment rating list:
http://fastgm.de/10min.html

Each engine plays 2700 games. I started a bit of data mining with this database, and the first results already give some interesting hints on the "style" of top engines. I computed the distribution of won and lost games as game length by engine. Also, I computed the median of won games and of lost games. The results are here, many people are probably interpreting this data better than me. One thing strikes: Komodo 10.1 seems to win games way faster than it loses games, in the meaning that won games are way shorter. Houdini 4 is the opposite, it loses games much faster than it wins games, the lost games are shorter. It can be related to strengths of engines as phases of the game, opening, midgame, endgame. I hope some people will find it interesting, and express opinions on what this means. I don't know how these results are related to the absolute strength of engines, but the relation is not determinant, as "weak" Critter, for example, seems similar here to "strong" Stockfish, at least as the "mean" goes.
===============

Komodo 10.1:

Median won game:77 moves
Median lost game:91 moves
===============

Stockfish 7

Median won game:81 moves
Median lost game:83 moves
===============

Houdini 4

Median won game:89 moves
Median lost game:82.5 moves
===============

Andscacs 0.872

Median won game:80 moves
Median lost game:80 moves
===============

Critter 1.6a

Median won game:82 moves
Median lost game:85 moves
===============

Your puzzles get harder and harder...

Do not Komodo and SF and Houdini datapoints reveal what you already suggested: the stronger the engine, the quicker the mate-path, and the longer the loss-path?

Critter performs on equal footing with SF and better than H, as both SF and Critter are attacking engines, while this particular version of H, H4, is more of a slow positional player, unlike H3,H2,etc., and attacking players do tend to win/lose quicker than positional players.

I do not know about Andscacs, it is knowledgeable engine, but winning on average quicker than SF is a bit strange; it is potentially related to some Andscacs idiosyncracy, or the particular pool of players favours in some way Andscacs winning pattern.

But, basically, there are too many caveats in there:

1. Andreas Strangmueller plays until mate, so the mate-numbers are somehow, or considerably inflated (20+ moves after a winning positions is achieved, and in rapid games move length is generally less because of more mistakes)

2. All conclusions depend on the pool of players, 10 in this case, including also Fizbo, Equinox, etc., it is quite possible that in a different pool of players above diagrams will look somewhat different, depending on easy/difficult opponents/engine versions

3. 10' TC is prone to more irrational results with smaller game sample than LTC.

Overall, apart from stronger engine wins quicker and loses slower, and attacking engines tend to win/lose quicker on average with comparable strength than more positional engines, I do not spot any other pattern.

Lyudmil Tsvetkov wrote:
Overall, apart from stronger engine wins quicker and loses slower, and attacking engines tend to win/lose quicker on average with comparable strength than more positional engines, I do not spot any other pattern.

I somehow concur, at least we see which engine is more attacking. Could you see some influence on the phase of the game on engines' results? I roughly equaled for strength these 5 engines at ultra-fast controls in an ongoing test, after I complete the round-robin, I will post results for roughly equal in strength engines. But at ultra-fast, which may distort the results.

Thanks for this test!

In my tests I also see that Andscacs tend to win faster than other engines quite often, mostly by attack. I think is related to how is tuned, and also to a capture extension that most engines does not have that favour seeing some types of tactics faster. This capture extension is something that Andscacs is tuned around, so not so easy to add to any engine.

And the results of Komodo for me are clearly related to evaluation, but I suppose that the default contemp also accounts for a noticeable part of the difference with other engines, at least for the wining part.

Laskos wrote:

Lyudmil Tsvetkov wrote:
Overall, apart from stronger engine wins quicker and loses slower, and attacking engines tend to win/lose quicker on average with comparable strength than more positional engines, I do not spot any other pattern.

I somehow concur, at least we see which engine is more attacking. Could you see some influence on the phase of the game on engines' results? I roughly equaled for strength these 5 engines at ultra-fast controls in an ongoing test, after I complete the round-robin, I will post results for roughly equal in strength engines. But at ultra-fast, which may distort the results.

Sorry, Kai, I do not quite get your question.
Where should I see endgame data?

If you mean, by onlooking engine games, which engine would win faster in the eg, I guess all eg wins are slow ones, usually the engine that presses more and is more positional in nature would perform better, but search should also play bigger role in eg, as complexity only decreases, and even if you do not have good eval, but sufficient search, you would see things coming. Probably in this way SF, which has far from optimal eval, plays quite good in the eg.

So no pattern for me in the eg, just that more positional engines, like Komodo, would squeeze more long-winding wins, and also deep-searchers, like SF, would do so.

But as we all know, the foundation for the eg is laid already in the mg.