Differences between top engines related to "style"
Posted: Fri Oct 07, 2016 4:25 pm
From the excellent FastGM Rating List (FGRL), I downloaded the database for 10 minutes + 6 seconds increment rating list:
http://fastgm.de/10min.html
Each engine plays 2700 games. I started a bit of data mining with this database, and the first results already give some interesting hints on the "style" of top engines. I computed the distribution of won and lost games as game length by engine. Also, I computed the median of won games and of lost games. The results are here, many people are probably interpreting this data better than me. One thing strikes: Komodo 10.1 seems to win games way faster than it loses games, in the meaning that won games are way shorter. Houdini 4 is the opposite, it loses games much faster than it wins games, the lost games are shorter. It can be related to strengths of engines as phases of the game, opening, midgame, endgame. I hope some people will find it interesting, and express opinions on what this means. I don't know how these results are related to the absolute strength of engines, but the relation is not determinant, as "weak" Critter, for example, seems similar here to "strong" Stockfish, at least as the "mean" goes.
===============
Komodo 10.1:
Median won game:77 moves
Median lost game:91 moves
===============
Stockfish 7
Median won game:81 moves
Median lost game:83 moves
===============
Houdini 4
Median won game:89 moves
Median lost game:82.5 moves
===============
Andscacs 0.872
Median won game:80 moves
Median lost game:80 moves
===============
Critter 1.6a
Median won game:82 moves
Median lost game:85 moves
===============
http://fastgm.de/10min.html
Each engine plays 2700 games. I started a bit of data mining with this database, and the first results already give some interesting hints on the "style" of top engines. I computed the distribution of won and lost games as game length by engine. Also, I computed the median of won games and of lost games. The results are here, many people are probably interpreting this data better than me. One thing strikes: Komodo 10.1 seems to win games way faster than it loses games, in the meaning that won games are way shorter. Houdini 4 is the opposite, it loses games much faster than it wins games, the lost games are shorter. It can be related to strengths of engines as phases of the game, opening, midgame, endgame. I hope some people will find it interesting, and express opinions on what this means. I don't know how these results are related to the absolute strength of engines, but the relation is not determinant, as "weak" Critter, for example, seems similar here to "strong" Stockfish, at least as the "mean" goes.
===============
Komodo 10.1:
Median won game:77 moves
Median lost game:91 moves
===============
Stockfish 7
Median won game:81 moves
Median lost game:83 moves
===============
Houdini 4
Median won game:89 moves
Median lost game:82.5 moves
===============
Andscacs 0.872
Median won game:80 moves
Median lost game:80 moves
===============
Critter 1.6a
Median won game:82 moves
Median lost game:85 moves
===============