I wonder how good their searches and evaluations. On one hand, we know where to focus to make those engines be stronger. On the other hand, I may pick some good things for my needs, say, create a new Xiangqi chess engine by using a good search function + my own Xiangqi structure and evaluation. (Even though Stockfish has the best components, its license is GPL, not suitable for some tasks).
To compare, I have created some new chess engines by swapping their searches and evaluations, then make a tournament for them all using Banksia. The result is in the below table. A name “Fable 5.2 chess 24hrs sf19eval” mean the engine Fable 5.2 chess 24hrs + Stockfish 19 NNUE evaluation.
For short: the strongest AI engine Fable 5.2 chess 24hrs is behind Stockfish 19 about 110 Elo on the search and 279 Elo on the evaluation.
More details on my blog.
Code: Select all
# Engine Elo 95 % interval Score
-----------------------------------------------------------------
1 Stockfish 19 +0 (anchor) 651.5
2 Fable 5.2 chess 24hrs sf19eval -110 -147 .. -78 583.5
3 Opus 5.5 chess 24hrs sf19eval -114 -148 .. -83 581
4 Fable 5.1 chess 24hrs sf19eval -159 -194 .. -125 549
5 Opus 5 chess 24hrs sf19eval -161 -194 .. -134 547
6 Sonnet 5.5 chess 24hrs sf19eval -195 -229 .. -164 521.5
7 Fable 5 chess 24hrs sf19eval -219 -256 .. -189 503
8 Astra 6 chess 24hrs sf19eval -272 -305 .. -243 460.5
9 Stockfish 19 fable52eval -279 -313 .. -249 455.5
10 Haiku 5.5 chess 24hrs sf19eval -299 -336 .. -267 439
11 Sonnet 5 chess 24hrs sf19eval -334 -370 .. -302 411.5
12 Fable 5.2 chess 24hrs -335 -372 .. -305 410.5
13 Opus 5.5 chess 24hrs -405 -443 .. -374 356.5
14 Sonnet 5.5 chess 24hrs -454 -492 .. -421 320.5
15 Fable 5.1 chess 24hrs -612 -657 .. -577 219.5
16 Opus 5 chess 24hrs -636 -681 .. -604 206.5
17 Astra 6 chess 24hrs -686 -731 .. -647 181
18 Fable 5 chess 24hrs -802 -859 .. -758 130
19 Haiku 5.5 chess 24hrs -1125 -1196 .. -1075 37.5
20 Sonnet 5 chess 24hrs -1138 -1209 .. -1076 35
