Fable 5.2 chess 24hrs: 3542 Elo, written from scratch by an AI in 24 hours

[Steve Maughan]

Hi Steve,

Fable 5.2 avx2 added to AI-List on R9: https://ipmanchess.yolasite.com/r9-7945 ... ngines.php

Looks like it needs +352Elo to reach Stockfish..

Ipman.

Cool - thanks for testing!

The good news is there is still some headroom in the benchmark for new AI models to improve further.

— Steve

[Bart Weststrate]

3288 makes more sense indeed. The question is or 3500 can be reached with such a small net. I don't think so.

[Steve Maughan]

3288 makes more sense indeed. The question is or 3500 can be reached with such a small net. I don't think so.

I was using the CCRL blitz scale, in which case Fable 5.2 is certainly over 3500 Elo.

— Steve

[Uri Blass]

3288 makes more sense indeed. The question is or 3500 can be reached with such a small net. I don't think so.

I was using the CCRL blitz scale, in which case Fable 5.2 is certainly over 3500 Elo.

— Steve

Looked at the CCRL blitz rating and if Fable5.2 is over 3500 elo then it means difference of less than 350 elo relative to stockfish19 8 cpu when I see 352 elo difference from the top in https://ipmanchess.yolasite.com/r9-7945 ... ngines.php

https://computerchess.org.uk/404/

[Raphexon]

3288 makes more sense indeed. The question is or 3500 can be reached with such a small net. I don't think so.

I was using the CCRL blitz scale, in which case Fable 5.2 is certainly over 3500 Elo.

— Steve

Looked at the CCRL blitz rating and if Fable5.2 is over 3500 elo then it means difference of less than 350 elo relative to stockfish19 8 cpu when I see 352 elo difference from the top in https://ipmanchess.yolasite.com/r9-7945 ... ngines.php

https://computerchess.org.uk/404/

CCRL is super drawish. 3500 elo on CCRL Blitz is like Stockfish 9?
Not too difficult for somebody using nnue.

[Bart Weststrate]

3288 makes more sense indeed. The question is or 3500 can be reached with such a small net. I don't think so.

I was using the CCRL blitz scale, in which case Fable 5.2 is certainly over 3500 Elo.

— Steve

Looked at the CCRL blitz rating and if Fable5.2 is over 3500 elo then it means difference of less than 350 elo relative to stockfish19 8 cpu when I see 352 elo difference from the top in https://ipmanchess.yolasite.com/r9-7945 ... ngines.php

https://computerchess.org.uk/404/

CCRL is super drawish. 3500 elo on CCRL Blitz is like Stockfish 9?
Not too difficult for somebody using nnue.

SF 9 is 3694

[Bart Weststrate]

3288 makes more sense indeed. The question is or 3500 can be reached with such a small net. I don't think so.

I was using the CCRL blitz scale, in which case Fable 5.2 is certainly over 3500 Elo.

— Steve

Clear, I never looked there but that is +500 above the 40/15 list. Didn't knew that.

[wwenioaguiar]

Steve, thanks for running this. It made me curious about the other half of the question: the engine Fable wrote plays at 3542, but how well do these models play when they have to choose every move themselves, with no engine and no search?

I run a small arena where LLM agents play rated games move by move (engines are not allowed there), so I have some early numbers. Small samples against a random mover, so take them as anecdotes:

- Gemini 3.5 Flash-Lite, given the move history, the FEN and the list of legal moves: 4 mates and 1 draw by repetition in 5 games.
- Gemma 4 26B, same setup: also 4 mates in 5, plus one draw by repetition while 24 points of material up. Even with the legal-move list in the prompt, 26 of its 176 answers were moves that were not on the list.
- Llama 3.2 3B: mostly legal moves, but aimless.

The pattern so far: they win material easily and then struggle to finish. Shuffling pieces in a won endgame until threefold repetition is the most common failure.

So right now the gap between "can write a strong engine" and "can play decent chess by itself" looks huge. Has anyone here measured the frontier models (Fable 5.2 included) playing directly, without tools?

(The arena is https://www.llmpvp.com, free, if someone wants to plug in a model.)

[Uri Blass]

3288 makes more sense indeed. The question is or 3500 can be reached with such a small net. I don't think so.

I was using the CCRL blitz scale, in which case Fable 5.2 is certainly over 3500 Elo.

— Steve

Looked at the CCRL blitz rating and if Fable5.2 is over 3500 elo then it means difference of less than 350 elo relative to stockfish19 8 cpu when I see 352 elo difference from the top in https://ipmanchess.yolasite.com/r9-7945 ... ngines.php

https://computerchess.org.uk/404/

CCRL is super drawish. 3500 elo on CCRL Blitz is like Stockfish 9?
Not too difficult for somebody using nnue.

SF 9 is 3694

I do not see stockfish9 there.

https://computerchess.org.uk/404/cgi/co ... +opponents

I see stockfish10 1 cpu with 3508 ccrl rating list

https://computerchess.org.uk/404/cgi/en ... _10_64-bit


3694 for SF9 is for FRC chess
https://computerchess.org.uk/404FRC/cgi ... _length=30