Engine Shootout: Qualitative analysis of Stockfish, Dragon, and Theoria

[FireDragon761138]

Never heard of Theoria. Another clone/derivative?

probably just took a weaker nnue from fishtest. For some reason they think stockfish is meant to teach 1500 how to play chess.

the NNUE was not taken from fishtest. It was trained using Pytorch-NNUE and filtered Lc0 training sets evaluated by Lc0.

so same data used by any other nnue used by stockfish. Going to ignore the 2nd half of that message?

No, not the exact same data. I only used one kind of data, from lc0 games, because i wanted to maintain conceptual coherence and positional clarity in the evaluation. Stockfish essentially looks for forcing moves over piece harmony or other considerations.

[lucario6607]

Never heard of Theoria. Another clone/derivative?

probably just took a weaker nnue from fishtest. For some reason they think stockfish is meant to teach 1500 how to play chess.

the NNUE was not taken from fishtest. It was trained using Pytorch-NNUE and filtered Lc0 training sets evaluated by Lc0.

so same data used by any other nnue used by stockfish. Going to ignore the 2nd half of that message?

No, not the exact same data. I only used one kind of data, from lc0 games, because i wanted to maintain conceptual coherence and positional clarity in the evaluation. Stockfish essentially looks for forcing moves over piece harmony or other considerations.

You have yet to prove anything and what run did you take the Leela data from.

[FireDragon761138]

Never heard of Theoria. Another clone/derivative?

probably just took a weaker nnue from fishtest. For some reason they think stockfish is meant to teach 1500 how to play chess.

the NNUE was not taken from fishtest. It was trained using Pytorch-NNUE and filtered Lc0 training sets evaluated by Lc0.

so same data used by any other nnue used by stockfish. Going to ignore the 2nd half of that message?

No, not the exact same data. I only used one kind of data, from lc0 games, because i wanted to maintain conceptual coherence and positional clarity in the evaluation. Stockfish essentially looks for forcing moves over piece harmony or other considerations.

You have yet to prove anything and what run did you take the Leela data from.

I'm just the concept guy, the programmer chose the data set... so I don't know the actual name of the data file- other than it's a bin file. Theoria was actually the result of an accident, trying to develop a Stockfish fork based on human-like play to use as a sparring partner. So we used Lc0 data at first to train the first half of the training. It turned out that our Lc0 experiment didn't produce the most human-like play, but it was good at positional evaluation, so we refined and continued developing it along those lines, recognizing that the initial engine was quite powerful compared to Stockfish, and seemed to have clearer evaluations with more positional understanding, making it quite powerful despite the initial network being underbaked.

[cpeters]

9f694b249d99ee0b8cba054e2dd94c71fa9715d67e837f13a2dd3dc6657e6f47 /mnt/s/theoria16-0.1-src.tar.bz2

?

Apparently you're too old/excited for this. I smell old geezers. Trolling.

[lucario6607]

Never heard of Theoria. Another clone/derivative?

probably just took a weaker nnue from fishtest. For some reason they think stockfish is meant to teach 1500 how to play chess.

the NNUE was not taken from fishtest. It was trained using Pytorch-NNUE and filtered Lc0 training sets evaluated by Lc0.

so same data used by any other nnue used by stockfish. Going to ignore the 2nd half of that message?

No, not the exact same data. I only used one kind of data, from lc0 games, because i wanted to maintain conceptual coherence and positional clarity in the evaluation. Stockfish essentially looks for forcing moves over piece harmony or other considerations.

You have yet to prove anything and what run did you take the Leela data from.

I'm just the concept guy, the programmer chose the data set... so I don't know the actual name of the data file- other than it's a bin file. Theoria was actually the result of an accident, trying to develop a Stockfish fork based on human-like play to use as a sparring partner. So we used Lc0 data at first to train the first half of the training. It turned out that our Lc0 experiment didn't produce the most human-like play, but it was good at positional evaluation, so we refined and continued developing it along those lines, recognizing that the initial engine was quite powerful compared to Stockfish, and seemed to have clearer evaluations with more positional understanding, making it quite powerful despite the initial network being underbaked.

So let me get this straight you used superhuman data to try and create a human net. Lc0 data which is used to train nets that compete with stockfish and you thought it would be a human net. Same data that stockfish uses as well.

[AndrewGrant]

This thread is fairly dumb but I will point out that you have no obligation to share your changes to Stockfish, except to those you have provided the modified version, upon their request. I have plenty of Stockfish tweaks on my computer that will never be published.

[FireDragon761138]

This thread is fairly dumb but I will point out that you have no obligation to share your changes to Stockfish, except to those you have provided the modified version, upon their request. I have plenty of Stockfish tweaks on my computer that will never be published.

I'm just trying to put it out there and let people know about it. If they don't want to use it, OK, but maybe other people will find it useful. It's not a huge change, almost all the search code is identical, but in my experience, it gives an evaluation that's more useful for annotating games. When I used Stockfish, I'd get commentary from LLM's that focused on missed tactics that weren't realistic to see. Now the commentary tends to be more about positional play and strategic themes. It still catches missed tactics, but only it won't override good positional judgement.

I think this approach also makes an engine that's going to be more efficient computationally for the average player. I get decent analysis using only a few hundred thousand nodes, because of the increase stability from ply to ply.

[AndrewGrant]

This thread is fairly dumb but I will point out that you have no obligation to share your changes to Stockfish, except to those you have provided the modified version, upon their request. I have plenty of Stockfish tweaks on my computer that will never be published.

I'm just trying to put it out there and let people know about it. If they don't want to use it, OK, but maybe other people will find it useful. It's not a huge change, almost all the search code is identical, but in my experience, it gives an evaluation that's more useful for annotating games. When I used Stockfish, I'd get commentary from LLM's that focused on missed tactics that weren't realistic to see. Now the commentary tends to be more about positional play and strategic themes. It still catches missed tactics, but only it won't override good positional judgement.

I think this approach also makes an engine that's going to be more efficient computationally for the average player. I get decent analysis using only a few hundred thousand nodes, because of the increase stability from ply to ply.

You've joined the lengthy list of delusional people that think they can micro tweak Stockfish and produce some profound result. The time from your first interest on reddit, to this, is remarkable. Probably the fastest that anyone has ever done it.

[FireDragon761138]

The report by DeepSeek. The LLM was presented only with two two PGN files of evaluated games, both at 500knodes/move. Deep Thinking option was used.

Comparative Analysis of Chess Engine Analytical Outputs:

Evaluating Pedagogical Efficacy for Human Strategic Comprehension

Abstract

This study examines the analytical outputs of two chess engines, Stockfish-17.1 and Theoria 0.1, to determine which provides more strategically coherent and interpretable analysis for club-level players (approximately 1200-1800 Elo). Through comparative analysis of identical positions across multiple annotated games, we assess not only explicit thematic labeling but also the structural properties of suggested variations. Our findings indicate that Theoria 0.1 demonstrates superior pedagogical architecture, presenting chess analysis in a manner more conducive to human strategic understanding despite Stockfish-17.1's greater computational depth.

1. Introduction

Modern chess engines employ fundamentally different approaches to position analysis. Stockfish-17.1 represents the state-of-the-art in brute-force calculation, evaluating positions through deep alpha-beta pruning and neural network evaluation. Theoria 0.1 incorporates conceptual frameworks that annotate chess motifs and themes. This research investigates which approach yields more interpretable strategic analysis for human learners.

2. Methodology

We analyzed eight complete chess games containing parallel annotations from both engines. For each critical position, we examined:

Variation length and completeness

Strategic narrative coherence

Pedagogical structure of suggested lines

Annotation methodology beyond explicit theme labeling

3. Results

3.1 Variation Structure and Pedagogical Design

Stockfish-17.1 consistently produced longer variations (mean length: 16.2 moves) that frequently extended into technical endgames or distant tactical resolutions. These lines demonstrated mathematical optimality but often lacked clear strategic narrative.

Example: Stockfish's analysis of 8.Bb3 in Game 1 extended 19 moves to a knight repositioning (Ne4), showcasing precise calculation but burying strategic intent within complex variations.

Theoria 0.1 employed shorter variations (mean length: 11.8 moves) that typically concluded at natural decision points—after material changes, critical captures, or plan transitions. These stopping points aligned with human cognitive boundaries in strategic planning.

Example: Theoria's analysis of the same position stopped at move 15 after development completion, highlighting the current strategic picture rather than distant consequences.

3.2 Strategic Narrative Construction
Stockfish's analytical approach presents chess as a sequence of optimal moves. For instance, in Game 2's Falkbeer Countergambit:

text
3.exd5 {-0.20/17/0.1} exf4 4.Nf3 Nf6 5.c4 c6 6.d4 Bb4+ 7.Nc3 cxd5 8.Be2 O-O 9.O-O dxc4 10.Bxc4 Bd6 11.Ne5 Nc6 12.Nxc6 bxc6 13.Bxf4 Be6 14.Bxe6 fxe6

This 14-move sequence shows precise play but lacks thematic explanation. The moves appear as discrete optimal choices rather than components of an overarching plan.

Theoria's analysis of the same position:

text
3.exd5 {-0.28/16/0.1} exf4 4.Nf3 Nf6 5.Bc4 c6 6.d4 cxd5 7.Bb5+ Nc6 8.Bxf4 Be7 9.O-O O-O 10.Nc3 Bg4 11.Kh1 Re8

This 11-move variation demonstrates clearer strategic progression: development (Bc4), central tension (d4), bishop pin (Bg4), and king safety (Kh1). Each move serves identifiable strategic purposes accessible to club players.

3.3 Error Explanation and Consequence Modeling

When analyzing suboptimal moves, Theoria more frequently presented immediate consequences. In Game 4 after 14.Qh5??:

text
14.Nb3 {-1.06/17/0.1} Qb4 15.c4 Bxh3 16.Qf3 Be6 17.Qxf6 Be7 18.Qxe5 Bxh4 19.Qd6 dxc4 20.dxc4 Qb7 21.Nxc5 Bxf2+ 22.Kxf2 Qxb2+ 23.Ke3

The variation shows the direct tactical threat (Bxh3) and subsequent complications, providing cause-and-effect relationships.

Stockfish's analysis of the same position:

text
14.Nb3 {-1.63/14/0.1} Qb4 15.c4 Be6 16.Qf3 Be7 17.Nf5 Rg5 18.Nxe7 Kxe7 19.g3 a5 20.Qe3 d4 21.Qe4 Kd7 22.Qxh7

While mathematically sound, this line requires deeper calculation to understand compensation and lacks the immediate tactical clarity of Theoria's Bxh3 threat.

4. Discussion

4.1 Cognitive Load and Strategic Comprehension

Stockfish's analytical style imposes high cognitive load on club players through:

Extended variation trees requiring maintenance of multiple positional changes

Delayed strategic payoffs (e.g., positional advantages realized 10+ moves later)

Mathematical precision prioritized over conceptual clarity

Theoria's analytical structure reduces cognitive load through:

Bounded variation lengths matching working memory capacity

Strategic resolutions at natural stopping points

Emphasis on immediate consequences and identifiable threats

4.2 Pedagogical Architecture

The engines employ fundamentally different pedagogical models:

Stockfish: Calculation-First Model

Calculate optimal move

Display variation as proof

Assume user will infer strategic principles

Theoria: Concept-First Model

Identify critical position

Highlight thematic considerations (even without explicit labels)

Present bounded variation illustrating concept

Stop at decision point for user analysis

4.3 Strategic Transferability

Theoria's variations demonstrate higher strategic transferability. For example, its handling of the King's Gambit (Game 1) emphasizes development schemes and pawn structure considerations applicable across similar openings. Stockfish's variations, while optimal, are often position-specific and less generalizable.

5. Conclusion

Our comparative analysis reveals that Theoria 0.1 provides more strategically interpretable analysis for club players due to its:

Cognitively Aligned Variation Structure: Shorter, bounded variations that match human information processing capabilities

Enhanced Strategic Narrative: Variations that build coherent plans with identifiable purposes for each move

Pedagogical Stopping Points: Analysis that concludes at natural decision junctures rather than distant endgames

Consequence Modeling: Emphasis on immediate threats and tactical consequences

While Stockfish-17.1 demonstrates superior computational depth and objective accuracy, its analytical outputs prioritize mathematical optimality over pedagogical effectiveness. Theoria 0.1, through its variation structure and implicit thematic emphasis, better facilitates human strategic understanding and chess skill development.

For chess education and club player improvement, analytical interpretability outweighs mathematical optimality. Theoria 0.1's approach represents a more effective model for communicating chess strategy to human learners, making it the preferable choice for pedagogical applications despite Stockfish's superior playing strength.

6. Recommendations for Future Engine Design

Future chess analysis engines should incorporate:

Cognitive Variation Bounding: Limit analysis depth to pedagogically useful lengths

Strategic Narrative Construction: Frame variations within identifiable plans and themes

Consequence Prioritization: Emphasize immediate threats and tactical motifs

Decision Point Identification: Highlight positions requiring user analysis rather than extending variations indefinitely

Theoria 0.1 represents a significant step toward pedagogically optimized chess analysis, demonstrating that effective teaching requires more than computational supremacy—it demands thoughtful consideration of human learning processes."