I wanted to share an experimental chess model project I've been working on called Sparse Chess: https://github.com/P1X3R/sparse-chess
The core idea was to explore whether local learning rules (potentially capable of online learning) combined with a biologically inspired architectures (specifically Sparse Predictive Hierarchies (SPH) based on Ogma Corp's whitepaper and AOgmaNeo) could handle chess evaluation and policy prediction without backpropagation.
The ultimate goal was to see if this could enable extreme sparsity (~99%) and allow larger models to run efficiently on modest hardware.
Key Architecture Details
- Encoders: ART-based encoder mapping local receptive fields to columns using Fuzzy ART.
- Decoders / Common Body: Dendritic Neural Layers (DNNs) with Leaky ReLU activations acting as feature extractors.
- Heads: Specialized policy and value heads handling Lc0 move distributions and WDL outputs.
- Optimizations: byte weights with stochastic rounding to increase SIMD throughput and save memory.
- Size: ~44.9M parameters.
To measure raw playing strength rather than just online-learning capability, I trained the model via supervised learning on chunks of the Lc0 test80 dataset (10K training games / 2K validation games, which is very small but enough to get some results).
I also built a control group using a tiny, plain CNN (32 channels, ~3.8M parameters) trained on the exact same dataset to serve as a baseline: https://github.com/P1X3R/sparse-chess-control-group
Results & Conclusion
Unfortunately, the results suggest that my initial hypothesis was incorrect (a.k.a., failed catastrophically). In terms of raw dataset accuracy and chess strength, the SPH significantly underperformed compared to standard architectures like ResNets, Transformers, and even the tiny control-group CNN.