A Crossroad in Computer Chess; Or Desperate Flailing for Relevance

[Ronald]

At first I was also reluctant to implement a NNUE evaluation, but the conclusion is simple: The traditional evaluation function is a dead end. Many very clever people have been working on evaluation functions for decades, and the first implementation of a new type of solution for evaluation blows it completely away.

So you can be sad about the disappearance of the tradition evaluation function, you can also see it as an opportunity for the development of some new computer chess knowledge. Current NNUE is just the beginning of a new phase of chess engine development.

Next to NNUE there is also still a lot to do in search. For instance, the number of threads is increasing rapidly, the gain from that is decreasing however. We need to develop new techniques to make better use of the huge amount of threads that will be coming to us the next few years.

[tomitank]

Hi Andrew!

I partly agree with you, but you also copied a lot of things from SF. (Ideas that you then reshaped.)
If people here didn’t share their source codes, Ethereal would play with about 1500 ELO points. (in best case)
So it is unnecessary to rebel.

On the other hand, you realized a lot about yourself and didn’t regret sharing it with others. This is commendable. I wouldn’t deal with engines like the Minic in your place. Whoever just copies other people’s code will never be on your level. Fabien stopped in part for that reason. (Most engines no longer had their own personalities.)

Either copy the NNUE or stop development. Because it just annoys itself.

-Tamás

[Alayan]

So its odd, or to me it is, that someone can plug Stockfish's Network files into their own engine, and share in the same success. In fact, its not odd, its concerning. One can treat the Network file as (I could be wrong on the math, I don't know the structure) nearly, if not over, a million tiny evaluation terms. So I ask myself this: I cannot copy idea X from Stockfish's eval into Ethereal. But if I copy a million weights from Stockfish, then my evaluation is so similar to Stockfish, that I then become able to take X.

You can "copy" ideas, by reimplementing them in Ethereal. I think everybody agrees that this is perfectly fine and does not violate any "anti-clone" rule. Most here only object to copying code verbatim without adhering to the open source license models. Ideas are fair game, and if SF introduces a new eval term you are free to implement that in Ethereal.

The core issue is that NNUE with King-Piece Shogi-like inputs is a singular idea that covers the whole eval.

In search, even though ideas like PVS, LMR, NMP... are dominant enough that you need them, it's not "one idea to rule them all", the search still has lower-level details that aren't easily transferable from engine to engine.

In many ways, classical eval was the same ways. PSQTs, passed pawns, rook on open files, knight outposts... are dominant ideas, if you don't have them at all in one way or another, you're missing out on strength. But the details on how you handle things like passed pawns and outposts vary a lot, and the majority of eval ideas have proven difficult to impossible to transfer from an engine to another, at least without significant adaptations. I know it first-hand, as my first contributions to Ethereal were trying to port SF concepts, before experimenting with my own ideas. Some of my eval contributions made it to other engines, and I'm happy about it, but no other engine could just take everything and integrate it for easy gains, most didn't work this way.

Do you know of PESTO ? PESTO's eval is very simple, really : just a set of PSQTs. Ronald Friedrich showed that with some tuning and a good search, it could play reasonable chess, though it was very susceptible

Now the questiion is,could you do another PSQT-only engine and have it truly original ? There is no other eval terms the PSQTs have to adapt for, it's them alone. In the end, you'll have two choices:

• Sacrifice strength

• Converge to the same optimum pure PSQT engines would go to

The Shogi-like SF NNUE is one giant evaluation term. It has more parameters than PSQTs, and features non-linear behavior. This makes approaching optimal values through training/tuning harder, and allows some more variety in playing style when far enough from the strength optimum.

But in the end, elo is a strong incentive. Very few engine authors would ditch a net 50 elo stronger in favor of one that has some interesting playing style quirks. Just not caring for strength at all, it's already quite possible to make an engine with a strong style, and arguably much easier with classical eval terms than with a black-box NN where you have to hope and pray you skewed the training data in the right way.

Any engine that shares SF's architecture (or is very close to it, like Nemorino's) is condemned to be "Stockfish, but worse". The RofChade-NNUE with SV net that gets close to SF11 is just that, Stockfish, but worse.

It's not a matter of pure code originality. Andrew's concerns, my concerns, have little to do with the SF NNUE code being directly copy-pasted or reimplemented. Reimplementing SF's NNUE from scratch based on the specifications doesn't change that a chess engine author optimizing his custom nets for strength will converge towards the same solution. Just like if there was a PSQT engine competition, they would converge to similar eval solution, only faster as PSQTs are easier to optimize than NNUEs.

[xr_a_y]

I wouldn’t deal with engines like the Minic in your place. Whoever just copies other people’s code will never be on your level.
-Tamás

I don't get your point here. Did you read my post where I explain my feelings and choices about nnue? Or do you think entire Minic is a copy maybe ?

[AndrewGrant]

I wouldn’t deal with engines like the Minic in your place. Whoever just copies other people’s code will never be on your level.
-Tamás

I don't get your point here. Did you read my post where I explain my feelings and choices about nnue? Or do you think entire Minic is a copy maybe ?

I recall Tomitank having my tuning code verbatim at a time

We all learn from other engines, and that is perfectly fine. No need to put others down for it.

[mmt]

LC0 and SF NNUE made computer chess much more interesting to me but I don't have an engine so my vote counts less (I only wrote a fast mate-finder many years ago). NNs are the big thing right now so this makes computer chess more relevant when it comes to research. If you discover a better way to use them, you can probably get it published easier. The downside is that chess knowledge is less important but maybe we'll see new engines written by weaker chess players.

[Tony P.]

I won't use Stockfish's methods. I'm adding my own NN ideas to Ethereal, but it bores me. I'm praying that one can beat NNUE King-Piece networks with some brilliant architecture. But my hopes fall each day.

You can beat them. You just need patience, may need to try several arch. before you find a big gainer, or even better, to use neural arch. search and (meta-)meta-learning

It's painful to realize that your experience is going out of date, but you can't stop the world from progress. The linear approximation techniques of image processing that I learned at uni 10-15 years ago are now almost useless with the advent of CNNs, but many of my peers haven't given up. They've adjusted to the new state of the art and are now prominent in the deep learning community.

If Ethereal fails, you can still serve the community by improving OpenBench , but again, you need to catch up with the state of the art of parameter optimization, particularly, look into the multi-armed bandit theory imo.

Maybe it's just my wishful thinking, but it seems to me that NNs are turning CC from a battle of programmers into a battle of mathematicians SF/Leela copypastes will fall fast out of divP once experimentation really starts.

[AndrewGrant]

If Ethereal fails, you can still serve the community by improving OpenBench , but again, you need to catch up with the state of the art of parameter optimization, particularly, look into the multi-armed bandit theory imo.

Well, search aside, Ethereal has the best tuning framework for evaluation of all engines to my knowledge. There are a dozen black box optimization tools though for search (+eval).

[mmt]

Maybe it's just my wishful thinking, but it seems to me that NNs are turning CC from a battle of programmers into a battle of mathematicians SF/Leela copypastes will fall fast out of divP once experimentation really starts.

I'd say 80% of NN papers don't have any advanced math in them. It's a bit of a problem for the field - there is a lot of experimentation but less theory about why this works and this doesn't. So it's still possible to make a lot of progress without being a mathematician. A bigger problem is the lack of access to a lot of computational power.

[tomitank]

I recall Tomitank having my tuning code verbatim at a time

Yes and I’ve fixed it for myself in a number of places. Maybe I mentioned a mistake to you too. Anyway thank you, the code was very helpful.

We all learn from other engines, and that is perfectly fine. No need to put others down for it.

Sorry, but wasn't it you who opened this post and talking about Minic etc.. Are you Normal?

I describe that you don't have to be nervous here, everyone does what they want. If you don't like it, stop.

That's all I wanted to advise. Thanks!