A Crossroad in Computer Chess; Or Desperate Flailing for Relevance

[mmt]

LC0 and SF NNUE made computer chess much more interesting to me but I don't have an engine so my vote counts less (I only wrote a fast mate-finder many years ago). NNs are the big thing right now so this makes computer chess more relevant when it comes to research. If you discover a better way to use them, you can probably get it published easier. The downside is that chess knowledge is less important but maybe we'll see new engines written by weaker chess players.

[Tony P.]

I won't use Stockfish's methods. I'm adding my own NN ideas to Ethereal, but it bores me. I'm praying that one can beat NNUE King-Piece networks with some brilliant architecture. But my hopes fall each day.

You can beat them. You just need patience, may need to try several arch. before you find a big gainer, or even better, to use neural arch. search and (meta-)meta-learning

It's painful to realize that your experience is going out of date, but you can't stop the world from progress. The linear approximation techniques of image processing that I learned at uni 10-15 years ago are now almost useless with the advent of CNNs, but many of my peers haven't given up. They've adjusted to the new state of the art and are now prominent in the deep learning community.

If Ethereal fails, you can still serve the community by improving OpenBench , but again, you need to catch up with the state of the art of parameter optimization, particularly, look into the multi-armed bandit theory imo.

Maybe it's just my wishful thinking, but it seems to me that NNs are turning CC from a battle of programmers into a battle of mathematicians SF/Leela copypastes will fall fast out of divP once experimentation really starts.

[AndrewGrant]

If Ethereal fails, you can still serve the community by improving OpenBench , but again, you need to catch up with the state of the art of parameter optimization, particularly, look into the multi-armed bandit theory imo.

Well, search aside, Ethereal has the best tuning framework for evaluation of all engines to my knowledge. There are a dozen black box optimization tools though for search (+eval).

[mmt]

Maybe it's just my wishful thinking, but it seems to me that NNs are turning CC from a battle of programmers into a battle of mathematicians SF/Leela copypastes will fall fast out of divP once experimentation really starts.

I'd say 80% of NN papers don't have any advanced math in them. It's a bit of a problem for the field - there is a lot of experimentation but less theory about why this works and this doesn't. So it's still possible to make a lot of progress without being a mathematician. A bigger problem is the lack of access to a lot of computational power.

[tomitank]

I recall Tomitank having my tuning code verbatim at a time

Yes and I’ve fixed it for myself in a number of places. Maybe I mentioned a mistake to you too. Anyway thank you, the code was very helpful.

We all learn from other engines, and that is perfectly fine. No need to put others down for it.

Sorry, but wasn't it you who opened this post and talking about Minic etc.. Are you Normal?

I describe that you don't have to be nervous here, everyone does what they want. If you don't like it, stop.

That's all I wanted to advise. Thanks!

[Tony P.]

I'd say 80% of NN papers don't have any advanced math in them. It's a bit of a problem for the field - there is a lot of experimentation but less theory about why this works and this doesn't. So it's still possible to make a lot of progress without being a mathematician. A bigger problem is the lack of access to a lot of computational power.

It was regarded as a problem when Leela was SOTA, but now the much easier trainable NNUE has shown convincingly that power isn't the bottleneck.

There are so many papers overall that the 20% with the math are quite enough. I simply have no time to read the rest

[chrisw]

So its odd, or to me it is, that someone can plug Stockfish's Network files into their own engine, and share in the same success. In fact, its not odd, its concerning. One can treat the Network file as (I could be wrong on the math, I don't know the structure) nearly, if not over, a million tiny evaluation terms. So I ask myself this: I cannot copy idea X from Stockfish's eval into Ethereal. But if I copy a million weights from Stockfish, then my evaluation is so similar to Stockfish, that I then become able to take X.

You can "copy" ideas, by reimplementing them in Ethereal. I think everybody agrees that this is perfectly fine and does not violate any "anti-clone" rule. Most here only object to copying code verbatim without adhering to the open source license models. Ideas are fair game, and if SF introduces a new eval term you are free to implement that in Ethereal.

The core issue is that NNUE with King-Piece Shogi-like inputs is a singular idea that covers the whole eval.

In search, even though ideas like PVS, LMR, NMP... are dominant enough that you need them, it's not "one idea to rule them all", the search still has lower-level details that aren't easily transferable from engine to engine.

In many ways, classical eval was the same ways. PSQTs, passed pawns, rook on open files, knight outposts... are dominant ideas, if you don't have them at all in one way or another, you're missing out on strength. But the details on how you handle things like passed pawns and outposts vary a lot, and the majority of eval ideas have proven difficult to impossible to transfer from an engine to another, at least without significant adaptations. I know it first-hand, as my first contributions to Ethereal were trying to port SF concepts, before experimenting with my own ideas. Some of my eval contributions made it to other engines, and I'm happy about it, but no other engine could just take everything and integrate it for easy gains, most didn't work this way.

Do you know of PESTO ? PESTO's eval is very simple, really : just a set of PSQTs. Ronald Friedrich showed that with some tuning and a good search, it could play reasonable chess, though it was very susceptible

Now the questiion is,could you do another PSQT-only engine and have it truly original ? There is no other eval terms the PSQTs have to adapt for, it's them alone. In the end, you'll have two choices:

• Sacrifice strength

• Converge to the same optimum pure PSQT engines would go to

The Shogi-like SF NNUE is one giant evaluation term. It has more parameters than PSQTs, and features non-linear behavior. This makes approaching optimal values through training/tuning harder, and allows some more variety in playing style when far enough from the strength optimum.

But in the end, elo is a strong incentive. Very few engine authors would ditch a net 50 elo stronger in favor of one that has some interesting playing style quirks. Just not caring for strength at all, it's already quite possible to make an engine with a strong style, and arguably much easier with classical eval terms than with a black-box NN where you have to hope and pray you skewed the training data in the right way.

Any engine that shares SF's architecture (or is very close to it, like Nemorino's) is condemned to be "Stockfish, but worse". The RofChade-NNUE with SV net that gets close to SF11 is just that, Stockfish, but worse.

It's not a matter of pure code originality. Andrew's concerns, my concerns, have little to do with the SF NNUE code being directly copy-pasted or reimplemented. Reimplementing SF's NNUE from scratch based on the specifications doesn't change that a chess engine author optimizing his custom nets for strength will converge towards the same solution. Just like if there was a PSQT engine competition, they would converge to similar eval solution, only faster as PSQTs are easier to optimize than NNUEs.

That's one big assumption. Error mostly being the old chestnut that there is one solution to chess, which may be true given infinite space/time, but we don't have that, and in any case there are probably infinite pathways to the 'one' solution.

I call this Frans Morsch syndrome. I remember speaking to him way back in the 1990s, he was saying "it's all over, just too many amateurs now". there were also "too many amateurs" with the LC0 explosion, too many amateurs with SF, now too many amateurs with NNUE. It's kind of annoying when suddenly there are a zillion and one 'competitors' competing when just before the target was in sight, but c'est la vie.

[AndrewGrant]

I recall Tomitank having my tuning code verbatim at a time

Yes and I’ve fixed it for myself in a number of places. Maybe I mentioned a mistake to you too. Anyway thank you, the code was very helpful.

We all learn from other engines, and that is perfectly fine. No need to put others down for it.

Sorry, but wasn't it you who opened this post and talking about Minic etc.. Are you Normal?

Uh, not quite. this thread was not a complaint about Minic or Rubi or Igel or Nemo or RofChade.
It was a concern about a trend, which I feel could have bad long term effects on computer chess.
I'm not accusing anyone of wrong doing here. I'm just talking about my thoughts.

[Alayan]

That's one big assumption. Error mostly being the old chestnut that there is one solution to chess, which may be true given infinite space/time, but we don't have that, and in any case there are probably infinite pathways to the 'one' solution.

Using the same evaluation architecture creates a lot of limitations. The ways to reach a given strength level within a given eval architecture and the same search are much more narrow than with different architectures.

The strong Leela-compatible nets out there that aren't directly based on Leela training and aim for maximum strength still come up as very similar in behavior.

[xr_a_y]

We all learn from other engines, and that is perfectly fine. No need to put others down for it.

Sorry, but wasn't it you who opened this post and talking about Minic etc.. Are you Normal?

I dont think Andrew talk specifically about Minic but more of nnue technology itself. Maybe I opened the door in using NNUE as an experiment (and more recently building nets for it using something else than sf qsearch, merging the nodchip learner inside Minic to do so) but that was just a matter of days for others to do the same. And again, inside Minic, this technology is an option and a way for me too learn in the nn world, certainly not a way to climb the ladder.