A Crossroad in Computer Chess; Or Desperate Flailing for Relevance

[brianr]

Just mentioning that there is also considerable art in the NN side of things.

Perhaps as my own chess was always so weak, it was relatively easy for me to focus on the programming side of it.
I imagine that might be more difficult for a strong chess player.

With engines playing at super-human levels for quite some time and now with no idea exactly what the nets are "seeing", our human understanding of the actual chess these monsters play is becoming even less consequential.

On the other hand, creating better data, growing better nets, and making them work with an effective search seems like an area with a lot of room for invention and development. So, I tend to think of chess itself as just the yardstick used to measure progress.

[smatovic]

Remember, remember, the 5th of December...

Remember, remember, the 5th of December,
the day the earth stood still,
with its AlphaBeta mill,
evaluation terms are no longer required,
neural networks are from now on desired,
handcrafted code ruled once the whole world,
automatically tuned weights from now on behold,
there is no human knowledge in need,
for the neural network to succeed,
where does this journey end - some people ask,
nobody really knows - but that's not the task,
the sky seems the limit,
so let us join the new spirit,
of this glorious journey,
in this new kind of tourney.

--
Srdja

http://talkchess.com/forum3/viewtopic.p ... 3&p=782095



--
Srdja

[towforce]

At the same time, this is me jumping up and down, waving my hands, saying "Hey, I've been working on Ethereal for 60hrs a week for the last 6 years. I've done all this work, spent all this time and energy. But now if you want to play at Stockfish's level, you just need to download the training code, feed in your evals, wait a few billion clock cycles, and presto, its done. Why should I bother?"

Being able to work 60 hours a week and produce good results from it puts you into a rare category: you have the capability to make yourself a successful person working for someone who will share a big part of their success with you or, better still, working for yourself.

There appears to be a missing ingredient, though: you shouldn't need me to tell you this.

There's one thing that all successful people have to do: jump. The parachute analogy is that you're in a safe place in an aeroplane, and you have to jump out of it and hurtle towards the ground at 160 MPH. In the moment you make your first jump, you go from being absolutely terrified to realising that it's the most exhilarating experience of your entire life! The moment of greatest danger is the moment of least fear.

[xr_a_y]

I would only be interested in testing engines with NNUE if they used their own net.

At present, I think that only Igel, Nemorino and Minic do this?

In order to be "pure" Minic for this (generate the "nascent nutrient" net), I had to merge the learner from Nodchip repo. Not that hard. But this is necessary in order to not depend on Stockfish search. Another alternative is to build an extenal workflow in pytorch or something else that does not depend on engine (search/eval) anymore.

In this sense I'd be glad to better understand how others did the own nets their are using!, because for now my merge of the Nodchip learner in Minic is very very slow (being single threaded ...) and I'd love to benefit from an external tool to build nets !

[Dokterchen]

I don't see any contradiction between tuning the classic evaluation function and a NNEU net, because the basis for creating a NNUE are positions which were previously evaluated with the classic evaluation function. I even find this interaction very exciting. If the improvement of the classical evaluation function shows an increase of 10 ELO, how does that affect the NNUE? Does that also improve by 10? Or even by 20? Or does that lead to -10?

[Ronald]

At first I was also reluctant to implement a NNUE evaluation, but the conclusion is simple: The traditional evaluation function is a dead end. Many very clever people have been working on evaluation functions for decades, and the first implementation of a new type of solution for evaluation blows it completely away.

So you can be sad about the disappearance of the tradition evaluation function, you can also see it as an opportunity for the development of some new computer chess knowledge. Current NNUE is just the beginning of a new phase of chess engine development.

Next to NNUE there is also still a lot to do in search. For instance, the number of threads is increasing rapidly, the gain from that is decreasing however. We need to develop new techniques to make better use of the huge amount of threads that will be coming to us the next few years.

[tomitank]

Hi Andrew!

I partly agree with you, but you also copied a lot of things from SF. (Ideas that you then reshaped.)
If people here didn’t share their source codes, Ethereal would play with about 1500 ELO points. (in best case)
So it is unnecessary to rebel.

On the other hand, you realized a lot about yourself and didn’t regret sharing it with others. This is commendable. I wouldn’t deal with engines like the Minic in your place. Whoever just copies other people’s code will never be on your level. Fabien stopped in part for that reason. (Most engines no longer had their own personalities.)

Either copy the NNUE or stop development. Because it just annoys itself.

-Tamás

[Alayan]

So its odd, or to me it is, that someone can plug Stockfish's Network files into their own engine, and share in the same success. In fact, its not odd, its concerning. One can treat the Network file as (I could be wrong on the math, I don't know the structure) nearly, if not over, a million tiny evaluation terms. So I ask myself this: I cannot copy idea X from Stockfish's eval into Ethereal. But if I copy a million weights from Stockfish, then my evaluation is so similar to Stockfish, that I then become able to take X.

You can "copy" ideas, by reimplementing them in Ethereal. I think everybody agrees that this is perfectly fine and does not violate any "anti-clone" rule. Most here only object to copying code verbatim without adhering to the open source license models. Ideas are fair game, and if SF introduces a new eval term you are free to implement that in Ethereal.

The core issue is that NNUE with King-Piece Shogi-like inputs is a singular idea that covers the whole eval.

In search, even though ideas like PVS, LMR, NMP... are dominant enough that you need them, it's not "one idea to rule them all", the search still has lower-level details that aren't easily transferable from engine to engine.

In many ways, classical eval was the same ways. PSQTs, passed pawns, rook on open files, knight outposts... are dominant ideas, if you don't have them at all in one way or another, you're missing out on strength. But the details on how you handle things like passed pawns and outposts vary a lot, and the majority of eval ideas have proven difficult to impossible to transfer from an engine to another, at least without significant adaptations. I know it first-hand, as my first contributions to Ethereal were trying to port SF concepts, before experimenting with my own ideas. Some of my eval contributions made it to other engines, and I'm happy about it, but no other engine could just take everything and integrate it for easy gains, most didn't work this way.

Do you know of PESTO ? PESTO's eval is very simple, really : just a set of PSQTs. Ronald Friedrich showed that with some tuning and a good search, it could play reasonable chess, though it was very susceptible

Now the questiion is,could you do another PSQT-only engine and have it truly original ? There is no other eval terms the PSQTs have to adapt for, it's them alone. In the end, you'll have two choices:

• Sacrifice strength

• Converge to the same optimum pure PSQT engines would go to

The Shogi-like SF NNUE is one giant evaluation term. It has more parameters than PSQTs, and features non-linear behavior. This makes approaching optimal values through training/tuning harder, and allows some more variety in playing style when far enough from the strength optimum.

But in the end, elo is a strong incentive. Very few engine authors would ditch a net 50 elo stronger in favor of one that has some interesting playing style quirks. Just not caring for strength at all, it's already quite possible to make an engine with a strong style, and arguably much easier with classical eval terms than with a black-box NN where you have to hope and pray you skewed the training data in the right way.

Any engine that shares SF's architecture (or is very close to it, like Nemorino's) is condemned to be "Stockfish, but worse". The RofChade-NNUE with SV net that gets close to SF11 is just that, Stockfish, but worse.

It's not a matter of pure code originality. Andrew's concerns, my concerns, have little to do with the SF NNUE code being directly copy-pasted or reimplemented. Reimplementing SF's NNUE from scratch based on the specifications doesn't change that a chess engine author optimizing his custom nets for strength will converge towards the same solution. Just like if there was a PSQT engine competition, they would converge to similar eval solution, only faster as PSQTs are easier to optimize than NNUEs.

[xr_a_y]

I wouldn’t deal with engines like the Minic in your place. Whoever just copies other people’s code will never be on your level.
-Tamás

I don't get your point here. Did you read my post where I explain my feelings and choices about nnue? Or do you think entire Minic is a copy maybe ?

[AndrewGrant]

I wouldn’t deal with engines like the Minic in your place. Whoever just copies other people’s code will never be on your level.
-Tamás

I don't get your point here. Did you read my post where I explain my feelings and choices about nnue? Or do you think entire Minic is a copy maybe ?

I recall Tomitank having my tuning code verbatim at a time

We all learn from other engines, and that is perfectly fine. No need to put others down for it.