I declare that HCE is dead...

[AndrewGrant]

mostly, unless you are a lunatic who will keep thousands of lines for a few points of elo. http://chess.grantnet.us/test/11953/ (https://github.com/AndyGrant/EtherealDe ... .09c8b0f9/) Tested deleting every single line of code needed for the HCE. All the eval terms, the hash tables, the incremental PSQT updates, the zobrist keys, everything. Result? Minus a few elo for deleting 1,800 lines of code. Not to mention the other 900 that would come from deleting my Tuner, which was supposedly my greatest effort in unique engine design.

So I won't delete it all. I'll take a few points of elo, and the promise that Ethereal will behave nicely in the end game. But from this moment on, I see absolutely zero reason to ever work on the HCE evaluation again, aside from what I am doing now: Which is deleting the portions of HCE that polluted the code base, and made things "ugly". I'll slim it down to the things that are reasonable, creating the best didactic engine evaluation from what I now consider Ancient History.

Do others find their HCE's now virtually pointless? Has there been a single HCE patch to pass Fishtesting when NNUE was enabled?

[jhellis3]

Yep, only exception I can think of is special case handling endgame / material key heuristics. General HCE - dead. I guess one could always try overlaying some stuff on top of the NN as well, but I imagine finding improvements there would be even more difficult than improving current HCE.

[carldaman]

Pointless only when trying to get the highest Elo overall. Others may prefer to have the ability to create specific playing personalities, whether human-like, anti-human, or Tal-like play-to-win attacking styles that the HCE could allow by means of tunable parameters.

I prefer to use such enhanced HCE-based engines for both sparring and analysis. The ideas produced are fantastic and this keeps my interest in chess going. Not everyone is a computer-assisted "correspondence" or engine-vs-engine server player.

However, very few developers are somehow interested in trading Elo/strength for style, in spite of having no aspirations of going for a #1-rated engine. Personally, I find the concept fascinating.

NNUE is great in its own right, producing amazing strength, but can become a dead end if it eventually stifles creativity.

[jonkr]

My first attempt at deleting all my HCE resulted in a -8 Elo test, but I expect if I keep trying it will be successful (possibly something bugged, or just I need to improve the nets in some way.)

For endgame I was successful at deleting all my hard-coded evaluation code (except tablebases) without losing elo, and with more endgame training it's stronger now. I already deleted my HCE tuning code.

However I plan to experiment with more net structures loosely inspired by HCE setup.

Also I do think it was cool how I used to be able to click on a piece and see exactly where the eval was coming from, or add specific terms and see if they improved play. But I already spent a lot of time on HCE and don't have time to do both.

[AndrewGrant]

I'm not sure if its depressing, or liberating.

[towforce]

HCE = Hand Crafted Evaluation.

In the time of the Genesis (the 20th century), there was an unstated assumption that everything needed to be known about the game of chess could be hand coded into an evaluation function. Now we know better. We have huge NNs, and they're still not capturing enough!

As to why huge NNs aren't capturing enough: I have my theory, which I know is controversial, and I cannot prove it (yet), but it seems to me that NNs are capturing a large number of simple patterns when a small number of complex patterns would be better.

[mvanthoor]

Do others find their HCE's now virtually pointless?

The tuner in combination with lots of evaluation terms in the HCE works like sort of a poor man's neural network.

While it may not be necessary in the end, you do need something to actually get your engine going. A good set of tapered PST's is enough. Still, you need to understand what the HCE does, before you go to work on a neural network, or you'll be developing something you don't even know what it is about.

I'm not sure if its depressing, or liberating.

It's frightening. The entire world is going full steam ahead throwing neural networks and AI against every problem imaginable, while many people don't know (or care) the massive impact this can have. In the Netherlands, we got a nice little lesson regarding that, but I'm not sure if anyone learned anything...

This: Computer says "No"

With neural networks in the system, nobody knows _why_ the computer says "No", but everybody assumes it's correct.

[chrisw]

When I last looked at Stockfish, it was flipping to the classical HCE evaluation when the PSQT was out of range, +/- 10 pawns or so. Which implies not dead.

[Raphexon]

I'm not sure if its depressing, or liberating.

Don't forget that the HCE is being replaced by the NN-trainer codebase that also needs to be maintained. One replaces the other.

I personally think it's a nice thing that the HCE can be completely replaced by NN + piece/square table (for lazyeval and increased resolution in winning positions)

It's also (imo) more elegant than throwing hundreds of terms together.

[Branko Radovanovic]

I see no evidence NNUE is inherently stronger than a (very) highly developed HCE. For Stockfish, the advent of NNUE did deliver an equivalent of 5 or so years of HCE development almost instantly, but for all we know, in a parallel universe where NNUE was never invented there might yet appear a 2025 HCE version of Stockfish which is stronger than the 2025 NNUE version of Stockfish.

From the perspective of "right here, right now", though - and that's what matters - developing HCE probably makes little sense if Elo is all one is after.

I like what Stoofvlees has been doing. If I understand correctly, it's hand-coded extraction of features with NN scoring. This sort of hybrid approach may be the most promising in the long run IMO.