So what do we miss in the traditional evaluation?

[mclane]

We have the passed pawn, mobility, kingsafety, piece value, pst, piece threats etc. There must be something big that we failed to include in the traditional eval as the gap of nnue and non-nnue is huge.

Like others already more or less indicated, it's the transition between structures and piece values/mobility/paths.
Usually we just evaluate for opening/middlegame/endgame phase, but this is a bit like Black and White only, while obviously NN and NNUE and even PST tuning have shown that they are interacting in such a complex way, that they always will prevail against handwritten eval.

Near tablebase positions might currently still be an exception though, I think with less and less pieces, concrete evaluation principles might still be better than abstract endgame 'knowledge' from current training, but that's just my opinion.

This „transition“ as you call it, is the plan the programs don’t do.
All this evaluation must lead to something that is more then the material win or the positional win.

[mclane]

Even more you consider WHY you put or move a piece here or there.
One primitive way is: I have to out it there be Ayse it is said to be good. Knight in the center gives good control.
But this is a low level thinking, as primitive as increasing the evaluation score for that piece.

Better: the knight on e5 could together with the queen on h5 and the rook coming via f1, f3, g3 create a mate net.
Because mate ends the game immediately instead of knight e5 increases evaluation only.

So next comes how to prepare this plan in the given position.

It can be you have to sac the main line (maximised score) at some stage for bringing your plan through.

[abgursu]

I think mobility may be undervalued in traditional evaluations.

When we think of the size of the evaluation data calculated for programs like LC0, I guess we are evaluating 0.1% of the terms that LC0 evaluates.
So in that case we are missing almost everything.
In the case of NNUE type evaluations, we might be examining as much as 1% of the terms.

Since SF nnue was not 1000 Elo stronger than SF, I guess that the additional terms don't add a gigantic amount, but they are clearly very important and more than 100 Elo stronger than the regular SF eval for nearly even positions.

So what do we gain?
We gain the freedom of not chasing down a zillion tedious terms.

So what do we lose?
We do not know what the darn thing is doing, so we gain zero knowledge about how to play better chess.

I was in same thought. I tried to set mobility values higher like a 1 billion times last year. I changed everything and everything but couldn't gained even any single ELO. I still think mobility must be higher, but I couldn't prove.

[Alayan]

The issue is differentiating "useless mobility" from "active/threatening mobility" and from "illusory mobility" where the piece must stay static to defend something.

[Harald]

May be this would help:
- Use evaluation patterns and features that are lasting longer and are not overruled by short tactics.
- Give the engine enough evaluation patterns that it is never lost in any situation in the game.
- Give it steps and bases in the position space that it can reach one by one and 'follow a plan'.
Like the people playing anti-computer chess. First close the center and one side with pawns.
Then move your pieces to the other side and attack the king.

[dkappe]

Some practical observations from the training of the Toga III net.

1) training the net on 930m d8 evals — no game result, it is already some 150 elo stronger than Toga II’s HCE, playing in the original Toga II search.
2) training the resulting net on the same data, but on a much lower learning rate and using lambda 0.5 — I.e. 50% game result — it gets another 50 elo.

Looking at #1, this yields a function approximation of d8 scores. Would you expect 150 extra elo by adding 8 ply?

[Harald]

Please have a look at another thread:
Playing with "The Secret of Chess"
http://www.talkchess.com/forum3/viewtop ... =2&t=76453

[Patrice Duhamel]

Can we learn something from NNUE networks, using some kind of "reverse engineering" ?

Maybe it's a bad idea but for example taking thousands of winning positions with passed pawns and looking at the highest values in hidden layers, then look at the inputs used to produce these values, is it possible to find new ideas for traditional evaluation ?

[Paloma]

I don't think so

[mclane]

The issue is differentiating "useless mobility" from "active/threatening mobility" and from "illusory mobility" where the piece must stay static to defend something.

The useless mobility is that, that is not supporting the plan.
The active threatening mobility most often is part of a plan.

The plan is IMO the key to the progress of neural net or nnue.