Chess Training: Human Brain v NN

[towforce]

I believe that chess NNs are building a large number of simple (shallow) patterns, whereas top humans are generating a small number of deep (complicated) patterns. I know, and accept, that some people disagree with this view, and that I personally cannot prove it.

Anyway, nobody will dispute that what humans are doing to learn good chess is very different from what NNs are doing. Here are some key facts:

* NNs are being trained using a number of positions that is many orders of magnitude larger than any human has ever seen

* top humans are still better than NNs at static (ply 1) evaluations

* it is sometimes said that the biggest difference between strong and weak players is in the endgame. Here's NNs are still relatively weak in static evaluations

So... what are top humans doing differently from NNs to enable them to still be better at static evaluations?

Here's what I think: having a really good, long look at a small number of positions is enabling their brains to uncover important patterns that the NNs cannot. Billions of positions looks like a lot of information, but in comparison with the number of possible chess positions, it's actually quite tiny.

It also reminds me of something important about chess: small differences in the position can make a big difference to the correct evaluation of it: an NN might have seen a similar position in training - but a small difference could make that evaluation wrong

As well as more complex patterns, another skill that the human behaviour of studying a small number of positions in depth might yield is the ability to know what factors are likely to determine what "small differences" are likely to make the difference between the current position being won or drawn/lost. If so, then it's not immediately obvious to me how we're going to train that skill into NNs.

[smatovic]

The neurons of a human brain are 'clocked' with 10 to 100 hertz and there are ~100 billion neurons with ~100 trillion weights present in total, not sure how much of these human neurons can be used for chess, but considering ~80 Knps with 50 million weights for Lc0, this seems to be a matter of NN size vs. NN inference speed, how the human NN encodes not only positional evaluation but also search heuristics would be interesting to know IMO.

--
Srdja

[towforce]

The neurons of a human brain are 'clocked' with 10 to 100 hertz and there are ~100 billion neurons with ~100 trillion weights present in total, not sure how much of these human neurons can be used for chess, but considering ~80 Knps with 50 million weights for Lc0, this seems to be a matter of NN size vs. NN inference speed, how the human NN encodes not only positional evaluation but also search heuristics would be interesting to know IMO.

--
Srdja

Lc0's search enables it to be vastly superior overall to top humans.

My point is that Lc0's NN has trained on billions of positions, whereas top human brains have only trained on a tiny fraction of that, and yet that NN doesn't have an ability to look at a position and quickly make an assessment that's close to the truth like top human brains do.

Are you saying that if ANNs were just bigger, then they could train on fewer positions and still be as good as humans? I think there's something important missing from the ANN training process. Among the missing items:

1. formation of important deep (complex) patterns

2. the ability to know, for a given position, what small differences could make big changes to the evaluation

[smatovic]

Maybe my point is the divison between tactical playout via MiniMax search and positional knowledge via evaluation heuristics. The majority of chess engines are divided since 1948 (Wiener) into these two parts. But IMO now with NNs present in CC the lines start to blurry further (considering selective AB search). How does or can a NN encode tactical playout, the AB plies, the sequence of a game, the game tree, in its neural network architecure? Merge Shannon's Type-A and Type-B strategy in a NN. Something like this. I don't have the answer.

--
Srdja

[towforce]

If you're going to have search, and not just static evaluation, then based on what I've said previously in this thread, maybe instead of:

1. choosing the move ordering

2. evaluating the position

... the NN should be ...

1. writing the move ordering algorithm based on the position

2. writing the evaluation function based on the position

... and doing both of these things on the fly.

That would be more human. That would be understanding the position and answering the question, "What's important here?"

[smatovic]

If you're going to have search, and not just static evaluation, then based on what I've said previously in this thread, maybe instead of:
..

Regarding your OP, I have the impression that the 'deep patterns' you describe in human brains are a mixture of search and static evaluation, take it as a multidimensional evaluation function over t, time - the sequence of a game, and m as permutations of the game tree. Imagine you expand the whole game tree at once, then look from above onto it, and then run a kind of t-m evaluation function on it, of course there must be some kind of pruning in practice. This is just an impression I have, dunno if such a t-m evaluation exists, and I am not expert enough to judge if such a mix is already present to a degree in Lc0's CNN or SF's NNUE networks.

If we say that humans learn less positions, have more 'weights' and outperform LC0 at ply 1 eval, then there must be some additional information stored in the human brain what is not stored in LC0's CNN.

--
Srdja

[towforce]

Search is so insignificant in human players that it's barely worth mentioning IMO. Computers have been out searching GMs since 1977 - link.

One superpower that GMs have is an ability to look at a position and know what to look for: what's going to be important in this position. Fast neural pathways have been developed by long and intense repetition that will guide them to this knowledge without having to do any conscious thinking.

Trained NNs will probably know roughly whether a position of this type is good or bad, but they probably won't know what to look for to determine the important differences between a "good position of this type" and a "bad position of this type". This is why it would be interesting to train NNs to write their own evaluation functions for a position (or at least pick the components of the evaluation function that will be important in the current position).

I'm not sure how you'd do that automatically right now: your big problem would be that there would be too many ways of getting the right answer (in terms of the final evaluation) with the wrong EF components for the current position. It's certainly something that could be done manually, though: remember that the threshold for becoming a GM is knowledge of about 50,000 different chess patterns.

Speaking of that old 50,000 number, the fact that today's NNs are being trained on billions of positions comes close to proving that they're learning - but they're not learning as well as top humans do.

[DrCliche]

Humans are simply learning different types of information that allows them to generalize better. You say that humans have seen far fewer positions than machines, but when a competent human evaluates a position, they extrapolate that position out to many variations by excluding (probably) irrelevant information, by thinking about relationships to previously studied positions, etc. Some of that process is willful and conscious, but some of that process is also automatic and unconscious.

So competent humans are explicitly learning efficient (for them) encodings via something akin to contrastive learning, and also chunking countless permutations of observed positions together by learning to exclude irrelevant features, among other techniques. There's no reason in principle an artificial neural network couldn't be made to do everything that competent humans are doing. It's really just an engineering problem.

(Though it's likely that the human brain's evaluation function is unnecessarily complicated and slow, and that on platforms that can efficiently search and track many positions simultaneously, a cheaper evaluation function is usually better. Consider that Stockfish NNUE seems to be inarguably superior to Leela at this point. Then again, Leela has failed to experiment with many potential innovations in slavish devotion to whatever some particular group of people thinks "zero" means.)

[towforce]

So competent humans are explicitly learning efficient (for them) encodings via something akin to contrastive learning, and also chunking countless permutations of observed positions together by learning to exclude irrelevant features, among other techniques. There's no reason in principle an artificial neural network couldn't be made to do everything that competent humans are doing. It's really just an engineering problem.

I basically agree - but how might one go about doing that?

[DrCliche]

I basically agree - but how might one go about doing that?

Look up ML papers on contrastive learning.

You want to use a network to encode chess positions into a (much smaller) latent space, where the loss function has a term that penalizes the distance between encodings of similar positions. (It is up to you to determine how you want to define "similar", though of course there are unsupervised methods.)

A second network can then evaluate a given chess position using its latent vector as input, rather than (or in addition to) the chess position itself. Here you would have a normal loss function that simply encourages evaluation accuracy, plus whatever regularization terms you find aid generalization.

You can either use strong engine evaluations as a training target (this is what Stockfish does), or use search to allow the evaluation function to essentially bootstrap itself, comparing one node evaluations against aggregated 800 node evaluations or whatever (this is what Leela does).

If you train both the encoder and the evaluator in tandem, you will likely end up with a system that "understands" chess positions in a more "humanlike" way. (And though not necessary for the functioning of a chess engine, you may find the encoder trains more stably if you also simultaneously train a decoder, whose job is to take a position's latent vector as input and output the actual chess position, or possible moves, or whatever.)