Bluefish vs Leela in TCEC , who will win? BF =170 Threads, Lc0 =2x GPU

[CMCanavessi]

People on this forum have said thatLCzero has a 40 MB pre calculated cache of moves. I wish that the TCEC would run an experiment letting SF use the latest Brianfish or else disabling LCzeros cache of moves.

LOL... Leela is open source, anyone can go and check for that. There's nothing like that.

[Robert Pope]

People on this forum have said thatLCzero has a 40 MB pre calculated cache of moves. I wish that the TCEC would run an experiment letting SF use the latest Brianfish or else disabling LCzeros cache of moves.

LOL... Leela is open source, anyone can go and check for that. There's nothing like that.

They are claiming that the net is in essence a book as well as an evaluation function. Of course, in many ways, Stockfish is as well, they just spend little time tuning the part that would help discriminate between lines in the opening.

[jp]

They are claiming that the net is in essence a book as well as an evaluation function. Of course, in many ways, Stockfish is as well, they just spend little time tuning the part that would help discriminate between lines in the opening.

But the size of the SF evaluation function is very small (the whole code is small), isn't it? They say the size of an Lc NN is 52 MB.

[Robert Pope]

They are claiming that the net is in essence a book as well as an evaluation function. Of course, in many ways, Stockfish is as well, they just spend little time tuning the part that would help discriminate between lines in the opening.

But the size of the SF evaluation function is very small (the whole code is small), isn't it? They say the size of an Lc NN is 52 MB.

Sure, that's why they make the claim.

But size isn't really a fair comparison (nothing ever is, right?). A NN evaluates by just applying a basic math calculation to all those numbers. Stockfish evaluation is done much more focused and efficiently. Stockfish can take two numbers and say "apply these two factors on a sliding scale depending on material." The NN has to learn that same information for every set of material combinations it runs across (if it learns it at all). So by being smarter in coding, they've accomplished the same thing with 2 numbers instead of 10000. Stockfish can be coded that a queen is a lot better than a pawn with just two numbers. The NN has to train that the queen it sees on d4 is better than a pawn on d4, and it can't generalize that to queens on f4 or h5 very well - for all it knows d4 is a "super-square" that gives queens extra powers. So it needs a lot more numbers in its evaluation matrix to be able to represent what Stockfish can do with a couple numbers. And it needs enough numbers that the information from item A doesn't get wiped out by adding the experience about item B.

Of course the flip side is that since Stockfish is coded so much more compactly and cleverly, that all the edge cases where those heuristics fail don't get evaluated properly. The NN can do just as well at the edge cases as it does in regular ones because it doesn't treat them any differently.

But claiming Lc0 has a built-in opening book because it uses more numbers in its eval is a stretch. More numbers and simple implementation vs few numbers with smart coding are just different ways of attacking the problem.

[corres]

NN is a "Black Box" what store huge number of connection between board positions and pre-evaluated worth of those position. The pre-evaluation were happened during the self-play learning. The result of self-play learning is contained by the NN file what Leela uses during searching.
In this course Leela gives a position as an input to NN and read out from NN as an output a probability
vector of move probabilities and probability number what refer to the winning chance of the game
from which the position originates.
The question is we can rebuild the position from the evaluation value of an AB engine and from the evaluation worth of an NN engine.
Evaluation of an AB engine is pure number with dimension of centipawn. Obviously a centipawn value practically says nothing about the position from what it was derived.
Evaluation of an NN engine contains the move vector what refers to each possible moves for black and white from that position. Can we rebuild the position from what that move vector was derived? Practically the answer is Yes.
In this sense the NN contains "a kind of database" as I was stated earlier.

[chrisw]

NN is a "Black Box" what store huge number of connection between board positions and pre-evaluated worth of those position. The pre-evaluation were happened during the self-play learning. The result of self-play learning is contained by the NN file what Leela uses during searching.
In this course Leela gives a position as an input to NN and read out from NN as an output a probability
vector of move probabilities and probability number what refer to the winning chance of the game
from which the position originates.
The question is we can rebuild the position from the evaluation value of an AB engine and from the evaluation worth of an NN engine.
Evaluation of an AB engine is pure number with dimension of centipawn. Obviously a centipawn value practically says nothing about the position from what it was derived.
Evaluation of an NN engine contains the move vector what refers to each possible moves for black and white from that position. Can we rebuild the position from what that move vector was derived? Practically the answer is Yes.

The answer is no. You have either not thought deeply enough about this, or you have an incorrect model of the policy matrix.

In this sense the NN contains "a kind of database" as I was stated earlier.

[corres]

NN is a "Black Box" what store huge number of connection between board positions and pre-evaluated worth of those position. The pre-evaluation were happened during the self-play learning. The result of self-play learning is contained by the NN file what Leela uses during searching.
In this course Leela gives a position as an input to NN and read out from NN as an output a probability
vector of move probabilities and probability number what refer to the winning chance of the game
from which the position originates.
The question is we can rebuild the position from the evaluation value of an AB engine and from the evaluation worth of an NN engine.
Evaluation of an AB engine is pure number with dimension of centipawn. Obviously a centipawn value practically says nothing about the position from what it was derived.
Evaluation of an NN engine contains the move vector what refers to each possible moves for black and white from that position. Can we rebuild the position from what that move vector was derived? Practically the answer is Yes.

The answer is no. You have either not thought deeply enough about this, or you have an incorrect model of the policy matrix.

In this sense the NN contains "a kind of database" as I was stated earlier.

You mix the "policy matrix" with the "value matrix".
Probability vector of move probabilities and the winning chance are contained by the "value head" (=value matrix).
"Policy head" (=policy matrix) is the helper for searching to the right way only.

[chrisw]

NN is a "Black Box" what store huge number of connection between board positions and pre-evaluated worth of those position. The pre-evaluation were happened during the self-play learning. The result of self-play learning is contained by the NN file what Leela uses during searching.
In this course Leela gives a position as an input to NN and read out from NN as an output a probability
vector of move probabilities and probability number what refer to the winning chance of the game
from which the position originates.
The question is we can rebuild the position from the evaluation value of an AB engine and from the evaluation worth of an NN engine.
Evaluation of an AB engine is pure number with dimension of centipawn. Obviously a centipawn value practically says nothing about the position from what it was derived.
Evaluation of an NN engine contains the move vector what refers to each possible moves for black and white from that position. Can we rebuild the position from what that move vector was derived? Practically the answer is Yes.

The answer is no. You have either not thought deeply enough about this, or you have an incorrect model of the policy matrix.

In this sense the NN contains "a kind of database" as I was stated earlier.

You mix the "policy matrix" with the "value matrix".
Probability vector of move probabilities and the winning chance are contained by the "value head" (=value matrix).
"Policy head" (=policy matrix) is the helper for searching to the right way only.

No. I don’t mix anything. Thank you for the beginners lecture.

If you think you can reconstitute the position from the policy matrix, you have an incorrect model of the policy matrix.

[corres]

NN is a "Black Box" what store huge number of connection between board positions and pre-evaluated worth of those position. The pre-evaluation were happened during the self-play learning. The result of self-play learning is contained by the NN file what Leela uses during searching.
In this course Leela gives a position as an input to NN and read out from NN as an output a probability
vector of move probabilities and probability number what refer to the winning chance of the game
from which the position originates.
The question is we can rebuild the position from the evaluation value of an AB engine and from the evaluation worth of an NN engine.
Evaluation of an AB engine is pure number with dimension of centipawn. Obviously a centipawn value practically says nothing about the position from what it was derived.
Evaluation of an NN engine contains the move vector what refers to each possible moves for black and white from that position. Can we rebuild the position from what that move vector was derived? Practically the answer is Yes.

The answer is no. You have either not thought deeply enough about this, or you have an incorrect model of the policy matrix.

In this sense the NN contains "a kind of database" as I was stated earlier.

You mix the "policy matrix" with the "value matrix".
Probability vector of move probabilities and the winning chance are contained by the "value head" (=value matrix).
"Policy head" (=policy matrix) is the helper for searching to the right way only.

No. I don’t mix anything. Thank you for the beginners lecture.
If you think you can reconstitute the position from the policy matrix, you have an incorrect model of the policy matrix.

With pleasure.

I wrote nothing about "policy matrix".
Please, stick to my text if you refer to it.

[chrisw]

NN is a "Black Box" what store huge number of connection between board positions and pre-evaluated worth of those position. The pre-evaluation were happened during the self-play learning. The result of self-play learning is contained by the NN file what Leela uses during searching.
In this course Leela gives a position as an input to NN and read out from NN as an output a probability
vector of move probabilities and probability number what refer to the winning chance of the game
from which the position originates.
The question is we can rebuild the position from the evaluation value of an AB engine and from the evaluation worth of an NN engine.
Evaluation of an AB engine is pure number with dimension of centipawn. Obviously a centipawn value practically says nothing about the position from what it was derived.
Evaluation of an NN engine contains the move vector what refers to each possible moves for black and white from that position. Can we rebuild the position from what that move vector was derived? Practically the answer is Yes.

The answer is no. You have either not thought deeply enough about this, or you have an incorrect model of the policy matrix.

In this sense the NN contains "a kind of database" as I was stated earlier.

You mix the "policy matrix" with the "value matrix".
Probability vector of move probabilities and the winning chance are contained by the "value head" (=value matrix).
"Policy head" (=policy matrix) is the helper for searching to the right way only.

No. I don’t mix anything. Thank you for the beginners lecture.
If you think you can reconstitute the position from the policy matrix, you have an incorrect model of the policy matrix.

With pleasure.

I wrote nothing about "policy matrix".
Please, stick to my text if you refer to it.

You prefer “move vector”. No problem. You wrote exactly this:
“Evaluation of an NN engine contains the move vector what refers to each possible moves for black and white from that position. Can we rebuild the position from what that move vector was derived? Practically the answer is Yes”.

Error 1. “NN” “contains” one sides “moves” only. Not black and white. Good luck with rebuilding a position with one sides moves only.

Error 2. A “move vector” is origin square to destination square. No information about piece type. Good luck with working out, by definition with the actual piece positions not known, whether e4e5 is a pawn, queen, rook or king move.

Error 3. Your “move vector”, policy matrix, call it what you like is a map of move probabilities for all possible chess moves, in chess, not just the position you don’t know and want to reconstruct. Some valid moves will have very low probabilities. Some non-existent moves will have indeterminate but finite probabilities. Good luck, absent the position, with disentangling what represents a move and what doesn’t.

Error 4. Your “move vector” is only a valid list of legal moves because you already know the legal moves, and you already only know the legal moves if you know the position. But you don’t, by definition, know what you are trying to reconstruct.

So, not only wrong (Pauli) applies. Bad model, insufficient thought, error on top of error, no logic. Leading to nonsense conclusion.

You’re welcome.