Lc0 51010

[Albert Silver]

btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.

By sheer coincidence, I was speaking to a very strong player (2700) yesterday, who said that Leela set to 1 node (not zero) beat him more often than not in blitz time control.

[cdani]

What AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.

That would be a continuum. Disagree. The term “material” has been shown to be meaningless. It was a useful heuristic in a world where we had no choice but to find heuristics to work with, and everybody settled on assessing the “mtrl” and adding it to the “psnl” to gave the “eval”, the latter being more or less “accurate”. It kind of worked, some people thought it was Deus, but it was basically a nonsense.
Zero approach confirms that material, king safety and mobility et al are artificial constructs, and Zero approach completely discards them for a holistic statistic, very good most of the time, but with glaringly imbecilic errors in many specific cases.
Everything that was known was wrong, even the words are wrong. Theory, based on wrong words, wrong assessment criteria, is, unless the space is small enough, also going to be wrong.

Of course material, king safety and mobility et al are artificial constructs, as you say, but we always knew this.

Well, some of us did, but if mainstream talkchess was anything to go by, very few. The constructs were treated as real. Adding all the constructs together with “correct” weights to give “accuracy” plus mantra “chess is tactics” was and is not only wrong, it’s actually nonsense.

They are just the best we can do as human players,

nothing to do with computers, these were human heuristics, we’re very good at heuristics.

unable to do millions of calculations like a NN. The NNs can show us specific positions where our general rules lead to a wrong conclusion,

that’s funny. I would say the opposite, the NNs show how we were right and chess programming “community” as represented by talkchess, was wrong.

but I don't know of any new rules that human players can use as a result of the NNs,

that’s a continuum assumption. Tear up the old rules and realise that SF is not god.

except maybe to put a little less weight on material vs mobility and king safety. Can you state even one new "rule" or principle that human players can use as a result of the NNs that will result in an increase in Elo rating?

well, since the NN’s are completely incapable of communicating to us any form of “why this and not that” other that “I ran it through the network and the probability number came out higher”, you’re not going to get any sub-concept information (NN doesn't have sub concepts), you’ll only, at this stage, be able to get overviews. How about, be brave, or barrel on into complexity, or, the robot you are facing is not invincible and makes mistakes too? btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.

The "truth" of a position is only reachable with brute force. NN engines and AB engines visit and "understand" a tiny part of the possible nodes. NN engines are better at some positions and AB engines at some others. In chess we have the advantage that AB engines are so advanced that NN engines
have trouble leaving much behind them, at least for the moment, unlike other fields where NN trashed best known techniques soon.

In general terms, NN are intrinsically stupid (as AB). Only that they are stupid in different ways, like becoming racist in curriculum analysis or saying wrong the capital of a country because they don't know when makes sense to consult some data instead of deducing it.

Will NN engines leave much behind AB engines? Why not, in understanding they are already clearly better. But chess search space seems to be relatively limited in the sense that a traditional engine will probably be able to give war still for quite some time yet.
In the game of Go seems not to be the case, but this does not make NN Go engines nowhere near perfect, just limited in ways that we are not able to formalize yet.

[lkaufman]

btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.

By sheer coincidence, I was speaking to a very strong player (2700) yesterday, who said that Leela set to 1 node (not zero) beat him more often than not in blitz time control.

I played it with depth set to 1, which resulted in a move always showing 50% win prob., which I assume means that it is the policy network move without an eval. I suppose 1 node would just play the same move but would show a realistic win prob. for it. In blitz I had no chance, so I can believe that a 2700 might come out behind. Probably my play at 2 hours plus 30 sec inc. is somewhat stronger than a 2700 playing blitz, so perhaps I might have a fighting chance with that much time vs policy network move, but at any rapid or near-rapid level I would be the underdog.

[jp]

btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.

By sheer coincidence, I was speaking to a very strong player (2700) yesterday, who said that Leela set to 1 node (not zero) beat him more often than not in blitz time control.

I played it with depth set to 1, which resulted in a move always showing 50% win prob., which I assume means that it is the policy network move without an eval. I suppose 1 node would just play the same move but would show a realistic win prob. for it. In blitz I had no chance, so I can believe that a 2700 might come out behind. Probably my play at 2 hours plus 30 sec inc. is somewhat stronger than a 2700 playing blitz, so perhaps I might have a fighting chance with that much time vs policy network move, but at any rapid or near-rapid level I would be the underdog.

How does nodes=1 make Lc play better than nodes=0?

What is the difference in strength of the two settings? For older networks, there's no way nodes=0 could beat a very strong player.

[lkaufman]

btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.

By sheer coincidence, I was speaking to a very strong player (2700) yesterday, who said that Leela set to 1 node (not zero) beat him more often than not in blitz time control.

I played it with depth set to 1, which resulted in a move always showing 50% win prob., which I assume means that it is the policy network move without an eval. I suppose 1 node would just play the same move but would show a realistic win prob. for it. In blitz I had no chance, so I can believe that a 2700 might come out behind. Probably my play at 2 hours plus 30 sec inc. is somewhat stronger than a 2700 playing blitz, so perhaps I might have a fighting chance with that much time vs policy network move, but at any rapid or near-rapid level I would be the underdog.

How does nodes=1 make Lc play better than nodes=0?

What is the difference in strength of the two settings? For older networks, there's no way nodes=0 could beat a very strong player.

As far as I can tell, nodes = 0, nodes = 1, and depth = 1 all play the same moves, which appear to be the policy network move, since they all show a zero centipawn score or 50% win prob. consistently. I tried network 11248 at depth 1 and it beat me soundly in a blitz game. When you say it couldn't beat a strong player, did you mean at standard tc or at blitz, and what is the basis for your statement?

[carldaman]

Excuse me for a moment, but where does one set nodes or depth for Leela? I'm looking at the uci options in the GUI, but I must be missing something.

Thanks in advance!

[Albert Silver]

Excuse me for a moment, but where does one set nodes or depth for Leela? I'm looking at the uci options in the GUI, but I must be missing something.

Thanks in advance!

It is something that must be set in the time control or match settings. The Fritz GUI does not support this, but Arena and Cute Chess do.

[Raphexon]

What AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.

That would be a continuum. Disagree. The term “material” has been shown to be meaningless. It was a useful heuristic in a world where we had no choice but to find heuristics to work with, and everybody settled on assessing the “mtrl” and adding it to the “psnl” to gave the “eval”, the latter being more or less “accurate”. It kind of worked, some people thought it was Deus, but it was basically a nonsense.
Zero approach confirms that material, king safety and mobility et al are artificial constructs, and Zero approach completely discards them for a holistic statistic, very good most of the time, but with glaringly imbecilic errors in many specific cases.
Everything that was known was wrong, even the words are wrong. Theory, based on wrong words, wrong assessment criteria, is, unless the space is small enough, also going to be wrong.

Of course material, king safety and mobility et al are artificial constructs, as you say, but we always knew this.

Well, some of us did, but if mainstream talkchess was anything to go by, very few. The constructs were treated as real. Adding all the constructs together with “correct” weights to give “accuracy” plus mantra “chess is tactics” was and is not only wrong, it’s actually nonsense.

They are just the best we can do as human players,

nothing to do with computers, these were human heuristics, we’re very good at heuristics.

unable to do millions of calculations like a NN. The NNs can show us specific positions where our general rules lead to a wrong conclusion,

that’s funny. I would say the opposite, the NNs show how we were right and chess programming “community” as represented by talkchess, was wrong.

but I don't know of any new rules that human players can use as a result of the NNs,

that’s a continuum assumption. Tear up the old rules and realise that SF is not god.

except maybe to put a little less weight on material vs mobility and king safety. Can you state even one new "rule" or principle that human players can use as a result of the NNs that will result in an increase in Elo rating?

well, since the NN’s are completely incapable of communicating to us any form of “why this and not that” other that “I ran it through the network and the probability number came out higher”, you’re not going to get any sub-concept information (NN doesn't have sub concepts), you’ll only, at this stage, be able to get overviews. How about, be brave, or barrel on into complexity, or, the robot you are facing is not invincible and makes mistakes too? btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.

The "truth" of a position is only reachable with brute force. NN engines and AB engines visit and "understand" a tiny part of the possible nodes. NN engines are better at some positions and AB engines at some others. In chess we have the advantage that AB engines are so advanced that NN engines
have trouble leaving much behind them, at least for the moment, unlike other fields where NN trashed best known techniques soon.

In general terms, NN are intrinsically stupid (as AB). Only that they are stupid in different ways, like becoming racist in curriculum analysis or saying wrong the capital of a country because they don't know when makes sense to consult some data instead of deducing it.

Will NN engines leave much behind AB engines? Why not, in understanding they are already clearly better. But chess search space seems to be relatively limited in the sense that a traditional engine will probably be able to give war still for quite some time yet.
In the game of Go seems not to be the case, but this does not make NN Go engines nowhere near perfect, just limited in ways that we are not able to formalize yet.

Chess also has the added advantage (for A/B) that it's discrete, it's a tactical game.
The strongest Go engines (like CrazyStone) were already using MCTS before Google.
And Go engines don't even need to search to be strong.
https://arxiv.org/abs/1511.06410

I'd be very surprised if a chess engine without search can ever be competitive with decent human players. (Roughly 2200-2250 ELO)
Then again, maybe I'm wrong and Lc0's static evaluation is already strong enough.

Ironically, draughts and checkers like games may also not be very suited for A/B (like Go) but due to their much smaller search space it's less of an issue. In the case of Checkers it wasn't an issue since it's weakly solved.
International Draughts(10x10) was a different story, in 2012 Scharzman beat Maximus. And while Maximus wasn't the strongest checkers engine, it still managed to reach an average depth of 24 ply!
https://jhorssen.home.xs4all.nl/Maximus ... aximus.pdf

24ply is insane because another study claims that Houdini 1.5 at 20 ply is as strong as Magnus.

https://pdfs.semanticscholar.org/047f/6 ... f58666.pdf

Obviously part of the reason why humans may have enjoyed a bigger advantage in draughts is that it never attracted the same attention of programmers that chess did. That's true but the same goes for the playerbase and a smaller playerbase is less likely to have Magnus level talents.

On the other hand, why I think draughts-like games (and Go*) are intrinsically harder for A/B search is that since every piece is the same it's much more of a positional game. And while chess has a bigger search space it also has many more obvious bad moves that are quickly pruned. You may have a queen that can move in 12 different ways but if 11 of those result in an immediate loss of queen (without gain) then it's really just a piece that can make 1 move.
On the other hand, since the search space is so small of draughts like games on 8x8 and 10x10 boards it's effectively "brute-forceable" by now.

*From what I've read that even Go on smaller boards with a smaller search space than chess aren't exactly suited for A/B.

[Raphexon]

Seems like I spoke too quickly and 0 node Leela is already quite strong at Blitz.

[chrisw]

What AZ and LZ showed is that there are serious weaknesses in existing A/B programs, which they can exploit. As for what it means for humans playing chess, the A/B engines already showed that mobility and king safety can compensate for material to a greater degree than was generally believed, and the NN engines are just extending that trend. When I look at opening analysis by Lc0, in general it seems more like what we currently believe to be the best lines than does analysis by A/B engines. I am amazed at how rapidly long lines of modern theory appear as the best line in Lc0 analysis.

That would be a continuum. Disagree. The term “material” has been shown to be meaningless. It was a useful heuristic in a world where we had no choice but to find heuristics to work with, and everybody settled on assessing the “mtrl” and adding it to the “psnl” to gave the “eval”, the latter being more or less “accurate”. It kind of worked, some people thought it was Deus, but it was basically a nonsense.
Zero approach confirms that material, king safety and mobility et al are artificial constructs, and Zero approach completely discards them for a holistic statistic, very good most of the time, but with glaringly imbecilic errors in many specific cases.
Everything that was known was wrong, even the words are wrong. Theory, based on wrong words, wrong assessment criteria, is, unless the space is small enough, also going to be wrong.

Of course material, king safety and mobility et al are artificial constructs, as you say, but we always knew this.

Well, some of us did, but if mainstream talkchess was anything to go by, very few. The constructs were treated as real. Adding all the constructs together with “correct” weights to give “accuracy” plus mantra “chess is tactics” was and is not only wrong, it’s actually nonsense.

They are just the best we can do as human players,

nothing to do with computers, these were human heuristics, we’re very good at heuristics.

unable to do millions of calculations like a NN. The NNs can show us specific positions where our general rules lead to a wrong conclusion,

that’s funny. I would say the opposite, the NNs show how we were right and chess programming “community” as represented by talkchess, was wrong.

but I don't know of any new rules that human players can use as a result of the NNs,

that’s a continuum assumption. Tear up the old rules and realise that SF is not god.

except maybe to put a little less weight on material vs mobility and king safety. Can you state even one new "rule" or principle that human players can use as a result of the NNs that will result in an increase in Elo rating?

well, since the NN’s are completely incapable of communicating to us any form of “why this and not that” other that “I ran it through the network and the probability number came out higher”, you’re not going to get any sub-concept information (NN doesn't have sub concepts), you’ll only, at this stage, be able to get overviews. How about, be brave, or barrel on into complexity, or, the robot you are facing is not invincible and makes mistakes too? btw, I read a couple of days ago, you wrote LCZero, set to nodes=0, policy move, would win against you. I very much doubt it. Policy errors. Statistically very good, but quite capable of telling you a cat is a panda with 99% certainty every so often.

The "truth" of a position is only reachable with brute force. NN engines and AB engines visit and "understand" a tiny part of the possible nodes. NN engines are better at some positions and AB engines at some others. In chess we have the advantage that AB engines are so advanced that NN engines
have trouble leaving much behind them, at least for the moment, unlike other fields where NN trashed best known techniques soon.

In general terms, NN are intrinsically stupid (as AB). Only that they are stupid in different ways, like becoming racist in curriculum analysis or saying wrong the capital of a country because they don't know when makes sense to consult some data instead of deducing it.

Will NN engines leave much behind AB engines? Why not, in understanding they are already clearly better. But chess search space seems to be relatively limited in the sense that a traditional engine will probably be able to give war still for quite some time yet.
In the game of Go seems not to be the case, but this does not make NN Go engines nowhere near perfect, just limited in ways that we are not able to formalize yet.

Chess also has the added advantage (for A/B) that it's discrete, it's a tactical game.
The strongest Go engines (like CrazyStone) were already using MCTS before Google.
And Go engines don't even need to search to be strong.
https://arxiv.org/abs/1511.06410

I'd be very surprised if a chess engine without search can ever be competitive with decent human players. (Roughly 2200-2250 ELO)
Then again, maybe I'm wrong and Lc0's static evaluation is already strong enough.

Ironically, draughts and checkers like games may also not be very suited for A/B (like Go) but due to their much smaller search space it's less of an issue. In the case of Checkers it wasn't an issue since it's weakly solved.
International Draughts(10x10) was a different story, in 2012 Scharzman beat Maximus. And while Maximus wasn't the strongest checkers engine, it still managed to reach an average depth of 24 ply!
https://jhorssen.home.xs4all.nl/Maximus ... aximus.pdf

24ply is insane because another study claims that Houdini 1.5 at 20 ply is as strong as Magnus.

https://pdfs.semanticscholar.org/047f/6 ... f58666.pdf

Obviously part of the reason why humans may have enjoyed a bigger advantage in draughts is that it never attracted the same attention of programmers that chess did. That's true but the same goes for the playerbase and a smaller playerbase is less likely to have Magnus level talents.

On the other hand, why I think draughts-like games (and Go*) are intrinsically harder for A/B search is that since every piece is the same it's much more of a positional game. And while chess has a bigger search space it also has many more obvious bad moves that are quickly pruned. You may have a queen that can move in 12 different ways but if 11 of those result in an immediate loss of queen (without gain) then it's really just a piece that can make 1 move.
On the other hand, since the search space is so small of draughts like games on 8x8 and 10x10 boards it's effectively "brute-forceable" by now.

*From what I've read that even Go on smaller boards with a smaller search space than chess aren't exactly suited for A/B.

There was always going to be a game whose rule system put it at the edge of what is and isn’t solvable. Add in a bunch of emotional reasons, even Soviet USA politicing, the fact that it’s the game played in the dominant white man culture, and that game happens to have been chess, white European version, and it stayed somehow as the main game, when others could well have taken over. Peak chess has been and gone, probably taking with it the other contenders too. I’m inclined to take the position it was all much overrated and we didn’t learn anything much generalisable elsewhere once the basic search algorithm was worked out. Neural nets can make some sort of visual order out of the chess board space, but again, so what?, making visual order out of real world is the thing, not chess boards. So long and thanks for all the fish.