Alphazero news

[hgm]

Either way, this proves my point that by introducing diversity to the opening moves does not seem to harm an engine's performance in any way, even if the engine is a perfect player.

It doesn't prove anything of the sort. (And which 'ways' are you distinguishing here anyway?) The existence of a single line that would lead to a position where fallible opponents would almost always blunder away the theoretical draw, and any deviation of that line bringing you to a postion from where fallible opponents would almost always lose (even to equal or somewhat weaker players) is not at all incompatible with all positions in these lines being theoretical draws (i.e. with perfect play).

Even in the case of a theoretical loss it is not clear at all that the theoretically best defence (delaying the mate as long as possible) would give the best chances for a 'swindle' against a fallible opponent. In fact ample examples exist where a 'perfect' computer player (having the checkmate within its horizon, which is tantamount to playing a solved game) did himself in by trying to delay a mate through immediate sacrifice of all its material, guaranteeing a loss even if the opponent would not have seen the mate. There is this famous story about this computer from before the Deep Blue era that suddenly sacrificed a Rook when playing a team of human GMs (thus giving them a trivial win), just because it was the only way to avoid a mate in 22 that the opponents would not have spotted in a million years.

[Gary Internet]

Until I can watch a TCEC superfinal style of match live on the internet, where the latest development version of SF plays AZ over 100 games starting from both sides of 50 different openings, where both engines run on whatever hardware their teams deem optimal, discussions like this are worthless. People trying desperately to theorise this, blame that, speculate about the other. Either let the engines play on a perfectly level playing field in a public setting, or stop going on about it.

That's why I love TCEC and CCC. We can actually see how Leela really performs against other engines and we can ignore the people trying to predict things based on "self play Elo" and other nonsense.

[Laskos]

If AZ can always play into closed openings from start position no matter what the opponent does, why should its performance on open openings be reflected in its Elo rating?

It is a question of philosophy. As 100% of the practical use of chess engines consists of analysis one can argue that a chess engine should be able to play good chess in any (reasonable) position...

That statement of Matthew is not true. A0 cannot play always as it likes the openings (closed, less tactics). With "Leela Ratio" of about 2-3, Lc0 is comparable to A0 playing against SF (10 or 8).

Short time control.

First test:
Lc0 No Book vs SF10 BookX.bin:
Score of lc0_v19_11261 vs SF10: 5 - 16 - 19 [0.362] 40
Elo difference: -98.07 +/- 79.65

SF10 used a varied BookX.bin, a good and small book. That is by all my experience a bad result for Lc0, although Lc0 plays all the games from the start by itself, no artifacts hindering it are introduced. The next test is underway: both play from 1 Initial Board position, no any books or openings. And then, both play from the so called "human openings". If these, last two, come significantly better for Lc0 than enabling SF10 with BookX.bin, then the lack of variety and the selection of "human openings" distorts the results compared to varied sorts of openings in BookX (for SF10 only, Lc0 is left alone to deal all from the beginning by itself). If these things happen (I will post the next results in say 2 hours), A0 CANNOT steer every game into convenient opening. In fact, I knew that from my experience with Lc0, so for me the only reliable result was that from TCEC "outrageous" openings. But let's see the next results.

[corres]

Until I can watch a TCEC superfinal style of match live on the internet, where the latest development version of SF plays AZ over 100 games starting from both sides of 50 different openings, where both engines run on whatever hardware their teams deem optimal, discussions like this are worthless. People trying desperately to theorise this, blame that, speculate about the other. Either let the engines play on a perfectly level playing field in a public setting, or stop going on about it.

That's why I love TCEC and CCC. We can actually see how Leela really performs against other engines and we can ignore the people trying to predict things based on "self play Elo" and other nonsense.

I agree with an addendum: Every result should be decided by the play of contenders without any "TCEC rules".

[Thomas A. Anderson]

If AZ can always play into closed openings from start position no matter what the opponent does, why should its performance on open openings be reflected in its Elo rating?

It is a question of philosophy. As 100% of the practical use of chess engines consists of analysis one can argue that a chess engine should be able to play good chess in any (reasonable) position...

That statement of Matthew is not true. A0 cannot play always as it likes the openings (closed, less tactics). With "Leela Ratio" of about 2-3, Lc0 is comparable to A0 playing against SF (10 or 8).
rom the beginning by itself). If these things happen (I will post the next results in say 2 hours), A0 CANNOT steer every game into convenient opening. In fact, I knew that from my experience with Lc0, so for me the only reliable result was that from TCEC "outrageous" openings. But let's see the next results.

I think you misunderstood the sentence, it starts with an "If" and was certainly meant as conjunctive construction. Common consensus seems to be that the "fairest" match condition is starting from the initial position and let SF use whatever book and with whatever diversification it wants to use. Same for AZ. I start considering different starting positions as different disciplines. Because we have no clue if AZ would be able to avoid running into some/many of that positions, it would be unfair testing AZ on a possibly very untrained battlefield.

[noobpwnftw]

Either way, this proves my point that by introducing diversity to the opening moves does not seem to harm an engine's performance in any way, even if the engine is a perfect player.

It doesn't prove anything of the sort. (And which 'ways' are you distinguishing here anyway?) The existence of a single line that would lead to a position where fallible opponents would almost always blunder away the theoretical draw, and any deviation of that line bringing you to a postion from where fallible opponents would almost always lose (even to equal or somewhat weaker players) is not at all incompatible with all positions in these lines being theoretical draws (i.e. with perfect play).

Even in the case of a theoretical loss it is not clear at all that the theoretically best defence (delaying the mate as long as possible) would give the best chances for a 'swindle' against a fallible opponent. In fact ample examples exist where a 'perfect' computer player (having the checkmate within its horizon, which is tantamount to playing a solved game) did himself in by trying to delay a mate through immediate sacrifice of all its material, guaranteeing a loss even if the opponent would not have seen the mate. There is this famous story about this computer from before the Deep Blue era that suddenly sacrificed a Rook when playing a team of human GMs (thus giving them a trivial win), just because it was the only way to avoid a mate in 22 that the opponents would not have spotted in a million years.

I failed see what you are trying to argue. Now I quote myself again:

There is no hard proof that by introducing diversity to its opening moves "weakens" the engine in any way, that's like saying Berlin is inferior to French, one could also make an argument saying that they are exactly equivalent and if some statistics are favoring one over the other, then those statistics are flawed, which both can be verified only if chess is solved.

that's like saying Berlin is inferior to French

Ways number one, which I think is BS.

saying that they are exactly equivalent

Ways number two, which you think is BS.

There is no hard proof that by introducing diversity to its opening moves "weakens" the engine in any way

Proves this point.

I do not care about how a perfect player may look stupid to you, I just argue that

We weakened AZ for diversity instead.

is nonsense.

For (1) with an approximation approach, the overall better one benefits more from diversity than the weaker one, for (2) even a perfect player would benefit from diversity against a imperfect player, for (3) the only case that it can be weakened is that the approximation is not uniformly better but heavily biased against diversity.

[Laskos]

If AZ can always play into closed openings from start position no matter what the opponent does, why should its performance on open openings be reflected in its Elo rating?

It is a question of philosophy. As 100% of the practical use of chess engines consists of analysis one can argue that a chess engine should be able to play good chess in any (reasonable) position...

That statement of Matthew is not true. A0 cannot play always as it likes the openings (closed, less tactics). With "Leela Ratio" of about 2-3, Lc0 is comparable to A0 playing against SF (10 or 8).
rom the beginning by itself). If these things happen (I will post the next results in say 2 hours), A0 CANNOT steer every game into convenient opening. In fact, I knew that from my experience with Lc0, so for me the only reliable result was that from TCEC "outrageous" openings. But let's see the next results.

I think you misunderstood the sentence, it starts with an "If" and was certainly meant as conjunctive construction. Common consensus seems to be that the "fairest" match condition is starting from the initial position and let SF use whatever book and with whatever diversification it wants to use. Same for AZ. I start considering different starting positions as different disciplines. Because we have no clue if AZ would be able to avoid running into some/many of that positions, it's would be unfair testing AZ on a possibly very untrained battlefield.

See the results:

Lc0 No Book vs SF10 BookX.bin:
Score of lc0_v191_11261 vs SF10: 5 - 16 - 19 [0.362] 40
Elo difference: -98.07 +/- 79.65


Lc0 No Book vs SF10 No Book (Initial Board position):
Score of lc0_v19_11261 vs SF10: 12 - 6 - 22 [0.575] 40
Elo difference: 52.51 +/- 73.05


Lc0 vs SF10 from "12 human openings":
Score of lc0_v19_11261 vs SF10: 8 - 5 - 27 [0.537] 40
Elo difference: 26.11 +/- 61.76

In the paper only the latter 2 are presented, but the first one, which is close to 120-150 Elo points lower for Lc0, without any hindering to its playing (it plays all by itself form the start position), is absent. And the almost deterministic "best Cerebellum book moves" with A0 diversification is again a practice playing into A0 strength. In fact the first result is the most relevant, as Lc0 "shouldn't be hindered by openings", and if the diversification comes from SF10, SF will be punished by Lc0 all by itself, right?

Aside from lack of diversification or a bit of diversification coming mostly from A0, all this relates too to the fact that A0 (Lc0) is a more specialized task solver than a regular engine, a thread I posted two or three months ago. So, take A0 away from what it specializes in, say with an opening book, and it will buckle its strength.

[Thomas A. Anderson]

Nailing it down to the initial position testing, the only thing you are missing is this the diversification-UCI parameter in the AZ vs. BF test. A test with LC0 against a standard engine with and without that diversification is what should be interesting, right?

Aside from lack of diversification or a bit of diversification coming mostly from A0, all this relates too to the fact that A0 (Lc0) is a more specialized task solver than a regular engine, a thread I posted two or three months ago. So, take A0 away from what it specializes in, say with an opening book, and it will buckle its strength.

The results from your short test 2 and 3, might indeed give the impression that playing strength AZ is depending more on the initial position. Instead of calling AZ because of that a "specialized task solver", I would say that the NN wasn't trained enough on that positions. What might be a consequence of the monocultural self-training process. If DM would have had the task to strike AB-Engines in TCEC positions, the training process would have been different I think.

[Laskos]

Nailing it down to the initial position testing, the only thing you are missing is this the diversification-UCI parameter in the AZ vs. BF test. A test with LC0 against a standard engine with and without that diversification is what should be interesting, right?

Aren't my results interesting?
I will not persevere, as you seem to have some agenda or ideology, I am not into this.

Congratulations anyway to DeepMind team for an important paper in "Science".

[jp]

If AZ can always play into closed openings from start position no matter what the opponent does, why should its performance on open openings be reflected in its Elo rating?

What makes you believe that?

It is a question of philosophy. As 100% of the practical use of chess engines consists of analysis one can argue that a chess engine should be able to play good chess in any (reasonable) position...

That statement of Matthew is not true. A0 cannot play always as it likes the openings.

Yes, I agree with Kai. Chess is not like that. No one or thing can "always play into closed openings no matter what the opponent does". That claim is not true.