AlphaZero Chess is not that strong ...

[mclane]

I have done computerchess in the last 38 years.
And we build this forum here to allow people to discuss about computerchess without beeing insulted by trolls.

Show me one GM/IM who replayed the 10 games without beeing impressed by it ! The grandmasters and im’s all know stockfish.

Believe me, any chess player can evaluate this situation.
And the reason is that chess players all over the world do not need to generate or evaluate 70.000.000 NPS to play chess.

[abulmo2]

A0 is fully deterministic. What would you say about the match (100 games) of two classical engines on single core played exclusively from chess starting position?
You wouldn't mind? If that is really the case, you have no clue about computer chess.

The question was not to use an opening book to add some variety, but to use the cerebellum book, that contains months of computation to find the best variations for stockfish.

Could you please show us how is this distinguishing of easy positions done in MCTS+NN and how much Elo does it bring?

MCTS+NN outputs a list of moves with their winning probablilities. It is much easier to find if you have a good enough best move than with alpha beta. In alphabeta you can only allocate more or less time depending on the program returning a stable or unstable score. This is obvioulsy less accurate.

I can tell you how much Elo does 1move/min costs SF playing against same SF with TC 40/40 - over 30Elo.

At 70 MNP/s ?

[Milos]

A0 is fully deterministic. What would you say about the match (100 games) of two classical engines on single core played exclusively from chess starting position?
You wouldn't mind? If that is really the case, you have no clue about computer chess.

The question was not to use an opening book to add some variety, but to use the cerebellum book, that contains months of computation to find the best variations for stockfish.

So you just avoid my question.

Could you please show us how is this distinguishing of easy positions done in MCTS+NN and how much Elo does it bring?

MCTS+NN outputs a list of moves with their winning probablilities. It is much easier to find if you have a good enough best move than with alpha beta. In alphabeta you can only allocate more or less time depending on the program returning a stable or unstable score. This is obvioulsy less accurate.

You seems to be confused. At the end of the search move is chosen based on the visit count not based on the winning probabilities. Probabilities are only used as a primer to guide the exploration.
So how is a visit count better estimation of a hard move than fail-high/low?
You are just making random claims with zero support.

I can tell you how much Elo does 1move/min costs SF playing against same SF with TC 40/40 - over 30Elo.

At 70 MNP/s ?

No at 25Mnps 32 threads on 16 cores (I don't have more powerful machine) from the chess starting position, 100 games, actually I got result 11+/88=/1- for TC 40/40 vs 1/1.
Do you really expect that this would drop below lets say below 25Elo at 70Mnps?
Next in the queue is the match of newest asmBrainfish with Cerebellum limited to 10 moves only, 6-men Syz and 16GB hash 40/40 TC vs SF8, 512MB hash, no book, no EGBT and 512MB Hash 1/1 TC, 100 games from chess starting position alternating colors.
I bet the difference is going to be higher than in case of A0 and SF8.

[shrapnel]

And nothing can turn the wheel back.

What Wheel ? Milos and Tsvetkov don't even believe in the Wheel, what to say of AlphaZero ?!

[shrapnel]

Its very obvious that the bean counter who computed 70.000.000 NPS
and came 23-27 searches deep were completely smashed away by the 80.000 NPS b-strategy neural network.

This is what all humans see, all humans who play chess, no matter which elo they have, see how stupid the bean counter stockfish looked.
With its pieces nearly in the beginning position while alpha zero had everything developed and lots of space, stockfish had to move backwards and almost had Zugzwang positions were it was helpless.

Ask the grandmasters who comment or the world chess champions who commented.

Stop trolling . What is the thing in the games you don’t understand ?

Alpha zero played very idealistic and speculative chess.
It played for a plan while stockfish had no clue what was going on.


The approach to create chess strength by alpha beta failed.
The paradigm came to an end.
There is now a new boss in town. And it plays a completely different chess.

Superb, Realistic Post.
Couldn't have put it better myself.
More than just Stockfish's defeat, I think what's really irking the Alpha Beta Lobby is the consummate ease with which AlphaZero won.
Even worse, and which was unforgivable from their point of view was that in some Games, Stockfish was made to appear practically stupid and noobish, especially with nearly all its Pieces stuck on the Back Rank.
Awesome Power Display by AlphaZero !
And as you rightly pointed out, one doesn't have to be a Programmer or even a very strong Chess Player to see that Stockfish was not just beaten but completely outclassed !
It wasn't just a routine win of a chess engine over another.

[shrapnel]

A0 wasn't able to find them in 72 of these games.[/b]
?

The 72 Draws Stockfish managed to eke out, which you are so proud of, only happened because, rather than Stockfish, it was AlphaZero which was handicapped by being given only 1 min/move.
As the Graph in the DeepMind Paper shows, strength of AlphaZero increases rapidly the more time it is given, much more so than Stockfish.
So, if it had been given 2 min/move, Stockfish would probably be looking at a 72-0 Score instead of 28-0 !
Put that in you Pipe and smoke it.

[Laskos]

The results of the match AZ vs SF8 is translated in +100 Elo for AZ.
The weakened part of Stockfish for this match is already known :
1) Only 1 GB hashtable (64GB to 128GB would have been more way more suitable)
2) 1 minute by move fixed time control
3) No opening book for Stockfish (AZ saved his opening knowledge from its previous games)
4) Only version 8 (the current development version is already about 40 Elo above)

With this 4 points upgraded to a regular level, the current Stockfish version would been already 100 Elo stronger.

One more thing, when you look at the graph : https://www.dropbox.com/s/kaq37zf0z66d6 ... g.png?dl=0
you see there's no progress from 300,000 to 700,000 steps. This AZ is already at his best !

To summarize a bit about what I read in this thread (aside that troll beating Stockfish).

1/ Elo terminology is a bit misleading here.

Take these 3 results:

+2 -0 =8
+20 -0 =80
+200 -0 =800

Elo-wise the difference is the same. What can be said about the last result is that the advantage is HUGE, and only the drawishness of the Chess is distorting Elo perceptions. Elo becomes a bit irrelevant, and something like Normalized Elo or WILO is more useful to describe the results more straightforwardly.

I don't know why people are forgetful about Checkers paradigm, where +2 -0 =98 result was considered decisive. Elo-wise it's +7 Elo points, more precisely 7 +/- 10 Elo (2SD) points, not even decisive.

And all in all, all your objections of handicapping SF8 are almost irrelevant.

2/ I hope DM will soon improve on their NN MCTS, for example by including chess specific features in training and by using stronger hardware in their actual games (it scales well). Then aside playing Stockfish in whatever favorable conditions (huge hash size and such ), let them play an almost deterministic super-A0 against Cerebellum enabled super-A0 in self-games. We might see that all our best opening books are junk full of blunders. And if a super-improved almost deterministic A0 will want to play almost the same openings with transpositions all the time, it might mean that other openings are blunders.

3/ Elo Plateauing.
The result does indicate an Elo plateau not far away. It is only a hint, but I hope DM will continue improve, as they did with Go, and we will see how much Elo they gain. Again, Elo becomes a bit of nuisance for measuring strength in these conditions.

4/ By introducing non-weakening small non-determinism (which anyway happens with parallel processing), I am curious to see the draw rate of improved A0 in self games at this long time control. Without an opening book, again, it's entire possible that all our opening books are junk distorting the outcome.

5/ I don't understand people's criticism of a preprint in arXiv. They came with an outstanding result and rushed to publicize it. Google or me, like Google, I also do my best to get publicity when I find something important and interesting. It's a normal practice, Google or whoever

[pilgrimdan]

The results of the match AZ vs SF8 is translated in +100 Elo for AZ.
The weakened part of Stockfish for this match is already known :
1) Only 1 GB hashtable (64GB to 128GB would have been more way more suitable)
2) 1 minute by move fixed time control
3) No opening book for Stockfish (AZ saved his opening knowledge from its previous games)
4) Only version 8 (the current development version is already about 40 Elo above)

With this 4 points upgraded to a regular level, the current Stockfish version would been already 100 Elo stronger.

One more thing, when you look at the graph : https://www.dropbox.com/s/kaq37zf0z66d6 ... g.png?dl=0
you see there's no progress from 300,000 to 700,000 steps. This AZ is already at his best !

To summarize a bit about what I read in this thread (aside that troll beating Stockfish).

1/ Elo terminology is a bit misleading here.

Take these 3 results:

+2 -0 =8
+20 -0 =80
+200 -0 =800

Elo-wise the difference is the same. What can be said about the last result is that the advantage is HUGE, and only the drawishness of the Chess is distorting Elo perceptions. Elo becomes a bit irrelevant, and something like Normalized Elo or WILO is more useful to describe the results more straightforwardly.

I don't know why people are forgetful about Checkers paradigm, where +2 -0 =98 result was considered decisive. Elo-wise it's +7 Elo points, more precisely 7 +/- 10 Elo (2SD) points, not even decisive.

And all in all, all your objections of handicapping SF8 are almost irrelevant.

2/ I hope DM will soon improve on their NN MCTS, for example by including chess specific features in training and by using stronger hardware in their actual games (it scales well). Then aside playing Stockfish in whatever favorable conditions (huge hash size and such ), let them play an almost deterministic super-A0 against Cerebellum enabled super-A0 in self-games. We might see that all our best opening books are junk full of blunders. And if a super-improved almost deterministic A0 will want to play almost the same openings with transpositions all the time, it might mean that other openings are blunders.

3/ Elo Plateauing.
The result does indicate an Elo plateau not far away. It is only a hint, but I hope DM will continue improve, as they did with Go, and we will see how much Elo they gain. Again, Elo becomes a bit of nuisance for measuring strength in these conditions.

4/ By introducing non-weakening small non-determinism (which anyway happens with parallel processing), I am curious to see the draw rate of improved A0 in self games at this long time control. Without an opening book, again, it's entire possible that all our opening books are junk distorting the outcome.

5/ I don't understand people's criticism of a preprint in arXiv. They came with an outstanding result and rushed to publicize it. Google or me, like Google, I also do my best to get publicity when I find something important and interesting. It's a normal practice, Google or whoever

thanks Kai for this post ... what you say ... makes sense to me ...

[Werewolf]

The approach to create chess strength by alpha beta failed.
The paradigm came to an end.
There is now a new boss in town. And it plays a completely different chess.

Any chess player in the world sees what was going on.
It seems to me the only person in this forum who is not willing to see the truth is you.
Alpha zero beated the search tree beasts.
The bean counters.
Yesterday Komodo , Houdini and stockfish were the best chess program chess programming of the last 50 years was capable to build, and suddenly today, within a day after the event, these whole 50 years was reset by a machine that taught itself how to play chess.

This is not correct.
You have seen ten (10!) games of Alpha Zero. The 10 best, cherry picked games. I am graded 2120 elo. If I showed you my 10 best games you'd honestly think I was 2500 elo. If I showed you my 10 worst you'd think I was 1400 elo.

You can't judge until you've seen all 100 games. So why are they hiding the other 90?

Secondly, as pointed out numerous times Stockfish was artificially weakened in about 5 (!!) different ways.

All they wanted was a strapline: "Alpha Zero crushes Stockfish".

And when people don't think, they get it.

[hgm]

Yeah, sure. It was 'artificially weakened' because it had to play his own moves, rather than a book playing on its behalf. How incredibly unfair and totally crippling... Especially since AlphaZero had to do the same. Totally outrageous to be subjected to the same match conditions as your opponent. If that isn't cheating, I would not know what is!