David Silver (Deepmind) inaccuracies

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Rebel
Posts: 4299
Joined: Thu Aug 18, 2011 10:04 am

David Silver (Deepmind) inaccuracies

Post by Rebel » Thu Dec 21, 2017 5:25 pm

David Silver (Deepmind) inaccuracies (1½ minute snippet).

https://www.youtube.com/watch?v=3kEMwF0 ... e=youtu.be

#1. TCEC technically is not the computer chess world championship. Stockfish isn't the ICGA world champion, Komodo is. Mark will not be amused.

#2. What does David Silver mean when he says: "this previous world champion"?

Lyudmil Tsvetkov
Posts: 6037
Joined: Tue Jun 12, 2012 10:41 am

Re: David Silver (Deepmind) inaccuracies

Post by Lyudmil Tsvetkov » Thu Dec 21, 2017 5:55 pm

Rebel wrote:David Silver (Deepmind) inaccuracies (1½ minute snippet).

https://www.youtube.com/watch?v=3kEMwF0 ... e=youtu.be

#1. TCEC technically is not the computer chess world championship. Stockfish isn't the ICGA world champion, Komodo is. Mark will not be amused.

#2. What does David Silver mean when he says: "this previous world champion"?
Thanks for the info.

I am happy I have watched no videos of theirs and read no papers of theirs. :D

Lyudmil Tsvetkov
Posts: 6037
Joined: Tue Jun 12, 2012 10:41 am

Re: David Silver (Deepmind) inaccuracies

Post by Lyudmil Tsvetkov » Thu Dec 21, 2017 6:38 pm

Btw., just to ask once more.

So far, the NN approach has only rendered more than modest results on a normal hardware: around 2400 at most in all cases(Giraffe, Romi, etc.).

Why would the very same approach suddenly perform much better on their hardware?

This simply lacks any sense, and it is all about:
- the hardware
- the weakened SF
- and, most importantly, the opening knowledge/book

SF algorithms, if adapted on their 5000 TPUs, would perform miracles.

I guess a 2400 elo engine should stay where it is: 800 elos below the top SF, Houdini and Komodo.

Sorry, but that is the simple reality.

Lyudmil Tsvetkov
Posts: 6037
Joined: Tue Jun 12, 2012 10:41 am

Re: David Silver (Deepmind) inaccuracies

Post by Lyudmil Tsvetkov » Thu Dec 21, 2017 6:57 pm

Btw., please just look closer at the 10 games provided, I just did that(thanks to Vincent for compiling the pgn):

- 4 games, numbers 3,5,6 and 10, feature a QID, where Alpha invariably pushes d4-d5 in the very same line on either move 6 or 7.
- in 2 games, SF cedes the pair of bishops as early as move 5 in a Ruy Lopez exchange, which is known to at most give equality to white
- 2 hopeless French played by SF

So that, in 8 games, opening knowledge decided for Alpha, and there were 5(!!) repeats, HALH of the published games/openings.

How many repeats are there in the rest of the unpublished games?
Is that the reason they don't want to publish them?
What if we see that Alpha won 13 games with the very same 6. d5 move in the QID out of a total of 28?

No those fully hopeless opening setups, what would the actual score be?
Given all SF handicaps?

Again, I can not believe a software for a 2400 level engine can perform miracles, simply count me out.

Lyudmil Tsvetkov
Posts: 6037
Joined: Tue Jun 12, 2012 10:41 am

Re: David Silver (Deepmind) inaccuracies

Post by Lyudmil Tsvetkov » Thu Dec 21, 2017 7:17 pm

Another major clue: in 6(!!) games out of 10, numbers 1,5,6,7,8 and 10, Alpha fianchettoes its bishop on Bg2.

How does it know Bg2 is a good move, the best actually?
Why does not it do the very same with black, not a single king side fianchetto, Bg7?
Is not the same pattern good for both white and black?

Why no fianchettos with black at all?

Well, I guess you know the answer: because openings books/human knowledge give advantage to white with such setup, but not black(all KIDs, Gruenfelds, etc. don't score very well overall in human databases).

I guess this is obvious proof they used human knowledge to train their openings.

If Alpha had trained and developed the Bg2/Bg7 pattern, why does it play it only with white then? Does not make sense, does it?

Because they rely on statistics and have no patterns at all, that is why. That is the quality of their NN.

Any guess why SF did not play a single king side fianchetto?
Well, I know the answer to this: because it is weak.

You want me to believe Alpha played strong chess: come one, it does not fianchetto with black on g7 and frequently prefers 1.d4(for the same reason, human statistics).

What do you think of that?
I think it is a shame.

User avatar
Eelco de Groot
Posts: 4027
Joined: Sun Mar 12, 2006 1:40 am
Location: Groningen

Re: David Silver (Deepmind) inaccuracies

Post by Eelco de Groot » Thu Dec 21, 2017 7:27 pm

Rebel wrote: #2. What does David Silver mean when he says: "this previous world champion"?
I think that he means that Stockfish 8 was the previous TCEC winner, which would be correct, although you can have doubts that TCEC is not a FIDE recognized championship. Houdini now being the TCEC champion. I don't think Mark Lefler would have liked much being Alpha Zero's guinea pig instead of SF, although he would have liked learning from the games.
Debugging is twice as hard as writing the code in the first
place. Therefore, if you write the code as cleverly as possible, you
are, by definition, not smart enough to debug it.
-- Brian W. Kernighan

User avatar
Rebel
Posts: 4299
Joined: Thu Aug 18, 2011 10:04 am

Re: David Silver (Deepmind) inaccuracies

Post by Rebel » Thu Dec 21, 2017 7:33 pm

Eelco de Groot wrote:
Rebel wrote: #2. What does David Silver mean when he says: "this previous world champion"?
I think that he means that Stockfish 8 was the previous TCEC winner, which would be correct, although you can have doubts that TCEC is not a FIDE recognized championship. Houdini now being the TCEC champion. I don't think Mark Lefler would have liked much being Alpha Zero's guinea pig instead of SF, although he would have liked learning from the games.
To me it sounds as brutal commerce, we AZ are now WC :wink:

User avatar
hgm
Posts: 23000
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: David Silver (Deepmind) inaccuracies

Post by hgm » Thu Dec 21, 2017 8:14 pm

Lyudmil Tsvetkov wrote:So far, the NN approach has only rendered more than modest results on a normal hardware: around 2400 at most in all cases(Giraffe, Romi, etc.).

Why would the very same approach suddenly perform much better on their hardware?
Because it actually is a very different approach.

In AlphaZero the NN is used to guide the search. In Giraffe it was oly used for evaluation. In Romi Chess no was used at all.

Lyudmil Tsvetkov
Posts: 6037
Joined: Tue Jun 12, 2012 10:41 am

Re: David Silver (Deepmind) inaccuracies

Post by Lyudmil Tsvetkov » Thu Dec 21, 2017 8:20 pm

hgm wrote:
Lyudmil Tsvetkov wrote:So far, the NN approach has only rendered more than modest results on a normal hardware: around 2400 at most in all cases(Giraffe, Romi, etc.).

Why would the very same approach suddenly perform much better on their hardware?
Because it actually is a very different approach.

In AlphaZero the NN is used to guide the search. In Giraffe it was oly used for evaluation. In Romi Chess no was used at all.
This is just surmising. You don't know how they did it.

Lyudmil Tsvetkov
Posts: 6037
Joined: Tue Jun 12, 2012 10:41 am

Re: David Silver (Deepmind) inaccuracies

Post by Lyudmil Tsvetkov » Thu Dec 21, 2017 8:23 pm

Rebel wrote:
Eelco de Groot wrote:
Rebel wrote: #2. What does David Silver mean when he says: "this previous world champion"?
I think that he means that Stockfish 8 was the previous TCEC winner, which would be correct, although you can have doubts that TCEC is not a FIDE recognized championship. Houdini now being the TCEC champion. I don't think Mark Lefler would have liked much being Alpha Zero's guinea pig instead of SF, although he would have liked learning from the games.
To me it sounds as brutal commerce, we AZ are now WC :wink:
When you think Rebel was once close to being it...

Perhaps I should challenge Alpha to a match, they might be running on theor 5000 TPUs, I guess I still have a fair chance, as it plays very weak in the opening. :D

Post Reply