recent article on alphazero ... 12/11/2017 ...

[trulses]

Alpha Zero is revolutionary I think.I will email them to ask if they will go commercial.Of course they probably are not as keen into chess as we are.Just think ,this thing destroys Stockfish and prob the current versions also.The possibilities are endless.TAL WAS CORRECT. Really I hope they go commercial or at least open source.

I think if they had commercialization in mind they would've already done something with their Go engine considering the massive hype around it and basically a free-reign on the market. The game is quite popular in Asia so I bet there are some people who would pay quite a bit for access to AlphaGo output.

Maybe it's not viable yet since it would have to be hosted on the GCP or wherever their TPUs are.

The last paper on AlphaZero, however, is very much an early draft, with just enough results to generate a lot of publicity, but not nearly enough details to be able to replicate. IIRC, Deepmind announced the fully peer-reviewed paper to be available soon (no date given). Maybe the Arxiv-version of the Alpha-Zero paper was rushed to generate the publicity during the London Chess Classic last week.

I think it might have been related to the NIPS conference, David Silver, one of the top guys at Deepmind held a talk on AG0 and A0 there. The talk is available on Aske Plaat's youtube channel.

edit: Merged comments

[Dariusz Orzechowski]

You will be right as soon as Google publishes their raw data (which is not going to happen). CERN even published few hundred terabytes of their raw data on the internet last year.

It is not customary in sciece to publish raw data. I never did it, and no one ever asked for it. People that want to reproduce the experiment should collect their own raw data.

But if you want to call it science, raw data should be available if someone wants to scrutinize published results. In same cases it's not practical to collect own raw data even if theoretically possible. I wonder what would you say if people behind Cassini project said that Newton Laws are well-known, everyone is free to build their own space probe, send it to Saturn and collect their own data. At the same time claiming that they found signs of life there.

The only reason I can see that would make it necessary to publish raw data is when the data analysis could be controversial or non-standard. I can very well imagine that this is the case in some of the CERN experiments, where discovery of a new particle often is claimed based on extremely scanty evidence of just a hand-ful of 'events'. It does't seem the case for the AlphaZero experiments.

AlphaZero claims are if fact non-standard otherwise no one would care. A lot of people would like to verify them. But in practice they cannot do this. Leela Zero project tries to do it at much smaller scale so the results cannot be directly compared anyway. Back of envelope calculations show that distributed Leela Zero project generates games around 10000 times slower than Google for recent AlphaZero. Suddenly instead of 34 hours, it takes 40 years. With raw data available it would take only some weeks-months to verify the result.

[hgm]

But if you want to call it science, raw data should be available if someone wants to scrutinize published results. In same cases it's not practical to collect own raw data even if theoretically possible. I wonder what would you say if people behind Cassini project said that Newton Laws are well-known, everyone is free to build their own space probe, send it to Saturn and collect their own data. At the same time claiming that they found signs of life there.

Well, that is how it works. If the data became clearly visible only after some standard image-enhancement method, there wouldn't be any reason for them to publish anything else than the enhanced photograph with the smiling martionsticking up his middle finger (of 3).

AlphaZero claims are if fact non-standard otherwise no one would care.

That is also bullshit. It is not about the claims. It is about the method used to arrive at these claims. If the method is counting won games to establish which player is stronger, there is for instance zero reason to publish the games themselves. Because it is beyond reasonable doubt a sane person will be able to distinguish a won game from a lost one, or be able to count. That applies irrespective of the prior likelihood of the claims (e.g. that a rat was able to beat Kasparove after having been raised on a diet of broccoli).

A lot of people would like to verify them. But in practice they cannot do this. Leela Zero project tries to do it at much smaller scale so the results cannot be directly compared anyway. Back of envelope calculations show that distributed Leela Zero project generates games around 10000 times slower than Google for recent AlphaZero. Suddenly instead of 34 hours, it takes 40 years. With raw data available it would take only some weeks-months to verify the result.

It would be nice if they published the parameters of the trained neural network. But they are under no obligation to do so. There is no conceivable way they could be wrong in their claim that the network they trained achieved the reported results, other than that they committed gross fraud and made up the entire thing. Considering their earlier achievements in Go there is no reason at all to believe the entire thing is a scam. If they could master Go, as was proven beyond any doubt by playing public games against human top players, then surely they must have no difficulty at all with a silly game like Chess. So why would you want to accuse them of fraud?

[George Tsavdaris]

Alpha Zero is revolutionary I think.I will email them to ask if they will go commercial.Of course they probably are not as keen into chess as we are.Just think ,this thing destroys Stockfish and prob the current versions also.The possibilities are endless.TAL WAS CORRECT. Really I hope they go commercial or at least open source.

They said they will publish a complete paper(in Nature i guess) after some time so they will give more details about the method so someone perhaps will make a similar program.

In the email tell them it is a matter of life and death since there are countless obsessed crazy computer Chess fans here that are dying to have it.

[Rein Halbersma]

It would be nice if they published the parameters of the trained neural network. But they are under no obligation to do so.

Reproducibility is the corner stone of science. For experiments where even a single run takes 1,000+ CPU years to run, not giving hyper parameters and having other researchers to a grid search to rediscover them, effectively precludes such reproductions.

There is no conceivable way they could be wrong in their claim that the network they trained achieved the reported results, other than that they committed gross fraud and made up the entire thing. Considering their earlier achievements in Go there is no reason at all to believe the entire thing is a scam. If they could master Go, as was proven beyond any doubt by playing public games against human top players, then surely they must have no difficulty at all with a silly game like Chess. So why would you want to accuse them of fraud?

This is a straw man. No one is accusing Deepmind of fraud. But simply saying "OK, we beat the Go champion, here's some great games of chess that beat Stockfish, and here's a sketch how we did it" -no matter how impressive- is simply not science, but an engineering triumph.

For this to be a valid scientific result, we need the full paper with details that make this reproducible. The fact that they did this for the earlier Go papers, is good enough to wait a little while longer for them to come up with the full paper for the AlphaZero project.

[CheckersGuy]

It seems a complete bullshit paper...

I have to agree. That small "paper" doesnt really qualify as a paper

[Dariusz Orzechowski]

But if you want to call it science, raw data should be available if someone wants to scrutinize published results. In same cases it's not practical to collect own raw data even if theoretically possible. I wonder what would you say if people behind Cassini project said that Newton Laws are well-known, everyone is free to build their own space probe, send it to Saturn and collect their own data. At the same time claiming that they found signs of life there.

Well, that is how it works. If the data became clearly visible only after some standard image-enhancement method, there wouldn't be any reason for them to publish anything else than the enhanced photograph with the smiling martionsticking up his middle finger (of 3).

No, it doesn't work like this, at least not in science. In such a case no one would accept it. Everyone would scream: give us more details! And then look really closely at provided data. Perhaps some people would be fine with just middle finger made of 3 pixels, I don't doubt it. But I certainly wouldn't.

That is also bullshit.

There is something wrong with that sentence.

It would be nice if they published the parameters of the trained neural network. But they are under no obligation to do so.

I agree.

So why would you want to accuse them of fraud?

As far as I know no one said anything about fraud. In science you just don't accept bold claims on insufficient evidence. Google may say whatever they want but it's not science.

[Milos]

Considering their earlier achievements in Go there is no reason at all to believe the entire thing is a scam. If they could master Go, as was proven beyond any doubt by playing public games against human top players, then surely they must have no difficulty at all with a silly game like Chess. So why would you want to accuse them of fraud?

This paragraph of yours is total and utter bullshit.
You really enjoy making apples and orangutans comparison as Tord would say.
Following your comparisons logic, match A0 vs SF was as if they played Lee in Go, where Lee would have been given 1min/move and being heavily and visibly drunk at the same time.

[peter]

Hi!

Reproducibility is the corner stone of science.

Google isn't interested in science if not to sell it.
Therefore there won't be a serious paper about the A0- testing, because the better the documentation would get, the clearer the methodical bias of the match would be to be seen.

The only reason for Google to go on with A0 testing and developing would be a hope for selling anything by any means as a follow up.
Scientific results aren't worth anything to Google per se, at least there's no reason for them to publicise them.

If A0 really was as good in chess already against reasonable opening theory of today I could imagine two economic models of selling scientific results:

Other programmers could get use of A0 at least as for testing their own programs against, and chessplayers could get opening lines from it, pity both would be worth renting or buying some time or insight in A0- playing only, if it could be kept as secret of the user's own too then.
(Rybka- Cluster by Rajlich and Cimiotti, Hydra by Donninger and PAL were two examples about direct connection between Hardware- Software- Know How and money in chess.)

And the second way, which is probably the only one to go for Google till now, is to earn real counting attention for some scientific publication, but not only from other scientists, but from popular- scientific audience as well, the latter one probably counting even more.

And Google always has to keep scientific results as their own, if they want to sell something based on the results directly, so don't expect really open and openminded documentation of A0, as long as you can't offer some business in exchange neither.

As long as I haven't seen a serious paper in a journal of any impact factor in mathematics, IT or at least AI, I wouldn't hope for any further public development of A0 at all.

Still hoping

[hgm]

Considering their earlier achievements in Go there is no reason at all to believe the entire thing is a scam. If they could master Go, as was proven beyond any doubt by playing public games against human top players, then surely they must have no difficulty at all with a silly game like Chess. So why would you want to accuse them of fraud?

This paragraph of yours is total and utter bullshit.
You really enjoy making apples and orangutans comparison as Tord would say.
Following your comparisons logic, match A0 vs SF was as if they played Lee in Go, where Lee would have been given 1min/move and being heavily and visibly drunk at the same time.

So what is your point? That we should suspect a big fraud if they manage to beat a drunk Lee after they managed to beat a sober one without any possibility of cheating?