Alphazero news

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Gian-Carlo Pascutto
Posts: 1184
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: Alphazero news

Post by Gian-Carlo Pascutto » Tue Dec 11, 2018 3:12 pm

Matthew, I have one last question regarding the implementation details that aren't explicit in the paper.

Normally, an MCTS search would do "tree reuse" from move to move, carrying forward the subtree that was actually chosen. But during the training, there is noise added to the root. If one does tree reuse, the effect of the noise is lessened. Leela Chess Zero decided to disable tree reuse because of that. Leela Zero has kept it enabled.

Could you clarify how it was done in DeepMind's version?

User avatar
Laskos
Posts: 9441
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Alphazero news

Post by Laskos » Tue Dec 11, 2018 3:29 pm

noobpwnftw wrote:
Tue Dec 11, 2018 2:30 pm
I find it interesting that the purpose of such journals are for people to have a place to share their discoveries among others, when they have no ability to reach the audience on their own. Why would anyone want to 'bar' their own papers from getting published anyway? If you don't want to get it published, don't even bother writing a paper on it.

Barring is what the journals supposed to do, if
many, many very solid researchers never in their life get to have a "Nature" or a "Science" paper.
then the problem is not with the researchers.

I do not see any necessity to have my emotions attached to such titles, but maybe that's not for everyone.
I don't understand a word of what you wrote. Top journals ("Nature" and "Science" are the top journals in sciences) want to be relevant, and their content to have high significance for current science and its development.
The simple fact that ML, especially reinforcement learning using deep networks proved above-human-capabilities first in Go and IIRC only later, with more involved techniques, in Pac-Man, is very noteworthy even for me, a complete amateur in this field. We are closer to understanding what humans are up to in daily life, Pac-Man being an ultra-simplified caricature of the daily life. Deep Mind paper on Go and this more generalized AlphaZero paper ARE significant, absolutely the level to be published in "Nature" and "Science". That Google/DeepMind have it easy to get such feats, that it's a PR stunt, marketing trick and so on, is not the business of these journals, as long as corruption is not involved. These journals, if they want to preserve their stature and authority, have to inform from specialists to amateurs in this field like me how the significant progress in this important research domain is going.

matthewlai
Posts: 793
Joined: Sun Aug 03, 2014 2:48 am
Location: London, UK
Contact:

Re: Alphazero news

Post by matthewlai » Tue Dec 11, 2018 4:12 pm

Gian-Carlo Pascutto wrote:
Tue Dec 11, 2018 3:12 pm
Matthew, I have one last question regarding the implementation details that aren't explicit in the paper.

Normally, an MCTS search would do "tree reuse" from move to move, carrying forward the subtree that was actually chosen. But during the training, there is noise added to the root. If one does tree reuse, the effect of the noise is lessened. Leela Chess Zero decided to disable tree reuse because of that. Leela Zero has kept it enabled.

Could you clarify how it was done in DeepMind's version?
It is enabled at all times unless we are testing something specific that will be affected by it. We introduce diversity in two ways - dirichlet noise and visit count sampling.

Dirichlet noise is used to modify prior at the root node before each search (if it's enabled). So if a subtree is reused, when the next search starts it will be added to the node that is now the root node.

Visit count sampling isn't really affected.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.

User avatar
hgm
Posts: 23714
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Alphazero news

Post by hgm » Tue Dec 11, 2018 4:13 pm

jp wrote:
Tue Dec 11, 2018 1:44 pm
That is not true. It's definitely possible to get the paper accepted without reviewers all being happy.


But the publication policies of Nature & Science probably shouldn't be the main topic here.
Oh yeah? How many Science or Nature papers do you have to your name that did not have the approval of the referees?
Milos wrote:
Tue Dec 11, 2018 1:39 pm
And regarding anonymous peer reviewing, it is never double-blind even when journal policy says so.
That you even mention 'double blind' here shows that you have no clue whatsoever how the peer-review system works, and are just shooting off your mouth as usual. Of course it is not double blind. I have refereed hundreds of papers for Journals like Science, Nature, Pys. Rev. Lett., Phys. Rev. A, J. Phys. B etc, and I always got to see the full title and author list as a referee. 'Anonymous referee' just means that the authors who submitted the paper don't know the referee. So that it would be impossible to affect their decision by threats or bribes, without actually bribing everyone in the field.

jp
Posts: 815
Joined: Mon Apr 23, 2018 5:54 am

Re: Alphazero news

Post by jp » Tue Dec 11, 2018 4:29 pm

hgm wrote:
Tue Dec 11, 2018 4:13 pm
jp wrote:
Tue Dec 11, 2018 1:44 pm
That is not true. It's definitely possible to get the paper accepted without reviewers all being happy.

But the publication policies of Nature & Science probably shouldn't be the main topic here.
Oh yeah?
Yes, yeah. The reviewers can be in disagreement. It happens. Depends how much the unhappy reviewer wants to fight.

This discussion of Nature & Science is sorta off topic, but if everyone wants to talk about it that's ok.

Milos
Posts: 3387
Joined: Wed Nov 25, 2009 12:47 am

Re: Alphazero news

Post by Milos » Tue Dec 11, 2018 4:31 pm

hgm wrote:
Tue Dec 11, 2018 4:13 pm
jp wrote:
Tue Dec 11, 2018 1:44 pm
That is not true. It's definitely possible to get the paper accepted without reviewers all being happy.


But the publication policies of Nature & Science probably shouldn't be the main topic here.
Oh yeah? How many Science or Nature papers do you have to your name that did not have the approval of the referees?
Milos wrote:
Tue Dec 11, 2018 1:39 pm
And regarding anonymous peer reviewing, it is never double-blind even when journal policy says so.
That you even mention 'double blind' here shows that you have no clue whatsoever how the peer-review system works, and are just shooting off your mouth as usual. Of course it is not double blind. I have refereed hundreds of papers for Journals like Science, Nature, Pys. Rev. Lett., Phys. Rev. A, J. Phys. B etc, and I always got to see the full title and author list as a referee. 'Anonymous referee' just means that the authors who submitted the paper don't know the referee. So that it would be impossible to affect their decision by threats or bribes, without actually bribing everyone in the field.
It seems with age you are becoming dyslexic. Kind of missed the whole part of the sentence starting with "even when" (or "all being happy" in jp's comment). There is quite bunch of journals that have a double blind policy. That fact you didn't hear about it reflects more on your lack of actual experience outside of your small self-centric world and the need of empty bragging than anything else.

User avatar
hgm
Posts: 23714
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Alphazero news

Post by hgm » Tue Dec 11, 2018 4:52 pm

jp wrote:
Tue Dec 11, 2018 4:29 pm
Yes, yeah. The reviewers can be in disagreement. It happens. Depends how much the unhappy reviewer wants to fight.
That did not answer my question. How many Science papers did you publish that way?

clumma
Posts: 177
Joined: Fri Oct 10, 2014 8:05 pm
Location: Berkeley, CA

Re: Alphazero news

Post by clumma » Tue Dec 11, 2018 5:39 pm

Laskos wrote:
Tue Dec 11, 2018 12:42 pm
Yes, but didn't they take SF8 + book itself, and not the full BrainFish with its UCI option? That would be a good match: A0 versus full BrainFish of early 2018 with varied openings UCI option, but they seem to not have done that.
This is an important question. Matthew, can you confirm whether Brainfish was used, or merely Stockfish with the Cerebellum polyglot book? It's my understanding that the Brainfish binary will use the stored evaluations in search, whereas SF + polyglot will not.

-Carl

clumma
Posts: 177
Joined: Fri Oct 10, 2014 8:05 pm
Location: Berkeley, CA

Re: Alphazero news

Post by clumma » Tue Dec 11, 2018 5:40 pm

matthewlai wrote:
Tue Dec 11, 2018 12:54 pm
We used the BrainFish player for the book moves, and switch to SF8 (for consistency with other results) once out of book. We did that to make sure we were using the opening book correctly, and we weren't aware of the BF UCI option for diversity, so we only tried enforcing diversity from the AZ side.
Oh no!

-Carl

shrapnel
Posts: 1198
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: Alphazero news

Post by shrapnel » Tue Dec 11, 2018 5:44 pm

matthewlai wrote:
Tue Dec 11, 2018 3:20 am
Cannot talk about anything unannounced unfortunately!
Your simple Statement is open to different interpretations.
But fair enough.
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

Post Reply