Gian-Carlo Pascutto wrote: ↑Tue Dec 11, 2018 4:12 pm
Matthew, I have one last question regarding the implementation details that aren't explicit in the paper.
Normally, an MCTS search would do "tree reuse" from move to move, carrying forward the subtree that was actually chosen. But during the training, there is noise added to the root. If one does tree reuse, the effect of the noise is lessened. Leela Chess Zero decided to disable tree reuse because of that. Leela Zero has kept it enabled.
Could you clarify how it was done in DeepMind's version?
It is enabled at all times unless we are testing something specific that will be affected by it. We introduce diversity in two ways - dirichlet noise and visit count sampling.
Dirichlet noise is used to modify prior at the root node before each search (if it's enabled). So if a subtree is reused, when the next search starts it will be added to the node that is now the root node.
Visit count sampling isn't really affected.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.
jp wrote: ↑Tue Dec 11, 2018 2:44 pm
That is not true. It's definitely possible to get the paper accepted without reviewers all being happy.
But the publication policies of Nature & Science probably shouldn't be the main topic here.
Oh yeah? How many Science or Nature papers do you have to your name that did not have the approval of the referees?
Milos wrote: ↑Tue Dec 11, 2018 2:39 pmAnd regarding anonymous peer reviewing, it is never double-blind even when journal policy says so.
That you even mention 'double blind' here shows that you have no clue whatsoever how the peer-review system works, and are just shooting off your mouth as usual. Of course it is not double blind. I have refereed hundreds of papers for Journals like Science, Nature, Pys. Rev. Lett., Phys. Rev. A, J. Phys. B etc, and I always got to see the full title and author list as a referee. 'Anonymous referee' just means that the authors who submitted the paper don't know the referee. So that it would be impossible to affect their decision by threats or bribes, without actually bribing everyone in the field.
jp wrote: ↑Tue Dec 11, 2018 2:44 pm
That is not true. It's definitely possible to get the paper accepted without reviewers all being happy.
But the publication policies of Nature & Science probably shouldn't be the main topic here.
Oh yeah? How many Science or Nature papers do you have to your name that did not have the approval of the referees?
Milos wrote: ↑Tue Dec 11, 2018 2:39 pmAnd regarding anonymous peer reviewing, it is never double-blind even when journal policy says so.
That you even mention 'double blind' here shows that you have no clue whatsoever how the peer-review system works, and are just shooting off your mouth as usual. Of course it is not double blind. I have refereed hundreds of papers for Journals like Science, Nature, Pys. Rev. Lett., Phys. Rev. A, J. Phys. B etc, and I always got to see the full title and author list as a referee. 'Anonymous referee' just means that the authors who submitted the paper don't know the referee. So that it would be impossible to affect their decision by threats or bribes, without actually bribing everyone in the field.
It seems with age you are becoming dyslexic. Kind of missed the whole part of the sentence starting with "even when" (or "all being happy" in jp's comment). There is quite bunch of journals that have a double blind policy. That fact you didn't hear about it reflects more on your lack of actual experience outside of your small self-centric world and the need of empty bragging than anything else.
Laskos wrote: ↑Tue Dec 11, 2018 1:42 pm
Yes, but didn't they take SF8 + book itself, and not the full BrainFish with its UCI option? That would be a good match: A0 versus full BrainFish of early 2018 with varied openings UCI option, but they seem to not have done that.
This is an important question. Matthew, can you confirm whether Brainfish was used, or merely Stockfish with the Cerebellum polyglot book? It's my understanding that the Brainfish binary will use the stored evaluations in search, whereas SF + polyglot will not.
matthewlai wrote: ↑Tue Dec 11, 2018 1:54 pm
We used the BrainFish player for the book moves, and switch to SF8 (for consistency with other results) once out of book. We did that to make sure we were using the opening book correctly, and we weren't aware of the BF UCI option for diversity, so we only tried enforcing diversity from the AZ side.
Laskos wrote: ↑Tue Dec 11, 2018 1:42 pm
Yes, but didn't they take SF8 + book itself, and not the full BrainFish with its UCI option? That would be a good match: A0 versus full BrainFish of early 2018 with varied openings UCI option, but they seem to not have done that.
This is an important question. Matthew, can you confirm whether Brainfish was used, or merely Stockfish with the Cerebellum polyglot book? It's my understanding that the Brainfish binary will use the stored evaluations in search, whereas SF + polyglot will not.
-Carl
Yes, it seems that's what they did. SF8 + Cerebellum best moves. It would have been better SF8 + Cerebellum varied moves (UCI option in BrainFish), and much better full BrainFish with varied openings. In fact, this A0 versus full BrainFish (varied openings) of early 2018 would have been a very interesting match. With possibly quite different result compared to usual results of the paper at full time control. Allowing only A0 for variety with only the best moves of Cerebellum, A0 will play close to its most trained lines. And, aside from storing evaluations in its search, in early 2018, the engine of BrainFish was close to SF9, not SF8.
Laskos wrote: ↑Tue Dec 11, 2018 1:42 pm
Yes, but didn't they take SF8 + book itself, and not the full BrainFish with its UCI option? That would be a good match: A0 versus full BrainFish of early 2018 with varied openings UCI option, but they seem to not have done that.
This is an important question. Matthew, can you confirm whether Brainfish was used, or merely Stockfish with the Cerebellum polyglot book? It's my understanding that the Brainfish binary will use the stored evaluations in search, whereas SF + polyglot will not.
-Carl
Yes, it seems that's what they did. SF8 + Cerebellum best moves. It would have been better SF8 + Cerebellum varied moves (UCI option in BrainFish), and much better full BrainFish with varied openings. In fact, this A0 versus full BrainFish (varied openings) of early 2018 would have been a very interesting match. With possibly quite different result compared to usual results of the paper at full time control. Allowing only A0 for variety with only the best moves of Cerebellum, A0 will play close to its most trained lines. And, aside from storing evaluations in its search, in early 2018, the engine of BrainFish was close to SF9, not SF8.
In fact, I mainly take as reliable result for A0 from varied openings the TCEC openings match against SF8:
A0 vs SF8
+17 =75 -8
+31 Elo points
Now, at first glance one can almost surely say that SF10 would have performed better, like:
SF10 vs SF8
+22 =73 -5
+60 Elo points
But it still doesn't mean I have a very high confidence that SF10 would beat A0 in 100 games from TCEC openings in their conditions. A0 (and Lc0) is not that "sensitive" to the regular opponent, be it SF8 or SF10, when in superiority. A0 vs inferior regular engine shows a compressed Elo difference (I showed a model-plot in another thread). But in that model, the Elo compression is hardly above a factor of 2 or so. So, I would be fairly confident that A0 and SF10 are quite closely matched playing from TCEC openings, maybe with a slight advantage of SF10.
But as Matthew said, this was the first version of A0, I don't know what they have in hand by now.
