Alphazero news

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
matthewlai
Posts: 789
Joined: Sun Aug 03, 2014 2:48 am
Location: London, UK
Contact:

Re: Alphazero news

Post by matthewlai » Tue Dec 11, 2018 11:56 am

jorose wrote:
Tue Dec 11, 2018 9:56 am
matthewlai wrote:
Tue Dec 11, 2018 3:20 am

Hi Matthew,

Nice to see you on the forum again!
I was a big fan of your probability based search. In fact, shortly after you published your thesis I read it and my only confusion was why you did it in chess and not Go, so I wrote my B.Sc. thesis doing exactly that. Naturally, I was wondering whether you got to try your probability based search further in AlphaZero or any other project?

With best regards,
Jonathan Rosenthal
Thanks! I am glad you found the idea interesting. It's something I wish I had time to explore further, because I really felt it makes more sense than depth-limited alpha-beta.

It wasn't used in AZ because AZ is based on MCTS, and my goal of the probability-based search was to basically "make alpha-beta more MCTS". MCTS already naturally shapes the tree the way a probability-based search would. It also makes incorporating prior much simpler. The only reason I didn't actually use MCTS was because, like everyone at the time, I didn't think it would be possible to use MCTS on a game that has so much sharp tactics. In fact, we weren't sure about that either until we actually tried AlphaZero on chess.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.

matthewlai
Posts: 789
Joined: Sun Aug 03, 2014 2:48 am
Location: London, UK
Contact:

Re: Alphazero news

Post by matthewlai » Tue Dec 11, 2018 12:13 pm

Thomas A. Anderson wrote:
Tue Dec 11, 2018 10:24 am
Matthew, thank you very much for participating in this thread! Everyone here should be grateful, that we are now getting the answers that were missing so much after releasing of the preliminary paper last December. Let's keep this discussion a technical one. With regards to this, I have a question, that came up while I was greedily digging through the papers. One number seems to be very odd to me: In the match against the SF with opening book (Brainfish), when AZ play black, it gets ~2/4 % wins against SF8/SF9, but ~18% wins when playing against the supposedly stronger Brainfish ?! Beside a docu bug, I can imagine two possible reasons: Either the number of games in this match has been very low or AZ discovered some poor opening book lines. The latter would be very interesting, especially how many and which lines are affected. While getting the game notation from these matches would be perfect, knowing how many games have been played in these tests would also be helpful.
Thanks again for your contribution, Matthew!
It's hard to say and I don't want to speculate much beyond what we have data to support, but my guess (and I can very well be wrong) is that there's much less diversity when SF uses the BF opening book. We already didn't have a lot of diversity from start position, but start position at least has several variations that are roughly equal and both AZ and SF have enough non-determinism (through multi-threaded scheduling mostly) that we still got reasonably diverse games. With the BF games we took out most of SF's non-determinism, and it's possible that SF just ends up playing a line that's not very good often, or something like that. In fact, we found that as we explained in the paper, if we force AZ to play some sub-optimal moves (in its opinion) to enforce diversity, we win even more games! I realise there's a lot of hand-waving here, but there are just too many possibilities.

I don't remember the number of games played, but it was more than high enough that the result is statistically significant.

We decided to release games from the start position and TCEC positions as the main result of the chess part of the paper because start position is more scientifically pure (they were actually playing the game of chess, not a game that's just like chess except you are forced to start from these positions), and from TCEC openings we show that we can play well even in openings that it wouldn't normally play.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.

duncan
Posts: 9407
Joined: Mon Jul 07, 2008 8:50 pm

Re: Alphazero news

Post by duncan » Tue Dec 11, 2018 12:18 pm

Laskos wrote:
Tue Dec 11, 2018 11:48 am

You have no idea how is to publish in "Nature" and "Science" (Deep Mind publishing in both on this same theme). HGM has a "Nature" paper. I don't have, although I do have (few) papers in the best journals of physics (Phys. Rev. Letters, for example).
Do you have a link ?

User avatar
Laskos
Posts: 8977
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Alphazero news

Post by Laskos » Tue Dec 11, 2018 12:23 pm

matthewlai wrote:
Tue Dec 11, 2018 12:13 pm
Thomas A. Anderson wrote:
Tue Dec 11, 2018 10:24 am
Matthew, thank you very much for participating in this thread! Everyone here should be grateful, that we are now getting the answers that were missing so much after releasing of the preliminary paper last December. Let's keep this discussion a technical one. With regards to this, I have a question, that came up while I was greedily digging through the papers. One number seems to be very odd to me: In the match against the SF with opening book (Brainfish), when AZ play black, it gets ~2/4 % wins against SF8/SF9, but ~18% wins when playing against the supposedly stronger Brainfish ?! Beside a docu bug, I can imagine two possible reasons: Either the number of games in this match has been very low or AZ discovered some poor opening book lines. The latter would be very interesting, especially how many and which lines are affected. While getting the game notation from these matches would be perfect, knowing how many games have been played in these tests would also be helpful.
Thanks again for your contribution, Matthew!
It's hard to say and I don't want to speculate much beyond what we have data to support, but my guess (and I can very well be wrong) is that there's much less diversity when SF uses the BF opening book. We already didn't have a lot of diversity from start position, but start position at least has several variations that are roughly equal and both AZ and SF have enough non-determinism (through multi-threaded scheduling mostly) that we still got reasonably diverse games. With the BF games we took out most of SF's non-determinism, and it's possible that SF just ends up playing a line that's not very good often, or something like that. In fact, we found that as we explained in the paper, if we force AZ to play some sub-optimal moves (in its opinion) to enforce diversity, we win even more games! I realise there's a lot of hand-waving here, but there are just too many possibilities.

I don't remember the number of games played, but it was more than high enough that the result is statistically significant.

We decided to release games from the start position and TCEC positions as the main result of the chess part of the paper because start position is more scientifically pure (they were actually playing the game of chess, not a game that's just like chess except you are forced to start from these positions), and from TCEC openings we show that we can play well even in openings that it wouldn't normally play.
I don't remember what Cerebellum book lines are chosen by what UI, but using for SF8 a regular polyglot opening book like the small, but good BookX.bin and with Lc0 without any book in Cutechess-Cli, I did get very varied openings. And a decrease of Lc0 strength of at least 50 Elo points compared to just playing from Initial Board position, but at short time controls. I think in Cutechess-Cli one has a random seed for a .bin book, but I don't remember well now.

pohl4711
Posts: 957
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Contact:

Re: Alphazero news

Post by pohl4711 » Tue Dec 11, 2018 12:35 pm

matthewlai wrote:
Tue Dec 11, 2018 12:13 pm
It's hard to say and I don't want to speculate much beyond what we have data to support, but my guess (and I can very well be wrong) is that there's much less diversity when SF uses the BF opening book.
That is correct. When playing with the default settings (one of the default options is BestBookMove=true), BrainFish plays always the best move out of the Cerebellum-Library, which means: BrainFish always plays the same move in the same position.

Regards - Stefan (SPCC (main test-website of Brainfish)) https://www.sp-cc.de/

User avatar
Laskos
Posts: 8977
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Alphazero news

Post by Laskos » Tue Dec 11, 2018 12:42 pm

pohl4711 wrote:
Tue Dec 11, 2018 12:35 pm
matthewlai wrote:
Tue Dec 11, 2018 12:13 pm
It's hard to say and I don't want to speculate much beyond what we have data to support, but my guess (and I can very well be wrong) is that there's much less diversity when SF uses the BF opening book.
That is correct. When playing with the default settings (one of the default options is BestBookMove=true), BrainFish plays always the best move out of the Cerebellum-Library, which means: BrainFish always plays the same move in the same position.

Regards - Stefan (SPCC (main test-website of Brainfish)) https://www.sp-cc.de/
Yes, but didn't they take SF8 + book itself, and not the full BrainFish with its UCI option? That would be a good match: A0 versus full BrainFish of early 2018 with varied openings UCI option, but they seem to not have done that.

matthewlai
Posts: 789
Joined: Sun Aug 03, 2014 2:48 am
Location: London, UK
Contact:

Re: Alphazero news

Post by matthewlai » Tue Dec 11, 2018 12:51 pm

Laskos wrote:
Tue Dec 11, 2018 12:23 pm
matthewlai wrote:
Tue Dec 11, 2018 12:13 pm
Thomas A. Anderson wrote:
Tue Dec 11, 2018 10:24 am
Matthew, thank you very much for participating in this thread! Everyone here should be grateful, that we are now getting the answers that were missing so much after releasing of the preliminary paper last December. Let's keep this discussion a technical one. With regards to this, I have a question, that came up while I was greedily digging through the papers. One number seems to be very odd to me: In the match against the SF with opening book (Brainfish), when AZ play black, it gets ~2/4 % wins against SF8/SF9, but ~18% wins when playing against the supposedly stronger Brainfish ?! Beside a docu bug, I can imagine two possible reasons: Either the number of games in this match has been very low or AZ discovered some poor opening book lines. The latter would be very interesting, especially how many and which lines are affected. While getting the game notation from these matches would be perfect, knowing how many games have been played in these tests would also be helpful.
Thanks again for your contribution, Matthew!
It's hard to say and I don't want to speculate much beyond what we have data to support, but my guess (and I can very well be wrong) is that there's much less diversity when SF uses the BF opening book. We already didn't have a lot of diversity from start position, but start position at least has several variations that are roughly equal and both AZ and SF have enough non-determinism (through multi-threaded scheduling mostly) that we still got reasonably diverse games. With the BF games we took out most of SF's non-determinism, and it's possible that SF just ends up playing a line that's not very good often, or something like that. In fact, we found that as we explained in the paper, if we force AZ to play some sub-optimal moves (in its opinion) to enforce diversity, we win even more games! I realise there's a lot of hand-waving here, but there are just too many possibilities.

I don't remember the number of games played, but it was more than high enough that the result is statistically significant.

We decided to release games from the start position and TCEC positions as the main result of the chess part of the paper because start position is more scientifically pure (they were actually playing the game of chess, not a game that's just like chess except you are forced to start from these positions), and from TCEC openings we show that we can play well even in openings that it wouldn't normally play.
I don't remember what Cerebellum book lines are chosen by what UI, but using for SF8 a regular polyglot opening book like the small, but good BookX.bin and with Lc0 without any book in Cutechess-Cli, I did get very varied openings. And a decrease of Lc0 strength of at least 50 Elo points compared to just playing from Initial Board position, but at short time controls. I think in Cutechess-Cli one has a random seed for a .bin book, but I don't remember well now.
It's true, there are many books that we could have chosen from. The problem is there wasn't one that we thought everyone will be happy with, and we do have a lot of "critics" (in quotes because they aren't the useful kind of critics) who will probably go into any book we choose, find a line that they think is bad using whatever their preferred analysis method is on the day, and say we deliberately chose that book because it's bad.

In the end we settled on Brainfish book just because it's actually generated by Stockfish, so it's about as "pure" as it gets.

I do believe that opening book will help quite a bit at short time control, since "intuitive play" is what AZ/Lc0 are good at. Though at the time controls we played with, SF does make some very reasonable opening moves (at least when it would still be in most books), and I'm not sure if it would have made a lot of difference.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.

matthewlai
Posts: 789
Joined: Sun Aug 03, 2014 2:48 am
Location: London, UK
Contact:

Re: Alphazero news

Post by matthewlai » Tue Dec 11, 2018 12:52 pm

pohl4711 wrote:
Tue Dec 11, 2018 12:35 pm
matthewlai wrote:
Tue Dec 11, 2018 12:13 pm
It's hard to say and I don't want to speculate much beyond what we have data to support, but my guess (and I can very well be wrong) is that there's much less diversity when SF uses the BF opening book.
That is correct. When playing with the default settings (one of the default options is BestBookMove=true), BrainFish plays always the best move out of the Cerebellum-Library, which means: BrainFish always plays the same move in the same position.

Regards - Stefan (SPCC (main test-website of Brainfish)) https://www.sp-cc.de/
Thanks! That is good to know.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.

matthewlai
Posts: 789
Joined: Sun Aug 03, 2014 2:48 am
Location: London, UK
Contact:

Re: Alphazero news

Post by matthewlai » Tue Dec 11, 2018 12:54 pm

Laskos wrote:
Tue Dec 11, 2018 12:42 pm
pohl4711 wrote:
Tue Dec 11, 2018 12:35 pm
matthewlai wrote:
Tue Dec 11, 2018 12:13 pm
It's hard to say and I don't want to speculate much beyond what we have data to support, but my guess (and I can very well be wrong) is that there's much less diversity when SF uses the BF opening book.
That is correct. When playing with the default settings (one of the default options is BestBookMove=true), BrainFish plays always the best move out of the Cerebellum-Library, which means: BrainFish always plays the same move in the same position.

Regards - Stefan (SPCC (main test-website of Brainfish)) https://www.sp-cc.de/
Yes, but didn't they take SF8 + book itself, and not the full BrainFish with its UCI option? That would be a good match: A0 versus full BrainFish of early 2018 with varied openings UCI option, but they seem to not have done that.
We used the BrainFish player for the book moves, and switch to SF8 (for consistency with other results) once out of book. We did that to make sure we were using the opening book correctly, and we weren't aware of the BF UCI option for diversity, so we only tried enforcing diversity from the AZ side.
Disclosure: I work for DeepMind on the AlphaZero project, but everything I say here is personal opinion and does not reflect the views of DeepMind / Alphabet.

Milos
Posts: 3316
Joined: Wed Nov 25, 2009 12:47 am

Re: Alphazero news

Post by Milos » Tue Dec 11, 2018 1:06 pm

Laskos wrote:
Tue Dec 11, 2018 11:48 am
noobpwnftw wrote:
Tue Dec 11, 2018 11:26 am
Is it because people who wrote those 'fully decent' papers are not Google-branded?

But in my view a 'decent paper' is still a decent paper even if it is published on a morning newspaper.
You have no idea how is to publish in "Nature" and "Science" (Deep Mind publishing in both on this same theme). HGM has a "Nature" paper. I don't have, although I do have (few) papers in the best journals of physics (Phys. Rev. Letters, for example). I don't know about morning newspapers, not exactly my field of expertise. Many, many very solid researchers never in their life get to have a "Nature" or a "Science" paper.
So you say, author's institution or group doesn't play any role in paper acceptance chances? Bunch of my colleagues have Nature publications and none of them had any when they were working at university or for smaller companies.

Post Reply