Alphazero news

[Laskos]

Ok, I browsed quickly the paper and the additional material.

All important results are against SF8, aside the one against SF9, but from 1 standard opening position. The Cerebellum book, "Human Openings" and TCEC openings are used by SF8. The results against SF8 + Cerebellum and against SF8 in TCEC openings suggest that this A0 in these conditions is somewhat weaker than SF10. Very good result is achieved against SF9, but the result is unreliable, being from 1 standard opening position. Very good result against SF8 from "Human Openings", but "Human Openings" in the older preprint were favorable to A0, so I have no very high confidence in this result.

All in all, in their conditions, A0 seems a bit weaker than SF10 from a normal, unbiased set of openings.
Hardware conditions are fair GPU-CPU wise. I have a higher by a factor of ~2.5 "Effective Leela Ratio" than they do, and Lc0 is still a bit weaker than SF10 at LTC. A0 is still significantly better than Leela with the best nets, but in their conditions, seems a bit weaker than SF10 (again, using normal set of openings). I mean, that older A0 in the paper and material presented, probably by today, they improved on it.

[hgm]

a Core i7 4770K has 45 GFlops per core, a gen 3 TPU has 45 TFlops which equals 1000x faster speed for the chosen task (per core)

But AlphaZero was using gen 1 TPUs, right? IIRC these had 0 GFlops, as the could not do floating point at all.

[mwyoung]

a Core i7 4770K has 45 GFlops per core, a gen 3 TPU has 45 TFlops which equals 1000x faster speed for the chosen task (per core)

But AlphaZero was using gen 1 TPUs, right? IIRC these had 0 GFlops, as the could not do floating point at all.

I was told gen 3. But it did not say in the information posted. Here is what was posted on the site.

For the games themselves, Stockfish used 44 CPU (central processing unit) cores and AlphaZero used a single machine with four TPUs and 44 CPU cores. Stockfish had a hash size of 32GB and used syzygy endgame tablebases.

BTW: I need to buy a 2080 ti since 2 TPUs are equal one 2080 ti.

[jp]

1) Match vs. old version of SF.

2) AZ vs. SF + 17,= 75,- 8. It's score with TCEC openings.

And still SF8 in (2), right?

That's plus 32 elo. If the opponent was actually SF9 (does anyone know?) that would be about what I'd expect from SF10 under roughly TCEC conditions.

It was SF8. The Jan 13 SF games were separate with no TCEC openings.

[jp]

I was told gen 3. But it did not say in the information posted. Here is what was posted on the site.

For the games themselves, Stockfish used 44 CPU (central processing unit) cores and AlphaZero used a single machine with four TPUs and 44 CPU cores. Stockfish had a hash size of 32GB and used syzygy endgame tablebases.

What do people think would happen if A0 used 4 TPUs and just 1 CPU core? How would that change its strength?
I see no "fairness" reason to give it 44 cores as well as the 4 TPUs.

[Javier Ros]

Ok, I browsed quickly the paper and the additional material.

All important results are against SF8, aside the one against SF9, but from 1 standard opening position. The Cerebellum book, "Human Openings" and TCEC openings are used by SF8. The results against SF8 + Cerebellum and against SF8 in TCEC openings suggest that this A0 in these conditions is somewhat weaker than SF10. Very good result is achieved against SF9, but the result is unreliable, being from 1 standard opening position. Very good result against SF8 from "Human Openings", but "Human Openings" in the older preprint were favorable to A0, so I have no very high confidence in this result.

All in all, in their conditions, A0 seems a bit weaker than SF10 from a normal, unbiased set of openings.
Hardware conditions are fair GPU-CPU wise. I have a higher by a factor of ~2.5 "Effective Leela Ratio" than they do, and Lc0 is still a bit weaker than SF10 at LTC. A0 is still significantly better than Leela with the best nets, but in their conditions, seems a bit weaker than SF10 (again, using normal set of openings). I mean, that older A0 in the paper and material presented, probably by today, they improved on it.

Once the article was sent for publication, it was submitted to the review of two or three referees, whose identity is kept secret, which will send their recommendations or demands to the authors for the article to be published. I suppose that among these recommendations they will have included the topic of the book of openings for Stockfish and the variety in the time control.

Since the article is based on the first version of A0, all the games played should have been done with this version.

If during this time they have advanced and have achieved better results with A0 version 2, this will probably lead to the publication of another article.

[sovaz1997]

All public games against Stockfish 8?



See it table (from https://deepmind.com/documents/260/alph ... eprint.pdf).
All puclic games here: https://deepmind.com/research/alphago/a ... resources/.

[hgm]

I was told gen 3. But it did not say in the information posted. Here is what was posted on the site.

Well, I don't know who 'told you' that, but the newly published paper states:

Each program was run on the hardware for which it was designed (23): Stockfish and Elmo used 44 central processing unit (CPU) cores (as in the TCEC world championship), whereas AlphaZero and AlphaGo Zero used a single machine with four first-generation TPUs and 44 CPU cores (24).

So it seems in fact that in terms of GFlops there was no advantage at all...

[jp]

If during this time they have advanced and have achieved better results with A0 version 2

It's not clear whether they are actively developing it. Maybe not?

[Javier Ros]

If during this time they have advanced and have achieved better results with A0 version 2

It's not clear whether they are actively developing it. Maybe not?

I don't know, but for the amount of games and news in the revised version of the article, I would say that they have been quite busy until the final version has been accepted.