TalkChess.com

Rebel wrote:Google exactly knows the hardware advantage they had during the match and it isn't mentioned in the paper.

It is mentioned exactly in the paper which hardware they used (4 TPUs). The TPUs and their performance has been described in other papers, to which they refer.

I assume they picked such a low number of TPUs to make the AlphaZero and Stockfish use similar hardware during the match, chip-area or Watt-wise. They used 5,000 CPUs for training, so it seems they could easily have used many more than 4 in the match. So they must have had a reason to pick 4, rather than 1, 40 or 400. Balancing the hardware would be a logical way to decide this.

hgm wrote:

Rebel wrote:Google exactly knows the hardware advantage they had during the match and it isn't mentioned in the paper.

It is mentioned exactly in the paper which hardware they used (4 TPUs). The TPUs and their performance has been described in other papers, to which they refer.

I assume they picked such a low number of TPUs to make the AlphaZero and Stockfish use similar hardware during the match, chip-area or Watt-wise. They used 5,000 CPUs for training, so it seems they could easily have used many more than 4 in the match. So they must have had a reason to pick 4, rather than 1, 40 or 400. Balancing the hardware would be a logical way to decide this.

We have been doing the math of these 4 TPU'S before. Even Mark Watkins [BB+] estimates AZ vs SF as:

BB+ wrote:but this does tend to indicate (at least) a 100x hardware edge for AlphaZero, possibly something more like 1000x or more.

Withholding information like this is scientific and intelectual fraud. Smart commerce? Sure.

hgm wrote:

Rebel wrote:Google exactly knows the hardware advantage they had during the match and it isn't mentioned in the paper.

It is mentioned exactly in the paper which hardware they used (4 TPUs). The TPUs and their performance has been described in other papers, to which they refer.

I assume they picked such a low number of TPUs to make the AlphaZero and Stockfish use similar hardware during the match, chip-area or Watt-wise. They used 5,000 CPUs for training, so it seems they could easily have used many more than 4 in the match. So they must have had a reason to pick 4, rather than 1, 40 or 400. Balancing the hardware would be a logical way to decide this.

or ... they wanted the least amount of an advantage ... in order to not lose one game ... 100 games without a lose ... sounds a lot better than 100 games with a lost ... so the 'setup' hardware and software was based on not losing one single game ... and whatever 'configuration' would support that ... that is what they went with ...

The 4 TPUS use just 150W. That's not a lot. And if AlphaZero does not use all the 64 CPU cores, which seems likely, they might use less power in total.

pilgrimdan wrote:

hgm wrote:

Rebel wrote:Google exactly knows the hardware advantage they had during the match and it isn't mentioned in the paper.

It is mentioned exactly in the paper which hardware they used (4 TPUs). The TPUs and their performance has been described in other papers, to which they refer.

I assume they picked such a low number of TPUs to make the AlphaZero and Stockfish use similar hardware during the match, chip-area or Watt-wise. They used 5,000 CPUs for training, so it seems they could easily have used many more than 4 in the match. So they must have had a reason to pick 4, rather than 1, 40 or 400. Balancing the hardware would be a logical way to decide this.

or ... they wanted the least amount of an advantage ... in order to not lose one game ... 100 games without a lose ... sounds a lot better than 100 games with a lost ... so the 'setup' hardware and software was based on not losing one single game ... and whatever 'configuration' would support that ... that is what they went with ...

They lost games in the other openings and in Shogi, so no.

Rebel wrote:We have been doing the math of these 4 TPU'S before. Even Mark Watkins [BB+] estimates AZ vs SF as:

BB+ wrote:but this does tend to indicate (at least) a 100x hardware edge for AlphaZero, possibly something more like 1000x or more.

Faulty math... You copied that quote from open-chess.org, so you must also seen how I rebutted that claim there, and how BB+ then conceded that the 100x/1000x might indeed not be the relevant measure. Talking about intellectual fraud...

Withholding information like this is scientific and intelectual fraud. Smart commerce? Sure.

But they don't withold it, do they? If they had, how would you know it now?

hgm wrote:

Rebel wrote:We have been doing the math of these 4 TPU'S before. Even Mark Watkins [BB+] estimates AZ vs SF as:

BB+ wrote:but this does tend to indicate (at least) a 100x hardware edge for AlphaZero, possibly something more like 1000x or more.

Faulty math... You copied that quote from open-chess.org, so you must also seen how I rebutted that claim there, and how BB+ then conceded that the 100x/1000x might indeed not be the relevant measure. Talking about intellectual fraud...

Indeed, I did not see his self-correction, my bad.

Let's go back then to our previous talks, one TPU delivers 180 TFLOPS, 4 makes 720 TFLOPS.

How much TIPS does a 64 CPU (no hyperthreading) 4 GHz deliver?

Rebel wrote:

hgm wrote:

Rebel wrote:We have been doing the math of these 4 TPU'S before. Even Mark Watkins [BB+] estimates AZ vs SF as:

BB+ wrote:but this does tend to indicate (at least) a 100x hardware edge for AlphaZero, possibly something more like 1000x or more.

Faulty math... You copied that quote from open-chess.org, so you must also seen how I rebutted that claim there, and how BB+ then conceded that the 100x/1000x might indeed not be the relevant measure. Talking about intellectual fraud...

Indeed, I did not see his self-correction, my bad.

Let's go back then to our previous talks, one TPU delivers 180 TFLOPS, 4 makes 720 TFLOPS.

How much TIPS does a 64 CPU (no hyperthreading) 4 GHz deliver?

http://www.zdnet.com/article/gpu-killer ... really-is/

An individual 2nd generation TPU chip delivers 45 teraflops. Each tensor processing unit, or TPU, consists of 4 TPU chips mounted on the same circuit board, for a total of 180 petaflops per TPU.

Adam Hair wrote:

Rebel wrote:

hgm wrote:

Rebel wrote:We have been doing the math of these 4 TPU'S before. Even Mark Watkins [BB+] estimates AZ vs SF as:

BB+ wrote:but this does tend to indicate (at least) a 100x hardware edge for AlphaZero, possibly something more like 1000x or more.

Faulty math... You copied that quote from open-chess.org, so you must also seen how I rebutted that claim there, and how BB+ then conceded that the 100x/1000x might indeed not be the relevant measure. Talking about intellectual fraud...

Indeed, I did not see his self-correction, my bad.

Let's go back then to our previous talks, one TPU delivers 180 TFLOPS, 4 makes 720 TFLOPS.

How much TIPS does a 64 CPU (no hyperthreading) 4 GHz deliver?

http://www.zdnet.com/article/gpu-killer ... really-is/

An individual 2nd generation TPU chip delivers 45 teraflops. Each tensor processing unit, or TPU, consists of 4 TPU chips mounted on the same circuit board, for a total of 180 petaflops per TPU.

I got my information from -

Each of these new TPU devices delivers up to 180 teraflops of floating-point performance.

https://www.blog.google/topics/google-c ... -learning/

Some else, RdM already said something about it, but I forgot.

AZ - 80,000 positions/second
SF - 70,000,000 positions/second

If these TPU's are so fast, then how do we need to interpret their way of positions/second ?

Or do they mean MCTS play-outs per second.