Fastest Cloud Computing For ML Nearly As Fast As Fastest Supercomputer
Posted: Thu Jul 30, 2020 10:04 am
I'm interested to compare ML offerings on on GCE (Google Compute Engine), AWS and Azure - the 3 biggest providers (maybe IBM and a few others as well), but looking at the GCE offering, I think I must be misunderstanding something?
They have a wide range of offerings, but their fastest one seems to be their TPU offering, which is not surprising given that most arithmetic on a TPU is half-precision, making it "not very useful" for tasks other than ML.
Here's their TPU pricing - link. Although a price isn't offered for this option, the fastest option offered is 2048 cores of their TPU V3. Here's why I am confused:
Wiki appears to be saying that a single V3 TPU offers 90 teraflops (link). According to the above pricing page, you can rent 2048 of those, for a total of 90*2048 = 184,320 teraflops, which is 184 petaflops. This is close to being as fast as the world's fastest supercomputer - link.
I've done some checks to see whether this could possibly be correct, and it could well be: a direct quote from this page - link: "Cloud TPU v3 Pod 100+ petaflops"
So on GCE you can apparently rent nearly as many flops as the world's fastest supercomputer will offer you. The world's fastest supercomputer cost a billion dollars (link).
They have a wide range of offerings, but their fastest one seems to be their TPU offering, which is not surprising given that most arithmetic on a TPU is half-precision, making it "not very useful" for tasks other than ML.
Here's their TPU pricing - link. Although a price isn't offered for this option, the fastest option offered is 2048 cores of their TPU V3. Here's why I am confused:
Wiki appears to be saying that a single V3 TPU offers 90 teraflops (link). According to the above pricing page, you can rent 2048 of those, for a total of 90*2048 = 184,320 teraflops, which is 184 petaflops. This is close to being as fast as the world's fastest supercomputer - link.
I've done some checks to see whether this could possibly be correct, and it could well be: a direct quote from this page - link: "Cloud TPU v3 Pod 100+ petaflops"
So on GCE you can apparently rent nearly as many flops as the world's fastest supercomputer will offer you. The world's fastest supercomputer cost a billion dollars (link).