Titan V and LCZero

[Werewolf]

It has been pointed out a few times on this forum that the strength of a GPU with LCZero is its memory, specifically bandwidth.

Most Nvidia cards use GDDR memory. The latest (actually now quite old) gen 10 cards use either GDDR5 or GDDR5X.

But the Titan V uses HBM2. It also has a better architecture (Volta as opposed to Pascal) and more cores.

I was wondering, for LCZero, how much faster would a Titan V be compared to a 1080 Ti?

Before anyone rushes out to buy one, beware there is a bug:

https://www.extremetech.com/extreme/266 ... imulations



As far as I know Nvidia haven't commented on this, fixed it, or even confirmed if it is a software or hardware issue.

[smatovic]

I am not into the code of LC0,
but with MCTS it should be possible to utilize multiple GPUs during search,
so, in future, a 2way or 4way setup could outperform a single Titan V again....

the Titan V has so called TPUs on chip, not sure if LC0 can benefit from these,
so it could be a waste of resources.
nevertheless, the Titan V is an beast in compute and memory performance.

--
Srdja

[Dann Corbit]

I am not into the code of LC0,
but with MCTS it should be possible to utilize multiple GPUs during search,
so, in future, a 2way or 4way setup could outperform a single Titan V again....

the Titan V has so called TPUs on chip, not sure if LC0 can benefit from these,
so it could be a waste of resources.
nevertheless, the Titan V is an beast in compute and memory performance.

--
Srdja

I guess it would be really powerful (Titan V) for LCZero.
The TPU operations run on tiny data (8 bit) and each halving doubles throughput (double precision, single precision, 16 bit precision, 8 bit).
The learning stuff is a natural fit to TPUs.
No reason why LCZero should be different than Alpha Zero.

[smatovic]

I guess it would be really powerful (Titan V) for LCZero.
The TPU operations run on tiny data (8 bit) and each halving doubles throughput (double precision, single precision, 16 bit precision, 8 bit).
The learning stuff is a natural fit to TPUs.
No reason why LCZero should be different than Alpha Zero.

My question is, are the TPUs currently used by Leela Zero or not.

I have no clue, but i can imagine this would need some special treatment in Leelas code.

Maybe Gary can elaborate on this?

--
Srdja

[AdminX]

Leela Chess Zero client on Google Colab's NVIDIA Tesla K80 GPU.



https://github.com/glinscott/leela-ches ... aboratory)

Didn't feel like upgrading my GPU just yet so this will have to do.

[AdminX]

Leela Chess Zero client on Google Colab's NVIDIA Tesla K80 GPU.



https://github.com/glinscott/leela-ches ... aboratory)

Didn't feel like upgrading my GPU just yet so this will have to do.

2018/04/18 17:16:53 Completed 7 games in 6m3.384693217s time

[Leto]

Leela Chess Zero client on Google Colab's NVIDIA Tesla K80 GPU.



https://github.com/glinscott/leela-ches ... aboratory)

Didn't feel like upgrading my GPU just yet so this will have to do.

2018/04/18 17:16:53 Completed 7 games in 6m3.384693217s time

And it's easy to set up following the directions here:
https://github.com/glinscott/leela-ches ... aboratory)

If you already have a Google account you could be up and running in less than 15 minutes. The session stops itself every 12 hours you just need to restart the session. If you restart it quickly after it stops you can probably get close to 1,500 games a day.

[Rubinus]

Cca 6 years old project Neoxium - http://www.neoxium.com/en/current_en.html
Unfortunatelly, author is dead. I test from Hiarcs 13.2 HTC, OS Win-mobile 5, options 1700-1750.

[AdminX]

Cca 6 years old project Neoxium - http://www.neoxium.com/en/current_en.html
Unfortunatelly, author is dead. I test from Hiarcs 13.2 HTC, OS Win-mobile 5, options 1700-1750.

Wow, that is pretty cool. Interesting project!

[Leto]

Cca 6 years old project Neoxium - http://www.neoxium.com/en/current_en.html
Unfortunatelly, author is dead. I test from Hiarcs 13.2 HTC, OS Win-mobile 5, options 1700-1750.

He died young, he would have been 47 this year.