LCZero GPU memory requirements?

[Geonerd]

Looking to by a 'basic' card to run the engine, a modest used GTX 1060, or just maybe a 1070 if I can find a screaming deal. (Anything wrong with a GTX980?) Is 3 GB enough, or should I pop for the 6 GB version. And do all the 3 GB 1060 models come with 128 fewer CUDA cores activated?

Thanks

[jorose]

As a disclaimer, I am not actually all too familiar with the Leela codebase.

That being said I don't think it should be an issue. First of all, keep in mind people are using neural nets with much larger images than chess boards. If you have an image which is 1024x1024 (around half of "full HD") than it is around 16'000 times the size of an 8x8.

The other point is that memory usage is dependent on batch size, which I believe is a parameter in Leela. If the memory is not enough you could reduce the batch size until it is enough. I don't really think you would need to do this with a card with that much memory though.

[jjoshua2]

3gb is plenty for using leela as long as you don't plan to do supervised learning to do experiments on your own. Only downside is they do all have a few less cuda cores. A gtx 980 is probably faster than a 1060 (~2k nps), which I think is more like a 970. But a 2070 (30k) is even faster than a 1080 ti (about 10k). And if a 2050 and 2060 come out they will be much faster than previous generations assuming they still have tensor cores like all the other 20xx cards.

[Geonerd]

3gb is plenty for using leela as long as you don't plan to do supervised learning to do experiments on your own. Only downside is they do all have a few less cuda cores. A gtx 980 is probably faster than a 1060 (~2k nps), which I think is more like a 970. But a 2070 (30k) is even faster than a 1080 ti (about 10k). And if a 2050 and 2060 come out they will be much faster than previous generations assuming they still have tensor cores like all the other 20xx cards.

Thanks! The recent Alpha-Zero releases have finally sparked an interest in Leela. I haven't been paying attention to gaming cards for several years.
Can I beg two more questions?
Can a single instance of LC0 make use of multiple GPU cards? If so, do they have to be matched?
How critical is GPU memory bandwidth for Leela? (I suspect there are a few cards out there that come with faster memory chips...)

[shrapnel]

I don't understand why you want to buy a Card which is already obsolete.
With the steady improvement of Lc0 and possible commercialization of AlphaZero, the Gamers with the fastest and best Graphic Cards (not the CPU any longer) will have the advantage.
If I was in the market for a new Graphic Card, I would buy the best I could afford...and hence put off upgrading for a longer time.

[Geonerd]

I don't understand why you want to buy a Card which is already obsolete.
With the steady improvement of Lc0 and possible commercialization of AlphaZero, the Gamers with the fastest and best Graphic Cards (not the CPU any longer) will have the advantage.
If I was in the market for a new Graphic Card, I would buy the best I could afford...and hence put off upgrading for a longer time.

Excuse me, but it's my money and I will spend it as I like. Computer chess is a minor hobby, and I have no need or desire to throw big chunks of cash at it. My main A/B computer is an 'obsolete' dual Opteron. It's got 24 cores, 48 gig(!) of ram, and it runs conventional engines as well as an 8c/16t Ryzen. It also makes a pretty darn good video and photo editing platform. Total cost was just under $300, an easily justified expense for an eccentric, cheap-ass build.

For now, I just want something to play around with, and a $100~150 GPU is entirely appropriate. And even if I really catch the NN bug, I'll likely wait to see how well Navi performs before buying a mid-high end card.

To my knowledge, there is no LC0 hardware FAQ that discusses the simple questions I've asked. (Maybe I'm overthinking and should just go with raw CUDA benchmarks?) If you don't want to answer, or don't know, fine, but FFS, please don't presume to lecture me. And BTW why aren't you bitching as Laskos for buying an OMG OBSOLETE!! 1070?

[jjoshua2]

3gb is plenty for using leela as long as you don't plan to do supervised learning to do experiments on your own. Only downside is they do all have a few less cuda cores. A gtx 980 is probably faster than a 1060 (~2k nps), which I think is more like a 970. But a 2070 (30k) is even faster than a 1080 ti (about 10k). And if a 2050 and 2060 come out they will be much faster than previous generations assuming they still have tensor cores like all the other 20xx cards.

Thanks! The recent Alpha-Zero releases have finally sparked an interest in Leela. I haven't been paying attention to gaming cards for several years.
Can I beg two more questions?
Can a single instance of LC0 make use of multiple GPU cards? If so, do they have to be matched?
How critical is GPU memory bandwidth for Leela? (I suspect there are a few cards out there that come with faster memory chips...)

GPU memory bandwidth doesn't really matter. It's mostly cuda cores * clock speed. And you can use multiple GPUs as long as they are roughly the same. like a 2080 and 2080 ti or a 1060 and 1070. Up to 4 GPUs works well now.
Now that the 2060 is out there's really no reason not to get that for Leela since its like ~10x faster than a 1070 or 1080, since it has tensor cores. Much better than getting a ton of 1060s (which someone has 6 even that scale well, but are still slower than a 2060) even if they are almost free just because of electricity costs.

[Geonerd]

GPU memory bandwidth doesn't really matter. It's mostly cuda cores * clock speed. And you can use multiple GPUs as long as they are roughly the same. like a 2080 and 2080 ti or a 1060 and 1070. Up to 4 GPUs works well now.
Now that the 2060 is out there's really no reason not to get that for Leela since its like ~10x faster than a 1070 or 1080, since it has tensor cores. Much better than getting a ton of 1060s (which someone has 6 even that scale well, but are still slower than a 2060) even if they are almost free just because of electricity costs.

Thanks. With the 20x0 cards performing so much faster running the fp16 back end, I'll be waiting for a hot sale on a 2060/70, or maybe the rumored 1660.

[jjoshua2]

NP. The 1660 won't have tensor cores it seems though. So 2060 is where the best value is.

[mwyoung]

Looking to by a 'basic' card to run the engine, a modest used GTX 1060, or just maybe a 1070 if I can find a screaming deal. (Anything wrong with a GTX980?) Is 3 GB enough, or should I pop for the 6 GB version. And do all the 3 GB 1060 models come with 128 fewer CUDA cores activated?

Thanks

HWINFO64 show I am only using 2.164 GB of memory when running Lc0. So 3 GB would be enough.