Sergio NN 40x512

[Hai]

Can we get some 40x512 nets?
That would also improve our understanding of NN development, NN training, GPU/CPU/NPU and RAM.
It could help to detect problems and bugs too.

If Sergio could release 40x512 NNs,
(maybe with 7-men tb rescoring and also
trained with some Chess960 opening/games),
that would be simply awesome and blow away the competition.

[Laskos]

Can we get some 40x512 nets?
That would also improve our understanding of NN development, NN training, GPU/CPU/NPU and RAM.
It could help to detect problems and bugs too.

If Sergio could release 40x512 NNs,
(maybe with 7-men tb rescoring and also
trained with some Chess960 opening/games),
that would be simply awesome and blow away the competition.

You will have to have a well scaling to 4 RTX 2080 Ti GPUs Lc0 engine and play very long time control games, for such a net to be better than some of the smaller nets. But it would be good in long analyses even on one RTX GPU.

[Hai]

Do you mean a smaller net, which has 5x or 10x more nodes per move, would get much more points than a 40x512 net?

[Laskos]

Do you mean a smaller net, which has 5x or 10x more nodes per move, would get much more points than a 40x512 net?

I am not sure what are these 44 points and the statistical significance of the datapoints. Smaller net is not that bad in these fixed nodes plots, if one considers that it's 2-3 times faster, right?

In LTC games, probably some 2-3x more nodes per second on a strong RTX GPU than a 40bx512 HugeNet. So, maybe close to T60 nets or 30bx384 nets. But in 3-4 years I guess GPUs will get significantly stronger, and the HugeNet and even larger might become the strongest in Chess to LTC. Also nets with different structures might change optimal sizes.

[Hai]

We have seen lots of scaling tests.
It means that LC0 Sergio 30x384, trained with T60 games, is already stronger with only 1000 nodes per move than the opponents with 10000 nodes per move .

[mwyoung]

Can we get some 40x512 nets?
That would also improve our understanding of NN development, NN training, GPU/CPU/NPU and RAM.
It could help to detect problems and bugs too.

If Sergio could release 40x512 NNs,
(maybe with 7-men tb rescoring and also
trained with some Chess960 opening/games),
that would be simply awesome and blow away the competition.

I think it would be fine. Regardless of what others say. The strongest NN at fast and slow time controls. Are the biggest and slowest NPS Neural Nets. In my testing. And that is tested in the real world against the other best engines. And tested on 1 RTX 2080ti. NPS is a very poor gauge on what is the best nets. You don't need 6 grand of graphic cards for them to top the charts.

So far 1 RTX has not yet reached it limits with the NN...

[Nordlandia]

From what i understand is this: it will take few months to generate one "40x512 Net" and considered outdated half-way through generation.

For many this is futile or wasted time. For others it can be exciting to test and analyse with ✔

[corres]

We have seen lots of scaling tests.
It means that LC0 Sergio 30x384, trained with T60 games, is already stronger with only 1000 nodes per move than the opponents with 10000 nodes per move

It is pity but in fixed time/move tests (what models better the real competitions than the tests with fixed nodes)
jHorthos's J13B.2-188 (320x24 "mid" Net) is the better for more than one RTX 2080 Ti so far.
From Sergio's nets the bests are the 256x20-t40 (normal) Nets: 1400 and 1541. So for a single RTX card these are the best choices.

[Eduard]

I like the sergio 256x20-t40 (normal). It is a very, very good network.

[shrapnel]

I like the sergio 256x20-t40 (normal). It is a very, very good network.

....for YOUR particular PC Configuration, not everyone.
To each, his own, certainly as far as NNs are concerned.