Sergio NN 40x512

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Hai
Posts: 528
Joined: Sun Aug 04, 2013 11:19 am

Sergio NN 40x512

Post by Hai » Sun Dec 29, 2019 7:37 pm

Can we get some 40x512 nets? :D :D :D
That would also improve our understanding of NN development, NN training, GPU/CPU/NPU and RAM.
It could help to detect problems and bugs too.

If Sergio could release 40x512 NNs,
(maybe with 7-men tb rescoring and also
trained with some Chess960 opening/games),
that would be simply awesome and blow away the competition.

User avatar
Laskos
Posts: 9834
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Sergio NN 40x512

Post by Laskos » Sun Dec 29, 2019 7:50 pm

Hai wrote:
Sun Dec 29, 2019 7:37 pm
Can we get some 40x512 nets? :D :D :D
That would also improve our understanding of NN development, NN training, GPU/CPU/NPU and RAM.
It could help to detect problems and bugs too.

If Sergio could release 40x512 NNs,
(maybe with 7-men tb rescoring and also
trained with some Chess960 opening/games),
that would be simply awesome and blow away the competition.
You will have to have a well scaling to 4 RTX 2080 Ti GPUs Lc0 engine and play very long time control games, for such a net to be better than some of the smaller nets. But it would be good in long analyses even on one RTX GPU.

Hai
Posts: 528
Joined: Sun Aug 04, 2013 11:19 am

Re: Sergio NN 40x512

Post by Hai » Sun Dec 29, 2019 8:01 pm

Do you mean a smaller net, which has 5x or 10x more nodes per move, would get much more points than a 40x512 net?

Image

User avatar
Laskos
Posts: 9834
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Sergio NN 40x512

Post by Laskos » Sun Dec 29, 2019 11:19 pm

Hai wrote:
Sun Dec 29, 2019 8:01 pm
Do you mean a smaller net, which has 5x or 10x more nodes per move, would get much more points than a 40x512 net?

Image

I am not sure what are these 44 points and the statistical significance of the datapoints. Smaller net is not that bad in these fixed nodes plots, if one considers that it's 2-3 times faster, right?

In LTC games, probably some 2-3x more nodes per second on a strong RTX GPU than a 40bx512 HugeNet. So, maybe close to T60 nets or 30bx384 nets. But in 3-4 years I guess GPUs will get significantly stronger, and the HugeNet and even larger might become the strongest in Chess to LTC. Also nets with different structures might change optimal sizes.

Hai
Posts: 528
Joined: Sun Aug 04, 2013 11:19 am

Re: Sergio NN 40x512

Post by Hai » Fri Jan 10, 2020 9:03 pm

We have seen lots of scaling tests.
It means that LC0 Sergio 30x384, trained with T60 games, is already stronger with only 1000 nodes per move than the opponents with 10000 nodes per move :D :D :D.

mwyoung
Posts: 1933
Joined: Wed May 12, 2010 8:00 pm

Re: Sergio NN 40x512

Post by mwyoung » Fri Jan 10, 2020 11:58 pm

Hai wrote:
Sun Dec 29, 2019 7:37 pm
Can we get some 40x512 nets? :D :D :D
That would also improve our understanding of NN development, NN training, GPU/CPU/NPU and RAM.
It could help to detect problems and bugs too.

If Sergio could release 40x512 NNs,
(maybe with 7-men tb rescoring and also
trained with some Chess960 opening/games),
that would be simply awesome and blow away the competition.
I think it would be fine. Regardless of what others say. The strongest NN at fast and slow time controls. Are the biggest and slowest NPS Neural Nets. In my testing. And that is tested in the real world against the other best engines. And tested on 1 RTX 2080ti. NPS is a very poor gauge on what is the best nets. You don't need 6 grand of graphic cards for them to top the charts.

So far 1 RTX has not yet reached it limits with the NN...
Professing themselves to be wise, they became fools,
take on me. Foes 0.

User avatar
Nordlandia
Posts: 2522
Joined: Fri Sep 25, 2015 7:38 pm
Location: Sortland, Norway

Re: Sergio NN 40x512

Post by Nordlandia » Sat Jan 11, 2020 2:27 am

From what i understand is this: it will take few months to generate one "40x512 Net" and considered outdated half-way through generation.

For many this is futile or wasted time. For others it can be exciting to test and analyse with ✔

corres
Posts: 2023
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: Sergio NN 40x512

Post by corres » Sat Jan 11, 2020 2:37 pm

Hai wrote:
Fri Jan 10, 2020 9:03 pm
We have seen lots of scaling tests.
It means that LC0 Sergio 30x384, trained with T60 games, is already stronger with only 1000 nodes per move than the opponents with 10000 nodes per move
It is pity but in fixed time/move tests (what models better the real competitions than the tests with fixed nodes)
jHorthos's J13B.2-188 (320x24 "mid" Net) is the better for more than one RTX 2080 Ti so far.
From Sergio's nets the bests are the 256x20-t40 (normal) Nets: 1400 and 1541. So for a single RTX card these are the best choices.

User avatar
Eduard
Posts: 273
Joined: Fri Oct 26, 2018 10:58 pm
Location: Germany
Full name: Eduard Nemeth
Contact:

Re: Sergio NN 40x512

Post by Eduard » Sat Jan 11, 2020 4:20 pm

I like the sergio 256x20-t40 (normal). It is a very, very good network.

shrapnel
Posts: 1261
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: Sergio NN 40x512

Post by shrapnel » Sun Jan 12, 2020 6:09 am

Eduard wrote:
Sat Jan 11, 2020 4:20 pm
I like the sergio 256x20-t40 (normal). It is a very, very good network.
....for YOUR particular PC Configuration, not everyone.
To each, his own, certainly as far as NNs are concerned.
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

Post Reply