CCC has serious hardware update!

[mwyoung]

mwyoung will come back with the "use a serious time control" argument. And I will search for some popcorn...

Indeed. But, I warn them of doing that, for this has greater implications, which they do not yet understand ....

And yes testing conditions matter in testing. That is why testing sites use different rating list for different time controls.

[Pi4Chess]

why talking about x cores vs n cores ? Just test x nodes vs n nodes with and without NNUE to see the difference ?

[Nay Lin Tun]

why talking about x cores vs n cores ? Just test x nodes vs n nodes with and without NNUE to see the difference ?

That testing condition is only useful on fixed number of cores only.

The quality of search coming from less number of cores is much much better than same nodes from multi-cores. ( e.g 1 core vs 8 cores)

[Jouni]

BTW why has Dragon much higher NPS than Stockfish now? Dragon 250 Mnps and SF 150 Mnps.

[mwyoung]

You can trash me, CCRL, and others. Because you do not like the facts. But the facts are not going away with how NNUE scales.

Lets look not at my data, but CCRL blitz data. And lets see how NNUE behaves to a typical A/B engine like Ethereal.

1. Ethereal 12.75 64-bit 8CPU 3537 +20 −20 39.9% +61.4 57.2% 683
2. Ethereal 12.75 64-bit 3377 +22 −22 64.8% −86.2 56.5% 600

CCRL rating difference for 1 to 8 cores. 160 Elo

1. Stockfish 12 64-bit 8CPU 3692 +15 −14 71.5% −147.4 54.5% 1532
2. Stockfish 12 64-bit 3639 +15 −14 79.0% −226.6 36.1% 2054

CCRL rating difference from 1 to 8 cores. 53 Elo

"So you start your initial core doubling at +70 per doubling"

There were many threads addressing the issues with CCRLs methods and how it compresses elo and distorts values due to the opponent pool. If you have data that contradicts the typical scaling, then you should post it. You need thousands of games in your sample. Share the conditions, the opening book, and make sure to understand cores vs threads.

I posted data that cleared ccrl. Under ccrl flawed testing thread. And as predicted you are trashing others for their data. I have more data that can be post from other testing sites. Showing the same lack of scaling. I can post that so you can trash them also.

[Uri Blass]

You can trash me, CCRL, and others. Because you do not like the facts. But the facts are not going away with how NNUE scales.

Lets look not at my data, but CCRL blitz data. And lets see how NNUE behaves to a typical A/B engine like Ethereal.

1. Ethereal 12.75 64-bit 8CPU 3537 +20 −20 39.9% +61.4 57.2% 683
2. Ethereal 12.75 64-bit 3377 +22 −22 64.8% −86.2 56.5% 600

CCRL rating difference for 1 to 8 cores. 160 Elo

1. Stockfish 12 64-bit 8CPU 3692 +15 −14 71.5% −147.4 54.5% 1532
2. Stockfish 12 64-bit 3639 +15 −14 79.0% −226.6 36.1% 2054

CCRL rating difference from 1 to 8 cores. 53 Elo

"So you start your initial core doubling at +70 per doubling"

There were many threads addressing the issues with CCRLs methods and how it compresses elo and distorts values due to the opponent pool. If you have data that contradicts the typical scaling, then you should post it. You need thousands of games in your sample. Share the conditions, the opening book, and make sure to understand cores vs threads.

I posted data that cleared ccrl. Under ccrl flawed testing thread. And as predicted you are trashing others for their data. I have more data that can be post from other testing sites. Showing the same lack of scaling. I can post that so you can trash them also.

The only proof to show lack of scaling of nnue engines is if some non nnue engine can beat them at longer time control.

draws prove nothing because it is possible that you do not earn rating above some elo by doubling the time control.

[mwyoung]

You can trash me, CCRL, and others. Because you do not like the facts. But the facts are not going away with how NNUE scales.

Lets look not at my data, but CCRL blitz data. And lets see how NNUE behaves to a typical A/B engine like Ethereal.

1. Ethereal 12.75 64-bit 8CPU 3537 +20 −20 39.9% +61.4 57.2% 683
2. Ethereal 12.75 64-bit 3377 +22 −22 64.8% −86.2 56.5% 600

CCRL rating difference for 1 to 8 cores. 160 Elo

1. Stockfish 12 64-bit 8CPU 3692 +15 −14 71.5% −147.4 54.5% 1532
2. Stockfish 12 64-bit 3639 +15 −14 79.0% −226.6 36.1% 2054

CCRL rating difference from 1 to 8 cores. 53 Elo

"So you start your initial core doubling at +70 per doubling"

There were many threads addressing the issues with CCRLs methods and how it compresses elo and distorts values due to the opponent pool. If you have data that contradicts the typical scaling, then you should post it. You need thousands of games in your sample. Share the conditions, the opening book, and make sure to understand cores vs threads.

I posted data that cleared ccrl. Under ccrl flawed testing thread. And as predicted you are trashing others for their data. I have more data that can be post from other testing sites. Showing the same lack of scaling. I can post that so you can trash them also.

The only proof to show lack of scaling of nnue engines is if some non nnue engine can beat them at longer time control.

draws prove nothing because it is possible that you do not earn rating above some elo by doubling the time control.

That is clearly not correct. And that is not the only way.
We have lots of data from myself and other testers showing the lack of scaling with more cores. Showing many doublings in cpu, but minimal increase in Elo. Not even 70 Elo.

[Alayan]

The obvious question is why SF classical isn't performing much closer to SF NNUE on high core count machine, then.

[mwyoung]

The obvious question is why SF classical isn't performing much closer to SF NNUE on high core count machine, then.

I guess because it is not a classical engine. SF classical is just getting crushed, by NNUE with 2 cores vs 16 for Stockfish classical. . NNUE scales like crap, but it is so strong it does not matter.

[brianr]

SF-NNUE scaled just fine going from 1 to 2, 4, and 8 threads (on 12 core box).
I did not test beyond 8, and the big tourney rigs have way more than 8 of course...
Results posted in SF-NNUE Discord several months ago, IIRC.