Stockfish with new NNUE architecture and bigger net released

[Cornfed]

Will they release it as SF14 I wonder.

Do not think so. The Elo gain is not that big... The "normal" LTC test in the SF-framework (with 60''+600ms thinkingtime) gave +5.85 Elo.

But, as written on abrok.eu, this new architecture and bigger netsize is a great base for further improvements. Thats the good news. Because, since the release of Stockfish 13, 3 months ago, the progress of SF was very small.

So...how is it a great "base for further improvements"?

Because they can tweek new network to gain another 5-10 elo..
They had been tweeking the same old network for 4-5 months and keep gaining small elo,? 5-10?.

Well, if it's now all only about 'tweaking networks'...perhaps it is time to just 'stick a fork' in Stockfish.
I mean, has reliance on NNUE basically brought an end to everything about engine advancement but...speedups?

[ernst]

Is anybody seeing a difference in processor behavior with this new net? For some reason when I go from 16 to 32 threads the processor (5950x) now won't do much boosting at all. At 16 threads it's at 4.1ghz; at 32, it's at like 3.6. This was not the behavior with the version from just a couple of days ago.

This new version gives a heavier load on the CPU, so you probably hit the power limit of your setup. Hence the lower clockspeed.

[mclane]

maybe nobody is interested because it is not stronger then the normal stockfish.

[MikeB]

test results against prior version of SF:

https://tests.stockfishchess.org/tests/ ... 3412d0921d
TC: 10s+0.1s, 1 thread
ELO: 21.74 +-3.4 (95%) LOS: 100.0%
Total: 10000 W: 1559 L: 934 D: 7507 Elo +21.74
Ptnml(0-2): 38, 701, 2972, 1176, 113

https://tests.stockfishchess.org/tests/ ... 3412d0925b
TC: 60s+0.6s, 1 thread
ELO: 5.85 +-1.7 (95%) LOS: 100.0%
Total: 20000 W: 1381 L: 1044 D: 17575 Elo +5.85
Ptnml(0-2): 27, 885, 7864, 1172, 52

https://tests.stockfishchess.org/tests/ ... 7940a03806
TC: 20s+0.2s, 8 threads
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 34272 W: 1610 L: 1452 D: 31210 Elo +1.60
Ptnml(0-2): 30, 1285, 14350, 1439, 32

https://tests.stockfishchess.org/tests/ ... 7940a03c72
TC: 60s+0.6s, 8 threads
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 45544 W: 1262 L: 1214 D: 43068 Elo +0.37
Ptnml(0-2): 12, 1129, 20442, 1177, 12

[Pi4Chess]

Very interesting results. So the new net architecture seems to be better with less time /cpu power compared the older one?
It's really counter - intuitive. So How to know it is the right direction to have global elo gain in LTC in the future ?

[cdani]

Very interesting results. So the new net architecture seems to be better with less time /cpu power compared the older one?
It's really counter - intuitive. So How to know it is the right direction to have global elo gain in LTC in the future ?

If I had one tenth part of the computer power of Stockfish guys, I would do several minutes test as LTC.

[Uri Blass]

Very interesting results. So the new net architecture seems to be better with less time /cpu power compared the older one?
It's really counter - intuitive. So How to know it is the right direction to have global elo gain in LTC in the future ?

It is not counter intuitive to get less elo with longer time control.

Practically when the level is higher then it is harder to get more elo.
The problem is that the results suggest that earning less elo is not only because of more draws.
When I compare test 1 and test 2 I find that there are less wins and more losses in test 2 that is not something natural that I expect to find
with a good improvement.

test 1:Total: 10000 W: 1559 L: 934 D: 7507 Elo +21.74
test 2:Total: 20000 W: 1381 L: 1044 D: 17575 Elo +5.85

[MikeB]

Very interesting results. So the new net architecture seems to be better with less time /cpu power compared the older one?
It's really counter - intuitive. So How to know it is the right direction to have global elo gain in LTC in the future ?

Indeed, for Pi Chess level hardware, this is a significant major improvement> I have already ported it to Droidfish and I see a major improvement in play at very fast tc. Which makes it more entertaining for human play since to makes more suitable moves for human play at very fast time control. The "old" nnue-stockfish would make weaker moves at very fast tc, this now plays more like human like moves at very fast tc. As an example, without an opening book. at very fast tc, Stockfish would have played 1. Nc3 at depth 6 in the opening start position. , now it plays 1.e4 at depth 6.
Will add some play by depth, play by node count options and few other features (like opening book and move randomization) and then port it to the Harmon Chess app.

[Cornfed]

Very interesting results. So the new net architecture seems to be better with less time /cpu power compared the older one?
It's really counter - intuitive. So How to know it is the right direction to have global elo gain in LTC in the future ?

It is not counter intuitive to get less elo with longer time control.

Practically when the level is higher then it is harder to get more elo.
The problem is that the results suggest that earning less elo is not only because of more draws.
When I compare test 1 and test 2 I find that there are less wins and more losses in test 2 that is not something natural that I expect to find
with a good improvement.

test 1:Total: 10000 W: 1559 L: 934 D: 7507 Elo +21.74
test 2:Total: 20000 W: 1381 L: 1044 D: 17575 Elo +5.85

Uri - what you say makes sense. good nets are a rising tide that lifts all boats. Thus we should see more draws than in pre-NNUE days....particularly, one would think, at slower TC's.

[Cornfed]

Also, I am curious - how does Stockfish come up with the new 'percentage' output?
ex:

move 9 of a sharp opening I am looking at:

+/= (55% +15%=79%-6%)

While I much prefer the idea over the old .34 (for example)...and I understand what it is saying...I am not sure of exactly HOW those percentages are arrived at. Anyone know for sure?