Has Stockfish NNUE reached it's limit now?

[Jouni]

No progress in any list for some weeks. May be next step is a bigger net!?

[Madeleine Birchfield]

No progress in any list for some weeks. May be next step is a bigger net!?

Sergio Vieri was the only person on the Stockfish team who really knew how to train nets, and when he suddenly stopped training nobody else could really step in and replicate his efforts. So they've had to start from scratch.

[Pi4Chess]

It seems there is no elo progress against Stockfish 7 since 22 October dev release. But may be SF 12 is too far from Stockfish 7 to see any improvement ?

https://nextchessmove.com/dev-builds

[RogerC]

To mesure a progression in engines ELO, first you have to play a lot of games as the differences between versions are near 1 ELO or less.

50000 games would be the minimum to have a good evaluation of progression (or regression). NCM chess dev-builds evaluations plays only 20000 games, and vs a very weak engine (SF7) so ELO is now biased by the big 78% of wins.

The best way to be sure that SF is progressing (or not) is the 60000 games vs SF12 at LTC that is played quite often. The last run was november 29 : SFdev was +30,61 ELO vs SF12 (https://github.com/glinscott/fishtest/w ... sion-Tests).

But sure, we can see that SF NNUE has stopped his rate of progression since November 01 (just +2 ELO in 1 month).

[Pi4Chess]

@RogerC Thx for the link and infos. There is some little plateau since late october. My bet is that evolution of NNUE is needed. May be changing the size/structure of the net ? Using multiple nets (by opening, by mid/late game) ? I really don't know but this is very interesting

[carldaman]

Or maybe go back to classical eval?

[Pi4Chess]

Anyone knowing the process of stockfish development can answer this question please :

Are the dev versions published before a new net is found retro tested in fishtest with the New net ? Would it be relevant ?

[Jouni]

Finally 14.12. version shows nice gain again! But not from NNUE.

[Nay Lin Tun]

Meanwhile SPRT 1000 nodes per move test showed Leela improved +40 elo from last season.

[Jouni]

I am really sceptical about 1000 nodes test. And FGRL shows only regression .