Stockfish 14 release round the corner

[Sopel]

Stockfish 14 may be released in couple of days. It is likely to have an 30-35 elo gain over SF 13.
SF14 with NNUE net trained from Lc0 training games. One gets to see best of both worlds!!

Won't that pose an issue for testing groups?
At present, we only test NNUE engines that have nets trained on their own games.

You're free to make CCRL even more irrelevant than it is today. This is not an issue relevant to the topic of Stockfish 14 release.

@pohl4711 draw rate is going up and there's nothing wrong with having releases more often

[Sopel]

I still use Stockfish 12 since v13 is no better when using a single cpu core.
I only have computers with dual core cpus and prefer just using a single core for my chess engine matches. A lot of my old engines only support 1 core anyway.
Until a new version is about +40 elo above v12 using 1 core, I will stay with v12.

this is wrong

[Madeleine Birchfield]

Won't that pose an issue for testing groups?
At present, we only test NNUE engines that have nets trained on their own games.

The fact that CCRL tested BBC 1.4, CGET tested Fire 8 NN, and every ratings list tested Nemorino 6.00, all engines that used a network trained on Stockfish games, shows otherwise.

[Graham Banks]

Won't that pose an issue for testing groups?
At present, we only test NNUE engines that have nets trained on their own games.

The fact that CCRL tested BBC 1.4, CGET tested Fire 8 NN, and every ratings list tested Nemorino 6.00, all engines that used a network trained on Stockfish games, shows otherwise.

I didn't realize that Nemorino 6.00 was trained on Stockfish games.
I've not tested BBC 1.4 myself, but it must be in the blitz list.

[Madeleine Birchfield]

https://github.com/official-stockfish/S ... /pull/3596

IMHO, this is too early. +30 Elo? Before SF 12 each new release had +50 Elo in SF-regression tests (SF 8:+76 Elo, SF 9:+58 Elo, SF 10:+54 Elo, SF11 +53 Elo):
https://github.com/glinscott/fishtest/w ... sion-Tests
Then we had the huge NNUE-Elo jump to SF 12 (SF 12: +133 Elo)
SF 13 was +35 Elo, but was only released as a reaction to FatFritz 2, which was promoted by Chessbase as "the strongest engine". So, I see no need, to release a new version until +50 Elo in regression-tests are confirmed. Mention, that the regression-tests are played with 60sec+600ms on singlecore in the SF-framework. So, if SF is playing/analyzing with more thinking-time and/or more cores, the Elo-gain is much smaller than 50 Elo. So, a +30 Elo gain in regression-tests is not enough, I think, even though, the Elo-progression is getting harder, because of the increasing draw-rates in the SF-framework.

In my tests vs. other engines, SF dev is now +34 Elo stronger than SF 13: https://www.sp-cc.de

After Fat Fritz 2 came out, the Stockfish team probably decided to release official versions more often to prevent commercial clones from being sold with misleading advertising.

[Madeleine Birchfield]

I didn't realize that Nemorino 6.00 was trained on Stockfish games.
I've not tested BBC 1.4 myself, but it must be in the blitz list.

Nemorino author said so himself:

Nemorino 6:
- Uses a net (nemorino_net_6.nnue) trained with training data generated with Nemorino partly using nets from Stockfish

http://talkchess.com/forum3/viewtopic.p ... 63#p875463

Nemorino only began using nets trained using only Nemorino with version 6.03..

[Graham Banks]

I didn't realize that Nemorino 6.00 was trained on Stockfish games.
I've not tested BBC 1.4 myself, but it must be in the blitz list.

Nemorino author said so himself:

Nemorino 6:
- Uses a net (nemorino_net_6.nnue) trained with training data generated with Nemorino partly using nets from Stockfish

http://talkchess.com/forum3/viewtopic.p ... 63#p875463

Nemorino only began using nets trained using only Nemorino with version 6.03..

Okay. He did say partly with 6.00, which is different than wholly.
It would explain why 6.05 is weaker though.

[Gabor Szots]

The fact that CCRL tested BBC 1.4... shows otherwise.

With BBC 1.4 it is clearly stated that an SF net was used. That was a unique experiment to show what an NNUE, implemented in a weakish engine, can achieve.

[AndrewGrant]

I didn't realize that Nemorino 6.00 was trained on Stockfish games.
I've not tested BBC 1.4 myself, but it must be in the blitz list.

Nemorino author said so himself:

Nemorino 6:
- Uses a net (nemorino_net_6.nnue) trained with training data generated with Nemorino partly using nets from Stockfish

http://talkchess.com/forum3/viewtopic.p ... 63#p875463

Nemorino only began using nets trained using only Nemorino with version 6.03..

Okay. He did say partly with 6.00, which is different than wholly.
It would explain why 6.05 is weaker though.

Should read this post: http://talkchess.com/forum3/viewtopic.p ... 13#p896213 (Skip to the 3rd paragraph). Current State of NNUE engines.

TL;DR: Stockfish using Leela evals/labels to train a network is, in some metrics of comparing, more or equally original than most other engines aside from Ethereal, Seer, and (? unknown) Komodo.

[Madeleine Birchfield]

Also, the latest Stockfish net is trained on T60 and T74 data. The only ratings list that has tested Leela with T60 and T74 nets is SPCC, and some ratings lists, such as FGRL, never even tested Leela in the first place.