I'm disappointed with Stockfish dev.

[Uri Blass]

[Event "Computer chess game"] [Site "DESKTOP-OFQ3C0P"] [Date "2023.02.10"] [Round "?"] [White"Stockfish_23020319_x64_avx2 "] [Black "Stockfish_23020319_x64_avx2 "] [Result "*"] [BlackElo "3535"] [Time "22:26:00"] [WhiteElo "3535"] [TimeControl "0+7"] [SetUp "1"] [FEN "1r3rk1/1bqnbpp1/p2ppn1B/1p6/4PP1Q/PNNB4/1PP3PP/1K1R3R b - - 0 16"] [Termination "unterminated"] [PlyCount "0"] [WhiteType "program"] [BlackType "human"] 30...Qb5 31.Qf3 Kg7 32.Rxe6 Rxe6 33.Nf5+ Kf7 34.Qh3 Kg8 35.Nxd4 Qe8 36.Nxe6 Qxe6 37.f4 a5 38.f5 Qe8 39.Qh4 Qf7 40.Qf2 Rd8 41.Ra1 b3 42.Rb1 a4 43.cxb3 axb3 44.Qe3 gxf5 45.gxf5 Kh8 46.d4 Qc4 47.Rxb3 Qf1+ 48.Qg1 Qf4 49.Qe1 Rxd4 50.Rb8+ Kg7 51.Qg3+ Qxg3 52.hxg3 Rxe4 53.Rc8 Rc4 54.Rxc7+ Kh6 55.Rc8 Kg5 56.Kg2 Kxf5 57.Kh3 Ke5 58.c6 Kd6 59.c7 Rxc7 60.Ra8 Ke6 61.Kg2 Kf7 62.Kf3 Rc3+ 63.Kf2 = (0.07 --) Depth: 46/66 00:00:35 698MN, tb=47011 The position is equal

Your line is wrong

In 30...Qb5 31.Qf3 Kg7 32.Rxe6?? is a mistake.

correct is 32.e5 Qxa6 33.Qxf6+ Kh6 34.g5+ Kh5 35.Qf3+ Kh4
and white can choose between 36.Qe4+ Kh5 37.h3 and 36.h3

[Chessqueen]

[Event "Computer chess game"] [Site "DESKTOP-OFQ3C0P"] [Date "2023.02.10"] [Round "?"] [White"Stockfish_23020319_x64_avx2 "] [Black "Stockfish_23020319_x64_avx2 "] [Result "*"] [BlackElo "3535"] [Time "22:26:00"] [WhiteElo "3535"] [TimeControl "0+7"] [SetUp "1"] [FEN "1r3rk1/1bqnbpp1/p2ppn1B/1p6/4PP1Q/PNNB4/1PP3PP/1K1R3R b - - 0 16"] [Termination "unterminated"] [PlyCount "0"] [WhiteType "program"] [BlackType "human"]

Your line is wrong

In 30...Qb5 31.Qf3 Kg7 32.Rxe6?? is a mistake.

correct is 32.e5 Qxa6 33.Qxf6+ Kh6 34.g5+ Kh5 35.Qf3+ Kh4
and white can choose between 36.Qe4+ Kh5 37.h3 and 36.h3

I only placed pgn in front of the first line enclosed with Brackets so the diagram can be shown as requested by Edward

[Eelco de Groot]

I read here https://github.com/official-stockfish/S ... 3f680e0677

Created by retraining the master net on a dataset composed of:
* Most of the previous best dataset filtered to remove positions likely having only one good move
* Adding training data from Leela T77 dec2021 rescored with 16tb of 7-piece tablebases

So I think no wonder that Eduard would be disappointed, the builders of this net throw out all the 'good testpositions' with this, singular or critical positions where there is only one good move. So you have to rely totally on your search for those but apparently they are so atypical that it just confuses the net building? I have no real explanation. It is just like throwing out all tactical positions apparently that works too.

[dkappe]

I read here https://github.com/official-stockfish/S ... 3f680e0677

Created by retraining the master net on a dataset composed of:
* Most of the previous best dataset filtered to remove positions likely having only one good move
* Adding training data from Leela T77 dec2021 rescored with 16tb of 7-piece tablebases

So I think no wonder that Eduard would be disappointed, the builders of this net throw out all the 'good testpositions' with this, singular or critical positions where there is only one good move. So you have to rely totally on your search for those but apparently they are so atypical that it just confuses the net building? I have no real explanation. It is just like throwing out all tactical positions apparently that works too.

But the search means that positions many ply deeper are evaluated. Generally, you don’t want to train on positions that poorly predict game outcome. So, for example, anything that would get swept up in a qsearch makes for poor training data. But qsearch is expensive, so lots of heuristics have to be tried. The Stockfish team has been experimenting with all manner of filtering. At least for some time, this will be the way forward.

[Eduard]

There are now 6 new nets from Linrock. And each network is said to be better than the previous network. That's a lot of elos. But: Can someone show me where the progress is based on practical positions? What can the new network do better than the old one (before Linrock)? I can not find it.

If you play Stockfish on the server without a book (I'll do an new experiment) then Stockfish dev will play most of the moves that are in books. I have seen that.

Example: If an opponent plays 1. e4. Here Stockfish will respond with either e6, e5 or c5. After 1...e5 the open defense with Nxe4 or a Marshall attack or a Berlin defense is very likely. After 1..c5 the Najdorf system is very likely to be played.

Aside from consuming a lot of time without a book, Stockfish dev just becomes well-known theory paths, am playing far into the middle game!

The old network can do that just as well as the new one, with the difference that it is tactically stronger. So the new net doesn't bring me any advantage if I play normal chess. Sure, with Bullet 10s+0.1s everything is a different game, especially when strange exotic short variants are to be played.

Just who needs that? In practice nobody!

So I am happy about the progress of some other engines. Komodo Dragon plays so well with my tournament book on PlayChess that he's almost impossible to beat (I don't have Dragon myself, but a friend does). Lc0 plays so strong with my Lc0 tournament book that you can reach well over 3000 serverelo with an RTX 2080, and place 1. Even Ethereal 14 is difficult to defeat. I would not have thought that.

Nice to see these progress on other engines.

[carldaman]

Hi Eduard,

Which SF NNUE network do you recommend (instead of the linrock ones)?
What network is Charisma using?

Thanks!

[Sopel]

There are now 6 new nets from Linrock. And each network is said to be better than the previous network. That's a lot of elos. But: Can someone show me where the progress is based on practical positions? What can the new network do better than the old one (before Linrock)? I can not find it.

You willingly deny facts and then ask us to provide more facts. How do we know the new facts will not be denied, too?

edit. Just in case it's unclear. What you're trying to do is make us succumb to selection bias. This forum could use a few lessons about it actually.

[syzygy]

I started cloning because I wasn't satisfied with Stockfish in my analysis.

Then start developing and contributing, not renaming the engine and editing your name into it.

[syzygy]

But: Can someone show me where the progress is based on practical positions? What can the new network do better than the old one (before Linrock)?

It has been known since forever that engine progress cannot be measured on single positions. You need to play games, MANY MANY games.

[Uri Blass]

But: Can someone show me where the progress is based on practical positions? What can the new network do better than the old one (before Linrock)?

It has been known since forever that engine progress cannot be measured on single positions. You need to play games, MANY MANY games.

I agree that you need to play many games to measure progress but
If the engine play better then it means that there are positions that it play better moves and Eduard asked for these positions.

Tests are only at bullet time control and usually with biased book UHO_XXL_+0.90_+1.19.epd
Testing patches only with biased book means that it is possible that some change is a regression when you play with positions that are evaluated as close to 0.00 and we even do not know it.

There is an improvement even with normal book at bullet time control based on regression tests but Eduard claims that there is no improvement at longer time control than bullet.

I do not know if he is right and the only way to test is by games at longer time control than bullet and with normal book.

Even at bullet a patch that may be no regression from a biased book may be a regression from normal book and I do not understand
the reason that the stockfish developers do not test every patch that pass for no regression with normal book(I can understand not testing progress with normal book because it is logical to think that the rating difference with normal book is smaller).

I think that if they test only patches that pass for no regression with normal book and 8 cores for engine and 60+0.6 time control with 8 cores then it may be better for stockfish's developement.