Has Stockfish NNUE reached it's limit now?

[Madeleine Birchfield]

Seriously, nobody else is doing it? The Discoard channel is always busy...

That's not what I said; I said that they had to start from scratch because Sergio Vieri left. The Stockfish team were SPSA tuning the network weights two months ago while Gary Linscott was building a pytorch trainer from the ground up, because they couldn't successfully follow Sergio's guidelines and train a better neural network than the last net trained by Sergio using the nodchip trainer. Because of that, progress has been extremely slow the past two months.

[M ANSARI]

Or maybe go back to classical eval?

Probably yes, but it will be a long long way to go now ( Probably 3 years + working).

In the last Sufi 19, it was NNUE that just stepped in time to save Stockfish team. ( Lc0 vs SF Classical ended up 59-41 in bonus match).

And in cccc, Classical lost to Lc0 in 2 pawns handicap.

[pgn][Event "Odds Ladder (3|2)"]
[Site "?"]
[Date "2020.12.14"]
[Round "1"]
[White "Lc0"]
[Black "Stockfish Classic"]
[Result "1-0"]
[ECO "A00"]
[GameDuration "00:06:15"]
[GameEndTime "2020-12-14T18:18:15.145 PST"]
[GameStartTime "2020-12-14T18:11:59.558 PST"]
[Opening "Anti-Borg (Desprez) Opening"]
[PlyCount "79"]
[TimeControl "180+2"]

1. h4 Nf6 2. h5 Nxh5 3. a4 Nf6 4. a5 Nc6 5. Nf3 Nxa5 6. Ng1 Nc6 7. Nf3 Ng8 8.
Ng1 Nb8 9. e4 g6 10. Nf3 d6 11. d4 Bg7 12. Bc4 h6 13. Nc3 e6 14. Be3 Nf6 15. Qe2
Ng4 16. Bf4 Nc6 17. O-O-O Qe7 18. Bg3 O-O 19. Bh4 Qe8 20. Nb5 Qd7 21. e5 d5 22.
Nh2 Nxh2 23. Rxh2 dxc4 24. Bf6 Nxe5 25. dxe5 Qxb5 26. Qe3 c3 27. bxc3 Qa6 28.
Rd3 g5 29. Rh5 Qa1+ 30. Kd2 Qf1 31. Rxg5 hxg5 32. Qxg5 Qxf2+ 33. Kc1 Qg1+ 34.
Rd1 Qe3+ 35. Qxe3 Rd8 36. Rxd8+ Kh7 37. Qe4+ Kh6 38. Qh4+ Kg6 39. Qg5+ Kh7 40.
Qxg7# 1-0 [/pgn]

Pretty amazing game! But looking at the game I actually spotted a forced mate with Bishop + Rook by sacrificing the Queen. I guess humans are still better at some things as we can notice the Bishop and Rook mate pattern. Of course white is still winning but this makes me wonder if in some positions LC0 still has some tactical awareness issues.

[d]r1b2rk1/ppp2pb1/q3pBpp/4P3/8/2P1Q3/2P2PPR/2KR4 w - - 0 28

[Kanizsa]

After the sacrifice of Queen, have you thought to Qa1+ and Rd8+ giving back Queen on Rook d1 ?

[Jouni]

Analysis by Crystal 110920:

28.Qxh6 Qa1+ 29.Kd2 Rd8+ 30.Ke3 Qxc3+ 31.Kf4 Qb4+ 32.c4 Bxh6+ 33.Rxh6 Qxc4+ 34.Kg5 Qf4+ 35.Kxf4 Rd4+ 36.Rxd4 g5+ 37.Kxg5 a6 38.Rh8#
+- (#11) Depth: 16/37 00:00:01 7130kN

[Andrew]

A move earlier

Analysis by Crystal 220819 64 BMI2:

27.bxc3 Qa4 28.Rd4 Qa1+ 29.Kd2 Qf1 30.Rxh6 b6 31.Bxg7 Kxg7 32.Rhh4 Rh8 33.Rxh8 Qe2+ 34.Kxe2 Ba6+ 35.Kd2 Rxh8 36.Qg5 Rh1 37.Qf6+ Kh6 38.Qh8+ Kg5 39.Qxh1 Kf5 40.Qh4 Kxe5 41.Qf4#
White mates: +- (#15) Depth: 38/31 00:02:23 929MN, tb=931

then later
27.bxc3 h5 28.Rxh5 Qa5 29.Rd4 Bxf6 30.Qh6 Bg5+ 31.Rxg5 Qa1+ 32.Kd2 Qh1 33.Qxh1 Kg7 34.Rh5 gxh5 35.Qxh5 Rd8 36.Qg5+ Kh7 37.Qxd8 f6 38.Rh4+ Kg7 39.Qxf6+ Kg8 40.Rh8#
White mates: +- (#14) Depth: 40/29 00:04:07 1540MN, tb=931

Move 26
26.Qe3 h5 27.Rxh5 c3 28.bxc3 Qa6 29.Rd3 Qa1+ 30.Kd2 Qf1 31.Rh4 Bd7 32.Bxg7 f5 33.Rh8+ Kf7 34.Rxd7+ Ke8 35.Bxf8 Qd1+ 36.Kxd1 Kxd7 37.Rh7+ Ke8 38.Qd2 Kxf8 39.Qd7 Kg8 40.Qf7#
White mates: +- (#15) Depth: 42/35 00:02:52 895MN, tb=4329

Move 25 is heading that way as well but will need someone with a faster machine (gave up on 4 threads after 30 min!!)

Andrew

[M ANSARI]

After the sacrifice of Queen, have you thought to Qa1+ and Rd8+ giving back Queen on Rook d1 ?

This position would probably be good to add to the puzzle rush catalog but this would be a puzzle around 1600 ELO to 1700 ELO rating. After the queen sac all you have to do is calculate that the nuisance checks end and after that mate is inevitable. Forced and only moves are generally easy to calculate for a human, but those types of moves are super easy for even the weakest engines. So I am just surprised that this type of forcing variation is missed by one of the strongest engines on earth. I do remember in the early days of Lc0 where it would miss simple tactics and although things have dramatically improved I am wondering if some remnants remain.

[M ANSARI]

Dpost

[M ANSARI]

DPost

[ernest]

Suddenly +10 in NCM testing for latest SF version. Really unexpected and weird !

The jump appears to be confirmed on NCM for the just last Stockfish dev (20201218)...

[OneTrickPony]

And in cccc, Classical lost to Lc0 in 2 pawns handicap.

This is an amazing game, SF NNUE is quite clueless about it as well it takes a lot of hand holding to convince it white is actually winning after 23...dxc4, Leela sees it instantly.
When you run the position after white's 23th move Stockfish thinks 23...dxc4 gives big advantage until depth 43 and only then switches to 43...a6.
It's fun going through the lines where SF on quite serious hardware (almost 60M nodes/s in those positions) is jumping from -3 to +5