Out of the Kai 450 positions, 99 not found by SF

[zullil]

Someone goofed.

I'm guessing the claimed SF eval must have been for low depths (e.g. <30 or so?).
What does Leela say? I'm now guessing this is another example of them not being all that different when you give them enough time.

Sf-dev did favor Black until depth 17, which was less than 1 second on my system:

info depth 2 seldepth 2 multipv 1 score cp 55 nodes 116267 nps 8304785 tbhits 0 time 14 pv b7c6 d5c6
info depth 3 seldepth 3 multipv 1 score cp 55 nodes 116907 nps 8350500 tbhits 0 time 14 pv b7c6 d5c6 d7b6
info depth 4 seldepth 4 multipv 1 score cp 62 nodes 117574 nps 8398142 tbhits 0 time 14 pv b7c6 d5c6 d7b6 a2a4
info depth 5 seldepth 5 multipv 1 score cp 62 nodes 118366 nps 8454714 tbhits 0 time 14 pv b7c6 d5c6 d7b6 a2a4 c8e6
info depth 6 seldepth 6 multipv 1 score cp 83 nodes 120292 nps 8019466 tbhits 0 time 15 pv b7c6 d5c6 d7b6 a2a4 a7a5
info depth 7 seldepth 7 multipv 1 score cp 105 nodes 122934 nps 8195600 tbhits 0 time 15 pv b7c6 d5c6 d7b6 a2a4 a7a5 d1b3
info depth 8 seldepth 11 multipv 1 score cp 53 nodes 129870 nps 8658000 tbhits 0 time 15 pv b7c6 d5c6 d7b6 a2a4 a7a5 b4a5 a8a5 d1b3 d8e8 b3b4
info depth 9 seldepth 14 multipv 1 score cp 59 nodes 134684 nps 8417750 tbhits 0 time 16 pv b7c6 d5c6 d7b6 a2a4 c8e6 a4a5 b6c8 e2a6 a8b8 b4b5 f6h4 d1e2 c8e7 a6b7
info depth 10 seldepth 18 multipv 1 score cp 50 nodes 190259 nps 9059952 tbhits 0 time 21 pv b7c6 d5c6 d7b6 a2a4 c8e6 a4a5 b6c8 b4b5 c8e7 d1a4 a7a6 e2c4 e6c4 a4c4 a6b5 c3b5
info depth 11 seldepth 22 multipv 1 score cp 54 nodes 297135 nps 9585000 tbhits 0 time 31 pv b7c6 d5c6 d7b6 a2a4 c8e6 a4a5 b6c8 b4b5 f8h8 d1a4 d8g8 g4g5 f6g5 f1g1 h8h5 e2f1 c8e7
info depth 12 seldepth 24 multipv 1 score cp 81 nodes 374259 nps 9848921 tbhits 0 time 38 pv b7c6 d5c6 d7b6 a2a4 c8e6 a4a5 b6c8 b4b5 f8h8 d1a4 d8g8 g4g5 f6g5 f1g1
info depth 13 seldepth 23 multipv 1 score cp 43 nodes 438944 nps 10208000 tbhits 0 time 43 pv b7c6 d5c6 d7b6 a2a4 c8e6 a4a5 b6c8 b4b5 f8h8 d1a4 d8g8 c3d5 e6d5 e4d5 c8e7 e2c4 g8e8 a4c2 f6h4 e1d3
info depth 14 seldepth 31 multipv 1 score cp 25 nodes 747908 nps 10533915 tbhits 0 time 71 pv b7c6 d5c6 d7b6 a2a4 c8e6 a4a5 b6c8 b4b5 f8h8 d1a4 d8g8 c3d5 e6d5 e4d5 c8e7 e2c4 g8e8 e1d3 e7g6 a1a2 e5e4 f1e1 e4e3 a2e2
info depth 15 seldepth 28 multipv 1 score cp 29 nodes 1027717 nps 10933159 tbhits 0 time 94 pv b7c6 d5c6 d7b6 a2a4 c8e6 a4a5 b6c8 b4b5 f8h8 c3d5 f6h4 e2c4 c8e7 e1d3 e7d5 c4d5 e6d5
info depth 16 seldepth 25 multipv 1 score cp 31 nodes 1130918 nps 10979786 tbhits 0 time 103 pv b7c6 d5c6 d7b6 a2a4 c8e6 a4a5 b6c8 b4b5 f8h8 d1a4 d8g8 c3d5 e6d5 e4d5 g8d5 g4g5 f6g5 f1g1 h8h5 e1d3 c8e7
info depth 17 seldepth 31 multipv 1 score cp 9 nodes 3313914 nps 12505335 tbhits 0 time 265 pv b7c6 d5c6 d7b6 a2a4 a7a5 b4a5 a8a5 d1b3 f8h8 e1d3 c8a6 b3b4 a6d3 b4a5 b6c4 a5b4 d3e2 c3e2 c4e3 f1b1 d6d5 g4g5 f6g5

[jp]

Sf-dev did favor Black until depth 17, which was less than 1 second on my system:

SF-non-dev favors Black far beyond that.

-1.02 depth 30 (here meaning in favor of Black).
-0.37 depth 31 (looks like it might change around here...).
-0.29 depth 32.
-0.34 depth 33 (stubbornly refusing to change its mind ).

[zullil]

Sf-dev did favor Black until depth 17, which was less than 1 second on my system:

SF-non-dev favors Black far beyond that.

-1.02 depth 30 (here meaning in favor of Black).
-0.37 depth 31 (looks like it might change around here...).

Wow, you're right. Sf-10 is very different than Sf-dev on this position. Black is considered better until here in my new search.

info depth 32 seldepth 54 multipv 1 score cp -12 nodes 2923895952 nps 20410286 hashfull 794 tbhits 0 time 143256 pv d7b8 b4b5 a7a6 a2a4 d8e8 e2c4 e8f7 c6b7 c8b7 b5a6 b8a6 e1d3 f7e7 h1h2 f6h4 c3e2 a6c5 d3c5 d6c5 a4a5 f8h8 a5a6 b7c8 f1g1 h4f2 d1f1 f2g1 f1g1 a8a7 e2c3 e7h4 c3e2

[jp]

SF-non-dev:

-1.02 depth 30 (here meaning in favor of Black).
-0.37 depth 31.
-0.29 depth 32.
-0.34 depth 33.

0.00 depth 34.

[jp]

A third position.

[d]r2q2k1/Np1b1rb1/3p1nn1/3Pp2p/4Ppp1/1Q1N1P2/PP2BBPP/2R2R1K b - - 0 20

Nxe4 is nothing special for NN programs:

Code: Select all

ScorpioNN 3.0-n_maddex:
00:06	        119k	20k	+0,10	Nf6xe4

AllieStein 0.4-n10.1:
00:06	        269k	41k	+0,28	Nf6xe4

Lc0 0.21.1-nT40.T8.610:
00:11	        104k	9k	+0,32	Nf6xe4 

SF doesn't choose Nxe4 at low depths. Assuming it really is the best move, I'd like to see at what depth it chooses it.

[zullil]

A third position.

[d]r2q2k1/Np1b1rb1/3p1nn1/3Pp2p/4Ppp1/1Q1N1P2/PP2BBPP/2R2R1K b - - 0 20

Nxe4 is nothing special for NN programs:

Code: Select all

ScorpioNN 3.0-n_maddex:
00:06	        119k	20k	+0,10	Nf6xe4

AllieStein 0.4-n10.1:
00:06	        269k	41k	+0,28	Nf6xe4

Lc0 0.21.1-nT40.T8.610:
00:11	        104k	9k	+0,32	Nf6xe4 

SF doesn't choose Nxe4 at low depths. Assuming it really is the best move, I'd like to see at what depth it chooses it.

Stockfish-dev (bench 3568210):

info depth 24 seldepth 44 multipv 1 score cp -19 lowerbound nodes 83200901 nps 19095914 hashfull 37 tbhits 0 time 4357 pv f6e4
info depth 22 currmove f6e4 currmovenumber 1
info depth 24 seldepth 44 multipv 1 score cp -3 lowerbound nodes 84628238 nps 19099128 hashfull 37 tbhits 0 time 4431 pv f6e4

[jp]

Stockfish-dev (bench 3568210):

info depth 24 seldepth 44 multipv 1 score cp -19 lowerbound nodes 83200901 nps 19095914 hashfull 37 tbhits 0 time 4357 pv f6e4
info depth 22 currmove f6e4 currmovenumber 1
info depth 24 seldepth 44 multipv 1 score cp -3 lowerbound nodes 84628238 nps 19099128 hashfull 37 tbhits 0 time 4431 pv f6e4

Oh, that's very quick. Again, SF-dev & SF-non-dev may be quite different. SF-non-dev is in the high 20s (depths) without choosing it. (I think it considers it around depth 18 but then moves to other moves.)

[zullil]

Stockfish-dev (bench 3568210):

info depth 24 seldepth 44 multipv 1 score cp -19 lowerbound nodes 83200901 nps 19095914 hashfull 37 tbhits 0 time 4357 pv f6e4
info depth 22 currmove f6e4 currmovenumber 1
info depth 24 seldepth 44 multipv 1 score cp -3 lowerbound nodes 84628238 nps 19099128 hashfull 37 tbhits 0 time 4431 pv f6e4

Oh, that's very quick. Again, SF-dev & SF-non-dev may be quite different. SF-non-dev is in the high 20s (depths) without choosing it. (I think it considers it around depth 18 but then moves to other moves.)

Sf-dev (without endgame tables) says 0.00:

info depth 57 seldepth 83 multipv 1 score cp 0 nodes 121115142796 nps 22306009 hashfull 1000 tbhits 0 time 5429709 pv f6e4 f3e4 f4f3 f2e3 f3e2 f1f7 g8f7 b3b7 f7g8 a7c6 d8e8 c1e1 a8a2 e1e2 g6f4 e3f4 e5f4 e2e1 f4f3 g2f3 g4f3 b7b6 e8f7 b6f2 d7h3 e4e5 d6e5 d3e5 g7e5 c6e5 f7d5 h1g1 d5b7 f2g3 b7g7 g3g7 g8g7 e5d3 h3f5 e1e3 f5d3 e3d3 a2b2 d3f3 g7h6 f3g3 h5h4 g3g4 h6h5 g4g2 b2b1 g1f2 b1c1 f2f3 c1c3 f3f4 c3c6 g2g5 h5h6 g5d5 c6c1 d5b5 c1c2 h2h3 c2g2 b5b6 h6h5 b6b5 h5g6 b5b7 g2g3 b7b6 g6h5 b6b5 h5g6

[jp]

Stockfish-dev (bench 3568210):
info depth 24 seldepth 44 multipv 1 pv f6e4

Sf-dev (without endgame tables) says 0.00:

info depth 57 seldepth 83 multipv 1 score cp 0 pv f6e4 f3e4

It looks like SF-non-dev doesn't choose ... Nxe4 before at least depth 40:

depth 39: ... Ne8 or ... Bh6 (+0.57 White advantage).

[jp]

[d]5rk1/NpR5/3p1q1b/3Pp2p/4Pn2/5P1b/PPQ2B1P/4N2K w - - 9 29

The Chessbase human games-trained NN is reported to have eval -1.22, and SF 0.00. (Nodes/Depth not known.)

SF-non-dev, depth=48:

0.00 29.Rxb7 Kh8 30.Nb5 Rg8 31.Bg3 h4 32.Bxf4 Bxf4 33.Qf2 Qh6 34.Qa7 Be3 35.Rh7+ Qxh7 36.Qxe3 Rg6 37.Nc3 Qd7 38.Qf2 Kg8 39.Nd1 Rg7 40.Qxh4 Qa7 41.Nf2 Bd7 42.Ng4 Bxg4 43.fxg4 Qxa2 44.Qd8+ Kh7 45.Qf6 Qa1 46.Qf5+ Rg6 47.Qf7+ Rg7

Louis, can you see what SF-dev or Lc says??