Hang a Rook?

[D Sceviour]

After looking at Bratko-Kopec with Stockfish 8, notice the following PV after a depth of 4 in BK #2:

[d]3r1k2/4npp1/1ppr3p/p6P/P2PPPP1/1NR5/5K2/2R5 w - - 0 1

Code: Select all

  4	+5.15	321	0:00.00	e5 g6 exd6 Rxd6
  3	+5.15	229	0:00.00	e5 g6 exd6
  2	+5.64	155	0:00.00	e5 b5
  1	+1.31	90	0:00.00	e5
  0	#

What is SF8 doing here by suggesting to hang a rook in the PV?

[hgm]

That is what you get when you prune a lot. Low depths with Stockfish mean nothing. The idea is that it will always reach high depth exactly because it prunes so much.

[D Sceviour]

That is what you get when you prune a lot. Low depths with Stockfish mean nothing. The idea is that it will always reach high depth exactly because it prunes so much.

Stockfish calculates something at low depths. BK #2 has already been analyzed to death, but this analysis of SF8 is absurd.

[hgm]

Sure, it calculates with the moves it doesn't prune. But if you then prune all reasonable moves...

What can you expect from a search of only 321 positions?

[Eelco de Groot]

Not sure if this actually happens, but after e5, the pawn can not be captured, moving the rook away is a quiet move and probably is LMPed. It is not clear to me why 1...Rxd4 is not calculated with a better score than +5 so better than g6.

The most recent development version even goes to + 637 centipawns but only 81 nodes calculated and depth 2 (After 1 e4-e5, 1...Ne7-g6 is of course also a funny move And both are quiet moves so it is not just a capture search here)

position fen 3r1k2/4npp1/1ppr3p/p6P/P2PPPP1/1NR5/5K2/2R5 w - - 0 1
go depth 20
info depth 1 seldepth 1 multipv 1 score cp 108 nodes 39 nps 39000 tbhits 0 time 1 pv e4e5
info depth 2 seldepth 2 multipv 1 score cp 637 nodes 81 nps 40500 tbhits 0 time 2 pv e4e5 e7g6 e5d6 g6f4 c3c6
info depth 3 seldepth 3 multipv 1 score cp 97 nodes 179 nps 44750 tbhits 0 time 4 pv e4e5 d6d4 b3d4 d8d4
info depth 4 seldepth 4 multipv 1 score cp 116 nodes 327 nps 81750 tbhits 0 time 4 pv d4d5 c6c5 f2f3 f7f6
info depth 5 seldepth 5 multipv 1 score cp 122 nodes 533 nps 106600 tbhits 0 time 5 pv d4d5 c6c5 c1g1 d8e8 g4g5
info depth 6 seldepth 6 multipv 1 score cp 137 nodes 1049 nps 174833 tbhits 0 time 6 pv d4d5 c6c5 f2f3 f8e8 c1g1 e8d7
info depth 7 seldepth 7 multipv 1 score cp 73 nodes 6189 nps 687666 tbhits 0 time 9 pv f4f5 f7f6 c1c2 f8g8 c3g3 d6d7 e4e5 f6e5
info depth 8 seldepth 8 multipv 1 score cp 66 nodes 11035 nps 919583 tbhits 0 time 12 pv f4f5 f7f6 c1b1 f8g8 c3g3 d6d7 e4e5 f6e5 d4e5
info depth 9 seldepth 12 multipv 1 score cp 81 nodes 21279 nps 1182166 tbhits 0 time 18 pv f2g3 d6d7 f4f5 b6b5 c1a1 b5b4 c3c5 d7a7 b3a5
info depth 10 seldepth 14 multipv 1 score cp 79 nodes 39490 nps 1410357 tbhits 0 time 28 pv f4f5 f7f6 e4e5 d6d5 c3e3 d5d7 e3c3 d7a7 f2f3 b6b5 a4b5 c6b5
info depth 11 seldepth 15 multipv 1 score cp 56 nodes 100973 nps 1771456 tbhits 0 time 57 pv f4f5 f7f6 f2e3 f8f7 e3f3 d6d7 c1e1 d7a7 e4e5 b6b5 a4b5 c6b5 e5e6 f7f8
info depth 12 seldepth 16 multipv 1 score cp 51 nodes 165423 nps 1858685 tbhits 0 time 89 pv e4e5 d6d5 f4f5 b6b5 c1a1 b5b4 c3c1 c6c5 c1c5 d5c5 d4c5 e7c6 e5e6 f7e6 f5e6 d8d5
info depth 13 seldepth 21 multipv 1 score cp 91 nodes 259053 nps 1992715 tbhits 0 time 130 pv d4d5 c6d5 e4e5 d6d7 b3d4 d8e8 f4f5 e7g8 c1e1 d7e7 c3e3 e8c8 e3b3 e7b7 f2f3 f7f6
info depth 14 seldepth 21 multipv 1 score cp 77 nodes 312354 nps 2041529 tbhits 0 time 153 pv d4d5 c6d5 e4e5 d6d7 b3d4 d8e8 f2f3 d7d8 c3b3 d8c8 c1c8 e8c8 b3b6 f8g8 f4f5 c8c4 b6b8 g8h7
info depth 15 seldepth 23 multipv 1 score cp 95 nodes 369958 nps 2078415 tbhits 0 time 178 pv d4d5 c6d5 e4e5 d6d7 b3d4 d8e8 c3b3 d7d8 b3b6 d8c8 c1c8 e8c8 f4f5 c8c4 b6b8 e7c8 f2e3 f8e8 b8a8 c4c3 e3f2 c3c4 d4f3
info depth 16 seldepth 27 multipv 1 score cp 99 nodes 457092 nps 2116166 tbhits 0 time 216 pv d4d5 c6d5 e4e5 d6d7 b3d4 d8e8 c3b3 e8c8 c1c8 e7c8 b3c3 c8a7 f2e3 d7b7 d4f5 b6b5 c3c5 b5a4 c5a5 a7c8 a5a8 b7c7 e3d4
info depth 17 seldepth 28 multipv 1 score cp 100 nodes 544085 nps 2142066 tbhits 0 time 254 pv d4d5 c6d5 e4e5 d6d7 b3d4 d8e8 f2f3 d7d8 c3b3 d8b8 f4f5 b8c8 c1c8 e8c8 b3b6 c8c3 f3e2 e7c8 b6c6 c3c6 d4c6 c8b6 c6a5
info depth 18 seldepth 32 multipv 1 score cp 104 nodes 672394 nps 2183097 tbhits 0 time 308 pv d4d5 c6d5 e4e5 d6d7 b3d4 d8e8 f2f3 d7d8 c3b3 d8b8 f4f5 b8c8 c1c8 e8c8 b3b6 c8c4 f3e3 f8e8 f5f6 g7f6 e5f6 e7c8 b6b8 e8d7 b8b7 d7e8
info depth 19 seldepth 34 multipv 1 score cp 87 nodes 871937 nps 2241483 tbhits 0 time 389 pv d4d5 c6d5 e4e5 d6d7 b3d4 d8e8 f2f3 d7d8 c3b3 d8b8 f4f5 b8c8 c1c8 e7c8 f3f4 c8d6 e5e6 d6c4 b3b5 f7e6 d4e6 f8f7 b5d5 c4b2 d5d7 e8e7 d7e7 f7e7 f4e5 b2a4 e6g7
info depth 20 seldepth 35 multipv 1 score cp 97 nodes 1325227 nps 2296753 tbhits 0 time 577 pv d4d5 c6d5 e4e5 d6d7 b3d4 d7b7 f4f5 b7b8 c3b3 d8c8 c1c8 e7c8 b3b5 c8e7 f2f3 f8e8 f5f6 g7f6 e5f6 e7g8 b5d5 b8c8 d5d6 c8c4 f3e3 c4a4 d6b6 a4a3 e3f4
bestmove d4d5 ponder c6d5

[JVMerlino]

Sure, it calculates with the moves it doesn't prune. But if you then prune all reasonable moves...

What can you expect from a search of only 321 positions?

Indeed. Perft 4 from the position is 622,922. Although, to be fair, I checked several other strong engines, and all of them reported <1K nodes at depth 4, and none of them had the rook hanging in the PV.

But, to also be fair, of course SF would never actually hang the rook in a game, even if it was depth limited, and who only gives an engine a couple of milliseconds before scrutinizing the PV?

[Ajedrecista]

Hello John:

Sure, it calculates with the moves it doesn't prune. But if you then prune all reasonable moves...

What can you expect from a search of only 321 positions?

Indeed. Perft 4 from the position is 622,922. Although, to be fair, I checked several other strong engines, and all of them reported <1K nodes at depth 4, and none of them had the rook hanging in the PV.

But, to also be fair, of course SF would never actually hang the rook in a game, even if it was depth limited, and who only gives an engine a couple of milliseconds before scrutinizing the PV?

I am not totally sure about the comparison with perft values: perft(1) of that position is 33 but depth 1 reports 90 nodes... of course I do not know exactly how a chess engine works.

Otherwise, I fully agree on the fact that very aggressive pruning of SF shows this behaviour, not considering it relevant. SF will not hang the rook with reasonable depths.

Just checking perft values and number of unique positions of Bratko-Kopec #2:

Code: Select all

JetChess 1.0.0.0:

    perft(1) =              33
    perft(2) =             793
    perft(3) =          26,013
    perft(4) =         622,922
    perft(5) =      20,077,998
    perft(6) =     486,503,398
    perft(7) =  15,329,960,358
    perft(8) = 376,726,075,275

positions(1) =              33
positions(2) =             793
positions(3) =          13,668
positions(4) =         198,137
positions(5) =       2,108,657
positions(6) =      23,173,657

I hope no typos.

Regards from Spain.

Ajedrecista.

[JVMerlino]

Hello John:

Sure, it calculates with the moves it doesn't prune. But if you then prune all reasonable moves...

What can you expect from a search of only 321 positions?

Indeed. Perft 4 from the position is 622,922. Although, to be fair, I checked several other strong engines, and all of them reported <1K nodes at depth 4, and none of them had the rook hanging in the PV.

But, to also be fair, of course SF would never actually hang the rook in a game, even if it was depth limited, and who only gives an engine a couple of milliseconds before scrutinizing the PV?

I am not totally sure about the comparison with perft values: perft(1) of that position is 33 but depth 1 reports 90 nodes... of course I do not know exactly how a chess engine works.

Otherwise, I fully agree on the fact that very aggressive pruning of SF shows this behaviour, not considering it relevant. SF will not hang the rook with reasonable depths.

Just checking perft values and number of unique positions of Bratko-Kopec #2:

Code: Select all

JetChess 1.0.0.0:

    perft(1) =              33
    perft(2) =             793
    perft(3) =          26,013
    perft(4) =         622,922
    perft(5) =      20,077,998
    perft(6) =     486,503,398
    perft(7) =  15,329,960,358
    perft(8) = 376,726,075,275

positions(1) =              33
positions(2) =             793
positions(3) =          13,668
positions(4) =         198,137
positions(5) =       2,108,657
positions(6) =      23,173,657

I hope no typos.

Regards from Spain.

Ajedrecista.

SF doesn't do any pruning at the root, but it does do qsearch after making all 33 root moves. So searching 90 nodes at depth 1 makes sense.

[mjlef]

Stockfish uses very aggressive pruning at low depths, even in PV nodes (where the principal variation comes from). After searching just a few moves, it tosses just about all quiet moves, making the search a bit more like a qsearch (which in most programs includes just captures, responses to check and perhaps some quiet checking moves and advanced pawn moves). Moving the rook to safety does not fit into those categories, so it just never sees a way to save the rook. One method to avoid this is after you search the first move, you note if the opponents response is the rook being captured. You mark the rook as a threatened piece, and make sure you include some quiet "safe" moves to see if the rook can escape capture. I have experimented with this in the past...and it is a close call to say if it helps or not.

Most programs prune less in PV nodes (some not at all) so you do not see this behavior. As depth increases, Stockfish includes more and more moves so it will understand it eventually.

This is just a reminder that "depth" means different things to different programs. You have to compare playing strength in timed games, and Stockfish has proven its strength.

[hgm]

One method to avoid this is after you search the first move, you note if the opponents response is the rook being captured. You mark the rook as a threatened piece, and make sure you include some quiet "safe" moves to see if the rook can escape capture. I have experimented with this in the past...and it is a close call to say if it helps or not.

The problem with this is that it often is difficult to identify the true threat from the line that refuted the first move. This because of the default move ordering, which performs captures in victim-value order. This masks the real threat by trading higher victims. E.g. if your Knight gets attacked by a Pawn, you would like to search moves that withdraw the Knight. But if the same position contains a QxQ, RxR or BxB capture, there are two problems. In the first place you would play the QxQ or RxR yourself as first move, and the opponent would first have to recapture the Q or R before he can take the Knight. This could be avoided by looking at the null-move refutation. (But you might not try null move in PV nodes.) But the second problem is that to refute the null move (or any other move that does not move away the Knight or capture its attacker) the opponent would now start with the QxQ, RxR, and only play out the PxN threat after you recapture the Q or R. This would mis-identify the threatened piece as the Q or R, and you would waste time by trying to withdraw those. This would avoid their trading, but it would still lose you the Knight!

So in identifying the threat, you would like to strip off the preceding trades of higher pieces. This would be easy in PV nodes, where you do get a PV. But in other nodes(or in a threat-identifying null move with null window) you would get a refutation three, where the threatened side plays the all-nodes. The opponent would refute your null with QxQ, to which you continue with something x Q, but this would fail low because you later lose the Knight. So you will be trying alternatives, but if these do not recapture the Queen, the opponent would withdraw it immediately, and you would fail low because you lost a Queen (even when you then could save the Knight). So you search a collection of moves, which fail low for different reasons. And you somehow have to conclude from that that the problem was the PxN threat.

This is not trivial. One could try something like identifying cut moves that raise the eval after the best capture in the following all node ('gain moves'), and propagate that towards the root along (cut-move, all-node) pairs that do not raise the eval. (QxQ, something x Q) would not raise the eval, so that the following PxN would be seen as the gain move in the node where QxQ is played. The gain move should then be considered the refutation of the preceding move, rather than the QxQ, so that as an alternative to that preceding move you try moving the N (and not the Q).