mcostalba wrote:syzygy wrote:Contrary to what you think, this is not speculation.
You also wrote "The reason why it is there is to handle the case where DTZ is not available and the root position is in the TBs and winning."
Isn't this the same case...
There is the case where DTZ is available. SF then uses DTZ at the root only to filter out the non-winning moves (or the losing moves if the root is a draw). This code looks at the 50-move counter and tries to be a bit smart (e.g. it will go for losing moves will high DTZ if the root position is lost but the opponent has not too much 50-move margin). In this case, TB probing within the tree is disabled.
The other case is where DTZ is not available. If the root position is in the TBs, WDL is used to filter out the moves that are known to be suboptimal. But now it is impossible to properly take into account the 50-move rule. Since in this case the filtering step cannot guarantee a win within the 50-move rule, the search continues to do TB probes after captures and pawn moves. So now the search searches for either mate or a winning zeroing move. If the root posiion is a TB win and the search is able to bridge the distance from the root position to the next winning zeroing move, then the engine will convert the TB win (but engine play will be very ugly, as any winning pawn move will be preferred over a mate in two or a 2-ply combination that wins material). The engine may fail to convert the win if the distance to the next zeroing move is too large. This approach should normally work well in pawn endgames, where the winning path to mate can be very long and complicated, but the paths to each next winning pawn move will often be much shorter.
In this second case (root in TBs and winning, DTZ not available), one could do still a bit more by probing WDL at all OR nodes to quickly prune the non-winning moves, but this increases the number of probes a lot. The net effect of this change would probably be positive in endgames where the winning path is relatively narrow (like KRPvKP) and negative in endgames where basically all paths win (like KBNvK). When I wrote the root probing code I decided to keep the WDL root-probing fallback simple. So I only needed to add the rule50_count() == 0 condition. Play will be ugly, but it is only a fallback anyway.
While I am at it:
The root filtering approach has the problem that it might keep fewer moves than what multipv requires. It also has the problem that it might keep only those moves that the user has himself filtered out with the searchmoves option (in which case no moves are left to search at all).
To fix this, I implemented "root ranking" in Cfish. It turns out to be much simpler than the filtering code.
The ranking step assigns a "rank" to each root move based on its DTZ value and the value of the 50-move counter. Moves that are winning are assigned the top rank. Moves that do not win because of the 50-move rule are ranked lower. Moves that draw are ranked below that, moves that draw with high DTZ are ranked even lower. Moves that lose with low DTZ are ranked at the bottom. All root moves are then sorted by rank (top ranked first).
The idea of the ranks is that the main multipv loop is modified to search all equally ranked moves together. So if there are 3 moves that win and multipv = 5, then the multipv loop first searches the first 3 moves to find the best one, then the remaining two top-ranked moves to find the second best one, then the last top-ranked move to find the third best move. Then multipv searches moves ranked below the top-ranked moves to find the fourth and fifth best moves.
With this approach, multipv always produces the right number of lines and there is no problem if searchmoves happens to eliminate all top-ranked moves.
If DTZ is not available, then the ranks can be assigned on the basis of WDL values. This is of course less than ideal, because it cannot properly take into account the 50-move rule.