Natural TB

[syzygy]

- Very difficult to find mates
- Return very high/low unnatural scores
- Sacrifice piece to simplify position in case the resulting position is win in TB: this is NOT what a good human player would do in such cases and so in unnatural for me.

What you should do is simply not use TBs.

But is a waste of resources to probe at the beginning just for consuming a draw score. So I have modified the probing at the end of the search to store in TT a special VALUE_TB_DRAW that in no other case can appear as value in a TT (apart form the usual and harmless SMP races).

So probing once immediately after entering the TB (thereby cutting the whole subtree) is a waste or resources, but searching the subtree and probing a thousand times at the leaves is not?

Well, I cannot help you, that's for sure.

[Evert]

Indeed there is one case where probing earlier is always correct and does not introduces artifacts(*), that is when WDL returns a draw position. In this case if we return immediately a draw score it can't be wrong.

Why is this any different from lost or won positions?

If you capture into a won TB position, then you can cut the tree there: the result of the sub-tree is known. Similar, if you capture into a lost TB position, then you are done: you know that position is lost.
If this eventually turns out to be a PV node you may want to show the line leading to the final position, but that can (and arguably should) be done at the root.

Now, it's a slightly different matter if the root is a TB position and you are unhappy with DTZ50 optimal play (humans would consider DTM50 as "optimal", but even then a human would prefer a longer, easy mate to a faster but harder one), but that is really a separate problem. I suppose that in that case, one could follow the DTZ50 optimal line to mate and set that as a bound on the score and do a mate-search: if there is a faster mate you try to find it. Without DTM tables of some sort I don't think there's a fully reliable way to find it though, and for playing strength it makes no difference because it does not change to result of the game.

[hgm]

The solution is to only use DTZ50 info to decide if the position in the root table is won considering the current value of the ply counter (which WDL cannot tell you, as it assumes 0 for the ply counter). Draws then can be pruned (or return 0, which is a guaranteed fail high/low when the root position is a win, and thus amounts to the same thing).

To reduce artifacts, you should search on for the wins losses, as you don't want to play by DTZ50 in that case. But this is just a matter to score the won probes for different conversion with a value representing the attractiveness of the conversion (and of course their distance from the root, in engines that do not automatically go for the shortest path to a gain, to assure progress). So conversion from KQBNKP to KBNKP would score lower than conversion to KQBNK, even though the search can force it earlier. And all would score lower than a mate. This would guarantee the engine would go for the 'heuristically best' won conversion it can force within its horizon (where mate is always best), rather than for the fastest (which almost always is a sacrifice).

[Michel]

If you capture into a won TB position, then you can cut the tree there: the result of the sub-tree is known. Similar, if you capture into a lost TB position, then you are done: you know that position is lost.

There is a small cosmetic difference in case you are looking for a mate at root. A cut at a TB win/loss may mask a forced mate deeper in the tree which could become part of the PV.

So if window contains mate scores it may make sense (for example in analysis) to enable some kind of mate searching mode.

But I do not understand Marco at all to be honest. He has always strongly rejected code for SF with purely cosmectic effect. I recall for example that he strongly supported Lucas when the latter wanted to rip out SF's endgame code.

Now he seems to busy replacing Ronald's conceptually simple TB search by a pile of hacks whose practical and theoretical effects are dubious to say the least.

[mcostalba]

Why is this any different from lost or won positions?

Because a WDL won/lost position could actually end up in a draw (rule 50 or repetition).

[Evert]

Why is this any different from lost or won positions?

Because a WDL won/lost position could actually end up in a draw (rule 50 or repetition).

How?
Optimal play by the winning side avoids repetitions.
The syzygy WDL tables tell you whether a position is won/draw/lost under the 50-move rule (given that the position was reached after a capture, ie, the 50-move counter was just reset). If it is won, you can return a winning score for this node (fail high, if the bounds don't ask for an exact score). If it is lost, you can return a losing score (fail low, if the bounds don't ask for an exact score).
There is nothing to gain by not returning a score; the node is a terminal node with a known result (win/draw/loss).

The exception, as noted, is at the root, where DTZ50 optimal play may not be what you want. In the tree, you don't care.

[mcostalba]

Now he seems to busy replacing Ronald's conceptually simple TB search by a pile of hacks whose practical and theoretical effects are dubious to say the least.

This project is called "Natural TB" and from the name you should be able to infer the target...and from the target you should be able to infer that it does not aim to replace anything.

It is just an experiment to see if we can have the cake and eat that too. I never stated that this approach leads to a more precise play of current syzygy. What I am trying to achieve is to verify if this approach can:

1) Lead to a game play that is ELO equivalent to current TB

2) Avoid current TB artifacts that sound unnatural (at least to me)

As a secondary side effect I am trying hard to avoid DTZ probing (still don't know if this will be possible without losing ELO) because I think that if only WDL tables are present we can easily have 5-men in memory (374M for WDL against 922M for WDL+DTZ)

[Michel]

So if window contains mate scores it may make sense (for example in analysis) to enable some kind of mate searching mode.

Actually if a TBWIN is scored lower than mate wouldn't searching deeper be triggered automatically by normal search?

If we are in a zero window search with a mate score then a TBWIN would be below alpha so we continue searching...

Edit: There is also TBLOSS to consider. Hmm.

[Evert]

If you capture into a won TB position, then you can cut the tree there: the result of the sub-tree is known. Similar, if you capture into a lost TB position, then you are done: you know that position is lost.

There is a small cosmetic difference in case you are looking for a mate at root. A cut at a TB win/loss may mask a forced mate deeper in the tree which could become part of the PV.

So if window contains mate scores it may make sense (for example in analysis) to enable some kind of mate searching mode.

Ok, sure. If you can show that the root position is won (or lost) it makes sense to switch to a special mate-finder (or delayer) search. During a normal search (where tablebases can help by reducing effort in positions where the result is known) you wouldn't do that though.

[Michel]

So we should think of a tbwin,draw,loss as a lowerbound, exact score, upperbound... Is this theoretically correct?