hgm wrote: ↑Sat Jan 31, 2026 4:30 pm
What good would searching do in an EGT you already have? The EGT can already be considered to represent the result of an infinitely deep search all the way to checkmate (or for DTZ, winning conversion). The 'search' process proposed by Koistinen is exactly what retrograde analysis already does.
Koistinen's idea was to evaluate the positions (with a regular evaluation function) to estimate whether they are won, drawn or lost, and then somehow use this information to compress the tables better. (I do not think this is a great idea, but it always comes up.)
Koistinen wrote:One idea is to build an evaluator by defining lots of properties of a position and then correlate them with the actual values.
Find some way to combine it all and then find out if searching a few ply helps any.
I guess having one evaluator per endgame class might help.
I don't know which way is best, I just find the question interesting.
It is also possible to have uncertainty part of the evaluation and then decide whether to look up the value based on uncertainty.
To store exceptions I think a multilayered approach is best, but testing is needed.
Same idea might work for dtz50, say having an evaluator which gives a range of dtz50 values.
The possible ways to do it are many, maybe you need some luck to find a good way.
So the proposal there was a custom evaluation function per table plus an N-ply search and then somehow encode in the tablebase file the delta with what this predictor produces.
Alternatively, if you generate DTZ tables, you could choose to set all positions with dtz <= N for some N to "don't care" for compression (for the DTZ table or even for the WDL table) [my generator actually has an option for this for DTZ tables; I don't think it saved that much space though]. Then you need to do a full-width N-ply search with a trivial evaluation function (returning +/-n if dtz = n <= N and 0 otherwise) before probing to check if dtz <= N. If so, you don't need to access the table. If not, the table will store the correct dtz value.
Syzygy tables more or less do this with N=1. If there is a winning capture, then you don't need to access the table. (And if there is a drawing capture, it also uses this information to improve compression). It does not do this for direct mate-in-1s, though.
DTZ pawnful tables also require the probing code to check whether there is a winning pawn move, which again means dtz=1 (and this can be done by probing WDL, obviously). WDL tables do not (because access speed is more important for WDL than for DTZ).
