Page 1 of 3

Stockfish Syzygy: how to interpret mates?

Posted: Sun Dec 01, 2013 6:48 pm
by Jouni
I still don't get it! What are displayed mates meaning? 3 examples here with 5 piece bases and latest compile:

[D]5k2/8/8/1p5p/1P5P/8/4K3/8 w - - mate in 30, but SF mate in 58
[D]R7/4k3/8/8/8/p7/r3p3/4K3 b - - mate in 22, but SF mate in 52
[D]1k6/8/1B5p/1R6/6K1/8/8/7r w - - mate in 23, but SF mate in 60

Can't engine simply display "mate found", because distance seems to be anything over 50 :?: . Are Syzygy bases useless for serious analysis?

Re: Stockfish Syzygy: how to interpret mates?

Posted: Sun Dec 01, 2013 9:19 pm
by syzygy
The present implementation does not allow you to "seriously" analyse 5- or 6-piece positions, but for that you can use e.g. http://www.k4it.de/index.php?topic=egtb&lang=en

For positions with more pieces than the installed tablebases have, a mate in 50+x moves means that a winning tablebase position can be reached in x moves.

Actually, it seems you are analysing 6-piece positions with 5-piece tablebases. So when you see mate in 53 it means 3 moves until a winning 5-piece position. UCI does not allow outputting "tablebase win after 3 moves".

This seems to be too confusing for most people, so I might change this to simply report very high values. The user will then not know the distance to the win, but stop complaining.

I will soon commit a patch that will let SF report a score indicating the tablebase result when searching a position with 5 or 6 pieces (i.e. a position that is in the tablebases).

Re: Stockfish Syzygy: how to interpret mates?

Posted: Sun Dec 01, 2013 9:35 pm
by syzygy
I guess that once a "tablebase win in x" is found, the engine could search on to try to find a mate. But this would require quite a lot of changes in SF in code with which I am not very familiar. If anybody wants to do this, I will not stop him or her.

Re: Stockfish Syzygy: how to interpret mates?

Posted: Sun Dec 01, 2013 11:38 pm
by syzygy
syzygy wrote:I will soon commit a patch that will let SF report a score indicating the tablebase result when searching a position with 5 or 6 pieces (i.e. a position that is in the tablebases).
Patch committed.

You will now see only a very high score if the search has found a tablebase win.

Once the root position is in the tablebases, the reported scores will now correspond to the tablebase scores, unless an actual mate has been found in which case you will see the mate score. If the position is a 50-move draw (i.e. not enough moves left to convert the win) you will see +0.01 or -0.01.

Re: Stockfish Syzygy: how to interpret mates?

Posted: Mon Dec 02, 2013 9:36 am
by ernest
syzygy wrote:Patch committed.
And how about Houdini 4 on the matter? 8-)

Re: Stockfish Syzygy: how to interpret mates?

Posted: Mon Dec 02, 2013 9:45 am
by Houdini
ernest wrote:
syzygy wrote:Patch committed.
And how about Houdini 4 on the matter? 8-)
Houdini 4 uses high scores for Syzygy wins, not mate scores.
Using mate scores is too confusing for the end-user.

Re: Stockfish Syzygy: how to interpret mates?

Posted: Mon Dec 02, 2013 10:03 am
by ernest
Merci, Robert !

Re: Stockfish Syzygy: how to interpret mates?

Posted: Mon Dec 02, 2013 11:07 am
by RJN
Houdini wrote:
ernest wrote:
syzygy wrote:Patch committed.
And how about Houdini 4 on the matter? 8-)
Houdini 4 uses high scores for Syzygy wins, not mate scores.
Using mate scores is too confusing for the end-user.
I wish it were an adjustable option, so that we end-users could decide when we wished to be confused or not :wink:

Re: Stockfish Syzygy: how to interpret mates?

Posted: Mon Dec 02, 2013 12:12 pm
by Houdini
RJN wrote:I wish it were an adjustable option, so that we end-users could decide when we wished to be confused or not :wink:
I am open to your suggestions.

Note that each UCI option uses valuable screen space in the engine configuration window, so there must be a clear added value for each option.

What is the value of knowing that the current position has a DTZ of 8, or 5, or 22? I suggest that it's a fairly meaningless statistic, wouldn't you agree?

Re: Stockfish Syzygy: how to interpret mates?

Posted: Mon Dec 02, 2013 6:15 pm
by hgm
syzygy wrote:UCI does not allow outputting "tablebase win after 3 moves".
Uh? How about "info string tablebase win after 3 moves"
This seems to be too confusing for most people, so I might change this to simply report very high values. The user will then not know the distance to the win, but stop complaining.
Indeed, I think this is better. A tablebase score is similar in character to a mate score, but you could report mate in 10000+x, rather than 50+x. The latter is too close to something real, and is not even an upper limit, as you very well might have to pass through multiple tablebases, as well as spend more than 50 moves in the same tablebase (if all P-slices are considered belonging to the same tablebase, as is usual).

Note that it quite usual that engines report mate scores that are not the fastest mate (for over-the-horizon mates, discovered by hash grafting), so I don't know why that would confuse the end user more than any other mate score. Mate scores always mean 'mate in at most ...'.
Houdini wrote:What is the value of knowing that the current position has a DTZ of 8, or 5, or 22? I suggest that it's a fairly meaningless statistic, wouldn't you agree?
I don't think so. It tells you when to expect progress. When it says DTC = 22 it is obvious that it will be tedious and I'd brew a cup of tea, when it says DTZ = 3, I can wait for 3 moves to see if the DTZ changes into a mate score, or what the new DTZ will be.

I would have WinBoard print DTC's reported by the engine with the other engine output...

Anyway, it seems to me that it is more an interface decision whether to show this to the user, and that the engine should dutifully report it. The enfine can pirnt

info dtc N ....

as PV info (which should be ignored by UCI-compliant interfaces that don't know what dtc means), or, more conservatively

info string dtc N

just before the PV info.