syzygy questions

[bob]

I am looking at syzygy for Crafty, and I presume I understand most of this but I wanted to ask to be certain there is not a surprise...

(1) obviously, as I did with Ken's DTC stuff in the 80's, you only probe with the 50 move counter is zero, as opposed to the Nalimov-like approach of only probing after a capture. This seems important for pawn pushes as you might be winning before a push, but drawing after a push so the zeroing push has to generate a probe. Correct?

(2) I presume there is no en passant consideration since that would make DTZ creation problematic, same position with and without EP possibility. IE the specific condition being a double pawn push that resets the 50 move counter and generates a probe, but what if the opponent wins with the ep capture and draws if he doesn't? So the probe after the double push should probably be skipped if there is a pawn that can actually make the ep capture (if the pawn can't be captured I would presume a probe is perfectly safe)???

(3) it seems that in the WDL case, there are only wins, draws and losses, with cursed wins and blessed losses just called "draws"? But in the DTC tables we get separate indications for draw and cursed win?

[syzygy]

I am looking at syzygy for Crafty, and I presume I understand most of this but I wanted to ask to be certain there is not a surprise...

(1) obviously, as I did with Ken's DTC stuff in the 80's, you only probe with the 50 move counter is zero, as opposed to the Nalimov-like approach of only probing after a capture. This seems important for pawn pushes as you might be winning before a push, but drawing after a push so the zeroing push has to generate a probe. Correct?

WDL50 is only accurate immediately after a zeroing move.

However, a DTC (or rather DTZ) value is always useful: it tells you the shortest distance to a zeroing move and the 50-move counter tells you how many moves you have left. By comparing the two you know whether a winning position is a real win or a cursed win.

(2) I presume there is no en passant consideration since that would make DTZ creation problematic, same position with and without EP possibility.

The tables do not include positions with ep rights (but of course do not ignore the en passant rule). The probing code takes care of en passant. (Basically by first probing as if ep is not possible, then correcting the result.)

So the probe after the double push should probably be skipped if there is a pawn that can actually make the ep capture (if the pawn can't be captured I would presume a probe is perfectly safe)???

No need to skip any probes, assuming you port all the probing code to Crafty's data structures and move generation functions.

The comments in tbprobe.cpp should help, but might leave some things a bit underdocumented. Feel free to ask.

(3) it seems that in the WDL case, there are only wins, draws and losses, with cursed wins and blessed losses just called "draws"? But in the DTC tables we get separate indications for draw and cursed win?

probe_wdl() returns -2,-1,0,1,2 for loss, blessed loss, draw, cursed win, win.

probe_dtz():

Code: Select all

// Probe the DTZ table for a particular position.
// If *success != 0, the probe was successful.
// The return value is from the point of view of the side to move:
//         n < -100 &#58; loss, but draw under 50-move rule
// -100 <= n < -1   &#58; loss in n ply &#40;assuming 50-move counter == 0&#41;
//         0	    &#58; draw
//     1 < n <= 100 &#58; win in n ply &#40;assuming 50-move counter == 0&#41;
//   100 < n        &#58; win, but draw under 50-move rule
//
// The return value n can be off by 1&#58; a return value -n can mean a loss
// in n+1 ply and a return value +n can mean a win in n+1 ply. This
// cannot happen for tables with positions exactly on the "edge" of
// the 50-move rule.
//
// This implies that if dtz > 0 is returned, the position is certainly
// a win if dtz + 50-move-counter <= 99. Care must be taken that the engine
// picks moves that preserve dtz + 50-move-counter <= 99.
//
// If n = 100 immediately after a capture or pawn move, then the position
// is also certainly a win, and during the whole phase until the next
// capture or pawn move, the inequality to be preserved is
// dtz + 50-movecounter <= 100.
//
// In short, if a move is available resulting in dtz + 50-move-counter <= 99,
// then do not accept moves leading to dtz + 50-move-counter == 100.
//

The "off by 1" thing has to do with the tables storing DTZ counts in full moves if you can normally afford to lose a ply without endangering the outcome. (So the tables store DTZ counts ply-accurate if there is at least one position that needs exactly 100 ply to be won.)

probe_wdl() is thread safe and is supposed to be called during the search.

probe_dtz() is not thread safe and is not supposed to be called during the search (but only at the root).

root_probe() shows one way of using DTZ at the root to filter out suboptimal moves (taking into account the 50-move rule!). The engine can then do a regular search on the remaining moves (Crafty's swindle mode). Playing only DTZ-optimal moves would lead to pawn pushes being preferred to mate in 2, etc.

If DTZ is only used to filter out suboptimal moves, there is a danger that the engine starts repeating positions. root_probe() therefore checks for a repetition in the game history. If there was one since the last zeroing move (played on the board), it will select DTZ-optimal moves to prevent the repeated position from occurring a third time.

[bob]

Seems straightforward, and so far all looks to be working. I still have some work to do to print out useful evaluation numbers, which I am still thinking about (since the DTZ number doesn't mean a lot in the grand scheme of things).

I had skimmed over some sort of en passant issue the stockfish guys were talking about here a while back and wanted to make sure that I didn't overlook something. Was that a problem with how they were accessing the data since they were rewriting everything???

[mcostalba]

I had skimmed over some sort of en passant issue the stockfish guys were talking about here a while back and wanted to make sure that I didn't overlook something. Was that a problem with how they were accessing the data since they were rewriting everything???

It was a syzygy bug.

[syzygy]

It was indeed a bug in the DTZ probing code and has been fixed.

[bob]

It was indeed a bug in the DTZ probing code and has been fixed.

Is there something new I should download, then??? My stuff is probably a month old or so???

Bob

[AndrewGrant]

I would guess the active github repo for it:

https://github.com/syzygy1/tb

Inlcudes a commit from ~1 month ago which fixed the enpass issue.

[syzygy]

I would guess the active github repo for it:

https://github.com/syzygy1/tb

Inlcudes a commit from ~1 month ago which fixed the enpass issue.

So you have to look in /interface.

The tbcore.c/.h and tbprobe.cpp/.h files are what is important.
It says .cpp, but it is really .c if you replace the SF-specific parts with the equivalent Crafty-based code.

The tbcore.c is #included in the tbprobe.cpp file which is ugly, but helps to separate the code that you need to adapt (tbprobe.cpp) from the code that in principle you don't need to touch (tbcore.c).

search.cpp is from (an old version of) SF that shows how the TBs could be used within the seach and at the root.

[bob]

I would guess the active github repo for it:

https://github.com/syzygy1/tb

Inlcudes a commit from ~1 month ago which fixed the enpass issue.

So you have to look in /interface.

The tbcore.c/.h and tbprobe.cpp/.h files are what is important.
It says .cpp, but it is really .c if you replace the SF-specific parts with the equivalent Crafty-based code.

The tbcore.c is #included in the tbprobe.cpp file which is ugly, but helps to separate the code that you need to adapt (tbprobe.cpp) from the code that in principle you don't need to touch (tbcore.c).

search.cpp is from (an old version of) SF that shows how the TBs could be used within the seach and at the root.

Where I am is here:

I have a position that is a cursed win (KNNKP, nalimov says mate in 97). I probe the position and I get the expected "cursed win" return. But then when I make one of the legal moves and probe, I get "tb win" and when I probe another move, I get not found. Most of the moves seem to produce the correct result.

I am looking at this.

But a question. The bug you fixed... was it in tbcore or elsewhere? IE do I need to throw out the tbprobe.c and start again or is the fix easy to insert? I will go back and look up the stockfish discussion here to see if there was a fix given...

[syzygy]

Where I am is here:

I have a position that is a cursed win (KNNKP, nalimov says mate in 97). I probe the position and I get the expected "cursed win" return. But then when I make one of the legal moves and probe, I get "tb win" and when I probe another move, I get not found. Most of the moves seem to produce the correct result.

I am looking at this.

Something is not yet working

Make sure you don't call probe_wdl_table() and probe_dtz_table() directly. The tables store incomplete data which is completed by the probe_wdl() and probe_dtz() functions.

But a question. The bug you fixed... was it in tbcore or elsewhere? IE do I need to throw out the tbprobe.c and start again or is the fix easy to insert? I will go back and look up the stockfish discussion here to see if there was a fix given...

It was in tbprobe.cpp. There were actually two bug fixes (the second "sign" bug was fixed in SF quite a while ago already):
https://github.com/syzygy1/tb/commit/68 ... 62d9762142
Instead of calling probe_ab() or probe_wdl() at linest 488-490 depending on the existence of ep rights, you could also simply call probe_wdl().

There is another small bug that Marco pointed out and was not yet fixed in my repository (just applied it):
https://github.com/syzygy1/tb/commit/c6 ... e948192dc8

List of all commits:
https://github.com/syzygy1/tb/commits/master