Question to syzygy author

[Dirt]

For me code should be easy to understand and to read

Well, I think you have succeeded in reducing the number of lines by a lot, but figuring out *why* it works correctly (if it actually does work correctly) is now a LOT harder. The original code had a clear logical plan (first ignore en passant, then fix it only as the last step, etc.).

And you have absolutely no need to teach me how to program. Really.

Now go figure it out yourself. I have had enough of correcting the bugs you are introducing even after I explain the errors that should be avoided.

Nonetheless, thank you for explaining this in a public place. It's been fascinating.

[stuwph]

And you have absolutely no need to teach me how to program. Really.

Ronald I would never read that intention out of Marco's post, never ...

hope you will change your mind and move on sharing your thoughts for the sake of chess programming and the pleasure to listen

Thanks to both of you for doing an amazing work

[Dirt]

And you have absolutely no need to teach me how to program. Really.

Ronald I would never read that intention out of Marco's post, never ...

Irrespective of Marco's intention, what started out as a documentation project turned into a re-write, and for goals that Ronald doesn't share. I'm not surprised he wanted to end this.

[mcostalba]

I have had enough of correcting the bugs you are introducing

Indeed there was another one.

The point is that DTZ files store a "don't care" value also in case the only possible legal moves are captures. For instance here:

[d]4K3/4q3/6p1/2k5/6p1/8/8/8 w - - 0 7

Only king capture is possible, but probing the dtz table returns -2 instead of -1.

So I needed to update again (for the fourth time!) the search() function. Incidentally, the ep move case is a subset of this "only capture possible" more general case so the new code gracefully handles also ep case (I hope you don't get angry because I wrote "gracefully").

Seriously, I think you really overreacted. That was totally unexpected by me. But of course I cannot continue alone, firstly because I am not able to figure out the code myself, second because it makes no sense to continue without author support. But if this thing stops here that would be really a pity because this topic is fascinating and I think we (me and you, or you and me if you prefer) are doing a great job that a lot of people appreciate (see just the topic views).

P.S:Hoping that you are still interested, please give a look at the updated code:

https://github.com/official-stockfish/S ... .cpp#L1236

Thanks!

[mcostalba]

I have found a possible serious bug in original TB implementation.

In this position:

[d]3K4/8/3k4/8/4p3/4B3/5P2/8 w - - 0 5

DTZ probing returns 1 instead of 15 that is the correct value.

What happens is that the double push f4 is erroneously detected as a win move.

After the push we have:

[d]3K4/8/3k4/8/4pP2/4B3/8/8 b - f3 0 5

And here the code misses the possible ep capture exf3.

The bug is in probe_dtz_no_ep() where here:

Code: Select all

  if (wdl > 0) {
    // Generate at least all legal non-capturing pawn moves
    // including non-capturing promotions.
    if (!pos.checkers())
      end = generate<NON_EVASIONS>(pos, stack);
    else
      end = generate<EVASIONS>(pos, stack);

    for (moves = stack; moves < end; moves++) {
      Move move = moves->move;
      if (type_of(pos.moved_piece(move)) != PAWN || pos.capture(move)
                || !pos.legal(move, ci.pinned))
        continue;
      pos.do_move(move, st, pos.gives_check(move, ci));
      int v = -probe_ab(pos, -2, -wdl + 1, success);     <-------------   This is wrong
      pos.undo_move(move);
      if (*success == 0) return 0;
      if (v == wdl)
        return v == 2 ? 1 : 101;   <---------------- Here we immediately return '1'
    }
  }

Is used probe_ab() that is blind to ep captures so it returns v == 2 (win) for position 3K4/8/3k4/8/4pP2/4B3/8/8 b - f3 0 5

Note that at the caller site the original position did not have any possible ep capture, so probe_dtz() returns immediately after calling probe_dtz_no_ep().

This is a possible serious bug.

I found it while debugging new code, I found some mismatches on returned values between mine and original code. Initially I debugged my code assuming I had the bug, but after some further investigation I found this one.

[Dirt]

I have found a possible serious bug in original TB implementation.

In this position:

[d]3K4/8/3k4/8/4p3/4B3/5P2/8 w - - 0 5

DTZ probing returns 1 instead of 15 that is the correct value.

What happens is that the double push f4 is erroneously detected as a win move.

Stockfish 6 never considers the double pawn push, not even at one ply. Why is that?

[mcostalba]

In encode_pawn() we have this to map a square into a lower one according to the "come later" adjustment.

Code: Select all

    
for (m = i; m < t; m++) {
      int p = pos[m];
      for (k = 0, j = 0; k < i; k++)  // Shouldn't be k < m ?
        j += (p > pos[k]);
      s += binomial[m - i][p - j - 8];
    }

I am wondering why the second loop stops at 'i' instead of 'm'. Currently only pieces in the previous groups are considered, but not previous pieces in the same current group.

The only other usage of this mapping technique, in the encoding of the leading group, we down-map a square even if a piece in the same leading group has a lower square.

[elcabesa]

Hi marco,

I'm just trying to understand whether or not my SYZYGY implementation has the bug you are reporting.

I'm using Fathom syzygy code and testing tb_probe_root with the reported position ( 3K4/8/3k4/8/4p3/4B3/5P2/8 w - - 0 5 ) tell me that f3 or f4 are not good moves and it removes them from root candidate moves.

after that it chooses Bf4+

is this the problem are you reporting? because it seems that fathom is not affected by the bug.

thank you for the help you could give to me.

[syzygy]

I have found a possible serious bug in original TB implementation.

In this position:

[d]3K4/8/3k4/8/4p3/4B3/5P2/8 w - - 0 5

DTZ probing returns 1 instead of 15 that is the correct value.

What happens is that the double push f4 is erroneously detected as a win move.

After the push we have:

[d]3K4/8/3k4/8/4pP2/4B3/8/8 b - f3 0 5

And here the code misses the possible ep capture exf3.

The bug is in probe_dtz_no_ep() where here:

You're right. Very annoying.

The code behind this site has the same bug:
https://syzygy-tables.info/?fen=3K4/8/3 ... _w_-_-_0_1
As long as the pawns stay where they are, the scores reported are totally confused. The DTZ 2 and DTZ 10 scores are wrong too.

Although this part of the code is functionally identical, my private engine correctly reports dtz=15. A DTZ-optimal sequence seems to be: Bb6 Ke6 Kc7 Kd5 Kd7 Ke5 Kc6 Kf5 Kd5 Kf4 Be3+ Kf3 Kd4 Ke2 Kxe4

I think the explanation for why my engine returns correct results is this: instead of filtering out en passant captures in probe_ab() and probe_dtz_no_ep(), in my engine probe_dtz() and probe_wdl() first set the ep flag to 0, then call probe_dtz_no_ep() and probe_ab(), then restore the ep flag and correct for ep. The double pawn push in probe_dtz_no_ep() sets the ep flag and therefore probe_ab() called on the position after f4 happens to look at the ep capture and finds that it is drawing.

I doubt that I did this intentionally.... but it might explain why I never caught this bug.

[syzygy]

Stockfish 6 never considers the double pawn push, not even at one ply. Why is that?

As you can see at the syzygy-tables.info link above, the probing code correctly identifies f4 as a draw. The problem occurs in positions with a double pawn push that would be winning but for an immediate ep capture. The problem also occurs in positions one ply earlier.

Since SF uses DTZ only for filtering at the root, and it still correctly distinguishes between winning, drawing and losing moves, this could only be a problem if the 50-move rule comes into play. Some moves that should have been filtered out because of too high DTZ might stay in because a double pawn push / en passant situation could incorrectly make probe_dtz() return a too low DTZ value.

I'm just trying to understand whether or not my SYZYGY implementation has the bug you are reporting.

I'm using Fathom syzygy code and testing tb_probe_root with the reported position ( 3K4/8/3k4/8/4p3/4B3/5P2/8 w - - 0 5 ) tell me that f3 or f4 are not good moves and it removes them from root candidate moves.

So the above also answers your question. The drawing and losing moves are still filtered out.