En passant problem in gaviota tablebases

[petero2]

While implementing syzygy tablebase support in Texel, as a sanity check I decided to compare the WDL values to the values returned by gaviota tablebases. In addition to the expected differences caused by 50-move draws, I also found a problem related to en passant captures.

If the only legal move in a position is an en passant capture, and this capture leads to a loss, the gaviota probing code incorrectly returns a draw score. This happens for example in the following positions:
[d]k1K5/8/8/8/4pP2/4Q3/8/8 b - f3 0 1
[d]8/8/8/8/Pp6/1K6/3N4/k7 b - a3 0 1
I'm not sure if this can be fixed efficiently in the probing code. Maybe it is better to detect this case in the calling code where you have access to the engine move generator and make/unmake move functions.

In total I found about 400 such positions, but some of them are not reachable from the initial chess position, for example:
[d]8/8/8/8/pP6/k7/8/KQ6 b - b3 0 1

[JVMerlino]

I am not able to reproduce this behavior. In each position, Myrddin correctly finds the en passant move and returns a Mate in N score for White.

jm

[michiguel]

While implementing syzygy tablebase support in Texel, as a sanity check I decided to compare the WDL values to the values returned by gaviota tablebases. In addition to the expected differences caused by 50-move draws, I also found a problem related to en passant captures.

If the only legal move in a position is an en passant capture, and this capture leads to a loss, the gaviota probing code incorrectly returns a draw score. This happens for example in the following positions:
[d]k1K5/8/8/8/4pP2/4Q3/8/8 b - f3 0 1
[d]8/8/8/8/Pp6/1K6/3N4/k7 b - a3 0 1
I'm not sure if this can be fixed efficiently in the probing code. Maybe it is better to detect this case in the calling code where you have access to the engine move generator and make/unmake move functions.

In total I found about 400 such positions, but some of them are not reachable from the initial chess position, for example:
[d]8/8/8/8/pP6/k7/8/KQ6 b - b3 0 1

I will take a look. For now, w/o looking at the code, and if I remember correctly, I can say the probing code should return "not available" (does it? if not, that is a bug). The TBs do not store en passant positions (though they are taken into account for building them of course) since it is not efficient to do so. Search will take care of it since if the position is returned "not available" it will go to the next ply.

Miguel

[michiguel]

While implementing syzygy tablebase support in Texel, as a sanity check I decided to compare the WDL values to the values returned by gaviota tablebases. In addition to the expected differences caused by 50-move draws, I also found a problem related to en passant captures.

If the only legal move in a position is an en passant capture, and this capture leads to a loss, the gaviota probing code incorrectly returns a draw score. This happens for example in the following positions:
[d]k1K5/8/8/8/4pP2/4Q3/8/8 b - f3 0 1
[d]8/8/8/8/Pp6/1K6/3N4/k7 b - a3 0 1
I'm not sure if this can be fixed efficiently in the probing code. Maybe it is better to detect this case in the calling code where you have access to the engine move generator and make/unmake move functions.

In total I found about 400 such positions, but some of them are not reachable from the initial chess position, for example:
[d]8/8/8/8/pP6/k7/8/KQ6 b - b3 0 1

I will take a look. For now, w/o looking at the code, and if I remember correctly, I can say the probing code should return "not available" (does it? if not, that is a bug). The TBs do not store en passant positions (though they are taken into account for building them of course) since it is not efficient to do so. Search will take care of it since if the position is returned "not available" it will go to the next ply.

Miguel

Geez, what I said is valid for castle, no en passant, which I included a sort of a one ply search inside the probing code. I need to take a closer look. I have not gone through this in a loong time.

Miguel

[petero2]

If the only legal move in a position is an en passant capture, and this capture leads to a loss, the gaviota probing code incorrectly returns a draw score.

Geez, what I said is valid for castle, no en passant, which I included a sort of a one ply search inside the probing code. I need to take a closer look. I have not gone through this in a loong time.

I believe the problem is in gtb-probe.c, function tb_probe_, line 1676, which does:

Code: Select all

/* chooses to ep or not */
dtm = bestx (stm, epscore, dtm);

This logic is incorrect if the non-ep score is a stalemate score, because in that case you can not choose to use the stalemate score, you are forced to play the losing ep capture.

[michiguel]

If the only legal move in a position is an en passant capture, and this capture leads to a loss, the gaviota probing code incorrectly returns a draw score.

Geez, what I said is valid for castle, no en passant, which I included a sort of a one ply search inside the probing code. I need to take a closer look. I have not gone through this in a loong time.

I believe the problem is in gtb-probe.c, function tb_probe_, line 1676, which does:

Code: Select all

/* chooses to ep or not */
dtm = bestx (stm, epscore, dtm);

This logic is incorrect if the non-ep score is a stalemate score, because in that case you can not choose to use the stalemate score, you are forced to play the losing ep capture.

I was just looking at that right now. Not sure that line is wrong, per se, but wrong in the context. Yes, there is a bug in the probing code, not in the tables. Basically, it chooses the score of the position before ep rather than executing the ep. But it is not possible because of a stalemate. Duh. It is equivalent of choosing a nullmove in this position.

I guess this escaped all the validation tests I performed because the values in the tables are correct. Arghh.

Miguel
PS: Thanks a lot Peter for you fantastic eye.

[petero2]

I am not able to reproduce this behavior. In each position, Myrddin correctly finds the en passant move and returns a Mate in N score for White.

Maybe Myrddin works like texel in this case and does not probe the root position, but only probes the positions after 1 ply. In texel if I start with the following position:
[d]k1K5/8/8/8/4p3/4Q3/5P2/8 w - - 0 1
and issue the following commands:

Code: Select all

uci
setoption name gaviotatbpath value /home/petero/chess/gtb
position fen k1K5/8/8/8/4p3/4Q3/5P2/8 w - - 0 1
setoption name multipv value 100
go infinite

I get the following result. Note the incorrect score for the f2f4 move:

Code: Select all

info depth 100 score mate 1 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 1 pv e3a3
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 2 pv e3e4
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 3 pv e3d4
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 4 pv e3c3
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 5 pv e3c5
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 6 pv e3g5
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 7 pv f2f3
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 8 pv e3d2
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 9 pv e3b3
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 10 pv e3b6
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 11 pv e3e1
info depth 100 score mate 3 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 12 pv e3e2
info depth 100 score mate 3 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 13 pv e3c1
info depth 100 score mate 3 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 14 pv e3h6
info depth 100 score mate 4 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 15 pv c8d7
info depth 100 score mate 4 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 16 pv e3f4
info depth 100 score mate 4 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 17 pv e3g3
info depth 100 score mate 4 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 18 pv e3h3
info depth 100 score mate 5 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 19 pv c8d8
info depth 100 score mate 22 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 20 pv e3a7
info depth 100 score mate 26 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 21 pv e3f3
info depth 100 score cp 0 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 22 pv f2f4
info depth 100 score cp 0 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 23 pv c8c7
info depth 100 score mate -14 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 24 pv e3d3

[JVMerlino]

I am not able to reproduce this behavior. In each position, Myrddin correctly finds the en passant move and returns a Mate in N score for White.

Maybe Myrddin works like texel in this case and does not probe the root position, but only probes the positions after 1 ply. In texel if I start with the following position:
[d]k1K5/8/8/8/4p3/4Q3/5P2/8 w - - 0 1
and issue the following commands:

Code: Select all

uci
setoption name gaviotatbpath value /home/petero/chess/gtb
position fen k1K5/8/8/8/4p3/4Q3/5P2/8 w - - 0 1
setoption name multipv value 100
go infinite

I get the following result. Note the incorrect score for the f2f4 move:

Code: Select all

info depth 100 score mate 1 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 1 pv e3a3
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 2 pv e3e4
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 3 pv e3d4
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 4 pv e3c3
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 5 pv e3c5
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 6 pv e3g5
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 7 pv f2f3
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 8 pv e3d2
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 9 pv e3b3
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 10 pv e3b6
info depth 100 score mate 2 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 11 pv e3e1
info depth 100 score mate 3 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 12 pv e3e2
info depth 100 score mate 3 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 13 pv e3c1
info depth 100 score mate 3 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 14 pv e3h6
info depth 100 score mate 4 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 15 pv c8d7
info depth 100 score mate 4 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 16 pv e3f4
info depth 100 score mate 4 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 17 pv e3g3
info depth 100 score mate 4 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 18 pv e3h3
info depth 100 score mate 5 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 19 pv c8d8
info depth 100 score mate 22 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 20 pv e3a7
info depth 100 score mate 26 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 21 pv e3f3
info depth 100 score cp 0 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 22 pv f2f4
info depth 100 score cp 0 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 23 pv c8c7
info depth 100 score mate -14 time 2143 nodes 3897 nps 1818 tbhits 2340 multipv 24 pv e3d3

Indeed you are correct. In the search, I do not probe TBs at the root. But I have implemented an "eval" command to perform a static eval, and it does return a draw score for this position:

[D]k1K5/8/8/8/4pP2/4Q3/8/8 b - f3 0 1

Very well spotted!

jm

[syzygy]

I believe the problem is in gtb-probe.c, function tb_probe_, line 1676, which does:

Code: Select all

/* chooses to ep or not */
dtm = bestx (stm, epscore, dtm);

This logic is incorrect if the non-ep score is a stalemate score, because in that case you can not choose to use the stalemate score, you are forced to play the losing ep capture.

I was just looking at that right now. Not sure that line is wrong, per se, but wrong in the context. Yes, there is a bug in the probing code, not in the tables. Basically, it chooses the score of the position before ep rather than executing the ep. But it is not possible because of a stalemate. Duh. It is equivalent of choosing a nullmove in this position.

At least in theory this oversight (when it is present in the generator and not just the probing code) can result in incorrect table entries, namely if there is a position in which the only winning move is a double pawn move that brings the opponent in a position in which his only legal move is the (losing) ep capture.

I'm reasonably sure there do not exist such positions with 5 pieces, but there might exist one with 6 pieces. I have tried to construct one in the past, but I don't remember if I found one or not. I think there are at most a few candidates.

Maybe someone wants to give it a try and find such a position.

[Arpad Rusz]

At least in theory this oversight (when it is present in the generator and not just the probing code) can result in incorrect table entries, namely if there is a position in which the only winning move is a double pawn move that brings the opponent in a position in which his only legal move is the (losing) ep capture.

I'm reasonably sure there do not exist such positions with 5 pieces, but there might exist one with 6 pieces. I have tried to construct one in the past, but I don't remember if I found one or not. I think there are at most a few candidates.

Maybe someone wants to give it a try and find such a position.

There is such a position (but with a checking first move):
[D]8/8/q7/k1K5/p7/8/1P6/7R w - - 0 1