Stockfish 1.7.1 displays weird mate scores

[zullil]

Thank you for pointing out this.
With your code change, "()" several positions are solved much faster.
Still I wonder why stockfish cannot solve this one.
[d]8/3Pb1p1/8/3P2P1/5P2/7k/4K3/8 w - - bm Kd3; id "E_E_T 007 - B vs L";
kind regards
Bernhard

Stockfish-1.7.1 (8 threads) does solve it, but not so quickly!

Code: Select all

Searching: 8/3Pb1p1/8/3P2P1/5P2/7k/4K3/8 w - -
infinite: 1 ponder: 0 time: 0 increment: 0 moves to go: 0
 2     +1.33   00:01      103 Kd3 Kg4 
 2     +1.49   00:01      148 Kf3 Kh4 
 3     +2.06   00:01      234 Kf3 Kh4 Ke4 
 4     +1.17   00:01      401 Kf3 Bd8 Ke4 Kg4 
 5     +1.54   00:01      705 Kf3 Bd8 d6 Kh4 Ke4 
 6     +1.41   00:01     1170 Kf3 Bd8 Ke4 Kg4 Ke5 g6 
 7     +1.41   00:01     3240 Kf3 Bd8 Ke4 Kg4 Ke5 Bc7+ d6 Ba5 g6 Bd8 
 8     +1.41   00:01     3943 Kf3 Bd8 Ke4 Kg4 Ke5 Bc7+ d6 Ba5 g6 Bd8 
 9  >  +1.49   00:01     5740 Kf3 Bd8 Ke4 Kg4 Ke5 Bc7+ d6 Ba5 Ke4 Bd8 Ke5 
 9  >  +1.58   00:01     6124 Kf3 Bd8 Ke4 Kg4 Ke5 Bc7+ d6 Ba5 Ke4 Bd8 Ke5 
 9     +1.58   00:01     7520 Kf3 Bd8 Ke4 Kg4 Ke5 Bb6 d6 g6 Kf6 Bd8+ Kxg6 Kxf4 
10  <  +1.41   00:01     9755 Kf3 Bd8 d6 Kh4 Ke4 Kg4 Ke5 g6 Ke4 
10  <  +1.25   00:01    12004 Kf3 Bd8 Ke4 Kg4 Ke5 Bb6 d6 g6 Kf6 Bd8+ Kxg6 Kxf4 
10     +1.29   00:01    15073 Kf3 Bd8 Ke4 Kg4 Ke5 Bb6 d6 Bd8 Ke4 Bb6 g6 Bd8 Ke5 
                              Bf6+ Ke4 Bd8 
11  <  +0.93   00:01    21143 Kf3 Bd8 d6 Kh4 Ke4 Kg4 Ke5 Bb6 g6 Bd8 f5 
11     +0.81   00:01    31775 Kf3 Bd8 d6 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bb6 Kd5 Kxf4 
12     +0.81   00:01    49298 Kf3 Bd8 Ke4 Kg4 Ke5 Bc7+ Ke6 Bd8 g6 Kxf4 Kf7 Ke5 
                              Kxg7 Kxd5 
13  <  +0.57   00:01    55385 Kf3 Bd8 Ke4 Kg4 Ke5 Bc7+ Ke6 Bd8 g6 Kxf4 Kf7 Bf6 
                              Ke8 Ke5 
13  <  +0.32   00:01    62513 Kf3 Bd8 Ke4 Kg4 Ke5 Bc7+ Ke6 Bd8 g6 Kxf4 Kf7 Bf6 
                              Ke8 Ke5 
13     +0.08   00:01    85316 Kf3 Kh4 Ke4 Kg4 Ke5 Bd8 d6 g6 Ke4 Bb6 d8=N Bxd8 
                              Ke5 Bb6 d7 Bd8 Ke4 Bb6 
14     +0.00   00:01   126625 Kf3 Kh4 Ke4 Kg4 Ke5 Bd8 d6 g6 Ke4 Bb6 d8=N Bxd8 
                              Ke5 Bb6 Ke4 Bd8 
15     +0.00   00:01   141272 Kf3 Kh4 Ke4 Kg4 Ke5 Bd8 d6 g6 Ke4 Bb6 d8=N Bxd8 
                              Ke5 Bb6 Ke4 Bd8 
16     +0.00   00:01   173150 Kf3 Kh4 Ke4 Kg4 Ke5 Bd8 d6 g6 Ke4 Bb6 d8=N Bxd8 
                              Ke5 Bb6 Kf6 Kxf4 Kxg6 Bd8 d7 Kg4 Kf7 Kxg5 Ke6 Kg4 
17     +0.00   00:01   216568 Kf3 Kh4 Ke4 Kg4 Ke5 Bd8 d6 g6 Ke4 Bb6 d8=N Bxd8 
                              Ke5 Kh4 Kd5 Kg4 Ke5 
18     +0.00   00:01   319242 Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
19     +0.00   00:01   401263 Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
20     +0.00   00:01   503897 Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
21     +0.00   00:01   643390 Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
22     +0.00   00:01   828428 Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
23     +0.00   00:01    1043k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
24     +0.00   00:01    1409k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
25     +0.00   00:01    1985k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
26     +0.00   00:01    2669k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
27     +0.00   00:01    3532k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
28     +0.00   00:02    4319k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
29     +0.00   00:02    5611k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
30     +0.00   00:02    7477k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
31     +0.00   00:03    9817k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
32     +0.00   00:03   13064k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
33     +0.00   00:04   17196k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
34     +0.00   00:05   22924k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
35     +0.00   00:06   29003k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
36     +0.00   00:08   37088k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
37     +0.00   00:09   44769k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
38     +0.00   00:11   54635k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
39     +0.00   00:14   69082k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
40     +0.00   00:16   83188k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
41     +0.00   00:21  110625k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
42     +0.00   00:28  144877k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
43     +0.00   00:34  183173k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
44     +0.00   00:42  232265k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
45     +0.00   00:50  279150k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
46     +0.00   01:00  336402k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
47     +0.00   01:15  435223k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
48     +0.00   01:30  523569k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
49     +0.00   01:54  709981k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
50     +0.00   02:17  868695k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
51     +0.00   02:34  971895k Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
52     +0.00   03:09    1215M Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
53     +0.00   03:41    1414M Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
54     +0.00   04:36    1833M Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
55     +0.00   05:24    2151M Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
56     +0.00   06:23    2553M Kf3 Kh4 Ke4 Kg4 Ke5 g6 Ke4 Bd8 Ke5 Be7 
56  >  +0.08   09:24    3908M Kd3 Kg3 Ke3 Kh4 Kd4 Kg4 Ke4 Bd8 Ke5 g6 d6 Kh5 Ke6 
                              Kg4 Kf7 Kxf4 Ke8 Kxg5 Kxd8 Kf6 Kc8 
56  >  +0.16   10:28    4357M Kd3 Kg3 Ke3 Kh4 Kd4 Kg4 Ke4 Bd8 Ke5 g6 d6 Kh5 Ke6 
                              Kg4 Kf7 Kxf4 Ke8 Kxg5 Kxd8 Kf6 Kc7 Kg7 d8=Q 
56  >  +0.32   12:03    5196M Kd3 Kg3 Ke3 Kh4 Kd4 Kg4 Ke4 Bd8 Ke5 g6 d6 Kh5 Ke6 
                              Kg4 Kf7 Kxf4 Ke8 Kxg5 Kxd8 Kf6 Kc7 Kg7 d8=Q 
56  >  +0.65   15:24    6884M Kd3 Kg3 Ke3 Kh4 Kd4 Kg4 Ke4 Bd8 Ke5 g6 d6 Kh5 Ke6 
                              Kg4 Kf7 Kxf4 Ke8 Kxg5 Kxd8 Kf6 Kc8 
56  >  +1.29   20:18    9756M Kd3 Kg3 Ke3 Kh4 Kd4 Kg4 Ke4 Bd8 Ke5 g6 d6 Kh5 Ke6 
                              Kg4 Kf7 Kxf4 Ke8 Kxg5 Kxd8 Kf6 Kc8 
56  >  +2.59   29:33   14805M Kd3 Kg3 Ke3 Kh4 Kd4 Kg4 Ke4 Bd8 Ke5 g6 d6 Kh5 Ke6 
                              Kg4 Kf7 Kxf4 Ke8 Kxg5 Kxd8 Kf6 Kc8 
56  >  +5.17   32:09   15852M Kd3 Kg3 Ke3 Kh4 Kd4 Kg4 Ke4 Bd8 Ke5 g6 d6 Kh5 Ke6 
                              Kg4 Kf7 Kxf4 Ke8 Kxg5 Kxd8 Kf6 Kc8 
56  > +10.34   40:18   19831M Kd3 Kg3 Ke3 Kh4 Kd4 Kg4 Ke4 Bd8 Ke5 g6 d6 Kh5 Ke6 
                              Kg4 Kf7 Kxf4 Ke8 Kxg5 Kxd8 Kf6 Kc8 
56  > +20.69   46:25   22737M Kd3 Kg3 Ke3 Kh4 Kd4 Kg4 Ke4 Bd8 Ke5 g6 d6 Kh5 Ke6 
                              Kg4 Kf7 Kxf4 Ke8 Kxg5 Kxd8 Kf6 Kc8 
56  > +41.37   52:44   26775M Kd3 Kg3 Ke3 Kh4 Kd4 Kg4 Ke4 Bd8 Ke5 Kg3 g6 Bh4 
                              Ke6 Kxf4 Kf7 Ke5 Kxg7 Kd6 Kh7 Kxd7 g7 Kd6 g8=Q 
                              Kc5 d6 Kb5 d7 Kc5 d8=Q Bxd8 Qf8+ 
56  > +82.75   57:12   29565M Kd3 Kg3 Ke3 Kh4 Kd4 Kg4 Ke4 Bd8 Ke5 Kg3 g6 Bh4 
                              Ke6 Kxf4 Kf7 Ke5 Kxg7 Kd6 Kh7 Kxd7 g7 Kd6 g8=Q 
                              Kc5 d6 Kb5 d7 Kc5 d8=Q Bxd8 Qf8+ 

Nodes: 73216001972
Nodes/second: 7716659
Best move: Kd3
Ponder move: Kg3

[zamar]

I've just sent Marco following patch which cures the first position so that future Stockfish versions won't suffer from this problem. Thanks again, Jouni!

rom a6cbb55275b9b7fde7bff71285d44ae743f07556 Mon Sep 17 00:00:00 2001
From: Joona Kiiski <joona.kiiski@gmail.com>
Date: Tue, 13 Apr 2010 18:25:50 +0300
Subject: [PATCH] Fix evasion pruning condition

Avoid incorrect mate scores in positions like

BK5/1R4b1/2k1Np2/3p3b/2p3pq/p1rB4/n2n1p2/8 w - -

Thanks for Jouni Uski for reporting the problem
---
src/search.cpp | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/src/search.cpp b/src/search.cpp
index efdb287..fff523c 100644
--- a/src/search.cpp
+++ b/src/search.cpp
@@ -1713,7 +1713,7 @@ namespace {

// Detect blocking evasions that are candidate to be pruned
evasionPrunable = isCheck
- && bestValue != -VALUE_INFINITE
+ && bestValue > value_mated_in(PLY_MAX)
&& !pos.move_is_capture(move)
&& pos.type_of_piece_on(move_from(move)) != KING
&& !pos.can_castle(pos.side_to_move());

[zamar]

In two other positions posted here, the output looks very ugly (I admit), but everything is fine.

When failing high, stockfish sees mate through transposition table. And those "mate in 60" and so on are "lowerbounds", and because mate scores are coming through TT, we can't check for fifty move rule.

However stockfish doesn't use transposition table in PV, so when it finally is able to get exact score it lacks depth to verify the mate (without the help of TT) to the end.

We could fix this by allowing use of TT also in PV (like in Crafty), but it could give rise to many other problems to deal with, so it's easier to live without.

[Uri Blass]

I've just sent Marco following patch which cures the first position so that future Stockfish versions won't suffer from this problem. Thanks again, Jouni!

rom a6cbb55275b9b7fde7bff71285d44ae743f07556 Mon Sep 17 00:00:00 2001
From: Joona Kiiski <joona.kiiski@gmail.com>
Date: Tue, 13 Apr 2010 18:25:50 +0300
Subject: [PATCH] Fix evasion pruning condition

Avoid incorrect mate scores in positions like

BK5/1R4b1/2k1Np2/3p3b/2p3pq/p1rB4/n2n1p2/8 w - -

Thanks for Jouni Uski for reporting the problem
---
src/search.cpp | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/src/search.cpp b/src/search.cpp
index efdb287..fff523c 100644
--- a/src/search.cpp
+++ b/src/search.cpp
@@ -1713,7 +1713,7 @@ namespace {

// Detect blocking evasions that are candidate to be pruned
evasionPrunable = isCheck
- && bestValue != -VALUE_INFINITE
+ && bestValue > value_mated_in(PLY_MAX)
&& !pos.move_is_capture(move)
&& pos.type_of_piece_on(move_from(move)) != KING
&& !pos.can_castle(pos.side_to_move());

I wonder if there is a measurable effect in elo for fixing this bug because it seems that the bug can cause stockfish to blunder and lose games instead of winning.

[d]2K5/BR4b1/3k1p2/3p3b/2pN2pq/p1rB4/n2n1p2/8 w - - 0 1

stockfish1.7 is going to choose the losing Rd7+ instead of the winning Nf5+

[Eelco de Groot]

Thank you for pointing out this.
With your code change, "()" several positions are solved much faster.
Still I wonder why stockfish cannot solve this one.
[d] 8/3Pb1p1/8/3P2P1/5P2/7k/4K3/8 w - - bm Kd3; id "E_E_T 007 - B vs L";
kind regards
Bernhard

Thanks Bernard for your message! I'm glad that somebody believes me I'm not really sure anymore now though, about what this is. I would rather not start compiling Stockfish myself at the moment, I do believe it is quite possible that you did see different behaviour though with the added () in your version. I would never have left this structure standing without the extra parentheses myself. But it seems unlikely this could be a Microsoft compiler bug so I'm not sure...

Thanks anyway Bernard,
Regards, Eelco

[Uri Blass]

In two other positions posted here, the output looks very ugly (I admit), but everything is fine.

When failing high, stockfish sees mate through transposition table. And those "mate in 60" and so on are "lowerbounds", and because mate scores are coming through TT, we can't check for fifty move rule.

However stockfish doesn't use transposition table in PV, so when it finally is able to get exact score it lacks depth to verify the mate (without the help of TT) to the end.

We could fix this by allowing use of TT also in PV (like in Crafty), but it could give rise to many other problems to deal with, so it's easier to live without.

I think that it is better if at least you fix the choice of the move

In other words if you have big fail high(not fail high by 0.1 pawns but by more than 1 pawn) trust it for the choice of the move even if it has fail low after it.

Otherwise stockfish may blunder by playing Qc5+ that draws instead of winning in the third position inspite of seeing the win and I can imagine that there are also practical cases when the big fail high does not lead to mate but stockfish blunder by not trusting it.

I wonder how often stockfish has a big fail high that is a bad move and note that even if a big fail high from 0.00 evaluation is practically for a move that leads to a draw by the 50 move rule you then you still lose nothing from trusting the fail high.


Uri

[Eelco de Groot]

I'm only half sure, but I just encountered this new code in 1.7.1 line 2319 in search.cpp
[...]
Extra () around the Boolean.

I could be wrong but it looks a bit suspicious and it also involves the matescores, could it be correct, that only the first part of this boolean is returned at the moment?

No, both versions are equivalent, it is like writing either "return a && b && c;" or "return (a && b && c);". The "return" statement does not require parentheses around the expression that follows, so they are optional. And "return a" itself is also not an expression because "return" is not an operator, so "return a && b && c;" is *not* evaluated like this (wrong code): "(return a) && b && c;".

If adding these parentheses changes behaviour then I would believe that there is something else being changed, too.

Sven

Hi Sven,

Yes, that's what I thought, that they should be logically equivalent. I don't quite trust the compiler on doing this exactly the same though because I had a similar issue with a condition being overlooked quite a while ago and it was not even a complicated line. But maybe I remembered wrong and there was a logical difference then. I did see Marco change

Code: Select all

bool deepChecks = depth == -OnePly && staticValue >= beta - PawnValueMidgame / 8;

to

Code: Select all

bool deepChecks = (depth == -OnePly && staticValue >= beta - PawnValueMidgame / 8);

which I think should also be equivalent logically? As I said I would never have entered the first line without extra parentheses myself but I trust nobody will believe there is a difference. I have too many windows opened to start testing or compiling at the moment. I did seem to spot the correct line that Joona is making changes to though!...

...I'm trying to make a version of Rainbow Serpent with the new search but adding all changes by hand, so it is going to take a while and there is no hurry because I'm not sure I should release it if it is not different enough.

If I start compiling Stockfish now, I'm sure I'm going to make things too complicated and mess it up even more than this thread.. With already three instances of WinMerge, one Shredder UCI, one Scid and one ChessDiag V1.01 open... Sorry guys if my analysis of the problem will be rejected or is confusing new people trying to compile Stockfish , but I don't think I can compile it at the moment

Eelco

[zamar]

I wonder how often stockfish has a big fail high that is a bad move and note that even if a big fail high from 0.00 evaluation is practically for a move that leads to a draw by the 50 move rule you then you still lose nothing from trusting the fail high.
Uri

Last time I tested enabling TT probe in PV (which fixes this behaviour), the difference was not measurable. And I've never so far observed this behaviour in real games, so there is no need to fix this. Accepting TT scores in PV causes so many difficult problems to deal with that I don't want it.

[Sven]

I did see Marco change

Code: Select all

bool deepChecks = depth == -OnePly && staticValue >= beta - PawnValueMidgame / 8;

to

Code: Select all

bool deepChecks = (depth == -OnePly && staticValue >= beta - PawnValueMidgame / 8);

which I think should also be equivalent logically?

This is a different animal. If someone is not quite sure whether the = or the == has stronger binding then it is absolutely ok to use parentheses for safety. But in "return expr;" operator precedence is not involved; it does not hurt to put parentheses around (expr) but one knows immediately that they are redundant and cannot change behaviour.

In fact == has higher precedence than = and also than && so that the parentheses do not change the result here but some people (including myself, and also many potential readers of foreign code) have to look that up in a table, which is never necessary in case of "return".

Nevertheless, it may be a matter of taste and style, of course.

Sven

[Ralph Stoesser]

Code: Select all

bool deepChecks = (depth == -OnePly && staticValue >= beta - PawnValueMidgame / 8);

is easier to read.

Adding extra parentheses to the return statement would not be easier to read.
Redundant parentheses are in most cases bad for clarity.
The above example is an exception from this rule.

All IMHO. (Beauty is in the eye of the beholder)