tt and mate scoring bug

[ericlangedijk]

Since the beginning my engine had a matescoring bug.
I hunted the source of that problem: using a tt score as an eval inside the search instead of the static evaluation.
I got that idea from (many) other chess engines.
This eval is then used for the logic of razoring, reversed futility pruning, nullmove pruning.
Something like this:

Code: Select all

            // use tt score as a better eval.
            if (ttHit
                && (ttEntry.bound == EXACT
                    || (ttEntry.bound == LOWER_BOUND && ttData.score >= stack->eval)
                    || (ttEntry.bound == UPPER_BOUND && ttData.score <= stack->eval)))
                stack->eval = ttEntry.score;

When I use this logic in my engine sometimes with deep mates it reports a pv with a ridiculous mate score far beyond the max searchdepth.

Any wizard around here who knows why, what and how to solve this?
(I saw 1 or 2 engines during my play-tests having the same bug).

[hgm]

Mate scores beyond the search depth are a normal phenomenon (known as 'grafting') when you allow overdeep hash hits to satisfy search requests.

E.g. in the following position
[d]8/8/8/3k4/R7/2K5/8/8 w - - 0 1
Fairy-Max gives the following analysis:

28 #13 156.8M 1:09.35 a4a5 d5d6 c3d4 d6e6 a5d5 e6f6 d5e5 f6g6 d4e4 g6f6 e4f4 f6g6 e5f5 g6h6 f5g5 h6h7 g5g2 h7h6
27 #13 100.0M 0:44.62 a4a5 d5d6 c3d4 d6e6 a5d5 e6f6 d5e5 f6g6 d4e4 g6f6 e4f4 f6g6 e5e6 g6g7 f4g5 g7f7 e6e5
26 #13 75.4M 0:33.59 a4a5 d5d6 c3d4 d6e6 a5d5 e6f6 d5e5 f6g6 d4e4 g6f6 e4f4 f6g6 e5e6 g6g7 f4g5 g7f7 e6e5
25 #14 37.8M 0:17.57 a4a5 d5e6 c3d4 e6d6 a5c5 d6e6 c5d5
24 #15 19.7M 0:09.25 a4a5 d5d6 c3d4 d6e6 a5a6 e6f5 a6c6 f5f4 c6c5 f4f3 c5f5 f3g4 d4e4 g4g3 e4e3 g3g4 f5b5
23 #17 12.9M 0:06.06 a4a5 d5d6 c3d4 d6e6 a5a6 e6e7 d4d5 e7f7 d5e5 f7e7 a6a7 e7d8 e5d6 d8e8 a7b7 e8f8 d6e6 f8g8 b7b6
22 #19 8.86M 0:04.10 a4a5 d5d6 c3d4 d6e6 a5a6 e6f5 a6b6 f5f4 b6b5 f4f3 b5f5 f3g4 d4e4 g4g3 e4e3
21 #24 7.23M 0:03.10 a4a5 d5d6 c3d4 d6e6 a5a6 e6e7 d4d5 e7f7 d5e5 f7e7 a6d6 e7f7 d6e6
21 #27 5.78M 0:02.42 a4c4 d5e5 c4c5 e5e6 c3d4 e6d6 c5c3 d6e6 c3c6 e6d7 d4d5 d7e7 d5e5 e7f7 c6c3
20 #27 4.62M 0:01.90 a4c4 d5e5 c4c5 e5e6 c3d4 e6d6 c5c3 d6e6 c3c6
20 #28 4.53M 0:01.85 a4b4 d5e5 b4b5 e5f6 c3d4 f6e6 b5c5 e6d6 c5c3 d6e6 c3c6 e6d7
20 +4.62 4.22M 0:01.75 a4a5 d5d6 c3d4 d6e6 a5a6 e6e7 d4d5 e7f7 d5e5 f7e7 a6a7 e7d8 e5e6 d8c8 e6d6 c8b8 a7c7 b8a8 d6e5 a8b8
19 +4.56 3.76M 0:01.54 a4a5 d5d6 c3d4 d6e6 a5a6 e6f5 a6d6 f5f4 d6f6 f4g5 f6f2
19 +4.55 2.70M 0:01.09 c3d3 d5e5 a4a5 e5e6 d3d4 e6f6 a5a6 f6f5 a6c6
18 +4.55 1.76M 0:00.71 c3d3 d5e5 a4a5 e5e6 d3d4 e6f6 a5a6 f6f5 a6c6
17 +4.54 1.29M 0:00.51 c3d3 d5e5 a4a5 e5e6 d3e4 e6d6 e4f5 d6c6 f5e5 c6d7 a5c5 d7e7 c5c7 e7e8 e5e4 e8f8 e4e5
16 +4.54 767095 0:00.31 c3d3 d5e5 a4a5 e5f4 a5c5 f4f3 c5c4 f3f2 c4f4 f2g2 d3e4 g2g3 e4e3 g3h3
15 +4.54 551486 0:00.21 c3d3 d5e5 a4a5 e5f4 a5c5 f4f3 c5c4 f3f2 d3e4 f2g2 c4c3 g2f2 c3b3
14 +4.54 359309 0:00.14 c3d3 d5e5 a4a5 e5f4 a5c5 f4f3 c5c4 f3f2 d3e4 f2g2 c4c3 g2f2
13 +4.52 209668 0:00.07 c3d3 d5e5 a4a5 e5f4 a5b5 f4f3 b5b4 f3f2 d3e4 f2g3 e4e3
12 +4.52 149534 0:00.06 c3d3 d5e5 a4a5 e5f4 a5b5 f4f3 b5b4 f3f2 d3e4 f2g3 e4e3
11 +4.51 93373 0:00.03 c3d3 d5e5 a4a5 e5f4 a5b5 f4g3 d3e4 g3g4 b5c5 g4g3 e4e5
10 +4.51 52561 0:00.01 c3d3 d5e5 a4a5 e5f4 a5b5 f4g3 d3e4 g3g4 b5c5 g4g3
9 +4.50 37186 0:00.01 c3d3 d5e5 a4a5 e5f4 a5b5 f4g3 d3e4 g3g4 e4e5
8 +4.48 18682 0:00.00 c3d3 d5e5 a4a5 e5f4 d3d4 f4f3 d4e5 f3e3
7 +4.49 7438 0:00.00 c3d3 d5e5 a4a5 e5f4 d3d4 f4f3 d4e5
6 +4.48 4338 0:00.00 c3d3 d5e5 a4a5 e5f4 d3d4 f4f3
5 +4.47 1853 0:00.00 c3d3 d5e5 d3e3 e5f5 e3d4
4 +4.47 421 0:00.00 c3d3 d5e5 d3e3 e5f5
3 +4.47 191 0:00.00 c3d3 d5e5 d3e3
2 +4.46 14 0:00.00 c3d3 d5e5
1 +4.47 7 0:00.00 c3d3
0 #

So it already finds a way to force mate in 28 at a search depth of 20 ply, while King capture for such a mate would occur only on ply 57. So the grafting nearly triples the search depth.

[ericlangedijk]

Ok that is kinda understandable. My maximum searchdepth is 128. And I get scores beyond that, sometimes larger than mate in 200 ply.
So I am not sure if that is hurting or a bug. What I do know is that when the engine starts spawning these huge mates, a mate is always there and it gets smaller soon until the real mate in X.
I disabled the "use tt score as eval" for now and I did only notice an increase of around +15 ELO. And it seems more stable.

[Aleks Peshkov]

There are too distinct kinds of evaluation: 1) positional -- traditionally measured in centipawns and 2) mate scores counted in plies (halfmoves). Encoding two distinct value types into one number is source of confusion. Traditional static evaluation returns former (centipawn score), various search heuristicts like futility pruning also make sense only with former. Using positional evaluation when side to move king is in check is source of problems. Adding plies to centipawns and adding centipawns to mate scores is undefined operation and nonsense in strict sense.

Code: Select all

if (!inCheck() && ttScore.isEval()) {
    if (ttBound == ExactScore
        || (ttBound == FailHigh && eval <= ttScore)
        || (ttBound == FailLow && ttScore <= eval)
    ) {
        eval = ttScore;
    }
}

[ericlangedijk]

Yes. Matescores are a range of its own.

[syzygy]

I disabled the "use tt score as eval" for now and I did only notice an increase of around +15 ELO. And it seems more stable.

+15 ELO sounds like an acceptable trade-off