See also my long post from today in this thread, to which you already replied. Whatever you choose from the given alternatives, you must change *something* at least because your current code as you have quoted it above is wrong w.r.t. mate scoring ...
fair enough, all I am saying is that evidence suggests something other than the actual handling of mate scores as the cause of the problem. Given that I wasn't even storing mate scores, I now think that conclusion makes sense. Do we agree on that? I'm gonna try and find an example of a situation that does not involve a mate score.
I'm gonna review a few things here...
1. For my current tests, I'm not using iterative deepening, nor am I storing hash moves. Nor am I doing qSearch. Strictly vanilla AlphaBeta pruning. with a transposition table.
2. The problem has only been observed with inexact depth matches in the hash probe (depth <= stored depth). Exact matches show no problems whatsoever.
3. My engine has passed all the perft tests and hash key verification tests I have thrown at it.
regards.
I think I have understood what you do.
However, I also think that you have not yet understood what HGM and I are saying. You are constructing a connection between your TT mate scoring bug, which really exists, and the "grafting" topic, which is a different issue - although TT-related, too - but does not necessarily involve any bug.
Sven
So the problem seems to be the storing of a position that is a mate, without storing a mate score. The co-incidence is that it happens to be a PV move for other reasons.
So how to know that it is a mate without going to the next ply to find out there are no responses? That is what I need to focus on
Sven Schüle wrote:You are constructing a connection between your TT mate scoring bug, which really exists, and the "grafting" topic, which is a different issue - although TT-related, too - but does not necessarily involve any bug.
Actually I do think these are two sides of the same coin. The point is that, because he messes with mate scores (subtracting the ply), he grafts wrong scores when he naively grafts them at another ply. If you mess up the mate scores, you should correct for the differece of the ply where you stored them, and the ply where you probe them, to make sure scores that were probed can be compared to scores that were obtained from search.
Sven Schüle wrote:You are constructing a connection between your TT mate scoring bug, which really exists, and the "grafting" topic, which is a different issue - although TT-related, too - but does not necessarily involve any bug.
Actually I do think these are two sides of the same coin. The point is that, because he messes with mate scores (subtracting the ply), he grafts wrong scores when he naively grafts them at another ply. If you mess up the mate scores, you should correct for the differece of the ply where you stored them, and the ply where you probe them, to make sure scores that were probed can be compared to scores that were obtained from search.
OK, I'm beginning to see your point.
This messing with the mate scores goes way back to the time when I was implementing my first recursive search, and I needed a way to make my engine choose a mate in N over a mate in N+1, even if the starting move comes later in the moveList. And I never bothered to re-examine that. Also, my strategy for determining check for the purpose of move legality checking is also quite old.
Now that I have things like iterative deepening and check extensions and hash tables etc at my disposal, I probably should rethink my approach to check and checkmate, rather than look for some kind of patch on the mate score.
So it'll take me a while longer, but I'll benefit in the end, touch wood.
Fguy64 wrote:Anyways, I appreciate the effort people make, but this thread has gone in a direction I never really intended. It started off with a question to Bob about what he meant by, and I paraphrase "I can't tell you how many times I'be tried to debug the problem, only to realise that it isn't a problem" I think, but I am not certain that he was referring to grafting, and at that time I was trying to ascertain whether his remark was relevant to the problem I was having, I thought it might be because my remarks clearly showed that while an incorrect score was being reported (a mate in 2 score instead of a mate in three score), the correct move was being played.
Initially you started this thread reporting problems related to hash table usage, where you got the same best move but different evaluations for few test positions after enabling hash table code.
Yesterday you brought up a different problem you have which seems to be clearly related to the special handling of mate scores. At least the problem looks very different from the previous one for me. You just used the same thread for discussing it.
So it surprises me that you say you didn't intend this thread to go into that direction. In fact it is kind of a different thread where people try to explain your wrong mate-in-2 score.
And if it turns out some day that both problems are in fact caused by only one single bug then you might still be ok with having discussed both in the same thread.
Sven
If you look again at my question to Bob from yesterday. I was in fact talking about a correct move but an incorrect eval, and thinking about grafting. It just so happened that the example I gave was a mating line, and the incorrect eval was a mate score. But at no time did I suggest or even suspect that the problem was with the handling of mate scores.themselves. That was HG's suggestion, several posts later. So my previous post to you stands.
The anomalies of grafting only occur with mate scores when the mate scores are not properly handled in the TT.
Let me summarize again all important (in my eyes) about mate scores and their handling.
Mate scores are completely different animal then usual scores (material + position) so we represent them usually as values close to some constant (let's call it CHECKMATE) big enough so that they are always greater (or less in the case of negative values) than any usual score. Let's emphasize once more that mate score is not just the score of actuall checkmate position but also evaluation of subtree whose PV ends with checkmate i. e. mate in N plies. So they can occur anywhere in the search tree and you have to treat them specially anywhere, not just in the checkmate node.
I see 2 different meaningfull ways how to treat them.
A) mate score in any given node of the tree means distance from this very node to the checkmate node at the end of PV of the given node.
This works following way: from the position of checkmate you return CHECKMATE constant and then during every transition up in the search tree you have to diminish (or augment) the score by 1. So you have to check all scores which you obtain in the search and when they are mate scores (it means for instance score > CHECKMATE - 500 or score < -CHECKMATE - 500) you have to adjust them (diminish positive and augment negative ones). However this is not enough because you have to adjust Alpha and Beta values as well if they happen to be mate scores, only just opposite way: augment positive and diminish negative values during transition down the searchtree (i. e. calling recursive AlphaBeta search). This is little bit messy but big advantage here is that you can store/retrieve such mate scores to/from TT without any adjusting.
B) mate score in any given node of the tree means distance from the root to the checkmate node at the end of PV of the given node.
This works like this: from the position of checkmate you return CHECKMATE constant - current searching depth. You do not have to modify neither mate scores nor Alpha/Beta when traversing the tree. But you have to adjust them when storing/retrieveing to/from TT so that the value which you are storing has meaning given above sub point A).
And that is all. Does anybody know any better way of dealing with mate scores?
My engines basical use method A, with the slight difference that they only adjust the negative scores. There is no need to adjust both; what is mate-in-N should become mate-in-(N+1) two ply earlier. So the score is only adjusted on one of the two plies. If you would adjust on both plies, some of the scores would never be used. (E.g. if -32000 is checkmated, +31999 would be mate-in-1, -31998 would be mated-in-1, and -31999 would not be used.)
I can add that adjusting alpha & beta in the opposite way (i.e. diminish them if they are sufficiently negative), will automatically cause mate-distance pruning: the search wil realize that if you have already found a mate-in-N at one level, you will have to find a mate-in-(N-1) two ply later to compete with it (so actualy a faster one), and expresses this by raising the 'beta bar'. In the end (i.e. deep enuogh in the tree) this will lead to a situation where beta will shift to below -INFINITY, the value at which you initialize your bestScore variable (when not in QS) before you have searched any moves. So you will get a beta cutoff before having searched any moves, similar to the stand-pat cutoff in QS. The only difference is that you don't fail high by standing pat, but by resigning. Even an immediate resign (which is what the -INFINITY score corresponds to; you cannot do worse than that) will be good enough to make this branch unacceptably poor for the opponent, because you already survived longer than he has to let you live in other branches. So resigning is by far the cheapest way to 'defeat' his goals.
So in method A you get mate-distance pruning for free.
hgm wrote:My engines basical use method A, with the slight difference that they only adjust the negative scores. There is no need to adjust both; what is mate-in-N should become mate-in-(N+1) two ply earlier. So the score is only adjusted on one of the two plies. If you would adjust on both plies, some of the scores would never be used. (E.g. if -32000 is checkmated, +31999 would be mate-in-1, -31998 would be mated-in-1, and -31999 would not be used.)
I can add that adjusting alpha & beta in the opposite way (i.e. diminish them if they are sufficiently negative), will automatically cause mate-distance pruning: the search wil realize that if you have already found a mate-in-N at one level, you will have to find a mate-in-(N-1) two ply later to compete with it (so actualy a faster one), and expresses this by raising the 'beta bar'. In the end (i.e. deep enuogh in the tree) this will lead to a situation where beta will shift to below -INFINITY, the value at which you initialize your bestScore variable (when not in QS) before you have searched any moves. So you will get a beta cutoff before having searched any moves, similar to the stand-pat cutoff in QS. The only difference is that you don't fail high by standing pat, but by resigning. Even an immediate resign (which is what the -INFINITY score corresponds to; you cannot do worse than that) will be good enough to make this branch unacceptably poor for the opponent, because you already survived longer than he has to let you live in other branches. So resigning is by far the cheapest way to 'defeat' his goals.
So in method A you get mate-distance pruning for free.
It is funny. I used method A before switching to B thus I should know it well yet I have somehow never realized that if you can checkmate your opponent in N plies then you are never going to checkmate him in (N-1) or (N+1) plies just because you are not the side to move on the end of the line .
Thanks for interesting remarks.
jwes wrote:The anomalies of grafting only occur with mate scores when the mate scores are not properly handled in the TT.
That I did not know. If someone said that earlier I missed it. It certainly explains a bit of the misunderstanding.
It does not imply that "anomalies of grafting do only occur with mate scores" but "anomalies of grafting as far as mate scores are involved do only occur when ... not properly handled ...".
And I would add that improper handling of mate scores in the TT is not much different from any other bug, it simply causes wrong behaviour, too.
Sven Schüle wrote:You are constructing a connection between your TT mate scoring bug, which really exists, and the "grafting" topic, which is a different issue - although TT-related, too - but does not necessarily involve any bug.
Actually I do think these are two sides of the same coin. The point is that, because he messes with mate scores (subtracting the ply), he grafts wrong scores when he naively grafts them at another ply. If you mess up the mate scores, you should correct for the differece of the ply where you stored them, and the ply where you probe them, to make sure scores that were probed can be compared to scores that were obtained from search.
OK, I'm beginning to see your point.
This messing with the mate scores goes way back to the time when I was implementing my first recursive search, and I needed a way to make my engine choose a mate in N over a mate in N+1, even if the starting move comes later in the moveList. And I never bothered to re-examine that. Also, my strategy for determining check for the purpose of move legality checking is also quite old.
Now that I have things like iterative deepening and check extensions and hash tables etc at my disposal, I probably should rethink my approach to check and checkmate, rather than look for some kind of patch on the mate score.
So it'll take me a while longer, but I'll benefit in the end, touch wood.
You keep insisting on giving the impression of having *partially* understood the comments
Your way of handling mate scores for the purpose of tree search is a correct method, you return something like -(10000-ply) at a "checkmated" node and propagate that up the tree as far as necessary. Nothing wrong with that, even after introducing iterative deepening and all the other stuff you mentioned. No need to change it. It corresponds to "method B" from Filip's recent post.
Storing values in the TT and retrieving from there is the only thing where your current code needs a change for the special case of mate scores, no matter if the score means "mated now", "mated in N plies", or "mate in N plies". This is not "some kind of patch" but an important and required bugfix based on a common and widely accepted method which also matches your mate score handling in the search. You just can't do without it, or you choose the alternative way given on the Bruce Moreland pages to which I already pointed you in another post.
Did we manage to create enough confusion for you? If not, say "more, please!"
.
