Stan Arts wrote:...
Respectively mate in 28, 27, 24, 22, 21, 20, 19, 18, 17, 17, 17 and finally 16 is found which I suppose is the correct solution. Robert Hyatt gets a point because I must admit my hash pv's are pretty crappy here.
Mate in 15 and the PV is 21 moves long??? Crappy software I wrote.... The only excuse I have is that the engine is in an heavy rewrite stage, so I have broke it badly... Go figure what's wrong with that...
Stan Arts wrote:...
Respectively mate in 28, 27, 24, 22, 21, 20, 19, 18, 17, 17, 17 and finally 16 is found which I suppose is the correct solution. Robert Hyatt gets a point because I must admit my hash pv's are pretty crappy here.
Mate in 15 and the PV is 21 moves long??? Crappy software I wrote.... The only excuse I have is that the engine is in an heavy rewrite stage, so I have broke it badly... Go figure what's wrong with that...
I bring the search function, I remove all the pruning and instead of calling the qsearch I generate a random number and I trasform in a Perft.
In this way I can check the move ordering, that every move would played one single time and no move would skipped.
I can check hash, killer and history and I debug the pruning function later.
PK wrote:Most programs have incorrect search. TSCP has correct one, but its strength is about 1700 Elo. Techniques that can bring a program from 1700 to 3200 Elo progressively introduce more and more incorrectness. The real trick is to use these techniques in such a way that all their errors are hidden.
Having said that, there are things that a search has to do correctly. Beside things that You mentioned (repetitions, draw by 50 move rule), many search bugs can be detected by looking at simple endgames. If a program doesn't checkmate with K+R vs K, it's obviously broken. If it cannot see a simple win like 8/1k3ppp/8/5PPP/K7/8/8/8 w - - 0 1 it has a problem. If it gets these two right, harder stuff can be found in endgame manuals. In general endgames are good for exposing search errors.
Your examples are not very good. whether you can mate with K+R vs K is not always a search issue. And playing any type of game where evaluation is involved hides search issues.
Best way to debug is to run a few positions to a fairly minimal depth (say 6 plies) and dump the search tree. Then check what is searched in what order (hash move, which captures, killers, etc. See what was reduced or extended or pruned. Etc.
[D]8/8/8/8/8/5k2/6R1/7K w - - 0 1
I think that even with no evaluation today programs should search deep enough to see the shortest mate in KR vs K so if your program does not see the shortest mate in less than a minute or see forced mate but cannot win then it clearly has a problem in the search that you need to fix.
Stockfish(3 cores) without tablebases see mate in 16 in few seconds.
Senpai(3 cores) needs more time but still see mate in 16 after 25 seconds without using a fast computer.
The problem is that produces a very large tree. Which is NOT the kind of tree you want to debug with. For example, suppose you can not find the mate in 16, but can only see a mate in 17. How do you debug that with tens of millions of lines of output if you dump the tree?
My point was that there are some simple positions that can be used to track down these sorts of bugs MUCH easier than using positions that tend to produce enormous trees. I've been debugging a Hsu-like singular extension test and I ran WAC against the SE version to find the smallest tree where SE would kick in. Then I could dump the tree to a file, and look at what was going on to see if I like what I see or not. In a big tree, this is hopeless.
