zullil wrote: Zenmastur wrote:
The point is that it doesn't take 26 plies to win the material back. So why does the engine fail to find the move until 26 plies are reached?
Forrest
You stated in another post that "I've been away from computer chess and chess in general for about 17 years." Many search enhancements have been introduced during that period. My understanding is that many extensions and reductions are now performed, so that at the completion of "depth" N, some root moves have been searched much more deeply than N plies, and perhaps some less than N. It's not just an alpha-beta search with iterative deepening anymore. And then there's the whole issue of multi-threading (which you wisely eliminated from consideration here by using a single core to test)...
Loius, I'm having trouble seeing what point you're trying to make with these comments. A little clarification would be helpful.
zullil wrote: ...
I'm sure Bob can comment more intelligently, but it seems to me that Crafty's behavior here is not the result of a "bug". The engine's "difficulty" finding the move is a consequence of the search design and evaluation parameters, which presumably have been tested to give strong results overall, though not on this one position.
I'm not going to get too hung up on the word(s) used to describe the problem. I'm of the opinion that the intent is to have the program find good moves as fast as possible. Nxe6 is a "good" move in this position. In fact, it might even be the "best" move. i.e. leads to the shortest win possible from this position. I have little doubt that it is a winning move. To that end, I'm conducting a little experiment, call it a pseudo-proof of the position. I would conduct a "true" proof by demonstration but Crafty isn't set up in such a way that it can perform the necessary steps. The first conjecture is that Nxe6 is a forced win for white. The pseudo-proof will consist of a book in which all possible moves by black are marked in such a way that under normal circumstances crafty won't play any of them. I.E. it has already played the line and found it wanting and has backed up a score to that position.
It would, of course, be best if all positions after black moves have mate in "x" scores. I'm not too hopeful I can achieve this simply because the routine that scores the positions isn't performing a true minimax on what it finds. To achieve this would require code changes and maybe even a change in the book structure itself. I'm not quite ready to tackle adding/changing the code just yet, so that will have to wait for another day.
The best way I can think to bring this about using the code, as is, is to construct a pgn file with game stubs that contain all responses to Nxe6. Create a book.bin from this file such that all moves are included. i.e. the command "book.bin create Nxe6reponses.pgn 999 1 0" should do the trick. Use this book to have Crafty play itself using each of the ending positions at least ten times. Take the pgn file with these games in it and make a book.bin from it. I.E. "book.bin create Nxe6games.pgn 50 1 0". This should give us a book with a few thousand position stemming from Nxe6 and all possible black responses. Now, just play tournaments with Crafty playing itself with all games starting at the position after Nxe6. If crafty has an "efficient" learning algorithm it shouldn't be long before it will have marked all positions after black moves as unplayable. I'm not sure if they will be marked as mates or not, you'll have to ask bob if this is possible with the current code, but if it is possible then assuming the time controls will allow a deep enough search you should see mate scores appearing in the book in short order. After a while all the scores will be mate scores if my conjecture is true.
It shouldn't take too much to change Crafty's code to make a "real" proof possible. But, I'm not ready to start on that just yet, I still have considerable catching up to do. In the mean time, I have completed a 100 game tournament as a starter file, created a book and then a 2,000 game tournament that I plan on making into the "main" book I'm going to use this evening. Then with this book in place the games will begin. I suspect that by the end of the weekend I'll have a "pseudo-proof".
The next conjecture to be tested is, "Is there any sequence of moves that leads to black retaining his material advantage to a depth of 26 plies including the first ply 16. Nxe6." I suspect that there is not, but this will have to wait, I think, until I'm competent enough to change the code and create an executable. This may take a while as I have other things on my to do list.
bob wrote:
For a search of > 5 seconds, you won't be able to measure the difference between a completely empty cache and one that has been "warmed up" by getting key parts of the program pre-loaded.
Finding Nxe6 after 70K nodes is not hard. But it introduces a measure of speculation that can be dangerous, because in 70K nodes there is NO way to see the ultimate win of material. You have to give big bonuses for the various pins, king can't castle, etc.
Let's avoid all speculation. The question then becomes "At what ply does Crafty (or any of the other programs) "believe" that it is getting it's material back AND can show it with a PV including the moves that regain the material?"
bob wrote:... Here's a good test to try:
take that position with one of the above programs (Rybka, etc) and look at the difference in score between Nxe6 and its second choice (perhaps use multi-PV). I just tried this in Crafty and the difference between the best move and Nxe6 is minuscule after a short search. The interesting question is, when does a program actually see it winning material. It takes Crafty to ply=20 (which is not very long). So not playing it at earlier plies is not what I would call a bug, I would call it just a circumstance. Many programs make the right move but for the wrong reason (make it accidentally). That doesn't always win games.
I agree that if it can't see that it gets it's material back prior to 20 plies, its just a circumstance. If it sees it at 20-plies, as you say, why wait until 26-plies to change the PV?
bob wrote:I was looking carefully, but more out of curiosity. Found an interesting thing (relative to crafty).
After Nxe6 fxe6 Qxe6 the score is +.74 (good for white). But after black plays Be7 to block the check, the score goes significantly negative. After some looking, it is the result of the bishop getting off of the d-file which now lets the white rook attack d7 (square adjacent to black king). I don't think this is reasonable. Tracy redid a lot of this a year or two ago, but I think this is a side-effect that we did not plan on. Got a bit of tuning to do as clearly moving the black bishop from d6 to e7 is NOT worth a pawn in terms of evaluation... and I am not searching tactically beyond that point for this analysis, so this is raw positional scoring that looks wrong.
More later but I suspect this is a bigger problem than originally suspected...
Well after that problem is fixed maybe it will shed further light on the situation.
I notice several people are making claims about how one program or another speculates. I'm not sure that is helpful. The big questions is when does it produce a PV that show its getting the material back or mating the black king. While we may not be a be to see some of the extensions at the end of PV are we can at least make educated guesses as to if the program "really" sees that its getting the material back or not. I don't think that short PV's that don't show this are of much value.
[D]rq2k2r/3n1ppp/p2bpnb1/8/Np1N4/1B3PP1/PP2Q2P/R1BR2K1 w kq - 1 16
16. Nxe6 fxe6 17. Qxe6+ Be7 18. Re1 Qd8 19. Bf4 Ng8
20. Rac1 Rc8 21. Rcd1 a5 22. Be5 Ngf6 23. Bd6 Ng8 24. Bc5 Rc7 25. Nb4
leads to this
[D]3qk1nr/2rnb1pp/1N2Q1b1/p1B5/1p6/1B3PP1/PP5P/3RR1K1 b k - 7 25
The best the computer can come up with is:
25. ... Kf8 26. Rxd7 Bxc5+ 27. Kg2 Qe8 28. Rxc7 Qxe6 29. Rxe6 Be8 30. Rc8 g6 31. Rexe8+ Kg7 32. Rxc5 h5 33. Rxa5 Ne7 34. Rxe7+ Kh6 35. Rb5 Rb8 36. Rxb4 Kg5 37. Kf2 Rd8 38. Rb5+ Kh6 39. Be6 Rd2+ 40. Kg1
This line seems to hold onto the material so a little more searching is required.
Regards,
Forrest
Only 2 defining forces have ever offered to die for you.....Jesus Christ and the American Soldier. One died for your soul, the other for your freedom.
