As a quick synopsis: Black plays the Pirc with 2. ...f5. Over time, white builds up a significant positional advantage. At move 50, most engines score the position at around +7 in favour of white, as does my own engine. However, after move 50, all my engine does is shuffle its rook back and forth, making no progress or pawn breaks, or anything that would be necessary to convert its huge advantage. Every now and then, it notices the 3-move repetition rule and shifts its king, and the game is eventually adjudicated as a draw.
Has anyone else experienced this problem? Is the lack of progress a problem with search or evaluation?
Perhaps looking at the PVs the engine prints can give you a clue. I guess it would see the gain of somematerial,otherwise it would not get to +7 in this position. There could be a backlash after the actual capture encouragig the engine togain the material only just before the horizon.
A delayed-lossbonus could help to cure this. 50-move blackmailmight hel as well.
It's an evaluation problem primarily (specifically that positions that constitute progress are evaluated less favorably than positions that aren't progressing), though one that in this case will probably be fixed by search. The problem you've got here is that progress first involves a step or two backwards before you can improve the static evaluation. i.e. h5 or f5 has to be played which will improve black's piece activity or possibly even sac a pawn but allow you to move forward. a deeper search will eventually overcome this evaluation valley. a more sophisticated evaluation may allow you to get out of the valley sooner.
the normal way that engines get past these sorts of situations if they can't solve it directly is by 50 move rule proximity. At some point seeing a large advantage and a 50 move rule draw coming your engine should play f5 or h5 just to avoid the draw as it still leaves a sizeable +ive evaluation. It's ugly, but would've allowed you to convert the win in this case - why did your engine eventually prefer the repeat to pushing a pawn?
a bit more detail for the position in particular should you try to tackle it by eval (but you should try to fix the 50 move rule thing besides as it'll help these situations in general):
position after 51 ... Rh7
[d]1k2bn2/2p1p2r/2p1P1pN/P1P3P1/1P3P1P/3N4/4R3/2K5 w - - 1 52
what you need to recognise as progress here is that h5 gxh5 f5 is an improvement
[d]1k2bn2/2p1p2r/2p1P2N/P1P2PPp/1P6/3N4/4R3/2K5 b - - 0 53
after that there's many varied possibilities for further progress/counterattacks/etc so it's possible this may not be enough (i.e. some lines white sacs more pawns for a piece so if it likes its pawns so much that it doesn't want to win a piece that could again create a problem)
My engine comes up with that after searching to depth 20. It seems to want to re-position the knight...
I'm not sure if this is a search depth issue, or an issue with my pruning or something else. Most of the evaluation of 702 comes from positional aspects, e.g. the safe mobility of the black pieces, white space, etc.
I'm not sure how evaluation can be improved to encourage sac'ing a pawn, so I will experiment to see if it is search pruning that is the issue.
kbhearn wrote:a bit more detail for the position in particular should you try to tackle it by eval (but you should try to fix the 50 move rule thing besides as it'll help these situations in general):
position after 51 ... Rh7
[d]1k2bn2/2p1p2r/2p1P1pN/P1P3P1/1P3P1P/3N4/4R3/2K5 w - - 1 52
what you need to recognise as progress here is that h5 gxh5 f5 is an improvement
[d]1k2bn2/2p1p2r/2p1P2N/P1P2PPp/1P6/3N4/4R3/2K5 b - - 0 53
after that there's many varied possibilities for further progress/counterattacks/etc so it's possible this may not be enough (i.e. some lines white sacs more pawns for a piece so if it likes its pawns so much that it doesn't want to win a piece that could again create a problem)
After testing, my engine scores the first position at 725 statically, and the second one at 670. I'm not entirely sure what aspect of the evaluation I need to fiddle with; perhaps candidate passed pawns?
My engine comes up with that after searching to depth 20. It seems to want to re-position the knight...
I'm not sure if this is a search depth issue, or an issue with my pruning or something else. Most of the evaluation of 702 comes from positional aspects, e.g. the safe mobility of the black pieces, white space, etc.
I'm not sure how evaluation can be improved to encourage sac'ing a pawn, so I will experiment to see if it is search pruning that is the issue.
- you could try reducing the magnitude of the penalty the no-mobility black pieces get a bit to facilitate transitioning that advantage to a different advantage
- you could recognise black's new h-pawn as weak somehow and give it a small penalty compared to a stronger passed pawn (this one might be tough - it is weak but i have a hard time giving algorithmic conditions to identify how none of the black pieces can come to its aid)
- you could give a bonus to have a fluid duo of pawns well advanced (f5+g5 both able to advance)
- you could try adjusting your backward pawn penalty to make f4+h4 less valuable and thus more willing to be sacced
- you could try measuring the lockedness of the position and give a bonus to the side with the advantage for having said advantage in a less locked up position (perhaps a multiplier)
recognising earlier in the line that the position is good will probably help with pruning decisions as well so if it wants to make the move early
the plan in that pv isn't unreasonable. i have to think when it gets closer to the point where it would actually play f5 it must give up on it. when you keep advancing through the game moves does f5 keep getting delayed further and further (indicating a reluctance to give up some of its positional goodies giving it that giant score) - like what's the pv when it decides to play 56. Kb1 (as from then on it seems to stick to pointless shuffling)
I've been pulling down the weights for everything apart from pawn structure to try to experiment with these ideas. However, I noticed that when I disabled my PST/PCSQ completely, my engine found h4h5 in a heartbeat. (f4f5 at depths 1-4, d3e1 at depths 6-11, h4h5 further on).
Perhaps scaling down the weight of the PST tables (which are mainly used for opening play) as the game progresses is the way to go?
