SEE problem

[Tord Romstad]

Crafty used (at least in past and at least in some specific situations) opposite bitscans for opposite sides.

Yes, that would work, and could also help to make things like the move generator completely symmetric. The problem is that (as far as I know) there is no elegant, efficient and portable way to do a reverse bitscan. Perhaps I should test if bitscanning by table lookup is fast enough.

But I think the source of the problem that in general dynamic things like SEE should not be comparable with static features.

IMHO search depth extensions on passed pawn pushes would lead to more precise evaluation of highly important and highly dynamic nature of passed pawns. Passed pawn push is a rare and irreversible move, so the search can not to explode.

The search may not quite explode, but in my experience extending all passed pawn pushes inflates the search tree far too much (even if only half-ply extensions are used, and reduces the playing strength). Currently I only extend pawn pushes to the 7th rank by half a ply.

Moreover, even if extending all passed pawn pushes were not too expensive, it wouldn't be good enough, because we have to evaluate positions with passed pawns in the quiescence search. Searching all passed pawn pushes in the quiescence search would also not be sufficient, because we also have to evaluate passed pawns for the side not to move.

Tord

[Tord Romstad]

I handle this by only using is_attacked() in my passed pawn eval.

That's not quite good enough, IMO, because it doesn't handle the very common and important case of X-ray attacks of a friendly or enemy rook behind the pawn. Using is_attacked() with the passed pawn removed from the board during the sliding attack generation is probably worth a try, though.

No path-dependency problems, and also SEE doesn't really handle passed pawns correctly. Say white has a pawn on e6, and white and black both attack e7 with a bishop. SEE will say it's safe to advance, but I'd prefer to save the pawn.

I'm not sure I understand what you're saying, but I think a passed pawn which can safely advance is almost always stronger than a passed pawn which cannot advance, even in cases where you don't want to push the pawn quite yet.

Tord

[hgm]

Foe backing (mixed-color X-rays) is not something that can be solved by SEE, as it implies the presence of other tactics coupled to the exchange under study. A Rook backed by an enemy Rook, like here, means that these Rooks can also capture each other. Here it happens to be bad to start with the foe-backed Rook, because it allows passage of the enemy Rook behind it to the target square. (It was basically a soft-pinned piece.) But this is only true because the interposed Rook was defended, or because your first capture with the other Rook defends it. If the alternative leading capture would have been a Bishop, and the foe-backed Rook would not have been defended, then starting with that Rook would be mandatory: if you captured with the Bishop first, the opponent would not recapture that Bishop, but simply take the undefended Rook. OTOH, the pinning Rook behind it could also be undefended, and you would be wasting time on calculating if it is safe to push the passer, while you can take a hanging Rook.

This whole issue becomes even worse when opposing Q and B or Q and R are aligned, because it means one side or the other has a good capture on a Queen. But in any case it is crucial if the pieces X-raying each other (and thus attacking each other) are defended, and by what (to know what else might start attacking the target square of the SEE when one of the sides decides to interject a trade of these pieces).

The best solution seems to exclude foe-backed pieces from the exchange alltogether. Or, of course, flag the situation as one that cannot be handled by SEE.

[FrancoisK]

Hi Tord,

I have encountered the very same problem when checking my eval for symetry. However it seems i did not find it bothering enough to try to solve it.

For me the only issue is that it gives false alarms when in eval symetry checking mode. I do not think it would be worth fixing or search's sake.

I have considered just desactivating the SEE test for passes pawns pushing (with an #ifdef) when my checking eval symetry #define is on.

Cheers
François

[Gerd Isenberg]

Crafty used (at least in past and at least in some specific situations) opposite bitscans for opposite sides.

Yes, that would work, and could also help to make things like the move generator completely symmetric. The problem is that (as far as I know) there is no elegant, efficient and portable way to do a reverse bitscan. Perhaps I should test if bitscanning by table lookup is fast enough.

I guess you don't really want to scan reverse for black versus forward for white to move, but only a vertical flipped bitscan. Otherwise you have the same flipped position asymmetry with X-rays from left/right on ranks. It is obvious that SEE fails up and then by design - but it should fail (or not) in flipped symmetrical manner.

Your idea to make an own semi-reverse bitscan by lookup (probably 16-bit lookups) is fine, because you may scan from eighth rank to first rank, but with same forward file-order. Alternately as mentioned, for that SEE-purpose it is quite simple to split the bitboard by square in three disjoint subsets - all ranks above, current rank and all ranks below and to swap below/above on color to move. This can even be done branchless with no memory effort and no special endian conversion instructions at all.

As mentioned, to look for batteries and X-rays in advance - with one additional magic-lookup for rook- and bishop attacks from the exchanging square ...

Gerd

[mjlef]

How about when there are two pieces attacking a square, always play out the piece that is not inline with an opponent sweep piece first fix this? I know it is a little more work, but it would happen very rarely.

The problem can exist with pawns as well (where a dicovered attack from a pawn move by a bishop can appear).

The main problem is not really for move ordering, it is for the qsearch where you can toss a whole move incorrectly.

There are a lot of shortcuts you can take in an SEE function. if the moving piece is <= value of the initial piece on the sqaure then you know the capture will be atleast equal, since you cans top after the first recapture ahead or equal. If the mover is > the taken piece, you can return a loss if the square is protected by an opponent pawn. I found just about 3-4 rules cover 85% or so of the SEE situations and make it lighting fast. You you just write out the possible capture sequences, some interesting facts concerning relative weights show up.

Mark

[Zach Wegner]

That's not quite good enough, IMO, because it doesn't handle the very common and important case of X-ray attacks of a friendly or enemy rook behind the pawn. Using is_attacked() with the passed pawn removed from the board during the sliding attack generation is probably worth a try, though.

I should have been clearer. I actually use floodfilling here, and the pawn is of course removed. I also get the attacks for every square up to and including the promotion square, similarly to the way Ed describes the Rebel eval.

No path-dependency problems, and also SEE doesn't really handle passed pawns correctly. Say white has a pawn on e6, and white and black both attack e7 with a bishop. SEE will say it's safe to advance, but I'd prefer to save the pawn.

I'm not sure I understand what you're saying, but I think a passed pawn which can safely advance is almost always stronger than a passed pawn which cannot advance, even in cases where you don't want to push the pawn quite yet.

That is true, but the distinction that I like to make is between a passed pawn that can advance "safely" as determined by SEE, but which could actually be unsafe because the opponent could sacrifice a minor or a rook, and a pawn that can advance safely because there are no opponent pieces attacking the next square at all.

I guess that's still rather unclear, I hope you can understand what I'm getting at.

[hgm]

How about when there are two pieces attacking a square, always play out the piece that is not inline with an opponent sweep piece first fix this?

The problem is that in real life, (i.e. were the opponent can play any capture, in stead of just the ones to a predetermined square), if you capture with the other pieces first, the opponent will respond by capturing your piece that was in line from behind. So counting on the idea that you could use that piece second is simply fooling yourself. It will no longer be there, as he will trade it first.

[Uri Blass]

This morning, I stumbled upon a symmetry bug in my evaluation function, in the following position:
[d]4r2k/pp4p1/2p1P3/2Q5/3p4/P2Br3/1PP4q/1K2R3 b - -
The reason turned out to be that the passed pawn on e6 is evaluated differently in the mirrored position. One of the factors considered by my passed pawn evaluation is whether the pawn can safely advance. I use the static exchange evaluator to test this. I discovered that in the above position, the pawn push e6-e7 is incorrectly classified as a non-losing move (SEE value 0), while in the mirrored position, the pawn push e3-e2 is correctly classified as losing (SEE value -1).

It's easy to see why this happens. My SEE function begins by finding the least valuable attacker. In the above position, there are only two attackers, and they are both of equal value (the rooks on e8 and e3). My SEE picks the rook on e3 first. Unfortunately, this means that the X-ray attack from white's rook on e1 gets uncovered, which causes in an incorrect SEE value. In other words, the SEE analyses the sequence e6-e7 Re3xe7 Re1xe7, and concludes that white is winning, and that e6-e7 is therefore a safe move.

Now, consider the mirrored position:
[d]1k2r3/1pp4Q/p2bR3/3P4/2q5/2P1p3/PP4P1/4R2K w - -
Here, my SEE picks the other rook first. The SEE analyses the sequence e3-e2 Re1xe2 Qc4xe2 Re6xe2 Re8xe2, and concludes that e3-e2 loses a pawn.

This is of course a highly annoying problem, and I see no elegant and efficient way to fix it. How do you make sure the correct attacker gets picked first in positions with X-ray-attacks?

Tord

I think that the solution is simply not to use SEE in the evaluation.
I wonder if you checked that SEE in the evaluation is better than alternatives that do not consider x rays.

If white has a passed pawn at e6 and white controls e7 you can give white some bonus in the evaluation.
If black also does not control e7 white can get another bonus.

Did you find that using SEE in the evaluation give better results relative to it?

Uri

[bob]

This is a known issue and this is just one of many ways it shows up. The problem is that our SEE is, by design, not a tree search, which would discover this trivially. But at a significant cost. But since SEE is already filled with errors (it ignores pinned pieces, overloaded pieces, discovered checks, etc) I don't think it is something worth worrying too much about...