Improving SF passer code

[Lyudmil Tsvetkov]

Now, you might take this as a continuation of the Lessons Learned thread, not to make it too long.

I hope not only SF but also others might find this interesting.

I have seen so many games SF lost against Komodo and other engines, especially in the endgame, because of lack of adequate connected passers knowledge, that it would be a mistake not to correct this.

Now, SF has very well tuned passers code in general.
It also has well tuned connected pawns code, which also influences passers.

So, SF sees connected passers only in the case, when the two or more passers are connected pawns according to SF definition.

SF definition for connected pawns is 2 pawns on the same rank on adjacent files, or a defended pawn.

Not quite so the official connected passers definition, you might want to check wikipedia for this. Wikipedia states, that connected passers are any two passers on adjacent files, regardless of whether they are on the same rank, or one of them is defended.

[d]6k1/8/8/5PP1/1P6/P7/8/6K1 w - - 0 1

so SF sees a3 and b4 above as connected passers, as a3 defends b4, and f5 and g5 as connected passers, as the two pawns are on the same rank next to each other

[d]6k1/8/1P6/8/6P1/8/P4P2/6K1 w - - 0 1
but does not see a2 and b6 as connected passers above, neither f2 and g4 as connected passers.

But both pairs really are connected passers, they are more useful than isolated passers, even though they are not next to each other currently, soon or at some point in time the less advanced passer may join the more advanced one

So my suggestion would be to somehow try to correct this behaviour, by giving some bonus, I am not sure how much, and not sure if rank-based or uniform, for the cases when the passers are on adjacent files, but not connected according to SF definition.

More or less, this would be equivalent to giving bonus to any passer with a friendly passer on an adjacent file, that is not connected according to SF definition, and not defending another passer. (the second condition is necessary, as, in the first diagram for example, the a3 pawn is not considered as a connected pawn, but actually functions as such, with possible advance on the next move to a4 to join b4)

I have really seen so many SF losses because of lack of such knowledge, that I would be surprised if this would not work, though of course the framework always does the opposite of what it is expected to do.
The bonus should be valid for both mg and eg.

How do you score your connected passers?

[mcostalba]

So my suggestion would be to somehow try to correct this behaviour, by giving some bonus, I am not sure how much, and not sure if rank-based or uniform, for the cases when the passers are on adjacent files, but not connected according to SF definition.

SF already gives a penalty to isolated pawns. Isn't it the same thing of what you are proposing? Penalize isolated passers is equivalent to bonus connected (as per your definition) passers.

[Lyudmil Tsvetkov]

So my suggestion would be to somehow try to correct this behaviour, by giving some bonus, I am not sure how much, and not sure if rank-based or uniform, for the cases when the passers are on adjacent files, but not connected according to SF definition.

SF already gives a penalty to isolated pawns. Isn't it the same thing of what you are proposing? Penalize isolated passers is equivalent to bonus connected (as per your definition) passers.

Hi Marco.

Isolated pawns are a different matter.
Here the problem is that the passers on both diagrams I posted deserve an almost equal bonus, but SF will not give connected pawns bonus to the passers on the second diagram.
But they are almost as dangerous as the passers on the first diagram.

Imagine the a2 pawn on the second diagram going to a5 and connecting to b6 passer, then it will receive connected passer bonus, but not before that, on a2. So we would like to take care of this discontinuity, as in most cases passers like a2 really join b6.

I posted this because I have seen too many games where SF mishandles this in the eg.

I remember Joerg pushing a patch along these or similar lines a year ago, that failed, but am not quite certain what the precise parameters and bonus of the patch were. Maybe Joerg could be so kind to refresh our memory a bit.

[Lyudmil Tsvetkov]

[d]6k1/P4p1p/8/2Pb4/8/4B1p1/8/6K1 w - - 0 1

For example, we have a position like this one.
Black has good winning chances, but, if SF does not consider the 3 black passers as connected, it will give very small score initially, as the pawns might not connect for quite a while.

There are many similar positions.

Maybe the bonus should be small, maybe just uniform, maybe just for the most advanced passer, seeing it for example as half connected with rank-based approach, etc. But some bonus is certainly due.

Actually, when I think now, I like more and more the idea of the most advanced passer that is not connected, but has another friendly passer on adjacent file, like g3 above, receiving 1/2 or 1/3 the bonus of normal connected passer in terms of rank. It seems just natural.

[mcostalba]

Isolated pawns are a different matter.

In SF a pawn is considered isolated when there are no friendly pawns on adjacent files.

Your new definition of connected passer is "that connected passers are any two passers on adjacent files, regardless of whether they are on the same rank, or one of them is defended."

This translates in "connected passers are any two non isolated passers"

[Lyudmil Tsvetkov]

Isolated pawns are a different matter.

In SF a pawn is considered isolated when there are no friendly pawns on adjacent files.

Your new definition of connected passer is "that connected passers are any two passers on adjacent files, regardless of whether they are on the same rank, or one of them is defended."

This translates in "connected passers are any two non isolated passers"

Right, but the point is to see that connected non-isolated passers according to general wiki definition, even when not connected according to SF definition, i.e. next to each other, are almost as strong as connected non-isolated passers according to SF definition.

[d]6k1/P4p1p/8/2Pb4/8/4B1p1/8/6K1 w - - 0 1

the problem is, what is the connected passer bonus for g3 above, as f7 and h7 passers make it such, but this is not seen in SF code

[mcostalba]

the problem is, what is the connected passer bonus for g3 above, as f7 and h7 passers make it such, but this is not seen in SF code

In SF there is a penalty for isolated pawns, so in your example:

[d]6k1/P4p1p/8/2Pb4/8/4B1p1/8/6K1 w - - 0 1

a7 pawn _already_ now gets less bonus than in this case:

[d]6k1/P4p1p/8/1P1b4/8/4B1p1/8/6K1 w - - 0 1

Becuase in first case it is isolated (and get penalized), in the second case it is not due to pawn in b5

[Lyudmil Tsvetkov]

the problem is, what is the connected passer bonus for g3 above, as f7 and h7 passers make it such, but this is not seen in SF code

In SF there is a penalty for isolated pawns, so in your example:

[d]6k1/P4p1p/8/2Pb4/8/4B1p1/8/6K1 w - - 0 1

a7 pawn _already_ now gets less bonus than in this case:

[d]6k1/P4p1p/8/1P1b4/8/4B1p1/8/6K1 w - - 0 1

Becuase in first case it is isolated (and get penalized), in the second case it is not due to pawn in b5

Right Marco, but the point is to see that the b5 pawn on your second diagram is almost as strong as if it were directly on b6; in some cases search will see easily this pawn can go to b6, but in others not.

So, you give a different bonus for 2 passers that are on adjacent files and next to each other, and 2 passers that are on adjacent files and not next to each other, even though none of the pairs contains isolated passers.

You would like to score passers on adjacent files next to each other and passers on adjacent files that are not next to each other almost the same.

I know this will not work on the framework, nothing works, but the point is that SF loses games because of that, and there should be a very subtle way of tuning/fixing that, currently no one can think of...

[Lyudmil Tsvetkov]

I post a subthread here, not to post a new thread.

Well, SF is trying desperately now to simplify its backward pawns, making them have exactly the same values as isolated pawns.
Whether this will succeed or not, is not that important, as at some point SF will have to go back and revise its backward code, making it subtler.

After readin how exactly SF scores backward pawns, here my suggestion for a first step towards improvement of SF backward pawns.

Well, SF scores backward pawns with ooposed flag, and that is very good and natural. The way it should be. It scores only pawns into the own camp, which is not perfect, but still almost optimal and very good.

Bad thing: it does not differentiate between ranks. Well, there might be very small distinction in terms of the penalty depending on ranks, but there is.

I see it as follows in SF code:

- keep current values for the 3rd rank
- increase with 1/8 the penalty for the 2nd rank
- decrease with 1/8 the penalty for the 4th rank

Second bad thing: SF thinks that connected pawns can not be backward, but this is not true.

Suggestion:

- penalise backward pawns when connected, only for the 2nd and 3rd ranks
- in this case, penalty should be 1/2 or so of the normal penalty for the respective rank, when the pawn is not connected

[d]6k1/1pp3p1/p4p2/P3p3/4P3/8/8/6K1 w - - 0 1
both b7 and f6 are backward, but with half the usual penalty for the respective ranks

I think such rules would be very useful, as backward pawns are an important eval factor, and you can not simply discard them like that.

Any feedback from SF?

[Lyudmil Tsvetkov]

Forgot to mention: when a pawn is backward and connected, it should be scored only for the 2nd and 3rd ranks, but not for the 4th rank, this is very important, this is because of the usual piece placements.