Kind of backward

[Lyudmil Tsvetkov]

I am certain square control is the least developed area in engine evaluation, almost non-existent.

And one can achieve quite a lot by applying good square control.

Similarly, backward pawns are one of the most neglected aspects of pawn features, although they are very important, so much important, that different kinds of backward pawns constitute more than 30%, and maybe even close to 50% of all available pawn features.

Of that enormous quantity of backward pawn features, engines consider only some 5 to 10%, so really a negligeable number.

You can do square control chiefly by pawn control and minor pieces control, similarly for backward pawns.

Here how you can do a basic, but very sound I think, form of considering a new kind of backward pawn, I will leave the definition to you.

Any pawn that has 2 enemy pawns to the left and to the right 2 ranks in front of it is considered backward.

For example, a pawn on the 2nd rank should have 2 enemy pawns on the 4th rank to the left and right in order to be considered backward, a pawn on the 3rd rank 2 enemy pawns to the left and right on the 5th rank, pawn on the 4th rank 2 enemy pawns to the left and right on the 6th rank, and a pawn on the 5th rank 2 enemy pawns to the left and right on the 7th rank.

There are no pawns of this type on the 6th rank, just on ranks 2-5.

[d]6k1/5p1p/2p1p3/6P1/p1pP4/4P3/PP3P2/6K1 w - - 0 1
b2 is backward on the 2nd rank, with enemy a4 and c4 pawns making it backward; d4 is backward on the 4th rank, with enemy c6 and e6 pawns making it such; and g5 is backward on the 5th rank, with enemy f7 and h7 pawns making it backward

You can not imagine how important is that, as pawns usually have to advance, and if they do not have that option or their advance is somehow thwarted, this is a bad condition, meaning a fair amount of passivity.
2 enemy pawns in front of such pawns mean precisely that: that the respective pawn's ability to advance is restricted.

[d]6k1/2p5/1p1p4/5p1p/1PP2P2/3P2PP/8/6K1 w - - 0 1
g3 is backward on the 3rd rank, with enemy f5 and h5 pawns making it such; c4 is backward on the 4th rank, with enemy b6 and d6 pawns making it such
Those pawns are restricted in their ability to advance, g3 can not move altogether, and c4 can move further only after an initial preparation, bringing the d3 white pawn to d4, so this deserves to be penalised somehow

Again, this is extremely important. The implication is that your reasonable pawn mobility is low, and low mobility always matters, even with pawns.

[d]6k1/6p1/5p1p/p1p4P/P2p1PP1/1P6/2P1P3/6K1 w - - 0 1
b3 is a backward pawn on the 3rd rank, with enemy a5 and c5 pawns making it backward; black d4 pawn is backward on the 5th rank, with enemy white c2 and e2 pawns making it backward; and g4 is of course and visibly backward on the 4th rank, with f6 and h6 pawns making it such

I would give that kind of backward pawn some 5-10cps standard values penalty, independently of the file and rank where it is, so a uniform penalty might apply.

Sorry, another stupid idea of mine, but just think about it: engines consider around 5 to 10% of all existing backward pawns, so really an enormous amount of reasonable eval is skipped even in top engines, while backward pawns are really very important.

Any thouhts on this?

[Lyudmil Tsvetkov]

Of course, such pawns are to be found only on files b through g.

[zullil]

Any thoughts on this?

Based on 5 seconds scrolling through your post---seems like a lot of redundancy with existing features, even with your "binders".

Shouldn't we be striving for independent (orthogonal ) features?

[Gerd Isenberg]

Any pawn that has 2 enemy pawns to the left and to the right 2 ranks in front of it is considered backward.

Not quite - even if the stop is controlled by two sentries, there might be two helpers controlling the stop as well, making an open pawn a real candidate.

[Lyudmil Tsvetkov]

Any thoughts on this?

Based on 5 seconds scrolling through your post---seems like a lot of redundancy with existing features, even with your "binders".

Shouldn't we be striving for independent (orthogonal ) features?

I am sure on the 6th second you would have changed your opinion.

If a bigger number of non-orthogonal and somewhat redundant features perform better than a fewer number of independent orthogonal ones, I would say no.

[Lyudmil Tsvetkov]

Any pawn that has 2 enemy pawns to the left and to the right 2 ranks in front of it is considered backward.

Not quite - even if the stop is controlled by two sentries, there might be two helpers controlling the stop as well, making an open pawn a real candidate.

And if the pawn is not open?
And not supported by 2 helpers, which would be the usual case?

On the other hand, 2 pawns controlling the square in front of an enemy pawn are a frequent phenomenon.

[jorose]

Hmm... Isn't it true that by definition if they are non orthogonal than you can use a linear combination of previous features in order to achieve the same evaluation? Thus your new features become redundant and do nothing aside from making it harder to tune your values.

Not to say your ideas are bad, but you need to be careful when you try to describe your heuristics. Try to keep them as simplistic and independent as possible.

[Lyudmil Tsvetkov]

Hmm... Isn't it true that by definition if they are non orthogonal than you can use a linear combination of previous features in order to achieve the same evaluation? Thus your new features become redundant and do nothing aside from making it harder to tune your values.

Not to say your ideas are bad, but you need to be careful when you try to describe your heuristics. Try to keep them as simplistic and independent as possible.

Orthogonal or non-orthogonal, you should account for the specific.

And in chess, there are thousands and tens of thousands specific evaluation features. If engines have just 100 eval terms, how are they going to account for those other specific terms? Simple tuning will not be sufficient.

And if you do not account for a vast majority of real, and important, features, influencing eval and the true score, how are you going to have good eval?

And eval, please do not forget, largely determines which moves the engine will pick up.

So I prefer a majority of more or less redundant terms, but covering specific areas, rather than a minority of general terms, that might be less redundant, but will cover a smaller number of eval cases.

Of course, the challenge will be to find terms that are as non-redundant as possible and applicable in as large number of specific cases as possible.

[jorose]

Evaluation terms are completely useless in vacuum really. I would prefer having general evaluation terms which are completely orthogonal and cover as many situations as possible. Then I would take a look at an area I feel an engine is weak (in your case you feel it does not handle backward pawns well) and replace a previous eval term with one or more orthogonal evaluation terms which better describe what you were thinking of. In this case however you are describing two very close terms, binders and your new definition of a backward pawn right after one another and proposing we throw them on top of each other.

May I suggest you try to dig deeper into your ideas and get closer to the core of them? If you are right that engines have trouble with this area of chess and you come up with more atomic terms then perhaps they might turn out to be really powerful concepts.

[zullil]

Evaluation terms are completely useless in vacuum really. I would prefer having general evaluation terms which are completely orthogonal and cover as many situations as possible. Then I would take a look at an area I feel an engine is week (in your case you feel it does not handle backward pawns well) and replace a previous eval term with one or more orthogonal evaluation terms which better describe what you were thinking of. In this case however you are describing two very close terms, binders and your new definition of a backward pawn right after one another and proposing we throw them on top of each other.

May I suggest you try to dig deeper into your ideas and get closer to the core of them? If you are right that engines have trouble with this area of chess and you come up with more atomic terms then perhaps they might turn out to be really powerful concepts.

Well said, but only because I agree with you.

Lyudmil and I have been discussing these issues for a while. I must admit that he has contributed far more ELO points to SF than I have.

BTW, I've never really understood if "orthogonal" as used here means more than "linearly independent". Of course, in linear algebra the former is stronger. But as you note, lack of independence suffices to generate redundancy.