Positional quiesence

[hgm]

The initial setup of orthodox Chess is all wrong for the King and Rooks: Kings don't belong on the central files, but need to be safely tucked away in a corner. This is why castling was invented, and why having castled should give a hefty positional bonus. As we all know, awarding such a bonus all at once, at the point where the King reaches its safe destination, can cause problems through horizon effect: when your opponent can castle, the engine will attempt all kind of silly positional, and perhaps even material sacrifices to push the castling over the horizon. (The bonus for good King safety is easily larger than a Pawn...) While of course in the long run the opponent will castle; there is no way you could prevent that. But the program does not know.

There also is a more subtle problem: if both can castle within the horizon, both get the bonus, and there is no net advantage. And castling is a quiet move, almost like null move. So when I castle, it gives the opponent the opportunity to castle as well. It is very usual in Chess that castling happens in subsequent moves, like it is infective. So rather than castling immediately, and allowing his opponent to do so as well, the program will try to push the opponent's castling over the horizon, by playing aggressive moves to which the opponent has to react, so it can castle on the last move before the horizon. So in stead of castling, it prefers other moves. The larger the castling bonus, the more reluctant it will be to castle!

Of course the conventional solution is to not award the castling bonus all at once, but distribute it over the preparation stages. Like emptying the squares between K and R, and awarding part of the Pawn-shield bonus it will get after castling already for having the rights to do so. An alternative would be to consider castling a non-quiet move, to be searched in Quiescence Search. That would prevent the trick of castling just before the horizon, as the reply castling would still be searched in QS. (As you can castle only once, and in most of the game not at all, this can never cause search explosion, and is not very expensive.) It would not help against making a sacrificial threat just before the horizon, and standing pat yourself after the opponent deals with it, though.

The reason I am bringing this up is that I am dealing with a Chess variant now (Chu Shogi) in which the initial position is all wrong on a vastly larger scale. All valuable sliders, such as Queen and Super-Rooks go in front (directly behind the Pawns) of the 4-rank deep initial army, and the light (Knight-class) pieces are tucked away on the back rank. So before you can safely engage the enemy, you have to swap that around in some massive 'vertical castling' operation, where you have the same problem that the setup is crowded with pieces in between them, and that there are no special moves that make them hop over each other, like castling in orthodox Chess allows the Rook to hop over the King.

So the obvious solution is to encourage advancing of the light stepping pieces on the back rank to 4th and 5th rank by giving an eval bonus for it through the PST. And make sure the bonus outweighs what can be earned from mobility by developing the valuable sliders, so that you won't start deploying them 'naked' in the center. (This is also a rule in orthodox Chess: develop the minors before your Queen and Rooks.) The light pieces have to step forward rank by rank, after creating room for them to do so, however. So there is automatically some distribution of the eventual bonus for being at the front. But even stepping a single rank needs a bonus that exceeds the PST + mobility increase for developing a slider, which usually can be done in a single move. Otherwise it would plan to develop 4 sliders in the 4 moves it can see up the the horizon (assumed to be at 8 ply), rather than move a single stepper up 4 ranks.

So even the distributed bonuses are pretty large, and as a result the horizon effect kicks in: no matter how large I make the bonus, it is very reluctant to advance the steppers in its rear guard, as these are quiet moves, and would allow the opponent to be the same. So in stead of actually playing them from the root, it starts to make pointless threats with its sliders in the center, too dangerous to ignore for the opponent, so that he can save its own good moves for just before the horizon. (And thus will never play them.) A bit similar to roaming the board with the Queen in Chess openings, making attacks on all kind of Pawns that can easily be protected by other Pawns, just to push development of the opponent's pieces over the horizon. The larger you make the bonus, the less inclined it will be to play the moves.

I wonder how this very fundamental problem can be solved. For material evaluation QS was invented to prevent saving your best capture for last, counting yourself rich in the illusion he would not retaliate. But it is not feasible to try every light-piece forward non-capture move in QS, as that would surely explode it. Without such a search for 'positional quiescence' (i.e. search until he runs out of moves with a spectacularly high positional score) you would get very much alternating scores between odd and even iterations, depending on who can do the last move, and ignore the opponent's equally good one that would follow.

Such alternating scores can be ameliorated by a 'tempo bonus', for who has the move. But for that to make sense, the value of the bonus should really adapt to the positional gain that is realistically possible to achieve with a move. Giving it the value gained by your best positional move would cause just the inverse alteration (since in fact it would be equivalent to doing one extra ply of search, except that you would not worry if the move was tactically sound.) So you would need to give it only half of it to dampen the oscillations of the score with increasing depth.

This suggests an alternative approach to positional quiescence: if the final ply before QS is met by stand-pat (i.e. returns the evaluation score after the move), you would not take that eval score, but the average of the score before and after the move. Or, in other words, if you stand pat in reply to a non-capture, don't use currentEval, but average it with the previousEval of the parent, (which was likely better, from the POV of the side standing pat), to decide if you would still like to stand pat. This would strongly discourage saving good positional moves for the last ply of the branch.

[PK]

Long time ago I have tried the following experiment: quiescence search that allows any move that improves piece-square table eval. Of course it was total failure, even though engine's moves looked really nice, especially in the opening. Perhaps doing so only for the moves that are especially good (castling, developing knights from first rank towards the center, rooks to 7th file, pawns trapping bishops) would make sense?

[hgm]

I guess it would certainly make sense for moves that can improve positional eval by about 1 Pawn. After all, we also consider capturing of Pawns in QS. I once considered to include Pawn pushes in QS, as creating passers can easily score such a bonus. Never actually tried it though.

For non-static bonuses (i.e. those that depend on location of opponent pieces) this might be dangerous, however, as termination is not guaranteed. For a static bonus you would sooner or later reach the maximum.

For Chu Shogi I don't think it could be a solution. There are 6 steppers (per side) that each need to be advance 4 ranks, and considering all these steps would cause QS to become 48 ply deep...

[Uri Blass]

I think that it is silly to give a big positional bonus for castling in the first place and more than a pawn is a big positional bonus.
There are positions when the king is totally safe inspite of not castling and part of them are in the opening theory.

For example
1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.Nc3 Nxe4

Here the right move is 5.Nxe4 and not 5.Bxf7+ Kxf7 6.Nxe4 d5 and black is better.

Edit:Maybe you can say that he white has problems to castle later so here is another example from opening theory:

1.e4 e6 2.d4 d5 3.Nc3 Bb4 4.e5 c5 5.a3 Bxc3+ 6.bxc3 Qa5 7.Bd2 Qa4
8.Qg4 Kf8
white may castle later and inspite of it the position is not considered bad for black.

I guess that
if you want to evaluate king safety then it is better to forget about the castling rule and evaluate it by other ways.

Edit 2:
being able to castle is better than not being able to castle so you can give a small positional bonus for being able to castle.

You can give also a small positional bouns for king at g1 or c1 or for rook at d1 or f1 but not more than it

[hgm]

I think that it is silly to give a big positional bonus for castling in the first place and more than a pawn is a big positional bonus.

Castling is just a means. You don't get the bonus for castling, you get it for the King Safety that you realize through castling. A King without Pawn shield on the central files is not very safe.

[Uri Blass]

I think that it is silly to give a big positional bonus for castling in the first place and more than a pawn is a big positional bonus.

Castling is just a means. You don't get the bonus for castling, you get it for the King Safety that you realize through castling. A King without Pawn shield on the central files is not very safe.

In this case you need a better king safety evaluation.
I think that no pawn shield is not a big problem that should have a big positional value of more than a pawn unless the opponent is also attacking squares near the king with more than one piece.

[Uri Blass]

Here is an example from position that I composed

position is almost equal based on stockfish or houdini and obviously they do not give a big penalty of more than a pawn for not having pawns near the king

[D]r1bq1rk1/ppp1bppp/2np1n2/4pP2/2B1P1P1/5N1P/PPPP4/RNBQ1RK1 w - - 6 1

[hgm]

But it could be worse. E.g. a half-open g-file.

[bob]

I think that it is silly to give a big positional bonus for castling in the first place and more than a pawn is a big positional bonus.
There are positions when the king is totally safe inspite of not castling and part of them are in the opening theory.

For example
1.e4 e5 2.Nf3 Nc6 3.Bc4 Nf6 4.Nc3 Nxe4

Here the right move is 5.Nxe4 and not 5.Bxf7+ Kxf7 6.Nxe4 d5 and black is better.

Edit:Maybe you can say that he white has problems to castle later so here is another example from opening theory:

1.e4 e6 2.d4 d5 3.Nc3 Bb4 4.e5 c5 5.a3 Bxc3+ 6.bxc3 Qa5 7.Bd2 Qa4
8.Qg4 Kf8
white may castle later and inspite of it the position is not considered bad for black.

I guess that
if you want to evaluate king safety then it is better to forget about the castling rule and evaluate it by other ways.

Edit 2:
being able to castle is better than not being able to castle so you can give a small positional bonus for being able to castle.

You can give also a small positional bouns for king at g1 or c1 or for rook at d1 or f1 but not more than it

I think it depends on what you mean by "bonus for castling."

If you mean a bonus just for making the move, that's most likely bad. If you mean a big bonus because center files are open and the f/g/h files are fine, and by castling you move from a very unsafe position to one that is much safer, then a big bonus makes sense. That's the way I evaluate castling, that is, by evaluating the safety of the king and if it is better after castling, normal minimax will choose to castle. And the worse the score is with the king in the center (the more open files, pieces attacking, etc) then the more "angst" the program has to get its king out of there.

I've always followed a GM quote I read somewhere, "castle if you want to, or if you must, but NOT just because you can..."

I do have a sort of "uncastle penalty" which is applied if you toss the right to castle away (such as moving the king). Because you give up a move option and that can be serious at times.

[hgm]

Note that how much king safety is worth is really only a tangent to the issue I am raising. The point really is that no matter how large a bonus it brings, the engine will still refuse to do such a quiet move if the opponent can do it too. (And then, in self-play...) I could set the king safety on a central file to -900, and it would still prefer to sacrifice all its minors rather than being the first to castle. It wants to make sure it can castle on the last move before the horizon. The more it earns by castling, the less likely it is it will ever do it (if the opponent does not do it first).