Underwhelming results from king safety evaluation

[clayt]

I'm currently working on adding a king safety heuristic to my evaluation function. I figured I would start with something dead simple and then see where I could go from there. My current implementation is more or less a direct implementation of the Attack Units approach from the wiki. It's implemented as follows:

1. Construct a neighborhood bitmask around the king consisting of the squares the king can see plus the three directly in front of the king.
2. For each attacking piece (knight, bishop, rook, queen), get a mask of the squares the piece can attack, and bitwise and it with the neighborhood mask
   to get the piece's king-attack set.
3. Multiply the number of ones in the piece's king attack set by a fixed score for each piece (2 for minor pieces, 3 for rooks, 5 for queens) and add to the total quantity of attack units.
4. Look up the bonus or malus for the given number of attack units and add it to the evaluation.
5. Repeat the process for the opposing side.

My implementation can be found here.

After tuning, my first confusing result was that the bonus given to an attack (after tuning) was much smaller than the examples shown in the wiki. The highest bonus given is only +7 centipawns for a large attack. Compare to Stockfish, which gave +500 centipawns for a large attack before it moved to NNUE.

When benchmarking, it turned out that my implementation resulted in a -10 Elo regression in playing strength. Clearly I've missed something - so what's wrong here? Some possibilities:

• Do I need to include pawn structure information in king safety?
• Have I misinterpreted the description on the wiki somehow?
• I'm currently using attacks. Should I instead consider legal moves (thus eliminating ally-attacks and pinned pieces)?

[clayt]

I just now thought of one possible logic error that may be responsible for some of my issues, though I haven't had the time to benchmark and see if it's the actual problem yet.

Here's my computation of the bitmask for the neighborhood surrounding the king:

Code: Select all

let b3_neighborhood: Bitboard = match ATTACKER {
    Color::White => Bitboard::new(0x00_0707_0707),
    Color::Black => Bitboard::new(0x07_0707_0700),
 };

let king_sq = g.king_sq(!ATTACKER);
let neighbor_mask = if (king_sq as u8) < (Square::B3 as u8) {
    b3_neighborhood >> ((Square::B3 as u8) - (king_sq as u8))
} else {
    b3_neighborhood << ((king_sq as u8) - (Square::B3 as u8))
};

The issue is that when the king is on the edge of the board, the mask overflows onto the next row, adding extra attacks to the neighborhood. I'll try fixing this shortly. However, I wouldn't be surprised if there are also other issues.

[clayt]

Looks like even after fixing that bug, the engine is still playing worse, so all the questions from my original post still stand. Currently I'm at a regression of -8 Elo, which is definitely not what I want.

[JoAnnP38]

Something I do differently, so you can take or reject at your discretion.

1. I don't count a square as being attacked if that square is defended by a friendly pawn.
2. I don't look at just the eight squares around the king and the three in front of that. I look at three concentric squares centered on the king and award a graduated bonus for attacking squares in a specific "ring". Tuning decides just how much an attack to a specific ring is valued.
3. I also consider the following elements parts of king safety: location of pawns in relation to king. I use those same concentric squares I used in king attacks and give bonuses for friendly pawns in those rings. Bigger bonuses for pawns in inner ring. Also, I give bonuses for preserving castling rights and a bonus whenever castling has been performed. My ideas on castling are a little controversial since awarding a castling bonus isn't really a "static evaluation feature".
4. King on open rank or half-open rank. Penalties for this. I also tried king adjacent to open rank but that didn't gain for me.

Here is a representation of the three masks I use to count king attacks and pawn shield. At first, I was iffy about looking at squares behind the king, but they seem to work fine.

3 3 3 3 3 3 3
3 2 2 2 2 2 3
3 2 1 1 1 2 3
3 2 1 K 1 2 3
3 2 1 1 1 2 3
3 2 2 2 2 2 3
3 3 3 3 3 3 3

[Whiskers]

I'm currently working on adding a king safety heuristic to my evaluation function. I figured I would start with something dead simple and then see where I could go from there. My current implementation is more or less a direct implementation of the Attack Units approach from the wiki. It's implemented as follows:

1. Construct a neighborhood bitmask around the king consisting of the squares the king can see plus the three directly in front of the king.
2. For each attacking piece (knight, bishop, rook, queen), get a mask of the squares the piece can attack, and bitwise and it with the neighborhood mask
   to get the piece's king-attack set.
3. Multiply the number of ones in the piece's king attack set by a fixed score for each piece (2 for minor pieces, 3 for rooks, 5 for queens) and add to the total quantity of attack units.
4. Look up the bonus or malus for the given number of attack units and add it to the evaluation.
5. Repeat the process for the opposing side.

My implementation can be found here.

After tuning, my first confusing result was that the bonus given to an attack (after tuning) was much smaller than the examples shown in the wiki. The highest bonus given is only +7 centipawns for a large attack. Compare to Stockfish, which gave +500 centipawns for a large attack before it moved to NNUE.

When benchmarking, it turned out that my implementation resulted in a -10 Elo regression in playing strength. Clearly I've missed something - so what's wrong here? Some possibilities:

• Do I need to include pawn structure information in king safety?
• Have I misinterpreted the description on the wiki somehow?
• I'm currently using attacks. Should I instead consider legal moves (thus eliminating ally-attacks and pinned pieces)?

I assume this is a problem with your trainer, specifically your training data not having enough positions with a ton of attacks to optimize those parameters well on.
If you copy Stockfish's king attack table and use it instead of yours, how does king safety score?

[alessandro]

The approach I use with AdaChess is a customization of the concept expressed in the wiki. The basic algorithm has almost identical idea.
However, I exclusively consider the squares around the King, and not three others forward (which frankly I do not understand why they should be taken into account).
In addition, the value of attacks is dynamic: the more squares a piece attacks, the more value it gains, which is fractionated each time. For example in this case
[d]5k2/2R5/8/8/7Q/5K2/8/8 w - - 0 1 the king's area consists of the squares E8, E7, F7, G7, G8. The white rook in C7 attacks 3 squares in the king's zone: E7, F7, G7. The base value for the rook attack starts with 20 centipawn. Since it attacks three squares the total value of the attack(for the rook) is (20/1) + (20/2) + (20/3). Therefore, the first square attacked gain full value to the attack. Attacking a second square adds half of the base value, attacking a third square adds an additional third, and so on.

Each piece is valued separately, so the queen in H4, attacking the E7 square, gets her full base value.

Under this system, the attack value of the various pieces should be lowered slightly from the original wiki value so as to avoid kamikaze attacks. I don't know how much ELO i get, but I like the idea and I have implemented it

[clayt]

I'm currently working on adding a king safety heuristic to my evaluation function. I figured I would start with something dead simple and then see where I could go from there. My current implementation is more or less a direct implementation of the Attack Units approach from the wiki. It's implemented as follows:

1. Construct a neighborhood bitmask around the king consisting of the squares the king can see plus the three directly in front of the king.
2. For each attacking piece (knight, bishop, rook, queen), get a mask of the squares the piece can attack, and bitwise and it with the neighborhood mask
   to get the piece's king-attack set.
3. Multiply the number of ones in the piece's king attack set by a fixed score for each piece (2 for minor pieces, 3 for rooks, 5 for queens) and add to the total quantity of attack units.
4. Look up the bonus or malus for the given number of attack units and add it to the evaluation.
5. Repeat the process for the opposing side.

My implementation can be found here.

After tuning, my first confusing result was that the bonus given to an attack (after tuning) was much smaller than the examples shown in the wiki. The highest bonus given is only +7 centipawns for a large attack. Compare to Stockfish, which gave +500 centipawns for a large attack before it moved to NNUE.

When benchmarking, it turned out that my implementation resulted in a -10 Elo regression in playing strength. Clearly I've missed something - so what's wrong here? Some possibilities:

• Do I need to include pawn structure information in king safety?
• Have I misinterpreted the description on the wiki somehow?
• I'm currently using attacks. Should I instead consider legal moves (thus eliminating ally-attacks and pinned pieces)?

I assume this is a problem with your trainer, specifically your training data not having enough positions with a ton of attacks to optimize those parameters well on.
If you copy Stockfish's king attack table and use it instead of yours, how does king safety score?

I quickly tried re-training with Leorik's training set instead of the Zurichess quiet-labeled set. Unfortunately, this led to a roughly 100 elo regression in both the version with king safety and without king safety. I'll probably end up trying a new approach which includes pawn structure information, maybe rejecting attacks on squares defended by pawns.

[jdart]

Even a basic king safety function should be better than no king safety. You need to make sure there are no bugs that prevent proper functioning. It's helpful to be able to print out eval details for a position, as Stockfish and other engines can do. I had a file of king attack positions, but I can't locate it now. It is worth looking at some positions, ranging from minimal attack (e.g. minor near the King) to multiple pieces attacking, plus positions with a pawn shield in front of the King, and positions with that shield damaged or missing. Just make sure the score looks more or less like what you'd expect. You can try some manual tuning and add auto-tuning later.

You may also want to do testing against an engine that does have king safety eval, or a neural net. Your win rate should go up with king safety implemented.

[jdart]

Here's an example of what can happen with poor king safety:
[pgn] [Event "ICS Rated blitz match"] [Site "nightmare-chess.nl"] [Date "2023.08.05"] [Round "-"] [White "ArasanX"] [Black "MatMoi"] [Result "1-0"] [WhiteElo "2873"] [BlackElo "2453"] [TimeControl "300+1"] 1. d4 Nf6 2. Nf3 d5 3. Bf4 c5 4. e3 Nc6 5. Nbd2 e6 6. c3 Bd6 7. Bg3 O-O 8. Bd3 c4 9. Bc2 h5 10. e4 Be7 11. e5 Ne8 12. h4 Qa5 13. Bf4 g6 14. g4 hxg4 15. h5 gxf3 16. hxg6 fxg6 17. Qxf3 Rf5 18. Bxf5 exf5 19. Qh3 Kf7 20. Qh8 Bf8 21. Rh7+ Ng7 22. Bh6 Qd8 23. Bxg7 Be7 24. Bf6+ {MatMoi resigns} 1-0 [/pgn]

[mathmoi]

Here's an example of what can happen with poor king safety:

MatMoi has *no* king safety.