A bizarre evaluation.

[lucasart]

Lesson: evaluation in non-quiet positions can give non-sensical results. And when the side to move can do a move that greatly improves the position for him, the position is by definition non-quiet.

Exactly! Looks like Larry some has trouble understanding the distinction between search and eval...

[hgm]

Yes, it is easily solved by search.
But a human can evaluate it instantly.
Why can't the computer?

His point is that he would like evaluation to be smarter for positions like this one.

Well, like I said, the problem seems to be that an unrealistically large penalty is given for a volatile aspect of the position: local safety of an uncastled King. Normally one would take a weighted average of the local safety in the current location and that after castling, the weights being determined by how fast you can castle (number of pieces between King and Rook). As the King fortress after O-O is in no way compromised, and there is nothing between King and Rook, this should give a pretty good KS in this position.

Of course there always is a risk in evaluating things by static heuristics. Castling might not be possible for dynamic reasons, e.g. the King being bound in protection to a minor on its other side, attacked by a Rook. Just like counting attackers and protectors, or running a SEE on every square is a poor substitute for QS. So I would not be surprised if searching casling in QS when the eval shows a poor KS in the current location would be a superior solution.

[cdani]

I don't know what you're going on about. Even with no search, and just looking at the static eval, SF reports only +0.8 for white (including 0.06 for tempo).

The position is talking about that it has bad evaluation is this one:

Code: Select all

Stockfish 7 64 BMI2 by T. Romstad, M. Costalba, J. Kiiski, G. Linscott
position fen rnbqk2r/ppp1ppbp/6p1/8/3PP3/1QP5/P4PPP/R1B1KBNR b KQkq - 0 1
eval
      Eval term |    White    |    Black    |    Total
                |   MG    EG  |   MG    EG  |   MG    EG
----------------+-------------+-------------+-------------
       Material |   ---   --- |   ---   --- |  0.16 -0.07
      Imbalance |   ---   --- |   ---   --- |  0.00  0.00
          Pawns |   ---   --- |   ---   --- |  0.10  0.01
        Knights |  0.06  0.00 |  0.06  0.00 |  0.00  0.00
         Bishop | -0.12 -0.32 | -0.09 -0.33 | -0.03  0.01
          Rooks | -0.27  0.00 | -0.19  0.00 | -0.09  0.00
         Queens |  0.00  0.00 |  0.00  0.00 |  0.00  0.00
       Mobility |  0.46  0.70 |  0.11  0.20 |  0.34  0.50
    King safety |  0.89 -0.06 | -1.24 -0.22 |  2.14  0.16
        Threats |  0.00  0.00 |  0.00  0.00 |  0.00  0.00
   Passed pawns |  0.00  0.00 |  0.00  0.00 |  0.00  0.00
          Space |  0.33  0.00 |  0.13  0.00 |  0.20  0.00
----------------+-------------+-------------+-------------
          Total |   ---   --- |   ---   --- |  2.82  0.63

Total Evaluation: 2.64 (white side)

d

 +---+---+---+---+---+---+---+---+
 | r | n | b | q | k |   |   | r |
 +---+---+---+---+---+---+---+---+
 | p | p | p |   | p | p | b | p |
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   | p |   |
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   |   |   |
 +---+---+---+---+---+---+---+---+
 |   |   |   | P | P |   |   |   |
 +---+---+---+---+---+---+---+---+
 |   | Q | P |   |   |   |   |   |
 +---+---+---+---+---+---+---+---+
 | P |   |   |   |   | P | P | P |
 +---+---+---+---+---+---+---+---+
 | R |   | B |   | K | B | N | R |
 +---+---+---+---+---+---+---+---+

Following my previous reasoning about the bad static evaluation of engines, even in this case this can be at least partially solved, and in fact I have something working on Andscacs related to it from 0.83.

I reduce attack value if there is less than two pieces attacking. Give it a try on Komodo. I hope if it works you will tell it

[mclane]

Yes, it is easily solved by search.
But a human can evaluate it instantly.
Why can't the computer?

His point is that he would like evaluation to be smarter for positions like this one.

We "solve" this with our eyes.
Engines have no eyes.

If you want engines to solve this you must implement botwinniks ideas about trajectories to make engines emulate watching chess board like humans do.

Read his books about the topic (I did very long time ago, Marty hirsch did also) and understand what he tried to program.

With our eyes we can see future development on the board without calculating moves, just by looking on the board.
Maybe implementing this LOOKING on the chess board additional to the search tree of today's chess programs could make interesting long range planning possible.

[melajara]

The concern is legitimate as EVERY game amounts to 1 ply search when evaluating the leaves of the search tree. As the score is retropropagated from the leaves to the root, the quality of the static evaluation from the leaves is paramount.

Of course, due to the actual depth of the search, nowadays leaves are hardly opening positions.

There is a huge discontinuity in king safety due to castling so it is somewhat normal than static evaluation is suremphasizing the king unsafety when castling didn't occur yet, hence the incentive for castling!

Now, maybe the score should be amended by a term representing immediate castling availability, but then we have the problem of potential unprotected pieces left from direct contact with the castling king and so on to assess the real availability of castling for the next move.

No easy fix for this issue, IMHO.

[zullil]

The point is that the position after 7.Qb3 is given an absurd evaluation by both engines. Never mind what search leads to this position, shouldn't the evaluation be a reasonable one? Any human who evaluated such a position as easiliy winning for White would be called a moron. Why should such poor evaluation work for engines? Yet it does.

I don't know what you're going on about. Even with no search, and just looking at the static eval, SF reports only +0.8 for white (including 0.06 for tempo). And this is perfectly normal, and can be explained by better space, center control etc. Of course, deeper search will show that the white advantage is smaller (as black is given a chance by search to play developpement moves):

Code: Select all

position fen rnbqk2r/ppp1ppbp/6p1/8/3PP3/2P5/P4PPP/R1BQKBNR w KQkq - 0 1
d  

 +---+---+---+---+---+---+---+---+
 | r | n | b | q | k |   |   | r |
 +---+---+---+---+---+---+---+---+
 | p | p | p |   | p | p | b | p |
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   | p |   |
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   |   |   |
 +---+---+---+---+---+---+---+---+
 |   |   |   | P | P |   |   |   |
 +---+---+---+---+---+---+---+---+
 |   |   | P |   |   |   |   |   |
 +---+---+---+---+---+---+---+---+
 | P |   |   |   |   | P | P | P |
 +---+---+---+---+---+---+---+---+
 | R |   | B | Q | K | B | N | R |
 +---+---+---+---+---+---+---+---+

Fen: rnbqk2r/ppp1ppbp/6p1/8/3PP3/2P5/P4PPP/R1BQKBNR w KQkq - 0 1
Key: 59157A21AC3BA42B
Checkers: 
eval
      Eval term |    White    |    Black    |    Total    
                |   MG    EG  |   MG    EG  |   MG    EG  
----------------+-------------+-------------+-------------
       Material |   ---   --- |   ---   --- |  0.16 -0.12 
      Imbalance |   ---   --- |   ---   --- |  0.00  0.00 
          Pawns |   ---   --- |   ---   --- |  0.10  0.01 
        Knights |  0.06  0.00 |  0.06  0.00 |  0.00  0.00 
         Bishop | -0.12 -0.32 | -0.09 -0.33 | -0.03  0.01 
          Rooks | -0.27  0.00 | -0.19  0.00 | -0.09  0.00 
         Queens |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
       Mobility |  0.41  0.61 |  0.11  0.20 |  0.29  0.41 
    King safety |  0.89 -0.06 |  0.78 -0.06 |  0.11  0.00 
        Threats |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
   Passed pawns |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
          Space |  0.33  0.00 |  0.13  0.00 |  0.20  0.00 
----------------+-------------+-------------+-------------
          Total |   ---   --- |   ---   --- |  0.74  0.34 

Total Evaluation: 0.80 (white side)

But after Qb3:

Code: Select all

Stockfish 120316 64 BMI2 by T. Romstad, M. Costalba, J. Kiiski, G. Linscott
position fen rnbqk2r/ppp1ppbp/6p1/8/3PP3/1QP5/P4PPP/R1B1KBNR b KQkq - 1 1
d

 +---+---+---+---+---+---+---+---+
 | r | n | b | q | k |   |   | r |
 +---+---+---+---+---+---+---+---+
 | p | p | p |   | p | p | b | p |
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   | p |   |
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   |   |   |
 +---+---+---+---+---+---+---+---+
 |   |   |   | P | P |   |   |   |
 +---+---+---+---+---+---+---+---+
 |   | Q | P |   |   |   |   |   |
 +---+---+---+---+---+---+---+---+
 | P |   |   |   |   | P | P | P |
 +---+---+---+---+---+---+---+---+
 | R |   | B |   | K | B | N | R |
 +---+---+---+---+---+---+---+---+

Fen: rnbqk2r/ppp1ppbp/6p1/8/3PP3/1QP5/P4PPP/R1B1KBNR b KQkq - 1 1
Key: D76CA3302163105A
Checkers: 
eval
      Eval term |    White    |    Black    |    Total    
                |   MG    EG  |   MG    EG  |   MG    EG  
----------------+-------------+-------------+-------------
       Material |   ---   --- |   ---   --- |  0.19 -0.07 
      Imbalance |   ---   --- |   ---   --- |  0.00  0.00 
          Pawns |   ---   --- |   ---   --- |  0.10  0.02 
        Knights |  0.06  0.00 |  0.06  0.00 |  0.00  0.00 
         Bishop | -0.12 -0.32 | -0.09 -0.33 | -0.03  0.01 
          Rooks | -0.27  0.00 | -0.19  0.00 | -0.09  0.00 
         Queens |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
       Mobility |  0.46  0.70 |  0.11  0.20 |  0.34  0.50 
    King safety |  0.89 -0.06 | -1.25 -0.22 |  2.14  0.16 
        Threats |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
   Passed pawns |  0.00  0.00 |  0.00  0.00 |  0.00  0.00 
          Space |  0.33  0.00 |  0.12  0.00 |  0.21  0.00 
----------------+-------------+-------------+-------------
          Total |   ---   --- |   ---   --- |  2.86  0.64 

Total Evaluation: 2.68 (white side)

[hgm]

Well, that is what I already said on the previous page: the misevaluation is due to king safety, or rather imagined lack thereof.

But as this misevaluation can only occur in positions with castling rights, fixing it would not result in any Elo gain, as Elo is typically determined in games where castling is not possible, because it already has been done.

[kgburcham]

I assume Larry is talking about this 2.63 7.Qb3 at one ply.


[d] rnbqk2r/ppp1ppbp/6p1/8/3PP3/2P5/P4PPP/R1BQKBNR w KQkq -

Engine: Stockfish 7Beta1 64 BMI2 (8192 MB)
by T. Romstad, M. Costalba, J. Kiiski, G.
1/01 0:00 +2.63 7.Qb3 (41) 41
2/02 0:00 +1.22 7.Bc4 Kf8 (188) 188
3/03 0:00 +1.18 7.Nf3 c6 8.h3 (310) 310
4/04 0:00 +1.44 7.Bf4 c6 8.Nf3 h6 (656) 328
5/05 0:00 +1.00 7.Bf4 Nc6 8.Nf3 e6 9.h3 (1.621) 810
6/07 0:00 +1.10 7.h4 e6 8.Nf3 Nc6 9.Bf4 b6 (3.376) 1125

[lkaufman]

I don't know what you're going on about. Even with no search, and just looking at the static eval, SF reports only +0.8 for white (including 0.06 for tempo).

The position is talking about that it has bad evaluation is this one:

Code: Select all

Stockfish 7 64 BMI2 by T. Romstad, M. Costalba, J. Kiiski, G. Linscott
position fen rnbqk2r/ppp1ppbp/6p1/8/3PP3/1QP5/P4PPP/R1B1KBNR b KQkq - 0 1
eval
      Eval term |    White    |    Black    |    Total
                |   MG    EG  |   MG    EG  |   MG    EG
----------------+-------------+-------------+-------------
       Material |   ---   --- |   ---   --- |  0.16 -0.07
      Imbalance |   ---   --- |   ---   --- |  0.00  0.00
          Pawns |   ---   --- |   ---   --- |  0.10  0.01
        Knights |  0.06  0.00 |  0.06  0.00 |  0.00  0.00
         Bishop | -0.12 -0.32 | -0.09 -0.33 | -0.03  0.01
          Rooks | -0.27  0.00 | -0.19  0.00 | -0.09  0.00
         Queens |  0.00  0.00 |  0.00  0.00 |  0.00  0.00
       Mobility |  0.46  0.70 |  0.11  0.20 |  0.34  0.50
    King safety |  0.89 -0.06 | -1.24 -0.22 |  2.14  0.16
        Threats |  0.00  0.00 |  0.00  0.00 |  0.00  0.00
   Passed pawns |  0.00  0.00 |  0.00  0.00 |  0.00  0.00
          Space |  0.33  0.00 |  0.13  0.00 |  0.20  0.00
----------------+-------------+-------------+-------------
          Total |   ---   --- |   ---   --- |  2.82  0.63

Total Evaluation: 2.64 (white side)

d

 +---+---+---+---+---+---+---+---+
 | r | n | b | q | k |   |   | r |
 +---+---+---+---+---+---+---+---+
 | p | p | p |   | p | p | b | p |
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   | p |   |
 +---+---+---+---+---+---+---+---+
 |   |   |   |   |   |   |   |   |
 +---+---+---+---+---+---+---+---+
 |   |   |   | P | P |   |   |   |
 +---+---+---+---+---+---+---+---+
 |   | Q | P |   |   |   |   |   |
 +---+---+---+---+---+---+---+---+
 | P |   |   |   |   | P | P | P |
 +---+---+---+---+---+---+---+---+
 | R |   | B |   | K | B | N | R |
 +---+---+---+---+---+---+---+---+

Following my previous reasoning about the bad static evaluation of engines, even in this case this can be at least partially solved, and in fact I have something working on Andscacs related to it from 0.83.

I reduce attack value if there is less than two pieces attacking. Give it a try on Komodo. I hope if it works you will tell it

That is worth a try, and if we like it I will so report here. But I think in the given position the problem has more to do with the fact that White has two safe queen checks available, although they are both harmless.

[bob]

Well, that is what I already said on the previous page: the misevaluation is due to king safety, or rather imagined lack thereof.

But as this misevaluation can only occur in positions with castling rights, fixing it would not result in any Elo gain, as Elo is typically determined in games where castling is not possible, because it already has been done.

It might not be just castling rights. Some try to evaluate safe checks. For example, from stockfish 7:

// Penalties for enemy's safe checks
const int QueenContactCheck = 89;
const int QueenCheck = 50;
const int RookCheck = 45;
const int BishopCheck = 6;
const int KnightCheck = 14;

In the above position, there are two safe checks since the queen can reach the diagonal bearing on the queen using two different squares. This is one of those positions where this evaluation term is not just wrong, but badly wrong. The search will repair this, since the penalty is so large. All it needs to do is play Bd7 if the pawn on b7 is not hanging, or else c6 or Nd7, etc. Since the threat is so easy to eliminate, without any positional loss, it works OK. So long as that Qb3 move is not at the END of the PV where it will certainly back up a really ugly score.

I've experimented with this "safe check penalty" idea but never found something I liked, and still do not do it.