Sanity check on piece-value tables

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

StuartRiffle
Posts: 25
Joined: Tue Apr 05, 2016 9:34 pm
Location: Canada

Sanity check on piece-value tables

Post by StuartRiffle »

Hi all,

I've been experimenting with generating material tables for use in Pigeon. Below is a heat map of some early results. Having never done this before, do numbers like these look sane?

(These tables are upside down (black's POV) because that's how my data was. Blue is more happy, red is more sad).

Image

The data was generated using about 4m games (CCRL 40/40, CEGT blitz, and an export of millbase from SCID). From each game, I discarded non-quiet positions, and also threw away anything after a promotion (figuring that play would mostly be tactical at that point). The result was about 100m test positions. I then used logistic regression to bake the tables.

Some features make perfect sense, but something major I didn't expect was the pronounced kingside bias you see with the bishop and the queen. (?)

A possibility: I didn't filter the input games by player strength, so perhaps these are the ghosts of unbalanced games, where the strong player was able to storm the castle and mate before the weaker player's king could get free?

If you have any insight, please let me know!
-Stuart
(Pigeon)
StuartRiffle
Posts: 25
Joined: Tue Apr 05, 2016 9:34 pm
Location: Canada

Re: Sanity check on piece-value tables

Post by StuartRiffle »

Oops, correction: I was trying to point out the bias in the knight and queen, not bishop and queen.
-Stuart
(Pigeon)
Dann Corbit
Posts: 12540
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Sanity check on piece-value tables

Post by Dann Corbit »

Did you use the whole game for the tables, or just the first few moves or something else?

The heat maps I produce look very different as a function of move number.

I use a tool written by Les Fernandez.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
StuartRiffle
Posts: 25
Joined: Tue Apr 05, 2016 9:34 pm
Location: Canada

Re: Sanity check on piece-value tables

Post by StuartRiffle »

Yes, pretty much all of the games, but only the strictly quiet positions, and I discarded promotions (and all positions thereafter).

I used Nelder-Mead to search the parameter space from within Pigeon.
-Stuart
(Pigeon)
Dann Corbit
Posts: 12540
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Sanity check on piece-value tables

Post by Dann Corbit »

StuartRiffle wrote:Yes, pretty much all of the games, but only the strictly quiet positions, and I discarded promotions (and all positions thereafter).

I used Nelder-Mead to search the parameter space from within Pigeon.
The approach I took was to limit the positions by bands.
So moves 1-N, N+1 - 2N, etc.
The tool actually lets me take multiple different views once I have read the raw PGN into the analysis engine.

I did not want to remove captures or promotions because they are an important class of move.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Adam Hair
Posts: 3226
Joined: Wed May 06, 2009 10:31 pm
Location: Fuquay-Varina, North Carolina

Re: Sanity check on piece-value tables

Post by Adam Hair »

Here is a copy of a post I made a couple of years ago:
Adam Hair wrote:These PST values were determined from ~450,000 long time control engine vs engine matches (2700+ CCRL Elo), using Larry Kaufman's "The Evaluation of Material Imbalances" as a guideline.

I found the average score for White when a White piece occupied a square for 6 plies during a game after leaving the opening book, and the same for the same Black piece in its comparable square. I adjusted the scores to remove White advantage, then converted to Elo. Then I assumed that 1 Elo equals 1 centipawn.

For midgame values, I looked at moves 12 through 40, with at least 1 rook,2 minors, and 4 pawns on each side and no material imbalance. For the endgame, only after move 40 was considered and no more than 1 rook, 2 minors, and 4 pawns on each side (no material imbalance).

The endgame statistics for some pieces/squares was sparse. So, for the endgame tables I used the more statistically significant information to find 3 dimensional surfaces to model the PST values.

I used the values with Dan Honeycutt's engine Cupcake, which has some but not much chess knowledge encoded (on purpose). In one test (see below), they seemed to be worth about 19 Elo over Cupcake's original PSTs.

Code: Select all

PST test
   # ENGINE            : RATING    POINTS  PLAYED    (%)
   1 Cupcake_modified4 &#58;    8.5    3044.5    6000   50.7% <--- PST derived from engine data
   2 Cupcake_modified3 &#58;    8.3    2819.0    5566   50.6%
   3 Cupcake_modified  &#58;    4.2    6637.0   13114   50.6%
   4 Cupcake_raw       &#58;    4.0    6567.0   12979   50.6%
   5 Cupcake_modified2 &#58;    1.5    3259.5    6524   50.0%
   6 Cupcake_model     &#58;   -3.0    3890.0    7800   49.9%
   7 Cupcake_current   &#58;  -10.4    3754.0    7725   48.6% <---- original values
   8 Cupcake_crafty    &#58;  -13.1    3283.0    6800   48.3%
I doubt these values would work for a more advanced engine, but they may work well for an engine with no knowledge.

Code: Select all

#define PAWN_VAL 100 
#define KNIGHT_VAL 300 
#define BISHOP_VAL 300 
#define ROOK_VAL 500 
#define ROOK_VAL 950

const int pstPawnMg&#91;64&#93; =
&#123;
//A1                                H1
0, 0, 0, 0, 0, 0, 0, 0
-1, -7, -11, -35, -13, 5, 3, -5
1, 1, -6, -19, -6, -7, -4, 10
1, 14, 8, 4, 5, 4, 10, 7
9, 30, 23, 31, 31, 23, 17, 11
21, 54, 72, 56, 77, 95, 71, 11
118, 121, 173, 168, 107, 82, -16, 22
 0, 0, 0, 0, 0, 0, 0, 0
//A8                                H8
&#125;;

const int pstPawnEg&#91;64&#93; =
&#123;
0, 0, 0, 0, 0, 0, 0, 0
-17, -17, -17, -17, -17, -17, -17, -17
-11, -11, -11, -11, -11, -11, -11, -11
-7, -7, -7, -7, -7, -7, -7, -7
16, 16, 16, 16, 16, 16, 16, 16
55, 55, 55, 55, 55, 55, 55, 55
82, 82, 82, 82, 82, 82, 82, 82
0, 0, 0, 0, 0, 0, 0, 0
&#125;;
   
const int pstKnightMg&#91;64&#93; =
&#123;
-99, -30, -66, -64, -29, -19, -61, -81
-56, -31, -28, -1, -7, -20, -42, -11
-38, -16, 0, 14, 8, 3, 3, -42
-14, 0, 2, 3, 19, 12, 33, -7
-14, -4, 25, 33, 10, 33, 14, 43
-22, 18, 60, 64, 124, 143, 55, 6
-34, 24, 54, 74, 60, 122, 2, 29
-60, 0, 0, 0, 0, 0, 0, 0
&#125;;
 
const int pstKnightEg&#91;64&#93; = 
&#123;
-99, -99, -94, -88, -88, -94, -99, -99
-81, -62, -49, -43, -43, -49, -62, -81
-46, -27, -15, -9, -9, -15, -27, -46
-22, -3, 10, 16, 16, 10, -3, -22
-7, 12, 25, 31, 31, 25, 12, -7
-2, 17, 30, 36, 36, 30, 17, -2
-7, 12, 25, 31, 31, 25, 12, -7
-21, -3, 10, 16, 16, 10, -3, -21
&#125;;
 
const int pstBishopMg&#91;64&#93; =  
&#123; 
-7, 12, -8, -37, -31, -8, -45, -67
15, 5, 13, -10, 1, 2, 0, 15
5, 12, 14, 13, 10, -1, 3, 4
1, 5, 23, 32, 21, 8, 17, 4
-1, 16, 29, 27, 37, 27, 17, 4
7, 27, 20, 56, 91, 108, 53, 44
-24, -23, 30, 58, 65, 61, 69, 11
0, 0, 0, 0, 0, 0, 0, 0
&#125;;

const int pstBishopEg&#91;64&#93; =
&#123;
-27, -21, -17, -15, -15, -17, -21, -27
-10, -4, 0, 2, 2, 0, -4, -10
2, 8, 12, 14, 14, 12, 8, 2
11, 17, 21, 23, 23, 21, 17, 11
14, 20, 24, 26, 26, 24, 20, 14
13, 19, 23, 25, 25, 23, 19, 13
8, 14, 18, 20, 20, 18, 14, 8
-2, 4, 8, 10, 10, 8, 4, -2
&#125;;

const int pstRookMg&#91;64&#93; =
&#123;
-2, -1, 3, 1, 2, 1, 4, -8
-26, -6, 2, -2, 2, -10, -1, -29
-16, 0, 3, -3, 8, -1, 12, 3
-9, -5, 8, 14, 18, -17, 13, -13
19, 33, 46, 57, 53, 39, 53, 16
24, 83, 54, 75, 134, 144, 85, 75
46, 33, 64, 62, 91, 89, 70, 104
84, 0, 0, 37, 124, 0, 0, 153
&#125;;

const int pstRookEg&#91;64&#93; =
&#123;
-32, -31, -30, -29, -29, -30, -31, -32
-27, -25, -24, -24, -24, -24, -25, -27
-15, -13, -12, -12, -12, -12, -13, -15
1, 2, 3, 4, 4, 3, 2, 1
15, 17, 18, 18, 18, 18, 17, 15
25, 27, 28, 28, 28, 28, 27, 25
27, 28, 29, 30, 30, 29, 28, 27
16, 17, 18, 19, 19, 18, 17, 16
&#125;;

const int pstQueenMg&#91;64&#93; =
&#123;
1, -10, -11, 3, -15, -51, -83, -13
-7, 3, 2, 5, -1, -10, -7, -2
-11, 0, 12, 2, 8, 11, 7, -6
-9, 5, 7, 9, 18, 17, 26, 4
-6, 0, 15, 25, 32, 9, 26, 12
-16, 10, 13, 25, 37, 30, 15, 26
1, 11, 35, 0, 16, 55, 39, 57
-13, 6, -42, 0, 29, 0, 0, 102
&#125;;

const int pstQueenEg&#91;64&#93; =
&#123;
-61, -55, -52, -50, -50, -52, -55, -61
-31, -26, -22, -21, -21, -22, -26, -31
-8, -3, 1, 3, 3, 1, -3, -8
9, 14, 17, 19, 19, 17, 14, 9
19, 24, 28, 30, 30, 28, 24, 19
23, 28, 32, 34, 34, 32, 28, 23
21, 26, 30, 31, 31, 30, 26, 21
12, 17, 21, 23, 23, 21, 17, 12
&#125;;

const int pstKingMg&#91;64&#93; =
&#123;
0, 0, 0, -9, 0, -9, 25, 0
-9, -9, -9, -9, -9, -9, -9, -9
-9, -9, -9, -9, -9, -9, -9, -9
-9, -9, -9, -9, -9, -9, -9, -9
-9, -9, -9, -9, -9, -9, -9, -9
-9, -9, -9, -9, -9, -9, -9, -9
-9, -9, -9, -9, -9, -9, -9, -9
-9, -9, -9, -9, -9, -9, -9, -9
&#125;;

const int pstKingEg&#91;64&#93; =
&#123;
-34, -30, -28, -27, -27, -28, -30, -34
-17, -13, -11, -10, -10, -11, -13, -17
-2, 2, 4, 5, 5, 4, 2, -2
11, 15, 17, 18, 18, 17, 15, 11
22, 26, 28, 29, 29, 28, 26, 22
31, 34, 37, 38, 38, 37, 34, 31
38, 41, 44, 45, 45, 44, 41, 38
42, 46, 48, 50, 50, 48, 46, 42
&#125;;
Couple of notes:
The bias towards the king side is definitely due to the preponderance of short side castling.
I used 0 in the middle game tables for any piece-square that lacked a significant number of occurrences (50 games minimum).
User avatar
hgm
Posts: 27790
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Sanity check on piece-value tables

Post by hgm »

This suggest it might be andvatageous to have two Queen and Knight PST (four if you also count the opening/end-game diversification), one for each King on a-file, the other on h-file. And then interpolate based on the file the King is actually in.

Xiangqi engines actually use something similar for the Horse PST, not based on King position (which is pretty much fixed there, because of the Palace), but on whether you have an advantage or disadvantage in attacking material. When you are ahead your Horses should strongly be biased forward, but when you are behind, withdrawing them in a defensive position should be strongly encouraged.
Dann Corbit
Posts: 12540
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Sanity check on piece-value tables

Post by Dann Corbit »

PST should be a function of game phase.

For instance, in the opening center pawns are more valuable and edge pawns are less valuable.

In the endgame, it is the edge pawns that are the most valuable.

So capture direction for greatest advantage can change.

Kings should hide in the corner until the endgame. In the endgame, they should centralize.

For many cases like this, a single PST will steer you wrong.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
User avatar
Evert
Posts: 2929
Joined: Sat Jan 22, 2011 12:42 am
Location: NL

Re: Sanity check on piece-value tables

Post by Evert »

hgm wrote:This suggest it might be andvatageous to have two Queen and Knight PST (four if you also count the opening/end-game diversification), one for each King on a-file, the other on h-file. And then interpolate based on the file the King is actually in.
That's actually not a bad idea. I might try that as a crude form of king-attack code.
I was thinking of something a bit different: under the assumption that the underlying table should be symmetric, extract the asymmetry under the assumption that it is due to the king's position (there'll be some contamination from centralised kings and long castling, but that can be dealt with) and use the asymmetry as a separate table shifted by the king's file. That probably works out to the same thing in the end.
Xiangqi engines actually use something similar for the Horse PST, not based on King position (which is pretty much fixed there, because of the Palace), but on whether you have an advantage or disadvantage in attacking material. When you are ahead your Horses should strongly be biased forward, but when you are behind, withdrawing them in a defensive position should be strongly encouraged.
That's quite a clever way to do that.
User avatar
hgm
Posts: 27790
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Sanity check on piece-value tables

Post by hgm »

Evert wrote:I was thinking of something a bit different: under the assumption that the underlying table should be symmetric, extract the asymmetry under the assumption that it is due to the king's position (there'll be some contamination from centralised kings and long castling, but that can be dealt with) and use the asymmetry as a separate table shifted by the king's file. That probably works out to the same thing in the end.
It would underestimate the magnitude of the asymmetry, unless the table was optimized under conditions where both sidea always castled King side. E.g. if they castled equally on both sides, there should not be an assymetry in the optimized tables at all.

If the difference in castling direction during PST optimization was large enough, you could apply a correction factor to the asymmetry to extrapolate it to 100%.