Stockfish - material balance/imbalance evaluation

[diep]

A lot of projects suddenly got funded when i and some others formulated new forms of parameter tuning and thought this would lead to selflearning and terminator chips.
Vincent

It is quite interesting. Have you actually published any of your ideas concerning new forms of parameter tuning? What is it about?

Tord, Remi and Bob are among the most influential persons in computer chess field. Why do you think they have produced "crap"? In the case of Remi I guess it was a clear reference to his QLR tool, right?

Can you at least explain why in your view they are so out of line? What is the "real" thing which "you and others" have invented?

Thanks,
Ben-Hur

QLR scales very bad and eats too much system time and was already outgunned in efficiency by some year 1983 type algorithm, or simple adaptation from it.

I'm always interested in investors, i don't get paid for this type of research currently. Even projects with a clear goal and pay back penalty when goal doesn't get reached could be acceptable (if the reward is big enough). I can forward you my resume if you're interested.

Realize first you have to succeed in tuning a lot of parameters. That eats big system time. After you succeeded there, you can try to optimize the proces to a more efficient way of tuning. The thing is that QLR fails directly when trying to scale to a lot of parameters.

Vincent

[grzegorzs]

To my calculation, as forwarded to several, the total system time used up for parameter tuning of the rybka* type engines must be roughly around 100 million cpu node hours, or at the expensive government hardware that's roughly a budget of $50 million.

A $50 million tuning project that can't even tune TM properly, ROFL .
I wish I could believe in Santa too .

I don't know what exactly you can't believe in but if I good remember in the past a lot of money was spent on computer chess.
Lets remind you Cray Blitz, Hitech, Deep Thought and Blue.
Hydra or whatever it was called also wasn't quite cheap project probably.
So we could ask question - when Kasparov was defeated, all interest in computer chess disappeared?
Why in the past someone was interested in spending money?

I don't believe in complicated conspiracy theory but we have to admit that generally speaking tuning process can be interesting for industry, medical advances and probably also for military development and so on and so on.

In recent years I observed progress in some programs absolutely not involved in any dirty job where progress was achieved only because of tuning. But we can't blindly exclude that any such "dirty job" not exists at all.

[diep]

V-

Do you think this underground government-backed 100 million cpunodehours $50 million chess engine tuning cabal used by the russian ippo* stooges is related in any way to that recent spy swap between US and Russia?

Of course there is a relationship.

The relationship is that the governments, both European as well as North-American ones, have increased the budgets of the intelligence agencies too much.

More than doubled.

So the 1200 organisations in USA that have intelligence gathering activities of one or another kind, companies excluded, really have a lot more to spend then in every direction. (for comparision Netherlands has 100 legal organisions and 1020+ illegal ones).

You see the result of that, both in computerchess as well as in spy catching.

The real question is of course what is espionage. For example i negotiated at a library software semi-government body near The Hague. The only thing they do is gather information from entire Europe regarding academic publications and writings, put it in a database, and ship all information to USA. Nothing comes back.

Is that espionage?

Vincent

[wgarvin]

I always enjoy reading Vincent's posts. They are a highly unique blend of computer chess info, programming info and weird conspiracy theories.
Keep em coming, Vincent!

[tungoctrung]

Sorry for my very late reply. I've recently found this topic.
I have some questions:
- In following code (current StockFish snapshot in Github), please notice code in bold format.

Code: Select all

namespace {
  //...
  // Polynomial material balance parameters
  const Value RedundantQueenPenalty = Value(320);
  const Value RedundantRookPenalty  = Value(554);
  
  const int LinearCoefficients[6] = { 1617, -162, -1172, -190, 105, 26 };

Code: Select all

/*
  const int QuadraticCoefficientsSameColor[][8] = {
  { 7, 7, 7, 7, 7, 7 }, { 39, 2, 7, 7, 7, 7 }, { 35, 271, -4, 7, 7, 7 },
  { 7, 25, 4, 7, 7, 7 }, { -27, -2, 46, 100, 56, 7 }, { 58, 29, 83, 148, -3, -25 } };

  const int QuadraticCoefficientsOppositeColor[][8] = {
  { 41, 41, 41, 41, 41, 41 }, { 37, 41, 41, 41, 41, 41 }, { 10, 62, 41, 41, 41, 41 },
  { 57, 64, 39, 41, 41, 41 }, { 50, 40, 23, -22, 41, 41 }, { 106, 101, 3, 151, 171, 41 } };
*/
  // Re-formatted:
  const int QuadraticCoefficientsSameColor[][8] = {
      {   7,   7,  7,   7,  7, 7 },
      {  39,   2,  7,   7,  7, 7 },
      {  35, 271, -4,   7,  7, 7 },
      {   7,  25,  4,   7,  7, 7 },
      { -27,  -2, 46, 100, 56, 7 },
      {  58,  29, 83, 148, -3, -25 } };

  const int QuadraticCoefficientsOppositeColor[][8] = {
      { 41,  41, 41,  41,  41, 41 },
      { 37,  41, 41,  41,  41, 41 },
      { 10,  62, 41,  41,  41, 41 },
      { 57,  64, 39,  41,  41, 41 },
      { 50,  40, 23, -22,  41, 41 },
      { 106, 101, 3, 151, 171, 41 } };

Please notice:
const int QuadraticCoefficientsSameColor[][8] = {
{ 7, 7, 7, 7, 7, 7 },
{ 39, 2, 7, 7, 7, 7 },
{ 35, 271, -4, 7, 7, 7 },
{ 7, 25, 4, 7, 7, 7 },
{ -27, -2, 46, 100, 56, 7 },
{ 58, 29, 83, 148, -3, -25 } };

const int QuadraticCoefficientsOppositeColor[][8] = {
{ 41, 41, 41, 41, 41, 41 },
{ 37, 41, 41, 41, 41, 41 },
{ 10, 62, 41, 41, 41, 41 },
{ 57, 64, 39, 41, 41, 41 },
{ 50, 40, 23, -22, 41, 41 },
{ 106, 101, 3, 151, 171, 41 } };

Code: Select all

template<Color Us>
int MaterialTable::imbalance(const int pieceCount[][8]) {
  //...
  // Redundancy of major pieces, formula based on Kaufman's paper
  // "The Evaluation of Material Imbalances in Chess"
  if (pieceCount[Us][ROOK] > 0)
      value -=  RedundantRookPenalty * (pieceCount[Us][ROOK] - 1)
              + RedundantQueenPenalty * pieceCount[Us][QUEEN];

  // Second-degree polynomial material imbalance by Tord Romstad
  for (pt1 = NO_PIECE_TYPE; [b]pt1 <= QUEEN[/b]; pt1++)
  {
      pc = pieceCount[Us][pt1];
      if (!pc)
          continue;

      v = LinearCoefficients[pt1];

      for (pt2 = NO_PIECE_TYPE; pt2 <= pt1; pt2++)
          v +=  QuadraticCoefficientsSameColor[pt1][pt2] * pieceCount[Us][pt2]
              + QuadraticCoefficientsOppositeColor[pt1][pt2] * pieceCount[Them][pt2];

      value += pc * v;
  }

Please notice:
for (pt2 = NO_PIECE_TYPE; pt2 <= pt1; pt2++)

Questions:
1. What is the meaning of RedundantQueenPenalty, RedundantRookPenalty and why they are used in following code?

Code: Select all

  if (pieceCount[Us][ROOK] > 0)
      value -=  RedundantRookPenalty * (pieceCount[Us][ROOK] - 1)
              + RedundantQueenPenalty * pieceCount[Us][QUEEN];

2. What is the meaning of LinearCoefficients values?

Code: Select all

  const int LinearCoefficients[6] = { 1617, -162, -1172, -190, 105, 26 };

3. Why do you use for (pt2 = NO_PIECE_TYPE; pt2 <= pt1; pt2++) ? I think it should be for (pt2 = NO_PIECE_TYPE; pt2 <= QUEEN; pt2++). Because if you keep the current code, program will scan thought the arrays QuadraticCoefficientsXXX as I mentioned in blue bold format. Numbers of loop iterations will be (1+2+3+4+5+6)=21 instead of 6x6=36. And red-italic values of QuadraticCoefficientsXXX arrays will not be calculated.
4. Why do you calculate
v += QuadraticCoefficientsSameColor[pt1][pt2] * pieceCount[Us][pt2] + QuadraticCoefficientsOppositeColor[pt1][pt2] * pieceCount[Them][pt2]; ?
Why not calculate
v += QuadraticCoefficientsSameColor[pt1][pt2] * pieceCount[Us][pt2] + QuadraticCoefficientsOppositeColor[pt1][pt2] * pieceCount[Us][pt2]; ?
Because the meaning of array name I understand here is Opposite color but not Opponent color.

Thanks & regards,

[Ferdy]

Questions:
1. What is the meaning of RedundantQueenPenalty, RedundantRookPenalty and why they are used in following code?

Imagine you have 2 rooks. So 2 rooks are doing same thing, both can only move horizontal and vertical. Perhaps most positions do not need 2 pieces functioning the same thing. So a small penalty is applied. May also depend on the material remaining on the board, pawn structures, number of pawns and others.

A very nice reference here.
http://home.comcast.net/~danheisman/Art ... alance.htm

[lkaufman]

In short, the idea is that pieces affect the value of each other. Here we give bonus and penalties for different duos (like bishop and knight, pawn and pawn, queen and rook, etc.). "Bishop pair" is a special piece in this implementation.

But as Marco already mentioned, this is not a big deal, maybe around +15 elo or sth like that. We expected more, but fine tuning was extremely difficult.

Is this +15 elo for the duos other than bishop pair and redundant major pieces, or does it include either (or both) of them? If it excludes those duos, I would say that +15 elo is pretty good. I have never been convinced that any duo other than those plus various (piece with pawn) terms have value. Possibly a slight negative for the knight pair may be justified. Perhaps some of the duos involving enemy pieces might pay off, I never did much work on that.

[tungoctrung]

Questions:
1. What is the meaning of RedundantQueenPenalty, RedundantRookPenalty and why they are used in following code?

Imagine you have 2 rooks. So 2 rooks are doing same thing, both can only move horizontal and vertical. Perhaps most positions do not need 2 pieces functioning the same thing. So a small penalty is applied. May also depend on the material remaining on the board, pawn structures, number of pawns and others.

A very nice reference here.
http://home.comcast.net/~danheisman/Art ... alance.htm

Thank you for your explanation, I've already read Kaufman paper before I post this question. I'm waiting StockFish developers to explain more clearly and exactly.

[Eelco de Groot]

Sorry for my very late reply. I've recently found this topic.
I have some questions:
- In following code (current StockFish snapshot in Github), please notice code in bold format.

Code: Select all

namespace {
  //...
  // Polynomial material balance parameters
  const Value RedundantQueenPenalty = Value(320);
  const Value RedundantRookPenalty  = Value(554);
  
  const int LinearCoefficients[6] = { 1617, -162, -1172, -190, 105, 26 };

Code: Select all

/*
  const int QuadraticCoefficientsSameColor[][8] = {
  { 7, 7, 7, 7, 7, 7 }, { 39, 2, 7, 7, 7, 7 }, { 35, 271, -4, 7, 7, 7 },
  { 7, 25, 4, 7, 7, 7 }, { -27, -2, 46, 100, 56, 7 }, { 58, 29, 83, 148, -3, -25 } };

  const int QuadraticCoefficientsOppositeColor[][8] = {
  { 41, 41, 41, 41, 41, 41 }, { 37, 41, 41, 41, 41, 41 }, { 10, 62, 41, 41, 41, 41 },
  { 57, 64, 39, 41, 41, 41 }, { 50, 40, 23, -22, 41, 41 }, { 106, 101, 3, 151, 171, 41 } };
*/
  // Re-formatted:
  const int QuadraticCoefficientsSameColor[][8] = {
      {   7,   7,  7,   7,  7, 7 },
      {  39,   2,  7,   7,  7, 7 },
      {  35, 271, -4,   7,  7, 7 },
      {   7,  25,  4,   7,  7, 7 },
      { -27,  -2, 46, 100, 56, 7 },
      {  58,  29, 83, 148, -3, -25 } };

  const int QuadraticCoefficientsOppositeColor[][8] = {
      { 41,  41, 41,  41,  41, 41 },
      { 37,  41, 41,  41,  41, 41 },
      { 10,  62, 41,  41,  41, 41 },
      { 57,  64, 39,  41,  41, 41 },
      { 50,  40, 23, -22,  41, 41 },
      { 106, 101, 3, 151, 171, 41 } };

Please notice:
const int QuadraticCoefficientsSameColor[][8] = {
{ 7, 7, 7, 7, 7, 7 },
{ 39, 2, 7, 7, 7, 7 },
{ 35, 271, -4, 7, 7, 7 },
{ 7, 25, 4, 7, 7, 7 },
{ -27, -2, 46, 100, 56, 7 },
{ 58, 29, 83, 148, -3, -25 } };

const int QuadraticCoefficientsOppositeColor[][8] = {
{ 41, 41, 41, 41, 41, 41 },
{ 37, 41, 41, 41, 41, 41 },
{ 10, 62, 41, 41, 41, 41 },
{ 57, 64, 39, 41, 41, 41 },
{ 50, 40, 23, -22, 41, 41 },
{ 106, 101, 3, 151, 171, 41 } };

Code: Select all

template<Color Us>
int MaterialTable::imbalance(const int pieceCount[][8]) {
  //...
  // Redundancy of major pieces, formula based on Kaufman's paper
  // "The Evaluation of Material Imbalances in Chess"
  if (pieceCount[Us][ROOK] > 0)
      value -=  RedundantRookPenalty * (pieceCount[Us][ROOK] - 1)
              + RedundantQueenPenalty * pieceCount[Us][QUEEN];

  // Second-degree polynomial material imbalance by Tord Romstad
  for (pt1 = NO_PIECE_TYPE; [b]pt1 <= QUEEN[/b]; pt1++)
  {
      pc = pieceCount[Us][pt1];
      if (!pc)
          continue;

      v = LinearCoefficients[pt1];

      for (pt2 = NO_PIECE_TYPE; pt2 <= pt1; pt2++)
          v +=  QuadraticCoefficientsSameColor[pt1][pt2] * pieceCount[Us][pt2]
              + QuadraticCoefficientsOppositeColor[pt1][pt2] * pieceCount[Them][pt2];

      value += pc * v;
  }

Please notice:
for (pt2 = NO_PIECE_TYPE; pt2 <= pt1; pt2++)

Questions:
1. What is the meaning of RedundantQueenPenalty, RedundantRookPenalty and why they are used in following code?

Code: Select all

  if (pieceCount[Us][ROOK] > 0)
      value -=  RedundantRookPenalty * (pieceCount[Us][ROOK] - 1)
              + RedundantQueenPenalty * pieceCount[Us][QUEEN];

2. What is the meaning of LinearCoefficients values?

Code: Select all

  const int LinearCoefficients[6] = { 1617, -162, -1172, -190, 105, 26 };

3. Why do you use for (pt2 = NO_PIECE_TYPE; pt2 <= pt1; pt2++) ? I think it should be for (pt2 = NO_PIECE_TYPE; pt2 <= QUEEN; pt2++). Because if you keep the current code, program will scan thought the arrays QuadraticCoefficientsXXX as I mentioned in blue bold format. Numbers of loop iterations will be (1+2+3+4+5+6)=21 instead of 6x6=36. And red-italic values of QuadraticCoefficientsXXX arrays will not be calculated.

Hello Mr. Trong,

I think you have answered already most of your questions, more or less. The numbers in the arrays in red indeed are not used in the calculations, I'm not quite sure if there is a purpose to the red values there but you are correct, they are not used because if you did, every possible combination of pieces would then be visited twice which would be not so efficient, all the values used would have to be half of what they are now, or at least the two corresponding values would have to add up to the number used now. So it is a slight optimization to traverse only half the array.

4. Why do you calculate
v += QuadraticCoefficientsSameColor[pt1][pt2] * pieceCount[Us][pt2] + QuadraticCoefficientsOppositeColor[pt1][pt2] * pieceCount[Them][pt2]; ?
Why not calculate
v += QuadraticCoefficientsSameColor[pt1][pt2] * pieceCount[Us][pt2] + QuadraticCoefficientsOppositeColor[pt1][pt2] * pieceCount[Us][pt2]; ?
Because the meaning of array name I understand here is Opposite color but not Opponent color.

Well no again I think you have answered it already, the meaning here is the number of corresponding pieces from the opponent, not the opposite color as in bishops on dark sques and on light squares. It is mainly in case of bishops that you might make that distinction, and maybe for pawns too, especialy in a blocked position they can be fixed to one type of square colour, but that distinction is not made in this table, it would probably make it too complicated and also in the rest of the eval the distinction between dark squared and light squared bishops is not made, only in very special cases like promoting passed pawns and bishops of the wrong colour, so it would probably not make enough sense to do that here if you don't do it elsewhere. But it would make an interesting experiment for sure, the original poster Ralph already made that suggestion at the beginning of the thread, he asked why not separate the two bishops here already because they might also have different relations with opponent pieces, especially with the bishops of the opponent.

I am sure the programmers could give more details, especially about the tuning but Tord is not frequenting here very often especially if it is summer in Norway he is maybe, like the bishops, running left and right cross country right now

Regards, Eelco

Thanks & regards,