Testing of Kaufman Material Values

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
mjlef
Posts: 1432
Joined: Thu Mar 30, 2006 12:08 pm
Contact:

Testing of Kaufman Material Values

Post by mjlef » Sat Apr 25, 2009 11:57 pm

Has anyone done some meaningful testing of the Larry Kaufman material values? In particular, does decreasing knight values as pawns come off the board or increasing rook values for the same seem to help?

Perhaps Bob has done a nice long, meaningful run on his machines?

Mark

bob
Posts: 20795
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Testing of Kaufman Material Values

Post by bob » Sun Apr 26, 2009 1:04 am

mjlef wrote:Has anyone done some meaningful testing of the Larry Kaufman material values? In particular, does decreasing knight values as pawns come off the board or increasing rook values for the same seem to help?

Perhaps Bob has done a nice long, meaningful run on his machines?

Mark
Tracy wrote some code to do the things Larry suggested. However we found nothing at all that made any difference. The things he suggested with respect to pawns and pieces, redundant pieces, coordinated pieces, etc, all had no significant effect, plus or minus.

May well work for others, but for us it was a non-working idea. Note that many terms are sort of redundant. If you evaluate mobility for a rook, then rook on open file scoring overlaps, since you have more mobility on an open file. And you have more mobility as pawns come off. So the overall effect probably is sensitive to what you are already evaluating...

Osipov Jury
Posts: 186
Joined: Mon Jan 21, 2008 1:07 pm
Location: Russia

Re: Testing of Kaufman Material Values

Post by Osipov Jury » Sun Apr 26, 2009 8:17 am

From my test: Rybka 3 without material imbalance values - minus 68 points Elo.

MattieShoes
Posts: 718
Joined: Fri Mar 20, 2009 7:59 pm

Re: Testing of Kaufman Material Values

Post by MattieShoes » Sun Apr 26, 2009 9:57 am

"material imbalance values"? By this, do you mean RP vs BN, etc?

Osipov Jury
Posts: 186
Joined: Mon Jan 21, 2008 1:07 pm
Location: Russia

Re: Testing of Kaufman Material Values

Post by Osipov Jury » Sun Apr 26, 2009 10:41 am

MattieShoes wrote:"material imbalance values"? By this, do you mean RP vs BN, etc?
Yes, I mean RP vs BN, PPP vs B or N, and full complex of another combinations. Rybka 3 calcullate this values in start of engine. Function contain approximately 200 lines of code.

Uri Blass
Posts: 8729
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Testing of Kaufman Material Values

Post by Uri Blass » Sun Apr 26, 2009 11:51 am

Osipov Jury wrote:From my test: Rybka 3 without material imbalance values - minus 68 points Elo.
I suspect that you can get most of this improvement without material imbalance tables only by changing the value of pieces.

Uri

bob
Posts: 20795
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Testing of Kaufman Material Values

Post by bob » Sun Apr 26, 2009 4:04 pm

MattieShoes wrote:"material imbalance values"? By this, do you mean RP vs BN, etc?
Yes. Ideas like a second knight is not as valuable as the first, due to redundancy, etc. Which is an extension of the usual BN vs RP is bad. But it is not quite as bad if you have a second knight as getting rid of one also removes a little bit of redundancy. Other ideas include "queen is more valuable with more pawns on the board, rooks are more valuable as pawns come off" and such.

I tested the queen idea using a bonus of 0 to 12 for each pawn on the board. This only does anything when you have a Q vs 2R for example. Since there could be 16 pawns, the bonus went up to 12*16. The result was absolutely no Elo change. I played 32,000 games for each of those bonus values, 0, 1, ..., 12. Zero change. All ended within 3 Elo of each other, which is well inside the error bar for 32,000 games.

bob
Posts: 20795
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Testing of Kaufman Material Values

Post by bob » Sun Apr 26, 2009 4:05 pm

Uri Blass wrote:
Osipov Jury wrote:From my test: Rybka 3 without material imbalance values - minus 68 points Elo.
I suspect that you can get most of this improvement without material imbalance tables only by changing the value of pieces.

Uri
I agree. We have done a few million games of testing to choose our current piece values. After that, none of Larry's suggestions have helped at all. Static values are far easier/faster to use than dynamic values. And if the static values are as accurate, there's no reason to not use them.

mjlef
Posts: 1432
Joined: Thu Mar 30, 2006 12:08 pm
Contact:

Re: Testing of Kaufman Material Values

Post by mjlef » Sun Apr 26, 2009 10:51 pm

Programs with material tables tend to encode lost of knowledge handled by rules in other programs (like a KB vs KP is most like a draw, and no real advantage to the B side). I wonder how much of the rating increase is due to these simple rules, versus the full board material imbalance terms?

User avatar
hgm
Posts: 24124
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Testing of Kaufman Material Values

Post by hgm » Mon Apr 27, 2009 8:34 am

My suspicion is that most of the Elo difference is caused by the Bishop pair. I assume that its value is implemented in the material tables; as it could be implemented there by simply initializing the tables differently, it would be silly to implement it by a seperate eval rule, which would take execution time. But of course any self-respecting program that does not use materia tables, still implements the Bishop pair through a dedicated rule.

So it is really a bit of a meaningless comparison. You cannot switch of the subtle effects of the material table without removing very basic knowledge that almost very engine has.

Post Reply