a Telltale position

[SuneF]

I feel that if a program does not see White's edge in the given position it is probably just undervaluing minor pieces. However a specific bonus for three minors vs. queen is the easiest fix here, I'm pretty sure your program must consistently overvalue the queen's side of this equation. It remains a mystery to me why the optimum value for minors (compared to majors) seems to be lower for engines than for humans. The values (for minors) that tested as best in Rybka are clearly too low to human eyes. That's why I always used Rybka 3 Human rather than the default for analysis of human games, it had the values much closer to what we human GMs think best.

Larry

When you say 'the value for minors' I assume you are speaking of the material score. In the evaluation, as you probably know too well, it all gets added up to a single number. When you have 3 minors vs a queen and you are just considering the material side of things it may seem the queen is overestimated, but the 3 minors will often add up to more when considering all the other terms - like mobility. If you consider the average value of the queen vs the average value of a minor it should get closer to a GMs assessment I think. It's just that estimating the average value of a piece is not so easy.
This is really one of the big problems with writing and tuning the evaluation, the terms are not orthogonal.

[michiguel]

I feel that if a program does not see White's edge in the given position it is probably just undervaluing minor pieces. However a specific bonus for three minors vs. queen is the easiest fix here, I'm pretty sure your program must consistently overvalue the queen's side of this equation. It remains a mystery to me why the optimum value for minors (compared to majors) seems to be lower for engines than for humans. The values (for minors) that tested as best in Rybka are clearly too low to human eyes. That's why I always used Rybka 3 Human rather than the default for analysis of human games, it had the values much closer to what we human GMs think best.

Larry

When you say 'the value for minors' I assume you are speaking of the material score. In the evaluation, as you probably know too well, it all gets added up to a single number. When you have 3 minors vs a queen and you are just considering the material side of things it may seem the queen is overestimated, but the 3 minors will often add up to more when considering all the other terms - like mobility. If you consider the average value of the queen vs the average value of a minor it should get closer to a GMs assessment I think. It's just that estimating the average value of a piece is not so easy.
This is really one of the big problems with writing and tuning the evaluation, the terms are not orthogonal.

I checked the terms of Gaviota evaluation. Based on pure material, the evaluation would be -0.10 (white's perspective, so black is better). But, when all the other terms are included (mobility etc.), the eval would be +0.60. So Sune's explanation applies. That is why, I cannot overvalue minors too much. But...

The reason why the eval is near neutral here (+0.07) is because I have a design problem (not a bug, there is no error in the code) in how to evaluate bad bishops. I see that -0.53 for bad bishops is clearly wrong in this position (I count all the pawns in the same color square and multiply by a factor). Here, all the non-blocked pawns in the second row should not be counted. I knew the term was too simplistic, but here it is exposed as really bad. Only d4 should contribute "a little" to make Bc1 "a tiny bit" bad. Maybe -0.10 but not more. That should bring the eval to ~+0.50, which is ok, IMHO.

So, Larry, my eval is poorly written, but for other reasons than material.

Miguel

[Ralph Stoesser]

I feel that if a program does not see White's edge in the given position it is probably just undervaluing minor pieces. However a specific bonus for three minors vs. queen is the easiest fix here, I'm pretty sure your program must consistently overvalue the queen's side of this equation. It remains a mystery to me why the optimum value for minors (compared to majors) seems to be lower for engines than for humans. The values (for minors) that tested as best in Rybka are clearly too low to human eyes. That's why I always used Rybka 3 Human rather than the default for analysis of human games, it had the values much closer to what we human GMs think best.

Larry

When you say 'the value for minors' I assume you are speaking of the material score. In the evaluation, as you probably know too well, it all gets added up to a single number. When you have 3 minors vs a queen and you are just considering the material side of things it may seem the queen is overestimated, but the 3 minors will often add up to more when considering all the other terms - like mobility. If you consider the average value of the queen vs the average value of a minor it should get closer to a GMs assessment I think. It's just that estimating the average value of a piece is not so easy.
This is really one of the big problems with writing and tuning the evaluation, the terms are not orthogonal.

If material evaluation is so error-prone why not drop it?
In a 60 games match, 2 seconds, Stockfish 2.0.1 non-material against Stockfish 2.0.1 material (default) I have

+12, -10, =38

Ok, this is admittedly pure luck (default plays stronger). but thinking about it, the non-material version is completely untuned!

[Christopher Conkie]

[d]8/8/8/1k3p2/p1p1pPp1/PpPpP1Pp/1P1P3P/QNK2NRR w - - 0 1

RobboLito 0085g3 x32:
1/3 00:00 5 0 +18.82 Rg2 hxg2
1/3 00:00 6 0 +25.17 Kd1
2/3 00:00 14 0 +25.32 Kd1 Kb6
3/5 00:00 32 0 +25.23 Kd1 Kb6 Ke1
4/5 00:00 59 0 +25.29 Kd1 Kb6 Ke1 Kb5
5/6 00:00 122 0 +25.39 Kd1 Kb6 Ke1 Kb5 Kf2
6/7 00:00 248 0 +25.49 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5
7/10 00:00 419 0 +22.08 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 hxg2 Kxg2
8/10 00:00 487 0 +22.08 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 hxg2 Kxg2
9/14+ 00:00 585 0 +22.16 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 hxg2 Kxg2
9/14+ 00:00 664 0 +22.24 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 hxg2 Kxg2
9/14+ 00:00 743 0 +22.36 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 hxg2 Kxg2
9/15 00:00 895 0 +22.48 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 hxg2 Kxg2 Kc5 h4 gxh3+ Rxh3
10/18 00:00 2,231 0 +22.46 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 Kb6 Kg1 hxg2 Kxg2 Kb5 h4 gxh3+ Rxh3
11/19+ 00:00 2,813 0 +22.54 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 Kb6 Kg1 hxg2 Kxg2 Kb5 h4 gxh3+ Rxh3
11/19+ 00:00 3,276 0 +22.62 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 Kb6 Kg1 hxg2 Kxg2 Kb5 h4 gxh3+ Rxh3
11/23 00:00 4,132 0 +22.68 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 Kb6 Kg1 hxg2 Kxg2 Ka5 h4 gxh3+ Rxh3 Kb5 Rh6
12/25+ 00:00 5,290 0 +22.76 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 Kb6 Kg1 hxg2 Kxg2 Ka5 h4 gxh3+ Rxh3 Kb5 Rh6
12/26+ 00:00 7,453 0 +22.84 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 Kb6 Kg1 hxg2 Kxg2 Ka5 h4 gxh3+ Rxh3 Kb5 Rh6
12/26+ 00:00 9,136 609,000 +22.96 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 Kb6 Kg1 hxg2 Kxg2 Ka5 h4 gxh3+ Rxh3 Kb5 Rh6
12/26 00:00 10,847 723,000 +22.98 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 Kb6 Kg1 hxg2 Kxg2 Kc5 h4 gxh3+ Kxh3 Kd5 g4 fxg4+ Kxg4

Houdini_w32_2CPU:
1/3 00:00 5 0 +18.81 Rg2 hxg2
1/3 00:00 6 0 +25.16 Kd1
2/3 00:00 14 0 +25.31 Kd1 Kb6
3/5 00:00 32 0 +25.23 Kd1 Kb6 Ke1
4/5 00:00 59 59,000 +25.28 Kd1 Kb6 Ke1 Kb5
5/6 00:00 122 122,000 +25.39 Kd1 Kb6 Ke1 Kb5 Kf2
6/7 00:00 248 24,000 +25.49 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5
7/10 00:00 324 32,000 +22.08 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 hxg2 Kxg2
8/10 00:00 392 39,000 +22.08 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 hxg2 Kxg2
9/15 00:00 547 54,000 +22.48 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 hxg2 Kxg2 Kc5 h4 gxh3+ Rxh3
10/21 00:00 1,462 132,000 +22.46 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 Kb6 Kg1 hxg2 Kxg2 Kb5 h4 gxh3+ Rxh3
11/23 00:00 2,384 198,000 +22.69 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 Kb6 Kg1 hxg2 Kxg2 Ka5 h4 gxh3+ Rxh3 Kb5 Rh6
12/25 00:00 5,099 339,000 +22.98 Kd1 Kb6 Ke1 Ka5 Kf2 Kb5 Rg2 Kb6 Kg1 hxg2 Kxg2 Kc5 h4 gxh3+ Kxh3 Kd5 g4 fxg4+ Kxg4

[Christopher Conkie]

[d]1r3r2/q5k1/4p1n1/1bPpPp1p/pPpR1Pp1/P1B1Q3/2B3PP/3R2K1 w - - 0 1

RobboLito 0085g3 x32:
1/4 00:00 46 0 -1.63 Bxf5 Rxf5
1/8 00:00 75 0 -0.18 Rxd5 exd5 e6+ Kg8 Rxd5
1/8 00:00 92 0 +0.97 R4d2
2/11 00:00 344 0 +0.87 R4d2 Bc6
3/11 00:00 635 0 +0.86 R4d2 Bc6 g3 Qd7
4/11 00:00 950 0 +0.85 R4d2 Bc6 g3 h4
5/13 00:00 1,620 0 +0.75 R4d2 Bc6 g3 h4 Bd4 Qd7


Houdini_w32_2CPU:
1/4 00:00 46 0 -1.26 Bxf5 Rxf5
1/8 00:00 75 0 -0.11 Rxd5 exd5 e6+ Kg8 Rxd5
1/8 00:00 100 0 +0.67 R4d2
2/11 00:00 452 7,000 +0.58 R4d2 Bc6
3/11 00:00 750 11,000 +0.56 R4d2 Bc6 g3 Qd7
4/11 00:00 1,072 14,000 +0.56 R4d2 Bc6 g3 h4
5/13 00:00 1,768 22,000 +0.47 R4d2 Bc6 g3 h4 Bd4 Qd7

[Christopher Conkie]

[d]7r/2k2p2/q1b1p1p1/p2pP1P1/n2P1P2/Q1p2B2/P1P2N1r/1RR2K2 w - - 0 44

RobboLito 0085g3 x32:
1/7 00:00 28 0 -1.36 Be2 Rxf2+ Kxf2 Rh2+ Kg3 Qxe2
1/15 00:00 123 0 +0.33 Nd3 Kc8 Qd6
2/15 00:00 245 0 +0.33 Nd3 Kc8 Qd6
3/15 00:00 534 0 -0.04 Nd3 Kc8 Bg2
4/21 00:00 1,765 0 0.00 Nd3 Kc8 Bg2 Qc4
5/21 00:00 2,872 0 -0.07 Nd3 Kc8 Bg2 Qc4 Qe7 Nb2
5/21 00:00 4,649 0 +1.70 Be2 Qa8 Qe7+ Kc8 Qxf7 Bd7 Qxg6 Nb2
6/21 00:00 5,433 0 +1.70 Be2 Qa8 Qe7+ Kc8 Qxf7 Bd7 Qxg6 Nb2
7/21 00:00 6,818 0 +1.67 Be2 Qa8 Qe7+ Kc8 Qxf7 Bd7 Qxg6 Nb2 Qf7 Qb7
8/21+ 00:00 8,066 0 +1.75 Be2 Qa8 Qe7+ Kc8 Qxf7 Bd7 Qxg6 Nb2 Qf7 Qb7
8/21+ 00:00 11,849 740,000 +1.83 Be2 Qa8 Qe7+ Kc8 Qxf7 Bd7 Qxg6 Nb2 Ng4 Rh1+ Kf2

Houdini_w32_2CPU:
1/7 00:00 28 0 -1.00 Be2 Rxf2+ Kxf2 Rh2+ Kg3 Qxe2
1/15 00:00 116 1,000 +0.20 Nd3 Kc8 Qd6
2/15 00:00 242 3,000 +0.20 Nd3 Kc8 Qd6
3/15 00:00 576 7,000 -0.01 Nd3 Kc8 Bg2
4/18 00:00 1,695 20,000 0.00 Nd3 Kc8 Bg2 Qc4
5/18 00:00 2,836 34,000 -0.03 Nd3 Kc8 Bg2 Qc4 Qe7 Nb2
5/18 00:00 4,537 53,000 +1.35 Be2 Qa8 Qe7+ Kc8 Qxf7 Bd7 Qxg6 Nb2
6/18 00:00 5,372 62,000 +1.35 Be2 Qa8 Qe7+ Kc8 Qxf7 Bd7 Qxg6 Nb2
7/18 00:00 6,867 78,000 +1.32 Be2 Qa8 Qe7+ Kc8 Qxf7 Bd7 Qxg6 Nb2 Qf7 Qb7
8/18+ 00:00 8,115 91,000 +1.41 Be2 Qa8 Qe7+ Kc8 Qxf7 Bd7 Qxg6 Nb2 Qf7 Qb7
8/21+ 00:00 12,638 108,000 +1.59 Be2 Qa8 Qe7+ Kc8 Qxf7 Bd7 Qxg6 Nb2 Ng4 Rh1+ Kf2

[Christopher Conkie]

[d]r3k2r/pp1n1p1p/2p3p1/3p1b2/1q1Pn1P1/2N1PQ1P/PP2NP2/2KR1B1R b - - 0 1

RobboLito 0085g3 x32:
1/8 00:00 95 0 +0.15 Nxc3 Nxc3
1/8 00:00 242 0 +0.46 Ng5
2/8 00:00 330 0 +0.37 Ng5 Qf4
3/12 00:00 1,117 0 +0.31 Ng5 Qf4 Ne6
4/12 00:00 1,530 0 +0.25 Ng5 Qf4 Ne6 Qh6
5/12 00:00 2,205 0 -0.03 Ng5 Qf4 Ne6 Qh6 Be4 Nxe4 dxe4
6/13 00:00 3,199 0 -0.03 Ng5 Qf4 Ne6 Qh6 Be4 Nxe4 dxe4
7/17 00:00 9,927 661,000 -0.38 Ng5 Qf4 Be4 Nxe4 Nxe4 Kb1 g5 Qf5
8/21 00:00 12,813 854,000 -0.38 Ng5 Qf4 Be4 Nxe4 Nxe4 Kb1 Nd2+ Ka1 Nc4 b3

Houdini_w32_2CPU:
1/8 00:00 98 0 +0.09 Nxc3 Nxc3
1/8 00:00 248 0 +0.29 Ng5
2/8 00:00 336 0 +0.23 Ng5 Qf4
3/13 00:00 1,131 0 +0.19 Ng5 Qf4 Ne6
4/13 00:00 1,547 96,000 +0.15 Ng5 Qf4 Ne6 Qh6
5/13 00:00 2,225 139,000 -0.01 Ng5 Qf4 Ne6 Qh6 Be4 Nxe4 dxe4
6/13 00:00 3,215 200,000 -0.01 Ng5 Qf4 Ne6 Qh6 Be4 Nxe4 dxe4
7/17 00:00 8,904 556,000 -0.24 Ng5 Qf4 Be4 Nxe4 Nxe4 Kb1 g5 Qf5
8/17 00:00 11,791 736,000 -0.24 Ng5 Qf4 Be4 Nxe4 Nxe4 Kb1 Nd2+ Ka1 Nc4 b3

[Christopher Conkie]

[d]r2qrb1k/1p1b2p1/p2ppn1p/8/3NP3/1BN5/PPP3QP/1K3RR1 w - - 0 1

RobboLito 0085g3 x32:
1/4 00:00 46 0 -1.77 Rxf6 Qxf6
1/5 00:00 60 0 -0.56 e5 dxe5
1/5 00:00 69 0 -0.26 Nf3
1/5 00:00 78 0 -0.13 Nce2
1/5 00:00 84 0 -0.09 Bc4
1/5 00:00 88 0 0.00 h4


Houdini_w32_2CPU:
1/4 00:00 46 0 -1.18 Rxf6 Qxf6
1/5 00:00 60 0 -0.20 e5 dxe5
1/5 00:00 69 0 0.00 Nf3
1/5 00:00 78 0 +0.06 Nce2
1/5 00:00 84 0 +0.09 Bc4
1/5 00:00 88 0 +0.14 h4

[Christopher Conkie]

[d]1kb5/1p1pN1K1/pP1P4/P7/8/8/8/3b4 w - - 17 9

RobboLito 0085g3 x32:
1/3 00:00 16 0 -2.99 Nxc8 Kxc8
1/3 00:00 17 0 -2.71 Kh8
1/4 00:00 22 0 -2.52 Kg8
1/5 00:00 26 0 -2.34 Kf8 Bb3
1/5 00:00 43 1,000 -2.21 Kf7 Bh5+ Kf6
1/5 00:00 56 1,000 -2.19 Kg6 Bb3 Kf6
1/5 00:00 62 2,000 -2.11 Kf6 Bh5
2/5 00:00 88 2,000 -2.18 Kf6 Bf3
3/6 00:00 163 5,000 -2.10 Kf6 Bf3 Ke5
4/8 00:00 276 8,000 -2.07 Kf6 Bf3 Ke5 Bc6 Nxc6+ bxc6
5/8 00:00 562 18,000 -2.20 Kf6 Bg4 Ke5 Be6 Ng6
6/9 00:00 913 28,000 -2.08 Kf6 Bg4 Ke5 Be6 Ng6 Bf7
7/10 00:00 1,542 48,000 -2.14 Kg6 Bb3 Kf6 Be6 Ke5 Bf7 Nf5
8/10 00:00 2,187 66,000 -2.08 Kg6 Bb3 Kf6 Bc4 Ke5 Be6 Ng6 Bf7
9/11 00:00 2,981 87,000 -2.14 Kg6 Bb3 Kf6 Bc4 Ke5 Bf7 Nf5 Be6 Ne7 Bf7


Houdini_w32_2CPU:
1/3 00:00 16 0 -2.80 Nxc8 Kxc8
1/3 00:00 17 0 -2.48 Kh8
1/4 00:00 21 0 -2.26 Kg8
1/5 00:00 25 0 -2.07 Kf8 Bb3
1/5 00:00 40 0 -1.92 Kf7 Bh5+ Kf6
1/5 00:00 51 0 -1.89 Kg6 Bb3 Kf6
1/5 00:00 57 0 -1.81 Kf6 Bh5
2/5 00:00 80 0 -1.89 Kf6 Bf3
3/6 00:00 155 0 -1.80 Kf6 Bf3 Ke5
4/8 00:00 267 0 -1.76 Kf6 Bf3 Ke5 Bc6 Nxc6+ bxc6
5/8 00:00 549 0 -1.91 Kf6 Bg4 Ke5 Be6 Ng6
6/9 00:00 965 0 -1.77 Kf6 Bg4 Ke5 Be6 Ng6 Bf7
7/10 00:00 1,561 0 -1.83 Kg6 Bb3 Kf6 Be6 Ke5 Bf7 Nf5
8/10 00:00 2,244 0 -1.77 Kg6 Bb3 Kf6 Bc4 Ke5 Be6 Ng6 Bf7
9/11 00:00 3,136 0 -1.83 Kg6 Bb3 Kf6 Bc4 Ke5 Bf7 Nf5 Be6 Ne7 Bf7

[Christopher Conkie]

[d]r3nk2/2qb1p2/3ppPp1/p3b2p/1p2PN1Q/1B1R4/PPP3PP/5RK1 b - - 0 1

RobboLito 0085g3 x32:
1/7 00:00 66 0 -1.70 Bxf4 Qxf4 Qc5+ Kh1
1/18 00:00 176 0 -1.25 Nxf6 Nxg6+ fxg6 Rxf6+ Kg7
1/18 00:00 202 0 -0.42 Bxb2
1/18 00:00 243 0 -0.27 Qa7+ Kh1 Nxf6
1/18 00:00 262 0 -0.11 Qb6+ Kh1 Nxf6
1/18 00:00 295 0 -0.10 Qc5+ Qf2 Qxf2+ Rxf2 Nxf6
2/18 00:00 348 0 -0.10 Qc5+ Qf2 Qxf2+ Rxf2 Nxf6
3/18 00:00 930 0 -0.10 Qc5+ Qf2 Qxf2+ Rxf2 Nxf6
3/18 00:00 1,737 0 -0.08 Rc8 Ne2 Qc5+ Qf2 Qxf2+ Kxf2 Nxf6
3/18 00:00 4,546 0 +0.15 a4 Bxe6 Qc5+ Kh1 Bxe6 Nxe6+ fxe6
4/18 00:00 4,858 0 +0.15 a4 Bxe6 Qc5+ Kh1 Bxe6 Nxe6+ fxe6

Houdini_w32_2CPU:
1/7 00:00 65 0 -1.35 Bxf4 Qxf4 Qc5+ Kh1
1/18 00:00 183 0 -0.91 Nxf6 Nxg6+ fxg6 Rxf6+ Kg7
1/18 00:00 209 0 -0.26 Bxb2
1/18 00:00 250 0 -0.16 Qa7+ Kh1 Nxf6
1/18 00:00 269 0 -0.06 Qb6+ Kh1 Nxf6
1/18 00:00 303 0 -0.05 Qc5+ Qf2 Qxf2+ Rxf2 Nxf6
2/18 00:00 357 10,000 -0.05 Qc5+ Qf2 Qxf2+ Rxf2 Nxf6
3/18 00:00 924 17,000 -0.05 Qc5+ Qf2 Qxf2+ Rxf2 Nxf6
3/18 00:00 1,797 33,000 -0.05 Rc8 Ne2 Qc5+ Qf2 Qxf2+ Kxf2 Nxf6
3/18 00:00 4,848 83,000 +0.09 a4 Bxe6 Qc5+ Kh1 Bxe6 Nxe6+ fxe6
4/18 00:00 5,215 88,000 +0.09 a4 Bxe6 Qc5+ Kh1 Bxe6 Nxe6+ fxe6