mistake/bug in evaluation or normal behaviore ?

MahmoudUthman · Post by **MahmoudUthman** » Thu Oct 20, 2016 12:21 am

In the 5th move my engine lets the knight on the left be captured , stock fish indicates that the right move is b5a3, is this normal or does it indicate an error/bug in the evaluation , and if so what could be the most likely cause ?
this was the output of the engine :

Code: Select all

position startpos moves g1f3 d7d5 b1c3 d5d4 c3b5 e7e5 e2e3 a7a6
go wtime 53440 btime 52394 winc 0 binc 0

info depth 1 score cp 0 EBF 0 nodes 62 time 1 nps 309062 pv f3e5
info depth 2 score cp -24 EBF 1 nodes 65 time 1 nps 15065 pv f3e5
info depth 3 score cp -24 EBF 2 nodes 145 time 1 nps 10145 pv f3e5
info depth 4 score cp -24 EBF 1 nodes 247 time 1 nps 352247 pv f3e5
info depth 5 score cp -24 EBF 0 nodes 65 time 1 nps 65 pv f3e5
info depth 6 score cp -24 EBF 1 nodes 65 time 1 nps 65 pv f3e5
info depth 7 score cp 1 EBF 2922 nodes 189962 time 168 nps 990152 pv e3d4
info depth 8 score cp -7 EBF 4 nodes 812097 time 772 nps 1446265 pv e3d4

bestmove e3d4

*also does the node count drop "d4→d5" indicate a bug in the tt or is this
normal ?
[pgn][Event "Arena Tournament TFTEST2"]
[Site "DESKTOP-435QOMN"]
[Date "2016.10.19"]
[Round "29"]
[White "TheFallen"]
[Black "Tscp181"]
[Result "1-0"]
[BlackElo "2200"]
[ECO "A00"]
[Opening "Dunst (Sleipner-Heinrichsen-Van Geet) Opening"]
[Time "23:35:41"]
[Variation "1...d5 2.Nf3"]
[WhiteElo "2200"]
[TimeControl "60"]
[Termination "normal"]
[PlyCount "99"]
[WhiteType "human"]
[BlackType "human"]

1. Nf3 {(Ng1-f3) +0.48/9 1} d5 {(d7d5 d2d4 g8f6 b1c3 b8c6 e2e3) -0.30/6 1}
2. Nc3 {(Nb1-c3) +0.38/9 2} d4 {(d5d4 c3b5 e7e5 f3e5 c7c6 b5a3 f8a3 b2a3)
-0.06/6 1} 3. Nb5 {(Nc3-b5) +0.13/9 1} e5 {(e7e5 f3e5 c7c6 b5a3 f8a3 b2a3)
-0.06/6 1} 4. e3 {(e2-e3) -0.24/8} a6 {(a7a6 e3d4 e5e4 b5c7 d8c7 f3e5 b8c6)
+0.52/6 1} 5. exd4 {(e3xd4) -0.07/8} axb5 {(a6b5 d4e5 c8d7 f1e2 b8c6 d2d4)
+0.54/6 1} 6. dxe5 {(d4xe5) -0.19/8} Qd5 {(d8d5 d2d4 c8d7 a2a3 b8c6)
+0.73/5 1} 7. d4 {(d2-d4) +0.03/8 1} Nc6 {(b8c6 c1e3 c8g4 f1e2 a8a2 a1a2
d5a2 e2b5 a2b2 b5c6 b7c6) +0.81/5 1} 8. c3 {(c2-c3) -0.06/7} Nge7 {(g8e7
f1e2 c8d7 d1d3 e7f5) +0.86/5 1} 9. Bd3 {(Bf1-d3) +0.02/7} Bg4 {(a8a2 a1a2
d5a2 d3b5 c8f5) +0.90/5 1} 10. h3 {(h2-h3) -0.07/7} Bxf3 {(g4f3 d1f3 d5f3
g2f3 b5b4) +1.25/5 1} 11. Qxf3 {(Qd1xf3) -0.34/9} Qxf3 {(d5f3 g2f3 b5b4
c1d2 e8c8 d3c4 b4c3 d2c3) +1.19/6 1} 12. gxf3 {(g2xf3) -0.44/10} b4 {(b5b4
c1d2 b4c3 d2c3 e7d5 d3e4) +1.20/6 1} 13. Bd2 {(Bc1-d2) -0.36/9 1} bxc3
{(b4c3 d2c3 e8c8 d3c4 e7d5 c4d5 d8d5) +1.23/6 1} 14. bxc3 {(b2xc3) -0.35/9
1} O-O-O {(e8c8 d3e4 h7h6 a1b1 e7d5) +1.33/5 1} 15. Bc4 {(Bd3-c4) -0.15/8
1} Nd5 {(e7d5 h1g1 h7h5 c4d5 d8d5 a1b1) +1.24/6 1} 16. Rb1 {(Ra1-b1)
-0.06/8} h5 {(h7h5 h1g1 h8g8 c4d5 d8d5 d2f4) +1.19/6 1} 17. Ke2 {(Ke1-e2)
-0.20/8 1} Rd7 {(d8d7 b1b3 h5h4 h1b1 d5b6) +1.32/5 1} 18. Bb5 {(Bc4-b5)
+0.09/7} Nde7 {(d5e7 d2g5 h5h4 g5e7 f8e7 b5c6 b7c6) +1.06/6 1} 19. a4
{(a2-a4) +0.14/8} Rd8 {(d7d8 d2g5 c6a5 h1g1 h8h7 e2e1) +1.07/6 1} 20. Rhg1
{(Rh1-g1) +0.05/8 1} Na5 {(c6a5 d2g5 h8g8 e2e1 d8d5 b5d3) +1.02/6 1} 21.
Bd3 {(Bb5-d3) +0.25/7} Rd7 {(d8d7 b1b5 a5c6 g1b1 b7b6) +1.18/5 1} 22. Be4
{(Bd3-e4) +0.44/7} Nac6 {(a5c6 b1b5 h8g8 d2g5 c6d8) +1.09/5} 23. Ra1
{(Rb1-a1) +0.30/7} g6 {(g7g6 e5e6 f7e6 e4c6 e7c6 g1g6) +1.23/5} 24. a5
{(a4-a5) +0.09/8} Bh6 {(f8h6 d2h6 h8h6 a5a6 b7a6 a1a6) +0.77/6} 25. a6
{(a5-a6) +0.90/9} bxa6 {(b7a6 a1a6 h6d2 a6a8 c6b8 e2d2 h8d8) +0.52/6} 26.
Rxa6 {(Ra1xa6) +0.78/9} Nb8 {(c6b8 a6a8 h6d2 e2d2 c7c5 g1b1) +0.39/6} 27.
Ra8 {(Ra6-a8) +0.75/9 1} Nd5 {(e7d5 e4d5 d7d5 f3f4 h8e8 a8a7) +0.29/6} 28.
Bxd5 {(Be4xd5) +3.05/10 2} Rxd5 {(d7d5 g1b1 c8d7 d2h6 h8h6 a8b8) -2.17/6}
29. Rb1 {(Rg1-b1) +3.30/10} Kd7 {(c8d7 d2h6 h8h6 b1b8 g6g5 b8b7 h6e6)
-2.25/7} 30. Bxh6 {(Bd2xh6) +3.50/10} Rxh6 {(h8h6 b1b8 d7e6 a8a6 e6d7 b8b7
h5h4) -2.36/6} 31. Rbxb8 {(Rb1xb8) +3.86/10} Kc6 {(d7c6 e2d3 d5b5 a8a6 b5b6
b8b6 c7b6 a6a7) -3.00/6} 32. Ra7 {(Ra8-a7) +3.96/9} h4 {(h5h4 e2d3 d5b5
b8b5 c6b5 a7c7) -4.06/6} 33. Rbb7 {(Rb8-b7) +5.23/10 1} Rd7 {(d5d7 c3c4
d7d4 b7c7 c6b6 c7f7 d4c4 e5e6) -3.27/7} 34. Kd3 {(Ke2-d3) +5.14/10} f6
{(f7f6 b7b8 d7h7 a7a6 c6d7 a6f6 h7e7) -3.91/6} 35. f4 {(f3-f4) +5.38/10 1}
fxe5 {(f6e5 f4e5 h6h7 c3c4 d7f7 d3e3) -3.72/6} 36. fxe5 {(f4xe5) +5.48/10}
Rh5 {(h6h5 d3e4 h5h7 e5e6 h7e7 e4e5) -3.85/6} 37. f4 {(f2-f4) +6.11/10} Rf5
{(h5f5 c3c4 f5f4 d4d5 d7d5 c4d5 c6d5 e5e6 d5e6 b7c7) -4.39/6} 38. Ke4
{(Kd3-e4) +6.91/10} Rff7 {(f5f7 b7b4 d7e7 d4d5 c6d7 d5d6 e7e6 a7c7)
-5.60/6} 39. Rb8 {(Rb7-b8) +8.10/10 1} Rde7 {(d7e7 a7a6 c6d7 d4d5 e7e8 e5e6
d7e7 b8e8 e7e8 e6f7 e8f7) -6.42/6} 40. Ra6+ {(Ra7-a6+) +8.41/10} Kd7 {(c6d7
d4d5 e7e8 e5e6 d7e7 b8e8 e7e8 e6f7 e8f7) -6.42/5} 41. d5 {(d4-d5) +9.43/11}
Rxf4+ {(f7f4 e4f4 e7f7 a6f6 f7g7 f6e6 g7f7 f4g5 f7f5 g5h4) -8.08/6} 42.
Kxf4 {(Ke4xf4) +11.25/10} Rf7+ {(e7f7 f4e4 f7f1 a6g6 f1e1 e4f4 e1f1 f4e3
d7e7) -8.42/6} 43. Ke4 {(Kf4-e4) +12.84/10} Rf2 {(f7f2 e5e6 d7e7 b8b7 f2e2
e4d3 e2f2 b7c7) -9.37/6} 44. Rxg6 {(Ra6xg6) +12.84/9} Ke7 {(d7e7 g6g7 f2f7
d5d6 c7d6 b8b7 e7e8 b7f7 d6e5 e4e5) -12.21/6} 45. Rg7+ {(Rg6-g7+) +M6/9}
Rf7 {(f2f7 d5d6 c7d6 b8b7 e7e8 b7f7 d6e5 c3c4 e8d8 e4e5) -12.44/6} 46. d6+
{(d5-d6+) +M5/2} cxd6 {(c7d6 e5d6 e7d6 g7f7 d6c5 f7c7 c5d6 c7h7 d6c5 h7h4)
-12.69/6} 47. Rb7+ {(Rb8-b7+) +M3/2} Ke8 {(e7e8 g7f7 d6e5 e4e5 e8d8 f7f8)
-M3/6} 48. Rgxf7 {(Rg7xf7) +M2/2} dxe5 {(d6e5 e4e5 e8d8 f7f8) -M2/5} 49.
Kxe5 {(Ke4xe5) +M1/2} Kd8 {(e8d8 b7b8) -M1/3} 50. Rb8# {(Rb7-b8+) +M0/1}
1-0
[/pgn]

bob · Post by **bob** » Thu Oct 20, 2016 2:19 am

MahmoudUthman wrote:In the 5th move my engine lets the knight on the left be captured , stock fish indicates that the right move is b5a3, is this normal or does it indicate an error/bug in the evaluation , and if so what could be the most likely cause ?
this was the output of the engine :
Code: Select all
position startpos moves g1f3 d7d5 b1c3 d5d4 c3b5 e7e5 e2e3 a7a6
go wtime 53440 btime 52394 winc 0 binc 0

info depth 1 score cp 0 EBF 0 nodes 62 time 1 nps 309062 pv f3e5
info depth 2 score cp -24 EBF 1 nodes 65 time 1 nps 15065 pv f3e5
info depth 3 score cp -24 EBF 2 nodes 145 time 1 nps 10145 pv f3e5
info depth 4 score cp -24 EBF 1 nodes 247 time 1 nps 352247 pv f3e5
info depth 5 score cp -24 EBF 0 nodes 65 time 1 nps 65 pv f3e5
info depth 6 score cp -24 EBF 1 nodes 65 time 1 nps 65 pv f3e5
info depth 7 score cp 1 EBF 2922 nodes 189962 time 168 nps 990152 pv e3d4
info depth 8 score cp -7 EBF 4 nodes 812097 time 772 nps 1446265 pv e3d4

bestmove e3d4
*also does the node count drop "d4→d5" indicate a bug in the tt or is this
normal ?
[pgn][Event "Arena Tournament TFTEST2"]
[Site "DESKTOP-435QOMN"]
[Date "2016.10.19"]
[Round "29"]
[White "TheFallen"]
[Black "Tscp181"]
[Result "1-0"]
[BlackElo "2200"]
[ECO "A00"]
[Opening "Dunst (Sleipner-Heinrichsen-Van Geet) Opening"]
[Time "23:35:41"]
[Variation "1...d5 2.Nf3"]
[WhiteElo "2200"]
[TimeControl "60"]
[Termination "normal"]
[PlyCount "99"]
[WhiteType "human"]
[BlackType "human"]

1. Nf3 {(Ng1-f3) +0.48/9 1} d5 {(d7d5 d2d4 g8f6 b1c3 b8c6 e2e3) -0.30/6 1}
2. Nc3 {(Nb1-c3) +0.38/9 2} d4 {(d5d4 c3b5 e7e5 f3e5 c7c6 b5a3 f8a3 b2a3)
-0.06/6 1} 3. Nb5 {(Nc3-b5) +0.13/9 1} e5 {(e7e5 f3e5 c7c6 b5a3 f8a3 b2a3)
-0.06/6 1} 4. e3 {(e2-e3) -0.24/8} a6 {(a7a6 e3d4 e5e4 b5c7 d8c7 f3e5 b8c6)
+0.52/6 1} 5. exd4 {(e3xd4) -0.07/8} axb5 {(a6b5 d4e5 c8d7 f1e2 b8c6 d2d4)
+0.54/6 1} 6. dxe5 {(d4xe5) -0.19/8} Qd5 {(d8d5 d2d4 c8d7 a2a3 b8c6)
+0.73/5 1} 7. d4 {(d2-d4) +0.03/8 1} Nc6 {(b8c6 c1e3 c8g4 f1e2 a8a2 a1a2
d5a2 e2b5 a2b2 b5c6 b7c6) +0.81/5 1} 8. c3 {(c2-c3) -0.06/7} Nge7 {(g8e7
f1e2 c8d7 d1d3 e7f5) +0.86/5 1} 9. Bd3 {(Bf1-d3) +0.02/7} Bg4 {(a8a2 a1a2
d5a2 d3b5 c8f5) +0.90/5 1} 10. h3 {(h2-h3) -0.07/7} Bxf3 {(g4f3 d1f3 d5f3
g2f3 b5b4) +1.25/5 1} 11. Qxf3 {(Qd1xf3) -0.34/9} Qxf3 {(d5f3 g2f3 b5b4
c1d2 e8c8 d3c4 b4c3 d2c3) +1.19/6 1} 12. gxf3 {(g2xf3) -0.44/10} b4 {(b5b4
c1d2 b4c3 d2c3 e7d5 d3e4) +1.20/6 1} 13. Bd2 {(Bc1-d2) -0.36/9 1} bxc3
{(b4c3 d2c3 e8c8 d3c4 e7d5 c4d5 d8d5) +1.23/6 1} 14. bxc3 {(b2xc3) -0.35/9
1} O-O-O {(e8c8 d3e4 h7h6 a1b1 e7d5) +1.33/5 1} 15. Bc4 {(Bd3-c4) -0.15/8
1} Nd5 {(e7d5 h1g1 h7h5 c4d5 d8d5 a1b1) +1.24/6 1} 16. Rb1 {(Ra1-b1)
-0.06/8} h5 {(h7h5 h1g1 h8g8 c4d5 d8d5 d2f4) +1.19/6 1} 17. Ke2 {(Ke1-e2)
-0.20/8 1} Rd7 {(d8d7 b1b3 h5h4 h1b1 d5b6) +1.32/5 1} 18. Bb5 {(Bc4-b5)
+0.09/7} Nde7 {(d5e7 d2g5 h5h4 g5e7 f8e7 b5c6 b7c6) +1.06/6 1} 19. a4
{(a2-a4) +0.14/8} Rd8 {(d7d8 d2g5 c6a5 h1g1 h8h7 e2e1) +1.07/6 1} 20. Rhg1
{(Rh1-g1) +0.05/8 1} Na5 {(c6a5 d2g5 h8g8 e2e1 d8d5 b5d3) +1.02/6 1} 21.
Bd3 {(Bb5-d3) +0.25/7} Rd7 {(d8d7 b1b5 a5c6 g1b1 b7b6) +1.18/5 1} 22. Be4
{(Bd3-e4) +0.44/7} Nac6 {(a5c6 b1b5 h8g8 d2g5 c6d8) +1.09/5} 23. Ra1
{(Rb1-a1) +0.30/7} g6 {(g7g6 e5e6 f7e6 e4c6 e7c6 g1g6) +1.23/5} 24. a5
{(a4-a5) +0.09/8} Bh6 {(f8h6 d2h6 h8h6 a5a6 b7a6 a1a6) +0.77/6} 25. a6
{(a5-a6) +0.90/9} bxa6 {(b7a6 a1a6 h6d2 a6a8 c6b8 e2d2 h8d8) +0.52/6} 26.
Rxa6 {(Ra1xa6) +0.78/9} Nb8 {(c6b8 a6a8 h6d2 e2d2 c7c5 g1b1) +0.39/6} 27.
Ra8 {(Ra6-a8) +0.75/9 1} Nd5 {(e7d5 e4d5 d7d5 f3f4 h8e8 a8a7) +0.29/6} 28.
Bxd5 {(Be4xd5) +3.05/10 2} Rxd5 {(d7d5 g1b1 c8d7 d2h6 h8h6 a8b8) -2.17/6}
29. Rb1 {(Rg1-b1) +3.30/10} Kd7 {(c8d7 d2h6 h8h6 b1b8 g6g5 b8b7 h6e6)
-2.25/7} 30. Bxh6 {(Bd2xh6) +3.50/10} Rxh6 {(h8h6 b1b8 d7e6 a8a6 e6d7 b8b7
h5h4) -2.36/6} 31. Rbxb8 {(Rb1xb8) +3.86/10} Kc6 {(d7c6 e2d3 d5b5 a8a6 b5b6
b8b6 c7b6 a6a7) -3.00/6} 32. Ra7 {(Ra8-a7) +3.96/9} h4 {(h5h4 e2d3 d5b5
b8b5 c6b5 a7c7) -4.06/6} 33. Rbb7 {(Rb8-b7) +5.23/10 1} Rd7 {(d5d7 c3c4
d7d4 b7c7 c6b6 c7f7 d4c4 e5e6) -3.27/7} 34. Kd3 {(Ke2-d3) +5.14/10} f6
{(f7f6 b7b8 d7h7 a7a6 c6d7 a6f6 h7e7) -3.91/6} 35. f4 {(f3-f4) +5.38/10 1}
fxe5 {(f6e5 f4e5 h6h7 c3c4 d7f7 d3e3) -3.72/6} 36. fxe5 {(f4xe5) +5.48/10}
Rh5 {(h6h5 d3e4 h5h7 e5e6 h7e7 e4e5) -3.85/6} 37. f4 {(f2-f4) +6.11/10} Rf5
{(h5f5 c3c4 f5f4 d4d5 d7d5 c4d5 c6d5 e5e6 d5e6 b7c7) -4.39/6} 38. Ke4
{(Kd3-e4) +6.91/10} Rff7 {(f5f7 b7b4 d7e7 d4d5 c6d7 d5d6 e7e6 a7c7)
-5.60/6} 39. Rb8 {(Rb7-b8) +8.10/10 1} Rde7 {(d7e7 a7a6 c6d7 d4d5 e7e8 e5e6
d7e7 b8e8 e7e8 e6f7 e8f7) -6.42/6} 40. Ra6+ {(Ra7-a6+) +8.41/10} Kd7 {(c6d7
d4d5 e7e8 e5e6 d7e7 b8e8 e7e8 e6f7 e8f7) -6.42/5} 41. d5 {(d4-d5) +9.43/11}
Rxf4+ {(f7f4 e4f4 e7f7 a6f6 f7g7 f6e6 g7f7 f4g5 f7f5 g5h4) -8.08/6} 42.
Kxf4 {(Ke4xf4) +11.25/10} Rf7+ {(e7f7 f4e4 f7f1 a6g6 f1e1 e4f4 e1f1 f4e3
d7e7) -8.42/6} 43. Ke4 {(Kf4-e4) +12.84/10} Rf2 {(f7f2 e5e6 d7e7 b8b7 f2e2
e4d3 e2f2 b7c7) -9.37/6} 44. Rxg6 {(Ra6xg6) +12.84/9} Ke7 {(d7e7 g6g7 f2f7
d5d6 c7d6 b8b7 e7e8 b7f7 d6e5 e4e5) -12.21/6} 45. Rg7+ {(Rg6-g7+) +M6/9}
Rf7 {(f2f7 d5d6 c7d6 b8b7 e7e8 b7f7 d6e5 c3c4 e8d8 e4e5) -12.44/6} 46. d6+
{(d5-d6+) +M5/2} cxd6 {(c7d6 e5d6 e7d6 g7f7 d6c5 f7c7 c5d6 c7h7 d6c5 h7h4)
-12.69/6} 47. Rb7+ {(Rb8-b7+) +M3/2} Ke8 {(e7e8 g7f7 d6e5 e4e5 e8d8 f7f8)
-M3/6} 48. Rgxf7 {(Rg7xf7) +M2/2} dxe5 {(d6e5 e4e5 e8d8 f7f8) -M2/5} 49.
Kxe5 {(Ke4xe5) +M1/2} Kd8 {(e8d8 b7b8) -M1/3} 50. Rb8# {(Rb7-b8+) +M0/1}
1-0
[/pgn]

Certainly does not look normal. What are your material piece values? This is tossing a knight for 2 pawns. Looks like an evaluation problem but there is no way to be sure without more analysis and data.

MahmoudUthman · Post by **MahmoudUthman** » Thu Oct 20, 2016 2:29 am

bob wrote:Certainly does not look normal. What are your material piece values? This is tossing a knight for 2 pawns. Looks like an evaluation problem but there is no way to be sure without more analysis and data.

material piece values

Code: Select all

100,300//knight,300//bishop,500,950

PST:Pawel Koziol's PST

cdani · Post by **cdani** » Thu Oct 20, 2016 2:10 pm

For these things I added to Andscacs (in debug mode) a detailed "eval" command to show the static eval like Stockfish has. This helps to see which parameters are the most relevant to the position, so to help to tune them.

So in this case I will try to see if in the relevant positions something seems to be out of place.

Sven · Post by **Sven** » Thu Oct 20, 2016 2:13 pm

MahmoudUthman wrote:In the 5th move my engine lets the knight on the left be captured , stock fish indicates that the right move is b5a3, is this normal or does it indicate an error/bug in the evaluation , and if so what could be the most likely cause ?
this was the output of the engine :
Code: Select all
position startpos moves g1f3 d7d5 b1c3 d5d4 c3b5 e7e5 e2e3 a7a6
go wtime 53440 btime 52394 winc 0 binc 0

info depth 1 score cp 0 EBF 0 nodes 62 time 1 nps 309062 pv f3e5
info depth 2 score cp -24 EBF 1 nodes 65 time 1 nps 15065 pv f3e5
info depth 3 score cp -24 EBF 2 nodes 145 time 1 nps 10145 pv f3e5
info depth 4 score cp -24 EBF 1 nodes 247 time 1 nps 352247 pv f3e5
info depth 5 score cp -24 EBF 0 nodes 65 time 1 nps 65 pv f3e5
info depth 6 score cp -24 EBF 1 nodes 65 time 1 nps 65 pv f3e5
info depth 7 score cp 1 EBF 2922 nodes 189962 time 168 nps 990152 pv e3d4
info depth 8 score cp -7 EBF 4 nodes 812097 time 772 nps 1446265 pv e3d4

bestmove e3d4

It is definitely a bug, an engine should not give a knight for two pawns in that position. But there is not enough information present to state whether it is caused by evaluation, search, or something else. Your engine only shows the first ply of its PV so we don't know which sequence of moves leads to the given score. If the PV starts with exd4 axb5 dxe5 Qd5, as in the game, then I would assume that you are using positional scores that may be too high, e.g. for central pawns or something like that. But it could also be a search problem, a TT bug, whatever.

First thing should be to try to reproduce the problem. Then you might proceed by disabling parts of the program and see whether the problem persists, e.g. disable TT, disable search features like LMR/null move/..., disable positional evaluation.

MahmoudUthman · Post by **MahmoudUthman** » Thu Oct 20, 2016 11:05 pm

cdani wrote:For these things I added to Andscacs (in debug mode) a detailed "eval" command to show the static eval like Stockfish has. This helps to see which parameters are the most relevant to the position, so to help to tune them.

So in this case I will try to see if in the relevant positions something seems to be out of place.

I'll be adding one after I finish refactoring the code of the engine in the mean while I've found two things that when disabled result in the correct result for the position (Knight and Pawn PST).

Sven Schüle wrote:It is definitely a bug, an engine should not give a knight for two pawns in that position. But there is not enough information present to state whether it is caused by evaluation, search, or something else. Your engine only shows the first ply of its PV so we don't know which sequence of moves leads to the given score. If the PV starts with exd4 axb5 dxe5 Qd5, as in the game, then I would assume that you are using positional scores that may be too high, e.g. for central pawns or something like that. But it could also be a search problem, a TT bug, whatever.

First thing should be to try to reproduce the problem. Then you might proceed by disabling parts of the program and see whether the problem persists, e.g. disable TT, disable search features like LMR/null move/..., disable positional evaluation.

PVs [before modifications] :

Code: Select all

info depth 1 score cp 0 EBF 62 nodes 62 time 1 nps 293062 pv f3e5
info depth 2 score cp 0 EBF 2 nodes 183 time 1 nps 262183 pv f3e5 a6b5
info depth 3 score cp 0 EBF 5 nodes 1002 time 4 nps 294502 pv f3e5 a6b5 f1b5
info depth 4 score cp 10 EBF 5 nodes 5394 time 13 nps 739163 pv f3e5 a6b5 f1b5 c7c6 b5d3
info depth 5 score cp 10 EBF 2 nodes 14304 time 27 nps 347896 pv f3e5 a6b5 f1b5 c7c6 b5d3
info depth 6 score cp -24 EBF 6 nodes 89018 time 132 nps 796321 pv f3e5 a6b5 f1b5 c7c6 b5c4 c8e6 e1g1
info depth 7 score cp 1 EBF 2 nodes 223399 time 354 nps 670076 pv e3d4 a6b5 d4e5 d8d5 d2d4 d5e4 c1e3 c8d7
info depth 8 score cp -8 EBF 2 nodes 571004 time 851 nps 1035501 pv e3d4 a6b5 d4e5 c8e6 f1b5 c7c6 b5d3 e6a2
info depth 9 score cp -2 EBF 2 nodes 1155570 time 1690 nps 1530227 pv e3d4 a6b5 d4e5 c8e6 d2d4 d8d7 a2a3 f8b4 c1d2 b4d2
info depth 10 score cp -18 EBF 6 nodes 6935039 time 7196 nps 7560091 pv f3e5 a6b5 f1b5 c7c6 b5c4 g8h6 d1h5 g7g6 e5g6 a8a5 h5a5
info depth 11 score cp 3 EBF 2 nodes 14034951 time 17056 nps 14393200 pv e3d4 a6b5 d4e5 c7c6 d2d4 c8e6 f1d3 e6a2 c1d2 b8d7 e1g1

trying the search without null move pruning and TT changed nothing , but in the end disabling the PST for Knights and pawns resulted in the correct result :

*Mobility disabled:
PVs (Pawn PST disabled):

Code: Select all

info depth 1 score cp 43 EBF 62 nodes 62 time 1 nps 300062 pv b5a3
info depth 2 score cp 20 EBF 2 nodes 185 time 2 nps 110685 pv b5a3 b8c6
info depth 3 score cp 43 EBF 5 nodes 1036 time 4 nps 126286 pv b5a3 f8a3 b2a3
info depth 4 score cp 29 EBF 6 nodes 7163 time 14 nps 847448 pv f3e5 a6b5 f1b5 c7c6 b5d3
info depth 5 score cp 29 EBF 4 nodes 29555 time 50 nps 646795 pv b5a3 f8a3 b2a3 b8c6 f1d3
info depth 6 score cp 3 EBF 1 nodes 57761 time 132 nps 674397 pv b5a3 f8a3 b2a3 b8c6 e3d4 e5d4
info depth 7 score cp 29 EBF 3 nodes 181575 time 384 nps 880973 pv b5a3 f8a3 b2a3 b8c6 e3d4 e5d4 f1d3
info depth 8 score cp -1 EBF 3 nodes 701195 time 964 nps 1212352 pv b5a3 f8a3 b2a3 b8c6 e3d4 e5d4 d1e2 g8e7 c1b2
info depth 9 score cp 22 EBF 1 nodes 862622 time 1727 nps 1234141 pv b5a3 f8a3 b2a3 b8c6 e3d4 e5d4 f1e2 g8f6 e1g1
info depth 10 score cp -1 EBF 7 nodes 6884698 time 6910 nps 7468834 pv b5a3 f8a3 b2a3 b8c6 e3d4 e5d4 f1d3 g8e7 e1g1 e8g8
info depth 11 score cp 7 EBF 0 nodes 6113930 time 11460 nps 6407948 pv b5a3 f8a3 b2a3 b8c6 c1b2 g8f6 e3d4 e5d4
info depth 12 score cp -16 EBF 29 nodes 180715390 time 146459 nps 181389354 pv b5a3 f8a3 b2a3 b8c6 c1b2 c8e6

*Mobility disabled
PVs(Knight PST disabled):

Code: Select all

info depth 1 score cp -53 EBF 62 nodes 62 time 1 nps 272062 pv b5d4
info depth 2 score cp -53 EBF 2 nodes 185 time 1 nps 194185 pv b5d4 e5d4
info depth 3 score cp -53 EBF 5 nodes 1002 time 3 nps 94668 pv b5d4 e5d4 e3d4
info depth 4 score cp -64 EBF 8 nodes 8968 time 15 nps 780234 pv f3e5 d4e3 d1f3 c8e6
info depth 5 score cp -53 EBF 2 nodes 22568 time 41 nps 530836 pv b5a3 f8a3 b2a3 c8g4 f1d3
info depth 6 score cp -65 EBF 3 nodes 86774 time 146 nps 621315 pv b5a3 f8a3 b2a3 d4e3 f2e3 c8g4
info depth 7 score cp -52 EBF 3 nodes 262945 time 410 nps 788066 pv e3d4 a6b5 d1e2 c7c6 e2e5 c8e6 f1d3
info depth 8 score cp -61 EBF 3 nodes 897148 time 1102 nps 1364419 pv b5a3 f8a3 b2a3 d4e3 f2e3 e5e4 f3d4 c8d7
info depth 9 score cp -63 EBF 1 nodes 1734362 time 2518 nps 2138030 pv b5d4 e5d4 f3d4 c7c5 d4f3 c8e6 f1e2 f8d6 e1g1
info depth 10 score cp -64 EBF 7 nodes 12603541 time 11431 nps 13139395 pv b5a3 f8a3 b2a3 d4e3 f2e3 e5e4 f3d4 d8h4 g2g3 h4e7
info depth 11 score cp -53 EBF 0 nodes 10272047 time 19248 nps 10603305 pv b5a3 f8a3 b2a3 d4e3 d2e3 d8d1 e1d1 e5e4 f3g5 e1g1

*Mobility disabled
PVs (Both disabled):

Code: Select all

info depth 1 score cp -35 EBF 62 nodes 62 time 1 nps 273062 pv b5a3
info depth 2 score cp -35 EBF 2 nodes 185 time 1 nps 155185 pv b5a3 f8a3
info depth 3 score cp -35 EBF 5 nodes 983 time 3 nps 137983 pv b5a3 f8a3 b2a3
info depth 4 score cp -60 EBF 5 nodes 5716 time 11 nps 540352 pv b5a3 f8a3 b2a3 c8f5
info depth 5 score cp -35 EBF 2 nodes 14268 time 26 nps 404037 pv b5a3 f8a3 b2a3 d4e3 f2e3
info depth 6 score cp -43 EBF 5 nodes 80960 time 111 nps 704743 pv b5a3 f8a3 b2a3 c8g4 f1d3 e8f8
info depth 7 score cp -35 EBF 1 nodes 100260 time 242 nps 560764 pv b5a3 f8a3 b2a3 d4e3 f2e3 e5e4 f3g1
info depth 8 score cp -56 EBF 5 nodes 588872 time 663 nps 1022689 pv b5a3 d4e3 f2e3 f8a3 b2a3 e5e4 f3d4 c8d7
info depth 9 score cp -40 EBF 1 nodes 657578 time 1206 nps 1014262 pv b5a3 d4e3 f2e3 f8a3 b2a3 e5e4 f3d4 d8h4 g2g3 h4g5
info depth 10 score cp -42 EBF 7 nodes 5237633 time 4825 nps 5747305 pv b5a3 d4e3 d2e3 d8d1 e1d1 f8a3 b2a3 e5e4 f3e5 c8e6 f1c4
info depth 11 score cp -40 EBF 1 nodes 6809595 time 9961 nps 7211381 pv b5a3 d4e3 f2e3 f8a3 b2a3 e3c3
info depth 12 score cp -42 EBF 10 nodes 72746781 time 59829 nps 73311571 pv b5a3 f8a3 b2a3 d4e3 d2e3 d8d1 e1d1

*Mobility Enabled:
PVs (Both disabled):

Code: Select all

info depth 1 score cp -62 EBF 62 nodes 62 time 1 nps 313062 pv b5a3
info depth 2 score cp -101 EBF 2 nodes 185 time 1 nps 306185 pv b5a3 c8f5
info depth 3 score cp -58 EBF 5 nodes 1063 time 3 nps 240396 pv b5a3 f8a3 b2a3
info depth 4 score cp -97 EBF 6 nodes 7265 time 15 nps 720598 pv f3e5 a6b5 f1b5 c7c6 b5d3
info depth 5 score cp -51 EBF 2 nodes 21545 time 40 nps 453695 pv b5a3 d4e3 d2e3 d8d1 e1d1 c8g4
info depth 6 score cp -82 EBF 4 nodes 91781 time 140 nps 624259 pv b5a3 b8c6 f1d3 c8e6 c2c3 f8c5
info depth 7 score cp -64 EBF 1 nodes 169177 time 355 nps 750734 pv b5a3 b8c6 f1c4 c8f5 d2d3 f8b4 c1d2 d8d6
info depth 8 score cp -72 EBF 4 nodes 819628 time 957 nps 1264145 pv b5a3 b8c6 f1d3 g8f6 e3d4 e5e4 d1e2 f8a3
info depth 9 score cp -71 EBF 2 nodes 1800474 time 2537 nps 2327474 pv b5a3 d4e3 d2e3 d8d1 e1d1 f8a3 b2a3 e5e4 f3g5 g8f6
info depth 10 score cp -57 EBF 9 nodes 17995406 time 15559 nps 18558372 pv f3e5 d4e3 d1f3 e3f2 e1f2 c8e6 f3b7 a6b5 f1b5

I checked the PST several times but I couldn't pinpoint cause , it's values look normal to me ! , any suggestions ?

Sven · Post by **Sven** » Fri Oct 21, 2016 1:09 am

MahmoudUthman wrote:trying the search without null move pruning and TT changed nothing , but in the end disabling the PST for Knights and pawns resulted in the correct result
[...]
I checked the PST several times but I couldn't pinpoint cause , it's values look normal to me ! , any suggestions ?

The PST itself may be correct but there may be sign errors when using it, or something like that? Mirroring?

You should implement the "eval" command as suggested. For this case you should make its output quite verbose, e.g. show the score for each single piece and each eval feature. This might require some rework of the evaluation function since you might want to store all those values that you want to be printed later in some variables so that "eval" calls the normal evaluation function and then prints the contents of some variables. Then play the original "wrong" PV to the end and save the output of "eval". Do this
1) for the version with the original evaluation,
2) for the version where you disabled some parts of the PST (use the same PV as in 1) of course).
Then compare the output. That should give you an idea which pieces get a dubious score.

kbhearn · Post by **kbhearn** » Fri Oct 21, 2016 1:39 am

You mentioned your material values used were 100,300,300,500,950 - combined with the piece-square table this brings the position after the N sac to close to even at least (2 pawns, 2 central pawns at +0.15 to +0.25, 1 N on a3 penalised at -0.20) - perhaps a few other terms might interact further and make it think giving up the N is a good trade at low depth. Usually the base piece values in an engine for a minor is a bit higher than 300 bringing minor+pawn a little closer to a rook and making compensation required already for piece for 3 pawns.

See
https://chessprogramming.wikispaces.com/Point+Value

and specifically the entries toward the bottom of the table there

edit: and actually pawel's linked PSTs specify N=325, B=335 if you were using that as a starting point

Sven · Post by **Sven** » Fri Oct 21, 2016 2:06 am

kbhearn wrote:You mentioned your material values used were 100,300,300,500,950 - combined with the piece-square table this brings the position after the N sac to close to even at least (2 pawns, 2 central pawns at +0.15 to +0.25, 1 N on a3 penalised at -0.20) - perhaps a few other terms might interact further and make it think giving up the N is a good trade at low depth.

For problems like this one you have to consider where the score comes from. It comes from the end position of the given PV (if PVs are collected during qsearch as well) or from a position reached from there by playing the optimal capture sequence (otherwise). Here I guess PVs are not collected within qsearch (since I don't see any capture sequence at the end of PVs). The end position of the final PV given above under "before modifications"

Code: Select all

info depth 11 score cp 3 EBF 2 nodes 14034951 time 17056 nps 14393200 pv e3d4 a6b5 d4e5 c7c6 d2d4 c8e6 f1d3 e6a2 c1d2 b8d7 e1g1

is this:
[d]r2qkbnr/1p1n1ppp/2p5/1p2P3/3P4/3B1N2/bPPB1PPP/R2Q1RK1 b kq - 0 10[/d]
Here black is a piece up vs. one pawn. That should be -200. Black to move has no obvious winning capture sequence (otherwise it would be even worse for White). White has some compensation through development and two central pawns but that can't compensate for the value of two pawns. So there must be some bug - maybe also in qsearch? -, not just some bad weights.

hgm · Post by **hgm** » Fri Oct 21, 2016 11:23 am

MahmoudUthman wrote:I'll be adding one after I finish refactoring the code of the engine in the mean while I've found two things that when disabled result in the correct result for the position (Knight and Pawn PST).

Note that it is usually very easy to make rare blunders go away by small changes to completely unrelated features. Like changing the evaluation a bit, or the move ordering.

So making the error go away is not a big feat, and is not even an indication that the error was related in anyway to the thing you added or disabled. You were lucky that the old search tree probed the error, and you cannot expect to be that lucky again after changing anything.

The proper way to solve problems like this is tracing the tree. I usually do that by selective printing of a list of moves and their scores in nodes of interest. By having a debug infra-structure like

Code: Select all

MOVE path&#91;MAXPLY&#93;;
int ply;

Search&#40;depth&#41;
&#123;
    ...
    path&#91;ply++&#93; = move;
    MakeMove&#40;move&#41;;
    score= -Search&#40;depth-1&#41;;
    UnMake&#40;);
    ply--;
    if&#40;NodeOnPath&#40;)) printf&#40;"%d&#58;%d %s %6d %6d\n", ply, depth, MoveToText&#40;move&#41;, score, bestScore&#41;;
    ...
&#125;

That allows you to zoom in on the node that causes the error pretty fast, even in a deep search (especially if you allow the sought path to be entered as arguments to the startup command). You just keep adding the move with the suspect score to the path, until you reach the node where the score is generated. Then you can add other print statements, under the same "if(NodeOnPath())" clause, to figure out how this score got to be so wrong.

mistake/bug in evaluation or normal behaviore ?

mistake/bug in evaluation or normal behaviore ?

Re: mistake/bug in evaluation or normal behaviore ?

Re: mistake/bug in evaluation or normal behaviore ?

Re: mistake/bug in evaluation or normal behaviore ?

Re: mistake/bug in evaluation or normal behaviore ?

Re: mistake/bug in evaluation or normal behaviore ?

Re: mistake/bug in evaluation or normal behaviore ?

Re: mistake/bug in evaluation or normal behaviore ?

Re: mistake/bug in evaluation or normal behaviore ?

Re: mistake/bug in evaluation or normal behaviore ?