Page 1 of 3

Stockfish zero evals

Posted: Fri May 02, 2014 8:59 pm
by lkaufman
It seems to me that I see an unusually high number of zero evaluations when analyzing with Stockfish. Does anyone know what causes this? Zero scores are often due to repetitions, but why would Stockfish see more repetitions than other engines? Or maybe it's something else.

Re: Stockfish zero evals

Posted: Fri May 02, 2014 9:06 pm
by Uri Blass
lkaufman wrote:It seems to me that I see an unusually high number of zero evaluations when analyzing with Stockfish. Does anyone know what causes this? Zero scores are often due to repetitions, but why would Stockfish see more repetitions than other engines? Or maybe it's something else.
searching deeper can explain seeing more 0's because if you search deep enough you can see the draw when the opponent does not see the draw because it does not see deep enough.

Maybe it is not the only explanation but my guess is that komodo depth n can also see more 0's than komodo depth n-1.

Re: Stockfish zero evals

Posted: Fri May 02, 2014 9:53 pm
by syzygy
My guess would be the same.

Re: Stockfish zero evals

Posted: Fri May 02, 2014 11:00 pm
by Joerg Oster
Yes, you are absolutely right. I have observed this many times as well.
But I have no explanation why this happens ...

Re: Stockfish zero evals

Posted: Fri May 02, 2014 11:14 pm
by Joerg Oster
I have seen this also at more shallow depths, so I don't think that's a good explanation.

And I never heard that by searching one or more plies deeper, maybe after implementing Null Move or LMR, an engine began to show more draw scores than before. Did you?

Re: Stockfish zero evals

Posted: Sat May 03, 2014 3:23 am
by Isaac
Joerg Oster wrote:I have seen this also at more shallow depths, so I don't think that's a good explanation.
I can confirm this. Someone on the TCEC analyzed about 30k games from the CCRL from engines rated over 3000 and having no more than 50 elo difference. He let Stockfish analyzing up to a fixed -shallow- depth (depth 14 if I remember well but I may be wrong) for a particular move number on every single game.
He then plotted the evaluation on the x-axis and the frequency on the y-axis.
As expected, "0.00" was the most common eval. If I remember well, it was around 11% of all eval values returned. The rest of the shape looked like a wide Gaussian distribution, not centered in 0 (drifted toward positive values).

Re: Stockfish zero evals

Posted: Sat May 03, 2014 5:29 am
by Lyudmil Tsvetkov
Joerg Oster wrote:I have seen this also at more shallow depths, so I don't think that's a good explanation.

And I never heard that by searching one or more plies deeper, maybe after implementing Null Move or LMR, an engine began to show more draw scores than before. Did you?
Maybe take a look at the below game, at the eval after move 52 for black.

[pgn][PlyCount "225"]
[MLNrOfMoves "112"]
[MLFlags "100100"]
[Date "2014.04.13"]
[Round "1"]
[White "Senpai 1.0"]
[Black "Stockfish 14041208"]
[Result "1-0"]
[EventDate "2014.??.??"]
[ECO "A49"]
[TimeControl "240+2"]

1. d4 {book} 1... Nf6 {book} 2. Nf3 {book} 2... g6 {book} 3. g3 {book} 3... Bg7
{book} 4. Bg2 {book} 4... c5 {book} 5. d5 {book} 5... O-O {book} 6. c4 {book}
6... d6 {book} 7. O-O {book} 7... b5 {book} 8. cxb5 {book} 8... a6 {book} 9.
bxa6 {+0.65/18 7.2s} 9... Bxa6 {-0.41/21 7.8s} 10. Nc3 {+0.72/20 8.0s} 10...
Nbd7 {-0.40/22 5.3s} 11. Qc2 {+0.72/20 28s} 11... Ng4 {-0.42/ 21 7.1s} 12. Rd1
{+0.90/18 6.1s} 12... Qb6 {-0.38/21 6.5s} 13. Bg5 {+0.93/17 7.0s} 13... Rfb8
{-0.37/22 10s} 14. Rab1 {+0.99/19 6.2s} 14... Qd8 {-0.38/23 5.4s} 15. h3
{+0.87/ 18 6.0s} 15... Nge5 {-0.37/24 5.4s} 16. Nxe5 {+0.87/20 6.5s} 16... Nxe5
{-0.37/24 5.5s} 17. b3 {+0.87/20 9.8s} 17... Bc8 {-0.37/24 5.6s} 18. Ne4
{+0.93/18 5.2s} 18... f5 {-0.36/22 14s} 19. Nc3 {+0.92/19 4.8s} 19... Bd7
{-0.35/23 5.6s} 20. Rdc1 {+0.91/19 4.9s} 20... Ra3 {-0. 30/21 11s} 21. Rb2
{+0.89/18 5.0s} 21... Nf7 {-0.22/22 10s} 22. Bd2 {+0.81/18 9.3s} 22... Rb7
{-0.22/21 5.6s} 23. Kf1 {+0.78/17 6.0s} 23... Qa8 {-0.23/22 16s} 24. Ra1
{+0.77/ 18 9.8s} 24... Qa6 {-0.23/21 5.5s} 25. Kg1 {+0.77/19 7.5s} 25... Bd4
{-0.23/21 11s} 26. e3 {+0.90/19 5.3s} 26... Bg7 {-0.24/21 8.9s} 27. f4
{+0.93/18 4.7s} 27... Rb8 {-0.31/18 8.7s} 28. Bf1 {+0.90/19 5.4s} 28... Qb7
{-0.37/21 14s} 29. Bc4 {+0.90/19 5.1s} 29... Re8 {-0.52/ 23 18s} 30. Re1
{+0.94/19 5.0s} 30... e5 {-0.31/21 16s} 31. Rbb1 {+1.13/16 4.3s} 31... Qa8
{-0.56/22 26s} 32. Rbd1 {+1.12/18 4.9s} 32... Qa5 {-0.53/21 11s} 33. Bd3
{+1.16/18 7. 9s} 33... Qb6 {-0.51/20 5.6s} 34. Kg2 {+1.13/18 3.8s} 34... e4
{-0.59/20 8.7s} 35. Bc4 {+1. 32/20 5.5s} 35... Qa5 {-0.68/24 9.8s} 36. Re2
{+1.43/21 3.9s} 36... Qb6 {-0.81/26 11s} 37. Bc1 {+1.44/22 4.0s} 37... Raa8
{-0.81/26 3.5s} 38. a4 {+1.46/21 4.7s} 38... Re7 {-0.85/24 7.0s} 39. Bb2
{+1.45/21 4.3s} 39... Nd8 {-0.88/24 5.7s} 40. Nb5 {+1.41/21 6.3s} 40... Nb7
{-0.87/25 3.4s} 41. Bxg7 {+1.42/21 4.3s} 41... Rxg7 {-0.87/22 5.7s} 42. Rf2
{+1.41/22 5.1s} 42... g5 {-0.74/21 8.8s} 43. fxg5 {+1.68/19 4.2s} 43... Rxg5
{-1.07/21 4.3s} 44. Qc3 {+1.75/19 5.1s} 44... Rg7 {-0.82/21 9.4s} 45. Kh2
{+1.72/19 3.9s} 45... Nd8 {-0.75/22 7.8s} 46. Be2 {+1.44/20 4.5s} 46... Nf7
{-0.86/23 4.2s} 47. Na3 {+1.36/22 4.9s} 47... Ne5 {-0.86/ 23 2.3s} 48. Nc4
{+1.36/21 1.8s} 48... Nxc4 {-0.96/24 11s} 49. Bxc4 {+1.38/20 5.2s} 49... Qd8
{-0.90/23 4.6s} 50. Rdf1 {+1.38/20 4.2s} 50... h6 {-0.89/21 5.1s} 51. Rg2
{+1.41/ 19 3.5s} 51... Kh7 {-0.86/21 2.2s} 52. Rgf2 {+1.31/20 27s} 52... Kg8
{0.00/35 2.3s} 53. Rf4 {+1.38/18 3.2s} 53... Kh7 {-0.79/21 2.4s} 54. Qe1
{+1.32/17 2.9s} 54... Qg5 {-0.79/20 3.2s} 55. Rg1 {+1.41/20 3.4s} 55... Qf6
{-0.82/21 1.5s} 56. Qa1 {+1.35/19 16s} 56... Qd8 {-0.81/ 22 1.5s} 57. Qb1
{+1.31/20 3.2s} 57... Qg5 {-0.80/20 3.5s} 58. Rg2 {+1.38/19 3.2s} 58... Qe7
{-0.81/21 3.1s} 59. Qb2 {+1.31/19 2.7s} 59... Qf7 {-0.85/21 1.3s} 60. Qa1
{+1.37/ 18 7.0s} 60... Qg6 {-0.85/22 2.3s} 61. Qf1 {+1.41/16 2.7s} 61... Qf7
{-0.82/23 3.6s} 62. Qf2 {+1.31/18 6.2s} 62... Qe7 {-0.82/22 1.3s} 63. Bb5
{+1.41/20 2.6s} 63... Bc8 {-0.95/23 3.5s} 64. Qb2 {+1.41/21 4.5s} 64... Rb8
{-1.04/20 2.4s} 65. Qc3 {+1.41/21 4.1s} 65... Bd7 {-0.98/19 2.4s} 66. Bxd7
{+1.38/20 3.7s} 66... Qxd7 {-1.18/21 2.4s} 67. Qc4 {+1.42/20 3.3s} 67... Rb4
{-1.11/19 2.4s} 68. Qc2 {+1.25/20 2.7s} 68... Qf7 {-1.25/20 2.4s} 69. Qd1
{+1.46/20 3.6s} 69... Qf6 {-1.17/19 2.1s} 70. a5 {+1.71/17 5.0s} 70... Qc3
{-1.44/20 2.7s} 71. Qh5 {+1.66/19 2.6s} 71... Qf6 {-1.44/22 2.4s} 72. Rxf5
{+2.09/20 4.4s} 72... Qg6 {-1. 37/22 1.8s} 73. Qxg6+ {+1.81/20 1.2s} 73... Kxg6
{-1.59/24 3.0s} 74. Rgf2 {+1.80/23 6.9s} 74... Rxb3 {-1.70/19 2.0s} 75. Rf6+
{+1.90/17 2.9s} 75... Kh7 {-1.46/20 1.5s} 76. Rxd6 {+1.73/19 3.9s} 76... Rxe3
{-1.84/20 3.7s} 77. g4 {+1.80/20 2.6s} 77... Rb7 {-1.84/20 2.4s} 78. Rb6
{+2.18/19 3.0s} 78... Rd7 {-1.84/20 1.3s} 79. d6 {+2.24/19 3.0s} 79... Ra3
{-2.36/21 3.5s} 80. a6 {+2.79/16 3.2s} 80... Kg6 {-2.22/21 1.7s} 81. Rc2
{+2.78/17 2. 7s} 81... h5 {-2.29/20 3.0s} 82. gxh5+ {+3.48/16 2.7s} 82... Kxh5
{-2.55/20 2.4s} 83. Rxc5+ {+3.36/18 1.5s} 83... Kh4 {-2.91/20 2.0s} 84. Rc7
{+3.64/18 4.1s} 84... Ra2+ {-4.26/20 2.8s} 85. Kg1 {+4.05/20 4.9s} 85... Rd8
{-4.48/22 2.3s} 86. a7 {+4.17/17 4.7s} 86... Kxh3 {-4. 61/22 2.5s} 87. Rb8
{+4.83/20 3.5s} 87... Rxd6 {-5.14/25 2.4s} 88. Rh7+ {+5.44/20 2. 4s} 88... Kg4
{-5.60/22 2.1s} 89. Rg7+ {+5.59/21 4.2s} 89... Kf5 {-5.68/23 2.6s} 90. Rf8+
{+5.70/19 3.9s} 90... Ke6 {-5.72/26 2.5s} 91. Rg6+ {+5.90/20 4.2s} 91... Ke5
{-5.72/27 1.3s} 92. Re8+ {+5.96/20 2.5s} 92... Kf4 {-5.84/27 3.2s} 93. Rxd6
{+5.98/21 3.8s} 93... Rxa7 {-5. 84/28 1.4s} 94. Rd4 {+5.99/18 2.4s} 94... Ra1+
{-5.90/28 3.7s} 95. Kf2 {+6.01/22 4.1s} 95... Ra2+ {-5.95/28 1.3s} 96. Ke1
{+6.09/17 2.1s} 96... Ra1+ {-5.95/27 2.4s} 97. Kd2 {+6.11/22 4.0s} 97... Ra2+
{-16.57/26 3.5s} 98. Kc3 {+6.13/22 3.9s} 98... Re2 {-9.40/26 2.4s} 99. Kc4
{+6.21/23 3.8s} 99... Kf5 {-22.41/22 1.9s} 100. Kd5 {+6.36/17 2.6s} 100... e3
{-27.41/27 2.8s} 101. Rf8+ {+10.07/21 3.7s} 101... Kg6 {-101.33/32 2.4s} 102.
Rg4+ {+M23/17 5.5s} 102... Kh7 {-M38/23 1.9s} 103. Rf3 {+M21/17 2.8s} 103... Rh2
{-M28/24 2.2s} 104. Rxe3 {+M19/15 0.97s} 104... Rh6 {-M18/25 2.3s} 105. Re7+
{+M17/15 0.92s} 105... Kh8 {-M16/1 0s} 106. Rc4 {+M15/17 2.7s} 106... Rg6
{-M14/30 1.7s} 107. Rc8+ {+M13/20 2.5s} 107... Rg8 {-M12/1 0s} 108. Rxg8+
{+M11/36 0.89s} 108... Kxg8 {-M10/1 0s} 109. Ke6 {+M9/99 0. 25s} 109... Kf8
{-M8/120 0.063s} 110. Kf6 {+M7/99 0s} 110... Kg8 {-M6/1 0s} 111. Re8+
{+M5/ 99 0s} 111... Kh7 {-M4/1 0s} 112. Rd8 {+M3/99 0s} 112... Kh6 {-M2/1 0s}
113. Rh8# {+M1/99 0s, White mates} 1-0
[/pgn]

SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.

Re: Stockfish zero evals

Posted: Sat May 03, 2014 8:06 am
by Michel
SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.
This is a well known issue. A good fix is known, but unfortunately testing has shown (in a statistical sense) that even this fix yields a tiny elo loss. The elo loss hasn't been measured very precisely but it should be around 1 elo.

Re: Stockfish zero evals

Posted: Sat May 03, 2014 8:51 am
by Ferdy
Lyudmil wrote:SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.
There is nothing wrong with returning 0.0 here as long as the engine has tried very hard exploring other moves and proves that this is the only best reply, rather than the engine returns 0.0 just because there is repetition. One way to see how the engine reacts is if we let it search the position except the move that repeats, will it find a better alternative move? During a game it is different since there is time constraint.

Re: Stockfish zero evals

Posted: Sat May 03, 2014 9:43 am
by carldaman
Michel wrote:
SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.
This is a well known issue. A good fix is known, but unfortunately testing has shown (in a statistical sense) that even this fix yields a tiny elo loss. The elo loss hasn't been measured very precisely but it should be around 1 elo.

Giving up 1 Elo point to fix this annoying issue would be Elo well spent!

Regards,
CL