Stockfish zero evals
Moderator: Ras
-
- Posts: 6214
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
- Full name: Larry Kaufman
Stockfish zero evals
It seems to me that I see an unusually high number of zero evaluations when analyzing with Stockfish. Does anyone know what causes this? Zero scores are often due to repetitions, but why would Stockfish see more repetitions than other engines? Or maybe it's something else.
-
- Posts: 10770
- Joined: Thu Mar 09, 2006 12:37 am
- Location: Tel-Aviv Israel
Re: Stockfish zero evals
searching deeper can explain seeing more 0's because if you search deep enough you can see the draw when the opponent does not see the draw because it does not see deep enough.lkaufman wrote:It seems to me that I see an unusually high number of zero evaluations when analyzing with Stockfish. Does anyone know what causes this? Zero scores are often due to repetitions, but why would Stockfish see more repetitions than other engines? Or maybe it's something else.
Maybe it is not the only explanation but my guess is that komodo depth n can also see more 0's than komodo depth n-1.
-
- Posts: 5672
- Joined: Tue Feb 28, 2012 11:56 pm
-
- Posts: 967
- Joined: Fri Mar 10, 2006 4:29 pm
- Location: Germany
- Full name: Jörg Oster
Re: Stockfish zero evals
Yes, you are absolutely right. I have observed this many times as well.
But I have no explanation why this happens ...
But I have no explanation why this happens ...
Jörg Oster
-
- Posts: 967
- Joined: Fri Mar 10, 2006 4:29 pm
- Location: Germany
- Full name: Jörg Oster
Re: Stockfish zero evals
I have seen this also at more shallow depths, so I don't think that's a good explanation.
And I never heard that by searching one or more plies deeper, maybe after implementing Null Move or LMR, an engine began to show more draw scores than before. Did you?
And I never heard that by searching one or more plies deeper, maybe after implementing Null Move or LMR, an engine began to show more draw scores than before. Did you?
Jörg Oster
-
- Posts: 265
- Joined: Sat Feb 22, 2014 8:37 pm
Re: Stockfish zero evals
I can confirm this. Someone on the TCEC analyzed about 30k games from the CCRL from engines rated over 3000 and having no more than 50 elo difference. He let Stockfish analyzing up to a fixed -shallow- depth (depth 14 if I remember well but I may be wrong) for a particular move number on every single game.Joerg Oster wrote:I have seen this also at more shallow depths, so I don't think that's a good explanation.
He then plotted the evaluation on the x-axis and the frequency on the y-axis.
As expected, "0.00" was the most common eval. If I remember well, it was around 11% of all eval values returned. The rest of the shape looked like a wide Gaussian distribution, not centered in 0 (drifted toward positive values).
-
- Posts: 6052
- Joined: Tue Jun 12, 2012 12:41 pm
Re: Stockfish zero evals
Maybe take a look at the below game, at the eval after move 52 for black.Joerg Oster wrote:I have seen this also at more shallow depths, so I don't think that's a good explanation.
And I never heard that by searching one or more plies deeper, maybe after implementing Null Move or LMR, an engine began to show more draw scores than before. Did you?
[pgn][PlyCount "225"]
[MLNrOfMoves "112"]
[MLFlags "100100"]
[Date "2014.04.13"]
[Round "1"]
[White "Senpai 1.0"]
[Black "Stockfish 14041208"]
[Result "1-0"]
[EventDate "2014.??.??"]
[ECO "A49"]
[TimeControl "240+2"]
1. d4 {book} 1... Nf6 {book} 2. Nf3 {book} 2... g6 {book} 3. g3 {book} 3... Bg7
{book} 4. Bg2 {book} 4... c5 {book} 5. d5 {book} 5... O-O {book} 6. c4 {book}
6... d6 {book} 7. O-O {book} 7... b5 {book} 8. cxb5 {book} 8... a6 {book} 9.
bxa6 {+0.65/18 7.2s} 9... Bxa6 {-0.41/21 7.8s} 10. Nc3 {+0.72/20 8.0s} 10...
Nbd7 {-0.40/22 5.3s} 11. Qc2 {+0.72/20 28s} 11... Ng4 {-0.42/ 21 7.1s} 12. Rd1
{+0.90/18 6.1s} 12... Qb6 {-0.38/21 6.5s} 13. Bg5 {+0.93/17 7.0s} 13... Rfb8
{-0.37/22 10s} 14. Rab1 {+0.99/19 6.2s} 14... Qd8 {-0.38/23 5.4s} 15. h3
{+0.87/ 18 6.0s} 15... Nge5 {-0.37/24 5.4s} 16. Nxe5 {+0.87/20 6.5s} 16... Nxe5
{-0.37/24 5.5s} 17. b3 {+0.87/20 9.8s} 17... Bc8 {-0.37/24 5.6s} 18. Ne4
{+0.93/18 5.2s} 18... f5 {-0.36/22 14s} 19. Nc3 {+0.92/19 4.8s} 19... Bd7
{-0.35/23 5.6s} 20. Rdc1 {+0.91/19 4.9s} 20... Ra3 {-0. 30/21 11s} 21. Rb2
{+0.89/18 5.0s} 21... Nf7 {-0.22/22 10s} 22. Bd2 {+0.81/18 9.3s} 22... Rb7
{-0.22/21 5.6s} 23. Kf1 {+0.78/17 6.0s} 23... Qa8 {-0.23/22 16s} 24. Ra1
{+0.77/ 18 9.8s} 24... Qa6 {-0.23/21 5.5s} 25. Kg1 {+0.77/19 7.5s} 25... Bd4
{-0.23/21 11s} 26. e3 {+0.90/19 5.3s} 26... Bg7 {-0.24/21 8.9s} 27. f4
{+0.93/18 4.7s} 27... Rb8 {-0.31/18 8.7s} 28. Bf1 {+0.90/19 5.4s} 28... Qb7
{-0.37/21 14s} 29. Bc4 {+0.90/19 5.1s} 29... Re8 {-0.52/ 23 18s} 30. Re1
{+0.94/19 5.0s} 30... e5 {-0.31/21 16s} 31. Rbb1 {+1.13/16 4.3s} 31... Qa8
{-0.56/22 26s} 32. Rbd1 {+1.12/18 4.9s} 32... Qa5 {-0.53/21 11s} 33. Bd3
{+1.16/18 7. 9s} 33... Qb6 {-0.51/20 5.6s} 34. Kg2 {+1.13/18 3.8s} 34... e4
{-0.59/20 8.7s} 35. Bc4 {+1. 32/20 5.5s} 35... Qa5 {-0.68/24 9.8s} 36. Re2
{+1.43/21 3.9s} 36... Qb6 {-0.81/26 11s} 37. Bc1 {+1.44/22 4.0s} 37... Raa8
{-0.81/26 3.5s} 38. a4 {+1.46/21 4.7s} 38... Re7 {-0.85/24 7.0s} 39. Bb2
{+1.45/21 4.3s} 39... Nd8 {-0.88/24 5.7s} 40. Nb5 {+1.41/21 6.3s} 40... Nb7
{-0.87/25 3.4s} 41. Bxg7 {+1.42/21 4.3s} 41... Rxg7 {-0.87/22 5.7s} 42. Rf2
{+1.41/22 5.1s} 42... g5 {-0.74/21 8.8s} 43. fxg5 {+1.68/19 4.2s} 43... Rxg5
{-1.07/21 4.3s} 44. Qc3 {+1.75/19 5.1s} 44... Rg7 {-0.82/21 9.4s} 45. Kh2
{+1.72/19 3.9s} 45... Nd8 {-0.75/22 7.8s} 46. Be2 {+1.44/20 4.5s} 46... Nf7
{-0.86/23 4.2s} 47. Na3 {+1.36/22 4.9s} 47... Ne5 {-0.86/ 23 2.3s} 48. Nc4
{+1.36/21 1.8s} 48... Nxc4 {-0.96/24 11s} 49. Bxc4 {+1.38/20 5.2s} 49... Qd8
{-0.90/23 4.6s} 50. Rdf1 {+1.38/20 4.2s} 50... h6 {-0.89/21 5.1s} 51. Rg2
{+1.41/ 19 3.5s} 51... Kh7 {-0.86/21 2.2s} 52. Rgf2 {+1.31/20 27s} 52... Kg8
{0.00/35 2.3s} 53. Rf4 {+1.38/18 3.2s} 53... Kh7 {-0.79/21 2.4s} 54. Qe1
{+1.32/17 2.9s} 54... Qg5 {-0.79/20 3.2s} 55. Rg1 {+1.41/20 3.4s} 55... Qf6
{-0.82/21 1.5s} 56. Qa1 {+1.35/19 16s} 56... Qd8 {-0.81/ 22 1.5s} 57. Qb1
{+1.31/20 3.2s} 57... Qg5 {-0.80/20 3.5s} 58. Rg2 {+1.38/19 3.2s} 58... Qe7
{-0.81/21 3.1s} 59. Qb2 {+1.31/19 2.7s} 59... Qf7 {-0.85/21 1.3s} 60. Qa1
{+1.37/ 18 7.0s} 60... Qg6 {-0.85/22 2.3s} 61. Qf1 {+1.41/16 2.7s} 61... Qf7
{-0.82/23 3.6s} 62. Qf2 {+1.31/18 6.2s} 62... Qe7 {-0.82/22 1.3s} 63. Bb5
{+1.41/20 2.6s} 63... Bc8 {-0.95/23 3.5s} 64. Qb2 {+1.41/21 4.5s} 64... Rb8
{-1.04/20 2.4s} 65. Qc3 {+1.41/21 4.1s} 65... Bd7 {-0.98/19 2.4s} 66. Bxd7
{+1.38/20 3.7s} 66... Qxd7 {-1.18/21 2.4s} 67. Qc4 {+1.42/20 3.3s} 67... Rb4
{-1.11/19 2.4s} 68. Qc2 {+1.25/20 2.7s} 68... Qf7 {-1.25/20 2.4s} 69. Qd1
{+1.46/20 3.6s} 69... Qf6 {-1.17/19 2.1s} 70. a5 {+1.71/17 5.0s} 70... Qc3
{-1.44/20 2.7s} 71. Qh5 {+1.66/19 2.6s} 71... Qf6 {-1.44/22 2.4s} 72. Rxf5
{+2.09/20 4.4s} 72... Qg6 {-1. 37/22 1.8s} 73. Qxg6+ {+1.81/20 1.2s} 73... Kxg6
{-1.59/24 3.0s} 74. Rgf2 {+1.80/23 6.9s} 74... Rxb3 {-1.70/19 2.0s} 75. Rf6+
{+1.90/17 2.9s} 75... Kh7 {-1.46/20 1.5s} 76. Rxd6 {+1.73/19 3.9s} 76... Rxe3
{-1.84/20 3.7s} 77. g4 {+1.80/20 2.6s} 77... Rb7 {-1.84/20 2.4s} 78. Rb6
{+2.18/19 3.0s} 78... Rd7 {-1.84/20 1.3s} 79. d6 {+2.24/19 3.0s} 79... Ra3
{-2.36/21 3.5s} 80. a6 {+2.79/16 3.2s} 80... Kg6 {-2.22/21 1.7s} 81. Rc2
{+2.78/17 2. 7s} 81... h5 {-2.29/20 3.0s} 82. gxh5+ {+3.48/16 2.7s} 82... Kxh5
{-2.55/20 2.4s} 83. Rxc5+ {+3.36/18 1.5s} 83... Kh4 {-2.91/20 2.0s} 84. Rc7
{+3.64/18 4.1s} 84... Ra2+ {-4.26/20 2.8s} 85. Kg1 {+4.05/20 4.9s} 85... Rd8
{-4.48/22 2.3s} 86. a7 {+4.17/17 4.7s} 86... Kxh3 {-4. 61/22 2.5s} 87. Rb8
{+4.83/20 3.5s} 87... Rxd6 {-5.14/25 2.4s} 88. Rh7+ {+5.44/20 2. 4s} 88... Kg4
{-5.60/22 2.1s} 89. Rg7+ {+5.59/21 4.2s} 89... Kf5 {-5.68/23 2.6s} 90. Rf8+
{+5.70/19 3.9s} 90... Ke6 {-5.72/26 2.5s} 91. Rg6+ {+5.90/20 4.2s} 91... Ke5
{-5.72/27 1.3s} 92. Re8+ {+5.96/20 2.5s} 92... Kf4 {-5.84/27 3.2s} 93. Rxd6
{+5.98/21 3.8s} 93... Rxa7 {-5. 84/28 1.4s} 94. Rd4 {+5.99/18 2.4s} 94... Ra1+
{-5.90/28 3.7s} 95. Kf2 {+6.01/22 4.1s} 95... Ra2+ {-5.95/28 1.3s} 96. Ke1
{+6.09/17 2.1s} 96... Ra1+ {-5.95/27 2.4s} 97. Kd2 {+6.11/22 4.0s} 97... Ra2+
{-16.57/26 3.5s} 98. Kc3 {+6.13/22 3.9s} 98... Re2 {-9.40/26 2.4s} 99. Kc4
{+6.21/23 3.8s} 99... Kf5 {-22.41/22 1.9s} 100. Kd5 {+6.36/17 2.6s} 100... e3
{-27.41/27 2.8s} 101. Rf8+ {+10.07/21 3.7s} 101... Kg6 {-101.33/32 2.4s} 102.
Rg4+ {+M23/17 5.5s} 102... Kh7 {-M38/23 1.9s} 103. Rf3 {+M21/17 2.8s} 103... Rh2
{-M28/24 2.2s} 104. Rxe3 {+M19/15 0.97s} 104... Rh6 {-M18/25 2.3s} 105. Re7+
{+M17/15 0.92s} 105... Kh8 {-M16/1 0s} 106. Rc4 {+M15/17 2.7s} 106... Rg6
{-M14/30 1.7s} 107. Rc8+ {+M13/20 2.5s} 107... Rg8 {-M12/1 0s} 108. Rxg8+
{+M11/36 0.89s} 108... Kxg8 {-M10/1 0s} 109. Ke6 {+M9/99 0. 25s} 109... Kf8
{-M8/120 0.063s} 110. Kf6 {+M7/99 0s} 110... Kg8 {-M6/1 0s} 111. Re8+
{+M5/ 99 0s} 111... Kh7 {-M4/1 0s} 112. Rd8 {+M3/99 0s} 112... Kh6 {-M2/1 0s}
113. Rh8# {+M1/99 0s, White mates} 1-0
[/pgn]
SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.
-
- Posts: 2292
- Joined: Mon Sep 29, 2008 1:50 am
Re: Stockfish zero evals
This is a well known issue. A good fix is known, but unfortunately testing has shown (in a statistical sense) that even this fix yields a tiny elo loss. The elo loss hasn't been measured very precisely but it should be around 1 elo.SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
Without ideas there is nothing to simplify.
-
- Posts: 4845
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Stockfish zero evals
There is nothing wrong with returning 0.0 here as long as the engine has tried very hard exploring other moves and proves that this is the only best reply, rather than the engine returns 0.0 just because there is repetition. One way to see how the engine reacts is if we let it search the position except the move that repeats, will it find a better alternative move? During a game it is different since there is time constraint.Lyudmil wrote:SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.
-
- Posts: 2283
- Joined: Sat Jun 02, 2012 2:13 am
Re: Stockfish zero evals
Michel wrote:This is a well known issue. A good fix is known, but unfortunately testing has shown (in a statistical sense) that even this fix yields a tiny elo loss. The elo loss hasn't been measured very precisely but it should be around 1 elo.SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.
Giving up 1 Elo point to fix this annoying issue would be Elo well spent!
Regards,
CL