Stockfish zero evals

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

lkaufman
Posts: 5960
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA

Stockfish zero evals

Post by lkaufman »

It seems to me that I see an unusually high number of zero evaluations when analyzing with Stockfish. Does anyone know what causes this? Zero scores are often due to repetitions, but why would Stockfish see more repetitions than other engines? Or maybe it's something else.
Uri Blass
Posts: 10267
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: Stockfish zero evals

Post by Uri Blass »

lkaufman wrote:It seems to me that I see an unusually high number of zero evaluations when analyzing with Stockfish. Does anyone know what causes this? Zero scores are often due to repetitions, but why would Stockfish see more repetitions than other engines? Or maybe it's something else.
searching deeper can explain seeing more 0's because if you search deep enough you can see the draw when the opponent does not see the draw because it does not see deep enough.

Maybe it is not the only explanation but my guess is that komodo depth n can also see more 0's than komodo depth n-1.
syzygy
Posts: 5557
Joined: Tue Feb 28, 2012 11:56 pm

Re: Stockfish zero evals

Post by syzygy »

My guess would be the same.
Joerg Oster
Posts: 937
Joined: Fri Mar 10, 2006 4:29 pm
Location: Germany

Re: Stockfish zero evals

Post by Joerg Oster »

Yes, you are absolutely right. I have observed this many times as well.
But I have no explanation why this happens ...
Jörg Oster
Joerg Oster
Posts: 937
Joined: Fri Mar 10, 2006 4:29 pm
Location: Germany

Re: Stockfish zero evals

Post by Joerg Oster »

I have seen this also at more shallow depths, so I don't think that's a good explanation.

And I never heard that by searching one or more plies deeper, maybe after implementing Null Move or LMR, an engine began to show more draw scores than before. Did you?
Jörg Oster
Isaac
Posts: 265
Joined: Sat Feb 22, 2014 8:37 pm

Re: Stockfish zero evals

Post by Isaac »

Joerg Oster wrote:I have seen this also at more shallow depths, so I don't think that's a good explanation.
I can confirm this. Someone on the TCEC analyzed about 30k games from the CCRL from engines rated over 3000 and having no more than 50 elo difference. He let Stockfish analyzing up to a fixed -shallow- depth (depth 14 if I remember well but I may be wrong) for a particular move number on every single game.
He then plotted the evaluation on the x-axis and the frequency on the y-axis.
As expected, "0.00" was the most common eval. If I remember well, it was around 11% of all eval values returned. The rest of the shape looked like a wide Gaussian distribution, not centered in 0 (drifted toward positive values).
Lyudmil Tsvetkov
Posts: 6052
Joined: Tue Jun 12, 2012 12:41 pm

Re: Stockfish zero evals

Post by Lyudmil Tsvetkov »

Joerg Oster wrote:I have seen this also at more shallow depths, so I don't think that's a good explanation.

And I never heard that by searching one or more plies deeper, maybe after implementing Null Move or LMR, an engine began to show more draw scores than before. Did you?
Maybe take a look at the below game, at the eval after move 52 for black.

[pgn][PlyCount "225"]
[MLNrOfMoves "112"]
[MLFlags "100100"]
[Date "2014.04.13"]
[Round "1"]
[White "Senpai 1.0"]
[Black "Stockfish 14041208"]
[Result "1-0"]
[EventDate "2014.??.??"]
[ECO "A49"]
[TimeControl "240+2"]

1. d4 {book} 1... Nf6 {book} 2. Nf3 {book} 2... g6 {book} 3. g3 {book} 3... Bg7
{book} 4. Bg2 {book} 4... c5 {book} 5. d5 {book} 5... O-O {book} 6. c4 {book}
6... d6 {book} 7. O-O {book} 7... b5 {book} 8. cxb5 {book} 8... a6 {book} 9.
bxa6 {+0.65/18 7.2s} 9... Bxa6 {-0.41/21 7.8s} 10. Nc3 {+0.72/20 8.0s} 10...
Nbd7 {-0.40/22 5.3s} 11. Qc2 {+0.72/20 28s} 11... Ng4 {-0.42/ 21 7.1s} 12. Rd1
{+0.90/18 6.1s} 12... Qb6 {-0.38/21 6.5s} 13. Bg5 {+0.93/17 7.0s} 13... Rfb8
{-0.37/22 10s} 14. Rab1 {+0.99/19 6.2s} 14... Qd8 {-0.38/23 5.4s} 15. h3
{+0.87/ 18 6.0s} 15... Nge5 {-0.37/24 5.4s} 16. Nxe5 {+0.87/20 6.5s} 16... Nxe5
{-0.37/24 5.5s} 17. b3 {+0.87/20 9.8s} 17... Bc8 {-0.37/24 5.6s} 18. Ne4
{+0.93/18 5.2s} 18... f5 {-0.36/22 14s} 19. Nc3 {+0.92/19 4.8s} 19... Bd7
{-0.35/23 5.6s} 20. Rdc1 {+0.91/19 4.9s} 20... Ra3 {-0. 30/21 11s} 21. Rb2
{+0.89/18 5.0s} 21... Nf7 {-0.22/22 10s} 22. Bd2 {+0.81/18 9.3s} 22... Rb7
{-0.22/21 5.6s} 23. Kf1 {+0.78/17 6.0s} 23... Qa8 {-0.23/22 16s} 24. Ra1
{+0.77/ 18 9.8s} 24... Qa6 {-0.23/21 5.5s} 25. Kg1 {+0.77/19 7.5s} 25... Bd4
{-0.23/21 11s} 26. e3 {+0.90/19 5.3s} 26... Bg7 {-0.24/21 8.9s} 27. f4
{+0.93/18 4.7s} 27... Rb8 {-0.31/18 8.7s} 28. Bf1 {+0.90/19 5.4s} 28... Qb7
{-0.37/21 14s} 29. Bc4 {+0.90/19 5.1s} 29... Re8 {-0.52/ 23 18s} 30. Re1
{+0.94/19 5.0s} 30... e5 {-0.31/21 16s} 31. Rbb1 {+1.13/16 4.3s} 31... Qa8
{-0.56/22 26s} 32. Rbd1 {+1.12/18 4.9s} 32... Qa5 {-0.53/21 11s} 33. Bd3
{+1.16/18 7. 9s} 33... Qb6 {-0.51/20 5.6s} 34. Kg2 {+1.13/18 3.8s} 34... e4
{-0.59/20 8.7s} 35. Bc4 {+1. 32/20 5.5s} 35... Qa5 {-0.68/24 9.8s} 36. Re2
{+1.43/21 3.9s} 36... Qb6 {-0.81/26 11s} 37. Bc1 {+1.44/22 4.0s} 37... Raa8
{-0.81/26 3.5s} 38. a4 {+1.46/21 4.7s} 38... Re7 {-0.85/24 7.0s} 39. Bb2
{+1.45/21 4.3s} 39... Nd8 {-0.88/24 5.7s} 40. Nb5 {+1.41/21 6.3s} 40... Nb7
{-0.87/25 3.4s} 41. Bxg7 {+1.42/21 4.3s} 41... Rxg7 {-0.87/22 5.7s} 42. Rf2
{+1.41/22 5.1s} 42... g5 {-0.74/21 8.8s} 43. fxg5 {+1.68/19 4.2s} 43... Rxg5
{-1.07/21 4.3s} 44. Qc3 {+1.75/19 5.1s} 44... Rg7 {-0.82/21 9.4s} 45. Kh2
{+1.72/19 3.9s} 45... Nd8 {-0.75/22 7.8s} 46. Be2 {+1.44/20 4.5s} 46... Nf7
{-0.86/23 4.2s} 47. Na3 {+1.36/22 4.9s} 47... Ne5 {-0.86/ 23 2.3s} 48. Nc4
{+1.36/21 1.8s} 48... Nxc4 {-0.96/24 11s} 49. Bxc4 {+1.38/20 5.2s} 49... Qd8
{-0.90/23 4.6s} 50. Rdf1 {+1.38/20 4.2s} 50... h6 {-0.89/21 5.1s} 51. Rg2
{+1.41/ 19 3.5s} 51... Kh7 {-0.86/21 2.2s} 52. Rgf2 {+1.31/20 27s} 52... Kg8
{0.00/35 2.3s} 53. Rf4 {+1.38/18 3.2s} 53... Kh7 {-0.79/21 2.4s} 54. Qe1
{+1.32/17 2.9s} 54... Qg5 {-0.79/20 3.2s} 55. Rg1 {+1.41/20 3.4s} 55... Qf6
{-0.82/21 1.5s} 56. Qa1 {+1.35/19 16s} 56... Qd8 {-0.81/ 22 1.5s} 57. Qb1
{+1.31/20 3.2s} 57... Qg5 {-0.80/20 3.5s} 58. Rg2 {+1.38/19 3.2s} 58... Qe7
{-0.81/21 3.1s} 59. Qb2 {+1.31/19 2.7s} 59... Qf7 {-0.85/21 1.3s} 60. Qa1
{+1.37/ 18 7.0s} 60... Qg6 {-0.85/22 2.3s} 61. Qf1 {+1.41/16 2.7s} 61... Qf7
{-0.82/23 3.6s} 62. Qf2 {+1.31/18 6.2s} 62... Qe7 {-0.82/22 1.3s} 63. Bb5
{+1.41/20 2.6s} 63... Bc8 {-0.95/23 3.5s} 64. Qb2 {+1.41/21 4.5s} 64... Rb8
{-1.04/20 2.4s} 65. Qc3 {+1.41/21 4.1s} 65... Bd7 {-0.98/19 2.4s} 66. Bxd7
{+1.38/20 3.7s} 66... Qxd7 {-1.18/21 2.4s} 67. Qc4 {+1.42/20 3.3s} 67... Rb4
{-1.11/19 2.4s} 68. Qc2 {+1.25/20 2.7s} 68... Qf7 {-1.25/20 2.4s} 69. Qd1
{+1.46/20 3.6s} 69... Qf6 {-1.17/19 2.1s} 70. a5 {+1.71/17 5.0s} 70... Qc3
{-1.44/20 2.7s} 71. Qh5 {+1.66/19 2.6s} 71... Qf6 {-1.44/22 2.4s} 72. Rxf5
{+2.09/20 4.4s} 72... Qg6 {-1. 37/22 1.8s} 73. Qxg6+ {+1.81/20 1.2s} 73... Kxg6
{-1.59/24 3.0s} 74. Rgf2 {+1.80/23 6.9s} 74... Rxb3 {-1.70/19 2.0s} 75. Rf6+
{+1.90/17 2.9s} 75... Kh7 {-1.46/20 1.5s} 76. Rxd6 {+1.73/19 3.9s} 76... Rxe3
{-1.84/20 3.7s} 77. g4 {+1.80/20 2.6s} 77... Rb7 {-1.84/20 2.4s} 78. Rb6
{+2.18/19 3.0s} 78... Rd7 {-1.84/20 1.3s} 79. d6 {+2.24/19 3.0s} 79... Ra3
{-2.36/21 3.5s} 80. a6 {+2.79/16 3.2s} 80... Kg6 {-2.22/21 1.7s} 81. Rc2
{+2.78/17 2. 7s} 81... h5 {-2.29/20 3.0s} 82. gxh5+ {+3.48/16 2.7s} 82... Kxh5
{-2.55/20 2.4s} 83. Rxc5+ {+3.36/18 1.5s} 83... Kh4 {-2.91/20 2.0s} 84. Rc7
{+3.64/18 4.1s} 84... Ra2+ {-4.26/20 2.8s} 85. Kg1 {+4.05/20 4.9s} 85... Rd8
{-4.48/22 2.3s} 86. a7 {+4.17/17 4.7s} 86... Kxh3 {-4. 61/22 2.5s} 87. Rb8
{+4.83/20 3.5s} 87... Rxd6 {-5.14/25 2.4s} 88. Rh7+ {+5.44/20 2. 4s} 88... Kg4
{-5.60/22 2.1s} 89. Rg7+ {+5.59/21 4.2s} 89... Kf5 {-5.68/23 2.6s} 90. Rf8+
{+5.70/19 3.9s} 90... Ke6 {-5.72/26 2.5s} 91. Rg6+ {+5.90/20 4.2s} 91... Ke5
{-5.72/27 1.3s} 92. Re8+ {+5.96/20 2.5s} 92... Kf4 {-5.84/27 3.2s} 93. Rxd6
{+5.98/21 3.8s} 93... Rxa7 {-5. 84/28 1.4s} 94. Rd4 {+5.99/18 2.4s} 94... Ra1+
{-5.90/28 3.7s} 95. Kf2 {+6.01/22 4.1s} 95... Ra2+ {-5.95/28 1.3s} 96. Ke1
{+6.09/17 2.1s} 96... Ra1+ {-5.95/27 2.4s} 97. Kd2 {+6.11/22 4.0s} 97... Ra2+
{-16.57/26 3.5s} 98. Kc3 {+6.13/22 3.9s} 98... Re2 {-9.40/26 2.4s} 99. Kc4
{+6.21/23 3.8s} 99... Kf5 {-22.41/22 1.9s} 100. Kd5 {+6.36/17 2.6s} 100... e3
{-27.41/27 2.8s} 101. Rf8+ {+10.07/21 3.7s} 101... Kg6 {-101.33/32 2.4s} 102.
Rg4+ {+M23/17 5.5s} 102... Kh7 {-M38/23 1.9s} 103. Rf3 {+M21/17 2.8s} 103... Rh2
{-M28/24 2.2s} 104. Rxe3 {+M19/15 0.97s} 104... Rh6 {-M18/25 2.3s} 105. Re7+
{+M17/15 0.92s} 105... Kh8 {-M16/1 0s} 106. Rc4 {+M15/17 2.7s} 106... Rg6
{-M14/30 1.7s} 107. Rc8+ {+M13/20 2.5s} 107... Rg8 {-M12/1 0s} 108. Rxg8+
{+M11/36 0.89s} 108... Kxg8 {-M10/1 0s} 109. Ke6 {+M9/99 0. 25s} 109... Kf8
{-M8/120 0.063s} 110. Kf6 {+M7/99 0s} 110... Kg8 {-M6/1 0s} 111. Re8+
{+M5/ 99 0s} 111... Kh7 {-M4/1 0s} 112. Rd8 {+M3/99 0s} 112... Kh6 {-M2/1 0s}
113. Rh8# {+M1/99 0s, White mates} 1-0
[/pgn]

SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.
Michel
Posts: 2272
Joined: Mon Sep 29, 2008 1:50 am

Re: Stockfish zero evals

Post by Michel »

SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.
This is a well known issue. A good fix is known, but unfortunately testing has shown (in a statistical sense) that even this fix yields a tiny elo loss. The elo loss hasn't been measured very precisely but it should be around 1 elo.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Stockfish zero evals

Post by Ferdy »

Lyudmil wrote:SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.
There is nothing wrong with returning 0.0 here as long as the engine has tried very hard exploring other moves and proves that this is the only best reply, rather than the engine returns 0.0 just because there is repetition. One way to see how the engine reacts is if we let it search the position except the move that repeats, will it find a better alternative move? During a game it is different since there is time constraint.
carldaman
Posts: 2283
Joined: Sat Jun 02, 2012 2:13 am

Re: Stockfish zero evals

Post by carldaman »

Michel wrote:
SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.
This is a well known issue. A good fix is known, but unfortunately testing has shown (in a statistical sense) that even this fix yields a tiny elo loss. The elo loss hasn't been measured very precisely but it should be around 1 elo.

Giving up 1 Elo point to fix this annoying issue would be Elo well spent!

Regards,
CL