Stockfish zero evals

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
lkaufman
Posts: 3722
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Stockfish zero evals

Post by lkaufman » Fri May 02, 2014 6:59 pm

It seems to me that I see an unusually high number of zero evaluations when analyzing with Stockfish. Does anyone know what causes this? Zero scores are often due to repetitions, but why would Stockfish see more repetitions than other engines? Or maybe it's something else.

Uri Blass
Posts: 8586
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Stockfish zero evals

Post by Uri Blass » Fri May 02, 2014 7:06 pm

lkaufman wrote:It seems to me that I see an unusually high number of zero evaluations when analyzing with Stockfish. Does anyone know what causes this? Zero scores are often due to repetitions, but why would Stockfish see more repetitions than other engines? Or maybe it's something else.
searching deeper can explain seeing more 0's because if you search deep enough you can see the draw when the opponent does not see the draw because it does not see deep enough.

Maybe it is not the only explanation but my guess is that komodo depth n can also see more 0's than komodo depth n-1.

syzygy
Posts: 4455
Joined: Tue Feb 28, 2012 10:56 pm

Re: Stockfish zero evals

Post by syzygy » Fri May 02, 2014 7:53 pm

My guess would be the same.

Joerg Oster
Posts: 688
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: Stockfish zero evals

Post by Joerg Oster » Fri May 02, 2014 9:00 pm

Yes, you are absolutely right. I have observed this many times as well.
But I have no explanation why this happens ...
Jörg Oster

Joerg Oster
Posts: 688
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: Stockfish zero evals

Post by Joerg Oster » Fri May 02, 2014 9:14 pm

I have seen this also at more shallow depths, so I don't think that's a good explanation.

And I never heard that by searching one or more plies deeper, maybe after implementing Null Move or LMR, an engine began to show more draw scores than before. Did you?
Jörg Oster

Isaac
Posts: 265
Joined: Sat Feb 22, 2014 7:37 pm

Re: Stockfish zero evals

Post by Isaac » Sat May 03, 2014 1:23 am

Joerg Oster wrote:I have seen this also at more shallow depths, so I don't think that's a good explanation.
I can confirm this. Someone on the TCEC analyzed about 30k games from the CCRL from engines rated over 3000 and having no more than 50 elo difference. He let Stockfish analyzing up to a fixed -shallow- depth (depth 14 if I remember well but I may be wrong) for a particular move number on every single game.
He then plotted the evaluation on the x-axis and the frequency on the y-axis.
As expected, "0.00" was the most common eval. If I remember well, it was around 11% of all eval values returned. The rest of the shape looked like a wide Gaussian distribution, not centered in 0 (drifted toward positive values).

Lyudmil Tsvetkov
Posts: 6052
Joined: Tue Jun 12, 2012 10:41 am

Re: Stockfish zero evals

Post by Lyudmil Tsvetkov » Sat May 03, 2014 3:29 am

Joerg Oster wrote:I have seen this also at more shallow depths, so I don't think that's a good explanation.

And I never heard that by searching one or more plies deeper, maybe after implementing Null Move or LMR, an engine began to show more draw scores than before. Did you?
Maybe take a look at the below game, at the eval after move 52 for black.

[pgn][PlyCount "225"]
[MLNrOfMoves "112"]
[MLFlags "100100"]
[Date "2014.04.13"]
[Round "1"]
[White "Senpai 1.0"]
[Black "Stockfish 14041208"]
[Result "1-0"]
[EventDate "2014.??.??"]
[ECO "A49"]
[TimeControl "240+2"]

1. d4 {book} 1... Nf6 {book} 2. Nf3 {book} 2... g6 {book} 3. g3 {book} 3... Bg7
{book} 4. Bg2 {book} 4... c5 {book} 5. d5 {book} 5... O-O {book} 6. c4 {book}
6... d6 {book} 7. O-O {book} 7... b5 {book} 8. cxb5 {book} 8... a6 {book} 9.
bxa6 {+0.65/18 7.2s} 9... Bxa6 {-0.41/21 7.8s} 10. Nc3 {+0.72/20 8.0s} 10...
Nbd7 {-0.40/22 5.3s} 11. Qc2 {+0.72/20 28s} 11... Ng4 {-0.42/ 21 7.1s} 12. Rd1
{+0.90/18 6.1s} 12... Qb6 {-0.38/21 6.5s} 13. Bg5 {+0.93/17 7.0s} 13... Rfb8
{-0.37/22 10s} 14. Rab1 {+0.99/19 6.2s} 14... Qd8 {-0.38/23 5.4s} 15. h3
{+0.87/ 18 6.0s} 15... Nge5 {-0.37/24 5.4s} 16. Nxe5 {+0.87/20 6.5s} 16... Nxe5
{-0.37/24 5.5s} 17. b3 {+0.87/20 9.8s} 17... Bc8 {-0.37/24 5.6s} 18. Ne4
{+0.93/18 5.2s} 18... f5 {-0.36/22 14s} 19. Nc3 {+0.92/19 4.8s} 19... Bd7
{-0.35/23 5.6s} 20. Rdc1 {+0.91/19 4.9s} 20... Ra3 {-0. 30/21 11s} 21. Rb2
{+0.89/18 5.0s} 21... Nf7 {-0.22/22 10s} 22. Bd2 {+0.81/18 9.3s} 22... Rb7
{-0.22/21 5.6s} 23. Kf1 {+0.78/17 6.0s} 23... Qa8 {-0.23/22 16s} 24. Ra1
{+0.77/ 18 9.8s} 24... Qa6 {-0.23/21 5.5s} 25. Kg1 {+0.77/19 7.5s} 25... Bd4
{-0.23/21 11s} 26. e3 {+0.90/19 5.3s} 26... Bg7 {-0.24/21 8.9s} 27. f4
{+0.93/18 4.7s} 27... Rb8 {-0.31/18 8.7s} 28. Bf1 {+0.90/19 5.4s} 28... Qb7
{-0.37/21 14s} 29. Bc4 {+0.90/19 5.1s} 29... Re8 {-0.52/ 23 18s} 30. Re1
{+0.94/19 5.0s} 30... e5 {-0.31/21 16s} 31. Rbb1 {+1.13/16 4.3s} 31... Qa8
{-0.56/22 26s} 32. Rbd1 {+1.12/18 4.9s} 32... Qa5 {-0.53/21 11s} 33. Bd3
{+1.16/18 7. 9s} 33... Qb6 {-0.51/20 5.6s} 34. Kg2 {+1.13/18 3.8s} 34... e4
{-0.59/20 8.7s} 35. Bc4 {+1. 32/20 5.5s} 35... Qa5 {-0.68/24 9.8s} 36. Re2
{+1.43/21 3.9s} 36... Qb6 {-0.81/26 11s} 37. Bc1 {+1.44/22 4.0s} 37... Raa8
{-0.81/26 3.5s} 38. a4 {+1.46/21 4.7s} 38... Re7 {-0.85/24 7.0s} 39. Bb2
{+1.45/21 4.3s} 39... Nd8 {-0.88/24 5.7s} 40. Nb5 {+1.41/21 6.3s} 40... Nb7
{-0.87/25 3.4s} 41. Bxg7 {+1.42/21 4.3s} 41... Rxg7 {-0.87/22 5.7s} 42. Rf2
{+1.41/22 5.1s} 42... g5 {-0.74/21 8.8s} 43. fxg5 {+1.68/19 4.2s} 43... Rxg5
{-1.07/21 4.3s} 44. Qc3 {+1.75/19 5.1s} 44... Rg7 {-0.82/21 9.4s} 45. Kh2
{+1.72/19 3.9s} 45... Nd8 {-0.75/22 7.8s} 46. Be2 {+1.44/20 4.5s} 46... Nf7
{-0.86/23 4.2s} 47. Na3 {+1.36/22 4.9s} 47... Ne5 {-0.86/ 23 2.3s} 48. Nc4
{+1.36/21 1.8s} 48... Nxc4 {-0.96/24 11s} 49. Bxc4 {+1.38/20 5.2s} 49... Qd8
{-0.90/23 4.6s} 50. Rdf1 {+1.38/20 4.2s} 50... h6 {-0.89/21 5.1s} 51. Rg2
{+1.41/ 19 3.5s} 51... Kh7 {-0.86/21 2.2s} 52. Rgf2 {+1.31/20 27s} 52... Kg8
{0.00/35 2.3s} 53. Rf4 {+1.38/18 3.2s} 53... Kh7 {-0.79/21 2.4s} 54. Qe1
{+1.32/17 2.9s} 54... Qg5 {-0.79/20 3.2s} 55. Rg1 {+1.41/20 3.4s} 55... Qf6
{-0.82/21 1.5s} 56. Qa1 {+1.35/19 16s} 56... Qd8 {-0.81/ 22 1.5s} 57. Qb1
{+1.31/20 3.2s} 57... Qg5 {-0.80/20 3.5s} 58. Rg2 {+1.38/19 3.2s} 58... Qe7
{-0.81/21 3.1s} 59. Qb2 {+1.31/19 2.7s} 59... Qf7 {-0.85/21 1.3s} 60. Qa1
{+1.37/ 18 7.0s} 60... Qg6 {-0.85/22 2.3s} 61. Qf1 {+1.41/16 2.7s} 61... Qf7
{-0.82/23 3.6s} 62. Qf2 {+1.31/18 6.2s} 62... Qe7 {-0.82/22 1.3s} 63. Bb5
{+1.41/20 2.6s} 63... Bc8 {-0.95/23 3.5s} 64. Qb2 {+1.41/21 4.5s} 64... Rb8
{-1.04/20 2.4s} 65. Qc3 {+1.41/21 4.1s} 65... Bd7 {-0.98/19 2.4s} 66. Bxd7
{+1.38/20 3.7s} 66... Qxd7 {-1.18/21 2.4s} 67. Qc4 {+1.42/20 3.3s} 67... Rb4
{-1.11/19 2.4s} 68. Qc2 {+1.25/20 2.7s} 68... Qf7 {-1.25/20 2.4s} 69. Qd1
{+1.46/20 3.6s} 69... Qf6 {-1.17/19 2.1s} 70. a5 {+1.71/17 5.0s} 70... Qc3
{-1.44/20 2.7s} 71. Qh5 {+1.66/19 2.6s} 71... Qf6 {-1.44/22 2.4s} 72. Rxf5
{+2.09/20 4.4s} 72... Qg6 {-1. 37/22 1.8s} 73. Qxg6+ {+1.81/20 1.2s} 73... Kxg6
{-1.59/24 3.0s} 74. Rgf2 {+1.80/23 6.9s} 74... Rxb3 {-1.70/19 2.0s} 75. Rf6+
{+1.90/17 2.9s} 75... Kh7 {-1.46/20 1.5s} 76. Rxd6 {+1.73/19 3.9s} 76... Rxe3
{-1.84/20 3.7s} 77. g4 {+1.80/20 2.6s} 77... Rb7 {-1.84/20 2.4s} 78. Rb6
{+2.18/19 3.0s} 78... Rd7 {-1.84/20 1.3s} 79. d6 {+2.24/19 3.0s} 79... Ra3
{-2.36/21 3.5s} 80. a6 {+2.79/16 3.2s} 80... Kg6 {-2.22/21 1.7s} 81. Rc2
{+2.78/17 2. 7s} 81... h5 {-2.29/20 3.0s} 82. gxh5+ {+3.48/16 2.7s} 82... Kxh5
{-2.55/20 2.4s} 83. Rxc5+ {+3.36/18 1.5s} 83... Kh4 {-2.91/20 2.0s} 84. Rc7
{+3.64/18 4.1s} 84... Ra2+ {-4.26/20 2.8s} 85. Kg1 {+4.05/20 4.9s} 85... Rd8
{-4.48/22 2.3s} 86. a7 {+4.17/17 4.7s} 86... Kxh3 {-4. 61/22 2.5s} 87. Rb8
{+4.83/20 3.5s} 87... Rxd6 {-5.14/25 2.4s} 88. Rh7+ {+5.44/20 2. 4s} 88... Kg4
{-5.60/22 2.1s} 89. Rg7+ {+5.59/21 4.2s} 89... Kf5 {-5.68/23 2.6s} 90. Rf8+
{+5.70/19 3.9s} 90... Ke6 {-5.72/26 2.5s} 91. Rg6+ {+5.90/20 4.2s} 91... Ke5
{-5.72/27 1.3s} 92. Re8+ {+5.96/20 2.5s} 92... Kf4 {-5.84/27 3.2s} 93. Rxd6
{+5.98/21 3.8s} 93... Rxa7 {-5. 84/28 1.4s} 94. Rd4 {+5.99/18 2.4s} 94... Ra1+
{-5.90/28 3.7s} 95. Kf2 {+6.01/22 4.1s} 95... Ra2+ {-5.95/28 1.3s} 96. Ke1
{+6.09/17 2.1s} 96... Ra1+ {-5.95/27 2.4s} 97. Kd2 {+6.11/22 4.0s} 97... Ra2+
{-16.57/26 3.5s} 98. Kc3 {+6.13/22 3.9s} 98... Re2 {-9.40/26 2.4s} 99. Kc4
{+6.21/23 3.8s} 99... Kf5 {-22.41/22 1.9s} 100. Kd5 {+6.36/17 2.6s} 100... e3
{-27.41/27 2.8s} 101. Rf8+ {+10.07/21 3.7s} 101... Kg6 {-101.33/32 2.4s} 102.
Rg4+ {+M23/17 5.5s} 102... Kh7 {-M38/23 1.9s} 103. Rf3 {+M21/17 2.8s} 103... Rh2
{-M28/24 2.2s} 104. Rxe3 {+M19/15 0.97s} 104... Rh6 {-M18/25 2.3s} 105. Re7+
{+M17/15 0.92s} 105... Kh8 {-M16/1 0s} 106. Rc4 {+M15/17 2.7s} 106... Rg6
{-M14/30 1.7s} 107. Rc8+ {+M13/20 2.5s} 107... Rg8 {-M12/1 0s} 108. Rxg8+
{+M11/36 0.89s} 108... Kxg8 {-M10/1 0s} 109. Ke6 {+M9/99 0. 25s} 109... Kf8
{-M8/120 0.063s} 110. Kf6 {+M7/99 0s} 110... Kg8 {-M6/1 0s} 111. Re8+
{+M5/ 99 0s} 111... Kh7 {-M4/1 0s} 112. Rd8 {+M3/99 0s} 112... Kh6 {-M2/1 0s}
113. Rh8# {+M1/99 0s, White mates} 1-0
[/pgn]

SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.

Michel
Posts: 2046
Joined: Sun Sep 28, 2008 11:50 pm

Re: Stockfish zero evals

Post by Michel » Sat May 03, 2014 6:06 am

SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.
This is a well known issue. A good fix is known, but unfortunately testing has shown (in a statistical sense) that even this fix yields a tiny elo loss. The elo loss hasn't been measured very precisely but it should be around 1 elo.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.

Ferdy
Posts: 4110
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Stockfish zero evals

Post by Ferdy » Sat May 03, 2014 6:51 am

Lyudmil wrote:SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.
There is nothing wrong with returning 0.0 here as long as the engine has tried very hard exploring other moves and proves that this is the only best reply, rather than the engine returns 0.0 just because there is repetition. One way to see how the engine reacts is if we let it search the position except the move that repeats, will it find a better alternative move? During a game it is different since there is time constraint.

carldaman
Posts: 1717
Joined: Sat Jun 02, 2012 12:13 am

Re: Stockfish zero evals

Post by carldaman » Sat May 03, 2014 7:43 am

Michel wrote:
SF expects draw by repetition after the first repetition that occurs in the position and returns 0.0 instead of the objective best move and score.
This is a well known issue. A good fix is known, but unfortunately testing has shown (in a statistical sense) that even this fix yields a tiny elo loss. The elo loss hasn't been measured very precisely but it should be around 1 elo.

Giving up 1 Elo point to fix this annoying issue would be Elo well spent!

Regards,
CL

Post Reply