Nemorino 6 (NNUE)

[Guenther]

a8=Q is not a bad move it's simply the best move as it immediately ends the game with a win. As long as tablebases and tablebase adjudication is used, there might be shorter ways to a win than mating.

...it immediately ends the game with an adjudication win.

Most of us are used to checkmate being needed to win. The flaw is with the TBs lacking DTM, but also with the engine that's unable to search out a checkmate on its own, one that would take precedence over the TBs.

Are we sure that Nemorino v6 isn't buggy in some ways?
CCRL is showing Nemorino v6 4CPU (after 90 games) being close to 200 Elo(!) below v6 single-core and about equal to v5 on 4CPU.

Nemorino 6.00 64-bit 3334 +16 −16 54.8%
Nemorino 5.00 64-bit 4CPU 3148 +18 −18 48.1%
Nemorino 6.00 64-bit 4CPU 3146 +64 −68 22.8%

What if the single cpu version played with NNUE? ;-)

[carldaman]

I was under the impression that v6 has NNUE switched on at all times. It's just a NNUE version.

[Florentino]

Are we sure that Nemorino v6 isn't buggy in some ways?
CCRL is showing Nemorino v6 4CPU (after 90 games) being close to 200 Elo(!) below v6 single-core and about equal to v5 on 4CPU.

Nemorino 6.00 64-bit 3334 +16 −16 54.8%
Nemorino 5.00 64-bit 4CPU 3148 +18 −18 48.1%
Nemorino 6.00 64-bit 4CPU 3146 +64 −68 22.8%

No, it's not buggy. The 4-CPU games were AFAIK played on a non-popcount CPU and therefore the OLD_CPU compile is playing here. There is much more performance punishment for NNUE without AVX2 as for classical evaluations. This results in a 4-5 time slowdown for the old cpu binary compared to AVX2 version.

[Guenther]

Are we sure that Nemorino v6 isn't buggy in some ways?
CCRL is showing Nemorino v6 4CPU (after 90 games) being close to 200 Elo(!) below v6 single-core and about equal to v5 on 4CPU.

Nemorino 6.00 64-bit 3334 +16 −16 54.8%
Nemorino 5.00 64-bit 4CPU 3148 +18 −18 48.1%
Nemorino 6.00 64-bit 4CPU 3146 +64 −68 22.8%

No, it's not buggy. The 4-CPU games were AFAIK played on a non-popcount CPU and therefore the OLD_CPU compile is playing here. There is much more performance punishment for NNUE without AVX2 as for classical evaluations. This results in a 4-5 time slowdown for the old cpu binary compared to AVX2 version.

If that is true (played on no popcount) , benching won't help anymore to get reliable reults over different hardware.
(It now would also depend on the kind of opponents)

The easiest solution would be to play NNUE games only on new hardware, other solutions would be to add the cpu now to the name,
which CCRL always tried to avoid, or allowing GUIs with different tc per player to make benching between NNUE and non_NNUE engines
reliable again for different hardware adaption against different opponents. The different tc would only be needed against non-NNUE
opponents.

[RubiChess]

Graham admitted that the Nemorino 4CPU gauntlet needs to be replayed as NNUE was off by some mistake in the settings. Happened also in the 1CPU RubiChess gauntlet where Nemorino had very bad results and Graham already replayed these games. Will make it into CCRL at next update.

But it is true: Different hardware is a problem for reliability.
NNUE engines will suffer more from running on old (non-AVX2) hardware than non-NNUE engines.
Or in other words: Playing NNUE on modern AVX2-aware hardware against non-NNUE engines will result in better Elo than running it on older hardware.

Regards, Andreas

[RubiChess]

Are we sure that Nemorino v6 isn't buggy in some ways?
CCRL is showing Nemorino v6 4CPU (after 90 games) being close to 200 Elo(!) below v6 single-core and about equal to v5 on 4CPU.

Nemorino 6.00 64-bit 3334 +16 −16 54.8%
Nemorino 5.00 64-bit 4CPU 3148 +18 −18 48.1%
Nemorino 6.00 64-bit 4CPU 3146 +64 −68 22.8%

No, it's not buggy. The 4-CPU games were AFAIK played on a non-popcount CPU and therefore the OLD_CPU compile is playing here. There is much more performance punishment for NNUE without AVX2 as for classical evaluations. This results in a 4-5 time slowdown for the old cpu binary compared to AVX2 version.

Especially for Grahams older machines I compiled a non-popcount binary that supports SSE2 and SSSE3 which should also be useful for NNUE and limit the perfomance punishment. Maybe an also an option for Nemorino.

Regards, Andreas

[JohnW]

I watched this game between Nemorino and Houdini 6 and I don't understand why Nemorino couldn't see that it was obviously a draw. It's evaluation was over 100 while Houdini 6 saw that it was draw for dozens of moves. I am no expert but it just seemed strange to me.

[pgn][Event "Nemorino 6-Houdini 6, Blitz 1.0min+1.0s"]
[Site "Home PC"]
[Date "2020.12.22"]
[Round "3"]
[White "Nemorino 6 (NNUE)"]
[Black "Houdini 6 x64-pext"]
[Result "1/2-1/2"]
[ECO "D06"]
[Annotator "0.33;0.10"]
[PlyCount "390"]
[TimeControl "60+1"]

{Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz 4008 MHz W=27.2 plies; 8,959kN/s; 4
TBAs B=25.6 plies; 23,108kN/s} 1. Nf3 {0.33/19 5 Both last book move} d5 {
0.10/20 4} 2. d4 {0.31/19 4} Bf5 {0.06/19 2 (Nf6)} 3. c4 {0.49/18 2} e6 {
0.07/18 0} 4. cxd5 {0.50/17 3 (Nc3)} exd5 {-0.03/18 1} 5. Qb3 {0.52/19 3 (Nc3)}
Nc6 {0.22/20 4} 6. Nc3 {0.61/19 3} Nf6 {-0.03/20 2 (Bb4)} 7. e4 {0.64/20 3}
dxe4 {0.18/22 4} 8. Ne5 {0.47/20 4} Nxe5 {0.13/21 2 (Be6)} 9. dxe5 {0.48/19 4}
Nd7 {0.04/19 1} 10. Bc4 {0.52/19 2} Qe7 {0.10/22 3} 11. e6 {0.45/18 3} Bxe6 {
0.14/23 2 (Nc5)} 12. Qxb7 {0.69/19 3} Rb8 {0.21/22 4 (Rd8)} 13. Qxc7 {0.55/19 4
} Qc5 {0.37/22 3} 14. Qxc5 {0.67/20 5} Nxc5 {0.40/22 2} 15. Bxe6 {0.70/18 2}
Nxe6 {0.19/20 1} 16. Nxe4 {0.64/19 2} f5 {0.22/22 4} 17. Nc3 {0.53/20 3 (Nd2)}
g6 {0.09/22 1} 18. O-O {0.59/20 8} Bg7 {0.09/22 1} 19. Rd1 {0.57/18 2} O-O {
0.21/22 11 (Bxc3)} 20. Kf1 {0.52/18 2 (g3)} Rfd8 {0.23/21 6} 21. Rxd8+ {
0.89/17 2} Nxd8 {0.26/20 3 (Rxd8)} 22. Nd1 {0.87/18 1 (g3)} Ne6 {0.09/20 1} 23.
Rb1 {0.94/18 1 (f4)} Rd8 {0.13/21 3} 24. Ke1 {1.10/19 3 (Nc3)} Kf7 {0.35/19 5
(Be5)} 25. Ne3 {1.30/16 1 (Bd2)} h5 {0.51/19 6 (f4)} 26. Nc4 {1.21/18 1 (Bd2)}
g5 {0.65/20 4} 27. Bd2 {1.21/18 1} f4 {0.51/19 2 (Rd4)} 28. b3 {1.52/18 2} g4 {
0.53/20 2} 29. Ba5 {1.55/18 2 (Kf1)} Rd4 {0.45/20 1 (Rd5)} 30. Rd1 {1.30/18 2
(Rc1)} Kg6 {0.41/20 1 (Ng5)} 31. Rxd4 {1.32/18 1 (Bb4)} Bxd4 {0.27/20 1 (Nxd4)}
32. Kf1 {1.49/17 2 (Ke2)} Ng5 {0.26/21 1 (Kf6)} 33. Nd6 {1.34/19 1 (Bd2)} Be5 {
0.50/22 3 (f3)} 34. Nb5 {1.44/21 1 (Nc8)} a6 {0.16/19 1 (f3)} 35. Nc7 {1.24/19
2 (Bc7)} Bxc7 {0.55/23 2} 36. Bxc7 {1.11/20 2} Kf5 {0.37/22 1} 37. Ke2 {
1.19/20 1} Ke4 {0.54/22 1} 38. b4 {1.10/21 1 (a4)} f3+ {0.65/21 1 (h4)} 39.
gxf3+ {1.41/23 1} Nxf3 {0.80/23 1} 40. a4 {1.33/25 1} Kd5 {0.84/23 1 (h4)} 41.
Kd3 {1.68/24 1} Ne1+ {1.22/24 2 (Nd4)} 42. Ke3 {1.62/22 1 (Kc3)} Nc2+ {0.33/22
1 (Kc4)} 43. Kf4 {1.96/25 1} Nxb4 {0.34/23 0} 44. Kg5 {1.90/25 1} Ke4 {0.32/27
2} 45. Kxh5 {1.76/28 1} Kf5 {0.32/31 1} 46. a5 {1.53/30 1 (Bb6)} Nd3 {0.32/31
1 (Nd5)} 47. Bb6 {1.57/31 1} Ne5 {0.32/28 0 (Ne1)} 48. Kh4 {1.67/30 1} Nf3+ {
0.32/30 1} 49. Kg3 {1.70/25 0} Nd2 {0.32/30 0 (Ng1)} 50. Bd8 {1.71/25 0 (Bc7)}
Ne4+ {0.31/30 1} 51. Kg2 {1.76/29 1} Nd2 {0.31/29 0} 52. Be7 {1.83/24 0} Nb3 {
0.31/31 2} 53. Bb4 {1.70/27 1} Nd4 {0.31/28 0} 54. Bd2 {1.67/26 1} Nf3 {
0.30/32 1} 55. Be3 {1.48/30 1} Ne5 {0.30/30 0} 56. Bb6 {1.59/28 1 (Bc5)} Nf3 {
0.29/33 2} 57. Bc5 {1.60/27 1} Nd2 {0.29/24 0} 58. Bb4 {1.42/29 1} Nf3 {
0.21/29 1} 59. Kg3 {1.51/28 1} Nd4 {0.17/30 1} 60. Bd2 {1.50/29 1 (Bc5)} Ne2+ {
0.13/31 1 (Nb3)} 61. Kh4 {1.46/29 1} Nd4 {0.13/29 0} 62. Be3 {1.49/28 1 (Bc3)}
Nb3 {0.20/32 2} 63. Bb6 {1.29/30 1} Nd2 {0.20/32 0} 64. Kg3 {1.28/30 1 (Kh5)}
Ne4+ {0.13/30 1} 65. Kg2 {1.43/30 1} Nd2 {0.13/30 0} 66. Bd4 {1.45/30 1 (Bc7)}
Nf3 {0.13/33 1} 67. Bb2 {1.43/28 1 (Bg7)} Kf4 {0.13/34 1 (Nd2)} 68. Bc1+ {
1.46/30 1} Kf5 {0.13/34 0} 69. Kg3 {1.43/27 1} Nd4 {0.13/35 1} 70. Bb2 {
1.44/29 1 (Be3)} Ne2+ {0.13/34 1} 71. Kh4 {1.47/28 1} Ng1 {0.13/34 0 (Kf4)} 72.
Bc3 {1.42/29 1 (Ba3)} Nf3+ {0.13/34 1} 73. Kg3 {1.48/30 1} Ng1 {0.13/35 0 (Ng5)
} 74. Ba1 {1.44/27 1 (Bb4)} Ne2+ {0.13/35 1} 75. Kh4 {1.54/29 1} Ng1 {0.12/35
0 (Nc1)} 76. Bg7 {1.43/29 2 (Bb2)} Nf3+ {0.13/34 1} 77. Kg3 {1.43/30 1} Ng1 {
0.12/32 0 (Nd2)} 78. f3 {2.15/29 1 (Bh8)} Nxf3 {0.13/34 1} 79. h3 {2.35/30 1}
Ne5 {0.13/30 0 (Nd2)} 80. Bxe5 {6.66/34 1} gxh3 {0.12/29 0 (Kxe5)} 81. Bd4 {
9.62/44 0 (Bd6)} Ke4 {0.12/34 1} 82. Bg1 {13.47/47 1 (Bb6)} Kd5 {0.12/36 6} 83.
Kxh3 {15.51/52 1} Kc6 {0.12/37 1 (Ke6)} 84. Bb6 {11.20/57 1} Kb7 {0.12/38 0
(Kd6)} 85. Kg4 {14.96/64 1 (Kg3)} Ka8 {0.12/37 1} 86. Kf5 {31.80/76 0 (Kf3)}
Kb8 {0.12/39 1 (Kb7)} 87. Ke5 {106.29/26 1} Kb7 {0.12/38 0 (Kc8)} 88. Kd6 {
13.59/49 0 (Ke4)} Kb8 {0.12/35 2 (Ka8)} 89. Kc6 {16.25/56 0} Ka8 {0.12/36 0
(Kc8)} 90. Be3 {106.29/24 1 (Bf2)} Kb8 {0.12/11 0} 91. Bf4+ {106.29/23 0 (Kc5)}
Ka7 {0.12/38 1 (Ka8)} 92. Bh6 {106.29/23 0 (Bd6)} Kb8 {0.12/40 1 (Ka8)} 93. Kb6
{106.29/23 0 (Kd5)} Ka8 {0.12/31 1} 94. Kxa6 {106.29/24 0} Kb8 {0.12/30 0} 95.
Bf8 {106.29/30 0 (Kb6)} Ka8 {0.12/33 1} 96. Kb5 {106.29/30 0} Ka7 {0.12/33 2}
97. Kb4 {106.30/27 1 (Be7)} Kb7 {0.12/31 3 (Kb8)} 98. Bc5 {109.70/41 13 (Bd6)}
Kb8 {0.12/33 1 (Ka8)} 99. Kb5 {110.90/21 0 (Bg1)} Ka8 {0.12/33 2} 100. Kb6 {
106.29/24 0} Kb8 {0.12/31 0} 101. Be3 {106.30/27 0 (a6)} Ka8 {0.12/32 2} 102.
Kc6 {106.29/27 0 (a6)} Kb8 {0.12/11 0} 103. Kb5 {106.30/28 1 (a6)} Kc8 {
0.12/34 2 (Kb7)} 104. Kb6 {106.29/25 0 (a6)} Kb8 {0.12/35 1} 105. Bc1 {106.29/
29 0 (Bf4+)} Ka8 {0.12/28 1} 106. Bb2 {106.29/29 1 (Kc6)} Kb8 {0.12/11 0} 107.
Ba3 {106.30/28 1 (a6)} Ka8 {0.12/36 1} 108. Kb5 {106.30/27 0 (Bb4)} Ka7 {
0.12/32 2 (Kb8)} 109. Bb2 {106.29/25 1} Kb8 {0.12/32 1 (Kb7)} 110. Ba1 {
114.74/23 1 (Be5+)} Ka7 {0.12/32 1 (Kb7)} 111. Bc3 {113.85/24 0 (Bd4+)} Kb8 {
0.12/33 1 (Kb7)} 112. Kb6 {111.81/25 1 (Bd4)} Ka8 {0.12/37 1} 113. Ba1 {
110.92/27 1 (Bd2)} Kb8 {0.12/11 0} 114. Bb2 {106.29/29 0 (Kc6)} Ka8 {0.12/33 2}
115. a6 {106.29/29 1 (Kc6)} Kb8 {0.12/11 0} 116. Bd4 {106.29/30 0 (Be5+)} Ka8 {
0.12/32 2} 117. Bc3 {106.29/31 1 (Ka5)} Kb8 {0.12/11 0} 118. Be1 {110.03/30 1
(Kc6)} Ka8 {0.12/30 2} 119. Bf2 {106.30/28 0 (Kc6)} Kb8 {0.12/11 0} 120. Bg1 {
106.29/30 1 (Kc6)} Ka8 {0.12/29 2} 121. Ka5 {106.31/30 1 (Be3)} Kb8 {0.12/11 0}
122. Ka4 {106.31/29 1 (a7+)} Ka8 {0.12/33 1} 123. Bc5 {106.29/26 1 (Ka3)} Kb8 {
0.12/11 0} 124. Kb5 {106.29/27 1 (Bd4)} Ka8 {0.12/32 3} 125. Kc6 {106.29/27 1
(Kb6)} Kb8 {0.12/11 0} 126. Kb6 {106.29/28 0 (Be3)} Ka8 {0.12/30 3} 127. Ba3 {
106.29/31 1 (Bb4)} Kb8 {0.12/11 0} 128. Ka5 {106.29/31 1} Ka7 {0.12/31 2 (Ka8)}
129. Bc5+ {106.30/29 1 (Bb4)} Kb8 {0.12/33 1} 130. Be3 {114.54/23 1 (Bf2)} Ka8
{0.12/35 1 (Kc7)} 131. Ka4 {114.34/24 1 (Kb4)} Kb8 {0.12/11 0} 132. Kb5 {
106.29/27 0 (a7+)} Ka8 {0.12/34 2} 133. Bc5 {106.29/29 1 (Kb6)} Kb8 {0.12/11 0}
134. Bd6+ {106.29/28 1 (Bb4)} Ka7 {0.12/31 2 (Ka8)} 135. Bf8 {106.29/29 1 (Bb4)
} Kb8 {0.12/33 1} 136. Kb4 {106.30/28 1 (Ka5)} Ka7 {0.12/29 1 (Kc7)} 137. Ka5 {
106.29/27 1 (Bc5+)} Kb8 {0.12/33 1 (Ka8)} 138. Bc5 {106.31/29 1 (Bd6+)} Ka8 {
0.12/33 1} 139. Kb6 {106.31/30 1 (Bb6)} Kb8 {0.12/32 0} 140. Kc6 {106.29/29 1
(Bd6+)} Ka8 {0.12/32 1} 141. Kd5 {106.29/28 1 (Bd4)} Kb8 {0.12/11 0} 142. Bf2 {
106.30/26 1 (a7+)} Ka8 {0.12/30 3} 143. Bd4 {114.54/22 1 (Kc6)} Kb8 {0.12/11 0}
144. Bg1 {106.40/25 1 (a7+)} Ka8 {0.12/34 2} 145. Ke5 {106.60/24 1 (Kc6)} Kb8 {
0.12/11 0} 146. a7+ {106.60/22 1} Kb7 {0.12/33 0} 147. Kd6 {106.09/29 0 (Ke6)}
Ka8 {0.12/31 3} 148. Kd7 {106.09/31 1 (Bd4)} Kb7 {0.12/11 0} 149. Ke6 {106.09/
32 2 (Bb6)} Ka8 {0.12/31 0} 150. Kd5 {106.09/32 1} Kb7 {0.12/11 0} 151. Bb6 {
106.09/29 1 (Bd4)} Ka8 {0.12/33 3} 152. Bf2 {106.09/31 1 (Ke5)} Kb7 {0.12/11 0}
153. Bg1 {106.09/30 1 (Bg3)} Ka8 {0.12/34 2} 154. Be3 {106.09/31 1 (Bb6)} Kb7 {
0.12/11 0} 155. Kc4 {106.09/31 1 (Kd6)} Ka8 {0.12/33 3} 156. Bf2 {106.09/30 1
(Kc5)} Kb7 {0.12/11 0} 157. Bd4 {106.09/30 4 (Kd5)} Ka8 {0.12/31 3} 158. Kb5 {
106.09/30 1 (Kb3)} Kb7 {0.12/11 0} 159. Bf2 {106.09/31 1 (Ka4)} Ka8 {0.12/34 2}
160. Kb4 {106.09/29 1 (Ka4)} Kb7 {0.12/11 0} 161. Bc5 {106.09/29 1 (Ka5)} Ka8 {
0.12/35 1} 162. Kb3 {106.09/29 1 (Bd4)} Kb7 {0.12/11 0} 163. Be3 {106.09/29 1
(Bd4)} Ka8 {0.12/34 3} 164. Kc4 {106.09/29 1} Kb7 {0.12/33 0} 165. Kc3 {
106.09/29 1 (Kd5)} Ka8 {0.12/32 1} 166. Bf2 {106.09/30 1 (Kb4)} Kb7 {0.12/11 0}
167. Bg1 {106.09/28 1 (Bc5)} Ka8 {0.12/36 1} 168. Bd4 {106.09/28 1 (Kb4)} Kb7 {
0.12/11 0} 169. Kb4 {106.09/28 2 (Kb2)} Ka8 {0.12/32 3} 170. Bb6 {106.09/30 1
(Bc3)} Kb7 {0.12/11 0} 171. Ka5 {106.09/30 1 (Be3)} Ka8 {0.12/30 3} 172. Kb5 {
106.09/31 1 (Bd4)} Kb7 {0.12/11 0} 173. Bc5 {106.09/32 1 (a8R)} Ka8 {0.12/33 2}
174. Ka4 {111.20/27 1 (Be3)} Kb7 {0.12/11 0} 175. Be3 {110.31/25 1 (a8Q+)} Ka8
{0.12/32 2} 176. Bd4 {106.09/30 0 (Kb4)} Kb7 {0.12/11 0} 177. Ka3 {109.42/25 1
(a8Q+)} Ka8 {0.12/34 1} 178. Kb4 {109.42/23 1 (Bc5)} Kb7 {0.12/11 0} 179. Kb3 {
109.42/26 1 (a8Q+)} Ka8 {0.12/35 1} 180. Ka3 {109.42/24 1 (Be3)} Kb7 {0.12/11 0
} 181. Ka2 {109.42/25 1 (a8R)} Ka8 {0.12/30 3} 182. Kb3 {106.49/28 1 (Bf2)} Kb7
{0.12/11 0} 183. Kc3 {108.53/25 1 (a8Q+)} Ka8 {0.12/34 3} 184. Be3 {106.10/27 1
} Kb7 {0.12/11 0} 185. Bb6 {106.09/28 1 (a8Q+)} Ka8 {0.12/29 1} 186. Kd2 {
106.09/29 1} Kb7 {0.12/11 0} 187. Kc1 {106.09/28 3 (Ke1)} Ka8 {0.12/35 1} 188.
Kc2 {106.09/26 1 (Kb1)} Kb7 {0.12/11 0} 189. Kd3 {106.09/29 1 (a8Q+)} Ka8 {
0.12/33 3} 190. Kc4 {105.10/30 5 (Kc3)} Kb7 {0.12/11 0} 191. Kb3 {105.99/31 1
(a8Q+)} Ka8 {0.12/31 2} 192. Ka4 {105.69/25 1 (Be3)} Kb7 {0.12/11 0} 193. Bd4 {
105.20/24 1 (a8Q+)} Ka8 {0.12/33 2} 194. Ka5 {105.00/26 0 (Ba1)} Kb7 {0.12/11 0
} 195. a8=Q+ {105.00/27 1} Kxa8 {0.12/9 0 Draw accepted} 1/2-1/2

[/pgn]

[carldaman]

I watched this game between Nemorino and Houdini 6 and I don't understand why Nemorino couldn't see that it was obviously a draw. It's evaluation was over 100 while Houdini 6 saw that it was draw for dozens of moves. I am no expert but it just seemed strange to me.

Were you using (syzygy) tablebases at all with Nemorino? If you were, then something is clearly wrong with its usage of TBs. If not, then it must be on the account of missing proper endgame knowledge, something that so many otherwise strong engines lack.

[Werner]

...perhaps they use one of the newer nets which are not compatible to version 6.0 ?

[Florentino]

I watched this game between Nemorino and Houdini 6 and I don't understand why Nemorino couldn't see that it was obviously a draw. It's evaluation was over 100 while Houdini 6 saw that it was draw for dozens of moves. I am no expert but it just seemed strange to me.

Nemorino 6 has embedded 4 men Tablebases and therefore scores these position with 0 if no tablebases are provided. Only explanantion I have is that you provided an invalid tablebase path, resulting in no tablebase use at all. Is this possible?