Leela (lack of) endgame technique?

[corres]

Be aware that the version of Leela playing over there has
1. No TB support
2. Not the best version ( best version of 10xxx net is about +25 elo better)
3. 50 moves rules bug in almost whole training
4. Terrible Time managment with pondering on setting (not properly tested)
These conditions cause leela about -100 elo.
...

...and engines do not use Opening Book (as it was AlphaZero-Stockfish 8 match), TC is rapid
(15min + 5sec/move), Stockfish use 8 GB memory but 4 X Tesla V100 use 64 GB memory -
so these circumstances give Leela ~200 Elo benefit...
At the end of Stage 2 we may take a little realer picture about the power of Leela.

[corres]

Leela Chess Zero - like AlphaZero - has no any "technique".
Chess engines based on Neural Network have only a memory with big capacity and a searching motor (MCTS)
to find appropriate move memorized during learning cycle for the analyzed position.
Physically the measure of NN memory is not big (some 100MB) but using filters make the memory virtually
more bigger.
If developers of Leela want to be Leela with good endgame knowledge they ought to growth the measure of NN and to play much more learning play.

[Uri Blass]

yes, sure, you are right that hardware-developement plays a major role in computerchess and the speed of progress as well !
OTOH: if u state that after around 3-4 months of training a NN gets a playing-strengh that is lets say 200 elo weaker than the top of the crop (a-b-engines (PC) after more than 25 years of developement) then its obvious to believe that this new approach made quicker progress than alpha-beta-prunning did and will substitute or merge with the "old" approach, isnt it?...oc we can speculate that there is a "wall" where NN's in chess stagnate and the best approach stays the a-b-prunning approach but only time can show us...i dont believe in it but i err too often to be sure
imo we witness the end of PURE a-b-engines within the next few years...just a personal point of view in respect of the data i get from the recent developement of NN's+chess and their meassured strengh.

I do not agree that after 3-4 months of training a NN gets playing strength that is 200 elo weaker than the top of a-b engines.
I think the NN simply use a significantly superior hardware.

The only fair comparison is to give the NN the same hardware (CPU and not GPU) because GPU today is significantly superior hardware for chess.

I believe that a-b engines can be significantly stronger if they use the GPU and not the CPU but they are not optimized to use the GPU today.

If I understand correctly stockfish today cannot even use the GPU because it is not programmed to do it so we cannot check what is stockfish's initial rating using the GPU.

Note that I expect worse initial rating for stockfish in case that somebody convert it to use the GPU but if the same person continue to develop stockfish GPU then I expect stockfish GPU to become better than Stockfish CPU after enough time of developement.

The point is that there may be changes in the algorithm that are productive with GPU and counterproductive with CPU(for example maybe it is possible to have some better evaluation function that is too expensive for the CPU but cheap with the GPU).

[Spliffjiffer]

hi Uri, maybe maybe future will show us
in terms of "fair comparison" though its much too personally to get ever consensus
sounds like Compare apples to pears to me

[muxecoid]

The last CCCC game against Pedone was disgusting. Any proper modern engine would win the endgame.

[MikeGL]

I just checked with SF and the endgame phase is just around +1.xx for Lc0.
Endgame Tablebase said that position is drawn, as strange as it may seem.

After 81...Rxe7 of black, this position is a TB draw.
No way for Lc0 to win this, even if it is a pawn up with a major piece vs. a minor piece.

[d]8/4r3/8/3B4/1k6/p7/K7/8 w - - 0 82

The last CCCC game against Pedone was disgusting. Any proper modern engine would win the endgame.

[muxecoid]

I mean earlier position. With rook and minor piece vs two minor pieces. If only exchanging minor pieces was delayed there should have been a way to win.

[OliverBr]

Hello together,
I (author of OliThink), I am very impressed about Leela Chess Zero.
It actually follows the path I was trying to do with Oliver. Create a strong chess engine with any chess knowledge whatever (except the rules, of course).
Fortunately I do now have a fine system, a MacBookPro 2018 15inch, with a powerful eGPU, GTX1080Ti. So (in High Sierra), I can use CUDA to get optimal performance from LC0.
Wit this hardware, Leela Chess Zero (0.24.1) completely and utterly owns and humiliates Stockfish 11 in midgame. Is this a surprise? It's difficult so say, because, honestly, the endgame, kind-a is strange. Whatever winning endgame, LC0 sacrifices pieces for nothing, only when 50-moves rule is threatening, it plays fine and wins. It is kind of funny in some way, as it appears like LC0 is torturing the losing side.

A fine example would be this game that was won since move 40:

Code: Select all

[Event "Computer Chess Game"]
[Site "Olivers-MacBook-2.local"]
[Date "2020.05.08"]
[Round "-"]
[White "Lc0 v0.24.1+git.dirty"]
[Black "Stockfish 11 64"]
[Result "1-0"]
[TimeControl "40/180"]
[Annotator "2. +0.29   1... -0.20"]

1. d4 d5 {-0.20/23 4} 2. c4 {+0.29/9 2.6} e6 {-0.06/24 5} 3. Nf3
{+0.28/11 6} Nf6 {-0.07/22 2.0} 4. g3 {+0.29/10 2.9} Be7 {+0.29/23 5} 5.
Bg2 {+0.29/10 2.5} O-O {+0.34/21 1.7} 6. O-O {+0.30/12 3} dxc4
{+0.42/24 0.3} 7. Qc2 {+0.32/12 3} b5 {+0.29/25 4} 8. a4 {+0.45/14 5} b4
{+0.22/26 0.1} 9. Nfd2 {+0.49/18 7} c6 {+0.00/30 0.1} 10. Nxc4
{+0.48/18 1.0} Qxd4 {+0.28/29 1.6} 11. Rd1 {+0.53/18 2.1} Qc5 {+0.00/30 4}
12. Be3 {+0.55/18 0.2} Qh5 {+0.15/23 2.5} 13. Nbd2 {+0.55/16 3} Ng4
{+0.16/24 2.0} 14. Nf3 {+0.43/15 14} Nxe3 {+0.00/31 0.2} 15. Nxe3
{+0.41/13 0.2} a6 {+0.00/28 1.4} 16. Nc4 {+0.75/12 11} Ra7 {+0.31/26 8} 17.
Rac1 {+0.81/14 4} c5 {+0.12/22 2.0} 18. Nfe5 {+0.82/12 1.9} g6
{+0.00/29 17} 19. h4 {+1.03/10 8} Bb7 {+0.00/28 15} 20. Qd3 {+1.08/14 5}
Bxg2 {+0.00/23 2.8} 21. Kxg2 {+1.05/17 0.4} Rc7 {-0.54/33 18} 22. b3
{+1.29/14 6} Qf5 {-0.59/25 2.0} 23. Qe3 {+1.56/13 5} Nc6 {-0.97/27 0.7} 24.
Nxc6 {+1.60/15 4} Rxc6 {-0.97/30 0.1} 25. Rd7 {+1.63/14 2.3} Bf6
{-1.22/32 17} 26. Rcd1 {+1.74/17 1.2} Bc3 {-1.16/25 1.7} 27. Qf3
{+1.77/12 8} Rfc8 {-1.18/31 0.1} 28. Nd6 {+1.75/13 7} Qxf3+ {-1.40/24 4}
29. Kxf3 {+1.76/14 0.2} Ra8 {-1.34/26 2.6} 30. e3 {+1.88/11 8} Rf8
{-1.47/29 7} 31. Nc4 {+1.94/11 6} Rfc8 {-1.49/31 0.2} 32. g4 {+2.00/11 9}
f5 {-1.57/27 12} 33. g5 {+1.98/13 0.9} e5 {-1.54/24 1.4} 34. Nd6
{+1.99/12 2.8} Rf8 {-1.77/26 5} 35. a5 {+2.08/13 4} e4+ {-2.84/27 12} 36.
Kf4 {+2.16/12 0.6} c4 {-3.06/28 6} 37. Nxc4 {+2.28/13 5} Rf7 {-3.30/26 0.2}
38. h5 {+2.44/15 6} gxh5 {-2.92/26 2.8} 39. Rxf7 {+2.52/15 4} Kxf7
{-2.98/24 2.1} 40. Nd6+ {+2.61/15 2.1} Kg6 {-2.95/26 0.3} 41. Nxf5
{+2.73/15 8} Re6 {-3.47/32 9} 42. Nh4+ {+2.84/15 4} Kf7 {-4.22/35 29} 43.
Rd7+ {+3.02/14 0.2} Re7 {-4.33/36 25} 44. Rxe7+ {+3.12/14 0.2} Kxe7
{-4.35/31 2.4} 45. Kxe4 {+3.20/13 10} Be1 {-4.75/35 5} 46. Kf3 {+3.30/13 8}
Bc3 {-5.37/37 23} 47. Ng2 {+3.60/11 46} Be5 {-5.91/37 0.2} 48. Nf4
{+4.27/9 12} Kf8 {-6.16/30 0.2} 49. Nxh5 {+4.94/8 21} Bc7 {-5.71/28 2.8}
50. Nf6 {+4.79/8 11} Bxa5 {-7.03/29 0.2} 51. Nxh7+ {+5.27/9 5} Kf7
{-7.54/29 1.9} 52. Nf6 {+5.76/8 7} Bb6 {-8.66/30 17} 53. Nd5 {+7.21/7 10}
Bd8 {-7.36/26 5} 54. Nxb4 {+11.54/7 4} Bxg5 {-8.81/30 18} 55. Nxa6
{+15.67/6 7} Bd8 {-9.86/28 1.2} 56. Nb4 {+22.78/6 6} Ke6 {-12.11/26 2.6}
57. Na2 {+21.51/6 4} Ke5 {-6.13/25 1.2} 58. Nc3 {+17.64/6 4} Ba5
{-8.58/27 7} 59. Na4 {+15.10/7 3} Kd5 {-9.92/29 4} 60. Nb2 {+16.73/6 2.7}
Bc7 {-10.91/28 4} 61. Nd3 {+19.40/6 2.6} Bd8 {-9.98/27 0.3} 62. Ne1
{+20.52/6 2.2} Bc7 {-11.79/29 3} 63. Nc2 {+20.61/6 1.9} Bd6 {-9.92/25 2.8}
64. Ne1 {+20.68/6 1.7} Bb8 {-11.85/29 0.2} 65. Ng2 {+21.74/6 1.5} Kc5
{-8.89/25 1.0} 66. Nf4 {+22.17/6 1.3} Kb4 {-5.73/21 0.5} 67. Ke4
{+24.19/5 1.1} Kc5 {-13.67/26 6} 68. Nd3+ {+23.80/6 1.0} Kd6
{-13.68/23 0.2} 69. Nc1 {+29.08/5 0.8} Ke6 {-11.70/25 0.9} 70. Ne2
{+30.67/5 0.7} Kd7 {-12.18/26 2.0} 71. Kd3 {+27.15/5 0.6} Bc7
{-11.18/27 2.5} 72. Ke4 {+27.36/5 0.5} Ba5 {-12.66/24 0.1} 73. f4
{+30.48/5 0.5} Ke7 {-12.75/27 1.1} 74. Ng3 {+26.94/5 0.5} Kd6
{-12.58/26 1.1} 75. Nf1 {+23.33/5 0.5} Kc5 {-5.76/17 0.2} 76. Kd3
{+29.02/5 0.5} Bc7 {-6.69/18 0.2} 77. Ke4 {+22.13/5 0.5} Bd8 {-8.06/22 0.5}
78. Nd2 {+33.62/5 0.5} Bf6 {-8.21/19 0.3} 79. Nf3 {+34.00/5 0.5} Bc3
{-9.82/18 0.2} 80. Kd3 {+25.30/5 0.5} Bg7 {-9.43/16 0.1} 81. Ng5
{+26.01/6 21} Kd6 {-12.85/30 10} 82. Nf3 {+23.32/6 18} Bf8 {-25.76/31 30}
83. Ke4 {+27.18/6 16} Ke6 {-47.72/32 17} 84. Nd4+ {+22.65/7 14} Kf7
{-63.75/37 29} 85. Ne2 {+26.77/7 12} Bc5 {-15.39/26 4} 86. Kd3
{+22.35/6 11} Kg6 {-47.97/38 2.2} 87. e4 {+26.45/6 9} Bf2 {-36.48/27 7} 88.
Nc3 {+27.59/6 8} Bc5 {-64.76/37 10} 89. Kc4 {+33.31/6 7} Be3 {-66.94/36 7}
90. Ne2 {+25.68/6 6} Kh5 {-69.76/33 1.7} 91. Kd5 {+28.08/6 6} Kg6
{-71.24/35 2.1} 92. b4 {+31.99/6 5} Kf7 {-64.23/29 5} 93. b5 {+32.91/6 5}
Ke8 {-1000.26/30 7} 94. f5 {+37.74/6 4} Bb6 {-1000.16/35 0.1} 95. e5
{+42.49/6 4} Bd8 {-1000.14/49 1.3} 96. Nf4 {+45.55/6 3} Kf7
{-1000.15/42 1.3} 97. Nh5 {+44.62/6 2.9} Kf8 {-1000.13/34 2.4} 98. Ng3
{+44.37/6 2.6} Kg7 {-1000.15/30 1.3} 99. Nf1 {+48.66/6 2.3} Kh6
{-58.87/23 1.4} 100. Kd6 {+52.05/6 2.0} Kg5 {-147.92/28 6} 101. f6
{+44.77/6 1.8} Ba5 {-1000.14/30 1.4} 102. Ne3 {+55.69/6 1.7} Bb4+
{-1000.14/28 4} 103. Ke6 {+57.44/6 1.5} Bc5 {-1000.14/29 1.4} 104. Nc4
{+61.88/5 1.3} Kf4 {-1000.12/29 1.5} 105. Nd6 {+58.77/5 1.2} Kg4
{-1000.13/34 1.4} 106. Nb7 {+60.70/5 1.1} Ba7 {-1000.13/36 2.2} 107. b6
{+50.44/5 1.0} Bxb6 {-1000.12/28 0.8} 108. Nd6 {+44.83/6 0.8} Be3
{-1000.13/26 1.3} 109. Nc4 {+27.70/5 0.9} Bh6 {-1000.14/28 2.2} 110. Nd6
{+41.89/5 0.7} Kf4 {-1000.16/35 2.0} 111. Nf5 {+16.03/6 0.7} Bf8
{-1000.17/42 0.7} 112. Ne7 {+22.93/6 0.6} Bh6 {-1000.18/40 1.5} 113. Nc6
{+31.52/6 0.8} Kg5 {-1000.19/34 1.2} 114. Nd4 {+20.10/6 0.8} Kg6
{-1000.17/36 0.6} 115. Nc6 {+13.70/6 0.7} Kg5 {-1000.16/35 0.9} 116. f7
{+17.32/6 0.7} Kg6 {-1000.14/42 1.3} 117. Ke7 {+16.08/6 0.6} Be3
{-1000.13/32 1.0} 118. e6 {+39.78/5 0.9} Bc5+ {-1000.12/38 0.7} 119. Kd7
{+15.81/6 0.8} Kf6 {-1000.14/49 1.5} 120. Nd4 {+7.38/7 0.7} Ba3
{-1000.13/50 2.4} 121. Ke8 {+9.20/8 11} Bd6 {-1000.14/44 4} 122. f8=Q+
{+20.35/8 12} Bxf8 {-1000.13/49 4} 123. Kxf8 {+18.03/9 6} Ke5
{-1000.11/59 0.1} 124. Ke7 {+34.22/8 22} Kd5 {-1000.10/53 4} 125. Kf6
{+47.60/7 10} Kd6 {-1000.10/47 4} 126. Nf5+ {+46.88/7 9} Kc6
{-1000.09/50 4} 127. Kf7 {+55.76/7 8} Kc5 {-1000.09/49 4} 128. e7
{+66.30/7 8} Kb4 {-1000.07/58 0.1} 129. Kg8 {+78.42/6 7} Kc3
{-1000.08/48 4} 130. Nh4 {+73.12/6 6} Kd4 {-1000.09/44 4} 131. Kf7
{+70.39/6 6} Kc3 {-1000.08/52 0.1} 132. Nf3 {+77.00/6 5} Kc4
{-1000.08/49 4} 133. Kg6 {+75.95/6 5} Kc5 {-1000.08/47 5} 134. Kf7
{+56.30/6 5} Kd5 {-1000.08/51 4} 135. e8=R {+91.83/6 4} Kc5 {-1000.11/45 4}
136. Rg8 {+83.00/6 4} Kc4 {-1000.12/46 4} 137. Kg7 {+73.16/6 4} Kb4
{-1000.12/48 4} 138. Re8 {+62.09/6 3} Kb3 {-1000.11/49 0.8} 139. Kh8
{+55.16/6 3} Kc4 {-1000.12/49 4} 140. Re1 {+42.46/6 2.9} Kb4
{-1000.12/52 4} 141. Ne5 {+44.28/6 2.6} Kc5 {-1000.13/51 4} 142. Kg8
{+42.33/6 2.4} Kd5 {-1000.12/53 4} 143. Ra1 {+41.26/6 2.3} Kxe5
{-1000.19/43 2.7} 144. Ra2 {+69.56/6 2.1} Ke4 {-1000.21/55 4} 145. Kg7
{+50.07/6 1.9} Kd3 {-1000.17/59 4} 146. Ra5 {+41.53/6 1.8} Kc4
{-1000.14/58 5} 147. Rf5 {+38.61/6 1.8} Kd3 {-1000.15/60 4} 148. Rf8
{+38.09/6 1.8} Kd4 {-1000.15/61 4} 149. Kf7 {+38.42/6 1.8} Kd3
{-1000.14/67 9} 150. Ra8 {+32.86/6 1.8} Kc3 {-1000.14/65 4} 151. Ke8
{+29.87/6 1.8} Kd4 {-1000.15/61 4} 152. Kd8 {+31.04/6 1.8} Ke4
{-1000.15/62 4} 153. Kc7 {+27.75/6 1.8} Kd4 {-1000.14/72 8} 154. Kb6
{+27.82/6 1.8} Ke5 {-1000.14/64 4} 155. Ka5 {+25.54/6 1.8} Ke4
{-1000.15/56 4} 156. Kb4 {+23.93/6 1.8} Ke3 {-1000.13/60 4} 157. Kc3
{+23.99/6 1.8} Kf4 {-1000.12/69 3} 158. Kc2 {+21.95/6 1.8} Ke5
{-1000.13/52 9} 159. Ra1 {+20.43/6 1.8} Kd5 {-1000.13/62 7} 160. Rh1
{+19.60/6 1.8} Ke5 {-1000.14/61 12} 161. Kb3 {+18.04/6 11} Ke4
{-1000.13/64 5} 162. Ka4 {+17.68/6 10} Kf5 {-1000.14/53 5} 163. Ka5
{+16.00/6 9} Ke4 {-1000.14/61 4} 164. Ka6 {+16.45/6 9} Kf3 {-1000.14/62 4}
165. Kb7 {+15.44/6 8} Kg2 {-1000.14/66 4} 166. Rh6 {+14.48/7 8} Kf3
{-1000.14/56 4} 167. Kb8 {+14.37/7 7} Ke4 {-1000.15/54 4} 168. Rh7
{+14.39/6 7} Kf5 {-1000.15/57 4} 169. Re7 {+13.46/6 6} Kf6 {-1000.15/65 4}
170. Re1 {+13.15/7 6} Kf5 {-1000.14/68 0.2} 171. Ka7 {+12.86/6 6} Kf4
{-1000.13/75 4} 172. Re8 {+12.59/6 5} Kg5 {-1000.13/70 4} 173. Rb8
{+12.18/6 5} Kf5 {-1000.14/63 4} 174. Ka6 {+12.96/6 5} Kg5 {-1000.14/59 4}
175. Rb2 {+12.88/6 4} Kf4 {-1000.15/53 4} 176. Kb5 {+12.57/6 4} Ke5
{-1000.13/59 4} 177. Kc4 {+13.03/6 4} Kf6 {-1000.12/65 0.2} 178. Kd5
{+14.74/6 4} Kf5 {-1000.11/72 0.1} 179. Rb1 {+13.55/6 3} Kf4
{-1000.11/62 4} 180. Kd4 {+10.13/7 3} Kf5 {-1000.11/62 4} 181. Re1
{+7.10/7 2.7} Kg4 {-1000.10/67 1.1} 182. Rf1 {+8.14/7 3} Kg5
{-1000.09/71 0.7} 183. Ke4 {+6.95/7 2.7} Kg6 {-1000.08/70 1.8} 184. Ke5
{+7.15/8 2.8} Kg7 {-1000.07/74 1.6} 185. Kf5 {+14.45/7 2.6} Kf7
{-1000.06/74 4} 186. Re1 {+128.00/4 2.0} Kg7 {-1000.05/107 2.2} 187. Re7+
{+128.00/3 4} Kf8 {-1000.04/245 0.1} 188. Kf6 {+128.00/4 2.5} Kg8
{-1000.03/245 0.1} 189. Re4 {+128.00/4 3} Kf8 {-1000.02/245 0.1} 190. Rd4
{+128.00/4 2.8} Kg8 {-1000.02/245 0.1} 191. Rh4 {+128.00/3 2.8} Kf8
{-1000.01/245 0.1} 192. Rh8# {+128.00/2 1.2}
{Xboard adjudication: Checkmate} 1-0

[Dann Corbit]

So happy to see the Olithink author back to talkchess! :
LC0 is about the same strength as SF on high end hardware, but on an average machine with a good GPU, LC0 kicks butt.
LC0 is especially good at analyzing quiet positions.

[Dann Corbit]

See the TCEC tournament for an example of LC0 verses SF on high end CPUs and GPUs