End of Era is there: SF is finally beaten!

[brianr]

http://www.computerchess.org.uk/ccrl/40 ... 089_64-bit

The CPU version is improving rapidly too.

Just looking to confirm that this is Leela on only one CPU (the BLAS backend I would think), not running with a GPU, correct?
Thanks.

[Graham Banks]

http://www.computerchess.org.uk/ccrl/40 ... 089_64-bit

The CPU version is improving rapidly too.

Just looking to confirm that this is Leela on only one CPU (the BLAS backend I would think), not running with a GPU, correct?
Thanks.

Correct. Gabor ran the gauntlet.

[chrisw]

There is no any "spectacular" and there is no any "plan" on the side of Leela.
In reality the position after 20.-c4 and the good continuation is in the 32930net of Leela.
To prove this I used the weak GTX 1060 to find the 21.Bg6 move.
From net30100 to net32000 Leela want to move 21.Be4, 21.Bxc4, 21.0-0.
But using net32500 Leela finds 21.Bg6 immediately!
So anywhere between net32000 and net32500 Leela has analysed yet this position during self play
and 21.Bg6 is in the NN of Leela.
Obviously Stockfish also would be much more stronger if it could use a good book instead of analysing every each position.

You have ZERO idea about how Neural Net engines work.

Does anybody? Can you make a non-tautological explanation in, say 300 words, “how neural net engines work” that we can cut out and put in a frame on the wall?

[Uri Blass]

...

There is no any "spectacular" and there is no any "plan" on the side of Leela.
In reality the position after 19.-c4 and the good continuation is in the 32930net of Leela.
To prove this I used the weak GTX 1060 to find the 20.Bg6 move.
From net30100 to net32000 Leela want to move 20.Be4, 20.Bxc4, 20.0-0.
But using net32500 Leela finds 20.Bg6 immediately!
So anywhere between net32000 and net32500 Leela has analysed yet this position during self play
and 20.Bg6 is in the NN of Leela.
Obviously Stockfish also would be much more stronger if it could use a good book instead of analysing every each position.


I am sorry but I miss-wrote the number of moves.
The text with good move numbers is above.

I agree 100 %. It has to calculate anew the opening every single game. LCZ has already done that before the games starts. Of course these have been the rules for the TCEC for years and it is impressive that a new engine can do what it is doing.

I agree 0%. The probability that LCZ between net32000 and net32500 has seen that specific position is much smaller than 0.000001%. Between net32000 and net32500, LCZ may somehow have learned to assess that position correctly, but that is no reason to believe that LCZ has seen exactly that position.

From net32000 and net32500, LCZ has learned new patterns and one or more of those patterns is what has allowed it to find the right move in that position (and in many other positions). This has nothing to do with having a "good book".

I partly agree.
Having a good book can help also in positions that you never saw in the past.
If you have a book with a lot of chess positions and evaluations of them then you can learn from the book to evaluate better also positions that are not in the book.

The idea that I have is to divide the positions to classes(let decide about classes based on material on the board and pawn structure and maybe it is possible to come with a better idea of division of the positions to classes).

If positions in the same class happen enough times in the database then you can calculate a bias for the class based on difference between static evaluation and evaluation after deep search for all positions in the same class and add the bias to the evaluation function when the program get a position in some class to evaluate.

Note that I believe this idea can work only with many positions otherwise a big part of the positions are in classes that do not happen enough times to get conclusions.

[MikeB]

There is no any "spectacular" and there is no any "plan" on the side of Leela.
In reality the position after 20.-c4 and the good continuation is in the 32930net of Leela.
To prove this I used the weak GTX 1060 to find the 21.Bg6 move.
From net30100 to net32000 Leela want to move 21.Be4, 21.Bxc4, 21.0-0.
But using net32500 Leela finds 21.Bg6 immediately!
So anywhere between net32000 and net32500 Leela has analysed yet this position during self play
and 21.Bg6 is in the NN of Leela.
Obviously Stockfish also would be much more stronger if it could use a good book instead of analysing every each position.

You have ZERO idea about how Neural Net engines work.

Does anybody? Can you make a non-tautological explanation in, say 300 words, “how neural net engines work” that we can cut out and put in a frame on the wall?

I certainly cannot - but for those who are willing to read more than 300 words , this is straight from the horse's mouth:

https://www.dropbox.com/s/rp2foutpse7l7 ... 5.pdf?dl=1 It is about succinct as it could be considering the subject matter. Pages 2 thru 6 and and Pages 9 thru 15 are the must reads - others pages can be skipped.

[jp]

I certainly cannot - but for those who are willing to read more than 300 words , this is straight from the horse's mouth:
It is about succinct as it could be considering the subject matter. Pages 2 thru 6 and and Pages 9 thru 15 are the must reads - others pages can be skipped.

I don't think that's what Chris meant. That's just saying what they did (and saying it very badly -- that's the v.1 before the reviewers forced them to make changes to make it slightly less bad). What they did is different from philosophical ideas of "how" that often make the mistake of anthropomorphizing computer code ("learn", "plan", "Understand", etc.).

[M ANSARI]

Here is an example of where SF and other AB engines are still superior. Lc0 had totally outplayed SF and tenaciously held an advantage and finally reached a totally winning simplified endgame.

[d]8/2b1k3/8/2pP4/2P1p1N1/8/5PK1/8 w - - 0 72


Here Lc0 played f3! forcing the exchange of the pawn and reaching a winning position. SF and Blue fish and Komodo all had mate scores or something around +153 which is almost the same. Lc0 also had an evaluation of around +9 ... so it seemed it was game over! However, Lc0 just couldn't see the win !!! And ended up drawing. To be fair, I tried the position out against SF and I too was not able to win the position, so it certainly is not an easy win as the Bishop can coordinate really well and you need some waiting moves to slowly let your King invade and close out blocking squares. However that should be trivial for an engine, especially if it has 6 piece EGTB's and there are only 7 pieces remaining. One more thing that needs to be looked at with Lc0. Here is the full game

[pgn] 1. d4 Nf6 2. c4 e6 3. Nf3 b6 4. g3 Ba6 5. b3 d5 6. Bg2 dxc4 7. Ne5 Bb4+ 8. Kf1 Bd6 9. Nxc4 Nd5 10. Bb2 Nc6 11. e4 Nde7 12. Nbd2 O-O 13. Kg1 e5 14. d5 Nd4 15. h4 c5 16. Nf3 Bxc4 17. bxc4 Nc8 18. Bh3 Re8 19. Kg2 Bf8 20. Bxc8 Qxc8 21. a4 f6 22. h5 Rb8 23. Nh4 Re7 24. Ng6 Rf7 25. a5 bxa5 26. Bc3 Qe8 27. Rxa5 f5 28. exf5 Rxf5 29. Nh4 Rf7 30. Re1 Qd7 31. Ra2 Bd6 32. Re4 Rbf8 33. Qg4 Qxg4 34. Rxg4 Rf6 35. Be1 R8f7 36. Re4 Kf8 37. Kh3 Kg8 38. Kg4 g6 39. hxg6 hxg6 40. Ra1 Kh7 41. Ra6 Kg7 42. Ra5 Nf5 43. Re2 Kf8 44. Ra6 Nh6+ 45. Kh3 g5 46. Ng2 g4+ 47. Kh2 Nf5 48. Nh4 Nd4 49. Re4 Nc2 50. Kg1 Nb4 51. Ra3 Nc2 52. Ra2 Nb4 53. Rae2 Nd3 54. Rd2 Nxe1 55. Rxe1 a5 56. Ra2 a4 57. Ng2 Rf3 58. Ne3 Ra7 59. Kg2 Ra6 60. Rea1 e4 61. Rxa4 Rxa4 62. Rxa4 Bxg3 63. Nxg4 Bd6 64. Ra6 Ke7 65. Ra7+ Kf8 66. Rh7 Rf4 67. Ne3 Ke8 68. Rh6 Kd7 69. Rg6 Bc7 70. Rg4 Rxg4+ 71. Nxg4 Ke7 72. f3 exf3+ 73. Kxf3 Ba5 74. Nf2 Bc3 75. Ne4 Bd4 76. Kf4 Bg1 77. Ke5 Bd4+ 78. Kf5 Kd7 79. Kg5 Kd8 80. Ng3 Ba1 81. Kg4 Kd7 82. Kf4 Bb2 83. Ke3 Bd4+ 84. Kd3 Kc7 85. Ne4 Kd8 86. Ng5 Kd7 87. Kd2 Kc8 88. Ne6 Bg1 89. Ng5 Bd4 90. Kc2 Be3 91. Ne4 Bd4 92. Nd6+ Kd7 93. Nb5 Bf2 94. Kb3 Kc8 95. Nc3 Kc7 96. Kc2 Bg1 97. Ne4 Be3 98. Kc3 Bd4+ 99. Kd3 Kc8 100. Ng5 Kd7 101. Kc2 Kc8 102. Ne6 Bg1 103. Kc3 Bf2 104. Ng5 Be3 105. Ne4 Kc7 106. Kc2 Kc8 107. Nd2 Bd4 108. Kd3 Kd7 109. Nf3 Bf6 110. Ke4 Kd6 111. Kf5 Bg7 112. Kg6 Bd4 113. Kf7 Ba1 114. Ke8 Bf6 115. Nd2 Ba1 116. Kd8 Bb2 117. Kc8 Ba1 118. Kb7 Bd4 119. Nb1 Ba1 120. Na3 Bb2 121. Nb5+ Kd7 122. Kb6 Bd4 123. d6 Bg1 124. Kb7 Bf2 125. Kb6 Bg1 126. Ka6 Bf2 127. Kb7 Be1 128. Kb6 Bf2 129. Ka7 Be1 130. Kb6 Bb4 131. Kb7 Bd2 132. Kb6 Bb4 133. Kb7 Bd2 134. Ka6 Bc1 135. Kb6 Be3 136. Ka7 Bc1 137. Kb6 Be3 138. Ka6 Bd4 139. Kb7 Bg1 140. Ka6 Kc6 141. Ka7 Bf2 142. Kb8 Bg3 143. Kc8 Bxd6 1/2-1/2[/pgn]

[corres]

There is no any "spectacular" and there is no any "plan" on the side of Leela.
In reality the position after 19.-c4 and the good continuation is in the 32930net of Leela.
To prove this I used the weak GTX 1060 to find the 20.Bg6 move.
From net30100 to net32000 Leela want to move 20.Be4, 20.Bxc4, 20.0-0.
But using net32500 Leela finds 20.Bg6 immediately!
So anywhere between net32000 and net32500 Leela has analysed yet this position during self play
and 20.Bg6 is in the NN of Leela.
Obviously Stockfish also would be much more stronger if it could use a good book instead of analysing every each position.

You have ZERO idea about how Neural Net engines work.

Thanks.
But if during self play LC0 never move in this position Bg6, from what she knows Bg6 is a good move?
For the exact answer we should look through the whole self play game between net32000 and net32500.
I am very curious to know the opinion of the main developers of LC0.
Registering the similarity between pattern is a very complex problem even in an NN system too.
If the developers of LC0 solved this problem I congratulate to they.

(continue)
There may be a place in the structure of Leela where the global chess problems can be manipulated: The policy head of Leela.
Policy head help the search to find possible good moves in certain situations. These "certain situations" are
the same as it is in AB engines: to seek for getting bishop pair/open lines to rocks/free pawns/etc., and to avoid double pawn/checks/discovered checks/opposite bishops/trapped figures/etc. In concrete only the developers of Leela know what are in the policy head of Leela. In generally the content of policy head can be changed automatically during the course of learning ("zero approaching") and manually too.
If in the policy head there is information about how Leela handles such a position being after the 19th move of Black and Leela uses policy head during normal play also she runs a sequence of search to decide the possibility of win/draw/loss of 20.Bg6 and if the possibility of win is rather high she will move 20.Bg6.
So in the case of 20.Bg6 there is no in the value head - that is Leela meets never the position existed after
19.-,c4 - Leela also can find the move 20.Bg6.
It is pity but Chris has right there is no any consequent and easy to understand write down about how Leela works. The developers of Leela handle in closed mode a lot of detailed information about Leela.
Although Leela is an opened project - supposedly...

[Alexander Schmidt]

You have ZERO idea about how Neural Net engines work.

Thanks.
But if during self play LC0 never move in this position Bg6, from what she knows Bg6 is a good move?

Leela learns patterns in self play games. In new positions she tries to find this patterns. That's why she is running faster with gpu which are optimized for calculating images. Leela don't save one single position in it's NN. She found something in the position which made her believe is winning. We will likely never know what it was.

[corres]

Leela learns patterns in self play games. In new positions she tries to find this patterns. That's why she is running faster with gpu which are optimized for calculating images. Leela don't save one single position in it's NN. She found something in the position which made her believe is winning. We will likely never know what it was.

NN of Leela does not consist of "policy head" only...