Super bullet chess: LC0 v0.21.0 network 41619 vs newest Stockfish dev 200319

[Hai]

Super bullet chess: LC0 v0.21.0 network 41619 vs newest Stockfish dev 200319

The 20 possible opening moves
40 games
Ponder on
LC0 with RTX 2080 Ti 32 kn/s
Stockfish with 4 cores @ 4.2 ghz
Time control: 1 min per game + 1 second per move.

Result:
LC0 26 points
Stockfish 14 points
+14 =24 -2
Winning percentage = 65.00%

= LC0 is 108 elo stronger than Stockfish.

http://www.mediafire.com/file/fa51ix07k ... 9.pgn/file

[pgn][Event "\[193\] LC0 (ID 41619) vs newest Stockfish"] [Site "Lokaler Computer"] [Date "2019.03.20"] [Round "3"] [White "Lc0 v0.21.0-rc2"] [Black "Stockfish 200319 64 POPCNT"] [Result "1-0"] [WhiteElo "3570"] [BlackElo "3500"] [PlyCount "331"] [EventDate "2019.??.??"] 1. a4 e5 2. e4 Nf6 3. Nc3 Bb4 4. Nf3 d6 5. Bc4 Bxc3 6. dxc3 O-O 7. Qe2 Nbd7 8. Bg5 h6 9. Bh4 Re8 10. Nd2 c6 11. f3 Qc7 12. O-O d5 13. Bd3 Nh5 14. Qe3 Ndf6 15. Rfe1 a5 16. Bf2 Nf4 17. Bf1 dxe4 18. Nxe4 N6d5 19. Qc5 Rd8 20. Rad1 b6 21. Qa3 Bf5 22. Qb3 Rab8 23. c4 Nb4 24. Rxd8+ Rxd8 25. c5 bxc5 26. Qc4 Bxe4 27. Qxe4 Rd5 28. h4 Qd6 29. b3 Ne6 30. Bg3 Rd4 31. Qxe5 Qxe5 32. Bxe5 Rd2 33. Re2 Rd1 34. Bc3 Rc1 35. h5 Kh8 36. f4 Kg8 37. f5 Nc7 38. Bd2 Rxc2 39. Bf4 Ncd5 40. Rxc2 Nxc2 41. Bd6 Nd4 42. Bc4 Nxf5 43. Bxc5 Nfe3 44. Be2 g6 45. Kf2 Nf5 46. hxg6 fxg6 47. Bc4 Kf7 48. b4 axb4 49. a5 Nfe7 50. Bxb4 Nc8 51. a6 Kf6 52. Bc5 g5 53. a7 Nxa7 54. Bxa7 Ke5 55. Be2 Kf4 56. Bb8+ Kf5 57. Bd3+ Ke6 58. Kf3 Kf6 59. Ke4 Ke6 60. Ba6 Kd7 61. Be5 Ke6 62. Bc8+ Ke7 63. Bg4 Nb6 64. Bd4 Nc4 65. Bc5+ Kf6 66. Bh5 Nd2+ 67. Kd3 Nb3 68. Bb6 c5 69. Bd1 Nd4 70. Ke4 Ne6 71. Ba5 Kg6 72. Bc2 Nf4 73. g3 Nh5 74. Be1 Kf7 75. Kf5 Ng7+ 76. Ke5 c4 77. Be4 c3 78. Bxc3 Nh5 79. Be1 Ng7 80. Bd5+ Kg6 81. Bc6 Kf7 82. Be4 Ne8 83. Bd3 Ng7 84. Bf2 g4 85. Bc2 Ne8 86. Be1 Nf6 87. Kf5 Ne8 88. Kxg4 Kf6 89. Bc3+ Ke6 90. Kh5 Nc7 91. Kxh6 Nb5 92. Bg7 Nd6 93. g4 Nf7+ 94. Kh5 Kd7 95. Bb3 Nd6 96. Kg6 Kc6 97. Bf8 Ne4 98. Kf5 Nd2 99. Bf7 Nf3 100. Bg8 Nd4+ 101. Kf6 Ne2 102. Bh6 Kc5 103. Bh7 Ng3 104. Ke5 Kc4 105. Bg8+ Kd3 106. Bf7 Ke2 107. Bd5 Nf1 108. g5 Nh2 109. Kf5 Nf1 110. g6 Ng3+ 111. Ke6 Nh5 112. Bc6 Kf2 113. Bb7 Ke2 114. Bc8 Kf3 115. Kf5 Ng3+ 116. Ke5 Nh5 117. Bb7+ Ke2 118. Ba8 Kf2 119. Kf5 Ke2 120. Kg5 Ng3 121. Bf8 Kd3 122. Bb7 Kd2 123. g7 Nf1 124. Ba8 Kc3 125. Kf6 Kd3 126. Ke7 Ne3 127. g8=Q Ke2 128. Bg7 Nf5+ 129. Kf6 Nxg7 130. Qxg7 Kd2 131. Qh7 Kc3 132. Qh6 Kc2 133. Qh5 Kd2 134. Qh4 Kc3 135. Qh8 Kd2 136. Qb8 Kc2 137. Ke5 Kc3 138. Kd5 Kd3 139. Qc8 Ke3 140. Qf8 Kd3 141. Qe8 Kc3 142. Qb8 Kd3 143. Bb7 Kc3 144. Bc8 Kd2 145. Qa8 Ke3 146. Qb7 Kf4 147. Qc7+ Ke3 148. Qd7 Kd3 149. Qe7 Kc3 150. Qb7 Kd3 151. Qc7 Ke3 152. Kd6 Kd2 153. Qd7 Kd3 154. Qe8 Kd2 155. Kd5 Kd3 156. Ba6+ Kc3 157. Bb7 Kb3 158. Bc8 Kc3 159. Kc5 Kb2 160. Kb6 Kb3 161. Kc5 Ka2 162. Kb4 Kb2 163. Bd7 Kc2 164. Qe2+ Kb1 165. Kb3 Ka1 166. Qa2# 1-0[/pgn]

[pgn][Event "\[193\] LC0 (ID 41619) vs newest Stockfish"] [Site "Lokaler Computer"] [Date "2019.03.20"] [Round "23"] [White "Lc0 v0.21.0-rc2"] [Black "Stockfish 200319 64 POPCNT"] [Result "1-0"] [ECO "A03"] [WhiteElo "3570"] [BlackElo "3500"] [PlyCount "315"] [EventDate "2019.??.??"] 1. f4 d5 2. Nf3 c5 3. e3 g6 4. Bb5+ Bd7 5. Bxd7+ Qxd7 6. Ne5 Qe6 7. Qe2 Nd7 8. Nxd7 Qxd7 9. d3 e6 10. Nd2 Ne7 11. e4 Bg7 12. Nf3 O-O 13. e5 a5 14. a4 Rfc8 15. c3 h5 16. h3 Nf5 17. Qf2 Bf8 18. g4 Ng7 19. c4 Rc7 20. Qe2 Be7 21. Bd2 Qc6 22. b3 b6 23. O-O-O Rd8 24. Bc3 Qa8 25. Kc2 Rcd7 26. Rd2 Qc6 27. Rc1 Kf8 28. Kb1 Ra8 29. Bb2 Rad8 30. Rg1 Kg8 31. Rc2 Kh7 32. Rf1 dxc4 33. dxc4 Rd3 34. Rc3 Rxc3 35. Bxc3 Rh8 36. Nd2 Kg8 37. Ne4 Kf8 38. Kb2 hxg4 39. hxg4 Rh4 40. Nf6 Rh3 41. Ka3 Ne8 42. Bb2 Rh1 43. Rf3 Rh4 44. Qf2 Rh6 45. Qg2 Nc7 46. f5 g5 47. Qe2 exf5 48. gxf5 Rxf6 49. exf6 Bxf6 50. Rd3 Ne8 51. Rd1 Ng7 52. Rd5 Ne8 53. Qh2 Bxb2+ 54. Kxb2 Kg7 55. Qe5+ Qf6 56. Kc2 Qxe5 57. Rxe5 Nf6 58. Kd3 Ng4 59. Re7 Kf6 60. Rb7 Kxf5 61. Rxf7+ Ke6 62. Rb7 Ne5+ 63. Ke4 Nd7 64. Rc7 Kd6 65. Rc8 Nf6+ 66. Kf5 Nh5 67. Rd8+ Kc7 68. Rd3 Nf4 69. Re3 Kd7 70. Kf6 Kd6 71. Re1 Kc7 72. Kxg5 Nd3 73. Rh1 Kc8 74. Kf5 Kb8 75. Rh7 Nc1 76. Rh3 Kc7 77. Ke5 Na2 78. Rh7+ Kc6 79. Rh6+ Kc7 80. Ke6 Nb4 81. Rh2 Nd3 82. Rh7+ Kc6 83. Ke7 Kc7 84. Kf8+ Kd6 85. Ke8 Nc1 86. Kd8 Nxb3 87. Kc8 Ke6 88. Kc7 Nd2 89. Kxb6 Nxc4+ 90. Kxc5 Ne3 91. Kb5 Kd6 92. Kxa5 Kc6 93. Kb4 Nc2+ 94. Kc3 Ne3 95. Rh5 Kd7 96. Kb4 Nc2+ 97. Kb5 Nd4+ 98. Kb6 Ne2 99. a5 Nf4 100. Rc5 Ne6 101. Rc1 Nf4 102. Kb7 Ke6 103. a6 Nd3 104. Rc3 Nf2 105. a7 Ne4 106. a8=Q Nxc3 107. Qd8 Ne4 108. Qb8 Kd5 109. Qa8 Nc3 110. Qd8+ Ke4 111. Kc6 Ne2 112. Kc5 Ke3 113. Kc4 Nf4 114. Qe8+ Kf3 115. Kd4 Ne2+ 116. Ke5 Kf2 117. Ke4 Ng3+ 118. Kf4 Ne2+ 119. Kg4 Nd4 120. Qe5 Ne2 121. Qe7 Ng1 122. Kf4 Ne2+ 123. Kf5 Kf3 124. Qd8 Ke3 125. Ke5 Kf2 126. Ke4 Ng3+ 127. Kf4 Ne2+ 128. Kg4 Ke3 129. Qe8+ Kd3 130. Kf3 Nc3 131. Qd8+ Kc4 132. Ke3 Nd5+ 133. Ke4 Nc3+ 134. Ke5 Nb5 135. Qc8+ Kd3 136. Qc6 Nc3 137. Qd7+ Kc4 138. Qd2 Nb5 139. Qb2 Kc5 140. Qb3 Kc6 141. Qc4+ Kb6 142. Kd5 Nc7+ 143. Kd6 Nb5+ 144. Kd7 Na7 145. Qd4+ Kb5 146. Qxa7 Kc4 147. Qb7 Kc3 148. Qc8+ Kd3 149. Qd8 Kd4 150. Ke8+ Ke3 151. Kf7 Ke2 152. Ke6 Kf3 153. Kf5 Kf2 154. Kf4 Ke2 155. Qd6 Kf1 156. Kf3 Ke1 157. Qd8 Kf1 158. Qd1# 1-0[/pgn]

[brianr]

With Ponder On and SF getting 4 cores, and Leela typically using 2 cores, just wondering if this was a 6 core system?
Thanks.

[ernest]

With Ponder On and SF getting 4 cores, and Leela typically using 2 cores, just wondering if this was a 6 core system?
Thanks.

Right !
And besides, Lc0-2080Ti vs. SF-4cores is quite a bit lopsided...

[mwyoung]

With Ponder On and SF getting 4 cores, and Leela typically using 2 cores, just wondering if this was a 6 core system?
Thanks.

Right !
And besides, Lc0-2080Ti vs. SF-4cores is quite a bit lopsided...

With Ponder On and SF getting 4 cores, and Leela typically using 2 cores, just wondering if this was a 6 core system?
Thanks.

Right !
And besides, Lc0-2080Ti vs. SF-4cores is quite a bit lopsided...

I would not think in terms of lopsidedness. The data is good
regardless of the hardware difference. I use 16 cores vs the same gpu. Is this an advantage for one program, or the other, or just right. Honestly I don't have proof one way or the other. It was my best guess when designing my testing system. They are such different anamials...

If you use price of hardware as a standard. Clearly Lc0 has the hardware advantage. Even on my testing system.

[ernest]

I would not think in terms of lopsidedness. The data is good
regardless of the hardware difference.

I agree !
However, to measure progress of Lc0, the "usual" way is to keep the similarity with the AlphaZero tests, where SF knodes/s were about x1000 the AlphaZero knodes/s (for Lc0, this corresponds to a LeelaRatio close to 1).
It is not easy to assess by how much SF improves with more cores vs. Lc0, because it also depends on time control, openings, ...

Finally, note that Hai has not yet responded on the number of cores of his testing machine (hopefully at least 6 !)

[mwyoung]

I would not think in terms of lopsidedness. The data is good
regardless of the hardware difference.

I agree !
However, to measure progress of Lc0, the "usual" way is to keep the similarity with the AlphaZero tests, where SF knodes/s were about x1000 the AlphaZero knodes/s (for Lc0, this corresponds to a LeelaRatio close to 1).
It is not easy to assess by how much SF improves with more cores vs. Lc0, because it also depends on time control, openings, ...

Finally, note that Hai has not yet responded on the number of cores of his testing machine (hopefully at least 6 !)

I disagree. There are many ways to measure progress. The ×1000 standard is a poor way to measure lc0 progress.imo. Because CPU based engines became less efficient as you add cpus and split the workload to more threads. This speed loss to the CPU engine is not reflected in the nps displayed by the CPU engine. It is like using a tachometer to tell how fast you are going. Without knowing what gear you are using.

[Hai]

Finally, note that Hai has not yet responded on the number of cores of his testing machine (hopefully at least 6 !)

My machine has only a 1/2 cpu core
I thought your core question was a joke.
Of course 6.
You can use also the search function here

[ernest]

Of course 6.

Of course ! Thanks !