Super bullet chess: LC0 v0.21.0 network 41619 vs newest Stockfish dev 200319

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Hai
Posts: 454
Joined: Sun Aug 04, 2013 11:19 am

Super bullet chess: LC0 v0.21.0 network 41619 vs newest Stockfish dev 200319

Post by Hai » Wed Mar 20, 2019 11:46 pm

Super bullet chess: LC0 v0.21.0 network 41619 vs newest Stockfish dev 200319

The 20 possible opening moves
40 games
Ponder on
LC0 with RTX 2080 Ti 32 kn/s
Stockfish with 4 cores @ 4.2 ghz
Time control: 1 min per game + 1 second per move.

Result:
LC0 26 points
Stockfish 14 points
+14 =24 -2
Winning percentage = 65.00%

= LC0 is 108 elo stronger than Stockfish.

http://www.mediafire.com/file/fa51ix07k ... 9.pgn/file




brianr
Posts: 356
Joined: Thu Mar 09, 2006 2:01 pm

Re: Super bullet chess: LC0 v0.21.0 network 41619 vs newest Stockfish dev 200319

Post by brianr » Wed Mar 20, 2019 11:57 pm

With Ponder On and SF getting 4 cores, and Leela typically using 2 cores, just wondering if this was a 6 core system?
Thanks.

ernest
Posts: 1855
Joined: Wed Mar 08, 2006 7:30 pm

Re: Super bullet chess: LC0 v0.21.0 network 41619 vs newest Stockfish dev 200319

Post by ernest » Thu Mar 21, 2019 12:35 am

brianr wrote:
Wed Mar 20, 2019 11:57 pm
With Ponder On and SF getting 4 cores, and Leela typically using 2 cores, just wondering if this was a 6 core system?
Thanks.
Right !
And besides, Lc0-2080Ti vs. SF-4cores is quite a bit lopsided...

mwyoung
Posts: 1636
Joined: Wed May 12, 2010 8:00 pm

Re: Super bullet chess: LC0 v0.21.0 network 41619 vs newest Stockfish dev 200319

Post by mwyoung » Thu Mar 21, 2019 1:09 am

ernest wrote:
Thu Mar 21, 2019 12:35 am
brianr wrote:
Wed Mar 20, 2019 11:57 pm
With Ponder On and SF getting 4 cores, and Leela typically using 2 cores, just wondering if this was a 6 core system?
Thanks.
Right !
And besides, Lc0-2080Ti vs. SF-4cores is quite a bit lopsided...
ernest wrote:
Thu Mar 21, 2019 12:35 am
brianr wrote:
Wed Mar 20, 2019 11:57 pm
With Ponder On and SF getting 4 cores, and Leela typically using 2 cores, just wondering if this was a 6 core system?
Thanks.
Right !
And besides, Lc0-2080Ti vs. SF-4cores is quite a bit lopsided...
I would not think in terms of lopsidedness. The data is good
regardless of the hardware difference. I use 16 cores vs the same gpu. Is this an advantage for one program, or the other, or just right. Honestly I don't have proof one way or the other. It was my best guess when designing my testing system. They are such different anamials...

If you use price of hardware as a standard. Clearly Lc0 has the hardware advantage. Even on my testing system.
Professing themselves to be wise, they became fools,
Take on me. foes 0

ernest
Posts: 1855
Joined: Wed Mar 08, 2006 7:30 pm

Re: Super bullet chess: LC0 v0.21.0 network 41619 vs newest Stockfish dev 200319

Post by ernest » Thu Mar 21, 2019 3:11 pm

mwyoung wrote:
Thu Mar 21, 2019 1:09 am
I would not think in terms of lopsidedness. The data is good
regardless of the hardware difference.
I agree !
However, to measure progress of Lc0, the "usual" way is to keep the similarity with the AlphaZero tests, where SF knodes/s were about x1000 the AlphaZero knodes/s (for Lc0, this corresponds to a LeelaRatio close to 1).
It is not easy to assess by how much SF improves with more cores vs. Lc0, because it also depends on time control, openings, ...

Finally, note that Hai has not yet responded on the number of cores of his testing machine (hopefully at least 6 !)

mwyoung
Posts: 1636
Joined: Wed May 12, 2010 8:00 pm

Re: Super bullet chess: LC0 v0.21.0 network 41619 vs newest Stockfish dev 200319

Post by mwyoung » Thu Mar 21, 2019 3:26 pm

ernest wrote:
Thu Mar 21, 2019 3:11 pm
mwyoung wrote:
Thu Mar 21, 2019 1:09 am
I would not think in terms of lopsidedness. The data is good
regardless of the hardware difference.
I agree !
However, to measure progress of Lc0, the "usual" way is to keep the similarity with the AlphaZero tests, where SF knodes/s were about x1000 the AlphaZero knodes/s (for Lc0, this corresponds to a LeelaRatio close to 1).
It is not easy to assess by how much SF improves with more cores vs. Lc0, because it also depends on time control, openings, ...

Finally, note that Hai has not yet responded on the number of cores of his testing machine (hopefully at least 6 !)
I disagree. There are many ways to measure progress. The ×1000 standard is a poor way to measure lc0 progress.imo. Because CPU based engines became less efficient as you add cpus and split the workload to more threads. This speed loss to the CPU engine is not reflected in the nps displayed by the CPU engine. It is like using a tachometer to tell how fast you are going. Without knowing what gear you are using.
Professing themselves to be wise, they became fools,
Take on me. foes 0

Hai
Posts: 454
Joined: Sun Aug 04, 2013 11:19 am

Re: Super bullet chess: LC0 v0.21.0 network 41619 vs newest Stockfish dev 200319

Post by Hai » Sat Mar 23, 2019 4:27 pm

ernest wrote:
Thu Mar 21, 2019 3:11 pm
Finally, note that Hai has not yet responded on the number of cores of his testing machine (hopefully at least 6 !)
My machine has only a 1/2 cpu core :lol: :lol: :lol:
I thought your core question was a joke.
Of course 6.
You can use also the search function here :wink:

ernest
Posts: 1855
Joined: Wed Mar 08, 2006 7:30 pm

Re: Super bullet chess: LC0 v0.21.0 network 41619 vs newest Stockfish dev 200319

Post by ernest » Sat Mar 23, 2019 7:40 pm

Hai wrote:
Sat Mar 23, 2019 4:27 pm
Of course 6.
Of course ! Thanks !

Post Reply