LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Hai
Posts: 446
Joined: Sun Aug 04, 2013 11:19 am

LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

Post by Hai » Sat Jun 15, 2019 6:18 pm

The two strongest LC0s:
LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

The 20 possible opening moves
40 games
Ponder on
LC0 40x256 with 1x RTX 2080 Ti
LC0 20x256 with 1x RTX 2080 Ti
Time control: 1 min per game + 10 seconds per move.

Result:
LC0 40x256 19.5 points
LC0 20x256 20.5 points
+1 =37 -2
Winning percentage = 48.75%

= LC0 40x256 is 9 elo weaker than LC0 20x256.
http://www.mediafire.com/file/eccp53cla ... 9.pgn/file

I'm very happy about this result because a month ago I have tested also the two best 40x256 and 20x256 LC0s and 40x256 was 75 elo weaker. So I avoided to tell that result, because it was not my intention to kill a new project before it starts. Sorry.

Take a look here if you missed some opinions from lkaufman:
viewtopic.php?f=6&t=70772

Congratulations to the 40x256 developers and supporters:)
I hope you can get more training games per day, because you have only 10% of what the main LC0 (20x256) is training per day!
http://157.230.189.191:8080

Hai
Posts: 446
Joined: Sun Aug 04, 2013 11:19 am

Re: LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

Post by Hai » Sat Jun 15, 2019 6:21 pm






User avatar
Laskos
Posts: 9118
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

Post by Laskos » Sat Jun 15, 2019 10:33 pm

Hai wrote:
Sat Jun 15, 2019 6:18 pm
The two strongest LC0s:
LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

The 20 possible opening moves
40 games
Ponder on
LC0 40x256 with 1x RTX 2080 Ti
LC0 20x256 with 1x RTX 2080 Ti
Time control: 1 min per game + 10 seconds per move.

Result:
LC0 40x256 19.5 points
LC0 20x256 20.5 points
+1 =37 -2
Winning percentage = 48.75%

= LC0 40x256 is 9 elo weaker than LC0 20x256.
http://www.mediafire.com/file/eccp53cla ... 9.pgn/file

I'm very happy about this result because a month ago I have tested also the two best 40x256 and 20x256 LC0s and 40x256 was 75 elo weaker. So I avoided to tell that result, because it was not my intention to kill a new project before it starts. Sorry.

Take a look here if you missed some opinions from lkaufman:
viewtopic.php?f=6&t=70772

Congratulations to the 40x256 developers and supporters:)
I hope you can get more training games per day, because you have only 10% of what the main LC0 (20x256) is training per day!
http://157.230.189.191:8080
Thanks, impressive!
What is this v0.40.2 engine? Where can one find it?

lkaufman
Posts: 3438
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

Post by lkaufman » Sun Jun 16, 2019 1:17 am

Hai wrote:
Sat Jun 15, 2019 6:18 pm
The two strongest LC0s:
LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

The 20 possible opening moves
40 games
Ponder on
LC0 40x256 with 1x RTX 2080 Ti
LC0 20x256 with 1x RTX 2080 Ti
Time control: 1 min per game + 10 seconds per move.

Result:
LC0 40x256 19.5 points
LC0 20x256 20.5 points
+1 =37 -2
Winning percentage = 48.75%

= LC0 40x256 is 9 elo weaker than LC0 20x256.
http://www.mediafire.com/file/eccp53cla ... 9.pgn/file

I'm very happy about this result because a month ago I have tested also the two best 40x256 and 20x256 LC0s and 40x256 was 75 elo weaker. So I avoided to tell that result, because it was not my intention to kill a new project before it starts. Sorry.

Take a look here if you missed some opinions from lkaufman:
viewtopic.php?f=6&t=70772

Congratulations to the 40x256 developers and supporters:)
I hope you can get more training games per day, because you have only 10% of what the main LC0 (20x256) is training per day!
http://157.230.189.191:8080
Glad to see it's a close competition now. One bit of advice: don't use time controls with small base to increment ratios like this 6 to 1, especially when testing Lc0. As you know, Lc0 sometimes takes ridiculous numbers of moves to win a won game. It is a total waste of testing time to be playing hundreds of moves in a resignable position at almost the same time limit as the first forty moves were played. Even for normal engines, the minimum ratio for efficient testing is 100 to 1 (i prefer 150 to 1 myself), but for Lc0 it is probably at least 300 to 1. You could easily double or triple the number of games per hour with no loss in quality. Maybe five minutes plus one second or ten minutes plus two seconds. Even half a second increment is way more than enough time to win these trivially won positions. I suppose if you are adjudicating aggressively then this isn't so much of an issue, but probably you are not.
Komodo rules!

Hai
Posts: 446
Joined: Sun Aug 04, 2013 11:19 am

Re: LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

Post by Hai » Sun Jun 16, 2019 10:18 am

Laskos wrote:
Sat Jun 15, 2019 10:33 pm
Hai wrote:
Sat Jun 15, 2019 6:18 pm
The two strongest LC0s:
LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

The 20 possible opening moves
40 games
Ponder on
LC0 40x256 with 1x RTX 2080 Ti
LC0 20x256 with 1x RTX 2080 Ti
Time control: 1 min per game + 10 seconds per move.

Result:
LC0 40x256 19.5 points
LC0 20x256 20.5 points
+1 =37 -2
Winning percentage = 48.75%

= LC0 40x256 is 9 elo weaker than LC0 20x256.
http://www.mediafire.com/file/eccp53cla ... 9.pgn/file

I'm very happy about this result because a month ago I have tested also the two best 40x256 and 20x256 LC0s and 40x256 was 75 elo weaker. So I avoided to tell that result, because it was not my intention to kill a new project before it starts. Sorry.

Take a look here if you missed some opinions from lkaufman:
viewtopic.php?f=6&t=70772

Congratulations to the 40x256 developers and supporters:)
I hope you can get more training games per day, because you have only 10% of what the main LC0 (20x256) is training per day!
http://157.230.189.191:8080
Thanks, impressive!
What is this v0.40.2 engine? Where can one find it?
After installing this: https://github.com/LeelaChessZero/lc0/releases
Delete that LC0.exe
and download here: http://157.230.189.191:8080 / Notice: Please update engine, that LC0.exe

You can ask questions "Joe MD" here: https://groups.google.com/forum/#!topic ... 5m-2JCdo-A

User avatar
Laskos
Posts: 9118
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

Post by Laskos » Sun Jun 16, 2019 10:28 am

Hai wrote:
Sun Jun 16, 2019 10:18 am
Laskos wrote:
Sat Jun 15, 2019 10:33 pm
Hai wrote:
Sat Jun 15, 2019 6:18 pm
The two strongest LC0s:
LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

The 20 possible opening moves
40 games
Ponder on
LC0 40x256 with 1x RTX 2080 Ti
LC0 20x256 with 1x RTX 2080 Ti
Time control: 1 min per game + 10 seconds per move.

Result:
LC0 40x256 19.5 points
LC0 20x256 20.5 points
+1 =37 -2
Winning percentage = 48.75%

= LC0 40x256 is 9 elo weaker than LC0 20x256.
http://www.mediafire.com/file/eccp53cla ... 9.pgn/file

I'm very happy about this result because a month ago I have tested also the two best 40x256 and 20x256 LC0s and 40x256 was 75 elo weaker. So I avoided to tell that result, because it was not my intention to kill a new project before it starts. Sorry.

Take a look here if you missed some opinions from lkaufman:
viewtopic.php?f=6&t=70772

Congratulations to the 40x256 developers and supporters:)
I hope you can get more training games per day, because you have only 10% of what the main LC0 (20x256) is training per day!
http://157.230.189.191:8080
Thanks, impressive!
What is this v0.40.2 engine? Where can one find it?
After installing this: https://github.com/LeelaChessZero/lc0/releases
Delete that LC0.exe
and download here: http://157.230.189.191:8080 / Notice: Please update engine, that LC0.exe

You can ask questions "Joe MD" here: https://groups.google.com/forum/#!topic ... 5m-2JCdo-A
Thanks, when back home, I will use your pointers. That 40b net can become a monster at long time controls and strong hardware (checked LTC on imo reliable test suites, improves like no other to long times per position).

User avatar
Guenther
Posts: 2871
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

Post by Guenther » Sun Jun 16, 2019 10:42 am

Hai wrote:
Sun Jun 16, 2019 10:18 am
Laskos wrote:
Sat Jun 15, 2019 10:33 pm
Hai wrote:
Sat Jun 15, 2019 6:18 pm
The two strongest LC0s:
LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

The 20 possible opening moves
40 games
Ponder on
LC0 40x256 with 1x RTX 2080 Ti
LC0 20x256 with 1x RTX 2080 Ti
Time control: 1 min per game + 10 seconds per move.

Result:
LC0 40x256 19.5 points
LC0 20x256 20.5 points
+1 =37 -2
Winning percentage = 48.75%

= LC0 40x256 is 9 elo weaker than LC0 20x256.
http://www.mediafire.com/file/eccp53cla ... 9.pgn/file

I'm very happy about this result because a month ago I have tested also the two best 40x256 and 20x256 LC0s and 40x256 was 75 elo weaker. So I avoided to tell that result, because it was not my intention to kill a new project before it starts. Sorry.

Take a look here if you missed some opinions from lkaufman:
viewtopic.php?f=6&t=70772

Congratulations to the 40x256 developers and supporters:)
I hope you can get more training games per day, because you have only 10% of what the main LC0 (20x256) is training per day!
http://157.230.189.191:8080
Thanks, impressive!
What is this v0.40.2 engine? Where can one find it?
After installing this: https://github.com/LeelaChessZero/lc0/releases
Delete that LC0.exe
and download here: http://157.230.189.191:8080 / Notice: Please update engine, that LC0.exe

You can ask questions "Joe MD" here: https://groups.google.com/forum/#!topic ... 5m-2JCdo-A
You seem not to understand that the confusing and bad/idiotic name 0.40.2 or 0.41.2 for the LC0.exe, or whatever, is just a gimmick name?
https://github.com/joeismad/lc0/commits/release/0.21 =>last 7 'commits'

Actually it just adjudicates draws earlier in training games, thus it is completely irrelevant for your games anyway...
Current foe list count : [93 - still rising]
http://rwbc-chess.de/chronology.htm

Hai
Posts: 446
Joined: Sun Aug 04, 2013 11:19 am

Re: LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

Post by Hai » Sun Jun 16, 2019 10:56 am

The graph is skyrocketing :twisted: :twisted: :twisted:
LC0 40x256
ID 118
graph.jpg
graph.jpg (97.42 KiB) Viewed 604 times
The development breaks through :wink: :wink: :wink:

Hai
Posts: 446
Joined: Sun Aug 04, 2013 11:19 am

Re: LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

Post by Hai » Mon Jul 01, 2019 1:53 am

lkaufman wrote:
Sun Jun 16, 2019 1:17 am
Hai wrote:
Sat Jun 15, 2019 6:18 pm
The two strongest LC0s:
LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

The 20 possible opening moves
40 games
Ponder on
LC0 40x256 with 1x RTX 2080 Ti
LC0 20x256 with 1x RTX 2080 Ti
Time control: 1 min per game + 10 seconds per move.

Result:
LC0 40x256 19.5 points
LC0 20x256 20.5 points
+1 =37 -2
Winning percentage = 48.75%

= LC0 40x256 is 9 elo weaker than LC0 20x256.
http://www.mediafire.com/file/eccp53cla ... 9.pgn/file

I'm very happy about this result because a month ago I have tested also the two best 40x256 and 20x256 LC0s and 40x256 was 75 elo weaker. So I avoided to tell that result, because it was not my intention to kill a new project before it starts. Sorry.

Take a look here if you missed some opinions from lkaufman:
viewtopic.php?f=6&t=70772

Congratulations to the 40x256 developers and supporters:)
I hope you can get more training games per day, because you have only 10% of what the main LC0 (20x256) is training per day!
http://157.230.189.191:8080
Glad to see it's a close competition now. One bit of advice: don't use time controls with small base to increment ratios like this 6 to 1, especially when testing Lc0. As you know, Lc0 sometimes takes ridiculous numbers of moves to win a won game. It is a total waste of testing time to be playing hundreds of moves in a resignable position at almost the same time limit as the first forty moves were played. Even for normal engines, the minimum ratio for efficient testing is 100 to 1 (i prefer 150 to 1 myself), but for Lc0 it is probably at least 300 to 1. You could easily double or triple the number of games per hour with no loss in quality. Maybe five minutes plus one second or ten minutes plus two seconds. Even half a second increment is way more than enough time to win these trivially won positions. I suppose if you are adjudicating aggressively then this isn't so much of an issue, but probably you are not.
I know the ridiculous numbers of moves to win a won game.
But it's not a total waste of testing time. Of course won endgames can be easily won very fast with low increment but if I would use five minutes plus one second or ten minutes plus two seconds, this would be clearly in favor of 20x256.
For example in the opening or in the middle game when only one or two seconds increment is left, 20x256 would win much more games. 40x256 is at the moment not improved enough, but it still can be improved much more than 20x256.
With 1 second increment it ends 1 vs 8 points and 32 draws, but with 10 seconds increment it ends 1 vs 2 and 37 draws.
So with more and more time, especially with enough time per move, 40x256 becomes stronger and stronger, while 20x256 is already playing with his full strength, because there are no improvements left.

80x1024 could be an interesting test too:)
Or 40x512.
I thing 256 is just to small.

lkaufman
Posts: 3438
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

Post by lkaufman » Mon Jul 01, 2019 5:06 am

Hai wrote:
Mon Jul 01, 2019 1:53 am
lkaufman wrote:
Sun Jun 16, 2019 1:17 am
Hai wrote:
Sat Jun 15, 2019 6:18 pm
The two strongest LC0s:
LC0 40x256 (ID 108) (v0.40.2-rc2) vs LC0 20x256 (ID 42549) (v0.21.2)

The 20 possible opening moves
40 games
Ponder on
LC0 40x256 with 1x RTX 2080 Ti
LC0 20x256 with 1x RTX 2080 Ti
Time control: 1 min per game + 10 seconds per move.

Result:
LC0 40x256 19.5 points
LC0 20x256 20.5 points
+1 =37 -2
Winning percentage = 48.75%

= LC0 40x256 is 9 elo weaker than LC0 20x256.
http://www.mediafire.com/file/eccp53cla ... 9.pgn/file

I'm very happy about this result because a month ago I have tested also the two best 40x256 and 20x256 LC0s and 40x256 was 75 elo weaker. So I avoided to tell that result, because it was not my intention to kill a new project before it starts. Sorry.

Take a look here if you missed some opinions from lkaufman:
viewtopic.php?f=6&t=70772

Congratulations to the 40x256 developers and supporters:)
I hope you can get more training games per day, because you have only 10% of what the main LC0 (20x256) is training per day!
http://157.230.189.191:8080
Glad to see it's a close competition now. One bit of advice: don't use time controls with small base to increment ratios like this 6 to 1, especially when testing Lc0. As you know, Lc0 sometimes takes ridiculous numbers of moves to win a won game. It is a total waste of testing time to be playing hundreds of moves in a resignable position at almost the same time limit as the first forty moves were played. Even for normal engines, the minimum ratio for efficient testing is 100 to 1 (i prefer 150 to 1 myself), but for Lc0 it is probably at least 300 to 1. You could easily double or triple the number of games per hour with no loss in quality. Maybe five minutes plus one second or ten minutes plus two seconds. Even half a second increment is way more than enough time to win these trivially won positions. I suppose if you are adjudicating aggressively then this isn't so much of an issue, but probably you are not.
I know the ridiculous numbers of moves to win a won game.
But it's not a total waste of testing time. Of course won endgames can be easily won very fast with low increment but if I would use five minutes plus one second or ten minutes plus two seconds, this would be clearly in favor of 20x256.
For example in the opening or in the middle game when only one or two seconds increment is left, 20x256 would win much more games. 40x256 is at the moment not improved enough, but it still can be improved much more than 20x256.
With 1 second increment it ends 1 vs 8 points and 32 draws, but with 10 seconds increment it ends 1 vs 2 and 37 draws.
So with more and more time, especially with enough time per move, 40x256 becomes stronger and stronger, while 20x256 is already playing with his full strength, because there are no improvements left.

80x1024 could be an interesting test too:)
Or 40x512.
I thing 256 is just to small.
I agree with you about the network size and about more time helping the larger network. It is obvious to me when analyzing with Lc0 on my 2080 that it rarely changes its analysis much after just a few seconds, and it cannot possibly be true that there is nothing left to discover in most chess positions after ten seconds! But I think it is just wrong to use roughly equal time per move in testing (I know CCRL and CEGT do it for reasons of historical compatibility, but it is terribly wasteful). If you fix the base to increment ratio at say 100 to 1 and just set the time limit to whatever will average the same amount of time per game as 1' + 10", I don't think your results for the larger network will be any worse. Having more time to think when the game is already essentially over is not as useful as having more time to think before that point.
Komodo rules!

Post Reply