Lc0 NN329 against Houdini 4 in 2 LTC games

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Lion
Posts: 531
Joined: Fri Mar 31, 2006 1:26 pm
Location: Switzerland

Re: Lc0 NN329 against Houdini 4 in 2 LTC games

Post by Lion »

Perfect, thanks!
shrapnel
Posts: 1339
Joined: Fri Nov 02, 2012 9:43 am
Location: New Delhi, India

Re: Lc0 NN329 against Houdini 4 in 2 LTC games

Post by shrapnel »

Well, this is embarrassing !
Komodo 11.2.2 x64 with Black, beat lc0 within 30 Moves :shock:
All Komodo settings were at default.
Well, I suppose its back to the Drawing board to try and find the best settings for lc0.
I've noticed that lco almost never utilizes the full 60 seconds when it is its turn to move.
It didn't seem to matter so much against Stockfish, but here this flaw is perhaps making it pay a heavy price.
lco seems very uncomfortable against Komodo.
Odd.
Anyway, here's the Game.
https://lichess.org/F5qUw9Z2
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: Lc0 NN329 against Houdini 4 in 2 LTC games

Post by Milos »

shrapnel wrote: Thu May 24, 2018 5:49 pm Well, this is embarrassing !
Komodo 11.2.2 x64 with Black, beat lc0 within 30 Moves :shock:
All Komodo settings were at default.
Well, I suppose its back to the Drawing board to try and find the best settings for lc0.
I've noticed that lco almost never utilizes the full 60 seconds when it is its turn to move.
It didn't seem to matter so much against Stockfish, but here this flaw is perhaps making it pay a heavy price.
lco seems very uncomfortable against Komodo.
Odd.
Anyway, here's the Game.
https://lichess.org/F5qUw9Z2
If you indeed gave all 8 cores of your CPU to Komodo it would be at least 400 Elo stronger than Lc0-cudnn on your precious 1080Ti even with best possible settings. So being killed in 30 moves while playing white is nothing out of ordinary.
If you were more informed you'd know what is the actual strength on Lc0 on particular hardware and wouldn't daydream of "putting Komodo, SF and H to dust" ;).
Lion
Posts: 531
Joined: Fri Mar 31, 2006 1:26 pm
Location: Switzerland

Re: Lc0 NN329 against Houdini 4 in 2 LTC games

Post by Lion »

shrapnel wrote: Thu May 24, 2018 5:49 pm Well, this is embarrassing !
Komodo 11.2.2 x64 with Black, beat lc0 within 30 Moves :shock:
All Komodo settings were at default.
Well, I suppose its back to the Drawing board to try and find the best settings for lc0.
I've noticed that lco almost never utilizes the full 60 seconds when it is its turn to move.
It didn't seem to matter so much against Stockfish, but here this flaw is perhaps making it pay a heavy price.
lco seems very uncomfortable against Komodo.
Odd.
Anyway, here's the Game.
https://lichess.org/F5qUw9Z2
I am not surprised.
I had several years ago had my own computer play on playchess.com the same variation with black and had many wins in a similar way.
So I am not surprised that LCO with no knowledge of the opening/variation falls the same way top engines loosed in the same opening/variation.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Lc0 NN329 against Houdini 4 in 2 LTC games

Post by Laskos »

With my more tactical settings for Lc0, which seemed to perform badly in short time control games, I played one game at 10 minutes + 10 seconds increment of Lc0 NN335 on GTX 1060 as White against latest SF dev on 4 i7 cores as Black, from standard opening position. It was a draw in Queen's Indian Defence.
https://lichess.org/3w2TWU3Z

Maybe, after all, the feeling that improving tactics by fiddling with parameters of Lc0 will be beneficial at LTC is not that far off. And this game was not even that long time control. Only one game played, will maybe play to four games later.
jp
Posts: 1470
Joined: Mon Apr 23, 2018 7:54 am

Re: Lc0 NN329 against Houdini 4 in 2 LTC games

Post by jp »

shrapnel wrote: Thu May 24, 2018 5:49 pm Well, this is embarrassing !
Komodo 11.2.2 x64 with Black, beat lc0 within 30 Moves :shock:
All Komodo settings were at default.
What is the default setting re. opening book?
yanquis1972
Posts: 1766
Joined: Wed Jun 03, 2009 12:14 am

Re: Lc0 NN329 against Houdini 4 in 2 LTC games

Post by yanquis1972 »

Laskos wrote: Thu May 24, 2018 10:19 pm With my more tactical settings for Lc0, which seemed to perform badly in short time control games, I played one game at 10 minutes + 10 seconds increment of Lc0 NN335 on GTX 1060 as White against latest SF dev on 4 i7 cores as Black, from standard opening position. It was a draw in Queen's Indian Defence.
https://lichess.org/3w2TWU3Z

Maybe, after all, the feeling that improving tactics by fiddling with parameters of Lc0 will be beneficial at LTC is not that far off. And this game was not even that long time control. Only one game played, will maybe play to four games later.
unless they're quite a radical departure from what i've seen suggested, or the TC is classical, i doubt it. messing around yesterdat with the WAC revised 200, i couldn't improve on a limited set that give leela the most difficulty, going from 5s to 60s+ with various extremes. imo an essential step in her core evolution simply hasn't taken place yet. but i'd love to be wrong!
shrapnel
Posts: 1339
Joined: Fri Nov 02, 2012 9:43 am
Location: New Delhi, India

Re: Lc0 NN329 against Houdini 4 in 2 LTC games

Post by shrapnel »

jp wrote: Thu May 24, 2018 10:50 pm What is the default setting re. opening book?
I'm using Sedat's 2017 .abk book.
It's around 7-8 Moves deep.
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis
shrapnel
Posts: 1339
Joined: Fri Nov 02, 2012 9:43 am
Location: New Delhi, India

Re: Lc0 NN329 against Houdini 4 in 2 LTC games

Post by shrapnel »

Laskos wrote: Thu May 24, 2018 10:19 pm With my more tactical settings for Lc0...
What Settings are you using, Mr Laskos ?
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Lc0 NN329 against Houdini 4 in 2 LTC games

Post by Laskos »

shrapnel wrote: Fri May 25, 2018 7:13 am
Laskos wrote: Thu May 24, 2018 10:19 pm With my more tactical settings for Lc0...
What Settings are you using, Mr Laskos ?
I tried my extreme settings, which are performing much better in tactical test-suites, but worse in positional ones. They are:

Scale thinking time=2.87
Cpuct MCTS option=9
First Play Urgency Reduction=-0.25

All games are from the initial standard opening position. These settings achieved one draw and 3 losses against SF dev on 4 cores at 10'+ 10''. Then I started to use CLOP settings and default settings, the results were similar, totally in 16 games at 10'+ 10'' with NN325, 6 draws and 10 losses against SF dev on 4 cores. I am unable to discern which settings are the best at this TC. The estimated rating of SF dev on 4 cores is 3600 CCRL 40/4' Elo. That of Lc0 NN335 on GTX 1060 about 3300 in these conditions. I will maybe wait until bigger nets (256x20) will start to train on new self-games, hope they will improve quickly, at least at their beginning. My impression is that 192x15 architecture was not fully exploited, they seem to run into over-fitting and some specific issues I don't understand.

Here is a game where Lc0 NN335 held a draw as Black from standard opening position:
https://lichess.org/XfEJUjWe