LC0 0.18rc2 released

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Javier Ros
Posts: 181
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

LC0 0.18rc2 released

Post by Javier Ros » Thu Sep 27, 2018 7:01 am

Second LC0 release candidate of 0.18 released
Change:
Fixed a severe race condition bug when only happened when --out-of-order-eval is on (which is the default value now).
Could produce incorrect search results.

https://github.com/LeelaChessZero/lc0/releases
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

User avatar
AdminX
Posts: 5104
Joined: Mon Mar 13, 2006 1:34 pm
Location: Acworth, GA
Contact:

Re: LC0 0.18rc2 released

Post by AdminX » Thu Sep 27, 2018 7:30 am

Thanks for the heads up.
"Good decisions come from experience, and experience comes from bad decisions."
__________________________________________________________________
Ted Summers

Lion
Posts: 509
Joined: Fri Mar 31, 2006 11:26 am
Location: Switzerland

Re: LC0 0.18rc2 released

Post by Lion » Thu Sep 27, 2018 11:47 am

Thank you.

I have been testing a "lot" 0.18rc1 and am wondering what influence had the mentioned "severe bug":

Severe bug fixed: Race condition when out-of-order-eval was enabled (and it was enabled by default)

Any idea what that means to actual LC0 testing?

crem
Posts: 98
Joined: Wed May 23, 2018 7:29 pm

Re: LC0 0.18rc2 released

Post by crem » Thu Sep 27, 2018 12:26 pm

Lion wrote:
Thu Sep 27, 2018 11:47 am
Thank you.

I have been testing a "lot" 0.18rc1 and am wondering what influence had the mentioned "severe bug":

Severe bug fixed: Race condition when out-of-order-eval was enabled (and it was enabled by default)

Any idea what that means to actual LC0 testing?
Two threads could update the same node simultaneously.
That could result in incorrect score update for a subtree, or e.g. some subtree being locked out of search (to be never considered).
With only two threads (default), it happened ~once per 1-2 minutes in my tests.
With more threads (e.g.) it happens every 10 seconds or so. The blunder vs Ethereal in TCEC bonus games (but not in CCCC) can be attributed to that bug.

Usually this bug reveals in a form of sudden blunder while better moves are more obvious.

crem
Posts: 98
Joined: Wed May 23, 2018 7:29 pm

Re: LC0 0.18rc2 released

Post by crem » Thu Sep 27, 2018 12:29 pm

crem wrote:
Thu Sep 27, 2018 12:26 pm
With more threads (e.g.) it happens every 10 seconds or so. The blunder vs Ethereal in TCEC bonus games (but not in CCCC) can be attributed to that bug.
For the blunder in CCCC game (where Leela put a rook under knight's attack and did not expect it to be taken), we still don't know the reason, and we cannot reproduce it.
It must be yet another bug.

cc2150dx
Posts: 173
Joined: Sat Nov 30, 2013 8:51 am
Location: Canada

Re: LC0 0.18rc2 released

Post by cc2150dx » Thu Sep 27, 2018 12:47 pm

I'm assuming that: cuda10.0-cudnn7.3-for-2080.zip is for the new 2080 nvidia cards and cuda.zip is for anything older?

Thanks
Komodo tester

crem
Posts: 98
Joined: Wed May 23, 2018 7:29 pm

Re: LC0 0.18rc2 released

Post by crem » Thu Sep 27, 2018 1:03 pm

cc2150dx wrote:
Thu Sep 27, 2018 12:47 pm
I'm assuming that: cuda10.0-cudnn7.3-for-2080.zip is for the new 2080 nvidia cards and cuda.zip is for anything older?

Thanks
As far as I know, this version works with older GPUs too, but requires CUDA 10, and very fresh drivers. It gives some improvement for fp16 but not for usual 32-bit cudnn backend.

For most people ordinary cuda build is good enough.

Nay Lin Tun
Posts: 510
Joined: Mon Jan 16, 2012 5:34 am

Re: LC0 0.18rc2 released

Post by Nay Lin Tun » Fri Sep 28, 2018 5:04 pm

Thanks for version 18. I noticed significant increase in speed in my GTX 1060 GPU. (average 2.5 k to 3.5 k) :D :)

Javier Ros
Posts: 181
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: LC0 0.18rc2 released

Post by Javier Ros » Fri Sep 28, 2018 11:23 pm

I had done a comparison test between lc0 0.18 rc1 and lc0 0.18 rc2 versions, using the Silver Test with 50 positions from

https://en.chessbase.com/post/test-your ... ings-suite

Lc0 version 0.18 rc1 with net 11261 on Asus Geforce GTX 1060 3 Gb against Stockfish 9 with 4 threads on i7 4790S with 4 cores and 8 threads, ponder Off, Hash tables 1 Gb. for both, under Arena gui. Time 5 minutes plus 3 seconds.
I want to experiment with tablebases support for lc0, 5 men syzygy, to see how it is working. The conclusion is that 5 men syzygy tablebases don't help much.
Stockfish 9 played without syzygy tablebases.
LeR=3300*875/4500000=0.64

Lc01811261Stockfish9Silver5plus3HT4CPU10 2018

1 Stockfish_9_x64_bmi +52 +27/=61/-12 57.50% 57.5/100
2 Lc01811261syzygy -52 +12/=61/-27 42.50% 42.5/100


lc0 crashes several times during the match. After that I repeat the complete test with the new lc0 v0.18 rc2. This time there were no crashes in the 100 games.

Lc018rc211261syzygy - Stockfish_9_x64_bmi2 : 42,5/100 9-24-67 (===1=1=====0=0===0=0=010=0=0===10====0=====0======1======000=======0=1======0=1===000==0=00====001=1)

Stockfish_9_x64_bmi2 - Lc018rc211261syzygy : 57,5/100 24-9-67 (===0=0=====1=1===1=1=101=1=1===01====1=====1======0======111=======1=0======1=0===111==1=11====110=0)

Exactly the same result!

The last game of the second match was amazing! A victory of lc0 with black pieces in the King's Indian Defense.



Complete tournament output from Arena

Lc018rc111261syzygy - Stockfish_9
https://sites.google.com/site/unoallavo ... ects=0&d=1

Lc018rc211261syzygy - Stockfish_9
https://sites.google.com/site/unoallavo ... ects=0&d=1
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

MikeB
Posts: 3201
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: LC0 0.18rc2 released

Post by MikeB » Sat Sep 29, 2018 1:25 am

Javier Ros wrote:
Fri Sep 28, 2018 11:23 pm
I had done a comparison test between lc0 0.18 rc1 and lc0 0.18 rc2 versions, using the Silver Test with 50 positions from

https://en.chessbase.com/post/test-your ... ings-suite

Lc0 version 0.18 rc1 with net 11261 on Asus Geforce GTX 1060 3 Gb against Stockfish 9 with 4 threads on i7 4790S with 4 cores and 8 threads, ponder Off, Hash tables 1 Gb. for both, under Arena gui. Time 5 minutes plus 3 seconds.
I want to experiment with tablebases support for lc0, 5 men syzygy, to see how it is working. The conclusion is that 5 men syzygy tablebases don't help much.
Stockfish 9 played without syzygy tablebases.
LeR=3300*875/4500000=0.64

Lc01811261Stockfish9Silver5plus3HT4CPU10 2018

1 Stockfish_9_x64_bmi +52 +27/=61/-12 57.50% 57.5/100
2 Lc01811261syzygy -52 +12/=61/-27 42.50% 42.5/100


lc0 crashes several times during the match. After that I repeat the complete test with the new lc0 v0.18 rc2. This time there were no crashes in the 100 games.

Lc018rc211261syzygy - Stockfish_9_x64_bmi2 : 42,5/100 9-24-67 (===1=1=====0=0===0=0=010=0=0===10====0=====0======1======000=======0=1======0=1===000==0=00====001=1)

Stockfish_9_x64_bmi2 - Lc018rc211261syzygy : 57,5/100 24-9-67 (===0=0=====1=1===1=1=101=1=1===01====1=====1======0======111=======1=0======1=0===111==1=11====110=0)

Exactly the same result!

The last game of the second match was amazing! A victory of lc0 with black pieces in the King's Indian Defense.



Complete tournament output from Arena

Lc018rc111261syzygy - Stockfish_9
https://sites.google.com/site/unoallavo ... ects=0&d=1

Lc018rc211261syzygy - Stockfish_9
https://sites.google.com/site/unoallavo ... ects=0&d=1
Thanks for sharing, 19. Qc1 is a clunker , game over, dev- McCain 10.0 ( not yet released).


Code: Select all

 dep	score	nodes	time	(not shown:  tbhits	knps	seldep)
 28	 -2.81 	5.65G	4:14.71	Rh7 c6 bxc6 Bg1 g3 dxc6 gxh2 Bxh2 h4 Nf2 Nh5 Kg1 h3 gxh3 Qh4 Qc2 Ne7 Ng4 Qxh3 Nd1 Kf8 Ndf2 Qh4 b5 Bg5 Bc4 Nf6 Qe2 Nxg4 Nxg4 Bxg4 
 28	 -2.75!	4.80G	3:36.10	Rh7 Bg1 g3 Rd1 gxh2 Bxh2 h4 Nf2 Nh5 Kg1 Qg5 Kf1 h3 gxh3 Ng3+ Ke1 Nh4 Rd3 Ng2+ Kd2 a5 Ng4 axb4 Rxb4 dxc5 Rb2! 
 27	 -2.65 	3.56G	2:39.28	Rh7 Bg1 g3 Rd1 gxh2 Bxh2 h4 c6 bxc6 Nf2 Nh5 Kg1 Qg5 Ng4 Ng3 dxc6 Bxg4 fxg4 Nxe2+ Nxe2 Qxg4 Qc4+ Kh8 Qd3 Rg7 Qf3 Ne7 Qxg4 
 27	 -2.51!	3.13G	2:19.93	Rh7 c6 bxc6 dxc6 g3 Bg1 gxh2 Bxh2 h4 Nf2 Nh5 Qc2 Ng3+ Kg1 Kh8 Nd5 h3 gxh3 Qh4 Bc4 Bg5 Rb3 Bxh3 Nxh3 Qxh3 Qg2 Qh5 b5 Nh4 Qf2 Rf8 Ra1! 
 27	 -2.42?	3.00G	2:14.02	Rh7 c6 bxc6 dxc6 g3 Bg1 gxh2 Bxh2 h4 Nf2 Nh5 Qc2 Ng3+ Kg1 Kh8 Nd5 h3 gxh3 Qh4 Bc4 Bg5 Rb3 Bxh3 Nxh3 Qxh3 Qg2 Qxg2+? 
 26	 -2.51!	2.94G	2:11.41	Rh7 c6 bxc6 dxc6 g3 Bg1 gxh2 Bxh2 h4 Nf2 Nh5 Kg1 h3 gxh3 Qh4 Qc2 Ne7 Ng4 Qxh3 Nd5! 
 26	 -2.48!	2.52G	1:51.75	Rh7 Bg1 g3 Rd1 gxh2 Bxh2 h4 Nf2 Nh5 Kg1 h3 gxh3 Ng3 c6 bxc6 dxc6 Be6 Qc2 Bg7 Ra1! 
 25	 -2.39 	2.05G	1:31.14	Rh7 Bg1 g3 Rd1 gxh2 Bxh2 h4 Nf2 Nh5 Kg1 h3 gxh3 Ng3 c6 bxc6 dxc6 Nxe2+ Nxe2 Nh4 Kh1 Nxf3 Rb3 Nxh2 Rg1+ Rg7 Rxg7+ Kxg7 Kxh2 Be6 Rf3 Bg5 
 25	 -2.07!	1.43G	1:03.43	Rh7 Be1 g3 Qb2 Rg7 c6 bxc6 dxc6 Nf8 Bd1 Ne8 h3 Qh4 Kg1 Nh7 Bb3+ Kh8 b5 Ng5 Bc4 Nxh3+ gxh3 Nf6 Nb4 Bxh3 Nbd5 Nh7 a4! 
 24	 -1.97 	1.17G	0:51.65	Rh7 Be1 g3 Qb2 Rg7 c6 bxc6 dxc6 Nf8 Bd1 Kh7 Ne2 h4 Bc2 Nh5 Nxe5 dxe5 Qxe5 Qg5 Qxg5 Rxg5 hxg3 Nxg3+ Bxg3 hxg3 Nxf4 Ne6 Nd5 Bg7 e5+ Kg8 Nf6+ Bxf6 exf6 Kf7 f4 Rh5+ Kg1 Kxf6 
 24	 -1.88!	809.6M	0:35.86	Rh7 Be1 g3 Qb2 Rg7 c6 bxc6 dxc6 Nf8 Bd1 h4 h3 Kh7 Bb3 Nh5 Rd1 Qe8 Nd5 Rb8 Rc1 Bxh3 Qa3! 
 23	 -1.78!	536.8M	0:23.42	Rh7 Be1 g3 Qb2 Rg7 c6 bxc6 dxc6 Nf8 Bd1 h4 Bb3+ Kh8 Qd2 Ne6 hxg3 hxg3 Bd5! 
 22	 -1.69 	437.4M	0:19.08	Rh7 Be1 g3 Qb2 Rg7 c6 bxc6 dxc6 h4 h3 Nh5 b5 Nf8 Rg1 Ne6 Nd5 Ng5 Nxe5 dxe5 Qxe5 Be6 Bc3 
 22	 -1.55!	340.9M	0:14.96	Rh7 Qd1 g3 Be1 h4 h3 Nf8 b5 Rg7 c6 bxc6 dxc6 Bxh3 Rg1! 
 21	 -1.45 	267.4M	0:11.83	Rh7 Qd1 g3 Be1 h4 h3 Nf8 b5 Rg7 c6 Nh5 cxb7 Bxb7 Nb4 Bc8 Rg1 Nh7 Nc6 Qg5 a4 Bd7 
 21	 -1.18!	196.5M	0:08.75	Rh7 Qd1 g3 hxg3 fxg3 Bxg3 h4 Be1 Nh5 b5 Ng3+ Kh2! 
 20	 -1.09 	166.4M	0:07.48	Rh7 Qd1 g3 Be1 h4 h3 Nh5 b5 Rg7 c6 Nf8 Nb4 bxc6 Nxc6 Qg5 Rg1 Bd7 a4 Nh7 Qc2 Qg6 
 20	 -0.65!	110.6M	0:05.04	Rh7 Qc2 g3 hxg3 fxg3 Bxg3 h4 Bf2 Nh5 c6 bxc6 dxc6 Ng3+ Kg1 Qg5 Rfe1! 
 19	 -0.56 	97.9M  	0:04.48	Rh7 Qc2 g3 hxg3 fxg3 Bxg3 h4 Bf2 Nh5 c6 Ng3+ Kg1 bxc6 dxc6 Be6 Nd5 Nxf1 Bxf1 h3 
 19	 -0.13!	87.1M  	0:04.00	Rh7 Qe1 g3 hxg3 fxg3 Bxg3 h4 Bf2 Nh5 Kg1 Qg5 Nb2! 
 18	 -0.04 	86.1M  	0:03.95	Rh7 Qe1 Bg5 a4 a6 a5 h4 fxg4 h3 g3 Nxg4 Qd1 Nxf2+ Nxf2 Bd7 Bh5 Qf6 c6 bxc6 dxc6 
 18	 -0.12!	79.4M  	0:03.64	Rh7 Qc2 Bg5 a4 g3 hxg3 fxg3 Bxg3 h4 Bf2 Nh5 Kg1 Ng3 Rfc1 Bxc1 Rxc1 Qg5 Nb2! 
 17	 -0.12 	40.6M  	0:01.87	Rg7 Qd1 gxf3 Bxf3 Ng4 h3 Nh4 Bxh4 Qxh4 Qe1 Qg5 cxd6 Ne3 dxc7 Rxc7 Rg1 Rg7 
 16	+0.14 	16.0M  	0:00.74	Bg5 a4 Rh7 Rg1 h4 fxg4 Nxg4 Bxg4 Bxg4 h3 Bh5 b5 b6 c6 Rf7 Kh2 a5 bxa6 
 15	+0.30 	10.5M  	0:00.50	Bg5 a4 Rh7 Rg1 h4 fxg4 Nxg4 Bxg4 Bxg4 h3 Bh5 Qb2 f3 gxf3 
 14	+0.33 	8.28M  	0:00.39	Bg5 a4 Rh7 Qd1 g3 hxg3 fxg3 Bxg3 h4 Bf2 Nh5 Kh2 Ng3 cxd6 
 13	+0.28 	5.05M  	0:00.24	h4 fxg4 Nxg4 Qd1 h3 gxh3 Nxf2+ Nxf2 a5 bxa5 Rg7 Ng4 Bg5 Qc2 Rxa5 cxd6 cxd6 
 12	+0.44 	3.24M  	0:00.16	Bg5 Qd1 Bh4 Bg1 Rg7 a4 Ne7 Qb3 a5 fxg4 hxg4 cxd6 cxd6 
 11	+0.38 	1.55M  	0:00.08	Bf8 a4 h4 fxg4 Nxg4 Qd1 Qg5 a5 Bd7 h3 Nxf2+ Rxf2 
 10	+0.30 	577741	0:00.03	h4 fxg4 Nxg4 Qd1 Qg5 a4 a5 Nb5 Bf8 cxd6 cxd6 bxa5 Rxa5 h3 Nxf2+ Rxf2 
  9	+0.34 	391453	0:00.02	h4 fxg4 Nxg4 Qd1 Nxf2+ Rxf2 Qg5 Bh5 a6 a4 Bd7 h3 Raf8 
  8	+0.32 	150111	0:00.01	h4 fxg4 Nxg4 Bxg4 Bxg4 h3 Bd7 b5 
  7	+0.19 	39666  	0:00.00	h4 fxg4 Nxg4 Qc2 Nxf2+ Nxf2 Qg5 a4 
  6	 -0.47 	22026  	0:00.00	h4 fxg4 Nxg4 a4 Nxf2+ Nxf2 
  5	 -0.38 	13706  	0:00.00	h4 fxg4 Nxg4 
  4	 -0.03 	7993    	0:00.00	c6 cxd6 Qxd6 dxc6 
  3	 -0.01 	3837    	0:00.00	Bd7 a3 gxf3 
  2	 -0.01 	3156    	0:00.00	Bd7 a3 gxf3 
  1	+0.35 	1084    	0:00.00	gxf3 
  0	# 

Post Reply