LCZero update (2)

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

User avatar
George Tsavdaris
Posts: 1627
Joined: Thu Mar 09, 2006 12:35 pm

Re: LCZero update

Post by George Tsavdaris »

CMCanavessi wrote:
George Tsavdaris wrote:
CMCanavessi wrote:New official version released:

https://github.com/glinscott/leela-ches ... s/tag/v0.4

Finally includes a windows build with all the dlls, and a working windows CPU-Only build as well
The graph starts perhaps to show a point of diminishing returns?
Is this somewhat worrisome for the project?

http://lczero.org/
The graph is misleading because of a bug that has now been solved (though, the graph still shows old data). There is around 300 "self-play elo" difference between network id 55 and the previous one, which the graphic does not show. The progress hasn't stalled at all.

Code: Select all

   # PLAYER                                 :  RATING  PLAYED    W    D    L   (%)  D(%)  OppAvg  OppN  OppDiv
 184 Leela Chess Zero Gen 33 (ID 55) x64    :  1632.2     139   92    9   38    69     6  1441.8    25    24.9
 200 Leela Chess Zero Gen 30 (ID 44) x64    :  1506.3     200  106   22   72    59    11  1427.0    25    25.0
 208 Leela Chess Zero Gen 24 x64            :  1401.3     200   84   22   94    48    11  1427.0    25    25.0
 213 Leela Chess Zero Gen 20 x64            :  1317.8     200   64   27  109    39    14  1427.0    25    25.0
 226 Leela Chess Zero Gen 16 x64            :  1229.3     200   47   26  127    30    13  1427.0    25    25.0
 228 Leela Chess Zero Gen 14 x64            :  1185.3     200   42   20  138    26    10  1427.0    25    25.0
 232 Leela Chess Zero Gen 12 x64            :  1094.2     250   64   26  160    31    10  1278.6    49    40.8
 244 Leela Chess Zero Gen 10 x64            :   860.1      92   53   11   28    64    12   654.5    23    23.0
 247 Leela Chess Zero Gen 8 x64             :   791.6      92   45   17   30    58    18   654.5    23    23.0
 252 Leela Chess Zero Gen 6 x64             :   597.4      92   31   18   43    43    20   654.5    23    23.0
 256 Leela Chess Zero Gen 4 x64             :   369.0     150   43   18   89    35    12   623.1    15    15.0
Yes but after the bug is fixed i see that the networks after 55(excluding ID58 that shows 0 games) have a decline in the rate of the increment of the ELO, as also even a decline in ELO the latest ID57!

Code: Select all

Id 	Network 	Elo 	Games 	Blocks 	Filters 	Time
58 	ce9535cc 	4095.81 	0 	6 	64 	2018-03-30 02:36:34.524385 -0400 EDT
57 	85b39b2e 	4228.57 	49520 	6 	64 	2018-03-29 22:38:14.304221 -0400 EDT
56 	b59b6355 	4245.95 	26061 	6 	64 	2018-03-29 18:32:26.819199 -0400 EDT
55 	4b3d9155 	4230.28 	52925 	6 	64 	2018-03-29 09:25:34.139643 -0400 EDT
After his son's birth they've asked him:
"Is it a boy or girl?"
YES! He replied.....
Guenther
Posts: 4718
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: LCZero update

Post by Guenther »

CMCanavessi wrote:
David Xu wrote:LCZero possesses a "-noise" command that applies Dirichlet noise to its move selection, thereby causing randomness in its play.
Quick correction: --noise does not cause randomness in its play, it just introduce some randomness in the search, but the moves introduced are evaluated and discarded if they are found to be not good enough.

The randomness in its play that you see in self-play training games is caused by another thing, temperature=1
Just a question. Should the noise option be used at all, or will it hurt a bit when using it in match play vs. other programs with given start positions repeated from both sides?
https://rwbc-chess.de

[Trolls n'existent pas...]
jkiliani
Posts: 143
Joined: Wed Jan 17, 2018 1:26 pm

Re: LCZero update

Post by jkiliani »

Guenther wrote:
CMCanavessi wrote:
David Xu wrote:LCZero possesses a "-noise" command that applies Dirichlet noise to its move selection, thereby causing randomness in its play.
Quick correction: --noise does not cause randomness in its play, it just introduce some randomness in the search, but the moves introduced are evaluated and discarded if they are found to be not good enough.

The randomness in its play that you see in self-play training games is caused by another thing, temperature=1
Just a question. Should the noise option be used at all, or will it hurt a bit when using it in match play vs. other programs with given start positions repeated from both sides?
Noise DOES cause randomness in play, just a lot less than temperature. If the scenario you're using doesn't need any randomness in play, you don't need to use --noise either.
Nay Lin Tun
Posts: 710
Joined: Mon Jan 16, 2012 6:34 am

Re: LCZero update

Post by Nay Lin Tun »

Arena tournment run on my i5 7400 3GZ Quad core CPU vs Nvidia 1060x 3GB

GPU version get about 3 times nps.

https://ibb.co/bDVmaS
Nay Lin Tun
Posts: 710
Joined: Mon Jan 16, 2012 6:34 am

Re: LCZero update

Post by Nay Lin Tun »

Leela Zero play worse and worse these days..

This is the last game I see in my machine on training games. Leela even drop queen for unknown reason 27. Qc4???.(playing again like too much random) . PS- I have version 4 network.
User avatar
George Tsavdaris
Posts: 1627
Joined: Thu Mar 09, 2006 12:35 pm

Re: LCZero update

Post by George Tsavdaris »

Nay Lin Tun wrote:Leela Zero play worse and worse these days..

This is the last game I see in my machine on training games. Leela even drop queen for unknown reason 27. Qc4???.(playing again like too much random) . PS- I have version 4 network.
27.Qc4????
It definitely has some serious bugs! Can you post it in the lc0 forum in order to inform its programmers about it? Because in the games i played it also dropped a Bishop for nothing too. It seems general and serious.
After his son's birth they've asked him:
"Is it a boy or girl?"
YES! He replied.....
Nay Lin Tun
Posts: 710
Joined: Mon Jan 16, 2012 6:34 am

Re: LCZero update

Post by Nay Lin Tun »

I did! Time to solve the bug! lol
Werewolf
Posts: 1996
Joined: Thu Sep 18, 2008 10:24 pm

Re: LCZero update

Post by Werewolf »

George Tsavdaris wrote:
Yes but after the bug is fixed i see that the networks after 55(excluding ID58 that shows 0 games) have a decline in the rate of the increment of the ELO, as also even a decline in ELO the latest ID57!
Yes I'd love to hear why ID 60 is so much weaker than ID 55.

Is it another bug? Do the bug fixes lower elo before it climbs again? There seems to be a very regular "one big step forward" and then "lots of little steps backwards".
koedem
Posts: 105
Joined: Fri Mar 18, 2016 10:45 pm

Re: LCZero update

Post by koedem »

Nay Lin Tun wrote:Leela Zero play worse and worse these days..

This is the last game I see in my machine on training games. Leela even drop queen for unknown reason 27. Qc4???.(playing again like too much random) . PS- I have version 4 network.
But this is a training game not a real game. Those always have been terrible and will be for quite a while since temperature is used. (which introduces A LOT of randomness so that one gets different games even with such a huge number of games, also of course to possibly escape local maxima)
User avatar
CMCanavessi
Posts: 1142
Joined: Thu Dec 28, 2017 4:06 pm
Location: Argentina

Re: LCZero update

Post by CMCanavessi »

Why people keep posting training games is beyond incredible... it's been said a million times already that they are NOT representative of real strenght.


Anyways, progress....

Code: Select all

 200 Leela Chess Zero ID 80 x64             :  1821.7     200   76   26   98    45    13  1865.3    25    25.0
 213 Leela Chess Zero ID 69 x64             :  1753.8     200   62   21  117    36    11  1865.3    25    25.0
 230 Leela Chess Zero Gen 33 (ID 55) x64    :  1669.2     400  185   35  180    51     9  1645.0    45    43.5
 252 Leela Chess Zero Gen 30 (ID 44) x64    :  1504.0     200  106   22   72    59    11  1424.7    25    25.0
 261 Leela Chess Zero Gen 24 x64            :  1399.7     200   84   22   94    48    11  1424.7    25    25.0
 266 Leela Chess Zero Gen 20 x64            :  1316.5     200   64   27  109    39    14  1424.7    25    25.0
 279 Leela Chess Zero Gen 16 x64            :  1228.3     200   47   26  127    30    13  1424.7    25    25.0
 281 Leela Chess Zero Gen 14 x64            :  1184.4     200   42   20  138    26    10  1424.7    25    25.0
 285 Leela Chess Zero Gen 12 x64            :  1093.6     250   64   26  160    31    10  1276.7    49    40.8
 297 Leela Chess Zero Gen 10 x64            :   860.0      92   53   11   28    64    12   654.4    23    23.0
 300 Leela Chess Zero Gen 8 x64             :   791.4      92   45   17   30    58    18   654.4    23    23.0
 305 Leela Chess Zero Gen 6 x64             :   597.2      92   31   18   43    43    20   654.4    23    23.0
 309 Leela Chess Zero Gen 4 x64             :   368.9     150   43   18   89    35    12   622.8    15    15.0

And the scoring progression of the gauntlet I run to test:

Code: Select all

ID 55: 61.0/200
ID 69: 72.5/200
ID 80: 89.0/200
Yet, a (somewhat serious) bug was discovered yesterday and major changes are under way, which when combined with the engine improvements that are about to be released, will make a huge jump in strenght in the days to come... we'll see how things develop.
Follow my tournament and some Leela gauntlets live at http://twitch.tv/ccls