Albert Silver wrote: ↑Mon Jun 04, 2018 9:02 pm
In view of the many settings in LC0, I ran a deep CLOP on three big ones at the same time. CLOP is designed to tune for the best playing performance, not best tactics or anything. The old default settings were nearly catastrophic giving NN369 a mere 109/200 solved in my corrected WAC set. The new settings yield... 159/200.
You can try them without any work, by downloading the latest LC0 June 4 build at
https://crem.xyz/lc0/ dubbed experimental.
Great! You mean your settings are the default settings of the latest LC0 experimental? And the test was at 10s/position?
I got similar results in tactics tuning on your suite WAC200, with my "extreme tactical settings". The problem was that it was degrading significantly positionally on my positional test suite Openings200.
NN373, GTX 1060 6GB, 10s/position
The default (31 May) LC0 tested in Poliglot on these suites was:
WAC200: 106/200
Openings200: 124/200
My "extreme tactical" LC0 (31 May) tested as:
WAC200: 156/200
Openings200: 106/200
Your settings with LC0 (4 June) experimental:
WAC200: 154/200
Openings200: 120/200
So, it seems that your settings from CLOP are not degrading positionally significantly, improving tactically greatly.
At ultra-fast games, my "extreme tactical" performed much worse than the old default. In games at 10'+ 10'' (comparable to time control used in test-suites solving), they both performed equally badly against Komodo 10.2, both 3.5/20. I will run now this experimental LC0 against Komodo 10.2 in 20 10' +10'' games (maybe first trying ultra-fast games). The sample will be small, but it's really a pain, these 20 games will take some 16 hours, partly because I use no adjudication, and some games can last very long time. LC0 can lose even completely drawn easy endgames, so I use no adjudication.
WIth the old default LC0, it had more chances to gain advantage in the opening against Komodo 10.2, which is a strong positional player too. But it often blundered elementarily, spoiling all its gains. With my "extreme tactical", it blundered less, but was often dominated positionally by Komodo 10.2. All in all, both came badly at 3.5/20, which gave me serious doubts about the scaling of LC0 (it performed on par or even worse than in fast games). When in Europe will be evening, I will report my results with your settings. Thanks for your efforts with CLOP!