CCRL 40/15, 40/2 and FRC lists updated (11th September 2021)

[connor_mcmonigle]

OTH I bet Stoofvlees is always tested with a different net.

OTOH I bet Stoofvlees has an author who povides them with all the components to test, and highlights when and what comprises a new version. Leela used to have a clear net to test: the last net of T10, T30, and T40. Nowadays the only clear upgrades representing it are the new binaries. The nets are all over the place, sprouting up in Discord, forums and more. Good luck trying to figure out which one presumably is the proper representative of the Leela project.

Have any of their key developers come forward to declare net XYZ as the current flagship of Leela? Demanding the tester go around digging through that is ridiculous. It is up to Leela's devs to provide a clear package to test. It is worth adding that because all the testers are free to choose and test what they want, there is no 'CCRL' policy on any engine or net.

You're spouting misinformation.
The network understood to be strongest at the time is packaged with each release: https://github.com/LeelaChessZero/lc0/r ... ag/v0.28.0. The networks are not "all over the place". It's completely reasonable to expect an engine to be tested with the network literally included in the release.

I'm afraid you're misinformed. The net that is packaged is a smaller demo net. It is not even the official strongest net. Much less the strongest net.

The strongest network depends on the hardware configuration+tc. It's not a "demo" net. The 15 block network packaged with the release was selected as to give good performance across a variety of platforms. It is definitely stronger than the old T40 network from 2 years ago.

[Rebel]

OTH I bet Stoofvlees is always tested with a different net.

OTOH I bet Stoofvlees has an author who povides them with all the components to test, and highlights when and what comprises a new version. Leela used to have a clear net to test: the last net of T10, T30, and T40. Nowadays the only clear upgrades representing it are the new binaries. The nets are all over the place, sprouting up in Discord, forums and more. Good luck trying to figure out which one presumably is the proper representative of the Leela project.

Have any of their key developers come forward to declare net XYZ as the current flagship of Leela? Demanding the tester go around digging through that is ridiculous. It is up to Leela's devs to provide a clear package to test. It is worth adding that because all the testers are free to choose and test what they want, there is no 'CCRL' policy on any engine or net.

https://lczero.org/dev/wiki/best-nets-for-lc0/

Did you even read that link? Because there is no clear nets designated to represent Leela in it. Instead it just says that there are a number of nets that can be recommended for different purposes. It isn't up to the tester to try to figure out which one should be the one to represent Leela. It is the developers we should be doing that. If they can't be bothered, then they can't complain when someone doesn't use the one they prefer.

Did you?

It's not that hard, you pick an advised net depending on a) the hardware and b) the time control you are planning to use.

[Albert Silver]

OTH I bet Stoofvlees is always tested with a different net.

OTOH I bet Stoofvlees has an author who povides them with all the components to test, and highlights when and what comprises a new version. Leela used to have a clear net to test: the last net of T10, T30, and T40. Nowadays the only clear upgrades representing it are the new binaries. The nets are all over the place, sprouting up in Discord, forums and more. Good luck trying to figure out which one presumably is the proper representative of the Leela project.

Have any of their key developers come forward to declare net XYZ as the current flagship of Leela? Demanding the tester go around digging through that is ridiculous. It is up to Leela's devs to provide a clear package to test. It is worth adding that because all the testers are free to choose and test what they want, there is no 'CCRL' policy on any engine or net.

https://lczero.org/dev/wiki/best-nets-for-lc0/

Did you even read that link? Because there is no clear nets designated to represent Leela in it. Instead it just says that there are a number of nets that can be recommended for different purposes. It isn't up to the tester to try to figure out which one should be the one to represent Leela. It is the developers we should be doing that. If they can't be bothered, then they can't complain when someone doesn't use the one they prefer.

Did you?

It's not that hard, you pick an advised net depending on a) the hardware and b) the time control you are planning to use.

If it's not that hard, then the Leela Devs can bloody well select one and send it. I cannot begin to imagine why it would fall on the tester to have to rummage through the site, find the wiki entry on all the nets, and then select one to represent the project. CCRL volunteers already provide thousands of hours of computer time, at the service of users and authors. I find it hard to believe that anyone would feel it was also their obligation to figure out what and where the engine they are testing is.

I can just see it in TCEC: "Hi, we would like to participate. You can use Google to find our entry. It's not that hard."

[Rebel]

You must remember from the FF1 rip-off that CPU ≠ GPU and that a 1060GTX (still 300 euro) is a different animal than the RTX 3090 following this benchmark the speed difference is a factor 6.5. Such giant speed fluctuations don't exist for the CPU. Hence Lc0 team made a guide line for optimal performance. It's perfect.

[Graham Banks]

We will just test Lc0 with the default net that comes with the release from now on.

[Rebel]

We will just test Lc0 with the default net that comes with the release from now on.

It's better to ask advice from knowledgeable people. Like I said, the choice of the net depends on the speed of the GPU and the time control one is planning to use.

[Graham Banks]

We will just test Lc0 with the default net that comes with the release from now on.

It's better to ask advice from knowledgeable people. Like I said, the choice of the net depends on the speed of the GPU and the time control one is planning to use.

No, because even then there will be disagreement.

We've made a group decision to stick with the net that comes packaged with each release, so no more experimenting with different nets from us.

[Rebel]

We will just test Lc0 with the default net that comes with the release from now on.

It's better to ask advice from knowledgeable people. Like I said, the choice of the net depends on the speed of the GPU and the time control one is planning to use.

No, because even then there will be disagreement.

We've made a group decision to stick with the net that comes packaged with each release, so no more experimenting with different nets from us.

But the default net is bad for a RTX 2080.

[Guenther]

We will just test Lc0 with the default net that comes with the release from now on.

It's better to ask advice from knowledgeable people. Like I said, the choice of the net depends on the speed of the GPU and the time control one is planning to use.

No, because even then there will be disagreement.

We've made a group decision to stick with the net that comes packaged with each release, so no more experimenting with different nets from us.

But the default net is bad for a RTX 2080.

Here I have to disagree. I don't think we know, if it is 'bad' for a RTX 2800 card. The main point though is you cannot make all people happy and
I think the solution to take the bundled net for testing by CCRL from now on is sensible.
There is no excuse then in the future for the LC0 team, if it is clear what CCRL is doing, except they won't care for CCRL anyway (which I don't know).

And if they would care a lot for testing organisations it should be also no problem to either bundle 2 or 3 diff. networks with a readme, or a suggestion for downloading 2 or 3 diff. nets for mainly used graphic cards (e.g. up to 1xxx, 2xxx and 3xxx cards).

[Rebel]

The default net is fom 1-1-2018