will Tcec allow Stockfish with a Leela net to play?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Wilson
Posts: 80
Joined: Tue Oct 29, 2019 2:20 am
Full name: Anthony Wilson

will Tcec allow Stockfish with a Leela net to play?

Post by Wilson » Thu Jun 17, 2021 5:08 am

if that happens other authors will do the same and Tcec will go downhill.

they need to make their "guidelines" mandatory rules, no exceptions, not even for Stockfish.

AndrewGrant
Posts: 1392
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: will Tcec allow Stockfish with a Leela net to play?

Post by AndrewGrant » Thu Jun 17, 2021 5:18 am

I'm fine with it. Its Leela data, but its still a Stockfish toolkit training the Networks, and a Stockfish implementation, and Stockfish is probably doing some preprocessing of that data. I'de prefer everyone go purely original ... but .... Stockfish with a Leela-data-trained Network is still as original, or more original, than everyone except: Ethereal, Seer, Komodo. [ to my knowledge of NNUEs ]

So before you ask TCEC to kick out Stockfish's Leela Net, you would have to ask them to kick out almost the entire NNUE field first.

Wilson
Posts: 80
Joined: Tue Oct 29, 2019 2:20 am
Full name: Anthony Wilson

Re: will Tcec allow Stockfish with a Leela net to play?

Post by Wilson » Thu Jun 17, 2021 6:11 am

AndrewGrant wrote:
Thu Jun 17, 2021 5:18 am
I'de prefer everyone go purely original ... but .... Stockfish with a Leela-data-trained Network is still as original, or more original, than everyone except: Ethereal, Seer, Komodo. [ to my knowledge of NNUEs ]
I thought you assumed Komodo used Stockfish's trainer since D. Kappe couldn't say and Mark and Larry ignored you while answered to others in the Komodo Dragon announcement thread. did I miss something?
AndrewGrant wrote:
Mon Nov 02, 2020 8:41 pm
dkappe wrote:
Mon Nov 02, 2020 8:40 pm
AndrewGrant wrote:
Mon Nov 02, 2020 8:35 pm

Are you not willing to say that Komodo is trained with separate code? I would think you would say so, because that would set Komodo apart.
Not for me to say.
Odd. I'm surprised by such a statement. Will Mark or Larry say?
Otherwise people will assume this is just SF's trainer, like most/all other NNUEs in play at the moment.
has Ethereal lost a lot of its originality as well?
AndrewGrant wrote:
Mon Nov 02, 2020 7:43 pm
Alayan wrote:
Mon Nov 02, 2020 7:41 pm
Good side : Leela is going to get kicked out of the next TCEC SuFi.
Bad side : training on Komodo data won't prevent Komodo Dragon from losing a lot of K's originality, for the most part becoming "Stockfish but worse".
+1

AndrewGrant
Posts: 1392
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: will Tcec allow Stockfish with a Leela net to play?

Post by AndrewGrant » Thu Jun 17, 2021 6:20 am

Wilson wrote:
Thu Jun 17, 2021 6:11 am
AndrewGrant wrote:
Thu Jun 17, 2021 5:18 am
I'de prefer everyone go purely original ... but .... Stockfish with a Leela-data-trained Network is still as original, or more original, than everyone except: Ethereal, Seer, Komodo. [ to my knowledge of NNUEs ]
I thought you assumed Komodo used Stockfish's trainer since D. Kappe couldn't say and Mark and Larry ignored you while answered to others in the Komodo Dragon announcement thread. did I miss something?
I do believe Komodo used Stockfish's trainer. I find it very unlikely it went any other way. However, I trust they used their own data, and their own implementation of the Network. Which scores them 2/3 on originality, inline with Stockfish with Leela nets. If I am wrong about it being Stockfish's trainer, then its 3/3 original. In either case, my statement stands. So you don't have to believe me or disbelieve me, either way my statement remains true, so this gotcha game is not worth anyone's time.
Wilson wrote:
Thu Jun 17, 2021 6:11 am
has Ethereal lost a lot of its originality as well?
Originality, no -- uniqueness, yes. SF networks have gotten more convoluted, but look back just a bit to a halfkp net, and Ethereal and Stockfish have technically the same evaluation function, just 20,000,000 different weights that define the function. But I manage to remain as original as possible with the halfkp, by not reusing anything in the Stockfish tool chain. My trainer is different. My factorizers are different. My implementation is different. My network encoding is different. My quantization is different. My UE portion of NNUE is different. The structure, 2x256 => 32 => 32 => 1 is NOT different.

---------------------------------------------------------------------------------------------------------------------------------------

But can you contend with my point, and not your qualms with me and Ethereal. If you count three points of originality: 1. Training data, 2. Training apparatus, 3. Implementation of the Network, then Stockfish-Leela-Network would score 2/3. Which is still more or equal to everyone but Ethereal, Seer, and Komodo.

Do you view the data as more important than the other two components?
Do you not agree with my three pillars or originality/uniqueness?

Michel
Posts: 2248
Joined: Sun Sep 28, 2008 11:50 pm

Re: will Tcec allow Stockfish with a Leela net to play?

Post by Michel » Thu Jun 17, 2021 6:25 am

Wilson wrote:
Thu Jun 17, 2021 5:08 am
if that happens other authors will do the same and Tcec will go downhill.

they need to make their "guidelines" mandatory rules, no exceptions, not even for Stockfish.
Fake news. Stockfish is not using a Leela net. I have reported this post.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.

AndrewGrant
Posts: 1392
Joined: Tue Apr 19, 2016 4:08 am
Location: U.S.A
Full name: Andrew Grant
Contact:

Re: will Tcec allow Stockfish with a Leela net to play?

Post by AndrewGrant » Thu Jun 17, 2021 6:38 am

Michel wrote:
Thu Jun 17, 2021 6:25 am
Wilson wrote:
Thu Jun 17, 2021 5:08 am
if that happens other authors will do the same and Tcec will go downhill.

they need to make their "guidelines" mandatory rules, no exceptions, not even for Stockfish.
Fake news. Stockfish is not using a Leela net. I have reported this post.
True and false. He really means a SF network trained using Leela data, but the nuance is not stated. Or perhaps its assumed that everyone has full knowledge. :shrug:

Modern Times
Posts: 2915
Joined: Thu Jun 07, 2012 9:02 pm

Re: will Tcec allow Stockfish with a Leela net to play?

Post by Modern Times » Thu Jun 17, 2021 6:47 am

Wilson wrote:
Thu Jun 17, 2021 5:08 am
if that happens other authors will do the same and Tcec will go downhill.

they need to make their "guidelines" mandatory rules, no exceptions, not even for Stockfish.
I guess they will re-write the rule to accommodate it.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

Michel
Posts: 2248
Joined: Sun Sep 28, 2008 11:50 pm

Re: will Tcec allow Stockfish with a Leela net to play?

Post by Michel » Thu Jun 17, 2021 6:48 am

AndrewGrant wrote:
Thu Jun 17, 2021 6:38 am
Michel wrote:
Thu Jun 17, 2021 6:25 am
Wilson wrote:
Thu Jun 17, 2021 5:08 am
if that happens other authors will do the same and Tcec will go downhill.

they need to make their "guidelines" mandatory rules, no exceptions, not even for Stockfish.
Fake news. Stockfish is not using a Leela net. I have reported this post.
True and false. He really means a SF network trained using Leela data, but the nuance is not stated. Or perhaps its assumed that everyone has full knowledge. :shrug:
Nuance means "subtle difference". We are definitely not talking about a subtle difference here. The difference between what the OP states and the truth is day and night.
Ideas=science. Simplification=engineering.
Without ideas there is nothing to simplify.

User avatar
pohl4711
Posts: 1690
Joined: Sat Sep 03, 2011 5:25 am
Location: Berlin, Germany
Full name: Stefan Pohl
Contact:

Re: will Tcec allow Stockfish with a Leela net to play?

Post by pohl4711 » Thu Jun 17, 2021 11:41 am

AndrewGrant wrote:
Thu Jun 17, 2021 5:18 am
Stockfish with a Leela-data-trained Network is still as original, or more original, than everyone except: Ethereal, Seer, Komodo. [ to my knowledge of NNUEs ]
That is exactly the point/the problem, I have as a tester of engines. Of course, it would be nice, to live in a perfect computerchess-world with 100% original engines, only. But we dont live in this world. And in modern times of open-source programming and net-learning, testing engines only, which we can be sure being 100% original, would mean, stop testing.
So, doing engine-testing in the real world of today: Where is the borderline between originality and cloning? Testing Stockfish, using Lc0-evals for its new net, but do not test Fire, using a net built on Stockfish-evals?
So, for me, I decided to test all these engines (and listing all engines, using a nnue-net built out of evals of another engine), except the many Stockfish-nearly-100%-clones like Shashchess, Eman, Honey etc. Not because, I want to ban these engines, but testing all these engines would mean distorting the whole ratinglist, because of too many Stockfishes in it.
Best solution? I dont know. But as a tester, I see no better way, doing my work.

I updated my ratinglist-site with this information below the ratinglist:
"Some engines are using a nnue-net based on evals of other engines. I decided to test these engines, too. As far as I know the follwing engines use nnue-nets based on evals of other engines (if I missed an engine, please contact me):
Fire 8.N, Fire 8.NN, Nemorino 6.00 (using Stockfish-based nnue net)
Fat Fritz 2, Stockfish (since 210615-version) (using Lc0-based nnue net)"

Modern Times
Posts: 2915
Joined: Thu Jun 07, 2012 9:02 pm

Re: will Tcec allow Stockfish with a Leela net to play?

Post by Modern Times » Thu Jun 17, 2021 12:59 pm

pohl4711 wrote:
Thu Jun 17, 2021 11:41 am

That is exactly the point/the problem, I have as a tester of engines. Of course, it would be nice, to live in a perfect computerchess-world with 100% original engines, only. But we dont live in this world. And in modern times of open-source programming and net-learning, testing engines only, which we can be sure being 100% original, would mean, stop testing.
So, doing engine-testing in the real world of today: Where is the borderline between originality and cloning? Testing Stockfish, using Lc0-evals for its new net, but do not test Fire, using a net built on Stockfish-evals?
Indeed. We recently added the following to our website for clarity. Although it was added only recently it has long been our approach:

** Please note that our role as a group is not to decide on the originality of engines as we have neither the legal or technical expertise. Viewers of the website must make their own judgements. Inclusion or non-inclusion of an engine should not be interpreted as a statement by us of the status of that engine **

That pre-dates NNUE and the new issues now arising as you've noted above ! What a minefield chess has become.
.

Opinions expressed here are my own, and not necessarily those of the CCRL Group.

Post Reply