My failed attempt to change TCEC NN clone rules

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
gonzochess75
Posts: 34
Joined: Mon Dec 10, 2018 2:29 pm
Full name: Adam Treat

Re: My failed attempt to change TCEC NN clone rules

Post by gonzochess75 » Thu Sep 19, 2019 3:55 pm

Robert Pope wrote:
Thu Sep 19, 2019 3:10 pm
If you agree with the above, then you need to have something in place to make sure engines are "different". It's not as big a deal when you just have a couple duplicates like with Lc0 and DeusX, and I don't have an issue with different groups that are trying different approaches, but when you can take GPL code and 20 people can churn out 20 flavors of the same engine, you need something in place to limit them, or you end up with just a ranking tournament of clones.
Allie is not a clone of Lc0. By far the most important part of code that Allie uses from Lc0 project is the cudann backend that Ankan wrote. Here is a direct quote from Ankan when I asked him if he had any concerns about Allie using this code and playing in TCEC:
"Hi. No, i don't have any issues. In fact I am happy that my code is used in multiple engines :)

Btw, congratulations on great performance by Allie at TCEC"
That code is great, very important and all praise to Ankan for writing it, but that code all by itself is a far cry from a chess engine. So you see Allie is not a clone or a copy of Lc0. And other developers are encouraged to use that cudann backend all they want to make actual NN chess engines. Feel free and enjoy.

That really should put this issue to bed IMO.

dkappe
Posts: 263
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: My failed attempt to change TCEC NN clone rules

Post by dkappe » Thu Sep 19, 2019 4:13 pm

MikeB wrote:
Thu Sep 19, 2019 3:49 pm

You certainly have made very strong argument that Allie is a clone , regardless of what anyone points to as being different. And NN (scorpioNN, Stoofvlees) or MCTS (KomodoMCTS) are not clones.

Not sure if it will change anything under the current infatuation state with anything neural.
If you run SIMEX on neural network engines (other than Stoofvlees), you get similarity of over 60%. That’s because they all derive from the alpha zero pseudo code which dictates certain neural network structures. They’re all “conceptual” clones of alpha zero, not lc0, which is itself a clone (though here clone is not a really useful term). Even a from scratch 100 line python implementation of mcts will produce the the same moves on the same net once multithreading is disabled. It’s in the nature of deterministic algorithms.

If you find this sort of hand waving argument from pictures compelling, I have a bridge to sell you.

dkappe
Posts: 263
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: My failed attempt to change TCEC NN clone rules

Post by dkappe » Thu Sep 19, 2019 4:27 pm

gonzochess75 wrote:
Thu Sep 19, 2019 2:15 pm

Since Allie has been playing in the tournaments I have asked Ankan and he has said - most recently yesterday - that he is happy that Allie is using his cudann backend code and wished congratulations of success in TCEC.
I’m sorry, the NVIDIA employee who contributed the CUDA code in the first place gave his blessing? Mind you, this is sort of like pointing to a shared math library as evidence of cloning — it has nothing to do with chess. So not a clone in intent or fact.

gonzochess75
Posts: 34
Joined: Mon Dec 10, 2018 2:29 pm
Full name: Adam Treat

Re: My failed attempt to change TCEC NN clone rules

Post by gonzochess75 » Thu Sep 19, 2019 4:54 pm

dkappe wrote:
Thu Sep 19, 2019 4:27 pm
gonzochess75 wrote:
Thu Sep 19, 2019 2:15 pm

Since Allie has been playing in the tournaments I have asked Ankan and he has said - most recently yesterday - that he is happy that Allie is using his cudann backend code and wished congratulations of success in TCEC.
I’m sorry, the NVIDIA employee who contributed the CUDA code in the first place gave his blessing? Mind you, this is sort of like pointing to a shared math library as evidence of cloning — it has nothing to do with chess. So not a clone in intent or fact.
Well, one of the top Leela devs told me Ankan is the most important person in the project and although he spent little time contributing, his code is really critical as noone else knows to code for GPU, and without him cuda backend wouldn't probably happen. So it seems quite relevant that Ankan is totally Ok and happy that Allie is using his code.

Gian-Carlo Pascutto
Posts: 1184
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: My failed attempt to change TCEC NN clone rules

Post by Gian-Carlo Pascutto » Thu Sep 19, 2019 4:59 pm

dkappe wrote:
Sun Sep 15, 2019 1:41 am
I’m not sure if you realize this, but lczero and lc0 are two distinct engines. The first, lczero, was the initial chess engine used by the leela chess project and was derived from a combination of the leela go engine and stockfish chess board logic. The second, lc0, was a rewrite from scratch.

I’m not sure from your post, but I think you’re referring to lc0 as “lczero?” They are distinct code bases. An engine derived from lczero could not be a clone of lc0 and vice versa.
FWIW the client code of lc0 still uses the Leela Zero code for non-CUDA backends. Or at least the code was derived from it, though it may also have been heavily improved/reworked since.

Whether this is relevant for these competitions where everything runs on RTX cards I'll leave in the middle, but it may effect lc0-CPU.

Gian-Carlo Pascutto
Posts: 1184
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: My failed attempt to change TCEC NN clone rules

Post by Gian-Carlo Pascutto » Thu Sep 19, 2019 5:12 pm

crem wrote:
Sat Sep 14, 2019 10:08 pm
NN-based engine consists of 3 parts:
1. Neural network.
2. Engine.
3. Training script.
You need 2 of 3 to be unique to be unique.

It’s so detached from reality! What is “training script” doing here? I was really perplexed. Who came up with those “2 of 3”? Training script is something that can be written in 1-2 hours (and I’m surprised no one did that to work around TCEC rules), it’s really very minor piece of work compared to other (others are very ambiguous too, I’ll post about them too a bit later).
I assume "training script" means all the code to train the network? Maybe if you do nothing else all day than write TensorFlow code for a living, you can write it in 1 hour. I still question that, though :-)

I can tell you exactly how long it took to write the code for Leela Zero (of which AFAIK lc0 still uses parts). It's part of the public record:
commit 2f7463d2cfba1b4617b3bd73bbdf3e1f52382429
Author: Gian-Carlo Pascutto <gcp@sjeng.org>
Date: Tue Oct 24 20:20:09 2017 +0200

Initial revision.

commit 9d91bdd3fd84ad22eb49129d41c89861ea1f1e98
Author: Gian-Carlo Pascutto <gcp@sjeng.org>
Date: Tue Nov 7 22:15:59 2017 +0100

Reverse full connected weights order.

So it took me 2 weeks. For Stoofvlees it took a few days, thanks to the experience from Leela and knowing that one should not use TensorFlow...

Now of course if they mean some batch file or script to launch your learning framework with some set of data, then the whole thing does become laughable. But at least that's not how I interpreted those rules.

Gian-Carlo Pascutto
Posts: 1184
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: My failed attempt to change TCEC NN clone rules

Post by Gian-Carlo Pascutto » Thu Sep 19, 2019 5:16 pm

dkappe wrote:
Sun Sep 15, 2019 7:58 pm
All of the architectures currently in common use in computer chess are derived from Deepmind’s Alpha Zero work.
Stacked ResNets are common in image recognition.

I assure you you'll find none of Alpha-Zero specific changes to *that* in Stoofvlees. The Alpha Zero changes are really how to map the resnet output into moves and scores. There's quite some room for improvement here.
Last edited by Gian-Carlo Pascutto on Thu Sep 19, 2019 5:19 pm, edited 1 time in total.

dkappe
Posts: 263
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: My failed attempt to change TCEC NN clone rules

Post by dkappe » Thu Sep 19, 2019 5:19 pm

Gian-Carlo Pascutto wrote:
Thu Sep 19, 2019 4:59 pm
dkappe wrote:
Sun Sep 15, 2019 1:41 am
I’m not sure if you realize this, but lczero and lc0 are two distinct engines. The first, lczero, was the initial chess engine used by the leela chess project and was derived from a combination of the leela go engine and stockfish chess board logic. The second, lc0, was a rewrite from scratch.

I’m not sure from your post, but I think you’re referring to lc0 as “lczero?” They are distinct code bases. An engine derived from lczero could not be a clone of lc0 and vice versa.
FWIW the client code of lc0 still uses the Leela Zero code for non-CUDA backends. Or at least the code was derived from it, though it may also have been heavily improved/reworked since.

Whether this is relevant for these competitions where everything runs on RTX cards I'll leave in the middle, but it may effect lc0-CPU.
Oops. Can a chess engine be a clone of a go engine, or is it clear to everyone by now that this disputed area of code is not chess specific?

This community reminds me of that South Park episode where 5 minutes into being stuck in the school house in a blizzard, they enthusiastically resort to cannibalism. Clone!

P.S. Gian-Carlo: big fan of your work. Really curious to know what’s under the covers with Stoofvlees.

Gian-Carlo Pascutto
Posts: 1184
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: My failed attempt to change TCEC NN clone rules

Post by Gian-Carlo Pascutto » Thu Sep 19, 2019 5:22 pm

dkappe wrote:
Thu Sep 19, 2019 5:19 pm
P.S. Gian-Carlo: big fan of your work. Really curious to know what’s under the covers with Stoofvlees.
Deep Sjeng, I thought that was somewhat obvious :D

There's nothing really "special" inside Stoofvlees (yet?! I think I'll need something to surpass the lc0 family). Someone on TCEC chat called it "write a Premier Division engine with these 10 easy tricks". It's just that I have some experience doing this kind of thing by now.

Edit: It must be the same for Daniel right? He's also been writing engines for ages.
Last edited by Gian-Carlo Pascutto on Thu Sep 19, 2019 5:26 pm, edited 1 time in total.

Gian-Carlo Pascutto
Posts: 1184
Joined: Sat Dec 13, 2008 6:00 pm
Contact:

Re: My failed attempt to change TCEC NN clone rules

Post by Gian-Carlo Pascutto » Thu Sep 19, 2019 5:25 pm

dkappe wrote:
Sun Sep 15, 2019 9:31 pm
They all use essentially the same A0 network types, as that is dictated by the PUCT search algorithm.
Really? Well, you probably want at least a move probability and some kind of winrate eval. That's not even an A0 innovation, although Alpha Zero was the first time I saw the networks for those outputs combined, which was a nice trick. (But I do not necessarily think that is the best approach depending on the nature of your data)

Using separate networks for policy and value was in the original Alpha Go paper, but had already been suggested on the computer-go mailing-list long before.

Post Reply