My failed attempt to change TCEC NN clone rules

[dkappe]

Since Allie has been playing in the tournaments I have asked Ankan and he has said - most recently yesterday - that he is happy that Allie is using his cudann backend code and wished congratulations of success in TCEC.

I’m sorry, the NVIDIA employee who contributed the CUDA code in the first place gave his blessing? Mind you, this is sort of like pointing to a shared math library as evidence of cloning — it has nothing to do with chess. So not a clone in intent or fact.

[gonzochess75]

Since Allie has been playing in the tournaments I have asked Ankan and he has said - most recently yesterday - that he is happy that Allie is using his cudann backend code and wished congratulations of success in TCEC.

I’m sorry, the NVIDIA employee who contributed the CUDA code in the first place gave his blessing? Mind you, this is sort of like pointing to a shared math library as evidence of cloning — it has nothing to do with chess. So not a clone in intent or fact.

Well, one of the top Leela devs told me Ankan is the most important person in the project and although he spent little time contributing, his code is really critical as noone else knows to code for GPU, and without him cuda backend wouldn't probably happen. So it seems quite relevant that Ankan is totally Ok and happy that Allie is using his code.

[Gian-Carlo Pascutto]

I’m not sure if you realize this, but lczero and lc0 are two distinct engines. The first, lczero, was the initial chess engine used by the leela chess project and was derived from a combination of the leela go engine and stockfish chess board logic. The second, lc0, was a rewrite from scratch.

I’m not sure from your post, but I think you’re referring to lc0 as “lczero?” They are distinct code bases. An engine derived from lczero could not be a clone of lc0 and vice versa.

FWIW the client code of lc0 still uses the Leela Zero code for non-CUDA backends. Or at least the code was derived from it, though it may also have been heavily improved/reworked since.

Whether this is relevant for these competitions where everything runs on RTX cards I'll leave in the middle, but it may effect lc0-CPU.

[Gian-Carlo Pascutto]

NN-based engine consists of 3 parts:
1. Neural network.
2. Engine.
3. Training script.
You need 2 of 3 to be unique to be unique.

It’s so detached from reality! What is “training script” doing here? I was really perplexed. Who came up with those “2 of 3”? Training script is something that can be written in 1-2 hours (and I’m surprised no one did that to work around TCEC rules), it’s really very minor piece of work compared to other (others are very ambiguous too, I’ll post about them too a bit later).

I assume "training script" means all the code to train the network? Maybe if you do nothing else all day than write TensorFlow code for a living, you can write it in 1 hour. I still question that, though

I can tell you exactly how long it took to write the code for Leela Zero (of which AFAIK lc0 still uses parts). It's part of the public record:
commit 2f7463d2cfba1b4617b3bd73bbdf3e1f52382429
Author: Gian-Carlo Pascutto <gcp@sjeng.org>
Date: Tue Oct 24 20:20:09 2017 +0200

Initial revision.

commit 9d91bdd3fd84ad22eb49129d41c89861ea1f1e98
Author: Gian-Carlo Pascutto <gcp@sjeng.org>
Date: Tue Nov 7 22:15:59 2017 +0100

Reverse full connected weights order.

So it took me 2 weeks. For Stoofvlees it took a few days, thanks to the experience from Leela and knowing that one should not use TensorFlow...

Now of course if they mean some batch file or script to launch your learning framework with some set of data, then the whole thing does become laughable. But at least that's not how I interpreted those rules.

[Gian-Carlo Pascutto]

All of the architectures currently in common use in computer chess are derived from Deepmind’s Alpha Zero work.

Stacked ResNets are common in image recognition.

I assure you you'll find none of Alpha-Zero specific changes to *that* in Stoofvlees. The Alpha Zero changes are really how to map the resnet output into moves and scores. There's quite some room for improvement here.

[dkappe]

I’m not sure if you realize this, but lczero and lc0 are two distinct engines. The first, lczero, was the initial chess engine used by the leela chess project and was derived from a combination of the leela go engine and stockfish chess board logic. The second, lc0, was a rewrite from scratch.

I’m not sure from your post, but I think you’re referring to lc0 as “lczero?” They are distinct code bases. An engine derived from lczero could not be a clone of lc0 and vice versa.

FWIW the client code of lc0 still uses the Leela Zero code for non-CUDA backends. Or at least the code was derived from it, though it may also have been heavily improved/reworked since.

Whether this is relevant for these competitions where everything runs on RTX cards I'll leave in the middle, but it may effect lc0-CPU.

Oops. Can a chess engine be a clone of a go engine, or is it clear to everyone by now that this disputed area of code is not chess specific?

This community reminds me of that South Park episode where 5 minutes into being stuck in the school house in a blizzard, they enthusiastically resort to cannibalism. Clone!

P.S. Gian-Carlo: big fan of your work. Really curious to know what’s under the covers with Stoofvlees.

[Gian-Carlo Pascutto]

P.S. Gian-Carlo: big fan of your work. Really curious to know what’s under the covers with Stoofvlees.

Deep Sjeng, I thought that was somewhat obvious

There's nothing really "special" inside Stoofvlees (yet?! I think I'll need something to surpass the lc0 family). Someone on TCEC chat called it "write a Premier Division engine with these 10 easy tricks". It's just that I have some experience doing this kind of thing by now.

Edit: It must be the same for Daniel right? He's also been writing engines for ages.

[Gian-Carlo Pascutto]

They all use essentially the same A0 network types, as that is dictated by the PUCT search algorithm.

Really? Well, you probably want at least a move probability and some kind of winrate eval. That's not even an A0 innovation, although Alpha Zero was the first time I saw the networks for those outputs combined, which was a nice trick. (But I do not necessarily think that is the best approach depending on the nature of your data)

Using separate networks for policy and value was in the original Alpha Go paper, but had already been suggested on the computer-go mailing-list long before.

[Gian-Carlo Pascutto]

Haha, I just had an idea inspired by events that happened in other engine competitions (unrelated to chess), which didn't disallow engines by same author or clone engines. What you could do is entering to TCEC some hundred clones of Allie.

Stoofvlees exists because at some point the CSVN thought it was OK to allow two engines from the same author (and ignored my objections over this). I entered two engines, and Stoofvlees nicely gave a full point to Deep Sjeng when they played each other.

Whether that was done fairly I'll leave in the middle, as part of the point is that you can't tell.

[Robert Pope]

Allie is not a clone of Lc0. By far the most important part of code that Allie uses from Lc0 project is the cudann backend that Ankan wrote.
...
That really should put this issue to bed IMO.

I didn't claim that Allie was a clone. I'm just pointing out why variety is good to have, and the need for tournament organizers to be clear about their requirements.