Is Allie a 'Leela net player'?

[Madeleine Birchfield]

A while ago, a new version of Rubichess came out that supported NNUE, and the Rubichess author called Rubichess a 'NNUE player', saying 'I'm using this "playing NNUE weight files" wording intentionally cause my testing shows that all the music is within the weight files, the rest of the engine is just like a radio or CD player translating this music for your ears.'

So this got me thinking about Allie. It is clear that Allie is not a clone of Leela in the same way that Fire or Houdini is a clone of Stockfish, as its search is very different form Leela's search, just as how Rubichess is not a clone of Stockfish. However, its neural network backend is the same as Leela's and every single network that has run on Allie (Stein, the various T30/40/60, et cetera) was trained using the Leela engine, so it seems to me that Allie's sole function is to playing Leela network weight files, in the same way that the new Rubichess (as well as Minic, Igel, Orion, et cetera) plays NNUE network weight files. As a result, it seems proper to call Allie a 'Leela net player' in the same way that the Rubichess author calls Rubichess a 'NNUE player'.

Relatedly, TCEC have updated the rules and guidelines to include this:

Guidelines for use of NNUE at TCEC:

1. NNUE code can be used and considered as if it was a library (even if it is not literally one), custom modifications are encouraged.

• All NNUE training data should be generated by the unique engine's own search and/or eval code.

To what extent can the Leela neural network backend be used by other engines such as Allie and considered as if it was a library (even if it is not literally one), in the same way that TCEC and the Minic author consider NNUE as a library?

[Tony P.]

I consider the S14 NN rules still relevant. If they're applied to CPU NNs too, then there can only be one NNUE player at TCEC - SF itself - unless someone writes a training algorithm that gains a ton of rating points over SF's, which is hard because the nets are small.

I won't be surprised if Leela net players other than Leela herself are barred from TCEC once there are enough non-Leela, non-SF NN architectures to replace them, so soon(tm).

[Alayan]

TCEC has said their NNUE rules will rely upon some code of honor from programmers, so I'm not hopeful on it making a big impact.

In any case, yes, Allie is a "Leela net player". Allie now displays a min-maxed eval, and it has deliberately gone with a different win% -> cp conversion formula than Leela. But looking at the archived eval graphs from Allie games in DivP and SuFi is revealing. The shapes are very similar to that of the commentating Leela, going up and down in the same spots, including similar fakebooms in drawn endgames.

And the "Stein" net that has been tolerated by TCEC for several seasons now wouldn't pass the TCEC NNUE guidelines, considering it used a lot of Leela training games. Which is a part of why Allie played so similarly to Leela, but having the very same net architecture and broadly similar search (as the net architecture itself has strong search implications) is a major factor too.

[chrisw]

TCEC has said their NNUE rules will rely upon some code of honor from programmers, so I'm not hopeful on it making a big impact.

In any case, yes, Allie is a "Leela net player". Allie now displays a min-maxed eval, and it has deliberately gone with a different win% -> cp conversion formula than Leela. But looking at the archived eval graphs from Allie games in DivP and SuFi is revealing. The shapes are very similar to that of the commentating Leela, going up and down in the same spots, including similar fakebooms in drawn endgames.

And the "Stein" net that has been tolerated by TCEC for several seasons now wouldn't pass the TCEC NNUE guidelines, considering it used a lot of Leela training games. Which is a part of why Allie played so similarly to Leela, but having the very same net architecture and broadly similar search (as the net architecture itself has strong search implications) is a major factor too.

Chess engine programmers really should have a pile of sand in the backyard plus a kiln and make their own CPUs

[Alayan]

You dislike me so you'll make the most stupid comparison just to contradict me without the slightest effort to think on the topic at hand. What a waste of time.

[dkappe]

I know gonzo likes to test Allie with Tiny Gyal, which was trained on Bad Gyal data. No leela data in sight.

I look forward to comparisons of a0lite eval graph and lc0’s.

[Madeleine Birchfield]

I know gonzo likes to test Allie with Tiny Gyal, which was trained on Bad Gyal data. No leela data in sight.

All that says is that Tiny Gyal and Bad Gyal are trained using pure supervised learning with purely outside data sources. Was Tiny Gyal/Bad Gyal trained using the Leela engine or did it use a different engine for training? If they were trained using the Leela engine, then if TCEC fairly applied its rules to all engines, then Allie couldn't use the Tiny Gyal/Bad Gyal nets. Otherwise, TCEC wouldn't have required Igel to find another net in order to participate in their Cup 7, because the current net that Igel uses doesn't have any Stockfish data, but it still required the Stockfish engine to train the net.

And Bad Gyal and Tiny Gyal are listed on the Leela website as networks compatible with Leela, just like Stein is. Thus they are Leela nets. Both Leela and Allie are Leela net players, but since Leela was developed first it deserves precedence.

[chrisw]

You dislike me so you'll make the most stupid comparison just to contradict me without the slightest effort to think on the topic at hand. What a waste of time.

If that’s a reply to me, hard to tell when you are in the snipping habit, I don’t dislike anybody, I have no idea who you are, never met you and reserve like/dislike for actual 3D people in real
life, usually not even then. Hate and despising are not words I use. Arguing with arguments, even creative sarcasm, is what happens on the internet, no? I mean how do you get personal with a screen of ascii?

[Tony P.]

All NNUE training data should be generated by the unique engine's own search and/or eval code.

Fwiw, I don't think it's the final version of the rule. As is, it's unenforceable, as training code doesn't have to be released, and if the organizers required that code, they'd have a hard time reproducing the training process to verify that the supplied code was the one used.

From the legal standpoint, the text output of chess engines can't be copyrighted. The ChessDB.cn output and snapshots are in the public domain, even though SF evals were used at the leaves. It's ethically questionable but certainly legal to train a net to replicate those evals. If the net doesn't attempt to replicate them but only uses them as a second opinion for a warm start and then evolves away from them to its own evals, it's not even unethical imo. What matters to me isn't what data is legitimately used but how it's used.

[dkappe]

I know gonzo likes to test Allie with Tiny Gyal, which was trained on Bad Gyal data. No leela data in sight.

All that says is that Tiny Gyal and Bad Gyal are trained using pure supervised learning with purely outside data sources. Was Tiny Gyal/Bad Gyal trained using the Leela engine or did it use a different engine for training? If they were trained using the Leela engine, then if TCEC fairly applied its rules to all engines, then Allie couldn't use the Tiny Gyal/Bad Gyal nets. Otherwise, TCEC wouldn't have required Igel to find another net in order to participate in their Cup 7, because the current net that Igel uses doesn't have any Stockfish data, but it still required the Stockfish engine to train the net.

And Bad Gyal and Tiny Gyal are listed on the Leela website as networks compatible with Leela, just like Stein is. Thus they are Leela nets. Both Leela and Allie are Leela net players, but since Leela was developed first it deserves precedence.

You have a misunderstanding of how training works in leela land. The engine does not perform the training. A tensorflow framework is used to train nets using either self play or other data. My training framework is written using pytorch, not tensorflow, but all of them look much like image classification resnet training frameworks with lots of boilerplate, I.e. if you had to call it a clone, it would be a MNIST clone. Otherwise you can call them Alphazero type networks, or Allie networks, or Scorpio networks, or a0lite networks, which is most appropriate, as the latest Bad Gyal networks are not compatible with leela.

The NNUE networks are pretty simple by comparison. The pytorch code is pretty minimal. Only thing missing is a fast qsearch. With that, a training run on a GPU should take minutes instead of hours.