TCEC season 13, 2 NN engines will be participating, Leela and Deus X

[Gian-Carlo Pascutto]

I don't think people would object DeusX participating if it was done right.

I'm not sure of that. It makes little sense to me that it is allowed and asmFish is not. Last time I measured (2 days ago), asmFish was +15 Elo over Stockfish master. Not to speak of the many Stockfish clones with different parameters.

I already gave the example about Komodo too.

[Gian-Carlo Pascutto]

Fact indeed.

So then, since you knew that in advance, instead of complaining, protect your work.

I am not the one complaining here. I have accepted it and moved on. "Unfortunately" someone dragged my work back into the computer chess area so here I am again now in talkchess shitposting again.

Albert is free to do what he did wrt Leela Zero Chess in TCEC. TCEC admins have the right to request the source code, but for already mentioned reasons it's irrelevant if he admits there have been no changes. He can "sell" it but any "customer" has the right to demand the source code and then give it away for free.

Been browsing the Lc0 pages, couldn't find a clear statement about the legal status, is it freeware, GPL?

Seems clear enough to me:
https://github.com/LeelaChessZero/lczero#license
https://github.com/LeelaChessZero/lc0#license

It's also on top of every engine source file.

I noticed the training code (not that relevant to this discussion) doesn't have the license set or a COPYING:
https://github.com/LeelaChessZero/lczero-training

But it's based on code from me and others that was GPLv3, so it's necessarily GPLv3 too:
https://github.com/LeelaChessZero/lczer ... process.py

[IQ]

free to do what he did wrt Leela Zero Chess in TCEC. TCEC admins have the right to request the source code, but for already mentioned reasons it's irrelevant if he admits there have been no changes. He can "sell" it but any "customer" has the right to demand the source code and then give it away for free.

This is true. It is legal. That does not make it ethical. If the major participants of TCEC voiced their concerns over this, TCEC might change their stance on this. A large part of their credibility and standing is due to the participation of the strongest chess engines and a common understanding of fairness. Their opinion has some weight so one could try to inform them that the time pressure applied by Albert Silver did not allow for careful deliberation if lc0 dev team is really ok with Deus X participating. You yourself seem very disillusioned with the potential outcomes of such an attempt, but shouldn't the current dev team at least try?

[Gian-Carlo Pascutto]

try to inform them that the time pressure applied by Albert Silver did not allow for careful deliberation if lc0 dev team is really ok with Deus X participating. You yourself seem very disillusioned with the potential outcomes of such an attempt, but shouldn't the current dev team at least try?

Shouldn't they make a statement to that effect then? So far all statements say it should go ahead, including this one from Alexander:

I know that that DeusX confusion is resolved now, but just to state my opinion for the record:

It is clear to me that failing to acknowledge usage of LCZero in the interview and announcement is not an oversight of some kind, but a deliberate attempt to hide the fact that it's derived from LCZero work, and to exaggerate an own share of work in the "DeusX" project. Simply speaking, it was more cheating than misunderstanding.

ASilver keeps telling that he described everything "perfectly clear" to Anton (TCEC admin), but given that I saw that interview, I strongly suspect that it was not so perfectly clear. It's easy to misrepresent the importance of different parts of the engine when talking to a non-specialist, and it seems to me that it was the case. I'm sure that if TCEC admins had any doubts, they would ask LCZero devs for an opinion.

There is also no doubt that ASilver should have notified LCZero devs somehow that he's going to submit Lc0+alternative net to TCEC as a separate entry. Again it seems to me that it was an explicit decision not to tell rather that something ASilver just forgot to think of.

That reason alone (trying to mislead TCEC administration and fans, not because of "no clones rule") would be enough to disqualify the engine, but as it would be bad PR for TCEC (because of that "mystery NN" hype), and because many people will be curious to see how they compare, it's actually better to keep it in the competition.


I do appreciate ASilver's effort to recover relationship between him and the LCZero community and I think we should continue to collaborate (i.e. ASilver did run lots of CLOPs which were very helpful and Lc0's default parameters are from ASilver's tunings), but I'd just like to say that what happened is not "okay".

[IQ]

Shouldn't they make a statement to that effect then? So far all statements say it should go ahead, including this one from Alexander:

I know that that DeusX confusion is resolved now, but just to state my opinion for the record:

It is clear to me that failing to acknowledge usage of LCZero in the interview and announcement is not an oversight of some kind, but a deliberate attempt to hide the fact that it's derived from LCZero work, and to exaggerate an own share of work in the "DeusX" project. Simply speaking, it was more cheating than misunderstanding.

ASilver keeps telling that he described everything "perfectly clear" to Anton (TCEC admin), but given that I saw that interview, I strongly suspect that it was not so perfectly clear. It's easy to misrepresent the importance of different parts of the engine when talking to a non-specialist, and it seems to me that it was the case. I'm sure that if TCEC admins had any doubts, they would ask LCZero devs for an opinion.

There is also no doubt that ASilver should have notified LCZero devs somehow that he's going to submit Lc0+alternative net to TCEC as a separate entry. Again it seems to me that it was an explicit decision not to tell rather that something ASilver just forgot to think of.

That reason alone (trying to mislead TCEC administration and fans, not because of "no clones rule") would be enough to disqualify the engine, but as it would be bad PR for TCEC (because of that "mystery NN" hype), and because many people will be curious to see how they compare, it's actually better to keep it in the competition.


I do appreciate ASilver's effort to recover relationship between him and the LCZero community and I think we should continue to collaborate (i.e. ASilver did run lots of CLOPs which were very helpful and Lc0's default parameters are from ASilver's tunings), but I'd just like to say that what happened is not "okay".

Yes, they should. Thats what i would hope this leads to. The severe time pressure put upon them has lead to a number of uncoordinated statements by various devs, including the one you quote above.The severe difficulties to get a hold of even a majority of devs in any collobarative project as well as the lack of careful reflection of what actually happened makes all these statements preliminary. Thats why they are well justified to review their position and if necessary reverse their previous stance in a joint statement. They do not have to, of course. I wanted to point out that they still can.

Apart from the legal ramifications or how the current lc0 dev team sees this, and you probaly will not want to answer this, which i can perfectly understand, but how do you view the situation personally?

[Gian-Carlo Pascutto]

you probaly will not want to answer this, which i can perfectly understand, but how do you view the situation personally?

1) Pretending LC0 with a different network is a new engine is, to say the least, disingenuous. I haven't watched the interview yet but the statements made on the TCEC website were at the very least highly misleading. This has now been clarified. Alexander gave a rather frank statement about how he feels about what happened. I don't see what I can add here.

2) Training a supervised network to get "good" performance in chess is not trivial (depending on some definition of "good" of course). It took some attempts during early Leela Zero Chess development to get one that was good enough to be sure the engine was working before launching the distributed effort, and at least one person kept insisting it was impossible . It's also much less complicated than writing the entire engine.

The discussion whether what Albert did was "hard" can be compared someone submitting an Elo-gaining patch to Stockfish, for example, or any of the improved clones of open source engines, etc. (Well, we don't know if Albert's network is actually stronger than the main net to begin with, but ignore that for the sake of this discussion...)

It is not at all easy to find an Elo gainer for Stockfish, and nobody should pretend they can just go out and do that themselves. It's still much less hard than writing an equally strong engine from scratch. Nobody who finds an Elo gainer for Stockfish should go and rename the engine and claim it as his own.

3) The real problem, IMHO, is with TCEC and other engine authors. Why is Leela allowed to enter twice, with only a different parametrization, and other engines are not allowed to do the same? This is, at the base, grossly unfair. More entries means a higher chance to win, that is basic statistics. TCEC walks a fine line between "tournament of someone's basement" and "de-facto world championship for engines". In my opinion, allowing both Deus X and Leela Zero Chess in the same tournament moves TCEC more towards the former status than the latter.

[IQ]

I haven't watched the interview yet

I agree with most of your points, but in order to fully understand what this is really about you should watch the video.

[Guenther]

you probaly will not want to answer this, which i can perfectly understand, but how do you view the situation personally?

1) Pretending LC0 with a different network is a new engine is, to say the least, disingenuous. I haven't watched the interview yet but the statements made on the TCEC website were at the very least highly misleading. This has now been clarified. Alexander gave a rather frank statement about how he feels about what happened. I don't see what I can add here.

from yesterday still:
http://www.chessdom.com/tcec-season-13- ... f-the-nns/

I see nothing clarified or resolved and the desire of the LC0 team to name it 'LC0 with a DeusX called NN created (with all unchanged LC0 tools BTW) by Albert Silver obviously was refused. May be TCEC cannot even distinguish between an engine and the used NN.

current text:

Two Neural Network engines in TCEC

In December 2017 Google subsidiary DeepMind shocked the chess world when it announced that its neural network AlphaZero had soundly defeated then-champion Stockfish 8 in internal tests. While many pointed out that Stockfish 8′s play settings were far from optimal, DeepMind had certainly demonstrated that traditional chess programs could be challenged by deep learning algorithms. It was clear that a new and powerful class of chess-playing software had emerged – the Neural Network (NN) chess engines.

For the first time ever the TCEC competition is going to see two NN engines competing. This will be Leela Chess Zero and DeusX.

Leela Chess Zero aka LC0 was the first public project that showed promising results. The engine is based on the Leela Zero Go engine, which in turn was based on Google’s AlphaGo Zero project, and it was spearheaded by one of Stockfish developers Gary Linscott. Currently the project is advancing fast, with multiple developers and a bigger network coming. People involved in the development share that LC0 is gaining ELO very fast and is ready for the top 10 or more.

Deus X is a new neural network that will be debuting this season. The author states that the version of the engine that will appear in 4th Division easily plays strong enough chess to climb from the lower divisions. However, he is now constructing what he believes will be a much stronger version that will be competitive in the Premier Division. More information on Deus X and an interview with the author will be available on Chessdom.com soon.

At least 3 times it still speaks of two neural network engines, when it is just one and the same with only a different NN...
There is one true sentence in the paragraphs buried by still the same old obfuscation from before.

Of course the uninformed masses won't care anyway and they would even welcome 20 renamed asmfishes as long as it wouldnt be too obvious,
even to them, which is very easy to acchieve of course. A slightly modified spasmfish is in the make already for TCEC14 - my lips are sealed

[frankp]

you probaly will not want to answer this, which i can perfectly understand, but how do you view the situation personally?

1) Pretending LC0 with a different network is a new engine is, to say the least, disingenuous. I haven't watched the interview yet but the statements made on the TCEC website were at the very least highly misleading. This has now been clarified. Alexander gave a rather frank statement about how he feels about what happened. I don't see what I can add here.

......

3) The real problem, IMHO, is with TCEC and other engine authors. Why is Leela allowed to enter twice, with only a different parametrization, and other engines are not allowed to do the same? This is, at the base, grossly unfair. More entries means a higher chance to win, that is basic statistics. TCEC walks a fine line between "tournament of someone's basement" and "de-facto world championship for engines". In my opinion, allowing both Deus X and Leela Zero Chess in the same tournament moves TCEC more towards the former status than the latter.

The new engine is no different to taking SF and changing the values of the evaluation parameters.
It is difficult not get personal about this, but the interview and comments appear disingenuous, if not misleading, to say the least.
AS could have changed the parameter values of any engine, including lc0, and simply played his own tournament and published the results.
So why the need to enter it into TCEC.
Perhaps there is more to come in this saga.

[Rebel]

Fact indeed.

So then, since you knew that in advance, instead of complaining, protect your work.

I am not the one complaining here. I have accepted it and moved on. "Unfortunately" someone dragged my work back into the computer chess area so here I am again now in talkchess shitposting again.

Albert is free to do what he did wrt Leela Zero Chess in TCEC. TCEC admins have the right to request the source code, but for already mentioned reasons it's irrelevant if he admits there have been no changes. He can "sell" it but any "customer" has the right to demand the source code and then give it away for free.

Well, since it's GPL Albert has the obligation to publish all of it, including the weights.