My failed attempt to change TCEC NN clone rules

[crem]

I wanted to bring up this topic several times already, but my drafts were too long to post.
Now, as Allie has good chances to go into finals, I think it’s time to bring this topic back.

Also I get frequent questions why don’t I bring this topic up if I don’t agree.

So, I tried to bring this topic up with TCEC administration, with no success.

Timeline:

March 10th (TCEC 15 just started, and Allie suddenly appeared there)

I contacted Anton Mihailov, TD director, telling that:

1. Current TCEC clone rules are not applied correctly:
Allie+Stein is a clone according to TCEC rules (both Allie is not unique relative to Lc0, and Stein is not unique comparing to Leela’s weights, see also http://talkchess.com/forum3/viewtopic.php?p=792755).
2. Rules themselves are poor and have to be changed.
3. I proposed various changes and ideas to the rules (with most of them Allie+Stein would be able to participate, and in some also “DeusX” could).
4. TCEC15 is already messed up, but let’s do that right for TCEC16, we have plenty of time (3 months until the new season start).

Anton responded:
1. This is very important, please keep writing.
2. Please keep it TOP SECRET! Noone should know!
(I tried to convince them that such discussions should be public, only got irrelevant answers, that I don’t know how to manage large communities)

I wrote lots of material, in different forms: one-line summary, one paragraph summary, diagrams, very detailed description etc.

In the end it was clear that noone from TCEC read even one-line summary.

March 29th
(not very relevant but for completeness)
We created a discord server, and Anton invited an undisclosed guest expert to the discussion, with whom we had an interesting, but short and not very relevant to TCEC rules discussion (because TCEC rules are up to TCEC team, really).

March 31th
I received the last message from Anton stating how important is this, and that I should continue writing my proposals.

April
I was pinging them periodically, with no reaction.

May 1st
I deleted the conversations (it was Google Doc and Discord server). Noone seemed to notice. Noone contacted me after that.

In the end I wasted ~15 hours of my time drawing all the diagrams and explanations of different detailization, all to save TCEC admins time.
From chats with them it was clear though that they didn’t even spend 5 minutes on that, they didn’t even read the 1-line summary.

Then TCEC-16 started with no changes at all.

At this point I decided that trying to convey any message to TCEC doesn't worth the effort.



Some screenshots from the document:

[Modern Times]

The whole clone and derivatives issue is a minefield. So many different opinions and viewpoints and "definitions". I would advise Anton to replace any clone "rules" with a simple statement "engines are entered at our sole discretion" then there are no arguments about rules being followed or not. Job done.

[crem]

The whole clone and derivatives issue is a minefield. So many different opinions and viewpoints and "definitions". I would advise Anton to replace any clone "rules" with a simple statement "engines are entered at our sole discretion" then there are no arguments about rules being followed or not. Job done.

I suggested that too as the first suggestion. He responded that rules have to be strict, unambiguous and cover all cases without need of any human interpretation.

[xr_a_y]

The whole clone and derivatives issue is a minefield. So many different opinions and viewpoints and "definitions". I would advise Anton to replace any clone "rules" with a simple statement "engines are entered at our sole discretion" then there are no arguments about rules being followed or not. Job done.

I suggested that too as the first suggestion. He responded that rules have to be strict, unambiguous and cover all cases without need of any human interpretation.

Looks like an impossible task to me, not only for NN engines.

And a lot thanks for your diagram, that are very clear.

[Modern Times]

I suggested that too as the first suggestion. He responded that rules have to be strict, unambiguous and cover all cases without need of any human interpretation.

Ridiculous response, that could never work

[crem]

I suggested that too as the first suggestion. He responded that rules have to be strict, unambiguous and cover all cases without need of any human interpretation.

Ridiculous response, that could never work

I cannot find exact response in logs, so it's possible that I exaggerated the response.
By my first proposal was to "define only vague rules, and they let admins apply common sense", and they didn't like it.

[noobpwnftw]

So to summarize, it's a story of how two dictators pretending to be open to opinions while one suggests the other shouldn't and the other suggesting to keep it up.

P.S. I was also told that I get to decide many things but in fact my opinion doesn't matter, either.

[Ozymandias]

I was also told that I get to decide many things but in fact my opinion doesn't matter

Sounds like you talked to some politician.

[Ferdy]

I wanted to bring up this topic several times already, but my drafts were too long to post.
Now, as Allie has good chances to go into finals, I think it’s time to bring this topic back.

Also I get frequent questions why don’t I bring this topic up if I don’t agree.

So, I tried to bring this topic up with TCEC administration, with no success.

Crosschecking your Alliestein image and TCEC rules I get this.

Code: Select all

Enter engine name? Alliestein

Rule 1: The code for training the NN, unique? (y/n) n

Rule 2: The NN weights file, unique? (y/n) y

Rule 3: The engine that executes the NN, unique? (y/n) n

TCEC Season 16, NN uniqueness test summary:
name: Alliestein
Pass Rule 1: n
Pass Rule 2: y
Pass Rule 3: n
Eligibility for participation: Fail
A minimum of 2 rules should pass!

Ref:
Rules/sections 16.b, 16.c and 16.d
https://tcec-chess.com/

Lets hear from TCEC TD.

[dkappe]

At some point a NN engine will beat lc0 in TCEC, and like the swallows returning to Capistrano, you will pen another missive about how they need to be banned. I look forward to your ongoing contortions.