will Tcec allow Stockfish with a Leela net to play?

[pohl4711]

Indeed. We recently added the following to our website for clarity. Although it was added only recently it has long been our approach:

** Please note that our role as a group is not to decide on the originality of engines as we have neither the legal or technical expertise. Viewers of the website must make their own judgements. Inclusion or non-inclusion of an engine should not be interpreted as a statement by us of the status of that engine **

That pre-dates NNUE and the new issues now arising as you've noted above ! What a minefield chess has become.

I think, each new development has pros and cons. Now, we see, that the opensource and open-framework development of Stockfish and Lc0 is not 100% a good thing, too. But IMHO, the pros are much more and much more important than the cons. But, nevertheless, it is important to understand, that they are cons. Not only pros. Like everywhere.

[Ferdy]

I am hoping one day there will be a competition or a rating list based on NN that is you take the best engine in search like stockfish and everybody is welcome to submit their NN. It will be a battle of NN. Or an engine based on stockfish with new NN architecture. Shogi in Japan is probably ahead of chess in terms of NN application.

[Raphexon]

I am hoping one day there will be a competition or a rating list based on NN that is you take the best engine in search like stockfish and everybody is welcome to submit their NN. It will be a battle of NN. Or an engine based on stockfish with new NN architecture. Shogi in Japan is probably ahead of chess in terms of NN application.

That competition already exists, it's called Fishtest. Winner gets the title of master net.

[noobpwnftw]

I am hoping one day there will be a competition or a rating list based on NN that is you take the best engine in search like stockfish and everybody is welcome to submit their NN. It will be a battle of NN. Or an engine based on stockfish with new NN architecture. Shogi in Japan is probably ahead of chess in terms of NN application.

That competition already exists, it's called Fishtest. Winner gets the title of master net.

Now I understand what this is all about.

[connor_mcmonigle]

I am hoping one day there will be a competition or a rating list based on NN that is you take the best engine in search like stockfish and everybody is welcome to submit their NN. It will be a battle of NN. Or an engine based on stockfish with new NN architecture. Shogi in Japan is probably ahead of chess in terms of NN application.

Well, this tournament sort of already exists for computer chess as well: it's called FishTest. Anyone can participate and the winner (likely) gets their network bundled with the next Stockfish release and their name immortalized in the list of Stockfish contributors. It also doubles as a sort of rating list.

(whoops, someone beat me to writing this, haha)

Stockfish with a Leela-data-trained Network is still as original, or more original, than everyone except: Ethereal, Seer, Komodo. [ to my knowledge of NNUEs ]

That is exactly the point/the problem, I have as a tester of engines. Of course, it would be nice, to live in a perfect computerchess-world with 100% original engines, only. But we dont live in this world. And in modern times of open-source programming and net-learning, testing engines only, which we can be sure being 100% original, would mean, stop testing.
So, doing engine-testing in the real world of today: Where is the borderline between originality and cloning? Testing Stockfish, using Lc0-evals for its new net, but do not test Fire, using a net built on Stockfish-evals?
So, for me, I decided to test all these engines (and listing all engines, using a nnue-net built out of evals of another engine), except the many Stockfish-nearly-100%-clones like Shashchess, Eman, Honey etc. Not because, I want to ban these engines, but testing all these engines would mean distorting the whole ratinglist, because of too many Stockfishes in it.
Best solution? I dont know. But as a tester, I see no better way, doing my work.

I updated my ratinglist-site with this information below the ratinglist:
"Some engines are using a nnue-net based on evals of other engines. I decided to test these engines, too. As far as I know the follwing engines use nnue-nets based on evals of other engines (if I missed an engine, please contact me):
Fire 8.N, Fire 8.NN, Nemorino 6.00 (using Stockfish-based nnue net)
Fat Fritz 2, Stockfish (since 210615-version) (using Lc0-based nnue net)"

The problem with testing Fire with a Stockfish network is that Fire is barely more original than the other Stockfish clones you listed. This is not controversial. You can ask any of the prominent Stockfish contributors for their opinion about the originality of Fire and hear something similar to the above. Furthermore, literally any engine now around 3100-3200 elo would instantly jump to 3300-3400 elo if they had such low standards that they were willing to copy Stockfish's entire evaluation function as Fire's "author" has done. Your rating list would end up ridiculously distorted were every author to have such low standards.

[kranium]

The problem with testing Fire with a Stockfish network is that Fire is barely more original than the other Stockfish clones you listed. This is not controversial. You can ask any of the prominent Stockfish contributors for their opinion about the originality of Fire and hear something similar to the above. Furthermore, literally any engine now around 3100-3200 elo would instantly jump to 3300-3400 elo if they had such low standards that they were willing to copy Stockfish's entire evaluation function as Fire's "author" has done. Your rating list would end up ridiculously distorted were every author to have such low standards.

that's absolutely not true Connor...
take a close look at Sugar for ex and you can see one-to-one correspondence line by line with source code from SF
that's not the case with Fire, and Fire is much weaker than Sugar for ex.

Fire's evaluation is not a 'copy' of SF. Anyone can see this, just compare the 2 sources...there is 0 overlap or similarity.
Fire's eval is based on Houdini, and Houdini's evaluation is unique and had absolutely nothing to do with SF's eval.
It's clear to me you haven't really looked closely, and are likely chiming in to support Andrew.

Fire's eval and search are completely different than SF
UCI interface is 100% different
Fire utilizes a unique asynchronous output throughout
there are thousands of differences...you making an analogy to any of those very closely copied SF clones is highly inaccurate to say the least
the part of Fire that is quite similar to SF is the time control, mainly because after testing many many different implementations for months,
there was nothing that worked as well.

"You can ask any of the prominent Stockfish contributors for their opinion about the originality of Fire and hear something similar to the above."
nonsense, I don't believe you've spoken to anybody...

You and Andrew are chums, I'm well aware of that...but it's time to stop with the persistent attacks

[kranium]

I'de prefer everyone go purely original ... but .... Stockfish with a Leela-data-trained Network is still as original, or more original, than everyone except: Ethereal, Seer, Komodo. [ to my knowledge of NNUEs ]

But can you contend with my point, and not your qualms with me and Ethereal. If you count three points of originality: 1. Training data, 2. Training apparatus, 3. Implementation of the Network, then Stockfish-Leela-Network would score 2/3. Which is still more or equal to everyone but Ethereal, Seer, and Komodo.

State-of-the-art chess programming ideas and techniques are well-known and well documented.
Almost all engines now use them...including Ethereal and Seer

Programmers should not be required to re-invent the wheel when it comes to NNUE

This is not required for EGTB files...everyone can download them and use them in their engines.
(and these TB files are normally generated by stockfish)

Testers should not be under pressure to eliminate certain engines because one loud-mouthed and arrogant
programmer has declared all engines near him in the rankings are not original, thereby eliminating all competition
and arrogantly proclaiming himself one of the only 3 original engines left.

Andrew has expressed doubts about Komodo originality and suggests it probably used Stockfish evals?
and guess what... that only leaves Ethereal and Seer!
It's quite easy now to understand Connor's motive and posts...

I urge everyone (especially top testers) to not be fooled...don't let this become the blind leading the blind. evaluate the engines, look at the source code, seek impartial advice (and that certainly excludes Andrew and Connor who are pursuing a personal agenda), and make up your own mind.

[connor_mcmonigle]

Fire's evaluation is not a 'copy' of SF. Anyone can see this, just compare the 2 sources...there is 0 overlap or similarity.
Fire's eval is based on Houdini, and Houdini's evaluation is unique and had absolutely nothing to do with SF's eval.
It's clear to me you haven't really looked closely, and are likely chiming in to support Andrew.

Well, the evaluation function was copied from Houdini before, but now it's been replaced with a copy of an old Stockfish network with NNUE implementation originating from SF. That's what I meant by "copying Stockfish's evaluation function". Regardless of whether we consider your old evaluation function (copied from Houdini with minimal modification) or the recent NNUE addition (copied from Stockfish/CFish without any modifications - a pretty big overlap) your position doesn't really improve.

Many Stockfish developers have voiced their annoyance with Fire and the attention it has received on other forums. Many have stated that they view it as a Stockfish derivative recognizing several of their specific patches in your search code. You'll have to ask them if you don't believe me.

[kranium]

I hope you realize you've just gone from

The problem with testing Fire with a Stockfish network is that Fire is barely more original than the other Stockfish clones you listed. This is not controversial. You can ask any of the prominent Stockfish contributors for their opinion about the originality of Fire and hear something similar to the above.

to

Many Stockfish developers have voiced their annoyance with Fire and the attention it has received on other forums. Many have stated that they view it as a Stockfish derivative recognizing several of their specific patches in your search code. You'll have to ask them if you don't believe me.

in a very very short time

and now I have to ask them about it?

But the truth is: as of today I'm aware of 1 SF contributer recognizing a piece of code he added to SF as a patch. It's can be found in the engines origins forum.
I don't believe anything else has ever been said or posted.

PS Ethereal github commit history contains a record of dozens of these SF patches being applied to Ethereal.

Your method and motives are become crystal clear Connor.

[kranium]

I'm happy to see Norman release his code to the chess community , I believe has has rectified any perceived or real transgressions and it clear his code does not have inordinate amount of SF code and he did contribute to the development of Stockfish NNUE. It is about as good as it gets given the differences of opinion. It is my opinion Fire should be able compete in all tournaments - no different that Fan Fritz 2 or 1 and many other engines.
I wish him well.

Within the Stockfish family Fire 8.1's highest depth-1 similarity is 64.21% with Stockfish 8 so I compared its sources with this version and I must agree with you. There are things which are common in many top open source engines like the previously mentioned ones and I would say that a couple of them are much more influenced by Stockfish than Fire is but that's it. If I didn't know Houdini is a Stockfish derivative and Fire is based on Houdini, I wouldn't think Fire's evaluation function could be considered derived from Stockfish at all. If anyone is interested I can make a post with the notes I took during the comparison.

Luckily we do have some fair and impartial programmers here who are not pursuing a an hidden agenda to disqualify competition