"It must be a clone of some sort..."

[mvanthoor]

Because of the topic "Is cloning a hobby?" ( http://talkchess.com/forum3/viewtopic.php?f=7&t=75040 ), I was thinking about how I react to new chess engines coming onto the scene.

The one thing that always fascinated me about chess engines is that they seem to the litmus test of programming: if you can write a decent chess engine, you must be a good programmer. At least, when I was around 15 a few centuries ago, everyone I knew that tinkered around with computers and some programming wanted to write their own chess engine.

I had my own chess engine, written in Borland Pascal. No matter that the only thing it could do was receive moves like "e2e4" and print an ASCII board. No matter that it only reached depth 3 or 4 and played horrible chess and made an illegal move now and then. No matter that the code was basically unmaintainable. Spaghetti would be nicer to untangle. The only thing that counted was the fact that I *did* have a chess engine, and *thus* I was a good programmer at 15, in the eyes of others. (I don't have that source code anymore. If I did, my eyes would probably start to bleed as soon as I looked at the first few functions.)

Writing a decent chess engine was very hard in the 70's and 80's. Computers were slow, there were only concepts (often only scientific papers), and getting help let alone source code was almost impossible... because they either were not available, or people didn't want to give you any help or source code, because chess engines were often commercial competitors. I never noticed, because in the early 80's, I'd just started saying "mama". I only know it because it's history.

With the advent of the internet for the general public it became easier in the 90's (and I did experienced some of that), and with open source engines, it became even easier in the 2000's. Information and source code are everywhere, and everybody and his grandma seems to have his or her own chess engine.

In short, back then, if you wanted to have your own chess engine, you had to be able to write it yourself.

Now, if a new chess engine appears on the scene and it starts out at 3000+ ELO, my first reaction basically is:

"This must be a clone of another engine that range."

I immediately feel like disregarding it and not following any development. I followed / am following the developments of some chess engines that started out really low, such as Stash, Nadya2 and now Wukong and BBC. I'm interested to see what these people who are writing engines from scratch can achieve. (It would even be more interesting if the engines would be in some other programming language than C or C++; in this case it is only Nadya2, in Prolog.)

If you're honest, what is *your* reaction to a new engine that gets onto the scene and starts out at 3000+ ELO?

I just can't believe that people slave away for a year or more, writing an engine from scratch, add huge amounts of functionality, work it up to 3.000 ELO using their own tournaments, and then call it "Just another Chess Engine, version 0.1", throw it onto GitHub, and then abandon it for 5 years or sometimes forever. (And yes, I've seen engines like that, and you probably will have too.) Why would someone put that much work into something and then basically throw it away? Because of that, I treat engines that start out extremely strong with quite some suspicion.

If I create something that is as much work to make as a chess engine from scratch (and it *IS* a lot of work), I want it to be mine, as perfect as I can make it, with fully commented code, as clean as I can get it, and with good documentation... even if the reason would be that if someone down the line wants to "verify that I can program / create software from scratch" ( see (*) ), I can just point them to the chess engine and documentation and tell them "I created that, and I wrote a book on it too. Go study it."

=== (small rant) ===

(*) In IT, it seems to be quite normal nowadays that companies ask for "some software you wrote from scratch in your own time" and hand it over to them with full documentation and instructions on how to compile it, so they can "verify that you can actually write and engineer software from scratch." (Maybe my education and degrees should be proof of that, but apparently not.) I have never heard of companies asking an architect: "Did you build some bridges in your free time? We'd like to see them, so we can verify you can actually engineer a bridge. Better yet, do you have a sky scraper lying around by any chance?"

[MikeB]

<snip>

=== (small rant) ===

(*) In IT, it seems to be quite normal nowadays that companies ask for "some software you wrote from scratch in your own time" and hand it over to them with full documentation and instructions on how to compile it, so they can "verify that you can actually write and engineer software from scratch." (Maybe my education and degrees should be proof of that, but apparently not.) I have never heard of companies asking an architect: "Did you build some bridges in your free time? We'd like to see them, so we can verify you can actually engineer a bridge. Better yet, do you have a sky scraper lying around by any chance?"

I have heard of that - but never actually worked in an organization that employed that method. This is a sign of a poorly performing HR/Recruiting function. The primary purpose of an interview should be to gauge how well that personality will fit in the work group, only those who qualified for the position should have made it that far in the first place. Small rant justified.

[Tony P.]

If you legally fork someone else's engine rated below 3300 and get it into a TCEC superfinal on your own, which certainly requires a lot of creativity, being an uneven battle vs the large SF and Leela teams, then you can likewise call it almost yours and surely not a clone.

The problem is that any successful idea open-sourced under a license compatible with GPL 3.0 can end up pulled into SF or Leela. So the options are to patch either of them instead, or not to open the source, or to deliberately choose an incompatible license like MPL 2.0 with Exhibit B.

In the latter two cases, you can only fork from permissively licensed engines, but there are plenty, some of which are 2900-3300 (Arasan, Gull/Roc, Cheng, Scorpio, FabChess). However, it's strange to work alone on those that are still in active development instead of joining forces with the original author.

[Joost Buijs]

Now, if a new chess engine appears on the scene and it starts out at 3000+ ELO, my first reaction basically is:

"This must be a clone of another engine that range."

I immediately feel like disregarding it and not following any development. I followed / am following the developments of some chess engines that started out really low, such as Stash, Nadya2 and now Wukong and BBC. I'm interested to see what these people who are writing engines from scratch can achieve. (It would even be more interesting if the engines would be in some other programming language than C or C++; in this case it is only Nadya2, in Prolog.)

If you're honest, what is *your* reaction to a new engine that gets onto the scene and starts out at 3000+ ELO?

My feeling about this subject is exactly the same. Developing a strong chess engine from scratch takes several years, even for somebody with experience. Nowadays every few weeks new engines seem to pop up with ratings > 3000 and this simply can not be done without grabbing a lot from other strong open-source engines.

My current engine (the 7th rewrite from scratch) sits after several years of working on it in the region of 3200 CCRL, it started out at 2500 Elo somewhere in (2011 I believe it was). Of course it uses all the basics, PVS with a hashtable, killers, history and LMR, but everything is my own development. It would not be to difficult to add a lot of ideas from Stockfish and add NNUE evaluation to pimp it up 100 or 200 Elo, but I simply don't want to do this because it would give me the feeling that it is not my own engine anymore.

[Dann Corbit]

It would not be to difficult to add a lot of ideas from Stockfish and add NNUE evaluation to pimp it up 100 or 200 Elo, but I simply don't want to do this because it would give me the feeling that it is not my own engine anymore.

There is a word for this. I believe it is called "integrity."

[Tony P.]

It would be interesting to see a differently structured NNUE (with different weights as a result). The fruit of neural architecture optimization must be hanging lower than the search enhancements of 'classical' engines. Given the relative ease of training a small net, it's boring to see it copypasted from SF.

[Wilson]

It would not be to difficult to add a lot of ideas from Stockfish and add NNUE evaluation to pimp it up 100 or 200 Elo, but I simply don't want to do this because it would give me the feeling that it is not my own engine anymore.

There is a word for this. I believe it is called "integrity."

This is an interesting answer Dann but I have a question: if we take a look at the sources of the top open source engines from CCRL or CEGT rating lists, we will see that the search function of every one of them is basically taken from Stockfish with some params tuning. We have, in the same order: Razoring, Reverse Futility Pruning, Null Move Pruning with Verification, ProbCut. Inside the moves loop: Late Move Pruning, Futility Pruning, SEE Pruning, Singular Extension, Late Move Reduction and so on. All with more or less the same implementation details, also for ProbCut for which papers have been published several years ago but everyone does it the Stockfish way. If we had a similarity tester for the search function, I believe we would have a similarity much higher than 70%.

So, my question: does that mean that these engines lack integrity?

Few months ago, I opened a thread asking what we were allowed to learn/take from open source engines (basically Stockfish since it's the strongest one) and I used the Singular Extension as example since that was the algorithm I was toying with at the time. In the end, I decided not to use it because I couldn't find an implementation different enough from SF's one that wasn't significantly weaker. If I had implemented SE the Stockfish way just to gain ELO points, then probably I would have went on taking things here and there and my engine wouldn't have been mine anymore. So, I prefer it to be relatively weak and to try new things. If my search was like everyone's else, I think the probability to find something new that works would be much lower.

[Henk]

Code of my engine must be original. Can't imagine anyone writing such bad code with such stupid ideas.

[Gabor Szots]

Clone: appearing suddenly, no significant progress later (or altogether abandoned soon).
A variation of this: sudden big improvement after a time, never gradually. Shows the author has no own ideas, maybe not even thoroughly understanding the code used.
Not necessarily very strong, code has been know to be stolen from weaker engines too.

I guess many times students do this to brag about making 'their' engine appear on the rating lists.

[Dann Corbit]

So, my question: does that mean that these engines lack integrity?

Let's start with all of those methods being well known for years, and nobody owns them.
So my answer is this:
If you achieved any of those goals while violating the license agreement of any code then you lack integrity.
If you read the code written by others, understood it, and wrote your own version, there is nothing wrong with that[*]

[*] though others would beg to differ. I have found that chess programmers are very jealous about their evaluation function, despite the fact that each and every term in their program was learned from chess books, chess experts, and chess programming articles. Personally, I do not see using ideas as wrong at all, but I suppose that at some point if you are just echoing the work of others it seems to lack creativity to me.