TalkChess.com

Recently (after 5 years) I tested the current top-engines on similarity (once again) because I wanted to know 2 things:

1. if the crazy days of the Fruit cloning (2005-2010) and Rybka3 cloning (2010-2015) were over and how the situation anno 2019 is now that Stockfish is the ideal target. And so I ran SIM3 from Don Daily checking the CCRL TOP-10. And the results were amazingly good, as it seems computer chess healed itself.

#2. I also wanted to know something else, since the immense elo progress that have been made the last 5 years, is it not likely that the top-engines increase in similarity? And if that is the case then: how much. In other words, a test if SIM3 is still a reliable tool. And looking at the situation of 2019 the numbers are looking good, the top-engines nicely curdle between 45-50%. So far so good.

And then something somewhat strange happened, as last 3 engines I added the latest versions of Fire, Booot and Fizbo to the test and Fizbo showed an incredible low similarity of 24% average. That's not necessarily wrong but a bit strange it is for 3300+ elo rated engine.

And I decided to have a closer look, long story short, I found a small (but mean) bug in SIM3. In Fail-high situations it not always recorded the right move. I fixed the bug and ran the test again and the result is remarkable, softly speaking that is, view the comparison between SIM3 and SIM4. Suddenly according to the legenda Xiphos with 64% is a clone / derivative of Stockfish, even Komodo with 61% is in the red zone. High similarity between the top-engines. And of course they are not clones / derivatives of each other. Several are open source, easy to compare.

As next step I ran several engines from the Rybka3 cloning (2010-2015) period to see how they would perform with the corrected version. And suddenly they are no clones / derivatives any longer! View the comparison, similarity is about in the same range as today.

This begs the question if programmers have been wrongly accused of cloning in the past because of a bug in SIM3 ??

Page including links and download - http://rebel13.nl/home/sim.html

Interesting discovery Ed. Lets try it.

Hmm, and the calendar says what?

Very very interesting!

Thanks Ed.

Rebel wrote: ↑Mon Apr 01, 2019 6:53 am This begs the question if programmers have been wrongly accused of cloning in the past because of a bug in SIM3 ??

Page including links and download - http://rebel13.nl/home/sim.html

I tried to log sim4.1 output here is the difference to my analyzer.
1. sim4.1 does not send ucinewgame command or restart the engine. In my analyzer engine is restarted after every position has been analyzed.

2. sim4.1 sends go depth <depth> and stop commands, this has a drawback, it might happen that after sending stop, some engines may extend more time than others and you save the bestmove sent by engine. In my analyzer I use go movetime <timems> then save the bestmove sent by engine. This is more reliable as I am not interrupting the search of the engine. But this would only work for those engines that complies the movetime command which the majority of strong engines implemented correctly. go movetime does not work for Fizbo, so I use go infinite then send stop if analysis time is reached. The way I collect the bestmove is thru the pv sent by the engine. That pv line should not have upperbound and lowerbound text in it. After the stop command the engine would send bestmove bm, but I will not use that bm, I use the first move in the pv at a time that I sent the stop command, to comply with the analysis time specified by the user.

Perhaps these differences may affect the overall results of the tests.

Using sim4.1 here are my results.
I ran Stockfish 10 twice at 200ms to establish a base line. This should be the highest similarity at 64.53.
1. Stockfish 10
2. Stockfish 10 r1

Result:
Now with this sim version we get the highest similarity of 64.53 as 100% clone, on my computer with that given analysis time and 1 thread per engine with 128 MB hash.

Rebel wrote: ↑Mon Apr 01, 2019 6:53 am ... Several are open source, easy to compare...

Even if the engine sources are different, they use more and more the same known to work techniques. I expect them to be more and more equal in the best lines that they find. So they are not clones or derivatives per se, but are partial derivatives in term of the ideas they use. Andscacs also, of course, even if I try to do everything in my own way.

Hi Ed

Is there any possibility to edit the positions of the tool and add or remove positions?

Thank you

Sergio Martinez wrote: ↑Tue Apr 02, 2019 9:03 am Hi Ed

Is there any possibility to edit the positions of the tool and add or remove positions?

Thank you

Yes, but you need a special version of SIM3 including the source code. I am not allowed to share it.

Rebel wrote: ↑Tue Apr 02, 2019 11:32 am

Sergio Martinez wrote: ↑Tue Apr 02, 2019 9:03 am Hi Ed

Is there any possibility to edit the positions of the tool and add or remove positions?

Thank you

Yes, but you need a special version of SIM3 including the source code. I am not allowed to share it.

Ok.

Thanks.