Adam Hair's article on Pairwise comparison of engines

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

User avatar
Rebel
Posts: 6997
Joined: Thu Aug 18, 2011 12:04 pm
Full name: Ed Schröder

Re: Adam Hair's article on Pairwise comparison of engines

Post by Rebel »

Roger Brown wrote:
Graham Banks wrote:
Robert Pope wrote:I believe it is: http://www.top-5000.nl/clone.htm
Interesting to note that Rybka and Fruit don't show up as a pair to be suspicious about.

Hello Graham,

You should be cautious when referring to other works.

The information presented doesn't attempt to validate or invalidate suspicions, so it really isn't as interesting as you think it is.

Later


The need to avoid false accusation is greater than the need to determine authors who break the rules slightly. In other words, it is better to let lesser offenders slip through than to make accusations against innocent authors.

This tool should not be used solely for determining derivatives and clones. Other methods should be used in conjunction with this tool. Ultimately, any accusation of cloning requires an examination of the code of the accused author.
Funny that you address Graham, not me. What I said goes a lot further than an observation Graham made looking at the data.

As for your comment that the SYM tool "isn't as interesting as you think it is" than know it is used by the CSVN to validate a participant, even in use by Chessbase, in use by CC lovers who like to play engine-engine tournaments. The SYM tool is a gift from heaven (Don perrhaps smiles now) in a chaotic time. It only needs to be used wise.
Henk
Posts: 7220
Joined: Mon May 27, 2013 10:31 am

Re: Adam Hair's article on Pairwise comparison of engines

Post by Henk »

In order to create a good clone just make some small changes to pass the similarity test. This should be part of the course "How to create an original chess engine in one day".

Can someone tell us which part of Stockfish code we need to modify to do this. Modifying only some parameters or constants would be best for we can't assume everybody can program in C++. We probably also need to know how to compile.
Ferdy
Posts: 4833
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: Adam Hair's article on Pairwise comparison of engines

Post by Ferdy »

Henk wrote:In order to create a good clone just make some small changes to pass the similarity test. This should be part of the course "How to create an original chess engine in one day".
:) but nowadays creating an engine is as easy as compiling the source code, and then release the source and binary.
Some people like it especially if it is strong, then they find a sparring partner to it that they know is no match, after which publish the results telling that
the new engine is very strong. It is even one of the top of the rating list :D, which is reasonable because it is indeed strong.
Henk
Posts: 7220
Joined: Mon May 27, 2013 10:31 am

Re: Adam Hair's article on Pairwise comparison of engines

Post by Henk »

I think best way to detect whether it is a clone or not is to look at how much time it took to develop it or ask for (much) older versions of the chess engine source code.

Next module for the course would be about 'How to create an original chess engine in one day that plays human like chess' and 'magicians best tricks to remote control chess engines'
Adam Hair
Posts: 3226
Joined: Wed May 06, 2009 10:31 pm
Location: Fuquay-Varina, North Carolina

Re: Adam Hair's article on Pairwise comparison of engines

Post by Adam Hair »

Roger Brown wrote:
Graham Banks wrote:
Robert Pope wrote:I believe it is: http://www.top-5000.nl/clone.htm
Interesting to note that Rybka and Fruit don't show up as a pair to be suspicious about.

Hello Graham,

You should be cautious when referring to other works.

The information presented doesn't attempt to validate or invalidate suspicions,
Correct.
Roger Brown wrote: so it really isn't as interesting as you think it is.

Later
What?!
:lol:
Roger Brown
Posts: 782
Joined: Wed Mar 08, 2006 9:22 pm

Re: Adam Hair's article on Pairwise comparison of engines

Post by Roger Brown »

Rebel wrote:
Roger Brown wrote:
Graham Banks wrote:
Robert Pope wrote:I believe it is: http://www.top-5000.nl/clone.htm
Interesting to note that Rybka and Fruit don't show up as a pair to be suspicious about.

Hello Graham,

You should be cautious when referring to other works.

The information presented doesn't attempt to validate or invalidate suspicions, so it really isn't as interesting as you think it is.

Later


The need to avoid false accusation is greater than the need to determine authors who break the rules slightly. In other words, it is better to let lesser offenders slip through than to make accusations against innocent authors.

This tool should not be used solely for determining derivatives and clones. Other methods should be used in conjunction with this tool. Ultimately, any accusation of cloning requires an examination of the code of the accused author.
Funny that you address Graham, not me. What I said goes a lot further than an observation Graham made looking at the data.

As for your comment that the SYM tool "isn't as interesting as you think it is" than know it is used by the CSVN to validate a participant, even in use by Chessbase, in use by CC lovers who like to play engine-engine tournaments. The SYM tool is a gift from heaven (Don perrhaps smiles now) in a chaotic time. It only needs to be used wise.

Hello Ed,

You seem to be reading something into my comment that is not there.

I have no doubt that the tool is useful, even without knowing the CSVN or Chessbase stories you indicated.

My point was limited to indicating that the conclusions drawn from its application needed careful analysis.

In fact, in making that point, I used the material from your site, in quotes no less.

I am not saying anything materially different from yourself.

Graham's comments are made within a context.

As are mine.

Later.
Roger Brown
Posts: 782
Joined: Wed Mar 08, 2006 9:22 pm

Re: Adam Hair's article on Pairwise comparison of engines

Post by Roger Brown »

Adam Hair wrote:
Roger Brown wrote:
Graham Banks wrote:
Robert Pope wrote:I believe it is: http://www.top-5000.nl/clone.htm
Interesting to note that Rybka and Fruit don't show up as a pair to be suspicious about.

Hello Graham,

You should be cautious when referring to other works.

The information presented doesn't attempt to validate or invalidate suspicions,
Correct.
Roger Brown wrote: so it really isn't as interesting as you think it is.

Later
What?!
:lol:

Hello Adam,

See my reply to Ed elsewhere.

Gee, you computer-chess guys are sooooooooooo sensitive.

:shock:

Later.
Adam Hair
Posts: 3226
Joined: Wed May 06, 2009 10:31 pm
Location: Fuquay-Varina, North Carolina

Re: Adam Hair's article on Pairwise comparison of engines

Post by Adam Hair »

Roger Brown wrote:
Roger Brown wrote: so it really isn't as interesting as you think it is.

Later
Adam Hair wrote: What?!
:lol:

Hello Adam,

See my reply to Ed elsewhere.

Gee, you computer-chess guys are sooooooooooo sensitive.

:shock:

Later.
Am not :evil:

:wink:
User avatar
Rebel
Posts: 6997
Joined: Thu Aug 18, 2011 12:04 pm
Full name: Ed Schröder

Re: Adam Hair's article on Pairwise comparison of engines

Post by Rebel »

Roger Brown wrote:
Adam Hair wrote:
Roger Brown wrote:
Graham Banks wrote:
Robert Pope wrote:I believe it is: http://www.top-5000.nl/clone.htm
Interesting to note that Rybka and Fruit don't show up as a pair to be suspicious about.

Hello Graham,

You should be cautious when referring to other works.

The information presented doesn't attempt to validate or invalidate suspicions,
Correct.
Roger Brown wrote: so it really isn't as interesting as you think it is.

Later
What?!
:lol:

Hello Adam,

See my reply to Ed elsewhere.

Gee, you computer-chess guys are sooooooooooo sensitive.

:shock:

Later.
I think the sensitive part started when you criticized Graham for asking a perfect sensible question because of course the SYM tool is all about suspicion, the reason why Don wrote the thing.

And about Adam, I think that guy is totally indifferent :wink: but you gotta love him anyway.
Adam Hair
Posts: 3226
Joined: Wed May 06, 2009 10:31 pm
Location: Fuquay-Varina, North Carolina

Re: Adam Hair's article on Pairwise comparison of engines

Post by Adam Hair »

Rebel wrote: And about Adam, I think that guy is totally indifferent :wink: but you gotta love him anyway.
Maybe a bit irreverent, but not necessarily indifferent.

I know that you know all of the following, Ed. My response is for others who may read this thread:

Two people that I have a great amount of respect for have looked at the similarity data and reached two opposite conclusions. Based on phylogenetic analysis, Miguel Ballicora concluded that the similarity data does not show an unreasonable amount of similarity between Rybka 1.0 Beta and Fruit 2.1. Based on a refinement of my methods, Mark Watkins found that early Rybka versions (starting with Rybka 1.0 Beta) "show abnormally large move-matching with Fruit 2.1" (found in Move similarity analysis in chess programs - preprint)

Personally, I think that the similarity data shows that Fruit influenced Rybka, and that is all that it shows. If a person was looking for indications that Rybka was copied from Fruit, the data would make them more suspicious. If a person was looking for clones and close derivatives, then they would ignore Rybka/Fruit and look at other pairs of engines (start with Ed's discovery of Loop 2007, Toga 1.1a, and Fruit 2.2.1).