Adam,
I scanned the article. Nice.
My thoughts concern: "There are 4851 pairs. The average matched move percentage was 45.16, the standard deviation of the data was 2.86. "
Looks like you included known clones/derivatives in the calculation of the standard deviation. I think their inclusion biases the standard deviation. Of course, it makes it smaller. Shouldn't the standard deviation be calculated without the known clones? It would make the number larger (maybe only slightly).
Adam Hair's article on Pairwise comparison of engines
Moderators: hgm, Rebel, chrisw
-
- Posts: 2056
- Joined: Mon Mar 13, 2006 2:31 am
- Location: North Carolina, USA
-
- Posts: 12542
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Adam Hair's article on Pairwise comparison of engines
In order to do that you have to presuppose the result you hope to establish (assuming that you do not really know every single clone).CRoberson wrote:Adam,
I scanned the article. Nice.
My thoughts concern: "There are 4851 pairs. The average matched move percentage was 45.16, the standard deviation of the data was 2.86. "
Looks like you included known clones/derivatives in the calculation of the standard deviation. I think their inclusion biases the standard deviation. Of course, it makes it smaller. Shouldn't the standard deviation be calculated without the known clones? It would make the number larger (maybe only slightly).
An interesting experiment would be to take known clones and see the match rate and standard deviations.
A further interesting experiment would be to find the highest match rate and standard deviations for engines known definitely not to be clones.
Without doing these experiments, I wonder what the controls are.
-
- Posts: 3226
- Joined: Wed May 06, 2009 10:31 pm
- Location: Fuquay-Varina, North Carolina
Re: Adam Hair's article on Pairwise comparison of engines
Thanks, Charles.CRoberson wrote:Adam,
I scanned the article. Nice.
Actually, their inclusion makes the standard deviation from the mean larger.CRoberson wrote: My thoughts concern: "There are 4851 pairs. The average matched move percentage was 45.16, the standard deviation of the data was 2.86. "
Looks like you included known clones/derivatives in the calculation of the standard deviation. I think their inclusion biases the standard deviation. Of course, it makes it smaller. Shouldn't the standard deviation be calculated without the known clones? It would make the number larger (maybe only slightly).
-
- Posts: 2041
- Joined: Wed Mar 08, 2006 8:30 pm
Re: Adam Hair's article on Pairwise comparison of engines
Would be nice to know what article !!!CRoberson wrote: I scanned the article.
-
- Posts: 558
- Joined: Sat Mar 25, 2006 8:27 pm
-
- Posts: 2041
- Joined: Wed Mar 08, 2006 8:30 pm
-
- Posts: 41473
- Joined: Sun Feb 26, 2006 10:52 am
- Location: Auckland, NZ
Re: Adam Hair's article on Pairwise comparison of engines
Interesting to note that Rybka and Fruit don't show up as a pair to be suspicious about.Robert Pope wrote:I believe it is: http://www.top-5000.nl/clone.htm
gbanksnz at gmail.com
-
- Posts: 6997
- Joined: Thu Aug 18, 2011 12:04 pm
- Full name: Ed Schröder
Re: Adam Hair's article on Pairwise comparison of engines
It's and indication but not more than that. The SYM tool pretty much with precision can detect a clone and even a close derivative. It can not proof an engine is clean.Graham Banks wrote:Interesting to note that Rybka and Fruit don't show up as a pair to be suspicious about.Robert Pope wrote:I believe it is: http://www.top-5000.nl/clone.htm
-
- Posts: 782
- Joined: Wed Mar 08, 2006 9:22 pm
Re: Adam Hair's article on Pairwise comparison of engines
Graham Banks wrote:Interesting to note that Rybka and Fruit don't show up as a pair to be suspicious about.Robert Pope wrote:I believe it is: http://www.top-5000.nl/clone.htm
Hello Graham,
You should be cautious when referring to other works.
The information presented doesn't attempt to validate or invalidate suspicions, so it really isn't as interesting as you think it is.
Later
The need to avoid false accusation is greater than the need to determine authors who break the rules slightly. In other words, it is better to let lesser offenders slip through than to make accusations against innocent authors.
This tool should not be used solely for determining derivatives and clones. Other methods should be used in conjunction with this tool. Ultimately, any accusation of cloning requires an examination of the code of the accused author.
-
- Posts: 1056
- Joined: Thu Mar 09, 2006 4:15 pm
- Location: Long Island, NY, USA
Re: Adam Hair's article on Pairwise comparison of engines
Adam,
Would it be unreasonable to test a 2nd copy of each engine to establish consistency?
I think you need to have some assurance that each engine is strongly consistent and will produce the same move from the same position the great majority of the time if given a 2nd chance.
After running a second version through the 8000+ positions, the 2 versions of each engine can be compared. If the versions don't have at least 95% matched moves, then I would not consider the engine consistent and possible disqualify the engine from the test.
-Norm
Would it be unreasonable to test a 2nd copy of each engine to establish consistency?
I think you need to have some assurance that each engine is strongly consistent and will produce the same move from the same position the great majority of the time if given a 2nd chance.
After running a second version through the 8000+ positions, the 2 versions of each engine can be compared. If the versions don't have at least 95% matched moves, then I would not consider the engine consistent and possible disqualify the engine from the test.
-Norm