Similarity tests

[Laskos]

With swans it's a bit different. All black swans I saw were dyed former white swans. No naturally black swans were observed. If I see again a black swan, the reasonable assumption is that it's a former white swan.

http://en.wikipedia.org/wiki/Black_swan

That is the problem with assumptions!


Bye
Ingo

What problem? If Yeti wasn't observed, and all supposed Yeties were in fact bears, it's a scientific hypothesis needed to be falsified that Yeties don't exist. By your arguments all empirical data is worthless.

[Uri Blass]

Reading recent posts I have got the impression that the similarity tool is regarded as a reliable tool for deciding if an engine is original or not. As far as I can remember, at its birth it was stated expressly that on its own it is not suitable for that purpose.
This is just a reminder to the community not to commit the error of judging everything by this tool alone.

It could give false negatives, as Ed showed, but I have not seen any false positive. So:

1/ If it passes Sim test, that may mean nothing.
2/ If it doesn't pass the Sim test, that means it's a clone or a derivative.

Of course you do not see any false positive because if you see positive you assume that it is a true positive.

There is no way to refute the claim that there are no false positives if you assume that every positive is a clone or a derivative.

How do you prove that B is not a derivative of A?

By other circumstantial evidence. All the positives, were positives of open source Fruit, after Fruit, positives of open source Strelka, after Strelka, positives of open source Ippo, after Ippo, positives pf open source SF, after SF.

There is no benefit of doubt in these cases. Show me a single closed source engine, which, being prior to the open source one different engine, is a positive with that later open source engine.

I wonder what is the evidence against naum except move similarity to strelka.

[IWB]

With swans it's a bit different. All black swans I saw were dyed former white swans. No naturally black swans were observed. If I see again a black swan, the reasonable assumption is that it's a former white swan.

http://en.wikipedia.org/wiki/Black_swan

That is the problem with assumptions!


Bye
Ingo

What problem? If Yeti wasn't observed, and all supposed Yeties were in fact bears, it's a scientific hypothesis needed to be falsified that Yeties don't exist. By your arguments all empirical data is worthless.

No, my argument was that you just can't be that sure about false positives. You answered with some arguments against lack swans and I proved you wrong. Ergo you should not be that sure about false positves.
Even I said,, that with a positive you have to do further investigation and should not rely on the Sim test.
Emirical data is fine, but how many did you test? 50? 100? That's nothing.

(My doubt is because in my job I do have (for different reasons) a false positive from time to time - and that is a bad thing!)

Bye
Ingo

[Laskos]

With swans it's a bit different. All black swans I saw were dyed former white swans. No naturally black swans were observed. If I see again a black swan, the reasonable assumption is that it's a former white swan.

http://en.wikipedia.org/wiki/Black_swan

That is the problem with assumptions!


Bye
Ingo

What problem? If Yeti wasn't observed, and all supposed Yeties were in fact bears, it's a scientific hypothesis needed to be falsified that Yeties don't exist. By your arguments all empirical data is worthless.

No, my argument was that you just can't be that sure about false positives. You answered with some arguments against lack swans and I proved you wrong. Ergo you should not be that sure about false positves.
Even I said,, that with a positive you have to do further investigation and should not rely on the Sim test.
Emirical data is fine, but how many did you test? 50? 100? That's nothing.

(My doubt is because in my job I do have (for different reasons) a false positive from time to time - and that is a bad thing!)

Bye
Ingo

About black swans I just made the assumption that they are all dyed white swans. Sure, I have 300 meters from me a lake with black swans, it was just an assumption.

Prove just one false positive in Sim, and all my theory is in shambles

[Rebel]

Reading recent posts I have got the impression that the similarity tool is regarded as a reliable tool for deciding if an engine is original or not. As far as I can remember, at its birth it was stated expressly that on its own it is not suitable for that purpose.
This is just a reminder to the community not to commit the error of judging everything by this tool alone.

It could give false negatives, as Ed showed, but I have not seen any false positive. So:

1/ If it passes Sim test, that may mean nothing.
2/ If it doesn't pass the Sim test, that means it's a clone or a derivative.

I believe that you haven't seen a false positive, but that doesn't mean there are non. (Because you haven't seen a black swan it doesnt mean there are non.)

It's extremely unlikely.

Sim test uses 8238 positions most of them with a balanced (non-forcing) nature where 3-4 reasonable good moves are available. The 8238 number works as a filter to weed out randomness.

Look at the below link when we reduce the 8238 number in steps to 7500 | 6000 | 4000 | 2000 | 1000 and 500 (follow the links in the menu left) and notice that even using 500 positions already gives a reasonable result, the highest measured fluctation is below 5%. With 4000 positions everything is below 1.5% which is a pretty much neglectable number considering what we are testing.

http://www.top-5000.nl/sim-7500.htm

[Rebel]

Reading recent posts I have got the impression that the similarity tool is regarded as a reliable tool for deciding if an engine is original or not. As far as I can remember, at its birth it was stated expressly that on its own it is not suitable for that purpose.
This is just a reminder to the community not to commit the error of judging everything by this tool alone.

It could give false negatives, as Ed showed, but I have not seen any false positive. So:

1/ If it passes Sim test, that may mean nothing.
2/ If it doesn't pass the Sim test, that means it's a clone or a derivative.

Zactly.

[Graham Banks]

Reading recent posts I have got the impression that the similarity tool is regarded as a reliable tool for deciding if an engine is original or not. As far as I can remember, at its birth it was stated expressly that on its own it is not suitable for that purpose.
This is just a reminder to the community not to commit the error of judging everything by this tool alone.

It could give false negatives, as Ed showed, but I have not seen any false positive. So:

1/ If it passes Sim test, that may mean nothing.
2/ If it doesn't pass the Sim test, that means it's a clone or a derivative.

Zactly.

So you'd use the 60% as a pretty definitive indication?

[Rebel]

Reading recent posts I have got the impression that the similarity tool is regarded as a reliable tool for deciding if an engine is original or not. As far as I can remember, at its birth it was stated expressly that on its own it is not suitable for that purpose.
This is just a reminder to the community not to commit the error of judging everything by this tool alone.

It could give false negatives, as Ed showed, but I have not seen any false positive. So:

1/ If it passes Sim test, that may mean nothing.
2/ If it doesn't pass the Sim test, that means it's a clone or a derivative.

Zactly.

So you'd use the 60% as a pretty definitive indication?

In general one can say that 60% and higher is a sign of non-originality, 65% and higher being a strong sign of cloning, that the grey area is from 55-60% and any engine below 55% has passed the test, meaning just a first indication of originality. meaning nothing in the end.

[Graham Banks]

Reading recent posts I have got the impression that the similarity tool is regarded as a reliable tool for deciding if an engine is original or not. As far as I can remember, at its birth it was stated expressly that on its own it is not suitable for that purpose.
This is just a reminder to the community not to commit the error of judging everything by this tool alone.

It could give false negatives, as Ed showed, but I have not seen any false positive. So:

1/ If it passes Sim test, that may mean nothing.
2/ If it doesn't pass the Sim test, that means it's a clone or a derivative.

Zactly.

So you'd use the 60% as a pretty definitive indication?

In general one can say that 60% and higher is a sign of non-originality, 65% and higher being a strong sign of cloning, that the grey area is from 55-60% and any engine below 55% has passed the test, meaning just a first indication of originality. meaning nothing in the end.

I wonder whether or not somebody has a pretty much up to date chart or dendogram.

[Sedat Canbaz]

Just my 2 cents over this issue,

I mean let's say I changed SCCT's current rule from 55 % to 60% limit and above
And plus I don't request no any +100 Elo improvement over based original engine....

Then I am afraid in that,
Mostly of the below engines should be definitely as SCCT top 20 participants
And only picking Naum or Fire or Critter...
Then there will be BIG double standard over the rest ones (I mean for the current below player's list)!!!

So...I have a question to all chess engine experts,
I mean for those who believes that the limit should be minimum 60 %
Is it good idea or not ???

Then I am afraid that almost all SCCT's Top 20 will be Fruitians/Rybkanians/Stockfishians !!!
That's why....SCCT's rule 55 % limitation + 100 Elo requesting is ideal ... !!!
Correct me please if I am wrong....

For example,
Here are a few ones (which mostly of them are between 55-59% comparing to Rybka):

Belka
Bouquet
BlackMamba
Critter
Cyclone xTreme
Equinox
Fritz (latest versions)
Fire
Heron
Houdini
Igorrit
IPPOLIT
Ivanhoe
LEOpard
Loop
Murka
Naum
Onno
RobboLito
Strelka
Tankist
Twinfish
Toga
Vitruvius

Note: only Houdini is different than all, and it should be counted as original work,
because R. Houdart was the 1 guy who managed to improve his engine much stronger than Rybka !!
Of course the authors of Stockfish,Komodo,Gull,Rybka are another great talents too
Actually all rest engine programmers who work mainly with own original ideas are another great talents too

And without to not mention this I cant,
Arasan seems to be the most original in top 20, my congratulations to Jon Dart !

Really that means a lot for me....

If anybody will say my view is wrong...its ok... no problem
Then I plan to participate with CM9 Gladiator too, but I will rename it to BabyMaster)))


And last,
I hope my current posting will be useful ....


Best,
Sedat