When will we see HOUDINI in official tournaments?

[Mike S.]

As after all the similarity software is meant to measure the similarity of the evaluation function.

But that's disturbing, because I assume - although not being a programmer - that currently much more software engineering intelligence is in the search code, not in the evaluations. I may be wrong but usually my guesses are good.

There was a document publically available (not from the original engine authors) which contained detailled analysis and descriptions of every evaluation term of the engine Toga. Insiders will be aware; I didn't keep the download link. It was comprehensive.

Does it make sense to base a similarity analysis on evaluation only, disregarding search code? To me this seems to be flawed, if being used to exclude engines from tournaments based on percentages. Eval may by 70% similar, search (not tested) may be MUCH less similar.

Maybe this needs to be reconsidered, but of course I am only talking from a spectator's viewpoint. Thanks for hearing me.

[IWB]

Ingo, Robolito is an Ippolit clone, thus not allowed. If an engine gives just one 60% hit with any other engine that engine is not allowed to play.

Still I doubt the relevance but then the CSVN allow Critter and Robo 0.85 would be allowed as it is below 60% to R3. If you claime R085 is too close to a Litto so is a Rybka ... Additionaly I still miss a R4/4.1 comparisons.
OK, you assuem the participating Critter will be below 60% but will the CVSN test ALL participating engines and how will they do that in case of a cluster engine ...
A law which can not be enforced equaly is usually considered useless and if you enforce it to others it is pure arbitrary! Right now it sounds like a LEX-RYBKA to me and not a "We don't want cheaters" rule!

I personaly think the concept is wrong. The decision the CVSN made about Rybka forces them to allow everything or nothing. Everything else is plainly wrong!

Bye
Ingo

PS: No one should get this wrong! I do not blame Critter of anything, actually I don't mind. I only want to show the absurdity of the tool-rule!

[BubbaTough]

If an engine gives just one 60% hit with any other engine that engine is not allowed to play.

I would assume if two engines have a 60+% match, then the first one of them that was published would still be allowed to play. It makes no sense for a derivative to force the original to be banned. A few other amendments that would make sense is if the match is with a program by the same author that is permissible, or if the first published engine gives permission to the other engine to play all is good.

It does sounds like if the 60% rule is actually in place, that Critter would have to get Houdini's permission to play, as Houdini1.5 was published before Critter1.4 I believe (unless as Graham said the version entered is not 1.4, and does fail the criteria). I haven't seen any indication that it is in place though. I kind of like the rule myself in that it is very concrete, authors can test it themselves, and it does not require sharing precious source code. Perhaps I wouldn't call it clone detection...some name like the "sufficient diversity" rule might carry less stigma. I would guess some of the Houdini disdain for the technique is from the close match of RobboLito 0.085d1 and Houdini 1.00, which carries massive negative implications in this community. If such matches did not imply lifelong disdain for the author and all his future work, and instead just implied the author had some work to do before he could enter tournaments, it might not receive such strong negative reactions from certain authors. After all, its widely acknowledged that a strong match does not ensure wrongdoing has been done. But it also makes sense that tournaments would want entrants that do not always pick the same moves.

-Sam

[Graham Banks]

............But it also makes sense that tournaments would want entrants that do not always pick the same moves.

-Sam

Indeed! Makes it much more interesting from a spectator's point of view.

[chrisw]

............But it also makes sense that tournaments would want entrants that do not always pick the same moves.

-Sam

Indeed! Makes it much more interesting from a spectator's point of view.

My solution would be to let everybody play on a giant all play all basis and then the warring factions (evangelists, left anti-capitalist, devil worshippers, technocrats, the bored, Hyatt fans, Vas fans, Semites and anti-Semites) can battle it out as to which of the many possible results cross tables is "gods own cross table". Politically incorrect cross tables can be sent to a new secret forum UCT (unmentionable cross tables).

[Uri Blass]

Ingo, Robolito is an Ippolit clone, thus not allowed. If an engine gives just one 60% hit with any other engine that engine is not allowed to play.

1) Houdini 1.0 (time: 100 ms scale: 1.0)
2) Houdini 1.5 (time: 100 ms scale: 1.0)
3) IPPOLIT 0.080a (time: 100 ms scale: 1.0)
4) RobboLito 0.09 (time: 100 ms scale: 1.0)
5) Strelka 5 (time: 100 ms scale: 1.0)

Code: Select all

        1     2     3     4     5
 1.  ----- 63.27 67.27 71.07 63.66
 2.  63.27 ----- 58.87 61.68 66.79
 3.  67.27 58.87 ----- 69.31 60.56
 4.  71.07 61.68 69.31 ----- 62.37
 5.  63.66 66.79 60.56 62.37 -----

Ed,How can authors check it?

Suppose A and B are private engines that participate in this tournament.

How can the authors check that the similiarity between their engines is not more than 60% when noone of them has access to the engine of the second author?

Even if we test only similiarity to public engines then there are many public engines and I do not think that you can expect an author to buy all the commercial versions and the commercial versions of the past to test that their engine has no more than 60% similiarity to Fritz5 or to Junior8 or to Genius3.

[Uri Blass]

I can add that I wonder if it is possible to get a list of original and non original program based on the 60%

Note that I can imagine the following case.

engine X.1 is original

someone copy engine X.1 but is careful to do enough changes to get similiarity of 59.9% and call his engine Y

Later the author of engine X make a small change and has version X.2
that he want to participate with it in a tournament.

After comparing X.2 with Y we find similiarity of 60.1% so people tell the author of engine X.1 that he is not allowed to participate in a tournament with X.2

I do not think that it is fair.

[michiguel]

Ingo, Robolito is an Ippolit clone, thus not allowed. If an engine gives just one 60% hit with any other engine that engine is not allowed to play.

1) Houdini 1.0 (time: 100 ms scale: 1.0)
2) Houdini 1.5 (time: 100 ms scale: 1.0)
3) IPPOLIT 0.080a (time: 100 ms scale: 1.0)
4) RobboLito 0.09 (time: 100 ms scale: 1.0)
5) Strelka 5 (time: 100 ms scale: 1.0)

Code: Select all

        1     2     3     4     5
 1.  ----- 63.27 67.27 71.07 63.66
 2.  63.27 ----- 58.87 61.68 66.79
 3.  67.27 58.87 ----- 69.31 60.56
 4.  71.07 61.68 69.31 ----- 62.37
 5.  63.66 66.79 60.56 62.37 -----

Ed,How can authors check it?

Suppose A and B are private engines that participate in this tournament.

How can the authors check that the similiarity between their engines is not more than 60% when noone of them has access to the engine of the second author?

Even if we test only similiarity to public engines then there are many public engines and I do not think that you can expect an author to buy all the commercial versions and the commercial versions of the past to test that their engine has no more than 60% similiarity to Fritz5 or to Junior8 or to Genius3.

I don't know how this is planned to be implemented, but it is not difficult to imagine the possible scenario: You do not need the engines to check this. You need the list of moves for a set of engines for a given set of positions. That will give you an idea if you are ok. The final test could be done with a secret set by the organization.

Miguel

[Harvey Williamson]

Ingo, Robolito is an Ippolit clone, thus not allowed. If an engine gives just one 60% hit with any other engine that engine is not allowed to play.

1) Houdini 1.0 (time: 100 ms scale: 1.0)
2) Houdini 1.5 (time: 100 ms scale: 1.0)
3) IPPOLIT 0.080a (time: 100 ms scale: 1.0)
4) RobboLito 0.09 (time: 100 ms scale: 1.0)
5) Strelka 5 (time: 100 ms scale: 1.0)

Code: Select all

        1     2     3     4     5
 1.  ----- 63.27 67.27 71.07 63.66
 2.  63.27 ----- 58.87 61.68 66.79
 3.  67.27 58.87 ----- 69.31 60.56
 4.  71.07 61.68 69.31 ----- 62.37
 5.  63.66 66.79 60.56 62.37 -----

Ed,How can authors check it?

Suppose A and B are private engines that participate in this tournament.

How can the authors check that the similiarity between their engines is not more than 60% when noone of them has access to the engine of the second author?

Even if we test only similiarity to public engines then there are many public engines and I do not think that you can expect an author to buy all the commercial versions and the commercial versions of the past to test that their engine has no more than 60% similiarity to Fritz5 or to Junior8 or to Genius3.

I don't know how this is planned to be implemented, but it is not difficult to imagine the possible scenario: You do not need the engines to check this. You need the list of moves for a set of engines for a given set of positions. That will give you an idea if you are ok. The final test could be done with a secret set by the organization.

Miguel

Only if all engines are supplied to the CSVN. How can they possibly test Rybka Cluster? Although the rules say the exe should be attached to the entry form?! This is a ridiculous request as many work on the engine right up to the start and during the tournament.

[Rebel]

Ingo, Robolito is an Ippolit clone, thus not allowed. If an engine gives just one 60% hit with any other engine that engine is not allowed to play.

Still I doubt the relevance but then the CSVN allow Critter and Robo 0.85 would be allowed as it is below 60% to R3. If you claime R085 is too close to a Litto so is a Rybka ... Additionaly I still miss a R4/4.1 comparisons.
OK, you assuem the participating Critter will be below 60% but will the CVSN test ALL participating engines and how will they do that in case of a cluster engine ...
A law which can not be enforced equaly is usually considered useless and if you enforce it to others it is pure arbitrary! Right now it sounds like a LEX-RYBKA to me and not a "We don't want cheaters" rule!

I personaly think the concept is wrong. The decision the CVSN made about Rybka forces them to allow everything or nothing. Everything else is plainly wrong!

Bye
Ingo

PS: No one should get this wrong! I do not blame Critter of anything, actually I don't mind. I only want to show the absurdity of the tool-rule!

I have the feeling you haven't got the point of the system. Here is an example for demonstration.

1. ProDeo 1.74 (default setting of 2012)
2. ProDeo 1.74 (doubled king safety)
3. ProDeo 1.74 (Q3 - tactical engine)
4. ProDeo 1.74 (Rebel 12 settings of 2003)

Code: Select all

       1      2      3      4
1.  -----  78.09   48.24  69.74
2.  78.09  -----   51.76  72.32
3.  48.24  51.76   -----  54.56
4.  69.74  72.32   54.56  -----

Even the old Rebel 12 positional settings from 9 years ago show enough similarities (69%) to link both programs to the same origin and author. Only the extreme settings of the Q3 - tactical engine (48%) is able to hide the ProDeo origins, but then these settings play considerable weaker.

Robbolito is linked to Ippolit from an unknown author, end of story.

Rybka then.

Code: Select all

  1) Fruit 2.1      (time: 100 ms  scale: 1.0)
  2) Rybka 1        (time: 100 ms  scale: 1.0)
  3) Rybka 2.3.2a   (time: 100 ms  scale: 1.0)
  4) Rybka 3        (time: 100 ms  scale: 1.0)
  5) Rybka 4        (time: 100 ms  scale: 1.0)

         1     2     3     4     5
  1.  ----- 54.43 52.63 47.83 47.58
  2.  54.43 ----- 61.92 52.86 52.68
  3.  52.63 61.92 ----- 57.71 56.26
  4.  47.83 52.86 57.71 ----- 59.31
  5.  47.58 52.68 56.26 59.31 -----

None of the Rybka versions show a 60% similarity with Fruit. Rybka 3 and 4 with 47% are above all suspicion.