My two cents on Rybka's disqualification

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: My two cents on Rybka's disqualification

Post by bob »

Dann Corbit wrote:
bob wrote:
Dann Corbit wrote:
bob wrote:
Dann Corbit wrote:
bob wrote:
Dann Corbit wrote:
bob wrote:NASCAR has a set of rules. After the race, the winner's car is impounded and gone over with a microscope. If a violation is found, it is disqualified. In track and field events, if a drug test is failed, titles are vacated, participants can be banned for a period of time including life. This is common in any sort of organization that handles contests. Pete Rose comes to mind in baseball. A manager that bets on baseball games. Baseball has a rule that managers and players can not bet because they have the ability to alter the result of a game by how they play or how they manage, which is a big conflict of interest. Pete Rose violated that rule. Where is he today? outside looking in, yet he was one of the best ball players of all time. And won't be allowed in the hall of fame because he violated a simple rule. And he didn't even bet on his own team or games his team played in. But the rule was a rule.

Every decision made on the planet does not go through the courts. Nothing would get done if that were required.
Yes, those are good points and especially in the case of Pete Rose, the outcome clearly has the force of law because his reputation is utterly ruined and his livelihood is destroyed.

I still don't like the process chosen by the ICGA. I do not know enough about baseball bylaws and processes to know if I do not like the handling of the Pete Rose process.
The problem is, when you apply to play in an ICGA event, you are required to sign an entry form that includes specific requirements a program must meet, and requirements an author must meet should a protest be filed.
Do you know where I can find a copy of this agreement?
I would like to read it, in particular for each of the years that Vas competed in the competition.

I am particularly interested in the exact wording of the agreement and the exact wording of the penalties.

It is quite possible that the ICGA acted properly if Vas has violated his agreements and if the ICGA acted as they spelled out in their contracts.
I believe the entry forms are kept on the ICGA web site. I do not know if all versions are present, or just the one for the most recent WCCC event. But they have all been pretty clear about the requirement for source code should a protest be filed.

Here is rule 2 which is where the current issue lies:
2. Each program must be the original work of the entering developers. Programming teams whose code is derived from or including game-playing code written by others must name all other authors, or the source of such code, in their submission details. Programs which are discovered to be close derivatives of others (e.g., by playing nearly all moves the same), may be declared invalid by the Tournament Director after seeking expert advice. For this purpose a listing of all game-related code running on the system must be available on demand to the Tournament Director.
I have never seen a formal statement about potential penalties for failing to follow rules. In general, you get kicked out of any tournament where that happens. This is a first, so far as I know, for a case that spanned several years.
Upon examination of the rules, including this one:
"2. Each program must be the original work of the entering developers. Programming teams whose code is derived from or including game-playing code written by others must name all other authors, or the source of such code, in their submission details. Programs which are discovered to be close derivatives of others (e.g., by playing nearly all moves the same), may be declared invalid by the Tournament Director after seeking expert advice. For this purpose a listing of all game-related code running on the system must be available on demand to the Tournament Director."

It was not at all clear to me that the rule includes inspection long after the tournament is completed (indeed, such a rule makes little sense in that context).

I think that the rule should be amended to make this clear.

On the other hand, it also does not say that the tournament director cannot demand the source code years later. So, in some sense Vas seems to have "sworn to what was bad for himself" by agreeing to a highly ambiguous document.

In my opinion he should have furnished the source code when demanded.

In going over the documents I found, I was unable to find any specific penalties, for example at this location:
http://www.grappa.univ-lille3.fr/icga/e ... .php?id=12
I think that was one of the things we chose to leave until the circumstance arose, rather than have long debates, and try to cover every conceivable case. It is easier to let it happen, then you have context for making the decision...
Doesn't that strike you are rather arbitary?
Nope. Done all the time. Otherwise you start to define ways to handle every possible case, most of which will never happen, and some that you failed to think of that will happen. How many events will you vacate? Will you vacate for all infractions or just certain ones. What if the protested program offers some explanations that have a bit of plausibility? What if he fails to respond at all? What if he even refuses to present the source? Etc. Too many cases to handle, the current rule is clear about what is allowed in an entry (original code, or author names if others provided some of the code) and what is required (source if demanded.) For every ACM event I want to, I had a copy of my source. In the early days, this was a printout. One of those was what I sent Carey years ago to try to resurrect a working copy of Cray Blitz from such a printout. In later days, it was source files on a laptop.

I complied with the rules as I understood them, and never had any problem except for the one protest that was investigated as carefully as the current case.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: My two cents on Rybka's disqualification

Post by bob »

Dann Corbit wrote:
bob wrote:
Dann Corbit wrote:
{Dann Corbit}
There is such a thing as 'Fair use'
Vas has given credit to both crafty and fruit in his documentation.
Now, I have no idea if fair use applies. Do you?
{Robert Hyatt}
Absolutely not. The ICGA rules are explicit. The program must be original, not containing parts of other programs unless those authors are listed on the entry form (and then those authors could not enter their own program, much like the Fruit/Toga issue where only one is allowed in an ICGA event).
You don't see any gray area here?
YBW is not original.
Alpha-beta is not original.
LMR is not original.
etc.
etc.
etc.
Why is it that 100% of all programs are not disqualified under this guideline?
Because the guideline is about _code_ and not _ideas_. I've only explained this a hundred times now. Containing "parts of other programs". That is a very clear statement, since a program is a set of source statements, and a part of that program is a subset of those lines of code. This is about code copying, not about idea copying... <again>
How would you tell if someone copied and pasted LMR and then edited it, verses learning it and writing their own version?

Isn't it even possible to learn the mistakes in an algorithm and publish a faulty version?

I am just saying that I think there are gray colors here and not only black and white.

I also want the competitions to be as fair and open as possible and I cannot say that I have any special insight as to how to achieve this. My interest in this controversy is that I want the process to be very fair for all of the parties involved. I think that having a fair process in place is more important than others might and that is my perogative.
If they copy/paste/edit-rewrite so that it is unrecognizible, that's perfectly acceptable. If they copy/paste, that is not acceptable at all.

If you write LMR, you will _not_ write the same source statements as someone else. That is a key point. It just does not happen. It is unlikely two people would write a simple bubble sort exactly the same way. And LMR and such is far more complex than a simple sort.

And no, if you look at the code in Crafty that was copied, I would think even a fairly weak programmer could see that:

ms = X(a,b,c,d)
if (ms != 99999) {
do something
}

and in function X, you look at the code and immediately notice that the return value is an integer, and it is multiplied by 2, so that it can _never_ return an odd number, specifically 99999. That is the kind of stuff Vas copied. Problem is, I can precisely explain why that code was written, and why it is no longer needed. And that it was left in accidentally when I made the change from when the code was written and needed, until I made the change that made it useless but forgot to remove it. How could the copier explain that? It is a smoking gun. And we had several.

So that excuse won't pass, any more than the "two people could write the same algorithm" and end up with the same source." Too much experience has shown that to _not_ be the case.

As far as fairness goes, this was as fair as it gets. David asked Vas to participate at every stage. He refused. I personally believe because he knew the truth and that it was going to be exposed for all to see, leaving no doubt.
kranium
Posts: 2129
Joined: Thu May 29, 2008 10:43 am

Re: My two cents on Rybka's disqualification

Post by kranium »

Dann Corbit wrote:
I have shown Norman some ways to improve fire.

The most important early steps are the removal of bugs.

I have identified about 500 problem areas, provided diagnostics that prove some of them are real problems, and explained to Norman the value of making some corrections.
Dann-
?

no, sorry!

these compiler 'warnings' that you pointed out in an email a couple days ago, have been well-known to me since the very beginning of my Ippolit development efforts...as it was i that added the ~10 #pragma warning (disable) statements to the very earliest chesslogik ippolit/robbolito source code 2-3 years ago.

what is very disheartening is to now to see you now calling these (mostly harmless) warnings 'bugs', and taking credit for 'showing' me how to improve Fire, and 'explaining' to me the value of eliminating them?

i have already eliminated 2-3 of these pragma warnings disables just in the last 2-3 releases alone...(before you even contacted me, on my own accord, as part of my ongoing effort to improve/perfect the codebase).

but now, 'you' have identified '500 problem areas', by sending an email pointing out these already known warnings?

sorry this is just painful to read...
i would have thought you were above such a grab for attention and recognition.
what is going on with you?

Norm
jdart
Posts: 4367
Joined: Fri Mar 10, 2006 5:23 am
Location: http://www.arasanchess.org

Re: My two cents on Rybka's disqualification

Post by jdart »

I was an early defender of Vas. The evidence came out gradually and the picture was also clouded by what now seems to have been a lot of untrue statements from the Vas. It now seems that indeed code was used without attribution or license.

I don't think the investigation was actually unfair although as the parent poster said there was a lot of in-public denunciation first, and some of the denouncers were on the panel as well, so it looks a bit like a witch hunt. I also note no other closed source program has been subjected to this degree of scrutiny. But I can't now really argue with the conclusions.

I'm a bit saddened by the whole thing. If the author were not making a chess engine but some other kind of software I really doubt we'd see the same degree of passion. He'd just get sued over code misuse - or maybe he wouldn't if it wasn't worth enough to someone.

--Jon
El Gringo
Posts: 118
Joined: Tue Nov 27, 2007 4:01 pm

Re: My two cents on Rybka's disqualification

Post by El Gringo »

Hi guys,


I think i just saw Vasik and Iwita at the bahamas...
They were singing a song from the Steve Millar Band..
ooohh ooohh take the money and run :lol: