Rybka ban thoughts

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Uri Blass
Posts: 8553
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Rybka ban thoughts

Post by Uri Blass » Wed Jun 29, 2011 11:33 pm

bob wrote:
Dann Corbit wrote:It appears to me that Vas has made some disappointing choices.

However, I suspect that every single person who has ever made a chess program has carefully examined someone else's source and read someone else's article and implemented their own version of it.

Maybe Vas's use was more extensive, but he has been given the death sentence for the same thing that every other chess programmer does.
That is simply, and utterly false. Every other programmer does NOT copy the source from someone else. This has never been about copying ideas, it has been about copying source code.

Simply put, it is necessary to use modern techniques like LMR, Null move, PVS search, etc. I guess further that not a single programmer here has invented all of his evaluation terms, but at least got ideas from other programmers, GMs, etc.
Key word: "ideas". Ideas != source code



An algorithm is not protected by copyright. If you read and understand an algorithm, you can make your own implementation of it. There is nothing wrong with that. I guess that nobody here really understands what that means, considering how people react. Do people really not understand what the definition of an algorithm is?

I also do not think that plagiarism has any bearing on the subject. Vas was not producing a published report, and he also did give credit to both Crafty and Fruit, stating that he had carefully studied them.
Not good enough. If you copy from another article, book, or etc, there is a precise citation rule that must be followed. Vas _explicitly_ stated that he copied _no_ code. And that statement was proven to be completely false.

It does seem to me that he did take excessive liberties, but still I think it is strange to talk about plagiarism for a chess engine binary. Plagiarism is an academic crime.
Simply read the ICGA tournament rules. The author must explicitly list the names of anyone that has source code in the program being entered. This was not done.


I really do not know for sure how much wrongdoing Vas has done, but I think to a large degree he was made a whipping boy for the same sins that all chess programmer commit.
You have made many statements in the past. The above goes to the _BOTTOM_ of the pile in terms of quality. "all chess programmers" do not copy the code of others. That statement is trivial to prove false. I am a chess programmer. I have not copied _any_ code from any other program. This is easy to verify. And it instantly disproves this "all chess programmers copy" nonsense...

Where is this coming from?

Vas created his own set of problems, by himself. The penalty could have been much lighter but he offered no justification for his actions, no circumstances that could be used to reduce the penalties, no nothing...



Shredder was not reverse engineered. Hiarcs was not reverse engineered, Junior was not reverse engineered. Because of Vas' ultimate success he was singled out. Was it petty jealousy?
You already know the answer to this. Vas started this process by claiming that Strelka was a reverse-engineered clone of Rybka. Since Strelka source was released, it was natural that someone would notice that it was quite similar to fruit. This wasn't a witch-hunt at all. It wasn't petty jealousy. It was stupidity.



My whole take on this can (of course) be completely wrong. But I feel deep disappointment on all sides of the argument.

IMO-YMMV
I do not think that Vas could avoid the investigation by not claiming that
Strelka was a reverse-engineered clone of Rybka

The fact that Strelka is a derivative of Rybka was obvious without looking at the code based on looking at the analysis of the chess engine and Vas had to explain why both strelka and Rybka gives almost the same output(I posted examples for it before the claim of Vas that strelka is based on reverse engineering of rybka).

My opinion is that saying nothing could not help him because the connection rybka-strelka was obvious even without his words.

User avatar
Harvey Williamson
Posts: 1765
Joined: Sun May 25, 2008 9:12 pm
Location: Media City, UK
Contact:

Re: Rybka ban thoughts

Post by Harvey Williamson » Wed Jun 29, 2011 11:34 pm

Sarciness wrote:
wgarvin wrote:
At least half of the panel members have never competed against Rybka at the WCCC or a similar event.
"At least half" is simply not good enough. The other (neutral) jurors could easily be pressured and persuaded by even one juror with a personal bias. To have nearly have of the jury as competitors of the defendant is a farce!
The panel were neither judge nor Jury they helped compile a report. The Panel was open to team Rybka. The Judges were the ICGA.

Uri Blass
Posts: 8553
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Rybka ban thoughts

Post by Uri Blass » Wed Jun 29, 2011 11:42 pm

wgarvin wrote:
Dann Corbit wrote: The "jury of his peers" consisted of his opponents.
At least half of the panel members have never competed against Rybka at the WCCC or a similar event.
Dann Corbit wrote: They simply do not have his source code.
That doesn't matter. Reading source code is one obvious way to find out what a program does, but there are other ways. Several versions of Rybka were reverse-engineered to understand their behaviour and compare it to other programs such as Crafty and Fruit 2.1. The sources of Strelka were also available, and in all of the parts I looked at, were a very close match with the behaviour of the disassembled Rybka code.

Anyway, Fruit 2.1's eval was a pretty close match with Rybka 1.0 Beta. Its one thing to borrow an idea or two and implement them in your own program. But its a very different thing to take the entire eval from Fruit 2.1, port it to a bitboard representation, and call it your own. That appears to be what Vas did. (And he ignored the many opportunities that were given to him, to comment on or correct this perception of the panel's, if he believed it to be inaccurate).
Dann Corbit wrote: If the algorithm Vas wrote was the same, there is nothing wrong with that. It is the implementation and not the algorithm that is protected. I simply do not believe that what is claimed to have been proven has been proven.
But where does the "algorithm" end, and the "implementation" begin? If his eval has nearly the exact same eval features, in the same order, and (in the early versions at least) even with the same score weights... its hard to say that only an "idea" has been copied.

I'm not actually claiming that he literally copy-and-pasted code from Fruit 2.1 into his program. Here is what I believe: I think he translated it into bitboard format, and probably typed it in with his own naming conventions and style and so on. Regardless, the result is still "derived" from Fruit 2.1. It probably took him a day or two, maybe longer -- but if he had truly made an original eval from scratch, it would have probably taken months of experimentation and hard work to produce an eval of that quality. Instead, it looks as if Vas skipped all of that effort by just copying what Fabien had done. He then claimed it was his original work, and gave evasive answers when asked about the similarity.
I disagree that fruit2.1's evaluation was a a pretty close match with Rybka 1.0 Beta.

It is a fact that fruit2.1 has endgame knowledge that Rybka1 beta does not have and Rybka also has different numbers in the evaluation.

Chess players who use programs for analysis did not see the similiarity between fruit and rybka.
They clearly saw the similiarity between rybka1 beta and strelka.

I do not claim that Vas did not copy part of fruit's evaluation but he modified it enough to be clearly different than fruit(the evaluation may be based on some common ideas but not every thing is the same.
For example
Rybka has material balance table that fruit does not have and the weights for middle game and endgame in Rybka are also different from fruit.

Adam Hair
Posts: 3201
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: Rybka ban thoughts

Post by Adam Hair » Thu Jun 30, 2011 1:30 am

Sarciness wrote:What Dann says echoes my sentiments. Vas and Rybka may be guilty... maybe in the worst sense, but this "trial" is not fair at all.

64% similarity does not sound like "proof beyond reasonable doubt" to me, much less "absolute proof" as Robert Hyatt seems to claim.

If this were a jury in a real trial they would be instantly dismissed becasue many of them have personal biases in regard to the defendant. This is a bullshit ruling which reflects badly on the ICGA. What they need is an independent and impartial analysis.

Vas has not defended himself and should do so in my opinion. His silence seems more like disillusionment with the computer chess world or even a guilty concience than "taking the high ground".
The similarity stated for Rybka 1.0 beta and Fruit 2.1 is 74.4%. If this assessment of evaluation feature overlap is sound, then, when you compare 74.4% to the mean of 31.3% and the fact that 74.4% is 7.5 standard deviations from the mean, it is very damning.

The question is in the determination of evaluation feature overlap. Mark Watkins, who does not appear to be a competitor of Vasik, made this determination. As he noted, this was a subjective measurement. Which means that the percentages are on shakier ground than possibly quantitative measurements. It does appear that there is ample information for interested and qualified individuals to assess Mr. Watkins' judgement.

Adam

Adam Hair
Posts: 3201
Joined: Wed May 06, 2009 8:31 pm
Location: Fuquay-Varina, North Carolina

Re: Rybka ban thoughts

Post by Adam Hair » Thu Jun 30, 2011 2:44 am

Dann Corbit wrote:I think it is time for me to leave computer chess now, because I am at odds with the community as a whole.
That would be a loss to the community as a whole.
A great loss.

Adam

Chan Rasjid
Posts: 567
Joined: Thu Mar 09, 2006 3:47 pm
Location: Singapore

Re: Rybka ban thoughts

Post by Chan Rasjid » Thu Jun 30, 2011 3:27 am

wgarvin wrote:
fern wrote:Ok, but then, how it is the result, so copied as you say, got so much superior perfomance?

F
No one is saying that Rybka is "just a clone" or something. Obviously Vas put a lot of effort into improving Rybka, and most of it is probably original code.

The problem for the ICGA is that to enter those tournaments, you have to affirm that all of it is your original code. And Vas did claim that, even though it wasn't true. He had an unfair advantage over the other competitors who legitimately wrote their own eval.
May I have your permission to differ (with a pinch of salt).

Vasik Rajlich is shrewd, sharp and smart - smarter than the rest of the pack. As he said, he went through the codes of Fruit forward and backward many times. He probably realised what made Fruit ticks and he clearly mentioned in a post that he was surprised that it was so easy for Fruit to be that strong and he clearly mentioned that he understood all of the code of Fruit. He probably sensed what it takes to make a top class program ELO wise. His competitors could not see what what he saw and they thought they were smarter than that and pursued different ways with different evaluations to that of Fruit - they simply failed and failed to catch up with Vasik who alone held the key as to how to gain ELO and left the pack far behind. So the pack later cover up their failures by saying "I don't want to copy ... clone ... whatever."

IMNSHO.
Rasjid.
Don't believe when you're told "There's no free lunch!" There is Linux.

Terry McCracken
Posts: 15844
Joined: Wed Aug 01, 2007 2:16 am
Location: Canada

Re: Rybka ban thoughts

Post by Terry McCracken » Thu Jun 30, 2011 3:38 am

Chan Rasjid wrote:
wgarvin wrote:
fern wrote:Ok, but then, how it is the result, so copied as you say, got so much superior perfomance?

F
No one is saying that Rybka is "just a clone" or something. Obviously Vas put a lot of effort into improving Rybka, and most of it is probably original code.

The problem for the ICGA is that to enter those tournaments, you have to affirm that all of it is your original code. And Vas did claim that, even though it wasn't true. He had an unfair advantage over the other competitors who legitimately wrote their own eval.
May I have your permission to differ (with a pinch of salt).

Vasik Rajlich is shrewd, sharp and smart - smarter than the rest of the pack. As he said, he went through the codes of Fruit forward and backward many times. He probably realised what made Fruit ticks and he clearly mentioned in a post that he was surprised that it was so easy for Fruit to be that strong and he clearly mentioned that he understood all of the code of Fruit. He probably sensed what it takes to make a top class program ELO wise. His competitors could not see what what he saw and they thought they were smarter than that and pursued different ways with different evaluations to that of Fruit - they simply failed and failed to catch up with Vasik who alone held the key as to how to gain ELO and left the pack far behind. So the pack later cover up their failures by saying "I don't want to copy ... clone ... whatever."

IMNSHO.
Rasjid.

You really believe that or are you forming a Rat Pact?

The point is he did more than go through the code, he directly copied and pasted large blocks of it into Rybka 1.0 Beta. That's a fact. Not so smart.
Terry McCracken

Chan Rasjid
Posts: 567
Joined: Thu Mar 09, 2006 3:47 pm
Location: Singapore

Re: Rybka ban thoughts

Post by Chan Rasjid » Thu Jun 30, 2011 3:51 am

Terry McCracken wrote:
Chan Rasjid wrote:
wgarvin wrote:
fern wrote:Ok, but then, how it is the result, so copied as you say, got so much superior perfomance?

F
No one is saying that Rybka is "just a clone" or something. Obviously Vas put a lot of effort into improving Rybka, and most of it is probably original code.

The problem for the ICGA is that to enter those tournaments, you have to affirm that all of it is your original code. And Vas did claim that, even though it wasn't true. He had an unfair advantage over the other competitors who legitimately wrote their own eval.
May I have your permission to differ (with a pinch of salt).

Vasik Rajlich is shrewd, sharp and smart - smarter than the rest of the pack. As he said, he went through the codes of Fruit forward and backward many times. He probably realised what made Fruit ticks and he clearly mentioned in a post that he was surprised that it was so easy for Fruit to be that strong and he clearly mentioned that he understood all of the code of Fruit. He probably sensed what it takes to make a top class program ELO wise. His competitors could not see what what he saw and they thought they were smarter than that and pursued different ways with different evaluations to that of Fruit - they simply failed and failed to catch up with Vasik who alone held the key as to how to gain ELO and left the pack far behind. So the pack later cover up their failures by saying "I don't want to copy ... clone ... whatever."

IMNSHO.
Rasjid.

You really believe that or are you forming a Rat Pact?

The point is he did more than go through the code, he directly copied and pasted large blocks of it into Rybka 1.0 Beta. That's a fact. Not so smart.
You really believe that!

You do not read English, do you?

Did you not read what Wylie wrote just few posts up?

Rasjid.
Don't believe when you're told "There's no free lunch!" There is Linux.

bob
Posts: 20472
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Rybka ban thoughts

Post by bob » Thu Jun 30, 2011 4:01 am

Dann Corbit wrote:
bob wrote:
Dann Corbit wrote:
rbarreira wrote:I find it strange that your post doesn't contain the word "code" even once.
All of the code used to commit Vas is from reverse engineering.

Have you seen the post I made some time ago that showed two completely different code bases in C produced identical assembly (even the control structures were different).

Vas has been convicted of stealing code in a court made up of his opponents. There is literally no Rybka code available.

From assembly, the only thing that can be demonstrated is the algorithm similarity. I recall that 64% similarity was a quoted figure somewhere.
Your example was no good. Why? Because if you want to construct such an example, it is not that hard to do. But the question is, does it happen _naturally_? The answer is no. There is a ton of academic research on plagiarism detection that discusses this issue to death...

64% makes no sense to me. The parts that I examined were 100% matches. Not close. Not approximate. Identical.
Clearly though, the code cannot be identical because the binary is 4MB in size and 300 Elo stronger than its opponents when it was released.

Some things must be identical (e.g. strcpy).

The process of having his opposition decide against him does not sit well with me. What sort of justice is that?
You _really_ make light of what happened. It was not "strcpy()" as you mention. Pick a version. Rybka 1? Fruit eval top to bottom with very little added. 2.3.2? 2/3 of fruit eval. That's just for starters...

And that is a _far_ cry from "strcpy()".

Nobody has ever said that Rybka X is a total copy of Fruit Y. Clearly things were added, although one might make that claim when talking about the pre-commercial Rybkas and Crafty, where the versions prior to the commercial 1.0 (and the beta version of that) had apparently copied almost _all_ of Crafty 19.0...

Do you think David Levy is "Vas' opposition?" His "opposition" simply presented the evidence, in as clear and concise a way as possible. The ICGA made the decision, and then handed out the punishment. Not his "opposition."

bob
Posts: 20472
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Rybka ban thoughts

Post by bob » Thu Jun 30, 2011 4:04 am

Dann Corbit wrote:
rbarreira wrote:
Dann Corbit wrote:Only that every single programmer here is guilty of the same crimes as Vas, to one degree or another.
Every single programmer here has directly copied non-public-domain code without attributing it to the original author, released this code as part of their own software and/or later denied doing it when directly asked about it?

Those are some big accusations especially when talking about "every programmer here".
The "jury of his peers" consisted of his opponents.
Sorry, but that is factually wrong. His "opponents" presented the evidence we collected. The judge/jury were ICGA people that are not actively competing. Big difference between reality and what you claimed...


They simply do not have his source code.

If the algorithm Vas wrote was the same, there is nothing wrong with that. It is the implementation and not the algorithm that is protected. I simply do not believe that what is claimed to have been proven has been proven.
Have you looked at the ICGA Wiki and read it _carefully_? I would say no, because your conclusion would not follow from that information.

Now, I do not think that Vas was blameless. I have made that clear.
He may be very guilty, he may even be a criminal {I am not yet convinced by anything that this is true}. But the methods used to dismiss him forever from computer chess were easily as bad as anything he has ever done, if not worse.
I don't think I have ever seen anyone say the judge/jury was worse that the person convicted... this has to be a first, and it is not a _good_ first. A new low-water mark IMHO.

Post Reply