Crafty accused of copying Fruit PST

[mclane]

is crafty a commercial program ?

What does that have to do with things?

rybka is.

crafty is just an open source chess program.
he/they (whichever programmer beside bob put this into the codes) can try out almost any idea or code, if the programmer
writes somewhere, code X by author y used in module z.

i am sure if there is such a stuff in crafty, whoever did it in,
there is somewhere a reference : taken from code x written by author y.

its something different if you sell an engine, take parts from others, do NOT
tell anybody about usage etc.

then you have exactly the thing apple is doing at court in the moment with samsung and HTC about patents and stolen stuff.

[UncombedCoconut]

And sources in china reported that a sack of rice has fallen over.

So 32 numbers (because the table is symmetric) are the same with 32 numbers from another program. For such easy stuff like psq tables this is not unusual, because the table represents a rough estimation of good and bad squares of a piece (e.g. a bishop).

I count 10: 1 back rank penalty, -1 global offset, and 10 squares once you use horizontal/vertical/rotational symmetry:

Code: Select all

XXXX
 XXX
  XX
   X

Does anyone know the heritage of the back rank penalty idea? I wouldn't be surprised if it predates both programs, but I haven't followed CC long enough.

[bob]

Only for the brain-dead...

When Zach posts this, it does make one wonder what's going on though:

Are you really going to say that your bishop PST is "not nearly as similar as Rybkas"? That is just absurd.

I can imagine some reasons why it might be the same. But waffling about it isn't helping your case at all.

http://rybkaforum.net/cgi-bin/rybkaforu ... ?tid=22635

My comment that he responded two was combining two points...

(1) there is a subtle difference between copying every evaluation term and multiplying each by the same constant as opposed to copying and adding/subtracting a constant. Doing the former doesn't change a thing. Pawns are worth 10x more, knights are worth 10x more, outpost knight is worth 10x more. Etc. Doing the latter actually changes scoring, particularly how the specific piece (whose PST values are, say, modified by subtracting 10) is valued. If you subtract (or add) 8 to every bishop PST value, then you just subtly altered the value of any bishop by that amount. Which will definitely change how the program plays... Scores will change. PV will change. Game results will change...

(2) Crafty has 16 PST tables. Fruit has 10. If you compare the PST stuff in both, what you find and list under the similarity column is this:

(a) bishop PST values in Crafty and Fruit are identical except that for Crafty, the values are +8 bigger than fruit.

When you look at things about PSTs that are dissimilar, the column looks like this;

(a) there are no MG/EG differences in Crafty. There is one set of values for each piece (except kings).

(b) except for bishops, PST values in the two programs are completely different.

(c) for Kings, Crafty has 6 PSTs, where Fruit has 2. Crafty's PSTs are based on whether there are pawns on the kingside, the queenside, or both. One doesn't necessarily want to centralize the king if the only remaining pawns are on the A/B files, for example.

(d) Crafty has no Rook PSTs at all.

Summary:

One unexpected (to me) similarity. Rest is completely different.

My comment seemed to be clear when I wrote it...

[bob]

Only for the brain-dead...

Why do you say that? You have to admit that when only the addition of one constant makes the tables identical all the way through then it seems that something is not quite kosher. What you are seeing is exactly what Vas was accused of. Actually with Rybka, a much more robust method where a constant was multiplied and then the resulting number trucated was required to get a similar match. Here all that was required was a single +8 constant and you have an identical match for all the values. If this was the ICGA method used to identify "plagiarism", then clearly Crafty or Fruit are guilty of the same thing Rybka is guilty of. I of course do not consider the tuning of available data as doing anything wrong. The piece value data sets have been known for eons in chess circles and dynamically tuning them in the course of a chess game is what makes a human or an engine super strong at playing chess. I just don't agree to how it is perfectly explainable for one program, yet is used to condemn another program.

This is simply wrong. If we had been comparing Crafty to Fruit, we would have quit as soon as the PSTs were examined. One that is close (offset by a constant). 15 others that are completely different, and kings done in exactly the same way, no rook PST in crafty at all. They don't look anything alike except for one set of 64 numbers out of 16 such sets. And then the evals in Crafty and Fruit do not look alike, they don't do the same things, in the same order. They don't even evaluate the same ideas everywhere...

So it is not "exactly the same thing Vas was accused of." Rybka's PST values match exactly through the fruit initialization code. All 10 of them. Every last number. Just scaled differently at initialization to account for different material values.

How you would conclude the two comparisons are the same is _really_ a stretch of the imagination...

[bob]

Well for one, the PST values have been the "smoking gun" in the ICGA report which claimed that Vas had violated rule 2 of their guideline. I do agree that PST values are not valuable in claiming "plagiarism" and I don't want this to sound that I consider the values in Crafty as being cloned from Fruit. All I am trying to say is that if the rules are applied equally to all participants, things would be much different. I am pretty sure if all participants were to open their source code, and have their code analyzed in the same zeal and bias that Rybka code was exposed to ... there would be some surprises.

Also this is what Bob posted regarding PST values ... so maybe he should retract this statement.

"To suggest that two different programmers wrote two different pieces of code, independently of each other, without having a set of target values to work toward, and end up with the same set of PST values is, without a doubt, one of the most ridiculous statements I have ever read."

Sorry, but this is wrong. The PSTs were discovered long after the investigation was started when Strelka came along. Zach found the stuff when we were all communicating via email, because CCC became impossible to use as a discussion forum back then.

If you read Mark's analysis, he even says that the Rybka PST data was "secondary". The Rybka Forum folks (and Ed/Chris/etc) brought up the PST data as they thought they had the best case of showing that Rybka did not copy PST values, and hence the investigation was flawed from the beginning. They eventually gave up as Zach's analysis held up.

As far as my statement, it still stands. I am certainly surprised that Crafty/Fruit is so similar, but I will add that "In only one PST". And even that is not quite true in that Fruit has two different PSTs for bishops, Crafty uses just one.

But if you look at my other quotes, you would find "finding one similar PST would not be that far-fetched. But finding perfect matches in all 10 is beyond normal probability."

I don't see why I would want to change that at all. I'm going to try to figure out how, exactly, this happened. Whether it be by automated scaling, or what, I do not currently know. But I'm going to investigate, out of curiosity.

[bob]

Well the report seems to use a lot of statistcal math analysis that some things are highly improbable and thus Rybka is guilty. Personally I think things like Rybka managed more ELO gain per certain period than is usual for a chess program as pure BS ... how this be claimed to be a valid criteria by anyone is remarkable, especially when that same person duplicates the ELO gain in later releases. If you take the same system of using statistical analysis as a marker for guilt, then the PST bishop table would categorize Crafty as being guilty as hell.

No. Just with a small degree. what about the other 15 PSTs that are completely different? What about the rest of the code, where Crafty's eval and Fruit's look nothing alike? Ditto for search. Etc.

You do like hyperbole, it seems...

[bob]

And sources in china reported that a sack of rice has fallen over.

So 32 numbers (because the table is symmetric) are the same with 32 numbers from another program. For such easy stuff like psq tables this is not unusual, because the table represents a rough estimation of good and bad squares of a piece (e.g. a bishop).

I count 10: 1 back rank penalty, -1 global offset, and 10 squares once you use horizontal/vertical/rotational symmetry:

Code: Select all

XXXX
 XXX
  XX
   X

Does anyone know the heritage of the back rank penalty idea? I wouldn't be surprised if it predates both programs, but I haven't followed CC long enough.

Development. Get pieces off their original squares as quickly as possible. Been in Crafty since just about day 1, as you have to do something to force development before starting an attack that will otherwise fail...

We do some of this in EvaluateDevelopment() today, instead...

We used this idea in Cray Blitz and even earlier programs..

It is probably mentioned in the Slate chapter of "Chess Skill ..."

[Desperado]

(1)...Doing the former doesn't change a thing. Pawns are worth 10x more, knights are worth 10x more, outpost knight is worth 10x more...

Hello Bob,

this isnt a good example. It doesnt matter if you add constants or
use multipliers. _Both_ will have effect on how the engine plays, aside
from the possibility the multiplier effects _every_ evaluation term.
(material scores... or whatever). If the multiplier is only added to
one feature like PST it will change the ratio between other eval terms
of course. But you know that,so i think the example was just unluckily choosen.

In addition, i dont see any problem because i think everyone can backtrace the development of crafty over many years. But on the
other hand, even if PST values are matching in 2 _different_ engines
the meaning of the values becomes _totally_ different, as long as
the values aren t independant from all other stuff used in an engine.


imho

cheers, michael

[Anton]

_snip_

This is like someone driving around in a counterfeit Porsche 911 and pointing their finger at a car with a similar license plate screw to that found on a genuine Porsche 911 and screaming foul play?! I suppose it makes them feel better about their own unethical decisions..

[bob]

(1)...Doing the former doesn't change a thing. Pawns are worth 10x more, knights are worth 10x more, outpost knight is worth 10x more...

Hello Bob,

this isnt a good example. It doesnt matter if you add constants or
use multipliers. _Both_ will have effect on how the engine plays, aside
from the possibility the multiplier effects _every_ evaluation term.
(material scores... or whatever). If the multiplier is only added to
one feature like PST it will change the ratio between other eval terms
of course. But you know that,so i think the example was just unluckily choosen.

In addition, i dont see any problem because i think everyone can backtrace the development of crafty over many years. But on the
other hand, even if PST values are matching in 2 _different_ engines
the meaning of the values becomes _totally_ different, as long as
the values aren t independant from all other stuff used in an engine.


imho

cheers, michael

If you multiply _every_ eval term by 10, you change nothing. Pawns are worth 10x. Each file advance is worth 10x. Nothing changes. I did the opposite of this in Crafty way back, going from millipawns to centipawns. And I simply divided all scores by 10.

But if you add 10 to just one, you do change someting. Even if you add 10 to everything, you change things because for a term that is 10, adding 10 doubles it. for a term that is 100, adding 10 is a small change.

There's a big difference between changing ALL, as was done in Rybka by using pawn=3200 as opposed to the more common 100, and just changing one PST by adding or subtracting a constant. That was my point.

I'm not so concerned that fruit/crafty match on just one 64-word PST. I'm going to fix that, just because it should not happen. I would be far more concerned if I had seen the same kind of Fruit/Rybka duplication on ALL PST values...