Critter 1.2 SEEMS to be a member of the Ippo family

[fern]

I think that what you say perhaps apply too, in a degree, to Vas. He copied -I want to believe that- what he considered important but not esential as a mark of originality. Maybe he thought as you: this or that piece of code is already a commodity, for what to expend time in trying something dfifferent; the wheel was invented long ago.
I insist: originality and novelty comes from the last cherry you putted in the cake. Flour, cream, chocolate, all that is just commodity.

fern

[lucasart]

With Don's similarity tool Sim03, I got the similarity matrix as follows:


1) Critter 1.2 (time: 100 ms scale: 1.0)
2) Houdini 1.5 (time: 100 ms scale: 1.0)
3) IvanHoe B47cB (time: 100 ms scale: 1.0)
4) Komodo 2.03 (time: 100 ms scale: 1.0)
5) Robbo 009 (time: 100 ms scale: 1.0)
6) Rybka 3 (time: 100 ms scale: 1.0)
7) Rybka 4 (time: 100 ms scale: 1.0)
8) Shredder 10 (time: 100 ms scale: 1.0)
9) Shredder 12 (time: 100 ms scale: 1.0)
10) Stockfish 2.1.1 (time: 100 ms scale: 1.0)


1 2 3 4 5 6 7 8 9 10
1. ----- 61.42 60.79 50.08 61.00 57.56 55.86 43.53 45.70 51.08
2. 61.42 ----- 62.36 49.60 62.55 55.85 54.39 43.28 45.99 51.04
3. 60.79 62.36 ----- 50.45 67.06 58.76 55.47 44.08 47.04 50.78
4. 50.08 49.60 50.45 ----- 50.46 51.92 50.52 44.11 45.96 47.52
5. 61.00 62.55 67.06 50.46 ----- 60.03 56.60 44.54 47.84 50.89
6. 57.56 55.85 58.76 51.92 60.03 ----- 58.93 45.39 48.79 50.85
7. 55.86 54.39 55.47 50.52 56.60 58.93 ----- 44.22 47.50 50.91
8. 43.53 43.28 44.08 44.11 44.54 45.39 44.22 ----- 51.92 43.98
9. 45.70 45.99 47.04 45.96 47.84 48.79 47.50 51.92 ----- 45.96
10. 51.08 51.04 50.78 47.52 50.89 50.85 50.91 43.98 45.96 -----




The dendrogram using IBM SPSS19, method being the average linkage between the groups and measure the Pearson correlation is



As one can see, Critter 1.2 is closer to open source Robbo 009 and IvanHoe 47 than Rybka 4 is to Rybka 3 or Robbo 009 to Rybka 3. At the same time, Komodo and Stockfish seem pretty clean of massive borrowing from Ippo & Co. code, at least in the eval. I put there Shredder 12 and Shredder 10 to highlight that they are totally unrelated to this mess. Besides that, the difference between Shredder 10 and 12 is something which I wanted to visualize.

To read the dendrogram: the distance on the horizontal axis to the common ancestor is the degree of relatedness, smaller the distance - more related. Ippo family clusters pretty clearly, at the same time preserving its relation with Rybka family.

All the best, and don't take my results as a hard proof.
Kai

This is absolutly ridiculous!
What are the figures in this matrix anyway ? do you even know ?

And so what ? I guess they show that they often play the same moves ? Perhaps it simply means that Critter and Ippolit play correct moves where Shredder doesn't ?

What's that got to do with the actual source code ?

[Laskos]

This is absolutly ridiculous!
What are the figures in this matrix anyway ? do you even know ?

And so what ? I guess they show that they often play the same moves ? Perhaps it simply means that Critter and Ippolit play correct moves where Shredder doesn't ?

What's that got to do with the actual source code ?

Sorry, I just arrived and have to reply to Enrico in a more serious fashion, but first I have to eat something. I do know what the numbers in this matrix are, and please keep the word "ridiculous" to the less factual posts, there are plenty of them on CCC. Komodo is playing wrong moves, this is my understanding of you.

Kai

[Sven]

Unfortunately, I have no access to sim03 (I could ask for, agreed).

Hi Enrico,

although you did not ask for it sim03 (Windows) from komodochess homepage.

Sven

[noctiferus]

Thx

[Laskos]

Hi.

Unfortunately, I have no access to sim03 (I could ask for, agreed).

1) Could you please post (or send me by mail) the dataset you used for clustering?

2) in alternative, could you run a clustering with complete and single linkage, please (just to look at stability of similarities)?

I use a different clustering softw, so I'd like to compare results: this is the reason that the first altrenative would be preferable.

Thx
Enrico

I will try to answer your first choice. The method I used for hierarchical clustering in SPSS is the "average linkage clustering" which takes into account _all_ of the elements in a vector. With my primary matrix, taking _complete_ or _single_ linkage (max or min) would pick only single elements. The matrix I used is exactly what I have posted, no more than that, in agglomerative recursion.

Code: Select all

  1) Critter 1.2 (time: 100 ms  scale: 1)
  2) Houdini 1.5 (time: 100 ms  scale: 1.0)
  3) IvanHoe B47cB (time: 100 ms  scale: 1.00)
  4) Komodo 2.03 (time: 100 ms  scale: 1)
  5) Robbo 009 (time: 100 ms  scale: 1.0)
  6) Rybka 3 (time: 100 ms  scale: 1.0)
  7) Rybka 4 (time: 100 ms  scale: 1.00)
  8) Shredder 10 (time: 100 ms  scale: 1.0)
  9) Shredder 12 (time: 100 ms  scale: 1.0)
 10) Stockfish 2.1.1 (time: 100 ms  scale: 1.0)

         1     2     3     4     5     6     7     8     9    10
  1.  ----- 61.42 60.79 50.08 61.00 57.56 55.86 43.53 45.70 51.08
  2.  61.42 ----- 62.36 49.60 62.55 55.85 54.39 43.28 45.99 51.04
  3.  60.79 62.36 ----- 50.45 67.06 58.76 55.47 44.08 47.04 50.78
  4.  50.08 49.60 50.45 ----- 50.46 51.92 50.52 44.11 45.96 47.52
  5.  61.00 62.55 67.06 50.46 ----- 60.03 56.60 44.54 47.84 50.89
  6.  57.56 55.85 58.76 51.92 60.03 ----- 58.93 45.39 48.79 50.85
  7.  55.86 54.39 55.47 50.52 56.60 58.93 ----- 44.22 47.50 50.91
  8.  43.53 43.28 44.08 44.11 44.54 45.39 44.22 ----- 51.92 43.98
  9.  45.70 45.99 47.04 45.96 47.84 48.79 47.50 51.92 ----- 45.96
 10.  51.08 51.04 50.78 47.52 50.89 50.85 50.91 43.98 45.96 -----

If you need the diagonal elements, it's not important what you put there, something reasonable for self-similarity, say 70 or 90 for all diagonal elements, it doesn't matter.

The correlation between my vectors of values is the following

[/URL]

This is actually used for hierarchical iterative clustering, given the Pearson correlation measure.

Now, if you want to play with the full data set, sequence and partition iteratively it, then I have to send you the "similarity.data" file (~400kB) which can be opened as a text file and viewed as algebraic notation of moves choosen for each engine. You can play with that all you want, I never did, but Miguel used it to post results, better ask him how to deal with this file.

I used "Between-groups linkage", "Within-group linkage" and several other with very stable same results, you could use linkage on (min max) with the full "similarity.data" file, but having only the 10x10 matrix I suspect the results would be unstable, therefore I am using average linkage. I don't know how to send you the "similarity.data" file", maybe by e-mail?

Kai

[Milos]

I believe that piece square table is only a small part of the evaluation.
there is also mobility evaluation and passed pawn evaluation and king safety evaluation.

I do not expect a big similiarity between 2 programs if they only share the same piece square table when they do not share other parts of the evaluation.

You obviously don't know much about sim03. It exactly measures only static evaluation i.e. PST. Nothing else.
Don introduced it coz it's the only thing from Rybka 3 he could find similar to Ippo (Larry probably insisted on it) despite all his tries. This is the bare truth behind all the pretensions that we can read on the forum...

[BubbaTough]

I believe that piece square table is only a small part of the evaluation.
there is also mobility evaluation and passed pawn evaluation and king safety evaluation.

I do not expect a big similiarity between 2 programs if they only share the same piece square table when they do not share other parts of the evaluation.

You obviously don't know much about sim03. It exactly measures only static evaluation i.e. PST. Nothing else.
Don introduced it coz it's the only thing from Rybka 3 he could find similar to Ippo (Larry probably insisted on it) despite all his tries. This is the bare truth behind all the pretensions that we can read on the forum...

Why do you say that? I certainly don't know much about it, but it looks like a 100ms search, not a static eval. And I don't see why you claim static eval = PST, and not static eval = whole eval.

-Sam

[Milos]

Why do you say that? I certainly don't know much about it, but it looks like a 100ms search, not a static eval. And I don't see why you claim static eval = PST, and not static eval = whole eval.

It might be a good idea to first read some good book on communications and noise statistics/correlation (I suggest S. Haykin's Communication systems) and read prior threads on this topic.
100ms search actually has so much randomness in the actual search time (searched node count) that any common search component between two engines gets lost in the noise. The most of the randomness comes from the engine's check time poling function.
The same is valid for the dynamic components of the evaluation - so called positional values (scores that depend on pieces interaction - king safety, different pawns structure bonuses, mobility, etc.). There again the noise kills the correlation info.
What remains is the static evaluation - material values + PST.
You can actually very easily test it in your engine. Make full eval disabled and only lazy eval (material+PST) active. And then let this simplified engine play against your normal engine version.
Start from longer fixed time per move TC (like 1s per move) and gradually reduce TC towards 50ms or even 20ms and check what's the impact on Elo difference. You'll get quite surprised .

[BubbaTough]

Why do you say that? I certainly don't know much about it, but it looks like a 100ms search, not a static eval. And I don't see why you claim static eval = PST, and not static eval = whole eval.

It might be a good idea to first read some good book on communications and noise statistics/correlation (I suggest S. Haykin's Communication systems) and read prior threads on this topic.
100ms search actually has so much randomness in the actual search time (searched node count) that any common search component between two engines gets lost in the noise. The most of the randomness comes from the engine's check time poling function.
The same is valid for the dynamic components of the evaluation - so called positional values (scores that depend on pieces interaction - king safety, different pawns structure bonuses, mobility, etc.). There again the noise kills the correlation info.
What remains is the static evaluation - material values + PST.
You can actually very easily test it in your engine. Make full eval disabled and only lazy eval (material+PST) active. And then let this simplified engine play against your normal engine version.
Start from longer fixed time per move TC (like 1s per move) and gradually reduce TC towards 50ms or even 20ms and check what's the impact on Elo difference. You'll get quite surprised .

Its fine if you want to make some noise argument. Sounds quite reasonable (except for the data that seems to indicate there are some strong correlations), I don't understand why you are artificially separating piece squares from other things. It seems extremely unlikely that whether an engine chooses to push a passed pawn only depends on piece square tables, and not passed pawn values. Or whether moving a knight to an outpost is only dependent on square values and not the value of an outpost, or doing a BxN move only depends on PST and not on things like double pawn values or isolated pawn values or such. Or pushing a pawn in front of the king is only based on PST and not king shelter scores. The whole things sounds ridiculous. I am not saying its not true, I really have no idea, but it sounds ridiculous. If noise is the main factor, fine then, it will decrease correlation. But to claim PST and material values are somehow special in that they pierce noise and nothing else does, well, that would be very unintuitive to me.

-Sam