Clone detection test

[Harvey Williamson]

Would anyone mind if this thread got relocated to the forum dedicated to Engine Origins, since that's ultimately what it's about? Seems thematically appropriate. Please PM me if you have objections.

Thanks
Jeremy

I have an objection because this particular thread is of a technical nature more than political. I view that other forum as non-technical and political and it probably would not attract the more serious engineers as much as this one would.

Besides, the subject of this thread was specifically designed to be provocative, but it's really about designing instrumentation to measure the playing style of chess programs.

I agree with Don.

Miguel

Me to! This is the kind of intelligent discussion that should be available for all to read. It would be ironic if this gets moved but a thread offering Robbo for sale is allowed to remain in the main forum.

[sockmonkey]

Would anyone mind if this thread got relocated to the forum dedicated to Engine Origins, since that's ultimately what it's about? Seems thematically appropriate. Please PM me if you have objections.

Thanks
Jeremy

I have an objection because this particular thread is of a technical nature more than political. I view that other forum as non-technical and political and it probably would not attract the more serious engineers as much as this one would.

Besides, the subject of this thread was specifically designed to be provocative, but it's really about designing instrumentation to measure the playing style of chess programs.

I agree with Don.

Miguel

Me to! This is the kind of intelligent discussion that should be available for all to read. It would be ironic if this gets moved but a thread offering Robbo for sale is allowed to remain in the main forum.

I'm glad we're all in agreement, then. The thread stays put.

Jeremy

[Hood]

Code: Select all

  846  sf_strong         sf16            
  758  doch-1.2          doch-1.0        
  734  robbo             ippolito        
  720  komodo            doch-1.2        
  706  sf15              sf14            
  687  komodo            doch-1.0        
  671  sf16              sf15            
  655  rybka             robbo           
  649  sf16              sf14            
  644  rybka             ippolito        
  639  sf14              glaurung        
  638  sf_strong         sf15            
  636  toga2             fruit           
  630  sf15              glaurung        
  617  sf_strong         sf14            
  600  sf_strong         glaurung        
  595  sf16              glaurung        

  594  rybka             doch-1.2        
  589  fruit             doch-1.0        
  582  rybka             doch-1.0        
  581  rybka             komodo          
  579  ippolito          doch-1.0        
  573  robbo             komodo          
  571  sf15              robbo           
  571  ippolito          doch-1.2        
  569  sf15              rybka           
  568  robbo             doch-1.2        
  565  sf_strong         rybka           
  564  toga2             sf15            
  563  sf14              ippolito        
  561  toga2             sf14            
  561  toga2             doch-1.0        
  560  toga2             glaurung        
  560  komodo            ippolito        
  559  sf14              robbo           
  559  robbo             doch-1.0        
  559  fruit             doch-1.2        
  558  sf14              rybka           
  557  sf_strong         robbo           
  556  sf15              ippolito        
  554  sf16              rybka           
  554  sf16              robbo           
  554  sf15              komodo          
  553  komodo            fruit           
  553  glaurung          fruit           
  551  sf14              doch-1.0        
  549  sf15              doch-1.0        
  548  sf14              fruit           
  546  toga2             doch-1.2        
  544  sf15              fruit           
  544  glaurung          doch-1.0        
  543  toga2             sf16            
  542  rybka             glaurung        
  541  sf16              doch-1.0        
  538  sf16              ippolito        
  537  toga2             ippolito        
  536  toga2             komodo          
  536  sf_strong         doch-1.0        
  536  sf14              komodo          
  535  toga2             robbo           
  534  sf16              fruit           
  534  ippolito          fruit           
  532  komodo            glaurung        
  531  sf_strong         ippolito        
  531  sf15              doch-1.2        
  530  robbo             fruit           
  528  glaurung          doch-1.2        
  527  sf_strong         komodo          
  527  sf16              doch-1.2        
  526  toga2             rybka           
  525  robbo             glaurung        
  524  toga2             sf_strong       
  524  sf_strong         doch-1.2        
  523  sf14              doch-1.2        
  523  rybka             fruit           
  521  ippolito          glaurung        
  519  sf16              komodo          
  514  sf_strong         fruit           

Hello,

I am newbie. :-) Greetings to all Posters.

My name is Hood, Robin Hood ;-) 007. :-) Hood is my user name.

I am sorry for the joke but for me that is getting a little bit funny.
I hope you will not kill me ;-) but... with all the respect to the correlation, bayesian probability and all statistical functions.

3= 1 + 1 + 1= 2 + 1 = 1 + 2 = 3 * 1 = 1 * 3 = 3 + 0 = 0 + 3 = 4 - 1 etc.

I have some doubts.

It is practicly alef 0 ( alef 0 is the number of natural figures) possibilities of getting the same move - result of calculating the mini max or other function by the engine and selecting the move.

So correlation will not give any of clone validation but it can be only the similarity criteria- (between engines).
 
Going further when we will get the better engines the similarity in moves selecting will be increasing.
The ideal player will choose the same moves in the same position if we agree with Tarrasch sentence that in the position there is the one the best move. 
So ideal players will choose the same move, is not it ?

Regards
Locksley ;-) 
Chris

[Don]

Hello,

I am newbie. Greetings to all Posters.

My name is Hood, Robin Hood 007. Hood is my user name.

I am sorry for the joke but for me that is getting a little bit funny.
I hope you will not kill me but... with all the respect to the correlation, bayesian probability and all statistical functions.

3= 1 + 1 + 1= 2 + 1 = 1 + 2 = 3 * 1 = 1 * 3 = 3 + 0 = 0 + 3 = 4 - 1 etc.

I have some doubts.

It is practicly alef 0 ( alef 0 is the number of natural figures) possibilities of getting the same move - result of calculating the mini max or other function by the engine and selecting the move.

So correlation will not give any of clone validation but it can be only the similarity criteria- (between engines).

Going further when we will get the better engines the similarity in moves selecting will be increasing.
The ideal player will choose the same moves in the same position if we agree with Tarrasch sentence that in the position there is the one the best move.
So ideal players will choose the same move, is not it ?

Regards
Locksley
Chris

Apparently not. Only when there is a single best move is it that happening. But in chess there are a lot of positions where the choice of move is purely one of style.

[MattieShoes]

Somebody (De Groot?) had a number like 1.75 good moves per typical position, on average. There are obviously lots with only one good move, but then there are lots with several possibilities. Then of course, you can come across positions with a definite best move that is not obvious to a limited depth tree search...

[Hood]

'Apparently not. Only when there is a single best move is it that happening. But in chess there are a lot of positions where the choice of move is purely one of style'

Style ... if it concerns human agree but in the case of program ?

I am trying to be the programmer and chess player. I am not chessprogramer but in that case I think it is the matter of the optimization function not the style.
Using the method of going 'ad absurdum' i am giving the example.

W: Kh1 B: Kg3 Qa2.
Here is mate in one. Qa1, Qb1, Qg2, Qh2
would you like to make the correllation function in that position , what conclusion could you make ?
One can say here are 4 equal moves, one can say that the best is Qa1 because it is the shortest move. I will be the latter. Moving the Q to a1 needs less energy.

Rgds
Chris

[Hood]

"So just take a look in the assembler code of the engine and you know it all. Simple as that. "

Is it really so simple ?
In Java and C family:
++i, i++, i += 1, i = i + 1 can be translated to the same instruction set or not.
In Cobol : compute i = i + 1, add 1 to i ; can be the same as above.

Are ++i, i++ cloning proves that time ?

It was the time before Fruit, when the engines reached some level,
then Fruit was published and there was a jump in the strength of almost all engines. Can we say that all of them are clones of Fruit somehow ?

I think similar can happened with Rybka 3 and post Rybka engines.
including Ippolit families, Komodo etc.
If the program is not modified for the long time the others are closing the gap.

Rgds
Chris

[Karatorian]

I think this is a really interesting idea. Besides the obvious usage as a guide to potential clones, there are many other applications that it could be put too. As suggested above, choosing very different analysis partners, or comparing programs to human grandmasters. One idea I had was that if you wanted to build an engine with multiple "personalities" or playing styles, that you could use the test to mesure how dissimilar they are.

The thing I find the most interesting is the prospect of comparing programs to human players. Simply choose a large random sample of positions from published games of your "test subject" and then run the test. It could be interesting.

One interesting thing you could do with this is to see how similar programs that have had design input from a specific player are to that player.

I was thinking that it was a pity that the test couldn't be (reasonably) applied to human players. Then I thought about it some more had an idea. If you built up a set of correlations between a set of programs and players, shoudn't it be possible to derive a set of correlations between the players, using the correlations between the programs themselves to translate the player-to-program corellations into player-to-player corellations?

I don't know enough about statistics (I really must learn more about it someday) to know if this is feasable, or if there's some error that makes it not work. Does anyone here who does know statistics care to comment?

If it is doable, I think it'd be an interesting project, to compare, say, various world champions, to each other. It would also be interesting to see how closely players from the various "schools" are correllated, etc.

[MattieShoes]

I think this is a really interesting idea. Besides the obvious usage as a guide to potential clones, there are many other applications that it could be put too. As suggested above, choosing very different analysis partners, or comparing programs to human grandmasters. One idea I had was that if you wanted to build an engine with multiple "personalities" or playing styles, that you could use the test to mesure how dissimilar they are.

The thing I find the most interesting is the prospect of comparing programs to human players. Simply choose a large random sample of positions from published games of your "test subject" and then run the test. It could be interesting.

One interesting thing you could do with this is to see how similar programs that have had design input from a specific player are to that player.

I was thinking that it was a pity that the test couldn't be (reasonably) applied to human players. Then I thought about it some more had an idea. If you built up a set of correlations between a set of programs and players, shoudn't it be possible to derive a set of correlations between the players, using the correlations between the programs themselves to translate the player-to-program corellations into player-to-player corellations?

I don't know enough about statistics (I really must learn more about it someday) to know if this is feasable, or if there's some error that makes it not work. Does anyone here who does know statistics care to comment?

If it is doable, I think it'd be an interesting project, to compare, say, various world champions, to each other. It would also be interesting to see how closely players from the various "schools" are correllated, etc.

Another interesting idea would be to have an engine learn a player's style from their games via some sort of automated eval tuning. That wouldn't be possible years ago because the engines were weaker than the uber-GMs... But as computers widen the gap, it might be possible to weaken them to play similar to a specific GM by building a custom book and using some sort of eval tuning.

[Hood]

yes, building correlation tables for:

- engines to humans
- humans to humans

will be very interested and more productive positively then engine to engine for clone hunting (in my shy opinion).

we can use nuclear energy for bad and good

Rgds