I created a utility called similar which measures how different one chess program is from others. It does this by running 2000 position from random games and noting how often the moves agree and as output returns the percentage of moves that match.
You can get it here: http://komodochess.com/pub/similar.zip
Here is some sample output, comparing Robbolito with a few other programs:
------ RobboLito version 0.084 (time: 100 ms) ------
69.25 Houdini 1.5 w32 (time: 100 ms)
66.90 Rybka 3 (time: 100 ms)
61.70 Stockfish 1.9.1 JA 64bit (time: 100 ms)
61.35 Stockfish 1.8 JA (time: 100 ms)
59.80 Komodo64 1.2 JA (time: 100 ms)
59.15 Komodo 1.0 (time: 100 ms)
58.95 Stockfish 1.7.1 64bit (time: 100 ms)
58.95 Stockfish 1.6 64bit (time: 100 ms)
57.00 Fruit 2.3.1 (time: 100 ms)
56.20 Fruit 2.1 (time: 100 ms)
I have not tested this on windows so I'm hoping to get some feedback specific to windows.
The similar.exe is designed to run on 64 bit windows and is actually a tcl script wrapped up with a tcl runtime using tclkit technology. I am also including the "starkit" which is platform independent, but requires a tclkit runtime for your platform. It is similar to a jar file and can be taken apart and inspected and modified if you wish - assuming you know how to work with starkit's and such. google for starkit and sdx.kit for more information.
Please let me know if you find this interesting or useful. Email me at drd@mit.edu
Don
Similarity Detector Available
Moderators: hgm, Rebel, chrisw
-
- Posts: 918
- Joined: Mon Jan 05, 2009 7:40 pm
- Location: Germany
- Full name: Engin Üstün
Re: clone tester available
ah, yeah now we can see how similar houdini it is from Robbolito
-
- Posts: 3019
- Joined: Wed Mar 08, 2006 9:57 pm
- Location: Rio de Janeiro, Brazil
Re: clone tester available
Have you tried this method comparing the default Rybka 3 to the Dynamic and Human flavors?
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."
-
- Posts: 10948
- Joined: Wed Jul 26, 2006 10:21 pm
- Full name: Kai Laskos
Re: clone tester available
Thanks Don, my first try, seems to work
C:\similar>similar.exe -r 1
------ Ivanhoe_B49jAx64 (time: 100 ms) ------
68.05 Houdini 1.5 x64 (time: 100 ms)
Now I will prepare a battery of engines
Kai
C:\similar>similar.exe -r 1
------ Ivanhoe_B49jAx64 (time: 100 ms) ------
68.05 Houdini 1.5 x64 (time: 100 ms)
Now I will prepare a battery of engines
Kai
-
- Posts: 2129
- Joined: Thu May 29, 2008 10:43 am
Re: clone tester available
wow, amazing...Don wrote:I created a utility called similar which measures how different one chess program is from others. It does this by running 2000 position from random games and noting how often the moves agree and as output returns the percentage of moves that match.
You can get it here: http://komodochess.com/pub/similar.zip
Here is some sample output, comparing Robbolito with a few other programs:
------ RobboLito version 0.084 (time: 100 ms) ------
69.25 Houdini 1.5 w32 (time: 100 ms)
66.90 Rybka 3 (time: 100 ms)
61.70 Stockfish 1.9.1 JA 64bit (time: 100 ms)
61.35 Stockfish 1.8 JA (time: 100 ms)
59.80 Komodo64 1.2 JA (time: 100 ms)
59.15 Komodo 1.0 (time: 100 ms)
58.95 Stockfish 1.7.1 64bit (time: 100 ms)
58.95 Stockfish 1.6 64bit (time: 100 ms)
57.00 Fruit 2.3.1 (time: 100 ms)
56.20 Fruit 2.1 (time: 100 ms)
I have not tested this on windows so I'm hoping to get some feedback specific to windows.
The similar.exe is designed to run on 64 bit windows and is actually a tcl script wrapped up with a tcl runtime using tclkit technology. I am also including the "starkit" which is platform independent, but requires a tclkit runtime for your platform. It is similar to a jar file and can be taken apart and inspected and modified if you wish - assuming you know how to work with starkit's and such. google for starkit and sdx.kit for more information.
Please let me know if you find this interesting or useful. Email me at drd@mit.edu
Don
komodo and stockfish are much more similar than I would have guessed.
i was genuinely under the apparently incorrect 'assumption' that they were very different...
weird...
(i.e i thought they would certainly fail any 'clone' test...?)
-
- Posts: 1154
- Joined: Fri Jun 23, 2006 5:18 am
Re: clone tester available
Now, thanks to this cool tool, you can be specific about such claims. Are you talking level 70 similarity, or closer to 60 level similarity . One thing that is fun about this method of comparison is it provides an objective metric of comparison...a welcome change of pace from the more wishy washy statements that are often flung around here on this topic.kranium wrote: wow, amazing...
komodo and stockfish are much more similar than I would have guessed.
-Sam
-
- Posts: 2129
- Joined: Thu May 29, 2008 10:43 am
Re: clone tester available
yes, in this case komodo seems absolutley equal to stockfish 1.8....according to CCRL....aaaargh!?BubbaTough wrote:Now, thanks to this cool tool, you can be specific about such claims. Are you talking level 70 similarity, or closer to 60 level similarity . One thing that is fun about this method of comparison is it provides an objective metric of comparison...a welcome change of pace from the more wishy washy statements that are often flung around here on this topic.kranium wrote: wow, amazing...
komodo and stockfish are much more similar than I would have guessed.
-Sam
wait!
much better to leave it to the amateur user...
please test independently
(i.e. test yourself if you want reliable/unbiased results).
Last edited by kranium on Mon Dec 27, 2010 2:19 am, edited 1 time in total.
-
- Posts: 736
- Joined: Wed Dec 02, 2009 9:48 am
Re: clone tester available
If I am not mistaken the numbers that Don is showing are relevant to how Roblito compares to Komodo and then Robolito compares to Stockfish. The numbers are not a comparison of Komodo to Stockfish.kranium wrote:wow, amazing...Don wrote:I created a utility called similar which measures how different one chess program is from others. It does this by running 2000 position from random games and noting how often the moves agree and as output returns the percentage of moves that match.
You can get it here: http://komodochess.com/pub/similar.zip
Here is some sample output, comparing Robbolito with a few other programs:
------ RobboLito version 0.084 (time: 100 ms) ------
69.25 Houdini 1.5 w32 (time: 100 ms)
66.90 Rybka 3 (time: 100 ms)
61.70 Stockfish 1.9.1 JA 64bit (time: 100 ms)
61.35 Stockfish 1.8 JA (time: 100 ms)
59.80 Komodo64 1.2 JA (time: 100 ms)
59.15 Komodo 1.0 (time: 100 ms)
58.95 Stockfish 1.7.1 64bit (time: 100 ms)
58.95 Stockfish 1.6 64bit (time: 100 ms)
57.00 Fruit 2.3.1 (time: 100 ms)
56.20 Fruit 2.1 (time: 100 ms)
I have not tested this on windows so I'm hoping to get some feedback specific to windows.
The similar.exe is designed to run on 64 bit windows and is actually a tcl script wrapped up with a tcl runtime using tclkit technology. I am also including the "starkit" which is platform independent, but requires a tclkit runtime for your platform. It is similar to a jar file and can be taken apart and inspected and modified if you wish - assuming you know how to work with starkit's and such. google for starkit and sdx.kit for more information.
Please let me know if you find this interesting or useful. Email me at drd@mit.edu
Don
komodo and stockfish are much more similar than I would have guessed.
i was genuinely under the apparently incorrect 'assumption' that they were very different...
weird...
(i.e i thought they would certainly fail any 'clone' test...?)
-
- Posts: 2129
- Joined: Thu May 29, 2008 10:43 am
Re: clone tester available
yes of course...tomgdrums wrote:If I am not mistaken the numbers that Don is showing are relevant to how Roblito compares to Komodo and then Robolito compares to Stockfish. The numbers are not a comparison of Komodo to Stockfish.kranium wrote:wow, amazing...Don wrote:I created a utility called similar which measures how different one chess program is from others. It does this by running 2000 position from random games and noting how often the moves agree and as output returns the percentage of moves that match.
You can get it here: http://komodochess.com/pub/similar.zip
Here is some sample output, comparing Robbolito with a few other programs:
------ RobboLito version 0.084 (time: 100 ms) ------
69.25 Houdini 1.5 w32 (time: 100 ms)
66.90 Rybka 3 (time: 100 ms)
61.70 Stockfish 1.9.1 JA 64bit (time: 100 ms)
61.35 Stockfish 1.8 JA (time: 100 ms)
59.80 Komodo64 1.2 JA (time: 100 ms)
59.15 Komodo 1.0 (time: 100 ms)
58.95 Stockfish 1.7.1 64bit (time: 100 ms)
58.95 Stockfish 1.6 64bit (time: 100 ms)
57.00 Fruit 2.3.1 (time: 100 ms)
56.20 Fruit 2.1 (time: 100 ms)
I have not tested this on windows so I'm hoping to get some feedback specific to windows.
The similar.exe is designed to run on 64 bit windows and is actually a tcl script wrapped up with a tcl runtime using tclkit technology. I am also including the "starkit" which is platform independent, but requires a tclkit runtime for your platform. It is similar to a jar file and can be taken apart and inspected and modified if you wish - assuming you know how to work with starkit's and such. google for starkit and sdx.kit for more information.
Please let me know if you find this interesting or useful. Email me at drd@mit.edu
Don
komodo and stockfish are much more similar than I would have guessed.
i was genuinely under the apparently incorrect 'assumption' that they were very different...
weird...
(i.e i thought they would certainly fail any 'clone' test...?)
it's all about whether Robolito is a clone or not...
other engines are exempt...i understand completely now.
thanx Tom!
-
- Posts: 6401
- Joined: Thu Mar 09, 2006 8:30 pm
- Location: Chicago, Illinois, USA
Re: clone tester available
Exactly because this is not a 1D plot, it is at least 2D. This like saying Chicago is 900 miles from NY, and Chicago is 1000 miles from Denver, so Denver is very close to NY. No!!tomgdrums wrote:If I am not mistaken the numbers that Don is showing are relevant to how Roblito compares to Komodo and then Robolito compares to Stockfish. The numbers are not a comparison of Komodo to Stockfish.kranium wrote:wow, amazing...Don wrote:I created a utility called similar which measures how different one chess program is from others. It does this by running 2000 position from random games and noting how often the moves agree and as output returns the percentage of moves that match.
You can get it here: http://komodochess.com/pub/similar.zip
Here is some sample output, comparing Robbolito with a few other programs:
------ RobboLito version 0.084 (time: 100 ms) ------
69.25 Houdini 1.5 w32 (time: 100 ms)
66.90 Rybka 3 (time: 100 ms)
61.70 Stockfish 1.9.1 JA 64bit (time: 100 ms)
61.35 Stockfish 1.8 JA (time: 100 ms)
59.80 Komodo64 1.2 JA (time: 100 ms)
59.15 Komodo 1.0 (time: 100 ms)
58.95 Stockfish 1.7.1 64bit (time: 100 ms)
58.95 Stockfish 1.6 64bit (time: 100 ms)
57.00 Fruit 2.3.1 (time: 100 ms)
56.20 Fruit 2.1 (time: 100 ms)
I have not tested this on windows so I'm hoping to get some feedback specific to windows.
The similar.exe is designed to run on 64 bit windows and is actually a tcl script wrapped up with a tcl runtime using tclkit technology. I am also including the "starkit" which is platform independent, but requires a tclkit runtime for your platform. It is similar to a jar file and can be taken apart and inspected and modified if you wish - assuming you know how to work with starkit's and such. google for starkit and sdx.kit for more information.
Please let me know if you find this interesting or useful. Email me at drd@mit.edu
Don
komodo and stockfish are much more similar than I would have guessed.
i was genuinely under the apparently incorrect 'assumption' that they were very different...
weird...
(i.e i thought they would certainly fail any 'clone' test...?)
Miguel