Similarity tester - 2nd generation - BETA

[Rebel]

I am working on the documentation of SIMEX 2.0 which is meant to be the successor of Don Dailey's famous SIM03 Similarity Tester.

SIMEX works the same way but is more user-friendly, has more features but the main advantage is that you are no longer limited to the build-in 8238 positions of SIM03 but can create your own positions using EPD, in the download 7 EPD sets are provided for demonstration purposes. SIMEX uses MEA from Ferdinand Mosca as a base.

http://rebel13.nl/misc/simex.html

The documentation is not even half-way and by this thread I take the opportunity to receive comments, ideas before an official release. The page will be frequently updated the coming days.

[Ferdy]

Is simex2 able to generate a matrix similar to sim03?
In sim03 I just use the -m option to generate the following.
Rename midgame.data to similarity.data then run

Code: Select all

sim03w32.exe -m >matrix.txt

matrix.txt

Code: Select all

sim version 3

  Key:

  1) Andscacs_0.93
  2) brainfish_2018
  3) D2019.1.36.50
  4) D2019.2.37.52
  5) Ethereal_11.25
  6) Laser_1.7
  7) Pedone_1.9
  8) Rodent_0.277
  9) RofChade_2.1
 10) RubiChess_1.5
 11) Schooner_2.0.34
 12) Stockfish_10
 13) Texel_1.07

         1     2     3     4     5     6     7     8     9    10    11    12    13
  1.  ----- 54.19 51.77 51.54 54.26 53.53 48.88 53.34 52.86 50.46 54.56 53.27 52.23
  2.  54.19 ----- 52.68 52.99 56.11 56.66 48.83 51.97 55.88 51.49 55.37 63.38 51.65
  3.  51.77 52.68 ----- 65.94 53.69 54.66 52.03 53.96 54.92 51.41 53.44 52.56 51.95
  4.  51.54 52.99 65.94 ----- 53.73 54.54 51.64 53.86 55.06 51.70 53.06 52.69 51.75
  5.  54.26 56.11 53.69 53.73 ----- 57.94 50.33 53.59 58.35 54.09 55.68 56.66 54.05
  6.  53.53 56.66 54.66 54.54 57.94 ----- 51.63 54.69 56.43 53.45 55.79 56.34 54.01
  7.  48.88 48.83 52.03 51.64 50.33 51.63 ----- 51.64 50.38 49.36 51.64 48.96 50.59
  8.  53.34 51.97 53.96 53.86 53.59 54.69 51.64 ----- 53.97 52.62 54.29 51.83 55.05
  9.  52.86 55.88 54.92 55.06 58.35 56.43 50.38 53.97 ----- 54.19 55.20 55.81 54.19
 10.  50.46 51.49 51.41 51.70 54.09 53.45 49.36 52.62 54.19 ----- 53.46 51.08 53.11
 11.  54.56 55.37 53.44 53.06 55.68 55.79 51.64 54.29 55.20 53.46 ----- 54.49 52.82
 12.  53.27 63.38 52.56 52.69 56.66 56.34 48.96 51.83 55.81 51.08 54.49 ----- 51.20
 13.  52.23 51.65 51.95 51.75 54.05 54.01 50.59 55.05 54.19 53.11 52.82 51.20 -----

I tried the midgame at 100ms, then use sim03 to generate the matrix. I need the matrix output to generate dendrogram.




And this is from your html output.



Deuteriums are suspects the same is true for Stockfish/Brainfish . The dendrogram agrees with it.

Will release the sim2dendro.py later.

[Rebel]

I completed the documentation.

From the command line:

simex2 data\similarity.data >report.txt

In report.txt you get something like SIM3:

Code: Select all

Andsc asmFi brain Equin Ether Hanni Laser Stock 
----- 51.42 49.40 48.07 50.08 47.41 50.42 49.83 
51.42 ----- 60.66 49.74 53.11 48.75 53.26 61.04 
49.40 60.66 ----- 49.53 52.48 48.07 51.58 62.28 
48.07 49.74 49.53 ----- 49.74 51.29 50.19 49.71 
50.08 53.11 52.48 49.74 ----- 47.89 53.43 52.74 
47.41 48.75 48.07 51.29 47.89 ----- 49.23 48.22 
50.42 53.26 51.58 50.19 53.43 49.23 ----- 52.18 
49.83 61.04 62.28 49.71 52.74 48.22 52.18 ----- 

But maybe it's more easy when I create an output file as input for your sim2dendro.

[Guenther]

I am working on the documentation of SIMEX 2.0 which is meant to be the successor of Don Dailey's famous SIM03 Similarity Tester.

SIMEX works the same way but is more user-friendly, has more features but the main advantage is that you are no longer limited to the build-in 8238 positions of SIM03 but can create your own positions using EPD, in the download 7 EPD sets are provided for demonstration purposes. SIMEX uses MEA from Ferdinand Mosca as a base.

http://rebel13.nl/misc/simex.html

The documentation is not even half-way and by this thread I take the opportunity to receive comments, ideas before an official release. The page will be frequently updated the coming days.

Thanks for this Ed and Ferdy (MEA). Hopefully I can fiddle around with this after summer with own sets of epds.

[Ferdy]

I completed the documentation.

From the command line:

simex2 data\similarity.data >report.txt

In report.txt you get something like SIM3:

Code: Select all

Andsc asmFi brain Equin Ether Hanni Laser Stock 
----- 51.42 49.40 48.07 50.08 47.41 50.42 49.83 
51.42 ----- 60.66 49.74 53.11 48.75 53.26 61.04 
49.40 60.66 ----- 49.53 52.48 48.07 51.58 62.28 
48.07 49.74 49.53 ----- 49.74 51.29 50.19 49.71 
50.08 53.11 52.48 49.74 ----- 47.89 53.43 52.74 
47.41 48.75 48.07 51.29 47.89 ----- 49.23 48.22 
50.42 53.26 51.58 50.19 53.43 49.23 ----- 52.18 
49.83 61.04 62.28 49.71 52.74 48.22 52.18 ----- 

But maybe it's more easy when I create an output file as input for your sim2dendro.

I think it is indeed better for specific input for sim2dendro.
Better is just a csv type formatting with or without a header. Example

engine1 name,-----,51.42,49.40, and so on
engine2 name,51.42,-----,60.66, and so on
and so on

You can just add it at the end of the report.txt so that they are just in one file. That should not be difficult to parse.

The repo is at https://github.com/fsmosca/Similarity-Dendrogram but it can only read sim3 matrix output currently.

If possible perhaps you can add in the report.txt the info on test epd and time used.

[Ovyron]

Is it possible to "reverse engineer" this to make engine personalities that increase their similarity to other engines or to human players? Say, Stockfish already plays kind of similar to Laser, to maybe it'd be possible to make changes to the code that increase this similarity, so we'd get some "Laser Personality", and it'd be possible that it'd play like Laser but with a higher playing strength.

[Rebel]

I completed the documentation.

From the command line:

simex2 data\similarity.data >report.txt

In report.txt you get something like SIM3:

Code: Select all

Andsc asmFi brain Equin Ether Hanni Laser Stock 
----- 51.42 49.40 48.07 50.08 47.41 50.42 49.83 
51.42 ----- 60.66 49.74 53.11 48.75 53.26 61.04 
49.40 60.66 ----- 49.53 52.48 48.07 51.58 62.28 
48.07 49.74 49.53 ----- 49.74 51.29 50.19 49.71 
50.08 53.11 52.48 49.74 ----- 47.89 53.43 52.74 
47.41 48.75 48.07 51.29 47.89 ----- 49.23 48.22 
50.42 53.26 51.58 50.19 53.43 49.23 ----- 52.18 
49.83 61.04 62.28 49.71 52.74 48.22 52.18 ----- 

But maybe it's more easy when I create an output file as input for your sim2dendro.

I think it is indeed better for specific input for sim2dendro.
Better is just a csv type formatting with or without a header. Example

engine1 name,-----,51.42,49.40, and so on
engine2 name,51.42,-----,60.66, and so on
and so on

You can just add it at the end of the report.txt so that they are just in one file. That should not be difficult to parse.

The repo is at https://github.com/fsmosca/Similarity-Dendrogram but it can only read sim3 matrix output currently.

If possible perhaps you can add in the report.txt the info on test epd and time used.

Made the changes, while creating the HTML a file "dendogram.csv" is also made. Contents:

Code: Select all

epd\simex.epd,8238,100
Andscacs 0.93,-----,51.67,50.21,48.83,50.21,48.28,50.48,44.92,49.69,50.87
asmFish 2017,51.67,-----,60.21,50.07,53.00,49.06,53.38,45.74,52.27,60.65
brainfish 2018,50.21,60.21,-----,49.54,52.43,48.21,51.33,45.49,51.57,62.40
Equinox 3.30,48.83,50.07,49.54,-----,49.94,51.91,50.33,47.01,50.68,49.97
Ethereal 11.25,50.21,53.00,52.43,49.94,-----,48.22,53.40,49.03,51.66,52.74
Hannibal 1.7,48.28,49.06,48.21,51.91,48.22,-----,49.43,43.84,49.12,48.51
Laser 1.7,50.48,53.38,51.33,50.33,53.40,49.43,-----,48.21,51.21,52.31
RubiChess 1.4,44.92,45.74,45.49,47.01,49.03,43.84,48.21,-----,47.08,44.69
Schooner 2.0.34,49.69,52.27,51.57,50.68,51.66,49.12,51.21,47.08,-----,50.53
Stockfish 10,50.87,60.65,62.40,49.97,52.74,48.51,52.31,44.69,50.53,-----

Header contains used EPD, number positions, movetime.

If that's workable for you re-download SIMEX.

http://rebel13.nl/dl/simex2.7z

[Rebel]

Added 8 new engines to the 2500ms statistic.

http://rebel13.nl/html/simex_2500ms_18_engines.html

Don't know what to think about the increasing orange markers. One can say that's only logical, more time (in general) means better moves.

[Ferdy]

I completed the documentation.

From the command line:

simex2 data\similarity.data >report.txt

In report.txt you get something like SIM3:

Code: Select all

Andsc asmFi brain Equin Ether Hanni Laser Stock 
----- 51.42 49.40 48.07 50.08 47.41 50.42 49.83 
51.42 ----- 60.66 49.74 53.11 48.75 53.26 61.04 
49.40 60.66 ----- 49.53 52.48 48.07 51.58 62.28 
48.07 49.74 49.53 ----- 49.74 51.29 50.19 49.71 
50.08 53.11 52.48 49.74 ----- 47.89 53.43 52.74 
47.41 48.75 48.07 51.29 47.89 ----- 49.23 48.22 
50.42 53.26 51.58 50.19 53.43 49.23 ----- 52.18 
49.83 61.04 62.28 49.71 52.74 48.22 52.18 ----- 

But maybe it's more easy when I create an output file as input for your sim2dendro.

I think it is indeed better for specific input for sim2dendro.
Better is just a csv type formatting with or without a header. Example

engine1 name,-----,51.42,49.40, and so on
engine2 name,51.42,-----,60.66, and so on
and so on

You can just add it at the end of the report.txt so that they are just in one file. That should not be difficult to parse.

The repo is at https://github.com/fsmosca/Similarity-Dendrogram but it can only read sim3 matrix output currently.

If possible perhaps you can add in the report.txt the info on test epd and time used.

Made the changes, while creating the HTML a file "dendogram.csv" is also made. Contents:

Code: Select all

epd\simex.epd,8238,100
Andscacs 0.93,-----,51.67,50.21,48.83,50.21,48.28,50.48,44.92,49.69,50.87
asmFish 2017,51.67,-----,60.21,50.07,53.00,49.06,53.38,45.74,52.27,60.65
brainfish 2018,50.21,60.21,-----,49.54,52.43,48.21,51.33,45.49,51.57,62.40
Equinox 3.30,48.83,50.07,49.54,-----,49.94,51.91,50.33,47.01,50.68,49.97
Ethereal 11.25,50.21,53.00,52.43,49.94,-----,48.22,53.40,49.03,51.66,52.74
Hannibal 1.7,48.28,49.06,48.21,51.91,48.22,-----,49.43,43.84,49.12,48.51
Laser 1.7,50.48,53.38,51.33,50.33,53.40,49.43,-----,48.21,51.21,52.31
RubiChess 1.4,44.92,45.74,45.49,47.01,49.03,43.84,48.21,-----,47.08,44.69
Schooner 2.0.34,49.69,52.27,51.57,50.68,51.66,49.12,51.21,47.08,-----,50.53
Stockfish 10,50.87,60.65,62.40,49.97,52.74,48.51,52.31,44.69,50.53,-----

Header contains used EPD, number positions, movetime.

If that's workable for you re-download SIMEX.

http://rebel13.nl/dl/simex2.7z

Downloaded that file an hour ago, no changes.

Code: Select all

Variety (forced nature of positions) overview

 1. 2862 (28.62%)
 2. 4468 (44.68%)
 3. 2669 (26.69%)

Positions 10000 Andsc Deute Xipho 
Andscacs 0.93   ----- 52.73 39.45 
Deuterium v2019.1.36.50                                          52.73 ----- 41.64 
Xiphos 0.5      39.45 41.64 ----- 

 1. Engine Andscacs 0.93 and Deuterium v2019.1.36.50 show a similarity of 52.73%
 2. Engine Deuterium v2019.1.36.50 and Xiphos 0.5 show a similarity of 41.64%
 3. Engine Andscacs 0.93 and Xiphos 0.5 show a similarity of 39.45%


Done...

[Rebel]

Downloaded that file an hour ago, no changes.

Check the file dendogram.csv, should be present.