I am working on the documentation of SIMEX 2.0 which is meant to be the successor of Don Dailey's famous SIM03 Similarity Tester.
SIMEX works the same way but is more user-friendly, has more features but the main advantage is that you are no longer limited to the build-in 8238 positions of SIM03 but can create your own positions using EPD, in the download 7 EPD sets are provided for demonstration purposes. SIMEX uses MEA from Ferdinand Mosca as a base.
http://rebel13.nl/misc/simex.html
The documentation is not even half-way and by this thread I take the opportunity to receive comments, ideas before an official release. The page will be frequently updated the coming days.
Similarity tester - 2nd generation - BETA
Moderators: hgm, Rebel, chrisw
-
- Posts: 6991
- Joined: Thu Aug 18, 2011 12:04 pm
Similarity tester - 2nd generation - BETA
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Similarity tester - 2nd generation - BETA
Is simex2 able to generate a matrix similar to sim03?
In sim03 I just use the -m option to generate the following.
Rename midgame.data to similarity.data then run
matrix.txt
I tried the midgame at 100ms, then use sim03 to generate the matrix. I need the matrix output to generate dendrogram.
And this is from your html output.
Deuteriums are suspects the same is true for Stockfish/Brainfish . The dendrogram agrees with it.
Will release the sim2dendro.py later.
In sim03 I just use the -m option to generate the following.
Rename midgame.data to similarity.data then run
Code: Select all
sim03w32.exe -m >matrix.txt
Code: Select all
sim version 3
Key:
1) Andscacs_0.93
2) brainfish_2018
3) D2019.1.36.50
4) D2019.2.37.52
5) Ethereal_11.25
6) Laser_1.7
7) Pedone_1.9
8) Rodent_0.277
9) RofChade_2.1
10) RubiChess_1.5
11) Schooner_2.0.34
12) Stockfish_10
13) Texel_1.07
1 2 3 4 5 6 7 8 9 10 11 12 13
1. ----- 54.19 51.77 51.54 54.26 53.53 48.88 53.34 52.86 50.46 54.56 53.27 52.23
2. 54.19 ----- 52.68 52.99 56.11 56.66 48.83 51.97 55.88 51.49 55.37 63.38 51.65
3. 51.77 52.68 ----- 65.94 53.69 54.66 52.03 53.96 54.92 51.41 53.44 52.56 51.95
4. 51.54 52.99 65.94 ----- 53.73 54.54 51.64 53.86 55.06 51.70 53.06 52.69 51.75
5. 54.26 56.11 53.69 53.73 ----- 57.94 50.33 53.59 58.35 54.09 55.68 56.66 54.05
6. 53.53 56.66 54.66 54.54 57.94 ----- 51.63 54.69 56.43 53.45 55.79 56.34 54.01
7. 48.88 48.83 52.03 51.64 50.33 51.63 ----- 51.64 50.38 49.36 51.64 48.96 50.59
8. 53.34 51.97 53.96 53.86 53.59 54.69 51.64 ----- 53.97 52.62 54.29 51.83 55.05
9. 52.86 55.88 54.92 55.06 58.35 56.43 50.38 53.97 ----- 54.19 55.20 55.81 54.19
10. 50.46 51.49 51.41 51.70 54.09 53.45 49.36 52.62 54.19 ----- 53.46 51.08 53.11
11. 54.56 55.37 53.44 53.06 55.68 55.79 51.64 54.29 55.20 53.46 ----- 54.49 52.82
12. 53.27 63.38 52.56 52.69 56.66 56.34 48.96 51.83 55.81 51.08 54.49 ----- 51.20
13. 52.23 51.65 51.95 51.75 54.05 54.01 50.59 55.05 54.19 53.11 52.82 51.20 -----
I tried the midgame at 100ms, then use sim03 to generate the matrix. I need the matrix output to generate dendrogram.
And this is from your html output.
Deuteriums are suspects the same is true for Stockfish/Brainfish . The dendrogram agrees with it.
Will release the sim2dendro.py later.
-
- Posts: 6991
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Similarity tester - 2nd generation - BETA
I completed the documentation.
From the command line:
simex2 data\similarity.data >report.txt
In report.txt you get something like SIM3:
But maybe it's more easy when I create an output file as input for your sim2dendro.
From the command line:
simex2 data\similarity.data >report.txt
In report.txt you get something like SIM3:
Code: Select all
Andsc asmFi brain Equin Ether Hanni Laser Stock
----- 51.42 49.40 48.07 50.08 47.41 50.42 49.83
51.42 ----- 60.66 49.74 53.11 48.75 53.26 61.04
49.40 60.66 ----- 49.53 52.48 48.07 51.58 62.28
48.07 49.74 49.53 ----- 49.74 51.29 50.19 49.71
50.08 53.11 52.48 49.74 ----- 47.89 53.43 52.74
47.41 48.75 48.07 51.29 47.89 ----- 49.23 48.22
50.42 53.26 51.58 50.19 53.43 49.23 ----- 52.18
49.83 61.04 62.28 49.71 52.74 48.22 52.18 -----
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 4605
- Joined: Wed Oct 01, 2008 6:33 am
- Location: Regensburg, Germany
- Full name: Guenther Simon
Re: Similarity tester - 2nd generation - BETA
Thanks for this Ed and Ferdy (MEA). Hopefully I can fiddle around with this after summer with own sets of epds.Rebel wrote: ↑Fri Aug 09, 2019 11:32 pm I am working on the documentation of SIMEX 2.0 which is meant to be the successor of Don Dailey's famous SIM03 Similarity Tester.
SIMEX works the same way but is more user-friendly, has more features but the main advantage is that you are no longer limited to the build-in 8238 positions of SIM03 but can create your own positions using EPD, in the download 7 EPD sets are provided for demonstration purposes. SIMEX uses MEA from Ferdinand Mosca as a base.
http://rebel13.nl/misc/simex.html
The documentation is not even half-way and by this thread I take the opportunity to receive comments, ideas before an official release. The page will be frequently updated the coming days.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Similarity tester - 2nd generation - BETA
I think it is indeed better for specific input for sim2dendro.Rebel wrote: ↑Sat Aug 10, 2019 8:25 pm I completed the documentation.
From the command line:
simex2 data\similarity.data >report.txt
In report.txt you get something like SIM3:But maybe it's more easy when I create an output file as input for your sim2dendro.Code: Select all
Andsc asmFi brain Equin Ether Hanni Laser Stock ----- 51.42 49.40 48.07 50.08 47.41 50.42 49.83 51.42 ----- 60.66 49.74 53.11 48.75 53.26 61.04 49.40 60.66 ----- 49.53 52.48 48.07 51.58 62.28 48.07 49.74 49.53 ----- 49.74 51.29 50.19 49.71 50.08 53.11 52.48 49.74 ----- 47.89 53.43 52.74 47.41 48.75 48.07 51.29 47.89 ----- 49.23 48.22 50.42 53.26 51.58 50.19 53.43 49.23 ----- 52.18 49.83 61.04 62.28 49.71 52.74 48.22 52.18 -----
Better is just a csv type formatting with or without a header. Example
engine1 name,-----,51.42,49.40, and so on
engine2 name,51.42,-----,60.66, and so on
and so on
You can just add it at the end of the report.txt so that they are just in one file. That should not be difficult to parse.
The repo is at https://github.com/fsmosca/Similarity-Dendrogram but it can only read sim3 matrix output currently.
If possible perhaps you can add in the report.txt the info on test epd and time used.
-
- Posts: 4556
- Joined: Tue Jul 03, 2007 4:30 am
Re: Similarity tester - 2nd generation - BETA
Is it possible to "reverse engineer" this to make engine personalities that increase their similarity to other engines or to human players? Say, Stockfish already plays kind of similar to Laser, to maybe it'd be possible to make changes to the code that increase this similarity, so we'd get some "Laser Personality", and it'd be possible that it'd play like Laser but with a higher playing strength.
Your beliefs create your reality, so be careful what you wish for.
-
- Posts: 6991
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Similarity tester - 2nd generation - BETA
Made the changes, while creating the HTML a file "dendogram.csv" is also made. Contents:Ferdy wrote: ↑Sun Aug 11, 2019 3:25 amI think it is indeed better for specific input for sim2dendro.Rebel wrote: ↑Sat Aug 10, 2019 8:25 pm I completed the documentation.
From the command line:
simex2 data\similarity.data >report.txt
In report.txt you get something like SIM3:But maybe it's more easy when I create an output file as input for your sim2dendro.Code: Select all
Andsc asmFi brain Equin Ether Hanni Laser Stock ----- 51.42 49.40 48.07 50.08 47.41 50.42 49.83 51.42 ----- 60.66 49.74 53.11 48.75 53.26 61.04 49.40 60.66 ----- 49.53 52.48 48.07 51.58 62.28 48.07 49.74 49.53 ----- 49.74 51.29 50.19 49.71 50.08 53.11 52.48 49.74 ----- 47.89 53.43 52.74 47.41 48.75 48.07 51.29 47.89 ----- 49.23 48.22 50.42 53.26 51.58 50.19 53.43 49.23 ----- 52.18 49.83 61.04 62.28 49.71 52.74 48.22 52.18 -----
Better is just a csv type formatting with or without a header. Example
engine1 name,-----,51.42,49.40, and so on
engine2 name,51.42,-----,60.66, and so on
and so on
You can just add it at the end of the report.txt so that they are just in one file. That should not be difficult to parse.
The repo is at https://github.com/fsmosca/Similarity-Dendrogram but it can only read sim3 matrix output currently.
If possible perhaps you can add in the report.txt the info on test epd and time used.
Code: Select all
epd\simex.epd,8238,100
Andscacs 0.93,-----,51.67,50.21,48.83,50.21,48.28,50.48,44.92,49.69,50.87
asmFish 2017,51.67,-----,60.21,50.07,53.00,49.06,53.38,45.74,52.27,60.65
brainfish 2018,50.21,60.21,-----,49.54,52.43,48.21,51.33,45.49,51.57,62.40
Equinox 3.30,48.83,50.07,49.54,-----,49.94,51.91,50.33,47.01,50.68,49.97
Ethereal 11.25,50.21,53.00,52.43,49.94,-----,48.22,53.40,49.03,51.66,52.74
Hannibal 1.7,48.28,49.06,48.21,51.91,48.22,-----,49.43,43.84,49.12,48.51
Laser 1.7,50.48,53.38,51.33,50.33,53.40,49.43,-----,48.21,51.21,52.31
RubiChess 1.4,44.92,45.74,45.49,47.01,49.03,43.84,48.21,-----,47.08,44.69
Schooner 2.0.34,49.69,52.27,51.57,50.68,51.66,49.12,51.21,47.08,-----,50.53
Stockfish 10,50.87,60.65,62.40,49.97,52.74,48.51,52.31,44.69,50.53,-----
If that's workable for you re-download SIMEX.
http://rebel13.nl/dl/simex2.7z
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 6991
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Similarity tester - 2nd generation - BETA
Added 8 new engines to the 2500ms statistic.
http://rebel13.nl/html/simex_2500ms_18_engines.html
Don't know what to think about the increasing orange markers. One can say that's only logical, more time (in general) means better moves.
http://rebel13.nl/html/simex_2500ms_18_engines.html
Don't know what to think about the increasing orange markers. One can say that's only logical, more time (in general) means better moves.
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Similarity tester - 2nd generation - BETA
Downloaded that file an hour ago, no changes.Rebel wrote: ↑Sun Aug 11, 2019 9:16 amMade the changes, while creating the HTML a file "dendogram.csv" is also made. Contents:Ferdy wrote: ↑Sun Aug 11, 2019 3:25 amI think it is indeed better for specific input for sim2dendro.Rebel wrote: ↑Sat Aug 10, 2019 8:25 pm I completed the documentation.
From the command line:
simex2 data\similarity.data >report.txt
In report.txt you get something like SIM3:But maybe it's more easy when I create an output file as input for your sim2dendro.Code: Select all
Andsc asmFi brain Equin Ether Hanni Laser Stock ----- 51.42 49.40 48.07 50.08 47.41 50.42 49.83 51.42 ----- 60.66 49.74 53.11 48.75 53.26 61.04 49.40 60.66 ----- 49.53 52.48 48.07 51.58 62.28 48.07 49.74 49.53 ----- 49.74 51.29 50.19 49.71 50.08 53.11 52.48 49.74 ----- 47.89 53.43 52.74 47.41 48.75 48.07 51.29 47.89 ----- 49.23 48.22 50.42 53.26 51.58 50.19 53.43 49.23 ----- 52.18 49.83 61.04 62.28 49.71 52.74 48.22 52.18 -----
Better is just a csv type formatting with or without a header. Example
engine1 name,-----,51.42,49.40, and so on
engine2 name,51.42,-----,60.66, and so on
and so on
You can just add it at the end of the report.txt so that they are just in one file. That should not be difficult to parse.
The repo is at https://github.com/fsmosca/Similarity-Dendrogram but it can only read sim3 matrix output currently.
If possible perhaps you can add in the report.txt the info on test epd and time used.Header contains used EPD, number positions, movetime.Code: Select all
epd\simex.epd,8238,100 Andscacs 0.93,-----,51.67,50.21,48.83,50.21,48.28,50.48,44.92,49.69,50.87 asmFish 2017,51.67,-----,60.21,50.07,53.00,49.06,53.38,45.74,52.27,60.65 brainfish 2018,50.21,60.21,-----,49.54,52.43,48.21,51.33,45.49,51.57,62.40 Equinox 3.30,48.83,50.07,49.54,-----,49.94,51.91,50.33,47.01,50.68,49.97 Ethereal 11.25,50.21,53.00,52.43,49.94,-----,48.22,53.40,49.03,51.66,52.74 Hannibal 1.7,48.28,49.06,48.21,51.91,48.22,-----,49.43,43.84,49.12,48.51 Laser 1.7,50.48,53.38,51.33,50.33,53.40,49.43,-----,48.21,51.21,52.31 RubiChess 1.4,44.92,45.74,45.49,47.01,49.03,43.84,48.21,-----,47.08,44.69 Schooner 2.0.34,49.69,52.27,51.57,50.68,51.66,49.12,51.21,47.08,-----,50.53 Stockfish 10,50.87,60.65,62.40,49.97,52.74,48.51,52.31,44.69,50.53,-----
If that's workable for you re-download SIMEX.
http://rebel13.nl/dl/simex2.7z
Code: Select all
Variety (forced nature of positions) overview
1. 2862 (28.62%)
2. 4468 (44.68%)
3. 2669 (26.69%)
Positions 10000 Andsc Deute Xipho
Andscacs 0.93 ----- 52.73 39.45
Deuterium v2019.1.36.50 52.73 ----- 41.64
Xiphos 0.5 39.45 41.64 -----
1. Engine Andscacs 0.93 and Deuterium v2019.1.36.50 show a similarity of 52.73%
2. Engine Deuterium v2019.1.36.50 and Xiphos 0.5 show a similarity of 41.64%
3. Engine Andscacs 0.93 and Xiphos 0.5 show a similarity of 39.45%
Done...
-
- Posts: 6991
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Similarity tester - 2nd generation - BETA
Check the file dendogram.csv, should be present.
90% of coding is debugging, the other 10% is writing bugs.