The STS (Strategic Test Suite) was created by Swamithan and Dann Corbit and contains 1500 carefully chosen positions with multiple good moves varying with bonus points in the range of 10 to 1. The 1500 positions were checked and double checked with the best software available at the time. And herein lies a problem, during the years engines has increased 300-400 elo points and the set has become outdated as one can see when run the STS set with about 12 random picked strong engines, running MEA we get:
Code: Select all
EPD : sts.epd
Time : 1000ms
Top Top Max Total Time Hash
Engine Score Rating Hits Pos Rate Score Rate ms Mb Cpu
1 Stockfish 11 13266 3536 1203 1500 0.802 15000 0.884 1000 64 1
2 Komodo 10 13089 3491 1193 1500 0.795 15000 0.873 1000 64 1
3 Ethereal 12 12876 3431 1171 1500 0.781 15000 0.858 1000 64 1
4 Laser 1.7 12868 3431 1164 1500 0.776 15000 0.858 1000 64 1
5 Xiphos 0.6 12815 3415 1149 1500 0.766 15000 0.854 1000 64 1
6 Andscacs 0.95 12774 3407 1161 1500 0.774 15000 0.852 1000 64 1
7 rofChade 2.2 12608 3363 1121 1500 0.747 15000 0.841 1000 64 1
8 lc0 12595 3359 1123 1500 0.749 15000 0.840 1000 64 1
9 Wasp 3.75 12417 3311 1099 1500 0.733 15000 0.828 1000 64 1
10 Arasan 21.3 12033 3208 1052 1500 0.701 15000 0.802 1000 64 1
11 Fire 7.1 11892 3172 1023 1500 0.682 15000 0.793 1000 64 1
12 RubiChess 1.4 11863 3164 1030 1500 0.687 15000 0.791 1000 64 1
Created with MEA
by
Ferdinand
Mosca
The tool tries to fix this problem, no more handcraft, all automatic. How it works:
1. I ran the 1500 positions with SF11, MultiPV=4, 20 cores, 60 seconds per move, result: sts-sf11.epd
2. I ran the 1500 positions with Lc0, MultiPV=4, 60 seconds per move, result: sts-lc0.epd
The best move gets 10 points and based on the evaluation score differences the points of next 3 moves are calculated.
3. Combine sts-sf11 and sts-lc0, result: sts-2020.epd. points are added when moves are equal, else moves are added.
The results, first sts-sf11.epd and of course SF11 is excluded.
Code: Select all
EPD : sts-sf11.epd
Time : 1000ms
Top Top Max Total Time Hash
Engine Score Rating Hits Pos Rate Score Rate ms Mb Cpu
1 lc0 12410 3307 1066 1500 0.711 15000 0.827 1000 64 1
2 Komodo 10 12210 3255 1032 1500 0.688 15000 0.814 1000 64 1
3 Ethereal 12 11991 3196 1011 1500 0.674 15000 0.799 1000 64 1
4 Xiphos 0.6 11903 3176 1008 1500 0.672 15000 0.794 1000 64 1
5 Laser 1.7 11859 3164 1001 1500 0.667 15000 0.791 1000 64 1
6 Andscacs 0.95 11850 3160 991 1500 0.661 15000 0.790 1000 64 1
7 rofChade 2.2 11780 3140 986 1500 0.657 15000 0.785 1000 64 1
8 Wasp 3.75 11417 3044 949 1500 0.633 15000 0.761 1000 64 1
9 Arasan 21.3 11083 2955 910 1500 0.607 15000 0.739 1000 64 1
10 Fire 7.1 11032 2939 903 1500 0.602 15000 0.735 1000 64 1
11 RubiChess 1.4 10762 2867 891 1500 0.594 15000 0.717 1000 64 1
Created with MEA
by
Ferdinand
Mosca
Second sts-lc0.epd and of course Lc0 is excluded.
Code: Select all
EPD : sts-lc0.epd
Time : 1000ms
Top Top Max Total Time Hash
Engine Score Rating Hits Pos Rate Score Rate ms Mb Cpu
1 Stockfish 11 12813 3415 1121 1500 0.747 15000 0.854 1000 64 1
2 Ethereal 12 12265 3271 1057 1500 0.705 15000 0.818 1000 64 1
3 Komodo 10 12250 3267 1055 1500 0.703 15000 0.817 1000 64 1
4 Laser 1.7 12158 3244 1040 1500 0.693 15000 0.811 1000 64 1
5 Xiphos 0.6 12052 3212 1043 1500 0.695 15000 0.803 1000 64 1
6 rofChade 2.2 11997 3200 1028 1500 0.685 15000 0.800 1000 64 1
7 Andscacs 0.95 11920 3180 1011 1500 0.674 15000 0.795 1000 64 1
8 Wasp 3.75 11543 3080 978 1500 0.652 15000 0.770 1000 64 1
9 Arasan 21.3 11334 3024 954 1500 0.636 15000 0.756 1000 64 1
10 RubiChess 1.4 11234 2995 942 1500 0.628 15000 0.749 1000 64 1
11 Fire 7.1 11150 2971 929 1500 0.619 15000 0.743 1000 64 1
Created with MEA
by
Ferdinand
Mosca
Code: Select all
EPD : sts-2020.epd
Time : 1000ms
Top Top Max Total Time Hash
Engine Score Rating Hits Pos Rate Score Rate ms Mb Cpu
1 Komodo 10 16194 3427 1025 1500 0.683 18888 0.857 1000 64 1
2 Xiphos 0.6 16062 3399 1011 1500 0.674 18888 0.850 1000 64 1
3 Ethereal 12 16044 3395 1010 1500 0.673 18888 0.849 1000 64 1
4 Laser 1.7 16041 3395 1000 1500 0.667 18888 0.849 1000 64 1
5 rofChade 2.2 16033 3395 986 1500 0.657 18888 0.849 1000 64 1
6 Andscacs 0.95 15911 3367 988 1500 0.659 18888 0.842 1000 64 1
7 Wasp 3.75 15654 3315 951 1500 0.634 18888 0.829 1000 64 1
8 Fire 7.1 15588 3299 901 1500 0.601 18888 0.825 1000 64 1
9 Arasan 21.3 15354 3251 909 1500 0.606 18888 0.813 1000 64 1
10 RubiChess 1.4 15045 3188 890 1500 0.593 18888 0.797 1000 64 1
Created with MEA
by
Ferdinand
Mosca