STS test suite and engine analysis interface

flok · Post by **flok** » Sun Dec 25, 2016 10:12 pm

Thanks!

Note that the program mail fail at the end. This can be resolved with the following patch:

--- sts_rating_v13.1.py.org	2016-12-25 20:33:16.742365027 +0100
+++ sts_rating_v13.1.py	2016-12-25 20:34:18.338337318 +0100
@@ -685,9 +685,12 @@
                             
 
     # Quit the engine
-    p.stdin.write("quit\n")
-    p.stdin.close()
-    p.communicate()
+    try:
+        p.stdin.write("quit\n")
+        p.stdin.close()
+        p.communicate()
+    except:
+        pass
 
     timeEnd = time.clock()

Embla (rev 3611):

Code: Select all

STS Rating v13.1
Number of cores: 4

Engine: Emblatrunk-NM3
Hash: 32, Threads: 1, time/pos: 1.000s

Number of positions in STS1-STS15_LAN_v3.epd: 1500
Max score = 1500 x 10 = 15000
Test duration: 00h:00m:01s
Expected time to finish: 00h:25m:45s

  STS ID   STS1   STS2   STS3   STS4   STS5   STS6   STS7   STS8   STS9  STS10  STS11  STS12  STS13  STS14  STS15    ALL
  NumPos    100    100    100    100    100    100    100    100    100    100    100    100    100    100    100   1500
 BestCnt     38     22     38     33     54     49     27     25     30     57     12     35     55     38     13    526
   Score    492    316    544    442    644    725    402    429    419    637    279    499    662    498    353   7341
Score(%)   49.2   31.6   54.4   44.2   64.4   72.5   40.2   42.9   41.9   63.7   27.9   49.9   66.2   49.8   35.3   48.9

:: STS ID and Titles ::
STS 01: Undermining
STS 02: Open Files and Diagonals
STS 03: Knight Outposts
STS 04: Square Vacancy
STS 05: Bishop vs Knight
STS 06: Re-Capturing
STS 07: Offer of Simplification
STS 08: Advancement of f/g/h Pawns
STS 09: Advancement of a/b/c Pawns
STS 10: Simplification
STS 11: Activity of the King
STS 12: Center Control
STS 13: Pawn Play in the Center
STS 14: Queens and Rooks to the 7th rank
STS 15: Avoid Pointless Exchange

:: Top 5 STS with high result ::
1. STS 06, 72.5%, "Re-Capturing"
2. STS 13, 66.2%, "Pawn Play in the Center"
3. STS 05, 64.4%, "Bishop vs Knight"
4. STS 10, 63.7%, "Simplification"
5. STS 03, 54.4%, "Knight Outposts"

:: Top 5 STS with low result ::
1. STS 11, 27.9%, "Activity of the King"
2. STS 02, 31.6%, "Open Files and Diagonals"
3. STS 15, 35.3%, "Avoid Pointless Exchange"
4. STS 07, 40.2%, "Offer of Simplification"
5. STS 09, 41.9%, "Advancement of a/b/c Pawns"

MikeB · Post by **MikeB** » Sun Dec 25, 2016 10:47 pm

flok wrote:Thanks!

Note that the program mail fail at the end. This can be resolved with the following patch:

Code: Select all

--- sts_rating_v13.1.py.org	2016-12-25 20:33:16.742365027 +0100
+++ sts_rating_v13.1.py	2016-12-25 20:34:18.338337318 +0100
@@ -685,9 +685,12 @@
                             
 
     # Quit the engine
-    p.stdin.write("quit\n")
-    p.stdin.close()
-    p.communicate()
+    try:
+        p.stdin.write("quit\n")
+        p.stdin.close()
+        p.communicate()
+    except:
+        pass
 
     timeEnd = time.clock()

Embla (rev 3611):

Code: Select all

STS Rating v13.1
Number of cores: 4

Engine: Emblatrunk-NM3
Hash: 32, Threads: 1, time/pos: 1.000s

Number of positions in STS1-STS15_LAN_v3.epd: 1500
Max score = 1500 x 10 = 15000
Test duration: 00h:00m:01s
Expected time to finish: 00h:25m:45s

  STS ID   STS1   STS2   STS3   STS4   STS5   STS6   STS7   STS8   STS9  STS10  STS11  STS12  STS13  STS14  STS15    ALL
  NumPos    100    100    100    100    100    100    100    100    100    100    100    100    100    100    100   1500
 BestCnt     38     22     38     33     54     49     27     25     30     57     12     35     55     38     13    526
   Score    492    316    544    442    644    725    402    429    419    637    279    499    662    498    353   7341
Score(%)   49.2   31.6   54.4   44.2   64.4   72.5   40.2   42.9   41.9   63.7   27.9   49.9   66.2   49.8   35.3   48.9

:: STS ID and Titles ::
STS 01: Undermining
STS 02: Open Files and Diagonals
STS 03: Knight Outposts
STS 04: Square Vacancy
STS 05: Bishop vs Knight
STS 06: Re-Capturing
STS 07: Offer of Simplification
STS 08: Advancement of f/g/h Pawns
STS 09: Advancement of a/b/c Pawns
STS 10: Simplification
STS 11: Activity of the King
STS 12: Center Control
STS 13: Pawn Play in the Center
STS 14: Queens and Rooks to the 7th rank
STS 15: Avoid Pointless Exchange

:: Top 5 STS with high result ::
1. STS 06, 72.5%, "Re-Capturing"
2. STS 13, 66.2%, "Pawn Play in the Center"
3. STS 05, 64.4%, "Bishop vs Knight"
4. STS 10, 63.7%, "Simplification"
5. STS 03, 54.4%, "Knight Outposts"

:: Top 5 STS with low result ::
1. STS 11, 27.9%, "Activity of the King"
2. STS 02, 31.6%, "Open Files and Diagonals"
3. STS 15, 35.3%, "Avoid Pointless Exchange"
4. STS 07, 40.2%, "Offer of Simplification"
5. STS 09, 41.9%, "Advancement of a/b/c Pawns"

that latest patch work for me for exactly the first 100 positions , it then errors out

Code: Select all

Mac-Pro:sts_rating_v13.1 michaelbyrne$ python2.7 /Applications/sts_rating_v13.1/sts_rating_v13.1.py -f STS1-STS15_LAN_v3.epd  -e Aristides-v1.0 --proto uci -h 128 --getrating
STS Rating v13.1

Number of cores: 24

Engine: Aristides-v1.0
Hash: 128, Threads: 1, MoveTime: 1.0s
Number of positions in STS1-STS15_LAN_v3.epd: 1500

Your bench : 2.678740s
My bench   : 2.553400s
Analysis Time to get CCRL 40/4 rating estimate : 210ms
Starting engine Aristides-v1.0 ...
id name: Aristides v1.0 64 POPCNT


Traceback (most recent call last):
  File "/Applications/sts_rating_v13.1/sts_rating_v13.1.py", line 966, in <module>
    main(sys.argv[1:])
  File "/Applications/sts_rating_v13.1/sts_rating_v13.1.py", line 960, in main
    stc, nmps, nSt, bSan, contempt)
  File "/Applications/sts_rating_v13.1/sts_rating_v13.1.py", line 527, in analyze_pos
    p.stdin.write("isready\n")
ValueError: I/O operation on closed file
Mac-Pro:sts_rating_v13.1 michaelbyrne$

flok · Post by **flok** » Sun Dec 25, 2016 10:51 pm

Is that with my patch? Looks like a different problem. As if the engine goes away (too soon?)

Ferdy · Post by **Ferdy** » Sun Dec 25, 2016 11:37 pm

flok wrote:Thanks!

Note that the program mail fail at the end. This can be resolved with the following patch:

Code: Select all

--- sts_rating_v13.1.py.org	2016-12-25 20:33:16.742365027 +0100
+++ sts_rating_v13.1.py	2016-12-25 20:34:18.338337318 +0100
@@ -685,9 +685,12 @@
                             
 
     # Quit the engine
-    p.stdin.write("quit\n")
-    p.stdin.close()
-    p.communicate()
+    try:
+        p.stdin.write("quit\n")
+        p.stdin.close()
+        p.communicate()
+    except:
+        pass
 
     timeEnd = time.clock()

Embla (rev 3611):

Code: Select all

STS Rating v13.1
Number of cores: 4

Engine: Emblatrunk-NM3
Hash: 32, Threads: 1, time/pos: 1.000s

Number of positions in STS1-STS15_LAN_v3.epd: 1500
Max score = 1500 x 10 = 15000
Test duration: 00h:00m:01s
Expected time to finish: 00h:25m:45s

  STS ID   STS1   STS2   STS3   STS4   STS5   STS6   STS7   STS8   STS9  STS10  STS11  STS12  STS13  STS14  STS15    ALL
  NumPos    100    100    100    100    100    100    100    100    100    100    100    100    100    100    100   1500
 BestCnt     38     22     38     33     54     49     27     25     30     57     12     35     55     38     13    526
   Score    492    316    544    442    644    725    402    429    419    637    279    499    662    498    353   7341
Score(%)   49.2   31.6   54.4   44.2   64.4   72.5   40.2   42.9   41.9   63.7   27.9   49.9   66.2   49.8   35.3   48.9

:: STS ID and Titles ::
STS 01: Undermining
STS 02: Open Files and Diagonals
STS 03: Knight Outposts
STS 04: Square Vacancy
STS 05: Bishop vs Knight
STS 06: Re-Capturing
STS 07: Offer of Simplification
STS 08: Advancement of f/g/h Pawns
STS 09: Advancement of a/b/c Pawns
STS 10: Simplification
STS 11: Activity of the King
STS 12: Center Control
STS 13: Pawn Play in the Center
STS 14: Queens and Rooks to the 7th rank
STS 15: Avoid Pointless Exchange

:: Top 5 STS with high result ::
1. STS 06, 72.5%, "Re-Capturing"
2. STS 13, 66.2%, "Pawn Play in the Center"
3. STS 05, 64.4%, "Bishop vs Knight"
4. STS 10, 63.7%, "Simplification"
5. STS 03, 54.4%, "Knight Outposts"

:: Top 5 STS with low result ::
1. STS 11, 27.9%, "Activity of the King"
2. STS 02, 31.6%, "Open Files and Diagonals"
3. STS 15, 35.3%, "Avoid Pointless Exchange"
4. STS 07, 40.2%, "Offer of Simplification"
5. STS 09, 41.9%, "Advancement of a/b/c Pawns"

Thanks for the info. I forgot to change it that it does not work for non-windows, in my other script, I am using like below.

Code: Select all

p.communicate('quit\n')
p.poll()
if p.returncode is None:
    print('Warning!! the process has not terminated yet')

I think even the following is enough which also works in windows.

Code: Select all

p.stdin.write("quit\n")
p.communicate()

flok · Post by **flok** » Wed Dec 28, 2016 9:56 pm

Hi,

The program says:

Code: Select all

4. STS 15, 36.7%, "Avoid Pointless Exchange"

Which pointless exchange is this?
Because I tried if adding SEE helped but apart from 150 elo reduction(!) it also did not change this result much:

Code: Select all

2. STS 15, 30.9%, "Avoid Pointless Exchange"

Ferdy · Post by **Ferdy** » Thu Dec 29, 2016 5:13 am

flok wrote:Hi,

The program says:
Code: Select all
4. STS 15, 36.7%, "Avoid Pointless Exchange"
Which pointless exchange is this?

STS 15 can be found in,

Code: Select all

STS1-STS15_LAN_v3.epd

There are 100 positions in this specific suite.

flok · Post by **flok** » Fri Feb 17, 2017 4:11 pm

Can you make the "press any key" configurable?
I'd like to run it in a batch every night.

Ferdy · Post by **Ferdy** » Sat Feb 18, 2017 1:29 am

flok wrote:Can you make the "press any key" configurable?
I'd like to run it in a batch every night.

Near the end of the script just remove the following.

Code: Select all

raw_input("Press enter key to exit")

Buttermilk · Post by **Buttermilk** » Fri Oct 25, 2019 4:28 pm

Serait-il possible de convertir le programme "STS Rating v13.1" fait avec Python 2.7.x et le rendre compatible avec Python 3.7.x ou Python 3.8.0. J'ai essayé le convertisseur "2to3" mais cela ne fonctionne pas. N'étant pas programmeur Python ...
Un grand merci d'avance pour celle ou celui qui répondra à mon attente.

Traduction "Google"
Would it be possible to convert the program "STS Rating v13.1" made with Python 2.7.x and make it compatible with Python 3.7.x or Python 3.8.0. I tried the converter "2to3" but it does not work. Not being a Python programmer ...
A big thank you in advance for the one who will meet my expectations.

Ferdy wrote: ↑Sun Dec 25, 2016 3:55 am
flok wrote:Any chance on a source-release so that it can be run on a non-exe system?
Try this v13.1, python src.
Code: Select all
STS Rating

Tested on Python 2.7.6, 2.7.11

v13.1
1. Remove dependency of cpu_info, cpu brand is no longer displayed.

v13
1. Also display app version in summary file
2. Modify reporting of number of cores, now no more physical
and no more logical cores, just number of cores
3. Added contempt for uci engines that supports such option
https://drive.google.com/file/d/0BwAOsu ... sp=sharing

Merry Christmas

Ferdy · Post by **Ferdy** » Fri Oct 25, 2019 7:09 pm

Buttermilk wrote: ↑Fri Oct 25, 2019 4:28 pm Serait-il possible de convertir le programme "STS Rating v13.1" fait avec Python 2.7.x et le rendre compatible avec Python 3.7.x ou Python 3.8.0. J'ai essayé le convertisseur "2to3" mais cela ne fonctionne pas. N'étant pas programmeur Python ...
Un grand merci d'avance pour celle ou celui qui répondra à mon attente.

Traduction "Google"
Would it be possible to convert the program "STS Rating v13.1" made with Python 2.7.x and make it compatible with Python 3.7.x or Python 3.8.0. I tried the converter "2to3" but it does not work. Not being a Python programmer ...
A big thank you in advance for the one who will meet my expectations.

Added in github, https://github.com/fsmosca/STS-Rating
Will revise it later.

STS test suite and engine analysis interface

Re: STS test suite and engine analysis interface

Re: STS test suite and engine analysis interface

Re: STS test suite and engine analysis interface

Re: STS test suite and engine analysis interface

Re: STS test suite and engine analysis interface

Re: STS test suite and engine analysis interface

Re: STS test suite and engine analysis interface

Re: STS test suite and engine analysis interface

Re: STS test suite and engine analysis interface

Re: STS test suite and engine analysis interface