Steve Maughan wrote:Slightly OT for this thread but how do you automate the STS test suite?
Thanks - Steve
I started working on STS from this post.
http://talkchess.com/forum/viewtopic.ph ... hlight=sts
I make couple of epd revisions on the move format, convert bm SAN to LAN, create c8 Opcodes for move score, create c9 Opcode for moves in LAN. Also fixed overall formatting on id values (id "STS(v10.0) ...), since this will be read by the tool for summary. The conversion of SAN move to LAN is thru the use of Python Chess Library (PCL) I use version 0.7.0. Each STS are then combined, now included in the uploaded file into one file. Created a tool to read this epd. The tool can only be used by uci engines. It is a python script converted to exe file using py2exe program. But this tool does not use PCL. Py2Exe cannot convert the script to exe when PCL is used. I am using windows. The epd reformatting were done using another script.
Code: Select all
1b1qrr2/1p4pk/1np4p/p3Np1B/Pn1P4/R1N3B1/1Pb2PPP/2Q1R1K1 b - - bm Bxe5; id "STS(v10.0) Simplification.001"; c0 "Bxe5=10, f4=3, Nc4=2"; c8 "10 3 2"; c9 "b8e5 f5f4 b6c4";
Sample batch file is also included.
Code: Select all
:: Run to get sts rating using --getrating option
:: when --getrating is used, num threads will be set to 1, and movetime will also be
:: set by the tool depending on the speed of your machine. On my machine without other loads
:: the movetime per pos used by the tool is 200ms. The tool will run a short benchmark
:: to measure your machine speed to get the movetime, before starting the test suite.
STS_Rating_v2 -f "STS1-STS15_LAN.epd" -e "stockfish 6.exe" -h 128 --getrating --log
:: Normal run to get score, movetime is in millisec
:: STS_Rating_v2 -f "STS1-STS15_LAN.epd" -e "CDrill 1905 v1.exe" -t 1 -h 128 --movetime 500
When --log is enabled, positions not solved are saved, also engine log is created.
Sample run: The tool uses 286 ms because my machine is running with other things. If no other load, the tool will use 200ms/pos.
Code: Select all
Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
id name Monarch 1.7
Hash: 128, Threads: 1, time/pos: 0.286s
Test duration: 00:10:23
Expected time to finish: 00:07:54
STS rating: 2041
STS ID STS1 STS2 STS3 STS4 STS5 STS6 STS7 STS8 STS9 STS10 STS11 STS12 STS13 STS14 STS15 ALL
NumPos 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 1500
BestCnt 32 42 40 48 46 52 32 33 29 44 25 43 47 47 21 581
Score 452 511 564 574 575 725 439 457 388 554 364 523 573 584 410 7693
Score(%) 45.2 51.1 56.4 57.4 57.5 72.5 43.9 45.7 38.8 55.4 36.4 52.3 57.3 58.4 41.0 51.3
Rating 1770 2032 2268 2313 2317 2985 1712 1792 1485 2224 1378 2086 2308 2357 1583 2041
Download:
http://www.mediafire.com/download/so49n ... ing_v2.rar
