STS with pv?

[chesskobra]

Where can I obtain the Strategic Test Suite (STS) analysed with a sufficiently strong engine (e.g., SF 15 ou 16) on modern hardware, preferably together with the pv (or even multi-pv) and eval? Or suggestions on how to run, what time control to use, etc. I have access to Processor - 13th generation Intel Core i7-13700 (16-core, cache 24MB, 2.1 GHz up to 5.1GHz)

[Jouni]

Better sent to trash and forget.

[chesskobra]

Could you elaborate?

Anyway, I am not an engine developer or tester (although I may use it to test some engine also), but I am curious about it. It seems to be included in LucasChess, hence I am even more curious.

If someone has any other collection of strategic positions (preferably fully analysed), or an STS equivalent for endgames that would also be appreciated.

[Ferdy]

Where can I obtain the Strategic Test Suite (STS) analysed with a sufficiently strong engine (e.g., SF 15 ou 16) on modern hardware, preferably together with the pv (or even multi-pv) and eval? Or suggestions on how to run, what time control to use, etc. I have access to Processor - 13th generation Intel Core i7-13700 (16-core, cache 24MB, 2.1 GHz up to 5.1GHz)

All positions are reanalyzed with sf15 at longer time at multipv 10.

I have been checking each positions if it is good or bad test position based on the analysis.

I created a web app to view its analysis, etc.



That app's data are backed by google sheet.



All the analysis are in csv format in my google drive.



I will try to find my python script to do the analysis, in case someone wants to extend the analysis to a deeper depths.

[Ferdy]

This is the latest sts pos in epd format from my google drive. This is based from the re-analyzed positions that are checked so far.

The scoring system points are proportional to the top 10 evaluations.

[chesskobra]

Thank you Ferdy. Yesterday I ran your script sts-rating.py 3 times on STS1-STS15_LAN_v3.epd from your github. I was a bit surprised by this:

(13 gen i7-13700, 32 GB RAM) Hash: 256, Threads: 8, time/pos: 2.000s - total score 88.6 %
(13 gen i7-13700, 32 GB RAM) Hash: 256, Threads: 16, time/pos: 8.000s - total score 88.9 %
(Old laptop, Dell Latitude, core i5, 4 GB RAM) Hash: 128, Threads: 1, time/pos: 2.000s - total score 88.3 %

So not must difference in score between a powerful machine with more time per move, more threads, and a low end laptop with small value of movetime. 1 thread. STS 9, 7, 11, 12 are consistently low scoring in all 3 tests.

Is there a way I can run analysis on the epd file and create a pgn with pv or include pv in epd as a comment? What tool on linux does this?

[Dann Corbit]

The pv is part of the EPD standard.
https://www.chessprogramming.org/Extend ... escription

[chesskobra]

I asked because I thought many epd came with only best move, but no pv. But now I look at STS1-STS15_LAN_v3.epd in ferdy's github, and it has the format:

Code: Select all

1kr5/3n4/q3p2p/p2n2p1/PppB1P2/5BP1/1P2Q2P/3R2K1 w - - bm f5; id "STS(v1.0) Undermine.001"; c0 "f5=10, Be5+=2, Bf2=3, Bg4=2"; c7 "f5 Be5+ Bf2 Bg4"; c8 "10 2 3 2"; c9 "f4f5 d4e5 d4f2 f3g4";

Here c7 seems to be the pv.

[Ferdy]

I asked because I thought many epd came with only best move, but no pv. But now I look at STS1-STS15_LAN_v3.epd in ferdy's github, and it has the format:

Code: Select all

1kr5/3n4/q3p2p/p2n2p1/PppB1P2/5BP1/1P2Q2P/3R2K1 w - - bm f5; id "STS(v1.0) Undermine.001"; c0 "f5=10, Be5+=2, Bf2=3, Bg4=2"; c7 "f5 Be5+ Bf2 Bg4"; c8 "10 2 3 2"; c9 "f4f5 d4e5 d4f2 f3g4";

Here c7 seems to be the pv.

It is a list of moves. The engine can gain points if its best move is one of those.

[Ferdy]

Is there a way I can run analysis on the epd file and create a pgn with pv ...

Here is a python script to do that.

- install python
- install python chess with "pip install chess"

main.py

Code: Select all

import chess
import chess.engine
import chess.pgn


def get_epd(fn):
    """Saves epd from text file to a list."""
    epds = []
    with open(fn) as f:
        for line in f:
            epd_line = line.rstrip()
            epds.append(epd_line)

    return epds


def main():
    mtsec = 0.1  # movetime in second

    epdfn = 'sts_v7_issued.epd'
    epd_lines = get_epd(epdfn)

    engfn = r"F:\Chess\Engines\stockfish\sf16\sf16.1.exe"
    engine = chess.engine.SimpleEngine.popen_uci(engfn)
    eng_name = engine.id['name']
    engine.configure({'Hash': 128})
    engine.configure({'Threads': 2})

    for epd_line in epd_lines:
        board = chess.Board()
        binfo = board.set_epd(epd_line)
        pos_id = binfo['id']
        
        sinfo = engine.analyse(board, chess.engine.Limit(time=mtsec))
        pv = sinfo['pv']

        game = chess.pgn.Game()
        game = game.from_board(board)
        game.headers["Event"] = pos_id
        game.headers["Annotator"] = f'{eng_name}, mtsec={mtsec}'
        game.add_line(pv)
        print(f'{game}\n\n')

    engine.quit()


if __name__ == '__main__':
    main()

command line

Code: Select all

python main.py >out.pgn

Modify mtsec, epdfn, enginefn, hash and threads to suit your needs.

Sample output

Code: Select all

[Event "STS(v1.0) Undermine.001"]
[Site "?"]
[Date "????.??.??"]
[Round "?"]
[White "?"]
[Black "?"]
[Result "*"]
[Annotator "Stockfish 16.1, mtsec=0.1"]
[FEN "1kr5/3n4/q3p2p/p2n2p1/PppB1P2/5BP1/1P2Q2P/3R2K1 w - - 0 1"]
[SetUp "1"]

1. f5 Nc7 2. Bg7 Qb6+ 3. Kh1 g4 4. Bg2 Nc5 5. Qxc4 exf5 6. Be5 Ne4 7. Bxe4 fxe4 *


[Event "STS(v1.0) Undermine.002"]
[Site "?"]
[Date "????.??.??"]
[Round "?"]
[White "?"]
[Black "?"]
[Result "*"]
[Annotator "Stockfish 16.1, mtsec=0.1"]
[FEN "1n5k/3q3p/pp1p2pB/5r2/1PP1Qp2/P6P/6P1/2R3K1 w - - 0 1"]
[SetUp "1"]

1. Bxf4 Nc6 2. Rd1 Rf6 3. Bh6 Rf5 4. Qe3 Kg8 5. Qxb6 g5 6. b5 *