Python Chess and TSCP

[chrisw]

Getting engines going in Python Chess is pretty easy, for example:

with the relevant engine in the right directory:
engine = chess.engine.SimpleEngine.popen_uci(rootdir + 'UCI-Chess-Engines/stockfish-10-win/Windows/' + 'stockfish_10_x64')

engine = chess.engine.SimpleEngine.popen_uci(rootdir + 'UCI-Chess-Engines/LCZero/lc0-v0.21.0-rc1-windows-cuda/' + 'lc0')

then, with board set .......

info = engine.analyse(board, chess.engine.Limit(time=0.10))


Does anyone have a suitable start and analyse command for TCSP, Tom's Simple Chess Engine, code example? Including what and how to send for initialisation.

Thanks in advance

[Daniel Shawul]

I think the only difference is you use popen_xboard for xboard engines instead of popen_uci -- most other
commands should be shared with a wrapper.

Btw TSCP does not support 'setboard' or 'analyze' commands so you will have to use another xboard engine.

[chrisw]

I think the only difference is you use popen_xboard for xboard engines instead of popen_uci -- most other
commands should be shared with a wrapper.

Btw TSCP does not support 'setboard' or 'analyze' commands so you will have to use another xboard engine.

That’s a good point. I just wanted some to set to one or two ply to test the bare network, no lookahead. Any suggestions for an easy download and quick setup?

[Daniel Shawul]

I think the only difference is you use popen_xboard for xboard engines instead of popen_uci -- most other
commands should be shared with a wrapper.

Btw TSCP does not support 'setboard' or 'analyze' commands so you will have to use another xboard engine.

That’s a good point. I just wanted some to set to one or two ply to test the bare network, no lookahead. Any suggestions for an easy download and quick setup?

Maybe you can use stockfish to search depth of 1? I have not tried it but substituting the time limit shown below with chess.engine.Limit(depth=1)
should work. I think some have tried to compare the policy network of lc0 with stockfish qsearch or depth=1. I think stockfish will be tactically
stronger event at depth=1 but lc0's policy network (atleast the current generation) should overwhelm stockfish_depth_1 positionally so as to win most games against it

Code: Select all

import chess
import chess.engine

engine = chess.engine.SimpleEngine.popen_uci("stockfish")

board = chess.Board()
info = engine.analyse(board, chess.engine.Limit(time=0.100))
print("Score:", info["score"])
# Score: +20

board = chess.Board("r1bqkbnr/p1pp1ppp/1pn5/4p3/2B1P3/5Q2/PPPP1PPP/RNB1K1NR w KQkq - 2 4")
info = engine.analyse(board, chess.engine.Limit(depth=20))
print("Score:", info["score"])
# Score: #1

engine.quit()

[chrisw]

I think the only difference is you use popen_xboard for xboard engines instead of popen_uci -- most other
commands should be shared with a wrapper.

Btw TSCP does not support 'setboard' or 'analyze' commands so you will have to use another xboard engine.

That’s a good point. I just wanted some to set to one or two ply to test the bare network, no lookahead. Any suggestions for an easy download and quick setup?

Maybe you can use stockfish to search depth of 1? I have not tried it but substituting the time limit shown below with chess.engine.Limit(depth=1)
should work. I think some have tried to compare the policy network of lc0 with stockfish qsearch or depth=1. I think stockfish will be tactically
stronger event at depth=1 but lc0's policy network (atleast the current generation) should overwhelm stockfish_depth_1 positionally so as to win most games against it

Code: Select all

import chess
import chess.engine

engine = chess.engine.SimpleEngine.popen_uci("stockfish")

board = chess.Board()
info = engine.analyse(board, chess.engine.Limit(time=0.100))
print("Score:", info["score"])
# Score: +20

board = chess.Board("r1bqkbnr/p1pp1ppp/1pn5/4p3/2B1P3/5Q2/PPPP1PPP/RNB1K1NR w KQkq - 2 4")
info = engine.analyse(board, chess.engine.Limit(depth=20))
print("Score:", info["score"])
# Score: #1

engine.quit()

ok, thanks, I’ll give it a whirl. Sf10 and Lc0 work fine using Python Chess, I still need to work out how to get a policy move and a value move at ply 0 out of LC0, preferably without having to hack and recompile it.

[Ferdy]

ok, thanks, I’ll give it a whirl. Sf10 and Lc0 work fine using Python Chess, I still need to work out how to get a policy move and a value move at ply 0 out of LC0, preferably without having to hack and recompile it.

To see it in Lc0 console, run lc0
uci
setoption name VerboseMoveStats value true
isready
ucinewgame
go nodes 0

To see it in a file, enable its log
setoption name LogFile value log.txt

Code: Select all

[...]
 b1c3  (36  ) N:       0 (+ 0) (P:  3.49%) (Q:  0.10609) (D:  0.000) (U: 0.10474) (Q+U:  0.21083) (V:  -.----) 
 a2a3  (204 ) N:       0 (+ 0) (P:  3.57%) (Q:  0.10609) (D:  0.000) (U: 0.10712) (Q+U:  0.21321) (V:  -.----) 
 c2c3  (259 ) N:       0 (+ 0) (P:  3.87%) (Q:  0.10609) (D:  0.000) (U: 0.11600) (Q+U:  0.22209) (V:  -.----) 
 e2e3  (317 ) N:       0 (+ 0) (P:  4.21%) (Q:  0.10609) (D:  0.000) (U: 0.12630) (Q+U:  0.23239) (V:  -.----) 
 g2g3  (374 ) N:       0 (+ 0) (P:  4.44%) (Q:  0.10609) (D:  0.000) (U: 0.13308) (Q+U:  0.23917) (V:  -.----) 
 c2c4  (264 ) N:       0 (+ 0) (P:  4.46%) (Q:  0.10609) (D:  0.000) (U: 0.13376) (Q+U:  0.23985) (V:  -.----) 
 g1f3  (159 ) N:       0 (+ 0) (P:  7.78%) (Q:  0.10609) (D:  0.000) (U: 0.23328) (Q+U:  0.33938) (V:  -.----) 
 d2d4  (293 ) N:       0 (+ 0) (P:  8.98%) (Q:  0.10609) (D:  0.000) (U: 0.26945) (Q+U:  0.37554) (V:  -.----) 
 e2e4  (322 ) N:       0 (+ 0) (P: 30.69%) (Q:  0.10609) (D:  0.000) (U: 0.92069) (Q+U:  1.02678) (V:  -.----) 

Perhaps you know all of it. Do you want to get those info via python-chess?

[chrisw]

ok, thanks, I’ll give it a whirl. Sf10 and Lc0 work fine using Python Chess, I still need to work out how to get a policy move and a value move at ply 0 out of LC0, preferably without having to hack and recompile it.

To see it in Lc0 console, run lc0
uci
setoption name VerboseMoveStats value true
isready
ucinewgame
go nodes 0

To see it in a file, enable its log
setoption name LogFile value log.txt

Code: Select all

[...]
 b1c3  (36  ) N:       0 (+ 0) (P:  3.49%) (Q:  0.10609) (D:  0.000) (U: 0.10474) (Q+U:  0.21083) (V:  -.----) 
 a2a3  (204 ) N:       0 (+ 0) (P:  3.57%) (Q:  0.10609) (D:  0.000) (U: 0.10712) (Q+U:  0.21321) (V:  -.----) 
 c2c3  (259 ) N:       0 (+ 0) (P:  3.87%) (Q:  0.10609) (D:  0.000) (U: 0.11600) (Q+U:  0.22209) (V:  -.----) 
 e2e3  (317 ) N:       0 (+ 0) (P:  4.21%) (Q:  0.10609) (D:  0.000) (U: 0.12630) (Q+U:  0.23239) (V:  -.----) 
 g2g3  (374 ) N:       0 (+ 0) (P:  4.44%) (Q:  0.10609) (D:  0.000) (U: 0.13308) (Q+U:  0.23917) (V:  -.----) 
 c2c4  (264 ) N:       0 (+ 0) (P:  4.46%) (Q:  0.10609) (D:  0.000) (U: 0.13376) (Q+U:  0.23985) (V:  -.----) 
 g1f3  (159 ) N:       0 (+ 0) (P:  7.78%) (Q:  0.10609) (D:  0.000) (U: 0.23328) (Q+U:  0.33938) (V:  -.----) 
 d2d4  (293 ) N:       0 (+ 0) (P:  8.98%) (Q:  0.10609) (D:  0.000) (U: 0.26945) (Q+U:  0.37554) (V:  -.----) 
 e2e4  (322 ) N:       0 (+ 0) (P: 30.69%) (Q:  0.10609) (D:  0.000) (U: 0.92069) (Q+U:  1.02678) (V:  -.----) 

Perhaps you know all of it. Do you want to get those info via python-chess?

thanks for those hints. First off I want benchmarks to test against, just raw network versus raw network, so, yes, it will using Python Chess library to do that. Likewise raw network versus 1-ply AB evaluations. Many things on TODO list

[Ferdy]

thanks for those hints. First off I want benchmarks to test against, just raw network versus raw network, so, yes, it will using Python Chess library to do that. Likewise raw network versus 1-ply AB evaluations. Many things on TODO list

Just in case you need it, sample code evaluating wac201.epd using lc0 without search (using a limit of nodes=0). Also show the top move policy value.

Code: Select all

# -*- coding: utf-8 -*-
"""
lco.py

Requirements:
    python 3.6 and up
    python-chess v0.26.0

"""


import chess
import chess.engine


def analyze_epd(eng_path, net_id, epdfn, movesec_lim, node_lim):
    """ Evaluate positions in epdfn using the engine in eng_path """
    engine = chess.engine.SimpleEngine.popen_uci(eng_path)
    eng_name = engine.id['name']
    
    # Set options
    engine.configure({"Threads": 1})
    engine.configure({"VerboseMoveStats": 'true'})
    limit = chess.engine.Limit(time=movesec_lim, nodes=node_lim)
    
    score = 0
    total_pos = 0
    
    with open(epdfn, 'r') as h:
        for line in h:
            total_pos += 1
            
            epd = line.strip()
            print(epd)                     
            pos, epd_info = chess.Board().from_epd(epd)            
                
            res = engine.play(pos, limit, info=32)       
            bm_uci = res.move
            bm_san = pos.san(bm_uci)
            print(bm_san)
            
            # Parse lc0 move stats that was enabled in VerboseMoveStats
            # a1a7  (19  ) N:       0 (+ 0) (P: 13.26%) (Q: -0.04170) (D:  0.000) (U: 0.39790) (Q+U:  0.35620) (V:  -.----)
            move_policy = res.info['string']
            p = move_policy.split(':')[2].strip().split(')')[0].strip()
            print('move policy: {}'.format(p))
            
            # Can only process epd with bm
            if "bm" in epd_info and bm_uci in epd_info["bm"]:
                print('success')
                score += 1
            else:
                print('failure')
                
            print()
            
    engine.quit()
    
    print('epdfile     : {}'.format(epdfn))
    print('engine name : {}'.format(eng_name))
    print('net id      : {}'.format(net_id))
    print('score       : {}/{} ({:0.2f}%)'.format(score, total_pos, 100.0*score/total_pos))
        

def main():
    epdfn = 'wac201.epd'  # Place it in same dir with this script
    
    # Place net_id in the same dir with lc0.exe
    eng_path = 'C:/engines/nn/lc0-v0.21.0-rc1-windows-blas/lc0.exe'
    net_id = '36089'
    movesec_lim = 1.0
    node_lim = 0  # Set to 0 to use the move policy only without search
    analyze_epd(eng_path, net_id, epdfn, movesec_lim, node_lim)


if __name__ == "__main__":
    main()

Result is something like this.

Code: Select all

[...]
3Q4/p3b1k1/2p2rPp/2q5/4B3/P2P4/7P/6RK w - - bm Qh8+; id "WAC.298";
Qd7
move policy: 20.24%
failure

b2b1r1k/3R1ppp/4qP2/4p1PQ/4P3/5B2/4N1K1/8 w - - bm g6; id "WAC.300";
fxg7+
move policy: 16.88%
failure

epdfile     : wac201.epd
engine name : Lc0 v0.21.0-rc1
net id      : 36089
score       : 29/201 (14.43%)

[chrisw]

thanks for those hints. First off I want benchmarks to test against, just raw network versus raw network, so, yes, it will using Python Chess library to do that. Likewise raw network versus 1-ply AB evaluations. Many things on TODO list

Just in case you need it, sample code evaluating wac201.epd using lc0 without search (using a limit of nodes=0). Also show the top move policy value.

Code: Select all

# -*- coding: utf-8 -*-
"""
lco.py

Requirements:
    python 3.6 and up
    python-chess v0.26.0

"""


import chess
import chess.engine


def analyze_epd(eng_path, net_id, epdfn, movesec_lim, node_lim):
    """ Evaluate positions in epdfn using the engine in eng_path """
    engine = chess.engine.SimpleEngine.popen_uci(eng_path)
    eng_name = engine.id['name']
    
    # Set options
    engine.configure({"Threads": 1})
    engine.configure({"VerboseMoveStats": 'true'})
    limit = chess.engine.Limit(time=movesec_lim, nodes=node_lim)
    
    score = 0
    total_pos = 0
    
    with open(epdfn, 'r') as h:
        for line in h:
            total_pos += 1
            
            epd = line.strip()
            print(epd)                     
            pos, epd_info = chess.Board().from_epd(epd)            
                
            res = engine.play(pos, limit, info=32)       
            bm_uci = res.move
            bm_san = pos.san(bm_uci)
            print(bm_san)
            
            # Parse lc0 move stats that was enabled in VerboseMoveStats
            # a1a7  (19  ) N:       0 (+ 0) (P: 13.26%) (Q: -0.04170) (D:  0.000) (U: 0.39790) (Q+U:  0.35620) (V:  -.----)
            move_policy = res.info['string']
            p = move_policy.split(':')[2].strip().split(')')[0].strip()
            print('move policy: {}'.format(p))
            
            # Can only process epd with bm
            if "bm" in epd_info and bm_uci in epd_info["bm"]:
                print('success')
                score += 1
            else:
                print('failure')
                
            print()
            
    engine.quit()
    
    print('epdfile     : {}'.format(epdfn))
    print('engine name : {}'.format(eng_name))
    print('net id      : {}'.format(net_id))
    print('score       : {}/{} ({:0.2f}%)'.format(score, total_pos, 100.0*score/total_pos))
        

def main():
    epdfn = 'wac201.epd'  # Place it in same dir with this script
    
    # Place net_id in the same dir with lc0.exe
    eng_path = 'C:/engines/nn/lc0-v0.21.0-rc1-windows-blas/lc0.exe'
    net_id = '36089'
    movesec_lim = 1.0
    node_lim = 0  # Set to 0 to use the move policy only without search
    analyze_epd(eng_path, net_id, epdfn, movesec_lim, node_lim)


if __name__ == "__main__":
    main()

Result is something like this.

Code: Select all

[...]
3Q4/p3b1k1/2p2rPp/2q5/4B3/P2P4/7P/6RK w - - bm Qh8+; id "WAC.298";
Qd7
move policy: 20.24%
failure

b2b1r1k/3R1ppp/4qP2/4p1PQ/4P3/5B2/4N1K1/8 w - - bm g6; id "WAC.300";
fxg7+
move policy: 16.88%
failure

epdfile     : wac201.epd
engine name : Lc0 v0.21.0-rc1
net id      : 36089
score       : 29/201 (14.43%)

Python Chess is cool. Have you stats for the value, zero search? I’ve also been running EPD test suites, it’s worth calculating what a completely random engine would score on the same suite, use as baseline. I don’t have my code to hand but it was along the lines of

if (random.random() < (1.0 / legal_move_count)) success += 1
like if the legal count is 2, then random will solve 50% of the time and so on.

Of course this varies by each EPD suite, but I was getting random solver rates over N epds, of 8-10% or so.

[Ferdy]

Python Chess is cool. Have you stats for the value, zero search?

At nodes = 0, the V is empty.

Code: Select all

rnbqkb1r/pppp1ppp/8/4P3/6n1/7P/PPPNPPP1/R1BQKBNR b KQkq - bm Ne3; id "WAC.007";
Nxe5
g4e5  (1094) N:       0 (+ 0) (P: 58.50%) (Q: -0.10036) (D:  0.000) (U: 1.75494) (Q+U:  1.65459) (V:  -.----) 
move policy  : 58.50%
value policy : -.----
failure

r4q1k/p2bR1rp/2p2Q1N/5p2/5p2/2P5/PP3PPP/R5K1 w - - bm Rf7; id "WAC.008";
Qxf8+
f6f8  (1323) N:       0 (+ 0) (P: 18.76%) (Q:  0.95506) (D:  0.000) (U: 0.56270) (Q+U:  1.51776) (V:  -.----) 
move policy  : 18.76%
value policy : -.----
failure

But at nodes = 100.

Code: Select all

rnbqkb1r/pppp1ppp/8/4P3/6n1/7P/PPPNPPP1/R1BQKBNR b KQkq - bm Ne3; id "WAC.007";
Nxe5
g4e5  (1094) N:       1 (+34) (P: 58.50%) (Q: -0.21314) (D:  0.000) (U: 0.04875) (Q+U: -0.16439) (V: -0.2131) 
move policy  : 58.50%
value policy : -0.2131
failure

r4q1k/p2bR1rp/2p2Q1N/5p2/5p2/2P5/PP3PPP/R5K1 w - - bm Rf7; id "WAC.008";
Rf7
e7f7  (1510) N:      11 (+ 5) (P: 11.33%) (Q:  0.99514) (D:  0.000) (U: 0.09167) (Q+U:  1.08681) (V:  0.9829) 
move policy  : 11.33%
value policy : 0.9829
success

Revised code, extracting the V.

Code: Select all

# -*- coding: utf-8 -*-
"""
lco.py

Requirements:
    python 3.6 and up
    python-chess v0.26.0

"""


import chess
import chess.engine


def analyze_epd(eng_path, net_id, epdfn, movesec_lim, node_lim):
    """ Evaluate positions in epdfn using the engine in eng_path """
    engine = chess.engine.SimpleEngine.popen_uci(eng_path)
    eng_name = engine.id['name']
    
    # Set options
    engine.configure({"Threads": 1})
    engine.configure({"VerboseMoveStats": 'true'})
    limit = chess.engine.Limit(time=movesec_lim, nodes=node_lim)
    
    score = 0
    total_pos = 0
    
    with open(epdfn, 'r') as h:
        for line in h:
            total_pos += 1
            
            epd = line.strip()
            print(epd)                     
            pos, epd_info = chess.Board().from_epd(epd)            
                
            res = engine.play(pos, limit, info=32)       
            bm_uci = res.move
            bm_san = pos.san(bm_uci)
            print(bm_san)
            
            # Parse lc0 move stats that was enabled in VerboseMoveStats
            # a1a7  (19  ) N:       0 (+ 0) (P: 13.26%) (Q: -0.04170) (D:  0.000) (U: 0.39790) (Q+U:  0.35620) (V:  -.----)
            policy = res.info['string']
            print(policy)
            p = policy.split(':')[2].strip().split(')')[0].strip()
            print('move policy  : {}'.format(p))
            v = policy.split(':')[7].strip().split(')')[0].strip()
            print('value policy : {}'.format(v))
            
            if "bm" in epd_info and bm_uci in epd_info["bm"]:
                print('success')
                score += 1
            else:
                print('failure')
                
            print()
            
    engine.quit()
    
    print('epdfile     : {}'.format(epdfn))
    print('engine name : {}'.format(eng_name))
    print('net id      : {}'.format(net_id))
    print('score       : {}/{} ({:0.2f}%)'.format(score, total_pos, 100.0*score/total_pos))
        

def main():
    epdfn = 'wac201.epd'
    
    # Place net_id in the same dir with lc0.exe
    eng_path = 'C:/engines/nn/lc0-v0.21.0-rc1-windows-blas/lc0.exe'
    net_id = '36089'
    movesec_lim = 1.0
    node_lim = 0  # Set to 0 to use the move policy only without search
    analyze_epd(eng_path, net_id, epdfn, movesec_lim, node_lim)


if __name__ == "__main__":
    main()