Database snapshot

[OneTrickPony]

Very interesting, thank you very much.
One question: When I find a leaf in the tree (position after last given move not found) does it mean that this was evaluated with Stockfish depth 22 and no other analysis starting from that position was performed?

[OneTrickPony]

It unfortunately wants to play a losing line in Vienna with black (and Vienna is its first choice). It's the same problem Leela has searching from the position after 4th move:

[pgn] 1.d4 Nf6 2.c4 e6 3.Nf3 d5 4.Nc3 dxc4 5.e4 Bb4 6.Bxc4 Nxe4 7.O-O Bxc3 8.bxc3 Nxc3 9.Qb3 Nd5 10.Ba3 f6 11.Rae1 Kf7 *[/pgn]

First choices of the book so far for black (with the transposition of 1...Nf6/e6/d5) and this position it only gives:
Bxd5 0 50.00

Unfortunately for black 12.Ng5+ is a forced win for white (which btw my SF sees at depth 22 already)

As an anecdote: I played a Lichess blitz game vs a cheater with white in this line and they have fallen into it (probably were using SF on shallow depths as beginners don't know how to change the default settings/use more threads etc.). No human goes into that line as it just looks very risky. They were banned by Lichess a while later.

[noobpwnftw]

Very interesting, thank you very much.
One question: When I find a leaf in the tree (position after last given move not found) does it mean that this was evaluated with Stockfish depth 22 and no other analysis starting from that position was performed?

Yes.

As an anecdote: I played a Lichess blitz game vs a cheater with white in this line and they have fallen into it (probably were using SF on shallow depths as beginners don't know how to change the default settings/use more threads etc.). No human goes into that line as it just looks very risky. They were banned by Lichess a while later.

Of course, there must be infinite number of holes, or chess would be solved.

I have a Lichess bot which uses this database as its opening book, so it could be tested on whether the self-improving method works in practice.

[Ovyron]

It unfortunately wants to play a losing line in Vienna with black (and Vienna is its first choice). It's the same problem Leela has searching from the position after 4th move:

[pgn]1.d4 Nf6 2.c4 e6 3.Nf3 d5 4.Nc3 dxc4 5.e4 Bb4 6.Bxc4 Nxe4 7.O-O Bxc3 8.bxc3 Nxc3 9.Qb3 Nd5 10.Ba3 f6 11.Rae1 Kf7 *[/pgn]

First choices of the book so far for black (with the transposition of 1...Nf6/e6/d5) and this position it only gives:
Bxd5 0 50.00

Unfortunately for black 12.Ng5+ is a forced win for white (which btw my SF sees at depth 22 already)

Thanks! This was exactly the thing I would have tried to find first by querying the database

My analysis deviates with 9...b5 with backsolved score of 0.36, but in general, this is a line black doesn't want to play (4...Be7!! is 0.00). I'm glad 1000 times less positions analyzed can hold.

[Ferdy]

Thanks, I downloaded your exe, ran it in some directory and pasted a fen. I got this error:

Traceback (most recent call last):
File "book_probe2.py", line 55, in <module>
File "book_probe2.py", line 50, in main
File "book_probe2.py", line 26, in probe_book
IndexError: list index out of range
[172] Failed to execute script book_probe2

Use fen and not epd.
If you only have epd, just add "0 1" at the end to make it a fen.

[Ovyron]

Use fen and not epd.
If you only have epd, just add "0 1" at the end to make it a fen.

Thanks, now it works.

This is really cool, I kind of feel like I downloaded some huge database that weighs just 9MB

[Ferdy]

Use fen and not epd.
If you only have epd, just add "0 1" at the end to make it a fen.

Thanks, now it works.

This is really cool, I kind of feel like I downloaded some huge database that weighs just 9MB

Also working on simple html interface, I hope to release it next week.

[Ovyron]

Thanks, I downloaded your exe, ran it in some directory and pasted a fen. I got this error:

Traceback (most recent call last):
File "book_probe2.py", line 55, in <module>
File "book_probe2.py", line 50, in main
File "book_probe2.py", line 26, in probe_book
IndexError: list index out of range
[172] Failed to execute script book_probe2

Use fen and not epd.
If you only have epd, just add "0 1" at the end to make it a fen.

Hi Ferdy, I tried that for this position:

[d]r1bqkb1r/pppp1ppp/2n2n2/4p1N1/2B1P3/8/PPPP1PPP/RNBQK2R b KQkq -

But entering:

r1bqkb1r/pppp1ppp/2n2n2/4p1N1/2B1P3/8/PPPP1PPP/RNBQK2R b KQkq - 0 1

Produces errors and it exits.

[Ferdy]

Thanks, I downloaded your exe, ran it in some directory and pasted a fen. I got this error:

Traceback (most recent call last):
File "book_probe2.py", line 55, in <module>
File "book_probe2.py", line 50, in main
File "book_probe2.py", line 26, in probe_book
IndexError: list index out of range
[172] Failed to execute script book_probe2

Use fen and not epd.
If you only have epd, just add "0 1" at the end to make it a fen.

Hi Ferdy, I tried that for this position:

[d]r1bqkb1r/pppp1ppp/2n2n2/4p1N1/2B1P3/8/PPPP1PPP/RNBQK2R b KQkq -

But entering:

r1bqkb1r/pppp1ppp/2n2n2/4p1N1/2B1P3/8/PPPP1PPP/RNBQK2R b KQkq - 0 1

Produces errors and it exits.

You found a bug. The exe expects to have a winrate but there is a move from the database that does not give a win rate and this is:

Code: Select all

{'uci': 'c6e7', 'san': 'Ne7', 'score': -29998, 'rank': 0, 'note': '? (01-01)'}

See there is no 'winrate'. Its winrate should have been 0.0 assuming winrate has a range [0, 100], %.

I can modify the exe to handle such error and not exit the program, but lets wait for noobpwnftw to fix it by sending the winrate even if it is zero.


The html interface handles it by itself.

[Ferdy]

Thanks, I downloaded your exe, ran it in some directory and pasted a fen. I got this error:

Traceback (most recent call last):
File "book_probe2.py", line 55, in <module>
File "book_probe2.py", line 50, in main
File "book_probe2.py", line 26, in probe_book
IndexError: list index out of range
[172] Failed to execute script book_probe2

Use fen and not epd.
If you only have epd, just add "0 1" at the end to make it a fen.

Hi Ferdy, I tried that for this position:

[d]r1bqkb1r/pppp1ppp/2n2n2/4p1N1/2B1P3/8/PPPP1PPP/RNBQK2R b KQkq -

But entering:

r1bqkb1r/pppp1ppp/2n2n2/4p1N1/2B1P3/8/PPPP1PPP/RNBQK2R b KQkq - 0 1

Produces errors and it exits.

New version of exe
https://drive.google.com/file/d/1_KM89V ... sp=sharing

You can now just input epd.

python source code.

Code: Select all

#!/usr/bin/env python3
"""
probe_chessdb_book.py

Probe book moves in ChessDB

Requirements:
    * Python 3
    * Requests
        pip install requests
        
Dev log:
    v0.1
    * Use http
    * Add pv of bestmove results
    * Add Database stats results
    * Use fen directly after ...board=
    * epd can now be entered
    * Handle errors in querying all&board
"""

import requests
import json
import time


def probe_book(fen):
    data = []
    
    # Query all moves
    base = 'http://www.chessdb.cn/cdb.php?action=queryall&json=1&board='
    url = base + fen
    
    try:
        r = requests.get(url)
        r.raise_for_status()
    except requests.exceptions.HTTPError as errh:
        print("Http Error:",errh)
        return
    except requests.exceptions.ConnectionError as errc:
        print("Error Connecting:",errc)
        return
    except requests.exceptions.Timeout as errt:
        print("Timeout Error:",errt)
        return
    except requests.exceptions.RequestException as err:
        print("OOps: Something Else",err)
        return
        
    jdata = r.text
    
    d = json.loads(jdata)
    if d['status'] == 'ok':
        
        for n in d['moves']:
            try:
                data.append([n['san'], int(n['score']), float(n['winrate'])])
            except KeyError:
                pass
                
    else:
        print('Position is not found!')
        
    if len(data) > 0:
        print('(1) Move Stats:')
        print('{:6s}  {:<5s}  {}'.format('Move', 'Score', 'Winrate'))
        for d in data:
            print('{:6s}  {:<5d}  {:0.2f}'.format(d[0], d[1], d[2]))
    
    # Query pv
    base = 'http://www.chessdb.cn/cdb.php?action=querypv&json=1&board='
    url = base + fen
    r = requests.get(url)
        
    jdata = r.text
    d = json.loads(jdata)

    if d['status'] == 'ok':
        score = d['score']
        depth = d['depth']
        pv = ' '.join(d['pvSAN'])
        
        print()
        print('(2) PV info of top move:')
        print('{}/{} pv {}'.format(score, depth, pv))
        
    # Query database stats
    print()
    print('(3) Database stats:')
    url = 'http://www.chessdb.cn/statsc.php?lang=1'
    try:
        r = requests.get(url)
        res = r.text
        sp = res.split('<table class="stats"><tr><td>DB Size ( Storage / Queue ) :</td><td style="text-align: right;">')[1]
        sp = sp.split('</td>')
        value = sp[0].strip()
        print('DB Size ( Storage / Queue ) : {}'.format(value))
    except Exception as err:
        print(err)
    
    
def main():  
    print('Probe ChessDB Book v0.1')
    while True:
        fen = input('enter fen or epd or q to quit? ')
        if fen.lower() == 'q':
            break
        probe_book(fen)
        time.sleep(2)
    

if __name__ == '__main__':
    main()

Sample run:

Code: Select all

Probe ChessDB Book v0.1
enter fen or epd or q to quit? r1bqkb1r/pppp1ppp/2n2n2/4p1N1/2B1P3/8/PPPP1PPP/RN
BQK2R b KQkq -
(1) Move Stats:
Move    Score  Winrate
d5      -5     49.62
Bc5     -175   37.04
Nxe4    -248   32.05
Qe7     -290   29.34
Nd5     -365   24.86
b5      -375   24.30
d6      -418   21.98
g6      -453   20.22
Na5     -461   19.83
Ba3     -473   19.26
a5      -487   18.61
Bb4     -509   17.62
Bd6     -512   17.49
a6      -526   16.88
b6      -589   14.37
Rb8     -599   14.00
Nd4     -612   13.53
Nb4     -612   13.53
Nb8     -636   12.71
Rg8     -651   12.21
Ng8     -721   10.11
Ke7     -725   10.00
Be7     -730   9.87
h6      -780   8.60
Ng4     -796   8.23
h5      -805   8.02
Nh5     -885   6.41

(2) PV info of top move:
-5/30 pv d5 exd5 Na5 Bb5+ c6 dxc6 bxc6 Bd3 Nd5 Nf3 Bd6 O-O O-O Re1 f5 Nxe5 Qf6 N
f3 g5 g3 f4 Nc3 fxg3 hxg3 Qxf3 Qxf3 Rxf3 Be4 Rf7 Nxd5

(3) Database stats:
DB Size ( Storage / Queue ) : 63.92 GB / 17.60 GB
enter fen or epd or q to quit?