Very interesting, thank you very much.
One question: When I find a leaf in the tree (position after last given move not found) does it mean that this was evaluated with Stockfish depth 22 and no other analysis starting from that position was performed?
Database snapshot
Moderators: hgm, Rebel, chrisw
-
- Posts: 157
- Joined: Tue Apr 30, 2013 1:29 am
Re: Database snapshot
It unfortunately wants to play a losing line in Vienna with black (and Vienna is its first choice). It's the same problem Leela has searching from the position after 4th move:
[pgn] 1.d4 Nf6 2.c4 e6 3.Nf3 d5 4.Nc3 dxc4 5.e4 Bb4 6.Bxc4 Nxe4 7.O-O Bxc3 8.bxc3 Nxc3 9.Qb3 Nd5 10.Ba3 f6 11.Rae1 Kf7 *[/pgn]
First choices of the book so far for black (with the transposition of 1...Nf6/e6/d5) and this position it only gives:
Bxd5 0 50.00
Unfortunately for black 12.Ng5+ is a forced win for white (which btw my SF sees at depth 22 already)
As an anecdote: I played a Lichess blitz game vs a cheater with white in this line and they have fallen into it (probably were using SF on shallow depths as beginners don't know how to change the default settings/use more threads etc.). No human goes into that line as it just looks very risky. They were banned by Lichess a while later.
[pgn] 1.d4 Nf6 2.c4 e6 3.Nf3 d5 4.Nc3 dxc4 5.e4 Bb4 6.Bxc4 Nxe4 7.O-O Bxc3 8.bxc3 Nxc3 9.Qb3 Nd5 10.Ba3 f6 11.Rae1 Kf7 *[/pgn]
First choices of the book so far for black (with the transposition of 1...Nf6/e6/d5) and this position it only gives:
Bxd5 0 50.00
Unfortunately for black 12.Ng5+ is a forced win for white (which btw my SF sees at depth 22 already)
As an anecdote: I played a Lichess blitz game vs a cheater with white in this line and they have fallen into it (probably were using SF on shallow depths as beginners don't know how to change the default settings/use more threads etc.). No human goes into that line as it just looks very risky. They were banned by Lichess a while later.
-
- Posts: 560
- Joined: Sun Nov 08, 2015 11:10 pm
Re: Database snapshot
Yes.OneTrickPony wrote: ↑Mon Jul 29, 2019 8:42 pm Very interesting, thank you very much.
One question: When I find a leaf in the tree (position after last given move not found) does it mean that this was evaluated with Stockfish depth 22 and no other analysis starting from that position was performed?
Of course, there must be infinite number of holes, or chess would be solved.OneTrickPony wrote: ↑Mon Jul 29, 2019 9:15 pm As an anecdote: I played a Lichess blitz game vs a cheater with white in this line and they have fallen into it (probably were using SF on shallow depths as beginners don't know how to change the default settings/use more threads etc.). No human goes into that line as it just looks very risky. They were banned by Lichess a while later.
I have a Lichess bot which uses this database as its opening book, so it could be tested on whether the self-improving method works in practice.
-
- Posts: 4556
- Joined: Tue Jul 03, 2007 4:30 am
Re: Database snapshot
Thanks! This was exactly the thing I would have tried to find first by querying the databaseOneTrickPony wrote: ↑Mon Jul 29, 2019 9:15 pm It unfortunately wants to play a losing line in Vienna with black (and Vienna is its first choice). It's the same problem Leela has searching from the position after 4th move:
[pgn]1.d4 Nf6 2.c4 e6 3.Nf3 d5 4.Nc3 dxc4 5.e4 Bb4 6.Bxc4 Nxe4 7.O-O Bxc3 8.bxc3 Nxc3 9.Qb3 Nd5 10.Ba3 f6 11.Rae1 Kf7 *[/pgn]
First choices of the book so far for black (with the transposition of 1...Nf6/e6/d5) and this position it only gives:
Bxd5 0 50.00
Unfortunately for black 12.Ng5+ is a forced win for white (which btw my SF sees at depth 22 already)
My analysis deviates with 9...b5 with backsolved score of 0.36, but in general, this is a line black doesn't want to play (4...Be7!! is 0.00). I'm glad 1000 times less positions analyzed can hold.
Your beliefs create your reality, so be careful what you wish for.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Database snapshot
Use fen and not epd.Ovyron wrote: ↑Mon Jul 29, 2019 7:41 pm Thanks, I downloaded your exe, ran it in some directory and pasted a fen. I got this error:
Traceback (most recent call last):
File "book_probe2.py", line 55, in <module>
File "book_probe2.py", line 50, in main
File "book_probe2.py", line 26, in probe_book
IndexError: list index out of range
[172] Failed to execute script book_probe2
If you only have epd, just add "0 1" at the end to make it a fen.
-
- Posts: 4556
- Joined: Tue Jul 03, 2007 4:30 am
Re: Database snapshot
Thanks, now it works.
This is really cool, I kind of feel like I downloaded some huge database that weighs just 9MB
Your beliefs create your reality, so be careful what you wish for.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Database snapshot
Also working on simple html interface, I hope to release it next week.
-
- Posts: 4556
- Joined: Tue Jul 03, 2007 4:30 am
Re: Database snapshot
Hi Ferdy, I tried that for this position:Ferdy wrote: ↑Tue Jul 30, 2019 12:47 amUse fen and not epd.Ovyron wrote: ↑Mon Jul 29, 2019 7:41 pm Thanks, I downloaded your exe, ran it in some directory and pasted a fen. I got this error:
Traceback (most recent call last):
File "book_probe2.py", line 55, in <module>
File "book_probe2.py", line 50, in main
File "book_probe2.py", line 26, in probe_book
IndexError: list index out of range
[172] Failed to execute script book_probe2
If you only have epd, just add "0 1" at the end to make it a fen.
[d]r1bqkb1r/pppp1ppp/2n2n2/4p1N1/2B1P3/8/PPPP1PPP/RNBQK2R b KQkq -
But entering:
r1bqkb1r/pppp1ppp/2n2n2/4p1N1/2B1P3/8/PPPP1PPP/RNBQK2R b KQkq - 0 1
Produces errors and it exits.
Your beliefs create your reality, so be careful what you wish for.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Database snapshot
You found a bug. The exe expects to have a winrate but there is a move from the database that does not give a win rate and this is:Ovyron wrote: ↑Tue Jul 30, 2019 3:37 pmHi Ferdy, I tried that for this position:Ferdy wrote: ↑Tue Jul 30, 2019 12:47 amUse fen and not epd.Ovyron wrote: ↑Mon Jul 29, 2019 7:41 pm Thanks, I downloaded your exe, ran it in some directory and pasted a fen. I got this error:
Traceback (most recent call last):
File "book_probe2.py", line 55, in <module>
File "book_probe2.py", line 50, in main
File "book_probe2.py", line 26, in probe_book
IndexError: list index out of range
[172] Failed to execute script book_probe2
If you only have epd, just add "0 1" at the end to make it a fen.
[d]r1bqkb1r/pppp1ppp/2n2n2/4p1N1/2B1P3/8/PPPP1PPP/RNBQK2R b KQkq -
But entering:
r1bqkb1r/pppp1ppp/2n2n2/4p1N1/2B1P3/8/PPPP1PPP/RNBQK2R b KQkq - 0 1
Produces errors and it exits.
Code: Select all
{'uci': 'c6e7', 'san': 'Ne7', 'score': -29998, 'rank': 0, 'note': '? (01-01)'}
I can modify the exe to handle such error and not exit the program, but lets wait for noobpwnftw to fix it by sending the winrate even if it is zero.
The html interface handles it by itself.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Database snapshot
New version of exeOvyron wrote: ↑Tue Jul 30, 2019 3:37 pmHi Ferdy, I tried that for this position:Ferdy wrote: ↑Tue Jul 30, 2019 12:47 amUse fen and not epd.Ovyron wrote: ↑Mon Jul 29, 2019 7:41 pm Thanks, I downloaded your exe, ran it in some directory and pasted a fen. I got this error:
Traceback (most recent call last):
File "book_probe2.py", line 55, in <module>
File "book_probe2.py", line 50, in main
File "book_probe2.py", line 26, in probe_book
IndexError: list index out of range
[172] Failed to execute script book_probe2
If you only have epd, just add "0 1" at the end to make it a fen.
[d]r1bqkb1r/pppp1ppp/2n2n2/4p1N1/2B1P3/8/PPPP1PPP/RNBQK2R b KQkq -
But entering:
r1bqkb1r/pppp1ppp/2n2n2/4p1N1/2B1P3/8/PPPP1PPP/RNBQK2R b KQkq - 0 1
Produces errors and it exits.
https://drive.google.com/file/d/1_KM89V ... sp=sharing
You can now just input epd.
python source code.
Code: Select all
#!/usr/bin/env python3
"""
probe_chessdb_book.py
Probe book moves in ChessDB
Requirements:
* Python 3
* Requests
pip install requests
Dev log:
v0.1
* Use http
* Add pv of bestmove results
* Add Database stats results
* Use fen directly after ...board=
* epd can now be entered
* Handle errors in querying all&board
"""
import requests
import json
import time
def probe_book(fen):
data = []
# Query all moves
base = 'http://www.chessdb.cn/cdb.php?action=queryall&json=1&board='
url = base + fen
try:
r = requests.get(url)
r.raise_for_status()
except requests.exceptions.HTTPError as errh:
print("Http Error:",errh)
return
except requests.exceptions.ConnectionError as errc:
print("Error Connecting:",errc)
return
except requests.exceptions.Timeout as errt:
print("Timeout Error:",errt)
return
except requests.exceptions.RequestException as err:
print("OOps: Something Else",err)
return
jdata = r.text
d = json.loads(jdata)
if d['status'] == 'ok':
for n in d['moves']:
try:
data.append([n['san'], int(n['score']), float(n['winrate'])])
except KeyError:
pass
else:
print('Position is not found!')
if len(data) > 0:
print('(1) Move Stats:')
print('{:6s} {:<5s} {}'.format('Move', 'Score', 'Winrate'))
for d in data:
print('{:6s} {:<5d} {:0.2f}'.format(d[0], d[1], d[2]))
# Query pv
base = 'http://www.chessdb.cn/cdb.php?action=querypv&json=1&board='
url = base + fen
r = requests.get(url)
jdata = r.text
d = json.loads(jdata)
if d['status'] == 'ok':
score = d['score']
depth = d['depth']
pv = ' '.join(d['pvSAN'])
print()
print('(2) PV info of top move:')
print('{}/{} pv {}'.format(score, depth, pv))
# Query database stats
print()
print('(3) Database stats:')
url = 'http://www.chessdb.cn/statsc.php?lang=1'
try:
r = requests.get(url)
res = r.text
sp = res.split('<table class="stats"><tr><td>DB Size ( Storage / Queue ) :</td><td style="text-align: right;">')[1]
sp = sp.split('</td>')
value = sp[0].strip()
print('DB Size ( Storage / Queue ) : {}'.format(value))
except Exception as err:
print(err)
def main():
print('Probe ChessDB Book v0.1')
while True:
fen = input('enter fen or epd or q to quit? ')
if fen.lower() == 'q':
break
probe_book(fen)
time.sleep(2)
if __name__ == '__main__':
main()
Sample run:
Code: Select all
Probe ChessDB Book v0.1
enter fen or epd or q to quit? r1bqkb1r/pppp1ppp/2n2n2/4p1N1/2B1P3/8/PPPP1PPP/RN
BQK2R b KQkq -
(1) Move Stats:
Move Score Winrate
d5 -5 49.62
Bc5 -175 37.04
Nxe4 -248 32.05
Qe7 -290 29.34
Nd5 -365 24.86
b5 -375 24.30
d6 -418 21.98
g6 -453 20.22
Na5 -461 19.83
Ba3 -473 19.26
a5 -487 18.61
Bb4 -509 17.62
Bd6 -512 17.49
a6 -526 16.88
b6 -589 14.37
Rb8 -599 14.00
Nd4 -612 13.53
Nb4 -612 13.53
Nb8 -636 12.71
Rg8 -651 12.21
Ng8 -721 10.11
Ke7 -725 10.00
Be7 -730 9.87
h6 -780 8.60
Ng4 -796 8.23
h5 -805 8.02
Nh5 -885 6.41
(2) PV info of top move:
-5/30 pv d5 exd5 Na5 Bb5+ c6 dxc6 bxc6 Bd3 Nd5 Nf3 Bd6 O-O O-O Re1 f5 Nxe5 Qf6 N
f3 g5 g3 f4 Nc3 fxg3 hxg3 Qxf3 Qxf3 Rxf3 Be4 Rf7 Nxd5
(3) Database stats:
DB Size ( Storage / Queue ) : 63.92 GB / 17.60 GB
enter fen or epd or q to quit?