Database snapshot

[noobpwnftw]

finally there exists a "Discover" program people can run, which loads any UCI-compatible engine and play games, contributing moves encountered during the process.

I tried to compile ChessDBDiscover and it seems to work. Created an engine dir and put engine.exe, a renamed stockfish dev engine, then run ChessDBDiscover.exe with default config values.



So what does the ChessDBDiscover.exe is doing? Does it play engine vs engine matches based on StartFEN in config file? If so how many games? Does it send the results to the cloud?

Yes, it keeps playing games and submitting results until you close the program.

[Damir]

For those who want to probe my database locally or for other unspecified reasons, here is a full database snapshot of my book project as of today:

ftp://ftp.chessdb.cn/pub/chessdb/data-s ... 190728.tar

The database contains about 3 billion unique chess positions, mostly connected to startpos, analyzed by Stockfish with no less than 22 plies at terminal node and has a very wide multi-pv exploration, the scores been back-propagated using a weighted averaging function, also for most of the positions there is a special field(encoded as 'a0a0') marking known shortest distance of the position from startpos.

Using this database snapshot is as simple as putting the data files under your database folder and launch the server, yet still, I'd recommend you to use the online API and make feature requests if you need any, since it is getting updated constantly and I have no plans to make such kind of snapshots very frequently(while waiting for a contributor to make incremental snapshots possible).

This database snapshot is released into the public domain.

Is it possible to download your database in PGN as PGN format ?

[Damir]

or somehow convert it with a tool to PGN?

[Ferdy]

Updated the probe_chessdb_book.exe at https://drive.google.com/file/d/1_KM89V ... sp=sharing

Positive mate score with missing winrate is now fixed. winrate is now calculated based on score returned using logistic function. So a mate in 1 has a 100% winrate now.

[d]r1bqkb1r/ppppnppp/5n2/4p1N1/2B1P3/8/PPPP1PPP/RNBQK2R w KQkq - 6 5

Code: Select all

enter fen or epd or q to quit? r1bqkb1r/ppppnppp/5n2/4p1N1/2B1P3/8/PPPP1PPP/RNBQK2R w KQkq - 6 5
(1) Move Stats:
Move    Score   Winrate
Bxf7#   29999   100.00 

(2) PV info of top move:
29999/1 pv Bxf7#

(3) Database stats:
DB Size ( Storage / Queue ) : 64.41 GB / 23.60 GB

[Ferdy]

finally there exists a "Discover" program people can run, which loads any UCI-compatible engine and play games, contributing moves encountered during the process.

I tried to compile ChessDBDiscover and it seems to work. Created an engine dir and put engine.exe, a renamed stockfish dev engine, then run ChessDBDiscover.exe with default config values.



So what does the ChessDBDiscover.exe is doing? Does it play engine vs engine matches based on StartFEN in config file? If so how many games? Does it send the results to the cloud?

Yes, it keeps playing games and submitting results until you close the program.

I tried to run ChessDBDiscover with the position in the image for not less 2 hours. This position is the last without book info.



How do I know if positions from the run are saved in cloud?

[noobpwnftw]

How do I know if positions from the run are saved in cloud?

It will be there, just currently the DB has a very long queue, need a while to process them.

[Ferdy]

Use fen and not epd.
If you only have epd, just add "0 1" at the end to make it a fen.

Thanks, now it works.

This is really cool, I kind of feel like I downloaded some huge database that weighs just 9MB

Also working on simple html interface, I hope to release it next week.

Html interface is released at https://github.com/fsmosca/ChessDB-Online-Book

Download the whole thing and open index.html.

Or try it at https://fsmosca.github.io/ChessDB-Online-Book/
Make sure to enable javascript.

[zullil]

Html interface is released at https://github.com/fsmosca/ChessDB-Online-Book

Download the whole thing and open index.html.

Or try it at https://fsmosca.github.io/ChessDB-Online-Book/
Make sure to enable javascript.

Works fine if I download and open index.html. But the version online you've linked to doesn't work for me.

[EDIT] OK, Chrome is saying "Insecure content blocked. This page is trying to load scripts from unauthenticated sources." If I allow Chrome to "load unsafe scripts", then it works.

[zullil]

How was the "winrate" for each position determined?

[Ferdy]

How was the "winrate" for each position determined?

I suspect it is logistic function based on score. The score is from
"the scores been back-propagated using a weighted averaging function" see first post of this thread.

In javascipt when winrate is missing (by design) for mate scores, I use

Code: Select all

// 2019-08-02: ChessDB does not return winrate when score is mate.
// So we will calculate winrate by logistic function
if (typeof winrate === "undefined") {
  const K = 1.2;
  winrate = 100 * 1/(1 + (10**(-K*score/400)));
  winrate = winrate.toFixed(2);
}

Even for non-mate scores, this function is close to the winrate from server.