Syzygy endgame tables: Generation and first impressions

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

User avatar
Mike S.
Posts: 1480
Joined: Thu Mar 09, 2006 5:33 am

Syzygy endgame tables: Generation and first impressions

Post by Mike S. »

(The following text is partially taken from Google translation as I am a very lazy man these days :mrgreen: )

First, a link to basic information: http://chessprogramming.wikispaces.com/Syzygy+Bases

It says that the Syzygy bases of Ronald de Man, take the 50 move rule into account - what the Nalimovs lack - and are very compact: Less than a gigabyte for 3/4/5-men.

These tables you can either download or create yourself. Who has a flat rate and a fast internet connection will probably like to use the following source:

http://chess.cygnitec.com/tablebases/syzygy/ (3/4/5-men)

Still no 7-Zip on board? http://7-zip.org/

Although not a single file, but a few only. Just unzip everything into the same directory - as always best to SSD or flash memory - and that is the Syzygy-Tablebase path. If I am not mistaken, it should be 290 files (except for any text files you do not need).

Alternatively, you can also generate them yourself. It took me here overnight on dual core i5 CPU just under 6 hours. Total RAM 4 GB (I do not know what is the minimum for five piece). The TCEC chatter Trym has kindly made ​​available a download that contains the software for the generation that is under the GNU license and a PERL environment:

http://www.solheimsvollen.net/TBGen/SZGen.zip (approx. 27 MB)

PERL is only necessary to use a perl script that can greatly simplify the generation. For operation of the tables is of course not necessary and could be uninstalled after a successful preparation of tables.

If you have PERL (*) installed and unpacked the rest in any directory - ideally in the planned future Syzygy directory - you open a command line window and enter:

run.pl --generate --max 5

Optionally, you can also number the CPU threads to be used, for example --threads 4 (whichever you have).

Warning: 6-men generation requires 16 GB of RAM ... and probably a lot of time.

First experience:

Coincidentally, the Syzyzgy-enabled Stockfish version I found (the download link I unfortunately forgot) is of the same date as the development version I have currently installed, 23.09.2013. Thus, it could almost be a 1:1 comparison with / without Syz. Bases. This I have done with ultra-short time control and using 10 endgame starting positions. Not all symmetrical, as is also queen against two rooks or minor piece against three pawns, all of course with more pawns.

Not surprisingly, all 20 test games were drawn. It was, however, interesting to see: 1 Technically flawless course, 2 in-depth observation no computing differences detected (Tables on flash memory card). The engines typically started with 18 depts, then followed with less material soon 20+ depths.

But that was on a single core, ponder off. That it may come on very large computing depths and / or with many cores to "phenomena", about that I cannot make a statement. But basically I think that first impression is quite confidence-inspiring.

Current sources of Syzygy-enabled Stockfish is available at https://github.com/syzygy1/stockfish
But so far, unfortunately no executables.
Regards, Mike
syzygy
Posts: 5774
Joined: Tue Feb 28, 2012 11:56 pm

Re: Syzygy endgame tables: Generation and first impressions

Post by syzygy »

This link set up by Joshua Shriver (see this post) may help to get the full 6-men tables using bittorrent.

Alternatively, they can be generated on a computer with at least 16GB RAM or more. I would recommend using the --threads option (e.g. --threads 4 for a 4-core cpu or --threads 8 for a 4-core cpu with HT). On a computer with less than 24GB, add an option --disk.

Generating the 6-men tables takes about 100x as long as generating the 3-4-5-men tables.

See also Kirill Kryukov's site.
Vinvin
Posts: 5299
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: Syzygy endgame tables: Generation and first impressions

Post by Vinvin »

Hello Ronald, is there some 7 pieces available ?
I'm looking for RPPRP Syzygy ...
syzygy
Posts: 5774
Joined: Tue Feb 28, 2012 11:56 pm

Re: Syzygy endgame tables: Generation and first impressions

Post by syzygy »

Vinvin wrote:Hello Ronald, is there some 7 pieces available ?
I'm looking for RPPRP Syzygy ...
They are not available. With a few modifications the generator should be able to handle 7 pieces, but memory requirements are pretty huge (1 TB of shared RAM). Using a HD for swap space will most likely lead to unbearable disk thrashing. Using an SSD for swap space will most likely wear out the SSD long before you are finished with all the subtables required for generating RPPRP.

The full 7-piece WDL set might take less than 10 TB and I expect the full 7-piece DTZ set to be of similar size or maybe even smaller. In some years this might be practical.
User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 4:27 pm

Re: Syzygy endgame tables: Generation and first impressions

Post by Don »

syzygy wrote:
Vinvin wrote:Hello Ronald, is there some 7 pieces available ?
I'm looking for RPPRP Syzygy ...
They are not available. With a few modifications the generator should be able to handle 7 pieces, but memory requirements are pretty huge (1 TB of shared RAM). Using a HD for swap space will most likely lead to unbearable disk thrashing. Using an SSD for swap space will most likely wear out the SSD long before you are finished with all the subtables required for generating RPPRP.

The full 7-piece WDL set might take less than 10 TB and I expect the full 7-piece DTZ set to be of similar size or maybe even smaller. In some years this might be practical.
Please let me know when the 32 piece database is ready :-) Seriously, I am very interested in implementing this in Komodo. Hopefully it will be soon.

Don
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
syzygy
Posts: 5774
Joined: Tue Feb 28, 2012 11:56 pm

Re: Syzygy endgame tables: Generation and first impressions

Post by syzygy »

Don wrote:Seriously, I am very interested in implementing this in Komodo. Hopefully it will be soon.
That's great. Should you have difficulties with my example code (I don't know if the comments I've added are sufficiently clear), then send me a pb or e-mail. (Same applies to anyone else trying to add support.)
User avatar
jshriver
Posts: 1360
Joined: Wed Mar 08, 2006 9:41 pm
Location: Morgantown, WV, USA

Re: Syzygy endgame tables: Generation and first impressions

Post by jshriver »

syzygy wrote: That's great. Should you have difficulties with my example code (I don't know if the comments I've added are sufficiently clear), then send me a pb or e-mail. (Same applies to anyone else trying to add support.)
Do you have a version of the probing code that doesn't use the stockfish source? It's probably lacking on my part but attempts so far have failed as it seems to require extra code from stockfish. Appears what's in the git for probing is actually the stockfish patch code. So I'm not sure what is from you and what is from stockfish, and how to separate it out.

I just need more hours in the day for this hobby :)
syzygy
Posts: 5774
Joined: Tue Feb 28, 2012 11:56 pm

Re: Syzygy endgame tables: Generation and first impressions

Post by syzygy »

jshriver wrote:
syzygy wrote:That's great. Should you have difficulties with my example code (I don't know if the comments I've added are sufficiently clear), then send me a pb or e-mail. (Same applies to anyone else trying to add support.)
Do you have a version of the probing code that doesn't use the stockfish source? It's probably lacking on my part but attempts so far have failed as it seems to require extra code from stockfish. Appears what's in the git for probing is actually the stockfish patch code. So I'm not sure what is from you and what is from stockfish, and how to separate it out.
My probing code does not need the Stockfish code, but it needs a move generator. I might at some point provide probing code that includes a basic move generator, but that shouldn't be used by engines because the required conversions between board representations would only slow down the engine.
User avatar
Sylwy
Posts: 4881
Joined: Fri Apr 21, 2006 4:19 pm
Location: IAȘI - the historical capital of MOLDOVA
Full name: Silvian Rucsandescu

Re: Some questions for Mr.Ronald de Man

Post by Sylwy »

Hello man !

I generated myself the full 5 men set. Just now I do some tests.
But some things are still unclear for me.
I know - face to Nalimov TBs - they are two advantages:
-a small size;
-knows the 50 moves rule.
Others advantages ?
But face to Gaviota TBs (gtb) ?

Also is still unclear for me if Syzygy Bases are a sort of bitbases ( loaded into the RAM memory) with general informations (sort win-draw-loss) or show us the distance to mate (access at the root) ? Maybe both ?
Otherwise the "thing" looks nice and works very-well in my tests !
THANK YOU !

Have a nice weekend !

SilvianR :wink:

Image
Modern Times
Posts: 3753
Joined: Thu Jun 07, 2012 11:02 pm

Re: Some questions for Mr.Ronald de Man

Post by Modern Times »

They are working very well for me with Stockfish.