What is the best way to obtain the 7-piece tablebases?

[daniel71]

I recomend the website Lichess to download all the Syzygy 3-4-5-6-7 tablebases. https://tablebase.lichess.ovh/tables/standard/
Jimbo I recommended http://www.downthemall.net/
Which I used and is fantastic!! I cannot recommend any software higher this this to get the files! Thanks Jimbo I
http://talkchess.com/forum3/viewtopic.p ... 68#p745007

Invest in fiberoptic internet service, otherwise look into gigabit cable service. I downloaded the 6 piece tablebases with 200 down/ 20 up service, without trouble, after giving up before without the download them all software. I had tried for months and gave up before Jimbo I posted and solved all my problems.

[syzygy]

(I do not need depth to win tables).

6 terabytes in practice.

Syzygy author at this forum told many times: RTBW files do not work correctly without RTBZ files

RTBW: 8.5 TiB
RDTBZ: 8.3 TiB
----------------
Total: 16.7 TiB

Wrong. RTBW files work fine without RTBZ files. The other way around, not.

RTBW files give win/draw/loss information (plus something about the 50-move rule which we can ignore to keep things simple). As long as you're happy with knowing that the position is a win, draw or loss, RTBW files are all you need.

RTBZ files lack win/draw/loss information. They give a number for each position, but this number cannot be interpreted without knowing whether the position is a win or draw or loss.

7-piece RTBW files do not work without 3/4/5/6-piece RTBW files.

If you want to let an engine probe the TBs during the search, the RTBW files should be on SSD or the search will slow down a lot. For RTBZ files this is much less critical.

[ernest]

Wrong. RTBW files work fine without RTBZ files. The other way around, not.

RTBW files give win/draw/loss information (plus something about the 50-move rule which we can ignore to keep things simple). As long as you're happy with knowing that the position is a win, draw or loss, RTBW files are all you need.

Thanks for the clarification, Ronald !

Now what is the difference in play when, say Stockfish, uses only RTBW files or uses both RTBW and RTBZ files ?
(is it "plays the same, except for the 50-move rule" ?)

[duncan]

If you want to let an engine probe the TBs during the search, the RTBW files should be on SSD or the search will slow down a lot. For RTBZ files this is much less critical.

My apologies for being off topic, but if you had access to Summit scientific supercomputer, how many piece tablebase would it be able to solve and how long would it take to calculate them ?


https://www.ornl.gov/news/ornl-launches ... ercomputer

OAK RIDGE, Tenn., June 8, 2018—The U.S. Department of Energy’s Oak Ridge National Laboratory today unveiled Summit as the world’s most powerful and smartest scientific supercomputer.

With a peak performance of 200,000 trillion calculations per second—or 200 petaflops, Summit will be eight times more powerful than ORNL’s previous top-ranked system, Titan.


The IBM AC922 system consists of 4,608 compute servers, each containing two 22-core IBM Power9 processors and six NVIDIA Tesla V100 graphics processing unit accelerators, interconnected with dual-rail Mellanox EDR 100Gb/s InfiniBand. Summit also possesses more than 10 petabytes of memory paired with fast, high-bandwidth pathways for efficient data movement. The combination of cutting-edge hardware and robust data subsystems marks an evolution of the hybrid CPU–GPU architecture successfully pioneered by the 27-petaflops Titan in 2012.

[Dann Corbit]

The Oak Ridge computer would have to build the Syzygy tablebase files the same way that Bojun Guo did.
The GPUs would be useless for the calculation unless the code was completely rewritten.
There is an enormous volume of regular CPU cores as well, so it would be possible to build them faster.
As far as 8-9-10... man tablebase files, I guess that the source would have to be rewritten in order to create those.

So the answer to your question is that the Oak Ridge supercomputer could build the tables faster than Bojun Guo did by some scale factor.
I guess that it would still take longer than just downloading the finished tables with a fast net connection.
Those computers would not be able to create bigger files because the software does not allow for that.
And they are tied up doing classified calculations so they are not available anyway, and the cost per hour if you were able to rent it would be frightening.

[mbabigian]

I recomend the website Lichess to download all the Syzygy 3-4-5-6-7 tablebases. https://tablebase.lichess.ovh/tables/standard/
Jimbo I recommended http://www.downthemall.net/
Which I used and is fantastic!! I cannot recommend any software higher this this to get the files! Thanks Jimbo I
viewtopic.php?f=2&t=66096&p=745068#p745007

I wrote my own downloader and pulled them from Lichess. The key difference was the process was fully automated. The program downloaded the files in the order I chose so I could get the most important ones first, and it SHA hashed the file when done. If the hash was incorrect, it automatically deleted and downloaded the file again. Once a correct version was downloaded, it moved it into a new directory. Downloading all but the 6x1 files took around 6 weeks.

I didn't debug the snot out of the program, but if there is interest, I could post it when I have time to pull together some basic documentation. Oh, and the program had the ability to resume. So when I needed my internet link for something important, I could stop and restart without losing ground on the current partially downloaded file. Same if the internet went down during, which happened a few times (just restart).

[nnnnnnnn]

This does sound useful.

[mbabigian]

If I didn't screw something up, the downloader and associated files/documentation can be found at: www.software.farseer.org

Hope it helps.
Mike

[nnnnnnnn]

Thank you, that looks useful for Windows users. I wonder if there is something like that for Mac.

[yurikvelo]

how many piece tablebase would it be able to solve and how long would it take to calculate them ?

http://tb7.chessok.com/articles/Top8DTM_eng

This article from the Lomonosov tablebase page estimates that the size of an 8-man tablebase will be ~100x larger than a 7-man table base and will take a minimum of 10,000 TB of disk space and 50 TB of RAM, limiting its completion to a top-10 super computer

Only top 10 supercomputers can solve 8-man problem in 2014.The first 1000-move mate is unlikely to be found until 2020 when a part of a TOP100 supercomputer may be allowed to be used for solving this task.

6 months * 100 [size factor] * (1.7 / 125) [computing speed] = 8.16 months on the world's fastest super computer.

Adding just one more piece to the 7-man tablebases multiplies the number of positions PER material distribution by a factor of about 57.

For example the 8-piece endgame of:

King + Queen + Rook + Knight vs. King + Queen + Rook + Knight

… is roughly 57 times as many arrangements as the 7-piece endgame of:

King + Queen + Rook + Knight vs. King + Queen + Rook

Not only that, there are more combinations of 8-piece endgames than there are 7-piece endgames. Fortunately, there is exploitable side-to-move symmetry in even tablebase endings when both sides have the same exact material.

So scale up the time it took to do the 7-piece tablebases by 57 times the ratio of 8-piece to 7-piece unique endgame counts, and you have your answer. It's at least 57 x 5 = 285 times as large. Put another way: each day it took to do the 7-man tablebases will be one year of 8-piece computation.