Lomonosov Endgame Tablebases

[tpoppins]

What difference does that make? 1,000 discs is as unusable as 5,000.

[jpqy]

You have the upcoming 16TB SSD's

http://www.pcper.com/news/Storage/FMS-2 ... center-SSD
http://www.vcpost.com/articles/89439/20 ... drives.htm
http://www.pcworld.com/article/3021886/ ... -here.html
http://www.tomsitpro.com/articles/sandi ... -1925.html

JP.

[Nordlandia]

You have the upcoming 16TB SSD's

http://www.pcper.com/news/Storage/FMS-2 ... center-SSD
http://www.vcpost.com/articles/89439/20 ... drives.htm
http://www.pcworld.com/article/3021886/ ... -here.html
http://www.tomsitpro.com/articles/sandi ... -1925.html

JP.

Ronald talked about "20 TB 7-Men Syzygy" in case with max compression.

[shrapnel]

You have the upcoming 16TB SSD's

http://www.pcper.com/news/Storage/FMS-2 ... center-SSD
http://www.vcpost.com/articles/89439/20 ... drives.htm
http://www.pcworld.com/article/3021886/ ... -here.html
http://www.tomsitpro.com/articles/sandi ... -1925.html

JP.

I have been using only SSDs for the past 2 years. No more obsolete HDD for me.
Anyway, if 100TBs required, Lomonosov is basically useless .

[velmarin]

There are not even a code of probe for engines or GUIs.
Aquarium or chess asssistant do not have overall support. Only online.
I think its utility for the moment is for the postal chess, or curiosity....
It is curious that with Android its access is free.
Or an emulator... Very well.
[/img]

[bob]

can you put 100 TB on a hard drive..? are there any PB drives yet?

We have a 4 petabyte raid array here today. UAB is looking at a 100 petabyte array for the campus. So it is doable. But you won't buy one of these at Best Buy, and you won't be writing a small check.

You can buy a 4u rack from Dell with 96 6tb drives. That's about 1/2 petabyte or so factoring in redundancy. We have 8 of these. It is anything but cheap, and it produces more heat than the typical household furnace.

[whereagles]

lol..

right. cheers

[Norbert Raimund Leisner]

A question to Aquarium or ChessAssistant owners:

Which chess engines are supporting Lomonosov EGTBs?

http://chessok.com/?page_id=27966 - it seems that the are only available via online-access to the ChessOK-server.

I asked the Convekta-company, but have got no reply from the support team until today.....

Norbert

[Cumnor]

They do supply with Aquarium a modified Stockfish that uses Lomonosov TBs
but its an old Stockfish not been update to the latest SF version.

Picture here http://www.screenleap.com/comprybka/scr ... onosov+tbs

[Zenmastur]

I have been using only SSDs for the past 2 years. No more obsolete HDD for me.
Anyway, if 100TBs required, Lomonosov is basically useless .

HDD's aren't obsolete just yet. If you arbitrarily restrict yourself to using just SSD's that's your problem!

Lomonosov may be useless to you but 7 man EGTB's aren't useless in general.

I recently did a survey of a 7 million game data base to determine the frequency of occurrence of various end games. It turns out that many end games never occurred in 7 million games. So those are pretty much a waste of time to generate and a waste of space to store. E.G. 6 man 5v1 are pretty worthless. They simply won't occur often enough to make a difference and even if they did occur more often the out-come won't be influenced by the use of an EGTB because they are so lopsided that any good engine can achieve the best possible outcome on it's own.

On the other hand some end games occur in a huge fraction of games (comparatively speaking) AND are complicated enough that engines don't play them well. I looked at all end games with 7 men or less, except 6v1 and 5v1, plus selected 8 man end games that I thought would have a high probability of occurrence. BY FAR the end game that is most likely to occur is krpp-vs-krp which occurred in 165,068 games out of 7,076,320 games. (I.E. in 2.33% of all games) This represents over 26% of ALL 7 man end games in a single file. None of the 8 man end game that I checked nor any of the 6 man end game had any where near this number.

So, the moral of this story is you don't need the entire set of 7 man EGTB's for them to be useful. Therefore the storage requirements can be considerably reduced compare to the full set while still providing good to excellent service.

Some may object to using a partial sets because a conversion by way of a pawn promotion MAY reference an EGTB that isn't included in a partial set. I say MAY because most conversions (85% or MORE emphasis added), occur by capture which reduces it to a 6 man EGTB. Some terminate in checkmate or stalemate and a few will convert through promotions. Almost all promotions will be to a queen. In many positions this will be enough to decide the game without further use of EGTBs. In other cases judicious selection of end games to be included in the partial set many of these will fall within the selected sub-set. So only a very small % of all probes will suffer from the effects of an incomplete data set.

One other thing to note:

If a particular end game such as krppkrp occurs in a particular percentage of all game, in this case 2.33%, this IS NOT the only games it will influence. This one EGTB is likely to be referenced during, and therefore influence, a huge fraction of all games played. The problem is that measuring the fraction of games influenced would require a very clever implementation OR use a lot of computer resources. Regardless of what the actual percentage of games it would influence if it were generally available, you can be assured that the number will be considerably larger than 2.33%. If I had to hazard a guess, I would put it in the 20% range at a minimum.

Size is important! So I checked the size of 25 uncompressed Lomosov 6 man EGTB's vice their SYZYGY size to see what compression factor could be expected for 7 man EGTB's. 37 to one seems to be a reasonable number. This would make the krppkrp file about 22 GB.

I looked at the top 100 7-man end games, by number of occurrences. They represent 95% of all 7-man endgames found in my data base. The top 56 represent 90%, the top 27 represents 80%, the top 14 represents 67% of all occurrences. The top 14 files also represents more games than all 6-man and fewer EGTB combined. I.E. the top 14 7-man EGTB accounted for 422,800 games out of a total of 7,076,320 games or about 6% of all games. Assuming approximately the same compression ratio for these files as for Syzygy they would occupy about 314 GB.

However, if we sort them by number of occurrences in the database per MB of storage space required we'll arrive at a slightly different set of files. With this ordering the top 17 files accounts for 66.6% of all occurrences, requires about 193 GB and has two addition files with queens to help with conversions by pawn promotions.

So, Lomonosov may be useless to you due to using only SSDs, but that doesn't mean that the general case of 7-man EGTB's is a useless endeavor for the rest of us.


Regards,

Forrest