6 Man Syzygy vs 5 Man Nalimov

[MikeB]

Not surprisingly , 6 Man Syzygy has a slight edge here.

Code: Select all

Current date : time (EDST)
04/19/16 : 09:09:57
Rank Name Elo + - games score oppo. draws 
1 Stockfish 160226 64bit 3209 5 5 14576 68% 3088 31% 
2 Senpai 1.0 3187 5 5 14576 65% 3088 28% 
3 Texel 1.05 64-bit 3129 4 4 14576 57% 3088 27% 
4 Hakkapeliitta 3.0 3107 4 4 14576 54% 3089 27% 
5 Crafty-25.1cM01 3091 3 3 39584 45% 3114 32% 
6 Crafty-25.1c 3089 3 3 39584 45% 3114 32% 
7 Crafty-25.1b 3079 5 5 13204 50% 3092 48% 
8 Crafty-SYZYGY 3067 7 7 5658 45% 3105 31% 
9 Crafty-25.0.1 3059 7 7 5658 43% 3107 33% 
10 Arasan 18.3 2982 4 4 14576 37% 3090 28% 

Crafty-SYZYGY is crafty 25.0.1 using 6 Man Syzygy Tb's vs crafty 25.0.1 using 5 Man Nalimov. Syzygy has the 5 Man on RAM disk and the 6 Man on SSD. The entire 5 man Nalimov is on RAM disk. Currently looking at 8 ELO difference , which is in the expected ball park.

Vote here if you like to see Syzygy table bases supported by Crafty.

[MikeB]

Should add even if add Syzygy table base support , it will continue to support Nalimov table bases - it would support both formats.

[Nordlandia]

It will continue to support Nalimov table bases - it would support both formats.

I experimented with Stockfish Matefinder couple of days ago and compared the results.

Houdini with Nalimov is able to display mate in x moves faster than Stockfish due to SF have to convert Syzygy to DTM.

[yurikvelo]

Syzygy has the 5 Man on RAM disk and the 6 Man on SSD.

Creating RAM disk is harmfull.

You decrease amount of RAM available for filesystem buffers (probing 6-man), while never probe 5-men.

Even without 6-men, loading Syzygy into RAM disk is pointless, do `cp /syzygy5/*.* > /dev/null' before starting tournament and get better perfomance.

Disk accesses are buffered by OS anyway. If you have RAM disk, accesses are buffered into available RAM, thus copying. If you have no RAM disk - you access buffer directly, without cloning it.

I experimented with Stockfish Matefinder couple of days ago and compared the results

DTM has no influence on ELO.
Slow probing speed and lack of DTZ decrease ELO.

[MikeB]

Creating RAM disk is harmfull.

You decrease amount of RAM available for filesystem buffers (probing 6-man), while never probe 5-men.

Even without 6-men, loading Syzygy into RAM disk is pointless, do `cp /syzygy5/*.* > /dev/null' before starting tournament and get better perfomance.

Disk accesses are buffered by OS anyway. If you have RAM disk, accesses are buffered into available RAM, thus copying. If you have no RAM disk - you access buffer directly, without cloning it.

Please expand on having RAM disk is harmful - in what way. Currently have way more Ram than I need so I have not experienced any ill affects , so I am curious why you say it's harmful. Will try your copying trick , that's seems interesting. On your third point, I should test performance with just SSD. May be just as good as ram.

[yurikvelo]

Currently have way more Ram than I need

You need 100+ GB RAM on Syzygy-6 buffers.
The more RAM you freeze for non-important staff, the more TB-6 probing will query HDD/SDD controller and less blocks found in RAM buffers/caches.

If you disable TB6 and use only TB5 probing - than keeping files in RAM disk will not prevent operating system from caching this files into its own filesystem cache/buffers.
You will not see neither increase nor decrease, since all Syzygy5 files accessed at least once will be cached into RAM.
There will be no actual queries to RAMDisk controller or regular HDD controller. You get just 1GB RAM wasted for nothing.

[bob]

Creating RAM disk is harmfull.

You decrease amount of RAM available for filesystem buffers (probing 6-man), while never probe 5-men.

Even without 6-men, loading Syzygy into RAM disk is pointless, do `cp /syzygy5/*.* > /dev/null' before starting tournament and get better perfomance.

Disk accesses are buffered by OS anyway. If you have RAM disk, accesses are buffered into available RAM, thus copying. If you have no RAM disk - you access buffer directly, without cloning it.

Please expand on having RAM disk is harmful - in what way. Currently have way more Ram than I need so I have not experienced any ill affects , so I am curious why you say it's harmful. Will try your copying trick , that's seems interesting. On your third point, I should test performance with just SSD. May be just as good as ram.

Here's the issue. With lots of memory, the egtbs (if on a normal disk) will end up cached in memory, and can be directly accessed. If you use a RAM disk you do this twice. You have the RAM disk, then files are read from RAM disk to regular memory where they are cached, wasting 1/2 of that memory. Not to mention the cost of copying from RAM disk to regular memory.

The cat trick is something I have been doing since forever. Both for opening books and for endgame tables (when I use them, which is quite rare). It gets them buffered in memory where they will stay if you are not running other things at the same time.

BTW I am not sure you are going to see much if any Elo gain here. I ran a test a few years back where I cluster tested with and without EGTBs and I found absolutely no gain (or loss) at all.

[BBauer]

There will be no actual queries to RAMDisk controller or regular HDD controller. You get just 1GB RAM wasted for nothing.

If you put 5-piece WDL files to RAM disk it is 380 MB, not 1GB.
These files can be fast accessed even if swaped out by OS.
There has up to now not shown any data for your claim.
Kind regards
Bernhard

[bob]

There will be no actual queries to RAMDisk controller or regular HDD controller. You get just 1GB RAM wasted for nothing.

If you put 5-piece WDL files to RAM disk it is 380 MB, not 1GB.
These files can be fast accessed even if swaped out by OS.
There has up to now not shown any data for your claim.
Kind regards
Bernhard

It is simply an obvious observation. Data will sit in memory TWICE. Once in the RAM drive, once in normal filesystem cache. It is certainly true. How much the lost memory hurts is a different issue.

[yurikvelo]

There will be no actual queries to RAMDisk controller or regular HDD controller. You get just 1GB RAM wasted for nothing.

If you put 5-piece WDL files to RAM disk it is 380 MB, not 1GB.
These files can be fast accessed even if swaped out by OS.
There has up to now not shown any data for your claim.
Kind regards
Bernhard

To clarify my claim is:

- If you have no TB6 queries (disabled, absent) you cannot detect whether or not you have RAMDisk or not. You just wasted 1GB RAM, but NPS drop is not detectable, since files are getting buffered into RAM, no matter if they are stored on SSD, HDD or RAMDisk. No gain, no penalty. Only wasted RAM for RAMDisk.

- If you do have Syzygy-6 probing, than you have very strong RAM deficit.
If you waste 1Gb for RAMDisk - deficit raise.
Exact NPS drop depends on a lot of factors - to waste 1Gb of 2 GB free is not the same as waste 1 Gb of 32GB free (50% loss of OS filesystem buffers vs 3% loss).