Syzygy on RAM Drive

[Laskos]

On a 1GB RAMDisk I put the 5-men Syzygy (both WDL and DTZ), and let the top engines play with and without Syzygy at 1' + 1'', on one core. Balanced endgame positions up to 12 men (had to suffer many draws with these balanced endgame positions).

Code: Select all

 1.  SF Syzygy RAM            	530.5/1000	148-87-765
 2.  SF NO Syzygy             	469.5/1000	87-148-765

Code: Select all

 1.  K7a Syzygy RAM           	514.0/1000	121-93-786
 2.  K7a NO Syzygy            	486.0/1000	93-121-786

Code: Select all

 1.  H4 Syzygy RAM            	520.0/1000	142-102-756
 2.  H4 NO Syzygy             	480.0/1000	102-142-756

Within error margins, but it seems that SF (Ronald's 22.05 build) gains more from Syzygy than the other two.

[Laskos]

On a 1GB RAMDisk I put the 5-men Syzygy (both WDL and DTZ), and let the top engines play with and without Syzygy at 1' + 1'', on one core. Balanced endgame positions up to 12 men (had to suffer many draws with these balanced endgame positions).

Code: Select all

 1.  SF Syzygy RAM            	530.5/1000	148-87-765
 2.  SF NO Syzygy             	469.5/1000	87-148-765

Code: Select all

 1.  K7a Syzygy RAM           	514.0/1000	121-93-786
 2.  K7a NO Syzygy            	486.0/1000	93-121-786

Code: Select all

 1.  H4 Syzygy RAM            	520.0/1000	142-102-756
 2.  H4 NO Syzygy             	480.0/1000	102-142-756

Within error margins, but it seems that SF (Ronald's 22.05 build) gains more from Syzygy than the other two.

1GB RAMDisk with the 5-men Syzygy, positions are 6 men (4+2 and 3+3) _hard_ TB wins. I get clearer and more significant statistically results. Games 1' + 1'' on one core:

Code: Select all

 1.  SF Syzygy RAM            	608.5/1000	426-209-365
 2.  SF NO Syzygy             	391.5/1000	209-426-365

Code: Select all

 1.  K7a Syzygy RAM           	599.0/1000	431-233-336
 2.  K7a NO Syzygy            	401.0/1000	233-431-336

Code: Select all

 1.  H4 Syzygy RAM            	568.5/1000	339-202-459
 2.  H4 NO Syzygy             	431.5/1000	202-339-459

On an 8GB RAMDisk I put the 5-men Nalimovs, and I get similar improvement of H4 as with Syzygy:

Code: Select all

Nalimov Cache 128MB
 1.  H4 Nalimov RAM              569.0/1000	379-241-380
 2.  H4 NO TB                 	431.0/1000	241-379-380

To note that on these 6 men hard win positions, 5-men on RAMDisk can improve the play by some 70 Elo points.

[AdminX]

Hello Kai,

What OS and software did you use to create the RAMDisk?

[Laskos]

Hello Kai,

What OS and software did you use to create the RAMDisk?

Hi Ted,
Win 7, Dataram RAMDisk soft, but there are plenty.

[AdminX]

Hello Kai,

What OS and software did you use to create the RAMDisk?

Hi Ted,
Win 7, Dataram RAMDisk soft, but there are plenty.

Thanks Kai,

I am going to test out ImDisk

http://www.tekrevue.com/tip/create-10-g ... k-windows/

http://reboot.pro/files/download/284-imdisk-toolkit/

[syzygy]

In general I can only advise against the use of RAM disks for storing TB files.

If you don't use a RAM disk, upon access the relevant parts of the TB files will be cached into RAM anyway in an area called the "page cache". Once certain data is in the page cache it will stay there even between engine restarts, unless the OS needs the RAM for something else.

If you use a RAM disk, there is less RAM left for the page cache. The OS will have to copy TB data from the RAM disk to the page cache, so everything uses RAM twice.

A very intelligent RAM disk implementation (forget about this on Windows, is my guess) might avoid this duplication and actually store files in the page cache. But even then it is likely that memory would be more efficiently used if you leave it to the OS to decide which TB files are needed in RAM.

[BBauer]

While you may be right in general it will not hurt too much if you use a 480 MB RAM Disk for 5-piece WDL. Especially if you have a slow hard disk.
When using 6-piece WDL and little memory bases may become swaped out and have to be reloaded. That may cost time.
However, if you have the bases on SSD and enough memory the situation may be different.

Kind regards
Bernhard

[Laskos]

In general I can only advise against the use of RAM disks for storing TB files.

If you don't use a RAM disk, upon access the relevant parts of the TB files will be cached into RAM anyway in an area called the "page cache". Once certain data is in the page cache it will stay there even between engine restarts, unless the OS needs the RAM for something else.

If you use a RAM disk, there is less RAM left for the page cache. The OS will have to copy TB data from the RAM disk to the page cache, so everything uses RAM twice.

A very intelligent RAM disk implementation (forget about this on Windows, is my guess) might avoid this duplication and actually store files in the page cache. But even then it is likely that memory would be more efficiently used if you leave it to the OS to decide which TB files are needed in RAM.

With 16GB RAM and 0.5GB WDL for 5 men, I don´t think that´s a problem. Having a slow HDD and no SSD, I tried to optimize the 5 men Syzygy for heavy use, for example on 6 men won, hard positions. About cached chunks, for 6 men, isn´t it a bit unfair that Syzygy can load GBs of that freely, while Nalimovs use limited cache?
By the way, do you think Nalimovs on RAMDisk (with large cache) should perform better than Nalimovs on HDD? It´s for the first time, now, on RAMDisk, that I see a clear benefit of Nalimovs.

[shrapnel]

RAM disks not worth it...SSDs are a much better option.

[syzygy]

In general I can only advise against the use of RAM disks for storing TB files.

If you don't use a RAM disk, upon access the relevant parts of the TB files will be cached into RAM anyway in an area called the "page cache". Once certain data is in the page cache it will stay there even between engine restarts, unless the OS needs the RAM for something else.

If you use a RAM disk, there is less RAM left for the page cache. The OS will have to copy TB data from the RAM disk to the page cache, so everything uses RAM twice.

A very intelligent RAM disk implementation (forget about this on Windows, is my guess) might avoid this duplication and actually store files in the page cache. But even then it is likely that memory would be more efficiently used if you leave it to the OS to decide which TB files are needed in RAM.

With 16GB RAM and 0.5GB WDL for 5 men, I don´t think that´s a problem. Having a slow HDD and no SSD, I tried to optimize the 5 men Syzygy for heavy use, for example on 6 men won, hard positions.

That's why I started with "in general". In your specific case there is no problem and it may help to some extent to get a more stable testing environment.

Btw, on Linux the 5-piece tables can be preloaded into RAM with a simple:

Code: Select all

$ cat /path/to/tbs/??????.rtbw >/dev/null

or if vmtouch is installed:

Code: Select all

$ vmtouch -t /path/to/tbs/??????.rtbw

The vmtouch command has other useful applications, for example to find out how much of each table is cached in RAM:

Code: Select all

$ vmtouch -m 10G -v /path/to/tbs/*.rtbw

About cached chunks, for 6 men, isn´t it a bit unfair that Syzygy can load GBs of that freely, while Nalimovs use limited cache?

I'm not in any way in competition with Eugene Nalimov. The concept of "fairness" seems to be foreign to the present topic.

By the way, do you think Nalimovs on RAMDisk (with large cache) should perform better than Nalimovs on HDD? It´s for the first time, now, on RAMDisk, that I see a clear benefit of Nalimovs.

If you have (literally) more than enough RAM then sure, but in general no. Nalimov files are also cached in the page cache (and then separately decompressed in the engine's TB cache). Loading them on RAM disk will help during the first accesses (it will not take much time to copy some relevant parts from RAM disk into the page cache) but not in the long run (RAM available for the page cache will be much smaller).