Probing tablebases through USB 3.0

[Nordlandia]

How worse off is tablebases through USB 3.0 interface compared through SATAIII?

Assuming both parties consult 5-men syzygy during match.

Do latency affect performance much?

[syzygy]

At longer time controls latency should not be a major issue when accessing 5-piece TBs since the relevant data will quickly be cached to RAM.

At shorter time controls the initial slower access will have more of an impact. (But I don't know how USB 3.0 compares to SATA in terms of latency.)

To eliminate latency, precache them with "type *.rtbw >NUL" from a DOS prompt before starting the game or games (no need to repeat between games).

[Nordlandia]

Thanks! after pre-cashing now i can i close cmd and run games without worrying about latency.

Ponder matches is optimal with identical hardware.

[Nordlandia]

Just one technical question - do precaching 5-men nullify the disadvantage of HDD compared to SSD?

For example it may sound plausible to precache tablebases in RAM on older computers to compensate for slower mechanical drives.

[syzygy]

Yes, if the machine has enough RAM it will nullify the difference between WDL tables on SSD and HDD.

For 6-piece tables you would need a lot of RAM though (and some patience while precaching).

Another trick: if the use of 6-piece tables in long searches slows down the PC too much, try disabling the swap file. Without swap file, Windows cannot page to disk the programs you are using and is forced instead to release some of the RAM used for caching TB data. (So TB access will slow down a bit when memory is low, but the computer remains responsive.)

If 6-piece WDL tables are on HDD, probe depth should be increased to reduce the number of probes. Stockfish will continue to probe 5-piece tables at all depths (the idea being that they are cached in RAM anyway). I believe Komodo does the same if "smart probing"' is enabled.

[Nordlandia]

Regarding 6-men. How much of an penalty is it to probe 6-men until RAM get full if paging file is on SSD?

I have plenty of space available on 850 EVO, so the question is if it's good idea to set paging file close to WDL size.

Hash: 16 GB DDR4

Primary storage: Samsung 960 EVO (tablebases stored here)

Secondary storage: Samsung 850 EVO

Once RAM get full, Samsung 850 EVO kicks in as secondary memory. The speed is about 500 MB/s

[syzygy]

Regarding 6-men. How much of an penalty is it to probe 6-men until RAM get full if paging file is on SSD?

I have plenty of space available on 850 EVO, so the question is if it's good idea to set paging file close to WDL size.

Windows will never page TB data from RAM to the paging file, since all TB data is already backed by the TB files. So if Windows decides to free up RAM currently used for caching TB data, it will simply release that memory. When it needs the same data again, it will reread it from the TB file.

So there is no reason to set the size of the paging file close to the size of the WDL tables.

The reason why things can get slow with heavy 6-piece probing is that Windows starts paging out your other programs. This can be prevented by disabling the page file. Without page file, Windows cannot page out the other programs and is forced to do what we want it to do: release the cached TB data.

It would be nice if there were a way to tell Windows that it should always prefer to release cached TB data over paging out active programs, but I am not aware of one.

What can help if SF + 6-piece TBs starts slowing down the PC is to set SyzygyPath to something non-existent and then change it back. This will unmap all mapped TB files. However, SyzygyPath cannot be changed while the engine is searching.

I might at some point add an option to Cfish to periodically unmap all mapped TB files.

[Nordlandia]

How do Lock pages in memory (large pages) react if paging file is disabled?

So my understanding is that RAM will eventually be filled up with cached TB, the difference is that at the same time windows release unneeded cached TBs and might probe them later on if needed. Is this correct?