Tablebase access using a Solid State Disk

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 6:43 pm

Tablebase access using a Solid State Disk

Post by sje » Thu Oct 16, 2014 1:52 am

Tablebase access using a Solid State Disk

Today I got my first SSD (Solid State Disk). It is an external unit with 240 GiB capacity and it connects using a USB3 interface. The data transfer rate ceiling is well over 400 MiB/second, limited mostly by the USB3 ceiling. The cost was about US$1/GiB.

Important to the topic of tablebase access, the SSD I/O operation rate is about 100,000 operations per second, meaning that my program can probe tablebase files far more rapidly when they're on the SSD vs on a typical SATA hard drive. The difference? The SSD operation rate is faster by a factor of about 60, although this will vary much according to particulars.

An SSD connected via a PCI-e interface would be slightly more than twice as fast as with a USB3 connection, but this would limit which machines I could use.

I should probably change Symbolic to adapt its tablebase probe strategy based on the I/O operation rate of the storage used by the tablebase files. No rush here, as Symbolic won't need it until it returns to server play sometime next year.

wgarvin
Posts: 838
Joined: Thu Jul 05, 2007 3:03 pm
Location: British Columbia, Canada

Re: Tablebase access using a Solid State Disk

Post by wgarvin » Thu Oct 16, 2014 9:44 pm

What's really nice about SSDs is that their "seek time" is pretty much zero. For some use cases that makes a really huge difference. If you get a quality one, the raw throughput will also be excellent.

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 6:43 pm

Re: Tablebase access using a Solid State Disk

Post by sje » Sat Oct 18, 2014 5:50 am

Yes, the reduction of overall latency is the Big Win when using an SSD for tablebase storage.

My current tablebase format needs only a single byte to convey the exact mate/lose distance. The future format will need to read a maximum of three bytes which could possibly span a block boundary.

In the first case, exactly one block is read; for the second case one or two blocks is read. Each logical block on an HFS+ (Mac OS/X default) filesystem is 4 KiB long, so most of time spent for a tablebase element transfer is wasted regardless of the storage medium. That's what limits the probe rate to about 100 KHz for my USB3/SSD setup.

An SSD connected to a 6 Mbps SATA3 interface will run about a third faster. An SSD connected via a PCI-e interface might be twice as fast with a non RAID configuration. A RAID could run even faster, but would have the same latency.

Beyond all that would be the use of a RAM disk, and that's actually feasible for the three and four man tablebases.

--------

I had also considered moving the opening book to an SSD. But there's little to be gained here because Symbolic hits the book file only once; this is done at program boot time to load the book into RAM.

Post Reply