7-men Syzygy attempt

[Sesse]

Even more importantly, NVMe disks (and basically all other storage, save for the DIMM form factors of Optane, which hardly exist yet) work by transferring data to RAM and then having the CPU read it from there. So it would basically be impossible to have storage faster than RAM, since it needs to go through RAM.

[Milos]

Even more importantly, NVMe disks (and basically all other storage, save for the DIMM form factors of Optane, which hardly exist yet) work by transferring data to RAM and then having the CPU read it from there. So it would basically be impossible to have storage faster than RAM, since it needs to go through RAM.

Not if NVMe is accessed through polling or NVMe is PCIe based.

[Sesse]

NVMe is PCIe-based by definition. As for polling… I hope you're not accessing your NVMe disk over PIO, that would be awfully slow :-)

[Milos]

NVMe is PCIe-based by definition. As for polling… I hope you're not accessing your NVMe disk over PIO, that would be awfully slow

Also, if you place communication queues on NVMe device and your device is faster than main memory you can be faster than main memory.

[Sesse]

I ported Roland's Cfish patches to my Stockfish branch:

https://git.sesse.net/?p=stockfish;a=co ... fbd53f4c17
https://git.sesse.net/?p=stockfish;a=co ... 82a3763797

They're only lightly tested, so beware.

[noobpwnftw]

After receiving Ronald's NUMA-aware buff, all 4v3 pawnless tables are expected to be built in about 10-15 days.

There are more about 50% more pawnful ones than pawnless ones in number, and they are usually 4x slower to build. So it will still be months away from having the full set.

[Nordlandia]

After receiving Ronald's NUMA-aware buff, all 4v3 pawnless tables are expected to be built in about 10-15 days.

There are more about 50% more pawnful ones than pawnless ones in number, and they are usually 4x slower to build. So it will still be months away from having the full set.

Shouldn't NUMA offset some of this slowness compared to running without NUMA?

Something equilavent to running a overclocked system vs stock speed.

[noobpwnftw]

After receiving Ronald's NUMA-aware buff, all 4v3 pawnless tables are expected to be built in about 10-15 days.

There are more about 50% more pawnful ones than pawnless ones in number, and they are usually 4x slower to build. So it will still be months away from having the full set.

Shouldn't NUMA offset some of this slowness compared to running without NUMA?

Something equilavent to running a overclocked system vs stock speed.

This estimation is based on performance using NUMA-aware generator.

Pawnful building is on another machine that does not suffer slowdown without it, probably due to better hardware configuration.

[Sesse]

After receiving Ronald's NUMA-aware buff, all 4v3 pawnless tables are expected to be built in about 10-15 days.

There are more about 50% more pawnful ones than pawnless ones in number, and they are usually 4x slower to build. So it will still be months away from having the full set.

This is actually amazingly fast :-) Just a few weeks ago, the general sentiment was “it will take a year or more to build all tables, it will probably be closer to 100 TB, nobody's doing it or interested in it”… and now we're talking something like 30–35 TB, full pawnless set two weeks out and the rest mere months away?

Something tells me I'll need to start thinking about disks…

[Rein Halbersma]

After receiving Ronald's NUMA-aware buff, all 4v3 pawnless tables are expected to be built in about 10-15 days.

There are more about 50% more pawnful ones than pawnless ones in number, and they are usually 4x slower to build. So it will still be months away from having the full set.

This is actually amazingly fast Just a few weeks ago, the general sentiment was “it will take a year or more to build all tables, it will probably be closer to 100 TB, nobody's doing it or interested in it”… and now we're talking something like 30–35 TB, full pawnless set two weeks out and the rest mere months away?

Something tells me I'll need to start thinking about disks…

What I like to know is how are people able to afford the enormous resources (~1.5 Tb RAM and dozens of cores) for such hobby computations? Or are they using idle company resources somehow?