Weird disc behaviour

[Modern Times]

C code and Windows. Memory leaking, throttling or Windows doing funny things. I would bet it will run flawlessly on Linux and C++ streams.

Of course. It is always Windows at fault, and Linux is always the answer.

[hgm]

Note that an SSD ('Solid-State Drive') is not a disc.

[Modern Times]

Note that an SSD ('Solid-State Drive') is not a disc.

Indeed not but I assumed he likely was running an SSD.

[Eelco de Groot]

Such as the "copy" instruction probably does, copying files goes with 440 million bytes per second.

440MB/s sounds like an SATA SSD - an HDD probably wouldn't reach that, and the SATA interface bandwidth tops out at 600MB/s. An NVMe SSD should be a lot higher with sequential access. As for why, I think it's the SLC cache of the SSD. 2 bio positions with 40 bytes is around 80GB. Typical PCs have between 16 and 64GB of RAM, so that wouldn't even fit unless you have a PC with unusually much RAM. 80GB is on the order of maginitude I'd expect for the SLC cache of a 1TB drive, though that size often also depends on how full the drive is.

Sounds like a good analysis. Ed could upgrade but then he would have to install everything again I think. If you have to do this every day it would be worth it for sure. If Ed's HP does not have such a NVMe (PCIe instead of SATA I read) typical upgrade prices Queensystems :

Meerprijs voor: 480GB SSD

- HP Z-Drive Turbo 512GB M.2 Sata € 59,00
- HP Z-Drive Turbo 256GB M.2 NVMe € 69,00 > 6 x sneller als normale SSD
- HP Z-Drive Turbo 512GB M.2 NVMe € 99,00 > 6 x sneller als normale SSD
- HP Z-Drive Turbo 1TB M.2 NVMe € 179,00 > 6 x sneller als normale SSD
- HP Z-Drive Turbo 2TB M.2 NVMe € 289,00 > 6 x sneller als normale SSD

But from what you say then also enough RAM

[petero2]

You both make sense.

It might help to explicitly buffer large chunks of data, and write those out without any reading in between.

Such as the "copy" instruction probably does, copying files goes with 440 million bytes per second.

This can be implemented by simply making the built-in FILE buffers larger, like this:

Code: Select all

        fp1 = fopen(bin_one, "rb");
        fp2 = fopen(bin_two, "rb");

        int bufSize = 1000000;
        setvbuf(fp1, malloc(bufSize), _IOFBF, bufSize);
        setvbuf(fp2, malloc(bufSize), _IOFBF, bufSize);

next:   x=fread(&k1,8,1,fp1); if (x<1) goto done;

The buffers will never be freed though, so a real implementation would have to keep track of the allocated memory and deallocate it when no longer needed.

[Ras]

Ed could upgrade but then he would have to install everything again I think.

When I changed my SATA SSD for an NVME drive, I used the same size and just cloned the whole installation with Clonezilla. Clonezilla was even smart enough to realise that the fstab entries would be off because the device names for SATA and NVMe partitions are different, so it offered to adjust that. Then again, I'm on Linux, and I'm not sure how easy Windows would take that.

[Rebel]

This can be implemented by simply making the built-in FILE buffers larger, like this:

Code: Select all

        fp1 = fopen(bin_one, "rb");
        fp2 = fopen(bin_two, "rb");

        int bufSize = 1000000;
        setvbuf(fp1, malloc(bufSize), _IOFBF, bufSize);
        setvbuf(fp2, malloc(bufSize), _IOFBF, bufSize);

next:   x=fread(&k1,8,1,fp1); if (x<1) goto done;

The buffers will never be freed though, so a real implementation would have to keep track of the allocated memory and deallocate it when no longer needed.

Will try that.