Nalimov errors

[jonathancolledge]

Hi all, I have generated the 5 man tablebases for Nalimov and every single one has a different checksum to those found online.

Md5 checksums from here: http://tablebase.sesse.net/3-4-5/ and one other place (I can't remember).

I have tried several builds of Tbgen, this was the one I was using first: http://rybkaforum.net/cgi-bin/rybkaforu ... ?tid=24196 (Third post linked to by Charles). I generated them using tbgen -m 4096 -p kpkp kppk, for example. I then compress with datacomp e filename filename filename, etc

I have also used the 32 bit tbgen here: http://www.open-aurec.com/wbforum/viewtopic.php?t=51842 and used the Wayback machine for the link http://web.archive.org/web/200608060734 ... /TBGEN.ZIP Also, I have tried this one too: http://talkchess.com/forum3/viewtopic.php?f=2&t=29745

All the same result. They all fail Md5 checking using fsum. They do seem to work in SCID vs PC though.

Any help please?

[Dann Corbit]

Compare some of the checksums you generate with your files with checksums generated from Nalimov files you download.
I guess they will be different.
Probably a different checksum algorithm?

[jonathancolledge]

Thanks, this is what I get :

C:\Users\Jonathan>CertUtil -hashfile "F:\Chess Tablebases\Generate Nalimov\datacomp-tbgen\Done\kbbk.nbb.emd" MD5
MD5 hash of F:\Chess Tablebases\Generate Nalimov\datacomp-tbgen\Done\kbbk.nbb.emd:
43b6d89227a40a1beaf579751bee3cb7
CertUtil: -hashfile command completed successfully.

C:\Users\Jonathan>CertUtil -hashfile "F:\Chess Tablebases\Nalimov 3-4-5 Downloaded\3-4-5\kbbk.nbb.emd" MD5
MD5 hash of F:\Chess Tablebases\Nalimov 3-4-5 Downloaded\3-4-5\kbbk.nbb.emd:
6d3bee7a798a4829fa97f93e52bc1f56
CertUtil: -hashfile command completed successfully.

I just wonder if different builds of Datacomp and tbgen will generate different, but working files.

[Dann Corbit]

It does seem worrying that they MD5 hashes for the same file are different.
I suspect it is default sizes for structs, perhaps a different packing rule.
Modern compilers tend to pack on wider block sizes.
Are the files that you built larger?

[jonathancolledge]

Great, thank you! Yes mine are larger. When I decompress the downloaded files, they are the same size as the generated files. I haven't checked the checksums, yet, but that looks like the answer.

[syzygy]

Great, thank you! Yes mine are larger. When I decompress the downloaded files, they are the same size as the generated files. I haven't checked the checksums, yet, but that looks like the answer.

It certainly won't have anything to do with modern compilers "tending to pack on wider block sizes".

However, the answer might be that you have compressed the files using datacomp.exe with a different block size.

[Dann Corbit]

Great, thank you! Yes mine are larger. When I decompress the downloaded files, they are the same size as the generated files. I haven't checked the checksums, yet, but that looks like the answer.

It certainly won't have anything to do with modern compilers "tending to pack on wider block sizes".

However, the answer might be that you have compressed the files using datacomp.exe with a different block size.

So there is a pragma pack in the Nalimov code to ensure packing on a stated boundary so that the change from 32 to 64 bit addressing (or other packing rule such as one byte boundaries for space saving) had no effect?
I did not see it in my copy of the code.

[Dann Corbit]

Great, thank you! Yes mine are larger. When I decompress the downloaded files, they are the same size as the generated files. I haven't checked the checksums, yet, but that looks like the answer.

It certainly won't have anything to do with modern compilers "tending to pack on wider block sizes".

However, the answer might be that you have compressed the files using datacomp.exe with a different block size.

So there is a pragma pack in the Nalimov code to ensure packing on a stated boundary so that the change from 32 to 64 bit addressing (or other packing rule such as one byte boundaries for space saving) had no effect?
I did not see it in my copy of the code.

https://docs.microsoft.com/en-us/cpp/pr ... w=msvc-160
The above link shows that even today the boundary is 8 byres for 32 bit and 16 for 64 bit compiles/

[syzygy]

So there is a pragma pack in the Nalimov code to ensure packing on a stated boundary so that the change from 32 to 64 bit addressing (or other packing rule such as one byte boundaries for space saving) had no effect?
I did not see it in my copy of the code.

Without ever having looked at the source code, I am confident that it doesn't do byte-by-byte writes of structs to disks. It would be really bad programming, and Nalimov is not a bad programmer. (And if it did, and the memory layout of a struct would change, programs trying to use the TB file would just crash.)

[Dann Corbit]

Well, nalimov has only one compression algorithm, so I think the most likely scenario is that the OP is comparing uncompressed TB files to compressed TB files.