Hi all, I have generated the 5 man tablebases for Nalimov and every single one has a different checksum to those found online.
Md5 checksums from here: http://tablebase.sesse.net/3-4-5/ and one other place (I can't remember).
I have tried several builds of Tbgen, this was the one I was using first: http://rybkaforum.net/cgi-bin/rybkaforu ... ?tid=24196 (Third post linked to by Charles). I generated them using tbgen -m 4096 -p kpkp kppk, for example. I then compress with datacomp e filename filename filename, etc
I have also used the 32 bit tbgen here: http://www.open-aurec.com/wbforum/viewtopic.php?t=51842 and used the Wayback machine for the link http://web.archive.org/web/200608060734 ... /TBGEN.ZIP Also, I have tried this one too: http://talkchess.com/forum3/viewtopic.php?f=2&t=29745
All the same result. They all fail Md5 checking using fsum. They do seem to work in SCID vs PC though.
Any help please?
Nalimov errors
Moderators: hgm, Rebel, chrisw
-
- Posts: 6
- Joined: Fri Apr 09, 2021 10:59 am
- Full name: Jonathan Colledge
-
- Posts: 12540
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Nalimov errors
Compare some of the checksums you generate with your files with checksums generated from Nalimov files you download.
I guess they will be different.
Probably a different checksum algorithm?
I guess they will be different.
Probably a different checksum algorithm?
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 6
- Joined: Fri Apr 09, 2021 10:59 am
- Full name: Jonathan Colledge
Re: Nalimov errors
Thanks, this is what I get :
C:\Users\Jonathan>CertUtil -hashfile "F:\Chess Tablebases\Generate Nalimov\datacomp-tbgen\Done\kbbk.nbb.emd" MD5
MD5 hash of F:\Chess Tablebases\Generate Nalimov\datacomp-tbgen\Done\kbbk.nbb.emd:
43b6d89227a40a1beaf579751bee3cb7
CertUtil: -hashfile command completed successfully.
C:\Users\Jonathan>CertUtil -hashfile "F:\Chess Tablebases\Nalimov 3-4-5 Downloaded\3-4-5\kbbk.nbb.emd" MD5
MD5 hash of F:\Chess Tablebases\Nalimov 3-4-5 Downloaded\3-4-5\kbbk.nbb.emd:
6d3bee7a798a4829fa97f93e52bc1f56
CertUtil: -hashfile command completed successfully.
I just wonder if different builds of Datacomp and tbgen will generate different, but working files.
C:\Users\Jonathan>CertUtil -hashfile "F:\Chess Tablebases\Generate Nalimov\datacomp-tbgen\Done\kbbk.nbb.emd" MD5
MD5 hash of F:\Chess Tablebases\Generate Nalimov\datacomp-tbgen\Done\kbbk.nbb.emd:
43b6d89227a40a1beaf579751bee3cb7
CertUtil: -hashfile command completed successfully.
C:\Users\Jonathan>CertUtil -hashfile "F:\Chess Tablebases\Nalimov 3-4-5 Downloaded\3-4-5\kbbk.nbb.emd" MD5
MD5 hash of F:\Chess Tablebases\Nalimov 3-4-5 Downloaded\3-4-5\kbbk.nbb.emd:
6d3bee7a798a4829fa97f93e52bc1f56
CertUtil: -hashfile command completed successfully.
I just wonder if different builds of Datacomp and tbgen will generate different, but working files.
-
- Posts: 12540
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Nalimov errors
It does seem worrying that they MD5 hashes for the same file are different.
I suspect it is default sizes for structs, perhaps a different packing rule.
Modern compilers tend to pack on wider block sizes.
Are the files that you built larger?
I suspect it is default sizes for structs, perhaps a different packing rule.
Modern compilers tend to pack on wider block sizes.
Are the files that you built larger?
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 6
- Joined: Fri Apr 09, 2021 10:59 am
- Full name: Jonathan Colledge
Re: Nalimov errors
Great, thank you! Yes mine are larger. When I decompress the downloaded files, they are the same size as the generated files. I haven't checked the checksums, yet, but that looks like the answer.
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: Nalimov errors
It certainly won't have anything to do with modern compilers "tending to pack on wider block sizes".jonathancolledge wrote: ↑Sun Apr 18, 2021 11:38 pm Great, thank you! Yes mine are larger. When I decompress the downloaded files, they are the same size as the generated files. I haven't checked the checksums, yet, but that looks like the answer.
However, the answer might be that you have compressed the files using datacomp.exe with a different block size.
-
- Posts: 12540
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Nalimov errors
So there is a pragma pack in the Nalimov code to ensure packing on a stated boundary so that the change from 32 to 64 bit addressing (or other packing rule such as one byte boundaries for space saving) had no effect?syzygy wrote: ↑Sat May 01, 2021 9:13 pmIt certainly won't have anything to do with modern compilers "tending to pack on wider block sizes".jonathancolledge wrote: ↑Sun Apr 18, 2021 11:38 pm Great, thank you! Yes mine are larger. When I decompress the downloaded files, they are the same size as the generated files. I haven't checked the checksums, yet, but that looks like the answer.
However, the answer might be that you have compressed the files using datacomp.exe with a different block size.
I did not see it in my copy of the code.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 12540
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Nalimov errors
https://docs.microsoft.com/en-us/cpp/pr ... w=msvc-160Dann Corbit wrote: ↑Sat May 01, 2021 10:49 pmSo there is a pragma pack in the Nalimov code to ensure packing on a stated boundary so that the change from 32 to 64 bit addressing (or other packing rule such as one byte boundaries for space saving) had no effect?syzygy wrote: ↑Sat May 01, 2021 9:13 pmIt certainly won't have anything to do with modern compilers "tending to pack on wider block sizes".jonathancolledge wrote: ↑Sun Apr 18, 2021 11:38 pm Great, thank you! Yes mine are larger. When I decompress the downloaded files, they are the same size as the generated files. I haven't checked the checksums, yet, but that looks like the answer.
However, the answer might be that you have compressed the files using datacomp.exe with a different block size.
I did not see it in my copy of the code.
The above link shows that even today the boundary is 8 byres for 32 bit and 16 for 64 bit compiles/
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: Nalimov errors
Without ever having looked at the source code, I am confident that it doesn't do byte-by-byte writes of structs to disks. It would be really bad programming, and Nalimov is not a bad programmer. (And if it did, and the memory layout of a struct would change, programs trying to use the TB file would just crash.)Dann Corbit wrote: ↑Sat May 01, 2021 10:49 pm So there is a pragma pack in the Nalimov code to ensure packing on a stated boundary so that the change from 32 to 64 bit addressing (or other packing rule such as one byte boundaries for space saving) had no effect?
I did not see it in my copy of the code.
-
- Posts: 12540
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Nalimov errors
Well, nalimov has only one compression algorithm, so I think the most likely scenario is that the OP is comparing uncompressed TB files to compressed TB files.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.