Help - can't clear bug with tt-move.

[Chan Rasjid]

So what Oliver Uwira hinted was right - "...protocol".

I have a startup TT size of 32MB and have polyglot hash size set to 64MB in order to test the setoption feature. In reallocating the size of the TT, the 3MB memory of the pawn hash table was wrongly deallocated without the pointer to the pawn's hash table reset to NULL; so pawn hashing was still enabled and it corrupts the memory that now belongs to the larger TT.

So the bug is finally resolved. Thanks to those who responded.

Best Regards,
Rasjid.

[hgm]

...
Another idea. What about the very end of the table? When I did my new "hash path" idea, I used buckets of size 8. But cleverly forgot that if you probe into a table of size 2^N entries, you need 2^N+7 entries to take care of the hash probe to that last entry + the next 7. Did you get the malloc() size corrected for the extra bucket entries? Or if you are going in a negative direction, can you every probe at entry 0 plus the 3 before it? Either way leads to trouble, obviously...

I always remember this somehow

I malloc(size +128) which covers 8 extra entries.

I loop through the bucket backwards but tests pointer-to-entry >= 1st-entry. So should be ok.

Rasjid.

Note that it is very detrimental to performance just looping through 8 consecutive entries, as in 7 out of 8 cases these would span two different cache lines. Much better is to make sure the allocation aligns buckets with cache lines, and loop through a single bucket circuarly. In that case the overflow problem at the end would not exist. (You might still have to allocate the extra 7 entries to create room to shift the allocation to get the desired alignment, though.)

Buckets of a single cache line (even if it is only 4 entries) usually also outperform buckets of multiple cache lines. The problem with multiple cache lines is that in most modern CPUs is not only that you need a very costly extra memory access, but also that it triggers the cache pre-fetch logic. Doing two cache-line fills from consecutive (or even close) addresses makes the CPU speculate that you will be sequentially stepping through memory, and will trigger an automatic third memory access to fetch the next cache line.

Note that Bob's cluster is built from rather obsolete CPUs (even Pentium Pro), which might not have such pre-fetchers, so that he cannot see this detrimental effect in his tests...

[bob]

...
Another idea. What about the very end of the table? When I did my new "hash path" idea, I used buckets of size 8. But cleverly forgot that if you probe into a table of size 2^N entries, you need 2^N+7 entries to take care of the hash probe to that last entry + the next 7. Did you get the malloc() size corrected for the extra bucket entries? Or if you are going in a negative direction, can you every probe at entry 0 plus the 3 before it? Either way leads to trouble, obviously...

I always remember this somehow

I malloc(size +128) which covers 8 extra entries.

I loop through the bucket backwards but tests pointer-to-entry >= 1st-entry. So should be ok.

Rasjid.

Note that it is very detrimental to performance just looping through 8 consecutive entries, as in 7 out of 8 cases these would span two different cache lines. Much better is to make sure the allocation aligns buckets with cache lines, and loop through a single bucket circuarly. In that case the overflow problem at the end would not exist. (You might still have to allocate the extra 7 entries to create room to shift the allocation to get the desired alignment, though.)

For me, this is not important. It is only done on PV nodes to save the path for that TT entry. I currently hash to a "bucket" already, which solves this issue, although the performance issue is effectively nil due to hardly any stores in the path table happening...

[hgm]

Oh, sure, for the path table performance is not an issue. But I think Chan is talking about main hash here. And he might even probe in QS, which makes it even more critical.

[Chan Rasjid]

Oh, sure, for the path table performance is not an issue. But I think Chan is talking about main hash here. And he might even probe in QS, which makes it even more critical.

I was replying to Bob about overwriting memory at the end or start of buckets. My bucket should be good - just 4 x 16 bytes aligned at 128 bytes boundary; just allocated + 128 bytes instead of + 64 bytes.

I picked up many chess programming tips here, except there is no man pages here on :-
"Howto - The Ippolit Chess Program".

Rasjid.