Clear or not to clear the hash tables

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

Cardoso
Posts: 362
Joined: Thu Mar 16, 2006 7:39 pm
Location: Portugal
Full name: Alvaro Cardoso

Clear or not to clear the hash tables

Post by Cardoso »

My engine sometimes makes a really bad move.
But when later I analise that position with the hash tables cleared it immediately sees the right move.
And that bad move goes to the end of the root move list.
So I'm pretty sure the entries in the hash table from the previous search are causing the problem.

When time is out I set a stopsearch flag, and at the top of the search if this flag is set I return zero.
Also when the stopsearch flag is set I store nothing in the hash tables.

Is this ok? Do you clear your hash tables before searching?
What value do you return when time is out?

Have you seen this king of behaviour in your engine?

Thanks in advance,
Alvaro
Robert Pope
Posts: 558
Joined: Sat Mar 25, 2006 8:27 pm

Re: Clear or not to clear the hash tables

Post by Robert Pope »

I thought of two things:

1. Are you validating the full hash key against the full key in the hashed position? You will get lots of false matches if you don't validate.

2. When Beaches played terrible moves, it was because of terminating in the middle of a search. The scores for the last root move searched are useless if you stopped the search early, so you have to either junk the last move searched, or only break out early at the root level. Possibly a cached root move is causing your first iteration to be for a high ply that takes too long, and then your program is forced to cancel early?
PK
Posts: 893
Joined: Mon Jan 15, 2007 11:23 am
Location: Warsza

Re: Clear or not to clear the hash tables

Post by PK »

Are You 100% sure that the move form hash / previous iteration is sorted first at the root? I remember a funny bug in an early version of Hopeless, when running it on the fast hardware for the first time: in a pawn endgame a history counter for some king move grew so high that it exceeded the value of promotion, so my program commited a spectacular suicide not searching the 2nd move.

Another thing that can be spoiled (i did it, too :oops: ) is to write into hash table a score from a timed-out search. Worse still, if such scores are not blocked from propagating downwards when the serach is being shut up.

After some hard experiences I introduced a global variable isTimedOut that controls hashing and value comparison.

Hope this helps,

pawel koziol
jwes
Posts: 778
Joined: Sat Jul 01, 2006 7:11 am

Re: Clear or not to clear the hash tables

Post by jwes »

It sounds like you have a bug in your hash code, possibly somehow reversing the sign of the value in hashes from previous iterations.
User avatar
pedrox
Posts: 1056
Joined: Fri Mar 10, 2006 6:07 am
Location: Basque Country (Spain)

Re: Clear or not to clear the hash tables

Post by pedrox »

My engine also maded moves really bad sometimes (1-2%). When I analyzed, I cleared hashtables and the error was not reproduced.

It was because of the random number used in Zobrist Keys, I used a proposed which appears on page Bruce Moreland, Where the number is not sufficiently random and I had to change.

Pedro
Cardoso
Posts: 362
Joined: Thu Mar 16, 2006 7:39 pm
Location: Portugal
Full name: Alvaro Cardoso

Re: Clear or not to clear the hash tables

Post by Cardoso »

Yes the best move from the previous iteration is the first to be searched.
And I also don't write to the hash if my stopsearch/timeout flag is set.

Alvaro
Cardoso
Posts: 362
Joined: Thu Mar 16, 2006 7:39 pm
Location: Portugal
Full name: Alvaro Cardoso

Re: Clear or not to clear the hash tables

Post by Cardoso »

Yes I do validate.
I've also tested for collisions and it happens one in 9 billion.

Alvaro
Cardoso
Posts: 362
Joined: Thu Mar 16, 2006 7:39 pm
Location: Portugal
Full name: Alvaro Cardoso

Re: Clear or not to clear the hash tables

Post by Cardoso »

Pedro, sou de Portugal.
E tu donde és?

Alvaro
User avatar
pedrox
Posts: 1056
Joined: Fri Mar 10, 2006 6:07 am
Location: Basque Country (Spain)

Re: Clear or not to clear the hash tables

Post by pedrox »

In a region called Basque Country in the north of Spain. I take and I make propaganda of a beautiful city, San Sebastían, pretty when it not rains :D

Best,

Pedro

Image
Mangar

Re: Clear or not to clear the hash tables

Post by Mangar »

hu, thats really often! Do you use a full 64 bit key to check it´s the right one or do you use only a 32 bit key for the index and a 32 bit key to check?

In Spike I never saw a hash collision until now. There are versions of Spike that didn´t test for a false move from hash that would have crashed on a false move. Never saw any crash on thousands of thes matches and never heard anybody mentioned such a case ...

There is a web page with "real" random numbers took from space radiaton. I copied the random numbers from there. Didn´t change anything but I like the idea ...

Greetings Volker