Hash cutoffs and analysis

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: Hash cutoffs and analysis

Post by Daniel Shawul »

hgm wrote:
Daniel Shawul wrote:Well you would know it is pv node if it is stored as EXACT in TT anyway so no need for node classification which i thought would be challenging.
That is not entirely true. It only means it was a PV node at the point where it was stored. That does not imply it will be a PV node now, as the current window might be different. You really have to test if the EXACT score lies between alpha and beta. (And if it does, it could only cause a cutoff if the TT entry was EXACT, so you won't have to test that separately.)
The windows will be wide enough anyway so the test is practically useless. Fruit had (-inf,inf) but i am sure an aspiration search of (-50,50) would still be more than enough. For fail lows/highs the test is compulsory because we are doing null window searches when using scout probably much less so with alpha-beta.
A depth 22 line when used for depth=3 search is useless because you are going to call eval().
Well, maybe you should not do that, then? If you are going to take the stand-pat you might as well take the hash cut. The PV won't continue beyond it either way.
Don't understand what you are saying here. We are not probing TT for the PV so its will be eval() at the tip (say after depth=4). Now you compare that with all those deep search values for the other moves or their eval scores for no cutoff you would see the PV will most likely be changing back and forth, unless its score was good also at shallow depth too.
User avatar
hgm
Posts: 27796
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Hash cutoffs and analysis

Post by hgm »

Daniel Shawul wrote:The windows will be wide enough anyway so the test is practically useless. Fruit had (-inf,inf) but i am sure an aspiration search of (-50,50) would still be more than enough. For fail lows/highs the test is compulsory because we are doing null window searches when using scout probably much less so with alpha-beta.
Well, I see it often enough that the score drops from one iteration to the next from -2 to -9...
Don't understand what you are saying here. We are not probing TT for the PV so its will be eval() at the tip (say after depth=4).
What I am saying is that maybe you should probe the TT in PV nodes with d=0. Can you name one argument for not probing it?
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Hash cutoffs and analysis

Post by bob »

Daniel Shawul wrote:I am surprised that this IS what you were looking for. It has been the norm since Fruit days to not do hash cutoffs at PVs for the sake of longer PVs. But I switched back to the old cutoff anywhere, because ironically that is better for analysis. You would not want to start from depth=0 every time which you will if you don't probe hash tables. So I am not sure it is a good move you made by not probing in TT. I have a problem in playing draws when cutting of at TT but I am not sure if everyone has this problem. If a move in TT is stored with depth=22 and score=6.0, then you have an immediate repeating draw, scorpio takes really long to switch them. This may be because of some peculiarities in the way i handle repeations but that is the only downside for me.
Just because Fruit does/did it, does NOT make it correct. Why on earth would anyone ignore a hash hit with an EXACT score, or fail to store an EXACT entry that could be used anywhere?

This is simply a form of laziness -- rather than addressing and fixing the problem, let's just side-step it and do something less efficient. I"ve heard this and "similar songs" many times in the past. IE "Don't store mate scores, they are problematic." They are not, if done correctly.
Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: Hash cutoffs and analysis

Post by Daniel Shawul »

hgm wrote:
Daniel Shawul wrote:The windows will be wide enough anyway so the test is practically useless. Fruit had (-inf,inf) but i am sure an aspiration search of (-50,50) would still be more than enough. For fail lows/highs the test is compulsory because we are doing null window searches when using scout probably much less so with alpha-beta.
Well, I see it often enough that the score drops from one iteration to the next from -2 to -9...
If it is an exact score, then it means you can use it as it is no matter with what alpha,beta you found it with. When you do aspiration, your windows are centered around it so it should fall within. Edit: Infact I am now wondering if check is needed at all not only practically but theoretically. If it is outside the window let it be, it will cause a cutoff. Who said an exact score should fall within whatever (alpha,beta) i have now ?
Don't understand what you are saying here. We are not probing TT for the PV so its will be eval() at the tip (say after depth=4).
What I am saying is that maybe you should probe the TT in PV nodes with d=0. Can you name one argument for not probing it?
Probing d=0 or not don't matter because you have a depth=20 move. Ofcourse you take the move and search it first but you are not going to cutoff with TT score so you all you have an eval() score. So that will cause the pv to change. Even if you use the score for the PV all of the other moves may not be stored in TT and only one move with eval() call that happened to have higher score is enough to disrupt the PV. What are you missing ?
User avatar
hgm
Posts: 27796
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Hash cutoffs and analysis

Post by hgm »

The idea was that if you don't do eval() but a hash probe at the end of the (not-so-deep) PV, the score would stay the same, and hence the PV would also stay the same. Even if you would not do depth bootstrapping, so that the depth the search reports is lower than the TT draft, and if you would overwrite the stored TT entries with this lower-draft result (which is inadvisable), the next iterations would pump up the depth to the original value without ever re-searching anything else than the PV nodes. The search is completely boxed in by the high-draft entries at the non-PV moves and at the (artificially shortened) end.

If indeed those entries are still there. But we are talking only about some 800 entries, most of them quite high draft, so the chances of that seem quite good.

Of course it is also possible to see this search of the PV that really should have been cutoff as a simple kludge to retrieve the PV from the hash table, rather than a real search. So an alternative approach would be to set a flag if you start searching a PV node that could have been cut off, and then when you return from the node and the flag is still set, suppress storing in the TT and return the TT score rather than what the search gave you. (And stumbling on a rep-draw would partly clear the flag, so that you would return the draw score, but still not store it in the hash table.)