a way to prevent broken PV's when using hash tables...

[sluijten]

I noticed that the percentage of exact entries in the hash table is quite low, this technique will not break the PV: only use null-move cut-offs and beta cut-offs from the hash table. You can still store exact results, but only use them for move ordering.

[bob]

I noticed that the percentage of exact entries in the hash table is quite low, this technique will not break the PV: only use null-move cut-offs and beta cut-offs from the hash table. You can still store exact results, but only use them for move ordering.

That will hurt more in endgames than you suspect. And it hurts somewhat overall, or you would never see a short PV in the first place...

[lucasart]

I noticed that the percentage of exact entries in the hash table is quite low, this technique will not break the PV: only use null-move cut-offs and beta cut-offs from the hash table. You can still store exact results, but only use them for move ordering.

That will hurt more in endgames than you suspect. And it hurts somewhat overall, or you would never see a short PV in the first place...

I also do the the same. So how do solve the 3 move problem if you use htable pruning at PV nodes ?

[hgm]

3 move problem?

You can pry the PV out of the hash table, or hash the entire PV in a separate table or overflow entry.

[bob]

I noticed that the percentage of exact entries in the hash table is quite low, this technique will not break the PV: only use null-move cut-offs and beta cut-offs from the hash table. You can still store exact results, but only use them for move ordering.

That will hurt more in endgames than you suspect. And it hurts somewhat overall, or you would never see a short PV in the first place...

I also do the the same. So how do solve the 3 move problem if you use htable pruning at PV nodes ?

First, describe the specific problem. If it is a short PV, then you can look at what I do to hash the PATH below an exact entry so that it can be grafted onto the PV when you get an EXACT hit.

If you are concerned about repetitions, and missing them because of a hit, or seeing them where you should not, you can't do a thing about it. You also get wrong hash results on non-EXACT entries. I don't see any reason to weaken an engine just to try to hide a _perceived_ problem...

[lucasart]

I noticed that the percentage of exact entries in the hash table is quite low, this technique will not break the PV: only use null-move cut-offs and beta cut-offs from the hash table. You can still store exact results, but only use them for move ordering.

That will hurt more in endgames than you suspect. And it hurts somewhat overall, or you would never see a short PV in the first place...

I also do the the same. So how do solve the 3 move problem if you use htable pruning at PV nodes ?

First, describe the specific problem. If it is a short PV, then you can look at what I do to hash the PATH below an exact entry so that it can be grafted onto the PV when you get an EXACT hit.

If you are concerned about repetitions, and missing them because of a hit, or seeing them where you should not, you can't do a thing about it. You also get wrong hash results on non-EXACT entries. I don't see any reason to weaken an engine just to try to hide a _perceived_ problem...

displaying a full PV is the least of my concerns. What I was worried about is the 3 repetition problem.

[bob]

I noticed that the percentage of exact entries in the hash table is quite low, this technique will not break the PV: only use null-move cut-offs and beta cut-offs from the hash table. You can still store exact results, but only use them for move ordering.

That will hurt more in endgames than you suspect. And it hurts somewhat overall, or you would never see a short PV in the first place...

I also do the the same. So how do solve the 3 move problem if you use htable pruning at PV nodes ?

First, describe the specific problem. If it is a short PV, then you can look at what I do to hash the PATH below an exact entry so that it can be grafted onto the PV when you get an EXACT hit.

If you are concerned about repetitions, and missing them because of a hit, or seeing them where you should not, you can't do a thing about it. You also get wrong hash results on non-EXACT entries. I don't see any reason to weaken an engine just to try to hide a _perceived_ problem...

displaying a full PV is the least of my concerns. What I was worried about is the 3 repetition problem.

There is _nothing_ you can do about that. Ignoring EXACT hits doesn't help a bit, because you still trust the fail-hi/fail-lo entries and they suffer from the same problem. Failing low or high because of either an overlooked repetition, or because you see an incorrect repetition score, is just as damaging.

This is a problem not worth worrying about. There is one well-known solution, that of not allowing matches when draft >= depth, and restrict it to draft == depth. But that is a performance-killer and will hurt more than it helps, from actual test measurements.

[lucasart]

I noticed that the percentage of exact entries in the hash table is quite low, this technique will not break the PV: only use null-move cut-offs and beta cut-offs from the hash table. You can still store exact results, but only use them for move ordering.

That will hurt more in endgames than you suspect. And it hurts somewhat overall, or you would never see a short PV in the first place...

I also do the the same. So how do solve the 3 move problem if you use htable pruning at PV nodes ?

First, describe the specific problem. If it is a short PV, then you can look at what I do to hash the PATH below an exact entry so that it can be grafted onto the PV when you get an EXACT hit.

If you are concerned about repetitions, and missing them because of a hit, or seeing them where you should not, you can't do a thing about it. You also get wrong hash results on non-EXACT entries. I don't see any reason to weaken an engine just to try to hide a _perceived_ problem...

displaying a full PV is the least of my concerns. What I was worried about is the 3 repetition problem.

There is _nothing_ you can do about that. Ignoring EXACT hits doesn't help a bit, because you still trust the fail-hi/fail-lo entries and they suffer from the same problem. Failing low or high because of either an overlooked repetition, or because you see an incorrect repetition score, is just as damaging.

This is a problem not worth worrying about. There is one well-known solution, that of not allowing matches when draft >= depth, and restrict it to draft == depth. But that is a performance-killer and will hurt more than it helps, from actual test measurements.

OK I am now testing the following TT pruning:

static inline bool can_return_tt(const TTEntry *tte, int depth, int alpha, int beta, int ply)
{
int tt_score = adjust_tt_score(tte->score, ply);

if (tte->type == ScoreExact)
// NB: if ply == 0, we don't want TT depth > depth, because of the 3 move rule
return (ply ? tte->depth >= depth : tte->depth == depth)
&& alpha < tt_score && tt_score < beta;
else
return (tte->depth >= depth
|| tt_score >= max(mate_in(MAX_PLY), beta)
|| tt_score < min(mated_in(MAX_PLY), beta))
&& ( (tte->type == ScoreLbound && tt_score >= beta)
|| (tte->type == ScoreUbound && tt_score <= alpha));
}

versus my previous one, which is the same but discarding the ScoreExact case (which inevitable comes from PV nodes, and onl;y prunes when encountered at PV nodes)

I'll probably need 200 games to decide, played 20 so far...

[tpetzke]

>>displaying a full PV is the least of my concerns. What I was worried about is the 3 repetition problem.

IMO you should have a 3 fold repetition detection that doesn't rely on a hash table. Before you probe the hash table you check for a repeated position (1 repetition is enough) and just return a draw or contempt score if it is repeated.

In such a case the hash table isn't probed at all and whatever is stored there doesn't interfere with the DRAWish evaluation of that node in the current path.

Thomas...

[lucasart]

>>displaying a full PV is the least of my concerns. What I was worried about is the 3 repetition problem.

IMO you should have a 3 fold repetition detection that doesn't rely on a hash table. Before you probe the hash table you check for a repeated position (1 repetition is enough) and just return a draw or contempt score if it is repeated.

In such a case the hash table isn't probed at all and whatever is stored there doesn't interfere with the DRAWish evaluation of that node in the current path.

Thomas...

Yes, that's one way to avoid the problem. Another way would be to zobrist into the key the number of times which the position was repeated, and then you can use htable as much as you like, and never have to think about 3 repetition in the htable code