Solving a fail low situation at the root

[bob]

Here's what I responded to:

Does Stockfish avoid fetching pv scores from the hash?

I responded "yes it does disallow exact hash hits at PV nodes."

From your comment, it seems that this is still being done. Getting rid of the hash PV reconstruction is a good idea. But not allowing exact hash matches is, IMHO, foolish when there is a very simple fix for this problem that works, without ignoring hash hits of any kind.

[gladius]

Here's what I responded to:

Does Stockfish avoid fetching pv scores from the hash?

I responded "yes it does disallow exact hash hits at PV nodes."

From your comment, it seems that this is still being done. Getting rid of the hash PV reconstruction is a good idea. But not allowing exact hash matches is, IMHO, foolish when there is a very simple fix for this problem that works, without ignoring hash hits of any kind.

We are going in circles here . Before last week, it *did* allow them. And reconstructed the PV from hash.

SF just started to disallow exact hash hits when the triangular array was introduced last week. That's what I was trying to point out.

RE: not allowing them, at least at the fishtest TC, there was no ELO benefit. So, that's why we decided not to go with a PV TT cache.

[xmas79]

Bob's PV Hash is an hashtable of paths which can be used to reconstruct the whole PV keeping hash hits in pv nodes. Nothing to do with TT hash.

[Joerg Oster]

Hi Joerg,

It seems that my first understanding of widening of bounds was totally wrong. I originally (wrongly) widened both bounds. When I saw Lucas's comment:
"set the unviolated bound half window"
I (wrongly) assumed that the unviolated bound would only be widened by half the violated one.

In my own testing I reduced the non-failing bound adjustment close to the point of not changing it at all, but I never arrived there.

I'm amazed that Lucas's patch doesn't cause a lot of search instability. Does a Stockfish TT hash entry keep track of the bounds? (I'm purposely not looking at Stockfish code until my engine is rewritten!) Does Stockfish avoid fetching pv scores from the hash?

Ron

Hi Ron,

regarding search instability, yes, I think i am now seeing more fail-high/fail-low situations when analyzing than before. But that's just my intuition, I didn't investigate it further.
Regarding your other points, I think they already have been answered by Gary.

[bob]

Here's what I responded to:

Does Stockfish avoid fetching pv scores from the hash?

I responded "yes it does disallow exact hash hits at PV nodes."

From your comment, it seems that this is still being done. Getting rid of the hash PV reconstruction is a good idea. But not allowing exact hash matches is, IMHO, foolish when there is a very simple fix for this problem that works, without ignoring hash hits of any kind.

We are going in circles here . Before last week, it *did* allow them. And reconstructed the PV from hash.

SF just started to disallow exact hash hits when the triangular array was introduced last week. That's what I was trying to point out.

RE: not allowing them, at least at the fishtest TC, there was no ELO benefit. So, that's why we decided not to go with a PV TT cache.

OK, first, the oldest stockfish I have is 1.8. Here is a direct comment from the code:

// Transposition table lookup. At PV nodes, we don't use the TT for
// pruning, but only for move ordering.

Which says that the TT is ignored (scores) at PV nodes as best I can understand.

Here's the code that goes with that:

if (!PvNode && tte && ok_to_use_TT(tte, depth, beta, ply))
{
ss->currentMove = ttMove; // Can be MOVE_NONE
return value_from_tt(tte->value(), ply);
}

The value is not used at all if this is a "PvNode" it seems?

Which directly contradicts what you wrote. I did not dig through later versions, however, so anything could have changed. I just remembered the above from discussions here in recent months.

For the record, I consider it silly to treat PV and non-PV nodes differently, period. But specifically, treating them differently regarding how the table is used really makes no sense.

the PV hash was NOT to improve Elo, as previously explained. It was to provide correct PVs when you get exact entry hits along the PV. The intent was to give a user a real pv that takes you from the root position to the endpoint that produced the terminal node score, each and every time, without those short PVs that one always gets when using EXACT matches along the PV...

[gladius]

Here's what I responded to:

Does Stockfish avoid fetching pv scores from the hash?

I responded "yes it does disallow exact hash hits at PV nodes."

From your comment, it seems that this is still being done. Getting rid of the hash PV reconstruction is a good idea. But not allowing exact hash matches is, IMHO, foolish when there is a very simple fix for this problem that works, without ignoring hash hits of any kind.

We are going in circles here . Before last week, it *did* allow them. And reconstructed the PV from hash.

SF just started to disallow exact hash hits when the triangular array was introduced last week. That's what I was trying to point out.

RE: not allowing them, at least at the fishtest TC, there was no ELO benefit. So, that's why we decided not to go with a PV TT cache.

OK, first, the oldest stockfish I have is 1.8. Here is a direct comment from the code:

// Transposition table lookup. At PV nodes, we don't use the TT for
// pruning, but only for move ordering.

Which says that the TT is ignored (scores) at PV nodes as best I can understand.

Here's the code that goes with that:

if (!PvNode && tte && ok_to_use_TT(tte, depth, beta, ply))
{
ss->currentMove = ttMove; // Can be MOVE_NONE
return value_from_tt(tte->value(), ply);
}

The value is not used at all if this is a "PvNode" it seems?

Which directly contradicts what you wrote. I did not dig through later versions, however, so anything could have changed. I just remembered the above from discussions here in recent months.

For the record, I consider it silly to treat PV and non-PV nodes differently, period. But specifically, treating them differently regarding how the table is used really makes no sense.

the PV hash was NOT to improve Elo, as previously explained. It was to provide correct PVs when you get exact entry hits along the PV. The intent was to give a user a real pv that takes you from the root position to the endpoint that produced the terminal node score, each and every time, without those short PVs that one always gets when using EXACT matches along the PV...

Ok, well, in the future, when you say "SF does this", please say the version you are referring to. Even as far back as SF 2.3.1, SF allowed this: https://github.com/official-stockfish/S ... h.cpp#L603.

I understand the reason behind the PV hash. I even implemented a similar version in SF to allow the PV pruning to remain. Then I measured PV pruning and it wasn't worth anything, so it was simpler to just turn it off and then the PV hash is not needed.

[Eelco de Groot]

OK, first, the oldest stockfish I have is 1.8. Here is a direct comment from the code:

// Transposition table lookup. At PV nodes, we don't use the TT for
// pruning, but only for move ordering.

Which says that the TT is ignored (scores) at PV nodes as best I can understand.

Here's the code that goes with that:

if (!PvNode && tte && ok_to_use_TT(tte, depth, beta, ply))
{
ss->currentMove = ttMove; // Can be MOVE_NONE
return value_from_tt(tte->value(), ply);
}

The value is not used at all if this is a "PvNode" it seems?

Which directly contradicts what you wrote. I did not dig through later versions, however, so anything could have changed. I just remembered the above from discussions here in recent months.

For the record, I consider it silly to treat PV and non-PV nodes differently, period. But specifically, treating them differently regarding how the table is used really makes no sense.

the PV hash was NOT to improve Elo, as previously explained. It was to provide correct PVs when you get exact entry hits along the PV. The intent was to give a user a real pv that takes you from the root position to the endpoint that produced the terminal node score, each and every time, without those short PVs that one always gets when using EXACT matches along the PV...

We had this discussion on Open Chess, in 2011, started by Uly from the Rybka forum, Robert also participated and I think Marco too. One result of part of that discussion was that Stockfish started to directly return from exact hash hits in PV nodes, because it makes backward analysis more responsive. This must have been roughly somewhere around Stockfish 2.1 I think, at least long before 2.3 JA which was from around 15-09-2012, according to my copy. We are now virtually at Stockfish 6.0 so this was a long time ago and many many code-changes. It only just recently changed back, to make PVs possible that show the leaf position, at least if the GUI allows such long PVs which I don't think they do. I'm afraid most correspondence chess players will not be very happy though with the behaviour for backward analysis, but they have not yet found out.

If Gary's SF version of Crafty's solution with a separate hashtable for PV nodes that he already made as an intermediate version of what is now implemented, discussion of which is on Fishcooking and elsewhere, would solve the short PVs AND return exact hits in PV nodes, then the code would be more complex but there certainly are practical advantages for real world use of Stockfish, as far as I understand it? But the correspondence chess players have not yet discovered this recent change I think...

[Eelco de Groot]

Correction for Gary: the discussion about returning hashhits in PV nodes was not in this thread that is more about Stockfish' Fail High behaviour, but around the same time and also on Open Chess, I found a post from Marco referring to Stockfish 2.0.1 so that must have been the first version.

Eelco

[bob]

Here's what I responded to:

Does Stockfish avoid fetching pv scores from the hash?

I responded "yes it does disallow exact hash hits at PV nodes."

From your comment, it seems that this is still being done. Getting rid of the hash PV reconstruction is a good idea. But not allowing exact hash matches is, IMHO, foolish when there is a very simple fix for this problem that works, without ignoring hash hits of any kind.

We are going in circles here . Before last week, it *did* allow them. And reconstructed the PV from hash.

SF just started to disallow exact hash hits when the triangular array was introduced last week. That's what I was trying to point out.

RE: not allowing them, at least at the fishtest TC, there was no ELO benefit. So, that's why we decided not to go with a PV TT cache.

OK, first, the oldest stockfish I have is 1.8. Here is a direct comment from the code:

// Transposition table lookup. At PV nodes, we don't use the TT for
// pruning, but only for move ordering.

Which says that the TT is ignored (scores) at PV nodes as best I can understand.

Here's the code that goes with that:

if (!PvNode && tte && ok_to_use_TT(tte, depth, beta, ply))
{
ss->currentMove = ttMove; // Can be MOVE_NONE
return value_from_tt(tte->value(), ply);
}

The value is not used at all if this is a "PvNode" it seems?

Which directly contradicts what you wrote. I did not dig through later versions, however, so anything could have changed. I just remembered the above from discussions here in recent months.

For the record, I consider it silly to treat PV and non-PV nodes differently, period. But specifically, treating them differently regarding how the table is used really makes no sense.

the PV hash was NOT to improve Elo, as previously explained. It was to provide correct PVs when you get exact entry hits along the PV. The intent was to give a user a real pv that takes you from the root position to the endpoint that produced the terminal node score, each and every time, without those short PVs that one always gets when using EXACT matches along the PV...

Ok, well, in the future, when you say "SF does this", please say the version you are referring to. Even as far back as SF 2.3.1, SF allowed this: https://github.com/official-stockfish/S ... h.cpp#L603.

I understand the reason behind the PV hash. I even implemented a similar version in SF to allow the PV pruning to remain. Then I measured PV pruning and it wasn't worth anything, so it was simpler to just turn it off and then the PV hash is not needed.

EXACT hits are rare. But they do happen. And each time you disallow one, you increase the cost of the search. I don't see any justification for doing that. The "but we can't measure any improvement" doesn't quite cut it for me, because the issue is quite obvious and the effect is a negative one, no matter how big or small it might be. However, everyone gets to make their own choices, good, bad or indifferent.

[gladius]

Here's what I responded to:

Does Stockfish avoid fetching pv scores from the hash?

I responded "yes it does disallow exact hash hits at PV nodes."

From your comment, it seems that this is still being done. Getting rid of the hash PV reconstruction is a good idea. But not allowing exact hash matches is, IMHO, foolish when there is a very simple fix for this problem that works, without ignoring hash hits of any kind.

We are going in circles here . Before last week, it *did* allow them. And reconstructed the PV from hash.

SF just started to disallow exact hash hits when the triangular array was introduced last week. That's what I was trying to point out.

RE: not allowing them, at least at the fishtest TC, there was no ELO benefit. So, that's why we decided not to go with a PV TT cache.

OK, first, the oldest stockfish I have is 1.8. Here is a direct comment from the code:

// Transposition table lookup. At PV nodes, we don't use the TT for
// pruning, but only for move ordering.

Which says that the TT is ignored (scores) at PV nodes as best I can understand.

Here's the code that goes with that:

if (!PvNode && tte && ok_to_use_TT(tte, depth, beta, ply))
{
ss->currentMove = ttMove; // Can be MOVE_NONE
return value_from_tt(tte->value(), ply);
}

The value is not used at all if this is a "PvNode" it seems?

Which directly contradicts what you wrote. I did not dig through later versions, however, so anything could have changed. I just remembered the above from discussions here in recent months.

For the record, I consider it silly to treat PV and non-PV nodes differently, period. But specifically, treating them differently regarding how the table is used really makes no sense.

the PV hash was NOT to improve Elo, as previously explained. It was to provide correct PVs when you get exact entry hits along the PV. The intent was to give a user a real pv that takes you from the root position to the endpoint that produced the terminal node score, each and every time, without those short PVs that one always gets when using EXACT matches along the PV...

Ok, well, in the future, when you say "SF does this", please say the version you are referring to. Even as far back as SF 2.3.1, SF allowed this: https://github.com/official-stockfish/S ... h.cpp#L603.

I understand the reason behind the PV hash. I even implemented a similar version in SF to allow the PV pruning to remain. Then I measured PV pruning and it wasn't worth anything, so it was simpler to just turn it off and then the PV hash is not needed.

EXACT hits are rare. But they do happen. And each time you disallow one, you increase the cost of the search. I don't see any justification for doing that. The "but we can't measure any improvement" doesn't quite cut it for me, because the issue is quite obvious and the effect is a negative one, no matter how big or small it might be. However, everyone gets to make their own choices, good, bad or indifferent.

Yes, I agree, they do happen. The tradeoff is keeping them, and adding all the code for the PV hash, or turning them off, and then you don't need a PV hash. There are good arguments for both approaches.

If it was my personal project, I probably would have added the PV hash because it's a fun piece of code. But for SF, simpler is better.