Interesting tidbit

[bob]

I started a run on this test this morning, just to see if the latest LMR/etc changes had made this extension more useful. The version 23.4R05 is effectively 23.3 (as released) but with the one-legal-reply-to-check extension turned on. There are a few cosmetic changes, as in 23.4 now has no SearchRoot() function, and no QuiesceChecks() function, as the search functions were combined for more simplicity, as were the quiesce functions. But they produce exactly the same node counts, perhaps a tiny bit faster.

In Crafty, I extend when I check the opponent, rather than extending to escape check, since that made things a bit more accurate. Now, in the normal search, if I check the opponent, I can _never_ get directly to Quiesce() since the search is extended, which gives a bit more accuracy than dropping into q-search when in check. This one-reply-to-check extension extends on the escape, which means two consecutive plies are extended, once to give the check, and once to escape the check if there is only one legal move.

Here's the two results:

Code: Select all

Crafty-23.4-1        2674    3    3 30000   65%  2559   22% 
Crafty-23.4R05       2649    4    4 21900   62%  2558   22% 

This extension is a fairly significant loss, as you can see. Not quite finished, but it has not yet been running an hour either. Roughly -25 Elo. Some have asked why I removed this (and other extensions). This gives a good idea of why. Note that in the above test the extension is a full ply since I removed the fractional ply stuff a while back, whereas the older versions used 3/4 ply which was a less costly extension, and therefore the Elo loss was less. But even when I tried 0.5 it was still worse than testing with 0.0 which disabled the extension completely.

I decided to run this since this idea is an obvious feature of singular extensions, in that if you only have one legal move to try in a position, it would always be extended. The difference is that it would be possible in some positions to only have one move but not be in check, particularly in endgames. I'd think, logically, that if SE was really going to help, this would be a place where it might make sense to extend. But apparently not. I have one more run with longer time controls, for verification.

[hgm]

I had the same experience in my Xiangqi engine HaQiKi D. Extending singular check-evasions (which in XQ are much more common than in Chess, as Kings do not move diagonally, and can be checked by Rooks during a large part of the game) cause a dramatic drop in Elo.

[Chan Rasjid]

Could it not be that this should not be treated just as a case of SE. It might be more appropriate to ask why we extend and ,specifically here, why extending check here is bad.

Rasjid

[bob]

Could it not be that this should not be treated just as a case of SE. It might be more appropriate to ask why we extend and ,specifically here, why extending check here is bad.

Rasjid

I was not doing this as a part of SE. It is a special-case of SE, obviously, and I thought I would test it since I removed all (except check) extensions but way before the more aggressive LMR and forward pruning stuff. I just inserted the very trivial code to make this happen to see how it would affect things. Badly, obviously. The reasoning has always been that if you only have one possible move, you are very tightly constrained and searching deeper should be worthwhile. We did this extension in Cray Blitz, in fact, although we eventually did SE as well. The only issue I have not tested (although the SE tests I did certainly caught this) was to extend whenever there is just one legal move, whether in check or not (the current test had to be in check first to even qualify). My SE experiments did not have that flaw and would extend any move if there was only one legal move to try, because it was trivially singular.

But so far, no SE has worked for me, and this 1-reply extension still hurts in the case of Crafty. YMMV of course, but it just doesn't work for me.

[michiguel]

I started a run on this test this morning, just to see if the latest LMR/etc changes had made this extension more useful. The version 23.4R05 is effectively 23.3 (as released) but with the one-legal-reply-to-check extension turned on. There are a few cosmetic changes, as in 23.4 now has no SearchRoot() function, and no QuiesceChecks() function, as the search functions were combined for more simplicity, as were the quiesce functions. But they produce exactly the same node counts, perhaps a tiny bit faster.

In Crafty, I extend when I check the opponent, rather than extending to escape check, since that made things a bit more accurate. Now, in the normal search, if I check the opponent, I can _never_ get directly to Quiesce() since the search is extended, which gives a bit more accuracy than dropping into q-search when in check. This one-reply-to-check extension extends on the escape, which means two consecutive plies are extended, once to give the check, and once to escape the check if there is only one legal move.

Here's the two results:

Code: Select all

Crafty-23.4-1        2674    3    3 30000   65%  2559   22% 
Crafty-23.4R05       2649    4    4 21900   62%  2558   22% 

This extension is a fairly significant loss, as you can see. Not quite finished, but it has not yet been running an hour either. Roughly -25 Elo. Some have asked why I removed this (and other extensions). This gives a good idea of why. Note that in the above test the extension is a full ply since I removed the fractional ply stuff a while back, whereas the older versions used 3/4 ply which was a less costly extension, and therefore the Elo loss was less. But even when I tried 0.5 it was still worse than testing with 0.0 which disabled the extension completely.

I decided to run this since this idea is an obvious feature of singular extensions, in that if you only have one legal move to try in a position, it would always be extended. The difference is that it would be possible in some positions to only have one move but not be in check, particularly in endgames. I'd think, logically, that if SE was really going to help, this would be a place where it might make sense to extend. But apparently not. I have one more run with longer time controls, for verification.

Does the tree average increase ~25%?

Miguel

[bob]

I started a run on this test this morning, just to see if the latest LMR/etc changes had made this extension more useful. The version 23.4R05 is effectively 23.3 (as released) but with the one-legal-reply-to-check extension turned on. There are a few cosmetic changes, as in 23.4 now has no SearchRoot() function, and no QuiesceChecks() function, as the search functions were combined for more simplicity, as were the quiesce functions. But they produce exactly the same node counts, perhaps a tiny bit faster.

In Crafty, I extend when I check the opponent, rather than extending to escape check, since that made things a bit more accurate. Now, in the normal search, if I check the opponent, I can _never_ get directly to Quiesce() since the search is extended, which gives a bit more accuracy than dropping into q-search when in check. This one-reply-to-check extension extends on the escape, which means two consecutive plies are extended, once to give the check, and once to escape the check if there is only one legal move.

Here's the two results:

Code: Select all

Crafty-23.4-1        2674    3    3 30000   65%  2559   22% 
Crafty-23.4R05       2649    4    4 21900   62%  2558   22% 

This extension is a fairly significant loss, as you can see. Not quite finished, but it has not yet been running an hour either. Roughly -25 Elo. Some have asked why I removed this (and other extensions). This gives a good idea of why. Note that in the above test the extension is a full ply since I removed the fractional ply stuff a while back, whereas the older versions used 3/4 ply which was a less costly extension, and therefore the Elo loss was less. But even when I tried 0.5 it was still worse than testing with 0.0 which disabled the extension completely.

I decided to run this since this idea is an obvious feature of singular extensions, in that if you only have one legal move to try in a position, it would always be extended. The difference is that it would be possible in some positions to only have one move but not be in check, particularly in endgames. I'd think, logically, that if SE was really going to help, this would be a place where it might make sense to extend. But apparently not. I have one more run with longer time controls, for verification.

Does the tree average increase ~25%?

Miguel

I'd have to re-run to answer that. I don't save log files, which is where the tree size info is kept. I could try it on something like WAC but I don't think that would be a good test since the extension definitely looks better there. More after I get some data...

[QED]

This one-reply-to-check extension extends on the escape, which means two consecutive plies are extended, once to give the check, and once to escape the check if there is only one legal move.

The side in check and with single legal move is probably in serious danger, because "always check, it might be mate" and because if there really was a mate somewhere, the side might be unable to avoid it. It is useful to extend here and hopefully get the resolution before horizon. But only if this is a PV or CUT node. Situations where the single evasion is key move due to deep tactics are very uncommon, therefore extending in ALL node may be expanding the futile subtree.

If I was going to test single evasion extension, I would first do unextended search, and only extend when returned value is above alpha.

Hmm, and may it be the check extension could be judged by analogous logic? First search unextended, and research extended only if the value was not above alpha (to find tactics, but not look too far if not necessary)? No, because of horizon effect. We need to extend checks unconditionally.

[bob]

This one-reply-to-check extension extends on the escape, which means two consecutive plies are extended, once to give the check, and once to escape the check if there is only one legal move.

The side in check and with single legal move is probably in serious danger, because "always check, it might be mate" and because if there really was a mate somewhere, the side might be unable to avoid it. It is useful to extend here and hopefully get the resolution before horizon. But only if this is a PV or CUT node. Situations where the single evasion is key move due to deep tactics are very uncommon, therefore extending in ALL node may be expanding the futile subtree.

If I was going to test single evasion extension, I would first do unextended search, and only extend when returned value is above alpha.

Hmm, and may it be the check extension could be judged by analogous logic? First search unextended, and research extended only if the value was not above alpha (to find tactics, but not look too far if not necessary)? No, because of horizon effect. We need to extend checks unconditionally.

I am not so sure about your last statement. I recently modified Crafty to only extend checks that SEE says do not lose material. Result was several elo gain. Not 20-30, but significant. I think it might have been +5-7 but am not sure without digging through a mountain of data.

[edwardyu]

I had the same experience in my Xiangqi engine HaQiKi D. Extending singular check-evasions (which in XQ are much more common than in Chess, as Kings do not move diagonally, and can be checked by Rooks during a large part of the game) cause a dramatic drop in Elo.

My Xiangqi engine EychessU extends singular check-evasions in PV only (including root). Once I found it help Elo but now I am re-testing to remove the PV singular check-evasion extension (but retaining root).

BTW, I also extend singular check-evasions in QS (increase the QScheck depth by -2), so that I can find continuous mating sequence. Is this a correct way of mate-finder, and will this hurt Elo in real games?

Edward Yu

[edwardyu]

I had the same experience in my Xiangqi engine HaQiKi D. Extending singular check-evasions (which in XQ are much more common than in Chess, as Kings do not move diagonally, and can be checked by Rooks during a large part of the game) cause a dramatic drop in Elo.

My Xiangqi engine EychessU extends singular check-evasions in PV only (including root). Once I found it help Elo but now I am re-testing to remove the PV singular check-evasion extension (but retaining root).

BTW, I also extend singular check-evasions in QS (increase the QScheck depth by -2), so that I can find continuous mating sequence. Is this a correct way of mate-finder, and will this hurt Elo in real games?

Edward Yu

Ah! My serious mistake/bug! If root is single reply, why not just play it without searching any further!