check extensions.

[bob]

I ran a few tests last night since I had recently seen a discussion about check extensions, and found some interesting results. First the data:

Code: Select all

Crafty-22.2R26-2  2587    5    5
Crafty-22.2R26-3  2599    4    4
Crafty-22.2R26-4  2608    5    5
Crafty-22.2R26-5  2579    4    4

Crafty-22.2R29-0  2597    5    5
Crafty-22.2R29-1  2597    5    5
Crafty-22.2R29-2  2598    4    4
Crafty-22.2R29-3  2597    5    5

Crafty-22.2R30-0  2597    5    5
Crafty-22.2R30-1  2594    5    5
Crafty-22.2R30-2  2598    4    4
Crafty-22.2R30-3  2597    5    5

OK, what are these? Crafty-22.2R26, R29 and R30 are identical versions of Crafty, except R26 varies the in-check extension from 1 (1/4 ply) to 5 (5/4 ply). R29 varies the one-legal-reply-to-check extension from 1/4 ply to 3/4 ply (default), R30 varies the mate-threat extension from 1/4 ply to 3/4 ply. In Crafty, the defaults are 4 (1 ply) for in-check, and 3 (3/4 ply) for the other two. Each of the above test versions played 32,000 games.

The rather surprising results were that the one-legal-reply and mate threat extensions appear to be completely useless, and I am going to remove them completely. The in check extension is more valuable, and surprisingly 1.0 plies seems to be the optimal value...

I had already discovered that the passed pawn push extension was no good, ditto for the recapture extension. It appears that the only one that is actually useful is the give-check or in-check (in Crafty it is actually a give-check extension where I extend when I give check rather than extending at the next ply when I escape check).

[Tord Romstad]

The rather surprising results were that the one-legal-reply and mate threat extensions appear to be completely useless, and I am going to remove them completely.

I still extend single replies (by half a ply, IIRC), but I am also not sure how useful they really are.

The in check extension is more valuable, and surprisingly 1.0 plies seems to be the optimal value...

I found an extension of half a ply of for checks to perform better the last time I tested. Of course, I don't test nearly as thoroughly as you do.

Tord

[hgm]

When I was developing uMax, I found that a Check extension hurt in in the end-game (when he King starts to be drawn towards the center). In fact, it hurt so much that the overall effect of having the check-extension during the entire game was almost nil. When I switched it off at the same time I dropped the King Safety terms from the eval, the check extension did have a huge positive effect (some 60 Elo, IIRC).

It did not strike me as too strange, as in end-games checking the King is usually not the way to make progress. So extending lines with long series of checks just draws search effort away from the lines that really matter.

[bob]

The rather surprising results were that the one-legal-reply and mate threat extensions appear to be completely useless, and I am going to remove them completely.

I still extend single replies (by half a ply, IIRC), but I am also not sure how useful they really are.

The in check extension is more valuable, and surprisingly 1.0 plies seems to be the optimal value...

I found an extension of half a ply of for checks to perform better the last time I tested. Of course, I don't test nearly as thoroughly as you do.

Tord

I am going for simplicity. If it doesn't help, it doesn't stay.

[bob]

When I was developing uMax, I found that a Check extension hurt in in the end-game (when he King starts to be drawn towards the center). In fact, it hurt so much that the overall effect of having the check-extension during the entire game was almost nil. When I switched it off at the same time I dropped the King Safety terms from the eval, the check extension did have a huge positive effect (some 60 Elo, IIRC).

It did not strike me as too strange, as in end-games checking the King is usually not the way to make progress. So extending lines with long series of checks just draws search effort away from the lines that really matter.

I have not tried that specific experiment, but will queue it up. I am re-running the current "checkless" tests at a longer time control to make sure there is no hidden trap here. Intuition says check extensions are more valuable at faster time controls because of the shallower depth, but I want to confirm this.

The only thing I will have to figure out is when to "turn it off" (normal check extension) since I don't suddenly turn off king safety, I phase it out with the "interpolation" approach with two evaluation scores. I suppose I could use the old stand-by of queen+piece is too little material to worry about, anything more and the extensions "stay".

[BubbaTough]

On the topic of measuring small ELO gains, I would be interested in knowing what the size of the benefit of Crafty's non-zero repetition values is. No need to do a special test just for me of course, but if you ever end up measuring it I would love it if you posted it.

-Sam

[bob]

On the topic of measuring small ELO gains, I would be interested in knowing what the size of the benefit of Crafty's non-zero repetition values is. No need to do a special test just for me of course, but if you ever end up measuring it I would love it if you posted it.

-Sam

It is primarily of use only for the cases of "swindle mode" stuff. If I find an EGTB draw, I would prefer a KRP vs KR draw as opposed to a KR vs KR draw, because the latter is a bit easier to defend. I think Eugene Nalimov suggested this after we talked about the way we scored draws in Cray Blitz, which was more complicated. The idea is that a +1 is a draw with winning chances if the opponent screws up. A -1 is a draw but you have to play correctly or you can lose, while a 0 is just a draw that should be easy.

I doubt it is worth much if anything against computers, but against humans it can make a difference where you might have to choose between a simple draw and one the human can blow...

[BubbaTough]

On the topic of measuring small ELO gains, I would be interested in knowing what the size of the benefit of Crafty's non-zero repetition values is. No need to do a special test just for me of course, but if you ever end up measuring it I would love it if you posted it.

-Sam

It is primarily of use only for the cases of "swindle mode" stuff. If I find an EGTB draw, I would prefer a KRP vs KR draw as opposed to a KR vs KR draw, because the latter is a bit easier to defend. I think Eugene Nalimov suggested this after we talked about the way we scored draws in Cray Blitz, which was more complicated. The idea is that a +1 is a draw with winning chances if the opponent screws up. A -1 is a draw but you have to play correctly or you can lose, while a 0 is just a draw that should be easy.

I doubt it is worth much if anything against computers, but against humans it can make a difference where you might have to choose between a simple draw and one the human can blow...

I see. my thought was that as a human, I value a position where I have the option to bail out into a repetition, and my opponent doesn't more than vice-versa. I have seen numerous computer games decided when the computer thought two moves were equivalent (both leading to perpetual) but they were not and the other computer took advantage. I have tried addressing that a little in my own engine, but the effect is so slight, if at all, I cannot tell if it is a benefit (or maybe even very minorly harmful).

-Sam

[jarkkop]

But a feature you remove now can turn out to be useful in the future Crafty, right? You can't be sure that these current removals are the absolute correct values if you make other changes or improvements in Crafty.

[bob]

But a feature you remove now can turn out to be useful in the future Crafty, right? You can't be sure that these current removals are the absolute correct values if you make other changes or improvements in Crafty.

Possible. But I don't want to leave junk lying around that definitely does nothing useful today, in the hope that it might do something useful next year... It is easy enough to add back if desired... I have had suspicions about these in the past and had throttled them back at times. Now I am convinced they are useless.