check extensions.

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
bob
Posts: 20347
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

check extensions.

Post by bob » Tue Oct 28, 2008 1:56 pm

I ran a few tests last night since I had recently seen a discussion about check extensions, and found some interesting results. First the data:

Code: Select all

Crafty-22.2R26-2  2587    5    5
Crafty-22.2R26-3  2599    4    4
Crafty-22.2R26-4  2608    5    5
Crafty-22.2R26-5  2579    4    4

Crafty-22.2R29-0  2597    5    5
Crafty-22.2R29-1  2597    5    5
Crafty-22.2R29-2  2598    4    4
Crafty-22.2R29-3  2597    5    5

Crafty-22.2R30-0  2597    5    5
Crafty-22.2R30-1  2594    5    5
Crafty-22.2R30-2  2598    4    4
Crafty-22.2R30-3  2597    5    5
OK, what are these? Crafty-22.2R26, R29 and R30 are identical versions of Crafty, except R26 varies the in-check extension from 1 (1/4 ply) to 5 (5/4 ply). R29 varies the one-legal-reply-to-check extension from 1/4 ply to 3/4 ply (default), R30 varies the mate-threat extension from 1/4 ply to 3/4 ply. In Crafty, the defaults are 4 (1 ply) for in-check, and 3 (3/4 ply) for the other two. Each of the above test versions played 32,000 games.

The rather surprising results were that the one-legal-reply and mate threat extensions appear to be completely useless, and I am going to remove them completely. The in check extension is more valuable, and surprisingly 1.0 plies seems to be the optimal value...

I had already discovered that the passed pawn push extension was no good, ditto for the recapture extension. It appears that the only one that is actually useful is the give-check or in-check (in Crafty it is actually a give-check extension where I extend when I give check rather than extending at the next ply when I escape check).

Tord Romstad
Posts: 1808
Joined: Wed Mar 08, 2006 8:19 pm
Location: Oslo, Norway

Re: check extensions.

Post by Tord Romstad » Tue Oct 28, 2008 2:59 pm

bob wrote:The rather surprising results were that the one-legal-reply and mate threat extensions appear to be completely useless, and I am going to remove them completely.
I still extend single replies (by half a ply, IIRC), but I am also not sure how useful they really are.
The in check extension is more valuable, and surprisingly 1.0 plies seems to be the optimal value...
I found an extension of half a ply of for checks to perform better the last time I tested. Of course, I don't test nearly as thoroughly as you do.

Tord

User avatar
hgm
Posts: 22327
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: check extensions.

Post by hgm » Tue Oct 28, 2008 3:26 pm

When I was developing uMax, I found that a Check extension hurt in in the end-game (when he King starts to be drawn towards the center). In fact, it hurt so much that the overall effect of having the check-extension during the entire game was almost nil. When I switched it off at the same time I dropped the King Safety terms from the eval, the check extension did have a huge positive effect (some 60 Elo, IIRC).

It did not strike me as too strange, as in end-games checking the King is usually not the way to make progress. So extending lines with long series of checks just draws search effort away from the lines that really matter.

bob
Posts: 20347
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: check extensions.

Post by bob » Tue Oct 28, 2008 3:26 pm

Tord Romstad wrote:
bob wrote:The rather surprising results were that the one-legal-reply and mate threat extensions appear to be completely useless, and I am going to remove them completely.
I still extend single replies (by half a ply, IIRC), but I am also not sure how useful they really are.
The in check extension is more valuable, and surprisingly 1.0 plies seems to be the optimal value...
I found an extension of half a ply of for checks to perform better the last time I tested. Of course, I don't test nearly as thoroughly as you do.

Tord
I am going for simplicity. If it doesn't help, it doesn't stay. :)

bob
Posts: 20347
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: check extensions.

Post by bob » Tue Oct 28, 2008 6:42 pm

hgm wrote:When I was developing uMax, I found that a Check extension hurt in in the end-game (when he King starts to be drawn towards the center). In fact, it hurt so much that the overall effect of having the check-extension during the entire game was almost nil. When I switched it off at the same time I dropped the King Safety terms from the eval, the check extension did have a huge positive effect (some 60 Elo, IIRC).

It did not strike me as too strange, as in end-games checking the King is usually not the way to make progress. So extending lines with long series of checks just draws search effort away from the lines that really matter.
I have not tried that specific experiment, but will queue it up. I am re-running the current "checkless" tests at a longer time control to make sure there is no hidden trap here. Intuition says check extensions are more valuable at faster time controls because of the shallower depth, but I want to confirm this.

The only thing I will have to figure out is when to "turn it off" (normal check extension) since I don't suddenly turn off king safety, I phase it out with the "interpolation" approach with two evaluation scores. I suppose I could use the old stand-by of queen+piece is too little material to worry about, anything more and the extensions "stay".

BubbaTough
Posts: 1154
Joined: Fri Jun 23, 2006 3:18 am

Re: check extensions.

Post by BubbaTough » Tue Oct 28, 2008 7:22 pm

On the topic of measuring small ELO gains, I would be interested in knowing what the size of the benefit of Crafty's non-zero repetition values is. No need to do a special test just for me of course, but if you ever end up measuring it I would love it if you posted it.

-Sam

bob
Posts: 20347
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: check extensions.

Post by bob » Tue Oct 28, 2008 7:46 pm

BubbaTough wrote:On the topic of measuring small ELO gains, I would be interested in knowing what the size of the benefit of Crafty's non-zero repetition values is. No need to do a special test just for me of course, but if you ever end up measuring it I would love it if you posted it.

-Sam
It is primarily of use only for the cases of "swindle mode" stuff. If I find an EGTB draw, I would prefer a KRP vs KR draw as opposed to a KR vs KR draw, because the latter is a bit easier to defend. :) I think Eugene Nalimov suggested this after we talked about the way we scored draws in Cray Blitz, which was more complicated. The idea is that a +1 is a draw with winning chances if the opponent screws up. A -1 is a draw but you have to play correctly or you can lose, while a 0 is just a draw that should be easy.

I doubt it is worth much if anything against computers, but against humans it can make a difference where you might have to choose between a simple draw and one the human can blow...

BubbaTough
Posts: 1154
Joined: Fri Jun 23, 2006 3:18 am

Re: check extensions.

Post by BubbaTough » Tue Oct 28, 2008 8:00 pm

bob wrote:
BubbaTough wrote:On the topic of measuring small ELO gains, I would be interested in knowing what the size of the benefit of Crafty's non-zero repetition values is. No need to do a special test just for me of course, but if you ever end up measuring it I would love it if you posted it.

-Sam
It is primarily of use only for the cases of "swindle mode" stuff. If I find an EGTB draw, I would prefer a KRP vs KR draw as opposed to a KR vs KR draw, because the latter is a bit easier to defend. :) I think Eugene Nalimov suggested this after we talked about the way we scored draws in Cray Blitz, which was more complicated. The idea is that a +1 is a draw with winning chances if the opponent screws up. A -1 is a draw but you have to play correctly or you can lose, while a 0 is just a draw that should be easy.

I doubt it is worth much if anything against computers, but against humans it can make a difference where you might have to choose between a simple draw and one the human can blow...
I see. my thought was that as a human, I value a position where I have the option to bail out into a repetition, and my opponent doesn't more than vice-versa. I have seen numerous computer games decided when the computer thought two moves were equivalent (both leading to perpetual) but they were not and the other computer took advantage. I have tried addressing that a little in my own engine, but the effect is so slight, if at all, I cannot tell if it is a benefit (or maybe even very minorly harmful).

-Sam

jarkkop
Posts: 197
Joined: Thu Mar 09, 2006 1:44 am
Location: Helsinki, Finland

Re: check extensions.

Post by jarkkop » Tue Oct 28, 2008 9:13 pm

But a feature you remove now can turn out to be useful in the future Crafty, right? You can't be sure that these current removals are the absolute correct values if you make other changes or improvements in Crafty.

bob
Posts: 20347
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: check extensions.

Post by bob » Tue Oct 28, 2008 9:44 pm

jarkkop wrote:But a feature you remove now can turn out to be useful in the future Crafty, right? You can't be sure that these current removals are the absolute correct values if you make other changes or improvements in Crafty.
Possible. But I don't want to leave junk lying around that definitely does nothing useful today, in the hope that it might do something useful next year... It is easy enough to add back if desired... I have had suspicions about these in the past and had throttled them back at times. Now I am convinced they are useless.

Post Reply