Extensions, anyone?

[Greg Strong]

The trend these days seems to be toward more and more agressive pruning and less extending. In fact, I seem to recall Dr. Hyatt saying in a recent post that, except for the check extension, all other extensions in Crafty were either no help or actually made the program weaker.

So I'm now wondering. Does anyone use any other extensions (besides check extension) to positive effect? I'd be curious to know what kind, and whether its rewarded with a full ply or a fractional ply...

[Dann Corbit]

The trend these days seems to be toward more and more agressive pruning and less extending. In fact, I seem to recall Dr. Hyatt saying in a recent post that, except for the check extension, all other extensions in Crafty were either no help or actually made the program weaker.

So I'm now wondering. Does anyone use any other extensions (besides check extension) to positive effect? I'd be curious to know what kind, and whether its rewarded with a full ply or a fractional ply...

Here are the Glaurung extensions:
"Check Extension (PV nodes)", "2", "2", SPIN, 0, 2},
"Check Extension (non-PV nodes)", "2", "2", SPIN, 0, 2},
"Single Reply Extension (PV nodes)", "1", "1", SPIN, 0, 2},
"Single Reply Extension (non-PV nodes)", "1", "1", SPIN, 0, 2},
"Pawn Push to 7th Extension (PV nodes)", "1", "1", SPIN, 0, 2},
"Pawn Push to 7th Extension (non-PV nodes)", "1", "1", SPIN, 0, 2},
"Recapture Extension (PV nodes)", "2", "2", SPIN, 0, 2},
"Pawn Endgame Extension (PV nodes)", "2", "2", SPIN, 0, 2},
"Pawn Endgame Extension (non-PV nodes)", "2", "2", SPIN, 0, 2},

I do not know how much Elo that they add. I guess some testers might be able to give you some feedback or you could do your own experiments.

Scorpio extends unit depth for pv nodes and single reply, mate threat, capturing promotion, and unstoppable passed pawn extensions.

There are also lesser extensions for checks and captures (capture as a function of piece type):
switch(PIECE(m_capture(move))) {
case queen:
case rook:
extend(UNITDEPTH);
break;
case bishop:
case knight:
extend(UNITDEPTH / 2);
break;
case pawn:
extend(UNITDEPTH / 4);
break;
}

[Tord Romstad]

Here are the Glaurung extensions:
"Check Extension (PV nodes)", "2", "2", SPIN, 0, 2},
"Check Extension (non-PV nodes)", "2", "2", SPIN, 0, 2},
"Single Reply Extension (PV nodes)", "1", "1", SPIN, 0, 2},
"Single Reply Extension (non-PV nodes)", "1", "1", SPIN, 0, 2},
"Pawn Push to 7th Extension (PV nodes)", "1", "1", SPIN, 0, 2},
"Pawn Push to 7th Extension (non-PV nodes)", "1", "1", SPIN, 0, 2},
"Recapture Extension (PV nodes)", "2", "2", SPIN, 0, 2},
"Pawn Endgame Extension (PV nodes)", "2", "2", SPIN, 0, 2},
"Pawn Endgame Extension (non-PV nodes)", "2", "2", SPIN, 0, 2},

Hi Dann,

You must either be looking at some very old Glaurung version, or a version you or somebody else has modified. These are the extensions used in Glaurung 2.2 (and in 2.1, IIRC):

Code: Select all

    { "Check Extension (PV nodes)", "2", "2", SPIN, 0, 2, {""} },
    { "Check Extension (non-PV nodes)", "1", "1", SPIN, 0, 2, {""} },
    { "Single Reply Extension (PV nodes)", "2", "2", SPIN, 0, 2, {""} },
    { "Single Reply Extension (non-PV nodes)", "2", "2", SPIN, 0, 2, {""} },
    { "Mate Threat Extension (PV nodes)", "0", "0", SPIN, 0, 2, {""} },
    { "Mate Threat Extension (non-PV nodes)", "0", "0", SPIN, 0, 2, {""} },
    { "Pawn Push to 7th Extension (PV nodes)", "1", "1", SPIN, 0, 2, {""} },
    { "Pawn Push to 7th Extension (non-PV nodes)", "1", "1", SPIN, 0, 2, {""} },
    { "Passed Pawn Extension (PV nodes)", "1", "1", SPIN, 0, 2, {""} },
    { "Passed Pawn Extension (non-PV nodes)", "0", "0", SPIN, 0, 2, {""} },
    { "Pawn Endgame Extension (PV nodes)", "2", "2", SPIN, 0, 2, {""} },
    { "Pawn Endgame Extension (non-PV nodes)", "2", "2", SPIN, 0, 2, {""} },

The unit is half-plies. In other words "1" means an extension by half a ply, and "2" an extension by a full ply. Most of the extensions should be easy to understand, with the possible exception of the last two. "Passed Pawn Extension" is an extension for all passed pawn pushes, and "Pawn Endgame Extension" an extension for a capture leading to a pawn endgame.

I do not know how much Elo that they add.

Neither do I. I almost never have the chance to run long test matches, so all my parameter settings (including the extensions) are based on little more than intuition and educated guesses. The probability that the default settings are optimal is infinitesimal.

Tord

[Greg Strong]

I do not know how much Elo that they add.

Neither do I. I almost never have the chance to run long test matches, so all my parameter settings (including the extensions) are based on little more than intuition and educated guesses. The probability that the default settings are optimal is infinitesimal.

Tord

Ok, I'll run some tests

I've got three pretty powerful machines at work just sitting around doing nothing at the moment. They're quad-core zeons with 4 gigs of ram. I'll run some gauntlet testing with the 10 Nunn positions against some other programs using Arena.

If you want to weigh in on what tests you think would be best, or what programs to put it up against, let me know. My inclination is to put it against about 4 other programs, playing both sides of each of the 10 Nunn positions, with game in 5 minute time controls, and tweek the extensions and see what happens...

Against which programs, I don't know. Thoughts anyone? It has to be programs that are already built for 32-bit Windows (sadly those machines aren't 64 bit) and it's easiest if it's engines that come with Arena 2.0, although adding new ones is no big deal. And it should be programs that are similar in strength. And, it would be really interesting to use programs that are either open-source, or that their extensions are configurable so we know what extensions the other programs are using. I will definitely be including the latest Crafty.

I'm open to suggestions on any of this. I've never really done anything quite like this before. I'll start 'em up some time tonight. I'll make sure all test results are available.

Greg

[mcostalba]

Ok, I'll run some tests

This would be great !!

I am VERY interested in your results. I have tested a bit and found that the most important is the endgame extension (whose condition I have also tweaked a bit in Stockfish).

When you post the results I will double check the optimal setup you will find with the current one and post the results too.

Thanks
Marco

t!

[bob]

I do not know how much Elo that they add.

Neither do I. I almost never have the chance to run long test matches, so all my parameter settings (including the extensions) are based on little more than intuition and educated guesses. The probability that the default settings are optimal is infinitesimal.

Tord

Ok, I'll run some tests

I've got three pretty powerful machines at work just sitting around doing nothing at the moment. They're quad-core zeons with 4 gigs of ram. I'll run some gauntlet testing with the 10 Nunn positions against some other programs using Arena.

If you want to weigh in on what tests you think would be best, or what programs to put it up against, let me know. My inclination is to put it against about 4 other programs, playing both sides of each of the 10 Nunn positions, with game in 5 minute time controls, and tweek the extensions and see what happens...

Against which programs, I don't know. Thoughts anyone? It has to be programs that are already built for 32-bit Windows (sadly those machines aren't 64 bit) and it's easiest if it's engines that come with Arena 2.0, although adding new ones is no big deal. And it should be programs that are similar in strength. And, it would be really interesting to use programs that are either open-source, or that their extensions are configurable so we know what extensions the other programs are using. I will definitely be including the latest Crafty.

I'm open to suggestions on any of this. I've never really done anything quite like this before. I'll start 'em up some time tonight. I'll make sure all test results are available.

Greg

This test is no good. An extension change is probably at best a matter of +/- 10 Elo and generally a lot less. To get the error bar down to something that is less than that is going to take months. I ran the 32,000 game test to test different extension amounts and discovered most don't help or hurt at all, although most were having a small negative effect except for checks which was a bit more significant and does help.

But it is going to take a _ton_ of games...

[bob]

Ok, I'll run some tests

This would be great !!

I am VERY interested in your results. I have tested a bit and found that the most important is the endgame extension (whose condition I have also tweaked a bit in Stockfish).

When you post the results I will double check the optimal setup you will find with the current one and post the results too.

Thanks
Marco

t!

I test with G2 all the time. If you have something you think is better, I can include it in a test run here to see if it works for large numbers of games...

[Greg Strong]

I thought that might well be the case, but I'm going to give it a try anyway just for the fun of it. If the extensions produce almost no change, then it still tells me something - it tells me not to waste much time with them.

I'm going to use Glaurung 2.2 as my test subject against Crafty 22.10, Scorpio Beta, Fruit 2.3.1, and Stockfish 1.2 (optimistic.)

Say, Bob, when you run a test set of 32,000 games, do you do it by starting them from 16,000 predefined positions?

[bob]

I thought that might well be the case, but I'm going to give it a try anyway just for the fun of it. If the extensions produce almost no change, then it still tells me something - it tells me not to waste much time with them.

I'm going to use Glaurung 2.2 as my test subject against Crafty 22.10, Scorpio Beta, Fruit 2.3.1, and Stockfish 1.2 (optimistic.)

Say, Bob, when you run a test set of 32,000 games, do you do it by starting them from 16,000 predefined positions?

No. I have about 4,000 positions, played once with black, once with white, using 4 opponents. so 4000 x 2 x 4 = 32,000

[zamar]

If you really want to play with extensions, you should start with the most interesting ones.

{ "Check Extension (PV nodes)", "2", "2", SPIN, 0, 2, {""} },
{ "Check Extension (non-PV nodes)", "1", "1", SPIN, 0, 2, {""} },

If you disable check extensions, program will lose quite many elo points.
This is the first to test, because you get statistically meaningful results quickly


{ "Single Reply Extension (PV nodes)", "2", "2", SPIN, 0, 2, {""} },
{ "Single Reply Extension (non-PV nodes)", "2", "2", SPIN, 0, 2, {""} },

This is quite obvious (disabling it makes no point), but this one can also be quite easy to measure. Some time ago I modified glaurung to extend half a ply when there were 2 or 3 replies, but results (8087 games: 3s base time + 0.1s/move, elodiff: -1) were unconvincing.

{ "Mate Threat Extension (PV nodes)", "0", "0", SPIN, 0, 2, {""} },
{ "Mate Threat Extension (non-PV nodes)", "0", "0", SPIN, 0, 2, {""} },

This can be used to tune engine to more attractive style of play, but my little tests suggest enabling it makes program weaker in general.

{ "Pawn Push to 7th Extension (PV nodes)", "1", "1", SPIN, 0, 2, {""} },
{ "Pawn Push to 7th Extension (non-PV nodes)", "1", "1", SPIN, 0, 2, {""} },

This happens so rarely (or game is already decided when it happens) that measuring value of it is almost impossible. Test this only if you can run say >100k games.

{ "Passed Pawn Extension (PV nodes)", "1", "1", SPIN, 0, 2, {""} },
{ "Passed Pawn Extension (non-PV nodes)", "0", "0", SPIN, 0, 2, {""} },

Hard to measure, you need a lot of games (30k?)

{ "Pawn Endgame Extension (PV nodes)", "2", "2", SPIN, 0, 2, {""} },
{ "Pawn Endgame Extension (non-PV nodes)", "2", "2", SPIN, 0, 2, {""} },

Haven't tested this one.

I agree with others that there is not much (if at all) to gain from extensions, instead there are other search parameters (like LMR & futility stuff) in glaurung which can (and should) be tuned.