Extensions, anyone?

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Extensions, anyone?

Post by mcostalba »

zamar wrote:
{ "Pawn Push to 7th Extension (PV nodes)", "1", "1", SPIN, 0, 2, {""} },
{ "Pawn Push to 7th Extension (non-PV nodes)", "1", "1", SPIN, 0, 2, {""} },

This happens so rarely (or game is already decided when it happens) that measuring value of it is almost impossible. Test this only if you can run say >100k games.
If it is true what you say, well, it is in any case NOT obviously true, at least for me.

Engine calculates 15-20 moves in advance also on non-pv lines, so Pawn Push to 7th Extension is quite common starting from a middle game position and became very common toward endgame (you have to consider secondary lines and 15 moves in advance).
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Extensions, anyone?

Post by mcostalba »

bob wrote: I test with G2 all the time. If you have something you think is better, I can include it in a test run here to see if it works for large numbers of games...

Thanks for your offer.....I am still waiting (when you''ll find the time, as you said) results for beta cut-off counters instead of node counter for root move ordering :-)

Marco
Tord Romstad
Posts: 1808
Joined: Wed Mar 08, 2006 9:19 pm
Location: Oslo, Norway

Re: Extensions, anyone?

Post by Tord Romstad »

Greg Strong wrote:
Tord Romstad wrote:
I do not know how much Elo that they add.
Neither do I. I almost never have the chance to run long test matches, so all my parameter settings (including the extensions) are based on little more than intuition and educated guesses. The probability that the default settings are optimal is infinitesimal.

Tord
Ok, I'll run some tests :D
Hello Gregory,

Thanks for the generous offer! I'll be very interested in seeing the results, but I should warn you not to expect anything exciting: As Bob points out, it is unlikely that changes in extensions will have sufficiently big impact on the strength to be measurable without thousands of games.

Tord
Tord Romstad
Posts: 1808
Joined: Wed Mar 08, 2006 9:19 pm
Location: Oslo, Norway

Re: Extensions, anyone?

Post by Tord Romstad »

bob wrote:I test with G2 all the time. If you have something you think is better, I can include it in a test run here to see if it works for large numbers of games...
That Stockfish 1.2 is stronger than Glaurung 2.2 seems almost 100% clear now. Have a look at the new CEGT rating list published today, for instance.

Tord
Tord Romstad
Posts: 1808
Joined: Wed Mar 08, 2006 9:19 pm
Location: Oslo, Norway

Re: Extensions, anyone?

Post by Tord Romstad »

zamar wrote:If you really want to play with extensions, you should start with the most interesting ones.

{ "Check Extension (PV nodes)", "2", "2", SPIN, 0, 2, {""} },
{ "Check Extension (non-PV nodes)", "1", "1", SPIN, 0, 2, {""} },

If you disable check extensions, program will lose quite many elo points.
Probably, but whether extending by half a ply or a full ply is better is an interesting question. As you can see above, Glaurung extends checks by only half a ply, except in PV nodes (which are very few).
{ "Single Reply Extension (PV nodes)", "2", "2", SPIN, 0, 2, {""} },
{ "Single Reply Extension (non-PV nodes)", "2", "2", SPIN, 0, 2, {""} },

This is quite obvious (disabling it makes no point), but this one can also be quite easy to measure.
I think (based on educated guesses, not on tests, as usual) that checks and single replies shouldn't both be extended by a full ply at non-PV nodes. In other words, if check extensions are increased to a full ply, single replies should be decreased to half a ply.

{ "Mate Threat Extension (PV nodes)", "0", "0", SPIN, 0, 2, {""} },
{ "Mate Threat Extension (non-PV nodes)", "0", "0", SPIN, 0, 2, {""} },

This can be used to tune engine to more attractive style of play, but my little tests suggest enabling it makes program weaker in general.
I don't even think it has any noticable effect on the style of play. It solves some test positions more quickly, that's all.
{ "Pawn Push to 7th Extension (PV nodes)", "1", "1", SPIN, 0, 2, {""} },
{ "Pawn Push to 7th Extension (non-PV nodes)", "1", "1", SPIN, 0, 2, {""} },

This happens so rarely (or game is already decided when it happens) that measuring value of it is almost impossible. Test this only if you can run say >100k games.
I think this extension is far more important than you believe. Pawn pushes to the 7th rank are not that unusual, and they often occur at a very critical moment of the game. The outcome of the game is very likely to depend on whether the pawn can promote in a few moves, and extending the move makes a lot of sense.
{ "Passed Pawn Extension (PV nodes)", "1", "1", SPIN, 0, 2, {""} },
{ "Passed Pawn Extension (non-PV nodes)", "0", "0", SPIN, 0, 2, {""} },

Hard to measure, you need a lot of games (30k?)
I think it helps, but as always, I am just guessing.
I agree with others that there is not much (if at all) to gain from extensions, instead there are other search parameters (like LMR & futility stuff) in glaurung which can (and should) be tuned.
I don't think there are any huge improvements to be found by tuning these parameters either. I guess that even after fine-tuning all the many search and evaluation parameters in Glaurung with millions of games, you wouldn't be able to improve the strength by more than 30-40 Elo points. For bigger improvements, you'd have to modify the source code.

Tord
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Extensions, anyone?

Post by bob »

mcostalba wrote:
bob wrote: I test with G2 all the time. If you have something you think is better, I can include it in a test run here to see if it works for large numbers of games...

Thanks for your offer.....I am still waiting (when you''ll find the time, as you said) results for beta cut-off counters instead of node counter for root move ordering :-)

Marco
As am I. :) We have had a "blast" user on the cluster that has been causing some havoc by searching a _monster_ file on all 128 nodes at the same time, which is causing some I/O issues where things time out and hang...

working on a fix...
zamar
Posts: 613
Joined: Sun Jan 18, 2009 7:03 am

Re: Extensions, anyone?

Post by zamar »

Tord Romstad wrote: I don't think there are any huge improvements to be found by tuning these parameters either. I guess that even after fine-tuning all the many search and evaluation parameters in Glaurung with millions of games, you wouldn't be able to improve the strength by more than 30-40 Elo points. For bigger improvements, you'd have to modify the source code.

Tord
I'm afraid you are right :(

As a little hobby I've tried to built an automated chess engine variable balancing system and used glaurung as a test engine. For first run I decided to balance glaurung's hand-made piece value table:

PawnValueMidgame 204
PawnValueEndgame 256
KnightValueMidgame 832
KnightValueEndgame 832
BishopValueMidgame 832
BishopValueEndgame 832
RookValueMidgame 1285
RookValueEndgame 1285
QueenValueMidgame 2560
QueenValueEndgame 2560

After 60k balancing games I got values:

204, 257, 822, 845, 833, 849, 1271, 1274, 2521, 2577

Test match: modified vs. original

+20761 =20862 -20363 (~2 elo points).

Balancing took a lot of time and was worth of almost nothing in elo points. And because test games were run at extreme lightning speed (1s + 0.1s/inc) and using ~2500 position epd-opening book, it's unclear if those values work at longer time control or with different opening book (or against other engines).
Joona Kiiski
User avatar
Greg Strong
Posts: 388
Joined: Sun Dec 21, 2008 6:57 pm
Location: Washington, DC

Re: Extensions, anyone?

Post by Greg Strong »

The first results are in. This test was of the mate threat extension. By default, Glaurung has no mate threat enabled. I tried that and the four other possabilities (half ply extension at PV only, half ply extension everywhere, full ply at pv and half everywhere else, and full ply everywhere.) The results are:

Code: Select all

                             Wins - Losses - Draws vs. Opponent

                   Crafty          Fruit        Scorpio      Stockfish        Total

Mate Thrt 0/0   12-4-4 (70%)   13-3-4 (75%)  17-1-2 (90%)   7-5-8 (55%)   49-13-18 (72.5%)

Mate Thrt 1/0   16-2-2 (85%)   11-6-3 (63%)  16-4-0 (80%)   6-7-7 (48%)   49-19-12 (68.75%)

Mate Thrt 1/1   13-1-6 (80%)   11-7-2 (60%)  16-1-3 (88%)   6-5-9 (53%)   46-14-20 (70%)

Mate Thrt 2/1   14-3-3 (78%)   11-6-3 (63%)  15-3-2 (80%)   7-5-8 (55%)   47-17-16 (68.75%)

Mate Thrt 2/2   13-3-4 (75%)   11-4-5 (68%)  12-3-5 (73%)  12-3-5 (73%)   48-13-19 (71.875%)
Yup, not exactly conclusive. It does look like the unbalanced extensions (extending more at PV nodes) is clearly worse than the balanced ones. It will be interesting to see if that continues to apply to the other extensions.

Next up ... Extending for pawn-to-7th-rank.
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Extensions, anyone?

Post by mcostalba »

Greg Strong wrote:

Code: Select all

                             Wins - Losses - Draws vs. Opponent

                   Crafty          Fruit        Scorpio      Stockfish        Total


Mate Thrt 2/2   13-3-4 (75%)   11-4-5 (68%)  12-3-5 (73%)  12-3-5 (73%)   48-13-19 (71.875%)
Sorry for the silly question but I don't understund who wins against who. The percent (75%, 68%, 73%, 73%) are the wins against the looses of Glaurung or of the opponent ?

In other words, if percent is > 50% it means it is Glaurung that won more or it was the opponent that won more?

Thanks
Marco
User avatar
Greg Strong
Posts: 388
Joined: Sun Dec 21, 2008 6:57 pm
Location: Washington, DC

Re: Extensions, anyone?

Post by Greg Strong »

Not a silly question - should have made the table more clear. The three numbers are wins-losses-draws for Glaurung, and the percentage is the percent of points Glaurung won, with draws being half a point. Or:

Percentage = (Glaurung wins + (Glaurung draws / 2)) / total games.