Texel tuning method with a small number of positions

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
PK
Posts: 835
Joined: Mon Jan 15, 2007 10:23 am
Location: Warsza
Contact:

Texel tuning method with a small number of positions

Post by PK » Thu Mar 23, 2017 11:00 pm

I managed to put together a quick and dirty implementation of Texel tuning method. In fact, right now it is nothing more than a tool to facilitate manual tuning (albeit guided by decreasing difference between evaluation and score, calculated with the sigmoid function from Peter's original post). So far, nothing new - except for the fact that I was too eager to try the algorithm and initially used something like 70.000 positions (sampled from the epd file by Alexandro Mosoi) instead of 7 millions. Surprisingly, even that was enough to show some gain - provided that I tuned only major evaluation terms. I mean, tuning piece/square tables like that would be out of the question (tried and failed) but for piece values it was already OK. About 120.000 positions allowed me to extract some smallish gain from passed pawn scores.

It probably means that if this tuning method is applied in descending order (from big to small terms) then one can shave quite a number of quiescence search calls by only gradually increasing the number of positions used.

jdart
Posts: 3842
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Texel tuning method with a small number of positions

Post by jdart » Fri Mar 24, 2017 2:55 am

The risk is that you have a good fit to those 70,000 positions but a poor fit to the larger universe of possible positions.

--Jon

PK
Posts: 835
Joined: Mon Jan 15, 2007 10:23 am
Location: Warsza
Contact:

Re: Texel tuning method with a small number of positions

Post by PK » Fri Mar 24, 2017 7:07 am

The risk is that you have a good fit to those 70,000 positions but a poor fit to the larger universe of possible positions.
Yes, I see that problem, that's why I run the short gauntlet tournaments of 1000 games afterwards, which sort of negates the time gain. However, tweaking 10 parameters I'd normally run 10 such gauntlets (probably longer ones, too), now I run one.

Another filtering idea would be to use a few batches of games, tuning for several iterations on one (picked randomly) and verifying the gain on, say, 5 others. Change would be accepted if all 5 control batches show the gain.
[/quote]

Ferdy
Posts: 4113
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Texel tuning method with a small number of positions

Post by Ferdy » Sat Mar 25, 2017 3:19 am

PK wrote:I managed to put together a quick and dirty implementation of Texel tuning method. In fact, right now it is nothing more than a tool to facilitate manual tuning (albeit guided by decreasing difference between evaluation and score, calculated with the sigmoid function from Peter's original post). So far, nothing new - except for the fact that I was too eager to try the algorithm and initially used something like 70.000 positions (sampled from the epd file by Alexandro Mosoi) instead of 7 millions. Surprisingly, even that was enough to show some gain - provided that I tuned only major evaluation terms. I mean, tuning piece/square tables like that would be out of the question (tried and failed) but for piece values it was already OK. About 120.000 positions allowed me to extract some smallish gain from passed pawn scores.

I don't understand that last statement, would you explain a little bit more on the following.
About 120.000 positions allowed me to extract some smallish gain from passed pawn scores.
Last edited by Ferdy on Sat Mar 25, 2017 3:25 am, edited 1 time in total.

Ferdy
Posts: 4113
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Texel tuning method with a small number of positions

Post by Ferdy » Sat Mar 25, 2017 3:25 am

jdart wrote:The risk is that you have a good fit to those 70,000 positions but a poor fit to the larger universe of possible positions.

--Jon
I believe there is nothing wrong with small number of training positions as long as the engine when tested on the actual game tests involving positions that were not in training sets, would perform better.

PK
Posts: 835
Joined: Mon Jan 15, 2007 10:23 am
Location: Warsza
Contact:

Re: Texel tuning method with a small number of positions

Post by PK » Sat Mar 25, 2017 8:16 am

@Ferdinand, here are the details:

Using a set of 70.000 positions I succeeded in tuning:

- piece values (split into midgame and endgame)
- bishop, knight and rook pair values
- a couple of imbalance values (exchange, rook for 2 minors, pawn+knight bonus)

Using a set of 120.000 positions I successfully tuned two values:

- a percentage by which our passed pawn score is decreased if a passer is blocked
- a percentage by which our passed pawn score is increased if we control a square in front of it

On both sets tuned values failed in the test tournament on:

- pawn piece/square table
- percentage adjustement of material value and mobility values
- fianchetto bonus

Ferdy
Posts: 4113
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Texel tuning method with a small number of positions

Post by Ferdy » Sat Mar 25, 2017 9:48 am

PK wrote:@Ferdinand, here are the details:

Using a set of 70.000 positions I succeeded in tuning:

- piece values (split into midgame and endgame)
- bishop, knight and rook pair values
- a couple of imbalance values (exchange, rook for 2 minors, pawn+knight bonus)

Using a set of 120.000 positions I successfully tuned two values:

- a percentage by which our passed pawn score is decreased if a passer is blocked
- a percentage by which our passed pawn score is increased if we control a square in front of it

On both sets tuned values failed in the test tournament on:

- pawn piece/square table
- percentage adjustement of material value and mobility values
- fianchetto bonus
That is better.

So on,

Code: Select all

- pawn piece/square table
- percentage adjustement of material value and mobility values
- fianchetto bonus
We may say that its values are already optimal.

Post Reply