Impressive Texel-tuning results

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
sandermvdb
Posts: 157
Joined: Sat Jan 28, 2017 12:29 pm
Location: The Netherlands

Impressive Texel-tuning results

Post by sandermvdb » Fri Jun 16, 2017 5:49 pm

About a month ago I tried to tune my evaluation values using the Texel-tuning method:

First I thought about how to create fens of millions of positions which should contain the actual result of that match. I used cute-chess at very fast time controls and pgn-extractor which created 3 files containing the positions: wins.epd, lost.epd and draws.epd.

Then I wrote some logic to import these fens and I implemented a local optimization routine as described on the cpw. My optimization routine is able to tune multiple arrays and per array a 'step' can be specified. I also made it possible to specify that certain values in the array can be skipped, values can be mirrored (left, right) and should sometimes be applied to both the white and black arrays.

The cpw describes that you should use the quiescence-search for this method but I wasn't happy with the speed of calculating the error which took about 20 seconds (see: http://talkchess.com/forum/viewtopic.php?t=64189). I also read that the author of zurichess used the Texel-tuning method and he had created several epd files which could be used as a testset: http://www.talkchess.com/forum/viewtopic.php?t=61427
This zip-file also contains a quiet.epd which only contains quiet positions so I thought that using this set I don't need the quiescence-search but I can call the evaluation funtion directly. This is way faster and can quite easily be made multi-threaded, which is what I did.

Next I calculated the scaling-factor which would give the lowest error: 1.3

My original evaluation function had an error of 0.0654. Tuned I am able to get it to 0.0586. These are the (sometimes obvious) major tweaks:

Code: Select all

- rook material          : 500 -> 645
- queen material         : 950 -> 1200
- pinned-pieces-penalty  : always 15 -> 0-80, depending on the pinned-piece type
- pawn-storm-bonus       : disabled
- pawn-shield-bonus      : way higher scores when the pawn is almost promoted, now the king is defending the pawn! :)
- king-safety-scores     : start at 50, then decreases, then increases, no clue why :P
- mobility               : bigger penalties and bonusses
- psqt                   : subtle but relevant tweaks
All in all this give an Elo increase of more than 100 elo points! See table below for prelimenary results (chess22k-ex has tuned values, chess22k has same logic without tuned values)

Code: Select all

Rank Name                          Elo     +/-   Games   Score   Draws
   1 chess22k-ex                   146      45     191   69.9%   26.7%
   2 Ruffian_105                    31      42     193   54.4%   29.0%
   3 Maverick 1.5                   -7      42     192   49.0%   29.2%
   4 chess22k                      -20      41     192   47.1%   29.7%
   5 AnMon_5.75                    -60      42     193   41.5%   26.9%
   6 chess22k 1.3                  -83      40     191   38.2%   35.6%
576 of 1500 games finished.

Some other things I like is that you can easily see which evaluation values are actually not used.
It is now also easy to add a new feature to the evalution function and see how it performs (assuming that a lower error values results in a higher elo-score).

I guess chess22k 1.4 will be released soon! :)

sasachess
Posts: 24
Joined: Wed Nov 05, 2014 10:28 am
Location: Italy
Contact:

Re: Impressive Texel-tuning results

Post by sasachess » Fri Jun 16, 2017 8:29 pm

I'm very happy to read your post! I remember our chat not many days ago. :wink:

sandermvdb
Posts: 157
Joined: Sat Jan 28, 2017 12:29 pm
Location: The Netherlands

Re: Impressive Texel-tuning results

Post by sandermvdb » Fri Jun 16, 2017 9:01 pm

sasachess wrote:I'm very happy to read your post! I remember our chat not many days ago. :wink:
If you are the author of gogobello, yes we had that chat! :)
Thanks for pointing me in this direction!

sandermvdb
Posts: 157
Joined: Sat Jan 28, 2017 12:29 pm
Location: The Netherlands

Re: Impressive Texel-tuning results

Post by sandermvdb » Sat Jun 17, 2017 5:34 am

Tournament is finished:

Code: Select all

Rank Name                          Elo     +/-   Games   Score   Draws
   1 chess22k-ex                   137      27     500   68.7%   26.6%
   2 Ruffian_105                    31      26     500   54.5%   28.2%
   3 Maverick 1.5                   -5      26     500   49.3%   26.2%
   4 chess22k                      -33      26     500   45.3%   30.2%
   5 AnMon_5.75                    -58      27     500   41.7%   23.8%
   6 chess22k 1.3                  -67      26     500   40.5%   31.0%

1500 of 1500 games finished.

brtzsnr
Posts: 433
Joined: Fri Jan 16, 2015 3:02 pm
Contact:

Re: Impressive Texel-tuning results

Post by brtzsnr » Sat Jun 17, 2017 8:08 am

Congrats.

I'm happy the training set worked for you. I believe you are the forth person (including me) reporting success with it. If you don't mind I will reference this thread somewhere on the repository.

I've been trying to improve that training set for the past few months with no success. I tried to add some non-quiet positions (and fixing the evaluation) but it was still a Elo loss.

sandermvdb
Posts: 157
Joined: Sat Jan 28, 2017 12:29 pm
Location: The Netherlands

Re: Impressive Texel-tuning results

Post by sandermvdb » Sat Jun 17, 2017 9:49 am

brtzsnr wrote:Congrats.

I'm happy the training set worked for you. I believe you are the forth person (including me) reporting success with it. If you don't mind I will reference this thread somewhere on the repository.

I've been trying to improve that training set for the past few months with no success. I tried to add some non-quiet positions (and fixing the evaluation) but it was still a Elo loss.
I don't mind and thanks for the set!
Please keep us informed if you have a better set :)

sasachess
Posts: 24
Joined: Wed Nov 05, 2014 10:28 am
Location: Italy
Contact:

Re: Impressive Texel-tuning results

Post by sasachess » Sat Jun 17, 2017 10:14 am

sandermvdb wrote:
sasachess wrote:I'm very happy to read your post! I remember our chat not many days ago. :wink:
If you are the author of gogobello, yes we had that chat! :)
Thanks for pointing me in this direction!
Yes I'm! :D

Post Reply