Page 1 of 1

Impressive Texel-tuning results

Posted: Fri Jun 16, 2017 5:49 pm
by sandermvdb
About a month ago I tried to tune my evaluation values using the Texel-tuning method:

First I thought about how to create fens of millions of positions which should contain the actual result of that match. I used cute-chess at very fast time controls and pgn-extractor which created 3 files containing the positions: wins.epd, lost.epd and draws.epd.

Then I wrote some logic to import these fens and I implemented a local optimization routine as described on the cpw. My optimization routine is able to tune multiple arrays and per array a 'step' can be specified. I also made it possible to specify that certain values in the array can be skipped, values can be mirrored (left, right) and should sometimes be applied to both the white and black arrays.

The cpw describes that you should use the quiescence-search for this method but I wasn't happy with the speed of calculating the error which took about 20 seconds (see: http://talkchess.com/forum/viewtopic.php?t=64189). I also read that the author of zurichess used the Texel-tuning method and he had created several epd files which could be used as a testset: http://www.talkchess.com/forum/viewtopic.php?t=61427
This zip-file also contains a quiet.epd which only contains quiet positions so I thought that using this set I don't need the quiescence-search but I can call the evaluation funtion directly. This is way faster and can quite easily be made multi-threaded, which is what I did.

Next I calculated the scaling-factor which would give the lowest error: 1.3

My original evaluation function had an error of 0.0654. Tuned I am able to get it to 0.0586. These are the (sometimes obvious) major tweaks:

Code: Select all

- rook material          : 500 -> 645
- queen material         : 950 -> 1200
- pinned-pieces-penalty  : always 15 -> 0-80, depending on the pinned-piece type
- pawn-storm-bonus       : disabled
- pawn-shield-bonus      : way higher scores when the pawn is almost promoted, now the king is defending the pawn! :)
- king-safety-scores     : start at 50, then decreases, then increases, no clue why :P
- mobility               : bigger penalties and bonusses
- psqt                   : subtle but relevant tweaks
All in all this give an Elo increase of more than 100 elo points! See table below for prelimenary results (chess22k-ex has tuned values, chess22k has same logic without tuned values)

Code: Select all

Rank Name                          Elo     +/-   Games   Score   Draws
   1 chess22k-ex                   146      45     191   69.9%   26.7%
   2 Ruffian_105                    31      42     193   54.4%   29.0%
   3 Maverick 1.5                   -7      42     192   49.0%   29.2%
   4 chess22k                      -20      41     192   47.1%   29.7%
   5 AnMon_5.75                    -60      42     193   41.5%   26.9%
   6 chess22k 1.3                  -83      40     191   38.2%   35.6%
576 of 1500 games finished.

Some other things I like is that you can easily see which evaluation values are actually not used.
It is now also easy to add a new feature to the evalution function and see how it performs (assuming that a lower error values results in a higher elo-score).

I guess chess22k 1.4 will be released soon! :)

Re: Impressive Texel-tuning results

Posted: Fri Jun 16, 2017 8:29 pm
by sasachess
I'm very happy to read your post! I remember our chat not many days ago. :wink:

Re: Impressive Texel-tuning results

Posted: Fri Jun 16, 2017 9:01 pm
by sandermvdb
sasachess wrote:I'm very happy to read your post! I remember our chat not many days ago. :wink:
If you are the author of gogobello, yes we had that chat! :)
Thanks for pointing me in this direction!

Re: Impressive Texel-tuning results

Posted: Sat Jun 17, 2017 5:34 am
by sandermvdb
Tournament is finished:

Code: Select all

Rank Name                          Elo     +/-   Games   Score   Draws
   1 chess22k-ex                   137      27     500   68.7%   26.6%
   2 Ruffian_105                    31      26     500   54.5%   28.2%
   3 Maverick 1.5                   -5      26     500   49.3%   26.2%
   4 chess22k                      -33      26     500   45.3%   30.2%
   5 AnMon_5.75                    -58      27     500   41.7%   23.8%
   6 chess22k 1.3                  -67      26     500   40.5%   31.0%

1500 of 1500 games finished.

Re: Impressive Texel-tuning results

Posted: Sat Jun 17, 2017 8:08 am
by brtzsnr
Congrats.

I'm happy the training set worked for you. I believe you are the forth person (including me) reporting success with it. If you don't mind I will reference this thread somewhere on the repository.

I've been trying to improve that training set for the past few months with no success. I tried to add some non-quiet positions (and fixing the evaluation) but it was still a Elo loss.

Re: Impressive Texel-tuning results

Posted: Sat Jun 17, 2017 9:49 am
by sandermvdb
brtzsnr wrote:Congrats.

I'm happy the training set worked for you. I believe you are the forth person (including me) reporting success with it. If you don't mind I will reference this thread somewhere on the repository.

I've been trying to improve that training set for the past few months with no success. I tried to add some non-quiet positions (and fixing the evaluation) but it was still a Elo loss.
I don't mind and thanks for the set!
Please keep us informed if you have a better set :)

Re: Impressive Texel-tuning results

Posted: Sat Jun 17, 2017 10:14 am
by sasachess
sandermvdb wrote:
sasachess wrote:I'm very happy to read your post! I remember our chat not many days ago. :wink:
If you are the author of gogobello, yes we had that chat! :)
Thanks for pointing me in this direction!
Yes I'm! :D