Laskas parameter optimizer

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Ferdy
Posts: 4527
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Laskas parameter optimizer

Post by Ferdy » Wed Dec 09, 2020 1:55 am

You may try Lakas using nevergrad framework to optimize you engine's search and evaluation parameters.

Sample command line from optimization comparison. Help is here. Optimizers that are supported so far.

Ferdy
Posts: 4527
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Lakas parameter optimizer

Post by Ferdy » Thu Jan 07, 2021 4:35 am


Joerg Oster
Posts: 815
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: Lakas parameter optimizer

Post by Joerg Oster » Thu Jan 07, 2021 10:14 am

Ferdy wrote:
Thu Jan 07, 2021 4:35 am
Run an optimization combining search and evaluation parameters with promising result.
Will you give them a try at Fishtest?
Jörg Oster

Ferdy
Posts: 4527
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Lakas parameter optimizer

Post by Ferdy » Thu Jan 07, 2021 3:13 pm

Joerg Oster wrote:
Thu Jan 07, 2021 10:14 am
Ferdy wrote:
Thu Jan 07, 2021 4:35 am
Run an optimization combining search and evaluation parameters with promising result.
Will you give them a try at Fishtest?
What should I do, I am not familiar with Fishtest.

Joerg Oster
Posts: 815
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: Lakas parameter optimizer

Post by Joerg Oster » Thu Jan 07, 2021 5:10 pm

Ferdy wrote:
Thu Jan 07, 2021 3:13 pm
Joerg Oster wrote:
Thu Jan 07, 2021 10:14 am
Ferdy wrote:
Thu Jan 07, 2021 4:35 am
Run an optimization combining search and evaluation parameters with promising result.
Will you give them a try at Fishtest?
What should I do, I am not familiar with Fishtest.
Fork the official repo. https://github.com/official-stockfish/Stockfish
Create a new branch with your changes.
Create an account at Fishtest. https://tests.stockfishchess.org/tests
Submit your test.

This guide https://github.com/glinscott/fishtest/w ... first-test may be of help.
Jörg Oster

Patrice Duhamel
Posts: 157
Joined: Sat May 25, 2013 9:17 am
Location: France
Full name: Patrice Duhamel
Contact:

Re: Laskas parameter optimizer

Post by Patrice Duhamel » Sat Jan 23, 2021 10:25 am

I tried Lakas with Cheese, to optimize the mobility for knight, bishop, rook and queen. (score = (mobility * scale) / 8 - translate)
I ran Lakas 4 times (for each pieces), 4 parameters by piece, and I used the same options as the example but for only 30 budgets and 100 games / budgets, at 10s+0.1s.

Result with tuned mobility : 2500 games at 10s+0.1s :

Code: Select all

Rank Name                      Elo    +    - games score oppo. draws 
   1 Cheese-30-dev               4    6    6  2500   51%    -4   43% 
   2 Cheese-30-dev-optimized    -4    6    6  2500   49%     4   43% 
Should I run Lakas with all 16 parameters at once ? and use more than 30 budgets ?
I think 100 games/budget are not enough, but it already takes lots of time (I can use only 4 threads).
Anything that can go wrong will go wrong.

Ferdy
Posts: 4527
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Laskas parameter optimizer

Post by Ferdy » Sat Jan 23, 2021 10:55 am

Patrice Duhamel wrote:
Sat Jan 23, 2021 10:25 am
I tried Lakas with Cheese, to optimize the mobility for knight, bishop, rook and queen. (score = (mobility * scale) / 8 - translate)
I ran Lakas 4 times (for each pieces), 4 parameters by piece, and I used the same options as the example but for only 30 budgets and 100 games / budgets, at 10s+0.1s.

Result with tuned mobility : 2500 games at 10s+0.1s :

Code: Select all

Rank Name                      Elo    +    - games score oppo. draws 
   1 Cheese-30-dev               4    6    6  2500   51%    -4   43% 
   2 Cheese-30-dev-optimized    -4    6    6  2500   49%     4   43% 
Should I run Lakas with all 16 parameters at once ? and use more than 30 budgets ?
I think 100 games/budget are not enough, but it already takes lots of time (I can use only 4 threads).
What do you mean by "I ran Lakas 4 times (for each pieces), 4 parameters by piece,"?
Should I run Lakas with all 16 parameters at once ? and use more than 30 budgets ?
Yes you can do that.

More games per budget is always better and more budgets are also better especially if number of parameters are high. These optimizations are really expensive.

Patrice Duhamel
Posts: 157
Joined: Sat May 25, 2013 9:17 am
Location: France
Full name: Patrice Duhamel
Contact:

Re: Laskas parameter optimizer

Post by Patrice Duhamel » Sat Jan 23, 2021 11:21 am

Ferdy wrote:
Sat Jan 23, 2021 10:55 am
What do you mean by "I ran Lakas 4 times (for each pieces), 4 parameters by piece,"?
I ran it to optimize Knight mobility, then Bishop mobility, rook, and queen. (4 parameters by piece)
Because with 16 parameters at once I guess it needs more budgets and it takes too much time.

It's better to use more parameters at the same time ?

I will try again later when I change my computer.
Anything that can go wrong will go wrong.

Ferdy
Posts: 4527
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Laskas parameter optimizer

Post by Ferdy » Sat Jan 23, 2021 11:50 am

Patrice Duhamel wrote:
Sat Jan 23, 2021 11:21 am
Ferdy wrote:
Sat Jan 23, 2021 10:55 am
What do you mean by "I ran Lakas 4 times (for each pieces), 4 parameters by piece,"?
I ran it to optimize Knight mobility, then Bishop mobility, rook, and queen. (4 parameters by piece)
Because with 16 parameters at once I guess it needs more budgets and it takes too much time.

It's better to use more parameters at the same time ?
Yes.

It has interrupt and resume feature. If you are busy at one point, just exit the tuning. Then continue with same command line that you had later. Better to save the command line in a batch file.

Include these command line options on your command line.

Code: Select all

--input-data-file out1.dat --output-data-file out1.dat
As tuning progresses, it will save the tuning data to out1.dat.
At startup it will read out1.dat and the optimizer would adjust its algo based on history on the out1.dat.

User avatar
xr_a_y
Posts: 1465
Joined: Sat Nov 25, 2017 1:28 pm
Location: France

Re: Laskas parameter optimizer

Post by xr_a_y » Sat Jan 23, 2021 3:28 pm

Hi, great scripts. Thanks.

A few suggestions :
- try to not call "python" directly inside the script (not working when python interpreter is named python for instance)
- make cutechess executable a parameter
- specifying depth and a time might confuse some engine (but I agree UCI protocol is respected)
- I think, for python3 at least, the popen shall look like this

Code: Select all

     process = Popen((str(tour_manager) + command).split(' '), stdout=PIPE, text=True)
I'm trying it now, will report if it gives some Elo.

Post Reply