Page 1 of 4
Laskas parameter optimizer
Posted: Wed Dec 09, 2020 2:55 am
by Ferdy
You may try
Lakas using
nevergrad framework to optimize you engine's search and evaluation parameters.
Sample
command line from
optimization comparison. Help is
here.
Optimizers that are supported so far.
Re: Lakas parameter optimizer
Posted: Thu Jan 07, 2021 5:35 am
by Ferdy
Re: Lakas parameter optimizer
Posted: Thu Jan 07, 2021 11:14 am
by Joerg Oster
Will you give them a try at Fishtest?
Re: Lakas parameter optimizer
Posted: Thu Jan 07, 2021 4:13 pm
by Ferdy
Joerg Oster wrote: ↑Thu Jan 07, 2021 11:14 am
Will you give them a try at Fishtest?
What should I do, I am not familiar with Fishtest.
Re: Lakas parameter optimizer
Posted: Thu Jan 07, 2021 6:10 pm
by Joerg Oster
Ferdy wrote: ↑Thu Jan 07, 2021 4:13 pm
Joerg Oster wrote: ↑Thu Jan 07, 2021 11:14 am
Will you give them a try at Fishtest?
What should I do, I am not familiar with Fishtest.
Fork the official repo.
https://github.com/official-stockfish/Stockfish
Create a new branch with your changes.
Create an account at Fishtest.
https://tests.stockfishchess.org/tests
Submit your test.
This guide
https://github.com/glinscott/fishtest/w ... first-test may be of help.
Re: Laskas parameter optimizer
Posted: Sat Jan 23, 2021 11:25 am
by Patrice Duhamel
I tried Lakas with Cheese, to optimize the mobility for knight, bishop, rook and queen. (score = (mobility * scale) / 8 - translate)
I ran Lakas 4 times (for each pieces), 4 parameters by piece, and I used the same options as the example but for only 30 budgets and 100 games / budgets, at 10s+0.1s.
Result with tuned mobility : 2500 games at 10s+0.1s :
Code: Select all
Rank Name Elo + - games score oppo. draws
1 Cheese-30-dev 4 6 6 2500 51% -4 43%
2 Cheese-30-dev-optimized -4 6 6 2500 49% 4 43%
Should I run Lakas with all 16 parameters at once ? and use more than 30 budgets ?
I think 100 games/budget are not enough, but it already takes lots of time (I can use only 4 threads).
Re: Laskas parameter optimizer
Posted: Sat Jan 23, 2021 11:55 am
by Ferdy
Patrice Duhamel wrote: ↑Sat Jan 23, 2021 11:25 am
I tried Lakas with Cheese, to optimize the mobility for knight, bishop, rook and queen. (score = (mobility * scale) / 8 - translate)
I ran Lakas 4 times (for each pieces), 4 parameters by piece, and I used the same options as the example but for only 30 budgets and 100 games / budgets, at 10s+0.1s.
Result with tuned mobility : 2500 games at 10s+0.1s :
Code: Select all
Rank Name Elo + - games score oppo. draws
1 Cheese-30-dev 4 6 6 2500 51% -4 43%
2 Cheese-30-dev-optimized -4 6 6 2500 49% 4 43%
Should I run Lakas with all 16 parameters at once ? and use more than 30 budgets ?
I think 100 games/budget are not enough, but it already takes lots of time (I can use only 4 threads).
What do you mean by "I ran Lakas 4 times (for each pieces), 4 parameters by piece,"?
Should I run Lakas with all 16 parameters at once ? and use more than 30 budgets ?
Yes you can do that.
More games per budget is always better and more budgets are also better especially if number of parameters are high. These optimizations are really expensive.
Re: Laskas parameter optimizer
Posted: Sat Jan 23, 2021 12:21 pm
by Patrice Duhamel
Ferdy wrote: ↑Sat Jan 23, 2021 11:55 am
What do you mean by "I ran Lakas 4 times (for each pieces), 4 parameters by piece,"?
I ran it to optimize Knight mobility, then Bishop mobility, rook, and queen. (4 parameters by piece)
Because with 16 parameters at once I guess it needs more budgets and it takes too much time.
It's better to use more parameters at the same time ?
I will try again later when I change my computer.
Re: Laskas parameter optimizer
Posted: Sat Jan 23, 2021 12:50 pm
by Ferdy
Patrice Duhamel wrote: ↑Sat Jan 23, 2021 12:21 pm
Ferdy wrote: ↑Sat Jan 23, 2021 11:55 am
What do you mean by "I ran Lakas 4 times (for each pieces), 4 parameters by piece,"?
I ran it to optimize Knight mobility, then Bishop mobility, rook, and queen. (4 parameters by piece)
Because with 16 parameters at once I guess it needs more budgets and it takes too much time.
It's better to use more parameters at the same time ?
Yes.
It has interrupt and resume feature. If you are busy at one point, just exit the tuning. Then continue with same command line that you had later. Better to save the command line in a batch file.
Include these command line options on your command line.
Code: Select all
--input-data-file out1.dat --output-data-file out1.dat
As tuning progresses, it will save the tuning data to out1.dat.
At startup it will read out1.dat and the optimizer would adjust its algo based on history on the out1.dat.
Re: Laskas parameter optimizer
Posted: Sat Jan 23, 2021 4:28 pm
by xr_a_y
Hi, great scripts. Thanks.
A few suggestions :
- try to not call "python" directly inside the script (not working when python interpreter is named python for instance)
- make cutechess executable a parameter
- specifying depth and a time might confuse some engine (but I agree UCI protocol is respected)
- I think, for python3 at least, the popen shall look like this
Code: Select all
process = Popen((str(tour_manager) + command).split(' '), stdout=PIPE, text=True)
I'm trying it now, will report if it gives some Elo.