You may try Lakas using nevergrad framework to optimize you engine's search and evaluation parameters.
Sample command line from optimization comparison. Help is here. Optimizers that are supported so far.
Laskas parameter optimizer
Moderators: hgm, Rebel, chrisw
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Lakas parameter optimizer
Run an optimization combining search and evaluation parameters with promising result.
-
- Posts: 937
- Joined: Fri Mar 10, 2006 4:29 pm
- Location: Germany
Re: Lakas parameter optimizer
Will you give them a try at Fishtest?Ferdy wrote: ↑Thu Jan 07, 2021 5:35 am Run an optimization combining search and evaluation parameters with promising result.
Jörg Oster
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Lakas parameter optimizer
What should I do, I am not familiar with Fishtest.Joerg Oster wrote: ↑Thu Jan 07, 2021 11:14 amWill you give them a try at Fishtest?Ferdy wrote: ↑Thu Jan 07, 2021 5:35 am Run an optimization combining search and evaluation parameters with promising result.
-
- Posts: 937
- Joined: Fri Mar 10, 2006 4:29 pm
- Location: Germany
Re: Lakas parameter optimizer
Fork the official repo. https://github.com/official-stockfish/StockfishFerdy wrote: ↑Thu Jan 07, 2021 4:13 pmWhat should I do, I am not familiar with Fishtest.Joerg Oster wrote: ↑Thu Jan 07, 2021 11:14 amWill you give them a try at Fishtest?Ferdy wrote: ↑Thu Jan 07, 2021 5:35 am Run an optimization combining search and evaluation parameters with promising result.
Create a new branch with your changes.
Create an account at Fishtest. https://tests.stockfishchess.org/tests
Submit your test.
This guide https://github.com/glinscott/fishtest/w ... first-test may be of help.
Jörg Oster
-
- Posts: 193
- Joined: Sat May 25, 2013 11:17 am
- Location: France
- Full name: Patrice Duhamel
Re: Laskas parameter optimizer
I tried Lakas with Cheese, to optimize the mobility for knight, bishop, rook and queen. (score = (mobility * scale) / 8 - translate)
I ran Lakas 4 times (for each pieces), 4 parameters by piece, and I used the same options as the example but for only 30 budgets and 100 games / budgets, at 10s+0.1s.
Result with tuned mobility : 2500 games at 10s+0.1s :
Should I run Lakas with all 16 parameters at once ? and use more than 30 budgets ?
I think 100 games/budget are not enough, but it already takes lots of time (I can use only 4 threads).
I ran Lakas 4 times (for each pieces), 4 parameters by piece, and I used the same options as the example but for only 30 budgets and 100 games / budgets, at 10s+0.1s.
Result with tuned mobility : 2500 games at 10s+0.1s :
Code: Select all
Rank Name Elo + - games score oppo. draws
1 Cheese-30-dev 4 6 6 2500 51% -4 43%
2 Cheese-30-dev-optimized -4 6 6 2500 49% 4 43%
I think 100 games/budget are not enough, but it already takes lots of time (I can use only 4 threads).
Anything that can go wrong will go wrong.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Laskas parameter optimizer
What do you mean by "I ran Lakas 4 times (for each pieces), 4 parameters by piece,"?Patrice Duhamel wrote: ↑Sat Jan 23, 2021 11:25 am I tried Lakas with Cheese, to optimize the mobility for knight, bishop, rook and queen. (score = (mobility * scale) / 8 - translate)
I ran Lakas 4 times (for each pieces), 4 parameters by piece, and I used the same options as the example but for only 30 budgets and 100 games / budgets, at 10s+0.1s.
Result with tuned mobility : 2500 games at 10s+0.1s :Should I run Lakas with all 16 parameters at once ? and use more than 30 budgets ?Code: Select all
Rank Name Elo + - games score oppo. draws 1 Cheese-30-dev 4 6 6 2500 51% -4 43% 2 Cheese-30-dev-optimized -4 6 6 2500 49% 4 43%
I think 100 games/budget are not enough, but it already takes lots of time (I can use only 4 threads).
Yes you can do that.Should I run Lakas with all 16 parameters at once ? and use more than 30 budgets ?
More games per budget is always better and more budgets are also better especially if number of parameters are high. These optimizations are really expensive.
-
- Posts: 193
- Joined: Sat May 25, 2013 11:17 am
- Location: France
- Full name: Patrice Duhamel
Re: Laskas parameter optimizer
I ran it to optimize Knight mobility, then Bishop mobility, rook, and queen. (4 parameters by piece)
Because with 16 parameters at once I guess it needs more budgets and it takes too much time.
It's better to use more parameters at the same time ?
I will try again later when I change my computer.
Anything that can go wrong will go wrong.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Laskas parameter optimizer
Yes.Patrice Duhamel wrote: ↑Sat Jan 23, 2021 12:21 pmI ran it to optimize Knight mobility, then Bishop mobility, rook, and queen. (4 parameters by piece)
Because with 16 parameters at once I guess it needs more budgets and it takes too much time.
It's better to use more parameters at the same time ?
It has interrupt and resume feature. If you are busy at one point, just exit the tuning. Then continue with same command line that you had later. Better to save the command line in a batch file.
Include these command line options on your command line.
Code: Select all
--input-data-file out1.dat --output-data-file out1.dat
At startup it will read out1.dat and the optimizer would adjust its algo based on history on the out1.dat.
-
- Posts: 1871
- Joined: Sat Nov 25, 2017 2:28 pm
- Location: France
Re: Laskas parameter optimizer
Hi, great scripts. Thanks.
A few suggestions :
- try to not call "python" directly inside the script (not working when python interpreter is named python for instance)
- make cutechess executable a parameter
- specifying depth and a time might confuse some engine (but I agree UCI protocol is respected)
- I think, for python3 at least, the popen shall look like this
I'm trying it now, will report if it gives some Elo.
A few suggestions :
- try to not call "python" directly inside the script (not working when python interpreter is named python for instance)
- make cutechess executable a parameter
- specifying depth and a time might confuse some engine (but I agree UCI protocol is respected)
- I think, for python3 at least, the popen shall look like this
Code: Select all
process = Popen((str(tour_manager) + command).split(' '), stdout=PIPE, text=True)