Yet another parameter tuner using optuna framework

Discussion of anything and everything relating to chess playing software and machines.

Moderators: Harvey Williamson, bob, hgm

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Kiudee
Posts: 27
Joined: Tue Feb 02, 2010 9:12 pm

Re: Yet another parameter tuner using optuna framework

Post by Kiudee » Wed Sep 16, 2020 11:01 pm

Thanks Ferdy for making your tool available! I like some of the plots you output for the results.
I wanted to point out that Optuna uses tree-structured parzen estimators as their model, which does not model interactions between parameters. Tools based on Gaussian processes (like the chess-tuning-tools or the tool released by thomasahle) take all interactions into account and thus are able to interpolate/extrapolate much more accurately.

Ferdy
Posts: 4295
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Yet another parameter tuner using optuna framework

Post by Ferdy » Wed Sep 16, 2020 11:07 pm

Kiudee wrote:
Wed Sep 16, 2020 11:01 pm
Thanks Ferdy for making your tool available! I like some of the plots you output for the results.
I wanted to point out that Optuna uses tree-structured parzen estimators as their model, which does not model interactions between parameters. Tools based on Gaussian processes (like the chess-tuning-tools or the tool released by thomasahle) take all interactions into account and thus are able to interpolate/extrapolate much more accurately.
Thanks for the info. The one with thomasahle takes a lot of memory tried it before. I have not yet tried your chess tuning tools. Will try it someday and compare it with optuna.

User avatar
mvanthoor
Posts: 439
Joined: Wed Jul 03, 2019 2:42 pm
Full name: Marcel Vanthoor

Re: Yet another parameter tuner using optuna framework

Post by mvanthoor » Thu Sep 17, 2020 7:31 am

At some point in the future, I'll have to look into this, or other similar tools. Thanks :)

Joerg Oster
Posts: 789
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: Yet another parameter tuner using optuna framework

Post by Joerg Oster » Thu Sep 17, 2020 8:37 am

Ferdy wrote:
Wed Sep 16, 2020 11:07 pm
Kiudee wrote:
Wed Sep 16, 2020 11:01 pm
Thanks Ferdy for making your tool available! I like some of the plots you output for the results.
I wanted to point out that Optuna uses tree-structured parzen estimators as their model, which does not model interactions between parameters. Tools based on Gaussian processes (like the chess-tuning-tools or the tool released by thomasahle) take all interactions into account and thus are able to interpolate/extrapolate much more accurately.
Thanks for the info. The one with thomasahle takes a lot of memory tried it before. I have not yet tried your chess tuning tools. Will try it someday and compare it with optuna.
Nevergrad might also be an interesting alternative.
It offers a wide variety of optimization methods,
and has a nice ask and tell interface.
Jörg Oster

jdart
Posts: 4013
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: Yet another parameter tuner using optuna framework

Post by jdart » Thu Sep 17, 2020 2:47 pm

Interesting. I also found this software:

https://github.com/automl/HpBandSter

which seems kind of similar. I have tried this sort of thing before especially for search parameters, but the problem I've found is that the effect of varying these can be quite small. So you are trying to find the optimum point but basically on a very "flat" surface, and furthermore with a method that produces noisy objective measures. It is therefore hard to get convergence.

chrisw
Posts: 3810
Joined: Tue Apr 03, 2012 2:28 pm

Re: Yet another parameter tuner using optuna framework

Post by chrisw » Thu Sep 17, 2020 3:30 pm

jdart wrote:
Thu Sep 17, 2020 2:47 pm
Interesting. I also found this software:

https://github.com/automl/HpBandSter

which seems kind of similar. I have tried this sort of thing before especially for search parameters, but the problem I've found is that the effect of varying these can be quite small. So you are trying to find the optimum point but basically on a very "flat" surface, and furthermore with a method that produces noisy objective measures. It is therefore hard to get convergence.
Yup, check. I was really quite surprised at just how dull search parameters proved to be, it's almost as if 'doing' a process is the most important bit and attempts to fine tune relatively fruitless. Compared to eval tuning where results plus/minus are semi-immediate and changes have strong effects. But search is not like eval, eval is grey scale accumulative, search is full of binary decision points (do I prune or not) and although the binary can be ameliorated to some extent (by adjusting how much depth left to search and so on, rather than yes/no cut/don't) there are still plenty of points where decision is full on binary, and a number that works for one case won't work for another etc. etc. Difficult problem.

Joerg Oster
Posts: 789
Joined: Fri Mar 10, 2006 3:29 pm
Location: Germany

Re: Yet another parameter tuner using optuna framework

Post by Joerg Oster » Thu Sep 17, 2020 4:28 pm

Ferdy wrote:
Wed Sep 16, 2020 10:05 pm
Joerg Oster wrote:
Wed Sep 16, 2020 2:30 pm
Ferdy wrote:
Wed Sep 16, 2020 1:17 pm
3. I'm not sure if parameter changes to a 'quick match' change all five parameters at a time, or just one?
Sorry I don't understand the question.
I guess he wants to know if the tuner changes all parameters at once or one by one for a new trial.
In the document they refer to this as relational sampling and independent sampling.
We can ask the optimizer either one by one or all at once. I did all at once. https://github.com/fsmosca/Optuna-Game- ... ner.py#L65
Which seems to be the most reasonable, imho. :)
Jörg Oster

No4b
Posts: 32
Joined: Thu Jun 18, 2020 1:21 pm
Location: Moscow
Full name: Alexander Litov

Re: Yet another parameter tuner using optuna framework

Post by No4b » Thu Sep 17, 2020 9:21 pm

Ferdy wrote:
Wed Sep 16, 2020 10:53 pm
No4b wrote:
Wed Sep 16, 2020 6:58 pm
Very interesting tool!
I will definitely try to work with it.

Am i understanding correctly that the engine itself obtain parameters via command line fe as "QueenValueOp=975"?
Not at the moment (will add it later), you need to modify the code around here https://github.com/fsmosca/Optuna-Game- ... er.py#L197
I mean should the chess engine itself (one that i want to tune) obtain parameters via command line fe as "QueenValueOp=975" ?
I want to know what to implement to work with your tool.

It is quite clear from your info that i should just write needed params in the tuner file itself, my question is how it (the tuner) communicate this params to the engine I want to tune. I tried to looked into the tuner code and my best guess is that tuner will run engine with smth like

Code: Select all

engine.exe "QueenValueOp=975" 
(so i should write some script into the engine to process lines like this), but i`m not sure... Thats why i`m asking.

Sorry if it is obvious and i`m being stupid. Im not that good at coding.

Ferdy
Posts: 4295
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Yet another parameter tuner using optuna framework

Post by Ferdy » Thu Sep 17, 2020 10:17 pm

No4b wrote:
Thu Sep 17, 2020 9:21 pm
Ferdy wrote:
Wed Sep 16, 2020 10:53 pm
No4b wrote:
Wed Sep 16, 2020 6:58 pm
Very interesting tool!
I will definitely try to work with it.

Am i understanding correctly that the engine itself obtain parameters via command line fe as "QueenValueOp=975"?
Not at the moment (will add it later), you need to modify the code around here https://github.com/fsmosca/Optuna-Game- ... er.py#L197
I mean should the chess engine itself (one that i want to tune) obtain parameters via command line fe as "QueenValueOp=975" ?
I want to know what to implement to work with your tool.

It is quite clear from your info that i should just write needed params in the tuner file itself, my question is how it (the tuner) communicate this params to the engine I want to tune.
I use cutechess-cli to set the suggested params to the engine.

Once I get the suggested param from the optimizer, I build a text string containing,

Code: Select all

option.PawnValueEn=92 option.BishopValueOp=324
Then add that in the main cutechess-cli command line.

Code: Select all

cutechess-cli.exe -engine cmd=deuterium.exe proto=uci name=deuterium_piecevalue option.PawnValueEn=92 option.BishopValueOp=324 -engine cmd=deuterium.exe proto=uci name=deuterium_default ...
It is at https://github.com/fsmosca/Optuna-Game- ... ner.py#L63

Ferdy
Posts: 4295
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Yet another parameter tuner using optuna framework

Post by Ferdy » Thu Sep 17, 2020 11:25 pm

@Joerg
Could you create a Stockfish where it exposes its material piece value and other evaluation param perhaps and a couple of search parameters. I can make it but may take sometime. I want to test optuna with it and later compare with other tuner like spsa and chess tuning tools and others perhaps. Stockfish params are well optimized, want to see how many elo optuna can increase or decrease. Or unoptimize it and try to optimize it back. Don't put a space in the uci option name and build an exe for modern. You can use the one with nnue but I prefer the one without it. Even a revised Stockfish 11 would do. Thanks.

Post Reply