Yet another parameter tuner using optuna framework

[Ferdy]

Created a parameter tuner which might be useful to your chess engine.

https://github.com/fsmosca/Optuna-Game-Parameter-Tuner

I got a promising optimization result on my engine Deuterium at https://github.com/fsmosca/Optuna-Game- ... mization-3
I thought with only couple of piece values it can at least equalize my default but it actually defeated my default.

The parameter to be optimized is hard-coded, you have to modify the tuner.py. The tournament manager is cutechess-cli which can be found in tourney_manager/cutechess folder. See the Setup section, you need to install those dependencies.

It is based on optuna framework at https://github.com/optuna/optuna

[Joerg Oster]

Thank you!
Looks very interesting.

[chrisw]

Created a parameter tuner which might be useful to your chess engine.

https://github.com/fsmosca/Optuna-Game-Parameter-Tuner

I got a promising optimization result on my engine Deuterium at https://github.com/fsmosca/Optuna-Game- ... mization-3
I thought with only couple of piece values it can at least equalize my default but it actually defeated my default.

The parameter to be optimized is hard-coded, you have to modify the tuner.py. The tournament manager is cutechess-cli which can be found in tourney_manager/cutechess folder. See the Setup section, you need to install those dependencies.

It is based on optuna framework at https://github.com/optuna/optuna

This looks extremely brilliant. I took a look at the Optuna site docs also. Trying to work out 'how it works', tell me if I understand it right, probably I don't of course.

1. You have a base engine, e1 and same engine, e2, except you can send parameter change instructions to e2

2. Uses cutechess tournament mode, with output the result of a quick match (25 rounds?)

3. I'm not sure if parameter changes to a 'quick match' change all five parameters at a time, or just one?

4. Optuna tries parameter changes (within your supplied range?), and using smart algorithm, decides where to concentrate it's fire, so to speak.

5. After N iterations, it gives you new values for the selected parameters.

One thing I don't understand are the supplied output graphs, and what they are showing (apart from Optuna deciding that PAWNVAlUE is the 'most important' tunable item.

One other thing. The Optuna presentation says it can handle the functions (cutechess output) in parallel, are you using that feature, or is cutechess script already running the tournament in parallel?

[Ferdy]

This looks extremely brilliant. I took a look at the Optuna site docs also. Trying to work out 'how it works', tell me if I understand it right, probably I don't of course.

1. You have a base engine, e1 and same engine, e2, except you can send parameter change instructions to e2

Yes they are the same engine, e1 which I call test engine and e2 as base engine. The e1 will always take the optimizer suggested param values. The e2 will always take the best param values. After a match if e1 wins that is more than 50% score, the param values used by e1 will become the best param values. This best param values then will be used by e2 in the next match. The e1 will take the new suggested values from the optimizer. The score of the match must be reported to the optimizer before taking its suggestions.

2. Uses cutechess tournament mode, with output the result of a quick match (25 rounds?)

Yes, number of game is settable, the more games the better.

3. I'm not sure if parameter changes to a 'quick match' change all five parameters at a time, or just one?

Sorry I don't understand the question.

4. Optuna tries parameter changes (within your supplied range?), and using smart algorithm, decides where to concentrate it's fire, so to speak.

Yes.

5. After N iterations, it gives you new values for the selected parameters.

Yes. They call it trials. A trial can be a 24-game match. After a trial we send the score of the match to the optimizer. After that it is safe to ask for the new suggested param values. Around here https://github.com/fsmosca/Optuna-Game- ... ner.py#L65

One thing I don't understand are the supplied output graphs, and what they are showing (apart from Optuna deciding that PAWNVAlUE is the 'most important' tunable item.

Slice plot: The y-axis is the score of the engine match, see the objective value label. The x-axis is the value of the parameter that has been tried. The color of the dot is the number of trials, trial number is high if it is dark, see the vertical legend at right. In here we can see if the tuner is making more suggestions that improves the objective value per parameter plot. We can also see here at what range of values has more dots or samples suggested. We can always see the y-axis if the dot performed well or not.


The contour: There is a vertical bar legend at right. The lighter it is, the better. It is the match result of the game match or objective value. Let's take a look at the plot in the intersection of y-axis QueenValueOp and the x-axis RookValueEn. See the light area around south-west, that is the area where QueenValueOp and RookValueEn has good score in the match. The dark area in the plot means that the result of the match on those parameter values are not good.

One other thing. The Optuna presentation says it can handle the functions (cutechess output) in parallel, are you using that feature, or is cutechess script already running the tournament in parallel?

I am using cutechess concurrency to run more than 1 game per trial or match.

I just successfully optimized the king safety weight, mobility weight and passer weight. I have tried improving it before without success but with optuna, I get promising result. See the method I use here. https://github.com/fsmosca/Optuna-Game- ... timization.

[Joerg Oster]

3. I'm not sure if parameter changes to a 'quick match' change all five parameters at a time, or just one?

Sorry I don't understand the question.

I guess he wants to know if the tuner changes all parameters at once or one by one for a new trial.
In the document they refer to this as relational sampling and independent sampling.

[Joerg Oster]

2. Uses cutechess tournament mode, with output the result of a quick match (25 rounds?)

Yes, number of game is settable, the more games the better.

The number of games per trial is probably dependent on the sensitivity of the parameters, no?
24 games seems very small in any case.

Btw, do you make use of the Pruning Algorithm by sending intermediate results of a trial?

[chrisw]

3. I'm not sure if parameter changes to a 'quick match' change all five parameters at a time, or just one?

Sorry I don't understand the question.

I guess he wants to know if the tuner changes all parameters at once or one by one for a new trial.
In the document they refer to this as relational sampling and independent sampling.

Yup, pretty much what I meant

[chrisw]

This looks extremely brilliant. I took a look at the Optuna site docs also. Trying to work out 'how it works', tell me if I understand it right, probably I don't of course.

1. You have a base engine, e1 and same engine, e2, except you can send parameter change instructions to e2

Yes they are the same engine, e1 which I call test engine and e2 as base engine. The e1 will always take the optimizer suggested param values. The e2 will always take the best param values. After a match if e1 wins that is more than 50% score, the param values used by e1 will become the best param values. This best param values then will be used by e2 in the next match. The e1 will take the new suggested values from the optimizer. The score of the match must be reported to the optimizer before taking its suggestions.

2. Uses cutechess tournament mode, with output the result of a quick match (25 rounds?)

Yes, number of game is settable, the more games the better.

3. I'm not sure if parameter changes to a 'quick match' change all five parameters at a time, or just one?

Sorry I don't understand the question.

As Georg Oster reply post.

4. Optuna tries parameter changes (within your supplied range?), and using smart algorithm, decides where to concentrate it's fire, so to speak.

Yes.

5. After N iterations, it gives you new values for the selected parameters.

Yes. They call it trials. A trial can be a 24-game match. After a trial we send the score of the match to the optimizer. After that it is safe to ask for the new suggested param values. Around here https://github.com/fsmosca/Optuna-Game- ... ner.py#L65

One thing I don't understand are the supplied output graphs, and what they are showing (apart from Optuna deciding that PAWNVAlUE is the 'most important' tunable item.

Slice plot: The y-axis is the score of the engine match, see the objective value label. The x-axis is the value of the parameter that has been tried. The color of the dot is the number of trials, trial number is high if it is dark, see the vertical legend at right. In here we can see if the tuner is making more suggestions that improves the objective value per parameter plot. We can also see here at what range of values has more dots or samples suggested. We can always see the y-axis if the dot performed well or not.


The contour: There is a vertical bar legend at right. The lighter it is, the better. It is the match result of the game match or objective value. Let's take a look at the plot in the intersection of y-axis QueenValueOp and the x-axis RookValueEn. See the light area around south-west, that is the area where QueenValueOp and RookValueEn has good score in the match. The dark area in the plot means that the result of the match on those parameter values are not good.

One other thing. The Optuna presentation says it can handle the functions (cutechess output) in parallel, are you using that feature, or is cutechess script already running the tournament in parallel?

I am using cutechess concurrency to run more than 1 game per trial or match.

I just successfully optimized the king safety weight, mobility weight and passer weight. I have tried improving it before without success but with optuna, I get promising result. See the method I use here. https://github.com/fsmosca/Optuna-Game- ... timization.

Looks very good. I still need to wrap my head around the graphs, will try later.

One thing, from your Github (and I don't want to flood you with suggested mods, I know how irritating that can be):

Second in order for the parameter values to be considered the best and replace the old best, it has to defeat the old best by more than 0.55 or 55% score. Normally this is only 0.5 or 50%.

I tried something like this in the past (except with random kicks to the parameters, not smart ones as you are doing with Optima), where the parameters, P1, give a better result than P0 - was to move P fractionally towards P1, rather than the full thing.
This could work in the case of the close 50-55% range. Just an idea, but I expect you are full of ideas already!

[Joerg Oster]

3. I'm not sure if parameter changes to a 'quick match' change all five parameters at a time, or just one?

Sorry I don't understand the question.

I guess he wants to know if the tuner changes all parameters at once or one by one for a new trial.
In the document they refer to this as relational sampling and independent sampling.

Yup, pretty much what I meant

If I understand it correctly, Optuna will eventually do both and also a mixture of both, to find out about the correlation of the parameters.

[No4b]

Very interesting tool!
I will definitely try to work with it.

Am i understanding correctly that the engine itself obtain parameters via command line fe as "QueenValueOp=975"?
And I need only one copy of the engine in the folder, tuner just will execute two copies of it with a different parameters set?