Ratosh wrote: ↑Fri Aug 30, 2019 9:09 am
It's a nice project, but i had issues:
- ahash from log file should not consider number of games and even concurrency, that way would be easy to run a 200 game session run using 100 games from previous run.
- Let me use options that are not seen as UCI options, i don't wanna to expose all possible tunable constants as options. I just need to define them using tool input options.
- Can't make it work with more than 100 games, tested with 6 options.
I have pushed a new version that should fix these issues. Thank you for the suggestions!
It turns out that skopt saves all previously "fitted models" for the gaussian optimization. I now delete all but the latest, which will hopefully mitigate Joerg Oster's memory leak problem.
Ratosh wrote: ↑Fri Aug 30, 2019 9:09 am
It's a nice project, but i had issues:
- ahash from log file should not consider number of games and even concurrency, that way would be easy to run a 200 game session run using 100 games from previous run.
- Let me use options that are not seen as UCI options, i don't wanna to expose all possible tunable constants as options. I just need to define them using tool input options.
- Can't make it work with more than 100 games, tested with 6 options.
I have pushed a new version that should fix these issues. Thank you for the suggestions!
It turns out that skopt saves all previously "fitted models" for the gaussian optimization. I now delete all but the latest, which will hopefully mitigate Joerg Oster's memory leak problem.
jorose wrote: ↑Fri Aug 30, 2019 7:08 am
Just wanted to mention I was successfully able to retune Winter's futility and static null move pruning margins with this tool. I measured a self play rating gain of +9 +- 3 Elo roughly.
How many games did you have to run during tunning to get this elo? I am trying to tune SEE evals in Igel, not clear yet how many games I need to run. Also, I presume for search it is better to use nodes limit per move I assume?
voffka wrote: ↑Thu Sep 05, 2019 3:16 am
thomasahle,
Very nice tool! Thx a lot for sharing it.
jorose,
jorose wrote: ↑Fri Aug 30, 2019 7:08 am
Just wanted to mention I was successfully able to retune Winter's futility and static null move pruning margins with this tool. I measured a self play rating gain of +9 +- 3 Elo roughly.
How many games did you have to run during tunning to get this elo? I am trying to tune SEE evals in Igel, not clear yet how many games I need to run. Also, I presume for search it is better to use nodes limit per move I assume?
Thanks!
I ran it for roughly 800 games if I recall correctly. I was running the games with time per move.
voffka wrote: ↑Thu Sep 05, 2019 9:57 am
Currently I am running 5000 games with nodes=800 to tune 4 parameters at once. I will report the status here once it is done.
Note that if you are doing that many games, it might be better to use the GBRT estimator by Jorg's experiements. This is done with the `-base-estimator GBRT` flag. See https://github.com/thomasahle/fastchess/issues/10 for the discussion.
I'm trying it now, using xboard protocol, and I'm not really successful because python-chess seems to be rejecting "usermove=1".
Anyone experience this before ?
I'm trying it now, using xboard protocol, and I'm not really successful because python-chess seems to be rejecting "usermove=1".
Anyone experience this before ?
Soon I'll be trying it with Myrddin, also an xboard engine. Hoping you have some success!
I'm trying it now, using xboard protocol, and I'm not really successful because python-chess seems to be rejecting "usermove=1".
Anyone experience this before ?
Soon I'll be trying it with Myrddin, also an xboard engine. Hoping you have some success!
Nope ... just switched to UCI and it works well. I'm trying to tune qfutilityMargin right now ... we'll see
I've done some kind of generic option implementation for both UCI and Xboard (and still compatible with my json / CLI thing).
