Tool for automatic black-box parameter optimization released

[Rémi Coulom]

Thanks a lot. Exactly the tool I was looking for for some time.
I haven't tried it yet but my plan is to hook it to cutechess-cli/bayeselo combo I am already using, and use a cluster to run many games in parallel.
Will this setup work? (probably GCP used it already). Thanks

Yes QLR is designed for cluster use. When you run the dummy program that comes with QLR without arguments, it explains the principle:

Code: Select all

 This is an example program for use with parallel optimization. In order to
 apply qlr algorithms to your own problem, you should write a program that
 behaves like this one.

 Arguments are:
  #1: processor id (symbolic name, typically a machine name to ssh to)
  #2: seed (integer)
  #3: parameter id of first parameter (symbolic name)
  #4: value of first parameter (float)
  #5: parameter id of second parameter (optional)
  #6: value of second parameter (optional)
  ...

 This program should write the game outcome to its output:
  W = win
  L = loss
  D = draw

 For instance:
  $ ./Dummy node-01 4 param 0.2
  W

Rémi

[Look]

Probably the quickest solution would be to take an existing UI like Winboard that handles the engine match.
So what is needed is a script that calls winboard (with the match settings as parameters) and finally parses the output pgn file created for the outcome of the match.

I took the other approach, Namely using an executable with pipes and calling cutechess-cli from it and read result from its (console) output, but unfortunately this seems way slower by a factor of something like 3, so as you said maybe best is to take code of a UI and change it according to specifications. I suppose cutechess-cli is best, since it is reasonably fast and AFAIK supports both WB and UCI. But maybe I can not play around with it, since internet connection is slow and apparently it requires Qt libraries which are huge. Even having setup cutechess-cli, one should consider how to call engines from it for parameters. Should they be exe files and parameters passed as UCI options or other ways, maybe use engines as static or dynamic libraries or even, the whole three(UI ,reference engine, candidate engine) all in same project.

[Rémi Coulom]

When I wrote this tool, I had the game of Go in mind, where playing a game in a few seconds makes little sense. I now understand that for tuning evaluation-function parameters of a chess program, you can play 1 or 2-ply games at a very high frequency. The simplest solution would probably consist in allowing a connection script to return several game results, in order to reduce the overhead of calling the script and starting winboard.

I'll do it for next version. But don't hold your breath, because I am very busy with other projects now.

What kind of game rate do the most extreme testers get? 100 per second? 1000 per second?

Rémi

[Zach Wegner]

For evaluation tuning I can usually run about 16 games/sec on an old quad. I'm sure I could run more, but I want them to be of high quality.

[Look]

When I wrote this tool, I had the game of Go in mind, where playing a game in a few seconds makes little sense. I now understand that for tuning evaluation-function parameters of a chess program, you can play 1 or 2-ply games at a very high frequency. The simplest solution would probably consist in allowing a connection script to return several game results, in order to reduce the overhead of calling the script and starting winboard.

I'll do it for next version. But don't hold your breath, because I am very busy with other projects now.

What kind of game rate do the most extreme testers get? 100 per second? 1000 per second?

Rémi

Really thanks for work you have done so far, but I do not think there is anything special you can do for its chess connection. It seems to me best would be UI developers who are specifically helping engine programming may take next steps over time, and create the connection and add facilities so that the whole process can go easily for tuning engine parameters. If you want to continue your work on this project, I suggest to try to improve its quality, as that would benefit all users.

Regards.

[diep]

hi,

If i play a few million games with diep, how many parameters you'd expect i can tune better with this manner?

So not theoretic worstcase of O ( n ^ 6 ) but average behaviour on computerchess.

Diep has a lot of parameters, many of which i suspect with slight changes would do better; stupid bruce force methods to tune all those thousands of parameters would fail by definition. I'd compress at a first attempt the parameters, so try just a few of them in turn.

That's not very great, but better than nothing i'd say.

Most interesting is of course default material value tuning.

diep has a default array of say 60 material values and a lot of rules, most of them are in arrays; so i'd estimate a few hundreds of parameters just for material. It's not that much of a difference with the year 2000, with respect to the number of parameters there.

I bet Diep was the first to have such a huge material evaluation already.

How many games would you expect needed to tune it better than its current default values?

Realize that majority of values, if you do small changes there, that the effect might not be measurable easily. In fact it's possible you'd need really a lot of games to measure the difference.

Material is not a complete smooth function initially, but it is the most important thing to tune well in an engine.

most important question: can the command line handle a couple of hundreds of parameters in linux?

How many characters can command line handle max there (i can set it bigger if it is easy to change)?

Of course i would start the test with the current values - my aim is not investigating the tuning algorithm - the aim is to squeeze out more elo by pushing a few buttons.

You are well aware how material works in chess; how many games are needed you estimate to get to something within say 10 elo of optimum values?

I can give you exact count on how many parameters it is in fact if that is helpful. Let's estimate for now quickly 256 parameters for material value calculation.

Thanks,
Vincent

[Don]

When I wrote this tool, I had the game of Go in mind, where playing a game in a few seconds makes little sense. I now understand that for tuning evaluation-function parameters of a chess program, you can play 1 or 2-ply games at a very high frequency. The simplest solution would probably consist in allowing a connection script to return several game results, in order to reduce the overhead of calling the script and starting winboard.

I'll do it for next version. But don't hold your breath, because I am very busy with other projects now.

What kind of game rate do the most extreme testers get? 100 per second? 1000 per second?

Rémi

I have a 6 core i7-980x and I can run 2730 games per minute at 1 ply - which is 45 games per second.

The tester is designed to be very low overhead, multi-processors, and fast as it was written in C.

Still, due to various overheads I have found that 1 ply searches are not very efficient, I can run 2 ply searches and get almost as many in. I have found that 3 ply is a reasonable compromise if I must get in a lot of really fast games in - way better quality than 1 ply without losing that much in quantity.

With 3 ply I can get about 1880 per minute or about 31 games per second.

For tuning evaluation, deeper is always better, but I have found that 5 ply is about the shortest "practical" level for this. If it helps a 5 ply search it usually helps at greater depths but that does not seem to be true for shorter searches - at least for Komodo.

With 5 ply searches I am already down to 811 games per minute or about 13.5 games per second.

For testing single processor programs I have found that I get better throughput telling the tester I am on 12 cores, even though I only have 6 cores. I'm pretty sure the reason is I/O overhead which no doubt causes the CPU to twiddle it's thumbs waiting for something to do.

[Daniel Shawul]

These are probably stupid questions but I will ask anyway.. The calculation of the hessian takes O(n^2). Is it the stochastic nature of the objective function which brought it O(n^6) ?
Also does it find the global optimal solution because I see that the hessian matrix is being computed ?

I tried the windows version and it runs perfect. However I could not compile
the linux version because of the pre-requisite libraries that I can't install. Can you or someone else provide a binary for linux 64 bit with the required dlls ?
I know that cutechess-cli can work without installing the whole QT package.

regards,
Daniel

[Rémi Coulom]

These are probably stupid questions but I will ask anyway.. The calculation of the hessian takes O(n^2). Is it the stochastic nature of the objective function which brought it O(n^6) ?

No. There are in fact two Hessians. A n*n Hessian of the quadratic regression. And a (n*(n+1)/2) * (n*(n+1)/2) matrix of the log-posterior with respect to parameters. I maximize the log-posterior with the Newton-Raphson method, with a cost cubic in the size of the Hessian. This is a very effective method when n is small, but not effective at all when n is large. I'll implement better methods.

Also does it find the global optimal solution because I see that the hessian matrix is being computed ?

I am not sure about the meaning of this question. Regression is local around the current estimated parameters. If the function to be optimized has local optima, then QLR may miss them.

I tried the windows version and it runs perfect. However I could not compile
the linux version because of the pre-requisite libraries that I can't install. Can you or someone else provide a binary for linux 64 bit with the required dlls ?
I know that cutechess-cli can work without installing the whole QT package.

regards,
Daniel

I can provide my 32-bit linux compilation. Not sure if you can run it in your 64-bit environment:
http://remi.coulom.free.fr/QLR/qlr-gui
I am not sure which additional dll you need, and you may probably be able to find them elsewhere.

Why can't you install libraries?

Rémi

[Daniel Shawul]

Evolutionary methods for optimization have been discussed here before so my first thought
was that yours was a GA. These can indeed find a global optima despite the users initial guesses
for the parameters. It takes a lot more objective function evaluations (games) though.
I dont know how important this is to the topic of chess evaluation terms as we have already have
a "good" estimate of the parameters before the experiment anyway..

For newton-raphson I recall that computation of the hessian matrix can be done in O(n^2 + n/2).
To decrease the cost of more function evaluations, we can use quasi-newton methods like BFGS.
Also We can use radial basis functions to approximate the response and determine optimal values
with much less number of samples. The reason for my question was that I had set a limit of n^2
samples per iteration for the methods I am familiar with.. I now understand that your method
makes many function calls per iteration but converges faster than the rest overall, which is the final goal.

regards,
Daniel

P.S : I don't have admin access to install software on the cluster. Cutechess-cli required QT
too , but Illari suggested that only 2 or more of dlls are required.
But don't worry about it, I will get somebody to do the comple for console QLR for 64 bit linux.
Thanks again.