To Ilari : CLOP + cutechess-cli

[ilari]

After 4 hours of hardcore hacking I was able to setup the thing on Windows 7 with Qt SDK mingw compiler.

Regarding your script I needed to change the following:

Code: Select all

sts = os.waitpid(process.pid, 0)

because on Windows there is not support for pid:

http://stackoverflow.com/questions/6647 ... -processes

So I workaround with:

Code: Select all

process.wait()

Oh, of course. I've now changed the script to use Popen.communicate() to get the output and to wait for the process to end. The new script is available here: http://koti.mbnet.fi/~ilaripih/bin/clop ... ess-cli.py

[ilari]

sts = os.waitpid(process.pid, 0)

Gee.I spent hours but I couldn't make it through apparently because of the bug you discovered.
I still have problems with cutechess-cli .json file which seems to ignore inisstring. Some uci engines take all the processors unless told to use threads=1.

Sorry about that. Unless they're something unrelated to the CLOP parameters you should NOT set the "initstr" values in the engines.json file. The "clop-cutechess-cli.py" script will send those parameters to cutechess-cli.

[ilari]

You may want to set:

Code: Select all

engine_param_cmd = 'setoption {name} value {value}'

in clop-cutechess.py instead of the shipped

Code: Select all

engine_param_cmd = 'setvalue {name} {value}'

The former seems the correct one for UCI engines, I don't know when is used the second one....

Actually the correct one for UCI options is 'setoption name {name} value {value}'.

Regarding the second syntax, I figured that some would want to optimize parameters that are not UCI options, and their engine might have its own syntax for setting these parameters. For example in Sloppy I'm now using the "setvalue {name} {value}" syntax.

[ilari]

...then I was unable to make the windows version of CLOP to launch the python glue script DummyScript.py, perhaps is because I missed something, anyhow after a good time spent googling+hacking around I workaround the issue converting the script in an exe file...

I don't think you have to do it, you just need to call the Python interpreter and give it the script as a parameter. So you'd have something like this in the .clop file:

Code: Select all

Script C:\Python26\python.exe clop-cutechess-cli.py

[Daniel Shawul]

Not to forget about winboard 2 engines which have "option value":) Anyway I run my first optimization overnight and got a small improvement with QLR. It was a self test against older version. It had difficulties but in the end came up with new values that improved it slightly.
Default scorpio:

Code: Select all

LAZY_MARGIN_MIN  150
LAZY_MARGIN_MAX 300

New values:

Code: Select all

LAZY_MARGIN_MIN  270
LAZY_MARGIN_MAX 425
Samples = 14213
TotalWeight = 10479.4
Wins = 4979
Draws = 4312
Losses = 4922
WinningRate = 0.502005

[Daniel Shawul]

I think communicate has problems.

Code: Select all

2011-10-16 09:13:05 id = 2 (machine1); Seed = 14567; Error: No Outcome.
StandardOutput:

StandardError:
Traceback (most recent call last):
  File "clop-cutechess-cli.py", line 119, in <module>
    sys.exit&#40;main&#40;))
  File "clop-cutechess-cli.py", line 103, in main
    for line in process.stdout&#58;
ValueError&#58; I/O operation on closed file

[ilari]

I think communicate has problems.

Code: Select all

2011-10-16 09:13:05 id = 2 (machine1); Seed = 14567; Error: No Outcome.
StandardOutput:

StandardError:
Traceback (most recent call last):
  File "clop-cutechess-cli.py", line 119, in <module>
    sys.exit&#40;main&#40;))
  File "clop-cutechess-cli.py", line 103, in main
    for line in process.stdout&#58;
ValueError&#58; I/O operation on closed file

Did you try the new script that I uploaded? Because it doesn't have the "for line in process.stdout" line anymore. Instead it parses the output returned by Popen.communicate()[0]

[Daniel Shawul]

My fault. I didn't want to loose the working directories I alread set, so I replaced only part of the code I though was problematic i.e around process.wait(). Using the new script now works as intended. Thanks.

[Daniel Shawul]

I think QLR has a very high tendency to get stuck in a local maxima. Since it does not use initial guesses it will have difficulty improving upon hand selected values. I don't know if CLOP has improvemetns over QLR in this regard.
Eg. Default mobility weight of 16 for all pieces.

Code: Select all

IntegerParameter KNIGHT_MOB  0 64
IntegerParameter BISHOP_MOB  0 64
IntegerParameter ROOK_MOB    0 64
IntegerParameter QUEEN_MOB   0 64

QLR came up with new values that almost doubles the default values (even 3x for the queen) that is clearly beaten by the default. It would have been great if it can initialize from user suggested values... It is early days but I don't think it can improve much.

Code: Select all

KNIGHT_MOB  31
BISHOP_MOB  25
ROOK_MOB    31
QUEEN_MOB   42
Samples = 692
TotalWeight = 475.101
Wins = 190
Draws = 175
Losses = 327
WinningRate = 0.401012

[Michel]

I think QLR has a very high tendency to get stuck in a local maxima.

I think I observed something similar. But I am not sure.

It would have been great if it can initialize from user suggested values...

Well it can, but currently this requires some manual input. First generate samples in a small window around the suggested values. Then enlarge the window to a wide one.