tuning info

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
elcabesa
Posts: 815
Joined: Sun May 23, 2010 11:32 am
Contact:

tuning info

Post by elcabesa » Wed Jan 03, 2018 10:20 am

Hi everyone,
I just finished a good session of tuning of my engine using a technique very similar to texel method, minimising the error of evaluation against game results.

I'd like now to try to tune some other parameters using an spsa like approach.
I have studied the perl spsa tuner and looked at fishtest code.
The spsa Perl tuner doesn't work very well with vajolet and I don't like to debug the uci interaction between them. We already have a very good tournament software, cutechess and I was asking myself if already exist some tool or fork of cutechess that allow to tune an engine.

Are you aware of some project or tuning script already using cutechess? Otherwise my idea is to add a tuner to cutechess.

Thank you all

User avatar
stegemma
Posts: 859
Joined: Mon Aug 10, 2009 8:05 pm
Location: Italy
Full name: Stefano Gemma
Contact:

Re: tuning info

Post by stegemma » Wed Jan 03, 2018 3:32 pm

elcabesa wrote:Hi everyone,
I just finished a good session of tuning of my engine using a technique very similar to texel method, minimising the error of evaluation against game results.

I'd like now to try to tune some other parameters using an spsa like approach.
I have studied the perl spsa tuner and looked at fishtest code.
The spsa Perl tuner doesn't work very well with vajolet and I don't like to debug the uci interaction between them. We already have a very good tournament software, cutechess and I was asking myself if already exist some tool or fork of cutechess that allow to tune an engine.

Are you aware of some project or tuning script already using cutechess? Otherwise my idea is to add a tuner to cutechess.

Thank you all
Does Vajolet have a parameter on the commadn line to load parameters from a file? I could test genetic optimization using Sabrina tournaments mode. I don't know SPSA but it could be added as a learning algorithm.
Author of Drago, Raffaela, Freccia, Satana, Sabrina.
http://www.linformatica.com

Daniel Shawul
Posts: 3758
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: tuning info

Post by Daniel Shawul » Wed Jan 03, 2018 3:53 pm

Clop + cutechess-cli is good for tuning search parameters.

It could be slow if you have too many parameters but often times there are just a handful of search parameters.

For example, I only have one futility_pruning_margin to tune in search.

Daniel

elcabesa
Posts: 815
Joined: Sun May 23, 2010 11:32 am
Contact:

Re: tuning info

Post by elcabesa » Thu Jan 04, 2018 6:05 pm

I don't like clop, but probably I can his approach.
write a python script for spsa that use cutechess to play game.. I'll give it a try

jdart
Posts: 3825
Joined: Fri Mar 10, 2006 4:23 am
Location: http://www.arasanchess.org

Re: tuning info

Post by jdart » Thu Jan 04, 2018 10:07 pm

There is a script that comes with CLOP that you can probably modify.

You might also take a look also at:

https://github.com/jdart1/arasan-chess/ ... sitions.py

Which shows how to invoke cutechess-cli and parse the results.

--Jon

Daniel Shawul
Posts: 3758
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: tuning info

Post by Daniel Shawul » Fri Jan 05, 2018 12:04 am

I think clop may support different optimization algorithms.

I see an C2SPSA.cpp in there though i am not sure what it is used for.

P.S: Does anybody know why latest cutechess-cli does not display results on stdout ?
It plays the game and saves it to a pgn but there is no result output on the screen.
I tried cutechess-cli 1.0 and 0.85, on ubuntu 17.10 -- Qt version 5.9.1

Daniel

petero2
Posts: 587
Joined: Mon Apr 19, 2010 5:07 pm
Location: Sweden
Contact:

Re: tuning info

Post by petero2 » Fri Jan 05, 2018 3:04 pm

Daniel Shawul wrote:P.S: Does anybody know why latest cutechess-cli does not display results on stdout ?
It plays the game and saves it to a pgn but there is no result output on the screen.
I tried cutechess-cli 1.0 and 0.85, on ubuntu 17.10 -- Qt version 5.9.1
I had this problem in fedora 24. I solved it by adding the following to ~/.config/QtProject/qtlogging.ini:

Code: Select all

[Rules]
*.debug=true

Daniel Shawul
Posts: 3758
Joined: Tue Mar 14, 2006 10:34 am
Location: Ethiopia
Contact:

Re: tuning info

Post by Daniel Shawul » Fri Jan 05, 2018 5:26 pm

It works like a charm!

Thanks a lot Peter!

Daniel

Ferdy
Posts: 4111
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: tuning info

Post by Ferdy » Sun Jan 07, 2018 2:27 pm

elcabesa wrote:The spsa Perl tuner doesn't work very well with vajolet and I don't like to debug the uci interaction between them.
Couple of notes when using spsa from
https://github.com/zamar/spsa/blob/master/spsa.pl

1. Be aware of the game adjudication on stalemate.
When your engine received a stalemate position what bestmove it prints?
Stockfish would print

Code: Select all

bestmove (none)
If your engine returns bestmove 0000,
then the code in line 446,

Code: Select all

$flag_stalemate = 1 if ($array[1] eq '(none)');
should be change to:

Code: Select all

$flag_stalemate = 1 if ($array[1] eq '0000');
2. When parsing engine output info line, make sure the splitted line converted into an array is properly read. At line 452,

Code: Select all

# Check for mate in one
if ($#array >= 9 && $array[0] eq 'info' && $array[1] eq 'depth' &&
    $array[7] eq 'score' && $array[8] eq 'mate' && $array[9] eq '1') 
{
    $flag_mate = 1;
    $winner = $engine_to_move;
}
With typical uci engine search output for example.

Code: Select all

info depth 1 score mate 1 nodes 20 nps 1818 tbhits 0 time 11 pv e2e4
The code above must be revised to:

Code: Select all

if ($#array >= 5 && $array[0] eq 'info' && $array[1] eq 'depth' &&
    $array[3] eq 'score' && $array[4] eq 'mate' && $array[5] eq '1') 
{
...
}
3. There is an unsafe code in line 461, revised this depending on your engine output.

Code: Select all

# Record score
if ($#array >= 7 && $array[0] eq 'info' && $array[1] eq 'depth' &&
    $array[7] eq 'score') 
{    
    $score = $array[9] if ($array[8] eq 'cp');
    $score = +100000   if ($array[8] eq 'mate' && $array[9] > 0);
    $score = -100000   if ($array&#91;8&#93; eq 'mate' && $array&#91;9&#93; < 0&#41;;
&#125;
That,

Code: Select all

$#array >= 7
should be change to:

Code: Select all

$#array >= 9

elcabesa
Posts: 815
Joined: Sun May 23, 2010 11:32 am
Contact:

Re: tuning info

Post by elcabesa » Sun Jan 07, 2018 5:05 pm

I don't like reinventing the well, we already have the beatiful cutechess-cli to play games.
I think I'll probably port spsa script to python and use cutechess to play games

Post Reply