ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

CLOP for Noisy Black-Box Parameter Optimization
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions Flat
View previous topic :: View next topic  
Author Message
Ferdinand Mosca



Joined: 10 Aug 2008
Posts: 453
Location: Philippines
PostPost subject: Re: CLOP for Noisy Black-Box Parameter Optimization    Posted: Sun Oct 23, 2011 10:47 pm Reply to topic Reply with quote
Quote:
You can not do it with the gui. But you can open the data file with the command-line version, and redirect output to a file, then plot it with a tool such as gnuplot, for instance.

I got this one, nice table.



Quote:
No, there is a problem with shrinking range. I will fix it. But you should not want to do that. CLOP is good at figuring out the right range. For tuning a parameter such as Queen value, just use a wide range (100-2000, for instance). It is very likely CLOP will do better by itself than if you try to tweak the range manually.

The only good reason to manually change the range during optimization is to start with a narrow range in order to indicate a good point in parameter space, then widen the range and let CLOP do its magic.

I will note that start with narrow then wider.



Quote:
You should not use such a narrow range. The more games, the more accurate the result. You can get an idea of how CLOP is uncertain about the value by looking at the interval over which it takes samples. But use a wide default interval.

One of my concerns here is to speed up the process, minimize playing large amount of games, and get a respectable value not necessarily very optimal. For example I am working on getting good approximate piece value of capablanca chess pieces such as the archbishop and chancellor. I am targetting to play only 5k games after that I will use what clop has got so far. Of course I will test the setting produced by clop vs my orig setting.
For normal chess I can only play around 10k games, stop the clop tunning and test clop setting vs my default. From this I got a promising result for clop tuned version, but as I increased the number of test games > 2k the advantage of clop-tuned engine is getting reduced thru the use of bayeselo. But I am satisfied with situation because clop-tuned engine is always leading. Also like a handicap to clop-tuned engine, I use clop to tune parameter vs. with engine 1, engine 2, and engine 3 only using xxx book, but when I do the one-on-one test, I use clop-tune engine vs engine 1, engine 2, engine 3, engine 4, engine 5, engine 6 using yyy book, and then test default engine vs engine 1 - 6 using also same yyy book even with this the clop-tuned engine leads as was mentioned.
Back to top
View user's profile Send private message
Display posts from previous:   
Subject Author Date/Time
CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Thu Sep 01, 2011 10:32 am
      Re: CLOP for Noisy Black-Box Parameter Optimization Joona Kiiski Fri Sep 02, 2011 8:52 am
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 02, 2011 9:45 am
                  Re: CLOP for Noisy Black-Box Parameter Optimization Marco Costalba Fri Sep 02, 2011 5:34 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Zach Wegner Fri Sep 02, 2011 4:44 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 02, 2011 7:58 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Rein Halbersma Fri Sep 02, 2011 10:07 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 03, 2011 9:24 am
                  Re: CLOP for Noisy Black-Box Parameter Optimization Rein Halbersma Sat Sep 03, 2011 5:01 pm
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 03, 2011 5:28 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Andres Valverde Sat Sep 03, 2011 10:28 am
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 03, 2011 10:40 am
      Re: CLOP for Noisy Black-Box Parameter Optimization Peter Österlund Sat Sep 03, 2011 6:54 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 03, 2011 7:34 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Ernest Bonnem Sun Sep 04, 2011 4:49 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Don Dailey Wed Sep 07, 2011 4:57 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Wed Sep 07, 2011 5:47 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Don Dailey Wed Sep 07, 2011 6:09 pm
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Wed Sep 07, 2011 6:26 pm
                        Re: CLOP for Noisy Black-Box Parameter Optimization F. Bluemers Wed Sep 07, 2011 6:39 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Wed Sep 07, 2011 8:19 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Thu Sep 08, 2011 4:44 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Larry Kaufman Fri Sep 09, 2011 3:36 am
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 09, 2011 6:11 am
                              Re: CLOP for Noisy Black-Box Parameter Optimization Larry Kaufman Fri Sep 09, 2011 2:43 pm
                                    Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 09, 2011 7:47 pm
                                          Re: CLOP for Noisy Black-Box Parameter Optimization Marco Costalba Sun Nov 06, 2011 8:24 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 8:30 am
                                          Re: CLOP for Noisy Black-Box Parameter Optimization Marco Costalba Sun Nov 06, 2011 8:50 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Peter Österlund Sun Nov 06, 2011 1:11 pm
                                                      Re: CLOP for Noisy Black-Box Parameter Optimization Marco Costalba Sun Nov 06, 2011 4:53 pm
                                                      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 5:14 pm
                                                            Re: CLOP for Noisy Black-Box Parameter Optimization Peter Österlund Sun Nov 06, 2011 11:11 pm
                                                                  Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Nov 07, 2011 11:18 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 5:05 pm
                                    Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 09, 2011 7:52 pm
                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 10, 2011 9:49 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Sun Sep 25, 2011 8:18 am
                                                      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Sep 25, 2011 8:47 am
                                                            Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Mon Sep 26, 2011 10:55 am
                                                                  Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Sep 26, 2011 1:40 pm
                                                                        Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Tue Sep 27, 2011 7:54 am
                                                                              Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Tue Sep 27, 2011 8:39 am
                                                                                    Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Thu Sep 29, 2011 2:04 pm
                                                                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Thu Sep 29, 2011 3:38 pm
                                                                        Re: CLOP for Noisy Black-Box Parameter Optimization Daniel Shawul Mon Oct 17, 2011 2:03 am
                                                                              Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Mon Oct 17, 2011 7:51 am
                                                                              Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Oct 17, 2011 7:56 am
                                                                                    Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Oct 17, 2011 8:02 am
                                                                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Oct 17, 2011 8:11 am
                                                                                    Re: CLOP for Noisy Black-Box Parameter Optimization Daniel Shawul Mon Oct 17, 2011 9:54 am
                                                                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Oct 17, 2011 11:10 am
      Re: CLOP for Noisy Black-Box Parameter Optimization david nash Wed Oct 05, 2011 12:19 am
      Re: CLOP - a trial for xiangqi engine Edward Yu Fri Oct 07, 2011 11:58 am
            Re: CLOP - a trial for xiangqi engine Rémi Coulom Fri Oct 07, 2011 1:17 pm
                  Re: CLOP - a trial for xiangqi engine Edward Yu Fri Oct 07, 2011 4:01 pm
                        Re: CLOP - a trial for xiangqi engine Rémi Coulom Fri Oct 07, 2011 4:30 pm
                              Re: CLOP - a trial for xiangqi engine Edward Yu Sat Oct 08, 2011 3:59 am
                                    Re: CLOP - a trial for xiangqi engine Rémi Coulom Sat Oct 08, 2011 7:49 am
                                          Re: CLOP - a trial for xiangqi engine Edward Yu Tue Oct 11, 2011 4:04 pm
                                                Re: CLOP - a trial for xiangqi engine Rémi Coulom Tue Oct 11, 2011 4:22 pm
                                                      Re: CLOP - a trial for xiangqi engine Daniel Shawul Tue Oct 11, 2011 4:49 pm
                                                            Re: CLOP - a trial for xiangqi engine Rémi Coulom Tue Oct 11, 2011 5:27 pm
                                                                  Issue.... Michel Van den Bergh Sat Oct 15, 2011 2:51 pm
                                                                        Re: Issue.... Rémi Coulom Sun Oct 16, 2011 6:30 am
                                                                              Re: Issue.... Michel Van den Bergh Sun Oct 16, 2011 8:10 am
                                                                                    Re: Issue.... Rémi Coulom Sun Oct 16, 2011 9:02 am
                                                                                          Re: Issue.... Michel Van den Bergh Sun Oct 16, 2011 3:10 pm
                                                                                          Re: Issue.... Rémi Coulom Sun Oct 16, 2011 7:25 pm
                              Re: CLOP Michel Van den Bergh Sat Oct 08, 2011 6:00 am
                                    Re: CLOP Rémi Coulom Sat Oct 08, 2011 7:53 am
                                          Re: CLOP Rémi Coulom Sat Oct 08, 2011 8:27 am
      Re: CLOP for Noisy Black-Box Parameter Optimization Ferdinand Mosca Tue Oct 11, 2011 8:22 am
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Tue Oct 11, 2011 1:38 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Ferdinand Mosca Sun Oct 23, 2011 7:39 pm
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Oct 23, 2011 8:55 pm
                              Re: CLOP for Noisy Black-Box Parameter Optimization Ferdinand Mosca Sun Oct 23, 2011 10:47 pm
                                    Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 9:42 am
                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Nov 04, 2011 9:58 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 10:17 am
                                                      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Nov 04, 2011 10:30 am
                                                            Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 10:44 am
                                                                  Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Nov 04, 2011 10:56 am
                                                                        Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 11:04 am
                                                                              Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Nov 04, 2011 11:56 am
                                                                                    Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 4:16 pm
                                                                                          Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 5:48 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Sergei S. Markoff Fri Nov 04, 2011 3:02 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Nov 05, 2011 7:45 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Sat Nov 05, 2011 10:49 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Ilari Pihlajisto Sat Nov 05, 2011 11:17 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Sun Nov 06, 2011 12:06 am
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 8:00 am
                              Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Sun Nov 06, 2011 10:20 am
                                    Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 5:10 pm
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads