ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

CLOP for Noisy Black-Box Parameter Optimization
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions Flat
View previous topic :: View next topic  
Author Message
Rémi Coulom



Joined: 24 Apr 2006
Posts: 350
PostPost subject: Re: CLOP - a trial for xiangqi engine    Posted: Sat Oct 08, 2011 7:49 am Reply to topic Reply with quote
edwardyu wrote:
Thanks Remi for the solution. Here is the modified script. Also the CLOP file has to be changed to use Replications 2.


Note that this approach won't work if you have more than one processor. In order to work with more than one processor, your script should use a different file name for each different seed, and the odd seed should wait for the even seed to complete before returning its result. But that is a bit complicated.

Quote:
A simple question: What statistics should I look for in order to decide that the experiment is converging and I can stop the experiment? Hessian? Eigenvectors? Max? or Win Rate?


There is no optimal way to stop: the more games you play, the more accurate the estimated optimal parameters. Confidence intervals on the win rate can give you a very vague idea of how close to optimal you may be, but it is still very difficult to tell. The plots in my paper can give you an idea of how close to optimal you may be, given a number of games played.
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Subject Author Date/Time
CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Thu Sep 01, 2011 10:32 am
      Re: CLOP for Noisy Black-Box Parameter Optimization Joona Kiiski Fri Sep 02, 2011 8:52 am
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 02, 2011 9:45 am
                  Re: CLOP for Noisy Black-Box Parameter Optimization Marco Costalba Fri Sep 02, 2011 5:34 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Zach Wegner Fri Sep 02, 2011 4:44 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 02, 2011 7:58 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Rein Halbersma Fri Sep 02, 2011 10:07 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 03, 2011 9:24 am
                  Re: CLOP for Noisy Black-Box Parameter Optimization Rein Halbersma Sat Sep 03, 2011 5:01 pm
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 03, 2011 5:28 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Andres Valverde Sat Sep 03, 2011 10:28 am
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 03, 2011 10:40 am
      Re: CLOP for Noisy Black-Box Parameter Optimization Peter Österlund Sat Sep 03, 2011 6:54 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 03, 2011 7:34 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Ernest Bonnem Sun Sep 04, 2011 4:49 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Don Dailey Wed Sep 07, 2011 4:57 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Wed Sep 07, 2011 5:47 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Don Dailey Wed Sep 07, 2011 6:09 pm
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Wed Sep 07, 2011 6:26 pm
                        Re: CLOP for Noisy Black-Box Parameter Optimization F. Bluemers Wed Sep 07, 2011 6:39 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Wed Sep 07, 2011 8:19 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Thu Sep 08, 2011 4:44 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Larry Kaufman Fri Sep 09, 2011 3:36 am
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 09, 2011 6:11 am
                              Re: CLOP for Noisy Black-Box Parameter Optimization Larry Kaufman Fri Sep 09, 2011 2:43 pm
                                    Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 09, 2011 7:47 pm
                                          Re: CLOP for Noisy Black-Box Parameter Optimization Marco Costalba Sun Nov 06, 2011 8:24 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 8:30 am
                                          Re: CLOP for Noisy Black-Box Parameter Optimization Marco Costalba Sun Nov 06, 2011 8:50 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Peter Österlund Sun Nov 06, 2011 1:11 pm
                                                      Re: CLOP for Noisy Black-Box Parameter Optimization Marco Costalba Sun Nov 06, 2011 4:53 pm
                                                      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 5:14 pm
                                                            Re: CLOP for Noisy Black-Box Parameter Optimization Peter Österlund Sun Nov 06, 2011 11:11 pm
                                                                  Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Nov 07, 2011 11:18 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 5:05 pm
                                    Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Sep 09, 2011 7:52 pm
                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Sep 10, 2011 9:49 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Sun Sep 25, 2011 8:18 am
                                                      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Sep 25, 2011 8:47 am
                                                            Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Mon Sep 26, 2011 10:55 am
                                                                  Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Sep 26, 2011 1:40 pm
                                                                        Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Tue Sep 27, 2011 7:54 am
                                                                              Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Tue Sep 27, 2011 8:39 am
                                                                                    Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Thu Sep 29, 2011 2:04 pm
                                                                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Thu Sep 29, 2011 3:38 pm
                                                                        Re: CLOP for Noisy Black-Box Parameter Optimization Daniel Shawul Mon Oct 17, 2011 2:03 am
                                                                              Re: CLOP for Noisy Black-Box Parameter Optimization Michel Van den Bergh Mon Oct 17, 2011 7:51 am
                                                                              Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Oct 17, 2011 7:56 am
                                                                                    Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Oct 17, 2011 8:02 am
                                                                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Oct 17, 2011 8:11 am
                                                                                    Re: CLOP for Noisy Black-Box Parameter Optimization Daniel Shawul Mon Oct 17, 2011 9:54 am
                                                                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Mon Oct 17, 2011 11:10 am
      Re: CLOP for Noisy Black-Box Parameter Optimization david nash Wed Oct 05, 2011 12:19 am
      Re: CLOP - a trial for xiangqi engine Edward Yu Fri Oct 07, 2011 11:58 am
            Re: CLOP - a trial for xiangqi engine Rémi Coulom Fri Oct 07, 2011 1:17 pm
                  Re: CLOP - a trial for xiangqi engine Edward Yu Fri Oct 07, 2011 4:01 pm
                        Re: CLOP - a trial for xiangqi engine Rémi Coulom Fri Oct 07, 2011 4:30 pm
                              Re: CLOP - a trial for xiangqi engine Edward Yu Sat Oct 08, 2011 3:59 am
                                    Re: CLOP - a trial for xiangqi engine Rémi Coulom Sat Oct 08, 2011 7:49 am
                                          Re: CLOP - a trial for xiangqi engine Edward Yu Tue Oct 11, 2011 4:04 pm
                                                Re: CLOP - a trial for xiangqi engine Rémi Coulom Tue Oct 11, 2011 4:22 pm
                                                      Re: CLOP - a trial for xiangqi engine Daniel Shawul Tue Oct 11, 2011 4:49 pm
                                                            Re: CLOP - a trial for xiangqi engine Rémi Coulom Tue Oct 11, 2011 5:27 pm
                                                                  Issue.... Michel Van den Bergh Sat Oct 15, 2011 2:51 pm
                                                                        Re: Issue.... Rémi Coulom Sun Oct 16, 2011 6:30 am
                                                                              Re: Issue.... Michel Van den Bergh Sun Oct 16, 2011 8:10 am
                                                                                    Re: Issue.... Rémi Coulom Sun Oct 16, 2011 9:02 am
                                                                                          Re: Issue.... Michel Van den Bergh Sun Oct 16, 2011 3:10 pm
                                                                                          Re: Issue.... Rémi Coulom Sun Oct 16, 2011 7:25 pm
                              Re: CLOP Michel Van den Bergh Sat Oct 08, 2011 6:00 am
                                    Re: CLOP Rémi Coulom Sat Oct 08, 2011 7:53 am
                                          Re: CLOP Rémi Coulom Sat Oct 08, 2011 8:27 am
      Re: CLOP for Noisy Black-Box Parameter Optimization Ferdinand Mosca Tue Oct 11, 2011 8:22 am
            Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Tue Oct 11, 2011 1:38 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Ferdinand Mosca Sun Oct 23, 2011 7:39 pm
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Oct 23, 2011 8:55 pm
                              Re: CLOP for Noisy Black-Box Parameter Optimization Ferdinand Mosca Sun Oct 23, 2011 10:47 pm
                                    Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 9:42 am
                                          Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Nov 04, 2011 9:58 am
                                                Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 10:17 am
                                                      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Nov 04, 2011 10:30 am
                                                            Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 10:44 am
                                                                  Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Nov 04, 2011 10:56 am
                                                                        Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 11:04 am
                                                                              Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Fri Nov 04, 2011 11:56 am
                                                                                    Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 4:16 pm
                                                                                          Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Fri Nov 04, 2011 5:48 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Sergei S. Markoff Fri Nov 04, 2011 3:02 pm
      Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sat Nov 05, 2011 7:45 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Sat Nov 05, 2011 10:49 pm
            Re: CLOP for Noisy Black-Box Parameter Optimization Ilari Pihlajisto Sat Nov 05, 2011 11:17 pm
                  Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Sun Nov 06, 2011 12:06 am
                        Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 8:00 am
                              Re: CLOP for Noisy Black-Box Parameter Optimization Jonatan Pettersson Sun Nov 06, 2011 10:20 am
                                    Re: CLOP for Noisy Black-Box Parameter Optimization Rémi Coulom Sun Nov 06, 2011 5:10 pm
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads