ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3)
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Tournaments and Matches Flat
View previous topic :: View next topic  
Author Message
Jesús Muñoz



Joined: 13 Jul 2011
Posts: 690
Location: Madrid, Spain.
PostPost subject: Re: Houdini 2.0: settings (Z, T3, Baracuda, Baracuda T3).    Posted: Mon Jul 09, 2012 9:46 am Reply to topic Reply with quote
Hello Robert:

Houdini wrote:
Tennison wrote:
T3 settings and Z settings seems (as in Sedat Rating List) a little bit better than original. A small plus to Z here but need more games to confirm because so close !

If you want to make the claims above, you need to play more games. For the moment everything is buried under the tower of statistical uncertainty.
For T3 and Z settings there is only one valid conclusion after 1000 games: they are not significantly different from the default settings.

On the Rybka Forum I published the results obtained with "z", "s" and your "T4" settings in 16´000 game matches against the default Houdini 2.0c.
Here are the results against the standard Houdini 2.0c.
- "s" scored 7797-8203 (38% draws), -8 Elo (+/- 4 Elo)
- "z" scored 7922-8078 (39% draws), -4 Elo (+/- 4 Elo)
- "T4" scored 7914-8086 (40% draws), -4 Elo (+/- 4 Elo)
Again, the only conclusion can be that these settings change very little to the objective strength of the engine.


I fully agree with you in view of the results of these three 16000-game matches. Statistical uncertainties mean everything in testing, IMHO. If I take the info of your three matches that you gave in the post I quote, I get more less these results:

Code:
s settings: +4757 -5163 =6080

(Error bars for 2-sigma confidence ~ 95.45% confidence) ~ -8.82 ± 4.33 Elo.
(LOS for s settings) ~ 0%.
(LOS > 95%): one engine must score 8082 points or more.


Code:
z settings: +4802 -4958 =6240

(Error bars for 2-sigma confidence ~ 95.45% confidence) ~ -3.39 ± 4.29 Elo.
(LOS for z settings) ~ 5.72%.
(LOS > 95%): one engine must score 8081.5 points or more.


Code:
T4 settings: +4714 -4886 =6400

(Error bars for 2-sigma confidence ~ 95.45% confidence) ~ -3.74 ± 4.26 Elo.
(LOS for T4 settings) ~ 3.96%.
(LOS > 95%): one engine must score 8081 points or more.


You may correct my results if you think they are wrong.

I guess that Houdini 3 is in the good path, only reading some of your reports. Could you give a rough estimate of the current Elo gain over Houdini 2.0c (or over Houdini 1.5a), please? Thanks in advance, and please keep up the good work!

@Ben: I wish you good luck, but remember that Robert did a lot of testing: for example, I think I read somewhere that he played circa three millions of games for tuning Houdini 1.5! Robert may correct me if I am wrong.

Regards from Spain.

Ajedrecista.
_________________
Six Fortran 95 tools.

Chess will never be solved.
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Subject Author Date/Time
Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Ben Tennison Fri Jul 06, 2012 11:15 am
      Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Ben Tennison Fri Jul 06, 2012 1:37 pm
            Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Ben Tennison Fri Jul 06, 2012 4:16 pm
                  Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Ben Tennison Sat Jul 07, 2012 7:14 am
                        Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Ben Tennison Sat Jul 07, 2012 11:53 am
                              Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Ben Tennison Mon Jul 09, 2012 8:01 am
                                    Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Ben Tennison Mon Jul 09, 2012 8:08 am
                                    Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Robert Houdart Mon Jul 09, 2012 8:53 am
                                          Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Ben Tennison Mon Jul 09, 2012 9:03 am
                                                Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) George Speight Mon Jul 09, 2012 10:44 am
                                          Re: Houdini 2.0: settings (Z, T3, Baracuda, Baracuda T3). Jesús Muñoz Mon Jul 09, 2012 9:46 am
                                          Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Uri Blass Mon Jul 09, 2012 10:55 am
                                                Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Robert Houdart Mon Jul 09, 2012 11:01 am
                                                Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Sedat Canbaz Mon Jul 09, 2012 11:15 am
                                                      Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Robert Houdart Mon Jul 09, 2012 11:35 am
                                                            Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Sedat Canbaz Mon Jul 09, 2012 11:54 am
                                                                  Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Robert Houdart Mon Jul 09, 2012 12:12 pm
                                                                        Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Sedat Canbaz Mon Jul 09, 2012 12:27 pm
                                                                              Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Robert Houdart Mon Jul 09, 2012 1:02 pm
                                                                                    Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Sedat Canbaz Mon Jul 09, 2012 3:30 pm
                                                            Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) George Speight Mon Jul 09, 2012 11:57 am
                                                                  Re: Houdini 2.0 : Settings (Z, T3, Baracuda, Baracuda T3) Kai Laskos Mon Jul 09, 2012 12:41 pm
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Tournaments and Matches

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads