ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Some thoughts on QS
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions Flat
View previous topic :: View next topic  
Author Message
Uri Blass



Joined: 08 Mar 2006
Posts: 5958
Location: Tel-Aviv Israel
PostPost subject: Re: Some thoughts on QS    Posted: Mon Jul 30, 2012 6:20 pm Reply to topic Reply with quote
Don wrote:
Uri Blass wrote:
I do not notice that komodo does better at longer time control based on rating lists.



Here are the current numbers using Houdini 2.0 and Komodo 5:

CCRL 40/4 - Houdini up 60 ELO
CCRL 40/40 - Houdini up 23 ELO

CEGT 4/40 - Houdini up 44 ELO
CEGT 40/20 - Houdini 2 ELO weaker than Komodo 5

In both cases you see we do pretty badly at short time controls and improve substantially at longer time controls which refutes Vincents idea that Komodo is optimized only to play fast time controls.

If you look at the CEGT list at 40/4 you will also see that Komodo 5 is coming out weaker than several other programs but at 40/20 it comes out as better than all other programs except for a virtual tie with Houdini 2.0c.

I think some of what we are observing is that Houdini is the wild card here. Most programs have ELO curves similar to Komodo, but Houdini and the other clones tend to play really well at fast time controls - so that is where you see most of the difference.


some comments when I look at the CCRL list

1)I think that usually you see smaller difference at longer time control
so comparing elo difference is not going to help here.

CCRL 40/40
Komodo 5 64-bit 3120 +24 −24
Movei 00.8.438 (10 10 10) 2672 +12 −12

difference 448 elo

CCRL 40/4

Komodo 5 64-bit 3217 +24 −23
Movei 00.8.438 (10 10 10) 2623 +9 −8

difference 594 elo

Note that Movei does relatively better than programs at similiar strength at long time control but I do not claim that it does better than Komodo at long time control because I think that difference in rating is meaningful only if one program is better at long time control and worse at blitz.

2)Even if we look only at Komodo5 and Houdini
I think that it is better to choose houdini1.5 and not houdini2 because 1.5 seems to be better than 2 at long time control.


and
If I use the CCRL 40/40 and 40/4 I get
40/4

Houdini 2.0c 64-bit 3276 +18 −18 69.3% −166.5 31.5%
Houdini 1.5a 64-bit 3235 +12 −12 68.7% −162.4 31.0%
Komodo 5 64-bit 3217 +24 −23 71.6% −188.4 28.6%

40/40

Houdini 1.5a 64-bit 3154 +16 −16 63.5% −89.9 43.4%
Houdini 2.0c 64-bit 3143 +16 −16 66.8% −117.9 36.8%
Komodo 5 64-bit 3120 +24 −24 61.5% −73.5 53.4%

The difference between houdini1.5a and Komodo5
is bigger at 40/40 and

3)even if we ignore houdini1.5a and look only at komodo5 and houdini2c the result is not significant statistically.

The 59 elo difference between houdini2c and komodo5 at 40/4 have a statistical error near 30 elo.

The 23 elo difference between houdini2c and komodo5 at 40/40 have a similiar statistical error.

The difference between the differences has a statistical error of more than 36 elo.
Back to top
View user's profile Send private message
Display posts from previous:   
Subject Author Date/Time
Some thoughts on QS H.G.Muller Thu Jul 19, 2012 9:20 pm
      Re: Some thoughts on QS Matthew R. Brades Thu Jul 19, 2012 9:25 pm
            Re: Some thoughts on QS Robert Hyatt Thu Jul 19, 2012 11:13 pm
                  Re: Some thoughts on QS Don Dailey Thu Jul 19, 2012 11:40 pm
                        Re: Some thoughts on QS Richard Vida Thu Jul 19, 2012 11:57 pm
                              Re: Some thoughts on QS Don Dailey Fri Jul 20, 2012 12:01 am
                        Re: Some thoughts on QS H.G.Muller Fri Jul 20, 2012 7:52 am
                        Re: Some thoughts on QS John Merlino Mon Jul 23, 2012 6:16 pm
                              Re: Some thoughts on QS Don Dailey Mon Jul 23, 2012 6:34 pm
                              Re: Some thoughts on QS Robert Hyatt Mon Jul 23, 2012 6:37 pm
                                    Re: Some thoughts on QS Vincent Diepeveen Tue Jul 24, 2012 9:10 am
                                          Re: Some thoughts on QS Uri Blass Tue Jul 24, 2012 9:21 am
                                                Re: Some thoughts on QS Vincent Diepeveen Tue Jul 24, 2012 12:36 pm
                                                      Re: Some thoughts on QS Ed Schroder Tue Jul 24, 2012 1:24 pm
                                                      Re: Some thoughts on QS Uri Blass Wed Jul 25, 2012 5:16 am
                                    Re: Some thoughts on QS Ed Schroder Tue Jul 24, 2012 1:16 pm
                                          Re: Some thoughts on QS Don Dailey Tue Jul 24, 2012 4:20 pm
                                                Re: Some thoughts on QS Robert Hyatt Tue Jul 24, 2012 9:27 pm
                                                      Re: Some thoughts on QS Don Dailey Tue Jul 24, 2012 10:05 pm
                                                            Re: Some thoughts on QS Vincent Diepeveen Wed Jul 25, 2012 1:01 am
                                                            Re: Some thoughts on QS Steve Maughan Wed Jul 25, 2012 1:41 am
                                                                  Re: Some thoughts on QS Ed Schroder Wed Jul 25, 2012 7:23 am
                                                                        Re: Some thoughts on QS Don Dailey Wed Jul 25, 2012 11:38 am
                                                                              Re: Some thoughts on QS Uri Blass Wed Jul 25, 2012 12:23 pm
                                                                                    Re: Some thoughts on QS Don Dailey Wed Jul 25, 2012 1:27 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Wed Jul 25, 2012 2:20 pm
                                                                                          Re: Some thoughts on QS Ed Schroder Wed Jul 25, 2012 7:09 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Wed Jul 25, 2012 7:13 pm
                                                                                          Re: Some thoughts on QS Ed Schroder Wed Jul 25, 2012 7:49 pm
                                                                                    Re: Some thoughts on QS Vincent Diepeveen Wed Jul 25, 2012 1:29 pm
                                                                                          Re: Some thoughts on QS Don Dailey Wed Jul 25, 2012 1:52 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Fri Jul 27, 2012 1:53 pm
                                                                                          Re: Some thoughts on QS Uri Blass Wed Jul 25, 2012 2:26 pm
                                                                                          Re: Some thoughts on QS Marcel van Kervinck Wed Jul 25, 2012 8:50 pm
                                                                                          Re: Some thoughts on QS Don Dailey Wed Jul 25, 2012 9:42 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Fri Jul 27, 2012 12:34 pm
                                                                                          Re: Some thoughts on QS Don Dailey Fri Jul 27, 2012 1:04 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Fri Jul 27, 2012 1:56 pm
                                                                                          Re: Some thoughts on QS Don Dailey Fri Jul 27, 2012 2:23 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Sat Jul 28, 2012 2:00 pm
                                                                                          Re: Some thoughts on QS Don Dailey Sat Jul 28, 2012 4:12 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Sat Jul 28, 2012 4:39 pm
                                                                                          Re: Some thoughts on QS Don Dailey Sat Jul 28, 2012 5:32 pm
                                                                                          Re: Some thoughts on QS H.G.Muller Sat Jul 28, 2012 6:04 pm
                                                                                          Re: Some thoughts on QS Don Dailey Sat Jul 28, 2012 6:31 pm
                                                                                          Re: Some thoughts on QS Don Dailey Sat Jul 28, 2012 6:36 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Sat Jul 28, 2012 6:45 pm
                                                                                          Re: Some thoughts on QS Don Dailey Sat Jul 28, 2012 7:01 pm
                                                                                          Re: Some thoughts on QS Uri Blass Sat Jul 28, 2012 8:45 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Sun Jul 29, 2012 10:45 pm
                                                                                          Re: Some thoughts on QS Don Dailey Mon Jul 30, 2012 1:33 am
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Mon Jul 30, 2012 12:46 pm
                                                                                          Re: Some thoughts on QS Don Dailey Mon Jul 30, 2012 1:42 pm
                                                                                          Re: Some thoughts on QS Uri Blass Mon Jul 30, 2012 3:46 pm
                                                                                          Re: Some thoughts on QS Don Dailey Mon Jul 30, 2012 5:23 pm
                                                                                          Re: Some thoughts on QS Robert Houdart Mon Jul 30, 2012 5:39 pm
                                                                                          Re: Some thoughts on QS Uri Blass Mon Jul 30, 2012 6:20 pm
                                                                                          Re: Some thoughts on QS Don Dailey Mon Jul 30, 2012 7:32 pm
                                                                                          Re: Some thoughts on QS Uri Blass Mon Jul 30, 2012 7:46 pm
                                                                                          Re: Some thoughts on QS Don Dailey Mon Jul 30, 2012 8:21 pm
                                                                                          Re: Some thoughts on QS Uri Blass Mon Jul 30, 2012 9:27 pm
                                                                                          Re: Some thoughts on QS Don Dailey Mon Jul 30, 2012 11:02 pm
                                                                                          Re: Some thoughts on QS Ronald de Man Tue Jul 31, 2012 12:18 am
                                                                                          Re: Some thoughts on QS Uri Blass Tue Jul 31, 2012 1:31 am
                                                                                          Re: Some thoughts on QS Sven Schüle Tue Jul 31, 2012 10:21 am
                                                                                          Re: Some thoughts on QS Uri Blass Tue Jul 31, 2012 11:10 am
                                                                                          Re: Some thoughts on QS Adam Hair Tue Jul 31, 2012 5:36 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Mon Jul 30, 2012 6:51 pm
                                                                                          Re: Some thoughts on QS Don Dailey Mon Jul 30, 2012 7:46 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Tue Jul 31, 2012 1:51 pm
                                                                                          Re: Some thoughts on QS Don Dailey Tue Jul 31, 2012 3:13 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Tue Jul 31, 2012 5:25 pm
                                                                                          Re: Some thoughts on QS Don Dailey Tue Jul 31, 2012 5:40 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Tue Jul 31, 2012 9:54 pm
                                                                                          Re: Some thoughts on QS Miguel A. Ballicora Tue Jul 31, 2012 10:12 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Tue Jul 31, 2012 10:22 pm
                                                                                          Re: Some thoughts on QS Uri Blass Wed Aug 01, 2012 4:17 am
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Wed Aug 01, 2012 9:57 am
                                                                                          Re: Some thoughts on QS Uri Blass Wed Aug 01, 2012 10:31 am
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Wed Aug 01, 2012 11:38 pm
                                                                                          Re: Some thoughts on QS Uri Blass Thu Aug 02, 2012 3:21 am
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Sat Aug 04, 2012 6:14 pm
                                                                                          Re: Some thoughts on QS Uri Blass Sun Aug 05, 2012 9:28 am
                                                                                          Re: Some thoughts on QS Robert Houdart Wed Aug 01, 2012 10:40 am
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Wed Aug 01, 2012 11:36 pm
                                                                                          Re: Some thoughts on QS Uri Blass Thu Aug 02, 2012 3:13 am
                                                                                          Re: Some thoughts on QS Uri Blass Sat Jul 28, 2012 6:54 pm
                                                                                          Re: Some thoughts on QS Ed Schroder Sat Jul 28, 2012 5:29 pm
                                                                                          Re: Some thoughts on QS Don Dailey Sat Jul 28, 2012 5:36 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Tue Jul 31, 2012 1:37 pm
                                                                                          Re: Some thoughts on QS Vincent Diepeveen Tue Jul 31, 2012 1:21 pm
                                                                        Re: Some thoughts on QS Gerd Isenberg Thu Jul 26, 2012 3:20 pm
                                                                        Re: Some thoughts on QS Don Dailey Thu Jul 26, 2012 3:35 pm
                                                                              Re: Some thoughts on QS Ed Schroder Thu Jul 26, 2012 5:27 pm
                                                                  Re: Some thoughts on QS Gerd Isenberg Thu Jul 26, 2012 3:57 pm
                                          Re: Some thoughts on QS Vincent Diepeveen Tue Jul 24, 2012 4:28 pm
                                                Re: Some thoughts on QS Don Dailey Tue Jul 24, 2012 4:40 pm
                                                      Re: Some thoughts on QS Vincent Diepeveen Wed Jul 25, 2012 12:08 pm
            Re: Some thoughts on QS H.G.Muller Fri Jul 20, 2012 7:30 am
      Re: Some thoughts on QS Vincent Diepeveen Fri Jul 20, 2012 1:02 am
            Re: Some thoughts on QS H.G.Muller Fri Jul 20, 2012 8:45 am
                  Re: Some thoughts on QS Vincent Diepeveen Fri Jul 20, 2012 11:33 am
                        Re: Some thoughts on QS H.G.Muller Fri Jul 20, 2012 12:06 pm
                              Re: Some thoughts on QS Vincent Diepeveen Fri Jul 20, 2012 12:50 pm
                                    Re: Some thoughts on QS H.G.Muller Fri Jul 20, 2012 1:23 pm
                                          Re: Some thoughts on QS Vincent Diepeveen Fri Jul 20, 2012 2:41 pm
                                                Re: Some thoughts on QS Matthew R. Brades Fri Jul 20, 2012 4:39 pm
                                          Re: Some thoughts on QS H.G.Muller Mon Jul 23, 2012 9:51 am
                                                Re: Some thoughts on QS H.G.Muller Thu Jul 26, 2012 5:33 pm
                                                      Re: Some thoughts on QS H.G.Muller Sat Jul 28, 2012 5:00 pm
            Re: Some thoughts on QS Uri Blass Fri Jul 20, 2012 4:48 pm
                  Re: Some thoughts on QS Vincent Diepeveen Fri Jul 20, 2012 6:30 pm
                        Re: Some thoughts on QS Uri Blass Fri Jul 20, 2012 9:22 pm
                              Re: Some thoughts on QS Vincent Diepeveen Fri Jul 20, 2012 9:33 pm
                                    Re: Some thoughts on QS Uri Blass Fri Jul 20, 2012 9:45 pm
                                          Re: Some thoughts on QS Vincent Diepeveen Fri Jul 20, 2012 10:10 pm
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: Programming and Technical Discussions

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads