ChessUSA.com TalkChess.com
Hosted by Your Move Chess & Games
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Who is stronger at chess? Computers or Humans?
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: General Topics Flat
View previous topic :: View next topic  

Who is stronger? Computer or Humans? How much?
Computer +500 elo
29%
 29%  [ 16 ]
Computer 301-500elo
27%
 27%  [ 15 ]
Computer 101-300 elo
32%
 32%  [ 18 ]
Computer 1-100 elo
3%
 3%  [ 2 ]
Equality
1%
 1%  [ 1 ]
Human 1-100 elo
3%
 3%  [ 2 ]
Human 101-300 elo
0%
 0%  [ 0 ]
Human 301-500 elo
0%
 0%  [ 0 ]
Human +500 elo
1%
 1%  [ 1 ]
Voted : 55
Total Votes : 55
- Poll expired -

Author Message
Milos Stanisavljevic



Joined: 25 Nov 2009
Posts: 1162
PostPost subject: Re: Who is stronger at chess? Computers or Humans?    Posted: Mon Feb 27, 2012 5:40 pm Reply to topic Reply with quote
Adam Hair wrote:
I used these time controls because I needed to know the change in Elo with thinking times in this range. I do not expect that the Elo increase holds for longer time controls. But this is what I had in mind when I said "shorter thinking times". However, due to search improvements and hardware improvements, I would assume that the depths reached at these time controls may not be inferior to those reached in the tests that determined the 50 to 70 Elo increase per doubling. Which I assumed occurred before How Computers Play Chess was published, where this "fact" was stated.

When the article was published EBFs were much higher at those depths (which you effectively test with your ultra-short TCs) than today. That's one reason you should not test at those TC's (at longer TC's average EBF is a much better representation of the strength of the program).
Second reason is that at ultra-short TCs (where time per move is much lower than 1sec) a time you dedicate to a move is much smaller than the real time engine spends searching. That's because the overhead of interface, time management, uncertainty of time measurement, SMP implementation, etc. plays much higher % in total time per move engine has. This reduces when time per move is increased and when time per move goes over 1 sec it becomes practically negligible.

Quote:
By the way, I think that extremely fast time controls are used in engine tuning in order to accumulate enough games in a reasonably amount of time in order to achieve some statistical significance. At least that is the rationale most authors use.

It's both. In order to see if the change is positive or not (to be included in the release) you need that absolute value of a change is higher than the variance of the measurement of that value. Variance you reduce by higher number of games (which is faster with ultra-short TCs) and absolute value you increase (exaggerate) with ultra-short TCs.

Quote:
In my tests, each engine played every other engine at the same time control. Based on those results, the opponents for a certain engine at a certain time control were determined. Those opponents were in a +/- 100 Elo interval from the intermediate rating for that engine/TC.

That is wrong setup.
An engine that you test at different TCs has to have same opponents all the time and its opponents need to have the same thinking time in all the matches, i.e. thinking time of the opponent has to be independent of the thinking time of the tested engine.
Therefore I suggest you select the pool of 5 test engines (with the strength range from the fastest TC of tested engine to slowest TC of tested engine) and test then against a single engine that plays at 3 TCs, for example at 1'+1'', 2'+2'' and 4'+4''.
P.S. All the tests should be done on a single core per engine (no SMP involved).
Back to top
View user's profile Send private message
Display posts from previous:   
Subject Author Date/Time
Who is stronger at chess? Computers or Humans? Maurizio Maglio Tue Feb 21, 2012 6:19 pm
      Re: Who is stronger at chess? Computers or Humans? Dr.Wael Deeb Tue Feb 21, 2012 6:28 pm
            Re: Who is stronger at chess? Computers or Humans? Maurizio Maglio Tue Feb 21, 2012 6:32 pm
      Re: Who is stronger at chess? Computers or Humans? Andrew R. Case Tue Feb 21, 2012 6:45 pm
            Re: Who is stronger at chess? Computers or Humans? Maurizio Maglio Tue Feb 21, 2012 6:50 pm
            Re: Who is stronger at chess? Computers or Humans? Julien MARCEL Tue Feb 21, 2012 7:01 pm
      Re: Who is stronger at chess? Computers or Humans? Sedat Canbaz Tue Feb 21, 2012 8:26 pm
            Re: Who is stronger at chess? Computers or Humans? Maurizio Maglio Tue Feb 21, 2012 8:45 pm
                  Re: Who is stronger at chess? Computers or Humans? Uri Zlatnik Tue Feb 21, 2012 10:08 pm
                        Re: Who is stronger at chess? Computers or Humans? Julien MARCEL Tue Feb 21, 2012 10:10 pm
                              Re: Who is stronger at chess? Computers or Humans? Uri Zlatnik Tue Feb 21, 2012 10:13 pm
                              Re: Who is stronger at chess? Computers or Humans? Dr.Wael Deeb Tue Feb 21, 2012 10:15 pm
                              Re: Who is stronger at chess? Computers or Humans? Sedat Canbaz Tue Feb 21, 2012 10:43 pm
                              Re: Who is stronger at chess? Computers or Humans? Uri Zlatnik Sun Feb 26, 2012 5:54 pm
                                    Re: Who is stronger at chess? Computers or Humans? Ignacio Garcia Mon Feb 27, 2012 2:03 am
                        Re: Who is stronger at chess? Computers or Humans? Dr.Wael Deeb Tue Feb 21, 2012 10:11 pm
                              Re: Who is stronger at chess? Computers or Humans? Uri Zlatnik Tue Feb 21, 2012 11:24 pm
                                    Re: Who is stronger at chess? Computers or Humans? Julien MARCEL Tue Feb 21, 2012 11:29 pm
                                    Re: Who is stronger at chess? Computers or Humans? Sedat Canbaz Tue Feb 21, 2012 11:32 pm
                                    Re: Who is stronger at chess? Computers or Humans? Maurizio Maglio Wed Feb 22, 2012 9:15 am
                                          Re: Who is stronger at chess? Computers or Humans? Sedat Canbaz Wed Feb 22, 2012 11:53 am
                                                Re: Who is stronger at chess? Computers or Humans? Uri Zlatnik Thu Feb 23, 2012 3:33 am
                                                      Re: Who is stronger at chess? Computers or Humans? Sedat Canbaz Thu Feb 23, 2012 8:49 am
                                                Re: Who is stronger at chess? Computers or Humans? Milos Stanisavljevic Thu Feb 23, 2012 5:14 pm
                                                      Re: Who is stronger at chess? Computers or Humans? Sedat Canbaz Thu Feb 23, 2012 7:43 pm
                                                            Re: Who is stronger at chess? Computers or Humans? Uri Zlatnik Thu Feb 23, 2012 11:15 pm
                                                            Re: Who is stronger at chess? Computers or Humans? Milos Stanisavljevic Fri Feb 24, 2012 4:09 am
                                                                  Re: Who is stronger at chess? Computers or Humans? Sedat Canbaz Fri Feb 24, 2012 10:48 am
                                                                        Re: Who is stronger at chess? Computers or Humans? Milos Stanisavljevic Fri Feb 24, 2012 1:10 pm
                                                                  Re: Who is stronger at chess? Computers or Humans? Sedat Canbaz Fri Feb 24, 2012 1:23 pm
                                                                        Re: Who is stronger at chess? Computers or Humans? Milos Stanisavljevic Fri Feb 24, 2012 1:51 pm
                                                                              Re: Who is stronger at chess? Computers or Humans? Sedat Canbaz Fri Feb 24, 2012 2:58 pm
                                                                                    Re: Who is stronger at chess? Computers or Humans? Sedat Canbaz Fri Feb 24, 2012 6:46 pm
                                                                  Re: Who is stronger at chess? Computers or Humans? Adam Hair Fri Feb 24, 2012 2:12 pm
                                                                        Re: Who is stronger at chess? Computers or Humans? Milos Stanisavljevic Fri Feb 24, 2012 2:39 pm
                                                                              Re: Who is stronger at chess? Computers or Humans? Adam Hair Fri Feb 24, 2012 5:10 pm
                                                                                    Re: Who is stronger at chess? Computers or Humans? Milos Stanisavljevic Fri Feb 24, 2012 10:25 pm
                                                                                          Re: Who is stronger at chess? Computers or Humans? Sedat Canbaz Sat Feb 25, 2012 1:17 am
                                                                                          Re: Who is stronger at chess? Computers or Humans? Milos Stanisavljevic Sat Feb 25, 2012 2:57 am
                                                                                          Re: Who is stronger at chess? Computers or Humans? Sedat Canbaz Sat Feb 25, 2012 9:42 am
                                                                                          Re: Who is stronger at chess? Computers or Humans? Adam Hair Sat Feb 25, 2012 3:34 am
                                                                                          Re: Who is stronger at chess? Computers or Humans? Milos Stanisavljevic Sun Feb 26, 2012 11:28 pm
                                                                                          Re: Who is stronger at chess? Computers or Humans? Adam Hair Mon Feb 27, 2012 4:39 am
                                                                                          Re: Who is stronger at chess? Computers or Humans? Milos Stanisavljevic Mon Feb 27, 2012 5:40 pm
                                                                                          Re: Who is stronger at chess? Computers or Humans? Adam Hair Tue Feb 28, 2012 2:04 am
                                                                                    Re: Who is stronger at chess? Computers or Humans? Norman Schmidt Sat Feb 25, 2012 3:36 am
                                                                                          Re: Who is stronger at chess? Computers or Humans? Adam Hair Sat Feb 25, 2012 3:46 am
                                                                                          Re: Who is stronger at chess? Computers or Humans? Norman Schmidt Sat Feb 25, 2012 3:52 am
                                                                                          Re: Who is stronger at chess? Computers or Humans? Norman Schmidt Sat Feb 25, 2012 4:05 am
                                                                                          Re: Who is stronger at chess? Computers or Humans? Adam Hair Mon Feb 27, 2012 4:49 am
                                                                                          Re: Who is stronger at chess? Computers or Humans? Ed Schroder Mon Feb 27, 2012 10:51 am
                                                                                          Re: Who is stronger at chess? Computers or Humans? Milos Stanisavljevic Mon Feb 27, 2012 4:19 pm
                                                                                          Re: Who is stronger at chess? Computers or Humans? Ed Schroder Mon Feb 27, 2012 7:21 pm
                                                                                          Re: Who is stronger at chess? Computers or Humans? Carl Bicknell Mon Feb 27, 2012 10:26 pm
                                                                                          Re: Who is stronger at chess? Computers or Humans? Oliver Brausch Tue Feb 28, 2012 4:15 am
                                                                                          Re: Who is stronger at chess? Computers or Humans? Carl Bicknell Tue Feb 28, 2012 8:58 am
                                                                                          Re: Who is stronger at chess? Computers or Humans? Ricardo Barreira Tue Feb 28, 2012 9:29 am
                                                                                          Re: Who is stronger at chess? Computers or Humans? Uri Zlatnik Fri Mar 09, 2012 9:47 pm
                                          Re: Who is stronger at chess? Computers or Humans? Guy Marfan Mon Feb 27, 2012 2:52 pm
                                    Re: Who is stronger at chess? Computers or Humans? Dr.Wael Deeb Wed Feb 22, 2012 10:43 am
                                    Re: Who is stronger at chess? Computers or Humans? Milos Stanisavljevic Thu Feb 23, 2012 5:27 pm
                              Re: Who is stronger at chess? Computers or Humans? Ricardo Barreira Wed Feb 22, 2012 9:24 am
                  Re: Who is stronger at chess? Computers or Humans? Sedat Canbaz Tue Feb 21, 2012 11:15 pm
      Re: Who is stronger at chess? Computers or Humans? Sean Evans Sun Feb 26, 2012 12:00 am
            Re: Who is stronger at chess? Computers or Humans? Sedat Canbaz Sun Feb 26, 2012 10:07 am
      Re: Who is stronger at chess? Computers or Humans? Eduard Nemeth Tue Feb 28, 2012 2:12 am
Post new topic    TalkChess.com Forum Index -> Computer Chess Club: General Topics

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum




Powered by phpBB © 2001, 2005 phpBB Group
Enhanced with Moby Threads