TalkChess.com
Hosted by Your Move Chess & Games

Author Message

Joined: 06 May 2009
Posts: 1638
Location: Fuquay-Varina, North Carolina

Post subject: Re: Critter 1.4a x64 SSE4 vs Houdini 2.0c Pro x64 match resu    Posted: Sat Mar 17, 2012 4:25 pm

Houdini wrote:
 Adam Hair wrote: So, using the results from Ingo's match between Critter 1.4a and Houdini 2.0c, we could compute a prediction interval. From Ingo's data, 52.67% +/- 7.04% or (45.63, 59.71) is a 95% prediction interval for the score of Brent's match. In fact, one might predict that there was approximately 98% chance that the score for Houdini would be less than 60% (based on Ingo's results).

Adam, that's incorrect. The confidence interval (+- 7%) is only valid with respect to the expected "true" value, not with respect to another random sample.

If you want to use Ingo's random sample to predict Brent's random sample, you have to multiply the uncertainty margin by 1.4: the confidence interval becomes 53% +- 10%. In other words, from Ingo's result you can deduce that another similar match will produce with 95% certainty a result between 43% and 63%. Brent's result lies comfortably within these boundaries.

A more straightforward way of looking at the results is as follows.
Ingo obtains 53 %, Brent obtains 60 %. Both results are perfectly compatible with the 55% +- 7% result predicted for a 100-game match and about 40 Elo established rating difference.

Robert

I am using the results of one random sample to predict (although after the fact) the results of another random sample. The computation of a prediction interval makes use of the sample mean, sample variance, and Student's T distribution, but not Sqrt(2).

If I were to compare Ingo's result to Brent's result, then Sqrt(2) would come into play.

As you noted, the two results are not at odds with each other. However, we probably would not have predicted Brent's results from looking solely at Ingo's results, which by the way are the most similar to Brent's conditions.

 Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First
Subject Author Date/Time
Brent M Fri Mar 16, 2012 2:50 pm
Robert Houdart Fri Mar 16, 2012 3:15 pm
Brent M Fri Mar 16, 2012 3:23 pm
Robert Houdart Fri Mar 16, 2012 4:08 pm
Adam Hair Sat Mar 17, 2012 1:02 pm
Robert Houdart Sat Mar 17, 2012 2:27 pm
Marijan Stipancic jr Sat Mar 17, 2012 3:37 pm
Robert Houdart Sun Mar 18, 2012 1:49 am
Marijan Stipancic jr Sun Mar 18, 2012 2:19 am
Bram Mourik Sun Mar 18, 2012 11:03 am
Robert Houdart Sun Mar 18, 2012 12:38 pm
Re: Critter 1.4a x64 SSE4 vs Houdini 2.0c Pro x64 match resu Adam Hair Sat Mar 17, 2012 4:25 pm
Robert Houdart Sat Mar 17, 2012 7:32 pm
Adam Hair Sat Mar 17, 2012 9:43 pm
Robert Houdart Sat Mar 17, 2012 10:43 pm
Adam Hair Sun Mar 18, 2012 12:26 am
Brent M Mon Mar 19, 2012 12:33 am
Robert Houdart Mon Mar 19, 2012 11:38 am
Brent M Mon Mar 19, 2012 12:24 pm
Richard Vida Mon Mar 19, 2012 12:57 pm
Brent M Mon Mar 19, 2012 1:03 pm
pal larkin Wed Mar 21, 2012 3:07 am
Richard Allbert Sun Mar 18, 2012 1:25 pm
Robert Houdart Sun Mar 18, 2012 2:03 pm
Richard Allbert Sun Mar 18, 2012 3:35 pm
Dr.Wael Deeb Sun Mar 18, 2012 5:59 pm

 Jump to: Select a forum Computer Chess Club Forums----------------Computer Chess Club: General TopicsComputer Chess Club: Tournaments and MatchesComputer Chess Club: Programming and Technical DiscussionsComputer Chess Club: Engine Origins Other Forums----------------Chess Thinkers ForumForum Help and Suggestions
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum