TalkChess.com :: View topic - Number 1 engine on long time controls

TalkChess.com
Hosted by Your Move Chess & Games

FAQ

Memberlist

Usergroups

Profile

Number 1 engine on long time controls

TalkChess.com Forum Index -> Computer Chess Club: General Topics

View previous topic :: View next topic

Author

Message

Jesús Muñoz

Joined: 13 Jul 2011
Posts: 692
Location: Madrid, Spain.

Post subject: Re: Number 1 engine on long time controls. Posted: Mon Feb 27, 2012 6:50 pm

Hello:

Milos wrote:

Uri Blass wrote:

I disagree about the following:
"if it is gaining more ELO per doubling that Houdni, then the logicial conclusion is that it WILL overtake it."

One example may be enough to prove that the conclusion is wrong and
I guess that houdini 32 bits gain more ELO per doubling than houdini 64 bits because of diminishing returns but houdini 32 bits is not going to overtake houdini 64 bits at long time control.

I think that you need to start with not equal time control but with time control that gives result that is close to 50%

If you find that the program that use more time earns more from doubling
in this case then it is more logical to think that it can beat the stronger program at long time control and even in this case it is not something that I feel sure about it.

Finally someone who understands and doesn't write the usual nonsense.
I agree with everything you write.
Don is as usual writing a load of crap and pretending he doesn't understand (for marketing reasons).
He uses extremely short time controls where the initial difference between 2 engines is exaggerated and then tries to imply the conclusion on something that is more than 3 orders of magnitude longer TC. This is at least 10 doublings and just for fun I extrapolated his table to long TCs (40moves/120mins) using the simplest linear extrapolation.
So lets see (40moves/120mins is 3min/move and is approximately equivalent to 6000sec+100sec - 10 doublings - Level 10)

Code:

Level where 00 is 6 + 0.1 and each successive level is double.
Komodo
HOUDINI gains
------- ------
Level 00 - +143.3
Leval 01 - +97.0 +46.3
Leval 02 - +74.6 +22.4
Level 03 - +52.8 +21.8
Level 04 - +39.5 +13.3
Level 05 - +27.0 +12.5
Level 06 - +14.5
Leval 07 - +2.0
Level 08 - -10.5
Level 09 - -23.0
Level 10 - -35.5

At 40moves/120mins Komodo should be 35.5 Elo stronger than Houdini. (even with shape-preserving cubic extrapolation, at the end Komodo should still be 22Elo stronger!)
In reality Komodo is not stronger at any time control, eventually there might be a point where it becomes equal (asymptotic point with extremely large time per move).

So Don, please stop with nonsense, not all the ppl here are idiots to believe your marketing...

I have seen that Milos has done an extrapolation and I also want to post my own one. With the data provided by Don:

Code:

Level where 00 is 6 + 0.1 and each successive level is double.

Komodo
HOUDINI gains
------- ------
Level 00 - +143.3
Leval 01 - +97.0 +46.3
Leval 02 - +74.6 +22.4
Level 03 - +52.8 +21.8
Level 04 - +39.5 +13.3
Level 05 - +27.0 +12.5

I have done an adjust by least squares by hand, with the only help of a Casio calculator (I hope no errors in my calculations). I explain what I have done a little more:

Code:

x axis ---> the level (time control).
y axis ---> Houdini Elo advantage over Komodo.

I did a rescale in the x axis in the following way: x axis is in logarithmic scale (basis e), and I add 1 to each level for avoid ln(0). So, when I put ln(1), it means Don's level 0, and so on; level 10 pointed out by Milos is ln(11) in my x axis. I calculated the least squares by hand with a system of equations, in this way:

Code:

Y(x) = m·ln(x) + n
Y numbers go to 'HOUDINI' column.

AX = B

Symmetric matrix of size 2x2: A = [N, SUM(x_i); SUM(x_i), SUM((x_i)²)]
Vector of size 2x1: X = [n, m]
Vector of size 2x1: B = [SUM(y_i), SUM((x_i)·(y_i))]

N is the number of (x, y) known data; here: N = 6.
SUM(x_i) = ln(1) + ln(2) + ... + ln(6) ~ 6.5793
SUM((x_i)²) =[ln(1)]² + [ln(2)]² + ... + [ln(6)]² ~ 9.4099
SUM(y_i) = 143.3 + 97 + ... + 27 = 434.2
SUM(x_i·y_i) = ln(1)·143.3 + ln(2)·97 + ... + ln(6)·27 ~ 334.3384

Det.(A) = N·SUM((x_i)²) - [SUM(x_i)]² ~ 13.1729

Solving by Cramer, I got (more less, roundings included):

Code:

n ~ 143.1793
m ~ -64.5781

So, renaming the level names to their original names (from 0 to 10 instead from 1 to 11) and adding a couple of levels, this is what I get:

Code:

OWN ADJUST:
===========

Level where 00 is 6 + 0.1 and each successive level is double.

Komodo
HOUDINI gains
------- ------
Level 00 - +143.2
Leval 01 - +98.4 +44.8
Leval 02 - +72.2 +26.2
Level 03 - +53.7 +18.5
Level 04 - +39.2 +14.5
Level 05 - +27.5 +11.7
Level 06 - +17.5 +10
Level 07 - +8.9 +8.6
Level 08 - +1.3 +7.6
Level 09 - -5.5 +6.8
Level 10 - -11.7 +6.2
Level 11 - -17.3 +5.6
Level 12 - -22.5 +5.2

So, I get smaller differences than Milos with my model. Given the fact that 2000 games are played in each level, I expect (at not very short time controls, where the draw ratio should rise with longer TC) uncertainties less than ±13 or ±14 Elo (with 95% confidence), that should not be forgotten. With the model I work: uncertainties are proportional to score(Houdini)*score(Komodo) - (draw_ratio)/4, so more draws mean less uncertainties. This model works fine in ranges of scores from 10% to 90%, which is the case here.

I wonder who of us (Milos and me) is more accurate in the extrapolation with the data given (I know that there are tons of ways of extrapolations). I see that in Milos' extrapolation, the missing numbers in 'Komodo gains' column are: {+12.5, +12.5, +12.5, +12.5, +12.5} (a very simple extrapolation, as Milos stated), which does not seem accurate for me because everyone can expect less gains with higher levels (in this sense, my extrapolation is a bit better). With cubic extrapolation, Milos gets -22 in 'HOUDINI' column (I assume that in level 10), which gets closer to my extrapolation, but is still a little far.

As each level doubles the previous level, maybe it could have more sense that my x axis was rescaled with logarithms of basis 2 (instead basis e), but it was much easier to me doing the math with natural logarithms. Comments, corrections, etc. are welcome, so please leave your impressions. Good luck to all the programmers with their engines!

Regards from Spain.

Ajedrecista.
_________________
Six Fortran 95 tools.

Chess will never be solved.

Last edited by Jesús Muñoz on Mon Feb 27, 2012 7:03 pm; edited 1 time in total

Display posts from previous:

Subject	Author	Date/Time
Number 1 engine on long time controls	Teodoriu Catalin	Fri Feb 24, 2012 7:17 pm
Re: NUMBER 1 ENGINE ON LONG TIME CONTROLS	Uri Blass	Fri Feb 24, 2012 7:52 pm
Re: NUMBER 1 ENGINE ON LONG TIME CONTROLS	Teodoriu Catalin	Sat Feb 25, 2012 8:02 am
Re: Number 1 engine on long time controls	Don Dailey	Fri Feb 24, 2012 10:01 pm
Re: Number 1 engine on long time controls	Lonnie Cook	Fri Feb 24, 2012 10:21 pm
Re: Number 1 engine on long time controls	Don Dailey	Fri Feb 24, 2012 11:16 pm
Re: Number 1 engine on long time controls	Robert Houdart	Fri Feb 24, 2012 11:40 pm
Re: Number 1 engine on long time controls	Don Dailey	Fri Feb 24, 2012 11:51 pm
Re: Number 1 engine on long time controls	Robert Houdart	Sat Feb 25, 2012 12:13 am
Re: Number 1 engine on long time controls	Ignacio Garcia	Sat Feb 25, 2012 12:29 am
Re: Number 1 engine on long time controls	Don Dailey	Sat Feb 25, 2012 12:30 am
Re: Number 1 engine on long time controls	Tom Giampietro	Sat Feb 25, 2012 1:05 am
Re: Number 1 engine on long time controls	Don Dailey	Sat Feb 25, 2012 1:40 am
Re: Number 1 engine on long time controls	Robert Houdart	Sat Feb 25, 2012 1:52 am
Re: Number 1 engine on long time controls	Peter Skinner	Sat Feb 25, 2012 5:54 am
Re: Number 1 engine on long time controls	Carl Bicknell	Sat Feb 25, 2012 8:57 am
Re: Number 1 engine on long time controls	Ben-Hur Carlos Langoni	Sat Feb 25, 2012 10:22 pm
Re: Number 1 engine on long time controls	Carl Bicknell	Sun Feb 26, 2012 8:00 am
Re: Number 1 engine on long time controls	Ben-Hur Carlos Langoni	Sun Feb 26, 2012 1:04 pm
Re: Number 1 engine on long time controls	George Speight	Sat Feb 25, 2012 9:37 am
Re: Number 1 engine on long time controls	Uri Blass	Sun Feb 26, 2012 10:43 am
Re: Number 1 engine on long time controls	Sedat Canbaz	Sun Feb 26, 2012 11:54 am
Re: Number 1 engine on long time controls	Ben-Hur Carlos Langoni	Sun Feb 26, 2012 1:17 pm
Re: Number 1 engine on long time controls	Sedat Canbaz	Sun Feb 26, 2012 2:05 pm
Re: Number 1 engine on long time controls	Sedat Canbaz	Sun Feb 26, 2012 2:57 pm
Re: Number 1 engine on long time controls	Don Dailey	Sun Feb 26, 2012 1:57 pm
Re: Number 1 engine on long time controls	Uri Blass	Sun Feb 26, 2012 2:33 pm
Re: Number 1 engine on long time controls	Ignacio Garcia	Sun Feb 26, 2012 3:41 pm
Re: Number 1 engine on long time controls	Don Dailey	Sun Feb 26, 2012 4:51 pm
Re: Number 1 engine on long time controls	Ignacio Garcia	Sun Feb 26, 2012 5:34 pm
Re: Number 1 engine on long time controls	Ricardo Barreira	Sun Feb 26, 2012 7:54 pm
Re: Number 1 engine on long time controls	Uri Blass	Sun Feb 26, 2012 10:53 pm
Re: Number 1 engine on long time controls	Don Dailey	Mon Feb 27, 2012 1:35 pm
Re: Number 1 engine on long time controls	Jouni Uski	Mon Feb 27, 2012 4:18 pm
Re: Number 1 engine on long time controls	Robert Houdart	Mon Feb 27, 2012 6:26 pm
Re: Number 1 engine on long time controls	Don Dailey	Mon Feb 27, 2012 6:50 pm
Re: Number 1 engine on long time controls	Larry Kaufman	Mon Feb 27, 2012 6:56 pm
Re: Number 1 engine on long time controls	Milos Stanisavljevic	Mon Feb 27, 2012 5:04 pm
Re: Number 1 engine on long time controls	Milos Stanisavljevic	Mon Feb 27, 2012 5:22 pm
Re: Number 1 engine on long time controls	Larry Kaufman	Mon Feb 27, 2012 6:31 pm
Re: Number 1 engine on long time controls	Don Dailey	Mon Feb 27, 2012 6:48 pm
Re: Number 1 engine on long time controls	Milos Stanisavljevic	Mon Feb 27, 2012 4:52 pm
Re: Number 1 engine on long time controls	Larry Kaufman	Mon Feb 27, 2012 5:11 pm
Re: Number 1 engine on long time controls	Milos Stanisavljevic	Mon Feb 27, 2012 5:19 pm
Re: Number 1 engine on long time controls	Don Dailey	Mon Feb 27, 2012 5:35 pm
Re: Number 1 engine on long time controls	Ignacio Garcia	Mon Feb 27, 2012 5:58 pm
Re: Number 1 engine on long time controls	Don Dailey	Mon Feb 27, 2012 6:38 pm
Re: Number 1 engine on long time controls	Ignacio Garcia	Tue Feb 28, 2012 2:40 pm
Re: Number 1 engine on long time controls.	Jesús Muñoz	Tue Feb 28, 2012 3:06 pm
Re: Number 1 engine on long time controls	Don Dailey	Tue Feb 28, 2012 9:06 pm
Re: Number 1 engine on long time controls	Ignacio Garcia	Wed Feb 29, 2012 11:22 am
Re: Number 1 engine on long time controls.	Jesús Muñoz	Mon Feb 27, 2012 6:50 pm
Re: Number 1 engine on long time controls.	Don Dailey	Tue Feb 28, 2012 12:30 am
Re: Number 1 engine on long time controls.	Jesús Muñoz	Tue Feb 28, 2012 11:31 am
Re: Number 1 engine on long time controls.	Don Dailey	Tue Feb 28, 2012 11:57 am
Re: Number 1 engine on long time controls.	Don Dailey	Tue Feb 28, 2012 12:06 pm
Re: Number 1 engine on long time controls.	Jouni Uski	Tue Feb 28, 2012 2:10 pm
Re: Number 1 engine on long time controls.	Don Dailey	Tue Feb 28, 2012 8:31 pm
Re: Number 1 engine on long time controls.	Bigler	Tue Feb 28, 2012 4:52 pm
Re: Number 1 engine on long time controls.	Larry Kaufman	Tue Feb 28, 2012 4:58 pm
Re: Number 1 engine on long time controls.	Bigler	Tue Feb 28, 2012 5:03 pm
Re: Number 1 engine on long time controls.	Larry Kaufman	Tue Feb 28, 2012 5:46 pm
Re: Number 1 engine on long time controls.	Terry McCracken	Wed Feb 29, 2012 5:37 am
Re: Number 1 engine on long time controls.	Carl Bicknell	Wed Feb 29, 2012 9:19 am
Re: Number 1 engine on long time controls.	Terry McCracken	Wed Feb 29, 2012 5:49 pm
Re: Number 1 engine on long time controls.	Dr.Wael Deeb	Wed Feb 29, 2012 5:59 pm
Re: Number 1 engine on long time controls.	Terry McCracken	Thu Mar 01, 2012 4:52 am
Re: Number 1 engine on long time controls.	Uri Blass	Tue Feb 28, 2012 6:05 pm
Re: Number 1 engine on long time controls	Bram Mourik	Tue Feb 28, 2012 9:01 pm
Re: Number 1 engine on long time controls	Don Dailey	Tue Feb 28, 2012 9:30 pm
Re: Number 1 engine on long time controls	Dr.Wael Deeb	Tue Feb 28, 2012 9:37 pm
Re: Number 1 engine on long time controls	Bram Mourik	Wed Feb 29, 2012 5:58 pm
Re: Number 1 engine on long time controls	Don Dailey	Wed Feb 29, 2012 6:25 pm
Re: Number 1 engine on long time controls	om prakash	Thu Mar 01, 2012 7:00 am
Re: Number 1 engine on long time controls	enrico fagiuoli	Wed Feb 29, 2012 8:43 pm
Re: Number 1 engine on long time controls	Leon Coleman	Wed Feb 29, 2012 5:23 am
Re: Number 1 engine on long time controls	Don Dailey	Wed Feb 29, 2012 12:47 pm
Re: Number 1 engine on long time controls	Vincent Diepeveen	Thu Mar 01, 2012 11:44 am
Re: Number 1 engine on long time controls	Uri Blass	Thu Mar 01, 2012 1:30 pm
Re: Number 1 engine on long time controls	Don Dailey	Thu Mar 01, 2012 1:34 pm
Re: Number 1 engine on long time controls	enrico fagiuoli	Thu Mar 01, 2012 2:57 pm
Re: Number 1 engine on long time controls	Don Dailey	Thu Mar 01, 2012 9:48 pm
Re: Number 1 engine on long time controls	enrico fagiuoli	Fri Mar 02, 2012 11:09 am
Re: Number 1 engine on long time controls	Vincent Diepeveen	Fri Mar 02, 2012 12:20 pm
Re: Number 1 engine on long time controls	Don Dailey	Fri Mar 02, 2012 12:26 pm
Re: Number 1 engine on long time controls	Vincent Diepeveen	Fri Mar 02, 2012 12:51 pm
Re: Number 1 engine on long time controls	Dr.Wael Deeb	Fri Mar 02, 2012 12:55 pm
Re: Number 1 engine on long time controls	Don Dailey	Fri Mar 02, 2012 1:49 pm
Re: Number 1 engine on long time controls	Dr.Wael Deeb	Fri Mar 02, 2012 3:10 pm
Re: Number 1 engine on long time controls	Don Dailey	Fri Mar 02, 2012 3:30 pm
Re: Number 1 engine on long time controls	Dr.Wael Deeb	Fri Mar 02, 2012 3:43 pm
Re: Number 1 engine on long time controls	Uri Blass	Fri Mar 02, 2012 8:14 pm
Re: Number 1 engine on long time controls	Dr.Wael Deeb	Fri Mar 02, 2012 8:31 pm
Re: Number 1 engine on long time controls	Uri Blass	Fri Mar 02, 2012 8:45 pm
Re: Number 1 engine on long time controls	Larry Kaufman	Fri Mar 02, 2012 8:58 pm
Re: Number 1 engine on long time controls	Dr.Wael Deeb	Fri Mar 02, 2012 9:26 pm
Re: Number 1 engine on long time controls	Dr.Wael Deeb	Fri Mar 02, 2012 9:23 pm
Re: Number 1 engine on long time controls	Carl Bicknell	Fri Mar 02, 2012 5:41 pm
Re: Number 1 engine on long time controls	Larry Kaufman	Fri Mar 02, 2012 6:59 pm
Re: Number 1 engine on long time controls	Dr.Wael Deeb	Fri Mar 02, 2012 7:10 pm
Re: Number 1 engine on long time controls	Carl Bicknell	Sat Mar 03, 2012 3:49 pm
Re: Number 1 engine on long time controls	Dr.Wael Deeb	Sun Mar 04, 2012 8:36 am
Re: Number 1 engine on long time controls	Robert Flesher	Fri Mar 02, 2012 9:28 pm
Re: Number 1 engine on long time controls	Dr.Wael Deeb	Fri Mar 02, 2012 9:55 pm
Re: Number 1 engine on long time controls	Jon Dart	Sat Feb 25, 2012 9:09 pm
Re: Number 1 engine on long time controls	Andrew R. Case	Sat Feb 25, 2012 9:14 pm
Re: Number 1 engine on long time controls	Gabor Szots	Thu Mar 01, 2012 2:19 pm
Re: Number 1 engine on long time controls	Larry Kaufman	Thu Mar 01, 2012 3:21 pm
Re: Number 1 engine on long time controls	om prakash	Thu Mar 01, 2012 4:16 pm
Re: Number 1 engine on long time controls	Larry Kaufman	Thu Mar 01, 2012 4:51 pm
Re: Number 1 engine on long time controls	om prakash	Thu Mar 01, 2012 4:54 pm
Re: Number 1 engine on long time controls	George Speight	Sun Mar 04, 2012 9:42 am
Re: Number 1 engine on long time controls	Jouni Uski	Sun Mar 04, 2012 11:35 am
Re: Number 1 engine on long time controls	Maurizio Maglio	Sun Mar 04, 2012 11:41 am
Re: Number 1 engine on long time controls	Don Dailey	Fri Mar 02, 2012 1:34 pm

TalkChess.com Forum Index -> Computer Chess Club: General Topics

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum