harware vs software advances

[Suji]

Because in 1995 I had a _better_ parallel search implementation than I do today. In fact, in 1989 I had that _same_ "better implementation."

Random questions of the day for Dr. Hyatt:

Since you had a better parallel search implementation 15+ years ago, you had to have changed it somewhere in between then and now.

First question: Why did you change implementations?

and

Second question: If you changed back to the "better implementation" now with current Crafty, would current Crafty's play improve?

[rbarreira]

Because in 1995 I had a _better_ parallel search implementation than I do today. In fact, in 1989 I had that _same_ "better implementation."

Random questions of the day for Dr. Hyatt:

Since you had a better parallel search implementation 15+ years ago, you had to have changed it somewhere in between then and now.

First question: Why did you change implementations?

and

Second question: If you changed back to the "better implementation" now with current Crafty, would current Crafty's play improve?

Well I am not him but I've seen him mentioning that a couple of times... The other parallel algorithm is DTS, explained at this article:

http://www.cis.uab.edu/info/faculty/hyatt/search.html

It is quite a bit more complex than the algorithms most people use, and requires a non-recursive search to be implemented correctly. An iterative search is, of course, a pain in the ass to implement, and that's just half of the work to get DTS working.

[mhull]

Because in 1995 I had a _better_ parallel search implementation than I do today. In fact, in 1989 I had that _same_ "better implementation."

Random questions of the day for Dr. Hyatt:

Since you had a better parallel search implementation 15+ years ago, you had to have changed it somewhere in between then and now.

First question: Why did you change implementations?

and

Second question: If you changed back to the "better implementation" now with current Crafty, would current Crafty's play improve?

Probably because hardware changed which makes parallel computing less efficient. There was true SMP up to 32-way with Cray crossbar technology (extremely expensive, e.g. Cray T90 series, the T932 cost $39 million) and later some limited SMP with intel and interleaved memory access, then along came NUMA (less efficient, but cheaper) and now MCMs (multi-chip modules) ganged up on dual and quad socket configurations, which might be some kind of hybrid of SMP and NUMA. Not sure how they handle memory across MCMs and sockets.

So it cannot be as efficient as it was in Cray days. Also, try to wrap your mind around 8 gigabytes of SRAM (Cray T90). Thats RAM faster than one can imagine even today. The speedup of programs would be immense, even running at lower clock speeds.

[Don]

By the way, please tell me why we have to pretend parallel programs did not exist in the mid 90's?

Are we reading the same posts? Bob was accusing you of treating parallel search as something new since 1995. Now you are accusing him of the same thing?

Show me where I said that.

Quoted above: "please tell me why we have to pretend parallel programs did not exist in the mid 90's?"

He went to great pains to say exactly the opposite.

My point is that if we are comparing actual HARDWARE advances since the mid 90's and clearly parallel programs existed, then it's blatantly unfair to disallow parallel technology from the 90's in this test but eagerly embrace parallel technology from 2010.

Are we actually interested in seeing how much we advanced or not? Or is that you guys only live in the Intel PC world? This discussion isn't about PC's and how much the Pentium has advanced, it's about hardware advancement in general. I was using 1824 processors long ago and 4 processor alpha's in the 90's.

I might be getting this wrong but I think Bob was using parallel processing in the late 80's! But running on 4 processors in 2010 is a huge advancement? This is totally not the case and I don't see how this trivial point escapes you.

If Bob is really interesting in measure ACTUAL hardware improvement we should be comparing the best parallel alpha system running in 1995 to the best chip we have running in 2010, the i7-980x which is a real product.

Bob, are you interested in a fair comparison or do you just want to compare the highly inferior Intel product to the very best chip we have today?

The alpha systems cost more than the Pentiums, but they were affordable (as high end systems go.)

[mhull]

By the way, please tell me why we have to pretend parallel programs did not exist in the mid 90's?

Are we reading the same posts? Bob was accusing you of treating parallel search as something new since 1995. Now you are accusing him of the same thing?

Show me where I said that.

Quoted above: "please tell me why we have to pretend parallel programs did not exist in the mid 90's?"

He went to great pains to say exactly the opposite.

My point is that if we are comparing actual HARDWARE advances since the mid 90's and clearly parallel programs existed, then it's blatantly unfair to disallow parallel technology from the 90's in this test but eagerly embrace parallel technology from 2010.

Are we actually interested in seeing how much we advanced or not? Or is that you guys only live in the Intel PC world? This discussion isn't about PC's and how much the Pentium has advanced, it's about hardware advancement in general. I was using 1824 processors long ago and 4 processor alpha's in the 90's.

I might be getting this wrong but I think Bob was using parallel processing in the late 80's! But running on 4 processors in 2010 is a huge advancement? This is totally not the case and I don't see how this trivial point escapes you.

If Bob is really interesting in measure ACTUAL hardware improvement we should be comparing the best parallel alpha system running in 1995 to the best chip we have running in 2010, the i7-980x which is a real product.

Bob, are you interested in a fair comparison or do you just want to compare the highly inferior Intel product to the very best chip we have today?

The alpha systems cost more than the Pentiums, but they were affordable (as high end systems go.)

A lot of poeple will naturally be interested in intel, since thats the platform they have played with in their own basement. And that's what crafty typically ran on in 1995. What he's doing now with new and old crafty will give some estimate on the benefits made in intel advances. It seems to me it's boiling down to NPS comparisons anyway, old versus new, from HW improvements, and then the delta will be software. That's what you're really after.

Now if one could slow down an old program and a new program on new hardware (introducing delay loops to lower average NPS), then test them on a cluster, you could calc the ELO on 1995 equivalent hardware. Take your pick of 1995 performance by dialing down the NPS to the 1995 level that hardware was recorded as producing at the time with that program. Don't know how hard that would be.

Maybe an x86 emulator could be hacked to accurately dial-down the effective Mhz. Then the old and new programs could be dialed down to 1995 speeds, but still tested on a cluster with thousands of games. Probably a tall order.

[Uri Blass]

To compare software, we need a 1995 program and 1995 hardware, and then the same 1995 program on 2010 hardware. But it doesn't need to be a lousy example of a 1995 program. Parallel search was old news in 1995. Genius never had that. So run it today and you immediately throw away 5/6 of the hardware advantage of today. Not reasonable, IMHO.

I think that it is not reasonable to take best of both worlds.

The combination of Genius and parallel search does not exist as software
so if you take parallel search as software from 1995 it means that you cannot take Genius3 and you need to find a program from 1995 when the programmer already used parallel search.

that's exactly what I have been saying...

It means that you can use Crafty of 1995 that is weaker than Genius of 1995 if you want to give the old program time advantage of 1000:1

Actually Crafty was not really weaker than genius in 1995. If you back up to the very early 90's (before Crafty) genius was a significant threat. But by 1995 I don't remember Genius really being an issue on chess servers or in tournaments...

I believe that Genius was better than Crafty on 1 cpu in 1995.
Genius was clearly the ssdf leading program in 1995 when ssdf used p90 for testing at 120/40 time control.

[mhull]

I'm pretty sure the Pentium 90 was the fastest processor at that time. Intel introduced the Pentium Pro family (to which the P200 belongs) in late 1995.

Yes, I'll assume that circa 1994 CG3 was running on the P90.

Any idea of the speed? I don't suppose anybody has a Pentium 90 laying around do they?

Don

I have an operational Pentium 90.

Good. What we need is a comparison on 2 different platforms, the P90 and the newest thing you have.

Are you willing to try the following experiment: ?

Find the oldest Crafty you can that runs on both platforms and run some kind of time test on both machines. Then do the same with the newest Crafty you can find. I don't know which old Crafty's you can still get but even if it's not very old it would still be useful. The challenge will be to find a program that still runs on both platforms.

I know that is probably a bit of work - but if you are willing to do it I would like to see the results.

What about running a Bochs IA32 emulator on modern hardware? Maybe it can be hacked to run at some arbitrary virtual clock speed.

[Uri Blass]

looking at the ssdf p90 results I find the following:

140 Chess Tiger 11.8 Pentium 90 MHz 2384 43 -43 261 50% 2385
145 Rebel 8.0 Pentium 90 MHz 2338 19 -19 1405 54% 2307
146 Rebel 9.0 Pentium 90 MHz 2337 23 -23 910 47% 2359
147 Hiarcs 6.0 Pentium 90 MHz 2335 18 -18 1427 50% 2332
148 Genius 5.0 DOS Pentium 90 MHz 2331 18 -18 1558 47% 2350
149 Hiarcs 5.0 Pentium 90 MHz 2328 38 -36 363 59% 2263
150 MChess Pro 6.0 Pentium 90 MHz 2313 17 -17 1706 45% 2347
151 Genius 3.0 Pentium 90 MHz 2310 25 -24 824 60% 2239
152 Rebel 6.0 Pentium 90 MHz 2306 31 -30 540 60% 2234
153 MChess Pro 5.0 Pentium 90 MHz 2305 26 -26 749 62% 2216
154 Genius 4.0 DOS Pentium 90 MHz 2304 23 -23 924 59% 2236
155 Rebel 7.0 Pentium 90 MHz 2297 25 -25 789 59% 2231
156 Nimzo 3.5 Pentium 90 MHz 2295 22 -22 998 46% 2325
157 Hiarcs 4.0 Pentium 90 MHz 2290 25 -25 786 54% 2261
158 Junior 4.0 Pentium 90 MHz 2289 22 -22 1035 42% 2343
159 Shredder 1.0 Pentium 90 MHz 2288 55 -55 165 50% 2291
160 Chessmaster 5000 Pentium 90 MHz 2287 49 -45 240 67% 2162
161 Nimzo 3.0 Pentium 90 MHz 2281 26 -25 767 58% 2223
162 Hiarcs 3.0 Pentium 90 MHz 2273 30 -30 545 56% 2228
170 Fritz 3.0 Pentium 90 MHz 2242 18 -18 1478 41% 2303
171 Fritz 4.0 Pentium 90 MHz 2234 40 -39 324 60% 2163

I believe that Rebel8,Hiarcs 5.0,MChess Pro 6.0 came after 1995

I can find for example that Hiarcs5.0 is from 1996
http://www.hiarcs.com/hiarcs_milestones.htm

[Don]

I'm pretty sure the Pentium 90 was the fastest processor at that time. Intel introduced the Pentium Pro family (to which the P200 belongs) in late 1995.

Yes, I'll assume that circa 1994 CG3 was running on the P90.

Any idea of the speed? I don't suppose anybody has a Pentium 90 laying around do they?

Don

I have an operational Pentium 90.

Good. What we need is a comparison on 2 different platforms, the P90 and the newest thing you have.

Are you willing to try the following experiment: ?

Find the oldest Crafty you can that runs on both platforms and run some kind of time test on both machines. Then do the same with the newest Crafty you can find. I don't know which old Crafty's you can still get but even if it's not very old it would still be useful. The challenge will be to find a program that still runs on both platforms.

I know that is probably a bit of work - but if you are willing to do it I would like to see the results.

What about running a Bochs IA32 emulator on modern hardware? Maybe it can be hacked to run at some arbitrary virtual clock speed.

But we cannot agree on what that speed should be. Bob thinks 1000 to 1 and I think 100 to 1 (based on on actual test with Rebel.)

[Don]

By the way, please tell me why we have to pretend parallel programs did not exist in the mid 90's?

Are we reading the same posts? Bob was accusing you of treating parallel search as something new since 1995. Now you are accusing him of the same thing?

Show me where I said that.

Quoted above: "please tell me why we have to pretend parallel programs did not exist in the mid 90's?"

He went to great pains to say exactly the opposite.

My point is that if we are comparing actual HARDWARE advances since the mid 90's and clearly parallel programs existed, then it's blatantly unfair to disallow parallel technology from the 90's in this test but eagerly embrace parallel technology from 2010.

Are we actually interested in seeing how much we advanced or not? Or is that you guys only live in the Intel PC world? This discussion isn't about PC's and how much the Pentium has advanced, it's about hardware advancement in general. I was using 1824 processors long ago and 4 processor alpha's in the 90's.

I might be getting this wrong but I think Bob was using parallel processing in the late 80's! But running on 4 processors in 2010 is a huge advancement? This is totally not the case and I don't see how this trivial point escapes you.

If Bob is really interesting in measure ACTUAL hardware improvement we should be comparing the best parallel alpha system running in 1995 to the best chip we have running in 2010, the i7-980x which is a real product.

Bob, are you interested in a fair comparison or do you just want to compare the highly inferior Intel product to the very best chip we have today?

The alpha systems cost more than the Pentiums, but they were affordable (as high end systems go.)

A lot of poeple will naturally be interested in intel, since thats the platform they have played with in their own basement. And that's what crafty typically ran on in 1995.

This might be interesting as a DIFFERENT issue, but it's not representative of hardware advances in actual Desktop computers. Nobody ever said this a Pentium contest.

What he's doing now with new and old crafty will give some estimate on the benefits made in intel advances. It seems to me it's boiling down to NPS comparisons anyway, old versus new, from HW improvements, and then the delta will be software. That's what you're really after.

That's not what I'm after and it's not about NPS as this is an extremely unreliable way to estimate what a program really does. It might be roughly ok but it can be off by a doubling or more.

Now if one could slow down an old program and a new program on new hardware (introducing delay loops to lower average NPS), then test them on a cluster, you could calc the ELO on 1995 equivalent hardware. Take your pick of 1995 performance by dialing down the NPS to the 1995 level that hardware was recorded as producing at the time with that program. Don't know how hard that would be.

The issue is that new program and old programs will not respond the same. I think it's roughly the same, but on a P90 rebel is 100 to 1 slower than on modern hardware, Bob thinks the "right" number is 1000 to 1.

This would imply that Bob could take the modern Crafty source code, go back in time to the P90 days, and get a 10 to 1 improvement just by rewriting the move generator. What he implies is so ridiculous as to be laughable.

You might get 2 or 3 times speedup if you start with a modern program that blows out the cache of the P90 or something but I did the reference test with an OLD program so that slowing it down 100 to 1 on new hardware (even a modern program would be reasonable.)

Maybe an x86 emulator could be hacked to accurately dial-down the effective Mhz. Then the old and new programs could be dialed down to 1995 speeds, but still tested on a cluster with thousands of games. Probably a tall order.

I think there is actually a utility for DOS that does something like this.