bob wrote:I do not quite see the point for all the tangents. It would seem to me we have a pretty good idea of what/how to test.
Says you.
Rebel is only 100 times faster on modern hardware. You add a zero to this and consider it a reasonable test.
I don't think it persuasive to introduce an orange in a comparison of apples. Crafty is apples to apples.
Rebel vs Rebel - apples to apples.
Crafty 1995 vs Crafty 2010 - apples vs oranges.
It's still going to be crafty 1995 vs crafty 1995 (new hardware), just like your Rebel test. The only difference is that crafty 1995 has more speedup potential than Reble because it (1995 version) can also benefit from 64-bit. Rebel can't.
That is why Crafty is a bad example. Bob has been saying that programs in the past were not optimized for current hardware. Well, Crafty _was_ being optimized for hardware of the future, in detriment to the current hardware of the time. He was saying this all the time in the 90's and there were countless threads about it. "64 bits will become standard" etc. etc.
Miguel
And your point would be? I did the same for parallel search if you recall. But I _designed_ Crafty to work on 32 bit machines. The original rotated stuff had a ton of 32 bit optimizations, with the idea that one day they would not be needed.
Somehow you seem to imply that planning for the future, having the foresight to predict that 64 bit processors would become standard, and such, is somehow a bad idea? I give raises to people that thing progressively, not fire 'em. That's a good characteristic to have, not a bad one. And for me, it worked. I had a competitive program in 1995. It gained more from 64 bit hardware than those that chose to stick with status quo. That is a problem?
What I wrote has no resemblance to what you understood, wanted to understand, or extrapolated from my words.
bob wrote:I do not quite see the point for all the tangents. It would seem to me we have a pretty good idea of what/how to test.
Says you.
Rebel is only 100 times faster on modern hardware. You add a zero to this and consider it a reasonable test.
I don't think it persuasive to introduce an orange in a comparison of apples. Crafty is apples to apples.
Rebel vs Rebel - apples to apples.
Crafty 1995 vs Crafty 2010 - apples vs oranges.
It's still going to be crafty 1995 vs crafty 1995 (new hardware), just like your Rebel test. The only difference is that crafty 1995 has more speedup potential than Reble because it (1995 version) can also benefit from 64-bit. Rebel can't.
That is why Crafty is a bad example. Bob has been saying that programs in the past were not optimized for current hardware. Well, Crafty _was_ being optimized for hardware of the future, in detriment to the current hardware of the time. He was saying this all the time in the 90's and there were countless threads about it. "64 bits will become standard" etc. etc.
Miguel
And your point would be? I did the same for parallel search if you recall. But I _designed_ Crafty to work on 32 bit machines. The original rotated stuff had a ton of 32 bit optimizations, with the idea that one day they would not be needed.
Somehow you seem to imply that planning for the future, having the foresight to predict that 64 bit processors would become standard, and such, is somehow a bad idea? I give raises to people that thing progressively, not fire 'em. That's a good characteristic to have, not a bad one. And for me, it worked. I had a competitive program in 1995. It gained more from 64 bit hardware than those that chose to stick with status quo. That is a problem?
What I wrote has no resemblance to what you understood, wanted to understand, or extrapolated from my words.
Miguel
Only in your world. You said Crafty was optimized for hardware of the future. That's wrong. It was _highly_ optimized for 32 bit processors. When 64 bits became available. those optimizations were removed (no longer necessary, obviously).
Would you say Slate was optimizing chess 4.x for future hardware? He _knew_ at the time that CDC would never be anything other than 60 bit words. Ah, he was designing for the Cray, even when he started on the CDC, before there was even an inkling there would be a Cray? Stupid programmer because he wrote chess 4.x in CDC assembly from top to bottom, which would make it just a tad hard to port to the Cray.
Or could it be he chose to use bitboards because they offered a lot of advantages, even if it did take two instructions to AND/OR/etc a full 64 bit word on a 60 bit processor, just like it did for me on the 32 bit processors?
If he went that route, what convinces you I didn't. I was pretty sure that at some point in time in the future, mainline PCs would follow the other existing 64 bit processors (alpha, SPARC, etc) and that would further help performance. But I didn't choose bitboards because of future 64 bit architectures. I chose bitboards because I wanted to see what they were capable of being used for, and how.
bob wrote:I do not quite see the point for all the tangents. It would seem to me we have a pretty good idea of what/how to test.
Says you.
Rebel is only 100 times faster on modern hardware. You add a zero to this and consider it a reasonable test.
I don't think it persuasive to introduce an orange in a comparison of apples. Crafty is apples to apples.
Rebel vs Rebel - apples to apples.
Crafty 1995 vs Crafty 2010 - apples vs oranges.
It's still going to be crafty 1995 vs crafty 1995 (new hardware), just like your Rebel test. The only difference is that crafty 1995 has more speedup potential than Reble because it (1995 version) can also benefit from 64-bit. Rebel can't.
That is why Crafty is a bad example. Bob has been saying that programs in the past were not optimized for current hardware. Well, Crafty _was_ being optimized for hardware of the future, in detriment to the current hardware of the time. He was saying this all the time in the 90's and there were countless threads about it. "64 bits will become standard" etc. etc.
Miguel
And your point would be? I did the same for parallel search if you recall. But I _designed_ Crafty to work on 32 bit machines. The original rotated stuff had a ton of 32 bit optimizations, with the idea that one day they would not be needed.
Somehow you seem to imply that planning for the future, having the foresight to predict that 64 bit processors would become standard, and such, is somehow a bad idea? I give raises to people that thing progressively, not fire 'em. That's a good characteristic to have, not a bad one. And for me, it worked. I had a competitive program in 1995. It gained more from 64 bit hardware than those that chose to stick with status quo. That is a problem?
I think his point is legitimate. It's a good thing that you made every effort to make it run well on 32 bits, but that does not imply that it runs BEST on 32 bit.
I don't think he implied that planning for 64 bit was a bad idea or the wrong decision. At least I didn't get that.
If I ONLY cared about 32 bit, I would definitely have not built a 64 bit program but I think you made the right choice and best compromise. I would always want to favor the future technology over the past, especially if it could make it run pretty well on the present.
bob wrote:I do not quite see the point for all the tangents. It would seem to me we have a pretty good idea of what/how to test.
Says you.
Rebel is only 100 times faster on modern hardware. You add a zero to this and consider it a reasonable test.
I don't think it persuasive to introduce an orange in a comparison of apples. Crafty is apples to apples.
Rebel vs Rebel - apples to apples.
Crafty 1995 vs Crafty 2010 - apples vs oranges.
It's still going to be crafty 1995 vs crafty 1995 (new hardware), just like your Rebel test. The only difference is that crafty 1995 has more speedup potential than Reble because it (1995 version) can also benefit from 64-bit. Rebel can't.
That is why Crafty is a bad example. Bob has been saying that programs in the past were not optimized for current hardware. Well, Crafty _was_ being optimized for hardware of the future, in detriment to the current hardware of the time. He was saying this all the time in the 90's and there were countless threads about it. "64 bits will become standard" etc. etc.
Miguel
And your point would be? I did the same for parallel search if you recall. But I _designed_ Crafty to work on 32 bit machines. The original rotated stuff had a ton of 32 bit optimizations, with the idea that one day they would not be needed.
Somehow you seem to imply that planning for the future, having the foresight to predict that 64 bit processors would become standard, and such, is somehow a bad idea? I give raises to people that thing progressively, not fire 'em. That's a good characteristic to have, not a bad one. And for me, it worked. I had a competitive program in 1995. It gained more from 64 bit hardware than those that chose to stick with status quo. That is a problem?
I think his point is legitimate. It's a good thing that you made every effort to make it run well on 32 bits, but that does not imply that it runs BEST on 32 bit.
I don't think he implied that planning for 64 bit was a bad idea or the wrong decision. At least I didn't get that.
Of course, I was praising him...
Miguel
If I ONLY cared about 32 bit, I would definitely have not built a 64 bit program but I think you made the right choice and best compromise. I would always want to favor the future technology over the past, especially if it could make it run pretty well on the present.
bob wrote:I do not quite see the point for all the tangents. It would seem to me we have a pretty good idea of what/how to test.
Says you.
Rebel is only 100 times faster on modern hardware. You add a zero to this and consider it a reasonable test.
I don't think it persuasive to introduce an orange in a comparison of apples. Crafty is apples to apples.
Rebel vs Rebel - apples to apples.
Crafty 1995 vs Crafty 2010 - apples vs oranges.
It's still going to be crafty 1995 vs crafty 1995 (new hardware), just like your Rebel test. The only difference is that crafty 1995 has more speedup potential than Reble because it (1995 version) can also benefit from 64-bit. Rebel can't.
That is why Crafty is a bad example. Bob has been saying that programs in the past were not optimized for current hardware. Well, Crafty _was_ being optimized for hardware of the future, in detriment to the current hardware of the time. He was saying this all the time in the 90's and there were countless threads about it. "64 bits will become standard" etc. etc.
Miguel
And your point would be? I did the same for parallel search if you recall. But I _designed_ Crafty to work on 32 bit machines. The original rotated stuff had a ton of 32 bit optimizations, with the idea that one day they would not be needed.
Somehow you seem to imply that planning for the future, having the foresight to predict that 64 bit processors would become standard, and such, is somehow a bad idea? I give raises to people that thing progressively, not fire 'em. That's a good characteristic to have, not a bad one. And for me, it worked. I had a competitive program in 1995. It gained more from 64 bit hardware than those that chose to stick with status quo. That is a problem?
I think his point is legitimate. It's a good thing that you made every effort to make it run well on 32 bits, but that does not imply that it runs BEST on 32 bit.
My simple statement: Given the choice of bitboards or mailbox on a 32 bit machine, I would choose bitboards _every_ time. For the same reason Slate chose bitboards on 60 bit hardware and had to deal with the 2 instruction overhead there as well. That penalty was not nearly as significant as you want to imagine. The original pentium could do two instructions at a time, using in-order execution. Often only one pipeline was busy, which is what eventually led to OOE and the pentium pro. But with that second pipe, we _always_ had that "paired" instruction to implement 64 bit AND with two 32 bit AND instructions. The penalty was not nearly as bad as you assume. And given that, and the advantages that bitboards bring to a program, it was the best approach for 32 bit hardware (better than mailbox) and is obviously an even _better_ choice for 64 bit machines. But it was _not_ a "bad choice" as you keep erroneously stating. I was there. I did 'em. They worked just fine. By December of 1994 I had compared a bitboard Crafty to a mailbox version. You can guess how that worked out, since every released version has been bitboard. It simply was not a poor performance decision in 1995. It was good on 32 bits and better on 64. But you can't overlook the "good on 32". So stop the nonsense about it was not the way to write a program for 32 bit processors. It most definitely was a good way.
I don't think he implied that planning for 64 bit was a bad idea or the wrong decision. At least I didn't get that.
If I ONLY cared about 32 bit, I would definitely have not built a 64 bit program but I think you made the right choice and best compromise. I would always want to favor the future technology over the past, especially if it could make it run pretty well on the present.
If I had known that no 64 bit processors would appear, I would _still_ have used bitboards. Slate certainly knew no 64 bit CDC would every be built, yet he wrote chess 4.x in CDC assembly language. Stupid? Based on his results, I would say no. Bitboards are, IMHO, the best solution for either 32 or 64 bit processors. Yes they are better still on the 64 bit hardware. But good enough on 32 bits that I would still write a chess program using that approach if 64 bits would never happen...
Don wrote:By the way, please tell me why we have to pretend parallel programs did not exist in the mid 90's?
Are we reading the same posts? Bob was accusing you of treating parallel search as something new since 1995. Now you are accusing him of the same thing?
Show me where I said that.
Quoted above: "please tell me why we have to pretend parallel programs did not exist in the mid 90's?"
He went to great pains to say exactly the opposite.
My point is that if we are comparing actual HARDWARE advances since the mid 90's and clearly parallel programs existed, then it's blatantly unfair to disallow parallel technology from the 90's in this test but eagerly embrace parallel technology from 2010.
Are we actually interested in seeing how much we advanced or not? Or is that you guys only live in the Intel PC world? This discussion isn't about PC's and how much the Pentium has advanced, it's about hardware advancement in general. I was using 1824 processors long ago and 4 processor alpha's in the 90's.
I might be getting this wrong but I think Bob was using parallel processing in the late 80's! But running on 4 processors in 2010 is a huge advancement? This is totally not the case and I don't see how this trivial point escapes you.
If Bob is really interesting in measure ACTUAL hardware improvement we should be comparing the best parallel alpha system running in 1995 to the best chip we have running in 2010, the i7-980x which is a real product.
Bob, are you interested in a fair comparison or do you just want to compare the highly inferior Intel product to the very best chip we have today?
The alpha systems cost more than the Pentiums, but they were affordable (as high end systems go.)
A lot of poeple will naturally be interested in intel, since thats the platform they have played with in their own basement. And that's what crafty typically ran on in 1995. What he's doing now with new and old crafty will give some estimate on the benefits made in intel advances. It seems to me it's boiling down to NPS comparisons anyway, old versus new, from HW improvements, and then the delta will be software. That's what you're really after.
Now if one could slow down an old program and a new program on new hardware (introducing delay loops to lower average NPS), then test them on a cluster, you could calc the ELO on 1995 equivalent hardware. Take your pick of 1995 performance by dialing down the NPS to the 1995 level that hardware was recorded as producing at the time with that program. Don't know how hard that would be.
Maybe an x86 emulator could be hacked to accurately dial-down the effective Mhz. Then the old and new programs could be dialed down to 1995 speeds, but still tested on a cluster with thousands of games. Probably a tall order.
There are utilities which will slow down your computer - I think they have been used to play very old games. I forgot what the utilities are called but they work pretty well.
I strongly suspect that increase in chess strength is exponential both in compute power and in chess algorithm software.
What I mean is that Houdini is probably 1000x stronger than the strongest chess program in 1994 {on like hardware}, just like the modern machines are 1000 times faster.
It is difficult to see this, since Elo is an exponential scale.
Information is accelerating at a super-exponential rate, just like compute power. If the strength of chess programs isn't also expanding in an exponential manner, then we probably aren't doing our jobs right.
Dann Corbit wrote:I strongly suspect that increase in chess strength is exponential both in compute power and in chess algorithm software.
What I mean is that Houdini is probably 1000x stronger than the strongest chess program in 1994 {on like hardware}, just like the modern machines are 1000 times faster.
What if the like hardware is old hardware? Here I'm assuming that the "path length" for processing a node is much longer in modern software than in old software, such that old software might process more nodes per second. The longer path length might not have enough depth to achieve a return on it's investment in intelligence. If this assumption is wrong, I still wouldn't bet that 1000x would be accurate.
The reverse is of course demonstrated to be true, that dumber but faster software cannot make its shorter path-length pay better dividends on modern hardware due to the limit of its investment in intelligence (path length).
The latter is already proven. The former may be more difficult to demonstrate, as has been discussed here already. Someone would have to invest the time to optimize new software for old hardware, which in reality might not comfortably fit within it, depending on the platform. For instance, try cramming a modern program into a unit that runs on a 68030 with a tiny hash memory and compete against a Mephisto or Fidelity program. I'm not sure that would work out well for the modern program. The Intel P90 might work, but one still might better put their money on Genius in that contest.
Dann Corbit wrote:I strongly suspect that increase in chess strength is exponential both in compute power and in chess algorithm software.
What I mean is that Houdini is probably 1000x stronger than the strongest chess program in 1994 {on like hardware}, just like the modern machines are 1000 times faster.
What if the like hardware is old hardware? Here I'm assuming that the "path length" for processing a node is much longer in modern software than in old software, such that old software might process more nodes per second. The longer path length might not have enough depth to achieve a return on it's investment in intelligence. If this assumption is wrong, I still wouldn't bet that 1000x would be accurate.
The reverse is of course demonstrated to be true, that dumber but faster software cannot make its shorter path-length pay better dividends on modern hardware due to the limit of its investment in intelligence (path length).
The latter is already proven. The former may be more difficult to demonstrate, as has been discussed here already. Someone would have to invest the time to optimize new software for old hardware, which in reality might not comfortably fit within it, depending on the platform. For instance, try cramming a modern program into a unit that runs on a 68030 with a tiny hash memory and compete against a Mephisto or Fidelity program. I'm not sure that would work out well for the modern program. The Intel P90 might work, but one still might better put their money on Genius in that contest.
That is a good point, and I am not sure how you would measure such a thing accurately, since with chess engines, they are *definitely* tuned for particular hardware combinations.
Dann Corbit wrote:I strongly suspect that increase in chess strength is exponential both in compute power and in chess algorithm software.
What I mean is that Houdini is probably 1000x stronger than the strongest chess program in 1994 {on like hardware}, just like the modern machines are 1000 times faster.
What if the like hardware is old hardware? Here I'm assuming that the "path length" for processing a node is much longer in modern software than in old software, such that old software might process more nodes per second. The longer path length might not have enough depth to achieve a return on it's investment in intelligence. If this assumption is wrong, I still wouldn't bet that 1000x would be accurate.
The reverse is of course demonstrated to be true, that dumber but faster software cannot make its shorter path-length pay better dividends on modern hardware due to the limit of its investment in intelligence (path length).
The latter is already proven. The former may be more difficult to demonstrate, as has been discussed here already. Someone would have to invest the time to optimize new software for old hardware, which in reality might not comfortably fit within it, depending on the platform. For instance, try cramming a modern program into a unit that runs on a 68030 with a tiny hash memory and compete against a Mephisto or Fidelity program. I'm not sure that would work out well for the modern program. The Intel P90 might work, but one still might better put their money on Genius in that contest.
That is a good point, and I am not sure how you would measure such a thing accurately, since with chess engines, they are *definitely* tuned for particular hardware combinations.
It's basically unresolvable. See the big argument I had with Bob Hyatt over this (I think it might be thread which is now almost a year old.) Do you run a new program on old hardware or an old program on new hardware to compare? You get different answers depending on how you figure this and the answer is highly subject to spin and manipulation. You can pick and choose the most favorable comparison to make your point.
What seems really clear is that software and hardware have both advanced enormously.
