Perhaps when you quit rambling around and making random comments, you might discover a few surprises. (1) test is in no way biased. It is as fair as I can imagine it being done. (2) I simply reported the results, as they are. No exaggeration, no hand waving. no making excuses. It simply is what it is. (3) this is the way I _alway_ report results.Don wrote:Hi Bob,
I think these numbers are proving that software is a bigger contributor to chess improvement than hardware. I am surprised that YOUR test (which I consider biased) is proving MY point in this case.
Don
As an aside, I don't see how my results prove _your_ point. I apparently got +360 from 1995 to 2010. But when going from P5/90 to E5345 single cpu, I see a 250x performance gain. There is at least two more doublings for using 6 cores rather than just one, taking me to 1500x faster NPS. Now I have to figure out a workable way to measure the actual Elo that gives, rather than using some useless estimate that after being done 10 times has _way_ too much error to be useful. I want a _real_ +Elo number for going from P5/90 to E5345 1 cpu, and then the full 8 cpus to approximate a 6 core i7.
bob wrote:This is quite early, but is perhaps a bit surprising. Looks like I have this 10x version working on the cluster (will have to wait for a complete run and check the PGN for oddities to make sure it is not losing on time excessively or anything). Here is results compared to 23.4, and some of the lower-rated programs in my test group:
I was thinking this would be much worse. To clarify what the above is...Code: Select all
Crafty-23.4 2703 4 4 30000 66% 2579 22% Crafty-23.3 2693 4 4 30000 65% 2579 22% Crafty-23.1 2622 4 4 30000 55% 2579 23% Glaurung 2.2 2606 3 3 60277 46% 2636 22% Toga2 2599 3 3 60275 45% 2636 23% Fruit 2.1 2501 3 3 60248 32% 2636 21% Glaurung 1.1 SMP 2444 3 3 60267 26% 2636 17% Crafty-10.18 2326 19 19 1327 20% 2580 14%
Everything is running on our cluster. This is the cluster I have used to post _all_ results here in recent years, it is hardware about 4 years old as previously mentioned. Crafty-10.18 is about 10% slower than it should be as I have yet to tackle the PGO stuff. Took a lot of work to make the old version work with more modern xboard protocol. Had a lot of fun with force and such.
I'll report the final results for this run, although this will not be the overall "final results." Got to make sure nothing odd is happening in the PGN, and then get the PGO working.
More later, but at least it seems to be playing... All this really measures is "how far behind is 1995 Crafty, giving everyone equal (and modern) hardware. I'd suspect it would not be as far behind if everyone was on a P5/90, will work on that angle later.