old crafty vs new crafty on new hardware.

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
bob
Posts: 20641
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: old crafty vs new crafty on new hardware. Some results.

Post by bob » Sun Sep 12, 2010 3:17 am

Don wrote:Hi Bob,

I think these numbers are proving that software is a bigger contributor to chess improvement than hardware. I am surprised that YOUR test (which I consider biased) is proving MY point in this case.

Don
Perhaps when you quit rambling around and making random comments, you might discover a few surprises. (1) test is in no way biased. It is as fair as I can imagine it being done. (2) I simply reported the results, as they are. No exaggeration, no hand waving. no making excuses. It simply is what it is. (3) this is the way I _alway_ report results.

As an aside, I don't see how my results prove _your_ point. I apparently got +360 from 1995 to 2010. But when going from P5/90 to E5345 single cpu, I see a 250x performance gain. There is at least two more doublings for using 6 cores rather than just one, taking me to 1500x faster NPS. Now I have to figure out a workable way to measure the actual Elo that gives, rather than using some useless estimate that after being done 10 times has _way_ too much error to be useful. I want a _real_ +Elo number for going from P5/90 to E5345 1 cpu, and then the full 8 cpus to approximate a 6 core i7.



bob wrote:This is quite early, but is perhaps a bit surprising. Looks like I have this 10x version working on the cluster (will have to wait for a complete run and check the PGN for oddities to make sure it is not losing on time excessively or anything). Here is results compared to 23.4, and some of the lower-rated programs in my test group:

Code: Select all

   Crafty-23.4        2703    4    4 30000   66%  2579   22% 
   Crafty-23.3        2693    4    4 30000   65%  2579   22% 
   Crafty-23.1        2622    4    4 30000   55%  2579   23% 
   Glaurung 2.2       2606    3    3 60277   46%  2636   22% 
   Toga2              2599    3    3 60275   45%  2636   23% 
   Fruit 2.1          2501    3    3 60248   32%  2636   21% 
   Glaurung 1.1 SMP   2444    3    3 60267   26%  2636   17% 
   Crafty-10.18       2326   19   19  1327   20%  2580   14% 
I was thinking this would be much worse. To clarify what the above is...

Everything is running on our cluster. This is the cluster I have used to post _all_ results here in recent years, it is hardware about 4 years old as previously mentioned. Crafty-10.18 is about 10% slower than it should be as I have yet to tackle the PGO stuff. Took a lot of work to make the old version work with more modern xboard protocol. Had a lot of fun with force and such.

I'll report the final results for this run, although this will not be the overall "final results." Got to make sure nothing odd is happening in the PGN, and then get the PGO working.

More later, but at least it seems to be playing... All this really measures is "how far behind is 1995 Crafty, giving everyone equal (and modern) hardware. I'd suspect it would not be as far behind if everyone was on a P5/90, will work on that angle later.

Post Reply