Good question: What % improvement is hardware vs. software?

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
jplchess
Posts: 115
Joined: Mon Jan 25, 2010 6:13 am

Good question: What % improvement is hardware vs. software?

Post by jplchess » Fri Jan 17, 2014 2:51 am

To the chess community:
What percent improvement is it for hardware and software? Together they make 100 percent.

Assume hardware goes from 1 Megahertz (late 1970's) to 2 Gigahertz (today).

If you wish, opening and endgame databases are optional. What percent is databases approximately as a 3rd variable; I prefer nothing because of Chess960 (except endgame database maybe).

Thanks for any input,
Jonathan Lee

Aaron Becker
Posts: 292
Joined: Tue Jul 07, 2009 2:56 am

Re: Good question: What % improvement is hardware vs. softwa

Post by Aaron Becker » Fri Jan 17, 2014 3:34 am

I don't think it's really possible to disentangle the effects of improved hardware and improved software, because they have evolved together. For example, modern bitboard move generation is a giant leap forward from 70's techniques, but it couldn't have practically been used on the hardware of the 70's because it relies on large cache-resident tables and efficient manipulation of 64 bit quantities. If you want to ask the question over a smaller time frame (say, since 2000), then I think the question starts to become more answerable.

User avatar
hgm
Posts: 23217
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Good question: What % improvement is hardware vs. softwa

Post by hgm » Fri Jan 17, 2014 7:22 am

Well, 2GHz today also typically means 4 CPUs in stead of 1, and 3 instructions per clock cycle in stead of 1 instruction per 3. So throw in another factor 36, for 72 GHz effective. So the speed went up by a factor 72,000 over 1MHz, or about 2^16. With the rule of thumb that each doubling gives you 70 Elo, the speed alone would bring 1120 Elo.

Aaron is right in pointing out that some software improvements are in fact only improvements because of more powerfull CPU instructions and larger memory sizes. So they are partly due to hardware improvements not factored in yet in the above calculation, which only took raw speed into account.

Michel
Posts: 2005
Joined: Sun Sep 28, 2008 11:50 pm

Re: Good question: What % improvement is hardware vs. softwa

Post by Michel » Fri Jan 17, 2014 7:36 am

the speed alone would bring 1120 Elo.
Well from what I remember these 1MHz computers were quite weak. Perhaps 1200-1500 Elo? Today's top engines are 3200-3300 Elo on CCRL. Even if you have to subtract a few hundred Elo to convert to human scale, it seems to me that a fair amount of Elo is due to software improvement as well.

How could it be otherwise? In the last year Stockfish has improved a couple of 100 Elo by playing almost 60 million testing games. Such resources were not available back in the 1MHz days.

mwyoung
Posts: 1310
Joined: Wed May 12, 2010 8:00 pm

Re: Good question: What % improvement is hardware vs. softwa

Post by mwyoung » Fri Jan 17, 2014 8:13 am

Michel wrote:
the speed alone would bring 1120 Elo.
Well from what I remember these 1MHz computers were quite weak. Perhaps 1200-1500 Elo? Today's top engines are 3200-3300 Elo on CCRL. Even if you have to subtract a few hundred Elo to convert to human scale, it seems to me that a fair amount of Elo is due to software improvement as well.

How could it be otherwise? In the last year Stockfish has improved a couple of 100 Elo by playing almost 60 million testing games. Such resources were not available back in the 1MHz days.
Most of that gain you would never see because the programs of the 80's would not scale very well with the extra speed. The programs were full width, with very primitive evaluation functions. If somehow you could scale up a Z80 or 6502. And most of the programs could only use very small amounts of Ram memory.
Professing themselves to be wise, they became fools,

User avatar
Laskos
Posts: 9047
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Good question: What % improvement is hardware vs. softwa

Post by Laskos » Fri Jan 17, 2014 8:40 am

jplchess wrote:To the chess community:
What percent improvement is it for hardware and software? Together they make 100 percent.

Assume hardware goes from 1 Megahertz (late 1970's) to 2 Gigahertz (today).

If you wish, opening and endgame databases are optional. What percent is databases approximately as a 3rd variable; I prefer nothing because of Chess960 (except endgame database maybe).

Thanks for any input,
Jonathan Lee
Maybe about 60% hardware, and since year 2000, about 60% software.

Vinvin
Posts: 4212
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: Good question: What % improvement is hardware vs. softwa

Post by Vinvin » Fri Jan 17, 2014 10:36 am

jplchess wrote:To the chess community:
What percent improvement is it for hardware and software? Together they make 100 percent.

Assume hardware goes from 1 Megahertz (late 1970's) to 2 Gigahertz (today).
It would be very interesting to have an accurate analyze about this question.
This 3 lists are very useful to make some ratings cross-comparison :
SSDF leader year by year but different hardware : http://en.wikipedia.org/wiki/Swedish_Ch ... nd_leaders
SSDF complete list : http://ssdf.bosjo.net/long.txt
CCRT complete list : http://www.computerchess.org.uk/ccrl/40 ... t_all.html

I'm already astonished by the low improvement (SSDF) from 1985 to 2008 : +1411 in 24 years = +58.8 Elo pts / year (hardware + software)

Code: Select all

1985 	Mephisto Amsterdam 	68000 12 MHz 	1827
...
2008 	Deep Rybka 3 	2GB Q6600 2.4 GHz 	3238

Sedat Canbaz
Posts: 3018
Joined: Thu Mar 09, 2006 10:58 am
Location: Antalya/Turkey
Contact:

Re: Good question: What % improvement is hardware vs. softwa

Post by Sedat Canbaz » Fri Jan 17, 2014 12:02 pm

Actually a few years ago i have run a similar hardawre test, for more details:
http://sedatchess.110mb.com/index.php?p=1_12

Of course, for more better conclusion we need more games, but however even this small test helps us to see the Elo diffrence,
e.g in case of doubling the processor speeds

As we see the Elo difference is approx. 300 Elo
And when we look at the speed difference between both hardwares ,AMD 3400 2.40GHz is approx 3 times faster than Celeron 1.70

So depending on engine versions,
Some engines gaining more Elo points, where some less from the processor speeds

And it's interesting to note under these conditions that 1 (one) core engines performed approx. 80-100 Elo, e.g in case of doubling the processor speed

Code: Select all

Rank  Hardware          Program               Elo   Games   Kn/s    Hardware User 
10 AMD 3400 2.40GHz     Rybka 3 w32 t1        3027   850    35      Sedat Canbaz
20 Celeron 1.70GHz      Rybka 3 w32 t1        2722   450    7       Sedat Canbaz 
*Elo Difference: 305

18 AMD 3400 2.40GHz     Toga II 1.4.2JD t1    2841   300    902     Sedat Canbaz
26 Celeron 1.70GHz      Toga II 1.4.2JD t1    2532   200    251     Sedat Canbaz
*Elo Difference: 309

23 AMD 3400 2.40GHz     Crafty 22.8 w32 t1    2657   400    1431    Sedat Canbaz
27 Celeron 1.70GHz      Crafty 22.8 w32 t1    2410   350    423     Sedat Canbaz
*Elo Difference: 247

Fritz Chess Benchmarks:
Hardware-Processor        Speed      Cores     kN/s
AMD Athlon 64 3400+       2.40 GHz     1       1367
Intel Celeron             1.70 GHz     1        506
*Processor Speed Difference: 2.70

Crafty 22.8 Benchmarks:
kN/s     Processor               Speed    EXE  Threads  Hardware User
1431694  AMD Athlon 64 3400+     2.40GHz  w32  1 CPU    Sedat Canbaz
423000   Intel Celeron           1.70GHz  w32  1 CPU    Sedat Canbaz
*Processor Speed Difference: 3.38
Strange indeed that Fritz bench gives 2.70 difference, where Crafty 22.8 bench says 3.38


One thing more,I have another Auto232 tournament:
https://sites.google.com/site/computers ... ct-auto232


Maybe these values will be useful, for those who are planing to update their hardwares ))


Processor Speed Difference between i7 970 @4.0GHz and i7 920 @3.33GHz: 1.66

Code: Select all

Hardware-Processor        Speed      Cores     kN/s
Intel Core i7 970       @ 4.00 GHz     6      17281
Intel Core i7 920       @ 3.30 GHz     4      10350


Rank  Name                        Elo    +    -  games  score oppo. draws
   3 Houdini 2.0b Pro x64 6c      3418   17   17  1049   68%  3305   46%
   7 Houdini 2.0b Pro x64 4c      3347   17   17  1030   51%  3344   47%
*Elo Difference: 71 

   6 Deep Rybka 4.1 x64 6c        3358   15   15  1363   59%  3305   54%
  15 Deep Rybka 4.1 x64 4c        3293   13   14  1602   47%  3314   56% 
*Elo Difference: 65 

  12 Fire 2.2 xTreme x64 6c       3311   23   22   530   52%  3297   63%
  13 Fire 2.2b xTreme GH x64 6c   3310   21   21   608   53%  3293   63%
  18 Fire 2.2 xTreme x64 4c       3275   16   16  1165   41%  3328   59% 
*Elo Difference: 36
Best,
Sedat

mwyoung
Posts: 1310
Joined: Wed May 12, 2010 8:00 pm

Re: Good question: What % improvement is hardware vs. softwa

Post by mwyoung » Sat Jan 18, 2014 12:37 am

Vinvin wrote:
jplchess wrote:To the chess community:
What percent improvement is it for hardware and software? Together they make 100 percent.

Assume hardware goes from 1 Megahertz (late 1970's) to 2 Gigahertz (today).
It would be very interesting to have an accurate analyze about this question.
This 3 lists are very useful to make some ratings cross-comparison :
SSDF leader year by year but different hardware : http://en.wikipedia.org/wiki/Swedish_Ch ... nd_leaders
SSDF complete list : http://ssdf.bosjo.net/long.txt
CCRT complete list : http://www.computerchess.org.uk/ccrl/40 ... t_all.html

I'm already astonished by the low improvement (SSDF) from 1985 to 2008 : +1411 in 24 years = +58.8 Elo pts / year (hardware + software)

Code: Select all

1985 	Mephisto Amsterdam 	68000 12 MHz 	1827
...
2008 	Deep Rybka 3 	2GB Q6600 2.4 GHz 	3238
Here is what that looks like in a chess game.

[pgn]

[Event "Blitz, 1m/40"]
[Site "I7CPU"]
[Date "2014.01.17"]
[Round "1"]
[White "Mephisto Amsterdam"]
[Black "Rybka 3"]
[Result "0-1"]
[ECO "D41"]
[PlyCount "104"]
[TimeControl "40/60:0/0:0/0"]

1. e4 {B 0} c6 {B 0} 2. c4 {B 0} d5 {B 0} 3. exd5 {B 0} cxd5 {B 0} 4. cxd5 {B 0
} Nf6 {B 0} 5. Nc3 {B 0} Nxd5 {B 0} 6. Nf3 {B 0} e6 {B 0} 7. Bc4 {B 0} Be7 {B 0
} 8. d4 {B 0} Nc6 {B 0} 9. O-O {B 0} O-O {B 0} 10. Re1 {B 0} Nxc3 {B 0} 11.
bxc3 {B 0} b6 {B 0} 12. Bd3 {B 0 Both last book move} Bb7 {0.25/11 4} 13. Qc2 {
0.50/1 5 (Bf4)} g6 {0.26/11 3 (h6)} 14. Bh6 {0.45/1 1 (Bf4)} Re8 {0.26/9 0} 15.
Qd2 {0.28/1 2 (Rab1)} Rc8 {0.19/11 3 (Qd6)} 16. Rab1 {0.08/1 1 (Bf4)} Bd6 {0.
11/12 3 (Na5)} 17. Bf4 {0.04/1 1 (h3)} Bxf4 {-0.08/12 6 (Na5)} 18. Qxf4 {0.24/
1 1} e5 {-0.05/13 2 (Ne7)} 19. Qh6 {0.12/1 1} Qf6 {-0.05/11 0} 20. Ng5 {0.24/1
1 (Be4)} Qg7 {-0.23/10 1 (exd4)} 21. Qxg7+ {-0.08/1 1 (Qh4)} Kxg7 {-0.05/9 0}
22. Be4 {-0.36/1 1 (d5)} exd4 {-0.64/12 2 (f6)} 23. cxd4 {-0.32/1 1} Ba6 {-0.
73/12 1 (f5)} 24. Bxc6 {-0.16/1 1} Rxe1+ {-0.73/10 0} 25. Rxe1 {0.01/1 0} Rxc6
{-0.73/10 0} 26. Nf3 {-0.50/1 1 (h3)} Bb7 {-1.35/14 3 (Kf6)} 27. Re7 {-0.80/1
1 (Ne5)} Kf6 {-1.57/14 2 (Rc1+)} 28. Re3 {-0.84/1 1} Rc1+ {-1.60/14 1} 29. Re1
{-1.16/1 1} Rc2 {-1.60/15 5 (Rc4)} 30. a3 {-1.68/1 1 (Ne5)} Bxf3 {-1.90/14 1
(Ra2)} 31. gxf3 {-1.36/1 1} Rd2 {-1.90/15 1 (Ra2)} 32. Re4 {-1.55/1 2 (Rc1)}
Rd3 {-1.91/16 2} 33. Rf4+ {-1.60/1 1 (Re3)} Ke6 {-1.98/14 1} 34. d5+ {-1.80/1
1 (Re4+)} Rxd5 {-1.97/15 2} 35. Re4+ {-1.72/1 1 (Kg2)} Re5 {-1.97/15 2 (Kf6)}
36. Ra4 {-1.76/1 1 (Rc4)} a5 {-2.14/13 2} 37. Kg2 {-1.80/1 1 (Rc4)} Kd5 {-2.50/
16 2 (f5)} 38. f4 {-1.88/1 1 (Rh4)} Re4 {-5.54/13 4} 39. Rxe4 {-2.16/1 1} Kxe4
{-6.59/12 1} 40. f5 {-2.64/1 1 (Kf1)} b5 {-11.18/9 1 (Kxf5)} 41. fxg6 {-3.32/1
6 (Kf1)} hxg6 {-19.84/9 2} 42. Kf1 {-4.20/1 6} Kd3 {-31.51/11 3} 43. Ke1 {-4.
80/1 6 (h4)} Kc2 {-5.11/9 0} 44. f4 {-6.32/1 6 (h4)} b4 {-30.59/12 9} 45. axb4
{-9.40/1 6} axb4 {-30.31/12 1} 46. Ke2 {-9.99/1 6} b3 {-32.33/10 2} 47. Kf3 {
-9.99/1 6 (h4)} Kd3 {-31.46/9 1 (b2)} 48. f5 {-9.76/1 6} g5 {-30.04/12 14
(gxf5)} 49. Kg4 {-9.08/1 6 (f6)} f6 {-34.77/10 1} 50. Kh5 {-9.99/1 6 (Kg3)} Ke4
{-35.29/10 1} 51. Kg6 {-9.99/1 6 (Kg4)} Ke5 {-37.37/12 4} 52. Kh6 {-9.99/1 5
(h3)} Kxf5 {-37.15/10 2 (b2) Time} 0-1

[/pgn]
Professing themselves to be wise, they became fools,

bob
Posts: 20369
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Good question: What % improvement is hardware vs. softwa

Post by bob » Sat Jan 18, 2014 6:09 am

Aaron Becker wrote:I don't think it's really possible to disentangle the effects of improved hardware and improved software, because they have evolved together. For example, modern bitboard move generation is a giant leap forward from 70's techniques, but it couldn't have practically been used on the hardware of the 70's because it relies on large cache-resident tables and efficient manipulation of 64 bit quantities. If you want to ask the question over a smaller time frame (say, since 2000), then I think the question starts to become more answerable.
For the record, bit board programs date back to the 60's. Samuel's checker program. For chess, chess 4.0 and beyond were bit boards, as was belle in 1978. there were 60 bit machines (CDC) and 64 bit machines (Cray) in the 70's, remember...

Post Reply