Good question: What % improvement is hardware vs. software?

[Aaron Becker]

I don't think it's really possible to disentangle the effects of improved hardware and improved software, because they have evolved together. For example, modern bitboard move generation is a giant leap forward from 70's techniques, but it couldn't have practically been used on the hardware of the 70's because it relies on large cache-resident tables and efficient manipulation of 64 bit quantities. If you want to ask the question over a smaller time frame (say, since 2000), then I think the question starts to become more answerable.

[hgm]

Well, 2GHz today also typically means 4 CPUs in stead of 1, and 3 instructions per clock cycle in stead of 1 instruction per 3. So throw in another factor 36, for 72 GHz effective. So the speed went up by a factor 72,000 over 1MHz, or about 2^16. With the rule of thumb that each doubling gives you 70 Elo, the speed alone would bring 1120 Elo.

Aaron is right in pointing out that some software improvements are in fact only improvements because of more powerfull CPU instructions and larger memory sizes. So they are partly due to hardware improvements not factored in yet in the above calculation, which only took raw speed into account.

[Michel]

the speed alone would bring 1120 Elo.

Well from what I remember these 1MHz computers were quite weak. Perhaps 1200-1500 Elo? Today's top engines are 3200-3300 Elo on CCRL. Even if you have to subtract a few hundred Elo to convert to human scale, it seems to me that a fair amount of Elo is due to software improvement as well.

How could it be otherwise? In the last year Stockfish has improved a couple of 100 Elo by playing almost 60 million testing games. Such resources were not available back in the 1MHz days.

[mwyoung]

the speed alone would bring 1120 Elo.

Well from what I remember these 1MHz computers were quite weak. Perhaps 1200-1500 Elo? Today's top engines are 3200-3300 Elo on CCRL. Even if you have to subtract a few hundred Elo to convert to human scale, it seems to me that a fair amount of Elo is due to software improvement as well.

How could it be otherwise? In the last year Stockfish has improved a couple of 100 Elo by playing almost 60 million testing games. Such resources were not available back in the 1MHz days.

Most of that gain you would never see because the programs of the 80's would not scale very well with the extra speed. The programs were full width, with very primitive evaluation functions. If somehow you could scale up a Z80 or 6502. And most of the programs could only use very small amounts of Ram memory.

[Laskos]

To the chess community:
What percent improvement is it for hardware and software? Together they make 100 percent.

Assume hardware goes from 1 Megahertz (late 1970's) to 2 Gigahertz (today).

If you wish, opening and endgame databases are optional. What percent is databases approximately as a 3rd variable; I prefer nothing because of Chess960 (except endgame database maybe).

Thanks for any input,
Jonathan Lee

Maybe about 60% hardware, and since year 2000, about 60% software.

[Vinvin]

To the chess community:
What percent improvement is it for hardware and software? Together they make 100 percent.

Assume hardware goes from 1 Megahertz (late 1970's) to 2 Gigahertz (today).

It would be very interesting to have an accurate analyze about this question.
This 3 lists are very useful to make some ratings cross-comparison :
SSDF leader year by year but different hardware : http://en.wikipedia.org/wiki/Swedish_Ch ... nd_leaders
SSDF complete list : http://ssdf.bosjo.net/long.txt
CCRT complete list : http://www.computerchess.org.uk/ccrl/40 ... t_all.html

I'm already astonished by the low improvement (SSDF) from 1985 to 2008 : +1411 in 24 years = +58.8 Elo pts / year (hardware + software)

Code: Select all

1985 	Mephisto Amsterdam 	68000 12 MHz 	1827
...
2008 	Deep Rybka 3 	2GB Q6600 2.4 GHz 	3238

[Sedat Canbaz]

Actually a few years ago i have run a similar hardawre test, for more details:
http://sedatchess.110mb.com/index.php?p=1_12

Of course, for more better conclusion we need more games, but however even this small test helps us to see the Elo diffrence,
e.g in case of doubling the processor speeds

As we see the Elo difference is approx. 300 Elo
And when we look at the speed difference between both hardwares ,AMD 3400 2.40GHz is approx 3 times faster than Celeron 1.70

So depending on engine versions,
Some engines gaining more Elo points, where some less from the processor speeds

And it's interesting to note under these conditions that 1 (one) core engines performed approx. 80-100 Elo, e.g in case of doubling the processor speed

Code: Select all

Rank  Hardware          Program               Elo   Games   Kn/s    Hardware User 
10 AMD 3400 2.40GHz     Rybka 3 w32 t1        3027   850    35      Sedat Canbaz
20 Celeron 1.70GHz      Rybka 3 w32 t1        2722   450    7       Sedat Canbaz 
*Elo Difference: 305

18 AMD 3400 2.40GHz     Toga II 1.4.2JD t1    2841   300    902     Sedat Canbaz
26 Celeron 1.70GHz      Toga II 1.4.2JD t1    2532   200    251     Sedat Canbaz
*Elo Difference: 309

23 AMD 3400 2.40GHz     Crafty 22.8 w32 t1    2657   400    1431    Sedat Canbaz
27 Celeron 1.70GHz      Crafty 22.8 w32 t1    2410   350    423     Sedat Canbaz
*Elo Difference: 247

Fritz Chess Benchmarks:
Hardware-Processor        Speed      Cores     kN/s
AMD Athlon 64 3400+       2.40 GHz     1       1367
Intel Celeron             1.70 GHz     1        506
*Processor Speed Difference: 2.70

Crafty 22.8 Benchmarks:
kN/s     Processor               Speed    EXE  Threads  Hardware User
1431694  AMD Athlon 64 3400+     2.40GHz  w32  1 CPU    Sedat Canbaz
423000   Intel Celeron           1.70GHz  w32  1 CPU    Sedat Canbaz
*Processor Speed Difference: 3.38

Strange indeed that Fritz bench gives 2.70 difference, where Crafty 22.8 bench says 3.38


One thing more,I have another Auto232 tournament:
https://sites.google.com/site/computers ... ct-auto232


Maybe these values will be useful, for those who are planing to update their hardwares ))


Processor Speed Difference between i7 970 @4.0GHz and i7 920 @3.33GHz: 1.66

Code: Select all

Hardware-Processor        Speed      Cores     kN/s
Intel Core i7 970       @ 4.00 GHz     6      17281
Intel Core i7 920       @ 3.30 GHz     4      10350


Rank  Name                        Elo    +    -  games  score oppo. draws
   3 Houdini 2.0b Pro x64 6c      3418   17   17  1049   68%  3305   46%
   7 Houdini 2.0b Pro x64 4c      3347   17   17  1030   51%  3344   47%
*Elo Difference: 71 

   6 Deep Rybka 4.1 x64 6c        3358   15   15  1363   59%  3305   54%
  15 Deep Rybka 4.1 x64 4c        3293   13   14  1602   47%  3314   56% 
*Elo Difference: 65 

  12 Fire 2.2 xTreme x64 6c       3311   23   22   530   52%  3297   63%
  13 Fire 2.2b xTreme GH x64 6c   3310   21   21   608   53%  3293   63%
  18 Fire 2.2 xTreme x64 4c       3275   16   16  1165   41%  3328   59% 
*Elo Difference: 36

Best,
Sedat

[mwyoung]

To the chess community:
What percent improvement is it for hardware and software? Together they make 100 percent.

Assume hardware goes from 1 Megahertz (late 1970's) to 2 Gigahertz (today).

It would be very interesting to have an accurate analyze about this question.
This 3 lists are very useful to make some ratings cross-comparison :
SSDF leader year by year but different hardware : http://en.wikipedia.org/wiki/Swedish_Ch ... nd_leaders
SSDF complete list : http://ssdf.bosjo.net/long.txt
CCRT complete list : http://www.computerchess.org.uk/ccrl/40 ... t_all.html

I'm already astonished by the low improvement (SSDF) from 1985 to 2008 : +1411 in 24 years = +58.8 Elo pts / year (hardware + software)

Code: Select all

1985 	Mephisto Amsterdam 	68000 12 MHz 	1827
...
2008 	Deep Rybka 3 	2GB Q6600 2.4 GHz 	3238

Here is what that looks like in a chess game.

[pgn]

[Event "Blitz, 1m/40"]
[Site "I7CPU"]
[Date "2014.01.17"]
[Round "1"]
[White "Mephisto Amsterdam"]
[Black "Rybka 3"]
[Result "0-1"]
[ECO "D41"]
[PlyCount "104"]
[TimeControl "40/60:0/0:0/0"]

1. e4 {B 0} c6 {B 0} 2. c4 {B 0} d5 {B 0} 3. exd5 {B 0} cxd5 {B 0} 4. cxd5 {B 0
} Nf6 {B 0} 5. Nc3 {B 0} Nxd5 {B 0} 6. Nf3 {B 0} e6 {B 0} 7. Bc4 {B 0} Be7 {B 0
} 8. d4 {B 0} Nc6 {B 0} 9. O-O {B 0} O-O {B 0} 10. Re1 {B 0} Nxc3 {B 0} 11.
bxc3 {B 0} b6 {B 0} 12. Bd3 {B 0 Both last book move} Bb7 {0.25/11 4} 13. Qc2 {
0.50/1 5 (Bf4)} g6 {0.26/11 3 (h6)} 14. Bh6 {0.45/1 1 (Bf4)} Re8 {0.26/9 0} 15.
Qd2 {0.28/1 2 (Rab1)} Rc8 {0.19/11 3 (Qd6)} 16. Rab1 {0.08/1 1 (Bf4)} Bd6 {0.
11/12 3 (Na5)} 17. Bf4 {0.04/1 1 (h3)} Bxf4 {-0.08/12 6 (Na5)} 18. Qxf4 {0.24/
1 1} e5 {-0.05/13 2 (Ne7)} 19. Qh6 {0.12/1 1} Qf6 {-0.05/11 0} 20. Ng5 {0.24/1
1 (Be4)} Qg7 {-0.23/10 1 (exd4)} 21. Qxg7+ {-0.08/1 1 (Qh4)} Kxg7 {-0.05/9 0}
22. Be4 {-0.36/1 1 (d5)} exd4 {-0.64/12 2 (f6)} 23. cxd4 {-0.32/1 1} Ba6 {-0.
73/12 1 (f5)} 24. Bxc6 {-0.16/1 1} Rxe1+ {-0.73/10 0} 25. Rxe1 {0.01/1 0} Rxc6
{-0.73/10 0} 26. Nf3 {-0.50/1 1 (h3)} Bb7 {-1.35/14 3 (Kf6)} 27. Re7 {-0.80/1
1 (Ne5)} Kf6 {-1.57/14 2 (Rc1+)} 28. Re3 {-0.84/1 1} Rc1+ {-1.60/14 1} 29. Re1
{-1.16/1 1} Rc2 {-1.60/15 5 (Rc4)} 30. a3 {-1.68/1 1 (Ne5)} Bxf3 {-1.90/14 1
(Ra2)} 31. gxf3 {-1.36/1 1} Rd2 {-1.90/15 1 (Ra2)} 32. Re4 {-1.55/1 2 (Rc1)}
Rd3 {-1.91/16 2} 33. Rf4+ {-1.60/1 1 (Re3)} Ke6 {-1.98/14 1} 34. d5+ {-1.80/1
1 (Re4+)} Rxd5 {-1.97/15 2} 35. Re4+ {-1.72/1 1 (Kg2)} Re5 {-1.97/15 2 (Kf6)}
36. Ra4 {-1.76/1 1 (Rc4)} a5 {-2.14/13 2} 37. Kg2 {-1.80/1 1 (Rc4)} Kd5 {-2.50/
16 2 (f5)} 38. f4 {-1.88/1 1 (Rh4)} Re4 {-5.54/13 4} 39. Rxe4 {-2.16/1 1} Kxe4
{-6.59/12 1} 40. f5 {-2.64/1 1 (Kf1)} b5 {-11.18/9 1 (Kxf5)} 41. fxg6 {-3.32/1
6 (Kf1)} hxg6 {-19.84/9 2} 42. Kf1 {-4.20/1 6} Kd3 {-31.51/11 3} 43. Ke1 {-4.
80/1 6 (h4)} Kc2 {-5.11/9 0} 44. f4 {-6.32/1 6 (h4)} b4 {-30.59/12 9} 45. axb4
{-9.40/1 6} axb4 {-30.31/12 1} 46. Ke2 {-9.99/1 6} b3 {-32.33/10 2} 47. Kf3 {
-9.99/1 6 (h4)} Kd3 {-31.46/9 1 (b2)} 48. f5 {-9.76/1 6} g5 {-30.04/12 14
(gxf5)} 49. Kg4 {-9.08/1 6 (f6)} f6 {-34.77/10 1} 50. Kh5 {-9.99/1 6 (Kg3)} Ke4
{-35.29/10 1} 51. Kg6 {-9.99/1 6 (Kg4)} Ke5 {-37.37/12 4} 52. Kh6 {-9.99/1 5
(h3)} Kxf5 {-37.15/10 2 (b2) Time} 0-1

[/pgn]

[bob]

I don't think it's really possible to disentangle the effects of improved hardware and improved software, because they have evolved together. For example, modern bitboard move generation is a giant leap forward from 70's techniques, but it couldn't have practically been used on the hardware of the 70's because it relies on large cache-resident tables and efficient manipulation of 64 bit quantities. If you want to ask the question over a smaller time frame (say, since 2000), then I think the question starts to become more answerable.

For the record, bit board programs date back to the 60's. Samuel's checker program. For chess, chess 4.0 and beyond were bit boards, as was belle in 1978. there were 60 bit machines (CDC) and 64 bit machines (Cray) in the 70's, remember...