Progress in 30 years by four intervals of 7-8 years

[Laskos]

I took 5 top engines of their time separated by 4 fairly equal intervals of 7-8 years since 1987, thanks to Ed's porting to UCI of his beautiful old works.
Hardware gains are for one core.

• Mephisto Roma (1987)
  CPU: 68020 14 MHz
  FIDE ELO: 2070
  
  Rebel 6.0 (1994)
  CPU: i486 66 MHz
  Hardware gain: 150 ELO points
  FIDE ELO: 2350
  
  Shredder 6PB (2002)
  CPU: Athlon 1500 MHz
  Hardware gain: 300 ELO points
  FIDE ELO: 2750
  
  Robbolito 0.085 (2009)
  CPU: Core 2 3000 MHz
  Hardware gain: 100 Elo points
  FIDE ELO: 3050
  
  Stockfish (2017)
  CPU: i7 4000 MHz
  Hardware gain: 70 Elo points
  FIDE ELO: 3300

Software gain (equal hardware, one i7 core):

Code: Select all

Games Completed = 100 of 100 (Avg game length = 144.497 sec)
Settings = RR/64MB/60000ms in 40 moves/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 3904 sec elapsed, 0 sec remaining
 1.  Rebel 6.0          (1994)	   64.5/100	   60-31-9  	(L: m=30 t=1 i=0 a=0)	(D: r=4 i=5 f=0 s=0 a=0)	(tpm=1268.5 d=7.10 nps=335726008)
 2.  Mephisto Roma      (1987)      35.5/100	   31-60-9  	(L: m=54 t=6 i=0 a=0)	(D: r=4 i=5 f=0 s=0 a=0)	(tpm=1385.3 d=1.00 nps=0)

100 ELO points

Code: Select all

Games Completed = 100 of 100 (Avg game length = 467.077 sec)
Settings = RR/64MB/120000ms+2000ms/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 11964 sec elapsed, 0 sec remaining
 1.  Shredder Paderborn (2002)      80.5/100	  74-13-13  	(L: m=11 t=2 i=0 a=0)	(D: r=8 i=3 f=0 s=2 a=0)	(tpm=3294.0 d=12.37 nps=1409398)
 2.  Rebel 6.0          (1994)      19.5/100	  13-74-13  	(L: m=70 t=4 i=0 a=0)	(D: r=8 i=3 f=0 s=2 a=0)	(tpm=3373.5 d=9.80  nps=125460712)

250 ELO points

Code: Select all

Games Completed = 100 of 100 (Avg game length = 253.156 sec)
Settings = RR/64MB/2000ms per move/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 6428 sec elapsed, 0 sec remaining
 1.  RobboLito 0.085    (2009)      92.0/100	   87-3-10  	(L: m=3 t=0 i=0 a=0)	(D: r=7 i=1 f=2 s=0 a=0)	(tpm=2001.4 d=18.02 nps=2700313)
 2.  Shredder Paderborn (2002)       8.0/100	   3-87-10  	(L: m=86 t=0 i=1 a=0)	(D: r=7 i=1 f=2 s=0 a=0)	(tpm=2031.7 d=12.46 nps=1425171)

420 ELO points

Code: Select all

Games Completed = 100 of 100 (Avg game length = 233.045 sec)
Settings = RR/64MB/60000ms+1000ms/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 5974 sec elapsed, 0 sec remaining
 1.  Stockfish 110117  (2017)       94.0/100	  89-1-10  	(L: m=1 t=0 i=0 a=0)	(D: r=6 i=2 f=1 s=1 a=0)	(tpm=1644.9 d=28.96 nps=1965019)
 2.  RobboLito 0.085   (2009)        6.0/100	  1-89-10  	(L: m=89 t=0 i=0 a=0)	(D: r=6 i=2 f=1 s=1 a=0)	(tpm=1852.4 d=16.50 nps=2572328)

480 ELO points


One can observe that software progression in 1987-2002 was fairly slow, slower than hardware progression. However, 2002-2017 saw extremely fast software progression and slow hardware progression. This is mostly due to such developments as Fruit/Rybka and Stockfish/Komodo.

In 30 years:

Total gain hardware: 150+300+100+70 = 620 ELO points
Total gain software: 100+250+420+480 = 1250 ELO points
Total gain hardware + software: 620+1250 = 1870 ELO points
FIDE ELO gain: 3300-2070 = 1230 ELO points
Compression of FIDE rating compared to computer rating: 1250/1870 = 0.67

[PK]

some of software progression is an effect of hardware progression - expressed by the possibility of running thousands of test games.

[Laskos]

some of software progression is an effect of hardware progression - expressed by the possibility of running thousands of test games.

I don't think it's a hardware progression. UI for testing at ultra-fast controls could have been written since first 8086 processors in 1980s. That it's only by 2005 that people started testing at ultra-fast time controls in many statistically significant games is programmers' issue.

PS A correction to the opening post:
Compression of FIDE rating compared to computer rating: 1230/1870 = 0.66

[mhull]

I took 5 top engines of their time separated by 4 fairly equal intervals of 7-8 years since 1987, thanks to Ed's porting to UCI of his beautiful old works.
Hardware gains are for one core.

• Mephisto Roma (1987)
  CPU: 68020 14 MHz
  FIDE ELO: 2070
  
  Rebel 6.0 (1994)
  CPU: i486 66 MHz
  Hardware gain: 150 ELO points
  FIDE ELO: 2350
  
  Shredder 6PB (2002)
  CPU: Athlon 1500 MHz
  Hardware gain: 300 ELO points
  FIDE ELO: 2750
  
  Robbolito 0.085 (2009)
  CPU: Core 2 3000 MHz
  Hardware gain: 100 Elo points
  FIDE ELO: 3050
  
  Stockfish (2017)
  CPU: i7 4000 MHz
  Hardware gain: 70 Elo points
  FIDE ELO: 3300

Software gain (equal hardware, one i7 core):

Code: Select all

Games Completed = 100 of 100 (Avg game length = 144.497 sec)
Settings = RR/64MB/60000ms in 40 moves/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 3904 sec elapsed, 0 sec remaining
 1.  Rebel 6.0          (1994)	   64.5/100	   60-31-9  	(L: m=30 t=1 i=0 a=0)	(D: r=4 i=5 f=0 s=0 a=0)	(tpm=1268.5 d=7.10 nps=335726008)
 2.  Mephisto Roma      (1987)      35.5/100	   31-60-9  	(L: m=54 t=6 i=0 a=0)	(D: r=4 i=5 f=0 s=0 a=0)	(tpm=1385.3 d=1.00 nps=0)

100 ELO points

Code: Select all

Games Completed = 100 of 100 (Avg game length = 467.077 sec)
Settings = RR/64MB/120000ms+2000ms/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 11964 sec elapsed, 0 sec remaining
 1.  Shredder Paderborn (2002)      80.5/100	  74-13-13  	(L: m=11 t=2 i=0 a=0)	(D: r=8 i=3 f=0 s=2 a=0)	(tpm=3294.0 d=12.37 nps=1409398)
 2.  Rebel 6.0          (1994)      19.5/100	  13-74-13  	(L: m=70 t=4 i=0 a=0)	(D: r=8 i=3 f=0 s=2 a=0)	(tpm=3373.5 d=9.80  nps=125460712)

250 ELO points

Code: Select all

Games Completed = 100 of 100 (Avg game length = 253.156 sec)
Settings = RR/64MB/2000ms per move/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 6428 sec elapsed, 0 sec remaining
 1.  RobboLito 0.085    (2009)      92.0/100	   87-3-10  	(L: m=3 t=0 i=0 a=0)	(D: r=7 i=1 f=2 s=0 a=0)	(tpm=2001.4 d=18.02 nps=2700313)
 2.  Shredder Paderborn (2002)       8.0/100	   3-87-10  	(L: m=86 t=0 i=1 a=0)	(D: r=7 i=1 f=2 s=0 a=0)	(tpm=2031.7 d=12.46 nps=1425171)

420 ELO points

Code: Select all

Games Completed = 100 of 100 (Avg game length = 233.045 sec)
Settings = RR/64MB/60000ms+1000ms/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 5974 sec elapsed, 0 sec remaining
 1.  Stockfish 110117  (2017)       94.0/100	  89-1-10  	(L: m=1 t=0 i=0 a=0)	(D: r=6 i=2 f=1 s=1 a=0)	(tpm=1644.9 d=28.96 nps=1965019)
 2.  RobboLito 0.085   (2009)        6.0/100	  1-89-10  	(L: m=89 t=0 i=0 a=0)	(D: r=6 i=2 f=1 s=1 a=0)	(tpm=1852.4 d=16.50 nps=2572328)

480 ELO points


One can observe that software progression in 1987-2002 was fairly slow, slower than hardware progression. However, 2002-2017 saw extremely fast software progression and slow hardware progression. This is mostly due to such developments as Fruit/Rybka and Stockfish/Komodo.

In 30 years:

Total gain hardware: 150+300+100+70 = 620 ELO points
Total gain software: 100+250+420+480 = 1250 ELO points
Total gain hardware + software: 620+1250 = 1870 ELO points
FIDE ELO gain: 3300-2070 = 1230 ELO points
Compression of FIDE rating compared to computer rating: 1250/1870 = 0.67

One question is how much software progress depends upon hardware, e.g. wider native registers (64-bit), larger memory for hashing and cache speed hacks.

IOW, could a modern program be adapted to squeeze into an MC68030 and demonstrate its superiority over a period MC68K program? Or could software advances even be effective in 16-bit and 8-bit architectures?

Could a programmer from the future have dominated with micros in the 1980s? Is there a way to demonstrate this?

[Dann Corbit]

One question is how much software progress depends upon hardware, e.g. wider native registers (64-bit), larger memory for hashing and cache speed hacks.

You can have wide integers even on an 8 bit machine, but they won't be nearly so fast.

IOW, could a modern program be adapted to squeeze into an MC68030 and demonstrate its superiority over a period MC68K program? Or could software advances even be effective in 16-bit and 8-bit architectures?

The Motorola 68x family all had 32 bit addressing. Way advanced above the intel 8088 and 8086 junk. An array based program would not suffer much but the clock speeds also were a lot slower then. You could, however address 4G RAM with the Motorola 68x processors.

Could a programmer from the future have dominated with micros in the 1980s? Is there a way to demonstrate this?

Yes, the search techniques get the branching factor down to 2 or better today. In the 80's it was around 6. A modern program transported to the motorola would blow the doors off of the software of the time.

Simple math.

[mhull]

Could a programmer from the future have dominated with micros in the 1980s? Is there a way to demonstrate this?

Yes, the search techniques get the branching factor down to 2 or better today. In the 80's it was around 6. A modern program transported to the motorola would blow the doors off of the software of the time.

Simple math.

Do you think the math shows that the software techniques required to achieve lower BF (and higher depth) would compensate for the potentially higher NPS of period software at both blitz and standard time controls?

[Vinvin]

...
Robbolito 0.085 (2009)
CPU: Core 2 3000 MHz
Hardware gain: 100 Elo points
FIDE ELO: 3050

Stockfish (2017)
CPU: i7 4000 MHz
Hardware gain: 70 Elo points
FIDE ELO: 3300
...

You could set 2 cores for the "Core 2" and 4 cores for the "i7" as they appear to be widely available in this config for the time being.

[Dann Corbit]

Could a programmer from the future have dominated with micros in the 1980s? Is there a way to demonstrate this?

Yes, the search techniques get the branching factor down to 2 or better today. In the 80's it was around 6. A modern program transported to the motorola would blow the doors off of the software of the time.

Simple math.

Do you think the math shows that the software techniques required to achieve lower BF (and higher depth) would compensate for the potentially higher NPS of period software at both blitz and standard time controls?

I don't understand your question.
The NPS will be about the same on period hardware for the old software and the new using mailbox. It may be that the magics bitboards would work better on 32 bit than the old techniques. Not sure about that.

Modern programs compiled for the 68K CPUs would blow the doors off the older programs on the same hardware.

It sounds to me like you may be asking for something else, though.

[S.Taylor]

I thought that Genius (3?) was a giant leap, as was Rybka. Also Hiarcs 7 was a big leap above the rest in its time.

[Dann Corbit]

I thought that Genius (3?) was a giant leap, as was Rybka. Also Hiarcs 7 was a big leap above the rest in its time.

Don't forget fruit
Good for the digestion