Progress in 30 years by four intervals of 7-8 years

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Laskos
Posts: 8963
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Progress in 30 years by four intervals of 7-8 years

Post by Laskos » Thu Jan 19, 2017 9:03 am

I took 5 top engines of their time separated by 4 fairly equal intervals of 7-8 years since 1987, thanks to Ed's porting to UCI of his beautiful old works.
Hardware gains are for one core.
  • Mephisto Roma (1987)
    CPU: 68020 14 MHz
    FIDE ELO: 2070

    Rebel 6.0 (1994)
    CPU: i486 66 MHz
    Hardware gain: 150 ELO points
    FIDE ELO: 2350

    Shredder 6PB (2002)
    CPU: Athlon 1500 MHz
    Hardware gain: 300 ELO points
    FIDE ELO: 2750

    Robbolito 0.085 (2009)
    CPU: Core 2 3000 MHz
    Hardware gain: 100 Elo points
    FIDE ELO: 3050

    Stockfish (2017)
    CPU: i7 4000 MHz
    Hardware gain: 70 Elo points
    FIDE ELO: 3300
Software gain (equal hardware, one i7 core):

Code: Select all

Games Completed = 100 of 100 (Avg game length = 144.497 sec)
Settings = RR/64MB/60000ms in 40 moves/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 3904 sec elapsed, 0 sec remaining
 1.  Rebel 6.0          (1994)	   64.5/100	   60-31-9  	(L: m=30 t=1 i=0 a=0)	(D: r=4 i=5 f=0 s=0 a=0)	(tpm=1268.5 d=7.10 nps=335726008)
 2.  Mephisto Roma      (1987)      35.5/100	   31-60-9  	(L: m=54 t=6 i=0 a=0)	(D: r=4 i=5 f=0 s=0 a=0)	(tpm=1385.3 d=1.00 nps=0)
100 ELO points

Code: Select all

Games Completed = 100 of 100 (Avg game length = 467.077 sec)
Settings = RR/64MB/120000ms+2000ms/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 11964 sec elapsed, 0 sec remaining
 1.  Shredder Paderborn (2002)      80.5/100	  74-13-13  	(L: m=11 t=2 i=0 a=0)	(D: r=8 i=3 f=0 s=2 a=0)	(tpm=3294.0 d=12.37 nps=1409398)
 2.  Rebel 6.0          (1994)      19.5/100	  13-74-13  	(L: m=70 t=4 i=0 a=0)	(D: r=8 i=3 f=0 s=2 a=0)	(tpm=3373.5 d=9.80  nps=125460712)
250 ELO points

Code: Select all

Games Completed = 100 of 100 (Avg game length = 253.156 sec)
Settings = RR/64MB/2000ms per move/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 6428 sec elapsed, 0 sec remaining
 1.  RobboLito 0.085    (2009)      92.0/100	   87-3-10  	(L: m=3 t=0 i=0 a=0)	(D: r=7 i=1 f=2 s=0 a=0)	(tpm=2001.4 d=18.02 nps=2700313)
 2.  Shredder Paderborn (2002)       8.0/100	   3-87-10  	(L: m=86 t=0 i=1 a=0)	(D: r=7 i=1 f=2 s=0 a=0)	(tpm=2031.7 d=12.46 nps=1425171)
420 ELO points

Code: Select all

Games Completed = 100 of 100 (Avg game length = 233.045 sec)
Settings = RR/64MB/60000ms+1000ms/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 5974 sec elapsed, 0 sec remaining
 1.  Stockfish 110117  (2017)       94.0/100	  89-1-10  	(L: m=1 t=0 i=0 a=0)	(D: r=6 i=2 f=1 s=1 a=0)	(tpm=1644.9 d=28.96 nps=1965019)
 2.  RobboLito 0.085   (2009)        6.0/100	  1-89-10  	(L: m=89 t=0 i=0 a=0)	(D: r=6 i=2 f=1 s=1 a=0)	(tpm=1852.4 d=16.50 nps=2572328)
480 ELO points


One can observe that software progression in 1987-2002 was fairly slow, slower than hardware progression. However, 2002-2017 saw extremely fast software progression and slow hardware progression. This is mostly due to such developments as Fruit/Rybka and Stockfish/Komodo.

In 30 years:

Total gain hardware: 150+300+100+70 = 620 ELO points
Total gain software: 100+250+420+480 = 1250 ELO points
Total gain hardware + software: 620+1250 = 1870 ELO points
FIDE ELO gain: 3300-2070 = 1230 ELO points
Compression of FIDE rating compared to computer rating: 1250/1870 = 0.67

PK
Posts: 785
Joined: Mon Jan 15, 2007 10:23 am
Location: Warsza
Contact:

Re: Progress in 30 years by four intervals of 7-8 years

Post by PK » Thu Jan 19, 2017 10:03 am

some of software progression is an effect of hardware progression - expressed by the possibility of running thousands of test games.

User avatar
Laskos
Posts: 8963
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Progress in 30 years by four intervals of 7-8 years

Post by Laskos » Thu Jan 19, 2017 3:50 pm

PK wrote:some of software progression is an effect of hardware progression - expressed by the possibility of running thousands of test games.
I don't think it's a hardware progression. UI for testing at ultra-fast controls could have been written since first 8086 processors in 1980s. That it's only by 2005 that people started testing at ultra-fast time controls in many statistically significant games is programmers' issue.

PS A correction to the opening post:
Compression of FIDE rating compared to computer rating: 1230/1870 = 0.66

User avatar
mhull
Posts: 12005
Joined: Wed Mar 08, 2006 8:02 pm
Location: Dallas, Texas
Full name: Matthew Hull

Re: Progress in 30 years by four intervals of 7-8 years

Post by mhull » Thu Jan 19, 2017 5:12 pm

Laskos wrote:I took 5 top engines of their time separated by 4 fairly equal intervals of 7-8 years since 1987, thanks to Ed's porting to UCI of his beautiful old works.
Hardware gains are for one core.
  • Mephisto Roma (1987)
    CPU: 68020 14 MHz
    FIDE ELO: 2070

    Rebel 6.0 (1994)
    CPU: i486 66 MHz
    Hardware gain: 150 ELO points
    FIDE ELO: 2350

    Shredder 6PB (2002)
    CPU: Athlon 1500 MHz
    Hardware gain: 300 ELO points
    FIDE ELO: 2750

    Robbolito 0.085 (2009)
    CPU: Core 2 3000 MHz
    Hardware gain: 100 Elo points
    FIDE ELO: 3050

    Stockfish (2017)
    CPU: i7 4000 MHz
    Hardware gain: 70 Elo points
    FIDE ELO: 3300
Software gain (equal hardware, one i7 core):

Code: Select all

Games Completed = 100 of 100 (Avg game length = 144.497 sec)
Settings = RR/64MB/60000ms in 40 moves/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 3904 sec elapsed, 0 sec remaining
 1.  Rebel 6.0          (1994)	   64.5/100	   60-31-9  	(L: m=30 t=1 i=0 a=0)	(D: r=4 i=5 f=0 s=0 a=0)	(tpm=1268.5 d=7.10 nps=335726008)
 2.  Mephisto Roma      (1987)      35.5/100	   31-60-9  	(L: m=54 t=6 i=0 a=0)	(D: r=4 i=5 f=0 s=0 a=0)	(tpm=1385.3 d=1.00 nps=0)
100 ELO points

Code: Select all

Games Completed = 100 of 100 (Avg game length = 467.077 sec)
Settings = RR/64MB/120000ms+2000ms/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 11964 sec elapsed, 0 sec remaining
 1.  Shredder Paderborn (2002)      80.5/100	  74-13-13  	(L: m=11 t=2 i=0 a=0)	(D: r=8 i=3 f=0 s=2 a=0)	(tpm=3294.0 d=12.37 nps=1409398)
 2.  Rebel 6.0          (1994)      19.5/100	  13-74-13  	(L: m=70 t=4 i=0 a=0)	(D: r=8 i=3 f=0 s=2 a=0)	(tpm=3373.5 d=9.80  nps=125460712)
250 ELO points

Code: Select all

Games Completed = 100 of 100 (Avg game length = 253.156 sec)
Settings = RR/64MB/2000ms per move/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 6428 sec elapsed, 0 sec remaining
 1.  RobboLito 0.085    (2009)      92.0/100	   87-3-10  	(L: m=3 t=0 i=0 a=0)	(D: r=7 i=1 f=2 s=0 a=0)	(tpm=2001.4 d=18.02 nps=2700313)
 2.  Shredder Paderborn (2002)       8.0/100	   3-87-10  	(L: m=86 t=0 i=1 a=0)	(D: r=7 i=1 f=2 s=0 a=0)	(tpm=2031.7 d=12.46 nps=1425171)
420 ELO points

Code: Select all

Games Completed = 100 of 100 (Avg game length = 233.045 sec)
Settings = RR/64MB/60000ms+1000ms/M 200000cp for 1000 moves, D 4000 moves/EPD:2moves_v1.epd(32000)
Time = 5974 sec elapsed, 0 sec remaining
 1.  Stockfish 110117  (2017)       94.0/100	  89-1-10  	(L: m=1 t=0 i=0 a=0)	(D: r=6 i=2 f=1 s=1 a=0)	(tpm=1644.9 d=28.96 nps=1965019)
 2.  RobboLito 0.085   (2009)        6.0/100	  1-89-10  	(L: m=89 t=0 i=0 a=0)	(D: r=6 i=2 f=1 s=1 a=0)	(tpm=1852.4 d=16.50 nps=2572328)
480 ELO points


One can observe that software progression in 1987-2002 was fairly slow, slower than hardware progression. However, 2002-2017 saw extremely fast software progression and slow hardware progression. This is mostly due to such developments as Fruit/Rybka and Stockfish/Komodo.

In 30 years:

Total gain hardware: 150+300+100+70 = 620 ELO points
Total gain software: 100+250+420+480 = 1250 ELO points
Total gain hardware + software: 620+1250 = 1870 ELO points
FIDE ELO gain: 3300-2070 = 1230 ELO points
Compression of FIDE rating compared to computer rating: 1250/1870 = 0.67
One question is how much software progress depends upon hardware, e.g. wider native registers (64-bit), larger memory for hashing and cache speed hacks.

IOW, could a modern program be adapted to squeeze into an MC68030 and demonstrate its superiority over a period MC68K program? Or could software advances even be effective in 16-bit and 8-bit architectures?

Could a programmer from the future have dominated with micros in the 1980s? Is there a way to demonstrate this?
Matthew Hull

Dann Corbit
Posts: 9492
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Progress in 30 years by four intervals of 7-8 years

Post by Dann Corbit » Thu Jan 19, 2017 6:38 pm

mhull wrote: One question is how much software progress depends upon hardware, e.g. wider native registers (64-bit), larger memory for hashing and cache speed hacks.
You can have wide integers even on an 8 bit machine, but they won't be nearly so fast.
IOW, could a modern program be adapted to squeeze into an MC68030 and demonstrate its superiority over a period MC68K program? Or could software advances even be effective in 16-bit and 8-bit architectures?
The Motorola 68x family all had 32 bit addressing. Way advanced above the intel 8088 and 8086 junk. An array based program would not suffer much but the clock speeds also were a lot slower then. You could, however address 4G RAM with the Motorola 68x processors.
Could a programmer from the future have dominated with micros in the 1980s? Is there a way to demonstrate this?
Yes, the search techniques get the branching factor down to 2 or better today. In the 80's it was around 6. A modern program transported to the motorola would blow the doors off of the software of the time.

Simple math.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

User avatar
mhull
Posts: 12005
Joined: Wed Mar 08, 2006 8:02 pm
Location: Dallas, Texas
Full name: Matthew Hull

Re: Progress in 30 years by four intervals of 7-8 years

Post by mhull » Thu Jan 19, 2017 7:04 pm

Dann Corbit wrote:
mhull wrote:Could a programmer from the future have dominated with micros in the 1980s? Is there a way to demonstrate this?
Yes, the search techniques get the branching factor down to 2 or better today. In the 80's it was around 6. A modern program transported to the motorola would blow the doors off of the software of the time.

Simple math.
Do you think the math shows that the software techniques required to achieve lower BF (and higher depth) would compensate for the potentially higher NPS of period software at both blitz and standard time controls?
Matthew Hull

Vinvin
Posts: 4200
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: Progress in 30 years by four intervals of 7-8 years

Post by Vinvin » Thu Jan 19, 2017 7:39 pm

Laskos wrote:...
Robbolito 0.085 (2009)
CPU: Core 2 3000 MHz
Hardware gain: 100 Elo points
FIDE ELO: 3050

Stockfish (2017)
CPU: i7 4000 MHz
Hardware gain: 70 Elo points
FIDE ELO: 3300
...
You could set 2 cores for the "Core 2" and 4 cores for the "i7" as they appear to be widely available in this config for the time being.

Dann Corbit
Posts: 9492
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Progress in 30 years by four intervals of 7-8 years

Post by Dann Corbit » Thu Jan 19, 2017 7:58 pm

mhull wrote:
Dann Corbit wrote:
mhull wrote:Could a programmer from the future have dominated with micros in the 1980s? Is there a way to demonstrate this?
Yes, the search techniques get the branching factor down to 2 or better today. In the 80's it was around 6. A modern program transported to the motorola would blow the doors off of the software of the time.

Simple math.
Do you think the math shows that the software techniques required to achieve lower BF (and higher depth) would compensate for the potentially higher NPS of period software at both blitz and standard time controls?
I don't understand your question.
The NPS will be about the same on period hardware for the old software and the new using mailbox. It may be that the magics bitboards would work better on 32 bit than the old techniques. Not sure about that.

Modern programs compiled for the 68K CPUs would blow the doors off the older programs on the same hardware.

It sounds to me like you may be asking for something else, though.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

S.Taylor
Posts: 8160
Joined: Thu Mar 09, 2006 2:25 am
Location: Jerusalem Israel

Re: Progress in 30 years by four intervals of 7-8 years

Post by S.Taylor » Thu Jan 19, 2017 8:13 pm

I thought that Genius (3?) was a giant leap, as was Rybka. Also Hiarcs 7 was a big leap above the rest in its time.

Dann Corbit
Posts: 9492
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Progress in 30 years by four intervals of 7-8 years

Post by Dann Corbit » Thu Jan 19, 2017 8:18 pm

S.Taylor wrote:I thought that Genius (3?) was a giant leap, as was Rybka. Also Hiarcs 7 was a big leap above the rest in its time.
Don't forget fruit
Good for the digestion
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Post Reply