Computer Chess Progress: Stockfish 7 vs Ruffian 1.0.5

[yurikvelo]

Thanks for the detailed CPU list. 2003 doesn't go down as modern in my book though

In fact, the 2003 CPU matches up quite nicely with Ruffian which is from 2002, so hardware/software kind of match.

I suggest this definition of what is modern x86-32 CPU in context of chess engines:

if speed ratio between modern engine (R4, H4, K9, S7) and old engine (top x86 engines 15-20 years ago) in single thread mode is the same - CPU is modern. If CPU favour kn/sec ratio for old engine - it's old.

I doubt if you run SF7-32-1 thread and Ruffius on Haswell and Centrino - speed difference will be the same.
But if you go back beyond Centrino, where small L1 cache, no L2, no SSE, low IPC - SF7 will slowdown more, because its techniques rely on these CPU features.

[bob]

Null move search has evolved from fixed R=1 and R=2 to a dynamic amount, so that is more recent. Reductions are more recent. Even singular extensions were around in the late 80's...

was it in general or for x86 computers?
Deep Thought, Blue obviously was SMP, but Ruffius, Fritz, early Rybka - were not

All mentioned features are not new and were known 20 years ago. Question was: how +600...700 elo was gained on the same hardware

Crafty was SMP in 1996 or so, and it was developed on a PC. Monty Newborn was using a parallel search on a minicomputer in the late 70's (Ostrich). I was using a parallel search in 1978 but on a dual-cpu Univac box, and then on the dual-cpu Cray in 1983. Jonathan Schaeffer used parallel search on Sun workstations and had a cluster version in the early 80's...

[yurikvelo]

Null move search has evolved from fixed R=1 and R=2 to a dynamic amount, so that is more recent. Reductions are more recent. Even singular extensions were around in the late 80's...

was it in general or for x86 computers?
Deep Thought, Blue obviously was SMP, but Ruffius, Fritz, early Rybka - were not

All mentioned features are not new and were known 20 years ago. Question was: how +600...700 elo was gained on the same hardware

Crafty was SMP in 1996 or so, and it was developed on a PC. Monty Newborn was using a parallel search on a minicomputer in the late 70's (Ostrich). I was using a parallel search in 1978 but on a dual-cpu Univac box, and then on the dual-cpu Cray in 1983. Jonathan Schaeffer used parallel search on Sun workstations and had a cluster version in the early 80's...

but top engines didn't use it, and we compare what provide gain against best late 1990, early 2000 engines (x86 only), not what is completely new and never known before

[bob]

Null move search has evolved from fixed R=1 and R=2 to a dynamic amount, so that is more recent. Reductions are more recent. Even singular extensions were around in the late 80's...

was it in general or for x86 computers?
Deep Thought, Blue obviously was SMP, but Ruffius, Fritz, early Rybka - were not

All mentioned features are not new and were known 20 years ago. Question was: how +600...700 elo was gained on the same hardware

Crafty was SMP in 1996 or so, and it was developed on a PC. Monty Newborn was using a parallel search on a minicomputer in the late 70's (Ostrich). I was using a parallel search in 1978 but on a dual-cpu Univac box, and then on the dual-cpu Cray in 1983. Jonathan Schaeffer used parallel search on Sun workstations and had a cluster version in the early 80's...

but top engines didn't use it, and we compare what provide gain against best late 1990, early 2000 engines (x86 only), not what is completely new and never known before

In 1996 Crafty finished either 3rd or 4th at the WMCCC event. No way to list parallel search as something new. Doesn't matter who used it, only when it was used. The "top engines" had authors that didn't think that multiple-cpu PCs would ever happen. I told them they were wrong. They were.

However, in 1994 Socrates was using parallel search to name just one well-known PC program.

[Joost Buijs]

Thanks for the detailed CPU list. 2003 doesn't go down as modern in my book though

In fact, the 2003 CPU matches up quite nicely with Ruffian which is from 2002, so hardware/software kind of match.

I suggest this definition of what is modern x86-32 CPU in context of chess engines:

if speed ratio between modern engine (R4, H4, K9, S7) and old engine (top x86 engines 15-20 years ago) in single thread mode is the same - CPU is modern. If CPU favour kn/sec ratio for old engine - it's old.

I doubt if you run SF7-32-1 thread and Ruffius on Haswell and Centrino - speed difference will be the same.
But if you go back beyond Centrino, where small L1 cache, no L2, no SSE, low IPC - SF7 will slowdown more, because its techniques rely on these CPU features.

I also think that when you compile Stockfish with one of the compilers from 20 years back (which is not possible without extensive modification) and run it for instance on a P200-MMX or on an even older CPU that the speed decrease will be a lot bigger than expected.
Like you said, very small and slow memory, no MMX, SSE or BMI, no L2 cache, very small L1 cache and no 64 bit will hurt the program a lot.
The old programs simply didn't rely on such features because they didn't exist.

Besides LMR, better tuning of evaluation features and pruning by trial and error, there is nothing new at the algorithmic front at all.
I wonder if LMR alone is worth 100 Elo, probably less, so the biggest gain is due to better tuning of features by bullet testing (which became feasible because the hardware was getting so much faster and cheaper).

[Vinvin]

Intel CPU history:

04.01.1996 - Pentium 166
10.06.1996 - Pentium 200
07.05.1997 - Pentium II 300
18.08.1997 - Pentium Pro 200
15.04.1998 - Pentium II 400
26.02.1999 - Pentium III 500
17.03.1999 - Xeon III 550
02.08.1999 - Pentium III 600
12.01.2000 - Pentium III 800
08.03.2000 - Pentium III 1000
20.11.2000 - Pentium 4 1500
23.04.2001 - Pentium 4 1700
02.06.2001 - Pentium 4 1800
27.08.2001 - Pentium 4 2000
07.01.2002 - Pentium 4 2200
02.04.2002 - Pentium 4 2400
26.08.2002 - Pentium 4 2600, FSB400
14.11.2002 - Pentium 4 3060, HT
12.03.2003 - Pentium M 1500 Centrino
23.09.2003 - Pentium 4 3200, FSB533

You run SF7 at pretty modern CPU (1M L2 cache, FSB400, SSE2)

Techniques which work well on 500-1000 kN/move might not work at 10-30 kn/move

P.S. I have alive PIII-500 (512 SDRAM, Windows XP) and could run some tests if there is interest

Some "iCOMP" values : https://en.wikipedia.org/wiki/ICOMP_(index)