Computer chess progress over the last 20 years!

[Joost Buijs]

Another thing to take into consideration is that 20 years ago compilers were a lot worse as they are now.
Although this is software improvement, it doesn't mean that this is engine improvement.
A factor of 2 (speed wise) in generated code efficiency can easily amount to ~70 Elo.
Maybe today a speed doubling is worth 60 Elo (due to diminishing returns), in the past it was clearly worth more.

[fierz]

Dear Joost,


I stated my "methods" in the end with all the links. I did not go to the SSDF list of 1997, but rather to the current one (http://ssdf.bosjo.net/long.txt) which includes all the old engines. There, it says

159 Hiarcs 6.0 49MB P200 MMX 2421 24 -24 829 56% 2378
161 Rebel 8.0 51MB P200 MMX 2412 21 -22 1011 47% 2436

Admittedly, Hiarcs is stronger, but actually 9 rating points are not relevant (and maybe random noise). I adjusted this 2412 rating of rebel upwards by 112 elo to arrive at my 2524 rating.

Again, I realize all this is a bit sketchy, but I just noticed that Ed Schröder is offering Rebel 6 and Rebel Century as UCI engines as free downloads. I'll run a match against Stockfish and see what I get.

cheers
Martin

[Vinvin]

I made a graph out of that list for a better view

http://www.fierz.ch/chessprogramprogress.png

It's pretty amazing how the progress seems to be quite relentless and steady. Perhaps there is a bit of a burst with Rybka, but apart from that the strength increase doesn't change a lot year over year.

Hello Martin, I just generate 2 lists
1) the latest unified rating list (CEGT+CCRL+WBEC+amateur+... = 2.26M games ), 32-bit and 64-bit engines inside.
2) a list based on RWBC + Le Fou numérique (428K games) only 32-bit engines.

Both list generate with Bayeselo because Ordo reports : "Database is not well connected by games..." .

Both list are calibrate with Shredder 12 = 2800 but due to few games in the second list, "Rybka 2.2n2" rating gives a wide difference : 2771 and 2739.

Unified : http://home.scarlet.be/vincentlejeune/c ... -03-15.txt
RWBC+FouNumerique : http://home.scarlet.be/vincentlejeune/c ... -03-15.txt

I'll look if I can merge both PGN databases without inconveniences.

Vincent

[Vinvin]

I made a graph out of that list for a better view

http://www.fierz.ch/chessprogramprogress.png

It's pretty amazing how the progress seems to be quite relentless and steady. Perhaps there is a bit of a burst with Rybka, but apart from that the strength increase doesn't change a lot year over year.

Hello Martin, I just generate 2 lists
1) the latest unified rating list (CEGT+CCRL+WBEC+amateur+... = 2.26M games ), 32-bit and 64-bit engines inside.
2) a list based on RWBC + Le Fou numérique (428K games) only 32-bit engines.
...

I forget to explain the point : there are a lot of old engines in the second list (Fritz 5.32, Shredder family, ...) but sometimes with very few games.

[Joost Buijs]

I stated my "methods" in the end with all the links. I did not go to the SSDF list of 1997, but rather to the current one (http://ssdf.bosjo.net/long.txt) which includes all the old engines.

Hi Martin,

Sorry, I have to admit that I didn't look carefully enough and assumed that you were using the old SSDF rating lists.
This doesn't alter the fact that I believe that an increase of 750 Elo for software improvements over the last 20 years is too high.
One of the problems might be that there are isolated rating-clusters within the rating lists because not all engines played each other.
Also changing opening books and changing hardware makes the data unreliable in the long run.

Like you said, one way to find out is to have the programs from 20 years ago play a match with current programs.
To rule out compiler optimization you need to recompile current programs with a compiler that was in use 20 years ago.
Features like 64 bit and intrinsics like popcnt and lzcnt should be disabled, when you use an old compiler these features won't be available at all.
To make it completely honest such a match should be run on hardware from 20 years ago, maybe there is still somebody with hardware like this stored in his attic.

Where the large improvement we saw over the last 20 years comes from is difficult to tell, LMR is a new thing, most other techniques in use today were already known.
In my eyes the biggest improvement is due to better tuning of evaluation parameters by 'bullet testing', this also enabled the use of more agressive pruning without making too much errors.
In a way improvment in hardware made 'bullet testing' feasible, in my eyes improvement in software is always closely related to improvement in hardware.
I fully agree with Bob that current hardware enables you to do things that were not doable in the past.

Joost