Who is stronger at chess? Computers or Humans?

[Milos]

What is your Elo estimation about for the current top 3 engines on on Intel Pentium MMX 233 ?

Do you mean that Houdini would be rated round 2700-2800 Elo points (on Intel Pentium MMX 233 ) ?

Code: Select all

Hardware-Processor        Speed      Cores     kN/s
2x Intel Xeon X5667     @ 4.60 GHz    12      28827
Intel Pentium MMX 233      260 MHz     1         61

In the computer rating list it would depend on the list basis rating, but in human Elo terms, yes I would say around 2800 Elo, or more precisely 2750+/-50Elo (for Houdini on Intel Pentium MMX 233 assuming also a proper opening book).

You could also test the thing by taking a top GM playing a match against your top machine (Intel Xeao X5667), but letting your machine play only on 1 core and having 3min/40 moves while giving 2h/40moves to a top GM.

[Sedat Canbaz]

Actually i was meaning for older auto machines (not on decent auto cars)

Usain Bolt would win,if Michael Schumacher in an Ford Model T

Saying Ford Model T is like saying chess on Charles Babbage's mechanical computer, therefore completely meaningless.

Any Top GMs of 2800 Elo would win,if Houdini,Rybka,Critter are on Intel Pentium MMX 233

Any top GM of 2800 Elo (out of a whole bunch of one ) would hardly ever have LOS greater then 50% in a longer (even) match against any of Houndini, Rybka, Critter engines on Intel Pentium MMX 233. That is also pretty obvious since (according to your own list) the fastest hardware today is not even 400 times faster than Pentium 233 in NPS, which in real time per fixed depth is probably below 250, which again is only 8 doublings or 400-480 Elo .

Doubling the processor speed,that means we can get extra approx.65 Elo

According to majority voting results,i see that Houdini 2.0c x64 should be rated around 3100 Elo

1)In case of aprox.300 Elo diffrence:

Code: Select all

--------------
Kns      Elo
--------------
14413    3100 
7206     3035 
3603     2970
1801     2905
900      2840
450      2775
225      2710
112      2645
56       2580

According to my estimations,Houdini 2.0c x64 should be rated at least 3300 Elo

2)In case of +500 Elo difference:

Code: Select all

--------------
Kns      Elo
--------------
28827    3300
14413    3235 
7206     3170
3603     3105
1801     3040
900      2975
450      2910
225      2845
112      2780
56       2715

Note also that Houdini,Rybka,Critter (on Intel Pentium MMX 233) will be played with:
-one core
-32 bit versions
-no sss4 versions
-no large pages
-less hashtables...

That means (in both cases),on Intel Pentium MMX 233:
-the above top engines are expecting to be performed approx. 2500-2600 Elo !!!


Hope this helps...


Best,
Sedat

[Milos]

Doubling the processor speed,that means we can get extra approx.65 Elo

According to majority voting results,i see that Houdini 2.0c x64 should be rated around 3100 Elo

1)In case of aprox.300 Elo diffrence:

Code: Select all

--------------
Kns      Elo
--------------
14413    3100 
7206     3035 
3603     2970
1801     2905
900      2840
450      2775
225      2710
112      2645
56       2580

According to my estimations,Houdini 2.0c x64 should be rated at least 3300 Elo

2)In case of +500 Elo difference:

Code: Select all

--------------
Kns      Elo
--------------
28827    3300
14413    3235 
7206     3170
3603     3105
1801     3040
900      2975
450      2910
225      2845
112      2780
56       2715

Note also that Houdini,Rybka,Critter (on Intel Pentium MMX 233) will be played with:
-one core
-32 bit versions
-no sss4 versions
-no large pages
-less hashtables...

That means (in both cases),on Intel Pentium MMX 233:
-the above top engines are expecting to be performed approx. 2500-2600 Elo !!!

First you have to understand a bit of computer architecture before making any analysis.
Frankly speaking with multiple cores NPS doesn't mean shit, you have to first figure out that.
12 times higher NPS on 12 cores in reality means only 120Elo points compared to single core and all that only if SMP is done really good for the given engine.
Then all the things that you listed (32b, no sse4, no large pages) already participate in NPS that you measured (they just increase rough speed, not quality of play). The only thing that plays a role is hash table size. However, since NPS is very low for Intel Pentium MMX 233, having relatively small hash is not a drawback at all (it's more like advantage, large hash would slow it down).
Finally, every doubling is not 65 Elo, there is a law of diminishing returns (if you search CCC programming forum, you'll find couple of discussion on that topic), where initially doubling was giving 70Elo, but today is more like 50Elo.

Also your starting point of 3300 Elo is too high, most probably it's around 3200 Elo in human rating.

So the real table would be something like:

Code: Select all

--------------
Kns      Elo
--------------
28827    3200 2x Intel Xeon X5667, 12 cores 
2400     3080 (single core = -120Elo)
1200     3030
600      2975
300      2915
150      2855
75       2790
37.5     2725

61       2770 Intel Pentium MMX 233  

[Adam Hair]

Actually i was meaning for older auto machines (not on decent auto cars)

Usain Bolt would win,if Michael Schumacher in an Ford Model T

Saying Ford Model T is like saying chess on Charles Babbage's mechanical computer, therefore completely meaningless.

Any Top GMs of 2800 Elo would win,if Houdini,Rybka,Critter are on Intel Pentium MMX 233

Any top GM of 2800 Elo (out of a whole bunch of one ) would hardly ever have LOS greater then 50% in a longer (even) match against any of Houndini, Rybka, Critter engines on Intel Pentium MMX 233. That is also pretty obvious since (according to your own list) the fastest hardware today is not even 400 times faster than Pentium 233 in NPS, which in real time per fixed depth is probably below 250, which again is only 8 doublings or 400-480 Elo .

Newer engines seem to get 100 to 120 Elo per doubling, at least with shorter thinking times.

[Milos]

Newer engines seem to get 100 to 120 Elo per doubling, at least with shorter thinking times.

Maybe you should look at your own rating list (pick an engine and check single vs. quad core for example for Elo difference) before making such hm, hm, funny claims .

P.S. Just to realize how funny is what you've just wrote, think of a consequence of it, it would mean not only there is no diminishing return, but there is an additional gain with speeding up of hardware. In physics that would be equivalent to perpetuum mobile claim...

[Sedat Canbaz]

Doubling the processor speed,that means we can get extra approx.65 Elo

According to majority voting results,i see that Houdini 2.0c x64 should be rated around 3100 Elo

1)In case of aprox.300 Elo diffrence:

Code: Select all

--------------
Kns      Elo
--------------
14413    3100 
7206     3035 
3603     2970
1801     2905
900      2840
450      2775
225      2710
112      2645
56       2580

According to my estimations,Houdini 2.0c x64 should be rated at least 3300 Elo

2)In case of +500 Elo difference:

Code: Select all

--------------
Kns      Elo
--------------
28827    3300
14413    3235 
7206     3170
3603     3105
1801     3040
900      2975
450      2910
225      2845
112      2780
56       2715

Note also that Houdini,Rybka,Critter (on Intel Pentium MMX 233) will be played with:
-one core
-32 bit versions
-no sss4 versions
-no large pages
-less hashtables...

That means (in both cases),on Intel Pentium MMX 233:
-the above top engines are expecting to be performed approx. 2500-2600 Elo !!!

First you have to understand a bit of computer architecture before making any analysis.
Frankly speaking with multiple cores NPS doesn't mean shit, you have to first figure out that.
12 times higher NPS on 12 cores in reality means only 120Elo points compared to single core and all that only if SMP is done really good for the given engine.
Then all the things that you listed (32b, no sse4, no large pages) already participate in NPS that you measured (they just increase rough speed, not quality of play). The only thing that plays a role is hash table size. However, since NPS is very low for Intel Pentium MMX 233, having relatively small hash is not a drawback at all (it's more like advantage, large hash would slow it down).
Finally, every doubling is not 65 Elo, there is a law of diminishing returns (if you search CCC programming forum, you'll find couple of discussion on that topic), where initially doubling was giving 70Elo, but today is more like 50Elo.

Also your starting point of 3300 Elo is too high, most probably it's around 3200 Elo in human rating.

So the real table would be something like:

Code: Select all

--------------
Kns      Elo
--------------
28827    3200 2x Intel Xeon X5667, 12 cores 
2400     3080 (single core = -120Elo)
1200     3030
600      2975
300      2915
150      2855
75       2790
37.5     2725

61       2770 Intel Pentium MMX 233  

First of all thanks for your all replays

Actually i respect your opinion,but however i dont agree with you

Yes...i am trying to understand this:
-I see that you are just a commentator-no more no less SORRY !

And its will be great if you publish some useful links(maybe yous),where other chess friends (including me) can benefit from your work

But next time no more comments please-instead of only bla,bla,bla,i prefer real facts !

But however,it seems you missed to visit/read some of my activities for Comptuter Chess

And here are some of them:

SCCT Auto232 Rating(same conditions,exception hardwares,see the Elo differences):
http://www.sedatcanbaz.com/chess/ratings/scct-auto232/

SCCT-PlayChess Rating May/October 2009 (see the Elo differences between i7 920 4.0GHz and QX 9650@3.83GHz):
http://www.sedatcanbaz.com/chess/playch ... ober-2009/

SCCT Hardware Tournament(same conditions,exception hardware speeds,see the Elo differences):
http://www.sedatcanbaz.com/chess/scct-h ... ournament/


Houdini 1.5a BenchMarks (see the differences between hardware speeds and LP ON/OFF)
http://www.sedatcanbaz.com/chess/1228-2/

Fritz Chess Benchmarks (see the hardware speed differences):
http://www.sedatcanbaz.com/chess/fritz- ... enchmarks/

SCCT's 3 Type Tournaments (same conditions,exception books,see the Elo differences):
http://www.sedatcanbaz.com/chess/sccts- ... urnaments/

One of SCCT Book Tournaments(same conditions,exception books,see the Elo differences):
http://www.sedatcanbaz.com/chess/tourna ... ournament/


SCCT- Which Hashtable size is better:
http://www.sedatcanbaz.com/chess/512mb128mb/

For more details about the differences between 32-bit,64-bit,SS42...versions,see also the useful sites e.g CCRL,CEGT,SWCR...


This time,i hope this helps...


A little note more (due to i am tired of comments):
-I have already announced a Challenge: Top GM to play with my machine (unfortunately no any contact so far,exception only comments...)

And no patience too see a such Challenge,remember also that in case of less than 500 Elo,i promised that i will donate the GMs with 5000 USD

But another issue is that,who will donate me,if i will be the Winner (i mean,if there will be more than 500 Elo)


That's All


Kind Regards,
Sedat

[Adam Hair]

Newer engines seem to get 100 to 120 Elo per doubling, at least with shorter thinking times.

Maybe you should look at your own rating list (pick an engine and check single vs. quad core for example for Elo difference) before making such hm, hm, funny claims .

P.S. Just to realize how funny is what you've just wrote, think of a consequence of it, it would mean not only there is no diminishing return, but there is an additional gain with speeding up of hardware. In physics that would be equivalent to perpetuum mobile claim...

Perhaps you should think about whether or not I would state something without anything to back up my statement. Also, I made no claim that the increase in Elo is linear as the number of doublings increase.

Take a moment to read what I wrote and realize that I am stating something that I have measured and with no intention to make you look bad.

When I arrive home in a few hours, I will present my data.

[Sedat Canbaz]

Actually the bellow table is quite good indicator about how to estimate the hardware Elo differences:

Code: Select all

Rank  Hardware          Program               Elo   Games   Kn/s    Hardware User 
04 QX6700@3.0GHz        Rybka 3 x64 t4        3217   600    230     Sedat Canbaz
05 AMD 4600 2.40GHz     Rybka 3 x64 t2        3114   550    95      Sedat Canbaz 
20 Celeron 1.70GHz      Rybka 3 w32 t1        2722   450    7       Sedat Canbaz

In other words, in my estimation:

Code: Select all

Hardware                Program               Elo   Games   Kn/s    Hardware User 
Intel Pentium MMX 233   Houdini 2.0c w32 t1   2600  
Intel Pentium MMX 233   Critter 1.4 w32 t1    2550
Intel Pentium MMX 233   Rybka 4.1 w32 t1      2550         
Intel Pentium MMX 233   Rybka 3 w32 t1        2500      

A little note more:
-I strongly believe that Houdini/Critter/Rybka (on Pentium MMX 233) will be not rated equal or stronger than Top GMs of 2800 Elo

And last (for our dear Milos):
-Do you think still that Michael Schumacher (on slowest auto machine) will become faster than Usain Bolt?!


Have a nice weekend,
Sedat

[Milos]

Perhaps you should think about whether or not I would state something without anything to back up my statement. Also, I made no claim that the increase in Elo is linear as the number of doublings increase.

Take a moment to read what I wrote and realize that I am stating something that I have measured and with no intention to make you look bad.

When I arrive home in a few hours, I will present my data.

I know you understand statistics fairly well as well as methods of chess testing, so I meant no disrespect and I don't think you invented the number. It's just highly probable that you made some error in your testing strategy that led to such a ridiculous result.
You see, there are quiet a bit of papers from 30 years ago till recent about search trees, diminishing returns, etc. and it's quite an established fact that speed doubling provides no more than 70 Elo gain. And that's the number from the past (before strong LMR and pruning). With more selective algorithms when increasing depths the EBF (effective branching factor) is increasing due to tree widening closer to the root. This increases even more diminishing returns than before.
Therefor having numbers like 120 Elo per doubling would be something so improbable as measuring some particles going faster then light. And even when really respectable institution and ppl perform serious measurements sooner or later an error is discovered.

[Sedat Canbaz]

Just my two cents,

First you have to understand a bit of computer architecture before making any analysis.

Actually first you need to learn a little bit that i7 architecture support SS42 versions,where SS42 does not work on older processors


Even SS42 versions do not work on my QX9650 architecture or on older machines too (including Pentium MMX 233)

Note also that 64-bit chess benchmarks provide/indicate more accurate measuring than Fritz Benchmarks
*Due to Fritz Benchmark is 32 bit tool

If you check the bellow results,you will notice what i mean:


Houdini 1.5a Benchmarks show that the hardware speed difference is x2,21

Code: Select all

kN/s    Mate Processor              Speed  Cores L.P  EXE  Hardware User
19389   10s  Intel Core i7 980X     @4.33GHz  6  ON   x64  Sedat Canbaz
 8749   31s  Intel Extreme QX9650   @3.66GHz  4  ON   x64  Sedat Canbaz

Fritz Benchmarks show that difference is x1,82

Code: Select all

Hardware-Processor        Speed      Cores     kN/s
Intel Core i7 980X      @ 4.33 GHz     6      18709
Intel Core 2 QX9650     @ 3.66 GHz     4      10253

Overall speed 2.21+1.82=4,03/2 = 2,015, that means i7 980X @4.33GHz is approx. two times faster than QX9650 @ 3.66 GHz

SCCT Auto232 Rating - Elo difference is approx.65-70 Elo:

Code: Select all

Rank  Name                          Elo    +    -  games  score oppo. draws
 
   3 Houdini 2.0b Pro x64 6c        3410   17   17  1049   68%  3297   46%
   6 Deep Rybka 4.1 x64 6c          3352   16   15  1201   60%  3294   55%
   9 Houdini 2.0b Pro x64 4c        3340   17   17  1030   51%  3337   47%
  15 Deep Rybka 4.1 x64 4c          3286   14   13  1602   47%  3306   56% 

More details,if you still have difficulties to understand me:


Houdini 1.5a Benchmarks show that the hardware speed difference is x37,26

Code: Select all

kN/s    Mate Processor              Speed  Cores L.P  EXE  Hardware User
33504   12s  2x Intel Xeon X5667    @4.54GHz 12  ON   x64  Kim Burcham
899     524s  AMD Athlon 64 3400+     2.40GHz  1  OFF  w32  Sedat Canbaz

Fritz Benchmarks show that diffrence is x21,08

Code: Select all

Hardware-Processor        Speed      Cores     kN/s
2x Intel Xeon X5667     @ 4.60 GHz    12      28827
AMD Athlon 64 3400+       2.40 GHz     1       1367

yes I would say around 2800 Elo, or more precisely 2750+/-50Elo (for Houdini on Intel Pentium MMX 233 assuming also a proper opening book).

Really ??? Believe me in case of such conditions,GMs of 2800 Elo will be performed approx. 150-200 Elo better than Houdini
I hope, you dont believe that Man vs Machine matches were not on Intel Pentium MMX 233 (in the early of 2000 years)

12 times higher NPS on 12 cores in reality means only 120Elo points compared to single core and all that only if SMP is done really good for the given engine.

Can you prove that ? but without comments next time please-i need serious data,testings by trusted sources

Best,
Sedat