Stockfish 5.0 x64 8CPU @CEGT 40/4

[snino64]

It is strange to note that stockfish 5 x64 2CPU has more elo than stockfish 5 x64 4CPU like Houdini 4 x64 4CPU have more elo than Houdini 4 x64 8CPU
Is this correct?
Is it better to use stockfish 2CPU O stockfish x64 8CPU instead of 4CPU?

1 Stockfish 5.0 x64 8CPU 3253 25 25 400 65.9% 3139 48.2%
2 Stockfish 5.0 x64 2CPU 3176 17 17 850 61.5% 3095 50.1%
3 Houdini 4.0 x64 4CPU 3174 12 12 2700 76.4% 2969 28.4%
4 Houdini 3.0 x64 4CPU 3171 10 10 4100 79.6% 2934 29.2%
5 Stockfish 5.0 x64 4CPU 3170 23 23 450 54.0% 3143 49.8%
6 Houdini 4.0 x64 8CPU 3170 52 52 100 51.0% 3163 42.0%
7 Komodo 7.0a x64 4CPU 3157 23 23 450 46.7% 3180 50.7%

[Uri Blass]

It is strange to note that stockfish 5 x64 2CPU has more elo than stockfish 5 x64 4CPU like Houdini 4 x64 4CPU have more elo than Houdini 4 x64 8CPU
Is this correct?
Is it better to use stockfish 2CPU O stockfish x64 8CPU instead of 4CPU?

1 Stockfish 5.0 x64 8CPU 3253 25 25 400 65.9% 3139 48.2%
2 Stockfish 5.0 x64 2CPU 3176 17 17 850 61.5% 3095 50.1%
3 Houdini 4.0 x64 4CPU 3174 12 12 2700 76.4% 2969 28.4%
4 Houdini 3.0 x64 4CPU 3171 10 10 4100 79.6% 2934 29.2%
5 Stockfish 5.0 x64 4CPU 3170 23 23 450 54.0% 3143 49.8%
6 Houdini 4.0 x64 8CPU 3170 52 52 100 51.0% 3163 42.0%
7 Komodo 7.0a x64 4CPU 3157 23 23 450 46.7% 3180 50.7%

3170+-23 is not weaker than 3176+-17

The first number can be easily 3185 when the second number can be easily 3165

[snino64]

True, but with two other cores should have more elo points. stockfish 5.0 x64 4CPU should be better, ie should be sandwiched between 2CPU and 8cpu, although the margin of error is large. The problem could be the small number of items for stockfish 4CPU.
Stockfish 4CPU performance should be much greater than Stockfish 2CPU.

[Sedat Canbaz]

It is strange to note that stockfish 5 x64 2CPU has more elo than stockfish 5 x64 4CPU like Houdini 4 x64 4CPU have more elo than Houdini 4 x64 8CPU
Is this correct?
Is it better to use stockfish 2CPU O stockfish x64 8CPU instead of 4CPU?

1 Stockfish 5.0 x64 8CPU 3253 25 25 400 65.9% 3139 48.2%
2 Stockfish 5.0 x64 2CPU 3176 17 17 850 61.5% 3095 50.1%
3 Houdini 4.0 x64 4CPU 3174 12 12 2700 76.4% 2969 28.4%
4 Houdini 3.0 x64 4CPU 3171 10 10 4100 79.6% 2934 29.2%
5 Stockfish 5.0 x64 4CPU 3170 23 23 450 54.0% 3143 49.8%
6 Houdini 4.0 x64 8CPU 3170 52 52 100 51.0% 3163 42.0%
7 Komodo 7.0a x64 4CPU 3157 23 23 450 46.7% 3180 50.7%

Actually a few years ago,
I already stated about why we see those Elo differences with adapted time controls:
http://www.talkchess.com/forum/viewtopi ... t&start=20
http://forum.computerschach.de/cgi-bin/ ... l?tid=4297


And the most strange thing:
CEGT shows 77 Elo difference (between Houdini 4c vs Houdini 1c)
CCRL shows 134 Elo difference (between Houdini 4c vs Houdini 1c)

And now,
On which results we should relay ??

Code: Select all

CEGT (adapted time control 40/4) Rating

Houdini 4.0 x64 4CPU 	      3174 Elo	
Houdini 4.0 x64 1CPU 	      3097 Elo
*Elo Difference 1c vs 4c        77 Elo

Houdini 2.0c x64 4CPU 	      3131 Elo
Houdini 2.0c x64 1CPU 	      3032 Elo
*Elo Difference 1c vs 4c        99 Elo

******************************************

CCRL (adapted time control 40/4) Rating

Houdini 4 64-bit 4CPU        3369 Elo
Houdini 4 64-bit 1CPU        3235 Elo
*Elo Difference 1c vs 4c      134 Elo


Houdini 2.0c 64-bit 4CPU     3273 Elo
Houdini 2.0c 64-bit 1CPU     3198 Elo
*Elo Difference 1c vs 4c       75 Elo

********************************************

SCCT (Auto232 mode 4m+2s) Rating

Houdini 2.0 Pro x64  4c      3363 Elo
Houdini 2.0 Pro x64 1c       3254 Elo
*Elo Difference 1c vs 4c      109 Elo

Btw,(once more I say: many thanks to both great teams)
CCRL and CEGT are one of the best ratings, especially I'm grateful that they test a lot of engines...

But however I think,
The both ratings have one same disadvantage: some of their MP's Engines Elo are not very accurate!
In other words, sometimes the adapted ratings don't give us very accurate MP's Elo results...

I believe this happens (differences in Elo) due to mainly:
1) Adapted time controls (played on different hardwares/processors)
2) They (CCRL/CEGT) are using a lot of different kind of opening books



Best,
Sedat

[Sedat Canbaz]

EDIT:

CCRL (adapted time control 40/4) Rating

Code: Select all

Houdini 4 64-bit 4CPU        3336 Elo
Houdini 4 64-bit 1CPU        3235 Elo
*Elo Difference 1c vs 4c      101 Elo

But anyway,
Still the there is a difference between CEGT and CCRL

[ThatsIt]

True, but with two other cores should have more elo points. stockfish 5.0 x64 4CPU should be better, ie should be sandwiched between 2CPU and 8cpu, although the margin of error is large. The problem could be the small number of items for stockfish 4CPU.
Stockfish 4CPU performance should be much greater than Stockfish 2CPU.

That's the reason why i wrote:
"Keep the less number of games for some engines in mind !"

Best wishes,
G.S.
(CEGT team)

[zullil]

Stockfish 4CPU performance should be much greater than Stockfish 2CPU.

It is. And this is not contradicted by the posted results. Please reread Uri's post.

[Sedat Canbaz]

Stockfish 4CPU performance should be much greater than Stockfish 2CPU.

It is. And this is not contradicted by the posted results. Please reread Uri's post.

No !

I don't agree with Uri's posting ... many experts are biased only only Elo error margins... but they forget that there can be other issues...

In other words,
It will never happen on same equal hardware conditions (with well optimized neutral short book)
I mean, 2 cores can not be the leader over 4 cores, even in 20 games match

Hopes this helps...

[zullil]

I mean, 2 cores can not be the leader over 4 cores, even in 10 games match

This statement is false. Might be "improbable", but surely not impossible.

[Sedat Canbaz]

I mean, 2 cores can not be the leader over 4 cores, even in 10 games match

This statement is false. Might be "improbable", but surely not impossible.

Ok...You are 100% right , exception: (I wrote in my previous posting) even in 20 games match