Stockfish with 16 threads --- big news?

[zullil]

Hmm: +6 ELO for 8 cpu and +50 ELO for 16 cpu. This means +400 ELO for 32 cpu !?

If I continue the sequence I get
+3000 elo for 64 cpu
+20,000 elo for 128 cpu
+100,000 elo for 256 cpu
+0,000,000 elo for 512 cpu

and then

-10,000,000 elo

[Joerg Oster]

is the smp patch a regression for one core? or am I reading the test results wrong

That's extremely unlikely given that it is a non-functional change on one core. It's probably just an unlucky run.

EDIT: Actually checking again I see that the test wasn't even finished. Never draw conclusions from an unfinished test (statistics 101).

Though it is a non-functional change on one core, I experience a small slowdown on my box.
But I guess even a small elo-loss on one core of 1 or 2 elo would be acceptable, given the huge gain for 16 cores.

Looking at what the code affected, I don't see how it would slow down anything in a non-parallel search... did you compile old and new yourself, using the same makefile and compiler, and tested with the same exact settings???

Yes.
Here is a comparison of both versions with standard bench command, 20 runs each simultaneously:

Code: Select all

Engine                          | Nodes/second
sf-default                      | 1682087.0 +- 3375.84
sf-smp                          | 1625325.0 +- 1962.36

Differences                     | 56762.0 +- 1505.0
Variance of the mean            | 336.53 ( 0.59 %)
Speed up                        | 3.49 %

I'd bet that is a cache-alignment issue or something similar, if you add anything to a data structure, you can see unexpected performance losses (or gains) just because of the alignment that changes so easily. That could probably be fixed by looking at what was added and analyzing the memory around the changes to see what is different.

Here's what was changed : https://github.com/zamar/Stockfish/comp ... ...942d67a
Only some conditions when nb threads >= 2

Looks like some new variables were introduced, however, which is exactly the kind of thing that can cause this mysterious performance alteration.

It seems to be rather hardware dependant. There is no slowdown on my Intel system.

Code: Select all

Engine                    | Nodes/second
sf-master                 | 2057867.5 +- 4265.22
sf-smp                    | 2055858.0 +- 4534.7

Differences               | 284.0 +- 1697.0
Variance of the mean      | 379.46 ( 133.61 %)
Speed up                  | 0.01 %

[styx]

looking forward to the new TCEC season!

[bob]

is the smp patch a regression for one core? or am I reading the test results wrong

That's extremely unlikely given that it is a non-functional change on one core. It's probably just an unlucky run.

EDIT: Actually checking again I see that the test wasn't even finished. Never draw conclusions from an unfinished test (statistics 101).

Though it is a non-functional change on one core, I experience a small slowdown on my box.
But I guess even a small elo-loss on one core of 1 or 2 elo would be acceptable, given the huge gain for 16 cores.

Looking at what the code affected, I don't see how it would slow down anything in a non-parallel search... did you compile old and new yourself, using the same makefile and compiler, and tested with the same exact settings???

Yes.
Here is a comparison of both versions with standard bench command, 20 runs each simultaneously:

Code: Select all

Engine                          | Nodes/second
sf-default                      | 1682087.0 +- 3375.84
sf-smp                          | 1625325.0 +- 1962.36

Differences                     | 56762.0 +- 1505.0
Variance of the mean            | 336.53 ( 0.59 %)
Speed up                        | 3.49 %

I'd bet that is a cache-alignment issue or something similar, if you add anything to a data structure, you can see unexpected performance losses (or gains) just because of the alignment that changes so easily. That could probably be fixed by looking at what was added and analyzing the memory around the changes to see what is different.

Here's what was changed : https://github.com/zamar/Stockfish/comp ... ...942d67a
Only some conditions when nb threads >= 2

Looks like some new variables were introduced, however, which is exactly the kind of thing that can cause this mysterious performance alteration.

It seems to be rather hardware dependant. There is no slowdown on my Intel system.

Code: Select all

Engine                    | Nodes/second
sf-master                 | 2057867.5 +- 4265.22
sf-smp                    | 2055858.0 +- 4534.7

Differences               | 284.0 +- 1697.0
Variance of the mean      | 379.46 ( 133.61 %)
Speed up                  | 0.01 %

That's very common. I see the same thing here regularly. Very difficult to predict. Sometimes difficult to fix as well.

[lkaufman]

Yes Komodo 9 with +20 ELO will be in very bad trouble. And there is already some +10 ELO patch after SF6 - unbelievable .

Actually I'm not too worried. The latest CCRL 40/40 list has SF6 ahead of Komodo 8 by 14 elo on 4 cpu and they are tied on one core. Weighted by number of games for SF6, Sf6 is about ten elo ahead of K8 at this tc. Current tests show that although pre-K9 is only about 20 elo ahead of k8 at ultra fast levels like 20" plus 0.1", at 3' plus 1" it is 30 elo plus after about 10,000 games, indicating that K9 scales better than K8. Based on this, K9 should be comfortably ahead of SF6 on CCRL 40/40 even if we release with no further gains (unlikely) and even if SF6 gains a few elo points, as I admit is likely.

[styx]

Yes Komodo 9 with +20 ELO will be in very bad trouble. And there is already some +10 ELO patch after SF6 - unbelievable .

Actually I'm not too worried. The latest CCRL 40/40 list has SF6 ahead of Komodo 8 by 14 elo on 4 cpu and they are tied on one core. Weighted by number of games for SF6, Sf6 is about ten elo ahead of K8 at this tc. Current tests show that although pre-K9 is only about 20 elo ahead of k8 at ultra fast levels like 20" plus 0.1", at 3' plus 1" it is 30 elo plus after about 10,000 games, indicating that K9 scales better than K8. Based on this, K9 should be comfortably ahead of SF6 on CCRL 40/40 even if we release with no further gains (unlikely) and even if SF6 gains a few elo points, as I admit is likely.

which is good. we all want to see a tough battle for the title

[JJJ]

And wait for the right time before releasing Komodo 9 Larry. Because you already have improved Komodo a lot just after a release.

[shrapnel]

And wait for the right time

As Stockfish always seems to be a jump or two ahead of Komodo, let's hope it doesn't become a case of "Waiting till the cows come home"

[JJJ]

And wait for the right time

As Stockfish always seems to be a jump or two ahead of Komodo, let's hope it doesn't become a case of "Waiting till the cows come home"

You know I m also hoping for a Houdini 5 when I m alone and cold in the night :p

[shrapnel]

You know I m also hoping for a Houdini 5 when I m alone and cold in the night :p

WOW ! Now THAT'S REAL Loneliness, if one needs a chess engine to keep warm at night !