Stockfish with 16 threads --- big news?

[shrapnel]

Looks like a patch submitted by Joona has generated a 50 point ELO gain for Stockfish running 16 threads. So perhaps a big step forward in Stockfish's multi-threaded searching. Seems very promising.

http://tests.stockfishchess.org/tests/v ... 3e9fac133b

I can get 16 Threads if I enable Hyper-Threading on my 8-Core i7 5960 X.
I use Windows 8.1.
Will my CPU also benefit ?

[zullil]

Looks like a patch submitted by Joona has generated a 50 point ELO gain for Stockfish running 16 threads. So perhaps a big step forward in Stockfish's multi-threaded searching. Seems very promising.

http://tests.stockfishchess.org/tests/v ... 3e9fac133b

I can get 16 Threads if I enable Hyper-Threading on my 8-Core i7 5960 X.
I use Windows 8.1.
Will my CPU also benefit ?

No man, you wasted your money on that i7. Too bad.

Seriously, no idea. Seems like the patch gives a small ELO boost for 8 threads, though much seems uncertain right now. Using hyperthreading is frowned upon, at least by some folks. Really don't want to open that can of worms. You'll have to test for yourself. If you have a compiler, just download the source code here: https://github.com/zamar/Stockfish/tree/smp

[bob]

is the smp patch a regression for one core? or am I reading the test results wrong

That's extremely unlikely given that it is a non-functional change on one core. It's probably just an unlucky run.

EDIT: Actually checking again I see that the test wasn't even finished. Never draw conclusions from an unfinished test (statistics 101).

Though it is a non-functional change on one core, I experience a small slowdown on my box.
But I guess even a small elo-loss on one core of 1 or 2 elo would be acceptable, given the huge gain for 16 cores.

Looking at what the code affected, I don't see how it would slow down anything in a non-parallel search... did you compile old and new yourself, using the same makefile and compiler, and tested with the same exact settings???

[Joerg Oster]

is the smp patch a regression for one core? or am I reading the test results wrong

That's extremely unlikely given that it is a non-functional change on one core. It's probably just an unlucky run.

EDIT: Actually checking again I see that the test wasn't even finished. Never draw conclusions from an unfinished test (statistics 101).

Though it is a non-functional change on one core, I experience a small slowdown on my box.
But I guess even a small elo-loss on one core of 1 or 2 elo would be acceptable, given the huge gain for 16 cores.

Looking at what the code affected, I don't see how it would slow down anything in a non-parallel search... did you compile old and new yourself, using the same makefile and compiler, and tested with the same exact settings???

Yes.
Here is a comparison of both versions with standard bench command, 20 runs each simultaneously:

Code: Select all

Engine                          | Nodes/second
sf-default                      | 1682087.0 +- 3375.84
sf-smp                          | 1625325.0 +- 1962.36

Differences                     | 56762.0 +- 1505.0
Variance of the mean            | 336.53 ( 0.59 %)
Speed up                        | 3.49 %

[bob]

is the smp patch a regression for one core? or am I reading the test results wrong

That's extremely unlikely given that it is a non-functional change on one core. It's probably just an unlucky run.

EDIT: Actually checking again I see that the test wasn't even finished. Never draw conclusions from an unfinished test (statistics 101).

Though it is a non-functional change on one core, I experience a small slowdown on my box.
But I guess even a small elo-loss on one core of 1 or 2 elo would be acceptable, given the huge gain for 16 cores.

Looking at what the code affected, I don't see how it would slow down anything in a non-parallel search... did you compile old and new yourself, using the same makefile and compiler, and tested with the same exact settings???

Yes.
Here is a comparison of both versions with standard bench command, 20 runs each simultaneously:

Code: Select all

Engine                          | Nodes/second
sf-default                      | 1682087.0 +- 3375.84
sf-smp                          | 1625325.0 +- 1962.36

Differences                     | 56762.0 +- 1505.0
Variance of the mean            | 336.53 ( 0.59 %)
Speed up                        | 3.49 %

I'd bet that is a cache-alignment issue or something similar, if you add anything to a data structure, you can see unexpected performance losses (or gains) just because of the alignment that changes so easily. That could probably be fixed by looking at what was added and analyzing the memory around the changes to see what is different.

[Dirt]

You'll have to test for yourself.

You neglected to explain how to test. Nodes/second is not the right metric.

[shrapnel]

Using hyperthreading is frowned upon, at least by some folks.

Agreed, but surely the relatively large ELO gain will surely more than compensate for the loss of ELO due to HT On ?
Anyway, I'll take your advice and make my own compiles and test.

[Vinvin]

is the smp patch a regression for one core? or am I reading the test results wrong

That's extremely unlikely given that it is a non-functional change on one core. It's probably just an unlucky run.

EDIT: Actually checking again I see that the test wasn't even finished. Never draw conclusions from an unfinished test (statistics 101).

Though it is a non-functional change on one core, I experience a small slowdown on my box.
But I guess even a small elo-loss on one core of 1 or 2 elo would be acceptable, given the huge gain for 16 cores.

Looking at what the code affected, I don't see how it would slow down anything in a non-parallel search... did you compile old and new yourself, using the same makefile and compiler, and tested with the same exact settings???

Yes.
Here is a comparison of both versions with standard bench command, 20 runs each simultaneously:

Code: Select all

Engine                          | Nodes/second
sf-default                      | 1682087.0 +- 3375.84
sf-smp                          | 1625325.0 +- 1962.36

Differences                     | 56762.0 +- 1505.0
Variance of the mean            | 336.53 ( 0.59 %)
Speed up                        | 3.49 %

I'd bet that is a cache-alignment issue or something similar, if you add anything to a data structure, you can see unexpected performance losses (or gains) just because of the alignment that changes so easily. That could probably be fixed by looking at what was added and analyzing the memory around the changes to see what is different.

Here's what was changed : https://github.com/zamar/Stockfish/comp ... ...942d67a
Only some conditions when nb threads >= 2

[zullil]

You'll have to test for yourself.

You neglected to explain how to test. Nodes/second is not the right metric.

Please see my reply. Specifically, the part about canned worms.

[shrapnel]

You'll have to test for yourself.

You neglected to explain how to test. Nodes/second is not the right metric.

Please see my reply. Specifically, the part about canned worms.

Hmmm.... maybe no worms in THIS Can !
I got 2-3 pretty MASSIVE wins against reasonably strong opponents after enabling HT on my i7 5960 X.
But then again, I usually win anyway, so can't say for sure.
I made the compiles from the Link you provided, using the Stockfish Rockwood BYO software.
Using 16 Threads with HT On may or may not be helping, but I certainly see no fall in ELO.
Here's one of the games... I was Black.
[Event "120s/Move"]
[Site "www.come2play.com"]
[Date "2015.02.16"]
[Round "?"]
[White "Stockfish"]
[Black "Stockfish 160215 64 BMI2"]
[Result "0-1"]
[ECO "C88"]
[PlyCount "69"]

{8192MB, KINGASAD V5.ctg, Anil} 1. e4 {0} e5 {B 0} 2. Nf3 {33} Nc6 {B 0} 3. Bb5
{22} a6 {B 0} 4. Ba4 {18} Nf6 {B 0} 5. O-O {22} Be7 {B 0} 6. Re1 {29} b5 {B 0}
7. Bb3 {19} O-O {B 0} 8. h3 {23} Bb7 {B 0} 9. d3 {19} d5 {B 0} 10. exd5 {25}
Nxd5 {B 0} 11. Nxe5 {21} Nd4 {B 0} 12. Nc3 {21} Nb4 {B 0} 13. Ne4 Nxb3 {0.00/
33 72} 14. axb3 {0.06/25 1} Qd5 {0.11/35 104} 15. f4 f6 {-0.27/34 69} 16. Nf3 {
-0.29/27 1} f5 {-0.42/37 89} 17. Nf2 {-0.31/25 1} Qd8 {-0.41/37 97} 18. Re3 Bd6
{-0.37/36 87} 19. c3 {-0.29/25 1} Nd5 {-0.39/37 76} 20. Re1 {-0.39/28 1} Bxf4 {
-0.43/37 90} 21. d4 {-0.39/23 1} Bg3 {-0.53/35 105} 22. Re6 {-0.46/30 8} Nf6 {
-0.40/35 96} 23. Nd3 {-0.42/25 1} Ne4 {-0.39/34 91} 24. Bf4 {-0.52/31 11} Qd7 {
-0.62/36 99} 25. d5 Qxd5 {-0.54/36 89} 26. Re5 {-0.54/25 1} Qf7 {-0.74/39 92}
27. Bxg3 {-0.72/30 2} Nxg3 {-0.66/39 87} 28. Nc5 {-0.67/27 1} Rad8 {-0.64/41 90
} 29. Nd4 {-0.64/25 1} Rd5 {-0.60/41 101} 30. Rxd5 Bxd5 {-0.88/37 78} 31. Rxa6
Re8 {-1.44/35 52} 32. Nde6 c6 {-1.46/35 33} 33. Nf4 {-1.46/28 1} Qe7 {-1.91/38
67} 34. Ncd3 {-1.78/36 22} Bf7 {-1.91/39 86} 35. Qf3 {-2.03/39 55} 0-1