Stockfish 8 - Double time control vs. 2 threads

[fastgm]

Stockfish 8
1 thread (5 minutes + 3 seconds) vs. 2 threads (2.5 minutes + 1.5 seconds)

Code: Select all

    Program                                Elo    +   -   Games   Score   Av.Op.  Draws
 ----------------------------------------------------------------------------------------
  1 Stockfish 8 - 1 thread    5' + 3"    : 3003   5   5   3000    51.0 %   2997   85.2 %
  2 Stockfish 8 - 2 threads 2.5' + 1.5"  : 2997   5   5   3000    49.0 %   3003   85.2 %


Av.Op. Elo = 3000

Result     : 1529.0/3000 (+251,=2556,-193)
Perf.      : 51.0 %
Margins    :
 68 %      : (+  0.3,-  0.3 %) -> [ 50.6, 51.3 %]
 95 %      : (+  0.7,-  0.7 %) -> [ 50.3, 51.7 %]
 99.7 %    : (+  1.0,-  1.0 %) -> [ 49.9, 52.0 %]

Elo        : 3007
Margins    :
 68 %      : (+  2,-  2) -> [3004,3009]
 95 %      : (+  5,-  5) -> [3002,3012]
 99.7 %    : (+  7,-  7) -> [3000,3014]


Games        :   3000 (finished)

White Wins   :    341 (11.4 %)
Black Wins   :    103 ( 3.4 %)
Draws        :   2556 (85.2 %)

White Perf.  : 54.0 %
Black Perf.  : 46.0 %


Individual statistics:

1 Stockfish 8 - 1 thread    5' + 3"    : 3003  3000 (+251,=2556,-193), 51.0 %
2 Stockfish 8 - 2 threads 2.5' + 1.5"  : 2997  3000 (+193,=2556,-251), 49.0 %

[ernest]

Very interesting, Andreas, thanks !!!

[Laskos]

Interesting as usual, thanks! From your previous thread we already know that doubling time at this time control is worth about 90 ELO points. Here the difference is 6+/-5 ELO points. Means 0.067 doublings. This means that the effective speed-up 1 -> 2 threads is 2^(1-0.067) ~ 1.91. For the first time it is measured so precisely for Lazy SMP engine. With YBW SMP like in Crafty time-to-depth can be used.

[ernest]

Interesting as usual, thanks! From your previous thread we already know that doubling time at this time control is worth about 90 ELO points. Here the difference is 6+/-5 ELO points. Means 0.067 doublings. This means that the effective speed-up 1 -> 2 threads is 2^(1-0.067) ~ 1.91. For the first time it is measured so precisely for Lazy SMP engine. With YBW SMP like in Crafty time-to-depth can be used.

These numbers, 90 Elo and 1.91, seem high.
Is it because they result from games engine vs same engine ?

[Laskos]

Interesting as usual, thanks! From your previous thread we already know that doubling time at this time control is worth about 90 ELO points. Here the difference is 6+/-5 ELO points. Means 0.067 doublings. This means that the effective speed-up 1 -> 2 threads is 2^(1-0.067) ~ 1.91. For the first time it is measured so precisely for Lazy SMP engine. With YBW SMP like in Crafty time-to-depth can be used.

These numbers, 90 Elo and 1.91, seem high.
Is it because they result from games engine vs same engine ?

The results of Komodo 9.3 self-play, which I would think also apply to Srockfish 8 self-play, were presented by Andreas here:
http://www.talkchess.com/forum/viewtopi ... 84&start=0
At this time control, about 93 ELO points for doublig of Komodo, hence 1.91 effective speed-up 1->2 threads for Stockfish 8. It's not unexpected, 1->2 threads scale very well, with Lazy or YBW SMP, but Andreas' measurement is the most precise one up to date for Lazy SMP. It can only be measured through games, not NPS, time-to-depth and other easier methods.

[Werewolf]

Interesting as usual, thanks! From your previous thread we already know that doubling time at this time control is worth about 90 ELO points. Here the difference is 6+/-5 ELO points. Means 0.067 doublings. This means that the effective speed-up 1 -> 2 threads is 2^(1-0.067) ~ 1.91. For the first time it is measured so precisely for Lazy SMP engine. With YBW SMP like in Crafty time-to-depth can be used.

Wow. That's a big upgrade from the standard a few years ago when Rybka ruled: "A doubling of cores is 1.7x faster"

This may have huge implications for hyper threading and monstrously big machines with high core counts.

[fastgm]

Stockfish 8
1 thread (5 minutes + 3 seconds) vs. 4 threads (75 seconds + 0.75 seconds)

Code: Select all

    Program                                Elo    +   -   Games   Score   Av.Op.  Draws
 ----------------------------------------------------------------------------------------
  1 Stockfish 8 - 1 thread   5' + 3"     : 3009   5   5   3000    52.6 %   2991   83.9 %
  2 Stockfish 8 - 4 threads 75" + 0.75"  : 2991   5   5   3000    47.4 %   3009   83.9 %


Wins   = 320
Draws  = 2516
Losses = 164
Av.Op. Elo = 3000

Result     : 1578.0/3000 (+320,=2516,-164)
Perf.      : 52.6 %
Margins    :
 68 %      : (+  0.4,-  0.4 %) -> [ 52.2, 53.0 %]
 95 %      : (+  0.7,-  0.7 %) -> [ 51.9, 53.3 %]
 99.7 %    : (+  1.1,-  1.1 %) -> [ 51.5, 53.7 %]

Elo        : 3018
Margins    :
 68 %      : (+  3,-  3) -> [3016,3021]
 95 %      : (+  5,-  5) -> [3013,3023]
 99.7 %    : (+  8,-  7) -> [3011,3026]


Games        :   3000 (finished)

White Wins   :    362 (12.1 %)
Black Wins   :    122 ( 4.1 %)
Draws        :   2516 (83.9 %)

White Perf.  : 54.0 %
Black Perf.  : 46.0 %


Individual statistics:

1 Stockfish 8 - 1 thread   5' + 3"     : 3009  3000 (+320,=2516,-164), 52.6 %
2 Stockfish 8 - 4 threads 75" + 0.75"  : 2991  3000 (+164,=2516,-320), 47.4 %


Engine                                Depth       Time   Games     Moves  Average
---------------------------------------------------------------------------------
Stockfish 8 - 1 thread   5' + 3"      28.60  349:27:58    3000    188602    6.67
Stockfish 8 - 4 threads 75" + 0.75"   26.32   90:17:50    3000    188629    1.72

[Laskos]

Stockfish 8
1 thread (5 minutes + 3 seconds) vs. 4 threads (75 seconds + 0.75 seconds)

Code: Select all

    Program                                Elo    +   -   Games   Score   Av.Op.  Draws
 ----------------------------------------------------------------------------------------
  1 Stockfish 8 - 1 thread   5' + 3"     : 3009   5   5   3000    52.6 %   2991   83.9 %
  2 Stockfish 8 - 4 threads 75" + 0.75"  : 2991   5   5   3000    47.4 %   3009   83.9 %


Wins   = 320
Draws  = 2516
Losses = 164
Av.Op. Elo = 3000

Result     : 1578.0/3000 (+320,=2516,-164)
Perf.      : 52.6 %
Margins    :
 68 %      : (+  0.4,-  0.4 %) -> [ 52.2, 53.0 %]
 95 %      : (+  0.7,-  0.7 %) -> [ 51.9, 53.3 %]
 99.7 %    : (+  1.1,-  1.1 %) -> [ 51.5, 53.7 %]

Elo        : 3018
Margins    :
 68 %      : (+  3,-  3) -> [3016,3021]
 95 %      : (+  5,-  5) -> [3013,3023]
 99.7 %    : (+  8,-  7) -> [3011,3026]


Games        :   3000 (finished)

White Wins   :    362 (12.1 %)
Black Wins   :    122 ( 4.1 %)
Draws        :   2516 (83.9 %)

White Perf.  : 54.0 %
Black Perf.  : 46.0 %


Individual statistics:

1 Stockfish 8 - 1 thread   5' + 3"     : 3009  3000 (+320,=2516,-164), 52.6 %
2 Stockfish 8 - 4 threads 75" + 0.75"  : 2991  3000 (+164,=2516,-320), 47.4 %


Engine                                Depth       Time   Games     Moves  Average
---------------------------------------------------------------------------------
Stockfish 8 - 1 thread   5' + 3"      28.60  349:27:58    3000    188602    6.67
Stockfish 8 - 4 threads 75" + 0.75"   26.32   90:17:50    3000    188629    1.72

Another important post, thanks. Factor of 4 in time is 2 doublings, from your data in another thread, the first one is worth 101 ELO points, the second 93 ELO points. In total 194 ELO points for quadrupling. The difference in this test is 18 ELO points, or 18/194 ~ 0.0928 quadruplings. The speed-up 1 --> 4 threads is 4^(1-0.0928) ~ 3.52. Again, the most precise measurement for Lazy SMP.

So,

1 --> 2 threads: 1.91
1 --> 4 threads: 3.52

[Milos]

So,

1 --> 2 threads: 1.91
1 --> 4 threads: 3.52

What I find amazing is that if you calculate efficiency according to Amdahl's Law it is exactly 95.5% both when going from 1->2 and 1->4 cores. Matching is on the second decimal!!! Amazing.

[Laskos]

So,

1 --> 2 threads: 1.91
1 --> 4 threads: 3.52

What I find amazing is that if you calculate efficiency according to Amdahl's Law it is exactly 95.5% both when going from 1->2 and 1->4 cores. Matching is on the second decimal!!! Amazing.

Very interesting observation. And in line to Amdahl's fit I did for Komodo 9.3 using Andreas' results:
http://www.talkchess.com/forum/viewtopi ... 4&start=45
Andreas posts very important, hardcore results which use huge CPU time.

From your observation, the predictions would be:

1 --> 8 threads: 6.1
1 --> 16 threads: 9.55
1 --> 32 threads: 13.4

That is somewhat higher than the old YBW numbers.