Stockfish 8 - Double time control vs. 2 threads

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Rebel, chrisw

fastgm
Posts: 818
Joined: Mon Aug 19, 2013 6:57 pm

Stockfish 8 - Double time control vs. 2 threads

Post by fastgm »

Stockfish 8
1 thread (5 minutes + 3 seconds) vs. 2 threads (2.5 minutes + 1.5 seconds)

Code: Select all

    Program                                Elo    +   -   Games   Score   Av.Op.  Draws
 ----------------------------------------------------------------------------------------
  1 Stockfish 8 - 1 thread    5' + 3"    : 3003   5   5   3000    51.0 %   2997   85.2 %
  2 Stockfish 8 - 2 threads 2.5' + 1.5"  : 2997   5   5   3000    49.0 %   3003   85.2 %


Av.Op. Elo = 3000

Result     : 1529.0/3000 (+251,=2556,-193)
Perf.      : 51.0 %
Margins    :
 68 %      : (+  0.3,-  0.3 %) -> [ 50.6, 51.3 %]
 95 %      : (+  0.7,-  0.7 %) -> [ 50.3, 51.7 %]
 99.7 %    : (+  1.0,-  1.0 %) -> [ 49.9, 52.0 %]

Elo        : 3007
Margins    :
 68 %      : (+  2,-  2) -> [3004,3009]
 95 %      : (+  5,-  5) -> [3002,3012]
 99.7 %    : (+  7,-  7) -> [3000,3014]


Games        :   3000 (finished)

White Wins   :    341 (11.4 %)
Black Wins   :    103 ( 3.4 %)
Draws        :   2556 (85.2 %)

White Perf.  : 54.0 %
Black Perf.  : 46.0 %


Individual statistics:

1 Stockfish 8 - 1 thread    5' + 3"    : 3003  3000 (+251,=2556,-193), 51.0 %
2 Stockfish 8 - 2 threads 2.5' + 1.5"  : 2997  3000 (+193,=2556,-251), 49.0 %
ernest
Posts: 2041
Joined: Wed Mar 08, 2006 8:30 pm

Re: Stockfish 8 - Double time control vs. 2 threads

Post by ernest »

Very interesting, Andreas, thanks !!!
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Stockfish 8 - Double time control vs. 2 threads

Post by Laskos »

Interesting as usual, thanks! From your previous thread we already know that doubling time at this time control is worth about 90 ELO points. Here the difference is 6+/-5 ELO points. Means 0.067 doublings. This means that the effective speed-up 1 -> 2 threads is 2^(1-0.067) ~ 1.91. For the first time it is measured so precisely for Lazy SMP engine. With YBW SMP like in Crafty time-to-depth can be used.
ernest
Posts: 2041
Joined: Wed Mar 08, 2006 8:30 pm

Re: Stockfish 8 - Double time control vs. 2 threads

Post by ernest »

Laskos wrote:Interesting as usual, thanks! From your previous thread we already know that doubling time at this time control is worth about 90 ELO points. Here the difference is 6+/-5 ELO points. Means 0.067 doublings. This means that the effective speed-up 1 -> 2 threads is 2^(1-0.067) ~ 1.91. For the first time it is measured so precisely for Lazy SMP engine. With YBW SMP like in Crafty time-to-depth can be used.
These numbers, 90 Elo and 1.91, seem high.
Is it because they result from games engine vs same engine ?
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Stockfish 8 - Double time control vs. 2 threads

Post by Laskos »

ernest wrote:
Laskos wrote:Interesting as usual, thanks! From your previous thread we already know that doubling time at this time control is worth about 90 ELO points. Here the difference is 6+/-5 ELO points. Means 0.067 doublings. This means that the effective speed-up 1 -> 2 threads is 2^(1-0.067) ~ 1.91. For the first time it is measured so precisely for Lazy SMP engine. With YBW SMP like in Crafty time-to-depth can be used.
These numbers, 90 Elo and 1.91, seem high.
Is it because they result from games engine vs same engine ?
The results of Komodo 9.3 self-play, which I would think also apply to Srockfish 8 self-play, were presented by Andreas here:
http://www.talkchess.com/forum/viewtopi ... 84&start=0
At this time control, about 93 ELO points for doublig of Komodo, hence 1.91 effective speed-up 1->2 threads for Stockfish 8. It's not unexpected, 1->2 threads scale very well, with Lazy or YBW SMP, but Andreas' measurement is the most precise one up to date for Lazy SMP. It can only be measured through games, not NPS, time-to-depth and other easier methods.
Werewolf
Posts: 1795
Joined: Thu Sep 18, 2008 10:24 pm

Re: Stockfish 8 - Double time control vs. 2 threads

Post by Werewolf »

Laskos wrote:Interesting as usual, thanks! From your previous thread we already know that doubling time at this time control is worth about 90 ELO points. Here the difference is 6+/-5 ELO points. Means 0.067 doublings. This means that the effective speed-up 1 -> 2 threads is 2^(1-0.067) ~ 1.91. For the first time it is measured so precisely for Lazy SMP engine. With YBW SMP like in Crafty time-to-depth can be used.

Wow. That's a big upgrade from the standard a few years ago when Rybka ruled: "A doubling of cores is 1.7x faster"

This may have huge implications for hyper threading and monstrously big machines with high core counts.
fastgm
Posts: 818
Joined: Mon Aug 19, 2013 6:57 pm

Stockfish 8 - Quadruple time control vs. 4 threads

Post by fastgm »

Stockfish 8
1 thread (5 minutes + 3 seconds) vs. 4 threads (75 seconds + 0.75 seconds)

Code: Select all

    Program                                Elo    +   -   Games   Score   Av.Op.  Draws
 ----------------------------------------------------------------------------------------
  1 Stockfish 8 - 1 thread   5' + 3"     : 3009   5   5   3000    52.6 %   2991   83.9 %
  2 Stockfish 8 - 4 threads 75" + 0.75"  : 2991   5   5   3000    47.4 %   3009   83.9 %


Wins   = 320
Draws  = 2516
Losses = 164
Av.Op. Elo = 3000

Result     : 1578.0/3000 (+320,=2516,-164)
Perf.      : 52.6 %
Margins    :
 68 %      : (+  0.4,-  0.4 %) -> [ 52.2, 53.0 %]
 95 %      : (+  0.7,-  0.7 %) -> [ 51.9, 53.3 %]
 99.7 %    : (+  1.1,-  1.1 %) -> [ 51.5, 53.7 %]

Elo        : 3018
Margins    :
 68 %      : (+  3,-  3) -> [3016,3021]
 95 %      : (+  5,-  5) -> [3013,3023]
 99.7 %    : (+  8,-  7) -> [3011,3026]


Games        :   3000 (finished)

White Wins   :    362 (12.1 %)
Black Wins   :    122 ( 4.1 %)
Draws        :   2516 (83.9 %)

White Perf.  : 54.0 %
Black Perf.  : 46.0 %


Individual statistics:

1 Stockfish 8 - 1 thread   5' + 3"     : 3009  3000 (+320,=2516,-164), 52.6 %
2 Stockfish 8 - 4 threads 75" + 0.75"  : 2991  3000 (+164,=2516,-320), 47.4 %


Engine                                Depth       Time   Games     Moves  Average
---------------------------------------------------------------------------------
Stockfish 8 - 1 thread   5' + 3"      28.60  349:27:58    3000    188602    6.67
Stockfish 8 - 4 threads 75" + 0.75"   26.32   90:17:50    3000    188629    1.72
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Stockfish 8 - Quadruple time control vs. 4 threads

Post by Laskos »

fastgm wrote:Stockfish 8
1 thread (5 minutes + 3 seconds) vs. 4 threads (75 seconds + 0.75 seconds)

Code: Select all

    Program                                Elo    +   -   Games   Score   Av.Op.  Draws
 ----------------------------------------------------------------------------------------
  1 Stockfish 8 - 1 thread   5' + 3"     : 3009   5   5   3000    52.6 %   2991   83.9 %
  2 Stockfish 8 - 4 threads 75" + 0.75"  : 2991   5   5   3000    47.4 %   3009   83.9 %


Wins   = 320
Draws  = 2516
Losses = 164
Av.Op. Elo = 3000

Result     : 1578.0/3000 (+320,=2516,-164)
Perf.      : 52.6 %
Margins    :
 68 %      : (+  0.4,-  0.4 %) -> [ 52.2, 53.0 %]
 95 %      : (+  0.7,-  0.7 %) -> [ 51.9, 53.3 %]
 99.7 %    : (+  1.1,-  1.1 %) -> [ 51.5, 53.7 %]

Elo        : 3018
Margins    :
 68 %      : (+  3,-  3) -> [3016,3021]
 95 %      : (+  5,-  5) -> [3013,3023]
 99.7 %    : (+  8,-  7) -> [3011,3026]


Games        :   3000 (finished)

White Wins   :    362 (12.1 %)
Black Wins   :    122 ( 4.1 %)
Draws        :   2516 (83.9 %)

White Perf.  : 54.0 %
Black Perf.  : 46.0 %


Individual statistics:

1 Stockfish 8 - 1 thread   5' + 3"     : 3009  3000 (+320,=2516,-164), 52.6 %
2 Stockfish 8 - 4 threads 75" + 0.75"  : 2991  3000 (+164,=2516,-320), 47.4 %


Engine                                Depth       Time   Games     Moves  Average
---------------------------------------------------------------------------------
Stockfish 8 - 1 thread   5' + 3"      28.60  349:27:58    3000    188602    6.67
Stockfish 8 - 4 threads 75" + 0.75"   26.32   90:17:50    3000    188629    1.72
Another important post, thanks. Factor of 4 in time is 2 doublings, from your data in another thread, the first one is worth 101 ELO points, the second 93 ELO points. In total 194 ELO points for quadrupling. The difference in this test is 18 ELO points, or 18/194 ~ 0.0928 quadruplings. The speed-up 1 --> 4 threads is 4^(1-0.0928) ~ 3.52. Again, the most precise measurement for Lazy SMP.

So,

1 --> 2 threads: 1.91
1 --> 4 threads: 3.52
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: Stockfish 8 - Quadruple time control vs. 4 threads

Post by Milos »

Laskos wrote: So,

1 --> 2 threads: 1.91
1 --> 4 threads: 3.52
What I find amazing is that if you calculate efficiency according to Amdahl's Law it is exactly 95.5% both when going from 1->2 and 1->4 cores. Matching is on the second decimal!!! Amazing.
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Stockfish 8 - Quadruple time control vs. 4 threads

Post by Laskos »

Milos wrote:
Laskos wrote: So,

1 --> 2 threads: 1.91
1 --> 4 threads: 3.52
What I find amazing is that if you calculate efficiency according to Amdahl's Law it is exactly 95.5% both when going from 1->2 and 1->4 cores. Matching is on the second decimal!!! Amazing.
Very interesting observation. And in line to Amdahl's fit I did for Komodo 9.3 using Andreas' results:
http://www.talkchess.com/forum/viewtopi ... 4&start=45
Andreas posts very important, hardcore results which use huge CPU time.

From your observation, the predictions would be:

1 --> 8 threads: 6.1
1 --> 16 threads: 9.55
1 --> 32 threads: 13.4

That is somewhat higher than the old YBW numbers.