Stockfish 8 - Double time control vs. 2 threads

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
fastgm
Posts: 415
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Stockfish 8 - Double time control vs. 2 threads

Post by fastgm » Tue Nov 15, 2016 7:26 pm

Stockfish 8
1 thread (5 minutes + 3 seconds) vs. 2 threads (2.5 minutes + 1.5 seconds)

Code: Select all

    Program                                Elo    +   -   Games   Score   Av.Op.  Draws
 ----------------------------------------------------------------------------------------
  1 Stockfish 8 - 1 thread    5' + 3"    : 3003   5   5   3000    51.0 %   2997   85.2 %
  2 Stockfish 8 - 2 threads 2.5' + 1.5"  : 2997   5   5   3000    49.0 %   3003   85.2 %


Av.Op. Elo = 3000

Result     : 1529.0/3000 (+251,=2556,-193)
Perf.      : 51.0 %
Margins    :
 68 %      : (+  0.3,-  0.3 %) -> [ 50.6, 51.3 %]
 95 %      : (+  0.7,-  0.7 %) -> [ 50.3, 51.7 %]
 99.7 %    : (+  1.0,-  1.0 %) -> [ 49.9, 52.0 %]

Elo        : 3007
Margins    :
 68 %      : (+  2,-  2) -> [3004,3009]
 95 %      : (+  5,-  5) -> [3002,3012]
 99.7 %    : (+  7,-  7) -> [3000,3014]


Games        :   3000 (finished)

White Wins   :    341 (11.4 %)
Black Wins   :    103 ( 3.4 %)
Draws        :   2556 (85.2 %)

White Perf.  : 54.0 %
Black Perf.  : 46.0 %


Individual statistics:

1 Stockfish 8 - 1 thread    5' + 3"    : 3003  3000 (+251,=2556,-193), 51.0 %
2 Stockfish 8 - 2 threads 2.5' + 1.5"  : 2997  3000 (+193,=2556,-251), 49.0 %

ernest
Posts: 1864
Joined: Wed Mar 08, 2006 7:30 pm

Re: Stockfish 8 - Double time control vs. 2 threads

Post by ernest » Wed Nov 16, 2016 7:51 pm

Very interesting, Andreas, thanks !!!

User avatar
Laskos
Posts: 9541
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Stockfish 8 - Double time control vs. 2 threads

Post by Laskos » Wed Nov 16, 2016 8:34 pm

Interesting as usual, thanks! From your previous thread we already know that doubling time at this time control is worth about 90 ELO points. Here the difference is 6+/-5 ELO points. Means 0.067 doublings. This means that the effective speed-up 1 -> 2 threads is 2^(1-0.067) ~ 1.91. For the first time it is measured so precisely for Lazy SMP engine. With YBW SMP like in Crafty time-to-depth can be used.

ernest
Posts: 1864
Joined: Wed Mar 08, 2006 7:30 pm

Re: Stockfish 8 - Double time control vs. 2 threads

Post by ernest » Wed Nov 16, 2016 10:22 pm

Laskos wrote:Interesting as usual, thanks! From your previous thread we already know that doubling time at this time control is worth about 90 ELO points. Here the difference is 6+/-5 ELO points. Means 0.067 doublings. This means that the effective speed-up 1 -> 2 threads is 2^(1-0.067) ~ 1.91. For the first time it is measured so precisely for Lazy SMP engine. With YBW SMP like in Crafty time-to-depth can be used.
These numbers, 90 Elo and 1.91, seem high.
Is it because they result from games engine vs same engine ?

User avatar
Laskos
Posts: 9541
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Stockfish 8 - Double time control vs. 2 threads

Post by Laskos » Thu Nov 17, 2016 8:06 am

ernest wrote:
Laskos wrote:Interesting as usual, thanks! From your previous thread we already know that doubling time at this time control is worth about 90 ELO points. Here the difference is 6+/-5 ELO points. Means 0.067 doublings. This means that the effective speed-up 1 -> 2 threads is 2^(1-0.067) ~ 1.91. For the first time it is measured so precisely for Lazy SMP engine. With YBW SMP like in Crafty time-to-depth can be used.
These numbers, 90 Elo and 1.91, seem high.
Is it because they result from games engine vs same engine ?
The results of Komodo 9.3 self-play, which I would think also apply to Srockfish 8 self-play, were presented by Andreas here:
http://www.talkchess.com/forum/viewtopi ... 84&start=0
At this time control, about 93 ELO points for doublig of Komodo, hence 1.91 effective speed-up 1->2 threads for Stockfish 8. It's not unexpected, 1->2 threads scale very well, with Lazy or YBW SMP, but Andreas' measurement is the most precise one up to date for Lazy SMP. It can only be measured through games, not NPS, time-to-depth and other easier methods.

Werewolf
Posts: 1198
Joined: Thu Sep 18, 2008 8:24 pm

Re: Stockfish 8 - Double time control vs. 2 threads

Post by Werewolf » Fri Nov 18, 2016 11:05 am

Laskos wrote:Interesting as usual, thanks! From your previous thread we already know that doubling time at this time control is worth about 90 ELO points. Here the difference is 6+/-5 ELO points. Means 0.067 doublings. This means that the effective speed-up 1 -> 2 threads is 2^(1-0.067) ~ 1.91. For the first time it is measured so precisely for Lazy SMP engine. With YBW SMP like in Crafty time-to-depth can be used.

Wow. That's a big upgrade from the standard a few years ago when Rybka ruled: "A doubling of cores is 1.7x faster"

This may have huge implications for hyper threading and monstrously big machines with high core counts.

fastgm
Posts: 415
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Stockfish 8 - Quadruple time control vs. 4 threads

Post by fastgm » Tue Nov 22, 2016 5:36 pm

Stockfish 8
1 thread (5 minutes + 3 seconds) vs. 4 threads (75 seconds + 0.75 seconds)

Code: Select all

    Program                                Elo    +   -   Games   Score   Av.Op.  Draws
 ----------------------------------------------------------------------------------------
  1 Stockfish 8 - 1 thread   5' + 3"     : 3009   5   5   3000    52.6 %   2991   83.9 %
  2 Stockfish 8 - 4 threads 75" + 0.75"  : 2991   5   5   3000    47.4 %   3009   83.9 %


Wins   = 320
Draws  = 2516
Losses = 164
Av.Op. Elo = 3000

Result     : 1578.0/3000 (+320,=2516,-164)
Perf.      : 52.6 %
Margins    :
 68 %      : (+  0.4,-  0.4 %) -> [ 52.2, 53.0 %]
 95 %      : (+  0.7,-  0.7 %) -> [ 51.9, 53.3 %]
 99.7 %    : (+  1.1,-  1.1 %) -> [ 51.5, 53.7 %]

Elo        : 3018
Margins    :
 68 %      : (+  3,-  3) -> [3016,3021]
 95 %      : (+  5,-  5) -> [3013,3023]
 99.7 %    : (+  8,-  7) -> [3011,3026]


Games        :   3000 (finished)

White Wins   :    362 (12.1 %)
Black Wins   :    122 ( 4.1 %)
Draws        :   2516 (83.9 %)

White Perf.  : 54.0 %
Black Perf.  : 46.0 %


Individual statistics:

1 Stockfish 8 - 1 thread   5' + 3"     : 3009  3000 (+320,=2516,-164), 52.6 %
2 Stockfish 8 - 4 threads 75" + 0.75"  : 2991  3000 (+164,=2516,-320), 47.4 %


Engine                                Depth       Time   Games     Moves  Average
---------------------------------------------------------------------------------
Stockfish 8 - 1 thread   5' + 3"      28.60  349:27:58    3000    188602    6.67
Stockfish 8 - 4 threads 75" + 0.75"   26.32   90:17:50    3000    188629    1.72

User avatar
Laskos
Posts: 9541
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Stockfish 8 - Quadruple time control vs. 4 threads

Post by Laskos » Tue Nov 22, 2016 7:52 pm

fastgm wrote:Stockfish 8
1 thread (5 minutes + 3 seconds) vs. 4 threads (75 seconds + 0.75 seconds)

Code: Select all

    Program                                Elo    +   -   Games   Score   Av.Op.  Draws
 ----------------------------------------------------------------------------------------
  1 Stockfish 8 - 1 thread   5' + 3"     : 3009   5   5   3000    52.6 %   2991   83.9 %
  2 Stockfish 8 - 4 threads 75" + 0.75"  : 2991   5   5   3000    47.4 %   3009   83.9 %


Wins   = 320
Draws  = 2516
Losses = 164
Av.Op. Elo = 3000

Result     : 1578.0/3000 (+320,=2516,-164)
Perf.      : 52.6 %
Margins    :
 68 %      : (+  0.4,-  0.4 %) -> [ 52.2, 53.0 %]
 95 %      : (+  0.7,-  0.7 %) -> [ 51.9, 53.3 %]
 99.7 %    : (+  1.1,-  1.1 %) -> [ 51.5, 53.7 %]

Elo        : 3018
Margins    :
 68 %      : (+  3,-  3) -> [3016,3021]
 95 %      : (+  5,-  5) -> [3013,3023]
 99.7 %    : (+  8,-  7) -> [3011,3026]


Games        :   3000 (finished)

White Wins   :    362 (12.1 %)
Black Wins   :    122 ( 4.1 %)
Draws        :   2516 (83.9 %)

White Perf.  : 54.0 %
Black Perf.  : 46.0 %


Individual statistics:

1 Stockfish 8 - 1 thread   5' + 3"     : 3009  3000 (+320,=2516,-164), 52.6 %
2 Stockfish 8 - 4 threads 75" + 0.75"  : 2991  3000 (+164,=2516,-320), 47.4 %


Engine                                Depth       Time   Games     Moves  Average
---------------------------------------------------------------------------------
Stockfish 8 - 1 thread   5' + 3"      28.60  349:27:58    3000    188602    6.67
Stockfish 8 - 4 threads 75" + 0.75"   26.32   90:17:50    3000    188629    1.72
Another important post, thanks. Factor of 4 in time is 2 doublings, from your data in another thread, the first one is worth 101 ELO points, the second 93 ELO points. In total 194 ELO points for quadrupling. The difference in this test is 18 ELO points, or 18/194 ~ 0.0928 quadruplings. The speed-up 1 --> 4 threads is 4^(1-0.0928) ~ 3.52. Again, the most precise measurement for Lazy SMP.

So,

1 --> 2 threads: 1.91
1 --> 4 threads: 3.52

Milos
Posts: 3387
Joined: Wed Nov 25, 2009 12:47 am

Re: Stockfish 8 - Quadruple time control vs. 4 threads

Post by Milos » Wed Nov 23, 2016 3:05 am

Laskos wrote: So,

1 --> 2 threads: 1.91
1 --> 4 threads: 3.52
What I find amazing is that if you calculate efficiency according to Amdahl's Law it is exactly 95.5% both when going from 1->2 and 1->4 cores. Matching is on the second decimal!!! Amazing.

User avatar
Laskos
Posts: 9541
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Stockfish 8 - Quadruple time control vs. 4 threads

Post by Laskos » Wed Nov 23, 2016 3:11 pm

Milos wrote:
Laskos wrote: So,

1 --> 2 threads: 1.91
1 --> 4 threads: 3.52
What I find amazing is that if you calculate efficiency according to Amdahl's Law it is exactly 95.5% both when going from 1->2 and 1->4 cores. Matching is on the second decimal!!! Amazing.
Very interesting observation. And in line to Amdahl's fit I did for Komodo 9.3 using Andreas' results:
http://www.talkchess.com/forum/viewtopi ... 4&start=45
Andreas posts very important, hardcore results which use huge CPU time.

From your observation, the predictions would be:

1 --> 8 threads: 6.1
1 --> 16 threads: 9.55
1 --> 32 threads: 13.4

That is somewhat higher than the old YBW numbers.

Post Reply