Some hyperthreading results

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
shrapnel
Posts: 1245
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: Some hyperthreading results

Post by shrapnel » Tue Sep 13, 2016 3:09 pm

I was going to write that using 16 Threads on my overclocked 8-Core machine doesn't help, so HT gives no benefit.
But seeing your post above, I'll try with 12 or 14 Threads and see...
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

User avatar
Laskos
Posts: 9473
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Some hyperthreading results

Post by Laskos » Wed Sep 14, 2016 4:38 am

bob wrote:
mjlef wrote:Kai,

How can both hyperthreading on and off be tested? Was this two identical machines or on 1 machine? I ask since I see a different nps with hyperthreading being off and using the half the thread, than hyperthreading on using half the threads. So I do not see how it can be tested on one machine.

If two identical machines could be setup and connected, one with hyperthreading on and one off, then that would rule out the issue. But nodes per second does not seem to be enough.

Mark
You can (a) go into the BIOS and disable hyper threading (on most machines, excluding apple) or (b) just run with N/2 threads and any recent/decent O/S will schedule each thread on a physical core.

For testing, you can easily play 4 threads vs 8 threads on one machine. 4 threads will run on 4 physical cores, 8 threads will use all 8 logical processors (2 per core). If you are paranoid, you can resort to thread affinity.
Well, it seems I was not paranoid enough. I too trusted the O/S (Windows 8.1 in my case), and just performed some careless checks, seeing a negligible 1% difference between setting or no affinity. But remembering that Mark saw significant NPS difference, yesterday I investigated a bit more, and discovered that I ruined the whole test by not being careful enough, and trusting the O/S. The difference in speed by setting affinity can be as large as 20%. Carefully setting required affinities, the correct results on my desktop (4 core i7 4790, 3.6GHz, Turbo Boost OFF, Dynamic Frequency OFF) are:

NPS improvement 4 -> 8 threads, 10 seconds per position, 150 positions:

Lazy:
  • Stockfish dev: 1.24
    Komodo 10.1: 1.28
    Andscacs 0.872: 1.19
YBW:
  • Houdini 4: 1.04
    Crafty 25.01: 1.17
Fritz Benchmark gives 1.20 factor improvement 4 -> 8 threads.

Direct matches of 8t vs 4t and 6t vs 4t shows that 6 threads on 4 physical cores gives the biggest gain:
Stockfish dev games at 10''+0.1'' :
  • Score of SF 8 threads vs SF 4 threads: 416 - 387 - 1197 [0.507] 2000
    ELO difference: 5.04 +/- 8.64

    Score of SF 6 threads vs SF 4 threads: 425 - 356 - 1219 [0.517] 2000
    ELO difference: 11.99 +/- 8.50
Hyperthreading with Lazy SF gives a beyond error margins advantage only for 6 threads on 4 physical cores. Similar results with Lazy Komodo. No benefit from hyperthreading strength-wise with YBW engines Houdini and Crafty.

Sorry for my erroneous first results, and thanks to Mark for pointing to a possible large problem with O/S allocation of threads.

Tdunbug
Posts: 47
Joined: Sun Mar 06, 2016 11:46 pm

Re: Some hyperthreading results

Post by Tdunbug » Wed Sep 14, 2016 11:58 am

So I am correct to assume from these results that possibly using 2/3 of the thread count on Lazy smp engines is ideal?

User avatar
Laskos
Posts: 9473
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Some hyperthreading results

Post by Laskos » Wed Sep 14, 2016 1:00 pm

Tdunbug wrote:So I am correct to assume from these results that possibly using 2/3 of the thread count on Lazy smp engines is ideal?
Yes, I guess so (with Stockfish at least). Although for very many cores and NUMA it might be different, say not using hyperthreads at all might be better. If you are using HT ON in BIOS and only the physical cores, make sure you set the affinity right.

shrapnel
Posts: 1245
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: Some hyperthreading results

Post by shrapnel » Wed Sep 14, 2016 2:59 pm

Laskos wrote:Yes, I guess so (with Stockfish at least). Although for very many cores and NUMA it might be different, say not using hyperthreads at all might be better. If you are using HT ON in BIOS and only the physical cores, make sure you set the affinity right.
So how many Threads to use with a 8-Core Processor ?
12, 14 or something else ?
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

lkaufman
Posts: 3733
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Some hyperthreading results

Post by lkaufman » Wed Sep 14, 2016 3:24 pm

Laskos wrote:
Tdunbug wrote:So I am correct to assume from these results that possibly using 2/3 of the thread count on Lazy smp engines is ideal?
Yes, I guess so (with Stockfish at least). Although for very many cores and NUMA it might be different, say not using hyperthreads at all might be better. If you are using HT ON in BIOS and only the physical cores, make sure you set the affinity right.
Wouldn't 3/4 be correct rather than 2/3 based on your results? I realize that the margin of error is probably larger than the difference between 3/4 and 2/3. Is there any theoretical reason why the ideal fraction should go up or down as cores increase?
Komodo rules!

User avatar
Laskos
Posts: 9473
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Some hyperthreading results

Post by Laskos » Wed Sep 14, 2016 3:47 pm

lkaufman wrote:
Laskos wrote:
Tdunbug wrote:So I am correct to assume from these results that possibly using 2/3 of the thread count on Lazy smp engines is ideal?
Yes, I guess so (with Stockfish at least). Although for very many cores and NUMA it might be different, say not using hyperthreads at all might be better. If you are using HT ON in BIOS and only the physical cores, make sure you set the affinity right.
Wouldn't 3/4 be correct rather than 2/3 based on your results? I realize that the margin of error is probably larger than the difference between 3/4 and 2/3. Is there any theoretical reason why the ideal fraction should go up or down as cores increase?
Yes, it will go down from my 3/4 for more cores to 1/2. The scaling 4 --> 8 (physical) is better than 16 --> 32 (physical), in other words, the scaling is below linear, and there is some point with many cores where a "weak" hyperthread doesn't add more than it harms due to SMP overhead. I presumed 2/3 for the range of roughly 8-16 physical cores.

User avatar
Laskos
Posts: 9473
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Some hyperthreading results

Post by Laskos » Wed Sep 14, 2016 3:52 pm

shrapnel wrote:
Laskos wrote:Yes, I guess so (with Stockfish at least). Although for very many cores and NUMA it might be different, say not using hyperthreads at all might be better. If you are using HT ON in BIOS and only the physical cores, make sure you set the affinity right.
So how many Threads to use with a 8-Core Processor ?
12, 14 or something else ?
I would guess 12 or 10. Depends on particular processor type and memory too.

lkaufman
Posts: 3733
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Some hyperthreading results

Post by lkaufman » Wed Sep 14, 2016 6:34 pm

Laskos wrote:
lkaufman wrote:
Laskos wrote:
Tdunbug wrote:So I am correct to assume from these results that possibly using 2/3 of the thread count on Lazy smp engines is ideal?
Yes, I guess so (with Stockfish at least). Although for very many cores and NUMA it might be different, say not using hyperthreads at all might be better. If you are using HT ON in BIOS and only the physical cores, make sure you set the affinity right.
Wouldn't 3/4 be correct rather than 2/3 based on your results? I realize that the margin of error is probably larger than the difference between 3/4 and 2/3. Is there any theoretical reason why the ideal fraction should go up or down as cores increase?
Yes, it will go down from my 3/4 for more cores to 1/2. The scaling 4 --> 8 (physical) is better than 16 --> 32 (physical), in other words, the scaling is below linear, and there is some point with many cores where a "weak" hyperthread doesn't add more than it harms due to SMP overhead. I presumed 2/3 for the range of roughly 8-16 physical cores.
That sounds correct to me. So probably there is no reason for me to use hyperthreading on 24 core machine. Should I use "affinities" on such a machine, and what would you guess is the likely benefit?
Komodo rules!

shrapnel
Posts: 1245
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: Some hyperthreading results

Post by shrapnel » Wed Sep 14, 2016 6:40 pm

Laskos wrote:]I would guess 12 or 10. Depends on particular processor type and memory too.
OK, thanks.
I suppose by memory you mean amount of Hash used ?
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

Post Reply