Threads-Test

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

fastgm
Posts: 818
Joined: Mon Aug 19, 2013 6:57 pm

Threads-Test

Post by fastgm »

Threads-Test

Test conditions:

Code: Select all

CPU:          Dual AMD Opteron 6376 (2x 16 Cores)
OS:           Windows 7 Professional 64-Bit
Tool:         Cutechess-Cli
Engine:       Stockfish DD 64 SSE4.2
Hash-Table:   128 MB    
Openings:     fq1500.pgn - 1500 different opening positions, changing colors (3000 games per match)
Time control: 15+0.05, 60+0.05 and 240+0.05 seconds per game
Per time control three matches: 1 versus 2 threads, 1 versus 4 threads and 1 versus 8 threads

Image

In the following PDF document you can find more detailed information:
http://www.fastgm.de/schach/Threads-Test-15-60-240.pdf
kranium
Posts: 2129
Joined: Thu May 29, 2008 10:43 am

Re: Threads-Test

Post by kranium »

cool, nice work
thx Andreas

it's nice to see some real data on this subject
is it fair to conclude:

1) extra threads help more in short TCs than in LTC?

2) as threads increase, it appears the graph should flatten out eventually...
i.e. at one point, adding extra cores/threads simply won't increase ELO in any meaningful way (law of diminishing returns)
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Threads-Test

Post by Laskos »

kranium wrote:cool, nice work
thx Andreas

it's nice to see some real data on this subject
is it fair to conclude:

1) extra threads help more in short TCs than in LTC?

2) as threads increase, it appears the graph should flatten out eventually...
i.e. at one point, adding extra cores/threads simply won't increase ELO in any meaningful way (law of diminishing returns)
Seems that 1) is the law of diminishing returns with time control and 2) it's unclear if it flattens (Amdahl's Law) or goes logarithmically (modified Gustafson's law).
Thanks for the test, Andreas.
Vinvin
Posts: 5228
Joined: Thu Mar 09, 2006 9:40 am
Full name: Vincent Lejeune

Re: Threads-Test

Post by Vinvin »

One thing is clear : when one lengthen the time control, the draw ratio rise.

Thank you !
Very interesting !
User avatar
lucasart
Posts: 3232
Joined: Mon May 31, 2010 1:29 pm
Full name: lucasart

Re: Threads-Test

Post by lucasart »

kranium wrote:cool, nice work
thx Andreas

it's nice to see some real data on this subject
is it fair to conclude:

1) extra threads help more in short TCs than in LTC?

2) as threads increase, it appears the graph should flatten out eventually...
i.e. at one point, adding extra cores/threads simply won't increase ELO in any meaningful way (law of diminishing returns)
1) Depends how you define "help". Maybe they help the same way, in terms of average time to depth. But we are simply looking at the normal diminishing returns, in elo terms, of increasing NPS on a single core test. So an effect completely unrelated to SMP. We need to measure average TTD to answer that question, not ELO.

2) Certainly. It's the compounding effect of (at least) two well known effects:
* SMP performance (expressed in terms of in 1/TTD=f(threads)) is less than linear with the number of threads. In theor 1/TTD should follow an Ahmdal's law. I remember Miguel made some pretty good fit with that.
* Even on a single threaded program, increasing NPS is less than linear in ELO terms. First time you double NPS, maybe you get +100 elo, second time, maybe only 90 elo etc.

And, as always, long TC flattens everything, because draw rate increases. This is inevitable. In a theoretical sense, when tc --> infinity, you get perfect play, hence 100% draws and 0 ELO difference.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.
fastgm
Posts: 818
Joined: Mon Aug 19, 2013 6:57 pm

Threads-Test - 2 vs. 4 and 4 vs. 8 threads

Post by fastgm »

In addition the results of 2 vs. 4 and 4 vs. 8 threads.

In the following PDF document you can find more detailed information:
http://www.fastgm.de/schach/Threads-Test-2-4-8.pdf

The PDF as image:
Image
fastgm
Posts: 818
Joined: Mon Aug 19, 2013 6:57 pm

Graph 1 vs. 2, 2 vs. 4 and 4 vs. 8 threads

Post by fastgm »

Image
fastgm
Posts: 818
Joined: Mon Aug 19, 2013 6:57 pm

Graph 1 vs. 2, 2 vs. 4, 4 vs. 8 and 8 vs. 16 threads

Post by fastgm »

New: 8 vs. 16 threads

Image

Image