Page 1 of 1

Threads-Test

Posted: Tue Mar 18, 2014 7:30 pm
by fastgm
Threads-Test

Test conditions:

Code: Select all

CPU:          Dual AMD Opteron 6376 (2x 16 Cores)
OS:           Windows 7 Professional 64-Bit
Tool:         Cutechess-Cli
Engine:       Stockfish DD 64 SSE4.2
Hash-Table:   128 MB    
Openings:     fq1500.pgn - 1500 different opening positions, changing colors (3000 games per match)
Time control: 15+0.05, 60+0.05 and 240+0.05 seconds per game
Per time control three matches: 1 versus 2 threads, 1 versus 4 threads and 1 versus 8 threads

Image

In the following PDF document you can find more detailed information:
http://www.fastgm.de/schach/Threads-Test-15-60-240.pdf

Re: Threads-Test

Posted: Wed Mar 19, 2014 1:13 am
by kranium
cool, nice work
thx Andreas

it's nice to see some real data on this subject
is it fair to conclude:

1) extra threads help more in short TCs than in LTC?

2) as threads increase, it appears the graph should flatten out eventually...
i.e. at one point, adding extra cores/threads simply won't increase ELO in any meaningful way (law of diminishing returns)

Re: Threads-Test

Posted: Wed Mar 19, 2014 3:03 am
by Laskos
kranium wrote:cool, nice work
thx Andreas

it's nice to see some real data on this subject
is it fair to conclude:

1) extra threads help more in short TCs than in LTC?

2) as threads increase, it appears the graph should flatten out eventually...
i.e. at one point, adding extra cores/threads simply won't increase ELO in any meaningful way (law of diminishing returns)
Seems that 1) is the law of diminishing returns with time control and 2) it's unclear if it flattens (Amdahl's Law) or goes logarithmically (modified Gustafson's law).
Thanks for the test, Andreas.

Re: Threads-Test

Posted: Sun Mar 23, 2014 11:41 pm
by Vinvin
One thing is clear : when one lengthen the time control, the draw ratio rise.

Thank you !
Very interesting !

Re: Threads-Test

Posted: Mon Mar 24, 2014 3:18 am
by lucasart
kranium wrote:cool, nice work
thx Andreas

it's nice to see some real data on this subject
is it fair to conclude:

1) extra threads help more in short TCs than in LTC?

2) as threads increase, it appears the graph should flatten out eventually...
i.e. at one point, adding extra cores/threads simply won't increase ELO in any meaningful way (law of diminishing returns)
1) Depends how you define "help". Maybe they help the same way, in terms of average time to depth. But we are simply looking at the normal diminishing returns, in elo terms, of increasing NPS on a single core test. So an effect completely unrelated to SMP. We need to measure average TTD to answer that question, not ELO.

2) Certainly. It's the compounding effect of (at least) two well known effects:
* SMP performance (expressed in terms of in 1/TTD=f(threads)) is less than linear with the number of threads. In theor 1/TTD should follow an Ahmdal's law. I remember Miguel made some pretty good fit with that.
* Even on a single threaded program, increasing NPS is less than linear in ELO terms. First time you double NPS, maybe you get +100 elo, second time, maybe only 90 elo etc.

And, as always, long TC flattens everything, because draw rate increases. This is inevitable. In a theoretical sense, when tc --> infinity, you get perfect play, hence 100% draws and 0 ELO difference.

Threads-Test - 2 vs. 4 and 4 vs. 8 threads

Posted: Wed Apr 02, 2014 6:30 pm
by fastgm
In addition the results of 2 vs. 4 and 4 vs. 8 threads.

In the following PDF document you can find more detailed information:
http://www.fastgm.de/schach/Threads-Test-2-4-8.pdf

The PDF as image:
Image

Graph 1 vs. 2, 2 vs. 4 and 4 vs. 8 threads

Posted: Fri Apr 04, 2014 12:37 pm
by fastgm
Image

Graph 1 vs. 2, 2 vs. 4, 4 vs. 8 and 8 vs. 16 threads

Posted: Tue May 06, 2014 6:45 pm
by fastgm
New: 8 vs. 16 threads

Image

Image