Threads-Test

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
fastgm
Posts: 409
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Threads-Test

Post by fastgm » Tue Mar 18, 2014 7:30 pm

Threads-Test

Test conditions:

Code: Select all

CPU:          Dual AMD Opteron 6376 (2x 16 Cores)
OS:           Windows 7 Professional 64-Bit
Tool:         Cutechess-Cli
Engine:       Stockfish DD 64 SSE4.2
Hash-Table:   128 MB    
Openings:     fq1500.pgn - 1500 different opening positions, changing colors (3000 games per match)
Time control: 15+0.05, 60+0.05 and 240+0.05 seconds per game
Per time control three matches: 1 versus 2 threads, 1 versus 4 threads and 1 versus 8 threads

Image

In the following PDF document you can find more detailed information:
http://www.fastgm.de/schach/Threads-Test-15-60-240.pdf

User avatar
kranium
Posts: 1824
Joined: Thu May 29, 2008 8:43 am

Re: Threads-Test

Post by kranium » Wed Mar 19, 2014 1:13 am

cool, nice work
thx Andreas

it's nice to see some real data on this subject
is it fair to conclude:

1) extra threads help more in short TCs than in LTC?

2) as threads increase, it appears the graph should flatten out eventually...
i.e. at one point, adding extra cores/threads simply won't increase ELO in any meaningful way (law of diminishing returns)

User avatar
Laskos
Posts: 9446
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Threads-Test

Post by Laskos » Wed Mar 19, 2014 3:03 am

kranium wrote:cool, nice work
thx Andreas

it's nice to see some real data on this subject
is it fair to conclude:

1) extra threads help more in short TCs than in LTC?

2) as threads increase, it appears the graph should flatten out eventually...
i.e. at one point, adding extra cores/threads simply won't increase ELO in any meaningful way (law of diminishing returns)
Seems that 1) is the law of diminishing returns with time control and 2) it's unclear if it flattens (Amdahl's Law) or goes logarithmically (modified Gustafson's law).
Thanks for the test, Andreas.

Vinvin
Posts: 4377
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: Threads-Test

Post by Vinvin » Sun Mar 23, 2014 11:41 pm

One thing is clear : when one lengthen the time control, the draw ratio rise.

Thank you !
Very interesting !

User avatar
lucasart
Posts: 3040
Joined: Mon May 31, 2010 11:29 am
Full name: lucasart
Contact:

Re: Threads-Test

Post by lucasart » Mon Mar 24, 2014 3:18 am

kranium wrote:cool, nice work
thx Andreas

it's nice to see some real data on this subject
is it fair to conclude:

1) extra threads help more in short TCs than in LTC?

2) as threads increase, it appears the graph should flatten out eventually...
i.e. at one point, adding extra cores/threads simply won't increase ELO in any meaningful way (law of diminishing returns)
1) Depends how you define "help". Maybe they help the same way, in terms of average time to depth. But we are simply looking at the normal diminishing returns, in elo terms, of increasing NPS on a single core test. So an effect completely unrelated to SMP. We need to measure average TTD to answer that question, not ELO.

2) Certainly. It's the compounding effect of (at least) two well known effects:
* SMP performance (expressed in terms of in 1/TTD=f(threads)) is less than linear with the number of threads. In theor 1/TTD should follow an Ahmdal's law. I remember Miguel made some pretty good fit with that.
* Even on a single threaded program, increasing NPS is less than linear in ELO terms. First time you double NPS, maybe you get +100 elo, second time, maybe only 90 elo etc.

And, as always, long TC flattens everything, because draw rate increases. This is inevitable. In a theoretical sense, when tc --> infinity, you get perfect play, hence 100% draws and 0 ELO difference.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.

fastgm
Posts: 409
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Threads-Test - 2 vs. 4 and 4 vs. 8 threads

Post by fastgm » Wed Apr 02, 2014 6:30 pm

In addition the results of 2 vs. 4 and 4 vs. 8 threads.

In the following PDF document you can find more detailed information:
http://www.fastgm.de/schach/Threads-Test-2-4-8.pdf

The PDF as image:
Image

fastgm
Posts: 409
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Graph 1 vs. 2, 2 vs. 4 and 4 vs. 8 threads

Post by fastgm » Fri Apr 04, 2014 12:37 pm

Image

fastgm
Posts: 409
Joined: Mon Aug 19, 2013 4:57 pm
Contact:

Graph 1 vs. 2, 2 vs. 4, 4 vs. 8 and 8 vs. 16 threads

Post by fastgm » Tue May 06, 2014 6:45 pm

New: 8 vs. 16 threads

Image

Image

Post Reply