threads vs processes

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
bob
Posts: 20555
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

threads vs processes

Post by bob » Wed Jul 16, 2008 9:03 pm

I just converted Crafty back to posix threads since we now have a decent POSIX threads library that replaced the old linux threads package. Took under one hour for reference, to show how close the two approaches are... Performance difference is exactly zero, except that EGTB is more efficient with threads since it then uses a shared LRU buffer chain which cuts down on I/O significantly as opposed to using 4 or 8 distinct buffers, one for each process.

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 6:43 pm

Re: threads vs processes

Post by sje » Wed Jul 16, 2008 9:59 pm

Relative efficiency is rather dependent upon the underlying OS implementation. Linux treats threads as lightweight processes, so there may not be much difference there. However, other systems like OpenBSD have a higher per process overhead, so threads could be the better deal.

bob
Posts: 20555
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: threads vs processes

Post by bob » Thu Jul 17, 2008 12:23 am

sje wrote:Relative efficiency is rather dependent upon the underlying OS implementation. Linux treats threads as lightweight processes, so there may not be much difference there. However, other systems like OpenBSD have a higher per process overhead, so threads could be the better deal.
I've run on all of 'em. Since copy-on-write was implemented in every paging system I have seen in the past 5+ years, I don't see any difference, except in the overhead to create the processes. Which I do exactly once in a game, so this means nothing. And the overhead between the linux Clone() (lightweight process creation) system call and the BSD fork() is _very_ minimal as it is, since the entire virtual memory address space is not duplicated on a fork() call today. In fact only the process table/descriptor/etc stuff is copied until a page of the virtual address space is modified by either process/thread...

User avatar
Zach Wegner
Posts: 1922
Joined: Wed Mar 08, 2006 11:51 pm
Location: Earth
Contact:

Re: threads vs processes

Post by Zach Wegner » Thu Jul 17, 2008 1:47 am

But isn't the main difference the fact that you can get rid of the BOARD * with processes? I wouldn't expect any sort of speed difference otherwise.

bob
Posts: 20555
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: threads vs processes

Post by bob » Thu Jul 17, 2008 5:23 pm

Zach Wegner wrote:But isn't the main difference the fact that you can get rid of the BOARD * with processes? I wouldn't expect any sort of speed difference otherwise.
I can't, no. I still split inside a process to get other processes to help, so I have to carry the board pointer around everywhere. When I split, I allocate split blocks for each process/thread that will be working to help, which needs the pointer to the (in the case of Crafty) TREE structure which contains the board and anything else that a thread might update when searching.

User avatar
Graham Banks
Posts: 33145
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Re: threads vs processes

Post by Graham Banks » Thu Jul 17, 2008 7:39 pm

Loop came out at one stage as being able to use either threads or processes.
It performed better using threads.
My email addresses:
gbanksnz at gmail.com
gbanksnz at yahoo.co.nz

User avatar
beachknight
Posts: 3533
Joined: Tue Jan 09, 2007 7:33 pm
Location: Antalya, Turkey
Contact:

Re: threads vs processes

Post by beachknight » Thu Jul 17, 2008 8:17 pm

Do I understand right? A new release of Crafty is on the sight?

best,
hi, merhaba, hallo HT

bob
Posts: 20555
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: threads vs processes

Post by bob » Thu Jul 17, 2008 8:40 pm

Graham Banks wrote:Loop came out at one stage as being able to use either threads or processes.
It performed better using threads.
I can't imagine why. These are two different ways to implement the same exact algorithm. As I mentioned, it took under an hour to convert back from processes to threads. It took about the same amount of time to convert to processes a couple of years ago...

bob
Posts: 20555
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: threads vs processes

Post by bob » Thu Jul 17, 2008 8:41 pm

beachknight wrote:Do I understand right? A new release of Crafty is on the sight?

best,
Once we verify that the windows version works, yes, although it is not very different from the released 22.1 except for this thread issue that should clean up all the issues with "smpnice=1" under windoze.

LoopList

Re: threads vs processes

Post by LoopList » Fri Jul 18, 2008 8:08 am

In order to get rid of the "board" pointer I implemented parallel processes. The engine became 10% faster in 32-bit mode and even 5% faster in 64-bit mode. But starting and terminating multiple slave processes is quite more complicated and ugly. Therefore the thread-powered engine with the additional "board" pointer was easier to develop and cleaner to tune. Furthermore, by using the "board" pointer I had the possibility to organise the "board"-class with private and public members. Passing "board" via const pointer enabled further security enhancements!

And DTS-like techniques are much too comlicated to be based on processes - communication overhead will increase compared to multiple threads. At least my research had shown me these results, perhaps I am wrong...

Fritz (Loop)

Post Reply