pthread weirdness

[CRoberson]

Pradu,

It has to do with threads being asynchronus. In normal programing,
the value in a memory location that a pointer points to is predictable
because the programming is synchronized (the calling function waits on
the called function to return). With threads, the calling function continues
and may iterate (in James' case) more than once before the thread can get started.

[Pradu]

Pradu,

It has to do with threads being asynchronus. In normal programing,
the value in a memory location that a pointer points to is predictable
because the programming is synchronized (the calling function waits on
the called function to return). With threads, the calling function continues
and may iterate (in James' case) more than once before the thread can get started.

I guess so, but when I was learning about pthreads and win32 threads all the example code look just the same. For example these websites:
http://www.llnl.gov/computing/tutorials ... #Compiling
or
http://www.mhpcc.edu/training/workshop2 ... /MAIN.html
http://math.arizona.edu/~swig/documentation/pthreads/
http://www.yolinux.com/TUTORIALS/LinuxT ... reads.html

Also I think the create thread function must return before another thread will be created in this loop. Here's an API reference:
http://cs.pub.ro/~apc/2003/resources/pt ... ers-14.htm

"The pthread_create() function creates a thread with the specified attributes and runs the C function start_routine in the thread with the single pointer argument specified. The new thread may, but will not always, begin running before pthread_create() returns. If pthread_create() completes successfully, the Pthread handle is stored in the contents of the location referred to by thread."

pthread_create() returns 0 if the thread starts normally and error codes if not. So I really don't see how the index can change before the thread is created. I'm not saying that either Prof. Hyatt or the websites are wrong or anything as I'm rather quite new to the field of threads and wouldn't know anyways. I just hoped that Prof. Hyatt could post some example code to show the right way of starting many threads with pthreads or win32 threads.

[sje]

What happens is that the newly created thread might not start until after the main program goes through more loop iterations. When the newly created thread eventually starts, it only then dereferences its input pointer -- and gets the wrong index value.

[Alessandro Scotti]

Hi Pradu,
I've randomly clicked on a couple of the links you posted and the situation there is different.
First, there are usually different variables passed to each thread. In our case, all the threads were given the address of the same variable which, for the purpose of assigning a unique id to each thread, is not correct.
Second and most important, all the examples I have seen wait for the threads to terminate before going out of the function (they "join" the threads). This basically means that local variables stay alive for the entire lifetime of the thread, so it's safe to reference them.
However, that rarely happens in practice and you want the threads to be free and go whenever they like without too many bounds!
My favorite approach is to create a thread_params structure that holds all of the thread initialization stuff, be it a single integer or a whole bunch of variables and allocate one instance for each thread. Then, it's the thread itself that deallocates it if/when it doesn't need it anymore.
Re: specifically how the bug works let's take for example the moment the first thread is created. You call pthread_create and as you noted the function does not return until the thread has been created. However, at this point you have now two active threads, the "main" thread and the thread you have just created. The thread being created means that it CAN run, not that it is running. You mustn't assume anything with respect to thread scheduling. So it happens that the main thread is resumed and let's say it terminates the loop and exits. Only now the child thread is activated and it attempts to reference a variable that does not exist anymore thru the (now invalid) pointer that it received at creation time... ops!

[Pradu]

Hi Pradu,
I've randomly clicked on a couple of the links you posted and the situation there is different.
First, there are usually different variables passed to each thread. In our case, all the threads were given the address of the same variable which, for the purpose of assigning a unique id to each thread, is not correct.

Oh my mistake, I thought the problem you guys were talking about was with &search_pool.t, but you guys were talking about &i. So to correct the posted code here I guess this would be right:

Code: Select all

void InitSearchThreads() {

   printf("initializing search threads...\n");
   for &#40;int i=1;i<NUM_PROCESSORS;i++) &#123;
      search_pool&#91;i&#93;.available=0;
      search_pool&#91;i&#93;.stop=0;
   &#125;





#ifdef EXE_64_BITS
U64 i;
#else
int i;
#endif
   for &#40;i=1;i<NUM_PROCESSORS;i++) &#123;
      printf&#40;"\t creating thread %d...\n",i&#41;;
      if &#40;pthread_create&#40;&search_pool&#91;i&#93;.t,NULL,InitSearchThread,&#40;void *&#41;i&#41;) &#123;
         printf&#40;"ERROR intializing thread %d.\n",i&#41;;
         exit&#40;1&#41;;
      &#125;
   &#125;   
   printf&#40;"search threads initialized.\n");
&#125;

void StopSearchThreads&#40;) &#123;

   for &#40;int i=1;i<NUM_PROCESSORS;i++) &#123;
      search_pool&#91;i&#93;.stop=1;
      pthread_join&#40;search_pool&#91;i&#93;.t,NULL&#41;;
   &#125;
      
&#125;

So if i increments before the thread starts, the value passed to the thread will also be wrong.

Second and most important, all the examples I have seen wait for the threads to terminate before going out of the function (they "join" the threads). This basically means that local variables stay alive for the entire lifetime of the thread, so it's safe to reference them.
However, that rarely happens in practice and you want the threads to be free and go whenever they like without too many bounds!
My favorite approach is to create a thread_params structure that holds all of the thread initialization stuff, be it a single integer or a whole bunch of variables and allocate one instance for each thread. Then, it's the thread itself that deallocates it if/when it doesn't need it anymore.

Yes this is what I do too malloc a structure and let the thread free it.

Re: specifically how the bug works let's take for example the moment the first thread is created. You call pthread_create and as you noted the function does not return until the thread has been created. However, at this point you have now two active threads, the "main" thread and the thread you have just created. The thread being created means that it CAN run, not that it is running. You mustn't assume anything with respect to thread scheduling. So it happens that the main thread is resumed and let's say it terminates the loop and exits. Only now the child thread is activated and it attempts to reference a variable that does not exist anymore thru the (now invalid) pointer that it received at creation time... ops!

Thanks Alessandro, Steven, Charles! I think I understand it now. This threading stuff can be tricky.

[bob]

I notice you have a loop iterating on i, and you are passing the _address_ of I to the new thread. By the time that thread executes, i will probably be at limit+1 which is almost certainly not what you want. Don't pass the address of that, pass the actual value. Note that the create function requires a pointer, but you can recast the value "i" to a (void *) and then in the new thread re-cast it back to an int. Then you are no longer passing a pointer to a value that will be changed many times by the time the thread has been created and gets scheduled.

I do something similar to James but I'm not quite understanding the above. Can you give a quick example of why passing the pointer value would be wrong? Also I guess you have to worry about the type of int as for 64-bits exe I guess pointers are 64-bits.
http://www.opengroup.org/pubs/online/79 ... reate.html

Also make absolutely certain you understand the difference between these:

volatile int X;
volatile int *X;
int * volatile X;
volatile int * volatile X;

they are absolutely not the same thing and you have to be sure that anything that can be changed in another thread is declared volatile or compiler optimization will kill you.

OK. In the parent process, you have a variable that is initially assigned the value "1"; You then create a thread and pass it the address of this variable. But that thread won't execute instantly. Meanwhile the parent increments the variable to 2, and passes that to the second newly created thread. If both now execute, they will both see a value of "2" since that is the value in memory and you passed them a pointer to that variable.

But in reality, the parent loop will likely complete before the first thread gets scheduled by the process scheduler, and every thread sees the same memory address with the same content, which is not what you intended...

[bob]

Pradu,

It has to do with threads being asynchronus. In normal programing,
the value in a memory location that a pointer points to is predictable
because the programming is synchronized (the calling function waits on
the called function to return). With threads, the calling function continues
and may iterate (in James' case) more than once before the thread can get started.

I guess so, but when I was learning about pthreads and win32 threads all the example code look just the same. For example these websites:
http://www.llnl.gov/computing/tutorials ... #Compiling
or
http://www.mhpcc.edu/training/workshop2 ... /MAIN.html
http://math.arizona.edu/~swig/documentation/pthreads/
http://www.yolinux.com/TUTORIALS/LinuxT ... reads.html

Also I think the create thread function must return before another thread will be created in this loop. Here's an API reference:
http://cs.pub.ro/~apc/2003/resources/pt ... ers-14.htm

There is a big difference between a thread being created, and it actually getting scheduled by the process scheduler and stuffed onto a CPU. That's the problem here...

"The pthread_create() function creates a thread with the specified attributes and runs the C function start_routine in the thread with the single pointer argument specified. The new thread may, but will not always, begin running before pthread_create() returns. If pthread_create() completes successfully, the Pthread handle is stored in the contents of the location referred to by thread."

pthread_create() returns 0 if the thread starts normally and error codes if not. So I really don't see how the index can change before the thread is created. I'm not saying that either Prof. Hyatt or the websites are wrong or anything as I'm rather quite new to the field of threads and wouldn't know anyways. I just hoped that Prof. Hyatt could post some example code to show the right way of starting many threads with pthreads or win32 threads.

[Tord Romstad]

I never used mutex locks as the overhead is beyond bearable and it murders efficiency if things are locked/unlocked frequently.

I have seen you make this claim before, but I have never been able to find anything faster than pthread_mutex_t locks for Glaurung. I made another experiment with my program last night, comparing the performance using pthread_mutex_t, OSSpinLock and Crafty's x86 assembly language locks. The tests were done on an iMac Core Duo 2 GHz running Mac OS X 10.4.9. The compiler was gcc 4.0.1.

I tried several positions, and the result was the same for all of them: pthread_mutex_t was fastest, and OSSpinLock and x86 assembly locks were about 2% slower. Am I doing something wrong, or is OS X somehow very different from Linux with respect to the performance of mutex locks?

Source code available on request, if somebody wants to repeat my experiment.

Tord

[sje]

My experience with pthread style mutexes on PowerPC and x86 is that they are very fast with little overhead. However, I have only used the simplest (default) mutex options and not the more complicated (e.g., counting) style.

Of course, it's always possible to code up some pathological case that makes mutex usage look bad. The question is whether or not a specific (and useful) strategy is faulty.

[Tord Romstad]

Hello Steven,

Good to see that I am not alone in finding pthread mutexes to be fast.

My experience with pthread style mutexes on PowerPC and x86 is that they are very fast with little overhead. However, I have only used the simplest (default) mutex options and not the more complicated (e.g., counting) style.

Me too. I use pthread_mutex_lock and pthread_mutex_unlock, but nothing else.

Tord