Writing to a Text File (Thread Safe)

[Steve Maughan]

In Maverick I'm writing a log of all the input and out put to a text file.

Since Maverick uses two thread - one to read the input and one to "
think" there is a possibility it tries to write to the same file at the same time. Is this a problem? How do others get around it?

Thanks,

Steve

[MrEdCollins]

Can you toggle a switch that both threads read?

When either thread wishes to write to the file, if first checks this okay_to_write switch. If this switch is set to 1 (yes) it temporarily sets the switch to 0 (no) and goes ahead and writes to the file. It then sets the switch back to 1 after the write is complete.

If either thread checks the switch and discovers that it is set to 0, this thread now knows the other thread must have just set the switch to 0. So now this thread just waits, and keeps checking the switch until it finds that it's back to 1 again.

Once it does find it back to 1, it works as described above... now THIS thread temporarily sets the switch to 0, writes to the file, and then sets it back to 1 when complete.

[sje]

In Symbolic, all normal output goes through a single routine which locks/unlocks a mutex which guards access to the standard output.

Code: Select all

void Driver::WriteLn(const std::string& str)
{
  outmutex.Lock();
  std&#58;&#58;cout << str << '\n' << std&#58;&#58;flush;
  outmutex.Unlock&#40;);
  Log&#40;std&#58;&#58;string&#40;"> ") + str&#41;;
&#125;

Note that any text sent to the above routine is echoed to the logfile. The logfile writer has its own mutex as it can be called at places other than writes to the standard output.

[sje]

When either thread wishes to write to the file, if first checks this okay_to_write switch. If this switch is set to 1 (yes) it temporarily sets the switch to 0 (no) and goes ahead and writes to the file. It then sets the switch back to 1 after the write is complete.

If either thread checks the switch and discovers that it is set to 0, this thread now knows the other thread must have just set the switch to 0. So now this thread just waits, and keeps checking the switch until it finds that it's back to 1 again.

Once it does find it back to 1, it works as described above... now THIS thread temporarily sets the switch to 0, writes to the file, and then sets it back to 1 when complete.

This will not work. If more than one thread has modification ability of a given shared resource, then there will be a race condition if the resource is not guarded from simultaneous access. In your scheme the race is not the output file writing, but rather the okay_to_write variable. In the time it takes between testing and setting that variable by one thread, another thread could sneak in and do the same. The test/set operation must be atomic with no chance of an interception. A mutex or spin lock can do this; a simple variable cannot.

[Henk]

Better write into a database. If you write into a file one thread has to wait until another releases the write lock.

So better write to different tables or records of a database.

[Don]

In Maverick I'm writing a log of all the input and out put to a text file.

Since Maverick uses two thread - one to read the input and one to "
think" there is a possibility it tries to write to the same file at the same time. Is this a problem? How do others get around it?

Thanks,

Steve

I think in linux a small write is atomic. There is an internal OS defined write buffer size (I think it's called PIPE_BUFFER_SIZE or something like that) which determines how much is written on one go. So if you had 100 different processes writing to the end of a text file I don't think they would get mixed together - but I'm not sure about the order they get written if that is an issue for you. I don't know how other OS's do it.

I believe the size of that is defined to be 512 and may be a POSIX standard.

Don

[syzygy]

I think in linux a small write is atomic. There is an internal OS defined write buffer size (I think it's called PIPE_BUFFER_SIZE or something like that) which determines how much is written on one go. So if you had 100 different processes writing to the end of a text file I don't think they would get mixed together - but I'm not sure about the order they get written if that is an issue for you. I don't know how other OS's do it.

I believe the size of that is defined to be 512 and may be a POSIX standard.

It is not sufficient (and in fact not so important from the programmer's point of view) that the OS implements atomic writes. The programmer usually does not invoke the OS directly. The question is what the C-library (or e.g. the C# runtime in case of C#) guarantees.

If you use streams (FILE *) in C, then POSIX guarantees that stream operations are atomic (link). So if two threads each perform a single fprintf(), there is a guarantee that they will be executed sequentially, i.e. the program will not crash, and the two output strings will not be interleaved. However, the moment one thread uses two separate fprintf()s that need to stay together, you will need to lock.

It seems streams in C++ before C++11 were not thread-safe, i.e. concurrent stream access could crash the program. C++11 guarantees the program will not crash, but characters may be interleaved (i.e. concurrent cout << "ab" and cout << "cd" may produce "acdb".)

[Steve Maughan]

Hi All,

Thanks for the answers. It would seem I need to "lock" the section of code. How would I do this in Windows? I have read about CRITICALSECTIONS and Windows Mutex - not sure if one is more suitable than the other.

Thanks,

Steve

[mar]

If you use threads, use CRITICAL_SECTION (equivalent to pthread_mutex_t). If you use processes, use CreateMutex.

[Steve Maughan]

Hi Martin,

THANKS!

Steve