Writing to a Text File (Thread Safe)

Discussion of chess software programming and technical issues.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
Steve Maughan
Posts: 1074
Joined: Wed Mar 08, 2006 7:28 pm
Location: Florida, USA
Contact:

Writing to a Text File (Thread Safe)

Post by Steve Maughan » Sat Aug 10, 2013 1:52 pm

In Maverick I'm writing a log of all the input and out put to a text file.

Since Maverick uses two thread - one to read the input and one to "
think" there is a possibility it tries to write to the same file at the same time. Is this a problem? How do others get around it?

Thanks,

Steve
http://www.chessprogramming.net - Maverick Chess Engine

MrEdCollins
Posts: 59
Joined: Mon May 02, 2011 10:12 pm
Location: Southern California
Contact:

Re: Writing to a Text File (Thread Safe)

Post by MrEdCollins » Sat Aug 10, 2013 2:12 pm

Can you toggle a switch that both threads read?

When either thread wishes to write to the file, if first checks this okay_to_write switch. If this switch is set to 1 (yes) it temporarily sets the switch to 0 (no) and goes ahead and writes to the file. It then sets the switch back to 1 after the write is complete.

If either thread checks the switch and discovers that it is set to 0, this thread now knows the other thread must have just set the switch to 0. So now this thread just waits, and keeps checking the switch until it finds that it's back to 1 again.

Once it does find it back to 1, it works as described above... now THIS thread temporarily sets the switch to 0, writes to the file, and then sets it back to 1 when complete.

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 6:43 pm

Re: Writing to a Text File (Thread Safe)

Post by sje » Sat Aug 10, 2013 2:13 pm

In Symbolic, all normal output goes through a single routine which locks/unlocks a mutex which guards access to the standard output.

Code: Select all

void Driver::WriteLn(const std::string& str)
{
  outmutex.Lock();
  std&#58;&#58;cout << str << '\n' << std&#58;&#58;flush;
  outmutex.Unlock&#40;);
  Log&#40;std&#58;&#58;string&#40;"> ") + str&#41;;
&#125;
Note that any text sent to the above routine is echoed to the logfile. The logfile writer has its own mutex as it can be called at places other than writes to the standard output.

User avatar
sje
Posts: 4675
Joined: Mon Mar 13, 2006 6:43 pm

Re: Writing to a Text File (Thread Safe)

Post by sje » Sat Aug 10, 2013 2:28 pm

MrEdCollins wrote:When either thread wishes to write to the file, if first checks this okay_to_write switch. If this switch is set to 1 (yes) it temporarily sets the switch to 0 (no) and goes ahead and writes to the file. It then sets the switch back to 1 after the write is complete.

If either thread checks the switch and discovers that it is set to 0, this thread now knows the other thread must have just set the switch to 0. So now this thread just waits, and keeps checking the switch until it finds that it's back to 1 again.

Once it does find it back to 1, it works as described above... now THIS thread temporarily sets the switch to 0, writes to the file, and then sets it back to 1 when complete.
This will not work. If more than one thread has modification ability of a given shared resource, then there will be a race condition if the resource is not guarded from simultaneous access. In your scheme the race is not the output file writing, but rather the okay_to_write variable. In the time it takes between testing and setting that variable by one thread, another thread could sneak in and do the same. The test/set operation must be atomic with no chance of an interception. A mutex or spin lock can do this; a simple variable cannot.

Henk
Posts: 6416
Joined: Mon May 27, 2013 8:31 am

Re: Writing to a Text File (Thread Safe)

Post by Henk » Sat Aug 10, 2013 3:24 pm

Better write into a database. If you write into a file one thread has to wait until another releases the write lock.

So better write to different tables or records of a database.

User avatar
Don
Posts: 5106
Joined: Tue Apr 29, 2008 2:27 pm

Re: Writing to a Text File (Thread Safe)

Post by Don » Sat Aug 10, 2013 8:22 pm

Steve Maughan wrote:In Maverick I'm writing a log of all the input and out put to a text file.

Since Maverick uses two thread - one to read the input and one to "
think" there is a possibility it tries to write to the same file at the same time. Is this a problem? How do others get around it?

Thanks,

Steve
I think in linux a small write is atomic. There is an internal OS defined write buffer size (I think it's called PIPE_BUFFER_SIZE or something like that) which determines how much is written on one go. So if you had 100 different processes writing to the end of a text file I don't think they would get mixed together - but I'm not sure about the order they get written if that is an issue for you. I don't know how other OS's do it.

I believe the size of that is defined to be 512 and may be a POSIX standard.

Don
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.

syzygy
Posts: 4602
Joined: Tue Feb 28, 2012 10:56 pm

Re: Writing to a Text File (Thread Safe)

Post by syzygy » Sat Aug 10, 2013 11:04 pm

Don wrote:I think in linux a small write is atomic. There is an internal OS defined write buffer size (I think it's called PIPE_BUFFER_SIZE or something like that) which determines how much is written on one go. So if you had 100 different processes writing to the end of a text file I don't think they would get mixed together - but I'm not sure about the order they get written if that is an issue for you. I don't know how other OS's do it.

I believe the size of that is defined to be 512 and may be a POSIX standard.
It is not sufficient (and in fact not so important from the programmer's point of view) that the OS implements atomic writes. The programmer usually does not invoke the OS directly. The question is what the C-library (or e.g. the C# runtime in case of C#) guarantees.

If you use streams (FILE *) in C, then POSIX guarantees that stream operations are atomic (link). So if two threads each perform a single fprintf(), there is a guarantee that they will be executed sequentially, i.e. the program will not crash, and the two output strings will not be interleaved. However, the moment one thread uses two separate fprintf()s that need to stay together, you will need to lock.

It seems streams in C++ before C++11 were not thread-safe, i.e. concurrent stream access could crash the program. C++11 guarantees the program will not crash, but characters may be interleaved (i.e. concurrent cout << "ab" and cout << "cd" may produce "acdb".)

User avatar
Steve Maughan
Posts: 1074
Joined: Wed Mar 08, 2006 7:28 pm
Location: Florida, USA
Contact:

Re: Writing to a Text File (Thread Safe)

Post by Steve Maughan » Sun Aug 11, 2013 12:51 am

Hi All,

Thanks for the answers. It would seem I need to "lock" the section of code. How would I do this in Windows? I have read about CRITICALSECTIONS and Windows Mutex - not sure if one is more suitable than the other.

Thanks,

Steve
http://www.chessprogramming.net - Maverick Chess Engine

mar
Posts: 2122
Joined: Fri Nov 26, 2010 1:00 pm
Location: Czech Republic
Full name: Martin Sedlak

Re: Writing to a Text File (Thread Safe)

Post by mar » Sun Aug 11, 2013 1:18 am

If you use threads, use CRITICAL_SECTION (equivalent to pthread_mutex_t). If you use processes, use CreateMutex.

User avatar
Steve Maughan
Posts: 1074
Joined: Wed Mar 08, 2006 7:28 pm
Location: Florida, USA
Contact:

Re: Writing to a Text File (Thread Safe)

Post by Steve Maughan » Sun Aug 11, 2013 1:21 am

Hi Martin,

THANKS!

Steve
http://www.chessprogramming.net - Maverick Chess Engine

Post Reply