In Maverick I'm writing a log of all the input and out put to a text file.
Since Maverick uses two thread - one to read the input and one to "
think" there is a possibility it tries to write to the same file at the same time. Is this a problem? How do others get around it?
Thanks,
Steve
Writing to a Text File (Thread Safe)
Moderators: hgm, Rebel, chrisw
-
- Posts: 1221
- Joined: Wed Mar 08, 2006 8:28 pm
- Location: Florida, USA
Writing to a Text File (Thread Safe)
http://www.chessprogramming.net - Maverick Chess Engine
-
- Posts: 59
- Joined: Tue May 03, 2011 12:12 am
- Location: Southern California
Re: Writing to a Text File (Thread Safe)
Can you toggle a switch that both threads read?
When either thread wishes to write to the file, if first checks this okay_to_write switch. If this switch is set to 1 (yes) it temporarily sets the switch to 0 (no) and goes ahead and writes to the file. It then sets the switch back to 1 after the write is complete.
If either thread checks the switch and discovers that it is set to 0, this thread now knows the other thread must have just set the switch to 0. So now this thread just waits, and keeps checking the switch until it finds that it's back to 1 again.
Once it does find it back to 1, it works as described above... now THIS thread temporarily sets the switch to 0, writes to the file, and then sets it back to 1 when complete.
When either thread wishes to write to the file, if first checks this okay_to_write switch. If this switch is set to 1 (yes) it temporarily sets the switch to 0 (no) and goes ahead and writes to the file. It then sets the switch back to 1 after the write is complete.
If either thread checks the switch and discovers that it is set to 0, this thread now knows the other thread must have just set the switch to 0. So now this thread just waits, and keeps checking the switch until it finds that it's back to 1 again.
Once it does find it back to 1, it works as described above... now THIS thread temporarily sets the switch to 0, writes to the file, and then sets it back to 1 when complete.
-
- Posts: 4675
- Joined: Mon Mar 13, 2006 7:43 pm
Re: Writing to a Text File (Thread Safe)
In Symbolic, all normal output goes through a single routine which locks/unlocks a mutex which guards access to the standard output.
Note that any text sent to the above routine is echoed to the logfile. The logfile writer has its own mutex as it can be called at places other than writes to the standard output.
Code: Select all
void Driver::WriteLn(const std::string& str)
{
outmutex.Lock();
std::cout << str << '\n' << std::flush;
outmutex.Unlock();
Log(std::string("> ") + str);
}
-
- Posts: 4675
- Joined: Mon Mar 13, 2006 7:43 pm
Re: Writing to a Text File (Thread Safe)
This will not work. If more than one thread has modification ability of a given shared resource, then there will be a race condition if the resource is not guarded from simultaneous access. In your scheme the race is not the output file writing, but rather the okay_to_write variable. In the time it takes between testing and setting that variable by one thread, another thread could sneak in and do the same. The test/set operation must be atomic with no chance of an interception. A mutex or spin lock can do this; a simple variable cannot.MrEdCollins wrote:When either thread wishes to write to the file, if first checks this okay_to_write switch. If this switch is set to 1 (yes) it temporarily sets the switch to 0 (no) and goes ahead and writes to the file. It then sets the switch back to 1 after the write is complete.
If either thread checks the switch and discovers that it is set to 0, this thread now knows the other thread must have just set the switch to 0. So now this thread just waits, and keeps checking the switch until it finds that it's back to 1 again.
Once it does find it back to 1, it works as described above... now THIS thread temporarily sets the switch to 0, writes to the file, and then sets it back to 1 when complete.
-
- Posts: 7220
- Joined: Mon May 27, 2013 10:31 am
Re: Writing to a Text File (Thread Safe)
Better write into a database. If you write into a file one thread has to wait until another releases the write lock.
So better write to different tables or records of a database.
So better write to different tables or records of a database.
-
- Posts: 5106
- Joined: Tue Apr 29, 2008 4:27 pm
Re: Writing to a Text File (Thread Safe)
I think in linux a small write is atomic. There is an internal OS defined write buffer size (I think it's called PIPE_BUFFER_SIZE or something like that) which determines how much is written on one go. So if you had 100 different processes writing to the end of a text file I don't think they would get mixed together - but I'm not sure about the order they get written if that is an issue for you. I don't know how other OS's do it.Steve Maughan wrote:In Maverick I'm writing a log of all the input and out put to a text file.
Since Maverick uses two thread - one to read the input and one to "
think" there is a possibility it tries to write to the same file at the same time. Is this a problem? How do others get around it?
Thanks,
Steve
I believe the size of that is defined to be 512 and may be a POSIX standard.
Don
Capital punishment would be more effective as a preventive measure if it were administered prior to the crime.
-
- Posts: 5566
- Joined: Tue Feb 28, 2012 11:56 pm
Re: Writing to a Text File (Thread Safe)
It is not sufficient (and in fact not so important from the programmer's point of view) that the OS implements atomic writes. The programmer usually does not invoke the OS directly. The question is what the C-library (or e.g. the C# runtime in case of C#) guarantees.Don wrote:I think in linux a small write is atomic. There is an internal OS defined write buffer size (I think it's called PIPE_BUFFER_SIZE or something like that) which determines how much is written on one go. So if you had 100 different processes writing to the end of a text file I don't think they would get mixed together - but I'm not sure about the order they get written if that is an issue for you. I don't know how other OS's do it.
I believe the size of that is defined to be 512 and may be a POSIX standard.
If you use streams (FILE *) in C, then POSIX guarantees that stream operations are atomic (link). So if two threads each perform a single fprintf(), there is a guarantee that they will be executed sequentially, i.e. the program will not crash, and the two output strings will not be interleaved. However, the moment one thread uses two separate fprintf()s that need to stay together, you will need to lock.
It seems streams in C++ before C++11 were not thread-safe, i.e. concurrent stream access could crash the program. C++11 guarantees the program will not crash, but characters may be interleaved (i.e. concurrent cout << "ab" and cout << "cd" may produce "acdb".)
-
- Posts: 1221
- Joined: Wed Mar 08, 2006 8:28 pm
- Location: Florida, USA
Re: Writing to a Text File (Thread Safe)
Hi All,
Thanks for the answers. It would seem I need to "lock" the section of code. How would I do this in Windows? I have read about CRITICALSECTIONS and Windows Mutex - not sure if one is more suitable than the other.
Thanks,
Steve
Thanks for the answers. It would seem I need to "lock" the section of code. How would I do this in Windows? I have read about CRITICALSECTIONS and Windows Mutex - not sure if one is more suitable than the other.
Thanks,
Steve
http://www.chessprogramming.net - Maverick Chess Engine
-
- Posts: 2559
- Joined: Fri Nov 26, 2010 2:00 pm
- Location: Czech Republic
- Full name: Martin Sedlak
Re: Writing to a Text File (Thread Safe)
If you use threads, use CRITICAL_SECTION (equivalent to pthread_mutex_t). If you use processes, use CreateMutex.
-
- Posts: 1221
- Joined: Wed Mar 08, 2006 8:28 pm
- Location: Florida, USA
Re: Writing to a Text File (Thread Safe)
Hi Martin,
THANKS!
Steve
THANKS!
Steve
http://www.chessprogramming.net - Maverick Chess Engine