Re: O_SYNC option doesn't work (2.4.18-3)

[Daniel Forrest <forrest@lmcg.wisc.edu>]

Gregory Giguashvili <Gregoryg@ParadigmGeo.com> writes:

>> The above sequence works OK, but it's still problematic since it
>> doesn't ensure that the data is coming from the server instead of
>> NFS cache. In one of my previous discussions on this list, I was
>> told to use the following technique to flush NFS buffers. It turns
>> out that acquiring write lock using fcntl (F_SETLK) interface acts
>> as NFS write barrier and flushes all the file NFS buffers. So, the
>> above sequence would result in the following one:
>> 
>> - lock_file (doesn't have to be a lockd lock)
>> - open_file (O_SYNC)
>> - if (read) fcntl (F_SETLK) (doing lock/unlock here to flush NFS buffers)
>> - read/write file
>> - if (write) fcntl (F_SETLK) (doing lock/unlock here to flush NFS buffers)
>> - close_file
>> - unlock_file

I have been working in a mixed Linux/Solaris environment and this is
what works for me.  (>40 machines reading/writing/modifying the same
files at the same time for hundreds of iterations with no errors.)

- open()

  To get a file descriptor that can be locked

- fcntl(F_SETLK)

  To ensure that all other clients have written their data to the file
  and to clear the client cache

- fchmod()

  File attributes (most importantly file size) were cached when the
  file was opened, these may change between that time and when the
  lock is granted, a side effect of fchmod() is to refresh the file
  attributes from the server

- lseek() / read() / write()

  If appending, it is important to lseek(SEEK_END) at this point to
  make sure you are at the true end of file

- close()

  This will flush any data written and remove the lock


If anyone experiments with this with systems other than Linux/Solaris
I would be interested in knowing if it works there also.

-- 
+----------------------------------+----------------------------------+
| Daniel K. Forrest                | Laboratory for Molecular and     |
| forrest@lmcg.wisc.edu            | Computational Genomics           |
| (608)262-9479                    | University of Wisconsin, Madison |
+----------------------------------+----------------------------------+