[Q] How to flush disk cache w/read-only filesystem w/o unmount&remount? (shared SAN filesystem)

[Q] How to flush disk cache w/read-only filesystem w/o unmount&remount? (shared SAN filesystem)

[Jim Lawson]

Greetings kernel-hackers -

I'm setting up a web farm with a number of Linux machines (ia32) attached
via FC-AL to a shared SAN.  There are 3 web servers, one of them mounts
the filesystem read/write, the other 2 read/only.  (Filesystem is
currently ext3.)

Everything works "fine", until of course I try to modify files on the r/w
server.  Then we start seeing cache incoherency - the r/o systems now have
"stale" information cached.  Old versions of files and directories are not
replaced.

I'm looking for a way to flush or invalidate the cache on the block
device/filesystem, so that the system is forced to go all the way to the
disk.  Unmounting and remounting would accomplish this, of course, but
that's tough to do in production.

This is a "read-mostly" application - I really only need to update the
filesystem once a day or so - but I'd like to find a nice way to do it,
without having to use NFS.

More info, if needed: Running Linux 2.4.18 (under Debian woody.)  Using
the qla2200 driver from Qlogic, although I don't think that matters - it
looks like the caching happens in the VFS somewhere...

I've tried blockdev --flushbufs, which appears to do a BLKFLSBUF, but that
seems to be equivalent to "sync" - just pushes the dirty buffers to disk,
which doesn't help me.

Looking in fs/dcache.c, I see shrink_dcache_sb(struct super_block *),
which *might* do what I want, but there doesn't seem to be any way to call
that from user-land.  And that probably just updates the dentries - I
don't know if that has any effect on the file data.

Hoping someone with more kernel experience can enlighten me -
Jim Lawson

Re: [Q] How to flush disk cache w/read-only filesystem w/o unmount&remount? (shared SAN filesystem)

[Andreas Dilger]

On Nov 05, 2002  10:18 -0500, Jim Lawson wrote:
> I'm looking for a way to flush or invalidate the cache on the block
> device/filesystem, so that the system is forced to go all the way to the
> disk.  Unmounting and remounting would accomplish this, of course, but
> that's tough to do in production.
> 
> I've tried blockdev --flushbufs, which appears to do a BLKFLSBUF, but that
> seems to be equivalent to "sync" - just pushes the dirty buffers to disk,
> which doesn't help me.
> 
> Looking in fs/dcache.c, I see shrink_dcache_sb(struct super_block *),
> which *might* do what I want, but there doesn't seem to be any way to call
> that from user-land.  And that probably just updates the dentries - I
> don't know if that has any effect on the file data.

You may be in luck - we likely need to have an ioctl to do this from
e2fsck because the ext3 htree repacking of bad directories is causing
a bunch of problems.  In theory, the startup scripts should reboot the
system in this case, but there have also been cases reported where
people ran "e2fsck -D" on an ro-mounted root and then read-write mounted
it again.

Cheers, Andreas
--
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: [Q] How to flush disk cache w/read-only filesystem w/o unmount&remount? (shared SAN filesystem)

[Anton Blanchard]

> You may be in luck - we likely need to have an ioctl to do this from
> e2fsck because the ext3 htree repacking of bad directories is causing
> a bunch of problems.  In theory, the startup scripts should reboot the
> system in this case, but there have also been cases reported where
> people ran "e2fsck -D" on an ro-mounted root and then read-write mounted
> it again.

It would be great if we could avoid the reboot, the biggest machines
I have access to can take 45 minutes to to pass firmware checks. I
hate it when fsck finds and error and causes another reboot :)

Anton