Block device swamping disk cache

Block device swamping disk cache

[marcus hall]

I am having problems writing a filesystem image to a device file.

I am running an arm version of the 2.5.59 kernel.

Writing the data to the device file eventually caused ps.nr_dirty to exceed
dirty_thresh in balance_dirty_pages(), yet when writeback_inodes() scans
all the superblocks, the inodes that it finds are all marked as memory_backed
(they are on a ramdisk), and the inode for the block device file doesn't
show up.

I did not see any place where mark_inode_dirty() gets called for the block
device..  Is there some other mechanism for flushing the disk cache allocated
to the device?  I end up with balance_dirty_pages() looping forever.

Just for grins, I added a call to mark_inode_dirty() in blkdev_commit_write()
(similar to generic_commit_write()), but this had no effect that I could
notice.  I did note that in blkdev_commit_write(), if I look at
file->f_dentry->d_inode, I see the inode form the /dev filesystem, yet
page->mapping->host is a different inode!  I don't know if this is a
problem or not, but it did surprise me..

Anyhow, is this a known issue with 2.5.59?  How is the disk cache allocated
to a block device supposed to get flushed?

Thanks!

Marcus Hall
marcus@tuells.org

Re: Block device swamping disk cache

[Christoph Hellwig]

On Tue, May 11, 2004 at 01:11:24PM -0600, marcus hall wrote:
> I am having problems writing a filesystem image to a device file.
> 
> I am running an arm version of the 2.5.59 kernel.

I don't think anyone here still remembers stoneage-aera kernels.

You're better off trying to reproduce it with 2.6.6 I guess.

Re: Block device swamping disk cache

[marcus hall]

On Tue, May 11, 2004 at 08:19:36PM +0100, Christoph Hellwig wrote:
> > I am running an arm version of the 2.5.59 kernel.
> 
> I don't think anyone here still remembers stoneage-aera kernels.
> 
> You're better off trying to reproduce it with 2.6.6 I guess.

Well, when we started the project, the 2.5.59 base was the most current.
Since then, there hasn't been time to try to update all of the drivers to
the 2.6 kernel.  I would like to eventually get there, but at this point
it's an unfunded mandate.

So, my question is just in general terms, what is supposed to occur to flush
blocks out to a block device?  Is the inode supposed to be marked as dirty
and queued on the superblock containing the block device file (sort of what
I expected to be seeing), or is there some other method?  I can transpose
between the 2.6 and 2.5 kernels enough to try to see what isn't happening,
unless the strategy has changed in the 2.6 kernel..

Marcus Hall
marcus@tuells.org

Re: Block device swamping disk cache

[Andrew Morton]

Christoph Hellwig <hch@infradead.org> wrote:
>
> On Tue, May 11, 2004 at 01:11:24PM -0600, marcus hall wrote:
> > I am having problems writing a filesystem image to a device file.
> > 
> > I am running an arm version of the 2.5.59 kernel.
> 
> I don't think anyone here still remembers stoneage-aera kernels.
> 
> You're better off trying to reproduce it with 2.6.6 I guess.

no, sorry, it'll still happen.  I haven't fixed the ramdisk driver yet.

The problem is that ->memory_backed means both "doesn't contribute
to dirty memory" and also "doesn't need writeback".

These concepts need to be split apart for the ramdisk driver.  I'll do it
for 2.6.7, promise.

Re: Block device swamping disk cache

[marcus hall]

On Tue, May 11, 2004 at 05:26:43PM -0700, Andrew Morton wrote:
> no, sorry, it'll still happen.  I haven't fixed the ramdisk driver yet.
> 
> The problem is that ->memory_backed means both "doesn't contribute
> to dirty memory" and also "doesn't need writeback".
> 
> These concepts need to be split apart for the ramdisk driver.  I'll do it
> for 2.6.7, promise.

Well, I believe that the inodes that are marked as memory_backed are
for the ramdisk, and that isn't really a problem.  The block device
that I am writing to is a compact flash, so it's going through the ide-disk
device.  I do not see this inode show up on any superblock's dirty queue
(since it doesn't appear that mark_inode_dirty() is being called for
it).  So the question I am asking is, what strategy is *supposed* to be
in place to flush the blocks out?  (Or is this a hole that isn't plugged?)

Marcus Hall
marcus@tuells.org

Re: Block device swamping disk cache

[Andrew Morton]

marcus hall <marcus@tuells.org> wrote:
>
> On Tue, May 11, 2004 at 05:26:43PM -0700, Andrew Morton wrote:
>  > no, sorry, it'll still happen.  I haven't fixed the ramdisk driver yet.
>  > 
>  > The problem is that ->memory_backed means both "doesn't contribute
>  > to dirty memory" and also "doesn't need writeback".
>  > 
>  > These concepts need to be split apart for the ramdisk driver.  I'll do it
>  > for 2.6.7, promise.
> 
>  Well, I believe that the inodes that are marked as memory_backed are
>  for the ramdisk, and that isn't really a problem.  The block device
>  that I am writing to is a compact flash, so it's going through the ide-disk
>  device.  I do not see this inode show up on any superblock's dirty queue
>  (since it doesn't appear that mark_inode_dirty() is being called for
>  it).  So the question I am asking is, what strategy is *supposed* to be
>  in place to flush the blocks out?  (Or is this a hole that isn't plugged?)

Not sure why you're stuck running Krufty Old Kernels, but this patch from
June 2003 should fix it.

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2003/06/13 17:43:10-07:00 akpm@digeo.com 
#   [PATCH] fix writeback for dirty ramdisk blockdev inodes
#   
#   Once the blockdev inode for /dev/ram0 is dirtied we have a memory-backed
#   inode on the blockdev superblock's s_dirty list.
#   
#   sync_sb_inodes() sees the memory-backed inode on the superblock and assumes
#   that all the other inodes on the superblock are also memory-backed.  This is
#   not true for the blockdev superblock!  We forget to write out dirty pages
#   against the following blockdevs.
#   
#   Fix this by just leaving the inode dirty and moving on to inspect the other
#   blockdev inodes on sb->s_io.
#   
#   (This is a little inefficient: an alternative is to leave dirtied
#   memory-backed inodes on inode_in_use, so nobody ever even considers them for
#   writeout.  But that introduces an inconsistency and is a bit kludgey).
# 
# fs/fs-writeback.c
#   2003/06/13 08:39:48-07:00 akpm@digeo.com +14 -1
#   fix writeback for dirty ramdisk blockdev inodes
# 
diff -Nru a/fs/fs-writeback.c b/fs/fs-writeback.c
--- a/fs/fs-writeback.c	Wed May 12 17:18:54 2004
+++ b/fs/fs-writeback.c	Wed May 12 17:18:54 2004
@@ -260,8 +260,21 @@
 		struct address_space *mapping = inode->i_mapping;
 		struct backing_dev_info *bdi = mapping->backing_dev_info;
 
-		if (bdi->memory_backed)
+		if (bdi->memory_backed) {
+			if (sb == blockdev_superblock) {
+				/*
+				 * Dirty memory-backed blockdev: the ramdisk
+				 * driver does this.
+				 */
+				list_move(&inode->i_list, &sb->s_dirty);
+				continue;
+			}
+			/*
+			 * Assume that all inodes on this superblock are memory
+			 * backed.  Skip the superblock.
+			 */
 			break;
+		}
 
 		if (wbc->nonblocking && bdi_write_congested(bdi)) {
 			wbc->encountered_congestion = 1;

Block device swamping disk cache

[marcus hall]

I am now having problems writing a filesystem image to a device file.

I am running an arm version of the 2.5.59 kernel.

Writing the data to the device file eventually caused ps.nr_dirty to exceed
dirty_thresh in balance_dirty_pages(), yet when writeback_inodes() scans
all the superblocks, the inodes that it finds are all marked as memory_backed
(they are on a ramdisk), and the inode for the block device file doesn't
show up.

I did not see any place where mark_inode_dirty() gets called for the block
device..  Is there some other mechanism for flushing the disk cache allocated
to the device?  I end up with balance_dirty_pages() looping forever.

Just for grins, I added a call to mark_inode_dirty() in blkdev_commit_write()
(similar to generic_commit_write()), but this had no effect that I could
notice.  I did note that in blkdev_commit_write(), if I look at
file->f_dentry->d_inode, I see the inode form the /dev filesystem, yet
page->mapping->host is a different inode!  I don't know if this is a
problem or not, but it did surprise me..

Anyhow, is this a known issue with 2.5.59?  How is the disk cache allocated
to a block device supposed to get flushed?

Thanks!

Marcus Hall
marcus@tuells.org