mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: David Howells <dhowells@redhat.com>
To: wilhelm.meier@fh-kl.de, linux-cachefs@redhat.com,
	nfsv4@linux-nfs.org, linux-kernel@vger.kernel.org
Cc: dhowells@redhat.com, steved@redhat.com
Subject: [PATCH 0/3] Fixes for FS-Cache and CacheFiles
Date: Fri, 20 Nov 2009 21:54:37 +0000	[thread overview]
Message-ID: <20091120215437.15448.26909.stgit@warthog.procyon.org.uk> (raw)
In-Reply-To: <11689.1258704981@redhat.com>

The associated patches provide three sets of changes:

 (1) More ways to get debugging information from the slow-work, FS-Cache and
     CacheFiles facilities.

 (2) Fixes for all facilities.

 (3) Some extensions for slow-work for users other than FS-Cache/CacheFiles
     that aren't included in this series, but upon which other patches have
     been built.

The patches can also be found in:

	http://people.redhat.com/~dhowells/fscache/patches/fscache-fixes.tar.bz2

The bugs fixed were found by running a simple stress tester:

	http://people.redhat.com/~dhowells/slurp.c

Build it and point it at a bunch of NFS (or AFS or whatever) directory trees,
and it forks and execs a copy of itself to handle each directory that it is
given.  Whilst scanning a directory, for each subdir of that directory a new
copy forked and exec'd, and then each file in that directory is read from end
to end.

This results in a large number of processes reading in parallel a large number
of files.  If the data set is large enough, it applies pressure to both RAM
space and cache space, and can cause OOMs to occur, and can cause cachefilesd
to run in parallel trying to cull the cache as well.

With these patches applied, you can see more counters in /proc/fs/fscache/stats
and new sources of information exist too:

 (*) /proc/slow_work_rq

	The items currently being processed by or currently queued for
	processing by the slow-work facility.

 (*) /proc/fs/fscache/objects

	A list of current fscache objects and their states.  This can be
	configured by loading a key into your session keyring before viewing
	the file, for instance:

		keyctl add user fscache:objlist KB @s

	This is described in Documentation/filesystems/caching/fscache.txt

David
---

David Howells (3):
      FS-Cache: Provide nop fscache_stat_d() if CONFIG_FSCACHE_STATS=n
      SLOW_WORK: Fix GFS2 to #include <linux/module.h> before using THIS_MODULE
      SLOW_WORK: Fix CIFS to pass THIS_MODULE to slow_work_register_user()


 fs/cifs/cifsfs.c      |    2 +-
 fs/fscache/internal.h |    1 +
 fs/gfs2/recovery.c    |    1 +
 3 files changed, 3 insertions(+), 1 deletions(-)


  reply	other threads:[~2009-11-20 21:54 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-11-19 17:20 [PATCH 00/28] " David Howells
2009-11-19 17:20 ` [PATCH 01/28] SLOW_WORK: Wait for outstanding work items belonging to a module to clear David Howells
2009-11-20  9:19   ` steve
2009-11-19 17:20 ` [PATCH 02/28] SLOW_WORK: Make slow_work_ops ->get_ref/->put_ref optional David Howells
2009-11-19 17:20 ` [PATCH 03/28] SLOW_WORK: Add support for cancellation of slow work David Howells
2009-11-19 17:20 ` [PATCH 04/28] SLOW_WORK: Add delayed_slow_work support David Howells
2009-11-19 17:20 ` [PATCH 05/28] SLOW_WORK: Allow the work items to be viewed through a /proc file David Howells
2009-11-19 17:21 ` [PATCH 06/28] SLOW_WORK: Allow the owner of a work item to determine if it is queued or not David Howells
2009-11-19 17:21 ` [PATCH 07/28] SLOW_WORK: Allow a requeueable work item to sleep till the thread is needed David Howells
2009-11-19 17:21 ` [PATCH 08/28] FS-Cache: Annotate slow-work runqueue proc lines for FS-Cache work items David Howells
2009-11-19 17:21 ` [PATCH 09/28] FS-Cache: Allow the current state of all objects to be dumped David Howells
2009-11-19 17:21 ` [PATCH 10/28] FS-Cache: Add counters for entry/exit to/from cache operation functions David Howells
2009-11-19 17:21 ` [PATCH 11/28] FS-Cache: Clear netfs pointers in cookie after detaching object, not before David Howells
2009-11-19 17:21 ` [PATCH 12/28] FS-Cache: Use radix tree preload correctly in tracking of pages to be stored David Howells
2009-11-19 17:21 ` [PATCH 13/28] FS-Cache: Permit cache retrieval ops to be interrupted in the initial wait phase David Howells
2009-11-19 17:21 ` [PATCH 14/28] FS-Cache: The object-available state can't rely on the cookie to be available David Howells
2009-11-19 17:21 ` [PATCH 15/28] FS-Cache: Fix lock misorder in fscache_write_op() David Howells
2009-11-19 17:21 ` [PATCH 16/28] FS-Cache: Don't delete pending pages from the page-store tracking tree David Howells
2009-11-19 17:22 ` [PATCH 17/28] FS-Cache: Handle read request vs lookup, creation or other cache failure David Howells
2009-11-19 17:22 ` [PATCH 18/28] FS-Cache: Handle pages pending storage that get evicted under OOM conditions David Howells
2009-11-19 17:22 ` [PATCH 19/28] FS-Cache: Add a retirement stat counter David Howells
2009-11-19 17:22 ` [PATCH 20/28] FS-Cache: Make sure FSCACHE_COOKIE_LOOKING_UP cleared on lookup failure David Howells
2009-11-19 17:22 ` [PATCH 21/28] FS-Cache: Start processing an object's operations on that object's death David Howells
2009-11-19 17:22 ` [PATCH 22/28] FS-Cache: Actually requeue an object when requested David Howells
2009-11-19 17:22 ` [PATCH 23/28] CacheFiles: Don't write a full page if there's only a partial page to cache David Howells
2009-11-19 17:22 ` [PATCH 24/28] CacheFiles: Handle truncate unlocking the page we're reading David Howells
2009-11-19 17:22 ` [PATCH 25/28] CacheFiles: Mark parent directory locks as I_MUTEX_PARENT to keep lockdep happy David Howells
2009-11-19 17:22 ` [PATCH 26/28] CacheFiles: Better showing of debugging information in active object problems David Howells
2009-11-19 17:22 ` [PATCH 27/28] CacheFiles: Catch an overly long wait for an old active object David Howells
2009-11-19 17:22 ` [PATCH 28/28] CacheFiles: Don't log lookup/create failing with ENOBUFS David Howells
2009-11-20  8:16 ` [PATCH 00/28] Fixes for FS-Cache and CacheFiles David Howells
2009-11-20 21:54   ` David Howells [this message]
2009-11-20 21:54     ` [PATCH 1/3] SLOW_WORK: Fix CIFS to pass THIS_MODULE to slow_work_register_user() David Howells
2009-11-20 21:54     ` [PATCH 2/3] SLOW_WORK: Fix GFS2 to #include <linux/module.h> before using THIS_MODULE David Howells
2009-11-20 21:54     ` [PATCH 3/3] FS-Cache: Provide nop fscache_stat_d() if CONFIG_FSCACHE_STATS=n David Howells
2009-11-20  8:18 ` [PATCH 00/28] Fixes for FS-Cache and CacheFiles David Howells

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20091120215437.15448.26909.stgit@warthog.procyon.org.uk \
    --to=dhowells@redhat.com \
    --cc=linux-cachefs@redhat.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=nfsv4@linux-nfs.org \
    --cc=steved@redhat.com \
    --cc=wilhelm.meier@fh-kl.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®