From: Trond Myklebust <trond.myklebust@fys.uio.no>
To: Ingo Molnar <mingo@elte.hu>
Cc: Andrew Morton <akpm@osdl.org>,
rlrevell@joe-job.com, linux-kernel@vger.kernel.org
Subject: Re: NFS client latencies
Date: Thu, 31 Mar 2005 09:54:51 -0500 [thread overview]
Message-ID: <1112280891.20211.29.camel@lade.trondhjem.org> (raw)
In-Reply-To: <20050331143930.GA4032@elte.hu>
to den 31.03.2005 Klokka 16:39 (+0200) skreiv Ingo Molnar:
> * Trond Myklebust <trond.myklebust@fys.uio.no> wrote:
>
> > > your patch works fine here - but there are still latencies in
> > > nfs_scan_commit()/nfs_scan_list(): see the attached 3.7 msec latency
> > > trace. It happened during a simple big-file writeout and is easily
> > > reproducible. Could the nfsi->commit list searching be replaced with a
> > > radix based approach too?
> >
> > That would be 100% pure overhead. The nfsi->commit list does not need
> > to be sorted and with these patches applied, it no longer is. In fact
> > one of the cleanups I still need to do is to get rid of those
> > redundant checks on wb->index (start is now always set to 0, and end
> > is always ~0UL).
> >
> > So the overhead you are currently seeing should just be that of
> > iterating through the list, locking said requests and adding them to
> > our private list.
>
> ah - cool! This was a 100 MB writeout so having 3.7 msecs to process
> 20K+ pages is not unreasonable. To break the latency, can i just do a
> simple lock-break, via the patch below?
Errm... That looks a bit unsafe. What is there then to stop another
process from removing "pos" while you are scheduling?
It seems to me that you should really start afresh in that case.
The good news, though, is that because requests on the "commit" list do
not remain locked for long without being removed from the list, you
should rarely have to skip them. IOW restarting from the head of the
list is pretty much the same as starting from where you left off.
Cheers,
Trond
> Ingo
>
> --- linux/fs/nfs/pagelist.c.orig
> +++ linux/fs/nfs/pagelist.c
> @@ -311,8 +311,9 @@ out:
> * You must be holding the inode's req_lock when calling this function
> */
> int
> -nfs_scan_list(struct list_head *head, struct list_head *dst,
> - unsigned long idx_start, unsigned int npages)
> +nfs_scan_list(struct nfs_inode *nfsi, struct list_head *head,
> + struct list_head *dst, unsigned long idx_start,
> + unsigned int npages)
> {
> struct list_head *pos, *tmp;
> struct nfs_page *req;
> @@ -327,6 +328,8 @@ nfs_scan_list(struct list_head *head, st
>
> list_for_each_safe(pos, tmp, head) {
>
> + cond_resched_lock(&nfsi->req_lock);
> +
> req = nfs_list_entry(pos);
>
> if (req->wb_index < idx_start)
> --- linux/fs/nfs/write.c.orig
> +++ linux/fs/nfs/write.c
> @@ -569,7 +569,7 @@ nfs_scan_commit(struct inode *inode, str
> int res = 0;
>
> if (nfsi->ncommit != 0) {
> - res = nfs_scan_list(&nfsi->commit, dst, idx_start, npages);
> + res = nfs_scan_list(nfsi, &nfsi->commit, dst, idx_start, npages);
> nfsi->ncommit -= res;
> if ((nfsi->ncommit == 0) != list_empty(&nfsi->commit))
> printk(KERN_ERR "NFS: desynchronized value of nfs_i.ncommit.\n");
> --- linux/include/linux/nfs_page.h.orig
> +++ linux/include/linux/nfs_page.h
> @@ -63,8 +63,8 @@ extern void nfs_release_request(struct n
>
> extern int nfs_scan_lock_dirty(struct nfs_inode *nfsi, struct list_head *dst,
> unsigned long idx_start, unsigned int npages);
> -extern int nfs_scan_list(struct list_head *, struct list_head *,
> - unsigned long, unsigned int);
> +extern int nfs_scan_list(struct nfs_inode *nfsi, struct list_head *,
> + struct list_head *, unsigned long, unsigned int);
> extern int nfs_coalesce_requests(struct list_head *, struct list_head *,
> unsigned int);
> extern int nfs_wait_on_request(struct nfs_page *);
--
Trond Myklebust <trond.myklebust@fys.uio.no>
next prev parent reply other threads:[~2005-03-31 14:55 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2005-03-29 23:04 Lee Revell
2005-03-29 23:18 ` Trond Myklebust
2005-03-29 23:32 ` Lee Revell
2005-03-29 23:34 ` Trond Myklebust
2005-03-29 23:37 ` Lee Revell
2005-03-30 8:02 ` Ingo Molnar
2005-03-30 14:11 ` Trond Myklebust
2005-03-30 14:20 ` Ingo Molnar
2005-03-30 19:53 ` Andrew Morton
2005-03-30 14:26 ` Lee Revell
2005-03-30 14:50 ` Trond Myklebust
2005-03-30 19:50 ` Lee Revell
2005-03-30 19:56 ` Andrew Morton
2005-03-30 21:14 ` Trond Myklebust
2005-03-31 2:26 ` Lee Revell
2005-03-31 2:39 ` Andrew Morton
2005-03-31 2:47 ` Lee Revell
2005-03-31 3:48 ` Trond Myklebust
2005-03-31 6:59 ` Ingo Molnar
2005-03-31 7:15 ` Ingo Molnar
2005-03-31 7:18 ` Andrew Morton
2005-03-31 7:30 ` Ingo Molnar
2005-03-31 11:58 ` Trond Myklebust
2005-03-31 12:34 ` Trond Myklebust
2005-03-31 13:58 ` Ingo Molnar
2005-03-31 14:32 ` Trond Myklebust
2005-03-31 14:39 ` Ingo Molnar
2005-03-31 14:50 ` Ingo Molnar
2005-04-01 2:28 ` Lee Revell
2005-04-01 4:30 ` Ingo Molnar
2005-04-01 16:16 ` Orion Poplawski
2005-04-01 16:33 ` Trond Myklebust
2005-04-01 21:18 ` Lee Revell
2005-03-31 14:54 ` Ingo Molnar
2005-03-31 15:00 ` Trond Myklebust
2005-03-31 14:54 ` Trond Myklebust [this message]
2005-03-31 14:58 ` Ingo Molnar
2005-03-31 15:06 ` Trond Myklebust
2005-03-31 15:10 ` Ingo Molnar
2005-03-31 16:00 ` Trond Myklebust
2005-03-31 15:10 ` Ingo Molnar
2005-03-31 7:03 ` Ingo Molnar
2005-03-31 7:39 ` Ingo Molnar
2005-03-31 7:48 ` Ingo Molnar
2005-03-31 7:58 ` Ingo Molnar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1112280891.20211.29.camel@lade.trondhjem.org \
--to=trond.myklebust@fys.uio.no \
--cc=akpm@osdl.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@elte.hu \
--cc=rlrevell@joe-job.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®