Re: CONFIG_PREEMPT and server workloads

[Andrea Arcangeli <andrea@suse.de>]

On Fri, Mar 19, 2004 at 10:52:03PM +0530, Dipankar Sarma wrote:
> On Thu, Mar 18, 2004 at 10:10:06PM -0800, Andrew Morton wrote:
> > The worst-case latency is during umount, fs/inode.c:invalidate_list() when
> > the filesystem has a zillion inodes in icache.  Measured 250 milliseconds
> > on a 256MB 2.7GHz P4 here.   OK, so don't do that.
> > 
> > The unavoidable worst case is in the RCU callbacks for dcache shrinkage -
> > I've seen 25 millisecond holdoffs on the above machine during filesystem
> > stresstests when RCU is freeing a huge number of dentries in softirq
> > context.
> 
> What filesystem stresstest was that ? 
> 
> > 
> > This if Hard To Fix.  Dipankar spent quite some time looking into it and
> > had patches, but I lost track of where they're at.
> 
> And I am still working on this on a larger scope/scale. Yes, I have
> a patch that hands over the rcu callbacks to a per-cpu kernel thread
> reducing the softirq time. However this is not really a solution to

why a per-cpu kernel thread when you can use a softirq?

> the overall problem, IMO. I am collecting some instrumentation
> data to understand softirq/rcu behavior during heavy loads and
> ways to counter long running softirqs.
> 
> Latency isn't the only issue. DoS on route cache is another
> issue that needs to be addressed. I have been experimenting
> with Robert Olsson's router test and should have some more results
> out soon.

why don't you simply interrupt rcu_do_batch after a dozen of callbacks?
if it gets interrupted you then go ahead and you splice the remaining
entries into another list for a tasklet, then the tasklet will be a
reentrant one, so the ksoftirqd will take care of the latency.

the only valid reason to use the timer irq instead of the tasklet in the
first place is to delay the rcu invocation and coalesce the work
together, but if there's too much work to do you must go back to the
tasklet way that has always been scheduler-friendy.