Re: [RFC] [PATCH] Include LRU in page count

[Daniel Phillips <phillips@arcor.de>]

On Monday 02 September 2002 02:17, Andrew Morton wrote:
> Daniel Phillips wrote:
> > 
> > On Monday 02 September 2002 01:08, Andrew Morton wrote:
> > > Daniel Phillips wrote:
> > > > Note that I changed the spin_lock in page_cache_release to a trylock, maybe
> > > > it's worth checking out the effect on contention.  With a little head
> > > > scratching we might be able to get rid of the spin_lock in lru_cache_add as
> > > > well.  That leaves (I think) just the two big scan loops.  I've always felt
> > > > it's silly to run more than one of either at the same time anyway.
> > >
> > > No way.  Take a look at http://samba.org/~anton/linux/2.5.30/
> > >
> > > That's 8-way power4, the workload is "dd from 7 disks
> > > dd if=/dev/sd* of=/dev/null bs=1024k".
> > >
> > > The CPU load in this situation was dominated by the VM.  The LRU list and page
> > > reclaim.  Spending more CPU in lru_cache_add() than in copy_to_user() is
> > > pretty gross.
> > 
> > Are we looking at the same thing?  The cpu load there is dominated by cpu_idle,
> > 89%.
> 
> Apart from that, dummy ;)  The absolute numbers are proportional
> to IO bandwidth.  And 7/8ths of a scsi disk per CPU isn't much.

OK, fine, my point here is that there's no real excuse for contention on the
lru lock.  Without departing radically from the current design the most you
can possibly spend on page reclaiming is 100% of one cpu, and we ought to
simply aim at getting the most bang for the buck out of that one cpu.  It
follows that we want only one copy of shrink_cache or refill_inactive running
at a time, otherwise we're tossing away cycles spinning on locks and getting
nothing back.

If lru_cache_add is costly then we can kill both the lock contention and the
cacheline pingponging by feeding the new pages into a local list, then
grafting the whole list onto the inactive list when we start to scan.  We
want to do something like that for nonsucky readahead as well.

-- 
Daniel