inode_lock heavily contended in 2.6.11

inode_lock heavily contended in 2.6.11

[Peter Chubb]

When running reaim7 on a 12-way IA64 on an ext2 filesystem on a ram
disc, I see very heavy contention on inode_lock.

lockstat output shows:

SPINLOCKS         HOLD            WAIT
  UTIL  CON    MEAN(  MAX )   MEAN(  MAX )(% CPU)     TOTAL NOWAIT SPIN RJECT  NAME
 46.8% 52.4%  1.9us( 130us)   20us(8073us)(21.5%)   5072151 47.6% 52.4%    0%  inode_lock
 15.9% 59.5%  3.8us(  61us)   18us(7067us)( 3.9%)    852983 40.5% 59.5%    0%    __sync_single_inode+0xf0
  9.2% 59.0%  1.2us(  25us)   20us(8073us)( 7.8%)   1596487 41.0% 59.0%    0%    generic_osync_inode+0xe0

 (etc).

Is anyone else seeing this on more realistic workloads?

-- 
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
The technical we do immediately,  the political takes *forever*

Re: inode_lock heavily contended in 2.6.11

[Andrew Morton]

Peter Chubb <peterc@gelato.unsw.edu.au> wrote:
>
> 
> When running reaim7 on a 12-way IA64 on an ext2 filesystem on a ram
> disc, I see very heavy contention on inode_lock.

Yes, that's a big global lock, protecting global resources.  I always
expected it to hit someone's fan, but it never did.

> lockstat output shows:
> 
> SPINLOCKS         HOLD            WAIT
>   UTIL  CON    MEAN(  MAX )   MEAN(  MAX )(% CPU)     TOTAL NOWAIT SPIN RJECT  NAME
>  46.8% 52.4%  1.9us( 130us)   20us(8073us)(21.5%)   5072151 47.6% 52.4%    0%  inode_lock
>  15.9% 59.5%  3.8us(  61us)   18us(7067us)( 3.9%)    852983 40.5% 59.5%    0%    __sync_single_inode+0xf0
>   9.2% 59.0%  1.2us(  25us)   20us(8073us)( 7.8%)   1596487 41.0% 59.0%    0%    generic_osync_inode+0xe0
> 
>  (etc).
> 
> Is anyone else seeing this on more realistic workloads?

An fsync/O_SYNC-intensive workload on a ram disk is likely to bring it out,
yes.  But once one has real disks under there, the acquisition frequency
will fall a lot.

Unless people are being all shy again, I don't think anyone has hit
significant inode_lock problems on more real-world things.