BKL removal from VFS

[Nakayama Shintaro <nakayama@tritech.co.jp>]

[-- Attachment #1: Type: Text/Plain, Size: 2241 bytes --]

Hi,

I've found great BKL contention when running multiple postmark
benchmarks. Here is the postmark results with lock contention
sampled by lockmeter.

	# of trans/sec		spinlock contention
1CPU	1197			--
4CPU	1549			62.2% (BKL=61.9%)
8CPU	1607			77.2% (BKL=77.0%)

	- kernel is 2.4.17
	- postmark params(number=5000 transactions=500000)
	- Xeon 550Mhz, 1G memory
	- virtually no disk I/O wait
	- concurrelty running 8 postmark processes

To improve postmark results, I removed BKL from contenting points
with modifications as follows.

- removed BKL from following operations
	i_op->create
	i_op->lookup
	i_op->link
	i_op->mknod
	i_op->mkdir
	i_op->unlink
	i_op->rmdir
	i_op->truncate
	i_op->setattr
	i_op->permission
	i_op->symlink
	f_op->readdir

	removed lock_kernel/unlock_kernel 
		->  LOCK_KERNEL/UNLOCK_KERNEL

- fine-grained ext2 locks

	global resource		protecting lock
	-----------------------------------------
	dir operation		i_sem, i_zombie
	inode, block bitmap	per-group lock
	inode meta data		per-group lock
	ext2 super block	super block lock

- changed VFS locking strategy
	fs_lock protects fs global operations such as sync,
	mount. fs_lock is read/write lock. Each entry into
	filesystem read-locks fs_lock, while global operations
	do write-locking.

- internaling BKL into some filesystems
	rootfs
	proc
	driverfs
	tmpfs	(not shmfs)
	binfmt_misc
	ext3
	nfs

	inserted lock_kernel/unlock_kernel
		-> fs_lock_kernel/fs_unlock_kernel

	other filesystems are not modified, thus dangerous to
	use currently (they have no protecting lock!)

By this modification, BKL contention was reduced greatly.

	# of trans/sec		spinlock contention
1CPU	2430			--
2CPU	4098			1.3%
4CPU	7529			3.2%
8CPU	11394			10.7%

	- 2.5.1 with BKL removal patch
	- postmark params(number=1000 transactions=200000)
	- others same as above

I've attached patch against 2.5.1 and m-postmark script. This 
patch is tested on 8-way SMP machine with postmark and dbench.
You can get patch and data from

	http://www.tritech.co.jp/members/nakayama/linux/vfs-bkl-2.5.1.diff.gz
	http://www.tritech.co.jp/members/nakayama/linux/results.tar.bz2

I'm not in the list, please cc any reply to me. Hope this helps
to improve Linux.

(This work is paid by Fujitsu)

[-- Attachment #2: vfs-bkl-2.5.1.diff.gz --]
[-- Type: application/octet-stream, Size: 30536 bytes --]