iosched: impact of streaming read on read-many-files

[Andrew Morton <akpm@digeo.com>]

Here we look at what affect a large streaming read has upon an operation
which reads many small files from the same disk.

A single streaming read was set up with:

	while true
	do
	        cat 512M-file > /dev/null
	done

and we measure how long it takes to read all the files from a 2.4.19 kernel
tree off the same disk with

	time (find kernel-tree -type f | xargs cat > /dev/null)



2.4.21-pre4:	31 minutes 30 seconds

2.5.61+hacks:	3 minutes 39 seconds

2.5.61+CFQ:	5 minutes 7 seconds (*)

2.5.61+AS:	17 seconds





* CFQ performed very strangely here.  Tremendous amount of seeking and a
  big drop in aggregate bandwidth.  See the vmstat 1 output from when the
  kernel tree read started up:


 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
 0  1   1240 125260   1176 109488    0    0 40744     0 1672   725  0  3 49 47
 0  1   1240  85892   1220 148788    0    0 39344     0 1651   693  0  3 49 48
 0  1   1240  45124   1260 189492    0    0 40744     0 1663   683  0  3 49 47
 1  1   1240   4544   1300 230068    0    0 40616     0 1661   837  0  4 49 47
 0  2   1348   3468    944 231696    0  108 40488   148 1671   800  0  4  4 91
 0  2   1348   2180    936 232920    0    0 40612    64 1668   789  0  4  0 96
 0  3   1348   4220    996 230648    0    0 11348     0 1256   352  0  2  0 98
 0  3   1348   4052   1064 230472    0    0  9012     0 1207   305  0  1  0 98
 0  4   1348   3596   1148 230580    0    0  6756     0 1171   247  0  1  0 99
 0  4   1348   4044   1148 229888    0    0  6344     0 1165   237  0  1  0 99
 1  3   1348   3708   1160 230212    0    0  7800     0 1187   255  0  1 21 78