A fix for Re: Severe I/O performance regression 2.6.6 to 2.6.7 or 2.6.8-rc3

A fix for Re: Severe I/O performance regression 2.6.6 to 2.6.7 or 2.6.8-rc3

[Berkley Shands]

Taking the 2.6.6-bk6/mm/readahead.c (a one line change from vanilla 2.6.6/mm/readahead.c)
and inserting it into 2.6.7 and 2.6.8-rc3 restores the thread read ahead functionality
and througput seen in the 2.6.6 kernel. This does however reduce the single stream read
performance from 426MB/Sec (under 2.6.8-rc3) to 386MB/Sec (same as 2.6.6).
The major functionality change was to try to detect "random" read performance,
if randomness is detected, then NEVER read ahead. In this case, two threads
using two different file descriptors to the same file read alternating strides
of the file. Since each thread did not do SEQUENTIAL reads, both were treated as random
accesses.

readahead.c needs to look at a little more context, or it needs to increase the size of the
window it uses to detect "randomness". In this case the read stride is 11KB.
A decent minimum window should be at least the stripe size for raid0.

The larger-than-phys-memory file read issue seems to have calmed down with the
readahead.c update. The 10MB/Sec is now 295MB/Sec (it should be 400+MB/Sec).
top shows
 08:08:27  up 14 min,  2 users,  load average: 1.34, 0.56, 0.21
75 processes: 73 sleeping, 2 running, 0 zombie, 0 stopped
CPU states:  cpu    user    nice  system    irq  softirq  iowait    idle
           total    0.3%    0.0%   41.5%   0.3%     3.1%   39.2%   15.2%
           cpu00    0.5%    0.0%   51.2%   0.7%     6.3%   39.4%    1.5%
           cpu01    0.3%    0.0%   31.8%   0.0%     0.0%   39.0%   28.7%
Mem:  7587576k av, 7575424k used,   12152k free,       0k shrd,    7676k buff
           20k active,            7391452k inactive
Swap: 2008116k av,   13496k used, 1994620k free                 7379716k cached

  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME CPU COMMAND
   57 root      15   0     0    0     0 RW    5.1  0.0   0:07   1 kswapd0

so kswapd is not out to lunch anymore. I will still have to figure out a peak
value for block size and RASize to see if I can improve large file read performance.

berkley

Re: A fix for Re: Severe I/O performance regression 2.6.6 to 2.6.7 or 2.6.8-rc3

[Andrew Morton]

Berkley Shands <berkley@cs.wustl.edu> wrote:
>
> Taking the 2.6.6-bk6/mm/readahead.c (a one line change from vanilla 2.6.6/mm/readahead.c)
>  and inserting it into 2.6.7 and 2.6.8-rc3 restores the thread read ahead functionality
>  and througput seen in the 2.6.6 kernel. This does however reduce the single stream read
>  performance from 426MB/Sec (under 2.6.8-rc3) to 386MB/Sec (same as 2.6.6).
>  The major functionality change was to try to detect "random" read performance,
>  if randomness is detected, then NEVER read ahead. In this case, two threads
>  using two different file descriptors to the same file read alternating strides
>  of the file. Since each thread did not do SEQUENTIAL reads, both were treated as random
>  accesses.

Let me attempt to clarify here.  You have two fd's open against the same
file and you are reading in the following manner:

fd1:   0-11k        33-44k        55-66k
fd2:         11-22k        44-55k

yes?

If correct, I think you're expecting a bit too much of the readahead
system.  If you have a real application which is using such access patterns
perhaps it should be taught to use POSIX_FADV_WILLNEED.