Linux VM use-once mechanisms don't seem to work.  Simple scenario like
streaming a file much greater than physical RAM size should be
identified to avoid trashing the page cache with useless data.

I know the VM cannot predict the future or assume anything about the
user's intent.  But this workload is simple and common, it should be
detected and better handled.

Test case:

Linux 2.6.20-16-lowlatency SMP PREEMPT x86_64 (also tried on 2.6.23-rc1)

- A file of 1/3 the RAM size is created, mapped and frequently accessed
(4 times).
- The test is run multiple times (4 total) to time it's execution.
- After the first run, other runs take much less time, because the file
is cached.
- A previously created file, 4 times the size of the RAM, is read or
copied.
- The test is re-run (2 times) to time it's execution.

To test:

$ make
# ./use-once-test.sh

Some big files will be created in your /tmp. They don't get erased after
the test to speedup multiple runs.

Results:

- The test execution time greatly increase after reading or copying the
large file.
- Frequently used data got kick out of the page cache and replaced with
useless read once data.
- Both the read only and copy (read + write) cases don't work.

I believe this clearly illustrate the slowdowns I experience after I
copy large files around my system.  All applications on my desktop are
jerky for some moments after that.  Watching a DVD is another example.

Base test:

1st run: 0m8.958s
2nd run: 0m3.442s
3rd run: 0m3.452s
4th run: 0m3.443s

Reading a large file test:

1st run: 0m8.997s
2nd run: 0m3.522s
`/tmp/large_file' -> `/dev/null'
3rd run: 0m8.999s    <<< page cache trashed
4th run: 0m3.440s

Copying (using cp) a large file test:

1st run: 0m8.979s
2nd run: 0m3.442s
`/tmp/large_file' -> `/tmp/large_file.copy'
3rd run: 0m13.814s    <<< page cache trashed
4th run: 0m3.455s

Copying (using fadvise_cp) a large file test:

1st run: 0m9.018s
2nd run: 0m3.444s
Copying large file...
3rd run: 0m14.024s    <<< page cache trashed
4th run: 0m3.449s

Copying (using splice-cp) a large file test:

1st run: 0m8.977s
2nd run: 0m3.442s
Copying large file...
3rd run: 0m14.118s    <<< page cache trashed
4th run: 0m3.456s

Possible solutions:

Various patches to fix the use-once mechanisms were discussed in the
past.  Some more that 6 years ago and some more recently.

http://lwn.net/2001/0726/a/2q.php3
http://lkml.org/lkml/2005/5/3/6
http://lkml.org/lkml/2006/7/17/192
http://lkml.org/lkml/2007/7/9/340
http://lkml.org/lkml/2007/7/21/219 (*1)

(*1) I have tested Peter's patch with some success.  It fix the read
case, but no the copy case.  Results: http://lkml.org/lkml/2007/7/24/527

Test programs and batch files are attached.


- Eric