Block Device Caching

[Markus Schaber <schabios@logi-track.com>]

Hello,

During our application testing, we noticed that our application (that
operates directly on a LVM volume) we noticed that it seems the read
data does not go into any cache.

Now we did some tests using dd blocksize=1M count=1000:

Using dd directly on the /dev/daten/testing lvm volume, we read about 95
MBytes/Seconds. Issuing multiple dds in sequence gives little variance in IO
speed (between 90 and 100 MB/sec).

When we create a file system on this volume, and mount it, and we create
a 1G file there, the dd gives us the same 95 MB/sec on the first read
after the mount, and approx. 480 MB/Sec on subsequent reads.

The machine runs Kernel 2.6.5 SMP on a dual SMT Itanium HP Box.

This lead us to the conclusion that block devices do not cache, but the
filesystem does. But subsequently, I ran some tests on my developer
machine (Pentium 4 Mobile Laptop).

dd'ing 16MB from an usb 1.1 stick present as /dev/sda, I got about
900k/sec on every read, so this seems to be uncached access, too.

When dd'ing 100MB from /dev/hda5, the first read gives about
22MBytes/Sek (which seems okay for a 2.5" IDE Disk), but subsequend
reads give about 389MBytes/sek (which is impossible to achieve using
such hardware). Interestingly, this happens on a mounted partition,
while when unmounting the partition, caching does not work. But for the
/dev/daten/testing above, mounting a filesystem on the partition does
not help in caching.

Can anyone give us a hint what's happening here, or even tell us how to
use a block device via the kernel caching (and memory mapping does not
work easily, as the production lvm volume is about 600 Gig on said
32-bit X86 machine.

Thanks,
Markus

PS: I subscribed to the list, so no Cc:-ing required.
-- 
markus schaber | dipl. informatiker
logi-track ag | rennweg 14-16 | ch 8001 zürich
phone +41-43-888 62 52 | fax +41-43-888 62 53
mailto:schabios@logi-track.com | www.logi-track.com