Re: Block Device Caching

[Markus Schaber <schabios@logi-track.com>]

Hi,

On Tue, 29 Jun 2004 16:46:22 -0600
Andreas Dilger <adilger@clusterfs.com> wrote:

>>[block device caching problems]
> When you close the block device it flushes the cache for that device
> (inode). If you kept the device open in some way (e.g. "sleep 10000000
> < /dev/hda5") then it should start caching the data between dd runs.

This sounds reasonable, and it works using hda5 on my developer machine:

root@kingfisher:~# dd if=/dev/hda5 of=/dev/null bs=1M count=100
100+0 Datensätze ein
100+0 Datensätze aus
104857600 bytes transferred in 4,644529 seconds (22576584 bytes/sec)
root@kingfisher:~# dd if=/dev/hda5 of=/dev/null bs=1M count=100
100+0 Datensätze ein
100+0 Datensätze aus
104857600 bytes transferred in 4,688006 seconds (22367207 bytes/sec)
root@kingfisher:~# sleep 1000000 </dev/hda5 &
[1] 17321
root@kingfisher:~# dd if=/dev/hda5 of=/dev/null bs=1M count=100
100+0 Datensätze ein
100+0 Datensätze aus
104857600 bytes transferred in 4,662113 seconds (22491433 bytes/sec)
root@kingfisher:~# dd if=/dev/hda5 of=/dev/null bs=1M count=100
100+0 Datensätze ein
100+0 Datensätze aus
104857600 bytes transferred in 0,271807 seconds (385779610 bytes/sec)

And it works on my USB memory stick:

root@kingfisher:~# dd if=/dev/sda of=/dev/null bs=1M count=100
31+1 Datensätze ein
31+1 Datensätze aus
32768000 bytes transferred in 36,011661 seconds (909927 bytes/sec)
root@kingfisher:~# dd if=/dev/sda of=/dev/null bs=1M count=100
31+1 Datensätze ein
31+1 Datensätze aus
32768000 bytes transferred in 37,133379 seconds (882441 bytes/sec)
root@kingfisher:~# sleep 1000000 </dev/sda &
[1] 17375
root@kingfisher:~# dd if=/dev/sda of=/dev/null bs=1M count=100
31+1 Datensätze ein
31+1 Datensätze aus
32768000 bytes transferred in 36,004170 seconds (910117 bytes/sec)
root@kingfisher:~# dd if=/dev/sda of=/dev/null bs=1M count=100
31+1 Datensätze ein
31+1 Datensätze aus
32768000 bytes transferred in 0,088144 seconds (371755027 bytes/sec)

But it seems to fail on our production machine LVM volume:

bear:~# dd if=/dev/daten/testing of=/dev/null bs=1M count=1000 
1000+0 records in
1000+0 records out
1048576000 bytes transferred in 10.779081 seconds (97278793 bytes/sec)
bear:~# dd if=/dev/daten/testing of=/dev/null bs=1M count=1000 
1000+0 records in
1000+0 records out
1048576000 bytes transferred in 10.773274 seconds (97331229 bytes/sec)
bear:~# sleep 1000000 </dev/daten/testing &
[1] 23588
bear:~# dd if=/dev/daten/testing of=/dev/null bs=1M count=1000 
1000+0 records in
1000+0 records out
1048576000 bytes transferred in 11.030774 seconds (95059149 bytes/sec)
bear:~# dd if=/dev/daten/testing of=/dev/null bs=1M count=1000 
1000+0 records in
1000+0 records out
1048576000 bytes transferred in 11.027699 seconds (95085656 bytes/sec)


top on bear displays:
top - 08:23:31 up 68 days,  5:14,  8 users,  load average: 0.08, 0.08, 0.03
Tasks:  81 total,   1 running,  79 sleeping,   0 stopped,   1 zombie
 Cpu0 :  0.0% us,  0.0% sy,  0.0% ni, 99.4% id,  0.0% wa,  0.6% hi,  0.0% si
 Cpu1 :  0.0% us,  0.3% sy,  0.0% ni, 99.7% id,  0.0% wa,  0.0% hi,  0.0% si
 Cpu2 :  0.0% us,  0.0% sy,  0.0% ni, 100.0% id,  0.0% wa,  0.0% hi,  0.0% si
 Cpu3 :  0.0% us,  0.0% sy,  0.0% ni, 100.0% id,  0.0% wa,  0.0% hi,  0.0% si
Mem:   3953624k total,  1120508k used,  2833116k free,   800248k buffers
Swap:  1048568k total,        0k used,  1048568k free,    24932k cached

So there clearly is enough free RAM to buffer 1 Gig of data (which, as I
said, works for a file when mounting /dev/daten/testing with ext3).

Did we do something wrong in our LVM setup? Do you need more info (such
as the output of some lv tools or the kernel config)?

Thanks for your efforts,
Markus Schaber

-- 
markus schaber | dipl. informatiker
logi-track ag | rennweg 14-16 | ch 8001 zürich
phone +41-43-888 62 52 | fax +41-43-888 62 53
mailto:schabios@logi-track.com | www.logi-track.com