* RAID1 oddity on 2.4.17+
@ 2002-01-07 23:39 Francois Romieu
2002-01-19 22:08 ` RAID1/promise 20265 trouble on 2.4.18-pre3-ac2 (was: RAID1 oddity on 2.4.17+) Francois Romieu
0 siblings, 1 reply; 2+ messages in thread
From: Francois Romieu @ 2002-01-07 23:39 UTC (permalink / raw)
To: linux-kernel
[-- Attachment #1: Type: text/plain, Size: 5270 bytes --]
Greetings,
hda: IC35L060AVER07-0, ATA DISK drive
hdc: IC35L060AVER07-0, ATA DISK drive
hde: IC35L060AVER07-0, ATA DISK drive
hdg: IC35L060AVER07-0, ATA DISK drive
$ lspci
00:00.0 Host bridge: Intel Corporation 82845 845 (Brookdale) Chipset Host Bridge
(rev 03)
00:01.0 PCI bridge: Intel Corporation 82845 845 (Brookdale) Chipset AGP Bridge
(rev 03)
00:1e.0 PCI bridge: Intel Corporation 82801BAM PCI (rev 12)
00:1f.0 ISA bridge: Intel Corporation 82801BA ISA Bridge (ICH2) (rev 12)
00:1f.1 IDE interface: Intel Corporation 82801BA IDE U100 (rev 12)
01:00.0 VGA compatible controller: ATI Technologies Inc Rage 128 PF
02:06.0 Ethernet controller: Standard Microsystems Corp [SMC] 83C170QF (rev 09)
02:07.0 Ethernet controller: Winbond Electronics Corp W89C940
02:0a.0 Ethernet controller: 3Com Corporation 3c905C-TX [Fast Etherlink] (rev
78)
02:0d.0 Multimedia audio controller: C-Media Electronics Inc CM8738 (rev 10)
02:0e.0 Unknown mass storage controller: Promise Technology, Inc. 20265 (rev 02)
$ cat /proc/interrupts
CPU0
0: 89814 XT-PIC timer
1: 4 XT-PIC keyboard
2: 0 XT-PIC cascade
5: 190 XT-PIC eth2
7: 1884 XT-PIC eth0
8: 1 XT-PIC rtc
9: 20487 XT-PIC ide2, ide3
10: 3851 XT-PIC eth1
12: 20 XT-PIC PS/2 Mouse
14: 21445 XT-PIC ide0
15: 22279 XT-PIC ide1
NMI: 0
LOC: 89777
ERR: 0
MIS: 0
$ cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
md0 : active raid1 hdc1[0] hda1[1]
1052160 blocks [2/2] [UU]
md1 : active raid1 hdc2[0] hda2[1]
2104448 blocks [2/2] [UU]
md2 : active raid1 hdc3[1] hda3[0]
56894080 blocks [2/2] [UU]
md3 : active raid1 hdg3[1] hde3[0]
56894080 blocks [2/2] [UU]
unused devices: <none>
$ dmesg | less
Linux version 2.4.17 (root@se4.intra.cogenit.fr) (gcc version egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)) #
6 Sun Jan 6 17:03:37 CET 2002
$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 1
model name : Intel(R) Pentium(R) 4 CPU 1.50GHz
stepping : 2
cpu MHz : 1513.513
cache size : 256 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm
bogomips : 3021.20
# partition table of /dev/hde
unit: sectors
/dev/hde1 : start= 63, size= 2104452, Id=fd, bootable
/dev/hde2 : start= 2104515, size= 4209030, Id=fd
/dev/hde3 : start= 6313545, size=113788395, Id=fd
/dev/hde4 : start= 0, size= 0, Id= 0
Ditto for hda, hdc, hdg.
CD-ROM present as hdb.
Output of this script (2.4.17) as first attachment
#!/bin/sh
for i in hdc hdg; do
hdparm -i /dev/$i
hdparm -t /dev/$i
sleep 1
done
for i in hdc3 hdg3; do
time mke2fs -j -O sparse_super -T largefile -m 0 -R stride=8 -v /dev/$i
sleep 1
done
sync
for i in hdc3 hdg3; do
mkdir /mnt/$i
mount /dev/$i /mnt/$i
date;
time dd if=/dev/zero of=/mnt/$i/foo bs=1024k count=1000
time sync
sleep 1
done
So far, so good.
Turn hdc into hda, hdg into hde, see second attachment.
So far, so good.
Same output available on request for test-2.4.18-pre1-andrew-last-low-latency.
I issue a reboot.
Now I build RAID1 arrays as:
raiddev /dev/md2
raid-level 1
nr-raid-disks 2
chunk-size 128k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda3
raid-disk 0
device /dev/hdc3
raid-disk 1
raiddev /dev/md3
raid-level 1
nr-raid-disks 2
chunk-size 128k
persistent-superblock 1
nr-spare-disks 0
device /dev/hde3
raid-disk 0
device /dev/hdg3
raid-disk 1
Please, see third attached file. Everything is fast.
/etc/fstab:
LABEL=/ / ext3 defaults 1 1
none /dev/pts devpts gid=5,mode=620 0 0
none /proc proc defaults 0 0
none /dev/shm tmpfs defaults 0 0
/dev/md0 swap swap defaults 0 0
/dev/cdrom /mnt/cdrom iso9660 noauto,owner,kudzu,ro 0
Now I run this script:
#!/bin/sh
for i in md2 md3; do
time mke2fs -j -O sparse_super -T largefile -m 0 -R stride=8 -v /dev/$i
sleep 1
done
sync
for i in md3 md2; do
mkdir /mnt/$i
mount /dev/$i /mnt/$i
date;
time dd if=/dev/zero of=/mnt/$i/foo bs=1024k count=1000
time sync
sleep 1
done
Please, see fourth attached file for output.
Question: where does the slowdown for operations on /dev/md3 come from ?
--
Ueimor
[-- Attachment #2: log-hdc-hdg --]
[-- Type: application/octet-stream, Size: 15442 bytes --]
[-- Attachment #3: log-hda-hde --]
[-- Type: application/octet-stream, Size: 15442 bytes --]
[-- Attachment #4: log-md2-md3 --]
[-- Type: application/octet-stream, Size: 14066 bytes --]
[-- Attachment #5: raid-build --]
[-- Type: text/plain, Size: 3453 bytes --]
$ cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
md3 : active raid1 hdg3[1] hde3[0]
56894080 blocks [2/2] [UU]
[>....................] resync = 4.2% (2418372/56894080) finish=24.9min speed=36422K/sec
md2 : active raid1 hdc3[1] hda3[0]
56894080 blocks [2/2] [UU]
[=======>.............] resync = 37.0% (21059712/56894080) finish=22.8min speed=26088K/sec
md0 : active raid1 hdc1[0] hda1[1]
1052160 blocks [2/2] [UU]
md1 : active raid1 hdc2[0] hda2[1]
2104448 blocks [2/2] [UU]
unused devices: <none>
$ vmstat 1
procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
0 0 3 0 482908 7000 13084 0 0 14 15 1171 1655 0 8 91
0 0 2 0 482908 7000 13084 0 0 0 0 2259 3663 0 26 74
0 0 2 0 482900 7008 13084 0 0 0 28 2248 3602 0 21 79
0 0 2 0 482900 7008 13084 0 0 0 0 2266 3544 0 23 77
0 0 2 0 482900 7008 13084 0 0 0 0 2269 3509 0 14 86
0 0 2 0 482900 7008 13084 0 0 0 0 2296 3655 0 24 76
0 0 2 0 482900 7008 13084 0 0 0 0 2334 3579 0 19 81
0 0 2 0 482900 7008 13084 0 0 0 108 2208 3410 0 20 80
0 0 2 0 482900 7008 13084 0 0 0 0 2215 3493 0 19 81
0 0 2 0 482900 7008 13084 0 0 0 0 2273 3512 0 21 79
0 0 2 0 482900 7008 13084 0 0 0 0 2323 3620 0 23 77
0 0 2 0 482900 7008 13084 0 0 0 0 2302 3619 0 26 74
0 0 2 0 482900 7008 13084 0 0 0 0 2315 3645 0 17 83
0 0 2 0 482900 7008 13084 0 0 0 0 2322 3659 0 25 75
0 0 2 0 482900 7008 13084 0 0 0 0 2286 3570 0 17 83
0 0 2 0 482900 7008 13084 0 0 0 0 2263 3561 0 22 78
0 0 2 0 482900 7008 13084 0 0 0 0 2294 3608 0 18 82
$ cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
md3 : active raid1 hdg3[1] hde3[0]
56894080 blocks [2/2] [UU]
[==>..................] resync = 12.6% (7210432/56894080) finish=23.0min speed=35899K/sec
md2 : active raid1 hdc3[1] hda3[0]
56894080 blocks [2/2] [UU]
[========>............] resync = 43.0% (24489792/56894080) finish=20.6min speed=26176K/sec
md0 : active raid1 hdc1[0] hda1[1]
1052160 blocks [2/2] [UU]
md1 : active raid1 hdc2[0] hda2[1]
2104448 blocks [2/2] [UU]
unused devices: <none>
$ cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
md3 : active raid1 hdg3[1] hde3[0]
56894080 blocks [2/2] [UU]
[=============>.......] resync = 67.2% (38241476/56894080) finish=10.6min speed=29192K/sec
md2 : active raid1 hdc3[1] hda3[0]
56894080 blocks [2/2] [UU]
[=================>...] resync = 86.1% (49040512/56894080) finish=5.2min speed=24700K/sec
md0 : active raid1 hdc1[0] hda1[1]
1052160 blocks [2/2] [UU]
md1 : active raid1 hdc2[0] hda2[1]
2104448 blocks [2/2] [UU]
unused devices: <none>
Jan 7 23:13:51 se4 kernel: md: syncing RAID array md2
Jan 7 23:24:50 se4 kernel: md: syncing RAID array md3
Jan 7 23:50:05 se4 kernel: md: md2: sync done.
Jan 7 23:56:47 se4 kernel: md: md3: sync done.
[-- Attachment #6: vmstat-dd-md2 --]
[-- Type: text/plain, Size: 1761 bytes --]
r b w swpd free buff cache si so bi bo in cs us sy id
1 1 1 0 2880 20216 385436 0 0 0 27060 590 30 0 8 92
0 1 0 0 3256 20216 385040 0 0 0 28296 594 32 0 10 90
1 0 2 0 2292 20232 386800 0 0 4 31300 586 265 0 25 75
0 1 2 0 2440 17808 388268 0 0 0 30032 586 503 0 41 59
0 1 2 0 2708 17808 388004 0 0 0 28004 602 31 0 9 91
0 1 1 0 3220 17808 387492 0 0 0 27604 595 36 0 11 89
1 0 2 0 2652 17812 389036 0 0 0 26924 608 49 0 18 82
1 0 2 0 2292 17844 388608 0 0 8 27824 480 1130 0 74 26
0 1 2 0 2292 17848 388496 0 0 0 29588 588 73 0 12 88
0 1 2 0 2420 17848 388368 0 0 0 27428 592 27 0 3 97
0 1 2 0 2796 17848 387992 0 0 0 27268 584 29 0 8 92
1 0 1 0 2292 17960 388928 0 0 4 11120 442 914 0 62 38
0 1 3 0 2516 17984 388136 0 0 0 31916 544 252 0 20 80
0 1 3 0 2772 17984 387880 0 0 0 27432 593 37 0 7 93
0 1 3 0 3028 17984 387624 0 0 0 30784 597 39 0 7 93
1 0 3 0 2356 18020 388776 0 0 0 29816 586 470 0 37 63
0 1 2 0 2576 18064 387892 0 0 4 27308 587 406 0 32 68
0 1 1 0 3296 18096 386516 0 0 0 32640 659 421 0 30 70
0 1 1 0 3296 18096 386516 0 0 0 27304 593 26 0 9 91
0 1 3 0 2452 18176 388072 0 0 0 28136 580 668 0 52 48
0 1 3 0 2708 18176 387816 0 0 0 27180 588 35 0 4 96
[-- Attachment #7: vmstat-dd-md3 --]
[-- Type: text/plain, Size: 1760 bytes --]
r b w swpd free buff cache si so bi bo in cs us sy id
0 1 2 0 3200 1852 480384 0 0 49 1580 165 50 1 5 94
0 1 2 0 3200 1852 480384 0 0 0 7824 172 14 0 3 97
0 1 2 0 3200 1852 480384 0 0 0 0 132 8 0 0 100
0 1 2 0 3200 1852 480384 0 0 0 2360 130 9 0 1 99
0 1 2 0 3200 1852 480384 0 0 0 0 130 6 0 0 100
0 1 2 0 3320 1860 480256 0 0 0 28 138 14 0 0 100
0 1 2 0 3320 1860 480256 0 0 0 7820 164 14 0 1 99
0 1 2 0 3316 1860 480256 0 0 0 0 132 10 0 1 99
0 1 2 0 3316 1860 480256 0 0 0 2356 129 9 0 1 99
0 1 2 0 3316 1860 480256 0 0 0 0 129 6 0 0 100
0 1 2 0 3316 1860 480256 0 0 0 0 135 8 0 0 100
0 1 2 0 3316 1860 480256 0 0 0 3968 130 10 0 1 99
0 1 2 0 3316 1860 480256 0 0 0 0 129 8 0 0 100
0 1 2 0 3316 1860 480256 0 0 0 7828 173 14 0 1 99
0 1 2 0 3316 1860 480256 0 0 0 0 132 6 0 1 99
0 1 3 0 3212 1876 480368 0 0 0 2704 128 181 0 9 91
0 1 2 0 3212 1876 480368 0 0 0 0 130 5 0 0 100
0 1 2 0 3212 1876 480368 0 0 0 0 134 7 0 1 99
0 1 2 0 3316 1876 480244 0 0 0 7696 166 14 0 1 99
0 1 2 0 3316 1876 480244 0 0 0 0 128 5 0 1 99
0 1 2 0 3316 1876 480244 0 0 0 2728 129 9 0 0 100
^ permalink raw reply [flat|nested] 2+ messages in thread* RAID1/promise 20265 trouble on 2.4.18-pre3-ac2 (was: RAID1 oddity on 2.4.17+)
2002-01-07 23:39 RAID1 oddity on 2.4.17+ Francois Romieu
@ 2002-01-19 22:08 ` Francois Romieu
0 siblings, 0 replies; 2+ messages in thread
From: Francois Romieu @ 2002-01-19 22:08 UTC (permalink / raw)
To: linux-kernel
Francois Romieu <romieu@cogenit.fr> :
[see <URL:http://www.cs.helsinki.fi/linux/linux-kernel/2002-01/0511.html>]
I put the same partitions on the 4 disks and built two similar RAID arrays.
o md10 is on the Intel controller and md20 on the Promise 20265
(I've dedicated these two arrays to testing, you're welcome).
# cat /proc/mdstat
[...]
md10 : active raid1 hdc3[1] hda3[0]
10241344 blocks [2/2] [UU]
[...]
md20 : active raid1 hdg3[1] hde3[0]
10241344 blocks [2/2] [UU]
o Boot-time parameters:
Kernel command line: BOOT_IMAGE=2.4.18-pre3-ac2 ro root=901 \
BOOT_FILE=/boot/bzImage-2.4.18-pre3-ac2 nmi_watchdog=1 root=/dev/md1 \
ide0=autotune ide1=autotune ide2=autotune ide3=autotune profile=2 devfs=nomount
o Some gross test:
sync; date; dd if=/dev/zero of=test bs=1024k count=500; sync; date
/dev/md10: 20s
/dev/md20: 240s+/-9s (<-- 10 times more *doh*)
Reproductibility: good.
o The first column is a difference in profile output when test is done
againt /dev/md10 and the second when the test is done on /dev/md20.
1 1 __block_prepare_write
0 2 __brelse
2 1 __journal_file_buffer
2 0 __journal_remove_journal_head
0 1 __kfree_skb
6 12 __make_request
0 1 __put_unused_buffer_head
1 2 __rdtsc_delay
1 1 __refile_buffer
1 3 __wake_up
1 0 balance_dirty_state
1 0 block_prepare_write
0 5 bread
0 1 d_lookup
1 0 deactivate_page_nolock
5060 30630 default_idle
0 1 do_anonymous_page
4 7 do_get_write_access
32 23 do_rw_disk
0 5 do_softirq
0 1 do_wp_page
0 2 ext3_block_to_path
2 1 ext3_commit_write
1 1 ext3_dirty_inode
2 1 ext3_do_update_inode
1 0 ext3_get_block_handle
1 1 ext3_get_branch
2 0 ext3_get_group_desc
11 6 ext3_get_inode_loc
3 2 ext3_mark_inode_dirty
5 5 ext3_new_block
0 1 ext3_releasepage
0 2 ext3_reserve_inode_write
0 1 ext3_writepage_trans_blocks
122 129 generic_file_write
2 2 generic_make_request
8 7 get_hash_table
8 39 handle_IRQ_event
2 1 ide_build_dmatable
8 4 ide_build_sglist
5 8 ide_dma_intr
22 19 ide_dmaproc
0 1 ide_do_request
25 27 ide_end_request
2 1 ide_get_queue
4 52 ide_intr
11 17 ide_wait_stat
4 2 journal_add_journal_head
3 5 journal_cancel_revoke
6 4 journal_commit_transaction
1 0 journal_destroy_revoke_caches
2 0 journal_dirty_data
5 4 journal_dirty_metadata
5 1 journal_dirty_sync_data
1 0 journal_free_journal_head
2 5 journal_get_write_access
0 1 journal_start
3 1 journal_stop
0 1 journal_unlock_journal_head
10 13 kfree
17 18 kmalloc
29 19 kmem_cache_alloc
8 13 kmem_cache_free
2 1 kmem_cache_grow
0 1 kmem_slab_destroy
1 0 ll_rw_block
0 13 pdc202xx_dmaproc
0 1 prune_icache
1 0 raid1_alloc_bh
3 1 raid1_alloc_r1bh
3 2 raid1_make_request
3 0 rmqueue
1 0 schedule
0 1 set_bh_page
10 4 start_request
0 1 sys_read
1 0 sys_write
2 0 system_call
1 0 try_to_free_buffers
3 1 unlock_page
1 1 walk_page_buffers
2 0 write_some_buffers
0 1 zap_page_range
0 2 zeromap_page_range
o Nothing special in dmesg output.
o ide settings:
# diff -u /proc/ide/hda/settings /proc/ide/hde/settings
--- /proc/ide/hda/settings Sat Jan 19 22:48:47 2002
+++ /proc/ide/hde/settings Sat Jan 19 22:48:47 2002
@@ -2,8 +2,8 @@
---- ----- --- --- ----
acoustic 0 0 254 rw
address 0 0 2 rw
-bios_cyl 7476 0 65535 rw
-bios_head 255 0 255 rw
+bios_cyl 119150 0 65535 rw
+bios_head 16 0 255 rw
bios_sect 63 0 63 rw
breada_readahead 128 0 255 rw
bswap 0 0 1 r
@@ -11,7 +11,7 @@
failures 0 0 65535 rw
file_readahead 124 0 16384 rw
ide_scsi 0 0 1 rw
-init_speed 69 0 69 rw
+init_speed 12 0 69 rw
io_32bit 3 0 3 rw
keepsettings 0 0 1 rw
lun 0 0 7 rw
# diff -u /proc/ide/hdc/settings /proc/ide/hdg/settings
--- /proc/ide/hdc/settings Sat Jan 19 22:49:35 2002
+++ /proc/ide/hdg/settings Sat Jan 19 22:49:35 2002
@@ -2,8 +2,8 @@
---- ----- --- --- ----
acoustic 0 0 254 rw
address 0 0 2 rw
-bios_cyl 7476 0 65535 rw
-bios_head 255 0 255 rw
+bios_cyl 119150 0 65535 rw
+bios_head 16 0 255 rw
bios_sect 63 0 63 rw
breada_readahead 128 0 255 rw
bswap 0 0 1 r
@@ -11,7 +11,7 @@
failures 0 0 65535 rw
file_readahead 124 0 16384 rw
ide_scsi 0 0 1 rw
-init_speed 69 0 69 rw
+init_speed 12 0 69 rw
io_32bit 3 0 3 rw
keepsettings 0 0 1 rw
lun 0 0 7 rw
o Thanks for your attention.
--
Ueimor
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2002-01-19 22:09 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2002-01-07 23:39 RAID1 oddity on 2.4.17+ Francois Romieu
2002-01-19 22:08 ` RAID1/promise 20265 trouble on 2.4.18-pre3-ac2 (was: RAID1 oddity on 2.4.17+) Francois Romieu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®