* processes hung in D (raid5/dm/ext3)
@ 2004-06-15 6:22 foo
2004-06-15 10:09 ` Andrew Morton
0 siblings, 1 reply; 19+ messages in thread
From: foo @ 2004-06-15 6:22 UTC (permalink / raw)
To: linux-kernel; +Cc: foo
I'm getting processes hung in the D state accessing the mount point of
the home directories on my file server - the same machine for which I
just described a problem about NFS, see:
Message-ID: <20040615000436.GA12516@porto.bmb.uga.edu>
or
http://www.uwsg.iu.edu/hypermail/linux/kernel/0406.1/1345.html
The filesystem in question is ext3 on LVM2 on software RAID5. The past
two nights in a row (I created the raid array on Sunday) system backups
or updatedb processes get hung accessing /export/home. Since this is a
frequently accessed directory, basically all of the processes on the
system eventually get hung requiring a reboot by sysrq. I manually
fscked both /export/home and / and both are fine after reboot.
The kernel is 2.6.7-rc3-bk4 with 2 patches:
http://www.uwsg.iu.edu/hypermail/linux/kernel/0406.1/0996.html
http://www.uwsg.iu.edu/hypermail/linux/kernel/0406.1/0889.html
The raid array and PV/VG/LVs were created under 2.6.5 with the 32/64
dm-ioctl.h patch (this is a 64-bit machine with a 32-bit userspace).
Please let me know if any other information would help in tracking this
down, I am *very* eager to get this fixed...
Thanks for your help,
-ryan
#
# Automatically generated make config: don't edit
#
CONFIG_X86_64=y
CONFIG_64BIT=y
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_RWSEM_GENERIC_SPINLOCK=y
CONFIG_X86_CMPXCHG=y
CONFIG_EARLY_PRINTK=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_GENERIC_ISA_DMA=y
#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_CLEAN_COMPILE=y
CONFIG_STANDALONE=y
#
# General setup
#
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
# CONFIG_POSIX_MQUEUE is not set
# CONFIG_BSD_PROCESS_ACCT is not set
CONFIG_SYSCTL=y
# CONFIG_AUDIT is not set
CONFIG_LOG_BUF_SHIFT=17
CONFIG_HOTPLUG=y
# CONFIG_IKCONFIG is not set
# CONFIG_EMBEDDED is not set
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
CONFIG_OBSOLETE_MODPARM=y
# CONFIG_MODVERSIONS is not set
# CONFIG_KMOD is not set
CONFIG_STOP_MACHINE=y
#
# Processor type and features
#
CONFIG_MK8=y
# CONFIG_MPSC is not set
# CONFIG_GENERIC_CPU is not set
CONFIG_X86_L1_CACHE_BYTES=64
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_TSC=y
CONFIG_X86_GOOD_APIC=y
CONFIG_MICROCODE=y
# CONFIG_X86_MSR is not set
# CONFIG_X86_CPUID is not set
CONFIG_X86_IO_APIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_MTRR=y
CONFIG_SMP=y
# CONFIG_PREEMPT is not set
# CONFIG_SCHED_SMT is not set
CONFIG_K8_NUMA=y
CONFIG_DISCONTIGMEM=y
CONFIG_NUMA=y
CONFIG_HAVE_DEC_LOCK=y
CONFIG_NR_CPUS=8
CONFIG_GART_IOMMU=y
CONFIG_SWIOTLB=y
CONFIG_X86_MCE=y
#
# Power management options
#
# CONFIG_PM is not set
#
# ACPI (Advanced Configuration and Power Interface) Support
#
CONFIG_ACPI=y
CONFIG_ACPI_BOOT=y
CONFIG_ACPI_INTERPRETER=y
# CONFIG_ACPI_AC is not set
# CONFIG_ACPI_BATTERY is not set
CONFIG_ACPI_BUTTON=y
CONFIG_ACPI_FAN=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_THERMAL=y
# CONFIG_ACPI_ASUS is not set
# CONFIG_ACPI_TOSHIBA is not set
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_BUS=y
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_PCI=y
CONFIG_ACPI_SYSTEM=y
#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set
#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_LEGACY_PROC=y
CONFIG_PCI_NAMES=y
#
# PCMCIA/CardBus support
#
# CONFIG_PCMCIA is not set
#
# PCI Hotplug Support
#
# CONFIG_HOTPLUG_PCI is not set
#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
# CONFIG_BINFMT_MISC is not set
CONFIG_IA32_EMULATION=y
# CONFIG_IA32_AOUT is not set
CONFIG_COMPAT=y
CONFIG_SYSVIPC_COMPAT=y
CONFIG_UID16=y
#
# Device Drivers
#
#
# Generic Driver Options
#
# CONFIG_FW_LOADER is not set
# CONFIG_DEBUG_DRIVER is not set
#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set
#
# Parallel port support
#
CONFIG_PARPORT=y
CONFIG_PARPORT_PC=y
CONFIG_PARPORT_PC_CML1=y
# CONFIG_PARPORT_SERIAL is not set
# CONFIG_PARPORT_PC_FIFO is not set
# CONFIG_PARPORT_PC_SUPERIO is not set
# CONFIG_PARPORT_OTHER is not set
# CONFIG_PARPORT_1284 is not set
#
# Plug and Play support
#
#
# Block devices
#
CONFIG_BLK_DEV_FD=y
# CONFIG_PARIDE is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
CONFIG_BLK_DEV_LOOP=y
# CONFIG_BLK_DEV_CRYPTOLOOP is not set
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_CARMEL is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=4096
CONFIG_BLK_DEV_INITRD=y
CONFIG_LBD=y
#
# ATA/ATAPI/MFM/RLL support
#
# CONFIG_IDE is not set
#
# SCSI device support
#
CONFIG_SCSI=y
CONFIG_SCSI_PROC_FS=y
#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
CONFIG_CHR_DEV_ST=y
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=y
# CONFIG_BLK_DEV_SR_VENDOR is not set
CONFIG_CHR_DEV_SG=y
#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
# CONFIG_SCSI_MULTI_LUN is not set
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set
#
# SCSI Transport Attributes
#
CONFIG_SCSI_SPI_ATTRS=y
# CONFIG_SCSI_FC_ATTRS is not set
#
# SCSI low-level drivers
#
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AACRAID is not set
# CONFIG_SCSI_AIC7XXX is not set
# CONFIG_SCSI_AIC7XXX_OLD is not set
# CONFIG_SCSI_AIC79XX is not set
# CONFIG_SCSI_ADVANSYS is not set
# CONFIG_SCSI_MEGARAID is not set
# CONFIG_SCSI_SATA is not set
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_EATA_PIO is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_PPA is not set
# CONFIG_SCSI_IMM is not set
CONFIG_SCSI_SYM53C8XX_2=y
CONFIG_SCSI_SYM53C8XX_DMA_ADDRESSING_MODE=1
CONFIG_SCSI_SYM53C8XX_DEFAULT_TAGS=16
CONFIG_SCSI_SYM53C8XX_MAX_TAGS=64
# CONFIG_SCSI_SYM53C8XX_IOMAPPED is not set
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_QLOGIC_ISP is not set
# CONFIG_SCSI_QLOGIC_FC is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
CONFIG_SCSI_QLA2XXX=y
# CONFIG_SCSI_QLA21XX is not set
# CONFIG_SCSI_QLA22XX is not set
# CONFIG_SCSI_QLA2300 is not set
# CONFIG_SCSI_QLA2322 is not set
# CONFIG_SCSI_QLA6312 is not set
# CONFIG_SCSI_QLA6322 is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_DEBUG is not set
#
# Multi-device support (RAID and LVM)
#
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
# CONFIG_MD_LINEAR is not set
# CONFIG_MD_RAID0 is not set
CONFIG_MD_RAID1=y
CONFIG_MD_RAID5=y
# CONFIG_MD_RAID6 is not set
# CONFIG_MD_MULTIPATH is not set
CONFIG_BLK_DEV_DM=y
# CONFIG_DM_CRYPT is not set
#
# Fusion MPT device support
#
CONFIG_FUSION=y
CONFIG_FUSION_MAX_SGE=40
CONFIG_FUSION_ISENSE=m
# CONFIG_FUSION_CTL is not set
#
# IEEE 1394 (FireWire) support
#
# CONFIG_IEEE1394 is not set
#
# I2O device support
#
# CONFIG_I2O is not set
#
# Networking support
#
CONFIG_NET=y
#
# Networking options
#
CONFIG_PACKET=y
CONFIG_PACKET_MMAP=y
# CONFIG_NETLINK_DEV is not set
CONFIG_UNIX=y
# CONFIG_NET_KEY is not set
CONFIG_INET=y
# CONFIG_IP_MULTICAST is not set
# CONFIG_IP_ADVANCED_ROUTER is not set
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE is not set
# CONFIG_ARPD is not set
CONFIG_SYN_COOKIES=y
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
# CONFIG_INET_IPCOMP is not set
# CONFIG_IPV6 is not set
# CONFIG_NETFILTER is not set
#
# SCTP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_SCTP is not set
# CONFIG_ATM is not set
# CONFIG_BRIDGE is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_NET_FASTROUTE is not set
# CONFIG_NET_HW_FLOWCONTROL is not set
#
# QoS and/or fair queueing
#
# CONFIG_NET_SCHED is not set
#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
CONFIG_NETDEVICES=y
# CONFIG_DUMMY is not set
CONFIG_BONDING=m
# CONFIG_EQUALIZER is not set
# CONFIG_TUN is not set
#
# ARCnet devices
#
# CONFIG_ARCNET is not set
#
# Ethernet (10 or 100Mbit)
#
# CONFIG_NET_ETHERNET is not set
#
# Ethernet (1000 Mbit)
#
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SK98LIN is not set
CONFIG_TIGON3=y
#
# Ethernet (10000 Mbit)
#
# CONFIG_IXGB is not set
# CONFIG_S2IO is not set
#
# Token Ring devices
#
# CONFIG_TR is not set
#
# Wireless LAN (non-hamradio)
#
# CONFIG_NET_RADIO is not set
#
# Wan interfaces
#
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PLIP is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not set
# CONFIG_SHAPER is not set
# CONFIG_NETCONSOLE is not set
#
# ISDN subsystem
#
# CONFIG_ISDN is not set
#
# Telephony Support
#
# CONFIG_PHONE is not set
#
# Input device support
#
CONFIG_INPUT=y
#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1600
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=1200
# CONFIG_INPUT_JOYDEV is not set
# CONFIG_INPUT_TSDEV is not set
# CONFIG_INPUT_EVDEV is not set
# CONFIG_INPUT_EVBUG is not set
#
# Input I/O drivers
#
# CONFIG_GAMEPORT is not set
CONFIG_SOUND_GAMEPORT=y
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
# CONFIG_SERIO_SERPORT is not set
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PARKBD is not set
# CONFIG_SERIO_PCIPS2 is not set
#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_INPUT_MOUSE is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set
#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_SERIAL_NONSTANDARD is not set
#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_ACPI=y
CONFIG_SERIAL_8250_NR_UARTS=4
# CONFIG_SERIAL_8250_EXTENDED is not set
#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
CONFIG_PRINTER=y
# CONFIG_LP_CONSOLE is not set
# CONFIG_PPDEV is not set
# CONFIG_TIPAR is not set
# CONFIG_QIC02_TAPE is not set
#
# IPMI
#
# CONFIG_IPMI_HANDLER is not set
#
# Watchdog Cards
#
CONFIG_WATCHDOG=y
# CONFIG_WATCHDOG_NOWAYOUT is not set
#
# Watchdog Device Drivers
#
CONFIG_SOFT_WATCHDOG=y
# CONFIG_ACQUIRE_WDT is not set
# CONFIG_ADVANTECH_WDT is not set
# CONFIG_ALIM1535_WDT is not set
# CONFIG_ALIM7101_WDT is not set
# CONFIG_SC520_WDT is not set
# CONFIG_EUROTECH_WDT is not set
# CONFIG_IB700_WDT is not set
# CONFIG_WAFER_WDT is not set
# CONFIG_I8XX_TCO is not set
# CONFIG_SC1200_WDT is not set
# CONFIG_SCx200_WDT is not set
# CONFIG_60XX_WDT is not set
# CONFIG_CPU5_WDT is not set
# CONFIG_W83627HF_WDT is not set
# CONFIG_W83877F_WDT is not set
# CONFIG_MACHZ_WDT is not set
#
# PCI-based Watchdog Cards
#
# CONFIG_PCIPCWATCHDOG is not set
# CONFIG_WDTPCI is not set
#
# USB-based Watchdog Cards
#
# CONFIG_USBPCWATCHDOG is not set
CONFIG_HW_RANDOM=y
# CONFIG_NVRAM is not set
CONFIG_RTC=y
# CONFIG_DTLK is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
#
# Ftape, the floppy tape device driver
#
CONFIG_AGP=y
CONFIG_AGP_AMD64=y
# CONFIG_AGP_INTEL_MCH is not set
# CONFIG_DRM is not set
# CONFIG_MWAVE is not set
# CONFIG_RAW_DRIVER is not set
CONFIG_HANGCHECK_TIMER=y
#
# I2C support
#
CONFIG_I2C=y
CONFIG_I2C_CHARDEV=y
#
# I2C Algorithms
#
# CONFIG_I2C_ALGOBIT is not set
# CONFIG_I2C_ALGOPCF is not set
#
# I2C Hardware Bus support
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
# CONFIG_I2C_AMD756 is not set
CONFIG_I2C_AMD8111=y
# CONFIG_I2C_I801 is not set
# CONFIG_I2C_I810 is not set
CONFIG_I2C_ISA=y
# CONFIG_I2C_NFORCE2 is not set
# CONFIG_I2C_PARPORT is not set
# CONFIG_I2C_PARPORT_LIGHT is not set
# CONFIG_I2C_PROSAVAGE is not set
# CONFIG_I2C_SAVAGE4 is not set
# CONFIG_SCx200_ACB is not set
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not set
# CONFIG_I2C_VIA is not set
# CONFIG_I2C_VIAPRO is not set
# CONFIG_I2C_VOODOO3 is not set
#
# Hardware Sensors Chip support
#
CONFIG_I2C_SENSOR=y
# CONFIG_SENSORS_ADM1021 is not set
# CONFIG_SENSORS_ASB100 is not set
# CONFIG_SENSORS_DS1621 is not set
# CONFIG_SENSORS_FSCHER is not set
# CONFIG_SENSORS_GL518SM is not set
# CONFIG_SENSORS_IT87 is not set
# CONFIG_SENSORS_LM75 is not set
# CONFIG_SENSORS_LM78 is not set
# CONFIG_SENSORS_LM80 is not set
# CONFIG_SENSORS_LM83 is not set
CONFIG_SENSORS_LM85=y
# CONFIG_SENSORS_LM90 is not set
# CONFIG_SENSORS_MAX1619 is not set
# CONFIG_SENSORS_VIA686A is not set
CONFIG_SENSORS_W83781D=y
# CONFIG_SENSORS_W83L785TS is not set
# CONFIG_SENSORS_W83627HF is not set
#
# Other I2C Chip support
#
# CONFIG_SENSORS_EEPROM is not set
# CONFIG_SENSORS_PCF8574 is not set
# CONFIG_SENSORS_PCF8591 is not set
# CONFIG_SENSORS_RTC8564 is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set
#
# Misc devices
#
# CONFIG_IBM_ASM is not set
#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set
#
# Digital Video Broadcasting Devices
#
# CONFIG_DVB is not set
#
# Graphics support
#
# CONFIG_FB is not set
# CONFIG_VIDEO_SELECT is not set
#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
# CONFIG_MDA_CONSOLE is not set
CONFIG_DUMMY_CONSOLE=y
#
# Sound
#
# CONFIG_SOUND is not set
#
# USB support
#
CONFIG_USB=y
# CONFIG_USB_DEBUG is not set
#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
# CONFIG_USB_BANDWIDTH is not set
# CONFIG_USB_DYNAMIC_MINORS is not set
#
# USB Host Controller Drivers
#
CONFIG_USB_EHCI_HCD=y
# CONFIG_USB_EHCI_SPLIT_ISO is not set
# CONFIG_USB_EHCI_ROOT_HUB_TT is not set
# CONFIG_USB_OHCI_HCD is not set
CONFIG_USB_UHCI_HCD=y
#
# USB Device Class drivers
#
# CONFIG_USB_BLUETOOTH_TTY is not set
# CONFIG_USB_ACM is not set
# CONFIG_USB_PRINTER is not set
CONFIG_USB_STORAGE=y
# CONFIG_USB_STORAGE_DEBUG is not set
# CONFIG_USB_STORAGE_DATAFAB is not set
# CONFIG_USB_STORAGE_FREECOM is not set
# CONFIG_USB_STORAGE_DPCM is not set
# CONFIG_USB_STORAGE_HP8200e is not set
# CONFIG_USB_STORAGE_SDDR09 is not set
# CONFIG_USB_STORAGE_SDDR55 is not set
# CONFIG_USB_STORAGE_JUMPSHOT is not set
#
# USB Human Interface Devices (HID)
#
# CONFIG_USB_HID is not set
#
# USB HID Boot Protocol drivers
#
# CONFIG_USB_KBD is not set
# CONFIG_USB_MOUSE is not set
# CONFIG_USB_AIPTEK is not set
# CONFIG_USB_WACOM is not set
# CONFIG_USB_KBTAB is not set
# CONFIG_USB_POWERMATE is not set
# CONFIG_USB_MTOUCH is not set
# CONFIG_USB_EGALAX is not set
# CONFIG_USB_XPAD is not set
# CONFIG_USB_ATI_REMOTE is not set
#
# USB Imaging devices
#
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_MICROTEK is not set
# CONFIG_USB_HPUSBSCSI is not set
#
# USB Multimedia devices
#
# CONFIG_USB_DABUSB is not set
#
# Video4Linux support is needed for USB Multimedia device support
#
#
# USB Network adaptors
#
# CONFIG_USB_CATC is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
# CONFIG_USB_USBNET is not set
#
# USB port drivers
#
# CONFIG_USB_USS720 is not set
#
# USB Serial Converter support
#
# CONFIG_USB_SERIAL is not set
#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
# CONFIG_USB_TIGL is not set
# CONFIG_USB_AUERSWALD is not set
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
# CONFIG_USB_LED is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_PHIDGETSERVO is not set
# CONFIG_USB_TEST is not set
#
# USB Gadget Support
#
# CONFIG_USB_GADGET is not set
#
# Firmware Drivers
#
# CONFIG_EDD is not set
#
# File systems
#
CONFIG_EXT2_FS=y
# CONFIG_EXT2_FS_XATTR is not set
CONFIG_EXT3_FS=y
# CONFIG_EXT3_FS_XATTR is not set
CONFIG_JBD=y
# CONFIG_JBD_DEBUG is not set
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
# CONFIG_XFS_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_ROMFS_FS is not set
# CONFIG_QUOTA is not set
# CONFIG_AUTOFS_FS is not set
# CONFIG_AUTOFS4_FS is not set
#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
# CONFIG_ZISOFS is not set
# CONFIG_UDF_FS is not set
#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=y
CONFIG_MSDOS_FS=y
CONFIG_VFAT_FS=y
# CONFIG_NTFS_FS is not set
#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
# CONFIG_DEVFS_FS is not set
# CONFIG_DEVPTS_FS_XATTR is not set
CONFIG_TMPFS=y
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_RAMFS=y
#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
#
# Network File Systems
#
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
# CONFIG_NFS_V4 is not set
# CONFIG_NFS_DIRECTIO is not set
CONFIG_NFSD=y
CONFIG_NFSD_V3=y
# CONFIG_NFSD_V4 is not set
CONFIG_NFSD_TCP=y
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=y
CONFIG_SUNRPC=y
# CONFIG_RPCSEC_GSS_KRB5 is not set
# CONFIG_SMB_FS is not set
# CONFIG_CIFS is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set
#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y
#
# Native Language Support
#
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=y
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
# CONFIG_NLS_CODEPAGE_850 is not set
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
CONFIG_NLS_ISO8859_1=y
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
# CONFIG_NLS_ISO8859_15 is not set
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
CONFIG_NLS_UTF8=y
#
# Profiling support
#
# CONFIG_PROFILING is not set
#
# Kernel hacking
#
CONFIG_DEBUG_KERNEL=y
# CONFIG_DEBUG_SLAB is not set
CONFIG_MAGIC_SYSRQ=y
CONFIG_DEBUG_SPINLOCK=y
# CONFIG_INIT_DEBUG is not set
# CONFIG_DEBUG_INFO is not set
# CONFIG_FRAME_POINTER is not set
# CONFIG_IOMMU_DEBUG is not set
#
# Security options
#
# CONFIG_SECURITY is not set
#
# Cryptographic options
#
# CONFIG_CRYPTO is not set
#
# Library routines
#
# CONFIG_CRC32 is not set
# CONFIG_LIBCRC32C is not set
Bootdata ok (command line is BOOT_IMAGE=Linux ro root=851 console=ttyS0,19200n8)
Linux version 2.6.7-rc3-bk4 (root@carignan) (gcc version 3.3.2 20030908 (Debian prerelease)) #6 SMP Sun Jun 13 00:37:31 EDT 2004
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000007fff0000 (usable)
BIOS-e820: 000000007fff0000 - 000000007ffff000 (ACPI data)
BIOS-e820: 000000007ffff000 - 0000000080000000 (ACPI NVS)
BIOS-e820: 00000000ff7c0000 - 0000000100000000 (reserved)
Scanning NUMA topology in Northbridge 24
Number of nodes 2 (10010)
Node 0 MemBase 0000000000000000 Limit 000000003fffffff
Node 1 MemBase 0000000040000000 Limit 000000007fff0000
Using node hash shift of 24
Bootmem setup node 0 0000000000000000-000000003fffffff
Bootmem setup node 1 0000000040000000-000000007fff0000
No mptable found.
On node 0 totalpages: 262143
DMA zone: 4096 pages, LIFO batch:1
Normal zone: 258047 pages, LIFO batch:16
HighMem zone: 0 pages, LIFO batch:1
On node 1 totalpages: 262128
DMA zone: 0 pages, LIFO batch:1
Normal zone: 262128 pages, LIFO batch:16
HighMem zone: 0 pages, LIFO batch:1
ACPI: RSDP (v002 ACPIAM ) @ 0x00000000000f4680
ACPI: XSDT (v001 A M I OEMXSDT 0x06000318 MSFT 0x00000097) @ 0x000000007fff0100
ACPI: FADT (v001 A M I OEMFACP 0x06000318 MSFT 0x00000097) @ 0x000000007fff0281
ACPI: MADT (v001 A M I OEMAPIC 0x06000318 MSFT 0x00000097) @ 0x000000007fff0380
ACPI: OEMB (v001 A M I OEMBIOS 0x06000318 MSFT 0x00000097) @ 0x000000007ffff040
ACPI: ASF! (v001 AMIASF AMDSTRET 0x00000001 INTL 0x02002026) @ 0x000000007fff3530
ACPI: DSDT (v001 0ABCF 0ABCF007 0x00000007 INTL 0x02002026) @ 0x0000000000000000
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Processor #0 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
Processor #1 15:5 APIC version 16
ACPI: IOAPIC (id[0x02] address[0xfec00000] global_irq_base[0x0])
IOAPIC[0]: Assigned apic_id 2
IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x03] address[0xfebfe000] global_irq_base[0x18])
IOAPIC[1]: Assigned apic_id 3
IOAPIC[1]: apic_id 3, version 17, address 0xfebfe000, GSI 24-27
ACPI: IOAPIC (id[0x04] address[0xfebff000] global_irq_base[0x1c])
IOAPIC[2]: Assigned apic_id 4
IOAPIC[2]: apic_id 4, version 17, address 0xfebff000, GSI 28-31
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
Checking aperture...
CPU 0: aperture @ 1ee0000000 size 64 MB
Aperture from northbridge cpu 0 beyond 4GB. Ignoring.
No AGP bridge found
Built 2 zonelists
Kernel command line: BOOT_IMAGE=Linux ro root=851 console=ttyS0,19200n8
Initializing CPU#0
PID hash table entries: 16 (order 4: 256 bytes)
time.c: Using 1.193182 MHz PIT timer.
time.c: Detected 1394.659 MHz processor.
Console: colour VGA+ 80x25
Memory: 2062444k/2097088k available (2625k kernel code, 0k reserved, 1205k data, 180k init)
Calibrating delay loop... 2736.12 BogoMIPS
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Mount-cache hash table entries: 256 (order: 0, 4096 bytes)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
Using local APIC NMI watchdog using perfctr0
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU0: AMD Opteron(tm) Processor 240 stepping 01
per-CPU timeslice cutoff: 1023.75 usecs.
task migration cache decay timeout: 2 msecs.
Booting processor 1/1 rip 6000 rsp 10040e3ff58
Initializing CPU#1
Calibrating delay loop... 2785.28 BogoMIPS
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
AMD Opteron(tm) Processor 240 stepping 01
Total of 2 processors activated (5521.40 BogoMIPS).
ENABLING IO-APIC IRQs
init IO_APIC IRQs
IO-APIC (apicid-pin) 2-0, 2-16, 2-17, 2-18, 2-19, 2-20, 2-21, 2-22, 2-23, 3-0, 3-1, 3-2, 3-3, 4-0, 4-1, 4-2, 4-3 not connected.
..TIMER: vector=0x31 pin1=2 pin2=-1
Using local APIC timer interrupts.
Detected 12.452 MHz APIC timer.
checking TSC synchronization across 2 CPUs: passed.
time.c: Using PIT/TSC based timekeeping.
Brought up 2 CPUs
CPU0: online
domain 0: span 01
groups: 01
domain 1: span 03
groups: 01 02
CPU1: online
domain 0: span 02
groups: 02
domain 1: span 03
groups: 02 01
NET: Registered protocol family 16
PCI: Using configuration type 1
mtrr: v2.0 (20020519)
ACPI: Subsystem revision 20040326
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (00:00)
PCI: Probing PCI hardware (bus 00)
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PCI1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.GOLA._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.GOLB._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 *5 6 7 9 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 *9 10 11 12 14 15)
SCSI subsystem initialized
usbcore: registered new driver usbfs
usbcore: registered new driver hub
IOAPIC[0]: Set PCI routing entry (2-16 -> 0xa9 -> IRQ 16 Mode:1 Active:1)
00:00:07[A] -> 2-16 -> IRQ 16
IOAPIC[0]: Set PCI routing entry (2-17 -> 0xb1 -> IRQ 17 Mode:1 Active:1)
00:00:07[B] -> 2-17 -> IRQ 17
IOAPIC[0]: Set PCI routing entry (2-18 -> 0xb9 -> IRQ 18 Mode:1 Active:1)
00:00:07[C] -> 2-18 -> IRQ 18
IOAPIC[0]: Set PCI routing entry (2-19 -> 0xc1 -> IRQ 19 Mode:1 Active:1)
00:00:07[D] -> 2-19 -> IRQ 19
IOAPIC[1]: Set PCI routing entry (3-3 -> 0xc9 -> IRQ 27 Mode:1 Active:1)
00:02:08[A] -> 3-3 -> IRQ 27
IOAPIC[1]: Set PCI routing entry (3-0 -> 0xd1 -> IRQ 24 Mode:1 Active:1)
00:02:08[B] -> 3-0 -> IRQ 24
IOAPIC[1]: Set PCI routing entry (3-1 -> 0xd9 -> IRQ 25 Mode:1 Active:1)
00:02:08[C] -> 3-1 -> IRQ 25
IOAPIC[1]: Set PCI routing entry (3-2 -> 0xe1 -> IRQ 26 Mode:1 Active:1)
00:02:08[D] -> 3-2 -> IRQ 26
IOAPIC[2]: Set PCI routing entry (4-0 -> 0xe9 -> IRQ 28 Mode:1 Active:1)
00:01:03[A] -> 4-0 -> IRQ 28
IOAPIC[2]: Set PCI routing entry (4-1 -> 0x32 -> IRQ 29 Mode:1 Active:1)
00:01:03[B] -> 4-1 -> IRQ 29
IOAPIC[2]: Set PCI routing entry (4-2 -> 0x3a -> IRQ 30 Mode:1 Active:1)
00:01:03[C] -> 4-2 -> IRQ 30
IOAPIC[2]: Set PCI routing entry (4-3 -> 0x42 -> IRQ 31 Mode:1 Active:1)
00:01:03[D] -> 4-3 -> IRQ 31
number of MP IRQ sources: 16.
number of IO-APIC #2 registers: 24.
number of IO-APIC #3 registers: 4.
number of IO-APIC #4 registers: 4.
testing the IO APIC.......................
IO APIC #2......
.... register #00: 02000000
....... : physical APIC id: 02
.... register #01: 00170011
....... : max redirection entries: 0017
....... : PRQ implemented: 0
....... : IO APIC version: 0011
.... register #02: 02000000
....... : arbitration: 02
.... IRQ redirection table:
NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
00 000 00 1 0 0 0 0 0 0 00
01 001 01 0 0 0 0 0 1 1 39
02 001 01 0 0 0 0 0 1 1 31
03 001 01 0 0 0 0 0 1 1 41
04 001 01 0 0 0 0 0 1 1 49
05 001 01 0 0 0 0 0 1 1 51
06 001 01 0 0 0 0 0 1 1 59
07 001 01 0 0 0 0 0 1 1 61
08 001 01 0 0 0 0 0 1 1 69
09 001 01 0 1 0 1 0 1 1 71
0a 001 01 0 0 0 0 0 1 1 79
0b 001 01 0 0 0 0 0 1 1 81
0c 001 01 0 0 0 0 0 1 1 89
0d 001 01 0 0 0 0 0 1 1 91
0e 001 01 0 0 0 0 0 1 1 99
0f 001 01 0 0 0 0 0 1 1 A1
10 001 01 1 1 0 1 0 1 1 A9
11 001 01 1 1 0 1 0 1 1 B1
12 001 01 1 1 0 1 0 1 1 B9
13 001 01 1 1 0 1 0 1 1 C1
14 000 00 1 0 0 0 0 0 0 00
15 000 00 1 0 0 0 0 0 0 00
16 000 00 1 0 0 0 0 0 0 00
17 000 00 1 0 0 0 0 0 0 00
IO APIC #3......
.... register #00: 03000000
....... : physical APIC id: 03
.... register #01: 00030011
....... : max redirection entries: 0003
....... : PRQ implemented: 0
....... : IO APIC version: 0011
.... register #02: 00000000
....... : arbitration: 00
.... IRQ redirection table:
NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
00 001 01 1 1 0 1 0 1 1 D1
01 001 01 1 1 0 1 0 1 1 D9
02 001 01 1 1 0 1 0 1 1 E1
03 001 01 1 1 0 1 0 1 1 C9
IO APIC #4......
.... register #00: 04000000
....... : physical APIC id: 04
.... register #01: 00030011
....... : max redirection entries: 0003
....... : PRQ implemented: 0
....... : IO APIC version: 0011
.... register #02: 00000000
....... : arbitration: 00
.... IRQ redirection table:
NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
00 001 01 1 1 0 1 0 1 1 E9
01 001 01 1 1 0 1 0 1 1 32
02 001 01 1 1 0 1 0 1 1 3A
03 001 01 1 1 0 1 0 1 1 42
IRQ to pin mappings:
IRQ0 -> 0:2
IRQ1 -> 0:1
IRQ3 -> 0:3
IRQ4 -> 0:4
IRQ5 -> 0:5
IRQ6 -> 0:6
IRQ7 -> 0:7
IRQ8 -> 0:8
IRQ9 -> 0:9
IRQ10 -> 0:10
IRQ11 -> 0:11
IRQ12 -> 0:12
IRQ13 -> 0:13
IRQ14 -> 0:14
IRQ15 -> 0:15
IRQ16 -> 0:16
IRQ17 -> 0:17
IRQ18 -> 0:18
IRQ19 -> 0:19
IRQ24 -> 1:0
IRQ25 -> 1:1
IRQ26 -> 1:2
IRQ27 -> 1:3
IRQ28 -> 2:0
IRQ29 -> 2:1
IRQ30 -> 2:2
IRQ31 -> 2:3
.................................... done.
PCI: Using ACPI for IRQ routing
PCI-DMA: Disabling IOMMU.
IA-32 Microcode Update Driver: v1.13 <tigran@veritas.com>
IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak Exp $
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
ACPI: Power Button (FF) [PWRF]
ACPI: Processor [CPU1] (supports C1, 8 throttling states)
ACPI: Processor [CPU2] (supports C1)
lp: driver loaded but no devices found
Real Time Clock Driver v1.12
hw_random: AMD768 system management I/O registers at 0x5000.
hw_random hardware driver 1.0.0 loaded
Software Watchdog Timer: 0.07 initialized. soft_noboot=0 soft_margin=60 sec (nowayout= 0)
Linux agpgart interface v0.100 (c) Dave Jones
Hangcheck: starting hangcheck timer 0.5.0 (tick is 180 seconds, margin is 60 seconds).
Serial: 8250/16550 driver $Revision: 1.90 $ 8 ports, IRQ sharing disabled
ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
parport0: PC-style at 0x378 [PCSPP(,...)]
lp0: using parport0 (polling).
Using anticipatory io scheduler
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
loop: loaded (max 8 devices)
tg3.c:v3.5 (May 25, 2004)
eth0: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:51:d6:c9
eth0: HostTXDS[0] RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1]
eth1: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:51:d6:ca
eth1: HostTXDS[0] RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1]
sym0: <875> rev 0x26 at pci 0000:03:04.0 irq 16
sym0: Symbios NVRAM, ID 7, Fast-20, SE, parity checking
sym0: open drain IRQ line driver, using on-chip SRAM
sym0: using LOAD/STORE-based firmware.
sym0: SCSI BUS has been reset.
scsi0 : sym-2.1.18j
Vendor: SONY Model: SDX-400C Rev: 0702
Type: Sequential-Access ANSI SCSI revision: 02
scsi(0:0:4:0): Beginning Domain Validation
sym0:4: wide asynchronous.
sym0:4: FAST-20 WIDE SCSI 40.0 MB/s ST (50.0 ns, offset 15)
scsi(0:0:4:0): Domain Validation skipping write tests
scsi(0:0:4:0): Ending Domain Validation
Vendor: OVERLAND Model: LIBRARYPRO Rev: 0417
Type: Medium Changer ANSI SCSI revision: 02
scsi(0:0:6:0): Beginning Domain Validation
sym0:6: wide asynchronous.
sym0:6: FAST-10 WIDE SCSI 20.0 MB/s ST (100.0 ns, offset 15)
scsi(0:0:6:0): Domain Validation skipping write tests
scsi(0:0:6:0): Ending Domain Validation
st: Version 20040403, fixed bufsize 32768, s/g segs 256
Attached scsi tape st0 at scsi0, channel 0, id 4, lun 0
st0: try direct i/o: yes (alignment 512 B), max page reachable by HBA 1048575
Attached scsi generic sg0 at scsi0, channel 0, id 4, lun 0, type 1
Attached scsi generic sg1 at scsi0, channel 0, id 6, lun 0, type 8
Fusion MPT base driver 3.01.06
Copyright (c) 1999-2004 LSI Logic Corporation
mptbase: Initiating ioc0 bringup
ioc0: 53C1030: Capabilities={Initiator}
mptbase: Initiating ioc1 bringup
ioc1: 53C1030: Capabilities={Initiator}
Fusion MPT SCSI Host driver 3.01.06
scsi1 : ioc0: LSI53C1030, FwRev=01030600h, Ports=1, MaxQ=255, IRQ=24
Vendor: SUPER Model: GEM318 Rev: 0
Type: Processor ANSI SCSI revision: 02
Attached scsi generic sg2 at scsi1, channel 0, id 8, lun 0, type 3
Vendor: SEAGATE Model: ST3146807LC Rev: 0006
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sda: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sda: drive cache: write back
sda: sda1
Attached scsi disk sda at scsi1, channel 0, id 9, lun 0
Attached scsi generic sg3 at scsi1, channel 0, id 9, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0006
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdb: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sdb: drive cache: write back
sdb: sdb1
Attached scsi disk sdb at scsi1, channel 0, id 10, lun 0
Attached scsi generic sg4 at scsi1, channel 0, id 10, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0006
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdc: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sdc: drive cache: write back
sdc: sdc1
Attached scsi disk sdc at scsi1, channel 0, id 11, lun 0
Attached scsi generic sg5 at scsi1, channel 0, id 11, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0007
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdd: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sdd: drive cache: write back
sdd: sdd1
Attached scsi disk sdd at scsi1, channel 0, id 12, lun 0
Attached scsi generic sg6 at scsi1, channel 0, id 12, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0007
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sde: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sde: drive cache: write back
sde: sde1
Attached scsi disk sde at scsi1, channel 0, id 13, lun 0
Attached scsi generic sg7 at scsi1, channel 0, id 13, lun 0, type 0
scsi2 : ioc1: LSI53C1030, FwRev=01030600h, Ports=1, MaxQ=255, IRQ=25
Vendor: SEAGATE Model: ST336607LW Rev: 0007
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdf: 71687372 512-byte hdwr sectors (36704 MB)
SCSI device sdf: drive cache: write back
sdf: sdf1 sdf2 sdf3 sdf4
Attached scsi disk sdf at scsi2, channel 0, id 0, lun 0
Attached scsi generic sg8 at scsi2, channel 0, id 0, lun 0, type 0
USB Universal Host Controller Interface driver v2.2
Initializing USB Mass Storage driver...
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
mice: PS/2 mouse device common for all mice
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
i2c /dev entries driver
md: raid1 personality registered as nr 3
md: raid5 personality registered as nr 4
raid5: measuring checksumming speed
generic_sse: 4256.000 MB/sec
raid5: using function: generic_sse (4256.000 MB/sec)
md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
device-mapper: 4.1.0-ioctl (2003-12-10) initialised: dm@uk.sistina.com
NET: Registered protocol family 2
IP: routing cache hash table of 8192 buckets, 128Kbytes
TCP: Hash tables configured (established 262144 bind 65536)
NET: Registered protocol family 1
NET: Registered protocol family 17
md: Autodetecting RAID arrays.
md: autorun ...
md: considering sde1 ...
md: adding sde1 ...
md: adding sdd1 ...
md: adding sdc1 ...
md: adding sdb1 ...
md: adding sda1 ...
md: created md0
md: bind<sda1>
md: bind<sdb1>
md: bind<sdc1>
md: bind<sdd1>
md: bind<sde1>
md: running: <sde1><sdd1><sdc1><sdb1><sda1>
raid5: device sde1 operational as raid disk 4
raid5: device sdd1 operational as raid disk 3
raid5: device sdc1 operational as raid disk 2
raid5: device sdb1 operational as raid disk 1
raid5: device sda1 operational as raid disk 0
raid5: allocated 5300kB for md0
raid5: raid level 5 set md0 active with 5 out of 5 devices, algorithm 2
RAID5 conf printout:
--- rd:5 wd:5 fd:0
disk 0, o:1, dev:sda1
disk 1, o:1, dev:sdb1
disk 2, o:1, dev:sdc1
disk 3, o:1, dev:sdd1
disk 4, o:1, dev:sde1
md: ... autorun DONE.
kjournald starting. Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 180k freed
Adding 2097136k swap on /dev/sdf2. Priority:-1 extents:1
EXT3 FS on sdf1, internal journal
kjournald starting. Commit interval 5 seconds
EXT3 FS on sdf3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-32, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-34, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-33, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-35, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-0, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-4, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-5, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-6, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-7, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-8, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-9, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-10, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-11, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-12, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-13, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-14, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-15, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-16, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-17, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-18, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-19, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-20, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-21, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-22, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-23, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-24, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-25, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-26, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-27, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-28, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-29, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-30, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-31, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
Ethernet Channel Bonding Driver: v2.6.0 (January 14, 2004)
bonding: MII link monitoring set to 100 ms
bonding: bond0: enslaving eth0 as an active interface with a down link.
bonding: bond0: enslaving eth1 as an active interface with a down link.
tg3: eth0: Link is up at 1000 Mbps, full duplex.
tg3: eth0: Flow control is off for TX and off for RX.
bonding: bond0: link status definitely up for interface eth0.
tg3: eth1: Link is up at 1000 Mbps, full duplex.
tg3: eth1: Flow control is off for TX and off for RX.
bonding: bond0: link status definitely up for interface eth1.
nfs warning: mount version older than kernel
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-15 6:22 processes hung in D (raid5/dm/ext3) foo
@ 2004-06-15 10:09 ` Andrew Morton
2004-06-15 15:00 ` foo
0 siblings, 1 reply; 19+ messages in thread
From: Andrew Morton @ 2004-06-15 10:09 UTC (permalink / raw)
To: foo; +Cc: linux-kernel
foo@porto.bmb.uga.edu wrote:
>
> I'm getting processes hung in the D state accessing the mount point of
> the home directories on my file server
Wait for it to happen again, then do
echo t > /proc/sysrq-trigger
dmesg -s 1000000 > foo
then send foo, foo.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-15 10:09 ` Andrew Morton
@ 2004-06-15 15:00 ` foo
2004-06-15 23:26 ` Andrew Morton
0 siblings, 1 reply; 19+ messages in thread
From: foo @ 2004-06-15 15:00 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel
On Tue, Jun 15, 2004 at 03:09:32AM -0700, Andrew Morton wrote:
> Wait for it to happen again, then do
>
> echo t > /proc/sysrq-trigger
> dmesg -s 1000000 > foo
>
> then send foo, foo.
Here you go. I did dump 2f /dev/null /export/home. The first time it
completed, the second it hung right away.
This is with 2.6.7-rc3-bk6 with no other patches, also compiled with a
newer gcc.
-ryan
Bootdata ok (command line is BOOT_IMAGE=Linux ro root=851 console=ttyS0,19200n8)
Linux version 2.6.7-rc3-bk6 (root@carignan) (gcc version 3.3.4 (Debian)) #1 SMP Tue Jun 15 02:35:08 EDT 2004
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000007fff0000 (usable)
BIOS-e820: 000000007fff0000 - 000000007ffff000 (ACPI data)
BIOS-e820: 000000007ffff000 - 0000000080000000 (ACPI NVS)
BIOS-e820: 00000000ff7c0000 - 0000000100000000 (reserved)
Scanning NUMA topology in Northbridge 24
Number of nodes 2 (10010)
Node 0 MemBase 0000000000000000 Limit 000000003fffffff
Node 1 MemBase 0000000040000000 Limit 000000007fff0000
Using node hash shift of 24
Bootmem setup node 0 0000000000000000-000000003fffffff
Bootmem setup node 1 0000000040000000-000000007fff0000
No mptable found.
On node 0 totalpages: 262143
DMA zone: 4096 pages, LIFO batch:1
Normal zone: 258047 pages, LIFO batch:16
HighMem zone: 0 pages, LIFO batch:1
On node 1 totalpages: 262128
DMA zone: 0 pages, LIFO batch:1
Normal zone: 262128 pages, LIFO batch:16
HighMem zone: 0 pages, LIFO batch:1
ACPI: RSDP (v002 ACPIAM ) @ 0x00000000000f4680
ACPI: XSDT (v001 A M I OEMXSDT 0x06000318 MSFT 0x00000097) @ 0x000000007fff0100
ACPI: FADT (v001 A M I OEMFACP 0x06000318 MSFT 0x00000097) @ 0x000000007fff0281
ACPI: MADT (v001 A M I OEMAPIC 0x06000318 MSFT 0x00000097) @ 0x000000007fff0380
ACPI: OEMB (v001 A M I OEMBIOS 0x06000318 MSFT 0x00000097) @ 0x000000007ffff040
ACPI: ASF! (v001 AMIASF AMDSTRET 0x00000001 INTL 0x02002026) @ 0x000000007fff3530
ACPI: DSDT (v001 0ABCF 0ABCF007 0x00000007 INTL 0x02002026) @ 0x0000000000000000
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Processor #0 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
Processor #1 15:5 APIC version 16
ACPI: IOAPIC (id[0x02] address[0xfec00000] global_irq_base[0x0])
IOAPIC[0]: Assigned apic_id 2
IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x03] address[0xfebfe000] global_irq_base[0x18])
IOAPIC[1]: Assigned apic_id 3
IOAPIC[1]: apic_id 3, version 17, address 0xfebfe000, GSI 24-27
ACPI: IOAPIC (id[0x04] address[0xfebff000] global_irq_base[0x1c])
IOAPIC[2]: Assigned apic_id 4
IOAPIC[2]: apic_id 4, version 17, address 0xfebff000, GSI 28-31
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
Checking aperture...
CPU 0: aperture @ 1ee0000000 size 64 MB
Aperture from northbridge cpu 0 beyond 4GB. Ignoring.
No AGP bridge found
Built 2 zonelists
Kernel command line: BOOT_IMAGE=Linux ro root=851 console=ttyS0,19200n8
Initializing CPU#0
PID hash table entries: 16 (order 4: 256 bytes)
time.c: Using 1.193182 MHz PIT timer.
time.c: Detected 1394.765 MHz processor.
Console: colour VGA+ 80x25
Memory: 2062532k/2097088k available (2540k kernel code, 0k reserved, 1204k data, 180k init)
Calibrating delay loop... 2736.12 BogoMIPS
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Mount-cache hash table entries: 256 (order: 0, 4096 bytes)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
Using local APIC NMI watchdog using perfctr0
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU0: AMD Opteron(tm) Processor 240 stepping 01
per-CPU timeslice cutoff: 1024.50 usecs.
task migration cache decay timeout: 2 msecs.
Booting processor 1/1 rip 6000 rsp 10040e3ff58
Initializing CPU#1
Calibrating delay loop... 2785.28 BogoMIPS
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
AMD Opteron(tm) Processor 240 stepping 01
Total of 2 processors activated (5521.40 BogoMIPS).
ENABLING IO-APIC IRQs
init IO_APIC IRQs
IO-APIC (apicid-pin) 2-0, 2-16, 2-17, 2-18, 2-19, 2-20, 2-21, 2-22, 2-23, 3-0, 3-1, 3-2, 3-3, 4-0, 4-1, 4-2, 4-3 not connected.
..TIMER: vector=0x31 pin1=2 pin2=-1
Using local APIC timer interrupts.
Detected 12.453 MHz APIC timer.
checking TSC synchronization across 2 CPUs: passed.
time.c: Using PIT/TSC based timekeeping.
Brought up 2 CPUs
CPU0: online
domain 0: span 01
groups: 01
domain 1: span 03
groups: 01 02
CPU1: online
domain 0: span 02
groups: 02
domain 1: span 03
groups: 02 01
NET: Registered protocol family 16
PCI: Using configuration type 1
mtrr: v2.0 (20020519)
ACPI: Subsystem revision 20040326
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (00:00)
PCI: Probing PCI hardware (bus 00)
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PCI1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.GOLA._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.GOLB._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 *5 6 7 9 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 *9 10 11 12 14 15)
SCSI subsystem initialized
usbcore: registered new driver usbfs
usbcore: registered new driver hub
IOAPIC[0]: Set PCI routing entry (2-16 -> 0xa9 -> IRQ 16 Mode:1 Active:1)
00:00:07[A] -> 2-16 -> IRQ 16
IOAPIC[0]: Set PCI routing entry (2-17 -> 0xb1 -> IRQ 17 Mode:1 Active:1)
00:00:07[B] -> 2-17 -> IRQ 17
IOAPIC[0]: Set PCI routing entry (2-18 -> 0xb9 -> IRQ 18 Mode:1 Active:1)
00:00:07[C] -> 2-18 -> IRQ 18
IOAPIC[0]: Set PCI routing entry (2-19 -> 0xc1 -> IRQ 19 Mode:1 Active:1)
00:00:07[D] -> 2-19 -> IRQ 19
IOAPIC[1]: Set PCI routing entry (3-3 -> 0xc9 -> IRQ 27 Mode:1 Active:1)
00:02:08[A] -> 3-3 -> IRQ 27
IOAPIC[1]: Set PCI routing entry (3-0 -> 0xd1 -> IRQ 24 Mode:1 Active:1)
00:02:08[B] -> 3-0 -> IRQ 24
IOAPIC[1]: Set PCI routing entry (3-1 -> 0xd9 -> IRQ 25 Mode:1 Active:1)
00:02:08[C] -> 3-1 -> IRQ 25
IOAPIC[1]: Set PCI routing entry (3-2 -> 0xe1 -> IRQ 26 Mode:1 Active:1)
00:02:08[D] -> 3-2 -> IRQ 26
IOAPIC[2]: Set PCI routing entry (4-0 -> 0xe9 -> IRQ 28 Mode:1 Active:1)
00:01:03[A] -> 4-0 -> IRQ 28
IOAPIC[2]: Set PCI routing entry (4-1 -> 0x32 -> IRQ 29 Mode:1 Active:1)
00:01:03[B] -> 4-1 -> IRQ 29
IOAPIC[2]: Set PCI routing entry (4-2 -> 0x3a -> IRQ 30 Mode:1 Active:1)
00:01:03[C] -> 4-2 -> IRQ 30
IOAPIC[2]: Set PCI routing entry (4-3 -> 0x42 -> IRQ 31 Mode:1 Active:1)
00:01:03[D] -> 4-3 -> IRQ 31
number of MP IRQ sources: 16.
number of IO-APIC #2 registers: 24.
number of IO-APIC #3 registers: 4.
number of IO-APIC #4 registers: 4.
testing the IO APIC.......................
IO APIC #2......
.... register #00: 02000000
....... : physical APIC id: 02
.... register #01: 00170011
....... : max redirection entries: 0017
....... : PRQ implemented: 0
....... : IO APIC version: 0011
.... register #02: 02000000
....... : arbitration: 02
.... IRQ redirection table:
NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
00 000 00 1 0 0 0 0 0 0 00
01 001 01 0 0 0 0 0 1 1 39
02 001 01 0 0 0 0 0 1 1 31
03 001 01 0 0 0 0 0 1 1 41
04 001 01 0 0 0 0 0 1 1 49
05 001 01 0 0 0 0 0 1 1 51
06 001 01 0 0 0 0 0 1 1 59
07 001 01 0 0 0 0 0 1 1 61
08 001 01 0 0 0 0 0 1 1 69
09 001 01 0 1 0 1 0 1 1 71
0a 001 01 0 0 0 0 0 1 1 79
0b 001 01 0 0 0 0 0 1 1 81
0c 001 01 0 0 0 0 0 1 1 89
0d 001 01 0 0 0 0 0 1 1 91
0e 001 01 0 0 0 0 0 1 1 99
0f 001 01 0 0 0 0 0 1 1 A1
10 001 01 1 1 0 1 0 1 1 A9
11 001 01 1 1 0 1 0 1 1 B1
12 001 01 1 1 0 1 0 1 1 B9
13 001 01 1 1 0 1 0 1 1 C1
14 000 00 1 0 0 0 0 0 0 00
15 000 00 1 0 0 0 0 0 0 00
16 000 00 1 0 0 0 0 0 0 00
17 000 00 1 0 0 0 0 0 0 00
IO APIC #3......
.... register #00: 03000000
....... : physical APIC id: 03
.... register #01: 00030011
....... : max redirection entries: 0003
....... : PRQ implemented: 0
....... : IO APIC version: 0011
.... register #02: 00000000
....... : arbitration: 00
.... IRQ redirection table:
NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
00 001 01 1 1 0 1 0 1 1 D1
01 001 01 1 1 0 1 0 1 1 D9
02 001 01 1 1 0 1 0 1 1 E1
03 001 01 1 1 0 1 0 1 1 C9
IO APIC #4......
.... register #00: 04000000
....... : physical APIC id: 04
.... register #01: 00030011
....... : max redirection entries: 0003
....... : PRQ implemented: 0
....... : IO APIC version: 0011
.... register #02: 00000000
....... : arbitration: 00
.... IRQ redirection table:
NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
00 001 01 1 1 0 1 0 1 1 E9
01 001 01 1 1 0 1 0 1 1 32
02 001 01 1 1 0 1 0 1 1 3A
03 001 01 1 1 0 1 0 1 1 42
IRQ to pin mappings:
IRQ0 -> 0:2
IRQ1 -> 0:1
IRQ3 -> 0:3
IRQ4 -> 0:4
IRQ5 -> 0:5
IRQ6 -> 0:6
IRQ7 -> 0:7
IRQ8 -> 0:8
IRQ9 -> 0:9
IRQ10 -> 0:10
IRQ11 -> 0:11
IRQ12 -> 0:12
IRQ13 -> 0:13
IRQ14 -> 0:14
IRQ15 -> 0:15
IRQ16 -> 0:16
IRQ17 -> 0:17
IRQ18 -> 0:18
IRQ19 -> 0:19
IRQ24 -> 1:0
IRQ25 -> 1:1
IRQ26 -> 1:2
IRQ27 -> 1:3
IRQ28 -> 2:0
IRQ29 -> 2:1
IRQ30 -> 2:2
IRQ31 -> 2:3
.................................... done.
PCI: Using ACPI for IRQ routing
PCI-DMA: Disabling IOMMU.
IA-32 Microcode Update Driver: v1.13 <tigran@veritas.com>
IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak Exp $
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
ACPI: Power Button (FF) [PWRF]
ACPI: Processor [CPU1] (supports C1, 8 throttling states)
ACPI: Processor [CPU2] (supports C1)
lp: driver loaded but no devices found
Real Time Clock Driver v1.12
hw_random: AMD768 system management I/O registers at 0x5000.
hw_random hardware driver 1.0.0 loaded
Software Watchdog Timer: 0.07 initialized. soft_noboot=0 soft_margin=60 sec (nowayout= 0)
Linux agpgart interface v0.100 (c) Dave Jones
Hangcheck: starting hangcheck timer 0.5.0 (tick is 180 seconds, margin is 60 seconds).
Serial: 8250/16550 driver $Revision: 1.90 $ 8 ports, IRQ sharing disabled
ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
parport0: PC-style at 0x378 [PCSPP(,...)]
lp0: using parport0 (polling).
Using anticipatory io scheduler
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
loop: loaded (max 8 devices)
tg3.c:v3.5 (May 25, 2004)
eth0: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:51:d6:c9
eth0: HostTXDS[0] RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1]
eth1: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:51:d6:ca
eth1: HostTXDS[0] RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1]
sym0: <875> rev 0x26 at pci 0000:03:04.0 irq 16
sym0: Symbios NVRAM, ID 7, Fast-20, SE, parity checking
sym0: open drain IRQ line driver, using on-chip SRAM
sym0: using LOAD/STORE-based firmware.
sym0: SCSI BUS has been reset.
scsi0 : sym-2.1.18j
Vendor: SONY Model: SDX-400C Rev: 0702
Type: Sequential-Access ANSI SCSI revision: 02
scsi(0:0:4:0): Beginning Domain Validation
sym0:4: wide asynchronous.
sym0:4: FAST-20 WIDE SCSI 40.0 MB/s ST (50.0 ns, offset 15)
scsi(0:0:4:0): Domain Validation skipping write tests
scsi(0:0:4:0): Ending Domain Validation
Vendor: OVERLAND Model: LIBRARYPRO Rev: 0417
Type: Medium Changer ANSI SCSI revision: 02
scsi(0:0:6:0): Beginning Domain Validation
sym0:6: wide asynchronous.
sym0:6: FAST-10 WIDE SCSI 20.0 MB/s ST (100.0 ns, offset 15)
scsi(0:0:6:0): Domain Validation skipping write tests
scsi(0:0:6:0): Ending Domain Validation
st: Version 20040403, fixed bufsize 32768, s/g segs 256
Attached scsi tape st0 at scsi0, channel 0, id 4, lun 0
st0: try direct i/o: yes (alignment 512 B), max page reachable by HBA 1048575
Attached scsi generic sg0 at scsi0, channel 0, id 4, lun 0, type 1
Attached scsi generic sg1 at scsi0, channel 0, id 6, lun 0, type 8
Fusion MPT base driver 3.01.06
Copyright (c) 1999-2004 LSI Logic Corporation
mptbase: Initiating ioc0 bringup
ioc0: 53C1030: Capabilities={Initiator}
mptbase: Initiating ioc1 bringup
ioc1: 53C1030: Capabilities={Initiator}
Fusion MPT SCSI Host driver 3.01.06
scsi1 : ioc0: LSI53C1030, FwRev=01030600h, Ports=1, MaxQ=255, IRQ=24
Vendor: SUPER Model: GEM318 Rev: 0
Type: Processor ANSI SCSI revision: 02
Attached scsi generic sg2 at scsi1, channel 0, id 8, lun 0, type 3
Vendor: SEAGATE Model: ST3146807LC Rev: 0006
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sda: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sda: drive cache: write back
sda: sda1
Attached scsi disk sda at scsi1, channel 0, id 9, lun 0
Attached scsi generic sg3 at scsi1, channel 0, id 9, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0006
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdb: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sdb: drive cache: write back
sdb: sdb1
Attached scsi disk sdb at scsi1, channel 0, id 10, lun 0
Attached scsi generic sg4 at scsi1, channel 0, id 10, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0006
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdc: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sdc: drive cache: write back
sdc: sdc1
Attached scsi disk sdc at scsi1, channel 0, id 11, lun 0
Attached scsi generic sg5 at scsi1, channel 0, id 11, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0007
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdd: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sdd: drive cache: write back
sdd: sdd1
Attached scsi disk sdd at scsi1, channel 0, id 12, lun 0
Attached scsi generic sg6 at scsi1, channel 0, id 12, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0007
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sde: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sde: drive cache: write back
sde: sde1
Attached scsi disk sde at scsi1, channel 0, id 13, lun 0
Attached scsi generic sg7 at scsi1, channel 0, id 13, lun 0, type 0
scsi2 : ioc1: LSI53C1030, FwRev=01030600h, Ports=1, MaxQ=255, IRQ=25
Vendor: SEAGATE Model: ST336607LW Rev: 0007
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdf: 71687372 512-byte hdwr sectors (36704 MB)
SCSI device sdf: drive cache: write back
sdf: sdf1 sdf2 sdf3 sdf4
Attached scsi disk sdf at scsi2, channel 0, id 0, lun 0
Attached scsi generic sg8 at scsi2, channel 0, id 0, lun 0, type 0
USB Universal Host Controller Interface driver v2.2
Initializing USB Mass Storage driver...
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
mice: PS/2 mouse device common for all mice
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
i2c /dev entries driver
md: raid1 personality registered as nr 3
md: raid5 personality registered as nr 4
raid5: measuring checksumming speed
generic_sse: 4256.000 MB/sec
raid5: using function: generic_sse (4256.000 MB/sec)
md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
device-mapper: 4.1.0-ioctl (2003-12-10) initialised: dm@uk.sistina.com
NET: Registered protocol family 2
IP: routing cache hash table of 8192 buckets, 128Kbytes
TCP: Hash tables configured (established 262144 bind 65536)
NET: Registered protocol family 1
NET: Registered protocol family 17
md: Autodetecting RAID arrays.
md: autorun ...
md: considering sde1 ...
md: adding sde1 ...
md: adding sdd1 ...
md: adding sdc1 ...
md: adding sdb1 ...
md: adding sda1 ...
md: created md0
md: bind<sda1>
md: bind<sdb1>
md: bind<sdc1>
md: bind<sdd1>
md: bind<sde1>
md: running: <sde1><sdd1><sdc1><sdb1><sda1>
raid5: device sde1 operational as raid disk 4
raid5: device sdd1 operational as raid disk 3
raid5: device sdc1 operational as raid disk 2
raid5: device sdb1 operational as raid disk 1
raid5: device sda1 operational as raid disk 0
raid5: allocated 5300kB for md0
raid5: raid level 5 set md0 active with 5 out of 5 devices, algorithm 2
RAID5 conf printout:
--- rd:5 wd:5 fd:0
disk 0, o:1, dev:sda1
disk 1, o:1, dev:sdb1
disk 2, o:1, dev:sdc1
disk 3, o:1, dev:sdd1
disk 4, o:1, dev:sde1
md: ... autorun DONE.
kjournald starting. Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 180k freed
Adding 2097136k swap on /dev/sdf2. Priority:-1 extents:1
EXT3 FS on sdf1, internal journal
kjournald starting. Commit interval 5 seconds
EXT3 FS on sdf3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-32, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-34, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-33, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-35, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-0, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-4, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-5, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-6, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-7, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-8, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-9, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-10, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-11, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-12, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-13, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-14, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-15, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-16, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-17, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-18, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-19, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-20, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-21, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-22, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-23, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-24, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-25, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-26, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-27, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-28, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-29, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-30, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-31, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
Ethernet Channel Bonding Driver: v2.6.0 (January 14, 2004)
bonding: MII link monitoring set to 100 ms
bonding: bond0: enslaving eth0 as an active interface with a down link.
bonding: bond0: enslaving eth1 as an active interface with a down link.
tg3: eth0: Link is up at 1000 Mbps, full duplex.
tg3: eth0: Flow control is off for TX and off for RX.
bonding: bond0: link status definitely up for interface eth0.
tg3: eth1: Link is up at 1000 Mbps, full duplex.
tg3: eth1: Flow control is off for TX and off for RX.
bonding: bond0: link status definitely up for interface eth1.
process `syslogd' is using obsolete setsockopt SO_BSDCOMPAT
SCSI-3 Opcodes & ASC/ASCQ Strings 3.01.06
mptbase: English readable SCSI-3 strings enabled:-)
isense: Registered SCSI-3 Opcodes & ASC/ASCQ Strings
st0: Block limits 2 - 16777215 bytes.
nfs warning: mount version older than kernel
SysRq : Show State
sibling
task PC pid father child younger older
init S 000017faefb01bbf 0 1 0 2 (NOTLB)
000001003ff8fd88 0000000000000006 000000790000d700 000001007ff9b070
0000000000000825 ffffffff803e23a0 000001007ff9b388 0000000000000246
0000000000000216 ffffffff8015a512
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377eb4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff8019188a>{do_select+906}
<ffffffff80186c64>{vfs_stat+68} <ffffffff80191340>{__pollwait+0}
<ffffffff80186bcb>{vfs_getattr+59} <ffffffff801ab861>{compat_sys_select+1105}
<ffffffff80123f81>{ia32_sysret+0}
migration/0 S 0000010001e0f760 0 2 1 3 (L-TLB)
0000010040e73ed8 0000000000000046 0000007302dadcc8 000001007ff9a030
00000000000000eb 000001003e614230 000001007ff9a348 0000010002dadcb8
0000010002dadcc0 0000000000000000
Call Trace:<ffffffff80133661>{complete+1} <ffffffff8013487c>{migration_thread+236}
<ffffffff80134790>{migration_thread+0} <ffffffff80134790>{migration_thread+0}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff80134790>{migration_thread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
ksoftirqd/0 S 00001742a3bbf0ed 0 3 1 4 2 (L-TLB)
0000010040e71f08 0000000000000046 0000010040e71e98 0000010001e6d0b0
000000000000083d ffffffff803e23a0 0000010001e6d3c8 0000010040e71e98
0000010001e12b20 0000000000000000
Call Trace:<ffffffff8013d0ff>{tasklet_action+111} <ffffffff8013d2a0>{ksoftirqd+0}
<ffffffff8013d2df>{ksoftirqd+63} <ffffffff8014e502>{kthread+146}
<ffffffff801115f7>{child_rip+8} <ffffffff8013d2a0>{ksoftirqd+0}
<ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}
migration/1 S 0000010040e1b5e0 0 4 1 5 3 (L-TLB)
0000010001e6bed8 0000000000000046 000000746a6b3cc8 0000010001e6c890
000000000000012a 000001003e614230 0000010001e6cba8 000001006a6b3cb8
000001006a6b3cc0 0000000000000001
Call Trace:<ffffffff8013487c>{migration_thread+236} <ffffffff80134790>{migration_thread+0}
<ffffffff80134790>{migration_thread+0} <ffffffff8014e502>{kthread+146}
<ffffffff801115f7>{child_rip+8} <ffffffff80134790>{migration_thread+0}
<ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}
ksoftirqd/1 S 00001659ca121eaf 0 5 1 6 4 (L-TLB)
0000010001e69f08 0000000000000046 0000010001e69e98 0000010001e6c070
0000000000000bbe 000001007ff9a850 0000010001e6c388 0000010001e69e98
0000010040e1e9a0 0000000000000000
Call Trace:<ffffffff8013d0ff>{tasklet_action+111} <ffffffff8013d2a0>{ksoftirqd+0}
<ffffffff8013d2df>{ksoftirqd+63} <ffffffff8014e502>{kthread+146}
<ffffffff801115f7>{child_rip+8} <ffffffff8013d2a0>{ksoftirqd+0}
<ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}
events/0 S 000017f74d04529d 0 6 1 8 7 5 (L-TLB)
0000010001e53e58 0000000000000046 0000000000000000 0000010040e410f0
00000000000001b5 ffffffff803e23a0 0000010040e41408 0000000000000001
0000000000000003 0000000000000000
Call Trace:<ffffffff80149df5>{worker_thread+261} <ffffffff8037748c>{thread_return+41}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff80149cf0>{worker_thread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
events/1 S 000017fb99b9a166 0 7 1 42 6 (L-TLB)
000001007ff7fe58 0000000000000046 0000007300000212 0000010040e408d0
0000000000000079 000001003dba32f0 0000010040e40be8 0000000000000001
0000000000000003 0000000000000000
Call Trace:<ffffffff80149df5>{worker_thread+261} <ffffffff8037748c>{thread_return+41}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff80149cf0>{worker_thread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
khelper S 00000012722456ce 0 8 6 9 (L-TLB)
0000010001e4fe58 0000000000000046 000001007f14dd38 0000010040e400b0
000000000000021a 000001007ff9a850 0000010040e403c8 0000000000000001
0000000000000003 0000000000000000
Call Trace:<ffffffff801115ef>{child_rip+0} <ffffffff80149df5>{worker_thread+261}
<ffffffff8037748c>{thread_return+41} <ffffffff80133440>{default_wake_function+0}
<ffffffff80133440>{default_wake_function+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff8014e502>{kthread+146}
<ffffffff801115f7>{child_rip+8} <ffffffff8014e540>{keventd_create_kthread+0}
<ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}
kacpid S 00000000087a8e53 0 9 6 40 8 (L-TLB)
000001007fefde58 0000000000000046 000001007fefde18 0000010001e8d130
0000000000002df5 000001007ff9a850 0000010001e8d448 0000000000000001
0000000000010000 0000010001e8d130
Call Trace:<ffffffff80146273>{do_sigaction+659} <ffffffff80145361>{sigprocmask+241}
<ffffffff80149df5>{worker_thread+261} <ffffffff8037748c>{thread_return+41}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
kblockd/0 S 000017f472fc5923 0 40 6 41 9 (L-TLB)
000001007fe93e58 0000000000000046 0000000000000216 0000010001ffa170
0000000000000797 ffffffff803e23a0 0000010001ffa488 0000000000000001
0000000000000003 0000000000000000
Call Trace:<ffffffff80149df5>{worker_thread+261} <ffffffff8037748c>{thread_return+41}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
kblockd/1 S 000017f478cb748d 0 41 6 54 40 (L-TLB)
0000010001fb9e58 0000000000000046 0000010040e56800 000001007fec71f0
00000000000004d9 000001007ff9a850 000001007fec7508 0000000000000001
0000000000000003 0000000000000000
Call Trace:<ffffffff80149df5>{worker_thread+261} <ffffffff8037748c>{thread_return+41}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
khubd S 000000001aca374c 0 42 1 56 7 (L-TLB)
0000010001f87f08 0000000000000046 0000000000000000 000001007fec69d0
0000000000002ff6 000001007ff9a850 000001007fec6ce8 0000000000000000
0000000000000000 0000000000000216
Call Trace:<ffffffff802d1871>{hub_thread+161} <ffffffff80133440>{default_wake_function+0}
<ffffffff801115f7>{child_rip+8} <ffffffff802d17d0>{hub_thread+0}
<ffffffff801115ef>{child_rip+0}
pdflush S 00000000246bfb3c 0 54 6 55 41 (L-TLB)
0000010001f85e98 0000000000000046 0000006a01e0f760 000001007fec61b0
000000000000064c 0000010040e410f0 000001007fec64c8 0000010040e410f0
0000010001e0f760 0000010001e100a0
Call Trace:<ffffffff8015cae0>{pdflush+0} <ffffffff8015c8c5>{__pdflush+261}
<ffffffff8015cae0>{pdflush+0} <ffffffff8015cafc>{pdflush+28}
<ffffffff8015cae0>{pdflush+0} <ffffffff8014e502>{kthread+146}
<ffffffff801115f7>{child_rip+8} <ffffffff8014e540>{keventd_create_kthread+0}
<ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}
pdflush S 000017fc7012083d 0 55 6 58 54 (L-TLB)
000001007fe51e98 0000000000000046 0000000000000000 000001007fec3230
0000000000000a30 000001007ff9a850 000001007fec3548 0000000000000000
000001007fe51e18 0000000000000400
Call Trace:<ffffffff8015cae0>{pdflush+0} <ffffffff8015c8c5>{__pdflush+261}
<ffffffff8015cae0>{pdflush+0} <ffffffff8015cafc>{pdflush+28}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
kswapd1 S 000017ed9504b784 0 56 1 57 42 (L-TLB)
0000010001f83eb8 0000000000000046 0000007300000000 0000010001fce2f0
0000000000020b32 000001007febf270 0000010001fce608 0000000000000202
00000000000000ec 0000000000000000
Call Trace:<ffffffff80164e6f>{kswapd+239} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff80131341>{schedule_tail+17}
<ffffffff801115f7>{child_rip+8} <ffffffff80164d80>{kswapd+0}
<ffffffff801115ef>{child_rip+0}
kswapd0 S 000001000000e798 0 57 1 170 56 (L-TLB)
000001007fe4feb8 0000000000000046 0000007800000000 0000010001fceb10
000000000001f8e0 000001006a752e90 0000010001fcee28 0000000000000202
00000000000000de 00000000000000e0
Call Trace:<ffffffff80164af2>{balance_pgdat+2} <ffffffff80164e6f>{kswapd+239}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff80131341>{schedule_tail+17} <ffffffff801115f7>{child_rip+8}
<ffffffff80164d80>{kswapd+0} <ffffffff801115ef>{child_rip+0}
aio/0 S 0000010001f9d800 0 58 6 59 55 (L-TLB)
000001007fe4de58 0000000000000046 0000006b7fe4de18 0000010001fcf330
000000000000046e 0000010040e410f0 0000010001fcf648 0000000000000001
0000000000010000 0000010001fcf330
Call Trace:<ffffffff80146273>{do_sigaction+659} <ffffffff80145361>{sigprocmask+241}
<ffffffff80149df5>{worker_thread+261} <ffffffff8037748c>{thread_return+41}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
aio/1 S 00000000246e1c84 0 59 6 58 (L-TLB)
0000010001f71e58 0000000000000046 0000010001f71e18 0000010001fd22b0
0000000000001224 000001007ff9a850 0000010001fd25c8 0000000000000001
0000000000010000 0000010001fd22b0
Call Trace:<ffffffff80146273>{do_sigaction+659} <ffffffff80145361>{sigprocmask+241}
<ffffffff80149df5>{worker_thread+261} <ffffffff8037748c>{thread_return+41}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
scsi_eh_0 S 00000000602b4d13 0 170 1 188 57 (L-TLB)
0000010040fbde28 0000000000000046 0000000000000246 000001007febe230
0000000000007288 000001007ff9a850 000001007febe548 0000000035c0ce5f
0000000000000000 0000000000000000
Call Trace:<ffffffff80376ced>{__down_interruptible+301} <ffffffff80133440>{default_wake_function+0}
<ffffffff80133493>{__wake_up_common+67} <ffffffff8037802b>{__down_failed_interruptible+53}
<ffffffff80298e13>{.text.lock.scsi_error+213} <ffffffff801115f7>{child_rip+8}
<ffffffff80298970>{scsi_error_handler+0} <ffffffff801115ef>{child_rip+0}
scsi_eh_1 S 00000002545d74a3 0 188 1 218 170 (L-TLB)
000001003fdbde28 0000000000000046 0000000000000246 000001007febb2b0
0000000000002b4d 000001007ff9a850 000001007febb5c8 000000003caba6fa
0000000000000000 0000000000000000
Call Trace:<ffffffff80376ced>{__down_interruptible+301} <ffffffff80133440>{default_wake_function+0}
<ffffffff80133493>{__wake_up_common+67} <ffffffff8037802b>{__down_failed_interruptible+53}
<ffffffff80298e13>{.text.lock.scsi_error+213} <ffffffff801115f7>{child_rip+8}
<ffffffff80298970>{scsi_error_handler+0} <ffffffff801115ef>{child_rip+0}
scsi_eh_2 S 0000000326cfd7ba 0 218 1 230 188 (L-TLB)
000001003fd59e28 0000000000000046 0000000000000246 000001007febaa90
0000000000002611 000001007ff9a850 000001007febada8 000000003ccbef01
0000000000000000 0000000000000000
Call Trace:<ffffffff80376ced>{__down_interruptible+301} <ffffffff80133440>{default_wake_function+0}
<ffffffff80133493>{__wake_up_common+67} <ffffffff8037802b>{__down_failed_interruptible+53}
<ffffffff80298e13>{.text.lock.scsi_error+213} <ffffffff801115f7>{child_rip+8}
<ffffffff80298970>{scsi_error_handler+0} <ffffffff801115ef>{child_rip+0}
kseriod S 0000000426f4b3d8 0 230 1 238 218 (L-TLB)
000001007fec5ef8 0000000000000046 0000006900000000 000001007feba270
00000000178c3ce3 0000010040e400b0 000001007feba588 00000000ffffffff
0000000000000001 0000000000000000
Call Trace:<ffffffff802edbf4>{serio_thread+244} <ffffffff80133440>{default_wake_function+0}
<ffffffff80131341>{schedule_tail+17} <ffffffff801115f7>{child_rip+8}
<ffffffff802edb00>{serio_thread+0} <ffffffff801115ef>{child_rip+0}
md0_raid5 S 000017f4796cf330 0 238 1 239 230 (L-TLB)
000001003fcf5ee8 0000000000000046 0000000000000202 000001007febf270
000000000000011b 000001007ff9a850 000001007febf588 000001003ffd5948
000001003ffd5980 000001003ffd59f4
Call Trace:<ffffffff802fdc6a>{raid5d+490} <ffffffff802fda89>{raid5d+9}
<ffffffff80303d81>{md_thread+417} <ffffffff80133440>{default_wake_function+0}
<ffffffff80131341>{schedule_tail+17} <ffffffff802fda80>{raid5d+0}
<ffffffff801115f7>{child_rip+8} <ffffffff802fda80>{raid5d+0}
<ffffffff80303be0>{md_thread+0} <ffffffff801115ef>{child_rip+0}
kjournald S 000017fb77fbef50 0 239 1 560 238 (L-TLB)
000001003f3c5e38 0000000000000046 0000010058b3f980 000001007febea50
000000000000036f ffffffff803e23a0 000001007febed68 0000000000000001
0000000000000003 000001003ffbdac8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00000976de89effe 0 560 1 561 239 (L-TLB)
000001003e085e38 0000000000000046 000001007f87af80 000001007fec2a10
0000000000002462 ffffffff803e23a0 000001007fec2d28 0000000000000001
0000000000000003 0000010040e4f6c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000096164d10d42 0 561 1 562 560 (L-TLB)
000001003dc63e38 0000000000000046 000000763dc63de8 0000010001e763f0
00000000000026b5 000001007ff9a850 0000010001e76708 0000000000000001
0000000000000003 0000010040e4f4c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 000009617118a193 0 562 1 563 561 (L-TLB)
000001003ddc5e38 0000000000000046 000000763ddc5de8 0000010001fd2ad0
00000000000015f4 000001007ff9a850 0000010001fd2de8 0000000000000001
0000000000000003 0000010040e4f2c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000096183755c0c 0 563 1 564 562 (L-TLB)
000001003f107e38 0000000000000046 000000773f107de8 0000010001e77430
0000000000002045 000001007ff9a850 0000010001e77748 0000000000000001
0000000000000003 0000010001e62ec8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000096178225e26 0 564 1 565 563 (L-TLB)
000001003dcfbe38 0000000000000046 000001003dbb49e0 0000010001fd32f0
0000000000002094 000001007ff9a850 0000010001fd3608 0000000000000001
0000000000000003 0000010001e62cc8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000176656f9a527 0 565 1 566 564 (L-TLB)
000001003e84be38 0000000000000046 000001001dea6c80 0000010001ffb1b0
00000000000002fc 000001007ff9a850 0000010001ffb4c8 0000000000000001
0000000000000003 0000010001e62ac8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 000009701981bade 0 566 1 567 565 (L-TLB)
000001003e641e38 0000000000000046 0000007371e57260 0000010001e8c0f0
0000000000012748 000001007febf270 0000010001e8c408 0000000000000001
0000000000000003 0000010001e628c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 000009717f79762d 0 567 1 568 566 (L-TLB)
000001003e0e9e38 0000000000000046 0000007371df64a0 000001007f9e4c90
00000000000035b4 000001007ff9a850 000001007f9e4fa8 0000000000000001
0000000000000003 0000010001e626c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00001743d3b35306 0 568 1 569 567 (L-TLB)
000001003e66be38 0000000000000046 000001006fb59ec0 000001007f9b8c50
0000000000001605 ffffffff803e23a0 000001007f9b8f68 0000000000000001
0000000000000003 0000010001e624c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00000968e46b17ba 0 569 1 570 568 (L-TLB)
000001003e075e38 0000000000000046 000001007a81f920 000001007f9b8430
000000000001be2f 000001007ff9a850 000001007f9b8748 0000000000000001
0000000000000003 0000010001e622c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000096933e1707b 0 570 1 571 569 (L-TLB)
000001003dd87e38 0000000000000046 0000010079c36980 000001007f9e54b0
0000000000018cab 000001007ff9a850 000001007f9e57c8 0000000000000001
0000000000000003 000001007ffa6ec8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00000969964f3cf9 0 571 1 572 570 (L-TLB)
000001003dca5e38 0000000000000046 000000763dca5de8 000001007f950e10
00000000000091d0 000001007ff9a850 000001007f951128 0000000000000001
0000000000000003 000001007ffa6cc8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00000969b431f4f3 0 572 1 573 571 (L-TLB)
000001003dfe7e38 0000000000000046 000000763dfe7de8 000001007f9e4470
0000000000008d33 000001007ff9a850 000001007f9e4788 0000000000000001
0000000000000003 000001007feb46c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00000969ca9247be 0 573 1 574 572 (L-TLB)
000001003e14be38 0000000000000046 0000010079c36440 000001007f9b2cd0
000000000000c0f7 000001007ff9a850 000001007f9b2fe8 0000000000000001
0000000000000003 000001007feb44c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000096a0692e7a6 0 574 1 575 573 (L-TLB)
000001003e149e38 0000000000000046 0000010079be2ec0 000001007f9f2d90
0000000000042b94 000001007ff9a850 000001007f9f30a8 0000000000000001
0000000000000003 000001007feb42c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000096ab3afede2 0 575 1 576 574 (L-TLB)
000001003e803e38 0000000000000046 00000100795b80e0 000001007f9505f0
000000000001e924 000001007ff9a850 000001007f950908 0000000000000001
0000000000000003 0000010001fe5ec8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000096b42c9907c 0 576 1 577 575 (L-TLB)
000001003dd53e38 0000000000000046 000001007878b740 000001007f9d25b0
00000000000135f1 000001007ff9a850 000001007f9d28c8 0000000000000001
0000000000000003 0000010001fe5cc8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000096b6b94417d 0 577 1 578 576 (L-TLB)
000001003e305e38 0000000000000046 000000763e305de8 000001007f9d2dd0
000000000000e345 000001007ff9a850 000001007f9d30e8 0000000000000001
0000000000000003 0000010001fe5ac8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000096b8c06b387 0 578 1 579 577 (L-TLB)
000001003e083e38 0000000000000046 000000763e083de8 000001007f951630
0000000000008ee6 000001007ff9a850 000001007f951948 0000000000000001
0000000000000003 0000010001fe58c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000096ba589b3de 0 579 1 580 578 (L-TLB)
000001003e721e38 0000000000000046 0000010077fcb740 000001003e723670
00000000000225a9 000001007ff9a850 000001003e723988 0000000000000001
0000000000000003 0000010001fe56c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000096c386dac3a 0 580 1 581 579 (L-TLB)
000001003e0efe38 0000000000000046 0000010076f35740 000001003e722e50
0000000000011412 000001007ff9a850 000001003e723168 0000000000000001
0000000000000003 0000010001fe54c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000096c683e0386 0 581 1 582 580 (L-TLB)
000001003e089e38 0000000000000046 000000763e089de8 000001003e722630
0000000000007e8d 000001007ff9a850 000001003e722948 0000000000000001
0000000000000003 0000010001fe52c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000096c8bb6f7d7 0 582 1 583 581 (L-TLB)
000001003e77be38 0000000000000046 00000100769f0860 000001003eeb56b0
000000000006575c 000001007ff9a850 000001003eeb59c8 0000000000000001
0000000000000003 000001007fed1ec8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000096d480f2cfa 0 583 1 584 582 (L-TLB)
000001003dd09e38 0000000000000046 000000763dd09de8 000001003eeb4e90
000000000000613f 000001007ff9a850 000001003eeb51a8 0000000000000001
0000000000000003 000001007fed1cc8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 000001007fed1a00 0 584 1 585 583 (L-TLB)
000001003e4f3e38 0000000000000046 0000007624d772c0 000001003eeb4670
0000000000014bf9 000001003d123170 000001003eeb4988 0000000000000001
0000000000000003 000001007fed1ac8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff80180f70>{end_bio_bh_io_sync+0}
<ffffffff801cee80>{commit_timeout+0} <ffffffff801115f7>{child_rip+8}
<ffffffff801ceea0>{kjournald+0} <ffffffff801115ef>{child_rip+0}
kjournald S 00000968bc48ecb3 0 585 1 586 584 (L-TLB)
000001003e3d9e38 0000000000000046 000001002d74a1a0 000001003e3db6f0
000000000001778f ffffffff803e23a0 000001003e3dba08 0000000000000001
0000000000000003 000001007fed18c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00000974f0dafff5 0 586 1 587 585 (L-TLB)
000001003e83fe38 0000000000000046 000000763e83fde8 000001003e3daed0
00000000000096ab 000001007ff9a850 000001003e3db1e8 0000000000000001
0000000000000003 000001007fed16c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000097506481ea5 0 587 1 588 586 (L-TLB)
000001003e63de38 0000000000000046 000001006e9af200 000001003e3da6b0
000000000004ee43 000001007ff9a850 000001003e3da9c8 0000000000000001
0000000000000003 000001007fed14c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00000975c19fa960 0 588 1 589 587 (L-TLB)
000001003e157e38 0000000000000046 00000100177da620 000001003e4f9730
00000000000284d2 000001007ff9a850 000001003e4f9a48 0000000000000001
0000000000000003 000001007fed12c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000097612db91c3 0 589 1 590 588 (L-TLB)
000001003e435e38 0000000000000046 000000763e435de8 000001003e4f8f10
0000000000009591 000001007ff9a850 000001003e4f9228 0000000000000001
0000000000000003 0000010001ff2ec8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 000009762ed04e39 0 590 1 591 589 (L-TLB)
000001003dc53e38 0000000000000046 000000773dc53de8 000001003e4f86f0
0000000000008b45 000001007ff9a850 000001003e4f8a08 0000000000000001
0000000000000003 0000010001ff2cc8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 000009764664377f 0 591 1 592 590 (L-TLB)
000001003deb5e38 0000000000000046 000000773deb5de8 000001003deb7770
0000000000002879 000001007ff9a850 000001003deb7a88 0000000000000001
0000000000000003 0000010001ff2ac8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000097648373d9d 0 592 1 593 591 (L-TLB)
000001003deb3e38 0000000000000046 000000773deb3de8 000001003deb6f50
00000000000015ff 000001007ff9a850 000001003deb7268 0000000000000001
0000000000000003 0000010001ff28c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 000009764abd6fa4 0 593 1 594 592 (L-TLB)
000001003e131e38 0000000000000046 000001002cd99e60 000001003deb6730
00000000000201f4 000001007ff9a850 000001003deb6a48 0000000000000001
0000000000000003 0000010001ff26c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00000968b83fefb2 0 594 1 595 593 (L-TLB)
000001003de23e38 0000000000000046 000000773de23de8 000001003de25070
0000000000001a18 ffffffff803e23a0 000001003de25388 0000000000000001
0000000000000003 0000010001ff24c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00000968bbdaa6c8 0 595 1 596 594 (L-TLB)
000001003e7e1e38 0000000000000046 000000773e7e1de8 000001003de24850
0000000000001171 ffffffff803e23a0 000001003de24b68 0000000000000001
0000000000000003 0000010001ff22c8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00000968bc4563a5 0 596 1 634 595 (L-TLB)
000001003e7f7e38 0000000000000046 000000773e7f7de8 000001003de24030
00000000000011aa ffffffff803e23a0 000001003de24348 0000000000000001
0000000000000003 000001007fed2ec8
Call Trace:<ffffffff801cf1ac>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801cee80>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceea0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
portmap S 00000001fffd6d6a 0 634 1 840 596 (NOTLB)
000001003e21de68 0000000000000006 0000007680000000 000001003e21f0f0
000000000000390b 000001007faa0950 000001003e21f408 0000000000000246
000001003e01e970 0000000000000002
Call Trace:<ffffffff80377eb4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff80191fd1>{do_poll+177} <ffffffff8019215e>{sys_poll+350}
<ffffffff80191340>{__pollwait+0} <ffffffff80123f81>{ia32_sysret+0}
syslogd S 000001007f907ec0 0 840 1 843 634 (NOTLB)
000001003dc0bd88 0000000000000006 0000007400000000 0000010001e76c10
0000000000000f32 000001007f9b34f0 0000010001e76f28 0000000000000216
0000000000000216 ffffffff8015a512
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff80313e05>{datagram_poll+21} <ffffffff8019188a>{do_select+906}
<ffffffff80191340>{__pollwait+0} <ffffffff801ab861>{compat_sys_select+1105}
<ffffffff80123f81>{ia32_sysret+0}
klogd R running task 0 843 1 846 840 (NOTLB)
rpc.statd S 0000001ce1b6dffc 0 846 1 857 843 (NOTLB)
000001003dde5d88 0000000000000006 3256cc8432343731 000001007f9b24b0
00000000000047ec ffffffff803e23a0 000001007f9b27c8 0000000000000216
0000000000000216 ffffffff8015a512
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff80330301>{tcp_poll+33} <ffffffff8019188a>{do_select+906}
<ffffffff80198a12>{destroy_inode+50} <ffffffff80191340>{__pollwait+0}
<ffffffff801ab861>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
exim S 00001706db11617f 0 857 1 864 846 (NOTLB)
000001003e5d9d88 0000000000000006 000001000000d700 000001007f9f35b0
0000000000007d07 ffffffff803e23a0 000001007f9f38c8 ffffffff8015a512
000001000000d700 0000000000000001
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff80330301>{tcp_poll+33} <ffffffff8019188a>{do_select+906}
<ffffffff80191340>{__pollwait+0} <ffffffff80154c01>{compat_sys_wait4+65}
<ffffffff801ab861>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
inetd S 0000001d2c815e9e 0 864 1 870 857 (NOTLB)
000001003e0b1d88 0000000000000006 0000000000000000 000001007f9d35f0
0000000000285906 ffffffff803e23a0 000001007f9d3908 0000000000000216
0000000000000216 ffffffff8015a512
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff80330301>{tcp_poll+33} <ffffffff8019188a>{do_select+906}
<ffffffff80191340>{__pollwait+0} <ffffffff8016c109>{unmap_region+409}
<ffffffff801ab861>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
rngd S 000017f3d0d7e2ea 0 870 1 880 864 (NOTLB)
000001003e219e68 0000000000000006 00000010ffffd36c 000001003e21e8d0
0000000000000f34 ffffffff803e23a0 000001003e21ebe8 0000000000000246
ffffffff80443c80 0000000000000001
Call Trace:<ffffffff8015a555>{__get_free_pages+37} <ffffffff80377eb4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff80191fd1>{do_poll+177}
<ffffffff8019215e>{sys_poll+350} <ffffffff80191340>{__pollwait+0}
<ffffffff80123f81>{ia32_sysret+0}
lpd S 000000a9189b63b6 0 880 1 890 870 (NOTLB)
000001003e23dd88 0000000000000006 000001000000d700 000001007fec21f0
0000000000003366 ffffffff803e23a0 000001007fec2508 ffffffff8015a512
000001000000d700 0000000000000001
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff8018a7d2>{pipe_poll+50} <ffffffff8019188a>{do_select+906}
<ffffffff80191340>{__pollwait+0} <ffffffff80154c01>{compat_sys_wait4+65}
<ffffffff801ab861>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
nfsd S 00001605f7be1cc7 0 890 1 892 880 (L-TLB)
000001003e0abd98 0000000000000046 0000007301cc8838 000001003e22b1b0
0000000000003a7a 000001003e22a170 000001003e22b4c8 0000000000000246
000001007fe7b8a0 000001003e0be180
Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377eb4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff8036e0bb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80145361>{sigprocmask+241} <ffffffff801f8d70>{nfsd+0}
<ffffffff801f8f47>{nfsd+471} <ffffffff801f8d70>{nfsd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801f8d70>{nfsd+0}
<ffffffff801f8d70>{nfsd+0} <ffffffff801115ef>{child_rip+0}
nfsd S 00001588597d8af1 0 892 1 896 890 (L-TLB)
000001003e4d1d98 0000000000000046 0000007301d8bbc0 000001003df551f0
000000000000fc14 000001003df541b0 000001003df55508 0000000000000246
000001007fe7d8a0 000001003e0be180
Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377eb4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff8036e0bb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80145361>{sigprocmask+241} <ffffffff801f8d70>{nfsd+0}
<ffffffff801f8f47>{nfsd+471} <ffffffff801f8d70>{nfsd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801f8d70>{nfsd+0}
<ffffffff801f8d70>{nfsd+0} <ffffffff801115ef>{child_rip+0}
nfsd S 0000172b46986b0f 0 896 1 891 892 (L-TLB)
000001003df43d98 0000000000000046 0000010001a00320 000001003dd3ea10
00000000000059bc ffffffff803e23a0 000001003dd3ed28 0000000000000246
000001007fe71ca0 000001003e0be180
Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377eb4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff8036e0bb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80145361>{sigprocmask+241} <ffffffff801f8d70>{nfsd+0}
<ffffffff801f8f47>{nfsd+471} <ffffffff801f8d70>{nfsd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801f8d70>{nfsd+0}
<ffffffff801f8d70>{nfsd+0} <ffffffff801115ef>{child_rip+0}
nfsd S 00001605f7be22db 0 891 1 897 896 (L-TLB)
000001003e2c7d98 0000000000000046 0000007301c984b0 000001003e22a170
0000000000000614 ffffffff803e23a0 000001003e22a488 0000000000000246
0000010001fa78a0 0000000000000216
Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377eb4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff8036e0bb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80145361>{sigprocmask+241} <ffffffff801f8d70>{nfsd+0}
<ffffffff801f8f47>{nfsd+471} <ffffffff801f8d70>{nfsd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801f8d70>{nfsd+0}
<ffffffff801f8d70>{nfsd+0} <ffffffff801115ef>{child_rip+0}
nfsd S 0000172b46983432 0 897 1 893 891 (L-TLB)
000001003e6dfd98 0000000000000046 0000007301523858 000001003dd3e1f0
00000000000001ee 000001007ff9a850 000001003dd3e508 0000000000000246
0000010001f9cca0 0000000000000216
Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377eb4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff8036e0bb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80145361>{sigprocmask+241} <ffffffff801f8d70>{nfsd+0}
<ffffffff801f8f47>{nfsd+471} <ffffffff801f8d70>{nfsd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801f8d70>{nfsd+0}
<ffffffff801f8d70>{nfsd+0} <ffffffff801115ef>{child_rip+0}
nfsd R running task 0 893 1 894 897 (L-TLB)
nfsd S 00001588597d9384 0 894 1 895 893 (L-TLB)
000001003e6d7d98 0000000000000046 0000010040a0aef0 000001003df541b0
00000000000000db 000001007ff9a850 000001003df544c8 0000000000000246
000001007fe808a0 0000000000000216
Call Trace:<ffffffff80377eb4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff8036e0bb>{svc_recv+987} <ffffffff80133440>{default_wake_function+0}
<ffffffff80133440>{default_wake_function+0} <ffffffff80145361>{sigprocmask+241}
<ffffffff801f8d70>{nfsd+0} <ffffffff801f8f47>{nfsd+471}
<ffffffff801f8d70>{nfsd+0} <ffffffff801115f7>{child_rip+8}
<ffffffff801f8d70>{nfsd+0} <ffffffff801f8d70>{nfsd+0}
<ffffffff801115ef>{child_rip+0}
nfsd S 000017fca38dfc47 0 895 1 899 894 (L-TLB)
000001003dd3dd98 0000000000000046 0000007301b2ad10 000001003dd3f230
0000000000000214 000001007ff9a850 000001003dd3f548 0000000000000246
0000010001f9bca0 0000000000000216
Call Trace:<ffffffff80141011>{del_timer_sync+49} <ffffffff80377eb4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff8036e0bb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80145361>{sigprocmask+241} <ffffffff801f8d70>{nfsd+0}
<ffffffff801f8f47>{nfsd+471} <ffffffff801f8d70>{nfsd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801f8d70>{nfsd+0}
<ffffffff801f8d70>{nfsd+0} <ffffffff801115ef>{child_rip+0}
lockd S 0000001df0430c1a 0 899 1 900 895 (L-TLB)
000001007f77fde8 0000000000000046 000001003e22a990 000001003e22a990
00000000000011a0 000001007ff9a850 000001003e22aca8 ffffffff8037748c
0000007700000001 0000000000000212
Call Trace:<ffffffff8037748c>{thread_return+41} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff8036c639>{svc_sock_release+569} <ffffffff8036e0bb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80377ff1>{__down_failed+53} <ffffffff80133440>{default_wake_function+0}
<ffffffff8020a699>{lockd+377} <ffffffff8020a520>{lockd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff8020a520>{lockd+0}
<ffffffff8020a520>{lockd+0} <ffffffff801115ef>{child_rip+0}
rpciod S 000001007f109ee8 0 900 1 902 899 (L-TLB)
000001007f109e88 0000000000000046 0000007840000000 000001007f9b9470
0000000000001312 000001003e22b1b0 000001007f9b9788 000001003e290c68
ffffffff8020a520 0000000000000008
Call Trace:<ffffffff8020a520>{lockd+0} <ffffffff8037809f>{__up_wakeup+53}
<ffffffff8020a520>{lockd+0} <ffffffff803694ea>{rpciod+1050}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff80131341>{schedule_tail+17} <ffffffff801115f7>{child_rip+8}
<ffffffff8020a520>{lockd+0} <ffffffff803690d0>{rpciod+0}
<ffffffff801115ef>{child_rip+0}
rpc.mountd S 00000001fffd73a1 0 902 1 905 900 (NOTLB)
000001003e613e68 0000000000000006 0000007300000000 000001003e615270
0000000000014334 000001007faa0950 000001003e615588 0000000000000246
000001007f7da430 0000000000000002
Call Trace:<ffffffff80377eb4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff80191fd1>{do_poll+177} <ffffffff8019215e>{sys_poll+350}
<ffffffff80191340>{__pollwait+0} <ffffffff80123f81>{ia32_sysret+0}
smartd S 0000170806fc9a7f 0 905 1 911 902 (NOTLB)
000001003e113ee8 0000000000000006 000001003ffba600 000001007faa0950
0000000000002ce5 ffffffff803e23a0 000001007faa0c68 0000000000000246
0000000000000009 ffffffff8027619b
Call Trace:<ffffffff8027619b>{blkdev_ioctl+955} <ffffffff80190cb6>{sys_ioctl+678}
<ffffffff80377eb4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff801541e1>{compat_sys_nanosleep+193} <ffffffff80123f81>{ia32_sysret+0}
sshd S 00001742a268c884 0 911 1 1283 914 905 (NOTLB)
000001003e59dd88 0000000000000006 000001000000d700 000001003e614a50
00000000000067ee ffffffff803e23a0 000001003e614d68 ffffffff8015a512
000001000000d700 0000000000000001
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff80330301>{tcp_poll+33} <ffffffff8019188a>{do_select+906}
<ffffffff80191340>{__pollwait+0} <ffffffff801ab861>{compat_sys_select+1105}
<ffffffff80123f81>{ia32_sysret+0}
ntpd S 000017fcc1e2c8a8 0 914 1 918 911 (NOTLB)
000001007f631d88 0000000000000006 0000007940000700 000001007f9f2570
00000000000003a2 000001007ff9a850 000001007f9f2888 0000000000000216
0000000000000216 ffffffff8015a512
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff80313e05>{datagram_poll+21} <ffffffff8019188a>{do_select+906}
<ffffffff80191340>{__pollwait+0} <ffffffff8011084f>{do_signal+175}
<ffffffff801ab861>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
atd S 00001708155c9ab5 0 918 1 921 914 (NOTLB)
000001003dce1ee8 0000000000000006 0000000000000000 000001007faa1170
0000000000005c07 000001007ff9a850 000001007faa1488 0000000000000246
0000000000000000 ffffffff80186c64
Call Trace:<ffffffff80186c64>{vfs_stat+68} <ffffffff80141011>{del_timer_sync+49}
<ffffffff80377eb4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff801541e1>{compat_sys_nanosleep+193} <ffffffff80123f81>{ia32_sysret+0}
cron R running task 0 921 1 925 918 (NOTLB)
irqbalance S 000017fc5ba4b8d4 0 925 1 935 921 (NOTLB)
000001003dbb7ee8 0000000000000006 0000000000000212 000001003e56c270
00000000000063f8 000001007ff9a850 000001003e56c588 0000000000000246
0000000000000216 00000100275c7d68
Call Trace:<ffffffff8016a7c7>{remove_vm_struct+231} <ffffffff8016c109>{unmap_region+409}
<ffffffff80377eb4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff801541e1>{compat_sys_nanosleep+193} <ffffffff80123f81>{ia32_sysret+0}
getty S 0000001eb703d85f 0 935 1 1492 925 (NOTLB)
000001003e551d38 0000000000000006 ffffffff804b4378 000001003db57330
000000000000c4e6 ffffffff803e23a0 000001003db57648 00000000fffd7a5e
000001003e551db8 ffffffff80141135
Call Trace:<ffffffff80141135>{del_singleshot_timer_sync+21} <ffffffff80377ebc>{schedule_timeout+188}
<ffffffff80377e25>{schedule_timeout+37} <ffffffff8026263b>{uart_write+699}
<ffffffff8024cb55>{read_chan+1109} <ffffffff8024d38c>{write_chan+556}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80133440>{default_wake_function+0} <ffffffff8024685f>{tty_read+303}
<ffffffff8017bca7>{vfs_read+199} <ffffffff8017bf19>{sys_read+73}
<ffffffff80123f81>{ia32_sysret+0}
sshd S 000001003f270e00 0 1283 911 1294 1304 (NOTLB)
000001003daf5c18 0000000000000006 0000007500001000 000001007faa0130
000000000000040a 000001003e582b90 000001007faa0448 ffffffff80382230
000001003db0d3a0 0000000000000374
Call Trace:<ffffffff801c46cd>{__ext3_journal_stop+45} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff801582b7>{generic_file_aio_write_nolock+2055}
<ffffffff8035dc17>{unix_stream_data_wait+263} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff8035ddf1>{unix_stream_recvmsg+369}
<ffffffff8030d564>{sock_aio_read+180} <ffffffff8017bb93>{do_sync_read+115}
<ffffffff8017ae77>{dentry_open+263} <ffffffff8017ad52>{filp_open+66}
<ffffffff8017bcba>{vfs_read+218} <ffffffff8017bf19>{sys_read+73}
<ffffffff80123f81>{ia32_sysret+0}
sshd S 000017fc6c343dc8 0 1294 1283 1295 (NOTLB)
000001003f269d88 0000000000000006 0000010040000700 000001003e582b90
0000000000001030 000001007ff9a850 000001003e582ea8 ffffffff8015a512
0000000000000030 0000000000000001
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff8024e4eb>{pty_write_room+43} <ffffffff8024d4e1>{normal_poll+305}
<ffffffff8019188a>{do_select+906} <ffffffff80191340>{__pollwait+0}
<ffffffff801ab861>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
zsh S 0000175e190daa09 0 1295 1294 1884 (NOTLB)
000001003dbfbf18 0000000000000006 0000007900000216 000001003dba22b0
000000000000bbac 000001006a6f96f0 000001003dba25c8 0000010040e7b280
0000010040e7b280 ffffffff80145361
Call Trace:<ffffffff80145361>{sigprocmask+241} <ffffffff80195891>{dput+33}
<ffffffff80123f81>{ia32_sysret+0} <ffffffff8010fde9>{sys_rt_sigsuspend+297}
<ffffffff801249fe>{sys32_rt_sigprocmask+206} <ffffffff801240d8>{ia32_ptregs_common+40}
sshd S 000001003f270300 0 1304 911 1306 1834 1283 (NOTLB)
000001003dacdc18 0000000000000006 0000007600001000 000001003e5833b0
00000000000003af 000001003e582370 000001003e5836c8 ffffffff80382230
000001003db0d3a0 0000000000000374
Call Trace:<ffffffff801c46cd>{__ext3_journal_stop+45} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff801582b7>{generic_file_aio_write_nolock+2055}
<ffffffff8035dc17>{unix_stream_data_wait+263} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff8035ddf1>{unix_stream_recvmsg+369}
<ffffffff8030d564>{sock_aio_read+180} <ffffffff8017bb93>{do_sync_read+115}
<ffffffff8017ae77>{dentry_open+263} <ffffffff8017ad52>{filp_open+66}
<ffffffff8017bcba>{vfs_read+218} <ffffffff8017bf19>{sys_read+73}
<ffffffff80123f81>{ia32_sysret+0}
sshd S 000017fade524bf8 0 1306 1304 1307 (NOTLB)
000001003d869d88 0000000000000006 000001000000d700 000001003e582370
00000000000011ff ffffffff803e23a0 000001003e582688 ffffffff8015a512
0000000008098460 0000000000000001
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff8024e4eb>{pty_write_room+43} <ffffffff8024d4e1>{normal_poll+305}
<ffffffff8019188a>{do_select+906} <ffffffff80191340>{__pollwait+0}
<ffffffff801ab861>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
zsh S 0000174b6cc3cd6f 0 1307 1306 1853 (NOTLB)
000001003d72df18 0000000000000006 0000000000000216 000001003db562f0
000000000000b46b ffffffff803e23a0 000001003db56608 000001007f75d280
000001007f75d280 ffffffff80145361
Call Trace:<ffffffff80145361>{sigprocmask+241} <ffffffff80195891>{dput+33}
<ffffffff80123f81>{ia32_sysret+0} <ffffffff8010fde9>{sys_rt_sigsuspend+297}
<ffffffff801249fe>{sys32_rt_sigprocmask+206} <ffffffff801240d8>{ia32_ptregs_common+40}
apcupsd S 000009632df819fd 0 1492 1 1538 935 (NOTLB)
000001003d0ddde8 0000000000000006 000001003e3f5cd8 000001003e21e0b0
000000000000b969 000001007ff9a850 000001003e21e3c8 ffffffff801584eb
00000000ffffdbdc 0000000000000038
Call Trace:<ffffffff801584eb>{generic_file_aio_write+107} <ffffffff801bbbb3>{ext3_file_write+35}
<ffffffff8013bdb2>{sys_wait4+642} <ffffffff80133440>{default_wake_function+0}
<ffffffff80133440>{default_wake_function+0} <ffffffff80154c01>{compat_sys_wait4+65}
<ffffffff80146273>{do_sigaction+659} <ffffffff8017bea7>{vfs_write+247}
<ffffffff8017bf99>{sys_write+73} <ffffffff80123f81>{ia32_sysret+0}
apcupsd S 000017fcc6b703d8 0 1538 1492 (NOTLB)
000001007c2d9d88 0000000000000006 0000010040001500 000001003dba32f0
00000000000010a5 000001007ff9a850 000001003dba3608 0000000000000246
0000001000000000 000000d000000000
Call Trace:<ffffffff80377eb4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff8019188a>{do_select+906} <ffffffff80191340>{__pollwait+0}
<ffffffff80246c5c>{tty_write+844} <ffffffff801ab861>{compat_sys_select+1105}
<ffffffff80123f81>{ia32_sysret+0}
sshd S 00001742a3a95af9 0 1834 911 1836 1304 (NOTLB)
000001002e86dc18 0000000000000006 0000000000001000 000001003db56b10
00000000000ab8bf 000001007ff9a850 000001003db56e28 ffffffff80382230
000001002f40e6a0 0000000000000374
Call Trace:<ffffffff801c46cd>{__ext3_journal_stop+45} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff801582b7>{generic_file_aio_write_nolock+2055}
<ffffffff8035dc17>{unix_stream_data_wait+263} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff8035ddf1>{unix_stream_recvmsg+369}
<ffffffff8030d564>{sock_aio_read+180} <ffffffff8017bb93>{do_sync_read+115}
<ffffffff8017ae77>{dentry_open+263} <ffffffff8017ad52>{filp_open+66}
<ffffffff8017bcba>{vfs_read+218} <ffffffff8017bf19>{sys_read+73}
<ffffffff80123f81>{ia32_sysret+0}
sshd S 000017fc00e962b2 0 1836 1834 1837 (NOTLB)
000001006e029d88 0000000000000006 000000740000d700 0000010001e8c910
0000000000009f32 000001003dba2ad0 0000010001e8cc28 ffffffff8015a512
0000000008099438 0000000000000001
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff8024e4eb>{pty_write_room+43} <ffffffff8024d4e1>{normal_poll+305}
<ffffffff8019188a>{do_select+906} <ffffffff80191340>{__pollwait+0}
<ffffffff801ab861>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
zsh S 00001743a8714734 0 1837 1836 1845 (NOTLB)
0000010028cf9f18 0000000000000006 0000000000000216 0000010001ffa990
000000000000d3bf ffffffff803e23a0 0000010001ffaca8 000001007f75d4c0
000001007f75d4c0 ffffffff80145361
Call Trace:<ffffffff80145361>{sigprocmask+241} <ffffffff80195891>{dput+33}
<ffffffff80123f81>{ia32_sysret+0} <ffffffff8010fde9>{sys_rt_sigsuspend+297}
<ffffffff801249fe>{sys32_rt_sigprocmask+206} <ffffffff801240d8>{ia32_ptregs_common+40}
zsh R running task 0 1845 1837 (NOTLB)
zsh S 000017fade51a9f9 0 1853 1307 1887 (NOTLB)
0000010024123d38 0000000000000006 0000007400000246 000001003e614230
00000000000054ed 000001003e582370 000001003e614548 000001007ffee680
000001000000e500 0000000000000216
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e25>{schedule_timeout+37}
<ffffffff8024a7a1>{opost_block+401} <ffffffff8024cb55>{read_chan+1109}
<ffffffff80168375>{do_wp_page+261} <ffffffff801699ff>{handle_mm_fault+399}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff8024685f>{tty_read+303} <ffffffff8017bca7>{vfs_read+199}
<ffffffff8017bf19>{sys_read+73} <ffffffff80123f81>{ia32_sysret+0}
top S 000017fc68844929 0 1884 1295 (NOTLB)
000001006d4f7d88 0000000000000006 0000007440001500 000001006a6f86b0
0000000000000121 000001007ff9a850 000001006a6f89c8 0000000000000246
000000103f2359d8 000000d000000000
Call Trace:<ffffffff80377eb4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff8019188a>{do_select+906} <ffffffff80191340>{__pollwait+0}
<ffffffff801ab861>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
dump D 000017f4796ce9b2 0 1887 1853 (NOTLB)
000001001ae8dbb8 0000000000000006 000001003ffd5948 000001006a6f96f0
0000000000009235 ffffffff803e23a0 000001006a6f9a08 ffffffff802fd643
0000000000000212 0000000000000001
Call Trace:<ffffffff802fd643>{raid5_unplug_device+291} <ffffffff80377d0b>{io_schedule+43}
<ffffffff80155f6a>{__lock_page+250} <ffffffff80155c40>{page_wake_function+0}
<ffffffff80155c40>{page_wake_function+0} <ffffffff801567f6>{do_generic_mapping_read+502}
<ffffffff80156a70>{file_read_actor+0} <ffffffff80156d14>{__generic_file_aio_read+372}
<ffffffff80156dfb>{generic_file_read+123} <ffffffff80169994>{handle_mm_fault+292}
<ffffffff80122ad8>{do_page_fault+440} <ffffffff80130948>{recalc_task_prio+424}
<ffffffff8037748c>{thread_return+41} <ffffffff8017bca7>{vfs_read+199}
<ffffffff8017bf19>{sys_read+73} <ffffffff80123f81>{ia32_sysret+0}
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-15 15:00 ` foo
@ 2004-06-15 23:26 ` Andrew Morton
2004-06-15 23:44 ` Evaldo Gardenali
` (3 more replies)
0 siblings, 4 replies; 19+ messages in thread
From: Andrew Morton @ 2004-06-15 23:26 UTC (permalink / raw)
To: foo; +Cc: linux-kernel
foo@porto.bmb.uga.edu wrote:
>
> On Tue, Jun 15, 2004 at 03:09:32AM -0700, Andrew Morton wrote:
> > Wait for it to happen again, then do
> >
> > echo t > /proc/sysrq-trigger
> > dmesg -s 1000000 > foo
> >
> > then send foo, foo.
>
> Here you go. I did dump 2f /dev/null /export/home. The first time it
> completed, the second it hung right away.
>
> This is with 2.6.7-rc3-bk6 with no other patches, also compiled with a
> newer gcc.
>
> ...
> dump D 000017f4796ce9b2 0 1887 1853 (NOTLB)
> 000001001ae8dbb8 0000000000000006 000001003ffd5948 000001006a6f96f0
> 0000000000009235 ffffffff803e23a0 000001006a6f9a08 ffffffff802fd643
> 0000000000000212 0000000000000001
> Call Trace:<ffffffff802fd643>{raid5_unplug_device+291} <ffffffff80377d0b>{io_schedule+43}
> <ffffffff80155f6a>{__lock_page+250} <ffffffff80155c40>{page_wake_function+0}
> <ffffffff80155c40>{page_wake_function+0} <ffffffff801567f6>{do_generic_mapping_read+502}
> <ffffffff80156a70>{file_read_actor+0} <ffffffff80156d14>{__generic_file_aio_read+372}
> <ffffffff80156dfb>{generic_file_read+123} <ffffffff80169994>{handle_mm_fault+292}
> <ffffffff80122ad8>{do_page_fault+440} <ffffffff80130948>{recalc_task_prio+424}
> <ffffffff8037748c>{thread_return+41} <ffffffff8017bca7>{vfs_read+199}
> <ffffffff8017bf19>{sys_read+73} <ffffffff80123f81>{ia32_sysret+0}
>
OK, well I'd be suspecting that either devicemapper or raid5 lost an I/O
completion, causing that page to never be unlocked.
Please try the latest -mm kernel, which has a few devicemapper changes,
although they are unlikely to fix this.
If it's possible to remove either raid5 or devicemapper from the picture,
that would help us find the problem.
Other than that, the chances of getting this fixed are proportional to your
skill in finding us a way of reproducing it. A good start would be to tell
us exactly which commands were used to set up the LVM and the raid array.
That way a raid/LVM ignoramus like me can take a look ;)
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-15 23:26 ` Andrew Morton
@ 2004-06-15 23:44 ` Evaldo Gardenali
2004-06-16 2:03 ` foo
` (2 subsequent siblings)
3 siblings, 0 replies; 19+ messages in thread
From: Evaldo Gardenali @ 2004-06-15 23:44 UTC (permalink / raw)
To: linux-kernel
[-- Attachment #1: Type: text/plain, Size: 461 bytes --]
Hi there
I always wondered if you guys would allow for killing a process like
this. I seldom have to reboot boxes with 30-40 users logged in doing
their work just because the evolution process of some user is stalled as
D. no dmesg error is given
Thanks
Evaldo
Em Ter, 2004-06-15 às 20:26, Andrew Morton escreveu:
> OK, well I'd be suspecting that either devicemapper or raid5 lost an I/O
> completion, causing that page to never be unlocked.
[-- Attachment #2: Esta é uma parte de mensagem assinada digitalmente --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-15 23:26 ` Andrew Morton
2004-06-15 23:44 ` Evaldo Gardenali
@ 2004-06-16 2:03 ` foo
2004-06-16 2:16 ` foo
2004-06-16 2:48 ` foo
3 siblings, 0 replies; 19+ messages in thread
From: foo @ 2004-06-16 2:03 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel
On Tue, Jun 15, 2004 at 04:26:07PM -0700, Andrew Morton wrote:
> > dump D 000017f4796ce9b2 0 1887 1853 (NOTLB)
> > 000001001ae8dbb8 0000000000000006 000001003ffd5948 000001006a6f96f0
> > 0000000000009235 ffffffff803e23a0 000001006a6f9a08 ffffffff802fd643
> > 0000000000000212 0000000000000001
> > Call Trace:<ffffffff802fd643>{raid5_unplug_device+291} <ffffffff80377d0b>{io_schedule+43}
> > <ffffffff80155f6a>{__lock_page+250} <ffffffff80155c40>{page_wake_function+0}
> > <ffffffff80155c40>{page_wake_function+0} <ffffffff801567f6>{do_generic_mapping_read+502}
> > <ffffffff80156a70>{file_read_actor+0} <ffffffff80156d14>{__generic_file_aio_read+372}
> > <ffffffff80156dfb>{generic_file_read+123} <ffffffff80169994>{handle_mm_fault+292}
> > <ffffffff80122ad8>{do_page_fault+440} <ffffffff80130948>{recalc_task_prio+424}
> > <ffffffff8037748c>{thread_return+41} <ffffffff8017bca7>{vfs_read+199}
> > <ffffffff8017bf19>{sys_read+73} <ffffffff80123f81>{ia32_sysret+0}
> >
>
> OK, well I'd be suspecting that either devicemapper or raid5 lost an I/O
> completion, causing that page to never be unlocked.
>
> Please try the latest -mm kernel, which has a few devicemapper changes,
> although they are unlikely to fix this.
Yeah, the changes I've seen recently don't look too promising as far as
this goes, but I'll try to give -mm a whirl tonight, since it seems easy
to reproduce.
> If it's possible to remove either raid5 or devicemapper from the picture,
> that would help us find the problem.
I don't see a way to do this, unfortunately... It's just this one mount
point that's giving me trouble.
> Other than that, the chances of getting this fixed are proportional to your
> skill in finding us a way of reproducing it. A good start would be to tell
> us exactly which commands were used to set up the LVM and the raid array.
> That way a raid/LVM ignoramus like me can take a look ;)
I can give you the exact commands I used. For the raid5 array:
mdadm -C /dev/md0 -c 128 -l 5 -n 5 /dev/sdg1 /dev/sdh1 /dev/sdi1
/dev/sdj1 /dev/sdk1
There were more disks in the box when I created it... Hopefully you
don't need 5x146GB SCSI disks to reproduce...
For LVM:
pvcreate -M2 /dev/md0
vgcreate vg0 /dev/md0
There's only one PV and one VG involved. The LVs were created like so:
lvcreate -L 25G -n home vg0
I don't know if it's a coincidence, but the "home" LV that's giving me
trouble is the first one I created, just as above. The filesystems were
created with:
mke2fs -j -b 4096 -R stride=32 /dev/vg0/home
Thanks for your help,
-ryan
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-15 23:26 ` Andrew Morton
2004-06-15 23:44 ` Evaldo Gardenali
2004-06-16 2:03 ` foo
@ 2004-06-16 2:16 ` foo
2004-06-16 12:57 ` Alasdair G Kergon
2004-06-16 2:48 ` foo
3 siblings, 1 reply; 19+ messages in thread
From: foo @ 2004-06-16 2:16 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel
On Tue, Jun 15, 2004 at 04:26:07PM -0700, Andrew Morton wrote:
> Other than that, the chances of getting this fixed are proportional to your
> skill in finding us a way of reproducing it. A good start would be to tell
> us exactly which commands were used to set up the LVM and the raid array.
> That way a raid/LVM ignoramus like me can take a look ;)
>
In case any of this is of interest...
# cat /proc/mdstat
Personalities : [raid1] [raid5]
md0 : active raid5 sde1[4] sdd1[3] sdc1[2] sdb1[1] sda1[0]
573487616 blocks level 5, 128k chunk, algorithm 2 [5/5] [UUUUU]
unused devices: <none>
# pvdisplay /dev/md0
--- Physical volume ---
PV Name /dev/md0
VG Name vg0
PV Size 546.92 GB / not usable 0
Allocatable yes
PE Size (KByte) 4096
Total PE 140011
Free PE 62699
Allocated PE 77312
PV UUID a76cj4-Fvnf-xkY8-bE0j-Wcdk-jtb0-mq9NbF
# vgdisplay vg0
--- Volume group ---
VG Name vg0
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 37
VG Access read/write
VG Status resizable
MAX LV 256
Cur LV 36
Open LV 36
Max PV 256
Cur PV 1
Act PV 1
VG Size 546.92 GB
PE Size 4.00 MB
Total PE 140011
Alloc PE / Size 77312 / 302.00 GB
Free PE / Size 62699 / 244.92 GB
VG UUID EameBy-n1m#-cwNa-rKqz-YqoM-hSEH-BKVeQQ
# lvdisplay
--- Logical volume ---
LV Name /dev/vg0/home
VG Name vg0
LV UUID yQFLXG-KLog-TdWQ-Rvis-35U1-jUj1-zO0h3M
LV Write Access read/write
LV Status available
# open 1
LV Size 25.00 GB
Current LE 6400
Segments 1
Allocation next free (default)
Read ahead sectors 0
Block device 253:0
[...35 more...]
-ryan
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-15 23:26 ` Andrew Morton
` (2 preceding siblings ...)
2004-06-16 2:16 ` foo
@ 2004-06-16 2:48 ` foo
2004-06-16 2:58 ` Andrew Morton
3 siblings, 1 reply; 19+ messages in thread
From: foo @ 2004-06-16 2:48 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel
On Tue, Jun 15, 2004 at 04:26:07PM -0700, Andrew Morton wrote:
> OK, well I'd be suspecting that either devicemapper or raid5 lost an I/O
> completion, causing that page to never be unlocked.
>
> Please try the latest -mm kernel, which has a few devicemapper changes,
> although they are unlikely to fix this.
OK, this was fun...
LILO 22.2 boot: linux-mm
Loading Linux-mm.................................
This is all I see on the serial console. The machine did boot, though;
a few minutes later I see this (and only this) from my syslog server:
xarello kernel: bonding: bond0: link status definitely up for interface
eth1.
Getty didn't come up on the serial console, and it's refusing ssh
requests, although it seems to be dropping lots of packets.
Now I have to go into work and hit the reset switch :(
-ryan
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-16 2:48 ` foo
@ 2004-06-16 2:58 ` Andrew Morton
2004-06-16 3:55 ` foo
2004-06-16 4:13 ` foo
0 siblings, 2 replies; 19+ messages in thread
From: Andrew Morton @ 2004-06-16 2:58 UTC (permalink / raw)
To: foo; +Cc: linux-kernel
foo@porto.bmb.uga.edu wrote:
>
> On Tue, Jun 15, 2004 at 04:26:07PM -0700, Andrew Morton wrote:
> > OK, well I'd be suspecting that either devicemapper or raid5 lost an I/O
> > completion, causing that page to never be unlocked.
> >
> > Please try the latest -mm kernel, which has a few devicemapper changes,
> > although they are unlikely to fix this.
>
> OK, this was fun...
>
> LILO 22.2 boot: linux-mm
> Loading Linux-mm.................................
>
> This is all I see on the serial console. The machine did boot, though;
> a few minutes later I see this (and only this) from my syslog server:
>
> xarello kernel: bonding: bond0: link status definitely up for interface
> eth1.
>
> Getty didn't come up on the serial console, and it's refusing ssh
> requests, although it seems to be dropping lots of packets.
Lovely. Please send over the kernel boot command line.
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-16 2:58 ` Andrew Morton
@ 2004-06-16 3:55 ` foo
2004-06-16 4:13 ` foo
1 sibling, 0 replies; 19+ messages in thread
From: foo @ 2004-06-16 3:55 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel
On Tue, Jun 15, 2004 at 07:58:22PM -0700, Andrew Morton wrote:
> foo@porto.bmb.uga.edu wrote:
> >
> > On Tue, Jun 15, 2004 at 04:26:07PM -0700, Andrew Morton wrote:
> > > OK, well I'd be suspecting that either devicemapper or raid5 lost an I/O
> > > completion, causing that page to never be unlocked.
> > >
> > > Please try the latest -mm kernel, which has a few devicemapper changes,
> > > although they are unlikely to fix this.
> >
> > OK, this was fun...
> >
> > LILO 22.2 boot: linux-mm
> > Loading Linux-mm.................................
> >
> > This is all I see on the serial console. The machine did boot, though;
> > a few minutes later I see this (and only this) from my syslog server:
> >
> > xarello kernel: bonding: bond0: link status definitely up for interface
> > eth1.
> >
> > Getty didn't come up on the serial console, and it's refusing ssh
> > requests, although it seems to be dropping lots of packets.
>
> Lovely. Please send over the kernel boot command line.
command line is auto BOOT_IMAGE=Linux ro root=851 console=ttyS0,19200n8
This is the same as for the mainline kernels I've been using.
-ryan
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-16 2:58 ` Andrew Morton
2004-06-16 3:55 ` foo
@ 2004-06-16 4:13 ` foo
2004-06-16 4:25 ` Andrew Morton
1 sibling, 1 reply; 19+ messages in thread
From: foo @ 2004-06-16 4:13 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel
On Tue, Jun 15, 2004 at 07:58:22PM -0700, Andrew Morton wrote:
> foo@porto.bmb.uga.edu wrote:
> >
> > On Tue, Jun 15, 2004 at 04:26:07PM -0700, Andrew Morton wrote:
> > > OK, well I'd be suspecting that either devicemapper or raid5 lost an I/O
> > > completion, causing that page to never be unlocked.
> > >
> > > Please try the latest -mm kernel, which has a few devicemapper changes,
> > > although they are unlikely to fix this.
> >
> > OK, this was fun...
> >
> > LILO 22.2 boot: linux-mm
> > Loading Linux-mm.................................
> >
> > This is all I see on the serial console. The machine did boot, though;
> > a few minutes later I see this (and only this) from my syslog server:
> >
> > xarello kernel: bonding: bond0: link status definitely up for interface
> > eth1.
> >
> > Getty didn't come up on the serial console, and it's refusing ssh
> > requests, although it seems to be dropping lots of packets.
>
> Lovely. Please send over the kernel boot command line.
Here's the dmesg, extracted from the logs.
klogd 1.4.1#10, log source = /proc/kmsg started.
Cannot find map file.
No module symbols loaded - kernel modules not enabled.
Bootdata ok (command line is BOOT_IMAGE=linux-mm ro root=851
console=ttyS0,19200n8)
Linux version 2.6.7-rc3-mm2 (root@carignan) (gcc version 3.3.4
(Debian)) #1 SMP Tue Jun 15 22:30:28 EDT 2004
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000007fff0000 (usable)
BIOS-e820: 000000007fff0000 - 000000007ffff000 (ACPI data)
BIOS-e820: 000000007ffff000 - 0000000080000000 (ACPI NVS)
BIOS-e820: 00000000ff7c0000 - 0000000100000000 (reserved)
Scanning NUMA topology in Northbridge 24
Number of nodes 2 (10010)
Node 0 MemBase 0000000000000000 Limit 000000003fffffff
Node 1 MemBase 0000000040000000 Limit 000000007fff0000
Using node hash shift of 24
Bootmem setup node 0 0000000000000000-000000003fffffff
Bootmem setup node 1 0000000040000000-000000007fff0000
No mptable found.
On node 0 totalpages: 262143
DMA zone: 4096 pages, LIFO batch:1
Normal zone: 258047 pages, LIFO batch:16
HighMem zone: 0 pages, LIFO batch:1
On node 1 totalpages: 262128
DMA zone: 0 pages, LIFO batch:1
Normal zone: 262128 pages, LIFO batch:16
HighMem zone: 0 pages, LIFO batch:1
ACPI: RSDP (v002 ACPIAM ) @
0x00000000000f4680
ACPI: XSDT (v001 A M I OEMXSDT 0x06000318 MSFT 0x00000097) @
0x000000007fff0100
ACPI: FADT (v001 A M I OEMFACP 0x06000318 MSFT 0x00000097) @
0x000000007fff0281
ACPI: MADT (v001 A M I OEMAPIC 0x06000318 MSFT 0x00000097) @
0x000000007fff0380
ACPI: OEMB (v001 A M I OEMBIOS 0x06000318 MSFT 0x00000097) @
0x000000007ffff040
ACPI: ASF! (v001 AMIASF AMDSTRET 0x00000001 INTL 0x02002026) @
0x000000007fff3530
ACPI: DSDT (v001 0ABCF 0ABCF007 0x00000007 INTL 0x02002026) @
0x0000000000000000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Processor #0 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
Processor #1 15:5 APIC version 16
ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
IOAPIC[0]: Assigned apic_id 2
IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x03] address[0xfebfe000] gsi_base[24])
IOAPIC[1]: Assigned apic_id 3
IOAPIC[1]: apic_id 3, version 17, address 0xfebfe000, GSI 24-27
ACPI: IOAPIC (id[0x04] address[0xfebff000] gsi_base[28])
IOAPIC[2]: Assigned apic_id 4
IOAPIC[2]: apic_id 4, version 17, address 0xfebff000, GSI 28-31
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
Using ACPI (MADT) for SMP configuration information
Checking aperture...
CPU 0: aperture @ 1ee0000000 size 64 MB
Aperture from northbridge cpu 0 beyond 4GB. Ignoring.
No AGP bridge found
Built 2 zonelists
Initializing CPU#0
Kernel command line: BOOT_IMAGE=linux-mm ro root=851
console=ttyS0,19200n8
PID hash table entries: 16 (order 4: 256 bytes)
time.c: Using 1.193182 MHz PIT timer.
time.c: Detected 1394.628 MHz processor.
Console: colour VGA+ 80x25
Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
Memory: 2056268k/2097088k available (2581k kernel code, 0k reserved,
1218k data, 180k init)
Calibrating delay loop... 2736.12 BogoMIPS
Mount-cache hash table entries: 256 (order: 0, 4096 bytes)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
Using local APIC NMI watchdog using perfctr0
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU0: AMD Opteron(tm) Processor 240 stepping 01
per-CPU timeslice cutoff: 1023.75 usecs.
task migration cache decay timeout: 2 msecs.
Booting processor 1/1 rip 6000 rsp 10001e39f58
Initializing CPU#1
Calibrating delay loop... 2785.28 BogoMIPS
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
AMD Opteron(tm) Processor 240 stepping 01
Total of 2 processors activated (5521.40 BogoMIPS).
ENABLING IO-APIC IRQs
..TIMER: vector=0x31 pin1=2 pin2=-1
Using local APIC timer interrupts.
Detected 12.452 MHz APIC timer.
checking TSC synchronization across 2 CPUs: passed.
time.c: Using PIT/TSC based timekeeping.
Brought up 2 CPUs
NET: Registered protocol family 16
PCI: Using configuration type 1
mtrr: v2.0 (20020519)
ACPI: Subsystem revision 20040326
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (00:00)
PCI: Probing PCI hardware (bus 00)
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 *5 6 7 9 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 *9 10 11 12 14 15)
SCSI subsystem initialized
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Using ACPI for IRQ routing
ACPI: PCI interrupt 0000:00:07.2[D] -> GSI 19 (level, low) -> IRQ 19
ACPI: PCI interrupt 0000:03:00.0[D] -> GSI 19 (level, low) -> IRQ 19
ACPI: PCI interrupt 0000:03:00.1[D] -> GSI 19 (level, low) -> IRQ 19
ACPI: PCI interrupt 0000:03:04.0[A] -> GSI 16 (level, low) -> IRQ 16
ACPI: PCI interrupt 0000:03:05.0[A] -> GSI 17 (level, low) -> IRQ 17
ACPI: PCI interrupt 0000:03:06.0[A] -> GSI 18 (level, low) -> IRQ 18
ACPI: PCI interrupt 0000:02:09.0[A] -> GSI 24 (level, low) -> IRQ 24
ACPI: PCI interrupt 0000:02:09.1[B] -> GSI 25 (level, low) -> IRQ 25
ACPI: PCI interrupt 0000:02:0a.0[A] -> GSI 24 (level, low) -> IRQ 24
ACPI: PCI interrupt 0000:02:0a.1[B] -> GSI 25 (level, low) -> IRQ 25
testing the IO APIC.......................
.................................... done.
PCI-DMA: Disabling IOMMU.
IA-32 Microcode Update Driver: v1.13 <tigran@veritas.com>
IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak Exp $
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
ACPI: Power Button (FF) [PWRF]
ACPI: Processor [CPU1] (supports C1, 8 throttling states)
ACPI: Processor [CPU2] (supports C1)
lp: driver loaded but no devices found
Real Time Clock Driver v1.12
hpet_acpi_add: no address or irqs in _CRS
hw_random: AMD768 system management I/O registers at 0x5000.
hw_random hardware driver 1.0.0 loaded
Software Watchdog Timer: 0.07 initialized. soft_noboot=0 soft_margin=60
sec (nowayout= 0)
Linux agpgart interface v0.100 (c) Dave Jones
Hangcheck: starting hangcheck timer 0.5.0 (tick is 180 seconds, margin
is 60 seconds).
Serial: 8250/16550 driver $Revision: 1.90 $ 8 ports, IRQ sharing
disabled
ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
parport0: PC-style at 0x378 [PCSPP(,...)]
lp0: using parport0 (polling).
Using anticipatory io scheduler
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
loop: loaded (max 8 devices)
tg3.c:v3.5 (May 25, 2004)
ACPI: PCI interrupt 0000:02:09.0[A] -> GSI 24 (level, low) -> IRQ 24
eth0: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)]
(PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:51:d6:c9
eth0: HostTXDS[0] RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0]
WireSpeed[1] TSOcap[1]
ACPI: PCI interrupt 0000:02:09.1[B] -> GSI 25 (level, low) -> IRQ 25
eth1: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)]
(PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:51:d6:ca
eth1: HostTXDS[0] RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0]
WireSpeed[1] TSOcap[1]
ACPI: PCI interrupt 0000:03:04.0[A] -> GSI 16 (level, low) -> IRQ 16
sym0: <875> rev 0x26 at pci 0000:03:04.0 irq 16
sym0: Symbios NVRAM, ID 7, Fast-20, SE, parity checking
sym0: open drain IRQ line driver, using on-chip SRAM
sym0: using LOAD/STORE-based firmware.
sym0: SCSI BUS has been reset.
scsi0 : sym-2.1.18j
Vendor: SONY Model: SDX-400C Rev: 0702
Type: Sequential-Access ANSI SCSI revision: 02
scsi(0:0:4:0): Beginning Domain Validation
sym0:4: wide asynchronous.
sym0:4: FAST-20 WIDE SCSI 40.0 MB/s ST (50.0 ns, offset 15)
scsi(0:0:4:0): Domain Validation skipping write tests
scsi(0:0:4:0): Ending Domain Validation
Vendor: OVERLAND Model: LIBRARYPRO Rev: 0417
Type: Medium Changer ANSI SCSI revision: 02
scsi(0:0:6:0): Beginning Domain Validation
sym0:6: wide asynchronous.
sym0:6: FAST-10 WIDE SCSI 20.0 MB/s ST (100.0 ns, offset 15)
scsi(0:0:6:0): Domain Validation skipping write tests
scsi(0:0:6:0): Ending Domain Validation
st: Version 20040403, fixed bufsize 32768, s/g segs 256
Attached scsi tape st0 at scsi0, channel 0, id 4, lun 0
st0: try direct i/o: yes (alignment 512 B), max page reachable by HBA
1048575
Attached scsi generic sg0 at scsi0, channel 0, id 4, lun 0, type 1
Attached scsi generic sg1 at scsi0, channel 0, id 6, lun 0, type 8
Fusion MPT base driver 3.01.07
Copyright (c) 1999-2004 LSI Logic Corporation
ACPI: PCI interrupt 0000:02:0a.0[A] -> GSI 24 (level, low) -> IRQ 24
mptbase: Initiating ioc0 bringup
ioc0: 53C1030: Capabilities={Initiator}
ACPI: PCI interrupt 0000:02:0a.1[B] -> GSI 25 (level, low) -> IRQ 25
mptbase: Initiating ioc1 bringup
ioc1: 53C1030: Capabilities={Initiator}
Fusion MPT SCSI Host driver 3.01.07
scsi1 : ioc0: LSI53C1030, FwRev=01030600h, Ports=1, MaxQ=255, IRQ=24
Vendor: SUPER Model: GEM318 Rev: 0
Type: Processor ANSI SCSI revision: 02
Attached scsi generic sg2 at scsi1, channel 0, id 8, lun 0, type 3
Vendor: SEAGATE Model: ST3146807LC Rev: 0006
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sda: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sda: drive cache: write back
sda: sda1
Attached scsi disk sda at scsi1, channel 0, id 9, lun 0
Attached scsi generic sg3 at scsi1, channel 0, id 9, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0006
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdb: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sdb: drive cache: write back
sdb: sdb1
Attached scsi disk sdb at scsi1, channel 0, id 10, lun 0
Attached scsi generic sg4 at scsi1, channel 0, id 10, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0006
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdc: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sdc: drive cache: write back
sdc: sdc1
Attached scsi disk sdc at scsi1, channel 0, id 11, lun 0
Attached scsi generic sg5 at scsi1, channel 0, id 11, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0007
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdd: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sdd: drive cache: write back
sdd: sdd1
Attached scsi disk sdd at scsi1, channel 0, id 12, lun 0
Attached scsi generic sg6 at scsi1, channel 0, id 12, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0007
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sde: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sde: drive cache: write back
sde: sde1
Attached scsi disk sde at scsi1, channel 0, id 13, lun 0
Attached scsi generic sg7 at scsi1, channel 0, id 13, lun 0, type 0
scsi2 : ioc1: LSI53C1030, FwRev=01030600h, Ports=1, MaxQ=255, IRQ=25
Vendor: SEAGATE Model: ST336607LW Rev: 0007
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdf: 71687372 512-byte hdwr sectors (36704 MB)
SCSI device sdf: drive cache: write back
sdf: sdf1 sdf2 sdf3 sdf4
Attached scsi disk sdf at scsi2, channel 0, id 0, lun 0
Attached scsi generic sg8 at scsi2, channel 0, id 0, lun 0, type 0
USB Universal Host Controller Interface driver v2.2
Initializing USB Mass Storage driver...
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
mice: PS/2 mouse device common for all mice
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
i2c /dev entries driver
md: raid1 personality registered as nr 3
md: raid5 personality registered as nr 4
raid5: measuring checksumming speed
generic_sse: 4256.000 MB/sec
raid5: using function: generic_sse (4256.000 MB/sec)
md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
device-mapper: 4.1.0-ioctl (2003-12-10) initialised: dm@uk.sistina.com
NET: Registered protocol family 2
IP: routing cache hash table of 8192 buckets, 128Kbytes
TCP: Hash tables configured (established 262144 bind 65536)
NET: Registered protocol family 1
NET: Registered protocol family 17
md: Autodetecting RAID arrays.
md: autorun ...
md: considering sde1 ...
md: adding sde1 ...
md: adding sdd1 ...
md: adding sdc1 ...
md: adding sdb1 ...
md: adding sda1 ...
md: created md0
md: bind<sda1>
md: bind<sdb1>
md: bind<sdc1>
md: bind<sdd1>
md: bind<sde1>
md: running: <sde1><sdd1><sdc1><sdb1><sda1>
raid5: device sde1 operational as raid disk 4
raid5: device sdd1 operational as raid disk 3
raid5: device sdc1 operational as raid disk 2
raid5: device sdb1 operational as raid disk 1
raid5: device sda1 operational as raid disk 0
raid5: allocated 5300kB for md0
raid5: raid level 5 set md0 active with 5 out of 5 devices, algorithm 2
RAID5 conf printout:
--- rd:5 wd:5 fd:0
disk 0, o:1, dev:sda1
disk 1, o:1, dev:sdb1
disk 2, o:1, dev:sdc1
disk 3, o:1, dev:sdd1
disk 4, o:1, dev:sde1
md: ... autorun DONE.
kjournald starting. Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 180k freed
Adding 2097136k swap on /dev/sdf2. Priority:-1 extents:1
EXT3 FS on sdf1, internal journal
kjournald starting. Commit interval 5 seconds
EXT3 FS on sdf3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-32, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-34, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-33, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-35, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-0, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-4, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-5, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-6, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-7, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-8, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-9, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-10, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-11, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-12, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-13, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-14, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-15, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-16, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-17, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-18, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-19, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-20, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-21, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-22, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-23, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-24, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-25, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-26, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-27, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-28, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-29, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-30, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-31, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
Ethernet Channel Bonding Driver: v2.6.0 (January 14, 2004)
bonding: MII link monitoring set to 100 ms
bonding: bond0: enslaving eth0 as an active interface with a down link.
bonding: bond0: enslaving eth1 as an active interface with a down link.
tg3: eth0: Link is up at 1000 Mbps, full duplex.
tg3: eth0: Flow control is off for TX and off for RX.
bonding: bond0: link status definitely up for interface eth0.
tg3: eth1: Link is up at 100 Mbps, full duplex.
tg3: eth1: Flow control is off for TX and off for RX.
bonding: bond0: link status definitely up for interface eth1.
process `syslogd' is using obsolete setsockopt SO_BSDCOMPAT
-ryan
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-16 4:13 ` foo
@ 2004-06-16 4:25 ` Andrew Morton
2004-06-16 4:41 ` foo
` (2 more replies)
0 siblings, 3 replies; 19+ messages in thread
From: Andrew Morton @ 2004-06-16 4:25 UTC (permalink / raw)
To: foo; +Cc: linux-kernel
foo@porto.bmb.uga.edu wrote:
>
> > Lovely. Please send over the kernel boot command line.
>
> Here's the dmesg, extracted from the logs.
So what do we conclude from this? It booted OK, but the serial console
broke, and perhaps the serial driver broke, and perhaps the tg3 driver
broke?
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-16 4:25 ` Andrew Morton
@ 2004-06-16 4:41 ` foo
2004-06-16 4:53 ` linux-kernel-owner
2004-06-16 7:22 ` tg3 wrong speed foo
2 siblings, 0 replies; 19+ messages in thread
From: foo @ 2004-06-16 4:41 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel
On Tue, Jun 15, 2004 at 09:25:16PM -0700, Andrew Morton wrote:
> So what do we conclude from this? It booted OK, but the serial console
> broke, and perhaps the serial driver broke, and perhaps the tg3 driver
> broke?
Something like that... The serial console was certainly broken. I'm
not so sure about the ethernet. If you see my other messages about the
serial console, the boot process always hangs right after the ethernet
comes up, and I have to hold down a key on the serial console for a few
seconds to get it going again. Maybe coincidence, maybe not. Also it
could be the bonding driver rather than tg3. Let me reboot the mainline
kernel again and I'll see if I get the same packet loss if I leave it
hung for a while.
-ryan
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-16 4:25 ` Andrew Morton
2004-06-16 4:41 ` foo
@ 2004-06-16 4:53 ` linux-kernel-owner
2004-06-16 7:22 ` tg3 wrong speed foo
2 siblings, 0 replies; 19+ messages in thread
From: linux-kernel-owner @ 2004-06-16 4:53 UTC (permalink / raw)
To: Andrew Morton
On Tue, Jun 15, 2004 at 09:25:16PM -0700, Andrew Morton wrote:
> So what do we conclude from this? It booted OK, but the serial console
> broke, and perhaps the serial driver broke, and perhaps the tg3 driver
> broke?
I didn't notice this before, but with -mm eth1 came up in 100Mbit mode,
so I guess there is something wrong with tg3. Since that seems to be
part of the problem, I won't try the experiment I just described unless
you think it would be useful.
-ryan
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: tg3 wrong speed
2004-06-16 4:25 ` Andrew Morton
2004-06-16 4:41 ` foo
2004-06-16 4:53 ` linux-kernel-owner
@ 2004-06-16 7:22 ` foo
2 siblings, 0 replies; 19+ messages in thread
From: foo @ 2004-06-16 7:22 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel
On Tue, Jun 15, 2004 at 09:25:16PM -0700, Andrew Morton wrote:
> So what do we conclude from this? It booted OK, but the serial console
> broke, and perhaps the serial driver broke, and perhaps the tg3 driver
> broke?
I just booted 2.6.7, and eth1 came up in 100Mbit mode as happened with
-mm, and packet loss ensued. I think maybe that the switch is only
allowing 1000Mbit operation. I rebooted into single user mode to
restore the old kernel, but the second time eth1 came up at 1000Mbit so
the networking works now. The serial console is no better than before,
but not completely broken like in -mm.
I'll leave 2.6.7 running today and see how it does.
-ryan
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-16 2:16 ` foo
@ 2004-06-16 12:57 ` Alasdair G Kergon
2004-06-16 19:00 ` foo
2004-06-17 2:28 ` foo
0 siblings, 2 replies; 19+ messages in thread
From: Alasdair G Kergon @ 2004-06-16 12:57 UTC (permalink / raw)
To: foo; +Cc: Andrew Morton, linux-kernel
On Tue, Jun 15, 2004 at 10:16:33PM -0400, foo@porto.bmb.uga.edu wrote:
> # pvdisplay /dev/md0
> # vgdisplay vg0
> # lvdisplay
LVM2/device-mapper diagnostics
==============================
If you use the 'lvs' reporting tool, we'll get fuller information more
concisely, so we can check for any anomalies.
[Unlikely to be any based on what you've posted so far, but worth
eliminating the possibility.]
Run:
vgs -v
lvs -v
pvs -v
Can you confirm that you're not running *any* LVM or device-mapper
commands during your backup sequence?
Confirm the versions by running:
lvm version
dmsetup targets
And dump the static device-mapper state, by running:
dmsetup -v table
dmsetup status
[I'll get that fixed to provide a column-based output too.]
And finally mount sysfs and run
ls -ld .../block/dm-*
[This lot ought to get scripted.]
Alasdair
--
agk@redhat.com
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-16 12:57 ` Alasdair G Kergon
@ 2004-06-16 19:00 ` foo
2004-06-16 19:15 ` Alasdair G Kergon
2004-06-17 2:28 ` foo
1 sibling, 1 reply; 19+ messages in thread
From: foo @ 2004-06-16 19:00 UTC (permalink / raw)
To: Alasdair G Kergon, Andrew Morton, linux-kernel
On Wed, Jun 16, 2004 at 01:57:54PM +0100, Alasdair G Kergon wrote:
> On Tue, Jun 15, 2004 at 10:16:33PM -0400, foo@porto.bmb.uga.edu wrote:
> > # pvdisplay /dev/md0
> > # vgdisplay vg0
> > # lvdisplay
>
> LVM2/device-mapper diagnostics
> ==============================
>
> If you use the 'lvs' reporting tool, we'll get fuller information more
> concisely, so we can check for any anomalies.
> [Unlikely to be any based on what you've posted so far, but worth
> eliminating the possibility.]
>
> Run:
> vgs -v
> lvs -v
> pvs -v
# vgs -v
Finding all volume groups
Finding volume group "vg0"
VG Attr Ext #PV #LV #SN VSize VFree VG UUID
vg0 wz-- 4.00M 1 36 0 546.92G 244.92G EameBy-n1m#-cwNa-rKqz-YqoM-hSEH-BKVeQQ
# lvs -v
Finding all logical volumes
LV VG #Seg Attr LSize Maj Min Origin Snap% Move Move% LV UUID
barolo12 vg0 1 -wn-ao 10.00G -1 -1 YZc9tO-y5wz-ssWJ-WSbI-spgV-Mc9y-6FLTne
barolo34 vg0 1 -wn-ao 10.00G -1 -1 pLm#IR-fDFS-7VPZ-oiI5-Y#Ry-Myat-CE3Pki
barolo5 vg0 1 -wn-ao 5.00G -1 -1 sRjOxq-s71y-mR0e-4Upn-b2Ai-#Og0-MFIEnK
barolo6 vg0 1 -wn-ao 5.00G -1 -1 zqQC0B-3fOk-EBIQ-acmD-iuoK-WJ75-gnUBZN
barolo7 vg0 1 -wn-ao 5.00G -1 -1 XO587u-M123-CLUb-SFsu-67c0-w37#-zoiHv4
barolo8 vg0 1 -wn-ao 5.00G -1 -1 r!EeKN-QMqm-7E3m-0ie#-WHXX-cH5V-1R5dY5
baroloA vg0 1 -wn-ao 5.00G -1 -1 Ew7Tqi-CC4h-IPao-7wVC-Haac-H!PU-916oNP
baroloB vg0 1 -wn-ao 5.00G -1 -1 Gx6Maq-8Noj-xkQH-Lmhc-t4!B-KEYk-e6QstK
baroloC vg0 1 -wn-ao 5.00G -1 -1 kJs2L5-sbfS-MGr0-3GE9-i#gw-#EqV-2nyYB5
baroloD vg0 1 -wn-ao 5.00G -1 -1 miH6a7-o3oE-Jus3-fBpu-Uk9y-HSj2-QR8mZJ
home vg0 1 -wn-ao 25.00G -1 -1 yQFLXG-KLog-TdWQ-Rvis-35U1-jUj1-zO0h3M
mail vg0 1 -wn-ao 1.00G -1 -1 3wldn3-gsCE-3s7u-ezf4-5bcj-vPGv-CKJkX!
merlot1 vg0 1 -wn-ao 5.00G -1 -1 MPrl1Z-CC2!-fOO7-qfzg-waGj-DsL9-bq1CZx
merlot2 vg0 1 -wn-ao 5.00G -1 -1 qWVGBV-xMEj-TWEW-RbQU-ANd3-CaR8-huHrwY
merlot3 vg0 1 -wn-ao 5.00G -1 -1 1YzfGJ-YMOq-oZFQ-0zWc-3dDY-uYPj-gJeOr!
merlot4 vg0 1 -wn-ao 5.00G -1 -1 DSchwn-x0ym-Q8Ru-zPQE-cPJs-MWxU-b#f0IT
merlot5 vg0 1 -wn-ao 5.00G -1 -1 H4k3kV-2rQC-!4It-FxcM-hC4E-!w1N-uU9EsY
merlot6 vg0 1 -wn-ao 5.00G -1 -1 y#YLZw-njzQ-6N2U-hzxK-LgzX-FHyX-Us51Cv
merlot7 vg0 1 -wn-ao 5.00G -1 -1 W3rBLv-6#MS-lDXO-l6FW-#r!4-iO7S-lSF!Oa
merlot8 vg0 1 -wn-ao 5.00G -1 -1 zQ86Hm-qu!I-J4XQ-!saa-lTZd-Yn8G-Q#!VRd
muscat1 vg0 1 -wn-ao 10.00G -1 -1 kDzHiI-NqU9-gJXY-JPNA-eBEK-H8dr-ZLTrkE
muscat2 vg0 1 -wn-ao 10.00G -1 -1 Ldv#Hl-ux9c-rKdS-bSw3-dS8I-HlMY-w3bYxr
muscat3 vg0 1 -wn-ao 10.00G -1 -1 4kPUY9-3aoD-rIB1-Fg5O-Ivh0-QL68-CvQYF5
muscat4 vg0 1 -wn-ao 10.00G -1 -1 OHMsUY-Or4N-9lK8-GXQ3-UBbN-1x60-3484SZ
muscat5 vg0 1 -wn-ao 10.00G -1 -1 jdOxGO-PCOg-IYxm-!U1D-n#p5-wygg-N1ZbuK
muscat6 vg0 1 -wn-ao 10.00G -1 -1 X!PH6U-q6jy-lqcX-e!x3-TgSr-cCyt-RhXdwM
muscat7 vg0 1 -wn-ao 10.00G -1 -1 i2JjoV-N85X-tGn4-YH0!-2433-iAg2-kqFBuJ
muscat8 vg0 1 -wn-ao 10.00G -1 -1 KiaWjv-CouY-rgtV-Xnqe-YWQP-#Odc-#eLOV3
software vg0 1 -wn-ao 20.00G -1 -1 jOWXFS-#QlU-el#j-vqtJ-dm6b-ntL0-aXoEHg
vbah vg0 1 -wn-ao 30.00G -1 -1 SG3vYW-OT26-SZhL-66f1-FrDl-Ml5N-YUfAe4
vcaa vg0 1 -wn-ao 5.00G -1 -1 b4B540-c!K!-kJUe-bGk9-slM6-qSOf-bqh1jB
vidb vg0 1 -wn-ao 1.00G -1 -1 Gf3dyd-1j#i-f2rf-LIzp-eZdr-jnrR-oyahVN
xarello1 vg0 1 -wn-ao 10.00G -1 -1 GbVm!D-OzE5-2Ecs-G4Mz-HPL5-7E31-fU5kBP
xarello2 vg0 1 -wn-ao 10.00G -1 -1 pPrVSQ-lyRz-t7Ml-hG4k-cFvg-jtMm-p0#Ot0
xarello3 vg0 1 -wn-ao 10.00G -1 -1 gsvzH3-SmcT-jp2q-RdHt-nO81-gquH-zqdaLH
xarello4 vg0 1 -wn-ao 10.00G -1 -1 dQLM2u-SdsS-VENi-rTA0-EM9T-KhHt-m39gwf
# pvs -v
Scanning for physical volume names
PV VG Fmt Attr PSize PFree PV UUID
/dev/md0 vg0 lvm2 a- 546.92G 244.92G a76cj4-Fvnf-xkY8-bE0j-Wcdk-jtb0-mq9NbF
> Can you confirm that you're not running *any* LVM or device-mapper
> commands during your backup sequence?
I'm using the same backup script that I was using before without LVM,
there are no snapshots or anything. It's just Amanda with a simple
script to run it.
> Confirm the versions by running:
> lvm version
# lvm version
LVM version: 2.00.06 (2003-08-20)
Library version: 1.00.05-ioctl-cvs (2003-09-01)
Driver version: 4.1.0
> dmsetup targets
The dmsetup I have installed doesn't have this command.
>
> And dump the static device-mapper state, by running:
> dmsetup -v table
> dmsetup status
# dmsetup -v table /dev/mapper/vg0-home
Name: vg0-home
State: ACTIVE
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 0
Number of targets: 1
UUID: EameByn1m#cwNarKqzYqoMhSEHBKVeQQyQFLXGKLogTdWQRvis35U1jUj1zO0h3M
# dmsetup -v status /dev/mapper/vg0-home
Name: vg0-home
State: ACTIVE
Tables present: LIVE
Open count: 1
Event number: 0
Major, minor: 253, 0
Number of targets: 1
UUID: EameByn1m#cwNarKqzYqoMhSEHBKVeQQyQFLXGKLogTdWQRvis35U1jUj1zO0h3M
0 52428800 linear
>
> [I'll get that fixed to provide a column-based output too.]
>
> And finally mount sysfs and run
> ls -ld .../block/dm-*
# ls -ld /sys/block/dm-*
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-0/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-1/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-10/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-11/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-12/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-13/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-14/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-15/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-16/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-17/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-18/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-19/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-2/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-20/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-21/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-22/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-23/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-24/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-25/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-26/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-27/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-28/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-29/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-3/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-30/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-31/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-32/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-33/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-34/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-35/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-4/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-5/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-6/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-7/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-8/
drwxr-xr-x 2 root root 0 Jun 16 03:11 /sys/block/dm-9/
-ryan
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-16 19:00 ` foo
@ 2004-06-16 19:15 ` Alasdair G Kergon
0 siblings, 0 replies; 19+ messages in thread
From: Alasdair G Kergon @ 2004-06-16 19:15 UTC (permalink / raw)
To: foo; +Cc: Andrew Morton, linux-kernel
On Wed, Jun 16, 2004 at 03:00:56PM -0400, foo@porto.bmb.uga.edu wrote:
[diagnostic output]
All looks normal - no accidentally-suspended devices.
Alasdair
--
agk@redhat.com
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: processes hung in D (raid5/dm/ext3)
2004-06-16 12:57 ` Alasdair G Kergon
2004-06-16 19:00 ` foo
@ 2004-06-17 2:28 ` foo
1 sibling, 0 replies; 19+ messages in thread
From: foo @ 2004-06-17 2:28 UTC (permalink / raw)
To: Alasdair G Kergon, Andrew Morton, linux-kernel
I just had another go at this with 2.6.7. I tried some different things
this time.
First I tried taring the filesystem, and that went OK. Then I tried find
-exec cat and that was OK, too. Next I did cat /dev/md0, and when that
was OK I did cat /dev/vg0/home and that also worked. Interestingly, cat
/dev/md0 gets about 120-130MB/s throughput whereas cat /dev/vg0/home
only gets 60-70MB/s.
Fianlly I tried dump again and that hung right away, trace below. I
also got an ls hung before I made the trace, but that doesn't seem to
show anything new.
A possibly interesting note about the serial console: when I triggered
the trace, apcupsd lost communication with the UPS, which is using the
other serial port (this is while the trace was printing to the console).
Communication resumed when it was done printing. The serial ports don't
share an interrupt, with each other or anything else.
There was no ethernet grief when I rebooted.
Bootdata ok (command line is BOOT_IMAGE=linux ro root=851 console=ttyS0,19200n8 single)
Linux version 2.6.7 (root@carignan) (gcc version 3.3.4 (Debian)) #1 SMP Wed Jun 16 02:55:47 EDT 2004
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000007fff0000 (usable)
BIOS-e820: 000000007fff0000 - 000000007ffff000 (ACPI data)
BIOS-e820: 000000007ffff000 - 0000000080000000 (ACPI NVS)
BIOS-e820: 00000000ff7c0000 - 0000000100000000 (reserved)
Scanning NUMA topology in Northbridge 24
Number of nodes 2 (10010)
Node 0 MemBase 0000000000000000 Limit 000000003fffffff
Node 1 MemBase 0000000040000000 Limit 000000007fff0000
Using node hash shift of 24
Bootmem setup node 0 0000000000000000-000000003fffffff
Bootmem setup node 1 0000000040000000-000000007fff0000
No mptable found.
On node 0 totalpages: 262143
DMA zone: 4096 pages, LIFO batch:1
Normal zone: 258047 pages, LIFO batch:16
HighMem zone: 0 pages, LIFO batch:1
On node 1 totalpages: 262128
DMA zone: 0 pages, LIFO batch:1
Normal zone: 262128 pages, LIFO batch:16
HighMem zone: 0 pages, LIFO batch:1
ACPI: RSDP (v002 ACPIAM ) @ 0x00000000000f4680
ACPI: XSDT (v001 A M I OEMXSDT 0x06000318 MSFT 0x00000097) @ 0x000000007fff0100
ACPI: FADT (v001 A M I OEMFACP 0x06000318 MSFT 0x00000097) @ 0x000000007fff0281
ACPI: MADT (v001 A M I OEMAPIC 0x06000318 MSFT 0x00000097) @ 0x000000007fff0380
ACPI: OEMB (v001 A M I OEMBIOS 0x06000318 MSFT 0x00000097) @ 0x000000007ffff040
ACPI: ASF! (v001 AMIASF AMDSTRET 0x00000001 INTL 0x02002026) @ 0x000000007fff3530
ACPI: DSDT (v001 0ABCF 0ABCF007 0x00000007 INTL 0x02002026) @ 0x0000000000000000
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Processor #0 15:5 APIC version 16
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
Processor #1 15:5 APIC version 16
ACPI: IOAPIC (id[0x02] address[0xfec00000] global_irq_base[0x0])
IOAPIC[0]: Assigned apic_id 2
IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
ACPI: IOAPIC (id[0x03] address[0xfebfe000] global_irq_base[0x18])
IOAPIC[1]: Assigned apic_id 3
IOAPIC[1]: apic_id 3, version 17, address 0xfebfe000, GSI 24-27
ACPI: IOAPIC (id[0x04] address[0xfebff000] global_irq_base[0x1c])
IOAPIC[2]: Assigned apic_id 4
IOAPIC[2]: apic_id 4, version 17, address 0xfebff000, GSI 28-31
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
Checking aperture...
CPU 0: aperture @ 1ee0000000 size 64 MB
Aperture from northbridge cpu 0 beyond 4GB. Ignoring.
No AGP bridge found
Built 2 zonelists
Kernel command line: BOOT_IMAGE=linux ro root=851 console=ttyS0,19200n8 single
Initializing CPU#0
PID hash table entries: 16 (order 4: 256 bytes)
time.c: Using 1.193182 MHz PIT timer.
time.c: Detected 1394.599 MHz processor.
Console: colour VGA+ 80x25
Memory: 2062532k/2097088k available (2541k kernel code, 0k reserved, 1205k data, 180k init)
Calibrating delay loop... 2736.12 BogoMIPS
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Mount-cache hash table entries: 256 (order: 0, 4096 bytes)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
Using local APIC NMI watchdog using perfctr0
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU0: AMD Opteron(tm) Processor 240 stepping 01
per-CPU timeslice cutoff: 1023.75 usecs.
task migration cache decay timeout: 2 msecs.
Booting processor 1/1 rip 6000 rsp 10040e3ff58
Initializing CPU#1
Calibrating delay loop... 2785.28 BogoMIPS
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
AMD Opteron(tm) Processor 240 stepping 01
Total of 2 processors activated (5521.40 BogoMIPS).
ENABLING IO-APIC IRQs
init IO_APIC IRQs
IO-APIC (apicid-pin) 2-0, 2-16, 2-17, 2-18, 2-19, 2-20, 2-21, 2-22, 2-23, 3-0, 3-1, 3-2, 3-3, 4-0, 4-1, 4-2, 4-3 not connected.
..TIMER: vector=0x31 pin1=2 pin2=-1
Using local APIC timer interrupts.
Detected 12.451 MHz APIC timer.
checking TSC synchronization across 2 CPUs: passed.
time.c: Using PIT/TSC based timekeeping.
Brought up 2 CPUs
CPU0: online
domain 0: span 01
groups: 01
domain 1: span 03
groups: 01 02
CPU1: online
domain 0: span 02
groups: 02
domain 1: span 03
groups: 02 01
NET: Registered protocol family 16
PCI: Using configuration type 1
mtrr: v2.0 (20020519)
ACPI: Subsystem revision 20040326
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (00:00)
PCI: Probing PCI hardware (bus 00)
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PCI1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.GOLA._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.GOLB._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 *5 6 7 9 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 *9 10 11 12 14 15)
SCSI subsystem initialized
usbcore: registered new driver usbfs
usbcore: registered new driver hub
IOAPIC[0]: Set PCI routing entry (2-16 -> 0xa9 -> IRQ 16 Mode:1 Active:1)
00:00:07[A] -> 2-16 -> IRQ 16
IOAPIC[0]: Set PCI routing entry (2-17 -> 0xb1 -> IRQ 17 Mode:1 Active:1)
00:00:07[B] -> 2-17 -> IRQ 17
IOAPIC[0]: Set PCI routing entry (2-18 -> 0xb9 -> IRQ 18 Mode:1 Active:1)
00:00:07[C] -> 2-18 -> IRQ 18
IOAPIC[0]: Set PCI routing entry (2-19 -> 0xc1 -> IRQ 19 Mode:1 Active:1)
00:00:07[D] -> 2-19 -> IRQ 19
IOAPIC[1]: Set PCI routing entry (3-3 -> 0xc9 -> IRQ 27 Mode:1 Active:1)
00:02:08[A] -> 3-3 -> IRQ 27
IOAPIC[1]: Set PCI routing entry (3-0 -> 0xd1 -> IRQ 24 Mode:1 Active:1)
00:02:08[B] -> 3-0 -> IRQ 24
IOAPIC[1]: Set PCI routing entry (3-1 -> 0xd9 -> IRQ 25 Mode:1 Active:1)
00:02:08[C] -> 3-1 -> IRQ 25
IOAPIC[1]: Set PCI routing entry (3-2 -> 0xe1 -> IRQ 26 Mode:1 Active:1)
00:02:08[D] -> 3-2 -> IRQ 26
IOAPIC[2]: Set PCI routing entry (4-0 -> 0xe9 -> IRQ 28 Mode:1 Active:1)
00:01:03[A] -> 4-0 -> IRQ 28
IOAPIC[2]: Set PCI routing entry (4-1 -> 0x32 -> IRQ 29 Mode:1 Active:1)
00:01:03[B] -> 4-1 -> IRQ 29
IOAPIC[2]: Set PCI routing entry (4-2 -> 0x3a -> IRQ 30 Mode:1 Active:1)
00:01:03[C] -> 4-2 -> IRQ 30
IOAPIC[2]: Set PCI routing entry (4-3 -> 0x42 -> IRQ 31 Mode:1 Active:1)
00:01:03[D] -> 4-3 -> IRQ 31
number of MP IRQ sources: 16.
number of IO-APIC #2 registers: 24.
number of IO-APIC #3 registers: 4.
number of IO-APIC #4 registers: 4.
testing the IO APIC.......................
IO APIC #2......
.... register #00: 02000000
....... : physical APIC id: 02
.... register #01: 00170011
....... : max redirection entries: 0017
....... : PRQ implemented: 0
....... : IO APIC version: 0011
.... register #02: 02000000
....... : arbitration: 02
.... IRQ redirection table:
NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
00 000 00 1 0 0 0 0 0 0 00
01 001 01 0 0 0 0 0 1 1 39
02 001 01 0 0 0 0 0 1 1 31
03 001 01 0 0 0 0 0 1 1 41
04 001 01 0 0 0 0 0 1 1 49
05 001 01 0 0 0 0 0 1 1 51
06 001 01 0 0 0 0 0 1 1 59
07 001 01 0 0 0 0 0 1 1 61
08 001 01 0 0 0 0 0 1 1 69
09 001 01 0 1 0 1 0 1 1 71
0a 001 01 0 0 0 0 0 1 1 79
0b 001 01 0 0 0 0 0 1 1 81
0c 001 01 0 0 0 0 0 1 1 89
0d 001 01 0 0 0 0 0 1 1 91
0e 001 01 0 0 0 0 0 1 1 99
0f 001 01 0 0 0 0 0 1 1 A1
10 001 01 1 1 0 1 0 1 1 A9
11 001 01 1 1 0 1 0 1 1 B1
12 001 01 1 1 0 1 0 1 1 B9
13 001 01 1 1 0 1 0 1 1 C1
14 000 00 1 0 0 0 0 0 0 00
15 000 00 1 0 0 0 0 0 0 00
16 000 00 1 0 0 0 0 0 0 00
17 000 00 1 0 0 0 0 0 0 00
IO APIC #3......
.... register #00: 03000000
....... : physical APIC id: 03
.... register #01: 00030011
....... : max redirection entries: 0003
....... : PRQ implemented: 0
....... : IO APIC version: 0011
.... register #02: 00000000
....... : arbitration: 00
.... IRQ redirection table:
NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
00 001 01 1 1 0 1 0 1 1 D1
01 001 01 1 1 0 1 0 1 1 D9
02 001 01 1 1 0 1 0 1 1 E1
03 001 01 1 1 0 1 0 1 1 C9
IO APIC #4......
.... register #00: 04000000
....... : physical APIC id: 04
.... register #01: 00030011
....... : max redirection entries: 0003
....... : PRQ implemented: 0
....... : IO APIC version: 0011
.... register #02: 00000000
....... : arbitration: 00
.... IRQ redirection table:
NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
00 001 01 1 1 0 1 0 1 1 E9
01 001 01 1 1 0 1 0 1 1 32
02 001 01 1 1 0 1 0 1 1 3A
03 001 01 1 1 0 1 0 1 1 42
IRQ to pin mappings:
IRQ0 -> 0:2
IRQ1 -> 0:1
IRQ3 -> 0:3
IRQ4 -> 0:4
IRQ5 -> 0:5
IRQ6 -> 0:6
IRQ7 -> 0:7
IRQ8 -> 0:8
IRQ9 -> 0:9
IRQ10 -> 0:10
IRQ11 -> 0:11
IRQ12 -> 0:12
IRQ13 -> 0:13
IRQ14 -> 0:14
IRQ15 -> 0:15
IRQ16 -> 0:16
IRQ17 -> 0:17
IRQ18 -> 0:18
IRQ19 -> 0:19
IRQ24 -> 1:0
IRQ25 -> 1:1
IRQ26 -> 1:2
IRQ27 -> 1:3
IRQ28 -> 2:0
IRQ29 -> 2:1
IRQ30 -> 2:2
IRQ31 -> 2:3
.................................... done.
PCI: Using ACPI for IRQ routing
PCI-DMA: Disabling IOMMU.
IA-32 Microcode Update Driver: v1.13 <tigran@veritas.com>
IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak Exp $
Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
ACPI: Power Button (FF) [PWRF]
ACPI: Processor [CPU1] (supports C1, 8 throttling states)
ACPI: Processor [CPU2] (supports C1)
lp: driver loaded but no devices found
Real Time Clock Driver v1.12
hw_random: AMD768 system management I/O registers at 0x5000.
hw_random hardware driver 1.0.0 loaded
Software Watchdog Timer: 0.07 initialized. soft_noboot=0 soft_margin=60 sec (nowayout= 0)
Linux agpgart interface v0.100 (c) Dave Jones
Hangcheck: starting hangcheck timer 0.5.0 (tick is 180 seconds, margin is 60 seconds).
Serial: 8250/16550 driver $Revision: 1.90 $ 8 ports, IRQ sharing disabled
ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
parport0: PC-style at 0x378 [PCSPP(,...)]
lp0: using parport0 (polling).
Using anticipatory io scheduler
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
loop: loaded (max 8 devices)
tg3.c:v3.6 (June 12, 2004)
eth0: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:51:d6:c9
eth0: HostTXDS[1] RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1]
eth1: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:51:d6:ca
eth1: HostTXDS[1] RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1]
sym0: <875> rev 0x26 at pci 0000:03:04.0 irq 16
sym0: Symbios NVRAM, ID 7, Fast-20, SE, parity checking
sym0: open drain IRQ line driver, using on-chip SRAM
sym0: using LOAD/STORE-based firmware.
sym0: SCSI BUS has been reset.
scsi0 : sym-2.1.18j
Vendor: SONY Model: SDX-400C Rev: 0702
Type: Sequential-Access ANSI SCSI revision: 02
scsi(0:0:4:0): Beginning Domain Validation
sym0:4: wide asynchronous.
sym0:4: FAST-20 WIDE SCSI 40.0 MB/s ST (50.0 ns, offset 15)
scsi(0:0:4:0): Domain Validation skipping write tests
scsi(0:0:4:0): Ending Domain Validation
Vendor: OVERLAND Model: LIBRARYPRO Rev: 0417
Type: Medium Changer ANSI SCSI revision: 02
scsi(0:0:6:0): Beginning Domain Validation
sym0:6: wide asynchronous.
sym0:6: FAST-10 WIDE SCSI 20.0 MB/s ST (100.0 ns, offset 15)
scsi(0:0:6:0): Domain Validation skipping write tests
scsi(0:0:6:0): Ending Domain Validation
st: Version 20040403, fixed bufsize 32768, s/g segs 256
Attached scsi tape st0 at scsi0, channel 0, id 4, lun 0
st0: try direct i/o: yes (alignment 512 B), max page reachable by HBA 1048575
Attached scsi generic sg0 at scsi0, channel 0, id 4, lun 0, type 1
Attached scsi generic sg1 at scsi0, channel 0, id 6, lun 0, type 8
Fusion MPT base driver 3.01.06
Copyright (c) 1999-2004 LSI Logic Corporation
mptbase: Initiating ioc0 bringup
ioc0: 53C1030: Capabilities={Initiator}
mptbase: Initiating ioc1 bringup
ioc1: 53C1030: Capabilities={Initiator}
Fusion MPT SCSI Host driver 3.01.06
scsi1 : ioc0: LSI53C1030, FwRev=01030600h, Ports=1, MaxQ=255, IRQ=24
Vendor: SUPER Model: GEM318 Rev: 0
Type: Processor ANSI SCSI revision: 02
Attached scsi generic sg2 at scsi1, channel 0, id 8, lun 0, type 3
Vendor: SEAGATE Model: ST3146807LC Rev: 0006
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sda: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sda: drive cache: write back
sda: sda1
Attached scsi disk sda at scsi1, channel 0, id 9, lun 0
Attached scsi generic sg3 at scsi1, channel 0, id 9, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0006
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdb: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sdb: drive cache: write back
sdb: sdb1
Attached scsi disk sdb at scsi1, channel 0, id 10, lun 0
Attached scsi generic sg4 at scsi1, channel 0, id 10, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0006
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdc: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sdc: drive cache: write back
sdc: sdc1
Attached scsi disk sdc at scsi1, channel 0, id 11, lun 0
Attached scsi generic sg5 at scsi1, channel 0, id 11, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0007
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdd: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sdd: drive cache: write back
sdd: sdd1
Attached scsi disk sdd at scsi1, channel 0, id 12, lun 0
Attached scsi generic sg6 at scsi1, channel 0, id 12, lun 0, type 0
Vendor: SEAGATE Model: ST3146807LC Rev: 0007
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sde: 286749488 512-byte hdwr sectors (146816 MB)
SCSI device sde: drive cache: write back
sde: sde1
Attached scsi disk sde at scsi1, channel 0, id 13, lun 0
Attached scsi generic sg7 at scsi1, channel 0, id 13, lun 0, type 0
scsi2 : ioc1: LSI53C1030, FwRev=01030600h, Ports=1, MaxQ=255, IRQ=25
Vendor: SEAGATE Model: ST336607LW Rev: 0007
Type: Direct-Access ANSI SCSI revision: 03
SCSI device sdf: 71687372 512-byte hdwr sectors (36704 MB)
SCSI device sdf: drive cache: write back
sdf: sdf1 sdf2 sdf3 sdf4
Attached scsi disk sdf at scsi2, channel 0, id 0, lun 0
Attached scsi generic sg8 at scsi2, channel 0, id 0, lun 0, type 0
USB Universal Host Controller Interface driver v2.2
Initializing USB Mass Storage driver...
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
mice: PS/2 mouse device common for all mice
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
i2c /dev entries driver
md: raid1 personality registered as nr 3
md: raid5 personality registered as nr 4
raid5: measuring checksumming speed
generic_sse: 4256.000 MB/sec
raid5: using function: generic_sse (4256.000 MB/sec)
md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
device-mapper: 4.1.0-ioctl (2003-12-10) initialised: dm@uk.sistina.com
NET: Registered protocol family 2
IP: routing cache hash table of 8192 buckets, 128Kbytes
TCP: Hash tables configured (established 262144 bind 65536)
NET: Registered protocol family 1
NET: Registered protocol family 17
md: Autodetecting RAID arrays.
md: autorun ...
md: considering sde1 ...
md: adding sde1 ...
md: adding sdd1 ...
md: adding sdc1 ...
md: adding sdb1 ...
md: adding sda1 ...
md: created md0
md: bind<sda1>
md: bind<sdb1>
md: bind<sdc1>
md: bind<sdd1>
md: bind<sde1>
md: running: <sde1><sdd1><sdc1><sdb1><sda1>
raid5: device sde1 operational as raid disk 4
raid5: device sdd1 operational as raid disk 3
raid5: device sdc1 operational as raid disk 2
raid5: device sdb1 operational as raid disk 1
raid5: device sda1 operational as raid disk 0
raid5: allocated 5300kB for md0
raid5: raid level 5 set md0 active with 5 out of 5 devices, algorithm 2
RAID5 conf printout:
--- rd:5 wd:5 fd:0
disk 0, o:1, dev:sda1
disk 1, o:1, dev:sdb1
disk 2, o:1, dev:sdc1
disk 3, o:1, dev:sdd1
disk 4, o:1, dev:sde1
md: ... autorun DONE.
kjournald starting. Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 180k freed
Adding 2097136k swap on /dev/sdf2. Priority:-1 extents:1
EXT3 FS on sdf1, internal journal
kjournald starting. Commit interval 5 seconds
EXT3 FS on sdf3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-32, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-34, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-33, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-35, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-0, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-4, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-5, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-6, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-7, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-8, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-9, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-10, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-11, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-12, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-13, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-14, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-15, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-16, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-17, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-18, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-19, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-20, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-21, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-22, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-23, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-24, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-25, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-26, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-27, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-28, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-29, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-30, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on dm-31, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
Ethernet Channel Bonding Driver: v2.6.0 (January 14, 2004)
bonding: MII link monitoring set to 100 ms
bonding: bond0: enslaving eth0 as an active interface with a down link.
bonding: bond0: enslaving eth1 as an active interface with a down link.
tg3: eth0: Link is up at 1000 Mbps, full duplex.
tg3: eth0: Flow control is off for TX and off for RX.
tg3: eth1: Link is up at 1000 Mbps, full duplex.
tg3: eth1: Flow control is off for TX and off for RX.
bonding: bond0: link status definitely up for interface eth0.
bonding: bond0: link status definitely up for interface eth1.
process `syslogd' is using obsolete setsockopt SO_BSDCOMPAT
SCSI-3 Opcodes & ASC/ASCQ Strings 3.01.06
mptbase: English readable SCSI-3 strings enabled:-)
isense: Registered SCSI-3 Opcodes & ASC/ASCQ Strings
st0: Block limits 2 - 16777215 bytes.
rpc.mountd[752]: segfault at 000000004b87ad6e rip 00000000080503e9 rsp 00000000ffffd684 error 4
nfsd: last server has exited
nfsd: unexporting all filesystems
rpc.mountd[3959]: segfault at 000000004b87ad6e rip 00000000080503e9 rsp 00000000ffffd494 error 4
nfsd: last server has exited
nfsd: unexporting all filesystems
rpc.mountd[3996]: segfault at 000000004b87ad6e rip 00000000080503e9 rsp 00000000ffffd494 error 4
nfsd: last server has exited
nfsd: unexporting all filesystems
rpciod: active tasks at shutdown?!
RPC: error 5 connecting to server localhost
RPC: failed to contact portmap (errno -5).
SysRq : Show State
sibling
task PC pid father child younger older
init S 00003e2596edc7f2 0 1 0 2 (NOTLB)
000001003ff8fd88 0000000000000006 0000007800000002 000001007ff9b070
0000000000000626 ffffffff803e23e0 000001007ff9b388 0000000000000246
0000000000000216 ffffffff8015a512
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377ef4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff801918ca>{do_select+906}
<ffffffff80186c64>{vfs_stat+68} <ffffffff80191380>{__pollwait+0}
<ffffffff80186bcb>{vfs_getattr+59} <ffffffff801ab8a1>{compat_sys_select+1105}
<ffffffff80123f81>{ia32_sysret+0}
migration/0 S 0000000000000000 0 2 1 3 (L-TLB)
0000010040e73ed8 0000000000000046 00000074349d5cc8 000001007ff9a030
00000000000001ec 000001003e29e8d0 000001007ff9a348 00000100349d5cb8
00000100349d5cc0 0000000000000000
Call Trace:<ffffffff8013487c>{migration_thread+236} <ffffffff80134790>{migration_thread+0}
<ffffffff80134790>{migration_thread+0} <ffffffff8014e502>{kthread+146}
<ffffffff801115f7>{child_rip+8} <ffffffff80134790>{migration_thread+0}
<ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}
ksoftirqd/0 S 0000000000000000 0 3 1 4 2 (L-TLB)
0000010040e71f08 0000000000000046 0000007d40e71e98 0000010001e6d0b0
00000000000000b7 000001003e61b370 0000010001e6d3c8 0000010040e71e98
0000010001e12b20 0000000000000000
Call Trace:<ffffffff8013d0ff>{tasklet_action+111} <ffffffff8013d2a0>{ksoftirqd+0}
<ffffffff8013d2df>{ksoftirqd+63} <ffffffff8014e502>{kthread+146}
<ffffffff801115f7>{child_rip+8} <ffffffff8013d2a0>{ksoftirqd+0}
<ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}
migration/1 S 0000000000000000 0 4 1 5 3 (L-TLB)
0000010001e6bed8 0000000000000046 000000795435fcc8 0000010001e6c890
00000000000000fc 000001007ed104b0 0000010001e6cba8 000001005435fcb8
000001005435fcc0 0000000000000001
Call Trace:<ffffffff8013487c>{migration_thread+236} <ffffffff80134790>{migration_thread+0}
<ffffffff80134790>{migration_thread+0} <ffffffff8014e502>{kthread+146}
<ffffffff801115f7>{child_rip+8} <ffffffff80134790>{migration_thread+0}
<ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}
ksoftirqd/1 S 00003e1fba66ad5c 0 5 1 6 4 (L-TLB)
0000010001e69f08 0000000000000046 0000010001e69e98 0000010001e6c070
000000000000010f 000001007ff9a850 0000010001e6c388 0000010001e69e98
0000010040e1e9a0 0000000000000000
Call Trace:<ffffffff8013d0ff>{tasklet_action+111} <ffffffff8013d2a0>{ksoftirqd+0}
<ffffffff8013d2df>{ksoftirqd+63} <ffffffff8014e502>{kthread+146}
<ffffffff801115f7>{child_rip+8} <ffffffff8013d2a0>{ksoftirqd+0}
<ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}
events/0 R running task 0 6 1 8 7 5 (L-TLB)
events/1 S 00003e267f613016 0 7 1 42 6 (L-TLB)
000001007ff7fe58 0000000000000046 0000007300000212 0000010040e408d0
000000000000039f 000001007ff9a850 0000010040e40be8 0000000000000001
0000000000000003 0000000000000000
Call Trace:<ffffffff80149df5>{worker_thread+261} <ffffffff803774bc>{thread_return+41}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff80149cf0>{worker_thread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
khelper S 000000067c85eea7 0 8 6 9 (L-TLB)
0000010001e4fe58 0000000000000046 000001007f459d38 0000010040e400b0
000000000000024d 000001007ff9a850 0000010040e403c8 0000000000000001
0000000000000003 0000000000000000
Call Trace:<ffffffff801115ef>{child_rip+0} <ffffffff80149df5>{worker_thread+261}
<ffffffff803774bc>{thread_return+41} <ffffffff80133440>{default_wake_function+0}
<ffffffff80133440>{default_wake_function+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff8014e502>{kthread+146}
<ffffffff801115f7>{child_rip+8} <ffffffff8014e540>{keventd_create_kthread+0}
<ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}
kacpid S 00000000087a440f 0 9 6 40 8 (L-TLB)
000001007fefde58 0000000000000046 000001007fefde18 0000010001e8d130
0000000000002db2 000001007ff9a850 0000010001e8d448 0000000000000001
0000000000010000 0000010001e8d130
Call Trace:<ffffffff80146273>{do_sigaction+659} <ffffffff80145361>{sigprocmask+241}
<ffffffff80149df5>{worker_thread+261} <ffffffff803774bc>{thread_return+41}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
kblockd/0 S 00003e224ad18831 0 40 6 41 9 (L-TLB)
0000010001fb9e58 0000000000000046 0000010040e59800 000001007fec29d0
000000000000010b ffffffff803e23e0 000001007fec2ce8 0000000000000001
0000000000000003 0000000000000000
Call Trace:<ffffffff80149df5>{worker_thread+261} <ffffffff803774bc>{thread_return+41}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
kblockd/1 S 00003e266f08347a 0 41 6 54 40 (L-TLB)
000001007fe95e58 0000000000000046 0000007300000202 000001007fec21b0
000000000000004f 0000010001fe8a50 000001007fec24c8 0000000000000001
0000000000000003 0000000000000000
Call Trace:<ffffffff80149df5>{worker_thread+261} <ffffffff803774bc>{thread_return+41}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
khubd S 000000001a81861e 0 42 1 56 7 (L-TLB)
000001007fe63f08 0000000000000046 0000000000000000 000001007febf230
0000000000002cc2 000001007ff9a850 000001007febf548 0000000000000000
0000000000000000 0000000000000216
Call Trace:<ffffffff802d18b1>{hub_thread+161} <ffffffff80133440>{default_wake_function+0}
<ffffffff801115f7>{child_rip+8} <ffffffff802d1810>{hub_thread+0}
<ffffffff801115ef>{child_rip+0}
pdflush S 00000000242315c5 0 54 6 55 41 (L-TLB)
0000010001f77e98 0000000000000046 0000006a01e0f760 0000010001fce2f0
0000000000000622 0000010040e410f0 0000010001fce608 0000010040e410f0
0000010001e0f760 0000010001e100a0
Call Trace:<ffffffff8015cae0>{pdflush+0} <ffffffff8015c8c5>{__pdflush+261}
<ffffffff8015cae0>{pdflush+0} <ffffffff8015cafc>{pdflush+28}
<ffffffff8015cae0>{pdflush+0} <ffffffff8014e502>{kthread+146}
<ffffffff801115f7>{child_rip+8} <ffffffff8014e540>{keventd_create_kthread+0}
<ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}
pdflush S 00003e266dbb5896 0 55 6 58 54 (L-TLB)
000001007fe5fe98 0000000000000046 0000007300000000 0000010001fceb10
000000000000055d 0000010001fe8a50 0000010001fcee28 0000000000000000
000001007fe5fe18 00000000000003cb
Call Trace:<ffffffff8015cae0>{pdflush+0} <ffffffff8015c8c5>{__pdflush+261}
<ffffffff8015cae0>{pdflush+0} <ffffffff8015cafc>{pdflush+28}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
kswapd1 S 00003e1d6861ae1a 0 56 1 57 42 (L-TLB)
0000010001f75eb8 0000000000000046 0000000000000000 0000010001fcf330
0000000000022d38 000001007ff9a850 0000010001fcf648 0000000000000202
00000000000000ef 0000000000000000
Call Trace:<ffffffff80164af2>{balance_pgdat+2} <ffffffff80164e6f>{kswapd+239}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff80131321>{schedule_tail+17} <ffffffff801115f7>{child_rip+8}
<ffffffff80164d80>{kswapd+0} <ffffffff801115ef>{child_rip+0}
kswapd0 S 0000000000000000 0 57 1 170 56 (L-TLB)
000001007fe5deb8 0000000000000046 0000007500000000 000001007feba2b0
00000000000227e7 0000010001e8c910 000001007feba5c8 0000000000000202
00000000000000eb 0000000000000100
Call Trace:<ffffffff80164e6f>{kswapd+239} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff80131321>{schedule_tail+17}
<ffffffff801115f7>{child_rip+8} <ffffffff80164d80>{kswapd+0}
<ffffffff801115ef>{child_rip+0}
aio/0 S 0000000000000000 0 58 6 59 55 (L-TLB)
000001007fe4be58 0000000000000046 0000006b7fe4be18 000001007febaad0
000000000000044b 0000010040e410f0 000001007febade8 0000000000000001
0000000000010000 000001007febaad0
Call Trace:<ffffffff80146273>{do_sigaction+659} <ffffffff80145361>{sigprocmask+241}
<ffffffff80149df5>{worker_thread+261} <ffffffff803774bc>{thread_return+41}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
aio/1 S 0000000024252125 0 59 6 58 (L-TLB)
0000010001f73e58 0000000000000046 0000010001f73e18 000001007febb2f0
000000000000147f 000001007ff9a850 000001007febb608 0000000000000001
0000000000010000 000001007febb2f0
Call Trace:<ffffffff80146273>{do_sigaction+659} <ffffffff80145361>{sigprocmask+241}
<ffffffff80149df5>{worker_thread+261} <ffffffff803774bc>{thread_return+41}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
<ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
<ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
<ffffffff801115ef>{child_rip+0}
scsi_eh_0 S 000000005fdbff82 0 170 1 188 57 (L-TLB)
000001003fdcde28 0000000000000046 0000000000000246 000001007fec71b0
0000000000003031 000001007ff9a850 000001007fec74c8 0000000035c1a774
0000000000000000 0000000000000000
Call Trace:<ffffffff80376d2d>{__down_interruptible+301} <ffffffff80133440>{default_wake_function+0}
<ffffffff80133493>{__wake_up_common+67} <ffffffff8037806b>{__down_failed_interruptible+53}
<ffffffff80298e53>{.text.lock.scsi_error+213} <ffffffff801115f7>{child_rip+8}
<ffffffff802989b0>{scsi_error_handler+0} <ffffffff801115ef>{child_rip+0}
scsi_eh_1 S 00000002446c8a8a 0 188 1 218 170 (L-TLB)
000001003fdade28 0000000000000046 0000000000000246 000001007feea950
0000000000002664 000001007ff9a850 000001007feeac68 000000003cab9668
0000000000000000 0000000000000000
Call Trace:<ffffffff80376d2d>{__down_interruptible+301} <ffffffff80133440>{default_wake_function+0}
<ffffffff80133493>{__wake_up_common+67} <ffffffff8037806b>{__down_failed_interruptible+53}
<ffffffff80298e53>{.text.lock.scsi_error+213} <ffffffff801115f7>{child_rip+8}
<ffffffff802989b0>{scsi_error_handler+0} <ffffffff801115ef>{child_rip+0}
scsi_eh_2 S 0000000316deda1b 0 218 1 230 188 (L-TLB)
0000010001fd3e28 0000000000000046 0000000000000246 000001007feeb170
0000000000002588 000001007ff9a850 000001007feeb488 000000003ccbede4
0000000000000000 0000000000000000
Call Trace:<ffffffff80376d2d>{__down_interruptible+301} <ffffffff80133440>{default_wake_function+0}
<ffffffff80133493>{__wake_up_common+67} <ffffffff8037806b>{__down_failed_interruptible+53}
<ffffffff80298e53>{.text.lock.scsi_error+213} <ffffffff801115f7>{child_rip+8}
<ffffffff802989b0>{scsi_error_handler+0} <ffffffff801115ef>{child_rip+0}
kseriod S 0000000416e9c96d 0 230 1 238 218 (L-TLB)
000001007feb9ef8 0000000000000046 0000006900000000 000001007fec6170
00000000178bebc3 0000010040e400b0 000001007fec6488 0000000000000001
0000000000000000 0000000000000000
Call Trace:<ffffffff802edc34>{serio_thread+244} <ffffffff80133440>{default_wake_function+0}
<ffffffff80131321>{schedule_tail+17} <ffffffff801115f7>{child_rip+8}
<ffffffff802edb40>{serio_thread+0} <ffffffff801115ef>{child_rip+0}
md0_raid5 S 00003e2670610809 0 238 1 239 230 (L-TLB)
000001007f7fdee8 0000000000000046 0000000000000202 0000010001fe8a50
0000000000000088 000001007ff9a850 0000010001fe8d68 000001003ffd5948
000001003ffd5980 000001003ffd59f4
Call Trace:<ffffffff802fdcaa>{raid5d+490} <ffffffff80303dc1>{md_thread+417}
<ffffffff80133440>{default_wake_function+0} <ffffffff80131321>{schedule_tail+17}
<ffffffff802fdac0>{raid5d+0} <ffffffff801115f7>{child_rip+8}
<ffffffff802fdac0>{raid5d+0} <ffffffff80303c20>{md_thread+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00003e255708fbc8 0 239 1 560 238 (L-TLB)
000001007f7dde38 0000000000000046 000000747f7ddde8 000001007feea130
00000000000002d8 ffffffff803e23e0 000001007feea448 0000000000000001
0000000000000003 000001003ffbdac8
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 000026122613b858 0 560 1 561 239 (L-TLB)
000001003f309e38 0000000000000046 000001007eafbec0 000001007f7a8b50
00000000000032f9 ffffffff803e23e0 000001007f7a8e68 0000000000000001
0000000000000003 000001003ffaccc8
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00002b440172e760 0 561 1 562 560 (L-TLB)
000001003e7b7e38 0000000000000046 000000763e7b7de8 000001007f6c73f0
0000000000001ffe 000001007ff9a850 000001007f6c7708 0000000000000001
0000000000000003 000001003ffacac8
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00002760eb1bb445 0 562 1 563 561 (L-TLB)
000001003e271e38 0000000000000046 000001003e271de8 000001007f6c63b0
00000000000021e0 000001007ff9a850 000001007f6c66c8 0000000000000001
0000000000000003 000001003ffac8c8
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 563 1 564 562 (L-TLB)
000001003e03fe38 0000000000000046 000000763e03fde8 000001007f63ac50
00000000000016a2 000001007f6904b0 000001007f63af68 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 564 1 565 563 (L-TLB)
000001003e7fde38 0000000000000046 000000763e7fdde8 000001007f64e3f0
00000000000016b9 000001007f6904b0 000001007f64e708 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00003e2557d33302 0 565 1 566 564 (L-TLB)
000001003dff3e38 0000000000000046 0000010067094560 000001007f6334b0
00000000000006d3 000001007ff9a850 000001007f6337c8 0000000000000001
0000000000000003 000001003ffac2c8
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 566 1 567 565 (L-TLB)
000001003dff1e38 0000000000000046 000000763dff1de8 000001007f6d4b90
0000000000001813 000001007f6904b0 000001007f6d4ea8 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 567 1 568 566 (L-TLB)
000001003f43fe38 0000000000000046 000000763f43fde8 000001007f63b470
000000000000164e 000001007f6904b0 000001007f63b788 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00003e20834b0b52 0 568 1 569 567 (L-TLB)
000001003f271e38 0000000000000046 000000763f271de8 000001007f632470
0000000000000406 000001007ff9a850 000001007f632788 0000000000000001
0000000000000003 000001007feb74c8
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 569 1 570 568 (L-TLB)
000001003e583e38 0000000000000046 000000763e583de8 000001007f6914f0
000000000000184a 000001007f6904b0 000001007f691808 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 570 1 571 569 (L-TLB)
000001003e0fde38 0000000000000046 000000763e0fdde8 000001007f690cd0
0000000000001662 000001007f6904b0 000001007f690fe8 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 571 1 572 570 (L-TLB)
000001003ff83e38 0000000000000046 000000763ff83de8 000001007f632c90
0000000000001843 000001007f6904b0 000001007f632fa8 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 572 1 573 571 (L-TLB)
000001003ff81e38 0000000000000046 000000763ff81de8 000001007f641530
0000000000001827 000001007f6904b0 000001007f641848 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 573 1 574 572 (L-TLB)
000001003e69fe38 0000000000000046 000000763e69fde8 000001007f63a430
0000000000001767 000001007f6904b0 000001007f63a748 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 574 1 575 573 (L-TLB)
000001003e90de38 0000000000000046 000000763e90dde8 0000010001fe8230
0000000000001658 000001007f6904b0 0000010001fe8548 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 575 1 576 574 (L-TLB)
000001003e7bbe38 0000000000000046 000000763e7bbde8 000001007f7a9370
000000000000176b 000001007f6904b0 000001007f7a9688 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 576 1 577 575 (L-TLB)
000001003e389e38 0000000000000046 000000763e389de8 0000010001fe9270
0000000000001945 000001007f6904b0 0000010001fe9588 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00000005ebfdef22 0 577 1 578 576 (L-TLB)
000001003e34fe38 0000000000000046 000001003e34fde8 000001007f7a8330
0000000001bb1857 000001007ff9a850 000001007f7a8648 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 000000064d0606d4 0 578 1 579 577 (L-TLB)
000001003e349e38 0000000000000046 000001003e349de8 000001003e34b570
000000003ccbf700 000001007ff9a850 000001003e34b888 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 579 1 580 578 (L-TLB)
000001003e64fe38 0000000000000046 000000763e64fde8 000001003e34ad50
00000000000023f3 000001007f6904b0 000001003e34b068 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 580 1 581 579 (L-TLB)
000001003dffde38 0000000000000046 000000763dffdde8 000001003e34a530
0000000000001bf7 000001007f6904b0 000001003e34a848 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 581 1 582 580 (L-TLB)
000001003f28fe38 0000000000000046 000000763f28fde8 000001003e0f15b0
0000000000001b47 000001007f6904b0 000001003e0f18c8 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 582 1 583 581 (L-TLB)
000001003e7a5e38 0000000000000046 000000763e7a5de8 000001003e0f0d90
0000000000001936 000001007f6904b0 000001003e0f10a8 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 583 1 584 582 (L-TLB)
000001003eaa3e38 0000000000000046 000000763eaa3de8 000001003e0f0570
0000000000001929 000001007f6904b0 000001003e0f0888 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 584 1 585 583 (L-TLB)
000001003e73de38 0000000000000046 000000763e73dde8 000001003e73f5f0
0000000000001784 000001007f6904b0 000001003e73f908 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 585 1 586 584 (L-TLB)
000001003e943e38 0000000000000046 000000763e943de8 000001003e73edd0
0000000000001720 000001007f6904b0 000001003e73f0e8 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 586 1 587 585 (L-TLB)
000001003e1d9e38 0000000000000046 000000763e1d9de8 000001003e73e5b0
00000000000018b2 000001007f6904b0 000001003e73e8c8 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 587 1 588 586 (L-TLB)
000001003e153e38 0000000000000046 000000763e153de8 000001003e155630
00000000000018de 000001007f6904b0 000001003e155948 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 588 1 589 587 (L-TLB)
000001003e2f1e38 0000000000000046 000000763e2f1de8 000001003e154e10
000000000000194e 000001007f6904b0 000001003e155128 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 589 1 590 588 (L-TLB)
000001003f1e7e38 0000000000000046 000000763f1e7de8 000001003e1545f0
00000000000016eb 000001007f6904b0 000001003e154908 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 590 1 591 589 (L-TLB)
000001003f1e1e38 0000000000000046 000000773f1e1de8 000001003f1e3670
000000000000181b 000001007f6904b0 000001003f1e3988 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 591 1 592 590 (L-TLB)
000001003e577e38 0000000000000046 000000773e577de8 000001003f1e2e50
000000000000179c 000001007f6904b0 000001003f1e3168 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 592 1 593 591 (L-TLB)
000001003e455e38 0000000000000046 000000773e455de8 000001003f1e2630
00000000000016ba 000001007f6904b0 000001003f1e2948 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 593 1 594 592 (L-TLB)
000001003e39fe38 0000000000000046 000000773e39fde8 000001003e4516b0
00000000000017b9 000001007f6904b0 000001003e4519c8 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 594 1 595 593 (L-TLB)
000001003e1bde38 0000000000000046 000000773e1bdde8 000001003e450e90
00000000000015fd 000001007f6904b0 000001003e4511a8 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 00000006691aead7 0 595 1 596 594 (L-TLB)
000001003e4dbe38 0000000000000046 000001003e4dbde8 000001003e450670
000000000b3c589b 000001007ff9a850 000001003e450988 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
kjournald S 0000000000000000 0 596 1 690 595 (L-TLB)
000001003e4d9e38 0000000000000046 000000773e4d9de8 000001007f6c6bd0
0000000000001c3a 000001007f6904b0 000001007f6c6ee8 0000000000000001
0000000000000003 0000000000000008
Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
<ffffffff801115ef>{child_rip+0}
syslogd S 0000000000000000 0 690 1 693 596 (NOTLB)
000001007f687d88 0000000000000006 0000007400000000 000001003e4a8ed0
0000000000000fdd 000001003e4a96f0 000001003e4a91e8 0000000000000216
0000000000000216 ffffffff8015a512
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
<ffffffff80313e45>{datagram_poll+21} <ffffffff801918ca>{do_select+906}
<ffffffff80191380>{__pollwait+0} <ffffffff801ab8a1>{compat_sys_select+1105}
<ffffffff80123f81>{ia32_sysret+0}
klogd R running task 0 693 1 696 690 (NOTLB)
rpc.statd S 0000275d80f36f11 0 696 1 707 693 (NOTLB)
000001003e3bfd88 0000000000000006 0000010034f5a3a0 000001003e931730
0000000000001f26 ffffffff803e23e0 000001003e931a48 0000000000000216
0000000000000216 ffffffff8015a512
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
<ffffffff80330341>{tcp_poll+33} <ffffffff801918ca>{do_select+906}
<ffffffff80191380>{__pollwait+0} <ffffffff801ab8a1>{compat_sys_select+1105}
<ffffffff80123f81>{ia32_sysret+0}
exim S 00003c9d5e980681 0 707 1 714 696 (NOTLB)
000001003e7c5d88 0000000000000006 000001000000d700 000001003e308f50
0000000000007e6c ffffffff803e23e0 000001003e309268 ffffffff8015a512
000001000000d700 0000000000000001
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
<ffffffff80330341>{tcp_poll+33} <ffffffff801918ca>{do_select+906}
<ffffffff80191380>{__pollwait+0} <ffffffff80154c01>{compat_sys_wait4+65}
<ffffffff801ab8a1>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
inetd S 0000000000000000 0 714 1 720 707 (NOTLB)
000001003e05bd88 0000000000000006 0000007800000000 000001003e49d070
00000000002a0f7f 000001003e308730 000001003e49d388 0000000000000216
0000000000000216 ffffffff8015a512
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
<ffffffff80330341>{tcp_poll+33} <ffffffff801918ca>{do_select+906}
<ffffffff80191380>{__pollwait+0} <ffffffff8016c109>{unmap_region+409}
<ffffffff801ab8a1>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
rngd S 00003e2325ef73e8 0 720 1 730 714 (NOTLB)
0000010040f27e68 0000000000000006 00000010ffffd36c 000001003e308730
0000000000000d85 ffffffff803e23e0 000001003e308a48 0000000000000246
ffffffff80443da0 0000000000000001
Call Trace:<ffffffff8015a555>{__get_free_pages+37} <ffffffff80377ef4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff80192011>{do_poll+177}
<ffffffff8019219e>{sys_poll+350} <ffffffff80191380>{__pollwait+0}
<ffffffff80123f81>{ia32_sysret+0}
lpd S 0000009c08291736 0 730 1 755 720 (NOTLB)
000001003e63dd88 0000000000000006 000001000000d700 000001003e49c850
0000000000002d11 ffffffff803e23e0 000001003e49cb68 ffffffff8015a512
000001000000d700 0000000000000001
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
<ffffffff8018a812>{pipe_poll+50} <ffffffff801918ca>{do_select+906}
<ffffffff80191380>{__pollwait+0} <ffffffff80154c01>{compat_sys_wait4+65}
<ffffffff801ab8a1>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
smartd S 00003c9e89326754 0 755 1 761 730 (NOTLB)
000001003e11fee8 0000000000000006 000001003ffba600 000001003e1210b0
00000000000035ab ffffffff803e23e0 000001003e1213c8 0000000000000246
0000000000000009 ffffffff802761db
Call Trace:<ffffffff802761db>{blkdev_ioctl+955} <ffffffff80190cf6>{sys_ioctl+678}
<ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff801541e1>{compat_sys_nanosleep+193} <ffffffff80123f81>{ia32_sysret+0}
sshd S 00003e224ac5a392 0 761 1 1090 764 755 (NOTLB)
000001003e80bd88 0000000000000006 000001000000d700 000001003e120890
000000000000c315 ffffffff803e23e0 000001003e120ba8 ffffffff8015a512
000001000000d700 0000000000000001
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
<ffffffff80330341>{tcp_poll+33} <ffffffff801918ca>{do_select+906}
<ffffffff80191380>{__pollwait+0} <ffffffff80154c01>{compat_sys_wait4+65}
<ffffffff801ab8a1>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
ntpd S 00003e276959b563 0 764 1 768 761 (NOTLB)
000001007ed8fd88 0000000000000006 0000007940fc0830 000001003e120070
0000000000000421 000001007ff9a850 000001003e120388 0000000000000216
0000000000000216 ffffffff8015a512
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
<ffffffff80313e45>{datagram_poll+21} <ffffffff801918ca>{do_select+906}
<ffffffff80191380>{__pollwait+0} <ffffffff8011084f>{do_signal+175}
<ffffffff801ab8a1>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
atd S 00003afba35fb0da 0 768 1 771 764 (NOTLB)
000001007f407ee8 0000000000000006 0000000000000000 000001007f6d4370
0000000000004080 000001007ff9a850 000001007f6d4688 0000000000000246
0000000000000000 ffffffff80186c64
Call Trace:<ffffffff80186c64>{vfs_stat+68} <ffffffff80141011>{del_timer_sync+49}
<ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff801541e1>{compat_sys_nanosleep+193} <ffffffff80123f81>{ia32_sysret+0}
cron S 00003e21579855b4 0 771 1 775 768 (NOTLB)
000001007f405ee8 0000000000000006 0000000000000000 000001007f6404f0
0000000000001261 ffffffff803e23e0 000001007f640808 0000000000000246
0000000000000000 ffffffff80186c64
Call Trace:<ffffffff80186c64>{vfs_stat+68} <ffffffff80141011>{del_timer_sync+49}
<ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff801541e1>{compat_sys_nanosleep+193} <ffffffff80123f81>{ia32_sysret+0}
irqbalance S 00003e270edcc3d0 0 775 1 785 771 (NOTLB)
000001003ea49ee8 0000000000000006 0000000000000212 000001003e29e0b0
0000000000006564 000001007ff9a850 000001003e29e3c8 0000000000000246
0000000000000216 0000010060208358
Call Trace:<ffffffff8016a7c7>{remove_vm_struct+231} <ffffffff8016c109>{unmap_region+409}
<ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff801541e1>{compat_sys_nanosleep+193} <ffffffff80123f81>{ia32_sysret+0}
zsh R running task 0 785 1 1797 775 (NOTLB)
sshd S 0000000000000000 0 1090 761 1092 1101 (NOTLB)
000001003ea33c18 0000000000000006 0000007500001000 000001003ea35130
00000000000003a1 0000010001e8c0f0 000001003ea35448 ffffffff80382270
000001003e180520 0000000000000374
Call Trace:<ffffffff801c470d>{__ext3_journal_stop+45} <ffffffff80377e65>{schedule_timeout+37}
<ffffffff801582b7>{generic_file_aio_write_nolock+2055}
<ffffffff8035dc57>{unix_stream_data_wait+263} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff8035de31>{unix_stream_recvmsg+369}
<ffffffff8030d5a4>{sock_aio_read+180} <ffffffff8017bb93>{do_sync_read+115}
<ffffffff8017ae77>{dentry_open+263} <ffffffff8017ad52>{filp_open+66}
<ffffffff8017bcba>{vfs_read+218} <ffffffff8017bf19>{sys_read+73}
<ffffffff80123f81>{ia32_sysret+0}
sshd S 0000000000000000 0 1092 1090 1093 (NOTLB)
000001003e9edd88 0000000000000006 000000750000d700 0000010001e8c0f0
0000000000000c00 000001007f640d10 0000010001e8c408 ffffffff8015a512
0000000008098450 0000000000000001
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
<ffffffff8024e52b>{pty_write_room+43} <ffffffff8024d521>{normal_poll+305}
<ffffffff801918ca>{do_select+906} <ffffffff80191380>{__pollwait+0}
<ffffffff801ab8a1>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
zsh S 00003d93b2c81863 0 1093 1092 1119 (NOTLB)
000001003de79f18 0000000000000006 0000000000000216 000001003dece9d0
0000000000001398 ffffffff803e23e0 000001003decece8 000001003e6b44c0
000001003e6b44c0 ffffffff80145361
Call Trace:<ffffffff80145361>{sigprocmask+241} <ffffffff801958d1>{dput+33}
<ffffffff80123f81>{ia32_sysret+0} <ffffffff8010fde9>{sys_rt_sigsuspend+297}
<ffffffff801249fe>{sys32_rt_sigprocmask+206} <ffffffff801240d8>{ia32_ptregs_common+40}
sshd S 0000000000000000 0 1101 761 1103 1090 (NOTLB)
000001003de0fc18 0000000000000006 0000007600001000 000001003e29f0f0
00000000000003f5 000001003decf1f0 000001003e29f408 ffffffff80382270
000001003e180a00 0000000000000374
Call Trace:<ffffffff801c470d>{__ext3_journal_stop+45} <ffffffff80377e65>{schedule_timeout+37}
<ffffffff801582b7>{generic_file_aio_write_nolock+2055}
<ffffffff8035dc57>{unix_stream_data_wait+263} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff8035de31>{unix_stream_recvmsg+369}
<ffffffff8030d5a4>{sock_aio_read+180} <ffffffff8017bb93>{do_sync_read+115}
<ffffffff8017ae77>{dentry_open+263} <ffffffff8017ad52>{filp_open+66}
<ffffffff8017bcba>{vfs_read+218} <ffffffff8017bf19>{sys_read+73}
<ffffffff80123f81>{ia32_sysret+0}
sshd S 00003e242c914ec0 0 1103 1101 1104 (NOTLB)
000001003dc05d88 0000000000000006 000000740000d700 000001003decf1f0
0000000000000e87 000001003e29e8d0 000001003decf508 ffffffff8015a512
0000000008098430 0000000000000001
Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
<ffffffff8024e52b>{pty_write_room+43} <ffffffff8024d521>{normal_poll+305}
<ffffffff801918ca>{do_select+906} <ffffffff80191380>{__pollwait+0}
<ffffffff801ab8a1>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}
zsh S 00003e242c9c72d2 0 1104 1103 8825 (NOTLB)
000001003dba3f18 0000000000000006 0000000000000216 000001003e29e8d0
0000000000009ded ffffffff803e23e0 000001003e29ebe8 000001003e6b4b80
000001003e6b4b80 ffffffff80145361
Call Trace:<ffffffff80145361>{sigprocmask+241} <ffffffff801958d1>{dput+33}
<ffffffff80123f81>{ia32_sysret+0} <ffffffff8010fde9>{sys_rt_sigsuspend+297}
<ffffffff801249fe>{sys32_rt_sigprocmask+206} <ffffffff801240d8>{ia32_ptregs_common+40}
zsh S 00003e1fc58e116c 0 1119 1093 8824 8727 (NOTLB)
000001003da01f18 0000000000000006 0000000000000216 000001003ea340f0
000000000000179b ffffffff803e23e0 000001003ea34408 000001003e6b4940
000001003e6b4940 ffffffff80145361
Call Trace:<ffffffff80145361>{sigprocmask+241} <ffffffff801958d1>{dput+33}
<ffffffff80123f81>{ia32_sysret+0} <ffffffff8010fde9>{sys_rt_sigsuspend+297}
<ffffffff801249fe>{sys32_rt_sigprocmask+206} <ffffffff801240d8>{ia32_ptregs_common+40}
apcupsd S 00000a987aa07cb0 0 1797 1 1896 4010 785 (NOTLB)
000001003d571de8 0000000000000006 000001003e2903d8 000001003e9306f0
000000000000b624 ffffffff803e23e0 000001003e930a08 ffffffff801584eb
00000000ffffdbdc 0000000000000038
Call Trace:<ffffffff801584eb>{generic_file_aio_write+107} <ffffffff801bbbf3>{ext3_file_write+35}
<ffffffff8013bdb2>{sys_wait4+642} <ffffffff80133440>{default_wake_function+0}
<ffffffff80133440>{default_wake_function+0} <ffffffff80154c01>{compat_sys_wait4+65}
<ffffffff80146273>{do_sigaction+659} <ffffffff8017bea7>{vfs_write+247}
<ffffffff8017bf99>{sys_write+73} <ffffffff80123f81>{ia32_sysret+0}
apcupsd R running task 0 1896 1797 (NOTLB)
portmap S 00003c7c39328cc8 0 4010 1 4021 1797 (NOTLB)
000001007f2e1e68 0000000000000006 0000001000000216 0000010034d8a3b0
0000000000001647 000001007ff9a850 0000010034d8a6c8 0000000000000246
000001003e293730 0000000000000006
Call Trace:<ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff80192011>{do_poll+177} <ffffffff8019219e>{sys_poll+350}
<ffffffff80191380>{__pollwait+0} <ffffffff80123f81>{ia32_sysret+0}
nfsd S 00003d33ef5e1212 0 4021 1 4022 4010 (L-TLB)
0000010034c71d98 0000000000000046 0000007340a62e88 000001007f6904b0
0000000000001dff 000001003e4a86b0 000001007f6907c8 0000000000000246
0000000000000256 0000010034c71d58
Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377ef4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff8036e0fb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80145361>{sigprocmask+241} <ffffffff801f8db0>{nfsd+0}
<ffffffff801f8f87>{nfsd+471} <ffffffff801f8db0>{nfsd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801f8db0>{nfsd+0}
<ffffffff801f8db0>{nfsd+0} <ffffffff801115ef>{child_rip+0}
nfsd S 00003d33ef5e1d69 0 4022 1 4025 4021 (L-TLB)
0000010034badd98 0000000000000046 0000007301b7b698 000001003e4a86b0
0000000000000122 ffffffff803e23e0 000001003e4a89c8 0000000000000246
000001003fd554a0 0000000000000216
Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377ef4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff8036e0fb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80145361>{sigprocmask+241} <ffffffff801f8db0>{nfsd+0}
<ffffffff801f8f87>{nfsd+471} <ffffffff801f8db0>{nfsd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801f8db0>{nfsd+0}
<ffffffff801f8db0>{nfsd+0} <ffffffff801115ef>{child_rip+0}
nfsd S 00003e0570804991 0 4025 1 4023 4022 (L-TLB)
0000010034c09d98 0000000000000046 0000010001b79788 000001003e309770
0000000000003076 000001007ff9a850 000001003e309a88 0000000000000246
000001003fd434a0 0000000000000216
Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377ef4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff8036e0fb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80145361>{sigprocmask+241} <ffffffff801f8db0>{nfsd+0}
<ffffffff801f8f87>{nfsd+471} <ffffffff801f8db0>{nfsd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801f8db0>{nfsd+0}
<ffffffff801f8db0>{nfsd+0} <ffffffff801115ef>{child_rip+0}
nfsd S 00003e057080e3f4 0 4023 1 4027 4025 (L-TLB)
000001003e08fd98 0000000000000046 00000100405be440 000001007febea10
0000000000001712 ffffffff803e23e0 000001007febed28 0000000000000246
000001007fe7c4a0 000001007f6b8cc0
Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377ef4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff8036e0fb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80145361>{sigprocmask+241} <ffffffff801f8db0>{nfsd+0}
<ffffffff801f8f87>{nfsd+471} <ffffffff801f8db0>{nfsd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801f8db0>{nfsd+0}
<ffffffff801f8db0>{nfsd+0} <ffffffff801115ef>{child_rip+0}
nfsd S 00003b3d1f1b4de6 0 4027 1 4024 4023 (L-TLB)
000001003e04bd98 0000000000000046 0000007340db8990 000001007fec6990
000000000000019c 000001007ff9a850 000001007fec6ca8 0000000000000246
000001007ffb24a0 0000000000000216
Call Trace:<ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff8036e0fb>{svc_recv+987} <ffffffff80133440>{default_wake_function+0}
<ffffffff80133440>{default_wake_function+0} <ffffffff80145361>{sigprocmask+241}
<ffffffff801f8db0>{nfsd+0} <ffffffff801f8f87>{nfsd+471}
<ffffffff801f8db0>{nfsd+0} <ffffffff801115f7>{child_rip+8}
<ffffffff801f8db0>{nfsd+0} <ffffffff801f8db0>{nfsd+0}
<ffffffff801115ef>{child_rip+0}
nfsd S 00003b90f03a78c3 0 4024 1 4026 4027 (L-TLB)
0000010001e77d98 0000000000000046 0000007301b8b030 000001007fec31f0
00000000000000ab ffffffff803e23e0 000001007fec3508 0000000000000246
000001007fe814a0 0000000000000216
Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377ef4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff8036e0fb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80145361>{sigprocmask+241} <ffffffff801f8db0>{nfsd+0}
<ffffffff801f8f87>{nfsd+471} <ffffffff801f8db0>{nfsd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801f8db0>{nfsd+0}
<ffffffff801f8db0>{nfsd+0} <ffffffff801115ef>{child_rip+0}
nfsd S 00003b3d1f1b6fe4 0 4026 1 4028 4024 (L-TLB)
000001003e6bfd98 0000000000000046 0000010001b7e610 000001003e49c030
000000000000066f ffffffff803e23e0 000001003e49c348 0000000000000246
000001007ffb44a0 000001007f6b8cc0
Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377ef4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff8036e0fb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80145361>{sigprocmask+241} <ffffffff801f8db0>{nfsd+0}
<ffffffff801f8f87>{nfsd+471} <ffffffff801f8db0>{nfsd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801f8db0>{nfsd+0}
<ffffffff801f8db0>{nfsd+0} <ffffffff801115ef>{child_rip+0}
nfsd S 00003b90f03a7214 0 4028 1 4030 4026 (L-TLB)
000001003e3bdd98 0000000000000046 0000007340d8c188 0000010034d8b3f0
0000000000000940 000001007fec31f0 0000010034d8b708 0000000000000246
000001007fe734a0 000001007f6b8cc0
Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377ef4>{schedule_timeout+180}
<ffffffff80141b20>{process_timeout+0} <ffffffff8036e0fb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80145361>{sigprocmask+241} <ffffffff801f8db0>{nfsd+0}
<ffffffff801f8f87>{nfsd+471} <ffffffff801f8db0>{nfsd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff801f8db0>{nfsd+0}
<ffffffff801f8db0>{nfsd+0} <ffffffff801115ef>{child_rip+0}
lockd S 0000000000000000 0 4030 1 4031 4028 (L-TLB)
000001007ed13de8 0000000000000046 000000763e61a330 000001003e61a330
000000000000111d 000001003e61ab50 000001003e61a648 ffffffff803774bc
0000007300000001 0000000000000212
Call Trace:<ffffffff803774bc>{thread_return+41} <ffffffff80377e65>{schedule_timeout+37}
<ffffffff8036c679>{svc_sock_release+569} <ffffffff8036e0fb>{svc_recv+987}
<ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
<ffffffff80378031>{__down_failed+53} <ffffffff80133440>{default_wake_function+0}
<ffffffff8020a6d9>{lockd+377} <ffffffff8020a560>{lockd+0}
<ffffffff801115f7>{child_rip+8} <ffffffff8020a560>{lockd+0}
<ffffffff8020a560>{lockd+0} <ffffffff801115ef>{child_rip+0}
rpciod S 0000000000000000 0 4031 1 4033 4030 (L-TLB)
000001007e893e88 0000000000000046 0000007440000000 000001007ed114f0
00000000000012cf 000001007f6904b0 000001007ed11808 000001003e5e8668
ffffffff8020a560 0000000000000008
Call Trace:<ffffffff8020a560>{lockd+0} <ffffffff803780df>{__up_wakeup+53}
<ffffffff8020a560>{lockd+0} <ffffffff8036952a>{rpciod+1050}
<ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801358f0>{autoremove_wake_function+0}
<ffffffff80131321>{schedule_tail+17} <ffffffff801115f7>{child_rip+8}
<ffffffff8020a560>{lockd+0} <ffffffff80369110>{rpciod+0}
<ffffffff801115ef>{child_rip+0}
rpc.mountd S 000027aa351b4f23 0 4033 1 4031 (NOTLB)
000001003e4c5e68 0000000000000006 0000001080000000 000001007f64f430
00000000000000ef ffffffff803e23e0 000001007f64f748 0000000000000246
0000010040f5f8f0 0000000000000002
Call Trace:<ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff80192011>{do_poll+177} <ffffffff8019219e>{sys_poll+350}
<ffffffff80191380>{__pollwait+0} <ffffffff80123f81>{ia32_sysret+0}
zsh S 00003e14d482c249 0 8727 1093 8775 1119 (NOTLB)
0000010024b3ff18 0000000000000006 0000000000000216 000001003e61b370
0000000000001960 ffffffff803e23e0 000001003e61b688 000001003e6b4940
000001003e6b4940 ffffffff80145361
Call Trace:<ffffffff80145361>{sigprocmask+241} <ffffffff801958d1>{dput+33}
<ffffffff80123f81>{ia32_sysret+0} <ffffffff8010fde9>{sys_rt_sigsuspend+297}
<ffffffff801249fe>{sys32_rt_sigprocmask+206} <ffffffff801240d8>{ia32_ptregs_common+40}
dump D 00003e14e808f7df 0 8775 8727 (NOTLB)
000001002ef31bb8 0000000000000006 000001003ffd5948 000001007f640d10
0000000000007df3 ffffffff803e23e0 000001007f641028 ffffffff802fd683
0000000000000212 0000000000000001
Call Trace:<ffffffff802fd683>{raid5_unplug_device+291} <ffffffff80377d3b>{io_schedule+43}
<ffffffff80155f6a>{__lock_page+250} <ffffffff80155c40>{page_wake_function+0}
<ffffffff80155c40>{page_wake_function+0} <ffffffff801567f6>{do_generic_mapping_read+502}
<ffffffff80156a70>{file_read_actor+0} <ffffffff80156d14>{__generic_file_aio_read+372}
<ffffffff80156dfb>{generic_file_read+123} <ffffffff80169994>{handle_mm_fault+292}
<ffffffff80122ad8>{do_page_fault+440} <ffffffff80130948>{recalc_task_prio+424}
<ffffffff803774bc>{thread_return+41} <ffffffff8017bca7>{vfs_read+199}
<ffffffff8017bf19>{sys_read+73} <ffffffff80123f81>{ia32_sysret+0}
sleep S 00003e1fc5e0bf41 0 8824 1119 (NOTLB)
0000010054f3fee8 0000000000000006 00000100115c0aa8 000001007febe1f0
00000000000e9920 000001007ff9a850 000001007febe508 0000000000000246
00000000ffffd724 0000000000000014
Call Trace:<ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
<ffffffff801541e1>{compat_sys_nanosleep+193} <ffffffff80123f81>{ia32_sysret+0}
ls D 00003e242d2c4ffe 0 8825 1104 (NOTLB)
00000100349d5d38 0000000000000006 000001003ffd5948 000001003e61ab50
00000000001349ac ffffffff803e23e0 000001003e61ae68 ffffffff802fd683
0000000000000216 000001003e26ed58
Call Trace:<ffffffff802fd683>{raid5_unplug_device+291} <ffffffff80377d3b>{io_schedule+43}
<ffffffff8017d4f8>{__wait_on_buffer+168} <ffffffff8017d310>{bh_wake_function+0}
<ffffffff8017d310>{bh_wake_function+0} <ffffffff801be1cb>{ext3_bread+91}
<ffffffff801bb348>{ext3_readdir+232} <ffffffff80191120>{filldir64+0}
<ffffffff80122ad8>{do_page_fault+440} <ffffffff8016b23b>{vma_merge+331}
<ffffffff80190db7>{vfs_readdir+119} <ffffffff80191294>{sys_getdents64+132}
<ffffffff80111441>{error_exit+0} <ffffffff80123f81>{ia32_sysret+0}
-ryan
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2004-06-17 2:30 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-06-15 6:22 processes hung in D (raid5/dm/ext3) foo
2004-06-15 10:09 ` Andrew Morton
2004-06-15 15:00 ` foo
2004-06-15 23:26 ` Andrew Morton
2004-06-15 23:44 ` Evaldo Gardenali
2004-06-16 2:03 ` foo
2004-06-16 2:16 ` foo
2004-06-16 12:57 ` Alasdair G Kergon
2004-06-16 19:00 ` foo
2004-06-16 19:15 ` Alasdair G Kergon
2004-06-17 2:28 ` foo
2004-06-16 2:48 ` foo
2004-06-16 2:58 ` Andrew Morton
2004-06-16 3:55 ` foo
2004-06-16 4:13 ` foo
2004-06-16 4:25 ` Andrew Morton
2004-06-16 4:41 ` foo
2004-06-16 4:53 ` linux-kernel-owner
2004-06-16 7:22 ` tg3 wrong speed foo
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®