On Mon, Jun 23, 2008 at 12:53 PM, Christopher S. Aker wrote: > adam radford wrote: >> >> On Mon, Jun 23, 2008 at 9:15 AM, Christopher S. Aker >> wrote: >> >>> It's happened on a number of different hosts, all of the same hardware >>> and >>> software configuration (Xen 3.2 64bit, 32bit pae dom0, 32bit pae domUs. >>> LVM >>> backend with 3ware hardware RAID-1). > >> >> >> A driver patch for older kernels including XenServer-4.1 is available >> here: >> >> http://www.3ware.com/KB/article.aspx?id=15243&cNode=6I1C6S > > Thanks, but unfortunately, from that link: > > "3ware 9000 series controllers are not affected by this issue." > > ... which is what we're using. This problem only appeared after rebooting > these machines into Xen. Some of affected boxes even ran 2.6.18 (non Xen) > for awhile without any problems. > > -Chris > I've seen cases where Xen DMA code (pci-dma-xen.c) sends I/O requests un-necessarily through the SWIOTLB code. Xen dma: avoid unnecessarily SWIOTLB bounce buffering: http://lists.xensource.com/archives/html/xen-changelog/2008-04/msg00008.html so you may need a similar bidirectional fix for the 3w-9xxx driver. Can you try this patch (in-line below and also attached): -Adam diff -Naur linux-2.6.18-xen-3.2.0/drivers/scsi/3w-9xxx.c linux-2.6.18-xen-3.2.0.new/drivers/scsi/3w-9xxx.c --- linux-2.6.18-xen-3.2.0/drivers/scsi/3w-9xxx.c 2008-02-12 02:44:33.000000000 -0800 +++ linux-2.6.18-xen-3.2.0.new/drivers/scsi/3w-9xxx.c 2008-06-23 15:42:02.000000000 -0700 @@ -1388,7 +1388,7 @@ if (cmd->use_sg == 0) goto out; - use_sg = pci_map_sg(pdev, cmd->request_buffer, cmd->use_sg, DMA_BIDIRECTIONAL); + use_sg = pci_map_sg(pdev, cmd->request_buffer, cmd->use_sg, cmd->sc_data_direction); if (use_sg == 0) { TW_PRINTK(tw_dev->host, TW_DRIVER, 0x1c, "Failed to map scatter gather list"); @@ -1415,7 +1415,7 @@ goto out; } - mapping = pci_map_single(pdev, cmd->request_buffer, cmd->request_bufflen, DMA_BIDIRECTIONAL); + mapping = pci_map_single(pdev, cmd->request_buffer, cmd->request_bufflen, cmd->sc_data_direction); if (mapping == 0) { TW_PRINTK(tw_dev->host, TW_DRIVER, 0x1d, "Failed to map page"); @@ -1423,7 +1423,7 @@ } cmd->SCp.phase = TW_PHASE_SINGLE; - cmd->SCp.have_data_in = mapping; + cmd->SCp.dma_handle = mapping; retval = mapping; out: return retval; @@ -2000,10 +2000,10 @@ switch(cmd->SCp.phase) { case TW_PHASE_SINGLE: - pci_unmap_single(pdev, cmd->SCp.have_data_in, cmd->request_bufflen, DMA_BIDIRECTIONAL); + pci_unmap_single(pdev, cmd->SCp.dma_handle, cmd->request_bufflen, cmd->sc_data_direction); break; case TW_PHASE_SGLIST: - pci_unmap_sg(pdev, cmd->request_buffer, cmd->use_sg, DMA_BIDIRECTIONAL); + pci_unmap_sg(pdev, cmd->request_buffer, cmd->use_sg, cmd->sc_data_direction); break; } } /* End twa_unmap_scsi_data() */