From: Shehjar Tikoo <shehjart@cse.unsw.edu.au>
To: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Cc: akpm@linux-foundation.org, tony.luck@intel.com,
linux-kernel@vger.kernel.org, linux-parisc@vger.kernel.org
Subject: Re: Panic in multiple kernels: IA64 SBA IOMMU: Culprit commit on Mar 28, 2008
Date: Thu, 06 Nov 2008 14:06:09 +1100 [thread overview]
Message-ID: <49125F21.8010806@cse.unsw.edu.au> (raw)
In-Reply-To: <20081106032605J.fujita.tomonori@lab.ntt.co.jp>
FUJITA Tomonori wrote:
> Sorry for the delay.
>
> CC'ed linux-parisc since the same problem could happen to parisc.
>
> On Tue, 04 Nov 2008 10:23:58 +1100
> Shehjar Tikoo <shehjart@cse.unsw.edu.au> wrote:
>
>> I've been observing kernel panics for the past week on
>> kernel versions 2.6.26, 2.6.27 but not on 2.6.24 and 2.6.25.
>>
>> The panic message says:
>>
>> arch/ia64/hp/common/sba_iommu.c: I/O MMU is out of mapping resources
>>
>> Using git-bisect, I've zeroed in on the commit that introduced this.
>> Please see the attached file for the commit.
>>
>> The workload consists of 2 tests:
>> 1. Single fio process writing a 1 TB file.
>> 2. 15 fio processes writing 15GB files each.
>>
>> The panic happens on both workloads. There is no stack trace after
>> the above message.
>>
>> Other info:
>> System is HP RX6600(16Gb RAM, 16 processors w/ dual cores and HT)
>> 20 SATA disks under software RAID0 with 6 TB capacity.
>> Silicon Image 3124 controller.
>> File system is XFS.
>>
>> I'd much appreciate some help in fixing this because this panic has
>> basically stalled my own work. I'd be willing to run more tests on my
>> setup to test any patches that possibly fix this issue.
>
> This patch modified the sba IOMMU driver to support LLDs' segment
> boundary limits properly.
>
> ATA hardware has poor segment boundary limit, 64KB. In addition, sba
> IOMMU driver uses size-aligned allocation algorithm. It means that
> it's difficult for the IOMMU driver to find an appropriate I/O address
> space. I think that you hit the allocation failure due to this problem
> (of course, it's possible that my change breaks the IOMMU driver but I
> can't find a problem so far).
>
> To make matters worse, sba IOMMU driver panic when the allocation
> fails. IIRC, only IA64 and parisc IOMMU drivers panic by default in
> the case of the allocation failure. I think that we need to change
> them to handle the failure properly.
>
> Can you try this? I've not fixed map_single failure yet but I think
> that you hit the failure allocation in map_sg path.
>
On 2.6.27, this patch seems to prevent the panic from happening for
both the tests I had described earlier. Do you need more info to
validate this? I will be running more tests with this patch over
the next few days, so we'll find out anyway.
Thanks
Shehjar
>
> diff --git a/arch/ia64/hp/common/sba_iommu.c b/arch/ia64/hp/common/sba_iommu.c
> index d98f0f4..8f44dc8 100644
> --- a/arch/ia64/hp/common/sba_iommu.c
> +++ b/arch/ia64/hp/common/sba_iommu.c
> @@ -676,12 +676,19 @@ sba_alloc_range(struct ioc *ioc, struct device *dev, size_t size)
> spin_unlock_irqrestore(&ioc->saved_lock, flags);
>
> pide = sba_search_bitmap(ioc, dev, pages_needed, 0);
> - if (unlikely(pide >= (ioc->res_size << 3)))
> - panic(__FILE__ ": I/O MMU @ %p is out of mapping resources\n",
> - ioc->ioc_hpa);
> + if (unlikely(pide >= (ioc->res_size << 3))) {
> + printk(KERN_WARNING "%s: I/O MMU @ %p is"
> + "out of mapping resources, %u %u %lx\n",
> + __func__, ioc->ioc_hpa, ioc->res_size,
> + pages_needed, dma_get_seg_boundary(dev));
> + return -1;
> + }
> #else
> - panic(__FILE__ ": I/O MMU @ %p is out of mapping resources\n",
> - ioc->ioc_hpa);
> + printk(KERN_WARNING "%s: I/O MMU @ %p is"
> + "out of mapping resources, %u %u %lx\n",
> + __func__, ioc->ioc_hpa, ioc->res_size,
> + pages_needed, dma_get_seg_boundary(dev));
> + return -1;
> #endif
> }
> }
> @@ -962,6 +969,7 @@ sba_map_single_attrs(struct device *dev, void *addr, size_t size, int dir,
> #endif
>
> pide = sba_alloc_range(ioc, dev, size);
> + BUG_ON(pide < 0);
>
> iovp = (dma_addr_t) pide << iovp_shift;
>
> @@ -1304,6 +1312,7 @@ sba_coalesce_chunks(struct ioc *ioc, struct device *dev,
> unsigned long dma_offset, dma_len; /* start/len of DMA stream */
> int n_mappings = 0;
> unsigned int max_seg_size = dma_get_max_seg_size(dev);
> + int idx;
>
> while (nents > 0) {
> unsigned long vaddr = (unsigned long) sba_sg_address(startsg);
> @@ -1402,9 +1411,13 @@ sba_coalesce_chunks(struct ioc *ioc, struct device *dev,
> vcontig_sg->dma_length = vcontig_len;
> dma_len = (dma_len + dma_offset + ~iovp_mask) & iovp_mask;
> ASSERT(dma_len <= DMA_CHUNK_SIZE);
> - dma_sg->dma_address = (dma_addr_t) (PIDE_FLAG
> - | (sba_alloc_range(ioc, dev, dma_len) << iovp_shift)
> - | dma_offset);
> + idx = sba_alloc_range(ioc, dev, dma_len);
> + if (idx < 0) {
> + dma_sg->dma_length = 0;
> + return -1;
> + }
> + dma_sg->dma_address = (dma_addr_t)(PIDE_FLAG | (idx << iovp_shift)
> + | dma_offset);
> n_mappings++;
> }
>
> @@ -1476,6 +1489,10 @@ int sba_map_sg_attrs(struct device *dev, struct scatterlist *sglist, int nents,
> ** Access to the virtual address is what forces a two pass algorithm.
> */
> coalesced = sba_coalesce_chunks(ioc, dev, sglist, nents);
> + if (coalesced < 0) {
> + sba_unmap_sg_attrs(dev, sglist, nents, dir, attrs);
> + return 0;
> + }
>
> /*
> ** Program the I/O Pdir
next prev parent reply other threads:[~2008-11-06 3:14 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-11-03 23:23 Shehjar Tikoo
2008-11-04 22:13 ` Luck, Tony
2008-11-06 3:01 ` Shehjar Tikoo
2008-11-05 18:26 ` FUJITA Tomonori
2008-11-06 3:06 ` Shehjar Tikoo [this message]
2008-11-07 3:49 ` FUJITA Tomonori
2008-11-07 16:58 ` Luck, Tony
2008-11-11 6:06 ` FUJITA Tomonori
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=49125F21.8010806@cse.unsw.edu.au \
--to=shehjart@cse.unsw.edu.au \
--cc=akpm@linux-foundation.org \
--cc=fujita.tomonori@lab.ntt.co.jp \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-parisc@vger.kernel.org \
--cc=tony.luck@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®