Re: [Bugme-new] [Bug 14003] New: Infinite loop on bootup while handling DMAR

Re: [Bugme-new] [Bug 14003] New: Infinite loop on bootup while handling DMAR

[Andrew Morton]

(switched to email.  Please respond via emailed reply-to-all, not via the
bugzilla web interface).

On Tue, 18 Aug 2009 14:54:22 GMT bugzilla-daemon@bugzilla.kernel.org wrote:

> http://bugzilla.kernel.org/show_bug.cgi?id=14003
> 
>            Summary: Infinite loop on bootup while handling DMAR

That's a box-killing post-2.6.30 regression.

>            Product: Drivers
>            Version: 2.5
>     Kernel Version: 2.6.31-rc6
>           Platform: All
>         OS/Version: Linux
>               Tree: Mainline
>             Status: NEW
>           Severity: normal
>           Priority: P1
>          Component: PCI
>         AssignedTo: drivers_pci@kernel-bugs.osdl.org
>         ReportedBy: bero@arklinux.org
>         Regression: Yes
> 
> 
> Booting 64bit 2.6.31-rc6 on a hp xw4600 results in an infinite loop of
> 
> DMAR:[DMA READ] Request device [ff:1f.7] fault addr fffffffffffff000
> DMAR:[fault reason 255] Unknown
> 
> even with iommu=soft (2.6.30 could boot with iommu=soft, but would result in
> the same error with hardware iommu enabled).
> 
> 2.6.30 reports this about the DMAR ACPI tables:
> 
> ACPI: DMAR 00000000defc247f 00158 (v01 COMPAQ BEARLX38 00000001      00000000)
> DMAR:Host address width 36
> DMAR:DRHD (flags: 0x00000000)base: 0x00000000fed90000
> DMAR:DRHD (flags: 0x00000001)base: 0x00000000fed93000
> DMAR:RMRR base: 0x00000000defd0000 end: 0x00000000defd0fff
> DMAR:RMRR base: 0x00000000defd1000 end: 0x00000000defd1fff
> DMAR:RMRR base: 0x00000000defd2000 end: 0x00000000defd2fff
> DMAR:RMRR base: 0x00000000defd3000 end: 0x00000000defd3fff
> DMAR:RMRR base: 0x00000000defd4000 end: 0x00000000defd4fff
> DMAR:RMRR base: 0x00000000defd5000 end: 0x00000000defd5fff
> DMAR:RMRR base: 0x00000000defd6000 end: 0x00000000defd6fff
> DMAR:RMRR base: 0x00000000defd7000 end: 0x00000000defd7fff
> Not all IO-APIC's listed under remapping hardware
> 
> Device ff:1f.7 doesn't actually exist, lspci -n says
> 
> 00:00.0 0600: 8086:29e0
> 00:01.0 0604: 8086:29e1
> 00:1a.0 0c03: 8086:2937 (rev 02)
> 00:1a.1 0c03: 8086:2938 (rev 02)
> 00:1a.2 0c03: 8086:2939 (rev 02)
> 00:1a.7 0c03: 8086:293c (rev 02)
> 00:1b.0 0403: 8086:293e (rev 02)
> 00:1c.0 0604: 8086:2940 (rev 02)
> 00:1c.4 0604: 8086:2948 (rev 02)
> 00:1c.5 0604: 8086:294a (rev 02)
> 00:1d.0 0c03: 8086:2934 (rev 02)
> 00:1d.1 0c03: 8086:2935 (rev 02)
> 00:1d.2 0c03: 8086:2936 (rev 02)
> 00:1d.7 0c03: 8086:293a (rev 02)
> 00:1e.0 0604: 8086:244e (rev 92)
> 00:1f.0 0601: 8086:2916 (rev 02)
> 00:1f.2 0106: 8086:2922 (rev 02)
> 01:00.0 0300: 1002:7187
> 01:00.1 0380: 1002:71a7
> 10:0b.0 0c00: 11c1:5811 (rev 61)
> 3f:00.0 0200: 14e4:167b (rev 02)
> 

Re: [Bugme-new] [Bug 14003] New: Infinite loop on bootup while handling DMAR

[Suresh Siddha]

On Wed, 2009-08-19 at 14:26 -0700, Andrew Morton wrote:
> (switched to email.  Please respond via emailed reply-to-all, not via the
> bugzilla web interface).
> 
> On Tue, 18 Aug 2009 14:54:22 GMT bugzilla-daemon@bugzilla.kernel.org wrote:
> 
> > http://bugzilla.kernel.org/show_bug.cgi?id=14003
> > 
> >            Summary: Infinite loop on bootup while handling DMAR
> 
> That's a box-killing post-2.6.30 regression.
> 
> > 
> > Booting 64bit 2.6.31-rc6 on a hp xw4600 results in an infinite loop of
> > 
> > DMAR:[DMA READ] Request device [ff:1f.7] fault addr fffffffffffff000
> > DMAR:[fault reason 255] Unknown

I think David has a fix queued up for this already. Please check
http://git.infradead.org/iommu-2.6.git/commit/0815565a

thanks,
suresh

Re: [Bugme-new] [Bug 14003] New: Infinite loop on bootup while handling DMAR

[David Woodhouse]

On Wed, 2009-08-19 at 14:26 -0700, Andrew Morton wrote:
> > http://bugzilla.kernel.org/show_bug.cgi?id=14003
> >            Summary: Infinite loop on bootup while handling DMAR
> 
> That's a box-killing post-2.6.30 regression.

It's a BIOS bug -- the user's BIOS is written by idiots who obviously
shipped it without any QA whatsoever.

As far as I can tell, 2.6.30 aborted early because of a _different_ BIOS
bug, but now we cope with that particular bug and we fall over the next
bug. Or just come across them in a different order.

The IOMMU on this board can _never_ have worked. Just disable it.

Or use a board with open source firmware available, and this kind of
crap won't happen. (At least if it does, you'll be able to fix it).

-- 
dwmw2

[PATCH] intel-iommu: Work around yet another BIOS bug

[David Woodhouse]

Yet another reason why trusting this stuff to the BIOS was a bad idea.

There now seem to be a bunch of BIOSes which report an IOMMU at a
physical address which just returns all ones. (Perhaps only when VT-d is
actually _disabled_ in the BIOS?)

Well done, Dell and HP -- although I didn't think it was possible, you
have _further_ lowered my already-unprintable opinion of closed source
BIOSes and BIOS engineers.

This patch makes the kernel detect this particularly brokenness and
abort early -- and fixes up the missing iounmap in the error paths which
I noticed while I was poking at it.

This should fix kernel.org bug #14003, which was being called a
'regression' -- I think because the IOMMU code used to trip over
_another_ BIOS bug earlier than this one, and that one _did_ cause it to
abort.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
---
 drivers/pci/dmar.c |   22 ++++++++++++++++++----
 1 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/dmar.c b/drivers/pci/dmar.c
index 7b287cb..380b60e 100644
--- a/drivers/pci/dmar.c
+++ b/drivers/pci/dmar.c
@@ -632,20 +632,31 @@ int alloc_iommu(struct dmar_drhd_unit *drhd)
 	iommu->cap = dmar_readq(iommu->reg + DMAR_CAP_REG);
 	iommu->ecap = dmar_readq(iommu->reg + DMAR_ECAP_REG);
 
+	if (iommu->cap == (uint64_t)-1 && iommu->ecap == (uint64_t)-1) {
+		/* Promote an attitude of violence to a BIOS engineer today */
+		WARN(1, "Your BIOS is broken; DMAR reported at address %llx returns all ones!\n"
+		     "BIOS vendor: %s; Ver: %s; Product Version: %s\n",
+		     drhd->reg_base_addr,
+		     dmi_get_system_info(DMI_BIOS_VENDOR),
+		     dmi_get_system_info(DMI_BIOS_VERSION),
+		     dmi_get_system_info(DMI_PRODUCT_VERSION));
+		goto err_unmap;
+	}
+
 #ifdef CONFIG_DMAR
 	agaw = iommu_calculate_agaw(iommu);
 	if (agaw < 0) {
 		printk(KERN_ERR
 		       "Cannot get a valid agaw for iommu (seq_id = %d)\n",
 		       iommu->seq_id);
-		goto error;
+		goto err_unmap;
 	}
 	msagaw = iommu_calculate_max_sagaw(iommu);
 	if (msagaw < 0) {
 		printk(KERN_ERR
 			"Cannot get a valid max agaw for iommu (seq_id = %d)\n",
 			iommu->seq_id);
-		goto error;
+		goto err_unmap;
 	}
 #endif
 	iommu->agaw = agaw;
@@ -665,7 +676,7 @@ int alloc_iommu(struct dmar_drhd_unit *drhd)
 	}
 
 	ver = readl(iommu->reg + DMAR_VER_REG);
-	pr_debug("IOMMU %llx: ver %d:%d cap %llx ecap %llx\n",
+	pr_info("IOMMU %llx: ver %d:%d cap %llx ecap %llx\n",
 		(unsigned long long)drhd->reg_base_addr,
 		DMAR_VER_MAJOR(ver), DMAR_VER_MINOR(ver),
 		(unsigned long long)iommu->cap,
@@ -675,7 +686,10 @@ int alloc_iommu(struct dmar_drhd_unit *drhd)
 
 	drhd->iommu = iommu;
 	return 0;
-error:
+
+ err_unmap:
+	iounmap(iommu->reg);
+ error:
 	kfree(iommu);
 	return -1;
 }
-- 
1.6.2.5


-- 
dwmw2

Re: [PATCH] intel-iommu: Work around yet another BIOS bug

[Andrew Morton]

On Thu, 20 Aug 2009 09:01:58 +0100 David Woodhouse <dwmw2@infradead.org> wrote:

> +	if (iommu->cap == (uint64_t)-1 && iommu->ecap == (uint64_t)-1) {
> +		/* Promote an attitude of violence to a BIOS engineer today */
> +		WARN(1, "Your BIOS is broken; DMAR reported at address %llx returns all ones!\n"
> +		     "BIOS vendor: %s; Ver: %s; Product Version: %s\n",
> +		     drhd->reg_base_addr,

Printing a u64 with %ll will (still) generate a warning on four architectures.

> +		     dmi_get_system_info(DMI_BIOS_VENDOR),
> +		     dmi_get_system_info(DMI_BIOS_VERSION),
> +		     dmi_get_system_info(DMI_PRODUCT_VERSION));
> +		goto err_unmap;
> +	}

Re: [PATCH] intel-iommu: Work around yet another BIOS bug

[David Woodhouse]

On Thu, 2009-08-20 at 02:44 -0700, Andrew Morton wrote:
> Printing a u64 with %ll will (still) generate a warning on four
> architectures.

Still that many? I thought we'd fixed them all already.

Certainly we fixed IA64, which is the last one I cared about in the
context of this particular code.

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation

Re: [PATCH] intel-iommu: Work around yet another BIOS bug

[Andrew Morton]

On Thu, 20 Aug 2009 12:14:37 +0100 David Woodhouse <dwmw2@infradead.org> wrote:

> On Thu, 2009-08-20 at 02:44 -0700, Andrew Morton wrote:
> > Printing a u64 with %ll will (still) generate a warning on four
> > architectures.
> 
> Still that many? I thought we'd fixed them all already.

You're right, they all seem to be fixed.  Nobody tells me nuttin.  Whee.

Re: [PATCH] intel-iommu: Work around yet another BIOS bug

[Matthew Wilcox]

On Thu, Aug 20, 2009 at 02:44:53AM -0700, Andrew Morton wrote:
> On Thu, 20 Aug 2009 09:01:58 +0100 David Woodhouse <dwmw2@infradead.org> wrote:
> 
> > +	if (iommu->cap == (uint64_t)-1 && iommu->ecap == (uint64_t)-1) {
> > +		/* Promote an attitude of violence to a BIOS engineer today */
> > +		WARN(1, "Your BIOS is broken; DMAR reported at address %llx returns all ones!\n"
> > +		     "BIOS vendor: %s; Ver: %s; Product Version: %s\n",
> > +		     drhd->reg_base_addr,
> 
> Printing a u64 with %ll will (still) generate a warning on four architectures.

We've got them all now.

$ grep -l int-l64 arch/*/include/asm/types.h
arch/alpha/include/asm/types.h
arch/ia64/include/asm/types.h
arch/mips/include/asm/types.h
arch/powerpc/include/asm/types.h
$ grep -l int-ll64 $(grep -l int-l64 arch/*/include/asm/types.h)
arch/alpha/include/asm/types.h
arch/ia64/include/asm/types.h
arch/mips/include/asm/types.h
arch/powerpc/include/asm/types.h

ie all architectures which use int-l64 only do so for the benefit of
userspace, and use int-ll64 within the kernel.  I did check this by hand
too ;-)

-- 
Matthew Wilcox				Intel Open Source Technology Centre
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."

Re: [PATCH] intel-iommu: Work around yet another BIOS bug

[Ray Lee]

On Thu, Aug 20, 2009 at 1:01 AM, David Woodhouse <dwmw2@infradead.org> wrote:
> +       if (iommu->cap == (uint64_t)-1 && iommu->ecap == (uint64_t)-1) {
> +               /* Promote an attitude of violence to a BIOS engineer today */
> +               WARN(1, "Your BIOS is broken; DMAR reported at address %llx returns all ones!\n"

Think about changing this to a warning that "Your IOMMU appears to be
disabled." All ones is, after all, the traditional hint that the
device is turned off.

Re: [PATCH] intel-iommu: Work around yet another BIOS bug

[David Woodhouse]

On Thu, 2009-08-20 at 12:29 -0700, Ray Lee wrote:
> Think about changing this to a warning that "Your IOMMU appears to be
> disabled." All ones is, after all, the traditional hint that the
> device is turned off.

Hints are all very well, but the BIOS provided an ACPI table explicitly
telling us that there was an active IOMMU at this location.

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation

Re: [PATCH] intel-iommu: Work around yet another BIOS bug

[Grant Grundler]

On Thu, Aug 20, 2009 at 08:32:20PM +0100, David Woodhouse wrote:
> On Thu, 2009-08-20 at 12:29 -0700, Ray Lee wrote:
> > Think about changing this to a warning that "Your IOMMU appears to be
> > disabled." All ones is, after all, the traditional hint that the
> > device is turned off.
> 
> Hints are all very well, but the BIOS provided an ACPI table explicitly
> telling us that there was an active IOMMU at this location.

Could that be to reserve address space that the "disabled" IOMMU
might still be responding to?

Ie the BIOS hides the control registers so the OS won't talk to the
device but the IOMMU might still attempt to lookup certain address ranges.

I'm more inclined to believe it's sloppiness on the part of the BIOS
writers but thought this might be an alternative explanation.

thanks,
grant

Re: [PATCH] intel-iommu: Work around yet another BIOS bug

[David Woodhouse]

On Thu, 2009-08-20 at 15:47 -0600, Grant Grundler wrote:
> 
> Could that be to reserve address space that the "disabled" IOMMU
> might still be responding to?
> 
> Ie the BIOS hides the control registers so the OS won't talk to the
> device but the IOMMU might still attempt to lookup certain address
> ranges.
> 
> I'm more inclined to believe it's sloppiness on the part of the BIOS
> writers but thought this might be an alternative explanation.

Nah, this is just the normal story: "We smoked too much crack and fried
our brains, and we don't do any QA on the crap we write because that
would leave fewer hours in the day for us to service our crack habit".

We _really_ need open source firmware.

Or at _least_ firmware written by competent engineers -- but I think
we've all fairly much given up on that happening by now?

-- 
dwmw2

Re: [PATCH] intel-iommu: Work around yet another BIOS bug

[Carl-Daniel Hailfinger]

On 20.08.2009 23:54, David Woodhouse wrote:
> On Thu, 2009-08-20 at 15:47 -0600, Grant Grundler wrote:
>   
>> I'm more inclined to believe it's sloppiness on the part of the BIOS
>> writers but thought this might be an alternative explanation.
>>     
>
> Nah, this is just the normal story: "We smoked too much crack and fried
> our brains, and we don't do any QA on the crap we write because that
> would leave fewer hours in the day for us to service our crack habit".
>
> We _really_ need open source firmware.
>
> Or at _least_ firmware written by competent engineers -- but I think
> we've all fairly much given up on that happening by now?
>   

coreboot is an open source (GPLv2) x86 lightweight firmware written in C
(with optional BIOS and EFI compatibility modules if anyone cares).
While it does not support all current x86 chipsets, it still does
support quite a few mainboards.
The developers are friendly and try to fix any reported bugs. There are
two big obstacles until x86 world domination, though: A dozen core
developers are simply not enough for the huge task (there are more open
specs than developers can keep up with), and some chipset vendors don't
open their specs. Anyone interested is invited to contribute and make
the world a better place.

Oh, and it supports packing a Linux kernel in the mainboard flash ROM
for diskless/networkless booting (that coreboot+Linux combination was
called LinuxBIOS some time ago).

More information at http://www.coreboot.org/


Regards,
Carl-Daniel

Re: [Bugme-new] [Bug 14003] New: Infinite loop on bootup while handling DMAR

[Faidon Liambotis]

David Woodhouse wrote:
> On Wed, 2009-08-19 at 14:26 -0700, Andrew Morton wrote:
>>> http://bugzilla.kernel.org/show_bug.cgi?id=14003
>>>            Summary: Infinite loop on bootup while handling DMAR
>> That's a box-killing post-2.6.30 regression.
> 
> It's a BIOS bug -- the user's BIOS is written by idiots who obviously
> shipped it without any QA whatsoever.
Matt may be wondering why he was addressed, since the bugzilla entry
mentions only a bug in HP's BIOS.

I'm experiencing the same bug on a newly bought Dell Optiplex 760 with
BIOS version A03, as explained in my mail in lkml,
<4A89CB52.4030008@debian.org>, subject "[regression, bisected] fails to
boot on Dell Optiplex 760 with VT-d enabled".

Thanks,
Faidon