On 03/08/2013 03:50 PM, Yinghai Lu wrote:
> On Thu, Mar 7, 2013 at 11:33 PM, WANG Chao <chaowang@redhat.com> wrote:
>> On 03/08/2013 03:27 PM, Yinghai Lu wrote:
>>> On Thu, Mar 7, 2013 at 11:20 PM, WANG Chao <chaowang@redhat.com> wrote:
>>>>>
>>>>> looks like your system DO have DMAR table, please enable dmar
>>>>> remapping in your kernel config.
>>>>
>>>> I've already got following config:
>>>> CONFIG_DMAR_TABLE=y
>>>> CONFIG_INTEL_IOMMU=y
>>>> CONFIG_IRQ_REMAP=y
>>>>
>>>> but I don't have intel_iommu=on in kernel cmdline. IIRC, iommu will prevent
>>>> 2nd kernel from booting ...
>>>
>>> Did you put intel_iommu=on on first and second cpu both?
>>
>> I tried, 2nd kernel didn't boot and keep splitting errors like these:
>> [    2.106939] DMAR: No ATSR found
>> [    2.110121] IOMMU 0 0xfed90000: using Queued invalidation
>> [    2.115522] IOMMU 1 0xfed91000: using Queued invalidation
>> [    2.120919] IOMMU: Setting RMRR:
>> [    2.124162] IOMMU: Setting identity map for device 0000:00:02.0 [0xab800000
>> - 0xaf9fffff]
>> [    2.133099] IOMMU: Setting identity map for device 0000:00:1d.0 [0xaac95000
>> - 0xaacb2fff]
>> [    2.141305] IOMMU: Setting identity map for device 0000:00:1a.0 [0xaac95000
>> - 0xaacb2fff]
>> [    2.149503] IOMMU: Setting identity map for device 0000:00:14.0 [0xaac95000
>> - 0xaacb2fff]
>> [    2.157690] IOMMU: Prepare 0-16MiB unity mapping for LPC
>> [    2.163011] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff
>> [Errors, here we go]
>> [    2.170932] dmar: DRHD: handling fault status reg 3
>> [    2.170933] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
>> [    2.182486] dmar: DMAR:[DMA Write] Request device [00:02.0] fault addr ffffe000
>> [    2.182486] DMAR:[fault reason 05] PTE Write access is not set
>> [    2.195705] dmar: DRHD: handling fault status reg 3
>> [    2.200570] dmar: DMAR:[DMA Read] Request device [00:02.0] fault addr ff873000
>> [    2.200570] DMAR:[fault reason 06] PTE Read access is not set
>> [    2.213618] dmar: DRHD: handling fault status reg 3
> 
> my Nehalem-EX and Westmere-EX is working with iommu enabled in second kernel.
> 
> what is 00:02.0 in your system?
This IOMMU issue is related to https://lkml.org/lkml/2012/11/26/814. We can
discuss this IOMMU issue in that thread.
Anyway 00:02.0 is a video card, the box is Ivy Bridge.
# lspci -s 00:02.0 -v
00:02.0 VGA compatible controller: Intel Corporation 3rd Gen Core processor
Graphics Controller (rev 09) (prog-if 00 [VGA controller])
        Subsystem: Intel Corporation Device 2211
        Flags: bus master, fast devsel, latency 0, IRQ 44
        Memory at afc00000 (64-bit, non-prefetchable) [size=4M]
        Memory at c0000000 (64-bit, prefetchable) [size=256M]
        I/O ports at 6000 [size=64]
        Expansion ROM at <unassigned> [disabled]
        Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
        Capabilities: [d0] Power Management version 2
        Capabilities: [a4] PCI Advanced Features
        Kernel driver in use: i915
> 
> Is your kernel upsteam kernel or redhat flavor one?
I'm using upstream vanilla kernel.
I put kernel config in attachment.


Is it expected to intel_iommu=on or crashkernel_low to make 2nd kernel boot in
3.9? Back in 3.8, it works just fine w/ only crashkernel param.

BTW, Have a good weekend!

Thanks,
WANG Chao

> 
> Thanks
> 
> Yinghai
>