Re: [RFC PATCH 0/6] Unify the Interrupt Mode and setup it as soon as possible

[Dou Liyang <douly.fnst@cn.fujitsu.com>]

At 03/30/2017 11:03 AM, Dou Liyang wrote:
> Hi Baoquan,
>
> At 03/30/2017 10:08 AM, Baoquan He wrote:
>> Hi Liyang,
>>
>> This is awesome. I planned to do this after kaslr back porting, glad to
>> see your posting. I like below diagram and the idea of patch 2/6
>> framework. Will review and see what I can do to help since rhel bug from
>> FJ is assigned to me.
>>
>
> Thanks very much for your join! We have investigated the bug almost
> half a year. :)
>
> In my opinion,
> If we plan to refactor the process of APIC initialization for the bug.
> There must be lots of work need to be done. This patchset is just the
> first step. When I test it, I am thinking about:
>
> 1. The check and logic in each enable and setup LAPIC/IOAPIC functions.
> 2. The process of IRQ remapping.
> 3. The check and init of APIC timer.
> 4. The relationship between the various switches, such as If
> the smp_found_config is 1, the acpi_lapic must be 1.
>
> And following work to me are:
>
> 1. Use more test cases to test.
> 2. learn the IOMMU.
> 3. trace the APIC timer code.
> 4. make the check logic more clear.
>
> Hope to be helpful to you.
>
>> Thanks for the effort!
>>
>> And add Joerg to this thread since he knows IOMMU very well.
>

ahh,

--cc joro@8bytes.org, not joro@8types.org

Thanks
Liyang

> oops, Yes, I forgot it, Thanks!
>
> Thanks
> Liyang
>
>>
>> Thanks
>> Baoquan
>>
>> On 03/29/17 at 10:55pm, Dou Liyang wrote:
>>> According to Ingo's and Eric's advice[1,2], Try my best to optimize the
>>> init of Interrupt Mode for x86.
>>>
>>> The MP specification defines three different interrupt modes as follows:
>>>
>>>  1. PIC Mode
>>>  2. Virtual Wire Mode
>>>  3. Symmetic I/O Mode
>>>
>>> Currently, In kernel,
>>>
>>> 1. Setup the Virtual Wire Mode during the IRQ initialization(
>>> step 1 in the following figure).
>>> 2. Enable and Setup the Symmetic I/O Mode either during the
>>> SMP-capabe system prepares CPUs(step 2) or during the UP system
>>> initializes itself(step 3).
>>>
>>>   start_kernel
>>> +---------------+
>>> |
>>> +--> .......
>>> |
>>> |    setup_arch
>>> +--> +-------+
>>> |
>>> |    init_IRQ
>>> +-> +--+-----+
>>> |      |        init_ISA_irqs
>>> |      +------> +-+--------+
>>> |                 |         +----------------+
>>> +--->             +------>  | 1.init_bsp_APIC|
>>> |     .......               +----------------+
>>> +--->
>>> |     rest_init
>>> +--->---+-----+
>>> |       |   kernel_init
>>> |       +> ----+-----+
>>> |              |   kernel_init_freeable
>>> |              +->  ----+-------------+
>>> |                       |     smp_prepare_cpus
>>> |                       +---> +----+---------+
>>> |                       |          |   +-------------------+
>>> |                       |          +-> |2.  apic_bsp_setup |
>>> |                       |              +-------------------+
>>> |                       |
>>> v                       |     smp_init
>>>                         +---> +---+----+
>>>                                   |    +-------------------+
>>>                                   +--> |3.  apic_bsp_setup |
>>>                                        +-------------------+
>>>
>>> The purpose of this patchset is Unifing these setup steps and
>>> executing as
>>> soon as possible as follows:
>>>
>>>    start_kernel
>>> ---------------+
>>> |
>>> |
>>> |
>>> |     init_IRQ
>>> +---->---+----+
>>> |        |
>>> |        |      +--------------------+
>>> |        +----> | 4. init_bsp_APIC   |
>>> |               +--------------------+
>>> v
>>>
>>> By the way, Also fix a bug about kexec[3].
>>>
>>>
>>> Some doubts, need help:
>>>
>>> 1. Patchset has influence on IOMMU in enable_IR_x2apic(). Not sure
>>> it can be in advance?
>>>
>>> 2. Due to
>>>
>>> Commit 8c3ba8d04924 ("x86, apic: ack all pending irqs when crashed/on
>>> kexec")
>>>
>>>  ..., patchset also needs TSC and uses the "cpu_khz" in
>>> setup_local_APIC().
>>> And a warning[4] will be triggered when crashed/on kexec. Not sure
>>> how to
>>> modify?
>>>
>>> [1]. https://lkml.org/lkml/2016/8/2/929
>>> [2]. https://lkml.org/lkml/2016/8/1/506
>>> [3]. https://lkml.org/lkml/2016/7/25/1118
>>> [4]. WARN_ON(max_loops <= 0) in setup_local_APIC()
>>>
>>> Dou Liyang (6):
>>>   x86/apic: Replace init_bsp_APIC() with apic_virture_wire_mode_setup()
>>>   x86/apic: Construct a framework for setuping APIC mode as soon as
>>>     possible
>>>   x86/apic: Extract APIC timer related code from apic_bsp_setup()
>>>   x86/apic: Make the APIC mode setup earlier for SMP-capable system
>>>   x86/apic: Make the APIC mode setup earlier for UP system
>>>   x86/apic: Remove the apic_virture_wire_mode_setup()
>>>
>>>  arch/x86/include/asm/apic.h    |   7 +-
>>>  arch/x86/include/asm/io_apic.h |   2 +
>>>  arch/x86/kernel/apic/apic.c    | 218
>>> ++++++++++++++++++++++++-----------------
>>>  arch/x86/kernel/apic/io_apic.c |   4 +-
>>>  arch/x86/kernel/irqinit.c      |   6 +-
>>>  arch/x86/kernel/smpboot.c      |  68 ++-----------
>>>  6 files changed, 149 insertions(+), 156 deletions(-)
>>>
>>> --
>>> 2.5.5
>>>
>>>
>>>
>>
>>
>>