Re: [PATCH] x86_64: restore mask_bits in msi shutdown

["Yinghai Lu" <yhlu.kernel@gmail.com>]

On Tue, Apr 15, 2008 at 1:12 AM, Andrew Morton
<akpm@linux-foundation.org> wrote:
>
> On Fri, 11 Apr 2008 16:26:11 -0700 Yinghai Lu <yhlu.kernel.send@gmail.com> wrote:
>
>  > I can not kexec RHEL 5.1 from 2.6.25-rc3 later
>  >
>  > caused by:
>  > commit 89d694b9dbe769ca1004e01db0ca43964806a611
>  > Author: Thomas Gleixner <tglx@linutronix.de>
>  > Date:   Mon Feb 18 18:25:17 2008 +0100
>  >
>  >     genirq: do not leave interupts enabled on free_irq
>  >
>  >     The default_disable() function was changed in commit:
>  >
>  >      76d2160147f43f982dfe881404cfde9fd0a9da21
>  >      genirq: do not mask interrupts by default
>  >
>  >     It removed the mask function in favour of the default delayed
>  >     interrupt disabling. Unfortunately this also broke the shutdown in
>  >     free_irq() when the last handler is removed from the interrupt for
>  >     those architectures which rely on the default implementations. Now we
>  >     can end up with a enabled interrupt line after the last handler was
>  >     removed, which can result in spurious interrupts.
>  >
>  >     Fix this by adding a default_shutdown function, which is only
>  >     installed, when the irqchip implementation does provide neither a
>  >     shutdown nor a disable function.
>  >
>  >     [@stable: affected versions: .21 - .24 ]
>  >
>  >
>  >
>  > for MSI, default_shutdown will call mask_bit for msi device. so all mask bits will
>  > left disabled after free_irq.
>  > then if kexec next kernel that only can use msi_enable bit.
>  > all device's MSI can not be used.
>  >
>  > So try to restore MSI mask bits that is saved before using msi in first kernel.
>  >
>  > Signed-off-by: Yinghai Lu <yhlu.kernel@gmail.com>
>  >
>  > Index: linux-2.6/arch/x86/kernel/io_apic_64.c
>  > ===================================================================
>  > --- linux-2.6.orig/arch/x86/kernel/io_apic_64.c
>  > +++ linux-2.6/arch/x86/kernel/io_apic_64.c
>  > @@ -2003,6 +2003,14 @@ static void set_msi_irq_affinity(unsigne
>  >  }
>  >  #endif /* CONFIG_SMP */
>  >
>  > +static void msi_shutdown(unsigned int irq)
>  > +{
>  > +     struct irq_desc *desc = irq_desc + irq;
>  > +
>  > +     msi_restore_mask_bits(irq);
>  > +     desc->status |= IRQ_MASKED;
>  > +}
>
>  I wonder if only x86_64 needs this treatment.
>
>
>  > Index: linux-2.6/drivers/pci/msi.c
>  > ===================================================================
>  > --- linux-2.6.orig/drivers/pci/msi.c
>  > +++ linux-2.6/drivers/pci/msi.c
>  > @@ -123,6 +123,31 @@ static void msix_flush_writes(unsigned i
>  >       }
>  >  }
>  >
>  > +void msi_restore_mask_bits(unsigned int irq)
>  > +{
>  > +     struct msi_desc *entry;
>  > +
>  > +     entry = get_irq_msi(irq);
>  > +     BUG_ON(!entry || !entry->dev);
>  > +     switch (entry->msi_attrib.type) {
>  > +     case PCI_CAP_ID_MSI:
>  > +             if (entry->msi_attrib.maskbit) {
>  > +                     int pos;
>  > +                     u32 mask_bits;
>  > +
>  > +                     pos = (long)entry->mask_base;
>  > +                     mask_bits = entry->orig_mask_bits;
>  > +                     pci_write_config_dword(entry->dev, pos, mask_bits);
>  > +             }
>  > +             break;
>  > +     case PCI_CAP_ID_MSIX:
>  > +             break;
>  > +     default:
>  > +             BUG();
>  > +             break;
>  > +     }
>  > +}
>
>  If no other architectures will ever need this, perhaps it is in the wrong
>  file.

need someone has powerpc (64?) etc platform with RHEL 5.1 with PCIe
NIC that is using MSI
to test if they have the same problem with kexec.

YH