[PATCH] 2.6.9: Only handle system NMIs on the BSP

[PATCH] 2.6.9: Only handle system NMIs on the BSP

[Maciej W. Rozycki]

Hello,

 While discussing races in the NMI handler's trailer fiddling with RTC
registers, I've discovered we incorrectly attempt to handle NMIs coming
from the system (memory errors, IOCHK# assertions, etc.) with all
processors even though the interrupts are only routed to the bootstrap
processor.  If one of these events coincides with a NMI watchdog tick it
may even be handled multiple times in parallel.

 Here is a fix that makes application processors ignore these events.  
They no longer access the NMI status bits at I/O port 0x61 which has also
the advantage of removing the contention on the port when the I/O APIC NMI
watchdog makes all processors arrive at the handler at the same time.

 For both the i386 and the x86_64.  Please apply.

  Maciej

Signed-off-by: Maciej W. Rozycki <macro@linux-mips.org>

patch-2.6.9-nmi-bsp-1
diff -up --recursive --new-file linux-2.6.9.macro/arch/i386/kernel/traps.c linux-2.6.9/arch/i386/kernel/traps.c
--- linux-2.6.9.macro/arch/i386/kernel/traps.c	2004-10-12 23:57:01.000000000 +0000
+++ linux-2.6.9/arch/i386/kernel/traps.c	2004-11-03 01:04:13.000000000 +0000
@@ -598,7 +598,11 @@ void die_nmi (struct pt_regs *regs, cons
 
 static void default_do_nmi(struct pt_regs * regs)
 {
-	unsigned char reason = get_nmi_reason();
+	unsigned char reason = 0;
+
+	/* Only the BSP gets external NMIs from the system.  */
+	if (!smp_processor_id())
+		reason = get_nmi_reason();
  
 	if (!(reason & 0xc0)) {
 		if (notify_die(DIE_NMI_IPI, "nmi_ipi", regs, reason, 0, SIGINT)
diff -up --recursive --new-file linux-2.6.9.macro/arch/x86_64/kernel/traps.c linux-2.6.9/arch/x86_64/kernel/traps.c
--- linux-2.6.9.macro/arch/x86_64/kernel/traps.c	2004-10-12 23:57:12.000000000 +0000
+++ linux-2.6.9/arch/x86_64/kernel/traps.c	2004-11-03 01:04:27.000000000 +0000
@@ -565,8 +565,12 @@ static void unknown_nmi_error(unsigned c
 
 asmlinkage void default_do_nmi(struct pt_regs * regs)
 {
-	unsigned char reason = inb(0x61);
+	unsigned char reason = 0;
 
+	/* Only the BSP gets external NMIs from the system.  */
+	if (!smp_processor_id())
+		reason = inb(0x61);
+ 
 	if (!(reason & 0xc0)) {
 		if (notify_die(DIE_NMI_IPI, "nmi_ipi", regs, reason, 0, SIGINT)
 								== NOTIFY_STOP)

Re: [PATCH] 2.6.9: Only handle system NMIs on the BSP

[Andi Kleen]

On Wed, Nov 03, 2004 at 02:07:43AM +0000, Maciej W. Rozycki wrote:
> Hello,
> 
>  While discussing races in the NMI handler's trailer fiddling with RTC
> registers, I've discovered we incorrectly attempt to handle NMIs coming
> from the system (memory errors, IOCHK# assertions, etc.) with all
> processors even though the interrupts are only routed to the bootstrap
> processor.  If one of these events coincides with a NMI watchdog tick it

Where is this documented/guaranteed that only APs get such NMIs? 

I don't see such a guarantee in the AMD documentation for example ... 

-Andi

Re: [PATCH] 2.6.9: Only handle system NMIs on the BSP

[Maciej W. Rozycki]

On Wed, 3 Nov 2004, Andi Kleen wrote:

> >  While discussing races in the NMI handler's trailer fiddling with RTC
> > registers, I've discovered we incorrectly attempt to handle NMIs coming
> > from the system (memory errors, IOCHK# assertions, etc.) with all
> > processors even though the interrupts are only routed to the bootstrap
> > processor.  If one of these events coincides with a NMI watchdog tick it
> 
> Where is this documented/guaranteed that only APs get such NMIs? 

 Only the BSP gets them (not APs) and it's we who arrange for that. ;-)  
We only enable LVT1 on the BSP -- APs have this input masked.  See 
setup_local_APIC().

> I don't see such a guarantee in the AMD documentation for example ... 

 It's specific to system software run.  You'd have to have a look at some
Linux documentation covering the area -- your best bet is the code itself.

  Maciej

Re: [PATCH] 2.6.9: Only handle system NMIs on the BSP

[Andi Kleen]

On Wed, Nov 03, 2004 at 03:04:06AM +0000, Maciej W. Rozycki wrote:
> On Wed, 3 Nov 2004, Andi Kleen wrote:
> 
> > >  While discussing races in the NMI handler's trailer fiddling with RTC
> > > registers, I've discovered we incorrectly attempt to handle NMIs coming
> > > from the system (memory errors, IOCHK# assertions, etc.) with all
> > > processors even though the interrupts are only routed to the bootstrap
> > > processor.  If one of these events coincides with a NMI watchdog tick it
> > 
> > Where is this documented/guaranteed that only APs get such NMIs? 
> 
>  Only the BSP gets them (not APs) and it's we who arrange for that. ;-)  
> We only enable LVT1 on the BSP -- APs have this input masked.  See 
> setup_local_APIC().

Ok, then it's fine for me. Thanks. 

-Andi