[PATCH for 2.6.19] [9/9] x86_64: Fix race in exit_idle

[Andi Kleen <ak@suse.de>]

When another interrupt happens in exit_idle the exit idle notifier
could be called an incorrect number of times.

Add a test_and_clear_bit_pda and use it handle the bit
atomically against interrupts to avoid this.

Pointed out by Stephane Eranian

Signed-off-by: Andi Kleen <ak@suse.de>

---
 arch/x86_64/kernel/process.c |    3 +--
 include/asm-x86_64/pda.h     |    9 +++++++++
 2 files changed, 10 insertions(+), 2 deletions(-)

Index: linux/include/asm-x86_64/pda.h
===================================================================
--- linux.orig/include/asm-x86_64/pda.h
+++ linux/include/asm-x86_64/pda.h
@@ -109,6 +109,15 @@ extern struct x8664_pda _proxy_pda;
 #define sub_pda(field,val) pda_to_op("sub",field,val)
 #define or_pda(field,val) pda_to_op("or",field,val)
 
+/* This is not atomic against other CPUs -- CPU preemption needs to be off */
+#define test_and_clear_bit_pda(bit,field) ({		\
+	int old__;						\
+	asm volatile("btr %2,%%gs:%c3\n\tsbbl %0,%0"		\
+	    : "=r" (old__), "+m" (_proxy_pda.field) 		\
+	    : "dIr" (bit), "i" (pda_offset(field)) : "memory");	\
+	old__;							\
+})
+
 #endif
 
 #define PDA_STACKOFFSET (5*8)
Index: linux/arch/x86_64/kernel/process.c
===================================================================
--- linux.orig/arch/x86_64/kernel/process.c
+++ linux/arch/x86_64/kernel/process.c
@@ -88,9 +88,8 @@ void enter_idle(void)
 
 static void __exit_idle(void)
 {
-	if (read_pda(isidle) == 0)
+	if (test_and_clear_bit_pda(0, isidle) == 0)
 		return;
-	write_pda(isidle, 0);
 	atomic_notifier_call_chain(&idle_notifier, IDLE_END, NULL);
 }