[PATCH] Pentium Pro - sysenter - doublefault

[Jim Houston <jim.houston@comcast.net>]

Hi Everyone,

I upgraded my Pentium Pro system to Redhat 9, installed a
linux-2.6.0-test3 kernel, and it fails with a double-fault when
init starts.

The code which decides if it is o.k. to use sysenter is broken for
some Pentium Pro cpus ,in particular, this bit of code from
arch/i386/kernel/cpu/intel.c:

	/* SEP CPUID bug: Pentium Pro reports SEP but doesn't have it */
	if ( c->x86 == 6 && c->x86_model < 3 && c->x86_mask < 3 )
		clear_bit(X86_FEATURE_SEP, c->x86_capability);

On my cpu model=1 and mask=9, it doesn't clear 86_FEATURE_SEP.
This results in a double-fault when init starts.  The double-fault
happens on the sysexit.  The new double-fault handler caught this
nicely, and I was able to debug this with kgdb.

The logic above is exactly what Intel says to do in "IA-32 Intel®
Architecture Software Developer's Manual, Volume 2: Instruction Set
Reference" on page 3-767.  It also says that sysenter was added to the
Pentium II.

I checked the Pentium Pro and Pentium II Specifications Update manuals
hoping to find the details to justify the "mask < 3" portion of the test
above. They both describe sysenter related errata but none which was
fixed in mask 3.

The attached patch avoids using sysenter on all Pentium Pro systems.

Jim Houston - Concurrent Computer Corp.


diff -urN linux-2.6.0-test3.orig/arch/i386/kernel/cpu/intel.c
linux-2.6.0-test3.new/arch/i386/kernel/cpu/intel.c
--- linux-2.6.0-test3.orig/arch/i386/kernel/cpu/intel.c	2003-08-20
10:30:14.000000000 -0400
+++ linux-2.6.0-test3.new/arch/i386/kernel/cpu/intel.c	2003-08-21
14:39:35.000000000 -0400
@@ -246,7 +246,7 @@
 	}
 
 	/* SEP CPUID bug: Pentium Pro reports SEP but doesn't have it */
-	if ( c->x86 == 6 && c->x86_model < 3 && c->x86_mask < 3 )
+	if ( c->x86 == 6 && c->x86_model < 3)
 		clear_bit(X86_FEATURE_SEP, c->x86_capability);
 	
 	/* Names for the Pentium II/Celeron processors