[PATCH] Secondary cpus boot-up for non defalut location built kernels

[PATCH] Secondary cpus boot-up for non defalut location built kernels

[Vivek Goyal]

[-- Attachment #1: Type: text/plain, Size: 1004 bytes --]

Hi,

This patch fixes the problem of secondary cpus boot up. This situation
is faced when kernel is built for non default locations like 16MB and
onwards. In this configuration, only primary cpu (BP) comes and
secondary cpus don't boot.

Problem occurs because in trampoline code, lgdt is not able to load the
GDT as it happens to be situated beyond 16MB. This is due to the fact
that cpu is still in real mode and default operand size is 16bit.

This patch uses lgdtl instead of lgdt to force operand size to 32
instead of 16.

Testing Details:
----------------

I have tested the patch on all combinations of following testing setup.

Kernel:
------
2.6.10-rc3-mm1

Hardware:
---------
1. 8way, PIII, 
2. 4way, Xeon, Hyper Threaded

Boot Loader:
-----------
1. Grub
2. LILO
3. Kexec

Memory Location Kernel built for:
--------------------------------
1MB
32MB

Kernel Image Type:
------------------
bzImage

Thanks
Vivek
-- 
Vivek Goyal
Linux Technology Center
India Software Labs
IBM India, Bangalore

[-- Attachment #2: boot_ap_for_nondefault_kernel.patch --]
[-- Type: text/plain, Size: 1278 bytes --]


This patch fixes the problem of secondary cpus not coming up over a reboot. 
This problem was seen when a kernel compiled for non default (16MB) location 
is booted.

Signed-off-by: Vivek Goyal <vgoyal@in.ibm.com>
---

 linux-2.6.10-rc3-mm1-changes-root/arch/i386/kernel/trampoline.S |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletion(-)

diff -puN arch/i386/kernel/trampoline.S~boot_ap_for_nondefault_kernel arch/i386/kernel/trampoline.S
--- linux-2.6.10-rc3-mm1-changes/arch/i386/kernel/trampoline.S~boot_ap_for_nondefault_kernel	2004-12-22 16:36:50.000000000 +0530
+++ linux-2.6.10-rc3-mm1-changes-root/arch/i386/kernel/trampoline.S	2004-12-22 16:47:54.000000000 +0530
@@ -51,8 +51,14 @@ r_base = .
 	movl	$0xA5A5A5A5, trampoline_data - r_base
 				# write marker for master knows we're running
 
+	/* GDT tables in non default location kernel can be beyond 16MB and
+	 * lgdt will not be able to load the address as in real mode default
+	 * operand size is 16bit. Use lgdtl instead to force operand size
+	 * to 32 bit.
+	 */
+
 	lidt	boot_idt - r_base	# load idt with 0, 0
-	lgdt	boot_gdt - r_base	# load gdt with whatever is appropriate
+	lgdtl	boot_gdt - r_base	# load gdt with whatever is appropriate
 
 	xor	%ax, %ax
 	inc	%ax		# protected mode (PE) bit
_

Re: [PATCH] Secondary cpus boot-up for non defalut location built kernels

[Eric W. Biederman]

Vivek Goyal <vgoyal@in.ibm.com> writes:

> Hi,
> 
> This patch fixes the problem of secondary cpus boot up. This situation
> is faced when kernel is built for non default locations like 16MB and
> onwards. In this configuration, only primary cpu (BP) comes and
> secondary cpus don't boot.
> 
> Problem occurs because in trampoline code, lgdt is not able to load the
> GDT as it happens to be situated beyond 16MB. This is due to the fact
> that cpu is still in real mode and default operand size is 16bit.

Which means that the cpu will load a 24bit linear address (3 bytes)
because it is acting like a 286.
 
> This patch uses lgdtl instead of lgdt to force operand size to 32
> instead of 16.

Sounds sane looking at the trampoline I suspect that people thought
it was using a 32bit address all along.  And the code only worked
because the data was stored little endian.  We probably want to apply
the same fix to the ldt instruction just about it.  Just in case
we ever decide to populate an idt at that point.

But the fix certainly looks good from here.

Eric


> Signed-off-by: Vivek Goyal <vgoyal@in.ibm.com>
> ---
> 
>  linux-2.6.10-rc3-mm1-changes-root/arch/i386/kernel/trampoline.S | 8 +++++++-
> 
>  1 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff -puN arch/i386/kernel/trampoline.S~boot_ap_for_nondefault_kernel
> arch/i386/kernel/trampoline.S
> 
> ---
> linux-2.6.10-rc3-mm1-changes/arch/i386/kernel/trampoline.S~boot_ap_for_nondefault_kernel
> 2004-12-22 16:36:50.000000000 +0530
> 
> +++ linux-2.6.10-rc3-mm1-changes-root/arch/i386/kernel/trampoline.S 2004-12-22
> 16:47:54.000000000 +0530
> 
> @@ -51,8 +51,14 @@ r_base = .
>  	movl	$0xA5A5A5A5, trampoline_data - r_base
>  				# write marker for master knows we're running
>  
> +	/* GDT tables in non default location kernel can be beyond 16MB and
> +	 * lgdt will not be able to load the address as in real mode default
> +	 * operand size is 16bit. Use lgdtl instead to force operand size
> +	 * to 32 bit.
> +	 */
> +
>  	lidt	boot_idt - r_base	# load idt with 0, 0
> -	lgdt	boot_gdt - r_base	# load gdt with whatever is appropriate
> +	lgdtl	boot_gdt - r_base	# load gdt with whatever is appropriate
>  
>  	xor	%ax, %ax
>  	inc	%ax		# protected mode (PE) bit
> _

Re: [PATCH] Secondary cpus boot-up for non defalut location built kernels

[Vivek Goyal]

[-- Attachment #1: Type: text/plain, Size: 1226 bytes --]


> > This patch fixes the problem of secondary cpus boot up. This situation
> > is faced when kernel is built for non default locations like 16MB and
> > onwards. In this configuration, only primary cpu (BP) comes and
> > secondary cpus don't boot.
> > 
> > Problem occurs because in trampoline code, lgdt is not able to load the
> > GDT as it happens to be situated beyond 16MB. This is due to the fact
> > that cpu is still in real mode and default operand size is 16bit.
> 
> Which means that the cpu will load a 24bit linear address (3 bytes)
> because it is acting like a 286.
>  
> > This patch uses lgdtl instead of lgdt to force operand size to 32
> > instead of 16.
> 
> Sounds sane looking at the trampoline I suspect that people thought
> it was using a 32bit address all along.  And the code only worked
> because the data was stored little endian.  

> We probably want to apply
> the same fix to the ldt instruction just about it.  Just in case
> we ever decide to populate an idt at that point.


Sounds good. Sending the patch again with suggested change included.

> 
> But the fix certainly looks good from here.
> 
> Eric
> 

-- 
Vivek Goyal
Linux Technology Center
India Software Labs
IBM India, Bangalore

[-- Attachment #2: boot_ap_for_nondefault_kernel.patch --]
[-- Type: text/plain, Size: 1328 bytes --]


This patch fixes the problem of secondary cpus not coming up over a reboot. 
This problem was seen when a kernel compiled for non default (16MB) location 
is booted.

Signed-off-by: Vivek Goyal <vgoyal@in.ibm.com>
---

 linux-2.6.10-rc3-mm1-changes-root/arch/i386/kernel/trampoline.S |   10 ++++++++--
 1 files changed, 8 insertions(+), 2 deletions(-)

diff -puN arch/i386/kernel/trampoline.S~boot_ap_for_nondefault_kernel arch/i386/kernel/trampoline.S
--- linux-2.6.10-rc3-mm1-changes/arch/i386/kernel/trampoline.S~boot_ap_for_nondefault_kernel	2004-12-22 16:36:50.000000000 +0530
+++ linux-2.6.10-rc3-mm1-changes-root/arch/i386/kernel/trampoline.S	2004-12-22 21:51:35.000000000 +0530
@@ -51,8 +51,14 @@ r_base = .
 	movl	$0xA5A5A5A5, trampoline_data - r_base
 				# write marker for master knows we're running
 
-	lidt	boot_idt - r_base	# load idt with 0, 0
-	lgdt	boot_gdt - r_base	# load gdt with whatever is appropriate
+	/* GDT tables in non default location kernel can be beyond 16MB and
+	 * lgdt will not be able to load the address as in real mode default
+	 * operand size is 16bit. Use lgdtl instead to force operand size
+	 * to 32 bit.
+	 */
+
+	lidtl	boot_idt - r_base	# load idt with 0, 0
+	lgdtl	boot_gdt - r_base	# load gdt with whatever is appropriate
 
 	xor	%ax, %ax
 	inc	%ax		# protected mode (PE) bit
_