mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* RE: [Fastboot] [PATCH] i386: move apic init in init_IRQs
@ 2005-10-31 17:04 Protasevich, Natalie
  2005-10-31 17:18 ` Zwane Mwaikambo
  2005-10-31 18:18 ` Eric W. Biederman
  0 siblings, 2 replies; 16+ messages in thread
From: Protasevich, Natalie @ 2005-10-31 17:04 UTC (permalink / raw)
  To: Eric W. Biederman, vgoyal
  Cc: Andrew Morton, fastboot, linux-kernel, Andi Kleen,
	Zwane Mwaikambo, Brown, Len

> Vivek Goyal <vgoyal@in.ibm.com> writes:
> > I have attached a patch with the mail which is now using 
> > boot_cpu_physical_apicid to hard set presence of boot cpu 
> instead of 
> > hard_smp_processor_id(). But the interesting questoin 
> remains why BIOS 
> > is not reporting the boot cpu.
> 
> 
> Ok.  I don't know if we care but I do know why we were not 
> seeing the report from the bios about your boot processor.  
> We record information about cpus for up to NR_CPUS, and since 
> you had a UP kernel NR_CPUS was one.
> 
> From your earlier boot log.
> 
> > ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled) 
> Processor #3 6:10 
> > APIC version 17
> > ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) 
> Processor #0 6:10 
> > APIC version 17
> > WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
> > ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled) 
> Processor #1 6:10 
> > APIC version 17
> > WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
> > ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled) 
> Processor #2 6:10 
> > APIC version 17
> > WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
> 
> So it looks like we have this problem completely fixed.  
> 
> I don't see a good way to ensure that we always record our 
> boot apicid when we boot a multiple processor system and only 
> use one processor.

Hi Eric,

There is another problem with that patch - it broke ES7000, I kept
getting timer panics. It turned out that check_timer() runs before the
actual APIC destination is set up. The IO-APIC uses
cpu_to_logical_apicid to find the destination - which needs
cpu_2_logical_apicid[] to be filled - which only happens after
processors are booted. At the time when check_timer() runs, it will
always be BAD_APICID (0xFF - broadcast) as the IO-APIC rte destination
for the timer, but ES7000 hardware happened not to support 0xFF so it
panics. I used bios_cpu_apicid[] to bring it up, but
cpu_to_logical_apicid is the only one that is kept up-to-date in the
hotplug case, so I cannot replace it in the cpu_mask_to_apicid(). 

There are probably some ways to fix this such as one below that I tried
(in mpparse.c):

        if (m->mpc_cpuflag & CPU_BOOTPROCESSOR) {
                Dprintk("    Bootup CPU\n");
                boot_cpu_physical_apicid = m->mpc_apicid;
+               cpu_2_logical_apicid[num_processors] = m->mpc_apicid;
        }
it  worked, but looks more like a kludge of course. I think IO-APIC
setup has to happen after processors were brought online and so is
check_timer(), if timer is connected through the IO-APIC.

--Natalie

 

^ permalink raw reply	[flat|nested] 16+ messages in thread

* RE: [Fastboot] [PATCH] i386: move apic init in init_IRQs
  2005-10-31 17:04 [Fastboot] [PATCH] i386: move apic init in init_IRQs Protasevich, Natalie
@ 2005-10-31 17:18 ` Zwane Mwaikambo
  2005-10-31 17:30   ` Eric W. Biederman
  2005-10-31 18:18 ` Eric W. Biederman
  1 sibling, 1 reply; 16+ messages in thread
From: Zwane Mwaikambo @ 2005-10-31 17:18 UTC (permalink / raw)
  To: Protasevich, Natalie
  Cc: Eric W. Biederman, vgoyal, Andrew Morton, fastboot, Linux Kernel,
	Andi Kleen, Brown, Len

On Mon, 31 Oct 2005, Protasevich, Natalie wrote:

> > Vivek Goyal <vgoyal@in.ibm.com> writes:
> > > I have attached a patch with the mail which is now using 
> > > boot_cpu_physical_apicid to hard set presence of boot cpu 
> > instead of 
> > > hard_smp_processor_id(). But the interesting questoin 
> > remains why BIOS 
> > > is not reporting the boot cpu.
> > 
> > 
> > Ok.  I don't know if we care but I do know why we were not 
> > seeing the report from the bios about your boot processor.  
> > We record information about cpus for up to NR_CPUS, and since 
> > you had a UP kernel NR_CPUS was one.
> > 
> > From your earlier boot log.
> > 
> > > ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled) 
> > Processor #3 6:10 
> > > APIC version 17
> > > ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) 
> > Processor #0 6:10 
> > > APIC version 17
> > > WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
> > > ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled) 
> > Processor #1 6:10 
> > > APIC version 17
> > > WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
> > > ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled) 
> > Processor #2 6:10 
> > > APIC version 17
> > > WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
> > 
> > So it looks like we have this problem completely fixed.  
> > 
> > I don't see a good way to ensure that we always record our 
> > boot apicid when we boot a multiple processor system and only 
> > use one processor.
> 
> Hi Eric,
> 
> There is another problem with that patch - it broke ES7000, I kept
> getting timer panics. It turned out that check_timer() runs before the
> actual APIC destination is set up. The IO-APIC uses
> cpu_to_logical_apicid to find the destination - which needs
> cpu_2_logical_apicid[] to be filled - which only happens after
> processors are booted. At the time when check_timer() runs, it will
> always be BAD_APICID (0xFF - broadcast) as the IO-APIC rte destination
> for the timer, but ES7000 hardware happened not to support 0xFF so it
> panics. I used bios_cpu_apicid[] to bring it up, but
> cpu_to_logical_apicid is the only one that is kept up-to-date in the
> hotplug case, so I cannot replace it in the cpu_mask_to_apicid(). 
> 
> There are probably some ways to fix this such as one below that I tried
> (in mpparse.c):
> 
>         if (m->mpc_cpuflag & CPU_BOOTPROCESSOR) {
>                 Dprintk("    Bootup CPU\n");
>                 boot_cpu_physical_apicid = m->mpc_apicid;
> +               cpu_2_logical_apicid[num_processors] = m->mpc_apicid;
>         }
> it  worked, but looks more like a kludge of course. I think IO-APIC
> setup has to happen after processors were brought online and so is
> check_timer(), if timer is connected through the IO-APIC.

Regarding IOAPIC setup I agree, Eric's patch is causing a few problems;

Total of 2 processors activated (14407.06 BogoMIPS).
checking TSC synchronization across 2 CPUs: passed.
softlockup thread 0 started up.
APIC error on CPU1: 00(40) <====
Brought up 2 CPUs

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Fastboot] [PATCH] i386: move apic init in init_IRQs
  2005-10-31 17:18 ` Zwane Mwaikambo
@ 2005-10-31 17:30   ` Eric W. Biederman
  2005-10-31 20:26     ` Zwane Mwaikambo
  0 siblings, 1 reply; 16+ messages in thread
From: Eric W. Biederman @ 2005-10-31 17:30 UTC (permalink / raw)
  To: Zwane Mwaikambo
  Cc: Protasevich, Natalie, vgoyal, Andrew Morton, fastboot,
	Linux Kernel, Andi Kleen, Brown, Len

Zwane Mwaikambo <zwane@arm.linux.org.uk> writes:

>
> Regarding IOAPIC setup I agree, Eric's patch is causing a few problems;
>
> Total of 2 processors activated (14407.06 BogoMIPS).
> checking TSC synchronization across 2 CPUs: passed.
> softlockup thread 0 started up.
> APIC error on CPU1: 00(40) <====
> Brought up 2 CPUs

Cool! Bug reports!

Zwane can I get a little more detail or is this just a warning?
I don't have enough information to understand what is happening
on your machine.

Eric


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Fastboot] [PATCH] i386: move apic init in init_IRQs
  2005-10-31 17:04 [Fastboot] [PATCH] i386: move apic init in init_IRQs Protasevich, Natalie
  2005-10-31 17:18 ` Zwane Mwaikambo
@ 2005-10-31 18:18 ` Eric W. Biederman
  1 sibling, 0 replies; 16+ messages in thread
From: Eric W. Biederman @ 2005-10-31 18:18 UTC (permalink / raw)
  To: Protasevich, Natalie
  Cc: vgoyal, Andrew Morton, fastboot, linux-kernel, Andi Kleen,
	Zwane Mwaikambo, Brown, Len

"Protasevich, Natalie" <Natalie.Protasevich@UNISYS.com> writes:

> Hi Eric,
>
> There is another problem with that patch - it broke ES7000, I kept
> getting timer panics. It turned out that check_timer() runs before the
> actual APIC destination is set up. The IO-APIC uses
> cpu_to_logical_apicid to find the destination - which needs
> cpu_2_logical_apicid[] to be filled - which only happens after
> processors are booted. At the time when check_timer() runs, it will
> always be BAD_APICID (0xFF - broadcast) as the IO-APIC rte destination
> for the timer, but ES7000 hardware happened not to support 0xFF so it
> panics. I used bios_cpu_apicid[] to bring it up, but
> cpu_to_logical_apicid is the only one that is kept up-to-date in the
> hotplug case, so I cannot replace it in the cpu_mask_to_apicid(). 
>
> There are probably some ways to fix this such as one below that I tried
> (in mpparse.c):
>
>         if (m->mpc_cpuflag & CPU_BOOTPROCESSOR) {
>                 Dprintk("    Bootup CPU\n");
>                 boot_cpu_physical_apicid = m->mpc_apicid;
> +               cpu_2_logical_apicid[num_processors] = m->mpc_apicid;
>         }
> it  worked, but looks more like a kludge of course. I think IO-APIC
> setup has to happen after processors were brought online and so is
> check_timer(), if timer is connected through the IO-APIC.

The first cpu is brought online much earlier than the rest.  So
we just need to setup a table for boot cpu earlier.  From the looks
of it mach-es700 won't work if you compile a uniprocessor kernel 
for it right now.

We need to do this a little later than in mptable but this should be a fairly
simple one or two line change.

If people keep breaking the subarchitectures by accident we might even 
inspire someone to make a comprehensible sub architecture implentation
on x86 one of these days.


Eric

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Fastboot] [PATCH] i386: move apic init in init_IRQs
  2005-10-31 17:30   ` Eric W. Biederman
@ 2005-10-31 20:26     ` Zwane Mwaikambo
  0 siblings, 0 replies; 16+ messages in thread
From: Zwane Mwaikambo @ 2005-10-31 20:26 UTC (permalink / raw)
  To: Eric W. Biederman
  Cc: Protasevich, Natalie, vgoyal, Andrew Morton, fastboot,
	Linux Kernel, Andi Kleen, Brown, Len

On Mon, 31 Oct 2005, Eric W. Biederman wrote:

> Zwane Mwaikambo <zwane@arm.linux.org.uk> writes:
> 
> >
> > Regarding IOAPIC setup I agree, Eric's patch is causing a few problems;
> >
> > Total of 2 processors activated (14407.06 BogoMIPS).
> > checking TSC synchronization across 2 CPUs: passed.
> > softlockup thread 0 started up.
> > APIC error on CPU1: 00(40) <====
> > Brought up 2 CPUs
> 
> Cool! Bug reports!
> 
> Zwane can I get a little more detail or is this just a warning?
> I don't have enough information to understand what is happening
> on your machine.

I just isolated which patch it was last night so i'm still not sure which 
part of it causes problems. The patch in question is;

i386-nmi_watchdog-merge-check_nmi_watchdog-fixes-from-x86_64.patch

This happens on both a dual P2-400 and a 3.6GHz P4 with HT enabled. What 
kind of information were you after?

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Fastboot] [PATCH] i386: move apic init in init_IRQs
  2005-10-31 18:31 Protasevich, Natalie
@ 2005-11-01  7:41 ` Eric W. Biederman
  0 siblings, 0 replies; 16+ messages in thread
From: Eric W. Biederman @ 2005-11-01  7:41 UTC (permalink / raw)
  To: Protasevich, Natalie
  Cc: vgoyal, Andrew Morton, fastboot, linux-kernel, Andi Kleen,
	Zwane Mwaikambo, Brown, Len

"Protasevich, Natalie" <Natalie.Protasevich@UNISYS.com> writes:

>> The first cpu is brought online much earlier than the rest.  
>> So we just need to setup a table for boot cpu earlier.  From 
>> the looks of it mach-es700 won't work if you compile a 
>> uniprocessor kernel for it right now.
>
> Yea, I didn't even try this - but I think it will produce the same
> result with regard to timer IOAPIC rte.
>
>> We need to do this a little later than in mptable but this 
>> should be a fairly simple one or two line change.
>
> Yes, it is maybe something like running map_cpu_to_logical_apicid() from
> APIC_init() just before the setup_IO_APIC().

The core piece of the puzzle is cpu_mask_to_apicid().  At the
time we setup the io_apic TARGET_CPUS will just be the bootstrap
processor.    So we might be able to get away with hard coding
the bootstrap processor in the non-SMP sections of the ioapic
startup code.

setup_ioapic_dest is going to fix things after we start the
cpus anyway so it should not be a problem.

Does that sound like a sane thing to do?

This code appears to affect all of the subarchitectures but the
default x86 one.  So it is clearly not just an ES7000 problem.

Now to figure out why Linus's laptop hates this patch...

Eric


^ permalink raw reply	[flat|nested] 16+ messages in thread

* RE: [Fastboot] [PATCH] i386: move apic init in init_IRQs
@ 2005-10-31 18:31 Protasevich, Natalie
  2005-11-01  7:41 ` Eric W. Biederman
  0 siblings, 1 reply; 16+ messages in thread
From: Protasevich, Natalie @ 2005-10-31 18:31 UTC (permalink / raw)
  To: Eric W. Biederman
  Cc: vgoyal, Andrew Morton, fastboot, linux-kernel, Andi Kleen,
	Zwane Mwaikambo, Brown, Len

 
> > Hi Eric,
> >
> > There is another problem with that patch - it broke ES7000, I kept 
> > getting timer panics. It turned out that check_timer() runs 
> before the 
> > actual APIC destination is set up. The IO-APIC uses 
> > cpu_to_logical_apicid to find the destination - which needs 
> > cpu_2_logical_apicid[] to be filled - which only happens after 
> > processors are booted. At the time when check_timer() runs, it will 
> > always be BAD_APICID (0xFF - broadcast) as the IO-APIC rte 
> destination 
> > for the timer, but ES7000 hardware happened not to support 
> 0xFF so it 
> > panics. I used bios_cpu_apicid[] to bring it up, but 
> > cpu_to_logical_apicid is the only one that is kept 
> up-to-date in the 
> > hotplug case, so I cannot replace it in the cpu_mask_to_apicid().
> >
> > There are probably some ways to fix this such as one below that I 
> > tried (in mpparse.c):
> >
> >         if (m->mpc_cpuflag & CPU_BOOTPROCESSOR) {
> >                 Dprintk("    Bootup CPU\n");
> >                 boot_cpu_physical_apicid = m->mpc_apicid;
> > +               cpu_2_logical_apicid[num_processors] = 
> m->mpc_apicid;
> >         }
> > it  worked, but looks more like a kludge of course. I think IO-APIC 
> > setup has to happen after processors were brought online and so is 
> > check_timer(), if timer is connected through the IO-APIC.
> 
> The first cpu is brought online much earlier than the rest.  
> So we just need to setup a table for boot cpu earlier.  From 
> the looks of it mach-es700 won't work if you compile a 
> uniprocessor kernel for it right now.

Yea, I didn't even try this - but I think it will produce the same
result with regard to timer IOAPIC rte.

> We need to do this a little later than in mptable but this 
> should be a fairly simple one or two line change.

Yes, it is maybe something like running map_cpu_to_logical_apicid() from
APIC_init() just before the setup_IO_APIC().
Thanks,
--Natalie

> If people keep breaking the subarchitectures by accident we 
> might even inspire someone to make a comprehensible sub 
> architecture implentation on x86 one of these days.
> 
> 
> Eric
> 

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Fastboot] [PATCH] i386: move apic init in init_IRQs
  2005-10-24 13:03         ` Vivek Goyal
  2005-10-24 15:36           ` Eric W. Biederman
@ 2005-10-25  7:17           ` Eric W. Biederman
  1 sibling, 0 replies; 16+ messages in thread
From: Eric W. Biederman @ 2005-10-25  7:17 UTC (permalink / raw)
  To: vgoyal; +Cc: Andrew Morton, fastboot, linux-kernel

Vivek Goyal <vgoyal@in.ibm.com> writes:
> I have attached a patch with the mail which is now using
> boot_cpu_physical_apicid to hard set presence of boot cpu instead of
> hard_smp_processor_id(). But the interesting questoin remains why BIOS is
> not reporting the boot cpu.


Ok.  I don't know if we care but I do know why we were not seeing
the report from the bios about your boot processor.  We record
information about cpus for up to NR_CPUS, and since you had
a UP kernel NR_CPUS was one.

>From your earlier boot log.

> ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled)
> Processor #3 6:10 APIC version 17
> ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> Processor #0 6:10 APIC version 17
> WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
> ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
> Processor #1 6:10 APIC version 17
> WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
> ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled)
> Processor #2 6:10 APIC version 17
> WARNING: NR_CPUS limit of 1 reached.  Processor ignored.

So it looks like we have this problem completely fixed.  

I don't see a good way to ensure that we always record our boot
apicid when we boot a multiple processor system and only use one
processor.

Eric




^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Fastboot] [PATCH] i386: move apic init in init_IRQs
  2005-10-24 13:03         ` Vivek Goyal
@ 2005-10-24 15:36           ` Eric W. Biederman
  2005-10-25  7:17           ` Eric W. Biederman
  1 sibling, 0 replies; 16+ messages in thread
From: Eric W. Biederman @ 2005-10-24 15:36 UTC (permalink / raw)
  To: vgoyal; +Cc: Andrew Morton, fastboot, linux-kernel

Vivek Goyal <vgoyal@in.ibm.com> writes:

> You are right. hard_smp_processor_id() is hard-coded to zero in case of a
> non SMP kernel (include/linux/smp.h) and that's why the problem is happening.
> I am booting a non-SMP capture kernel. In case of kexec on panic, we can very
> well boot on a cpu whose id is not zero.
>
> I have attached a patch with the mail which is now using
> boot_cpu_physical_apicid to hard set presence of boot cpu instead of
> hard_smp_processor_id(). But the interesting questoin remains why BIOS is
> not reporting the boot cpu.

Ok this looks good.  But it raises a couple of followup questions.
- Are there other places that use hard_smp_processor_id in 
  in a uniprocessor kernel?
- Does x86_64 have this same problem?

Anyway it looks like we have this working which is a big step forward
in having a reliable kdump mechanism.

Eric

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Fastboot] [PATCH] i386: move apic init in init_IRQs
  2005-10-22 15:23       ` Eric W. Biederman
@ 2005-10-24 13:03         ` Vivek Goyal
  2005-10-24 15:36           ` Eric W. Biederman
  2005-10-25  7:17           ` Eric W. Biederman
  0 siblings, 2 replies; 16+ messages in thread
From: Vivek Goyal @ 2005-10-24 13:03 UTC (permalink / raw)
  To: Eric W. Biederman; +Cc: Andrew Morton, fastboot, linux-kernel

On Sat, Oct 22, 2005 at 09:23:34AM -0600, Eric W. Biederman wrote:
> Vivek Goyal <vgoyal@in.ibm.com> writes:

[..]

> 
> 
> >> apic_id_registered expands to:
> >> static inline int apic_id_registered(void)
> >> {
> >> return physid_isset(GET_APIC_ID(apic_read(APIC_ID)), phys_cpu_present_map);
> >> }
> >> 
> >> Which indicates to me that the code that, there is something
> >> wrong in the logic of:
> >> 	if (!check_phys_apicid_present(boot_cpu_physical_apicid)) {
> >> 		printk("weird, boot CPU (#%d) not listed by the BIOS.\n",
> >> 				boot_cpu_physical_apicid);
> >> 		physid_set(hard_smp_processor_id(), phys_cpu_present_map);
> >> 	}
> >> 
> >> Currently we are refering to the boot cpus apicid with 3 different expressions
> >> one of them appears to be wrong.
> >> 
> >
> > Looks like apic_id_registered() is failing. I had put two debug printk()
> > statements and to my surprise hard_smp_processor_id() is returning different
> > value then GET_APIC_ID(apic_read(APIC_ID)).
> >
> > source code of hard_smp_processor_id() shows that it is also reading APIC_ID
> > register only. Then how can two values be different. (Until and unless
> > somebody modified the value in between two reads).
> 
> It appears the buggy expression is hard_smp_processor_id.  Quite
> possibly because it doesn't call apic_read() and instead open codes
> it.
> 
> boot_cpu_physical_apicid also returns apicid #1, before we have
> a problem.
> 
> So either we want to change hard_smp_processor_id to use apic_read()
> or we can just use boot_cpu_physical_apicid when fixing the apicid present
> bitmap.
> 
> > I am pasting another failure log with my debug messages(prefixed with "Debug:").
> > My debug patch is also attached with the mail.
> 
> See above but I am pretty certain we know enough to get farther.  For
> testing you may want to hard code your first kernel to use the second
> cpu.
> 
> The fact that hard_smp_processor_id gets the wrong value makes me wonder
> if your kernel will boot all of the way once we get past this problem.
> 
> I suspect if you disassemble the code for hard_smp_processor_id we
> will see the compiler doing the wrong thing.


You are right. hard_smp_processor_id() is hard-coded to zero in case of a
non SMP kernel (include/linux/smp.h) and that's why the problem is happening.
I am booting a non-SMP capture kernel. In case of kexec on panic, we can very
well boot on a cpu whose id is not zero.

I have attached a patch with the mail which is now using
boot_cpu_physical_apicid to hard set presence of boot cpu instead of
hard_smp_processor_id(). But the interesting questoin remains why BIOS is
not reporting the boot cpu.

Thanks
Vivek


o Removes the unnecessary call to local_irq_disable().

o Kdump was failing while second kernel was coming up. Check for presence
  of boot cpu apic id was failing in (apic_id_registered), hence hitting
  BUG().

o This should not have failed because before calling setup_local_APIC(), it is
  ensured that even if BIOS has not reported boot cpu, then hard set the
  prence of it. Problem happens because of usage of hard_smp_processor_id()
  which is hardcoded to zero in case of non SMP kernel. In kdump case second
  kernel can boot on a cpu whose boot cpu id is not zero. 

o Using boot_cpu_physical_apicid instead to hard set the presence of boot cpu.

Signed-off-by: Vivek Goyal <vgoyal@in.ibm.com>
---

 linux-2.6.14-rc4-mm1-16M-root/arch/i386/kernel/apic.c |    3 +--
 1 files changed, 1 insertion(+), 2 deletions(-)

diff -puN arch/i386/kernel/apic.c~kdump-i386-apic-verification-failure-fix arch/i386/kernel/apic.c
--- linux-2.6.14-rc4-mm1-16M/arch/i386/kernel/apic.c~kdump-i386-apic-verification-failure-fix	2005-10-24 17:40:08.000000000 +0530
+++ linux-2.6.14-rc4-mm1-16M-root/arch/i386/kernel/apic.c	2005-10-24 18:19:53.000000000 +0530
@@ -1055,7 +1055,6 @@ void __init setup_boot_APIC_clock(void)
 	using_apic_timer = 1;
 
 	local_irq_save(flags);
-	local_irq_disable();
 
 	calibration_result = calibrate_APIC_clock();
 	/*
@@ -1299,7 +1298,7 @@ int __init APIC_init(void)
 	if (!check_phys_apicid_present(boot_cpu_physical_apicid)) {
 		printk("weird, boot CPU (#%d) not listed by the BIOS.\n",
 				boot_cpu_physical_apicid);
-		physid_set(hard_smp_processor_id(), phys_cpu_present_map);
+		physid_set(boot_cpu_physical_apicid, phys_cpu_present_map);
 	}
 
 	/*
_


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Fastboot] [PATCH] i386: move apic init in init_IRQs
  2005-10-22 14:52     ` Vivek Goyal
@ 2005-10-22 15:23       ` Eric W. Biederman
  2005-10-24 13:03         ` Vivek Goyal
  0 siblings, 1 reply; 16+ messages in thread
From: Eric W. Biederman @ 2005-10-22 15:23 UTC (permalink / raw)
  To: vgoyal; +Cc: Andrew Morton, fastboot, linux-kernel

Vivek Goyal <vgoyal@in.ibm.com> writes:

> On Fri, Oct 21, 2005 at 08:45:12AM -0600, Eric W. Biederman wrote:
>> Vivek Goyal <vgoyal@in.ibm.com> writes:
>> 
>
> [..]
>
>> >> +	/*
>> >> +	 * Should not be necessary because the MP table should list the boot
>> >> +	 * CPU too, but we do it for the sake of robustness anyway.
>> >> +	 * Makes no sense to do this check in clustered apic mode, so skip it
>> >> +	 */
>> >> +	if (!check_phys_apicid_present(boot_cpu_physical_apicid)) {
>> >> +		printk("weird, boot CPU (#%d) not listed by the BIOS.\n",
>> >> +				boot_cpu_physical_apicid);
>> >
>> >
>> > I am testing kdump on i386 and I am hitting this message while second kernel
>> > is booting. I am doing testing with 2.6.14-rc4-mm1. Logs are pasted below.
>> 
>> The check has been there for a while.  All it is saying is that
>> our boot cpu has apicid #1.   So I suspect you are either on
>> an Opteron system or a hyperthreaded Xeon system.
>> 
>
> I am using Pentium. No hyperthreading.

Weird.  I would have that the BIOS would have listed the second cpu...
It might be worth tracking down later why the message appears but
the real problem is that the map is not getting filled in.

>> > Also kdump testing fails almost 50% of the time on my machine with
>> > 2.6.14-rc4-mm1.  It works fine with 2.6.14-rc4 though.
>> 
>> Is the failure that happens 50% represented by the bootlog below?
>> 
>
> Yes. But this problem is not happening all the time. Now in 4 trials
> I got it once again. The message in all the failures remains the same. 

It seems to only happen when this all starts on the second cpu,
with apic id #1.  Which explains the randomness.


>> apic_id_registered expands to:
>> static inline int apic_id_registered(void)
>> {
>> return physid_isset(GET_APIC_ID(apic_read(APIC_ID)), phys_cpu_present_map);
>> }
>> 
>> Which indicates to me that the code that, there is something
>> wrong in the logic of:
>> 	if (!check_phys_apicid_present(boot_cpu_physical_apicid)) {
>> 		printk("weird, boot CPU (#%d) not listed by the BIOS.\n",
>> 				boot_cpu_physical_apicid);
>> 		physid_set(hard_smp_processor_id(), phys_cpu_present_map);
>> 	}
>> 
>> Currently we are refering to the boot cpus apicid with 3 different expressions
>> one of them appears to be wrong.
>> 
>
> Looks like apic_id_registered() is failing. I had put two debug printk()
> statements and to my surprise hard_smp_processor_id() is returning different
> value then GET_APIC_ID(apic_read(APIC_ID)).
>
> source code of hard_smp_processor_id() shows that it is also reading APIC_ID
> register only. Then how can two values be different. (Until and unless
> somebody modified the value in between two reads).

It appears the buggy expression is hard_smp_processor_id.  Quite
possibly because it doesn't call apic_read() and instead open codes
it.

boot_cpu_physical_apicid also returns apicid #1, before we have
a problem.

So either we want to change hard_smp_processor_id to use apic_read()
or we can just use boot_cpu_physical_apicid when fixing the apicid present
bitmap.

> I am pasting another failure log with my debug messages(prefixed with "Debug:").
> My debug patch is also attached with the mail.

See above but I am pretty certain we know enough to get farther.  For
testing you may want to hard code your first kernel to use the second
cpu.

The fact that hard_smp_processor_id gets the wrong value makes me wonder
if your kernel will boot all of the way once we get past this problem.

I suspect if you disassemble the code for hard_smp_processor_id we
will see the compiler doing the wrong thing.

Eric

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Fastboot] [PATCH] i386: move apic init in init_IRQs
  2005-10-21 14:45   ` Eric W. Biederman
  2005-10-21 16:53     ` Albert Herranz
@ 2005-10-22 14:52     ` Vivek Goyal
  2005-10-22 15:23       ` Eric W. Biederman
  1 sibling, 1 reply; 16+ messages in thread
From: Vivek Goyal @ 2005-10-22 14:52 UTC (permalink / raw)
  To: Eric W. Biederman; +Cc: Andrew Morton, fastboot, linux-kernel

On Fri, Oct 21, 2005 at 08:45:12AM -0600, Eric W. Biederman wrote:
> Vivek Goyal <vgoyal@in.ibm.com> writes:
> 

[..]

> >> +	/*
> >> +	 * Should not be necessary because the MP table should list the boot
> >> +	 * CPU too, but we do it for the sake of robustness anyway.
> >> +	 * Makes no sense to do this check in clustered apic mode, so skip it
> >> +	 */
> >> +	if (!check_phys_apicid_present(boot_cpu_physical_apicid)) {
> >> +		printk("weird, boot CPU (#%d) not listed by the BIOS.\n",
> >> +				boot_cpu_physical_apicid);
> >
> >
> > I am testing kdump on i386 and I am hitting this message while second kernel
> > is booting. I am doing testing with 2.6.14-rc4-mm1. Logs are pasted below.
> 
> The check has been there for a while.  All it is saying is that
> our boot cpu has apicid #1.   So I suspect you are either on
> an Opteron system or a hyperthreaded Xeon system.
> 

I am using Pentium. No hyperthreading.

> > Also kdump testing fails almost 50% of the time on my machine with
> > 2.6.14-rc4-mm1.  It works fine with 2.6.14-rc4 though.
> 
> Is the failure that happens 50% represented by the bootlog below?
> 

Yes. But this problem is not happening all the time. Now in 4 trials
I got it once again. The message in all the failures remains the same. 

 
> The problem bootlog appears to be a glitch in the handling
> of apicids on the boot cpu that the BIOS does not report to the
> kernel.
> 
> > Second kernel is unable to come up. earlyprintk on serial console showed
> > a kernel BUG in setup_local_APIC(). Details are included in the logs below.
> 
> > Second kernel boot log.
> 
> The BUG is weird.  I don't think apic.c even goes to line 1479.
> Unless the BUG is inline in one of the other functions called
> by setup_local_APIC() .
> 
> 	/*
> 	 * Double-check whether this APIC is really registered.
> 	 */
> 	if (!apic_id_registered())
> 		BUG();
> 
> 
> apic_id_registered expands to:
> static inline int apic_id_registered(void)
> {
> 	return physid_isset(GET_APIC_ID(apic_read(APIC_ID)), phys_cpu_present_map);
> }
> 
> Which indicates to me that the code that, there is something
> wrong in the logic of:
> 	if (!check_phys_apicid_present(boot_cpu_physical_apicid)) {
> 		printk("weird, boot CPU (#%d) not listed by the BIOS.\n",
> 				boot_cpu_physical_apicid);
> 		physid_set(hard_smp_processor_id(), phys_cpu_present_map);
> 	}
> 
> Currently we are refering to the boot cpus apicid with 3 different expressions
> one of them appears to be wrong.
> 

Looks like apic_id_registered() is failing. I had put two debug printk()
statements and to my surprise hard_smp_processor_id() is returning different
value then GET_APIC_ID(apic_read(APIC_ID)).

source code of hard_smp_processor_id() shows that it is also reading APIC_ID
register only. Then how can two values be different. (Until and unless
somebody modified the value in between two reads).

I am pasting another failure log with my debug messages(prefixed with "Debug:").
My debug patch is also attached with the mail.

Second kernel boot log
---------------------

I'm in purgatory
Linux version 2.6.14-rc4-mm1-16M (root@llm01.in.ibm.com) (gcc version 3.4.3 20041212 (Red Hat 3.4.3-9.EL4)) #2 PREEMPT Sat Oct 22 18:44:25 IST 2005
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000100 - 000000000009d000 (usable)
 BIOS-e820: 000000000009d000 - 00000000000a0000 (reserved)
 BIOS-e820: 0000000000100000 - 000000002fffa480 (usable)
 BIOS-e820: 000000002fffa480 - 0000000030000000 (ACPI data)
 BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
user-defined physical RAM map:
 user: 0000000000000000 - 00000000000a0000 (usable)
 user: 0000000001000000 - 000000000142d000 (usable)
 user: 00000000014cd400 - 0000000005000000 (usable)
0MB HIGHMEM available.
80MB LOWMEM available.
found SMP MP-table at 0009e140
early console enabled
DMI 2.1 present.
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled)
Processor #3 6:10 APIC version 17
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Processor #0 6:10 APIC version 17
WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
Processor #1 6:10 APIC version 17
WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled)
Processor #2 6:10 APIC version 17
WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
Enabling APIC mode:  Flat.  Using 2 I/O APICs
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at 10000000 (gap: 05000000:fb000000)
Built 1 zonelists
Initializing CPU#0
Kernel command line: ro root=/dev/sda7 rhgb console=ttyS0,38400 irqpoll init 3 earlyprintk=ttyS0,38400 memmap=exactmap memmap=640K@0K memmap=4276K@16384K memmap=60619K@21301K elfcorehdr=21300K
Misrouted IRQ fixup and polling support enabled
This may significantly impact system performance
weird, boot CPU (#1) not listed by the BIOS.
Debug:Harsetting cpu apic id 0 to be present
Debug: APIC id being queried is 1
------------[ cut here ]------------
kernel BUG at ÿÿÿÿ:1479!
invalid operand: 0000 [#1]
PREEMPT
last sysfs file:
Modules linked in:
CPU:    0
EIP:    0060:[<c1012b32>]    Not tainted VLI
EFLAGS: 00010046   (2.6.14-rc4-mm1-16M)
EIP is at setup_local_APIC+0x41/0x1a7
eax: 00000000   ebx: 00040011   ecx: 00000c5b   edx: c1344201
esi: 00000011   edi: c13a9800   ebp: 01445007   esp: c13b5fbc
ds: 007b   es: 007b   ss: 0068
Process swapper (pid: 0, threadinfo=c13b4000 task=c133faa0)
Stack: c12e8774 00000001 c101ac40 00000000 01429900 c13c1c49 c12e8ac0 00000000
       00000003 c13b66cf c12e5d7d c13eddc0 c133ba5c 00000078 c13b6342 c13eddc0
       c1000199
Call Trace:
 [<c101ac40>] printk+0x17/0x1b
 [<c13c1c49>] APIC_init+0x5a/0x10a
 [<c13b66cf>] start_kernel+0xb3/0x1cd
 [<c13b6342>] unknown_bootoption+0x0/0x1b6
Code: c1 c1 e8 18 0f b6 f3 83 e0 0f 89 44 24 04 e8 0f 81 00 00 a1 20 d0 ff ff c1 e8 18 83 e0 0f 0f a3 05 e0 03 3f c1 19 c0 85 c0 75 02 <0f> 0b c7 05 e0 d0 ff ff ff ff ff ff 8b 0d c4 03 3f c1 a1 d0 d0
 <0>Kernel panic - not syncing: Attempted to kill the idle task!


Debug Patch
----------


 linux-2.6.14-rc4-mm1-16M-root/arch/i386/kernel/apic.c                   |    2 ++
 linux-2.6.14-rc4-mm1-16M-root/include/asm-i386/mach-default/mach_apic.h |    1 +
 2 files changed, 3 insertions(+)

diff -puN arch/i386/kernel/apic.c~apic-debug arch/i386/kernel/apic.c
--- linux-2.6.14-rc4-mm1-16M/arch/i386/kernel/apic.c~apic-debug	2005-10-22 18:37:28.000000000 +0530
+++ linux-2.6.14-rc4-mm1-16M-root/arch/i386/kernel/apic.c	2005-10-22 18:42:50.000000000 +0530
@@ -1299,6 +1299,8 @@ int __init APIC_init(void)
 	if (!check_phys_apicid_present(boot_cpu_physical_apicid)) {
 		printk("weird, boot CPU (#%d) not listed by the BIOS.\n",
 				boot_cpu_physical_apicid);
+		printk("Debug:Harsetting cpu apic id %d to be present\n",
+				hard_smp_processor_id());
 		physid_set(hard_smp_processor_id(), phys_cpu_present_map);
 	}
 
diff -puN include/asm-i386/mach-default/mach_apic.h~apic-debug include/asm-i386/mach-default/mach_apic.h
--- linux-2.6.14-rc4-mm1-16M/include/asm-i386/mach-default/mach_apic.h~apic-debug	2005-10-22 18:38:42.000000000 +0530
+++ linux-2.6.14-rc4-mm1-16M-root/include/asm-i386/mach-default/mach_apic.h	2005-10-22 18:44:10.000000000 +0530
@@ -111,6 +111,7 @@ static inline int check_phys_apicid_pres
 
 static inline int apic_id_registered(void)
 {
+	printk("Debug: APIC id being queried is %d\n", GET_APIC_ID(apic_read(APIC_ID)));
 	return physid_isset(GET_APIC_ID(apic_read(APIC_ID)), phys_cpu_present_map);
 }
 
_

/proc/cpuinfo output
--------------------

[root@llm01 ~]# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 10
model name      : Pentium III (Cascades)
stepping        : 1
cpu MHz         : 699.365
cache size      : 1024 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse
bogomips        : 1400.68

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 10
model name      : Pentium III (Cascades)
stepping        : 1
cpu MHz         : 699.365
cache size      : 1024 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse
bogomips        : 1398.47

processor       : 2
vendor_id       : GenuineIntel
cpu family      : 6
model           : 10
model name      : Pentium III (Cascades)
stepping        : 1
cpu MHz         : 699.365
cache size      : 1024 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse
bogomips        : 1398.47

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 10
model name      : Pentium III (Cascades)
stepping        : 1
cpu MHz         : 699.365
cache size      : 1024 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse
bogomips        : 1398.48


Thanks
Vivek

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Fastboot] [PATCH] i386: move apic init in init_IRQs
  2005-10-21 16:53     ` Albert Herranz
@ 2005-10-21 18:01       ` Eric W. Biederman
  0 siblings, 0 replies; 16+ messages in thread
From: Eric W. Biederman @ 2005-10-21 18:01 UTC (permalink / raw)
  To: Albert Herranz; +Cc: vgoyal, Andrew Morton, fastboot, linux-kernel

Albert Herranz <albert_herranz@yahoo.es> writes:

>> > Should the local_irq_disable() call go away onece
>> local_irq_save() got
>> > introduced.
>> 
>> Nope.  The irqs need to be disabled.  The save just
>> allows this
>> to be called in a context where irqs start out
>> disabled.  It is
>> just a save.
>
> local_irq_save() also disables interrupts.

Bah.  I was thinking and reading local_save_flags()....

So yes that does make the local_irq_disable redundant.

I still used to seeing:
save_flags();
cli();
...
restore_flags();

Eric



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Fastboot] [PATCH] i386: move apic init in init_IRQs
  2005-10-21 14:45   ` Eric W. Biederman
@ 2005-10-21 16:53     ` Albert Herranz
  2005-10-21 18:01       ` Eric W. Biederman
  2005-10-22 14:52     ` Vivek Goyal
  1 sibling, 1 reply; 16+ messages in thread
From: Albert Herranz @ 2005-10-21 16:53 UTC (permalink / raw)
  To: Eric W. Biederman, vgoyal; +Cc: Andrew Morton, fastboot, linux-kernel

> > Should the local_irq_disable() call go away onece
> local_irq_save() got
> > introduced.
> 
> Nope.  The irqs need to be disabled.  The save just
> allows this
> to be called in a context where irqs start out
> disabled.  It is
> just a save.

local_irq_save() also disables interrupts.

Cheers,
Albert




		
______________________________________________ 
Renovamos el Correo Yahoo! 
Nuevos servicios, más seguridad 
http://correo.yahoo.es

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Fastboot] [PATCH] i386: move apic init in init_IRQs
  2005-10-21 13:33 ` [Fastboot] " Vivek Goyal
@ 2005-10-21 14:45   ` Eric W. Biederman
  2005-10-21 16:53     ` Albert Herranz
  2005-10-22 14:52     ` Vivek Goyal
  0 siblings, 2 replies; 16+ messages in thread
From: Eric W. Biederman @ 2005-10-21 14:45 UTC (permalink / raw)
  To: vgoyal; +Cc: Andrew Morton, fastboot, linux-kernel

Vivek Goyal <vgoyal@in.ibm.com> writes:

> Hi Eric,
>
> I had a couple of observations.
>
> [..]
>>  #ifdef CONFIG_X86_IO_APIC
>>  	{
>> @@ -1046,9 +1050,11 @@ static unsigned int calibration_result;
>>  
>>  void __init setup_boot_APIC_clock(void)
>>  {
>> +	unsigned long flags;
>>  	apic_printk(APIC_VERBOSE, "Using local APIC timer interrupts.\n");
>>  	using_apic_timer = 1;
>>  
>> +	local_irq_save(flags);
>>  	local_irq_disable();
>>  
>
> Should the local_irq_disable() call go away onece local_irq_save() got
> introduced.

Nope.  The irqs need to be disabled.  The save just allows this
to be called in a context where irqs start out disabled.  It is
just a save.

>> +	/*
>> +	 * Should not be necessary because the MP table should list the boot
>> +	 * CPU too, but we do it for the sake of robustness anyway.
>> +	 * Makes no sense to do this check in clustered apic mode, so skip it
>> +	 */
>> +	if (!check_phys_apicid_present(boot_cpu_physical_apicid)) {
>> +		printk("weird, boot CPU (#%d) not listed by the BIOS.\n",
>> +				boot_cpu_physical_apicid);
>
>
> I am testing kdump on i386 and I am hitting this message while second kernel
> is booting. I am doing testing with 2.6.14-rc4-mm1. Logs are pasted below.

The check has been there for a while.  All it is saying is that
our boot cpu has apicid #1.   So I suspect you are either on
an Opteron system or a hyperthreaded Xeon system.

> Also kdump testing fails almost 50% of the time on my machine with
> 2.6.14-rc4-mm1.  It works fine with 2.6.14-rc4 though.

Is the failure that happens 50% represented by the bootlog below?

The problem bootlog appears to be a glitch in the handling
of apicids on the boot cpu that the BIOS does not report to the
kernel.

> Second kernel is unable to come up. earlyprintk on serial console showed
> a kernel BUG in setup_local_APIC(). Details are included in the logs below.

> Second kernel boot log.

The BUG is weird.  I don't think apic.c even goes to line 1479.
Unless the BUG is inline in one of the other functions called
by setup_local_APIC() .

	/*
	 * Double-check whether this APIC is really registered.
	 */
	if (!apic_id_registered())
		BUG();


apic_id_registered expands to:
static inline int apic_id_registered(void)
{
	return physid_isset(GET_APIC_ID(apic_read(APIC_ID)), phys_cpu_present_map);
}

Which indicates to me that the code that, there is something
wrong in the logic of:
	if (!check_phys_apicid_present(boot_cpu_physical_apicid)) {
		printk("weird, boot CPU (#%d) not listed by the BIOS.\n",
				boot_cpu_physical_apicid);
		physid_set(hard_smp_processor_id(), phys_cpu_present_map);
	}

Currently we are refering to the boot cpus apicid with 3 different expressions
one of them appears to be wrong.

That is as far as I can get at the moment.

Eric


>
> # SysRq : Trigger a crashdump
> I'm in purgatory
> Linux version 2.6.14-rc4-mm1-16M (root@llm01.in.ibm.com) (gcc version 3.4.3
> 20041212 (Red Hat 3.4.3-9.EL4)) #1 PREEMPT Wed Oct 19 13:55:24 IST 2005
> BIOS-provided physical RAM map:
>  BIOS-e820: 0000000000000100 - 000000000009d000 (usable)
>  BIOS-e820: 000000000009d000 - 00000000000a0000 (reserved)
>  BIOS-e820: 0000000000100000 - 000000002fffa480 (usable)
>  BIOS-e820: 000000002fffa480 - 0000000030000000 (ACPI data)
>  BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
> user-defined physical RAM map:
>  user: 0000000000000000 - 00000000000a0000 (usable)
>  user: 0000000001000000 - 000000000142d000 (usable)
>  user: 00000000014cd400 - 0000000004000000 (usable)
> 0MB HIGHMEM available.
> 64MB LOWMEM available.
> found SMP MP-table at 0009e140
> early console enabled
> DMI 2.1 present.
> ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled)
> Processor #3 6:10 APIC version 17
> ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> Processor #0 6:10 APIC version 17
> WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
> ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
> Processor #1 6:10 APIC version 17
> WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
> ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled)
> Processor #2 6:10 APIC version 17
> WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
> ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
> IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
> ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
> IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
> ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> Enabling APIC mode:  Flat.  Using 2 I/O APICs
> Using ACPI (MADT) for SMP configuration information
> Allocating PCI resources starting at 10000000 (gap: 04000000:fc000000)
> Built 1 zonelists
> Initializing CPU#0
> Kernel command line: ro root=/dev/sda7 rhgb console=ttyS0,38400 irqpoll init 3
> earlyprintk=ttyS0,38400 memmap=exactmap memmap=640K@0K memmap=4276K@16384K
> memmap=44235K@21301K elfcorehdr=21300K
> Misrouted IRQ fixup and polling support enabled
> This may significantly impact system performance
> weird, boot CPU (#1) not listed by the BIOS.
> ------------[ cut here ]------------
> kernel BUG at ÿÿÿÿ:1479!
> invalid operand: 0000 [#1]
> PREEMPT
> last sysfs file:
> Modules linked in:
> CPU:    0
> EIP:    0060:[<c1012b17>]    Not tainted VLI
> EFLAGS: 00010046   (2.6.14-rc4-mm1-16M)
> EIP is at setup_local_APIC+0x26/0x18c
> eax: 00000000   ebx: 00040011   ecx: 00000c06   edx: 00000000
> esi: 00000011   edi: c13a9800   ebp: 01445007   esp: c13b5fbc
> ds: 007b   es: 007b   ss: 0068
> Process swapper (pid: 0, threadinfo=c13b4000 task=c133faa0)
> Stack: c13a9800 01445007 c101ac30 00000000 01429900 c13c1c49 c12e8a4c 00000001
>        00000003 c13b66cf c12e5d5d c13eddc0 c133b9fc 00000078 c13b6342 c13eddc0
>        c1000199
> Call Trace:
>  [<c101ac30>] printk+0x17/0x1b
>  [<c13c1c49>] APIC_init+0x5a/0xf6
>  [<c13b66cf>] start_kernel+0xb3/0x1cd
>  [<c13b6342>] unknown_bootoption+0x0/0x1b6
> Code: e4 f7 0f 30 c3 56 53 83 ec 0c 8b 1d 30 d0 ff ff a1 20 d0 ff ff c1 e8 18 0f
> b6 f3 83 e0 0f 0f a3 05 e0 03 3f c1 19 c0 85 c0 75 02 <0f> 0b c7 05 e0 d0 ff ff
> ff ff ff ff 8b 0d c4 03 3f c1 a1 d0 d0
>  <0>Kernel panic - not syncing: Attempted to kill the idle task!
>
>
> Thanks
> Vivek

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [Fastboot] [PATCH] i386: move apic init in init_IRQs
  2005-10-04 15:04 Eric W. Biederman
@ 2005-10-21 13:33 ` Vivek Goyal
  2005-10-21 14:45   ` Eric W. Biederman
  0 siblings, 1 reply; 16+ messages in thread
From: Vivek Goyal @ 2005-10-21 13:33 UTC (permalink / raw)
  To: Eric W. Biederman; +Cc: Andrew Morton, fastboot, linux-kernel

Hi Eric,

I had a couple of observations.

[..]
>  #ifdef CONFIG_X86_IO_APIC
>  	{
> @@ -1046,9 +1050,11 @@ static unsigned int calibration_result;
>  
>  void __init setup_boot_APIC_clock(void)
>  {
> +	unsigned long flags;
>  	apic_printk(APIC_VERBOSE, "Using local APIC timer interrupts.\n");
>  	using_apic_timer = 1;
>  
> +	local_irq_save(flags);
>  	local_irq_disable();
>  

Should the local_irq_disable() call go away onece local_irq_save() got
introduced.


>  	calibration_result = calibrate_APIC_clock();
> @@ -1057,7 +1063,7 @@ void __init setup_boot_APIC_clock(void)
>  	 */
>  	setup_APIC_timer(calibration_result);
>  
> -	local_irq_enable();
> +	local_irq_restore(flags);
>  }
>  

[..]
>  
>  	verify_local_APIC();
>  
> +	/*
> +	 * Should not be necessary because the MP table should list the boot
> +	 * CPU too, but we do it for the sake of robustness anyway.
> +	 * Makes no sense to do this check in clustered apic mode, so skip it
> +	 */
> +	if (!check_phys_apicid_present(boot_cpu_physical_apicid)) {
> +		printk("weird, boot CPU (#%d) not listed by the BIOS.\n",
> +				boot_cpu_physical_apicid);


I am testing kdump on i386 and I am hitting this message while second kernel
is booting. I am doing testing with 2.6.14-rc4-mm1. Logs are pasted below.

Also kdump testing fails almost 50% of the time on my machine with
2.6.14-rc4-mm1.  It works fine with 2.6.14-rc4 though.

Second kernel is unable to come up. earlyprintk on serial console showed
a kernel BUG in setup_local_APIC(). Details are included in the logs below.

Second kernel boot log.

# SysRq : Trigger a crashdump
I'm in purgatory
Linux version 2.6.14-rc4-mm1-16M (root@llm01.in.ibm.com) (gcc version 3.4.3 20041212 (Red Hat 3.4.3-9.EL4)) #1 PREEMPT Wed Oct 19 13:55:24 IST 2005
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000100 - 000000000009d000 (usable)
 BIOS-e820: 000000000009d000 - 00000000000a0000 (reserved)
 BIOS-e820: 0000000000100000 - 000000002fffa480 (usable)
 BIOS-e820: 000000002fffa480 - 0000000030000000 (ACPI data)
 BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
user-defined physical RAM map:
 user: 0000000000000000 - 00000000000a0000 (usable)
 user: 0000000001000000 - 000000000142d000 (usable)
 user: 00000000014cd400 - 0000000004000000 (usable)
0MB HIGHMEM available.
64MB LOWMEM available.
found SMP MP-table at 0009e140
early console enabled
DMI 2.1 present.
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x03] enabled)
Processor #3 6:10 APIC version 17
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Processor #0 6:10 APIC version 17
WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
Processor #1 6:10 APIC version 17
WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled)
Processor #2 6:10 APIC version 17
WARNING: NR_CPUS limit of 1 reached.  Processor ignored.
ACPI: IOAPIC (id[0x0e] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 14, version 17, address 0xfec00000, GSI 0-15
ACPI: IOAPIC (id[0x0d] address[0xfec01000] gsi_base[16])
IOAPIC[1]: apic_id 13, version 17, address 0xfec01000, GSI 16-31
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
Enabling APIC mode:  Flat.  Using 2 I/O APICs
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at 10000000 (gap: 04000000:fc000000)
Built 1 zonelists
Initializing CPU#0
Kernel command line: ro root=/dev/sda7 rhgb console=ttyS0,38400 irqpoll init 3 earlyprintk=ttyS0,38400 memmap=exactmap memmap=640K@0K memmap=4276K@16384K memmap=44235K@21301K elfcorehdr=21300K
Misrouted IRQ fixup and polling support enabled
This may significantly impact system performance
weird, boot CPU (#1) not listed by the BIOS.
------------[ cut here ]------------
kernel BUG at ÿÿÿÿ:1479!
invalid operand: 0000 [#1]
PREEMPT
last sysfs file:
Modules linked in:
CPU:    0
EIP:    0060:[<c1012b17>]    Not tainted VLI
EFLAGS: 00010046   (2.6.14-rc4-mm1-16M)
EIP is at setup_local_APIC+0x26/0x18c
eax: 00000000   ebx: 00040011   ecx: 00000c06   edx: 00000000
esi: 00000011   edi: c13a9800   ebp: 01445007   esp: c13b5fbc
ds: 007b   es: 007b   ss: 0068
Process swapper (pid: 0, threadinfo=c13b4000 task=c133faa0)
Stack: c13a9800 01445007 c101ac30 00000000 01429900 c13c1c49 c12e8a4c 00000001
       00000003 c13b66cf c12e5d5d c13eddc0 c133b9fc 00000078 c13b6342 c13eddc0
       c1000199
Call Trace:
 [<c101ac30>] printk+0x17/0x1b
 [<c13c1c49>] APIC_init+0x5a/0xf6
 [<c13b66cf>] start_kernel+0xb3/0x1cd
 [<c13b6342>] unknown_bootoption+0x0/0x1b6
Code: e4 f7 0f 30 c3 56 53 83 ec 0c 8b 1d 30 d0 ff ff a1 20 d0 ff ff c1 e8 18 0f b6 f3 83 e0 0f 0f a3 05 e0 03 3f c1 19 c0 85 c0 75 02 <0f> 0b c7 05 e0 d0 ff ff ff ff ff ff 8b 0d c4 03 3f c1 a1 d0 d0
 <0>Kernel panic - not syncing: Attempted to kill the idle task!


Thanks
Vivek

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2005-11-01  7:42 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2005-10-31 17:04 [Fastboot] [PATCH] i386: move apic init in init_IRQs Protasevich, Natalie
2005-10-31 17:18 ` Zwane Mwaikambo
2005-10-31 17:30   ` Eric W. Biederman
2005-10-31 20:26     ` Zwane Mwaikambo
2005-10-31 18:18 ` Eric W. Biederman
  -- strict thread matches above, loose matches on Subject: below --
2005-10-31 18:31 Protasevich, Natalie
2005-11-01  7:41 ` Eric W. Biederman
2005-10-04 15:04 Eric W. Biederman
2005-10-21 13:33 ` [Fastboot] " Vivek Goyal
2005-10-21 14:45   ` Eric W. Biederman
2005-10-21 16:53     ` Albert Herranz
2005-10-21 18:01       ` Eric W. Biederman
2005-10-22 14:52     ` Vivek Goyal
2005-10-22 15:23       ` Eric W. Biederman
2005-10-24 13:03         ` Vivek Goyal
2005-10-24 15:36           ` Eric W. Biederman
2005-10-25  7:17           ` Eric W. Biederman

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®