From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760680AbYFZNKv (ORCPT ); Thu, 26 Jun 2008 09:10:51 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1761359AbYFZM5S (ORCPT ); Thu, 26 Jun 2008 08:57:18 -0400 Received: from e28smtp06.in.ibm.com ([59.145.155.6]:45227 "EHLO e28esmtp06.in.ibm.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1762750AbYFZM5Q (ORCPT ); Thu, 26 Jun 2008 08:57:16 -0400 Date: Thu, 26 Jun 2008 18:28:20 +0530 From: Gautham R Shenoy To: Rusty Russell Cc: "Zhang, Yanmin" , Mike Travis , Vegard Nossum , Adrian Bunk , Srivatsa Vaddagiri , linux-kernel@vger.kernel.org, "Rafael J. Wysocki" , "Zhang, Yanmin" , Heiko Carstens Subject: Re: v2.6.26-rc7: BUG: unable to handle kernel NULL pointer dereference Message-ID: <20080626125820.GA30144@in.ibm.com> Reply-To: ego@in.ibm.com References: <20080622125633.GA8166@damson.getinternet.no> <200806241136.52430.rusty@rustcorp.com.au> <1214294783.25608.75.camel@ymzhang> <200806242314.51656.rusty@rustcorp.com.au> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <200806242314.51656.rusty@rustcorp.com.au> User-Agent: Mutt/1.5.17 (2007-11-01) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jun 24, 2008 at 11:14:51PM +1000, Rusty Russell wrote: > On Tuesday 24 June 2008 18:06:23 Zhang, Yanmin wrote: > > On Tue, 2008-06-24 at 11:36 +1000, Rusty Russell wrote: > > > On Tuesday 24 June 2008 02:58:44 Mike Travis wrote: > > > > Rusty Russell wrote: > > > > > On Monday 23 June 2008 02:29:07 Vegard Nossum wrote: > > > > >> And the (cpu < nr_cpu_ids) fails because the CPU has just been > > > > >> offlined (or failed to initialize, but it's the same thing), while > > > > >> NR_CPUS is the value that was compiled in as CONFIG_NR_CPUS (so the > > > > >> former check will always be true). > > > > >> > > > > >> I don't think it is valid to ask for a per_cpu() variable on a CPU > > > > >> which does not exist, though > > > > > > > > > > Yes it is. As long as cpu_possible(cpu), per_cpu(cpu) is valid. > > > > > > > > > > The number check should be removed: checking cpu_possible() is > > > > > sufficient. > > > > > > > > > > Hope that helps, > > > > > Rusty. > > > > > > > > I don't see a check for index being out of range in cpu_possible(). > > > > > > You're right. It assumes cpu is < NR_CPUS. Hmm, I have no idea what's > > > going on. nr_cpu_ids (ignore that it's a horrible name for a bad idea) > > > should be fine to test against. > > > > > > Vegard's analysis is flawed: just because cpu is offline, it still must > > > be < nr_cpu_ids, which is based on possible cpus. Unless something crazy > > > is happening, but a quick grep doesn't reveal anyone manipulating > > > nr_cpu_ids. > > > > > > If changing this fixes the bug, something else is badly wrong... > > > Rusty. > > > > In function _cpu_up, the panic happens when calling > > __raw_notifier_call_chain at the second time. Kernel doesn't panic when > > calling it at the first time. If just say because of nr_cpu_ids, that's > > not right. > > > > By checking source codes, I find function do_boot_cpu is the culprit. > > Consider below call chain: > > _cpu_up=>__cpu_up=>smp_ops.cpu_up=>native_cpu_up=>do_boot_cpu. > > > > So do_boot_cpu is called in the end. In do_boot_cpu, if boot_error==true, > > cpu_clear(cpu, cpu_possible_map) is executed. So later on, when _cpu_up > > calls __raw_notifier_call_chain at the second time to report > > CPU_UP_CANCELED, because this cpu is already cleared from > > cpu_possible_map, get_cpu_sysdev returns NULL. > > > > Many resources are related to cpu_possible_map, so it's better not to > > change it. > > > > Below patch against 2.6.26-rc7 fixes it by removing the bit clearing in > > cpu_possible_map. > > > > Vegard, would you like to help test it? > > > > Signed-off-by: Zhang Yanmin  > > > > --- > > > > diff -Nraup linux-2.6.26-rc7/arch/x86/kernel/smpboot.c > > linux-2.6.26-rc7_cpuhotplug/arch/x86/kernel/smpboot.c --- > > linux-2.6.26-rc7/arch/x86/kernel/smpboot.c 2008-06-24 09:03:54.000000000 > > +0800 +++ linux-2.6.26-rc7_cpuhotplug/arch/x86/kernel/smpboot.c 2008-06-24 > > 09:04:45.000000000 +0800 @@ -996,7 +996,6 @@ do_rest: > > #endif > > cpu_clear(cpu, cpu_callout_map); /* was set by do_boot_cpu() */ > > cpu_clear(cpu, cpu_initialized); /* was set by cpu_init() */ > > - cpu_clear(cpu, cpu_possible_map); > > cpu_clear(cpu, cpu_present_map); Nice catch. While we're at it, is the clearing of cpu from the cpu_present_map necessary if cpu_up failed for 'cpu' ? > > per_cpu(x86_cpu_to_apicid, cpu) = BAD_APICID; > > } > > Nice catch. Basically, cpu_possible_map should only be cleared at boot, and > probably not even then. > > Acked-by: Rusty Russell > > Thanks, > Rusty. -- Thanks and Regards gautham