From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754670AbcILEFz (ORCPT ); Mon, 12 Sep 2016 00:05:55 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:52323 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751227AbcILEFx (ORCPT ); Mon, 12 Sep 2016 00:05:53 -0400 DMARC-Filter: OpenDMARC Filter v1.3.1 smtp.codeaurora.org 3E40F61EA2 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=pass smtp.mailfrom=joonwoop@codeaurora.org Date: Sun, 11 Sep 2016 21:05:47 -0700 From: Joonwoo Park To: Zefan Li Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Tejun Heo Subject: Re: [PATCH] cpuset: handle race between CPU hotplug and cpuset_hotplug_work Message-ID: <20160912040547.k3qtivdcet6p4mel@codeaurora.org> References: <1473381684-9537-1-git-send-email-joonwoop@codeaurora.org> <57D6177F.8000904@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <57D6177F.8000904@huawei.com> User-Agent: Mutt/1.6.1-neo (2016-06-11) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Sep 12, 2016 at 10:48:31AM +0800, Zefan Li wrote: > Cc: Tejun > > On 2016/9/9 8:41, Joonwoo Park wrote: > > Discrepancy between cpu_online_mask and cpuset's effective CPU masks on > > cpuset hierarchy is inevitable since cpuset defers updating of > > effective CPU masks with workqueue while nothing prevents system from > > doing CPU hotplug. For that reason guarantee_online_cpus() walks up > > the cpuset hierarchy until it finds intersection under the assumption > > that top cpuset's effective CPU mask intersects with cpu_online_mask > > even under such race. > > > > However a sequence of CPU hotplugs can open a time window which is none > > of effective CPUs in the top cpuset intersects with cpu_online_mask. > > > > For example when there are 4 possible CPUs 0-3 where only CPU0 is online: > > > > ======================== =========================== > > cpu_online_mask top_cpuset.effective_cpus > > ======================== =========================== > > echo 1 > cpu2/online. > > CPU hotplug notifier woke up hotplug work but not yet scheduled. > > [0,2] [0] > > > > echo 0 > cpu0/online. > > The workqueue is still runnable. > > [2] [0] > > ======================== =========================== > > > > Now there is no intersection between cpu_online_mask and > > top_cpuset.effective_cpus. Thus invoking sys_sched_setaffinity() at > > this moment can cause following: > > > > Unable to handle kernel NULL pointer dereference at virtual address 000000d0 > > ------------[ cut here ]------------ > > Kernel BUG at ffffffc0001389b0 [verbose debug info unavailable] > > Internal error: Oops - BUG: 96000005 [#1] PREEMPT SMP > > Modules linked in: > > CPU: 2 PID: 1420 Comm: taskset Tainted: G W 4.4.8+ #98 > > task: ffffffc06a5c4880 ti: ffffffc06e124000 task.ti: ffffffc06e124000 > > PC is at guarantee_online_cpus+0x2c/0x58 > > LR is at cpuset_cpus_allowed+0x4c/0x6c > > > > Process taskset (pid: 1420, stack limit = 0xffffffc06e124020) > > Call trace: > > [] guarantee_online_cpus+0x2c/0x58 > > [] cpuset_cpus_allowed+0x4c/0x6c > > [] sched_setaffinity+0xc0/0x1ac > > [] SyS_sched_setaffinity+0x98/0xac > > [] el0_svc_naked+0x24/0x28 > > > > The top cpuset's effective_cpus are guaranteed to be identical to online > > CPUs eventually. Hence fall back to online CPU mask when there is no > > intersection between top cpuset's effective_cpus and online CPU mask. > > > > Signed-off-by: Joonwoo Park > > Cc: Li Zefan > > Cc: cgroups@vger.kernel.org > > Cc: linux-kernel@vger.kernel.org > > Thanks for fixing this! > > Acked-by: Zefan Li > Cc: # 3.17+ > Thanks for reviewing. Shortly I will send v2 which has few grammar error fixes in the changelog. No code change has made. Joonwoo