From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1752581AbcADCYn (ORCPT <rfc822;w@1wt.eu>);
	Sun, 3 Jan 2016 21:24:43 -0500
Received: from szxga03-in.huawei.com ([119.145.14.66]:42050 "EHLO
	szxga03-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1751536AbcADCYk (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Sun, 3 Jan 2016 21:24:40 -0500
Subject: Re: [PATCH] cpuset: fix cpus_allowed mask for offline/online CPUs
To: "Chen, Yu C" <yu.c.chen@intel.com>,
        "cgroups@vger.kernel.org" <cgroups@vger.kernel.org>
References: <2f805346f87680dddfa9253dc9031db3b6ead8ad.1451648318.git.yu.c.chen@intel.com>
 <5689D049.4040500@huawei.com>
 <36DF59CE26D8EE47B0655C516E9CE640286AC772@shsmsx102.ccr.corp.intel.com>
CC: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Vlastimil Babka" <vbabka@suse.cz>, Rik van Riel <riel@redhat.com>,
        Joonsoo Kim <iamjoonsoo.kim@lge.com>,
        David Rientjes <rientjes@google.com>,
        "Vishnu Pratap Singh" <vishnu.ps@samsung.com>,
        Pintu Kumar <pintu.k@samsung.com>,
        "Michal Nazarewicz" <mina86@mina86.com>, Mel Gorman <mgorman@suse.de>,
        "Gortmaker, Paul (Wind River)" <paul.gortmaker@windriver.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Tim Chen <tim.c.chen@linux.intel.com>, Hugh Dickins <hughd@google.com>,
        Tejun Heo <tj@kernel.org>
From: Zefan Li <lizefan@huawei.com>
Message-ID: <5689D6F3.2090007@huawei.com>
Date: Mon, 4 Jan 2016 10:20:35 +0800
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101
 Thunderbird/38.1.0
MIME-Version: 1.0
In-Reply-To: <36DF59CE26D8EE47B0655C516E9CE640286AC772@shsmsx102.ccr.corp.intel.com>
Content-Type: text/plain; charset="windows-1252"; format=flowed
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.177.19.236]
X-CFilter-Loop: Reflected
X-Mirapoint-Virus-RAPID-Raw: score=unknown(0),
	refid=str=0001.0A020203.5689D714.000E,ss=1,re=0.000,recu=0.000,reip=0.000,cl=1,cld=1,fgs=0,
	ip=0.0.0.0,
	so=2013-05-26 15:14:31,
	dmn=2013-03-21 17:37:32
X-Mirapoint-Loop-Id: 1ba69feb647c3f48cf6df360892c2742
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

>> On 2016/1/1 20:09, Chen Yu wrote:
>>> Commit be4c9dd7aee5 ("cpuset: enable onlined cpu/node in effective
>>> masks") leverages cpuset's cpus_allowed and its parent's
>>> effective_cpus to calculate the new_cpus by:
>>>
>>> cpumask_and(&new_cpus, cs->cpus_allowed,
>>> parent_cs(cs)->effective_cpus);
>>>
>>> However cpus_allowed will also be updated after the CPU is offline, in
>>> hotplug_update_tasks_legacy, so when the CPU is online again, it will
>>> use the old cpus_allowed mask to calculate the new_cpus, thus new_cpus
>>> will get incorrect value after each round of offline/online.
>>>
>>> This problem is found on ubuntu 15.10 with cpuset mounted:
>>>
>>> 1. echo 0 > /sys/devices/system/cpu/cpu2/online
>>> 2. echo 1 > /sys/devices/system/cpu/cpu2/online
>>> 3. cat /sys/fs/cgroup/cpuset/cpuset.cpus
>>>      0-3
>>> 4. cat /sys/fs/cgroup/cpuset/user.slice/cpuset.cpus
>>>      0-1,3
>>> 5. taskset -c 2 ls
>>>      taskset: failed to set pid 0's affinity: Invalid argument
>>>
>>
>> This is the expected behavior...In legacy hierachy onlining an offlined cpu
>> won't restore cpuset configurations automatically.
> [Yu] Ah, I see, we were just a little confused with it before.
>   So if we want to online an offlined cpu in legacy hierarchy, we
> should not only echo 1 to 'online' sysfs, but also restore the mask manually?

Right. You have to take care of this by yourself.

> It seems that we should not rely on /sys/devices/system/cpu/cpu2/online,
> if cpuset has been mounted.