From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2D97C04A68 for ; Thu, 28 Jul 2022 18:57:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230471AbiG1S5j (ORCPT ); Thu, 28 Jul 2022 14:57:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51724 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229456AbiG1S5h (ORCPT ); Thu, 28 Jul 2022 14:57:37 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 452EB18B31 for ; Thu, 28 Jul 2022 11:57:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1659034655; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fjIYWUhlmn952PQamHyhfurxBgzciXNRkjegJKiOJW4=; b=PcCi3gU55ywwQ9YonL0ILXdb0EOs5LWocH3kKjiDfvUvvLEN02PCPfshU72Ex0TugBiit8 2z5bCNICnrybAASirJGMfJahvnq46AQrz059TAN+b50PXecOr+BhqtLn5jWRhQucdGTKUx 6N41JdkvgzLpwC4nzOkmujBAFB+53h8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-647-sdLg74s2Nye9Wp21UThKiA-1; Thu, 28 Jul 2022 14:57:31 -0400 X-MC-Unique: sdLg74s2Nye9Wp21UThKiA-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4515E102BCDB; Thu, 28 Jul 2022 18:57:30 +0000 (UTC) Received: from [10.22.9.86] (unknown [10.22.9.86]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4B68040315E; Thu, 28 Jul 2022 18:57:29 +0000 (UTC) Message-ID: <1ae1cc6c-dca9-4958-6b22-24a5777c5e8d@redhat.com> Date: Thu, 28 Jul 2022 14:57:28 -0400 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0 Subject: Re: [PATCH 1/2] cgroup/cpuset: Keep current cpus list if cpus affinity was explicitly set Content-Language: en-US To: Tejun Heo Cc: Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , Valentin Schneider , Zefan Li , Johannes Weiner , cgroups@vger.kernel.org, linux-kernel@vger.kernel.org References: <20220728005815.1715522-1-longman@redhat.com> From: Waiman Long In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 7/28/22 13:23, Tejun Heo wrote: > Hello, > > On Wed, Jul 27, 2022 at 08:58:14PM -0400, Waiman Long wrote: >> It was found that any change to the current cpuset hierarchy may reset >> the cpus_allowed list of the tasks in the affected cpusets to the >> default cpuset value even if those tasks have cpus affinity explicitly >> set by the users before. That is especially easy to trigger under a >> cgroup v2 environment where writing "+cpuset" to the root cgroup's >> cgroup.subtree_control file will reset the cpus affinity of all the >> processes in the system. >> >> That is especially problematic in a nohz_full environment where the >> tasks running in the nohz_full CPUs usually have their cpus affinity >> explicitly set and will behave incorrectly if cpus affinity changes. >> >> Fix this problem by adding a flag in the task structure to indicate that >> a task has their cpus affinity explicitly set before and make cpuset >> code not to change their cpus_allowed list unless the user chosen cpu >> list is no longer a subset of the cpus_allowed list of the cpuset itself. >> >> With that change in place, it was verified that tasks that have its >> cpus affinity explicitly set will not be affected by changes made to >> the v2 cgroup.subtree_control files. > I think the underlying cause here is cpuset overwriting the cpumask the user > configured but that's a longer discussion. > >> +/* >> + * Don't change the cpus_allowed list if cpus affinity has been explicitly >> + * set before unless the current cpu list is not a subset of the new cpu list. >> + */ >> +static int cpuset_set_cpus_allowed_ptr(struct task_struct *p, >> + const struct cpumask *new_mask) >> +{ >> + if (p->cpus_affinity_set && cpumask_subset(p->cpus_ptr, new_mask)) >> + return 0; >> + >> + p->cpus_affinity_set = 0; >> + return set_cpus_allowed_ptr(p, new_mask); >> +} > I wonder whether the more predictable behavior would be always not resetting > the cpumask if it's a subset of the new_mask. There can be a counter argument that if a user found out that there is not enough cpus in a cpuset to meet its performance target, one can always increase the number of cpus in the cpuset. Generalizing this behavior to all the tasks irrespective if they have explicitly set cpus affinity before will disallow this use case. > Also, shouldn't this check > p->cpus_mask instead of p->cpus_ptr? You are right. I should have used cpus_mask instead. Will send out a v2. Cheers, Longman