From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3D55C14C59C for ; Fri, 27 Dec 2024 07:59:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735286373; cv=none; b=eHiVP01t9rMGTq+R/g10ye4GdId50dA/YA7NUmm2fzetGWsMsvddiQMP/yGcp+AeanPn+2baN64YmVQgTZIdD0LoaykdatY1iglQGjWrMzXNcKlfGKBippWvz2tRTNN2P8Kk2vWtWF2Je8Zg6d/nqSqWeu42C7nR7c2kknbKSDo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1735286373; c=relaxed/simple; bh=dADSKLIn3X/Fz80H7azbpMEXNU7KmMJG7jgUaYiGWBE=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=nNnG1ST4IpIxCcaenY+XKiTFhu883CSh2P2/EhCRUnVhrpacbvuJesMg1gC5C4sWh7JqLgD/yJelSpPmEjMdqboWNeKa/QpBKld1vcWtXzw5WuXtmJETdrrnhRHs3oIuUH1JU6UjLBjGRrEOYvfQvWFDloeehrOMCJt7GsGKTaY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=YxeKeXdL; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="YxeKeXdL" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-21661be2c2dso75816535ad.1 for ; Thu, 26 Dec 2024 23:59:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1735286369; x=1735891169; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=TLV5QK9hyxUI41973tuagJWxunw1Iurn1BKj80JparE=; b=YxeKeXdLirGO8Upk4UpV8TynuBi2rgRrMJ8I5+b8lJsrlD/8r8yCtzENlkGl5Y1YwU vZKly7dX9ApxaFhC8vQPmdkcK5TBVcRYwXol+z/JIxsapSBNfdU+3MsP3sENQrau43gl 2N07yUGl8EWZI0IEx/Ib6EPCovkZGqg1XA57DbaNfD9FkOxJ44mDLvLiGXt08RrZaSdo R44Lj5VMxvWwJLUD6VO8jLhksEwJOjsi3Ar08CK1X/VuNz+SMzXMRhegXK0yDEP69Aad l40eQQMv4gf9HbdfrQ6JjuRNCZcfgM2mHLhuPLOIS823vlBmwes5aSi9GhNEhbL6LFFS 3uPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735286369; x=1735891169; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=TLV5QK9hyxUI41973tuagJWxunw1Iurn1BKj80JparE=; b=tFKDEE3Ncma4xYCZNTsN1hC9brEC+ubPmsbo51eJKuEER36frlHkZK0adLRVKGdyuL TDD+xtwfsgn0pVkL4/s3lkbHA8hXgJyBgw03vpg+ckBv8BDoDjqaVIG8j5O5PBY0N5Z1 WG4aFK5gdRNsCyCzCJXkELz71nfPBrwAVE92C/pXQZoWWd3xCsB7DocXckKr9PGF1r1P 5/H4vRME3nwb1TkXP4DpeREaOSNENppE+qqEfAN1n3/UpkeEMKn5v43pne0WijlqNU2p YIIx2m0WelWDNaRq1gJgd3JzKwlcentoD47S1U/wdhEbq8f0ajac0mH2PmOqZUZoLwin fWHg== X-Forwarded-Encrypted: i=1; AJvYcCWes8coRcCsv95UtMJgCGAnG8MIYZJX0RaMRGjMfQBgUf5hqpylTqUWQTBlELllLM5teAnpREr7dUWqwt4=@vger.kernel.org X-Gm-Message-State: AOJu0YzQ2RncZhqjGgrD7WbgLFhuT/6f/RrAIXeIpAiFWJKJrgd3Ip6A eGxD24pmG0Tq2ibTcZL5Dpus1srAxK10UTCYe9FKcDPZUdO9SesG0ONaPXttZoY= X-Gm-Gg: ASbGnctv0XEcTTrFmLvd/R/uc45sZ7jTwE7BZ9JiVLnQs+klcF0CDndy7aTKA0PDgDp 5K/zQto9dvY3kOfT/au5SEAlytP6f8rUy/8rL2FVOkTbad/WxrBV8NPcK3AEqN5Si3WvdWPMySI rOu6aAMovMZp+vy2uWHS52eNjhFZmMawXFUu5bz4yyrDFUm84KhgF8/MNWKUqyB3HpNIjcsvHMu u9jKC8AerreuL2pmb+fZWCt4r4BSeJFZZEraFiHVhx8zWf4yRq8q2fgSxeUkPyyzAlU1Ms7rVLa ZX0= X-Google-Smtp-Source: AGHT+IG3WT3aHxlb+GMt5QHvrbesrSggHoQPJs6ZiCrBOwbSyC7FhmyEeoUQyRTEm65BssHcNBqi1Q== X-Received: by 2002:a05:6a21:3987:b0:1e1:dbfd:582b with SMTP id adf61e73a8af0-1e5e0461e18mr42270649637.15.1735286369552; Thu, 26 Dec 2024 23:59:29 -0800 (PST) Received: from [10.255.195.17] ([139.177.225.237]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-842b1ce1fd5sm12920680a12.25.2024.12.26.23.59.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 26 Dec 2024 23:59:29 -0800 (PST) Message-ID: Date: Fri, 27 Dec 2024 15:59:22 +0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 3/3] sched/fair: Ensure select housekeeping cpus in task_numa_find_cpu To: K Prateek Nayak , mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, vschneid@redhat.com Cc: chengming.zhou@linux.dev, linux-kernel@vger.kernel.org References: <20241216122357.1276382-1-zhouchuyi@bytedance.com> <20241216122357.1276382-4-zhouchuyi@bytedance.com> From: Chuyi Zhou In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hello, 在 2024/12/27 12:40, K Prateek Nayak 写道: > Hello Chuyi, > > On 12/23/2024 6:28 PM, Chuyi Zhou wrote: >> >> >> 在 2024/12/18 14:21, K Prateek Nayak 写道: >>> Hello Chuyi, >>> >>> On 12/16/2024 5:53 PM, Chuyi Zhou wrote: >>>> [..snip..] >>>> @@ -2081,6 +2081,12 @@ numa_type numa_classify(unsigned int >>>> imbalance_pct, >>>>       return node_fully_busy; >>>>   } >>>> +static inline bool numa_migrate_test_cpu(struct task_struct *p, int >>>> cpu) >>>> +{ >>>> +    return cpumask_test_cpu(cpu, p->cpus_ptr) && >>>> +            housekeeping_cpu(cpu, HK_TYPE_DOMAIN); >>>> +} >>>> + >>>>   #ifdef CONFIG_SCHED_SMT >>>>   /* Forward declarations of select_idle_sibling helpers */ >>>>   static inline bool test_idle_cores(int cpu); >>>> @@ -2168,7 +2174,7 @@ static void task_numa_assign(struct >>>> task_numa_env *env, >>>>           /* Find alternative idle CPU. */ >>>>           for_each_cpu_wrap(cpu, cpumask_of_node(env->dst_nid), >>>> start + 1) { >>> >>> Can we just do: >>> >>>      for_each_cpu_and(cpu, cpumask_of_node(env->dst_nid), >>> housekeeping_cpumask(HK_TYPE_DOMAIN)) { >>>          ... >>>      } >>> >>> and avoid adding numa_migrate_test_cpu(). Thoughts? >> >> Make sense, but now there doesn't seem to be an API like >> for_each_cpu_wrap_and(). >> >> Do you think the following is better? >> >> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c >> index 855df103f4dd..4792ef672738 100644 >> --- a/kernel/sched/fair.c >> +++ b/kernel/sched/fair.c >> @@ -2167,9 +2167,9 @@ static void task_numa_assign(struct >> task_numa_env *env, >>                  int start = env->dst_cpu; >> >>                  /* Find alternative idle CPU. */ >> -               for_each_cpu_wrap(cpu, cpumask_of_node(env->dst_nid), >> start + 1) { >> +               for_each_cpu_and(cpu, cpumask_of_node(env->dst_nid), >> housekeeping_cpumask(HK_TYPE_DOMAIN)) { >>                          if (cpu == env->best_cpu || !idle_cpu(cpu) || > > "start" is set to "env->dst_cpu" is already taken care here with the > first comparison. > >> -                           !cpumask_test_cpu(cpu, env->p->cpus_ptr)) { >> +                               cpu == start || !cpumask_test_cpu(cpu, >> env->p->cpus_ptr)) { >>                                  continue; >>                          } >> > > I think the for_each_cpu_wrap() was used to reduce contention for xchg > operation below. Perhaps we can have a per-cpu temporary mask (like > load_balance_mask) if we want to reduce the xchg contention and break > this into cpumask_and() + for_each_cpu_wrap() steps. I'm not sure if > any of the existing masks (load_balance_mask, select_rq_mask, > should_we_balance_tmpmask) can be safely reused. Otherwise, perhaps we > can make a case for for_each_cpu_and_wrap() with this use case. > for_each_cpu_and_wrap() is a good idea, but it might be slightly off-topic for this subject. Perhaps we should stick with this implementation for now and see what others think about v2. Thanks.