From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A4AB26ED59 for ; Sat, 16 May 2026 17:16:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778951789; cv=none; b=UJDRw6UUdnN4Mfb3UdUzaP3ypkJA5P1rMpOB/SiokbBON390IdQg6raHzBetDDZknuyC0rXXSBDIt0286dSG2BCOsP9HRLOdoy/PHq2SpRCVzoV+Llqj8Gbl33A4xa8qZ2T6PlLnRv6GK9kBQAY9fK+VTxrQMNIrYtGg1dF2jm0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778951789; c=relaxed/simple; bh=Yh4mLswY2BOsWlyU4qPWw6VnQmNCC3VxHDWuBiMi3v0=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=twlFhUWS2/U5UvkYDuskwRrJx1SLvw4wum1+/em082td4NOr1W8uaMs1PdwAwxW7YzZO4Fp1l8ZWYRY9MP43HniCQ8udqRtXzaupNN+Cd7F23u5y6gbmooO1DAuo3uqpGr7OI4mOJC1oS1svD79s+bkNaRmNThnDShewAb6lvAA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=qgOKkc3W; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="qgOKkc3W" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64GAKqS0642549; Sat, 16 May 2026 17:16:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=ip/7TH Btl1dVstDBQrQnQJFECKD8u941ZYDTz2Rym+4=; b=qgOKkc3WFwh8YiRHPA/XJP izFpJPMewkZce24HPsja5pX0u8GnVBtGy4XdP1crtgtVLynmpFDrGQ1ZuEACOuLz 2cycZNTTkS6IsgvYAm/UJ1CWJYHd3BVF6vb+IxWfpdUp305ZLDWb62OIduZtv6vL IsORvZrjrLHBuFElrfgD1Yg7pBj31Iu3zx1I9m6W0wxV6Wrk+36YDpl+r8wcIUM/ zfCFKJQtCequdtl6FeSCUHx2pZn3OLF0khdD8S9i1ydnbQQrfDqU87x7NKIQHftT ZrdaDwPlmhWQt4iWqR7O+a6OpcKIGvnh7QHWwkwOHXLDVA+UoGR2xGIfcyGOss3A == Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4e6h74hngn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 May 2026 17:16:03 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 64GH96hh023030; Sat, 16 May 2026 17:16:02 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4e5kvcrh7t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 16 May 2026 17:16:02 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 64GHG03e29950718 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 16 May 2026 17:16:00 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AC40320043; Sat, 16 May 2026 17:16:00 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4730F20040; Sat, 16 May 2026 17:15:56 +0000 (GMT) Received: from [9.124.211.133] (unknown [9.124.211.133]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Sat, 16 May 2026 17:15:55 +0000 (GMT) Message-ID: <0aa37c32-4501-4d4d-881c-494845915b4d@linux.ibm.com> Date: Sat, 16 May 2026 22:45:54 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 1/5] sched/fair: Drop redundant RCU read lock in NOHZ kick path To: Andrea Righi Cc: K Prateek Nayak , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Christian Loehle , Phil Auld , Koba Ko , Felix Abecassis , Balbir Singh , Joel Fernandes , linux-kernel@vger.kernel.org References: <20260509180955.1840064-1-arighi@nvidia.com> <20260509180955.1840064-2-arighi@nvidia.com> <4b04aade-8474-4e37-991e-16f2faedaf0c@linux.ibm.com> Content-Language: en-US From: Shrikanth Hegde In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-Reinject: loops=2 maxloops=12 X-Authority-Analysis: v=2.4 cv=ffCdDUQF c=1 sm=1 tr=0 ts=6a08a653 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=IkcTkHD0fZMA:10 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=U7nrCbtTmkRpXpFmAIza:22 a=VnNF1IyMAAAA:8 a=KKAkSRfTAAAA:8 a=7CQSdrXTAAAA:8 a=zd2uoN0lAAAA:8 a=Ikd4Dj_1AAAA:8 a=FNqVcPaf4uiWvacgghQA:9 a=QEXdDO2ut3YA:10 a=cvBusfyB2V15izCimMoJ:22 a=a-qgeE7W1pNrGK8U0ZQC:22 X-Proofpoint-ORIG-GUID: 6J5LBvD0UPAfu7GLNdT5CVos3uBuUQtC X-Proofpoint-GUID: YEJNGYH1dRrYTVWBaTYNeUWJBKwSqZEQ X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTE2MDE3MCBTYWx0ZWRfX5TWFisJQT5qi RuvulFkpEKz0wMy5urLn/FAhRLYX4fBi2Jo4qkbU0LSpgBmSjTfPTt34q4PA0EKtXYKqXtlsx4s eRXxY5HivTjuoQbCyrsjUp1k88IA8deIs0CHsrTZxzJRls2rJHlmEdjeuR6sx9QzHVM3UygSWid xi6KOVBi7hl4Rc+ScsmcS1Zx+GuPBkFUhgkIgR0jn8irXCuRjKmLMXAivFLpYEZ0eri2wslaRlI LL1oLG3bk//BnKW7mk7H99SGpHVzPq5NUOF3ld0yLTCMMIENo4YBTgr0yM5w8hUzO7mLqMDATVl 5YSK5Pcwen8zIRG6uJaQS7s9WsMuf3+T7/NrH3PHS0zxsBffSpSj1Bw71aAkO1MkW65hS0iPjNL iuo8OUZK36Z55qJhs9e7kcsxVUwkYq4Z8b/a1D2VNxbXO03uWyR0E5XtCZrP4QyPFGj6HNF479I p5ZI5UfX0I6MyiUyB6g== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-16_02,2026-05-15_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 spamscore=0 phishscore=0 suspectscore=0 adultscore=0 clxscore=1015 impostorscore=0 lowpriorityscore=0 bulkscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2605130000 definitions=main-2605160170 On 5/16/26 11:15 AM, Andrea Righi wrote: > Hi Shrikanth, > > On Fri, May 15, 2026 at 12:19:16PM +0530, Shrikanth Hegde wrote: >> >> >> On 5/9/26 11:37 PM, Andrea Righi wrote: >>> nohz_balancer_kick() is reached from sched_balance_trigger(), which is >>> called from sched_tick(). sched_tick() runs with IRQs disabled, so the >>> additional rcu_read_lock/unlock() used around sched_domain accesses in >>> this path is redundant. Rely on the existing IRQ-disabled context (and >>> the rcu_dereference_all() checking) instead. >>> >>> The same applies to set_cpu_sd_state_idle(), called from the idle entry >>> path with IRQs disabled, and to set_cpu_sd_state_busy(), reachable via >>> nohz_balance_exit_idle() from two contexts: nohz_balancer_kick() (IRQs >>> disabled, as above) and sched_cpu_deactivate() (the CPUHP_AP_ACTIVE >>> teardown, which runs under cpus_write_lock(), so it cannot race with >>> sched-domain rebuilds). In both cases the rcu_dereference_all() >>> validation is sufficient. >>> >>> No functional change intended. >>> >> >> For this patch, few more comments below. >> >> Reviewed-by: Shrikanth Hegde >> >>> Cc: Vincent Guittot >>> Cc: Dietmar Eggemann >>> Suggested-by: K Prateek Nayak >>> Reviewed-by: K Prateek Nayak >>> Signed-off-by: Andrea Righi >> >> >>> @@ -12868,17 +12860,13 @@ static void nohz_balancer_kick(struct rq *rq) >>> static void set_cpu_sd_state_busy(int cpu) >>> { >>> struct sched_domain *sd; >>> - >>> - rcu_read_lock(); >>> sd = rcu_dereference_all(per_cpu(sd_llc, cpu)); >>> if (!sd || !sd->nohz_idle) >>> - goto unlock; >>> + return; >>> sd->nohz_idle = 0; >>> atomic_inc(&sd->shared->nr_busy_cpus); >>> -unlock: >>> - rcu_read_unlock(); >>> } >>> void nohz_balance_exit_idle(struct rq *rq) >>> @@ -12897,17 +12885,13 @@ void nohz_balance_exit_idle(struct rq *rq) >>> static void set_cpu_sd_state_idle(int cpu) >>> { >>> struct sched_domain *sd; >>> - >>> - rcu_read_lock(); >>> sd = rcu_dereference_all(per_cpu(sd_llc, cpu)); >>> if (!sd || sd->nohz_idle) >>> - goto unlock; >>> + return; >>> sd->nohz_idle = 1; >>> atomic_dec(&sd->shared->nr_busy_cpus); >>> -unlock: >>> - rcu_read_unlock(); >>> } >>> /* >> >> I was looking at other users of sd_llc, i.e test_idle_core and set_idle_core. >> They have rcu_dereference_all. So callers need not call rcu_read_lock/unlock if >> the irq disabled/preempt_disabled. >> >> One more place would be update_idle_core. I think it is called with interrupt disabled >> in __schedule path. > > Good point, __update_idle_core() reaches set_next_task_idle() via > pick_next_task() in __schedule(), and __schedule() disables IRQs before that > path. > > Since set_idle_cores()/test_idle_cores() use rcu_dereference_all(), the > rcu_read_lock/unlock() pair in __update_idle_core() is indeed redundant. I can > send a follow-up patch for this. > Thanks. >> >> And in sched_ext, scx_idle_update_selcpu_topology, It seems to be tied to cpu hotplug and >> by same logic of cpus_write_lock held, one could remove redundant rcu_read_lock there as well. >> >> No? > > For scx_idle_update_selcpu_topology() it's a bit more nuanced, if I'm not > missing anything: > - the helpers it uses (llc_weight/llc_span/numa_weight/numa_span) use plain > rcu_dereference(), so simply dropping rcu_read_lock() in the caller would > trip the lockdep check. They'd need to be converted to rcu_dereference_all() > first; > - the two call sites have different protection: > - handle_hotplug() runs from a CPU hotplug callback, so cpus_write_lock() > is held, serializes against sched-domain rebuilds, > - scx_enable() only holds cpus_read_lock(), which doesn't on > its own prevent cpuset sched-domain rebuilds (those run under > cpus_read_lock() too). > > I think this one needs a separate, more careful patch. Maybe we should keep this > series scoped to the NOHZ kick path and address those as follow-ups? > > Thanks, > -Andrea Yes. That makes sense.