From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8B1A219A2A3 for ; Mon, 26 Jan 2026 18:46:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769453176; cv=none; b=c9JEJGkV4vBov7wRI46INsDp0ubDfx6MhVFa7L9T0Zt1A98t7HpnrE2WR18WA24tIpvD3WGCFRLFHjzkg3GyAVSSWL+/a3fM5ibRK0VRio+oJeWZsmc2QS9PTDGa5XNJhrJ9EuU4JLHm8qzP5o0PJcChOzn3mvuNKqQ6BzqAjDg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769453176; c=relaxed/simple; bh=5DtrhTNr3EkVu79eUPv2FFCqa0H4oC6Ij3l9OjdP7V4=; h=Message-ID:Date:MIME-Version:From:Subject:To:Cc:References: In-Reply-To:Content-Type; b=dqpjqPwAM5XN+qvg8CbohDPR5GzkLKpkcJpZAbabcW0mNSNTnX/rhtffdKU58ALV78h1ri3OEz+L1canzqoJybw/zmx8YUvQg2LpsU2UI6qbOhSKIBZBQlHNrhU6VKC2+LToeMtemZEN/7XqMTOkmIgtS9m9WCDRQ3Xj2RYdETA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=ri7NDGwl; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="ri7NDGwl" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60QCJVTK029492; Mon, 26 Jan 2026 18:45:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=UApyn8 jZ2uSBW8kkJCInVEdeViOxDzcl5eZxDTzsoC4=; b=ri7NDGwlJIxCjYVBO8vGPD h1+bbw3Sro+383+YV18Fe1p3h2PRPAHzyOcPbK7+5JkKCMRJ+vZiEamvV5N7Bggr rG5G1uYHIqSf8IcoIgI5TuDhS59NIrs9pltv+JWjtK5wWHscBD82IfTith+Xe30r 4bconC2XNGP1gOUvgr3hm60Nr31T/PXS+qNJH56rkwtKtbOcRFu1CbsR+2nIX4Qp 16PKrgFvdDAG61tFtE2bwUlvPck9pp6IViI9Bk+Rrz90IsyBgN25AvFWwLagc9Q9 1XNvIO5HatPJb8O88ZpIGTa0tgmzy9952HmdtNvu9vC8119mhpkncwJ0E4uWqVpw == Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnk6smkw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 26 Jan 2026 18:45:54 +0000 (GMT) Received: from m0360083.ppops.net (m0360083.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60QIcYDc026661; Mon, 26 Jan 2026 18:45:53 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnk6smks-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 26 Jan 2026 18:45:53 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60QGRM09019844; Mon, 26 Jan 2026 18:45:52 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4bw9dmwqrc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 26 Jan 2026 18:45:52 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60QIjoqw29557104 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 26 Jan 2026 18:45:50 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4587620043; Mon, 26 Jan 2026 18:45:50 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7ACEA20040; Mon, 26 Jan 2026 18:45:46 +0000 (GMT) Received: from [9.87.135.102] (unknown [9.87.135.102]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 26 Jan 2026 18:45:46 +0000 (GMT) Message-ID: <5d1451b7-bd01-4dac-a32b-9e5c9fe41db7@linux.ibm.com> Date: Tue, 27 Jan 2026 00:15:45 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird From: Shrikanth Hegde Subject: Re: [PATCH v4] sched/clock: Avoid false sharing for sched_clock_irqtime To: Wangyang Guo Cc: linux-kernel@vger.kernel.org, Benjamin Lei , Tim Chen , Tianyou Li , K Prateek Nayak , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider References: <20260126021401.1490163-1-wangyang.guo@intel.com> Content-Language: en-US In-Reply-To: <20260126021401.1490163-1-wangyang.guo@intel.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-GUID: N5qFt92R9SCN1UTrYvV5V-FgL7sVK3uV X-Proofpoint-ORIG-GUID: g9qMpCgo6P99CW9Vh4BY-0liKvCRJkqt X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTI2MDE1NiBTYWx0ZWRfX767882zNa3it 6gTHrBnwC2DI5UBneCXZfl29lF35kFMfgxr0hd1VLOIAjVmNowv++gqSA5HO11QsLS722SLCumH aG5e1aVH0zjW/qGjmu1Z6ko/uYDs3GKAv62/Ue7TqpNpoDzYCjuYJch0OMpoSQOSR3GRWDE6WqR TCMRLqyE5OS0S04M/MA4M5o4IJ1qO2hg5EbatKzBa0Zg0mrUtWeLGJach3Jj3+nTY15C81RSpbG BrELkfgYT+uDsPfYyUwWRGTpV0PlTR6E9vIcepRuyp79MrxGppd2N0rSUqgZ6y55ftNqn8538ij R8pnyFCtGcxr2YA51tJQjdUZTJZVDZuv8aW+9Gkd3FPkhOkKYSzt5CIUXGIOEJveIqLvFHZtCVu 3dv2X+jVC/0UNxWcO0KB0QgmujG7cGInNT+IhPfCv4aBj59gzPQGajo3BXWz5NJerLcGWGFDNWh 7Oi+wuWJWewYJ99sjUA== X-Authority-Analysis: v=2.4 cv=AMiVTGgp c=1 sm=1 tr=0 ts=6977b662 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=IkcTkHD0fZMA:10 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VwQbUJbxAAAA:8 a=zd2uoN0lAAAA:8 a=QyXUC8HyAAAA:8 a=JfrnYn6hAAAA:8 a=VnNF1IyMAAAA:8 a=kZQ_F2YaFlPHM--c9wMA:9 a=QEXdDO2ut3YA:10 a=1CNFftbPRP8L7MoqJWF3:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.20,FMLib:17.12.100.49 definitions=2026-01-26_04,2026-01-26_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0 phishscore=0 bulkscore=0 suspectscore=0 adultscore=0 clxscore=1015 priorityscore=1501 lowpriorityscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601260156 Hi Wangyang. On 1/26/26 7:44 AM, Wangyang Guo wrote: > Read-mostly sched_clock_irqtime may share the same cacheline with > frequently updated nohz struct. Make it as static_key to avoid > false sharing issue. > > Details: > In kernel 6.14, we observed ~3% cycles hotspots in irqtime_account_irq > when running SPECjbb2015 in a 2-sockets system. Most of cycles spent > in reading sched_clock_irqtime, which is a read-mostly var. > IMHO, this changelog is meant for latest sched/core. having 6.14 data won't be that relevant. Also, changelog should be mentioning essence of calling disable in sched_clock_init_late which was in v3 discussions. > perf c2c (cachelien view) shows it has false sharing with nohz struct: > Num RmtHitm LclHitm Offset records Symbol > 6.25% 0.00% 0.00% 0x0 4 [k] _nohz_idle_balance.isra.0 > 18.75% 100.00% 0.00% 0x8 14 [k] nohz_balance_exit_idle > 6.25% 0.00% 0.00% 0x8 8 [k] nohz_balance_enter_idle > 6.25% 0.00% 0.00% 0xc 8 [k] sched_balance_newidle > 6.25% 0.00% 0.00% 0x10 31 [k] nohz_balancer_kick > 6.25% 0.00% 0.00% 0x20 16 [k] sched_balance_newidle > 37.50% 0.00% 0.00% 0x38 50 [k] irqtime_account_irq > 6.25% 0.00% 0.00% 0x38 47 [k] account_process_tick > 6.25% 0.00% 0.00% 0x38 12 [k] account_idle_ticks > > Offsets: > * 0x0 -- nohz.idle_cpu_mask (r) > * 0x8 -- nohz.nr_cpus (w) > * 0x38 -- sched_clock_irqtime (r), not in nohz, but share cacheline > > The layout in /proc/kallsyms can also confirm that: > ffffffff88600d40 b nohz > ffffffff88600d68 B arch_needs_tick_broadcast > ffffffff88600d6c b __key.264 > ffffffff88600d6c b __key.265 > ffffffff88600d70 b dl_generation > ffffffff88600d78 b sched_clock_irqtime > > With the patch applied, irqtime_account_irq hotspot disappear. > > --- Please dont put a --- here. git am will strip everything after this and actual diff. So the your signed-off-by and other details won't be captured. If the intention was keep version history, put it after the signed-off-by for example: https://lore.kernel.org/lkml/20260120113246.27987-2-kprateek.nayak@amd.com/ > V4 -> V3: > - Avoid creating a new workqueue to disable static_key > - Specify kernel version for c2c result in changelog > > V2 -> V3: > - Use static_key instead of a __read_mostly var. > > V1 -> V2: > - Use __read_mostly instead of __cacheline_aligned to avoid wasting > spaces. > > History: > v3: https://lore.kernel.org/all/20260116023945.1849329-1-wangyang.guo@intel.com/ > v2: https://lore.kernel.org/all/20260113074807.3404180-1-wangyang.guo@intel.com/ > v1: https://lore.kernel.org/all/20260113022958.3379650-1-wangyang.guo@intel.com/ > prev discussions: https://lore.kernel.org/all/20251211055612.4071266-1-wangyang.guo@intel.com/T/#u > > Suggested-by: K Prateek Nayak > Suggested-by: Peter Zijlstra > Suggested-by: Shrikanth Hegde > Reported-by: Benjamin Lei > Reviewed-by: Tim Chen > Reviewed-by: Tianyou Li > Signed-off-by: Wangyang Guo > --- > arch/x86/kernel/tsc.c | 2 -- > kernel/sched/clock.c | 3 +++ > kernel/sched/cputime.c | 8 ++++---- > kernel/sched/sched.h | 4 ++-- > 4 files changed, 9 insertions(+), 8 deletions(-) > > diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c > index 87e749106dda..9a62e18d1bff 100644 > --- a/arch/x86/kernel/tsc.c > +++ b/arch/x86/kernel/tsc.c > @@ -1142,7 +1142,6 @@ static void tsc_cs_mark_unstable(struct clocksource *cs) > tsc_unstable = 1; > if (using_native_sched_clock()) > clear_sched_clock_stable(); > - disable_sched_clock_irqtime(); > pr_info("Marking TSC unstable due to clocksource watchdog\n"); > } > > @@ -1212,7 +1211,6 @@ void mark_tsc_unstable(char *reason) > tsc_unstable = 1; > if (using_native_sched_clock()) > clear_sched_clock_stable(); > - disable_sched_clock_irqtime(); > pr_info("Marking TSC unstable due to %s\n", reason); > > clocksource_mark_unstable(&clocksource_tsc_early); > diff --git a/kernel/sched/clock.c b/kernel/sched/clock.c > index f5e6dd6a6b3a..2ae4fbf13431 100644 > --- a/kernel/sched/clock.c > +++ b/kernel/sched/clock.c > @@ -173,6 +173,7 @@ notrace static void __sched_clock_work(struct work_struct *work) > scd->tick_gtod, __gtod_offset, > scd->tick_raw, __sched_clock_offset); > > + disable_sched_clock_irqtime(); > static_branch_disable(&__sched_clock_stable); > } > > @@ -238,6 +239,8 @@ static int __init sched_clock_init_late(void) > > if (__sched_clock_stable_early) > __set_sched_clock_stable(); > + else > + disable_sched_clock_irqtime(); /* disable if clock unstable. */ > > return 0; > } > diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c > index 7097de2c8cda..959a86206c64 100644 > --- a/kernel/sched/cputime.c > +++ b/kernel/sched/cputime.c > @@ -12,6 +12,8 @@ > > #ifdef CONFIG_IRQ_TIME_ACCOUNTING > > +DEFINE_STATIC_KEY_FALSE(sched_clock_irqtime); > + > /* > * There are no locks covering percpu hardirq/softirq time. > * They are only modified in vtime_account, on corresponding CPU > @@ -25,16 +27,14 @@ > */ > DEFINE_PER_CPU(struct irqtime, cpu_irqtime); > > -int sched_clock_irqtime; > - > void enable_sched_clock_irqtime(void) > { > - sched_clock_irqtime = 1; > + static_branch_enable(&sched_clock_irqtime); > } > > void disable_sched_clock_irqtime(void) > { > - sched_clock_irqtime = 0; > + static_branch_disable(&sched_clock_irqtime); > } > > static void irqtime_account_delta(struct irqtime *irqtime, u64 delta, > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h > index adfb6e3409d7..ec963314287a 100644 > --- a/kernel/sched/sched.h > +++ b/kernel/sched/sched.h > @@ -3172,11 +3172,11 @@ struct irqtime { > }; > > DECLARE_PER_CPU(struct irqtime, cpu_irqtime); > -extern int sched_clock_irqtime; > +DECLARE_STATIC_KEY_FALSE(sched_clock_irqtime); > > static inline int irqtime_enabled(void) > { > - return sched_clock_irqtime; > + return static_branch_likely(&sched_clock_irqtime); > } > > /*