From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B1C3386543; Tue, 7 Jul 2026 21:32:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783459970; cv=none; b=FcBnN0dF+BQL5+Uha5wzmE1d9iq8MSRZzCg4YRWVjHS47JLEuq9Xbf15a239PTi/8SGEWSBIbnRKvB7U2pMGYaP24V2exAJ36eDXdhsirtAkoTs0SiizHpqHyS+bw+HqaBA0zbYoznNjydtE9faC4iE8dKMj0o1LwetvC3wyId0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783459970; c=relaxed/simple; bh=lX6T4z6Z3f3pJGca7pmzAOot8QIzjoQe9qpBStxiOJE=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=IiJTghrcQFmZgMY5EwdXZcJ+Ct9QOjAyKkXt8XmI2/XI85ZvUc+LRKyEMW0fMAILMqX+8mWl+KsxKw9msCLOPv6pvUVaVLMoPo1KqxYkLmFBUOwvDNE8hJLFZVT2iGX/QOseP5A9ptOoh0riexuy+0UK4y4xFRS9JXWmAacV3zk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=uaoT7yez; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=vsQdHteh; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="uaoT7yez"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="vsQdHteh" Date: Tue, 07 Jul 2026 21:32:39 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1783459962; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=79zCSRuri63fba3HCDCI+ThlHUZEBALIjbU3iXKResU=; b=uaoT7yezi8OPPM4RvnclYKyDuOWAOGNRKJ5JFjeP2eLcpR0c9zNbUr7gCPJE2Bj+7N37Ba kHU2PbinNG08Ms0zVV0KfikupIO18MUxZOWWhNHBmkc7RaUvjrHSQZM7w5yEsc4HlBzd3w T+xpWu2cPbm8CpT/XRhw8HDuQiPAbiKb1NxyTq9A4BdI9pQEWB4M8PVso8Q1hs7lnz0fG8 ilo3Zf9zVXK8U5OlLp0mxY37i3TC3MapPuS22VuQEMtwAjX9mMJVGezPInrYZCBCAzdM2x DKfEPZfEIPfmlxC32vhQfHdpCuKwHiWIA21bRd5OHF1CjdBmlnnm9wAaMA/tpA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1783459962; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=79zCSRuri63fba3HCDCI+ThlHUZEBALIjbU3iXKResU=; b=vsQdHtehiOZYCuSm55F2BqmPnYs47+khZ0fWoK+zipP3W60Xh8oW6pGAdRfH+D6Awlrn9X HDVaGU9tJSr1xYDw== From: "tip-bot2 for Usama Arif" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: timers/core] hrtimer: Don't take cpu_base::lock in hrtimer_get_next_event() when hres_active Cc: Usama Arif , Thomas Gleixner , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20260707153800.542394-1-usama.arif@linux.dev> References: <20260707153800.542394-1-usama.arif@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <178345995993.744054.3760104924144576805.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the timers/core branch of tip: Commit-ID: c4415c993fc2c8bdf2cf0bfbcbb1ac0e0f7a9eaf Gitweb: https://git.kernel.org/tip/c4415c993fc2c8bdf2cf0bfbcbb1ac0e0f7= a9eaf Author: Usama Arif AuthorDate: Tue, 07 Jul 2026 08:38:00 -07:00 Committer: Thomas Gleixner CommitterDate: Tue, 07 Jul 2026 23:26:58 +02:00 hrtimer: Don't take cpu_base::lock in hrtimer_get_next_event() when hres_acti= ve hrtimer_get_next_event() runs on every tick-stop decision via get_next_timer_interrupt() -> cmp_next_hrtimer_event(). When high resolution timers are active it must return KTIME_MAX -- the caller documents and depends on this. The function takes cpu_base->lock, checks hres_active, and returns KTIME_MAX. Taking the lock is not required because cpu_base->hres_active is only written by the local CPU in hrtimer_switch_to_hres() from hard interrupt context and in hrtimers_cpu_starting() during bring-up. All callers of hrtimer_get_next_event() reach it from the tick-stop / cpuidle paths with interrupts disabled on that CPU. No writer can therefore race with the read, so an unlocked hres_active check is stable and the lock can be skipped in this case. On a 176-thread AMD EPYC 9D64 running a production workload, bucketing callers of native_queued_spin_lock_slowpath(), the slowpath had 199 samples in total, of which 54 are attributed to hrtimer_get_next_event(), i.e. this accounts for ~27% of slowpath hits on this specific workload. Move the hres_active check before the lock guard region to address this. [ tglx: Massaged change log and comments ] Signed-off-by: Usama Arif Signed-off-by: Thomas Gleixner Link: https://patch.msgid.link/20260707153800.542394-1-usama.arif@linux.dev --- kernel/time/hrtimer.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c index 9c320ab..697816d 100644 --- a/kernel/time/hrtimer.c +++ b/kernel/time/hrtimer.c @@ -1786,13 +1786,21 @@ EXPORT_SYMBOL_GPL(__hrtimer_get_remaining); ktime_t hrtimer_get_next_event(void) { struct hrtimer_cpu_base *cpu_base =3D this_cpu_ptr(&hrtimer_bases); - ktime_t expires =3D KTIME_MAX; =20 - guard(raw_spinlock_irqsave)(&cpu_base->lock); - if (!hrtimer_hres_active(cpu_base)) - expires =3D __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_ALL); + /* + * When HRES is active cmp_next_hrtimer_event() expects KTIME_MAX. + * + * cpu_base->hres_active is written only by the local CPU in + * hrtimer_switch_to_hres() from hard interrupt context and in + * hrtimers_cpu_starting() during CPU bring-up, and all callers reach + * this with interrupts disabled on the same CPU, so an unlocked read is + * stable without holding the lock. + */ + if (hrtimer_hres_active(cpu_base)) + return KTIME_MAX; =20 - return expires; + guard(raw_spinlock_irqsave)(&cpu_base->lock); + return __hrtimer_get_next_event(cpu_base, HRTIMER_ACTIVE_ALL); } =20 /**