From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-dl1-f54.google.com (mail-dl1-f54.google.com [74.125.82.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D383124A076 for ; Fri, 3 Apr 2026 05:11:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.54 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775193110; cv=none; b=I2NXiUjm3pwhusUEogViEsq9IsgRgLC1iaiDgbJcL6Aw8gIJhihT3J7hkeQgzTAwFVJXRBdgDY+onvlfLLmVzsMQS3vehJk3Pk4RhYzANg7sSlb9Di7gpQ/ZRdrSFSiyuo9DnuBpWrL2YCr5WxAWctsx05kaO4DpIiMONogbhdg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775193110; c=relaxed/simple; bh=FOsweVg0nOvJJKdboKfjpyDi7q0dzPZxpa2Ok7l22k4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=aHY0JN4Sj2PGdonAY0ZU9lvhrxF7DGsztJStG6UlMGJeCU6PZ0ZxXe9WmTv8Ds8bX/51aNae2VE9P02pN6LfDQXYt5gHXBjHQucFCcORdIM1TacZFBBaKAfD9PfqG2ugBi6aUH4pGGCD/Vxzm8IBrGspfD1t0uCxZzhnETLTNi0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=wbinvd.org; spf=pass smtp.mailfrom=wbinvd.org; dkim=pass (2048-bit key) header.d=wbinvd.org header.i=@wbinvd.org header.b=TuJ29xIr; arc=none smtp.client-ip=74.125.82.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=wbinvd.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=wbinvd.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wbinvd.org header.i=@wbinvd.org header.b="TuJ29xIr" Received: by mail-dl1-f54.google.com with SMTP id a92af1059eb24-12bfa7fe691so918367c88.0 for ; Thu, 02 Apr 2026 22:11:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wbinvd.org; s=wbinvd; t=1775193108; x=1775797908; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=3hHsurcH57Wpr/XvbFvlepPsT1GbzIaJORFfjYT8fSw=; b=TuJ29xIrotaFSGjhe+ULamGstwQTJLkw9AWnTQbXOCuxv+f8z/1Kkq1YiIyHbvS2v0 9ETRm1UergVkLV+6rjVFeVdcysqVjvD71ZZWQJ7aSJv8tRAx7/F7EgMxmA+B3wLEpfaK K0FKpPdBjeK9/IoDYX/wwmtn8MhHAn4goL4iBXxqbtQAvZ1trKbnzYYOtP+ZaqX1L3RV 86MioOUOvn31TeAv+K0itOMn0nenfqREhu7tKHzw095VjimgOvOgOhKnLvwDbLWR1nvL g0BPgCU35t+OH04Q3kfS7uIF9KhgQ+CVpr8fxKSB/H6B9jfdE/p77S+7hxNYve96pH42 hOWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775193108; x=1775797908; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3hHsurcH57Wpr/XvbFvlepPsT1GbzIaJORFfjYT8fSw=; b=UJL77ZU8QBKji3lp2DeYZ0lCKlZPlbofPTyZpq1U3G8J7zqANTfMQGoy6Oo4voVyO8 FoUoAFLF9Aq022eD1wy7o72ZCdqva+xhdKQKE7smY7K418wM8dICGoA3w9VJ0RftZ4Rq 3NRDd3klLgMWBTWGkunNU2AnLQEV+CcuWWfIixMg3o/Po2R6LcWdRLchzdkJl7vDuk9Q CG3DLfv5XztemTsFV8nNFr1MxSFnGuoBV/rg8En6lkmCsaRFx8/LF7npEFg6gPmgzY3b NsD/xoZjKGUynCvlduIbQQW+HNK04KUEhLV2urt9Jq03jRt+ZHEQeK8I3jh+ng1w5QHU 6lcw== X-Forwarded-Encrypted: i=1; AJvYcCVE5kGNYu115Doppdh8Z0RiHEanI89hMNPXrEVYgJNuMqJVwrZT2Wo2BMQNmTtPm/PfnIUTqMZ8mxBvJAQ=@vger.kernel.org X-Gm-Message-State: AOJu0YwrviMCoZKymAnd/HAkQJp9KBV+jUfVwvPV1WY49Q231uOPUeq+ vVmCW7RmpWnqKCXhz24imWrBWRm3MO1GpX8sbZrOBr95oxFQI/6+ojmt9ukuRERHdAE= X-Gm-Gg: ATEYQzxfGGkTmXz0qljZZur2lyHUVzQx4IZNHrgXU8c/CdCEbhBGu5JNApteldibYKy P9UQyOEZfDFHesLVyNQpXzv1sPvjgf7Nh3J6MVqXIuvgdy4jEZX9rabKrXYpg7mYYp3EobN2LW9 BbEn8M+7ghOu0Xxmzx8EyRo3GNhLrc7s0AMFhs/LcpJHuZYPH89aazpfg4/95eInp+aY5In7lHM YxUtaXkJ+jZaTtfkadADc2dv75SFvaF1ngJ1mL+1Aq6EeZmZlOf86HILXhJfEr9rC68TL5Sw3ge lJ9jcNaQQXLeWDvOyFCBAk4yhJ3BW0BU0ujGNtMg6AFAsysvDbPNcsrgYFLUgmZMxR7uk/vd0FJ jT79VbufybUVMxKJqCnGhKBcyPIUDJ9ZRtVZEX5JjYqzs1ebsHF2DPY2NKsrO2id7W/izkF/HuY patAoTYDuNxjkkTDpnUSnzeMhSxQ== X-Received: by 2002:a05:7022:6713:b0:12b:f881:d923 with SMTP id a92af1059eb24-12bfb76037amr782329c88.28.1775193107675; Thu, 02 Apr 2026 22:11:47 -0700 (PDT) Received: from mozart.vkv.me ([2001:5a8:468b:d015:b0b6:acb3:9e6a:b5df]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-12bedd70f18sm3846194c88.6.2026.04.02.22.11.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Apr 2026 22:11:46 -0700 (PDT) Date: Thu, 2 Apr 2026 22:11:43 -0700 From: Calvin Owens To: Thomas Gleixner Cc: Borislav Petkov , Petr Mladek , linux-kernel@vger.kernel.org, arighi@nvidia.com, yaozhenguo1@gmail.com, tj@kernel.org, feng.tang@linux.alibaba.com, lirongqing@baidu.com, realwujing@gmail.com, hu.shengming@zte.com.cn, dianders@chromium.org, joel.granados@kernel.org, Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Frederic Weisbecker , Anna-Maria Behnsen , x86@kernel.org Subject: Re: [PATCH] clockevents: Prevent timer interrupt starvation Message-ID: References: <87v7ejetl1.ffs@tglx> <875x6a913n.ffs@tglx> <20260401163435.GGac1JG42tWmsCKL37@fat_crate.local> <87jyup70ka.ffs@tglx> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <87jyup70ka.ffs@tglx> On Thursday 04/02 at 19:07 +0200, Thomas Gleixner wrote: > Calvin reported an odd NMI watchdog lockup which claims that the CPU locked > up in user space. He provided a reproducer, which set's up a timerfd based > timer and then rearms it in a loop with an absolute expiry time of 1ns. > > As the expiry time is in the past, the timer ends up as the first expiring > timer in the per CPU hrtimer base and the clockevent device is programmed > with the minimum delta value. If the machine is fast enough, this ends up > in a endless loop of programming the delta value to the minimum value > defined by the clock event device, before the timer interrupt can fire, > which starves the interrupt and consequently triggers the lockup detector > because the hrtimer callback of the lockup mechanism is never invoked. > > As a first step to prevent this, avoid reprogramming the clock event device > when: > - a forced minimum delta event is pending > - the new expiry delta is less then or equal to the minimum delta > > Thanks to Calvin for providing the reproducer and to Borislav for testing > and providing data from his Zen5 machine. > > The problem is not limited to Zen5, but depending on the underlying > clock event device (e.g. TSC deadline timer on Intel) and the CPU speed > not necessarily observable. > > This change serves only as the last resort and further changes will be made > to prevent this scenario earlier in the call chain. > > Reported-by: Calvin Owens > Signed-off-by: Thomas Gleixner > --- > P.S: I'm working on the other changes, but wanted to get this out ASAP > for testing. Unfortunately the AMD boxes won't boot with this: one gives me no video console output, the other gets to userspace but hangs trying to mount the rootfs and then prints hard lockup traces with idle stacks. Sorry not to have more info yet, I'll have time tomorrow to sit down and get more data for you. If there's anything specific that you'd like me grab just let me know. Thanks, Calvin > --- > include/linux/clockchips.h | 2 ++ > kernel/time/clockevents.c | 37 +++++++++++++++++++++++-------------- > 2 files changed, 25 insertions(+), 14 deletions(-) > > --- a/include/linux/clockchips.h > +++ b/include/linux/clockchips.h > @@ -80,6 +80,7 @@ enum clock_event_state { > * @shift: nanoseconds to cycles divisor (power of two) > * @state_use_accessors:current state of the device, assigned by the core code > * @features: features > + * @next_event_forced: True if the last programming was a forced event > * @retries: number of forced programming retries > * @set_state_periodic: switch state to periodic > * @set_state_oneshot: switch state to oneshot > @@ -108,6 +109,7 @@ struct clock_event_device { > u32 shift; > enum clock_event_state state_use_accessors; > unsigned int features; > + unsigned int next_event_forced; > unsigned long retries; > > int (*set_state_periodic)(struct clock_event_device *); > --- a/kernel/time/clockevents.c > +++ b/kernel/time/clockevents.c > @@ -172,6 +172,7 @@ void clockevents_shutdown(struct clock_e > { > clockevents_switch_state(dev, CLOCK_EVT_STATE_SHUTDOWN); > dev->next_event = KTIME_MAX; > + dev->next_event_forced = 0; > } > > /** > @@ -224,13 +225,7 @@ static int clockevents_increase_min_delt > return 0; > } > > -/** > - * clockevents_program_min_delta - Set clock event device to the minimum delay. > - * @dev: device to program > - * > - * Returns 0 on success, -ETIME when the retry loop failed. > - */ > -static int clockevents_program_min_delta(struct clock_event_device *dev) > +static int __clockevents_program_min_delta(struct clock_event_device *dev) > { > unsigned long long clc; > int64_t delta; > @@ -263,13 +258,7 @@ static int clockevents_program_min_delta > > #else /* CONFIG_GENERIC_CLOCKEVENTS_MIN_ADJUST */ > > -/** > - * clockevents_program_min_delta - Set clock event device to the minimum delay. > - * @dev: device to program > - * > - * Returns 0 on success, -ETIME when the retry loop failed. > - */ > -static int clockevents_program_min_delta(struct clock_event_device *dev) > +static int __clockevents_program_min_delta(struct clock_event_device *dev) > { > unsigned long long clc; > int64_t delta = 0; > @@ -293,6 +282,21 @@ static int clockevents_program_min_delta > #endif /* CONFIG_GENERIC_CLOCKEVENTS_MIN_ADJUST */ > > /** > + * clockevents_program_min_delta - Set clock event device to the minimum delay. > + * @dev: device to program > + * > + * Returns 0 on success, -ETIME when the retry loop failed. > + */ > +static int clockevents_program_min_delta(struct clock_event_device *dev) > +{ > + if (dev->next_event_forced) > + return 0; > + > + dev->next_event_forced = 1; > + return __clockevents_program_min_delta(dev); > +} > + > +/** > * clockevents_program_event - Reprogram the clock event device. > * @dev: device to program > * @expires: absolute expiry time (monotonic clock) > @@ -324,6 +328,11 @@ int clockevents_program_event(struct clo > return dev->set_next_ktime(expires, dev); > > delta = ktime_to_ns(ktime_sub(expires, ktime_get())); > + > + /* Don't reprogram when a forced event is pending */ > + if (dev->next_event_forced && delta <= (int64_t)dev->min_delta_ns) > + return 0; > + > if (delta <= 0) > return force ? clockevents_program_min_delta(dev) : -ETIME; >