From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-dl1-f41.google.com (mail-dl1-f41.google.com [74.125.82.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C09B45C0B for ; Sat, 4 Apr 2026 00:15:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.41 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775261718; cv=none; b=TTuCPBbBzCQstE28OhZTkXOHbaHI/M9IB7ev/W4mOSA6BLWXxZKX/d+jeOLdHVtcvRvH5Rv52qosDpsAHzMO5eIPT0DLoB73fSpb9JglAU5kDVcl1jxrvwW3hzTC4rp0PLKMiyDG/1LJNbjCt53BQ3bCjP9UZnG/panMk6kF1Nw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775261718; c=relaxed/simple; bh=aoh8219+9gpSOSgukF3YhT3dbLC1QEzpQbe6/GbL4ds=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Z/fmFoJVzf8CfDqQgW552X0toi/MpAIrp9AWdp8tbLo/IsdQ552JixfYmbp5dkaxVEUGv9HripWJ8Qm6Xyw47/RcgRc6KVTnKRdat/WooW24zS3LRdexuPriZUsOiZCw42ybRLyUT3KSp1I5QhCY4GBEMYNVUjC5ydH8t2duBiM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=wbinvd.org; spf=pass smtp.mailfrom=wbinvd.org; dkim=pass (2048-bit key) header.d=wbinvd.org header.i=@wbinvd.org header.b=cJ/ox5nM; arc=none smtp.client-ip=74.125.82.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=wbinvd.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=wbinvd.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=wbinvd.org header.i=@wbinvd.org header.b="cJ/ox5nM" Received: by mail-dl1-f41.google.com with SMTP id a92af1059eb24-126ea4b77adso2913052c88.1 for ; Fri, 03 Apr 2026 17:15:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wbinvd.org; s=wbinvd; t=1775261714; x=1775866514; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=JtITzDHsLCHifOjJIbcpD7IFnznn22JXL525z1JOpiE=; b=cJ/ox5nMNWGN8iicyqjb8pcyOY/47/2Pak0H2czqCDbB72XjKQYLHpXi67LmZraWUU KV/KlzoKufisnjuPFWMA84wejuKSgl+9bGotbzpEEYipAviFZU8QCofbC3ZY9e4Ih2Df YS7Idq4DnG5tdcFXjMORQxTBR4CzhI6wWLbjpotMeiZALpSz5d6E3mGPHeVSMTecYdg3 nRv5QvEayI1E5gk5qZ7hrP+584N3a0C2Ra3mldxSqyRce1TPrFizaza0k81sF/L/LnU6 UazMP9ChHqeWyE4JaWaeTTd/ll/OUlYOmg/GMI8ocI/LHmXHScen/roDswEcEG92ne1X 1oVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775261714; x=1775866514; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JtITzDHsLCHifOjJIbcpD7IFnznn22JXL525z1JOpiE=; b=cjQPllBE7yjVW6+ATK5X88BPk60VefWDMaiptiq4sFuuOBj957KXBpRrqT4WCt7Wej meUsWh6f1EUWW4SWaahRrAmNPvybkogNtp6sKUVcxZoNWEGR+VGVm62wRwrtvvzK2tJ+ nBMbnwGNTThPLOAbYVGeCcQBSHSEjEan68pAZ+En5U8o9vvILCJqxo9wPS95vcsCq2gp GeYyGaKn80iQKz/cDCgc8K25Sq/78H1/SjqskLXrmh/IrxJE6BV5HhAkt3AOkJoptfFV Yb13YUxzlCC/Dep4EONZpX3pshZ9F5FwKMUI0JhQkGtQsYk828F0bhEeKBCoFtO859QZ 3Suw== X-Forwarded-Encrypted: i=1; AJvYcCVlAbe6+0snLmAe6vyKKjTJPH3cx8BqEjnH7mx4mkRnEKzTbISiysnd0CKVyqnaMYOy0/H/36pZmBUrkDQ=@vger.kernel.org X-Gm-Message-State: AOJu0YxuR2NXSjFlaYBAmJ8sCrADBDbfj8E4hstMAE5BQO2gMnEf7S0I v549wTHrGRtOyo3KvR2kPtKUB/DlCHNhTKTSjIFjM5LR9qVRpUKFLB7WlS3Q4cZkbqU= X-Gm-Gg: AeBDiet16E87Ri/wNceDAf3uDlLS+ecHQ1PANpRZ0JUZU84aVeeT/r2dP+0sxAL3DU2 wZ+l5b5CA9acV0R2ESrPfujlcaR3AbU4Zxpq7U9qBYe1YiTX2ZTPB7kUjmyS7nuyfy9yK3Qttmo CRK7CRjUKV2EiLGcFNP2n6hOS07anPmObzYsViCUiqL1QO0c0m4VSS4PEU1kVN8+kyeOfI6OqEp 8GqqkH9EwmxPq47wrLA0jtUX8boK4OlCKLBgUIHx4+uwFLc0OrcRldy34R42PH47LLGbPLdWC2f T7daqMJ2sf9M21iUK0WRyAl6Y7/GBsUG1Sbrl3M3rIgjZ9OUi7G9YKJ5fXoYCzUwvCUjYpgB0y8 pOG/3gumjIgP1yqAa/ZQ64TeDqXuVBXYK1z3f+t+h/SWOWyqOB8NFzVmRa/jO9AY1pn78DEurWy 3MhBogvFJVi5R+QH6Eu2rL6QfVkA== X-Received: by 2002:a05:7022:62aa:b0:127:3f2a:af21 with SMTP id a92af1059eb24-12bfb70eae1mr2110818c88.15.1775261714261; Fri, 03 Apr 2026 17:15:14 -0700 (PDT) Received: from mozart.vkv.me ([2001:5a8:468b:d015:7a23:62aa:25cd:2269]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-12bedd920b9sm5049460c88.8.2026.04.03.17.15.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Apr 2026 17:15:13 -0700 (PDT) Date: Fri, 3 Apr 2026 17:15:10 -0700 From: Calvin Owens To: Thomas Gleixner Cc: Borislav Petkov , Petr Mladek , linux-kernel@vger.kernel.org, arighi@nvidia.com, yaozhenguo1@gmail.com, tj@kernel.org, feng.tang@linux.alibaba.com, lirongqing@baidu.com, realwujing@gmail.com, hu.shengming@zte.com.cn, dianders@chromium.org, joel.granados@kernel.org, Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider , Frederic Weisbecker , Anna-Maria Behnsen , x86@kernel.org Subject: Re: [PATCH] clockevents: Prevent timer interrupt starvation Message-ID: References: <87v7ejetl1.ffs@tglx> <875x6a913n.ffs@tglx> <20260401163435.GGac1JG42tWmsCKL37@fat_crate.local> <87jyup70ka.ffs@tglx> <87bjg06r7t.ffs@tglx> <87zf3j6f88.ffs@tglx> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <87zf3j6f88.ffs@tglx> On Friday 04/03 at 21:00 +0200, Thomas Gleixner wrote: > On Fri, Apr 03 2026 at 08:58, Calvin Owens wrote: > > On Friday 04/03 at 16:41 +0200, Thomas Gleixner wrote: > >> I'm an idiot. When I polished the patch up, I dropped the hunks which > >> clear the flag in the interrupt handler and tired brain did not notice > >> despite checking five times in a row. Updated version below. > > > > That did it, both AMD machines survive the reproducer and are well > > behaved afterwards. > > Thank you and sorry for the nuisance. > > > If you like: > > > > Tested-By: Calvin Owens > > I will probably post a slightly different version similar to the one I > sent in the reply to Peter and if you have time then I would appreciate > a tested-by on that final to be polished version. I will take a look. > Btw, I'm really curious how you deduced the reproducer from systemd > code. I assume you figured somehow out which program triggered the > behaviour and then inspected the source to find something fishy. Can you > provide a pointer to the code in question? If they really do what your > reproducer does, then this code needs to be fixed too :) I pulled the text that was executing when the NMI fired out of the dump: 00 ba 38 03 00 00 48 8d 35 ce 40 18 00 48 8d 3d 16 41 18 00 e8 11 14 e8 ff b8 f4 ff ff ff e9 6d ff ff ff 0f 1f 80 00 00 00 00 0f b6 4f 2f 48 8d 15 e5 5f 26 00 48 89 c8 83 e0 03 48 c1 e0 05 48 ...and searched for it in systemd-networkd and all its libs. It appears in one spot in libsystemd-shared-259.so in path_hash_func(), so that must be where the userspace %ip was when the NMI fired. Unfortunately that has too many callers: I couldn't narrow it down meaningfully from there. Despite staring at a lot of timer code in systemd, I haven't yet found anything concrete that might cause buggy behavior. But, it stuck out at me that the detritus on the stack wasn't futex() or poll() or read() related. It seemed wildly improbable that the NMI would have just happened to catch systemd-networkd running like that, I guessed it was probably spinning around timerfd_settime() in userspace when the NMI fired (with calls to path_hash_func() somehow in-between). My initial guess was that the trigger was something about waiting on the timer in a different thread than it was set on. I started to write that out as a small reproducer, but almost jokingly thought, "well, I should just try setting them blindly first and see if that works", and then my head exploded when it actually did :) I've tried overloading the machine, and triggering some unrealistically large time steps back and forth underneath it. But I can't get systemd to stick itself in any sort of loop like that, or even set a single timer expiry to an unreasonable value. I think I will set up a little BPF thing to force systemd-networkd to dump core if it makes timerfd_settime() calls too quickly or with abstime arguments in the past, hopefully from the core I can work out what was going on. But any better suggestions are welcome. Thanks, Calvin