From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out30-118.freemail.mail.aliyun.com (out30-118.freemail.mail.aliyun.com [115.124.30.118]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2BC1C3D3B3 for ; Thu, 11 Jun 2026 07:18:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.118 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781162335; cv=none; b=pkRnil+EI20Yhv/kRXJ85Ae4NqIFOXyjgaknBvfHSYopElujx24QQPUFFEaR/Fo0uiojMtnumaJyNLyaG3SLX8rR7fwUIyDI11lu2NPOE74Nnm1mafn8NIzamiScmIlfLFI8Oj4WC3eFVFJfFbHC5MnsU3uUMV0iyCXL1nxVo/Y= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781162335; c=relaxed/simple; bh=nASsBpSbUdIyjUDpkLiVRh4hXb+0qzgMU4zOmMbc1Go=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=dtT3V9NnBin2b+9bIgLOY4OHnRRJh5f26Hp6xz74woiCpTMy9YHf5ftVvrF9v6LfKQVQZvs5odLXNMc4DGjlmV/mruW2bZm5IoeHmIRe0RzeX5Xuk7r6S30hdX7dtitsojIhqdRipgHapKqPI5JqfNQAdBkCymE+m+CKVzQ73v8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=wBkAhnFM; arc=none smtp.client-ip=115.124.30.118 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="wBkAhnFM" DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1781162329; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=d+wwjKUtYJODOeQlLu6/4XTlUk9UW2fMnrTvY3Vxjr0=; b=wBkAhnFMS8hwsg5oEtIkCUCVMeM5yFNl02n9KFNvePEZxzZgaZZfiNkVPgfl8LxIQecY8dDyeDu92QZfLTpR033KOKYsCWueRTUFWesSrMhQ/6f6UIRPQZ9wU4G7UArVnoofRfQpKaH1mVZvj6X9G0T/8X0lw+HZVCefOFQBmig= X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam033037033178;MF=qinyuntan@linux.alibaba.com;NM=1;PH=DS;RN=9;SR=0;TI=SMTPD_---0X4d-Gz1_1781162327; Received: from banye.tbsite.net(mailfrom:qinyuntan@linux.alibaba.com fp:SMTPD_---0X4d-Gz1_1781162327 cluster:ay36) by smtp.aliyun-inc.com; Thu, 11 Jun 2026 15:18:48 +0800 From: Qinyun Tan To: qinyuntan@linux.alibaba.com, Tony Luck , Reinette Chatre , x86@kernel.org Cc: Dave Martin , James Morse , Babu Moger , Xunlei Pang , linux-kernel@vger.kernel.org Subject: [BUG] resctrl: using smp_processor_id() in preemptible code in __l3_mon_event_count() via mbm_handle_overflow() during CPU hotplug Date: Thu, 11 Jun 2026 15:18:47 +0800 Message-ID: <20260611071847.1552322-1-qinyuntan@linux.alibaba.com> X-Mailer: git-send-email 2.43.7 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Hi all, While stress-testing resctrl under heavy CPU hotplug on an AMD platform with a DEBUG_PREEMPT/LOCKDEP kernel, I hit a recurring splat originating from the MBM overflow handler. Analysis suggests it is a latent issue in the generic fs/resctrl code (not AMD specific), and it is still present in current mainline (the code is identical). Environment: - Kernel 6.6.x + CONFIG_DEBUG_PREEMPT=y, LOCKDEP=y, PROVE_LOCKING=y - x86 platform with multiple L3 monitor domains, MBM enabled - Trigger: continuous CPU online/offline storm while MBM monitoring runs Splat (representative): BUG: using smp_processor_id() in preemptible [00000000] code: kworker/225:1/3750 caller is __l3_mon_event_count+0x73/0xb70 CPU: 432 PID: 3750 Comm: kworker/225:1 ... +debug Workqueue: events mbm_handle_overflow Call Trace: check_preemption_disabled+0xd1/0xe0 __l3_mon_event_count+0x73/0xb70 __mon_event_count+0x1c4/0x940 mbm_update_one_event+0xc2/0x300 mbm_handle_overflow+0x115/0x2f0 process_one_work+0x814/0x1790 worker_thread+0x726/0x1320 ... Note "kworker/225:1" running on CPU 432: the per-CPU worker bound to CPU225 was unbound and migrated when CPU225 went offline. Root cause: - mbm_over is a per-domain delayed_work, scheduled via schedule_delayed_work_on(d->mbm_work_cpu, ...), i.e. a bound work that is expected to run on a CPU of domain @d. - __l3_mon_event_count() reads per-CPU MBM MSRs and therefore relies on running on a CPU of @d. It does: int cpu = smp_processor_id(); ... if (!cpumask_test_cpu(cpu, &d->hdr.cpu_mask)) return -EINVAL; - In the read-from-sysfs path this invariant is provided by smp_call_function_any() (IPI, preemption disabled, runs on a domain CPU). In the overflow path the invariant is provided only *implicitly* by the work being a bound per-CPU kworker (is_percpu_thread() exempts smp_processor_id()). - When d->mbm_work_cpu goes offline, the workqueue unbinds the per-CPU worker (is_percpu_thread() becomes false) and the pending/just-woken work runs on a foreign CPU. The implicit invariant breaks: (a) smp_processor_id() in preemptible context -> DEBUG_PREEMPT splat; (b) the foreign CPU is not in d->hdr.cpu_mask -> cpumask_test_cpu() fails -> the MBM read for that domain/tick is silently skipped. cpus_read_lock() held in mbm_handle_overflow() does not help: the unbind/migration happens before the lock is taken, and the lock neither disables preemption nor prevents migration. Impact (believed low / mostly harmless): - Self-healing: at the end of each tick mbm_handle_overflow() re-picks an online d->mbm_work_cpu and reschedules, so the next tick runs correctly. - Production kernels (DEBUG_PREEMPT=n): no warning; at worst one missed MBM update for the affected domain on the hotplug tick. No crash. - No cross-domain corruption: @d is fixed via container_of(); the cpumask_test_cpu() guard turns the foreign-CPU case into a skip, not a bad read. - So the practical damage is a transient one-tick MBM accounting gap plus DEBUG_PREEMPT noise under hotplug; but the noise can mask other splats in CI/debug kernels, which is why I am reporting it. Reproduced reliably only by forcing this rare window at high frequency (continuous hotplug + active MBM). I do not have a good fix in mind. The read in __l3_mon_event_count() fundamentally assumes it runs on a CPU of the domain, but during hotplug the overflow work can be migrated off that CPU; neither the cpus_read_lock() held here nor the existing cpumask_test_cpu() guard addresses the preemptible-context use of smp_processor_id() itself. I would appreciate your guidance on how this should best be addressed. I can provide the full log and a reproducer on request. Thanks, Qinyun Tan