From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 352AC3B8D50 for ; Sun, 28 Jun 2026 21:45:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782683137; cv=none; b=fyP9GANatn+qkNPDfuVNLu9A0KtHtJni/084F8L96O+/CEUDAodoN6hy+21L7xtoAbJ6TT8PDaMGUljRsY56MImL1tV1YSfj24+uc1WfePbj3vyOa1gvOD+YQbC1qaYpJQ3nWhPHQ42c7gH8uOGj+TT+xD/DRGrjbsTCwbOBtuo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782683137; c=relaxed/simple; bh=BZC0rGREYDdz8ACkjyFyW5Qa+P9wYb1QujVYCeT996o=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Eoe+I2YifV/i2AmbD0nnqP0yqH7Gk3LnOhZiEhtmICfahkacsNCLKAKsr0cMIaqRGNuLx7Be3kC68fRnt8/uRBB65jbg3kDWq/IJ19PiThgdFA/txMoxCw+QaxiBoWPEXZ5wAJcG2bQm0Ao1E8lOElwKICFjwxt+x58VlWr48lU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ScNyX3oa; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=MvK6/5rN; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ScNyX3oa"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="MvK6/5rN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1782683135; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=SiJhyXhNBfsAzo8nGoH5kDNlW+30TFjfN78Hcf1BrTc=; b=ScNyX3oaWvNeraRba9l46q7kmnb7+N15lswlWZQ38h8drBfBhswyLvHcgR7jBuioPa/4Rr 4Xp2B/Y44glPYPQ73sQn1IVkWy2njGYrEumVsGwkLY6S/GQqmyheHbUk+7Lm+HRLaaJTV+ SKn5UCnwX3eCHLAReN42poUXrZiPe5g= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-78-1hXQdEC4NO2wjwyDD2tMBQ-1; Sun, 28 Jun 2026 17:45:33 -0400 X-MC-Unique: 1hXQdEC4NO2wjwyDD2tMBQ-1 X-Mimecast-MFC-AGG-ID: 1hXQdEC4NO2wjwyDD2tMBQ_1782683133 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-47132f8a98aso1238213f8f.1 for ; Sun, 28 Jun 2026 14:45:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1782683132; x=1783287932; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=SiJhyXhNBfsAzo8nGoH5kDNlW+30TFjfN78Hcf1BrTc=; b=MvK6/5rNdfREaJJv8tyikinnG+Puh3pARhCbPX6zMDqxS3XOx+WbZ+0CgnGaVQUfKn AfyQkC2bNiIZ3MtvlDv1YLnj8HvlnlUasOcmgXnNiw9tbK1EqyeE3JAIIwXGRNumtmEG 8tUWr+hrBv8VehZLkDpKj0jEjC2q7MKrs/dOaZI6StlPY6IKNiBRgYNi5GjOiU9oGJVg XdlhhhHgvBKA0sVDMZZr0RVpDINVEH/8b+g37CTtlEVEuNxtoPnaRGfrWpmrTGKdtomG HZ/vtqThNK7v26UNutfPKBwUL5u5FI0ncrUsj3raFG1Fuj7cJW8cRoB6eUzqcFeinck8 RDPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782683132; x=1783287932; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SiJhyXhNBfsAzo8nGoH5kDNlW+30TFjfN78Hcf1BrTc=; b=Il3LvogC2cMb2GLpmBN6Wj5MSIm3n6q/T9q7Arjq71jKiUwzYBt3XjXCKldijIy1ix 86Fs1/uWpjetL0NptyAfjbAYqEFQIkflWTUnUgkAM5RopIJ/g0LD/yfkM8uK3AJ8X6Or pke40LZAI351q+GUrXXtzqwZsi9jCOKHKyqF6cyr13eZS+vHs1njp+Ra4+xPzGe/5Kmv M5S4wUllxtMpfGqXgbEKvtZLAge+aMmPm8aY2xV56sM61uzHZA7lVVE81X/OzQhOBKXI Xx/PKAvsOCKw2LDDol2jolItkpCcUw+VwmdbYZNAi82HkONPqJPTRxurFZaT9Lx89vL2 CmNw== X-Gm-Message-State: AOJu0Yy6vKVAM4vwaHKflu7o+hKXDAMdp4hWXPY0DJsMEU8b0h7zHIkJ GV6a9E/pexDa0agtT6h2/udju9kJrbhf8DatoFqfxZNziuhxAXUshZs1fC7wGaYfNGJbq7hDK8R CoTxB4P4KWdlGhOTjMdBv/F0LBJcWNFuWeTtUGH4jaVw0+DrL715Ox91U5i89dkZ+3ZuEhWHLbl 2xi1A8hQNGHwTFTfz+rTAP0d3uBSueMPGOUa9pdiJ8gIs= X-Gm-Gg: AfdE7clGok0aBIpGojhdCbJeK7JvDcrROltqKC6IuEDG0drYfCKf2A2f/jn9BE1FKLs GMQlbsuvX4+dwqoqt3yamFQxTByIMId09HXqNSwkXY1RqyUdnzIxcLvLsa5rmwPAZkbj92d9Cq5 6NNZleoyJ/yERh7twGtXpof7+0ipciXWX0u5PeSliIQgVTiWddOecYaUddVrPDhm7c7lQM/Id3u jEaHUexlJMvRZVt+S29vUt3cmx8ViZZsm52vj0FYZ5GfZN29WMDgo2oRaKODM7dzKy8bnYrvMvE 3WUVQ5Bq7RO08qB6vNfaBkylcSm6oaCX9XKG9Avd5qln+3oWfR3aeOoWzgO1RIbjpqwc92wfCeX d X-Received: by 2002:a05:6000:4285:b0:45e:fa7b:a7d2 with SMTP id ffacd0b85a97d-46fb6bf2caamr14037859f8f.7.1782683132521; Sun, 28 Jun 2026 14:45:32 -0700 (PDT) X-Received: by 2002:a05:6000:4285:b0:45e:fa7b:a7d2 with SMTP id ffacd0b85a97d-46fb6bf2caamr14037797f8f.7.1782683131891; Sun, 28 Jun 2026 14:45:31 -0700 (PDT) Received: from redhat.com ([31.187.78.70]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-46f978dc0a9sm20747231f8f.15.2026.06.28.14.45.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 28 Jun 2026 14:45:31 -0700 (PDT) Date: Sun, 28 Jun 2026 17:45:27 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: David Hildenbrand , Miaohe Lin , Naoya Horiguchi , Andrew Morton , Oscar Salvador , Andi Kleen , Hidehiro Kawai , Rik van Riel , Vlastimil Babka , Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Hao Li , Kiryl Shutsemau , Byungchul Park , linux-mm@kvack.org, linux-cxl@vger.kernel.org, David Hildenbrand Subject: [PATCH 1/2] mm: memory-failure: use RCU to fix HWPoison flag race Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Non-atomic page flag operations (page->flags.f &= ~mask, __set_bit, __clear_bit) can race with atomic TestSetPageHWPoison() in memory_failure(). The non-atomic RMW reads flags, memory_failure() atomically sets HWPoison, then the RMW writes back the old value without HWPoison - clobbering the bit. Add synchronize_rcu() + retry helpers for setting and clearing HWPoison, and convert all memory_failure() call sites to use them. Follow-up patches wrap non-atomic page flag operations in rcu_read_lock/rcu_read_unlock so that synchronize_rcu() drains in-flight callers. Note: the MCE handler in arch/x86/kernel/cpu/mce/core.c also calls SetPageHWPoison() and is subject to the same race. It cannot use the drain helpers (MCE context cannot call synchronize_rcu()). For recoverable MCE errors, memory_failure() is queued via work items (kill_me_maybe/kill_me_never) and will re-set the bit via test_and_set_hwpoison_drain_rcu() if it was clobbered. The mce_panic() path sets HWPoison for kdump right before panic() so the race should not matter there. The MCG_STATUS_SEAM_NR path does not queue memory_failure(), but the affected page belongs to a TDX guest whose CPU core has already been marked dead - the page is not subject to concurrent non-atomic flag operations in the buddy allocator, so the race does not trigger. Fixes: 6a46079cf57a ("HWPOISON: The high level memory error handler in the VM v7") Suggested-by: David Hildenbrand Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: Cursor:gpt-5.4-xhigh-fast --- mm/memory-failure.c | 54 ++++++++++++++++++++++++++++++++++++++------- 1 file changed, 46 insertions(+), 8 deletions(-) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index ee42d4361309..351f8bbda248 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -76,6 +76,44 @@ static int sysctl_enable_soft_offline __read_mostly = 1; atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); +/* + * Drain any in-flight non-atomic page flag operations that could + * clobber a concurrently set HWPoison bit. Retries until the bit sticks. + */ +static void set_hwpoison_drain_rcu(struct page *p) +{ + do { + synchronize_rcu(); + } while (!TestSetPageHWPoison(p)); +} + +/* + * Drain any in-flight non-atomic page flag operations that could + * restore the HWPoison bit from stale data. Retries until it stays clear. + */ +static void clear_hwpoison_drain_rcu(struct page *p) +{ + do { + synchronize_rcu(); + } while (TestClearPageHWPoison(p)); +} + +static bool test_and_set_hwpoison_drain_rcu(struct page *p) +{ + bool was_set = TestSetPageHWPoison(p); + + set_hwpoison_drain_rcu(p); + return was_set; +} + +static bool test_and_clear_hwpoison_drain_rcu(struct page *p) +{ + bool was_set = TestClearPageHWPoison(p); + + clear_hwpoison_drain_rcu(p); + return was_set; +} + static bool hw_memory_failure __read_mostly = false; static DEFINE_MUTEX(mf_mutex); @@ -211,7 +249,7 @@ static bool page_handle_poison(struct page *page, bool hugepage_or_freepage, boo return false; } - SetPageHWPoison(page); + set_hwpoison_drain_rcu(page); if (release) put_page(page); page_ref_inc(page); @@ -1756,7 +1794,7 @@ static int mf_generic_kill_procs(unsigned long long pfn, int flags, * Use this flag as an indication that the dax page has been * remapped UC to prevent speculative consumption of poison. */ - SetPageHWPoison(&folio->page); + set_hwpoison_drain_rcu(&folio->page); /* * Unlike System-RAM there is no possibility to swap in a @@ -1801,7 +1839,7 @@ int mf_dax_kill_procs(struct address_space *mapping, pgoff_t index, goto unlock; if (!pre_remove) - SetPageHWPoison(page); + set_hwpoison_drain_rcu(page); /* * The pre_remove case is revoking access, the memory is still @@ -1878,7 +1916,7 @@ static unsigned long __folio_free_raw_hwp(struct folio *folio, bool move_flag) head = llist_del_all(raw_hwp_list_head(folio)); llist_for_each_entry_safe(p, next, head, node) { if (move_flag) - SetPageHWPoison(p->page); + set_hwpoison_drain_rcu(p->page); else num_poisoned_pages_sub(page_to_pfn(p->page), 1); kfree(p); @@ -2390,7 +2428,7 @@ int memory_failure(unsigned long pfn, int flags) if (hugetlb) goto unlock_mutex; - if (TestSetPageHWPoison(p)) { + if (test_and_set_hwpoison_drain_rcu(p)) { res = -EHWPOISON; if (flags & MF_ACTION_REQUIRED) res = kill_accessing_process(current, pfn, flags); @@ -2420,7 +2458,7 @@ int memory_failure(unsigned long pfn, int flags) } else { /* We lost the race, try again */ if (retry) { - ClearPageHWPoison(p); + clear_hwpoison_drain_rcu(p); retry = false; goto try_again; } @@ -2441,7 +2479,7 @@ int memory_failure(unsigned long pfn, int flags) /* filter pages that are protected from hwpoison test by users */ folio_lock(folio); if (hwpoison_filter(p)) { - ClearPageHWPoison(p); + clear_hwpoison_drain_rcu(p); folio_unlock(folio); folio_put(folio); res = -EOPNOTSUPP; @@ -2761,7 +2799,7 @@ int unpoison_memory(unsigned long pfn) } folio_put(folio); - if (TestClearPageHWPoison(p)) { + if (test_and_clear_hwpoison_drain_rcu(p)) { folio_put(folio); ret = 0; } -- MST