From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-yw1-f174.google.com (mail-yw1-f174.google.com [209.85.128.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DECAB430CE5 for ; Mon, 24 Aug 2026 14:04:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.174 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787580242; cv=none; b=AvdjnDI7iiDMQSmNLrbBmdhRO+bMmhwvKCI/2HezaDlLRm/3EoIiPBxWXDRJsdiOWupzyCkpKN61Zr50RJlXb9UHrsyC/67mll2fAMFJ5/IPMFnTF51ARcky0dKOv72HSQUuEMID4He2PGLwjuIFwSGcVUQr7X04z5wjkuFp/JM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787580242; c=relaxed/simple; bh=LN4S+uhNeXEkUev4MtTwRUrSM4Q5tR/xe+5lbgofppg=; h=Date:From:To:cc:Subject:In-Reply-To:Message-ID:References: MIME-Version:Content-Type; b=fLAg7Ulhz0oL4OLr0MM2PwTjAEKIJubFBiO1omBQEqGDj3qzayO3n9eSdNquqOAQjxaHgoPZaJ4uWSAYED/cjdPlKttkqTLv5H55Msyw8p5CD+R8F49fsS3s8Xt62Vne8hkYipw06TT1gOtFGLYJ/2LLa/ARXJmpo+eMZPjS+xQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=rtl52Ag6; arc=none smtp.client-ip=209.85.128.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="rtl52Ag6" Received: by mail-yw1-f174.google.com with SMTP id 00721157ae682-836cb2fa1bcso52633077b3.1 for ; Mon, 24 Aug 2026 07:04:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1787580240; x=1788185040; darn=vger.kernel.org; h=content-type:mime-version:references:message-id:in-reply-to:subject :cc:to:from:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=gLqnUltfGMwKBemLbA/06+FuqdHtLjtQ/H9OD8DkJLA=; b=rtl52Ag6X87b4zggXTJD7rgDtSYz03nSHK2+Pnjivk9XsTrG/xr0dPmc00ofbZztON G6/piNskoQTvXOwyCzz2tTj1fcti2eCb3O/cKFy5ejNiy/nZspB4YvWAf7ad7c1GbCx3 8Vxd3w/SWRN9LlPKXU70AdbcYilK3QfI9rtujvxBb96jDNmSNgrbbecDXiIdROQTVOW1 GzDdjcg48xiPXr/AKK1YS2VMOJ01t5Z8qMiMb496nMlIpIJYFFB+Pt8l/iiMXhommpWQ fa6ZCXVE1OwyXsL5fcId8kProLFoFo9xNuM9Qtd5EE2JLDfMHQn9cEeAXo1bN6wAY1nR b/+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787580240; x=1788185040; h=content-type:mime-version:references:message-id:in-reply-to:subject :cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=gLqnUltfGMwKBemLbA/06+FuqdHtLjtQ/H9OD8DkJLA=; b=qOP/z9ZoaTLXhRTxCx38KOZ3j5RRaO+Myy5YaiNm7ymIvvuZdyCmKH58nCEAnYHTdR Go32B2d1xQ/CMk6GNPR5h+DAJ5WQrIYu2uhnEkGNFFBqy1U1yTBG6lA8WG47gCEtPWz6 h0seCgavAjyRpkSmlS+Kb0n7/KCo7LB6Ezbb96qujxytxfqHWuu9+xb+lQnisNKYNbeG CCG8dje2md0z0MIk5tprbVYQ4xyQK/QF1WSZbz6YqQCP9HX/NgVzmbwc7NBZehONsPCn NqVGGkBEHzdzbNHXnRWXEQ/efQg/NwFIq26dTPiIaNWF5wFB/Dn7yW+tD8aeWU0Sb9RR ddMA== X-Forwarded-Encrypted: i=1; AHgh+RrHIfhqhDyjIMIv59YyzrdCObfXvPhbIjXdUHUIZ0ZZN7YDnyQElWXnByFVgfOZxy2ilS64tRRRsBSxO1Q=@vger.kernel.org X-Gm-Message-State: AFuF++ksg8AtVtD4UNLV0jpoAWfGO53E2QcYc9xrYzbhvoqOBAdLBmBK hLt+PH/KA/Q9ZaXzDxgw2SHnQ6PyTd5OFRS/4qDytbybaISA+hcXsnKR1FG4FRkYrA== X-Gm-Gg: AR+sD11+dIkoKe5sHgisJI2DMXvHsuyLB9qWiIY2VM18GQIYS09wGy3jX8n60Zlefew l10j+t+nKXrZE1fsPB+iO8pN5migcW/rU5lnCI9F/Z+KebEdsQhrxHUjv/YEzkCcX6h14JsmdtA zctNDGZHcMm5mfy+mfWdyDWdvYO6bdYd15dPywQvVr4T9ArkAZCFbCSENqVvMa7bQHB0bG2UcdT sDqSd+Wp1/vhpaKX8vsLlbPm2EcvwNXjYexhkSy2x1WvYPG3UcFIgI84ftHrX9IbKYgLU0veETG YnQVdLrPqWs3W0owA/mu/AU+I5D2+CH/s4fXzJeR3q9aRk79PIxFQxbAr1FHdjjfI/MFUs6dtvU jLdthWBL3PNRr9L3jGR705ir4MRyVEtqxM4hh26wfAIjsEuU1BEX69IhTZXaKokXxrZgde/aUwi 8C7bMxbhtFyvWa6ysxwTM8ZqghFFT128A4RaOF+qO1+MLRQrFWD8p6VbQvxuSnBPotEJWYTT3cv wq2tWwwZER1HbgQ3+uHJGigpGWigVkmIlo+PIbZyosSE0dk X-Received: by 2002:a05:690c:6004:b0:822:854d:4563 with SMTP id 00721157ae682-84c9682009cmr74201097b3.8.1787580237568; Mon, 24 Aug 2026 07:03:57 -0700 (PDT) Received: from darker.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id 00721157ae682-84cab851290sm33478997b3.35.2026.08.24.07.03.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Aug 2026 07:03:56 -0700 (PDT) Date: Mon, 24 Aug 2026 07:03:52 -0700 (PDT) From: Hugh Dickins To: Andrew Morton cc: Ackerley Tng , Alexander Viro , Baolin Wang , Barry Song , Binbin Wu , Christian Brauner , Christoph Hellwig , Christoph Lameter , Claudio Imbrenda , David Hildenbrand , JP Kobryn , Jan Kara , Jens Axboe , Johannes Weiner , Kairui Song , Kiryl Shutsemau , Lance Yang , Leonardo Bras , Lorenzo Stoakes , Marcelo Tosatti , Matthew Wilcox , Mel Gorman , Miaohe Lin , Michal Hocko , Minchan Kim , Muchun Song , Oscar Salvador , Peter Zijlstra , Qi Zheng , Rik van Riel , Sebastian Andrzej Siewior , Shakeel Butt , Suren Baghdasaryan , Vlastimil Babka , Yang Shi , Yu Zhao , Zach O'Keefe , Zi Yan , linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 05/25] mm/fbatch: lru_add_del_folio()+folio_add_lru() after clear_lru() In-Reply-To: <14a16945-529b-8bc0-ab38-3ea97e54e223@google.com> Message-ID: <9c538ff2-7150-1021-df1e-fbc10f0bb92a@google.com> References: <14a16945-529b-8bc0-ab38-3ea97e54e223@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Most callers of folio_test_clear_lru() then proceed to remove the folio from its lru, and add it back at the end when they're done (if still in use). But isolate_migratepages_block() and check_move_unevictable_pages() sometimes decide against, and release immediately with a folio_set_lru(). Which usually works fine: but there's now a small chance that while they held the folio with lru bit cleared, an lru_add fbatch drain came along, and had to skip that folio because its lru bit was transiently cleared (previously, the lru_add fbatch drain relied on finding lru bit never yet set). This risks leaving that folio off lru, unreclaimable until freed. Fix such cases by trying lru_add_del_folio() (which only takes action and returns true if the folio was on an lru_add fbatch), then folio_add_lru() if it succeeded: invalidating the old fbatch slot, appending in a new one. Signed-off-by: Hugh Dickins --- mm/compaction.c | 13 +++++++++++-- mm/vmscan.c | 16 +++++++++------- 2 files changed, 20 insertions(+), 9 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index a049415512c6..9e045a90ba21 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1204,7 +1204,13 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, !cc->alloc_contig)) { low_pfn += folio_nr_pages(folio) - 1; nr_scanned += folio_nr_pages(folio) - 1; - folio_set_lru(folio); + if (lru_add_del_folio(folio)) { + lruvec_unlock_irqrestore(locked, flags); + folio_add_lru(folio); + locked = NULL; + } else { + folio_set_lru(folio); + } goto isolate_fail_put; } } @@ -1293,7 +1299,10 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, if (locked) lruvec_unlock_irqrestore(locked, flags); if (folio) { - folio_set_lru(folio); + if (lru_add_del_folio(folio)) + folio_add_lru(folio); + else + folio_set_lru(folio); folio_put(folio); } diff --git a/mm/vmscan.c b/mm/vmscan.c index 4ca9775ceee8..7da12ffbd3bd 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -8021,17 +8021,19 @@ void check_move_unevictable_folios(struct folio_batch *fbatch) folio_clear_unevictable(folio); lruvec_add_folio(lruvec, folio); pgrescued += nr_pages; + } else if (lru_add_del_folio(folio)) { + lruvec_unlock_irq(lruvec); + folio_add_lru(folio); + lruvec = NULL; } - folio_set_lru(folio); + if (lruvec) + folio_set_lru(folio); } - if (lruvec) { - __count_vm_events(UNEVICTABLE_PGRESCUED, pgrescued); - __count_vm_events(UNEVICTABLE_PGSCANNED, pgscanned); + if (lruvec) lruvec_unlock_irq(lruvec); - } else if (pgscanned) { - count_vm_events(UNEVICTABLE_PGSCANNED, pgscanned); - } + count_vm_events(UNEVICTABLE_PGRESCUED, pgrescued); + count_vm_events(UNEVICTABLE_PGSCANNED, pgscanned); } EXPORT_SYMBOL_GPL(check_move_unevictable_folios); -- 2.51.0