From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9859387371 for ; Thu, 20 Aug 2026 04:57:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787201829; cv=none; b=JeD6Bn0vzggEWyuuW2MQqsK+0N6yNc23IbkpToANXMSQtm/Is+nG4GlAedLFyYPMqA55Jr+2WCQzruQ3YhmHp6fx+zucw53uFxY2Hc+CyqFaEHvnTsUw6l4CH4wIteG8N2TlvDdZ40TwTTGDiNkT8nMaVSVrsfIaUn09e8GOuP0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787201829; c=relaxed/simple; bh=TSZk6F9Zw8iSkjmTGUUIVvN5ixlyLuF9ep8kx50IgmE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fnTwpzUJVPuJR4rlIxyEQgKt5LXX57pdQScStTs8AWStK+cHdCx46Ls25FWQFvQsanhbnfHHQFTeE42FTUNbrXUQyMoqWtEUtZjukYjsuQDSGVQEVy/LkUPWppi1TPD3v0lBmVkt7n2nwTiU/0wxi59/eo7lwWyoNPJYQOGVhwI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WEu0ANyV; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WEu0ANyV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E08BD1F00A3D; Thu, 20 Aug 2026 04:57:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1787201827; bh=L/wIz6ML7okFnhQHn92A6v7algKS0sqhHpGu7c56urE=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=WEu0ANyVg5sufXEXgQXZ8Au81gfHVhKXozFhGTGmNlmE7UxANI9x5BG3Z2rDfPLkb j/r1D31sKJvTTs4HsmlK64aOy0CzjHta2ur1HdOq37AakH4TMc+/BD1VD6sx5ZRh6c sF50jzK1+3pZl/G4Z27dHqW6eh2HbvTtlPr7skKofsYJXZopMcNH1uIJu4OL7SK4XJ PrUQ760Joovr8SgYju+nlsYQxwNJhGbnsAcdlKH7OGINshuSU1kiFY/+WrEYk0SjGG yRV1Xv9XtG6mFWbJ9Tt6ko2TxBLG4vwKEVcktLC3MQOgt13+KAowh7VwLLlzJq3Jun ra5gjG+aCJCfQ== From: "Barry Song (Xiaomi)" To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: axelrasmussen@google.com, baolin.wang@linux.alibaba.com, baoquan.he@linux.dev, chenridong@xiaomi.com, david@kernel.org, hannes@cmpxchg.org, kasong@tencent.com, lianux.mm@gmail.com, linux-kernel@vger.kernel.org, ljs@kernel.org, lyugaofei@xiaomi.com, mhocko@kernel.org, qi.zheng@linux.dev, shakeel.butt@linux.dev, stevensd@chromium.org, wangzicheng@honor.com, weixugc@google.com, yuanchu@google.com, "Barry Song (Xiaomi)" Subject: [PATCH 2/3] mm/mglru: improve scan_folios() exhaustion detection Date: Thu, 20 Aug 2026 12:56:02 +0800 Message-Id: <20260820045603.68809-3-baohua@kernel.org> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260820045603.68809-1-baohua@kernel.org> References: <20260820045603.68809-1-baohua@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Commit 16b475d2ac3c ("mm/mglru: avoid reclaim type fall back when isolation makes no progress") uses scanned == 0 to determine whether scan_folios() has exhausted a reclaim type. However, this is not always sufficient. It is possible for scanned > 0, while the oldest reclaimable generation is exhausted after the first scan_folios() call. We detect early_stop in scan_folios(). If we stop early for any reason, it means the current reclaim type is not exhausted yet. If early_stop is never reached, it means we have exhausted the current oldest generation without hitting any scanning limit. Another issue is that if the lruvec has 4 generations, we might have exhausted the oldest generation while the second oldest generation is still reclaimable. In that case, this type is not exhausted yet. Add an exhausted output argument to scan_folios() so it can explicitly report whether the reclaimable lists for the current type have been exhausted. Signed-off-by: Barry Song (Xiaomi) --- mm/vmscan.c | 28 ++++++++++++++++++++++------ 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index d5cc30b667ad..1f2e574b0061 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4721,7 +4721,8 @@ static bool isolate_folio(struct lruvec *lruvec, struct folio *folio, struct sca static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec, struct scan_control *sc, int type, int tier, - struct list_head *list, int *isolatedp) + struct list_head *list, int *isolatedp, + bool *exhausted) { int i; int gen; @@ -4732,12 +4733,15 @@ static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec, int skipped = 0; unsigned long remaining = nr_to_scan; struct lru_gen_folio *lrugen = &lruvec->lrugen; + bool early_stop = false; VM_WARN_ON_ONCE(nr_to_scan > MAX_LRU_BATCH); VM_WARN_ON_ONCE(!list_empty(list)); - if (get_nr_gens(lruvec, type) == MIN_NR_GENS) + if (get_nr_gens(lruvec, type) == MIN_NR_GENS) { + *exhausted = true; return 0; + } gen = lru_gen_from_seq(lrugen->min_seq[type]); @@ -4768,8 +4772,10 @@ static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec, skipped_zone += delta; } - if (!--remaining || max(isolated, skipped_zone) >= MIN_LRU_BATCH) + if (!--remaining || max(isolated, skipped_zone) >= MIN_LRU_BATCH) { + early_stop = true; break; + } } if (skipped_zone) { @@ -4778,8 +4784,10 @@ static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec, skipped += skipped_zone; } - if (!remaining || isolated >= MIN_LRU_BATCH) + if (!remaining || isolated >= MIN_LRU_BATCH) { + early_stop = true; break; + } } item = PGSCAN_KSWAPD + reclaimer_offset(sc); @@ -4790,6 +4798,13 @@ static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec, scanned, skipped, isolated, type ? LRU_INACTIVE_FILE : LRU_INACTIVE_ANON); + /* + * If we didn't stop early, all reclaimable folios in the current + * generation have been scanned. We are exhausted if this is the last + * reclaimable generation. + */ + *exhausted = !early_stop && + lrugen->min_seq[type] + MIN_NR_GENS == lrugen->max_seq; *isolatedp = isolated; return scanned; } @@ -4847,11 +4862,12 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec, bool type_fallback_allowed = !is_single_type_reclaim(swappiness); int type = get_type_to_scan(lruvec, swappiness); int total_scanned = 0, scanned, tier; + bool exhausted; retry: tier = get_tier_idx(lruvec, type); scanned = scan_folios(nr_to_scan, lruvec, sc, - type, tier, list, isolated); + type, tier, list, isolated, &exhausted); total_scanned += scanned; if (*isolated) { @@ -4864,7 +4880,7 @@ static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec, * We are running out of the current reclaim type. Fall back to * the other type if allowed. */ - if (!scanned && type_fallback_allowed) { + if (exhausted && type_fallback_allowed) { type = !type; type_fallback_allowed = false; goto retry; -- 2.34.1