From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE8BC424664 for ; Wed, 12 Aug 2026 12:18:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786537121; cv=none; b=TsnZno8wc8Pw709anCT4cLxawJM6hucz6H5fxcYv5a/0TprZmQ6ifMTmrCmVxxD41E2HUBfBhEvC9Yu3MTczQsqfGSao88XO1jXdwl1UtithLT9Cqxm5xuNE6SZEydAx1Xzx0ty2abtmVSFQ5MJdmv60xTxXL6kmOxm8yvVrCOU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786537121; c=relaxed/simple; bh=YHjzv4WB4NI+PX/Via7hripVyOrsxTsZTgw7P+y4oXc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jWBouIAJpGEkJr7V20+Qj5cvoAOvGc3X5x8IadQcUamUjTf2hralacf7abvVG1q4md2vLWvtuDoOYGTEgb+2227/Q7VVLw3pQ2wnVZphbYuVLD7aLUCsEMdT+Cf8LYcr6mt26WgefoNtFILqEGG41JxdZDVUM0Mp99oIbPI/524= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=JkxTYDQC; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="JkxTYDQC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A29271F000E9; Wed, 12 Aug 2026 12:18:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1786537120; bh=c/HS6jeoNkCyji18aZKv6rudVAWLmw8SzAirPydXjGI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=JkxTYDQCLOuxfjIVK740eZCL70XL0BrkAWHVF8LXo5EPShGtbz6Wlsgzcn9T8kzjh 8Zka2Q3iylt2GAX9xst0IU8Y/Rcx5fCLyE+KF005pPWfFdu8KtlUCNPFACO4dfA3DX ElrwzhHLbC5iGlbsLcI4M8ojeYUzgn1Vc5w34z/tsQWKkpFUYbovv04YXLaUvBEN66 Nj3g34foMtUhtdisxEiU1aZD/VD73VXAj3c/YuOiYcaIWYZgBWnIgeWkUnRuO6WcIn OMhxsX0gcoe/ZdgTA83FBiTH7O0XFRvPoa9WEpVx1EUbGFNZYcUNeCCVQK4RbqJaPy ethfRhdyn/pcw== From: "Barry Song (Xiaomi)" To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: axelrasmussen@google.com, chenridong@xiaomi.com, david@kernel.org, hannes@cmpxchg.org, kasong@tencent.com, lianux.mm@gmail.com, linux-kernel@vger.kernel.org, ljs@kernel.org, lyugaofei@xiaomi.com, mhocko@kernel.org, qi.zheng@linux.dev, shakeel.butt@linux.dev, stevensd@chromium.org, wangzicheng@honor.com, weixugc@google.com, yuanchu@google.com, zhangbo56@xiaomi.com, baolin.wang@linux.alibaba.com, baoquan.he@linux.dev, "Barry Song (Xiaomi)" Subject: [RFC PATCH v4 09/16] mm/mglru: move folios from oldest gen to second-oldest gen from head to tail Date: Wed, 12 Aug 2026 20:16:51 +0800 Message-Id: <20260812121658.69965-10-baohua@kernel.org> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260812121658.69965-1-baohua@kernel.org> References: <20260812121658.69965-1-baohua@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit For reclamation, it makes sense to reclaim folios from tail to head, as folios near the head are relatively hot. However, when moving folios from the oldest generation to the second-oldest generation, using the tail-to-head order would effectively cause a cold/hot inversion. Signed-off-by: Barry Song (Xiaomi) --- mm/vmscan.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 27623d3bad95..17357b16d1a7 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -190,8 +190,20 @@ struct scan_control { prefetchw(&prev->_field); \ } \ } while (0) +#define prefetchw_next_lru_folio(_folio, _base, _field) \ + do { \ + if ((_folio)->lru.next != _base) { \ + struct folio *next; \ + \ + next = list_entry((_folio)->lru.next, \ + struct folio, lru); \ + prefetchw(&next->_field); \ + } \ + } while (0) + #else #define prefetchw_prev_lru_folio(_folio, _base, _field) do { } while (0) +#define prefetchw_next_lru_folio(_folio, _base, _field) do { } while (0) #endif /* @@ -3931,9 +3943,10 @@ static bool inc_min_seq(struct lruvec *lruvec, int type, int swappiness) for (zone = 0; zone < MAX_NR_ZONES; zone++) { struct list_head *head = &lrugen->folios[old_gen][type][zone]; unsigned long protected[MAX_NR_TIERS] = {}, delta = 0; + struct list_head *pos = head->next; - while (!list_empty(head)) { - struct folio *folio = lru_to_folio(head); + while (pos != head) { + struct folio *folio = list_entry(pos, struct folio, lru); long nr_pages = folio_nr_pages(folio); int refs = folio_lru_refs(folio); bool workingset = folio_test_workingset(folio); @@ -3944,6 +3957,8 @@ static bool inc_min_seq(struct lruvec *lruvec, int type, int swappiness) VM_WARN_ON_ONCE_FOLIO(folio_is_file_lru(folio) != type, folio); VM_WARN_ON_ONCE_FOLIO(folio_zonenum(folio) != zone, folio); + prefetchw_next_lru_folio(folio, head, flags); + pos = pos->next; new_gen = __folio_inc_gen(folio, old_gen, &gen_increased); if (gen_increased) { delta += nr_pages; -- 2.34.1