From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out30-110.freemail.mail.aliyun.com (out30-110.freemail.mail.aliyun.com [115.124.30.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C6B432F748 for ; Thu, 16 Jul 2026 10:46:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.110 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784198803; cv=none; b=P20A4xKMRJfz0p/wF5/0A8x9y1Hm6P0yU8Y1yO9FHODaWj8xrU1ress81qppYEQunov6v9pvytDHgxxefygaghTPaTlWehS6zxINWbb5EQxlHZQQmeQj9Zq8CkDXNkX4UELT6Dwkc0fAHjXl69DLCnVY/LvpzIyGV+Zagh0PRfQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784198803; c=relaxed/simple; bh=kgvvFk1/N8koG+ixTLGNRHAnuvwgpOcuS8hcolM2npA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bs+PO6wTeeNdUNmzAnULxqjD8Heq4pUeC1y16dpgP9sav4n9lfrL8Hh8ttRXYL2Uq59rGa1yjVmr4m/yKhLEdhVsSmdThqF7J0TjbOBe63wdz3hl09uEhS9CVf8o5/eFr0YbZD8Y6xSy/BdpIyS3vkmNF6nWSjFpxotPBBZ+fBo= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=ak401igr; arc=none smtp.client-ip=115.124.30.110 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="ak401igr" DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1784198792; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=Y8dvsppUCclol6edmZSY/pkLFvmywa1k8Hyk3xrw9tY=; b=ak401igrhkpqvz3zfDbdDWENe8yLVqBPFuNVSR5DM6r4x5Dzki3c08egoi4hof0MK4TTiXAX2MJrLXYvrTZTS314RRCkT7vhzAvFc8O7tgf46waHwz6LwLCk3n0S1SZNRb0+zEIczJHW1GoD2zvXLriWIA8fe95OWK/XTKSg3q0= X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R171e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=maildocker-contentspam011083073210;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=21;SR=0;TI=SMTPD_---0X7E4pTJ_1784198790; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0X7E4pTJ_1784198790 cluster:ay36) by smtp.aliyun-inc.com; Thu, 16 Jul 2026 18:46:30 +0800 From: Baolin Wang To: akpm@linux-foundation.org, david@kernel.org, ljs@kernel.org, hannes@cmpxchg.org Cc: riel@surriel.com, liam@infradead.org, vbabka@kernel.org, harry@kernel.org, jannh@google.com, lance.yang@linux.dev, kasong@tencent.com, qi.zheng@linux.dev, shakeel.butt@linux.dev, baohua@kernel.org, axelrasmussen@google.com, yuanchu@google.com, weixugc@google.com, mhocko@kernel.org, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/2] mm: mglru: promote mapped executable folios after first usage Date: Thu, 16 Jul 2026 18:46:19 +0800 Message-ID: <7e3a5f6bdf70e0237027025f4593a3874d411f65.1784197559.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Classical LRU protects mapped executable file folios through commit 8cab4754d24a0 ("vmscan: make mapped executable pages the first class citizen") and commit c909e99364c8 ("vmscan: activate executable pages after first usage"), giving executable code a better chance to stay in memory, avoiding IO thrashing and improving workload performance. However, MGLRU's protection of mapped executable file folios is less reliable. Although shrink_folio_list() checks references, the access flag of mapped executable file folios may have already been checked and cleared by lru_gen_look_around() or walk_mm(). Additionally, folio_update_gen() or lru_gen_set_refs() only sets the 'PG_referenced' flag for mapped executable file folios, which causes shrink_folio_list() to ignore the first usage of these mapped executable file folios and reclaim them easily. Follow the classical LRU's logic, promoting mapped executable file folios after their first usage in folio_update_gen() and lru_gen_set_refs(), giving executable code a better chance to stay in memory. On my 32-core Arm machine, with the memcg limit set to 2G, running 'make -j32' to build kernel showed some improvement in sys time. base patched 9248.543s 7861.579s Signed-off-by: Baolin Wang --- include/linux/mm_inline.h | 6 ++++++ mm/vmscan.c | 41 ++++++++++++++++++++++++--------------- 2 files changed, 31 insertions(+), 16 deletions(-) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index b5c4dc0f3fe3..30fdf1056312 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -30,6 +30,12 @@ static inline int folio_is_file_lru(const struct folio *folio) return !folio_test_swapbacked(folio); } +static inline bool folio_is_exec_file(const struct folio *folio, + const vma_flags_t *vma_flags) +{ + return vma_flags_test(vma_flags, VMA_EXEC_BIT) && folio_is_file_lru(folio); +} + static __always_inline void __update_lru_size(struct lruvec *lruvec, enum lru_list lru, enum zone_type zid, long nr_pages) diff --git a/mm/vmscan.c b/mm/vmscan.c index de62899c108d..fa808721b64d 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -835,11 +835,15 @@ enum folio_references { * with PG_active set. In contrast, the aging (page table walk) path uses * folio_update_gen(). */ -static bool lru_gen_set_refs(struct folio *folio) +static bool lru_gen_set_refs(struct folio *folio, const vma_flags_t *vma_flags) { /* see the comment on LRU_REFS_FLAGS */ if (!folio_test_referenced(folio) && !folio_test_workingset(folio)) { set_mask_bits(&folio->flags.f, LRU_REFS_MASK, BIT(PG_referenced)); + /* Activate file-backed executable folios after first usage. */ + if (folio_is_exec_file(folio, vma_flags)) + return true; + return false; } @@ -851,7 +855,7 @@ static bool lru_gen_set_refs(struct folio *folio) return true; } #else -static bool lru_gen_set_refs(struct folio *folio) +static bool lru_gen_set_refs(struct folio *folio, const vma_flags_t *vma_flags) { return false; } @@ -886,7 +890,7 @@ static enum folio_references folio_check_references(struct folio *folio, if (!referenced_ptes) return FOLIOREF_RECLAIM; - return lru_gen_set_refs(folio) ? FOLIOREF_ACTIVATE : FOLIOREF_KEEP; + return lru_gen_set_refs(folio, &vma_flags) ? FOLIOREF_ACTIVATE : FOLIOREF_KEEP; } referenced_folio = folio_test_clear_referenced(folio); @@ -914,7 +918,7 @@ static enum folio_references folio_check_references(struct folio *folio, /* * Activate file-backed executable folios after first usage. */ - if (vma_flags_test(&vma_flags, VMA_EXEC_BIT) && folio_is_file_lru(folio)) + if (folio_is_exec_file(folio, &vma_flags)) return FOLIOREF_ACTIVATE; return FOLIOREF_KEEP; @@ -2119,7 +2123,7 @@ static void shrink_active_list(unsigned long nr_to_scan, * IO, plus JVM can create lots of anon VM_EXEC folios, * so we ignore them here. */ - if (vma_flags_test(&vma_flags, VMA_EXEC_BIT) && folio_is_file_lru(folio)) { + if (folio_is_exec_file(folio, &vma_flags)) { nr_rotated += folio_nr_pages(folio); list_add(&folio->lru, &l_active); continue; @@ -3188,7 +3192,7 @@ static bool positive_ctrl_err(struct ctrl_pos *sp, struct ctrl_pos *pv) ******************************************************************************/ /* promote pages accessed through page tables */ -static int folio_update_gen(struct folio *folio, int gen) +static int folio_update_gen(struct folio *folio, int gen, const vma_flags_t *vma_flags) { unsigned long new_flags, old_flags = READ_ONCE(folio->flags.f); @@ -3196,10 +3200,15 @@ static int folio_update_gen(struct folio *folio, int gen) /* see the comment on LRU_REFS_FLAGS */ if (!folio_test_referenced(folio) && !folio_test_workingset(folio)) { + /* Activate file-backed executable folios after first usage. */ + if (folio_is_exec_file(folio, vma_flags)) + goto promote; + set_mask_bits(&folio->flags.f, LRU_REFS_MASK, BIT(PG_referenced)); return -1; } +promote: do { /* lru_gen_del_folio() has isolated this page? */ if (!(old_flags & LRU_GEN_MASK)) @@ -3428,8 +3437,8 @@ static bool suitable_to_scan(int total, int young) return young * n >= total; } -static void walk_update_folio(struct lru_gen_mm_walk *walk, struct folio *folio, - int new_gen, bool dirty) +static void walk_update_folio(struct lru_gen_mm_walk *walk, struct vm_area_struct *vma, + struct folio *folio, int new_gen, bool dirty) { int old_gen; @@ -3442,10 +3451,10 @@ static void walk_update_folio(struct lru_gen_mm_walk *walk, struct folio *folio, folio_mark_dirty(folio); if (walk) { - old_gen = folio_update_gen(folio, new_gen); + old_gen = folio_update_gen(folio, new_gen, &vma->flags); if (old_gen >= 0 && old_gen != new_gen) update_batch_size(walk, folio, old_gen, new_gen); - } else if (lru_gen_set_refs(folio)) { + } else if (lru_gen_set_refs(folio, &vma->flags)) { old_gen = folio_lru_gen(folio); if (old_gen >= 0 && old_gen != new_gen) folio_activate(folio); @@ -3518,7 +3527,7 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, continue; if (last != folio) { - walk_update_folio(walk, last, gen, dirty); + walk_update_folio(walk, args->vma, last, gen, dirty); last = folio; dirty = false; @@ -3531,7 +3540,7 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, walk->mm_stats[MM_LEAF_YOUNG] += nr; } - walk_update_folio(walk, last, gen, dirty); + walk_update_folio(walk, args->vma, last, gen, dirty); last = NULL; if (i < PTRS_PER_PTE && get_next_vma(PMD_MASK, PAGE_SIZE, args, &start, &end)) @@ -3609,7 +3618,7 @@ static void walk_pmd_range_locked(pud_t *pud, unsigned long addr, struct vm_area goto next; if (last != folio) { - walk_update_folio(walk, last, gen, dirty); + walk_update_folio(walk, vma, last, gen, dirty); last = folio; dirty = false; @@ -3623,7 +3632,7 @@ static void walk_pmd_range_locked(pud_t *pud, unsigned long addr, struct vm_area i = i > MIN_LRU_BATCH ? 0 : find_next_bit(bitmap, MIN_LRU_BATCH, i) + 1; } while (i <= MIN_LRU_BATCH); - walk_update_folio(walk, last, gen, dirty); + walk_update_folio(walk, vma, last, gen, dirty); lazy_mmu_mode_disable(); spin_unlock(ptl); @@ -4258,7 +4267,7 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr) continue; if (last != folio) { - walk_update_folio(walk, last, gen, dirty); + walk_update_folio(walk, vma, last, gen, dirty); last = folio; dirty = false; @@ -4270,7 +4279,7 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr) young += nr; } - walk_update_folio(walk, last, gen, dirty); + walk_update_folio(walk, vma, last, gen, dirty); lazy_mmu_mode_disable(); -- 2.47.3