From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67BB4367F42 for ; Wed, 15 Apr 2026 11:14:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776251694; cv=none; b=EwUGXFjRAac/ij4ItgykAU565VysBTxG5lRz92FrydJSfmmt716MTj3urpIi3eNDaOQYGJcf8ciE8q9lEFj3tIUNt3I7V7vcioeHCW6Rb+iqS+JRXZzhepeRTYxgGQEl2H31M9e4nScZeEz1RQrRp3PmJKWNrgOzvDdxwI9WJt4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776251694; c=relaxed/simple; bh=56/m/uTeNN9wB78un6OO3se3/EIjy7omNkiIDPY+1+o=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=e4T29C2HXmgMDYJR/s2Sl97n9Htg7TINOG5OkPxMuKw6rY/w+cZ1kzkSg3S0grb7AgL7N25u0acmdwj3KiiMfcUrlRcHD1XO7itezkm5vYlj3+IP6MUka1lP90fSUpnfSjUsG3cLyN3EpYa5kyzT+uNYiB7vueNqoKSwGzPio4c= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=KVazzK0V; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="KVazzK0V" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-2b25cf1b5f0so44350875ad.3 for ; Wed, 15 Apr 2026 04:14:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1776251693; x=1776856493; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UUwLFe4sJwTmvK33kSzO0AkxCRTRozJKSvcgK9DIV10=; b=KVazzK0VZfBqNnwCyx9s4RW+G4ihrsBj400ANV/6sAdq1NsAhiYBBuRqEcCkLCSbTi Za+KJcfgsFDm3S6eFYuouHSpjSIxB9cZxKuISFrjuIc7oWJk9w9M7l0a7hShGEqQZTXH gWDM3J1/k3ugFxESZf2cfjN4ajtHhil2emaOem6zHEYsqUukB+ArEjgLSGUbrdKsXJx0 Fn0vB+meUjfKzqLtMAemphfu2/HX+bqonCbUXCsZrSDh5FiFeTkm4MjYyD58a1b3YSZt 6LKbq7qJJqAENjlKC9tFMZWAJi6hGaBIj3kBraqwpGjjkwUpVhux63gImIvHInyd2WOG FPmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776251693; x=1776856493; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=UUwLFe4sJwTmvK33kSzO0AkxCRTRozJKSvcgK9DIV10=; b=SUKIp72l6wbCia0S/H+g6vAmn9f2FCUkvxheCGCpqrnbdhV5/j0eYx29GVQz6/ubxK x97JDZ8Z/ruZvV5A1u2z06b4EWmgLwt2aoNhT5PGJUyz2+Zbee5nu9kyR+iTgBTnITZJ TCpHSSb1hKt781W247IKasXizVW5wCUelPYe6/9j6zmi6UF4IS/Ab/xexf8ll2wCseww IfnEwf2Rk81qLKZs0Aa4OcKqHlqzpYfngdzlwge8F3eBZAnePOmnDN2MXl6us7tE+uUh CnxibvNYE9LmesHTw0Sm6lVA72pJ3gCmG3kH0VwkFgQmQgF6N8sic8p7HRZ+NJiqZ/Z0 gbVQ== X-Forwarded-Encrypted: i=1; AFNElJ/uKFbIBB2nLM2Cjxg9RHXNSsudTs0za+H/pW6oOHDR81COGZA8EdpRydROZaBl9cIPXSBL9JDS6YY/aZY=@vger.kernel.org X-Gm-Message-State: AOJu0YwrkJwECCvYw0NWyTLZ+5MDHPsBMPm2Ilh044VSdzHL0yBL3ydr id0rdxOQQef5yJ1vdebSV4UcWlbZCMcAek4dbXvl+M2OuEiW3nTZuSmeZVpWpdNZUA8= X-Gm-Gg: AeBDievHameZ0NdcOBd2PMD9G9MIQYOr68UxbRMoY9Igb+TndWMq/HsZPUMiGsGU8sV +4nqO+5ldlmNm/9Zxhv3XzEdugVgfRCAv+D/KSDC9XCuy6P4bFZCMofsxZTtVD+KaLcPiQYPiQ7 tUe1myLUl+pZXWgIJYs5K0/xY3iwi7MZKum31NLcos3wg179DMTR5asVmfi3DOMklklNR+opwzS +CVYhD53OJ2dhD3O1mTB3FkdMGmGJyLDgzahJbzYCq3Uy8mI0wSqf8WAKq4D6h0LvzFPRNyZGuK rCnHlcRcqmaR10TQ/eIz/sRkeRwEIrn4GQKaDbE683h8ur7k6ogy9pbHXBLVCgTUy+/GMyY3jla jz5LJghA3NfMn/WZ/jyYWoKqnv1lyaoiOZnhfAdg2uG7ZUqLTOdKLccUtuUHhOQHEfVSNTYckdh 8gYs5Psn+VLjNh0kFk2PI0nUeCzOuJ0nab9ZXrYSJSB6k= X-Received: by 2002:a17:903:37c6:b0:2b2:49a7:a5bc with SMTP id d9443c01a7336-2b2d5a60603mr217754215ad.39.1776251692524; Wed, 15 Apr 2026 04:14:52 -0700 (PDT) Received: from n232-176-004.byted.org ([36.110.163.96]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2b477fd3724sm19509485ad.0.2026.04.15.04.14.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Apr 2026 04:14:52 -0700 (PDT) From: Muchun Song To: Andrew Morton , David Hildenbrand , Muchun Song , Oscar Salvador , Michael Ellerman , Madhavan Srinivasan Cc: Lorenzo Stoakes , "Liam R . Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Nicholas Piggin , Christophe Leroy , aneesh.kumar@linux.ibm.com, joao.m.martins@oracle.com, linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, Muchun Song Subject: [PATCH v2 5/6] mm/sparse-vmemmap: Fix missing architecture-specific page table sync Date: Wed, 15 Apr 2026 19:14:11 +0800 Message-Id: <20260415111412.1003526-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20260415111412.1003526-1-songmuchun@bytedance.com> References: <20260415111412.1003526-1-songmuchun@bytedance.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit On x86-64, vmemmap_populate() normally calls sync_global_pgds() to keep the page tables in sync. However, when vmemmap optimization for compound devmaps is enabled, vmemmap_populate_compound_pages() is called directly from __populate_section_memmap(), bypassing the architecture- specific vmemmap_populate() entirely. This skips the sync on x86-64 and can later trigger vmemmap-access faults. Fix this by moving the vmemmap_can_optimize() dispatch from __populate_section_memmap() into the generic helpers -- vmemmap_populate_basepages() and vmemmap_populate_hugepages(). This way, the architecture vmemmap_populate() is always invoked first, ensuring any arch-specific post-population steps (e.g. sync_global_pgds()) are executed before returning. Architectures that override vmemmap_populate() (e.g. powerpc) handle the optimization dispatch in their own implementation instead. Fixes: 4917f55b4ef9 ("mm/sparse-vmemmap: improve memory savings for compound devmaps") Signed-off-by: Muchun Song --- arch/powerpc/include/asm/book3s/64/radix.h | 6 ------ arch/powerpc/mm/book3s64/radix_pgtable.c | 16 ++++++++++----- mm/sparse-vmemmap.c | 24 +++++++++++----------- 3 files changed, 23 insertions(+), 23 deletions(-) diff --git a/arch/powerpc/include/asm/book3s/64/radix.h b/arch/powerpc/include/asm/book3s/64/radix.h index bde07c6f900f..2600defa2dc2 100644 --- a/arch/powerpc/include/asm/book3s/64/radix.h +++ b/arch/powerpc/include/asm/book3s/64/radix.h @@ -357,11 +357,5 @@ int radix__remove_section_mapping(unsigned long start, unsigned long end); #define vmemmap_can_optimize vmemmap_can_optimize bool vmemmap_can_optimize(struct vmem_altmap *altmap, struct dev_pagemap *pgmap); #endif - -#define vmemmap_populate_compound_pages vmemmap_populate_compound_pages -int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn, - unsigned long start, - unsigned long end, int node, - struct dev_pagemap *pgmap); #endif /* __ASSEMBLER__ */ #endif diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/book3s64/radix_pgtable.c index 568500343e5f..21fece355fbb 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -1109,7 +1109,10 @@ static inline pte_t *vmemmap_pte_alloc(pmd_t *pmdp, int node, return pte_offset_kernel(pmdp, address); } - +static int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn, + unsigned long start, + unsigned long end, int node, + struct dev_pagemap *pgmap); int __meminit radix__vmemmap_populate(unsigned long start, unsigned long end, int node, struct vmem_altmap *altmap, struct dev_pagemap *pgmap) @@ -1122,6 +1125,9 @@ int __meminit radix__vmemmap_populate(unsigned long start, unsigned long end, in pmd_t *pmd; pte_t *pte; + if (vmemmap_can_optimize(altmap, pgmap)) + return vmemmap_populate_compound_pages(page_to_pfn((struct page *)start), + start, end, node, pgmap); /* * If altmap is present, Make sure we align the start vmemmap addr * to PAGE_SIZE so that we calculate the correct start_pfn in @@ -1303,10 +1309,10 @@ static pte_t * __meminit vmemmap_compound_tail_page(unsigned long addr, return pte; } -int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn, - unsigned long start, - unsigned long end, int node, - struct dev_pagemap *pgmap) +static int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn, + unsigned long start, + unsigned long end, int node, + struct dev_pagemap *pgmap) { /* * we want to map things as base page size mapping so that diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c index f5245647afee..7f684ed3479e 100644 --- a/mm/sparse-vmemmap.c +++ b/mm/sparse-vmemmap.c @@ -296,10 +296,16 @@ static int __meminit vmemmap_populate_range(unsigned long start, return 0; } +static int __meminit vmemmap_populate_compound_pages(unsigned long start, + unsigned long end, int node, + struct dev_pagemap *pgmap); + int __meminit vmemmap_populate_basepages(unsigned long start, unsigned long end, int node, struct vmem_altmap *altmap, struct dev_pagemap *pgmap) { + if (vmemmap_can_optimize(altmap, pgmap)) + return vmemmap_populate_compound_pages(start, end, node, pgmap); return vmemmap_populate_range(start, end, node, altmap, -1, 0); } @@ -411,6 +417,9 @@ int __meminit vmemmap_populate_hugepages(unsigned long start, unsigned long end, pud_t *pud; pmd_t *pmd; + if (vmemmap_can_optimize(altmap, pgmap)) + return vmemmap_populate_compound_pages(start, end, node, pgmap); + for (addr = start; addr < end; addr = next) { next = pmd_addr_end(addr, end); @@ -453,7 +462,6 @@ int __meminit vmemmap_populate_hugepages(unsigned long start, unsigned long end, return 0; } -#ifndef vmemmap_populate_compound_pages /* * For compound pages bigger than section size (e.g. x86 1G compound * pages with 2M subsection size) fill the rest of sections as tail @@ -491,14 +499,14 @@ static pte_t * __meminit compound_section_tail_page(unsigned long addr) return pte; } -static int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn, - unsigned long start, +static int __meminit vmemmap_populate_compound_pages(unsigned long start, unsigned long end, int node, struct dev_pagemap *pgmap) { unsigned long size, addr; pte_t *pte; int rc; + unsigned long start_pfn = page_to_pfn((struct page *)start); if (reuse_compound_section(start_pfn, pgmap)) { pte = compound_section_tail_page(start); @@ -544,26 +552,18 @@ static int __meminit vmemmap_populate_compound_pages(unsigned long start_pfn, return 0; } -#endif - struct page * __meminit __populate_section_memmap(unsigned long pfn, unsigned long nr_pages, int nid, struct vmem_altmap *altmap, struct dev_pagemap *pgmap) { unsigned long start = (unsigned long) pfn_to_page(pfn); unsigned long end = start + nr_pages * sizeof(struct page); - int r; if (WARN_ON_ONCE(!IS_ALIGNED(pfn, PAGES_PER_SUBSECTION) || !IS_ALIGNED(nr_pages, PAGES_PER_SUBSECTION))) return NULL; - if (vmemmap_can_optimize(altmap, pgmap)) - r = vmemmap_populate_compound_pages(pfn, start, end, nid, pgmap); - else - r = vmemmap_populate(start, end, nid, altmap, pgmap); - - if (r < 0) + if (vmemmap_populate(start, end, nid, altmap, pgmap)) return NULL; return pfn_to_page(pfn); -- 2.20.1