From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF93E282F34 for ; Wed, 13 May 2026 13:07:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778677656; cv=none; b=Bu/AHrAbkzS98IDEdqBIxi922ASM/+maVQVXAxsf5XoLUVNbDMD6RaaIg7quJCz4X1ft+v5b9Z+gR7oSrwLNR+jocrms/Am9XEYtQVYZP4kTIyd0Kc32y93RBtzmsm9Nn1CZGSSwCqpgfzREGrTGnR+u+5/pSQoMswE5I4SbzwA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778677656; c=relaxed/simple; bh=CTgTl2Bfdl7K7r8/m+ascYiZIIyrAlGaWurITpSVuO4=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=KKGoHwhmxm0N68Q0ZABPF4gYlldivcNSQ8YL0tmG43zoZo8p6jWcSVxkTyG4uU+izeU3rIa46cVv+i8XWVqI2SVW9ChhjswYVC1PdHPS7bZSTohpltyHfFI+pTVLIPkYMUU7eTP8n6/0TRQ24vnmUf/tTRCzuF+AhNwFIhkCsKI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=YacAMG/c; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="YacAMG/c" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2ba0fc8b1f0so43405375ad.3 for ; Wed, 13 May 2026 06:07:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1778677653; x=1779282453; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=J8NpoUoHQ+aQSSKrd0jYtj6P28R/h0KlfJ/MEbWicLs=; b=YacAMG/cShOqMZuzrrfY48N7XPXQN8XFw/wE5VJE9pVeCiDodOLn0mj1GMUlNHE9Q3 4HG+Q5XU9oS3+ZrBrBJpHbXtT2ltNrPB3ORJsy7QVBqc8i5i+vpevPBrexNQ371y9SJU Qo1N+cYWlIIN/t5uTnGtli1AiURdQaC6cJOWJomVzmkjEsgwsnXNrN0+ldb3Nk5iSkr9 SF/TFpBdtCnn6hRMKLkJZ2WLW3jxUrwSLGs8iu1iivWnPTDAsKhFugyViTkdhczVjmoc ijwi69F3DlEm8Ytq4Gdw/+q7lJdzGa90TAq4iNBLtpE+tx0ATW+xevvspeZtY20jYbAi RJ4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778677653; x=1779282453; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=J8NpoUoHQ+aQSSKrd0jYtj6P28R/h0KlfJ/MEbWicLs=; b=ey+HjclY/Llyj158MS0oMz2iOc0sjQhU1UJZirqhuf/F66pH2kIN9VCt9ddgsRy09o r37Xuu6KpxET3LUKR7Jp7lsu0auEvFcKaKlCbP/XCGXc399rYOro/spvcrbekUTrLCo+ n8Zsa+7nj+dAy5tVeiRxqVz+kp6NzyrRo3ae6GHjN+LI1XfxTfrG3VyTLc+iX+qCycjp JJM5vYZiFMjc2EWS5VENjHmtxXI/J6r2t0/Qj7PVWvKge5ffXpjNvV0sWRPI9uA0+5x1 DKf5FpIZc0Y/9lwb+Br1Kk8GEk/b9HByeJpwcy+JofcMatZnJgajqoX+V03hi1vZAfN6 DL9Q== X-Forwarded-Encrypted: i=1; AFNElJ/E0omhl60AC9lNg4L0bYp21uG9yNzxS+TG2PNEeFdSn4oLinF+shuLT0M7GzKNLwOLMQaCNEXuesYdW2I=@vger.kernel.org X-Gm-Message-State: AOJu0YwSqyQPcYDKMZvWmUlv+hWis6x6VWDbi8B6PteRLLHUaAWU5Ega b9I+yRLC1PMSSOH21ICReNJM5luM9yKUlCKYM302FVlfnUOZR4f5ihboH9OOV/VA0k4= X-Gm-Gg: Acq92OGnnuN28KNcvc793YY8hNDRMR0pn5QHAJJu7DeRXt6akY7YmxHhuX5L8v/CEvM VMnBwpmC/91/xigib1riC+CFanKdbgQGR1XdqBDkTwhiMEuJvv/qoUM4uk8R2BV8fuJnnNQ6CJy v+IXnFXsplgob/U14gkq+EDs4JQ8pWNCoxt1HGmzkPE3GHYP+65YQR2ivhhifjPoh59loHGLTuB qWgT1TOMQ8aWj+943T/9wrZuI0G/LP6SyI2yBQkrmvRqozNQ5NVU4p2etEhJXII/HlbDBXYW1T2 Tow7D+j2rkXj5ZiHYsJQ/A1FCE9ouHURozYlobNxoDA7Hwt5gkD3ruzZhU0NAgw8tmNIYPHwCLd QJjm2u6puRdq8FVR3/ZUrs5hSe4/tbl38SJZlAZjxY5N1PJsW7X/vX1l2Qg6hpsOqhKawFvyFVn DrR4KqF8Y0tKKBDlmhV2WQKBJ7CV1+hHkpX43m0v3SxLnzI84GApItOgcHaLM= X-Received: by 2002:a17:903:1b4f:b0:2b0:9c2b:641d with SMTP id d9443c01a7336-2bd2f4dbc06mr31307215ad.2.1778677652955; Wed, 13 May 2026 06:07:32 -0700 (PDT) Received: from PXLDJ45XCM.bytedance.net ([61.213.176.6]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2baf1e90854sm166641925ad.66.2026.05.13.06.07.27 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 13 May 2026 06:07:32 -0700 (PDT) From: Muchun Song To: Andrew Morton , David Hildenbrand , Muchun Song , Oscar Salvador , Michael Ellerman , Madhavan Srinivasan Cc: Lorenzo Stoakes , "Liam R . Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Nicholas Piggin , Christophe Leroy , Ackerley Tng , Frank van der Linden , aneesh.kumar@linux.ibm.com, joao.m.martins@oracle.com, linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, Muchun Song Subject: [PATCH v2 00/69] mm: Generalize HVO for HugeTLB and device DAX Date: Wed, 13 May 2026 21:04:28 +0800 Message-ID: <20260513130542.35604-1-songmuchun@bytedance.com> X-Mailer: git-send-email 2.50.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit In this series, HVO is redefined as Hugepage Vmemmap Optimization: a general vmemmap optimization model for large hugepage-backed mappings, rather than a HugeTLB-only implementation detail. The existing code grew around the original HugeTLB-specific HVO path, while device DAX developed similar but separate vmemmap optimization handling. As a result, the current implementation carries duplicated logic, boot-time special cases, and subsystem-specific interfaces around what is fundamentally the same sparse-vmemmap optimization. This series generalizes that optimization into a common framework used by both HugeTLB and device DAX. The first few patches include some minor bug fixes found during AI-aided review of the current code. These fixes are not the main goal of the series, but the later refactoring and unification work depends on them, so they are included here as preparatory changes. The series then reworks the relevant early boot and sparse initialization paths, introduces a generic section-based sparse-vmemmap optimization infrastructure, switches HugeTLB and device DAX over to the shared implementation, and removes the old special-case code. At a high level, the series does the following: - apply a small set of preparatory bug fixes - reorder early boot and sparse initialization so optimized vmemmap setup has the required zone and pageblock state - introduce generic section-based vmemmap optimization infrastructure - switch HugeTLB and device DAX to the shared implementation - consolidate HVO enablement and naming - remove obsolete HugeTLB-specific boot-time and architecture-specific optimization code - rewrite the documentation around the unified design This brings a few concrete benefits: - HugeTLB and device DAX share one vmemmap optimization framework, reducing duplicated logic and long-term maintenance overhead - when CONFIG_DEFERRED_STRUCT_PAGE_INIT is disabled, optimized struct pages can skip the usual memmap_init() initialization work, which helps reduce boot-time overhead - all architectures that support HVO benefit from the generic sparse-vmemmap optimization path without extra architecture-specific preinit handling - device DAX improves its struct page savings further by dropping the extra reserved tail page - shared vmemmap tail pages are mapped read-only, improving robustness I have only built and tested this series on x86. I do not currently have a powerpc test environment, so any testing or feedback on powerpc would be much appreciated. Changes since v1: - rebased onto current next tree - added the preparatory minor bug fixes found during AI-aided review - added further refactoring on top of the new infrastructure Muchun Song (69): mm/hugetlb: Fix boot panic with CONFIG_DEBUG_VM and HVO bootmem pages mm/hugetlb_vmemmap: Fix __hugetlb_vmemmap_optimize_folios() powerpc/mm: Fix wrong addr_pfn tracking in compound vmemmap population mm/hugetlb: Initialize gigantic bootmem hugepage struct pages earlier mm/mm_init: Simplify deferred_free_pages() migratetype init mm/sparse: Panic on memmap and usemap allocation failure mm/sparse: Move subsection_map_init() into sparse_init() mm/mm_init: Defer sparse_init() until after zone initialization mm/mm_init: Defer hugetlb reservation until after zone initialization mm/mm_init: Remove set_pageblock_order() call from sparse_init() mm/sparse: Move sparse_vmemmap_init_nid_late() into sparse_init_nid() mm/hugetlb_cma: Validate hugetlb CMA range by zone at reserve time mm/hugetlb: Refactor early boot gigantic hugepage allocation mm/hugetlb: Free cross-zone bootmem gigantic pages after allocation mm/hugetlb_vmemmap: Move bootmem HVO setup to early init mm/hugetlb: Remove obsolete bootmem cross-zone checks mm/sparse-vmemmap: Remove sparse_vmemmap_init_nid_late() mm/hugetlb: Remove unused bootmem cma field mm/mm_init: Make __init_page_from_nid() static mm/sparse-vmemmap: Drop VMEMMAP_POPULATE_PAGEREF mm: Rename vmemmap optimization macros around folio semantics mm/sparse: Drop power-of-2 size requirement for struct mem_section mm/sparse-vmemmap: track compound page order in struct mem_section mm/mm_init: Skip initializing shared vmemmap tail pages mm/sparse-vmemmap: Initialize shared tail vmemmap pages on allocation mm/sparse-vmemmap: Support section-based vmemmap accounting mm/sparse-vmemmap: Support section-based vmemmap optimization mm/hugetlb: Use generic vmemmap optimization macros mm/sparse: Mark memblocks present earlier mm/hugetlb: Switch HugeTLB to section-based vmemmap optimization mm/sparse: Remove section_map_size() mm/mm_init: Factor out pfn_to_zone() as a shared helper mm/sparse: Remove SPARSEMEM_VMEMMAP_PREINIT mm/sparse: Inline usemap allocation into sparse_init_nid() mm/hugetlb: Remove HUGE_BOOTMEM_HVO mm/hugetlb: Remove HUGE_BOOTMEM_CMA mm/sparse-vmemmap: Factor out shared vmemmap page allocation mm/sparse-vmemmap: Introduce CONFIG_SPARSEMEM_VMEMMAP_OPTIMIZATION mm/sparse-vmemmap: Switch DAX to vmemmap_shared_tail_page() powerpc/mm: Switch DAX to vmemmap_shared_tail_page() mm/sparse-vmemmap: Drop the extra tail page from DAX reservation mm/sparse-vmemmap: Switch DAX to section-based vmemmap optimization mm/sparse-vmemmap: Unify DAX and HugeTLB population paths mm/sparse-vmemmap: Remove the unused ptpfn argument powerpc/mm: Make vmemmap_populate_compound_pages() static mm/sparse-vmemmap: Map shared vmemmap tail pages read-only powerpc/mm: Map shared vmemmap tail pages read-only mm/sparse-vmemmap: Inline vmemmap_populate_address() into its caller mm/hugetlb_vmemmap: Remove vmemmap_wrprotect_hvo() mm/sparse: Simplify section_nr_vmemmap_pages() mm/sparse-vmemmap: Introduce vmemmap_nr_struct_pages() powerpc/mm: Drop powerpc vmemmap_can_optimize() mm/sparse-vmemmap: Drop vmemmap_can_optimize() mm/sparse-vmemmap: Drop @pgmap from vmemmap population APIs mm/sparse: Decouple section activation from ZONE_DEVICE mm: Redefine HVO as Hugepage Vmemmap Optimization mm/sparse-vmemmap: Consolidate HVO enable checks mm/hugetlb: Make HVO optimizable checks depend on generic logic mm/sparse-vmemmap: Localize init_compound_tail() mm/mm_init: Check zone consistency on optimized vmemmap sections mm/hugetlb: Drop boot-time HVO handling for gigantic folios mm/hugetlb: Simplify hugetlb_folio_init_vmemmap() mm/hugetlb: Initialize the full bootmem hugepage in hugetlb code mm/mm_init: Factor out compound page initialization mm/mm_init: Make __init_single_page() static mm/cma: Move CMA pageblock initialization into cma_activate_area() mm/cma: Move init_cma_pageblock() into cma.c mm/mm_init: Initialize pageblock migratetype in memmap init helpers Documentation/mm: Rewrite vmemmap_dedup.rst for unified HVO .../admin-guide/kernel-parameters.txt | 2 +- Documentation/admin-guide/mm/hugetlbpage.rst | 4 +- .../admin-guide/mm/memory-hotplug.rst | 2 +- Documentation/admin-guide/sysctl/vm.rst | 3 +- Documentation/arch/powerpc/index.rst | 1 - Documentation/arch/powerpc/vmemmap_dedup.rst | 101 ---- Documentation/mm/vmemmap_dedup.rst | 217 ++------ arch/arm64/mm/mmu.c | 5 +- arch/loongarch/mm/init.c | 5 +- arch/powerpc/include/asm/book3s/64/radix.h | 12 - arch/powerpc/mm/book3s64/radix_pgtable.c | 154 +----- arch/powerpc/mm/hugetlbpage.c | 11 +- arch/powerpc/mm/init_64.c | 1 + arch/powerpc/mm/mem.c | 5 +- arch/riscv/mm/init.c | 5 +- arch/s390/mm/init.c | 5 +- arch/x86/Kconfig | 1 - arch/x86/entry/vdso/vdso32/fake_32bit_build.h | 1 - arch/x86/mm/init_64.c | 5 +- drivers/dax/Kconfig | 1 + fs/Kconfig | 6 +- include/linux/hugetlb.h | 23 +- include/linux/memory_hotplug.h | 12 +- include/linux/mm.h | 44 +- include/linux/mm_types.h | 3 +- include/linux/mmzone.h | 151 ++++-- include/linux/page-flags-layout.h | 2 + include/linux/page-flags.h | 31 +- kernel/bounds.c | 5 + mm/Kconfig | 9 +- mm/bootmem_info.c | 5 +- mm/cma.c | 18 +- mm/hugetlb.c | 337 ++++-------- mm/hugetlb_cma.c | 41 +- mm/hugetlb_cma.h | 4 +- mm/hugetlb_vmemmap.c | 266 +-------- mm/hugetlb_vmemmap.h | 64 +-- mm/internal.h | 72 ++- mm/memory-failure.c | 6 +- mm/memory_hotplug.c | 22 +- mm/memremap.c | 4 +- mm/mm_init.c | 241 ++++----- mm/sparse-vmemmap.c | 511 ++++++------------ mm/sparse.c | 129 +---- mm/util.c | 2 +- scripts/gdb/linux/mm.py | 6 +- 46 files changed, 743 insertions(+), 1812 deletions(-) delete mode 100644 Documentation/arch/powerpc/vmemmap_dedup.rst base-commit: e98d21c170b01ddef366f023bbfcf6b31509fa83 -- 2.54.0