From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ot1-f49.google.com (mail-ot1-f49.google.com [209.85.210.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C5C3C2B2D7 for ; Mon, 2 Feb 2026 00:55:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.49 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769993730; cv=none; b=PorYvlfOwGFqB4SGCUHkRHXnH+Jx6lYB5HgPLDihTdvtprrkG4AiiGosdQTl/hfaOaaAbUeTYPkZznxYpN+dSn5DcH4eu7unP31q8wyS/QP/QIbRDBEf/9YiDDcXGQ/iE+olpTzeFWHBvH8ooQk6k1XrC+fP3MD++doM6plitWA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769993730; c=relaxed/simple; bh=tHX4WQFJIeYtUYVy5/VngmcQAec3VgSH37nUuse3pdI=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=GKkAxbyzZgUMC9tTVitZJKeZqCkK+r/aW8mKBBoiSDAqpszN8enZjsqTm3Nfj6cO0bcu57Z/TWN0w3L5lT8QFU9QhKkyKHacTHqq2Wb7xYAmTa20qivJugDMZb9gLJu7SjbbenY4bvbjTRNzEbdRc4bWpdVhYrXUgis/bHCk4ns= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=mspPs/ij; arc=none smtp.client-ip=209.85.210.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mspPs/ij" Received: by mail-ot1-f49.google.com with SMTP id 46e09a7af769-7d1890f7cefso3323807a34.3 for ; Sun, 01 Feb 2026 16:55:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1769993727; x=1770598527; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=S6S79S/dxGzOUWakad8EjOaI91nR+6xoxuxHJIInMSk=; b=mspPs/ijBOZTu+7+xu3u8yuqXzc4gm81be6zFdN0MQrWpF67WDhDtucG2kV6R2dgqa /OQ5k4z7GHP2wt7TRQ74o+O8+S7tujEMfZz0A/YxOBwJet9KCcmZK86U/pAWk6u+4Ikl yWjCaJQijJEfCjMfy5N3TlDgKNqLvtvLaLv4Wvp+pbjhUThivMMRvkVIVoiO8t0oZKgU XTOGOv/pW1a7TI1689ImxE57KUHwM5AlIJIy/MKjO7oOClF6p/jXUIzUuBawGBcHppui L+3pfP6dzey0XGPGMrUOT69fYJ5IRozKicgzMJ+BVl0zVtIh78LnutZl9OvIqQCxBrvB tYrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769993727; x=1770598527; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=S6S79S/dxGzOUWakad8EjOaI91nR+6xoxuxHJIInMSk=; b=uD+xDj5e6UmlWax3pVHjseCtckux5TIX79gGAxV5KZN1630C45QQ9ryH4+mrTQPS+k Qi9I+ZrSZvg1/FV3Dd+GHqxM9lzj/+ZssjDf0HzorQ5pbiFQk3xK5LpGHXNKFatemwxv Zb4LAsmriSITxKXJptbnZBfHGfxnuREC1f91YwhW4Q9J6XtzDPYin8C3V8qr+jSO9yOi DQG2pppL833HZNdVcQgK5cF7sBhi9y+OabXWZ4kcRYRGBhws0clevYnM3z9UkCGyG/Rj 73Z84MmULP62tzbC/2jNQ1FudquFHJQuqsCfH8c7Nx/lw6vUNj6QBqIWf60rYqabD4+b Weag== X-Forwarded-Encrypted: i=1; AJvYcCU/9TF0gXjccxlBhj7yCJWey74sB0OpibDZXHbeQwjXT2bqE48gCxHl9Zpba44PjezQMc8aPs1FhOT/Dec=@vger.kernel.org X-Gm-Message-State: AOJu0YxCJTOLnH6ptAA5e190Qm2ZNx4DbYk4Q+HYUNQ4petQD2ODa76z A68hEx3nrKP6To6a9w10VN8GvaZinKWpAUT81iq8Ppv40kdow6HuXx/B X-Gm-Gg: AZuq6aJy1jvBCqZXXQna+djSqeekzWDeHvUWOU0RbDpGNXjRj8baM4IVp4WfTIzTPdB 3xw4NL7X6A40oyKdNSMmnZp3+SHlV8mLvA7QGytmWqZU+laxloa33NvmOPWR94j9OgvUVWoH5UN NhrDN7uQk1TDKhwhichuPBCjqUoVrzQZTHxUuhgMaOii4Ai2QUQMudPN0HoXrqoUglvNr6gdO8D LE7iVNF9xfnsqWY4lXbl3c0RpYuF2WoZVHIwVhgs9nVdYGZMuXTp7+6CzIcZu9UzBapozemB9iP m4WYgPvz14EXZQkEnP8AsuFUoeQIcy4E+8AX2McLwOBL3G/jZNWj0zTjzv26e4lG7Ix+R5M4iHd 3s/eOLL042e3rV9Slu66A+z3mKYTnlPtQx12jHMDgg7VY7Q+e8nBhugvKIOTXX+nvuniu//0UCf utjtwm X-Received: by 2002:a05:6830:6add:b0:7cf:e539:dcf0 with SMTP id 46e09a7af769-7d1a52974b0mr6198948a34.10.1769993727529; Sun, 01 Feb 2026 16:55:27 -0800 (PST) Received: from localhost ([2a03:2880:10ff::]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7d18c69f197sm9598342a34.12.2026.02.01.16.55.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Feb 2026 16:55:26 -0800 (PST) From: Usama Arif To: ziy@nvidia.com, Andrew Morton , David Hildenbrand , lorenzo.stoakes@oracle.com, linux-mm@kvack.org Cc: hannes@cmpxchg.org, riel@surriel.com, shakeel.butt@linux.dev, kas@kernel.org, baohua@kernel.org, dev.jain@arm.com, baolin.wang@linux.alibaba.com, npache@redhat.com, Liam.Howlett@oracle.com, ryan.roberts@arm.com, vbabka@suse.cz, lance.yang@linux.dev, linux-kernel@vger.kernel.org, kernel-team@meta.com, Usama Arif Subject: [RFC 00/12] mm: PUD (1GB) THP implementation Date: Sun, 1 Feb 2026 16:50:17 -0800 Message-ID: <20260202005451.774496-1-usamaarif642@gmail.com> X-Mailer: git-send-email 2.47.3 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit This is an RFC series to implement 1GB PUD-level THPs, allowing applications to benefit from reduced TLB pressure without requiring hugetlbfs. The patches are based on top of f9b74c13b773b7c7e4920d7bc214ea3d5f37b422 from mm-stable (6.19-rc6). Motivation: Why 1GB THP over hugetlbfs? ======================================= While hugetlbfs provides 1GB huge pages today, it has significant limitations that make it unsuitable for many workloads: 1. Static Reservation: hugetlbfs requires pre-allocating huge pages at boot or runtime, taking memory away. This requires capacity planning, administrative overhead, and makes workload orchastration much much more complex, especially colocating with workloads that don't use hugetlbfs. 4. No Fallback: If a 1GB huge page cannot be allocated, hugetlbfs fails rather than falling back to smaller pages. This makes it fragile under memory pressure. 4. No Splitting: hugetlbfs pages cannot be split when only partial access is needed, leading to memory waste and preventing partial reclaim. 5. Memory Accounting: hugetlbfs memory is accounted separately and cannot be easily shared with regular memory pools. PUD THP solves these limitations by integrating 1GB pages into the existing THP infrastructure. Performance Results =================== Benchmark results of these patches on Intel Xeon Platinum 8321HC: Test: True Random Memory Access [1] test of 4GB memory region with pointer chasing workload (4M random pointer dereferences through memory): | Metric | PUD THP (1GB) | PMD THP (2MB) | Change | |-------------------|---------------|---------------|--------------| | Memory access | 88 ms | 134 ms | 34% faster | | Page fault time | 898 ms | 331 ms | 2.7x slower | Page faulting 1G pages is 2.7x slower (Allocating 1G pages is hard :)). For long-running workloads this will be a one-off cost, and the 34% improvement in access latency provides significant benefit. ARM with 64K PAGE_SZIE supports 512M PMD THPs. In meta, we have a CPU bound workload running on a large number of ARM servers (256G). I enabled the 512M THP settings to always for a 100 servers in production (didn't really have high expectations :)). The average memory used for the workload increased from 217G to 233G. The amount of memory backed by 512M pages was 68G! The dTLB misses went down by 26% and the PID multiplier increased input by 5.9% (This is a very significant improvment in workload performance). A significant number of these THPs were faulted in at application start when were present across different VMAs. Ofcourse getting these 512M pages is easier on ARM due to bigger PAGE_SIZE and pageblock order. I am hoping that these patches for 1G THP can be used to provide similar benefits for x86. I expect workloads to fault them in at start time when there is plenty of free memory available. Previous attempt by Zi Yan ========================== Zi Yan attempted 1G THPs [2] in kernel version 5.11. There have been significant changes in kernel since then, including folio conversion, mTHP framework, ptdesc, rmap changes, etc. I found it easier to use the current PMD code as reference for making 1G PUD THP work. I am hoping Zi can provide guidance on these patches! Major Design Decisions ====================== 1. No shared 1G zero page: The memory cost would be quite significant! 2. Page Table Pre-deposit Strategy PMD THP deposits a single PTE page table. PUD THP deposits 512 PTE page tables (one for each potential PMD entry after split). We allocate a PMD page table and use its pmd_huge_pte list to store the deposited PTE tables. This ensures split operations don't fail due to page table allocation failures (at the cost of 2M per PUD THP) 3. Split to Base Pages When a PUD THP must be split (COW, partial unmap, mprotect), we split directly to base pages (262,144 PTEs). The ideal thing would be to split to 2M pages and then to 4K pages if needed. However, this would require significant rmap and mapcount tracking changes. 4. COW and fork handling via split Copy-on-write and fork for PUD THP triggers a split to base pages, then uses existing PTE-level COW infrastructure. Getting another 1G region is hard and could fail. If only a 4K is written, copying 1G is a waste. Probably this should only be done on CoW and not fork? 5. Migration via split Split PUD to PTEs and migrate individual pages. It is going to be difficult to find a 1G continguous memory to migrate to. Maybe its better to not allow migration of PUDs at all? I am more tempted to not allow migration, but have kept splitting in this RFC. Reviewers guide =============== Most of the code is written by adapting from PMD code. For e.g. the PUD page fault path is very similar to PMD. The difference is no shared zero page and the page table deposit strategy. I think the easiest way to review this series is to compare with PMD code. Test results ============ 1..7 # Starting 7 tests from 1 test cases. # RUN pud_thp.basic_allocation ... # pud_thp_test.c:169:basic_allocation:PUD THP allocated (anon_fault_alloc: 0 -> 1) # OK pud_thp.basic_allocation ok 1 pud_thp.basic_allocation # RUN pud_thp.read_write_access ... # OK pud_thp.read_write_access ok 2 pud_thp.read_write_access # RUN pud_thp.fork_cow ... # pud_thp_test.c:236:fork_cow:Fork COW completed (thp_split_pud: 0 -> 1) # OK pud_thp.fork_cow ok 3 pud_thp.fork_cow # RUN pud_thp.partial_munmap ... # pud_thp_test.c:267:partial_munmap:Partial munmap completed (thp_split_pud: 1 -> 2) # OK pud_thp.partial_munmap ok 4 pud_thp.partial_munmap # RUN pud_thp.mprotect_split ... # pud_thp_test.c:293:mprotect_split:mprotect split completed (thp_split_pud: 2 -> 3) # OK pud_thp.mprotect_split ok 5 pud_thp.mprotect_split # RUN pud_thp.reclaim_pageout ... # pud_thp_test.c:322:reclaim_pageout:Reclaim completed (thp_split_pud: 3 -> 4) # OK pud_thp.reclaim_pageout ok 6 pud_thp.reclaim_pageout # RUN pud_thp.migration_mbind ... # pud_thp_test.c:356:migration_mbind:Migration completed (thp_split_pud: 4 -> 5) # OK pud_thp.migration_mbind ok 7 pud_thp.migration_mbind # PASSED: 7 / 7 tests passed. # Totals: pass:7 fail:0 xfail:0 xpass:0 skip:0 error:0 [1] https://gist.github.com/uarif1/bf279b2a01a536cda945ff9f40196a26 [2] https://lore.kernel.org/linux-mm/20210224223536.803765-1-zi.yan@sent.com/ Signed-off-by: Usama Arif Usama Arif (12): mm: add PUD THP ptdesc and rmap support mm/thp: add mTHP stats infrastructure for PUD THP mm: thp: add PUD THP allocation and fault handling mm: thp: implement PUD THP split to PTE level mm: thp: add reclaim and migration support for PUD THP selftests/mm: add PUD THP basic allocation test selftests/mm: add PUD THP read/write access test selftests/mm: add PUD THP fork COW test selftests/mm: add PUD THP partial munmap test selftests/mm: add PUD THP mprotect split test selftests/mm: add PUD THP reclaim test selftests/mm: add PUD THP migration test include/linux/huge_mm.h | 60 ++- include/linux/mm.h | 19 + include/linux/mm_types.h | 5 +- include/linux/pgtable.h | 8 + include/linux/rmap.h | 7 +- mm/huge_memory.c | 535 +++++++++++++++++++++- mm/internal.h | 3 + mm/memory.c | 8 +- mm/migrate.c | 17 + mm/page_vma_mapped.c | 35 ++ mm/pgtable-generic.c | 83 ++++ mm/rmap.c | 96 +++- mm/vmscan.c | 2 + tools/testing/selftests/mm/Makefile | 1 + tools/testing/selftests/mm/pud_thp_test.c | 360 +++++++++++++++ 15 files changed, 1197 insertions(+), 42 deletions(-) create mode 100644 tools/testing/selftests/mm/pud_thp_test.c -- 2.47.3