Re: [LSF/MM/BPF TOPIC] 64k (or 16k) base page size on x86

[Nadia Chambers <nadia.yvette.chambers@ik.me>]

On Thu, Feb 19, 2026 at 03:08:51PM +0000, Kiryl Shutsemau wrote:
> No, there's no new hardware (that I know of). I want to explore what page size
> means.
> The kernel uses the same value - PAGE_SIZE - for two things:
>   - the order-0 buddy allocation size;
>   - the granularity of virtual address space mapping;
> I think we can benefit from separating these two meanings and allowing
> order-0 allocations to be larger than the virtual address space covered by a
> PTE entry.
> The main motivation is scalability. Managing memory on multi-terabyte
> machines in 4k is suboptimal, to say the least.

I apologise for my delay responding. I've been out of the loop a while.

As far as I know, the ABI compatibility algorithms are original to Hugh
Dickins from his 2001 larpage patch. I forward ported it in 2003 and
renamed it to „pgcl“ because „large pages“ had a major nameclash with
the „large pages“ of various superpaging efforts. Babaoğlu and Joy in
1979 introduced CLSIZE as part of their VM work on the VAX, though I've
heard suggestions it might have actually been McKusick or Karels, hence
the terminology „page clustering“ and the use of the abbreviation „pgcl“
from taking the „page“ abbreviation „pg“ from „NBPG“ and the „cluster“
abbreviation „cl“ from „CLSIZE“, as there is some lineage and priority
to their efforts, though they did not preserve the 512 B -aligned VAX
ABI (corrections from McKusick are also welcome, though this mailing
list might be a bit far afield for casually cc:'ing him).

The Linux parts of the superpaging efforts renamed superpages to „huge
pages“, though the well-known existing literature and code from e.g.
Talluri & Hill (1992) and Navarro, Iyer, Druschel & Cox (2001), where the
Cox of the second paper is even the Alan Cox known, in addition to other
things, for his extensive contributions to Linux. I'm unsure of the
origin of the nomenclature „huge pages“, but suspect Rohit Seth, from
whom I've not seen recent posts to Linux mailing lists, though like all
the other historical contributors, it would also be good to hear from
him in this thread. I strongly suspect the ABI compatibility algorithms
were original to Hugh and have cc:'d him in case he wants clarify the
origins or correct any of my commentary involving his work or otherwise
comment on things.

My 2003 code can be found at
https://mirrors.edge.kernel.org/pub/linux/kernel/people/wli/vm/pgcl/
I don't know of a link to Hugh Dickins' code from 2001, but it was
a couple of years earlier and far better-written than my forward ports or
attempts thereat of his code.

Scalability is worth being more specific about. Trimming a constant
factor is potentially worthwhile, but often one might want to go about
trying to make a difference in the computational complexity. In my 2003
IBM-internal whitepaper (dhansen had to have seen it, but it may not
have been memorable enough) and some commentaries on the subject since,
I've described it differently from scalability per se, varying by
whether the page size spectrum is dense or sparse:

	1. The sparse case is perhaps trivial, where the log of the
		clustering factor (your ilog2(PG_SIZE/PTE_SIZE) without
		a named macro for it that I know of, Hugh's
		PAGE_MMUSHIFT) gets shaved off the allocation order of
		the first nontrivial superpage/„huge page“ size, and
		maybe still-larger ones, but the small constant
		subtracted becomes progressively less significant as
		they continue their growth. I think there is
		doubly-exponential drop-off in availability as page
		orders increase, barring tactics to combat it.
		Empirically, orders up to maybe 2 or 3 aren't too bad,
		so choosing a PAGE_MMUSHIFT so the first nontrivial
		superpage size is order 2 or 3 can be reasonable, though
		with page size spectra as sparse as e.g. x86-64
		(„Sprungfaktor“ might be worth borrowing from German) of
		2**9 == 512, it's already 256 KiB, which might start
		raising zeroing latency and internal fragmentation of
		concerns, which can be algorithmically addressed via
		e.g. tail packing the pagecache into MMUPAGE_SIZE
		-aligned fragments for the perhaps common case of many
		small files and slab allocating many things now
		allocated as pages and more.

	2. The dense page size spectrum case can go about mapping
		individual PAGE_SIZE/PG_SIZE -sized allocation units
		with many different translation sizes. This provides a
		guarantee that so long as the size and alignment allow,
		the superpages/„huge pages“ up to PAGE_SIZE/PG_SIZE will
		always successfully be allocated without fear of
		external fragmentation. This guarantee was of particular
		industrial importance in the „forward-looking to
		64-bit“ aspects for proposing it as the solution to
		32-bit large memory's struct page array virtualspace
		footprint problems, in part because at the time „64-bit“
		meant IA64, which meant a dense page size spectrum with
		a Sprungfaktor of 4 like MIPS and the VHPT inverted
		pagetable, perhaps more like SPARC. (Here, XKVA, later
		done by Ingo as „4/4“, was the „32-bit hack“ to be
		avoided according to the advice I was given and the page
		clustering strategy was intended to be the
		forward-looking solution to the struct page array.) The
		interim since I worked on it has been long enough that
		priorities differ now.

These more specific motivations may have more force than „scalability.“

As far as completeness, in discussions about mechanical coding assistance
of a certain type, I used page clustering as an example of something
some of the algorithms might have trouble with. While trying on real x86
laptop hardware was late and not the subject of the majority of the
thought, it was able to get that up to running a GUI and starting up
some Electron apps in the start-up at login list before anything goes
wrong. The regression testing matrix in QEMU was 20 architectures with
PAGE_MMUSHIFT values of 0, 2, 4, and 6, for 80 cells, with the
regression tests being LTP and something called mm-stress, which is I
think mostly a script running various arguments for stress-ng. I don't
think QEMU cuts it and some sort of real hardware would be far better to
use if one can put together the multi-architecture testing lab with
terminal concentrators, serial consoles, and remote power controls etc.
Brewing up a kernel de novo as a vehicle for it all almost seemed like a
good idea at one point. The Coq, CBMC etc. models they produced are also
a bit weak for checking it all, and I was particularly dissatisfied with
their not directly representing user address spaces in the models, never
mind the concrete C (and I guess Rust is in the mix these days too)
rather often leaving code unswept right beside code it just swept, and
the stack top code looked off, too, and environments weren't being
inherited properly, but it seemed better to approach by extending the
verification than directly fixing it by hand. I had started trying to
add Dartagnan and some other Coq etc. packages in on it that are also
kernel-relevant before certain disruptions, but it got renamed to Dat3M
and some unusual Java things are ongoing with it that make it harder to
get interoperating the rest of the Coq-based tools. Perhaps the
inevitable and always obligatory human review has to expand to make up
for where formal modelling and the thus-far-not-entirely-well-grounded
machine reasoning systems cannot yet tread. While there's been something
of an interruption or disruption so that it's disorganised, the residue
or detritus of the experimentation with mechanically-generated code
ended up at git@git.sr.ht:~nadiayvette/linux-pgcl which is rather far
from a finished product, but if we're posting things that merely get to
a shell that doesn't crash immediately but does eventually, well, it's
of that kind.

In 2003 I thought it was by far the most fun project I ever worked on.
It would be great if I could contribute to efforts along these lines.


-- nyc