From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 694613D4130 for ; Wed, 25 Feb 2026 16:34:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772037293; cv=none; b=DvSeQ1HaqLC6ebbPUk0gAtjT65BIS01Q4GDIlYmTTtWM9uPByGkj7+WN4Qjs66uRdC0i2ySMlnhh6nADc4SZ1zOD/Ogxlyqq1rQQQ4YOJKmHQzS3mCXWLIJi80PGPQSs26kFApQFCVoeuHOO/BztpbkRFd8/axSZAmY3ae8gMyI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772037293; c=relaxed/simple; bh=CRgIeXxfHd1+gygX8ykBVi2D3sWpCWeB04roUVjy0lw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=RO5YsyByFCRLtEWQ7vcIXgs1hDxESWHeSHH9iwDprIdbXO9LAN0cvRy5Zae4z6edrsHTJMsLBm841I+Ys7FP1DDa+kQFUd8fFra9alsxhIl2nSfKYnZ9ry3l4GZoTG6BTXEYMqKJiyDNiNcP34ippm0jJ3upLl0kNvfgxo4YGbM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jackmanb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=xe6Zzqlp; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jackmanb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="xe6Zzqlp" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-4836cc0b38eso23335425e9.2 for ; Wed, 25 Feb 2026 08:34:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1772037290; x=1772642090; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=S3owdeu4kE4g7S0Z4PekGxXbX8IdWlNVillVf61xXvw=; b=xe6Zzqlp+kVzqx5YLJMWQleiSl+pCwY8zBppplR5htcoiDqmnS6q9QcGGoqDBl+2aR TcRTcoZgIaOMT0G9neGMTK080aJ8LOQwgboH1ZdvB26zIM0IbzkE2vxTmsC9OnKa7vNj goe7J9nJ7+TLFsiWzqW6Y4CEY2xeVkHzNNhKg/R5ADQ5yb1T6GWkiWS1eSPTdTBib6Xy Mgo3GAeN/v1WcKAtzAX4acmGfulicvlxeiejh6Ml8snKBKcxhwmeT8/EUYv/VmhlI1oZ yPWWZZgKAsYEv8ySpfhoSPtEQvfvQ4fIo7N1AiZwzdAfKLhWaNu0kqbB6EUqAJGrx3Fc XcIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772037290; x=1772642090; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=S3owdeu4kE4g7S0Z4PekGxXbX8IdWlNVillVf61xXvw=; b=ccFlRDZWGJPvqoMKhlzCPSJDPpjcrznMAkg/KgKYzTPJhmvAsN0ThFfyfqx35H6q9g tpxvwCJ/KB0eUbumHoFJK4YR7ABbatw5RqqPAsaikN+HxBm7KRW34YPYk630fSEaOMw6 Mu+bEIORqYTQSTilHKVllMRJQL7YYRn9eDdNErCYUms9QNL6iGxK7gbuycxGndp1xj9T 4tL+ZUMh5NKbd/CBsC65Imx/r+HQ6epfs7K1ZAoxHv9E/Zik47wMZFdGlY02U71Op7da HJQ5BFdkjm3WXNwyxqPfSEx6Muz6zH/j5fMLukSucR5LMj272sNxIaDXGdFpxrmQKW6Y xrHw== X-Forwarded-Encrypted: i=1; AJvYcCXhEh0+lDPwa2m/VNoOI3roW2cfcdCdE1/KvcI2SIRpYjqaFVvFaYVk6nH/cPi5oA1s4KO1W0DC8QD36eE=@vger.kernel.org X-Gm-Message-State: AOJu0YxaU4wZIgLIEa5is1Y/9yWys3/3wUg0AjDivw8FYQmhUzP6vQHF RiBiP3FO0VZh1SNYuzWE1WNwNXHMR8N1MEg6cjHD+G6MW9nhqJ8U2Ced+9L+zcWuvY4VD6QMJw4 DyJ5vTWiQnf6yhA== X-Received: from wmo14.prod.google.com ([2002:a05:600c:230e:b0:483:41b6:adeb]) (user=jackmanb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4589:b0:480:1e9e:f9b with SMTP id 5b1f17b1804b1-483a95dea3emr345095145e9.16.1772037289584; Wed, 25 Feb 2026 08:34:49 -0800 (PST) Date: Wed, 25 Feb 2026 16:34:35 +0000 In-Reply-To: <20260225-page_alloc-unmapped-v1-0-e8808a03cd66@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260225-page_alloc-unmapped-v1-0-e8808a03cd66@google.com> X-Mailer: b4 0.14.3 Message-ID: <20260225-page_alloc-unmapped-v1-10-e8808a03cd66@google.com> Subject: [PATCH RFC 10/19] mm: add definitions for allocating unmapped pages From: Brendan Jackman To: Borislav Petkov , Dave Hansen , Peter Zijlstra , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Vlastimil Babka , Wei Xu , Johannes Weiner , Zi Yan Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, x86@kernel.org, rppt@kernel.org, Sumit Garg , derkling@google.com, reijiw@google.com, Will Deacon , rientjes@google.com, "Kalyazin, Nikita" , patrick.roy@linux.dev, "Itazuri, Takahiro" , Andy Lutomirski , David Kaplan , Thomas Gleixner , Brendan Jackman , Yosry Ahmed Content-Type: text/plain; charset="utf-8" Create __GFP_UNMAPPED, which requests pages that are not present in the direct map. Since this feature has a cost (e.g. more freelists), it's behind a kconfig. Unlike other conditionally-defined GFP flags, it doesn't fall back to being 0. This prevents building code that uses __GFP_UNMAPPED but doesn't depend on the necessary kconfig, since that would lead to invisible security issues. Create a freetype flag to record that pages on the freelists with this flag are unmapped. This is currently only needed for MIGRATE_UNMOVABLE pages, so the freetype encoding remains trivial. Also create the corresponding pageblock flag to record the same thing. To keep patches from being too overwhelming, the actual implementation is added separately, this is just types, Kconfig boilerplate, etc. Signed-off-by: Brendan Jackman --- include/linux/freetype.h | 41 +++++++++++++++++++++++++++++++++-------- include/linux/gfp_types.h | 26 ++++++++++++++++++++++++++ include/trace/events/mmflags.h | 9 ++++++++- mm/Kconfig | 4 ++++ 4 files changed, 71 insertions(+), 9 deletions(-) diff --git a/include/linux/freetype.h b/include/linux/freetype.h index 11bd6d2b94349..3b0d41b8c857f 100644 --- a/include/linux/freetype.h +++ b/include/linux/freetype.h @@ -2,6 +2,7 @@ #ifndef _LINUX_FREETYPE_H #define _LINUX_FREETYPE_H +#include #include #include @@ -64,30 +65,56 @@ static inline bool migratetype_is_mergeable(int mt) return mt < MIGRATE_PCPTYPES; } +enum { + /* Defined unconditionally as a hack to avoid a zero-width bitfield. */ + FREETYPE_UNMAPPED_BIT, + NUM_FREETYPE_FLAGS, +}; + /* * A freetype is the index used to identify free lists. This consists of a * migratetype, and other bits which encode orthogonal properties of memory. */ typedef struct { - int migratetype; + unsigned int migratetype : order_base_2(MIGRATE_TYPES); + unsigned int flags : NUM_FREETYPE_FLAGS; } freetype_t; +#ifdef CONFIG_PAGE_ALLOC_UNMAPPED +#define FREETYPE_UNMAPPED BIT(FREETYPE_UNMAPPED_BIT) +#define NUM_UNMAPPED_FREETYPES 1 +#else +#define FREETYPE_UNMAPPED 0 +#define NUM_UNMAPPED_FREETYPES 0 +#endif + /* * Return a dense linear index for freetypes that have lists in the free area. * Return -1 for other freetypes. */ static inline int freetype_idx(freetype_t freetype) { + /* For FREETYPE_UNMAPPED, only MIGRATE_UNMOVABLE has an index. */ + if (freetype.flags & FREETYPE_UNMAPPED) { + VM_WARN_ON_ONCE(freetype.flags & ~FREETYPE_UNMAPPED); + if (!IS_ENABLED(CONFIG_PAGE_ALLOC_UNMAPPED)) + return -1; + if (freetype.migratetype != MIGRATE_UNMOVABLE) + return -1; + return MIGRATE_TYPES; + } + /* No other flags are supported. */ + VM_WARN_ON_ONCE(freetype.flags); + return freetype.migratetype; } -/* No freetype flags actually exist yet. */ -#define NR_FREETYPE_IDXS MIGRATE_TYPES +/* One for each migratetype, plus one for MIGRATE_UNMOVABLE-FREETYPE_UNMAPPED */ +#define NR_FREETYPE_IDXS (MIGRATE_TYPES + NUM_UNMAPPED_FREETYPES) static inline unsigned int freetype_flags(freetype_t freetype) { - /* No flags supported yet. */ - return 0; + return freetype.flags; } static inline bool freetypes_equal(freetype_t a, freetype_t b) @@ -100,10 +127,8 @@ static inline freetype_t migrate_to_freetype(enum migratetype mt, { freetype_t freetype; - /* No flags supported yet. */ - VM_WARN_ON_ONCE(flags); - freetype.migratetype = mt; + freetype.flags = flags; return freetype; } diff --git a/include/linux/gfp_types.h b/include/linux/gfp_types.h index 814bb2892f99b..1f4f49bacb5b4 100644 --- a/include/linux/gfp_types.h +++ b/include/linux/gfp_types.h @@ -56,6 +56,9 @@ enum { ___GFP_NOLOCKDEP_BIT, #endif ___GFP_NO_OBJ_EXT_BIT, +#ifdef CONFIG_PAGE_ALLOC_UNMAPPED + ___GFP_UNMAPPED_BIT, +#endif ___GFP_LAST_BIT }; @@ -97,6 +100,10 @@ enum { #define ___GFP_NOLOCKDEP 0 #endif #define ___GFP_NO_OBJ_EXT BIT(___GFP_NO_OBJ_EXT_BIT) +#ifdef CONFIG_PAGE_ALLOC_UNMAPPED +#define ___GFP_UNMAPPED BIT(___GFP_UNMAPPED_BIT) +/* No #else - __GFP_UNMAPPED should never be a nop. Break the build if it isn't supported. */ +#endif /* * Physical address zone modifiers (see linux/mmzone.h - low four bits) @@ -293,6 +300,25 @@ enum { /* Disable lockdep for GFP context tracking */ #define __GFP_NOLOCKDEP ((__force gfp_t)___GFP_NOLOCKDEP) +/* + * Allocate pages that aren't present in the direct map. If the caller changes + * direct map presence, it must be restored to the previous state before freeing + * the page. (This is true regardless of __GFP_UNMAPPED). + * + * This uses the mermap (when __GFP_ZERO), so it's only valid to allocate with + * this flag where that's valid, namely from process context after the mermap + * has been initialised for that process. This also means that the allocator + * leaves behind stale TLB entries in the mermap region. The caller is + * responsible for ensuring they are flushed as needed. + * + * This is currently incompatible with __GFP_MOVABLE and __GFP_RECLAIMABLE, but + * only because of allocator implementation details, if a usecase arises this + * restriction could be dropped. + */ +#ifdef CONFIG_PAGE_ALLOC_UNMAPPED +#define __GFP_UNMAPPED ((__force gfp_t)___GFP_UNMAPPED) +#endif + /* Room for N __GFP_FOO bits */ #define __GFP_BITS_SHIFT ___GFP_LAST_BIT #define __GFP_BITS_MASK ((__force gfp_t)((1 << __GFP_BITS_SHIFT) - 1)) diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index a6e5a44c9b429..bb365da355b3a 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -61,11 +61,18 @@ # define TRACE_GFP_FLAGS_SLAB #endif +#ifdef CONFIG_PAGE_ALLOC_UNMAPPED +# define TRACE_GFP_FLAGS_UNMAPPED TRACE_GFP_EM(UNMAPPED) +#else +# define TRACE_GFP_FLAGS_UNMAPPED +#endif + #define TRACE_GFP_FLAGS \ TRACE_GFP_FLAGS_GENERAL \ TRACE_GFP_FLAGS_KASAN \ TRACE_GFP_FLAGS_LOCKDEP \ - TRACE_GFP_FLAGS_SLAB + TRACE_GFP_FLAGS_SLAB \ + TRACE_GFP_FLAGS_UNMAPPED #undef TRACE_GFP_EM #define TRACE_GFP_EM(a) TRACE_DEFINE_ENUM(___GFP_##a##_BIT); diff --git a/mm/Kconfig b/mm/Kconfig index bd49eb9ef2165..ccf1cda90cf4a 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1503,3 +1503,7 @@ config MERMAP_KUNIT_TEST If unsure, say N. endmenu + +config PAGE_ALLOC_UNMAPPED + bool "Support allocating pages that aren't in the direct map" if COMPILE_TEST + default COMPILE_TEST -- 2.51.2