From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA1933ECBE8 for ; Wed, 25 Feb 2026 16:34:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.74 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772037301; cv=none; b=FGMbtS0dkUMeKEXjmLzkOZhCop2HlINfPeg/aOXJFwQjcSF38xQixn1ySIUccg6/LlHyRAMp2DagjNoGxghSIOW0Q56Z7qiAC8Hr+9LFtkYqxcm/3/yGLnoLOzm3xPo8pLl0pszJ6y7z63+ayZ70f1FksGzYROlvyKhG800ZZik= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772037301; c=relaxed/simple; bh=VJZGXNNp92gP9BwsFi6v8jVgYl1WPnDx6Fbp1Vutrhw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=V7aPLtotrLJz16XMObn1EEOcTsA7bnBNp+H6LfgI4/U2FOBIMP6xbUbjhKnPbL8tY53dU2DIMtsUAZJLEubWKHy7aaKbocnESp4LzplBKLHL25f46Z7S6irHgneoOtRmaZtRyyxvQSlvk8H//88PhgyNqJp3rrsivEPOJ994nCI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jackmanb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=wcErE7JB; arc=none smtp.client-ip=209.85.208.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jackmanb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="wcErE7JB" Received: by mail-ed1-f74.google.com with SMTP id 4fb4d7f45d1cf-65f767a8d6cso1576630a12.2 for ; Wed, 25 Feb 2026 08:34:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1772037298; x=1772642098; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/X71c0eq0chbAJtrestek7XlSR8mXWIyGEkHLGooRRo=; b=wcErE7JBCQN7z39SDomQB9u9tFrqwTO/kYdpjhEtT4a65hI7jnSiBtL2B0YtTAnBUH low4oac0hID0bwWYcioP3pFzWIbgF0xj2i8qQmg7AhNzZANmnLpfrxuouaMfOynOVc0b rxiCJWDUfE9XlTWQW5gOkerLzsWmz4c3NTucplZmaJAZleKtCri3UIbcAKFO1+DK2MSe BsgctOA4Hw5/19gQBGs0I0bzYqF4AQwFNAPWAQ6sY8bgHOrb+6BCczNbOoEha8KpT5IV 9ZdGgjocpyPcx8DBfbMjpurIkBABhhVzJrsboPVkPP/WNmrjd6iJDBm/Mrewnr8a/zZ9 p6mQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772037298; x=1772642098; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/X71c0eq0chbAJtrestek7XlSR8mXWIyGEkHLGooRRo=; b=rCKHwysV73L73EQQ+MjO/84cBSvS2/OH5nfsiCt13BRWJZlHUOVMBa58IwU4KN8MLu xrtjHxDQFzLtY68KSwH7q35p3tEH4zPmWt36NJGOVGQSEXIso01kgE5B4QFdzGjMEREY Kh0k+VRCUGVsQhpS92CpcNU2D0atpt99craIuSxT0jmuPRjpqDxKpLGMp1ppOxDhZPE4 LyKM/7u3z9POEmAxnL/7ILVcNP8DSH8/aF4iYxu0p7fxVj//fuOYMqV7dvDN2LZf3wRX Lxt39omfQldxCWnhskaOT/ORZ+rIMeg9n/hb+FQTyp18dUKnWBRfc83AJBpA90tSmw9q u2nA== X-Forwarded-Encrypted: i=1; AJvYcCXlH4cd+rgwf8l4UIKiANb1vdQnYwTvAM9GeadcGaINN3ZD/ikpi7OCy02+yW+lhmg1nu8YvryvLWXb7Ss=@vger.kernel.org X-Gm-Message-State: AOJu0YwcIp5JNaTREtu0nx7IgPm4w6WNMftx7FCRb7W3CGEkwCTSEkPi xc4zTnALCFnXJndXWeAD7Mg/+yfqmIe4niBnAPQ9IHdWbXk2BbPqfdFivhdUJbZ8AgsWb+RSuhC A0NVN3LkpjA2Xig== X-Received: from edyr17.prod.google.com ([2002:aa7:cfd1:0:b0:654:503e:5179]) (user=jackmanb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6402:1e8a:b0:65c:3684:3294 with SMTP id 4fb4d7f45d1cf-65ea4ee919emr10937951a12.11.1772037297847; Wed, 25 Feb 2026 08:34:57 -0800 (PST) Date: Wed, 25 Feb 2026 16:34:41 +0000 In-Reply-To: <20260225-page_alloc-unmapped-v1-0-e8808a03cd66@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260225-page_alloc-unmapped-v1-0-e8808a03cd66@google.com> X-Mailer: b4 0.14.3 Message-ID: <20260225-page_alloc-unmapped-v1-16-e8808a03cd66@google.com> Subject: [PATCH RFC 16/19] mm/page_alloc: introduce ALLOC_NOBLOCK From: Brendan Jackman To: Borislav Petkov , Dave Hansen , Peter Zijlstra , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Vlastimil Babka , Wei Xu , Johannes Weiner , Zi Yan Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, x86@kernel.org, rppt@kernel.org, Sumit Garg , derkling@google.com, reijiw@google.com, Will Deacon , rientjes@google.com, "Kalyazin, Nikita" , patrick.roy@linux.dev, "Itazuri, Takahiro" , Andy Lutomirski , David Kaplan , Thomas Gleixner , Brendan Jackman , Yosry Ahmed Content-Type: text/plain; charset="utf-8" This flag is set unless we can be sure the caller isn't in an atomic context. The allocator will soon start needing to call set_direct_map_* APIs which cannot be called with IRQs off. It will need to do this even before direct reclaim is possible. Despite the fact that, in principle, ALLOC_NOBLOCK is distinct from __GFP_DIRECT_RECLAIM, in order to avoid introducing a GFP flag, just infer the former based on whether the caller set the latter. This means that, in practice, ALLOC_NOBLOCK is just !__GFP_DIRECT_RECLAIM, except that it is not influenced by gfp_allowed_mask. This could change later, though. Call it ALLOC_NOBLOCK in order to try and mitigate confusion vs the recently-removed ALLOC_NON_BLOCK, which meant something different. Signed-off-by: Brendan Jackman --- mm/internal.h | 1 + mm/page_alloc.c | 29 ++++++++++++++++++++++------- 2 files changed, 23 insertions(+), 7 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 5be53d25c89b7..6f2eacf3d8f2c 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1390,6 +1390,7 @@ unsigned int reclaim_clean_pages_from_list(struct zone *zone, #define ALLOC_HIGHATOMIC 0x200 /* Allows access to MIGRATE_HIGHATOMIC */ #define ALLOC_TRYLOCK 0x400 /* Only use spin_trylock in allocation path */ #define ALLOC_KSWAPD 0x800 /* allow waking of kswapd, __GFP_KSWAPD_RECLAIM set */ +#define ALLOC_NOBLOCK 0x1000 /* Caller may be atomic */ /* Flags that allow allocations below the min watermark. */ #define ALLOC_RESERVES (ALLOC_HARDER|ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 42b807faca5fe..5576bd6a26b7b 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -4646,6 +4646,8 @@ gfp_to_alloc_flags(gfp_t gfp_mask, unsigned int order) (gfp_mask & (__GFP_HIGH | __GFP_KSWAPD_RECLAIM)); if (!(gfp_mask & __GFP_DIRECT_RECLAIM)) { + alloc_flags |= ALLOC_NOBLOCK; + /* * Not worth trying to allocate harder for __GFP_NOMEMALLOC even * if it can't schedule. @@ -4839,14 +4841,13 @@ check_retry_cpuset(int cpuset_mems_cookie, struct alloc_context *ac) static inline struct page * __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, - struct alloc_context *ac) + struct alloc_context *ac, unsigned int alloc_flags) { bool can_direct_reclaim = gfp_mask & __GFP_DIRECT_RECLAIM; bool can_compact = can_direct_reclaim && gfp_compaction_allowed(gfp_mask); bool nofail = gfp_mask & __GFP_NOFAIL; const bool costly_order = order > PAGE_ALLOC_COSTLY_ORDER; struct page *page = NULL; - unsigned int alloc_flags; unsigned long did_some_progress; enum compact_priority compact_priority; enum compact_result compact_result; @@ -4898,7 +4899,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, * kswapd needs to be woken up, and to avoid the cost of setting up * alloc_flags precisely. So we do that now. */ - alloc_flags = gfp_to_alloc_flags(gfp_mask, order); + alloc_flags |= gfp_to_alloc_flags(gfp_mask, order); /* * We need to recalculate the starting point for the zonelist iterator @@ -5124,6 +5125,18 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, return page; } +static inline unsigned int init_alloc_flags(gfp_t gfp_mask, unsigned int flags) +{ + /* + * If the caller allowed __GFP_DIRECT_RECLAIM, they can't be atomic. + * Note this is a separate determination from whether direct reclaim is + * actually allowed, it must happen before applying gfp_allowed_mask. + */ + if (!(gfp_mask & __GFP_DIRECT_RECLAIM)) + flags |= ALLOC_NOBLOCK; + return flags; +} + static inline bool prepare_alloc_pages(gfp_t gfp_mask, unsigned int order, int preferred_nid, nodemask_t *nodemask, struct alloc_context *ac, gfp_t *alloc_gfp, @@ -5205,7 +5218,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid, struct list_head *pcp_list; struct alloc_context ac; gfp_t alloc_gfp; - unsigned int alloc_flags = ALLOC_WMARK_LOW; + unsigned int alloc_flags = init_alloc_flags(gfp, ALLOC_WMARK_LOW); int nr_populated = 0, nr_account = 0; /* @@ -5346,7 +5359,7 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, int preferred_nid, nodemask_t *nodemask) { struct page *page; - unsigned int alloc_flags = ALLOC_WMARK_LOW; + unsigned int alloc_flags = init_alloc_flags(gfp, ALLOC_WMARK_LOW); gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */ struct alloc_context ac = { }; @@ -5391,7 +5404,7 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, */ ac.nodemask = nodemask; - page = __alloc_pages_slowpath(alloc_gfp, order, &ac); + page = __alloc_pages_slowpath(alloc_gfp, order, &ac, alloc_flags); out: if (memcg_kmem_online() && (gfp & __GFP_ACCOUNT) && page && @@ -7911,11 +7924,13 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t gfp_flags, int nid, unsigned */ gfp_t alloc_gfp = __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC | __GFP_COMP | gfp_flags; - unsigned int alloc_flags = ALLOC_TRYLOCK; + unsigned int alloc_flags = init_alloc_flags(alloc_gfp, ALLOC_TRYLOCK); struct alloc_context ac = { }; struct page *page; VM_WARN_ON_ONCE(gfp_flags & ~__GFP_ACCOUNT); + VM_WARN_ON_ONCE(!(alloc_flags & ALLOC_NOBLOCK)); + /* * In PREEMPT_RT spin_trylock() will call raw_spin_lock() which is * unsafe in NMI. If spin_trylock() is called from hard IRQ the current -- 2.51.2