From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6EFDF3E557E for ; Sun, 26 Jul 2026 22:23:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.69 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785104638; cv=none; b=hpWyZcw31joqKbhwWqCJRhFarBBsXtrg+6SBnRTySQHvnAknmCI5+VGM4ueohdcEbqIijHT2Bo46ETFQV4dWheqYs3bl7+E8Ok3xGej79oJYV8gWIfOCz9U+MAN9RoJ5GHVwFAHUroQrU5XghLcc33r4nXVWNbve9M/Tk+55TJo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785104638; c=relaxed/simple; bh=Q8TsfVp/Lw9WBN2Zbx8568QKkggnjUZmnB7wU4ipxEM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GiyJdhd17RQZO8453Ncq9WmEbevePnfBY+ab/FKH0SCQHrM8oufbKUtbVxJN9jbiHZZIpzn4AVfTwbpcYn/o5V/7/u3bUGHNq9jJwP2hkCeodJucGu2fV9lzuVU1RBwdS65KlEcJLbpL+cj9riik4tjDgm0lbDNzLuVQAzFq8Dc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jackmanb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=h7g+6hqJ; arc=none smtp.client-ip=209.85.128.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jackmanb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="h7g+6hqJ" Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-496b408a5c0so7838175e9.0 for ; Sun, 26 Jul 2026 15:23:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1785104632; x=1785709432; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=cwoLB2iOFwVZoH8Cy33gw5t1pZIEJdBAZWDo2kjEDd0=; b=h7g+6hqJDMN9PGiXqb0zdoi/c80Hliow+kBBxDPOGZtSkWCGkEwcypg+hBq3I6Owtl SsFj7xcs6IA0oLaFf0uHni9mSnntk0krkvjaAHX9oGg3zJo3FBUIHD0L4112UVMbbtS5 MmbNz6Jrm6edg+ssOMnothkVtjh8S9+eGz2m8w482vOOvHC914T/YQjguwnnrqPr3VOV VDOt2UHYzAhpugf2YFS82N69VnADs1i3/l7hAkH/1iXSWvcwd4+Eqp+h2Y7uDchfV65e DWwdRR7AMV/SavWoPCx8ExhJf+YZDsa+3VQPDwvWbEyOfb5QQg1KKINIBu6I5GWFJ9Hh 5Pnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785104632; x=1785709432; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=cwoLB2iOFwVZoH8Cy33gw5t1pZIEJdBAZWDo2kjEDd0=; b=mm03wvv+0rxJf1gfPoXfmQlCFzCdJtnEfKBs0uCBNRst2ZsZIPEhi7rQoAc/2ZUWrx 4rX9YgIrIfvxGNYv83NMVSIJCnZGjnYI4PNpGNuP2pJV6Eapa9HfjxkLvRk4tYKkrbLp ZalU7oxUY5S9ovjACPVnIN72E6VSgadxriNM2lRtVZC5ZVV0mm9UuM6BsQZdWX3JDJ2h TB3+BBXI+ifaF7FMUII5okcpu3XgzAGMn7/2YpVAvI23c/vWkWlZ3+8K33vTUn0pGvaf 6pv6gb9+N/j5CbolGuXjno3K3FMV8iRj3W5aDT39GorprW0BuxfvjDanvnj/xzefb5qX wCeg== X-Forwarded-Encrypted: i=1; AHgh+RopZWMrVOz+AqrDSNEqQ3txNrrDVYE63wlGAHqvdBfpnqxi5FfHCD+qsgPa3NmFbEhguQMawkRekOIjcxY=@vger.kernel.org X-Gm-Message-State: AOJu0YwxtbwtIVnaO8L/pBcGO5xXgHsu2Q5B+ql+gJ5gYtSYSu7k6/LO xjZlZBWXCbd1AQcs4WkaIjsoWiALFzFdVcSz9kvuP1QoceV2soZgp3OAFXAAmyBe4YsiWsBM17C fqauZZ/W8b/ThYQ== X-Received: from wmox16.prod.google.com ([2002:a05:600c:1790:b0:495:4d5f:df26]) (user=jackmanb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:6986:b0:495:4dea:f7f5 with SMTP id 5b1f17b1804b1-496b5726cccmr92611625e9.29.1785104631662; Sun, 26 Jul 2026 15:23:51 -0700 (PDT) Date: Sun, 26 Jul 2026 22:22:53 +0000 In-Reply-To: <20260726-page_alloc-unmapped-v3-0-6f5729aa9832@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260726-page_alloc-unmapped-v3-0-6f5729aa9832@google.com> X-Mailer: b4 0.16-dev Message-ID: <20260726-page_alloc-unmapped-v3-20-6f5729aa9832@google.com> Subject: [PATCH v3 20/26] mm/page_alloc: introduce ALLOC_NOBLOCK From: Brendan Jackman To: Borislav Petkov , Dave Hansen , Peter Zijlstra , Andrew Morton , David Hildenbrand , Vlastimil Babka , Mike Rapoport , Wei Xu , Johannes Weiner , Zi Yan , Lorenzo Stoakes Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, x86@kernel.org, rppt@kernel.org, Sumit Garg , Will Deacon , rientjes@google.com, "Kalyazin, Nikita" , patrick.roy@linux.dev, "Itazuri, Takahiro" , Andy Lutomirski , David Kaplan , Thomas Gleixner , Yosry Ahmed , Patrick Bellasi , Reiji Watanabe , Sean Christopherson , Brendan Jackman Content-Type: text/plain; charset="utf-8" This flag is set unless we can be sure the caller isn't in an atomic context. The allocator will soon start needing to call set_direct_map_* APIs which cannot be called with IRQs off. It will need to do this even before direct reclaim is possible. Despite the fact that, in principle, ALLOC_NOBLOCK is distinct from __GFP_DIRECT_RECLAIM, in order to avoid introducing a GFP flag, just infer the former based on whether the caller set the latter. This means that, in practice, ALLOC_NOBLOCK is just !__GFP_DIRECT_RECLAIM, except that it is not influenced by gfp_allowed_mask. This requires some rather ugly plumbing to get it into the slowpath without clobbering it. Call it ALLOC_NOBLOCK in order to try and mitigate confusion vs the recently-removed ALLOC_NON_BLOCK, which meant something different. Signed-off-by: Brendan Jackman --- mm/page_alloc.c | 23 +++++++++++++++++++---- mm/page_alloc.h | 1 + 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index fb522a09f2e60..ecdd3780192e8 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5286,6 +5286,19 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, return page; } +/* Set alloc flags before applying gfp_allowed_mask. */ +static inline unsigned int init_alloc_flags(gfp_t gfp_mask, unsigned int alloc_flags) +{ + /* + * If the caller allowed __GFP_DIRECT_RECLAIM, they can't be atomic. + * Note this is a separate determination from whether direct reclaim is + * actually allowed, it must happen before applying gfp_allowed_mask. + */ + if (!(gfp_mask & __GFP_DIRECT_RECLAIM)) + alloc_flags |= ALLOC_NOBLOCK; + return alloc_flags; +} + static inline bool prepare_alloc_pages(gfp_t gfp_mask, unsigned int order, int preferred_nid, nodemask_t *nodemask, struct alloc_context *ac, gfp_t *alloc_gfp, @@ -5364,8 +5377,10 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid, struct zoneref *z; struct per_cpu_pages *pcp; struct list_head *pcp_list; - struct alloc_context ac; - unsigned int alloc_flags = ALLOC_WMARK_LOW; + struct alloc_context ac = { + .alloc_flags = init_alloc_flags(gfp, ALLOC_DEFAULT), + }; + unsigned int alloc_flags = ac.alloc_flags | ALLOC_WMARK_LOW; int nr_populated = 0, nr_account = 0; /* @@ -5584,9 +5599,9 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, struct page *page; gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */ struct alloc_context ac = { - .alloc_flags = alloc_flags, + .alloc_flags = init_alloc_flags(gfp, alloc_flags), }; - unsigned int fastpath_alloc_flags = alloc_flags; + unsigned int fastpath_alloc_flags = ac.alloc_flags; /* Other flags could be supported later if needed. */ if (WARN_ON(alloc_flags & ~(ALLOC_NOLOCK | ALLOC_NO_CODETAG))) diff --git a/mm/page_alloc.h b/mm/page_alloc.h index 2307b173a459f..02db12c9a1dc2 100644 --- a/mm/page_alloc.h +++ b/mm/page_alloc.h @@ -75,6 +75,7 @@ */ #define ALLOC_UNMAPPED 0x2000 #endif +#define ALLOC_NOBLOCK 0x4000 /* Caller may be atomic */ /* Flags that allow allocations below the min watermark. */ #define ALLOC_RESERVES (ALLOC_HARDER|ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|ALLOC_OOM) -- 2.54.0