From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qt1-f180.google.com (mail-qt1-f180.google.com [209.85.160.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4247E358367 for ; Thu, 4 Jun 2026 23:52:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=209.85.160.180 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780617168; cv=pass; b=omuHUAxmWcN6G+wy2cT85iQ9zccumSHe0jRdqMC0k6gVmiuGmpqtldt3T79CVtiTzkX90COC1WmHeTDoXoIytwafpSFb3Rjo1WiWEI/Dx3gPu1vhlixb8TkaaNKuSyk3hxI9a0xXXqz/K0UZtoXlvVBQTj1gOplHTc4fCPDHdDQ= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780617168; c=relaxed/simple; bh=JOZpyLhRcNbEi91yk9W05cxv7C+2vdRwz8ZhJjMqN7I=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=uULqoaqLuMQTR5mvIi/JBuP22U0DCxYCd+9gLWkyezpGw1VwyD5L/jfgCIDPnnyM27KHhzwFNSB5nHHp8pi2ohI7fizHb71gMirPmeEq6azPE83HbhV4/3ZgEK4KbahnnwVirX9em4Wqck2/OkawbdwCKuBWD0zBG4DBZWk48Rg= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=TOh+nIU3; arc=pass smtp.client-ip=209.85.160.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="TOh+nIU3" Received: by mail-qt1-f180.google.com with SMTP id d75a77b69052e-51765531803so259091cf.0 for ; Thu, 04 Jun 2026 16:52:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1780617165; cv=none; d=google.com; s=arc-20240605; b=fYG/BCNef8fFm5brCO7BI466MpKxe3zGITjmtVQCvXxZraoOsA4AGYtodjacyStQCp AlPl3Gpy6Po71g9NIwWs+kNtfZ4sBvawP5UChn+QcSl4RWlbLjxmpD4MphmhtBkR7k+Y P9iCviYCRKiooJA3TMl/gerRx1vSEqi+DF+akLN0+Fy0D9v0AGF3xyvfgyOCt+YrURwV uQymh6+EnsdN0sZX5ZC90B58jlgFUZSnKbnitdDv44BRWH/xnC9eJ4qLfnPSal68vTlM bn9bgT4P9VDryfmMjevUOh5aXxaJNgK8spwl13UWeQolJtwkPcQ5O0Yxrwgq3n7X4PLA 826g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=9qJVUjgRvfBeIKnlJo5ncmrdnqgqaZI84EFOu3Ayy7g=; fh=qv0lUGk4xj6kHCrvSqLBifDD7aD0KmfvcZ9osfb7I3Y=; b=g38PTRssdkCpSClCW5hTJ0jDHgZz52MKWEQvot0V/CmpPc58sG3L+hB9ufFOh/eUJH 0sR14pjWnkcKFvuiHl9dfFTneoyOdKFpLdbe5dLraFlNrJQDCuelPHNWVdwpyvolB6aw FJ4Vf4W71403eZsN4GrWBThoGg86AZwDyyCj9mBMThtuIxTD7DuMsiuC9c8Q2sWlvsnY +A59NkvJhM9fY8K/QmYVR0wi+B45N8UgVNIcDHGx8IYsTTkj/0SbFJAdaD0KKNXr7i7y WpCinYoAQFfBiQ50wdJWV6pj2stTmwqGYDIkD9Eoeh5wUvtdtazBt2/gAJFmRt06I+he C+Xw==; darn=vger.kernel.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1780617165; x=1781221965; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=9qJVUjgRvfBeIKnlJo5ncmrdnqgqaZI84EFOu3Ayy7g=; b=TOh+nIU3vqXF036nnENdAXChJGsnopTu3SzRWi1Ambx5fUQ/HKW2b6qj7CPFAQekTO XmKBSdMZzg64puoly2Ot/ZFcNuwLwsEqRXF0vJMt4EuwqteKRSXe7Oyno7tlqmtKFRsi 1f6ijtJYiNomRg3bkQfpM7KgHub1wMs8rP6Znw+psuQVmTbrD9XS9z/c2ms4hlR4Ni/S 47OfKiYhnwsQElCdHfHcibX6vqbTLBIVyJSM68A+rhqP0Twkr3cmsbz4CWjxOM2Xsafy dMEdWuHG9FtSZ+9AAyeAkVC35tCg2WoaKRqtmYMf6J9Js3/ZIUXgGLsTzbjZU5X5HzE0 5H6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780617165; x=1781221965; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=9qJVUjgRvfBeIKnlJo5ncmrdnqgqaZI84EFOu3Ayy7g=; b=hjBq14a/DJDaK5fE9AZVTdXUpQ9vf5gguhAaa9xr1sS+6aLepAP77JfJOpm8/RK7jX QofxKI3Tak4am/BFcCkEGrma5cRLNB916LKlssXJCSuDq2lV2RZAVBnP9d/mpOHnM7zO E+7EzlFwn7iQfrHFU8HtGwHxZN939S3QfDsJReRr2+XD26yo5K8zMYBMu+H90m+Awy2s ZYigRcs77D7bBC+u/pCVdFgD17vBC7gyYUvAfJS5nH7pv3raDYF/4359r4vwAd8E9zCM Sik90GNZUNtgniGSMpqlAem6TF9MTP05ssAE0LBDzNpfN2fGch5mur0HOR+DHo4aiNu3 CL0w== X-Forwarded-Encrypted: i=1; AFNElJ8qyJX9L7pdv8kkMifD6f8ksO/s5cU2ayiwObftR70eJ1ILvklt6Slz2wiuhaKvTmYB4DSPE37mewjV5rc=@vger.kernel.org X-Gm-Message-State: AOJu0Yy8b6dPHJ3kluYjhhcPQPbvnRw13E4OGHDsWdBh5d8iFg6ivHmP /VqnVv5jikbj9NNQBQeVeJYC6ygaFcAhBXsKKEqEwpHQUXH0pSqEVVEF9x3ntpyx4TJbfhTXhBd W+pMYk1gAo4YNt1EUYCiWOgFNnfPIAZQrOln4Mw5lh5/tvsOT2NMfgy0N X-Gm-Gg: Acq92OF9+R/MtFrEYqmrem5epe8T2jfRW7b53BV5qmHTc8qBwldDsyc5kwfnfcAu9ui 6aw4RzwP5i+iHje6L1Nd/X5OenbmZfrOsKJOWYjBuZITnmXM65zfkir1lNzpL0AMIrhahA6u9Lz UAbZ8D2ih4eRfGrKFR9CcbsGOSNZfEvCDDt7rbdGEP1BWMhzImFIwr86vfzvYKvumhBo1AtCGyS 1ZCzVLaPkwL0v2x9GMjRuPTIQ3oY8ddC43MZk/QpCZoRnc3Eos4Nz9PA8BKD64YwhHYnj/FXWxS uA0XqqxqHUNSQ/14 X-Received: by 2002:a05:622a:900a:b0:516:4f62:85ea with SMTP id d75a77b69052e-517984d8be7mr897491cf.21.1780617164336; Thu, 04 Jun 2026 16:52:44 -0700 (PDT) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <20260506022256.32664-1-hao.ge@linux.dev> <20260508171251.9bfb5e833859090d4480e222@linux-foundation.org> <20260526190015.d4edd406962fcc3a1ee4cb53@linux-foundation.org> <0e835109-9068-464f-88e2-c1847cd34c71@linux.dev> In-Reply-To: From: Suren Baghdasaryan Date: Thu, 4 Jun 2026 16:52:32 -0700 X-Gm-Features: AVVi8CfiDxsr9NDLNh0OqCMX2AsKHAD1fybPu2nOdA9CvIkGyhMZuKmWDIrdpdY Message-ID: Subject: Re: [PATCH v5] mm/alloc_tag: replace fixed-size early PFN array with dynamic linked list To: Hao Ge Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kent Overstreet , Roman Gushchin Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, Jun 3, 2026 at 7:47=E2=80=AFPM Hao Ge wrote: > > > On 2026/6/4 00:54, Suren Baghdasaryan wrote: > > On Tue, May 26, 2026 at 10:22=E2=80=AFPM Hao Ge wrot= e: > >> > >> On 2026/5/27 10:00, Andrew Morton wrote: > >>> On Fri, 8 May 2026 17:12:51 -0700 Andrew Morton wrote: > >>> > >>>> On Wed, 6 May 2026 10:22:56 +0800 Hao Ge wrote: > >>>> > >>>>> Pages allocated before page_ext is available have their codetag lef= t > >>>>> uninitialized. Track these early PFNs and clear their codetag in > >>>>> clear_early_alloc_pfn_tag_refs() to avoid "alloc_tag was not set" > >>>>> warnings when they are freed later. > >>>>> > >>>>> Currently a fixed-size array of 8192 entries is used, with a warnin= g if > >>>>> the limit is exceeded. However, the number of early allocations dep= ends > >>>>> on the number of CPUs and can be larger than 8192. > >>>>> > >>>>> Replace the fixed-size array with a dynamically allocated linked li= st > >>>>> of pfn_pool structs. Each node is allocated via alloc_page() and ma= pped > >>>>> to a pfn_pool containing a next pointer, an atomic slot counter, an= d a > >>>>> PFN array that fills the remainder of the page. > >>>>> > >>>>> The tracking pages themselves are allocated via alloc_page(), which > >>>>> would trigger __pgalloc_tag_add() -> alloc_tag_add_early_pfn() and > >>>>> recurse indefinitely. Introduce __GFP_NO_CODETAG (reuses the > >>>>> %__GFP_NO_OBJ_EXT bit) and pass gfp_flags through pgalloc_tag_add() > >>>>> so that the early path can skip recording allocations that carry th= is > >>>>> flag. > >>>> AI review asked a couple of things. I have a feeling we saw at leas= t > >>>> one of these, so probably already dealt with. > >>>> https://sashiko.dev/#/patchset/20260506022256.32664-1-hao.ge@l= inux.dev > >> Hi Andrew > >> > >> My apologies. I'm also waiting for Suren's review. He may have been ti= ed > >> up lately > >> > >> and might not have time to get to this. > >> > >> > >> Sashiko raised two issues this time. I've already responded to the fir= st > >> one. > >> > >> See the link below: > >> > >> https://lore.kernel.org/all/0b9969e2-b208-46c2-a9a5-bf620239275a@linux= .dev/ > >> > >> If I haven't missed any details, it should be a false positive. > > That seems to be the case. I wonder why Sashiko did not consider > > that... CC'ing Roman to see if Sashiko can be improved (unless we both > > are missing something). > > > >> > >> As for the second point, let me address it. > >> > >> The early PFN tracking window is entirely within mm_core_init(), > >> > >> which is called from start_kernel(): > >> > >> start_kernel() > >> > >> mm_core_init() > >> > >> memblock_free_all(); > >> > >> mem_init() //start early PFN tracking > >> > >> kmem_cache_init() // SLUB bootstra= p + > >> kmalloc caches > >> ... > >> page_ext_init() // clears > >> alloc_tag_add_early_pfn_ptr > >> > >> ... > >> > >> rest_init() //spawns kernel_init thread > >> > >> > >> kernel_init() =E2=86=92 kernel_init_freeable() // separate = thread, later > >> > >> smp_init() // secondary CPUs > >> come online here > >> > >> Within the early PFN window (mem_init() to page_ext_init()): > >> > >> 1. We are still in start_kernel(), single CPU. The buddy allocator > >> > >> was just initialized from memblock and should have plenty of free > >> > >> pages, so alloc_page() would likely be satisfied from the fast > >> > >> path. If so, the __GFP_NOFAIL without __GFP_DIRECT_RECLAIM > >> > >> check in the slowpath would not be reached. > >> > >> 2. Since only the boot CPU is running, alloc_page() targets the > >> > >> boot node, which has memory. So even if __GFP_THISNODE were > >> > >> inherited, it would not fail on the boot node during this window. > >> > >> > >> So Sashiko's analysis applies to the general case, and indeed the issu= es > >> > >> he raised could occur there. > >> > >> However, in the early boot scenario, I believe the current patch is sa= fe, > >> > >> even though it is not fully generic (after all, no one can predict > >> future use cases). > >> > >> Therefore, I agree with his suggestion that using a clean mask like > >> GFP_NOWAIT | __GFP_NOWARN. > > Hi Suren > > I've been thinking about the GFP flags issue for the past few days. > There are actually a couple of issues > > with the suggestion of using GFP_NOWAIT | __GFP_NOWARN. > > First, GFP_NOWAIT already includes __GFP_NOWARN, so it's redundant. > > Second, GFP_NOWAIT also includes __GFP_KSWAPD_RECLAIM, which is exactly > the same issue he flagged > > previously with GFP_ATOMIC =E2=80=94 it can still trigger wakeup_kswapd()= and > acquire scheduler locks, leading to > > potential deadlock in the same scenario he described. > > So I think __GFP_HIGH | __GFP_NO_CODETAG is the right choice. Make sense. TBH if we are failing to allocate a 0-order page during the early init stage then something is seriously wrong with the system. > > Since this runs under rcu_read_lock(), we can't have __GFP_DIRECT_RECLAIM= . > > And since Sashiko pointed out the scheduler lock concern with > __GFP_KSWAPD_RECLAIM, > > we can't have that either. > > I have posted the v6 revision, would you please kindly review it at your > convenience? > > https://lore.kernel.org/all/20260604024008.46592-1-hao.ge@linux.dev/ > > Thanks > > Best Regards > > Hao > > > This sounds good to me. With that change feel free to add: > > > > Acked-by: Suren Baghdasaryan > > > >> > >> In any case, I will wait for your and Suren's feedback. You may have > >> different opinions on this matter. > >> > >> > >> Thanks > >> > >> Best Regards > >> > >> Hao > >> > >> > >>> Please? > >>> > >>> Also, this patch has no evidence of human review. > >>> > >>> > >>> From: Hao Ge > >>> Subject: mm/alloc_tag: replace fixed-size early PFN array with dynami= c linked list > >>> Date: Wed, 6 May 2026 10:22:56 +0800 > >>> > >>> Pages allocated before page_ext is available have their codetag left > >>> uninitialized. Track these early PFNs and clear their codetag in > >>> clear_early_alloc_pfn_tag_refs() to avoid "alloc_tag was not set" war= nings > >>> when they are freed later. > >>> > >>> Currently a fixed-size array of 8192 entries is used, with a warning = if > >>> the limit is exceeded. However, the number of early allocations depe= nds > >>> on the number of CPUs and can be larger than 8192. > >>> > >>> Replace the fixed-size array with a dynamically allocated linked list= of > >>> pfn_pool structs. Each node is allocated via alloc_page() and mapped= to a > >>> pfn_pool containing a next pointer, an atomic slot counter, and a PFN > >>> array that fills the remainder of the page. > >>> > >>> The tracking pages themselves are allocated via alloc_page(), which w= ould > >>> trigger __pgalloc_tag_add() -> alloc_tag_add_early_pfn() and recurse > >>> indefinitely. Introduce __GFP_NO_CODETAG (reuses the %__GFP_NO_OBJ_E= XT > >>> bit) and pass gfp_flags through pgalloc_tag_add() so that the early p= ath > >>> can skip recording allocations that carry this flag. > >>> > >>> Link: https://lore.kernel.org/20260506022256.32664-1-hao.ge@linux.dev > >>> Signed-off-by: Hao Ge > >>> Suggested-by: Suren Baghdasaryan > >>> Cc: Brendan Jackman > >>> Cc: Johannes Weiner > >>> Cc: Kent Overstreet > >>> Cc: Michal Hocko > >>> Cc: Vlastimil Babka > >>> Cc: Zi Yan > >>> Signed-off-by: Andrew Morton > >>> --- > >>> > >>> include/linux/alloc_tag.h | 4 > >>> lib/alloc_tag.c | 145 +++++++++++++++++++++++----------= --- > >>> mm/page_alloc.c | 12 +- > >>> 3 files changed, 102 insertions(+), 59 deletions(-) > >>> > >>> --- a/include/linux/alloc_tag.h~mm-alloc_tag-replace-fixed-size-early= -pfn-array-with-dynamic-linked-list > >>> +++ a/include/linux/alloc_tag.h > >>> @@ -163,11 +163,11 @@ static inline void alloc_tag_sub_check(u > >>> { > >>> WARN_ONCE(ref && !ref->ct, "alloc_tag was not set\n"); > >>> } > >>> -void alloc_tag_add_early_pfn(unsigned long pfn); > >>> +void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t gfp_flags); > >>> #else > >>> static inline void alloc_tag_add_check(union codetag_ref *ref, str= uct alloc_tag *tag) {} > >>> static inline void alloc_tag_sub_check(union codetag_ref *ref) {} > >>> -static inline void alloc_tag_add_early_pfn(unsigned long pfn) {} > >>> +static inline void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t = gfp_flags) {} > >>> #endif > >>> > >>> /* Caller should verify both ref and tag to be valid */ > >>> --- a/lib/alloc_tag.c~mm-alloc_tag-replace-fixed-size-early-pfn-array= -with-dynamic-linked-list > >>> +++ a/lib/alloc_tag.c > >>> @@ -767,60 +767,95 @@ static __init bool need_page_alloc_taggi > >>> * their codetag uninitialized. Track these early PFNs so we can c= lear > >>> * their codetag refs later to avoid warnings when they are freed. > >>> * > >>> - * Early allocations include: > >>> - * - Base allocations independent of CPU count > >>> - * - Per-CPU allocations (e.g., CPU hotplug callbacks during smp_i= nit, > >>> - * such as trace ring buffers, scheduler per-cpu data) > >>> - * > >>> - * For simplicity, we fix the size to 8192. > >>> - * If insufficient, a warning will be triggered to alert the user. > >>> + * Each page is cast to a pfn_pool: the first few bytes hold metadat= a > >>> + * (next pointer and slot count), the remainder stores PFNs. > >>> + */ > >>> +struct pfn_pool { > >>> + struct pfn_pool *next; > >>> + atomic_t count; > >>> + unsigned long pfns[]; > >>> +}; > >>> + > >>> +#define PFN_POOL_SIZE ((PAGE_SIZE - offsetof(= struct pfn_pool, pfns)) / \ > >>> + sizeof(unsigned long)) > >>> + > >>> +/* > >>> + * Skip early PFN recording for a page allocation. Reuses the > >>> + * %__GFP_NO_OBJ_EXT bit. Used by __alloc_tag_add_early_pfn() to av= oid > >>> + * recursion when allocating pages for the early PFN tracking list > >>> + * itself. > >>> * > >>> - * TODO: Replace fixed-size array with dynamic allocation using > >>> - * a GFP flag similar to ___GFP_NO_OBJ_EXT to avoid recursion. > >>> + * Codetags of the pages allocated with __GFP_NO_CODETAG should be > >>> + * cleared (via clear_page_tag_ref()) before freeing the pages to pr= event > >>> + * alloc_tag_sub_check() from triggering a warning. > >>> */ > >>> -#define EARLY_ALLOC_PFN_MAX 8192 > >>> +#define __GFP_NO_CODETAG __GFP_NO_OBJ_EXT > >>> > >>> -static unsigned long early_pfns[EARLY_ALLOC_PFN_MAX] __initdata; > >>> -static atomic_t early_pfn_count __initdata =3D ATOMIC_INIT(0); > >>> +static struct pfn_pool *current_pfn_pool __initdata; > >>> > >>> -static void __init __alloc_tag_add_early_pfn(unsigned long pfn) > >>> +static void __init __alloc_tag_add_early_pfn(unsigned long pfn, gfp_= t gfp_flags) > >>> { > >>> - int old_idx, new_idx; > >>> + struct pfn_pool *pool; > >>> + int idx; > >>> > >>> do { > >>> - old_idx =3D atomic_read(&early_pfn_count); > >>> - if (old_idx >=3D EARLY_ALLOC_PFN_MAX) { > >>> - pr_warn_once("Early page allocations before pag= e_ext init exceeded EARLY_ALLOC_PFN_MAX (%d)\n", > >>> - EARLY_ALLOC_PFN_MAX); > >>> - return; > >>> + pool =3D READ_ONCE(current_pfn_pool); > >>> + if (!pool || atomic_read(&pool->count) >=3D PFN_POOL_SI= ZE) { > >>> + gfp_t gfp =3D gfp_flags & ~(__GFP_DIRECT_RECLAI= M | GFP_ZONEMASK); > >>> + struct page *new_page =3D alloc_page(gfp | __GF= P_NO_CODETAG); > >>> + struct pfn_pool *new; > >>> + > >>> + if (!new_page) { > >>> + pr_warn_once("early PFN tracking page a= llocation failed\n"); > >>> + return; > >>> + } > >>> + new =3D page_address(new_page); > >>> + new->next =3D pool; > >>> + atomic_set(&new->count, 0); > >>> + if (cmpxchg(¤t_pfn_pool, pool, new) !=3D = pool) { > >>> + clear_page_tag_ref(new_page); > >>> + __free_page(new_page); > >>> + continue; > >>> + } > >>> + pool =3D new; > >>> } > >>> - new_idx =3D old_idx + 1; > >>> - } while (!atomic_try_cmpxchg(&early_pfn_count, &old_idx, new_id= x)); > >>> + idx =3D atomic_read(&pool->count); > >>> + if (idx >=3D PFN_POOL_SIZE) > >>> + continue; > >>> + if (atomic_cmpxchg(&pool->count, idx, idx + 1) =3D=3D i= dx) > >>> + break; > >>> + } while (1); > >>> > >>> - early_pfns[old_idx] =3D pfn; > >>> + pool->pfns[idx] =3D pfn; > >>> } > >>> > >>> -typedef void alloc_tag_add_func(unsigned long pfn); > >>> +typedef void alloc_tag_add_func(unsigned long pfn, gfp_t gfp_flags); > >>> static alloc_tag_add_func __rcu *alloc_tag_add_early_pfn_ptr __ref= data =3D > >>> RCU_INITIALIZER(__alloc_tag_add_early_pfn); > >>> > >>> -void alloc_tag_add_early_pfn(unsigned long pfn) > >>> +void alloc_tag_add_early_pfn(unsigned long pfn, gfp_t gfp_flags) > >>> { > >>> alloc_tag_add_func *alloc_tag_add; > >>> > >>> if (static_key_enabled(&mem_profiling_compressed)) > >>> return; > >>> > >>> + /* Skip allocations for the tracking list itself to avoid recur= sion. */ > >>> + if (gfp_flags & __GFP_NO_CODETAG) > >>> + return; > >>> + > >>> rcu_read_lock(); > >>> alloc_tag_add =3D rcu_dereference(alloc_tag_add_early_pfn_ptr)= ; > >>> if (alloc_tag_add) > >>> - alloc_tag_add(pfn); > >>> + alloc_tag_add(pfn, gfp_flags); > >>> rcu_read_unlock(); > >>> } > >>> > >>> static void __init clear_early_alloc_pfn_tag_refs(void) > >>> { > >>> - unsigned int i; > >>> + struct pfn_pool *pool, *next; > >>> + struct page *page; > >>> + int i; > >>> > >>> if (static_key_enabled(&mem_profiling_compressed)) > >>> return; > >>> @@ -829,37 +864,45 @@ static void __init clear_early_alloc_pfn > >>> /* Make sure we are not racing with __alloc_tag_add_early_pfn(= ) */ > >>> synchronize_rcu(); > >>> > >>> - for (i =3D 0; i < atomic_read(&early_pfn_count); i++) { > >>> - unsigned long pfn =3D early_pfns[i]; > >>> + for (pool =3D current_pfn_pool; pool; pool =3D next) { > >>> + int nr_pfns =3D atomic_read(&pool->count); > >>> + > >>> + for (i =3D 0; i < nr_pfns; i++) { > >>> + unsigned long pfn =3D pool->pfns[i]; > >>> > >>> - if (pfn_valid(pfn)) { > >>> - struct page *page =3D pfn_to_page(pfn); > >>> - union pgtag_ref_handle handle; > >>> - union codetag_ref ref; > >>> - > >>> - if (get_page_tag_ref(page, &ref, &handle)) { > >>> - /* > >>> - * An early-allocated page could be fre= ed and reallocated > >>> - * after its page_ext is initialized bu= t before we clear it. > >>> - * In that case, it already has a valid= tag set. > >>> - * We should not overwrite that valid t= ag with CODETAG_EMPTY. > >>> - * > >>> - * Note: there is still a small race wi= ndow between checking > >>> - * ref.ct and calling set_codetag_empty= (). We accept this > >>> - * race as it's unlikely and the extra = complexity of atomic > >>> - * cmpxchg is not worth it for this deb= ug-only code path. > >>> - */ > >>> - if (ref.ct) { > >>> + if (pfn_valid(pfn)) { > >>> + union pgtag_ref_handle handle; > >>> + union codetag_ref ref; > >>> + > >>> + if (get_page_tag_ref(pfn_to_page(pfn), = &ref, &handle)) { > >>> + /* > >>> + * An early-allocated page coul= d be freed and reallocated > >>> + * after its page_ext is initia= lized but before we clear it. > >>> + * In that case, it already has= a valid tag set. > >>> + * We should not overwrite that= valid tag > >>> + * with CODETAG_EMPTY. > >>> + * > >>> + * Note: there is still a small= race window between checking > >>> + * ref.ct and calling set_codet= ag_empty(). We accept this > >>> + * race as it's unlikely and th= e extra complexity of atomic > >>> + * cmpxchg is not worth it for = this debug-only code path. > >>> + */ > >>> + if (ref.ct) { > >>> + put_page_tag_ref(handle= ); > >>> + continue; > >>> + } > >>> + > >>> + set_codetag_empty(&ref); > >>> + update_page_tag_ref(handle, &re= f); > >>> put_page_tag_ref(handle); > >>> - continue; > >>> } > >>> - > >>> - set_codetag_empty(&ref); > >>> - update_page_tag_ref(handle, &ref); > >>> - put_page_tag_ref(handle); > >>> } > >>> } > >>> > >>> + next =3D pool->next; > >>> + page =3D virt_to_page(pool); > >>> + clear_page_tag_ref(page); > >>> + __free_page(page); > >>> } > >>> } > >>> #else /* !CONFIG_MEM_ALLOC_PROFILING_DEBUG */ > >>> --- a/mm/page_alloc.c~mm-alloc_tag-replace-fixed-size-early-pfn-array= -with-dynamic-linked-list > >>> +++ a/mm/page_alloc.c > >>> @@ -1255,7 +1255,7 @@ void __clear_page_tag_ref(struct page *p > >>> /* Should be called only if mem_alloc_profiling_enabled() */ > >>> static noinline > >>> void __pgalloc_tag_add(struct page *page, struct task_struct *task= , > >>> - unsigned int nr) > >>> + unsigned int nr, gfp_t gfp_flags) > >>> { > >>> union pgtag_ref_handle handle; > >>> union codetag_ref ref; > >>> @@ -1269,17 +1269,17 @@ void __pgalloc_tag_add(struct page *page > >>> * page_ext is not available yet, record the pfn so we= can > >>> * clear the tag ref later when page_ext is initialize= d. > >>> */ > >>> - alloc_tag_add_early_pfn(page_to_pfn(page)); > >>> + alloc_tag_add_early_pfn(page_to_pfn(page), gfp_flags); > >>> if (task->alloc_tag) > >>> alloc_tag_set_inaccurate(task->alloc_tag); > >>> } > >>> } > >>> > >>> static inline void pgalloc_tag_add(struct page *page, struct task_= struct *task, > >>> - unsigned int nr) > >>> + unsigned int nr, gfp_t gfp_flags) > >>> { > >>> if (mem_alloc_profiling_enabled()) > >>> - __pgalloc_tag_add(page, task, nr); > >>> + __pgalloc_tag_add(page, task, nr, gfp_flags); > >>> } > >>> > >>> /* Should be called only if mem_alloc_profiling_enabled() */ > >>> @@ -1312,7 +1312,7 @@ static inline void pgalloc_tag_sub_pages > >>> #else /* CONFIG_MEM_ALLOC_PROFILING */ > >>> > >>> static inline void pgalloc_tag_add(struct page *page, struct task_= struct *task, > >>> - unsigned int nr) {} > >>> + unsigned int nr, gfp_t gfp_flags) {} > >>> static inline void pgalloc_tag_sub(struct page *page, unsigned int= nr) {} > >>> static inline void pgalloc_tag_sub_pages(struct alloc_tag *tag, un= signed int nr) {} > >>> > >>> @@ -1867,7 +1867,7 @@ inline void post_alloc_hook(struct page > >>> > >>> set_page_owner(page, order, gfp_flags); > >>> page_table_check_alloc(page, order); > >>> - pgalloc_tag_add(page, current, 1 << order); > >>> + pgalloc_tag_add(page, current, 1 << order, gfp_flags); > >>> } > >>> > >>> static void prep_new_page(struct page *page, unsigned int order, g= fp_t gfp_flags, > >>> _ > >>>