From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-lf1-f44.google.com (mail-lf1-f44.google.com [209.85.167.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EAF4D1ACEA9 for ; Wed, 18 Dec 2024 14:26:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.44 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734532011; cv=none; b=X7zS7vBKZ9owXRgQJfSJmlgpvl81htDLEsz/itFeVppY4O9ZIC9yNFfErQ85pGc5g1Vq80D7cOOgpmybRGe5+FfIsJVJ0sizpr+r9GDUZ33Z2fl1iUygXAd2FwHo65VwPKIL8cTAIxO2AMRKvM/ArbN989+zEF7/B6aank6xarw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734532011; c=relaxed/simple; bh=6N9xSUt/+atRBtfdzFq3mHK8VIfr7SEgnNjj4IRkd+c=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=VABGAZweJbn8gJRM8gYj7L48Y3Loceij+bPsLHPszSJZhSlekhYoEBbwTuThmTIXiFCi0ewpzuVCV1uNNAOgIUFvHAG0Cc/6UNPZVMtd6GS38KYM49NeTObCTsuLSq8EvyAwS1yBDneJWzgmhsuTlbQ0ywOk2Ukc2N/Rz4qAfDE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XA3ydLpV; arc=none smtp.client-ip=209.85.167.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XA3ydLpV" Received: by mail-lf1-f44.google.com with SMTP id 2adb3069b0e04-540201cfedbso6507312e87.3 for ; Wed, 18 Dec 2024 06:26:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734532007; x=1735136807; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=70sWeERMptzwKALDyA5C6mRflO5NDr8MtSi8T8llx40=; b=XA3ydLpVLyV2kXzXYHiYgtkRpYs5DgXLGuXaLBX4wL1JMAQ8zpIcmNexe/0oHGogL5 zrhYQmz3uh3LqcCNU9XMKPS2WCxr04f9mAE0gT11jbmvpe6qrjwPGQi1abNQmcjx/jqH Wegwm2OIXyclsGDOmraGoXUj+r5FatzHMK603+NwqUbDhcPgbdj8DOrGmDr9kTPR9BNf 1+4FLJfcFAP+RQZ9VT7YIJZAttN1tbKMwmJlmR1NCYxG8UdYC4HgLt0Lc1CPVSHXJ3LA mPByxLXQTXpGmH8Sb0ngYhoUVc1aFQ3goCmFxxP+uv596D2NpLzLv+z9ZH0Rddc/Uz6H Rs3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734532007; x=1735136807; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=70sWeERMptzwKALDyA5C6mRflO5NDr8MtSi8T8llx40=; b=mcWhHqhasXojNTEQOolJYluCCtngShQZPqjuRIceOktTWncMHn6Bo3zwxhA9g9VrFj Fpz3Cz/hA6Onco6FFuxhEDxA5OPvmEd0JwqSEGN5v06qEDqwLwsiE50KL7SAFToKp3oo e6LddBkA0WdxuN7IME4D+FD18cJyzCIAjJgpTK+LIw3EHMWRN5QDa/9xA1srZ7mN0sie 5Xc6ohdFgX4mGTMutZSgIiULSsEwi4IP6k6iKY27DqDhHUemLUFiuHHaZrsXwgUiMBv7 MsyJKKOYTBvPqwLk9Oc/i58oo9fcQsNXiLpuZCzay2SYODPpfpvoJfZpofwJTixBoXVw mLaw== X-Forwarded-Encrypted: i=1; AJvYcCWHNz5+CapeRWbq97fGa3dmS0A2+1a0VMp68VUF+vKKdq8bk2HMBNasZKsrrqRnicvu4vT+EP/AMCnH9vc=@vger.kernel.org X-Gm-Message-State: AOJu0YxH+hkX7EYgaHov4FG+FJ9pj0Hf9N/hJhAcvkIiTri62A/vBsR3 e6vOyaAIVwxPALfKgfWjuhmVyMhFf0wKJHrNaxeuQGoScwRfgxk/ X-Gm-Gg: ASbGncsuBXDp3tgR9NxFWwBcsqfyTVm91MamG/iXRs2Q8U57tkWTXE62e4wlWQPUWQO fo8gAKNe2yMJ+8a0gngrWxxkhnho8ry5Rmw0uaGOylDmOPtbxBvledeXw/1ft3S/IY/HHGX84pl uxH8oahZ1inV49BtLqcNGNgGLji/5azHwTkle0/EobN2GW2etXHNRKg0eaiORYWC4k23SKDGpOu g3Ia0mtIFOXR4yhvJIdcsyaHOdHXFe0WP6vbolsC0BjVB8od5yxTybdIXUWw8c2oHhL958paAxS yCmhd78iLR9X+76n6TBVI5f8jC4CBl6R4kMkri198w== X-Google-Smtp-Source: AGHT+IEpIIEIJ2UXiiLBdwl8jw7i267QzdNiytJ3jSLNDR3hUDDJnoXHvBFmKiclhZAWiuXzUvG2cA== X-Received: by 2002:a05:6512:1254:b0:53e:3a01:cf4f with SMTP id 2adb3069b0e04-541e674691amr1196803e87.14.1734532006593; Wed, 18 Dec 2024 06:26:46 -0800 (PST) Received: from ?IPV6:2001:678:a5c:1202:4fb5:f16a:579c:6dcb? (soda.int.kasm.eu. [2001:678:a5c:1202:4fb5:f16a:579c:6dcb]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-54120ba9a33sm1482892e87.98.2024.12.18.06.26.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 18 Dec 2024 06:26:45 -0800 (PST) Message-ID: Date: Wed, 18 Dec 2024 15:26:42 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 02/12] mm: pgtable: introduce generic p4d_alloc_one() and p4d_free() To: Qi Zheng , peterz@infradead.org, tglx@linutronix.de, david@redhat.com, jannh@google.com, hughd@google.com, yuzhao@google.com, willy@infradead.org, muchun.song@linux.dev, vbabka@kernel.org, lorenzo.stoakes@oracle.com, akpm@linux-foundation.org, rientjes@google.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org References: Content-Language: en-US, sv-SE From: Klara Modin In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi, On 2024-12-14 10:02, Qi Zheng wrote: > Several architectures (arm64, riscv, x86) define p4d_alloc_one() as a > wrapper for get_zeroed_page() and p4d_free() as a wrapper for free_page(). > > For these architectures, provide a generic implementation in > asm-generic/pgalloc.h and convert them to use it. And like other levels > of page tables, add statistics for P4D level page table. > > For s390, it also defines p4d_alloc_one() and p4d_free(), but it uses its > own logic, so skip it. > > Signed-off-by: Qi Zheng > --- > arch/arm64/include/asm/pgalloc.h | 15 ++++----- > arch/riscv/include/asm/pgalloc.h | 25 ++++++--------- > arch/x86/include/asm/pgalloc.h | 16 ++++------ > arch/x86/mm/pgtable.c | 3 ++ > include/asm-generic/pgalloc.h | 55 ++++++++++++++++++++++++++++++++ > include/linux/mm.h | 16 ++++++++++ > 6 files changed, 98 insertions(+), 32 deletions(-) > > diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h > index e75422864d1bd..679c530549327 100644 > --- a/arch/arm64/include/asm/pgalloc.h > +++ b/arch/arm64/include/asm/pgalloc.h > @@ -15,6 +15,8 @@ > > #define __HAVE_ARCH_PGD_FREE > #define __HAVE_ARCH_PUD_FREE > +#define __HAVE_ARCH_P4D_ALLOC_ONE > +#define __HAVE_ARCH_P4D_FREE > #include > > #define PGD_SIZE (PTRS_PER_PGD * sizeof(pgd_t)) > @@ -87,19 +89,16 @@ static inline void pgd_populate(struct mm_struct *mm, pgd_t *pgdp, p4d_t *p4dp) > > static inline p4d_t *p4d_alloc_one(struct mm_struct *mm, unsigned long addr) > { > - gfp_t gfp = GFP_PGTABLE_USER; > + if (!pgtable_l5_enabled()) > + return NULL; > > - if (mm == &init_mm) > - gfp = GFP_PGTABLE_KERNEL; > - return (p4d_t *)get_zeroed_page(gfp); > + return __p4d_alloc_one(mm, addr); > } > > static inline void p4d_free(struct mm_struct *mm, p4d_t *p4d) > { > - if (!pgtable_l5_enabled()) > - return; > - BUG_ON((unsigned long)p4d & (PAGE_SIZE-1)); > - free_page((unsigned long)p4d); > + if (pgtable_l5_enabled()) > + __p4d_free(mm, p4d); > } > > #define __p4d_free_tlb(tlb, p4d, addr) p4d_free((tlb)->mm, p4d) > diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h > index f52264304f772..bb6e1c5f1fb19 100644 > --- a/arch/riscv/include/asm/pgalloc.h > +++ b/arch/riscv/include/asm/pgalloc.h > @@ -14,6 +14,8 @@ > #ifdef CONFIG_MMU > #define __HAVE_ARCH_PUD_ALLOC_ONE > #define __HAVE_ARCH_PUD_FREE > +#define __HAVE_ARCH_P4D_ALLOC_ONE > +#define __HAVE_ARCH_P4D_FREE > #include > > static inline void riscv_tlb_remove_ptdesc(struct mmu_gather *tlb, void *pt) > @@ -118,21 +120,10 @@ static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pud, > #define p4d_alloc_one p4d_alloc_one > static inline p4d_t *p4d_alloc_one(struct mm_struct *mm, unsigned long addr) > { > - if (pgtable_l5_enabled) { > - gfp_t gfp = GFP_PGTABLE_USER; > - > - if (mm == &init_mm) > - gfp = GFP_PGTABLE_KERNEL; > - return (p4d_t *)get_zeroed_page(gfp); > - } > + if (!pgtable_l5_enabled) > + return NULL; > > - return NULL; > -} > - > -static inline void __p4d_free(struct mm_struct *mm, p4d_t *p4d) > -{ > - BUG_ON((unsigned long)p4d & (PAGE_SIZE-1)); > - free_page((unsigned long)p4d); > + return __p4d_alloc_one(mm, addr); > } > > #define p4d_free p4d_free > @@ -145,8 +136,12 @@ static inline void p4d_free(struct mm_struct *mm, p4d_t *p4d) > static inline void __p4d_free_tlb(struct mmu_gather *tlb, p4d_t *p4d, > unsigned long addr) > { > - if (pgtable_l5_enabled) > + if (pgtable_l5_enabled) { > + struct ptdesc *ptdesc = virt_to_ptdesc(p4d); > + > + pagetable_p4d_dtor(ptdesc); > riscv_tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); > + } > } > #endif /* __PAGETABLE_PMD_FOLDED */ > > diff --git a/arch/x86/include/asm/pgalloc.h b/arch/x86/include/asm/pgalloc.h > index dcd836b59bebd..d9bc6cae77c9e 100644 > --- a/arch/x86/include/asm/pgalloc.h > +++ b/arch/x86/include/asm/pgalloc.h > @@ -8,6 +8,8 @@ > > #define __HAVE_ARCH_PTE_ALLOC_ONE > #define __HAVE_ARCH_PGD_FREE > +#define __HAVE_ARCH_P4D_ALLOC_ONE > +#define __HAVE_ARCH_P4D_FREE > #include > > static inline int __paravirt_pgd_alloc(struct mm_struct *mm) { return 0; } > @@ -149,20 +151,16 @@ static inline void pgd_populate_safe(struct mm_struct *mm, pgd_t *pgd, p4d_t *p4 > > static inline p4d_t *p4d_alloc_one(struct mm_struct *mm, unsigned long addr) > { > - gfp_t gfp = GFP_KERNEL_ACCOUNT; > + if (!pgtable_l5_enabled()) > + return NULL; > > - if (mm == &init_mm) > - gfp &= ~__GFP_ACCOUNT; > - return (p4d_t *)get_zeroed_page(gfp); > + return __p4d_alloc_one(mm, addr); > } > > static inline void p4d_free(struct mm_struct *mm, p4d_t *p4d) > { > - if (!pgtable_l5_enabled()) > - return; > - > - BUG_ON((unsigned long)p4d & (PAGE_SIZE-1)); > - free_page((unsigned long)p4d); > + if (pgtable_l5_enabled()) > + return __p4d_free(mm, p4d); > } > > extern void ___p4d_free_tlb(struct mmu_gather *tlb, p4d_t *p4d); > diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c > index 69a357b15974a..3d6e84da45b24 100644 > --- a/arch/x86/mm/pgtable.c > +++ b/arch/x86/mm/pgtable.c > @@ -94,6 +94,9 @@ void ___pud_free_tlb(struct mmu_gather *tlb, pud_t *pud) > #if CONFIG_PGTABLE_LEVELS > 4 > void ___p4d_free_tlb(struct mmu_gather *tlb, p4d_t *p4d) > { > + struct ptdesc *ptdesc = virt_to_ptdesc(p4d); > + > + pagetable_p4d_dtor(ptdesc); > paravirt_release_p4d(__pa(p4d) >> PAGE_SHIFT); > paravirt_tlb_remove_table(tlb, virt_to_page(p4d)); > } > diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h > index 7c48f5fbf8aa7..dbf61819b3581 100644 > --- a/include/asm-generic/pgalloc.h > +++ b/include/asm-generic/pgalloc.h > @@ -215,6 +215,61 @@ static inline void pud_free(struct mm_struct *mm, pud_t *pud) > > #endif /* CONFIG_PGTABLE_LEVELS > 3 */ > > +#if CONFIG_PGTABLE_LEVELS > 4 > + > +static inline p4d_t *__p4d_alloc_one_noprof(struct mm_struct *mm, unsigned long addr) > +{ > + gfp_t gfp = GFP_PGTABLE_USER; > + struct ptdesc *ptdesc; > + > + if (mm == &init_mm) > + gfp = GFP_PGTABLE_KERNEL; > + gfp &= ~__GFP_HIGHMEM; > + > + ptdesc = pagetable_alloc_noprof(gfp, 0); > + if (!ptdesc) > + return NULL; > + > + pagetable_p4d_ctor(ptdesc); > + return ptdesc_address(ptdesc); > +} > +#define __p4d_alloc_one(...) alloc_hooks(__p4d_alloc_one_noprof(__VA_ARGS__)) > + > +#ifndef __HAVE_ARCH_P4D_ALLOC_ONE > +/** > + * p4d_alloc_one - allocate memory for a P4D-level page table > + * @mm: the mm_struct of the current context > + * > + * Allocate memory for a page table using %GFP_PGTABLE_USER for user context > + * and %GFP_PGTABLE_KERNEL for kernel context. > + * > + * Return: pointer to the allocated memory or %NULL on error > + */ > +static inline p4d_t *p4d_alloc_one_noprof(struct mm_struct *mm, unsigned long addr) > +{ > + return __p4d_alloc_one_noprof(mm, addr); > +} > +#define p4d_alloc_one(...) alloc_hooks(p4d_alloc_one_noprof(__VA_ARGS__)) > +#endif > + > +static inline void __p4d_free(struct mm_struct *mm, p4d_t *p4d) > +{ > + struct ptdesc *ptdesc = virt_to_ptdesc(p4d); > + > + BUG_ON((unsigned long)p4d & (PAGE_SIZE-1)); > + pagetable_p4d_dtor(ptdesc); > + pagetable_free(ptdesc); > +} > + > +#ifndef __HAVE_ARCH_P4D_FREE > +static inline void p4d_free(struct mm_struct *mm, pud_t *p4d) Should this perhaps be p4d_t *p4d rather than pud_t *p4d? Otherwise I get this build error: In file included from /home/klara/git/linux/arch/riscv/include/asm/kfence.h:8, from /home/klara/git/linux/mm/kfence/core.c:34: /home/klara/git/linux/include/asm-generic/pgalloc.h: In function ‘p4d_free’: /home/klara/git/linux/include/asm-generic/pgalloc.h:252:24: error: passing argument 2 of ‘__p4d_free’ from incompatible pointer type [-Wincompatible-pointer-types] 252 | __p4d_free(mm, p4d); | ^~~ | | | pud_t * /home/klara/git/linux/include/asm-generic/pgalloc.h:244:60: note: expected ‘p4d_t *’ but argument is of type ‘pud_t *’ 244 | static inline void __p4d_free(struct mm_struct *mm, p4d_t *p4d) | ~~~~~~~^~~ In file included from /home/klara/git/linux/arch/riscv/include/asm/kfence.h:8, from /home/klara/git/linux/mm/kfence/report.c:22: /home/klara/git/linux/include/asm-generic/pgalloc.h: In function ‘p4d_free’: /home/klara/git/linux/include/asm-generic/pgalloc.h:252:24: error: passing argument 2 of ‘__p4d_free’ from incompatible pointer type [-Wincompatible-pointer-types] 252 | __p4d_free(mm, p4d); | ^~~ | | | pud_t * /home/klara/git/linux/include/asm-generic/pgalloc.h:244:60: note: expected ‘p4d_t *’ but argument is of type ‘pud_t *’ 244 | static inline void __p4d_free(struct mm_struct *mm, p4d_t *p4d) | ~~~~~~~^~~ Regards, Klara Modin > +{ > + __p4d_free(mm, p4d); > +} > +#endif > + > +#endif > + > #ifndef __HAVE_ARCH_PGD_FREE > static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd) > { > diff --git a/include/linux/mm.h b/include/linux/mm.h > index 5e73e53c34e9e..807a12ed8ec96 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -3237,6 +3237,22 @@ static inline void pagetable_pud_dtor(struct ptdesc *ptdesc) > lruvec_stat_sub_folio(folio, NR_PAGETABLE); > } > > +static inline void pagetable_p4d_ctor(struct ptdesc *ptdesc) > +{ > + struct folio *folio = ptdesc_folio(ptdesc); > + > + __folio_set_pgtable(folio); > + lruvec_stat_add_folio(folio, NR_PAGETABLE); > +} > + > +static inline void pagetable_p4d_dtor(struct ptdesc *ptdesc) > +{ > + struct folio *folio = ptdesc_folio(ptdesc); > + > + __folio_clear_pgtable(folio); > + lruvec_stat_sub_folio(folio, NR_PAGETABLE); > +} > + > extern void __init pagecache_init(void); > extern void free_initmem(void); >