[PATCH] arm64/mm: Re-organize arch_make_huge_pte()

[PATCH] arm64/mm: Re-organize arch_make_huge_pte()

[Anshuman Khandual]

Core HugeTLB defines a fallback definition for arch_make_huge_pte(), which
calls platform provided pte_mkhuge(). But if any platform already provides
an override for arch_make_huge_pte(), then it does not need to provide the
helper pte_mkhuge().

arm64 override for arch_make_huge_pte() calls pte_mkhuge() internally, thus
creating an impression, that both of these callbacks are being used in core
HugeTLB and hence required to be defined. This drops off pte_mkhuge() which
was never required to begin with as there could not be any section mappings
at the PTE level. Re-organize arch_make_huge_pte() based on requested page
size and create the entry for the applicable page table level as needed. It
also removes a redundancy of clearing PTE_TABLE_BIT bit followed by setting
both PTE_TABLE_BIT and PTE_VALID bits (via PTE_TYPE_MASK) in the pte, while
creating CONT_PTE_SIZE size entries.

Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: Ard Biesheuvel <ardb@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
This applies on v6.12-rc5

 arch/arm64/include/asm/pgtable.h |  5 -----
 arch/arm64/mm/hugetlbpage.c      | 21 ++++++++++++++++-----
 2 files changed, 16 insertions(+), 10 deletions(-)

diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index c329ea061dc9..fa4c32a9f572 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -438,11 +438,6 @@ static inline void __set_ptes(struct mm_struct *mm,
 	}
 }
 
-/*
- * Huge pte definitions.
- */
-#define pte_mkhuge(pte)		(__pte(pte_val(pte) & ~PTE_TABLE_BIT))
-
 /*
  * Hugetlb definitions.
  */
diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
index 5f1e2103888b..3215adf48a1b 100644
--- a/arch/arm64/mm/hugetlbpage.c
+++ b/arch/arm64/mm/hugetlbpage.c
@@ -361,14 +361,25 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags)
 {
 	size_t pagesize = 1UL << shift;
 
-	entry = pte_mkhuge(entry);
-	if (pagesize == CONT_PTE_SIZE) {
-		entry = pte_mkcont(entry);
-	} else if (pagesize == CONT_PMD_SIZE) {
+	switch (pagesize) {
+#ifndef __PAGETABLE_PMD_FOLDED
+	case PUD_SIZE:
+		entry = pud_pte(pud_mkhuge(pte_pud(entry)));
+		break;
+#endif
+	case CONT_PMD_SIZE:
 		entry = pmd_pte(pmd_mkcont(pte_pmd(entry)));
-	} else if (pagesize != PUD_SIZE && pagesize != PMD_SIZE) {
+		fallthrough;
+	case PMD_SIZE:
+		entry = pmd_pte(pmd_mkhuge(pte_pmd(entry)));
+		break;
+	case CONT_PTE_SIZE:
+		entry = pte_mkcont(entry);
+		break;
+	default:
 		pr_warn("%s: unrecognized huge page size 0x%lx\n",
 			__func__, pagesize);
+		break;
 	}
 	return entry;
 }
-- 
2.30.2

Re: [PATCH] arm64/mm: Re-organize arch_make_huge_pte()

[Catalin Marinas]

On Tue, 29 Oct 2024 10:15:29 +0530, Anshuman Khandual wrote:
> Core HugeTLB defines a fallback definition for arch_make_huge_pte(), which
> calls platform provided pte_mkhuge(). But if any platform already provides
> an override for arch_make_huge_pte(), then it does not need to provide the
> helper pte_mkhuge().
> 
> arm64 override for arch_make_huge_pte() calls pte_mkhuge() internally, thus
> creating an impression, that both of these callbacks are being used in core
> HugeTLB and hence required to be defined. This drops off pte_mkhuge() which
> was never required to begin with as there could not be any section mappings
> at the PTE level. Re-organize arch_make_huge_pte() based on requested page
> size and create the entry for the applicable page table level as needed. It
> also removes a redundancy of clearing PTE_TABLE_BIT bit followed by setting
> both PTE_TABLE_BIT and PTE_VALID bits (via PTE_TYPE_MASK) in the pte, while
> creating CONT_PTE_SIZE size entries.
> 
> [...]

Applied to arm64 (for-next/misc), thanks!

[1/1] arm64/mm: Re-organize arch_make_huge_pte()
      https://git.kernel.org/arm64/c/f8192813dcbe

I think we can now get pte_mkcont() to only set PTE_CONT without
touching PTE_TYPE_PAGE. Previously it wasn't possible because
pte_mkhuge() was clearing the PTE_TABLE_BIT and we added it back via
pte_mkcont(). Give it a try and send an additional patch cleaning up
pte_mkcont() if this works. Thanks.

-- 
Catalin

Re: [PATCH] arm64/mm: Re-organize arch_make_huge_pte()

[Anshuman Khandual]

On 11/1/24 23:05, Catalin Marinas wrote:
> On Tue, 29 Oct 2024 10:15:29 +0530, Anshuman Khandual wrote:
>> Core HugeTLB defines a fallback definition for arch_make_huge_pte(), which
>> calls platform provided pte_mkhuge(). But if any platform already provides
>> an override for arch_make_huge_pte(), then it does not need to provide the
>> helper pte_mkhuge().
>>
>> arm64 override for arch_make_huge_pte() calls pte_mkhuge() internally, thus
>> creating an impression, that both of these callbacks are being used in core
>> HugeTLB and hence required to be defined. This drops off pte_mkhuge() which
>> was never required to begin with as there could not be any section mappings
>> at the PTE level. Re-organize arch_make_huge_pte() based on requested page
>> size and create the entry for the applicable page table level as needed. It
>> also removes a redundancy of clearing PTE_TABLE_BIT bit followed by setting
>> both PTE_TABLE_BIT and PTE_VALID bits (via PTE_TYPE_MASK) in the pte, while
>> creating CONT_PTE_SIZE size entries.
>>
>> [...]
> 
> Applied to arm64 (for-next/misc), thanks!
> 
> [1/1] arm64/mm: Re-organize arch_make_huge_pte()
>       https://git.kernel.org/arm64/c/f8192813dcbe
> 
> I think we can now get pte_mkcont() to only set PTE_CONT without
> touching PTE_TYPE_PAGE. Previously it wasn't possible because
> pte_mkhuge() was clearing the PTE_TABLE_BIT and we added it back via
> pte_mkcont(). Give it a try and send an additional patch cleaning up
> pte_mkcont() if this works. Thanks.

Yes, it does indeed work. Sent the following patch for the same.

https://lore.kernel.org/all/20241104041617.3804617-1-anshuman.khandual@arm.com/