From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 56CA1449B27; Tue, 21 Jul 2026 16:24:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784651064; cv=none; b=N8h7Fyzf8yLh2SjOzn2TvK9cN8H1PfpvB4e/cdwDCkfyWkXq0VzGegaxbvr61teov+dS/1/uUpiXOlWkN+4Uf/3b0Xu+zf2HC59rcnXJ2azI2gBpcLQrZdZfzYeDTlE4HhsAyzqgjGrGaXJ/JKiV/LO15UQk6OR6hhX9PDqFCb0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784651064; c=relaxed/simple; bh=WA9gsppc/aQ/EE5uHkeXjo1TTDNI7O1i8DReN2UVGOM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ckL7gUGwk/kEA+s1DE7uePH6aHLc1yU6seKd5fNXmBJ+TTq2MIN8bhunlgQyyUD9tcVmlWY0yaCETt5hF7nr1Qal3GeLWFiO3pa0yqHThYXl/6EaYUuRjRDAwh0rTKPAP8GJ+G08+tllsPG6ufOlEAHiIUFoIv3rJjjZlrJ+cEU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jpW44nJz; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jpW44nJz" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 832A61F000E9; Tue, 21 Jul 2026 16:24:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1784651062; bh=MLtKmgePsVMTaeM3LRrhXggCl62wc5rCPPnL48yidIo=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=jpW44nJzttT3pWAEhrIKz5B4sMBsyhohfB0x3ncv13vYzuBX+eor5vRrEJsAn1GEC 8buYNNPOKdsYbmQq4LUYFKbNl9hjm9cLuyrrmuj2vS7L92z3roHNEuY3rwGa7SLFla mVWiAz9B/42AJBbg/tlAC/rzlktFkH1KdlYTHVvbh+z6r7JDzdaFEoJe4k3cYL4Vn0 CEM3jZKRS8JnGvVzs60HuN1d6KQxlrY56SsI3GVHxg1BN5fUoLnKf3K0RBbDod0UQk MUO1T9opH9P+jmqCFdx3mbZ949Kz8DemL18bQ0aug9DwEw51oCZgNWGYZpjANkk1fk cdcD3utSbh+tg== From: "Mike Rapoport (Microsoft)" Date: Tue, 21 Jul 2026 19:23:29 +0300 Subject: [PATCH RFC 06/18] x86/mm/pat: introduce an inline helper to check if alias needs update Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20260721-generic-set-memory-v0-1-v1-6-2c1fc62306b3@kernel.org> References: <20260721-generic-set-memory-v0-1-v1-0-2c1fc62306b3@kernel.org> In-Reply-To: <20260721-generic-set-memory-v0-1-v1-0-2c1fc62306b3@kernel.org> To: linux-mm@kvack.org Cc: Albert Ou , Andrew Morton , Andy Lutomirski , Borislav Petkov , Brendan Jackman , Catalin Marinas , Dave Hansen , David Hildenbrand , Huacai Chen , Ingo Molnar , Ingo Molnar , Kevin Brodsky , "Liam R. Howlett" , Lorenzo Stoakes , Michal Hocko , Mike Rapoport , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , "H. Peter Anvin" , Suren Baghdasaryan , Thomas Gleixner , Vlastimil Babka , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, loongarch@lists.linux.dev, x86@kernel.org X-Mailer: b4 0.16-dev Once core parts of change_page_attr() would be moved to common code, the processing of alias mappings would be out of line and would require a function call for each modified PTE even if the primary target of change_page_attr() does not have alias mappings. Calling a function per PTE just to find out it has nothing to do is expensive and resulted in ~30% regression in instrumented cpa-test. Although cpa-test is a microbenchmark, ~30% regression is still a lot and can be easily avoided by inlining the checks that gates processing of the alias mapping. Introduce cpa_should_update_alias() inline helper that checks if the alias update is required. Since on 64-bit an alias could be in the high kernel mapping, pull __cpa_pfn_in_highmap() along and rename it to pfn_is_kernel(). No functional change intended. Assisted-by: Copilot:claude-opus-4.8 Signed-off-by: Mike Rapoport (Microsoft) --- arch/x86/include/asm/set_memory.h | 50 +++++++++++++++++++++++++++++++ arch/x86/mm/pat/set_memory.c | 62 +++------------------------------------ 2 files changed, 54 insertions(+), 58 deletions(-) diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_memory.h index 4362c26aa992..332ec300c2ed 100644 --- a/arch/x86/include/asm/set_memory.h +++ b/arch/x86/include/asm/set_memory.h @@ -3,12 +3,62 @@ #define _ASM_X86_SET_MEMORY_H #include +#include #include #include #define set_memory_rox set_memory_rox int set_memory_rox(unsigned long addr, int numpages); +#ifdef CONFIG_X86_64 +/* + * The kernel image is mapped into two places in the virtual address space + * (addresses without KASLR, of course): + * + * 1. The kernel direct map (0xffff880000000000) + * 2. The "high kernel map" (0xffffffff81000000) + * + * We actually execute out of #2. If we get the address of a kernel symbol, it + * points to #2, but almost all physical-to-virtual translations point to #1. + * + * This is so that we can have both a directmap of all physical memory *and* + * take full advantage of the limited (s32) immediate addressing range (2G) + * of x86_64. + * + * See Documentation/arch/x86/x86_64/mm.rst for more detail. + */ +static inline bool pfn_is_kernel(unsigned long pfn) +{ + unsigned long spfn = __pa_symbol(_text) >> PAGE_SHIFT; + /* Do not reference a physical address outside the kernel. */ + unsigned long epfn = __pa_symbol(roundup(_brk_end, PMD_SIZE) - 1) >> PAGE_SHIFT; + + return pfn >= spfn && pfn <= epfn; +} +#else +static inline bool pfn_is_kernel(unsigned long pfn) +{ + /* There is no highmap on 32-bit */ + return false; +} +#endif + +static inline bool cpa_should_update_alias(unsigned long vaddr, + unsigned long pfn) +{ + /* Primary is not in the direct map, its direct map alias needs update */ + if (vaddr < PAGE_OFFSET || + vaddr >= PAGE_OFFSET + (max_pfn_mapped << PAGE_SHIFT)) + return true; + + /* direct map page that is also part of the kernel highmap */ + if ((vaddr < (unsigned long)_text || vaddr >= _brk_end) && + pfn_is_kernel(pfn)) + return true; + + return false; +} + /* * The set_memory_* API can be used to change various attributes of a virtual * address range. The attributes include: diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index ee40962d4ee9..2a087dcedf84 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -224,61 +224,6 @@ within(unsigned long addr, unsigned long start, unsigned long end) return addr >= start && addr < end; } -#ifdef CONFIG_X86_64 - -static inline int -within_inclusive(unsigned long addr, unsigned long start, unsigned long end) -{ - return addr >= start && addr <= end; -} - -/* - * The kernel image is mapped into two places in the virtual address space - * (addresses without KASLR, of course): - * - * 1. The kernel direct map (0xffff880000000000) - * 2. The "high kernel map" (0xffffffff81000000) - * - * We actually execute out of #2. If we get the address of a kernel symbol, it - * points to #2, but almost all physical-to-virtual translations point to #1. - * - * This is so that we can have both a directmap of all physical memory *and* - * take full advantage of the limited (s32) immediate addressing range (2G) - * of x86_64. - * - * See Documentation/arch/x86/x86_64/mm.rst for more detail. - */ - -static inline unsigned long highmap_start_pfn(void) -{ - return __pa_symbol(_text) >> PAGE_SHIFT; -} - -static inline unsigned long highmap_end_pfn(void) -{ - /* Do not reference physical address outside the kernel. */ - return __pa_symbol(roundup(_brk_end, PMD_SIZE) - 1) >> PAGE_SHIFT; -} - -static bool __cpa_pfn_in_highmap(unsigned long pfn) -{ - /* - * Kernel text has an alias mapping at a high address, known - * here as "highmap". - */ - return within_inclusive(pfn, highmap_start_pfn(), highmap_end_pfn()); -} - -#else - -static bool __cpa_pfn_in_highmap(unsigned long pfn) -{ - /* There is no highmap on 32-bit */ - return false; -} - -#endif - /* * See set_mce_nospec(). * @@ -1830,7 +1775,7 @@ static int __cpa_process_fault(struct cpa_data *cpa, unsigned long vaddr, cpa->pfn = __pa(vaddr) >> PAGE_SHIFT; return 0; - } else if (__cpa_pfn_in_highmap(cpa->pfn)) { + } else if (pfn_is_kernel(cpa->pfn)) { /* Faults in the highmap are OK, so do not warn: */ return -EFAULT; } else { @@ -1968,7 +1913,7 @@ static int cpa_process_alias(struct cpa_data *cpa) * to touch the high mapped kernel as well: */ if (!within(vaddr, (unsigned long)_text, _brk_end) && - __cpa_pfn_in_highmap(cpa->pfn)) { + pfn_is_kernel(cpa->pfn)) { unsigned long temp_cpa_vaddr = (cpa->pfn << PAGE_SHIFT) + __START_KERNEL_map - phys_base; alias_cpa = *cpa; @@ -2026,7 +1971,8 @@ static int __change_page_attr_set_clr(struct cpa_data *cpa, int primary) if (ret) goto out; - if (primary && !(cpa->flags & CPA_NO_CHECK_ALIAS)) { + if (primary && !(cpa->flags & CPA_NO_CHECK_ALIAS) && + cpa_should_update_alias(__cpa_addr(cpa, cpa->curpage), cpa->pfn)) { ret = cpa_process_alias(cpa); if (ret) goto out; -- 2.53.0