From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A61043DA5C; Thu, 24 Sep 2026 10:06:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244377; cv=none; b=T5r7M3l1p0djG0KB2FIPZzmmsEPNLN/CafbP29qAsu+EtkKlsmkILuXrJV8DFGFLdBcQvTovdhtpRnDmI/h+ZqQ3Y7YVgQj9A+xMmxAa5rz2CYEOymZxXIMHtODeNSe6OXR1+orBCnBRTrMkfqri3GWhMN2Ei2l88Eb25SRDaPE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244377; c=relaxed/simple; bh=B/ilBvQaukranZL4YbLeq5rqFwWJqKrxYlAeRc3LaKE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tNO7sQ8/X/FzfeVP/JdFChXDT1Q/Z5IolbgYL1HzKoqiehf1OEeple0bzksFjbanM8UuFr6rN8TjW8Ty2WyDDaxshMOXd/g68PDZ1O4devlbUIDSHqUlVAkV4pYWBDS7OJaXTSzAv2tm09ZIJ44ytdgM2Ac8j9q2V6g+Y9FojmA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ch5tx7zD; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ch5tx7zD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A26BD1F00893; Thu, 24 Sep 2026 10:05:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244367; bh=6i5Bn/p9hoYkPkAS4fE0FC3u08lww1IyLVqKIgULjmw=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Ch5tx7zDK2am4maKTos8NryZBOb9Q9yKA2U1aj8MpTofoC0f4brL3bAeytSaOvGYy fDsWrt8SVwJVn9N1tSC9N7RcwWe3yGqb9p6XSr09zdW6qYzPndPqljKfmUQl7WxIAO ulbER1KfvK1A+1r06J85qrf9/l7nA8SJoiENU7EMpivAr8GTvf0biNlSXhA5R8ZS1o PqzwZEI1d+rqcIXxLuUvmNB7qS+eDjNsrYtizjnVC25Jm9KuFhBltE4xoGZbO84fYQ zMCmmYyUuQTLuB4dwHz/aOgbihC2Fsm2XyqIyVf9nCSvR/8f40Pbdy6G6PuW0zk3WW sY7AF4kTeKdMw== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 01/14] mm: Add an allocator for CoCo shared memory Date: Thu, 24 Sep 2026 15:35:16 +0530 Message-ID: <20260924100529.1398790-2-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Confidential-computing guests may require memory shared with the host to be aligned and transitioned in units larger than PAGE_SIZE. Several DMA users need struct page-backed allocations satisfying these requirements. Provide a common allocator instead of requiring each user to open-code this sequence. Add alloc_cc_shared_pages() and its node-aware variant. The allocator rounds the requested size to the architecture's shared granule, allocates suitably aligned contiguous pages and transitions the complete range to shared state. It also preserves the caller's GFP policy. A private-to-shared transition may alter memory contents. Mask __GFP_ZERO from the underlying allocation and when requested, clear the complete transitioned range after cc_make_shared() succeeds. Return the page and transitioned size so free_cc_shared_pages() can restore the complete range to private state before freeing it. If private state cannot be established, retain the allocation instead of returning a possibly shared page to the buddy allocator. Also provide the shared-granule geometry and byte-oriented transition helpers used by the allocator and by callers managing their own backing memory. Cc: Andrew Morton Signed-off-by: Aneesh Kumar K.V (Arm) --- include/linux/cc_shared.h | 39 +++++++ mm/Makefile | 1 + mm/cc_shared.c | 232 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 272 insertions(+) create mode 100644 include/linux/cc_shared.h create mode 100644 mm/cc_shared.c diff --git a/include/linux/cc_shared.h b/include/linux/cc_shared.h new file mode 100644 index 000000000000..5f8db7c468c5 --- /dev/null +++ b/include/linux/cc_shared.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_CC_SHARED_H +#define _LINUX_CC_SHARED_H + +#include +#include + +struct page; + +struct cc_shared_pages { + struct page *page; + size_t shared_size; +}; + +struct cc_shared_layout { + size_t requested_size; + size_t shared_size; + size_t alignment; +}; + +/* + * Architectures may override this to return the granule used for transitions + * between private and shared memory. The value must be a power of two and no + * smaller than PAGE_SIZE. + */ +size_t arch_cc_shared_granule_size(void); + +size_t cc_shared_granule_size(void); +int cc_shared_calc_layout(size_t requested, struct cc_shared_layout *layout); +bool cc_shared_range_valid(phys_addr_t base, size_t size); +int cc_make_shared(void *addr, size_t size); +int cc_make_private(void *addr, size_t size); +int alloc_cc_shared_pages_node(int nid, gfp_t gfp, + size_t requested, struct cc_shared_pages *mem); +int alloc_cc_shared_pages(gfp_t gfp, + size_t requested, struct cc_shared_pages *mem); +void free_cc_shared_pages(struct cc_shared_pages *mem); + +#endif /* _LINUX_CC_SHARED_H */ diff --git a/mm/Makefile b/mm/Makefile index e7245cb88c66..6e6544428422 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -56,6 +56,7 @@ obj-y := filemap.o mempool.o oom_kill.o fadvise.o \ compaction.o show_mem.o \ interval_tree.o list_lru.o workingset.o \ debug.o gup.o mmap_lock.o vma_init.o $(mmu-y) +obj-y += cc_shared.o # Give 'page_alloc' its own module-parameter namespace page-alloc-y := page_alloc.o diff --git a/mm/cc_shared.c b/mm/cc_shared.c new file mode 100644 index 000000000000..85e16f4504b8 --- /dev/null +++ b/mm/cc_shared.c @@ -0,0 +1,232 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2026 ARM Ltd. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +size_t __weak arch_cc_shared_granule_size(void) +{ + return PAGE_SIZE; +} + +size_t cc_shared_granule_size(void) +{ + size_t granule = arch_cc_shared_granule_size(); + + if (WARN_ON_ONCE(granule < PAGE_SIZE || !is_power_of_2(granule))) + return PAGE_SIZE; + + return granule; +} +EXPORT_SYMBOL_GPL(cc_shared_granule_size); + +int cc_shared_calc_layout(size_t requested, struct cc_shared_layout *layout) +{ + size_t granule, rounded; + + if (!requested || !layout) + return -EINVAL; + + granule = cc_shared_granule_size(); + if (check_add_overflow(requested, granule - 1, &rounded)) + return -EOVERFLOW; + + rounded = ALIGN_DOWN(rounded, granule); + layout->requested_size = requested; + layout->shared_size = rounded; + layout->alignment = granule; + + return 0; +} +EXPORT_SYMBOL_GPL(cc_shared_calc_layout); + +bool cc_shared_range_valid(phys_addr_t base, size_t size) +{ + size_t granule = cc_shared_granule_size(); + + if (!size) + return false; + + return IS_ALIGNED(base, granule) && IS_ALIGNED(size, granule); +} +EXPORT_SYMBOL_GPL(cc_shared_range_valid); + +static int cc_validate_transition(void *addr, size_t size) +{ + phys_addr_t phys; + + if (!addr || !size || !PAGE_ALIGNED(addr) || + !virt_addr_valid(addr)) + return -EINVAL; + + phys = page_to_phys(virt_to_page(addr)); + if (!cc_shared_range_valid(phys, size)) + return -EINVAL; + + return 0; +} + +int cc_make_shared(void *addr, size_t size) +{ + int ret = cc_validate_transition(addr, size); + + if (ret) + return ret; + + return set_memory_decrypted((unsigned long)addr, size >> PAGE_SHIFT); +} + +int cc_make_private(void *addr, size_t size) +{ + int ret = cc_validate_transition(addr, size); + + if (ret) + return ret; + + return set_memory_encrypted((unsigned long)addr, size >> PAGE_SHIFT); +} + +static int __alloc_cc_shared_pages_node(int nid, gfp_t gfp, + size_t requested, + struct cc_shared_pages *mem) +{ + struct cc_shared_layout layout; + struct page *page; + unsigned int order; + bool zero = gfp & __GFP_ZERO; + int ret; + + ret = cc_shared_calc_layout(requested, &layout); + if (ret) + return ret; + + order = get_order(layout.shared_size); + if (order > MAX_PAGE_ORDER) + return -EINVAL; + + /* + * State transitions require a linear-map address and may modify memory. + * Allocate from low memory and defer requested zeroing until afterwards. + */ + gfp &= ~(__GFP_HIGHMEM | __GFP_ZERO); + if (nid == NUMA_NO_NODE) + page = alloc_pages(gfp, order); + else + page = alloc_pages_node(nid, gfp, order); + if (!page) + return -ENOMEM; + + ret = cc_make_shared(page_address(page), layout.shared_size); + if (ret) { + if (!cc_make_private(page_address(page), layout.shared_size)) + __free_pages(page, order); + else + pr_warn_ratelimited("leaking %zu bytes with uncertain shared state\n", + layout.shared_size); + return ret; + } + + if (zero) + memset(page_address(page), 0, layout.shared_size); + + mem->page = page; + mem->shared_size = layout.shared_size; + return 0; +} + +/** + * alloc_cc_shared_pages_node - allocate memory that can be shared + * @nid: NUMA node from which to allocate, or %NUMA_NO_NODE + * @gfp: allocation flags + * @requested: number of bytes requested; must be nonzero + * @mem: storage for the allocated page and the size of the shared range + * + * Allocate at least @requested bytes and make the allocation shared when + * memory encryption is active. A memory-state transition requires a valid + * linear-map address, so such allocations never come from high memory + * + * The shared range may be rounded up to the architecture's transition + * granule. On success, @mem->shared_size records the actual size that was + * made shared and must be retained unchanged for free_cc_shared_pages(). + * @mem is not modified on failure. + * + * Return: 0 on success, or a negative error code on failure. + */ +int alloc_cc_shared_pages_node(int nid, gfp_t gfp, + size_t requested, + struct cc_shared_pages *mem) +{ + struct page *page; + unsigned int order; + + if (!mem || !requested) + return -EINVAL; + + if (cc_platform_has(CC_ATTR_MEM_ENCRYPT)) + return __alloc_cc_shared_pages_node(nid, gfp, requested, mem); + + order = get_order(requested); + if (order > MAX_PAGE_ORDER) + return -EINVAL; + + if (nid == NUMA_NO_NODE) + page = alloc_pages(gfp, order); + else + page = alloc_pages_node(nid, gfp, order); + if (!page) + return -ENOMEM; + + mem->page = page; + mem->shared_size = requested; + return 0; +} +EXPORT_SYMBOL_GPL(alloc_cc_shared_pages_node); + +/** + * alloc_cc_shared_pages - allocate memory that can be shared + * @gfp: allocation flags + * @requested: number of bytes requested; must be nonzero + * @mem: storage for the allocated page and the size of the shared range + * + * Equivalent to alloc_cc_shared_pages_node() with %NUMA_NO_NODE. + * + * Return: 0 on success, or a negative error code on failure. + */ +int alloc_cc_shared_pages(gfp_t gfp, + size_t requested, struct cc_shared_pages *mem) +{ + return alloc_cc_shared_pages_node(NUMA_NO_NODE, gfp, requested, mem); +} +EXPORT_SYMBOL_GPL(alloc_cc_shared_pages); + +void free_cc_shared_pages(struct cc_shared_pages *mem) +{ + if (!mem || !mem->page) + return; + + if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT)) + goto free_pages; + + if (cc_make_private(page_address(mem->page), mem->shared_size)) { + pr_warn_ratelimited("leaking %zu bytes that cannot be made private\n", + mem->shared_size); + return; + } + +free_pages: + __free_pages(mem->page, get_order(mem->shared_size)); + mem->page = NULL; + mem->shared_size = 0; +} +EXPORT_SYMBOL_GPL(free_cc_shared_pages); -- 2.43.0