From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 695E94AA01B; Mon, 21 Sep 2026 14:50:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002225; cv=none; b=objR3rzv8Hvo8XL6z8TlvwEUCQobx6Cj12LNsk+ggaTqMRwzJmxjRbkGvg71D53mbHrpo2MLA4VJjOfMGbPx4US47bXyM1Cf1xCY1Z9IVtPgIfJCdlSu7oTQSYdLb//palzXpfK2etzo8djLvMspTlPG64z7rRySLtinUosqQDo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002225; c=relaxed/simple; bh=n60ywsDXdJuMaFjQxbfN22Ty1YGsfuy5RFFHaRZGs2s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZZWSTUSr3D+C7dGucXg5T8rrosZ7yXvnKp2J5yRkfwduKnyt946YUeTH/ddVntnpRX7aaIHjrCp4Sn9WVsyO4hucEBtHlYbORdQM0OJH2X5l9YKxSRw45TUqfQ/4AOxQi28gKdl/EbXxTYa2kRMh8dp2RfI9wbLcJyEyVHCJ9TY= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HhhVBHd7; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HhhVBHd7" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5EAD21F00893; Mon, 21 Sep 2026 14:50:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002224; bh=2XD9KbmPFTDvG/++FINRoeRjfo1Owk1SK7Ef5Ny4pkE=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=HhhVBHd70NwE6tFDSR7EWwSk5kUn7dx1kha7j/ncW7XELIYaoWokPSNoeR4Gb7csH manH/KE1JbmJ0i/7twyR8DQ5XrSNGVGXxOuKAD+RPrR+xXRboXZTQPlJ3QmEqu859i HZU0YFWzKtkC9U5gYk+hQL+Nk1Q/jBfPYKwtVIz5itpVENE+9110tsSKuHgFvohU87 NP77rfhWL7u4mvtugGaq0ws9V+9f5uQ3MDPuE4Cb8F3CfiH0z1Pbw4bXnIUOU/GE2O JQkgUxAsfSs1Z5A0oGnQxHSA6wqJm37P5biOyVae7LUBXaPY5G/IlRwsFO2EbC8dFE osNmzMILFDf9Q== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 09/13] swiotlb: Align shared IO TLB pools to the shared granule size Date: Mon, 21 Sep 2026 20:18:43 +0530 Message-ID: <20260921144847.501151-10-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Align and size the early memblock pool to the common CoCo shared granule. Use the same rounded extent when the pool is transitioned and when it is released. Re-zero the extent after a successful private-to-shared transition because the architecture operation may change memory contents. This deliberately leaves the slot count unchanged: any rounded tail belongs to the pool allocation but is not advertised as allocatable SWIOTLB space. Replace the default and dynamic pool set_memory calls with the helpers so alignment validation and architecture dispatch are kept in one place. As before, pages are intentionally leaked if their private state cannot be restored safely. Signed-off-by: Aneesh Kumar K.V (Arm) --- kernel/dma/swiotlb.c | 42 +++++++++++++++++++++++++----------------- 1 file changed, 25 insertions(+), 17 deletions(-) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index ded7016a46a7..9928d75efc2d 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -22,6 +22,7 @@ #include #include +#include #include #include #include @@ -38,7 +39,6 @@ #include #include #include -#include #include #include #include @@ -369,16 +369,18 @@ void __init swiotlb_update_mem_attributes(void) if (!mem->nslabs || mem->late_alloc) return; - bytes = PAGE_ALIGN(mem->nslabs << IO_TLB_SHIFT); + + bytes = ALIGN(mem->nslabs << IO_TLB_SHIFT, cc_shared_granule_size()); if (io_tlb_default_mem.cc_shared) { int ret; - ret = set_memory_decrypted((unsigned long)mem->vaddr, - bytes >> PAGE_SHIFT); + ret = cc_make_shared(mem->vaddr, bytes); if (ret) { pr_warn("Failed to decrypt default memory pool, disabling it\n"); swiotlb_mark_pool_used(mem); + } else { + memset(mem->vaddr, 0, bytes); } } } @@ -436,8 +438,8 @@ static void __init *swiotlb_memblock_alloc(unsigned long nslabs, unsigned int flags, int (*remap)(void *tlb, unsigned long nslabs)) { - size_t bytes = PAGE_ALIGN(nslabs << IO_TLB_SHIFT); void *tlb; + size_t bytes = ALIGN(nslabs << IO_TLB_SHIFT, cc_shared_granule_size()); /* * By default allocate the bounce buffer memory from low memory, but @@ -445,9 +447,9 @@ static void __init *swiotlb_memblock_alloc(unsigned long nslabs, * memory encryption. */ if (flags & SWIOTLB_ANY) - tlb = memblock_alloc(bytes, PAGE_SIZE); + tlb = memblock_alloc(bytes, cc_shared_granule_size()); else - tlb = memblock_alloc_low(bytes, PAGE_SIZE); + tlb = memblock_alloc_low(bytes, cc_shared_granule_size()); if (!tlb) { pr_warn("%s: Failed to allocate %zu bytes tlb structure\n", @@ -456,7 +458,7 @@ static void __init *swiotlb_memblock_alloc(unsigned long nslabs, } if (remap && remap(tlb, nslabs) < 0) { - memblock_free(tlb, PAGE_ALIGN(bytes)); + memblock_free(tlb, bytes); pr_warn("%s: Failed to remap %zu bytes\n", __func__, bytes); return NULL; } @@ -578,7 +580,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask, swiotlb_adjust_nareas(num_possible_cpus()); retry: - order = get_order(nslabs << IO_TLB_SHIFT); + order = get_order(ALIGN(nslabs << IO_TLB_SHIFT, cc_shared_granule_size())); nslabs = SLABS_PER_PAGE << order; while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) { @@ -587,6 +589,8 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask, if (vstart) break; order--; + if (order < get_order(cc_shared_granule_size())) + break; nslabs = SLABS_PER_PAGE << order; retried = true; } @@ -626,8 +630,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask, goto error_slots; if (io_tlb_default_mem.cc_shared) { - rc = set_memory_decrypted((unsigned long)vstart, - (nslabs << IO_TLB_SHIFT) >> PAGE_SHIFT); + rc = cc_make_shared(vstart, nslabs << IO_TLB_SHIFT); if (rc) { leak_pages = true; goto error_decrypt; @@ -667,11 +670,11 @@ void __init swiotlb_exit(void) pr_info("tearing down default memory pool\n"); tbl_vaddr = (unsigned long)phys_to_virt(mem->start); - tbl_size = PAGE_ALIGN(mem->end - mem->start); + tbl_size = ALIGN(mem->end - mem->start, cc_shared_granule_size()); slots_size = PAGE_ALIGN(array_size(sizeof(*mem->slots), mem->nslabs)); if (io_tlb_default_mem.cc_shared) { - if (set_memory_encrypted(tbl_vaddr, tbl_size >> PAGE_SHIFT)) + if (cc_make_private((void *)tbl_vaddr, tbl_size)) leak_pages = true; } @@ -711,12 +714,15 @@ void __init swiotlb_exit(void) static struct page *alloc_dma_pages(gfp_t gfp, size_t bytes, u64 phys_limit, unsigned long attrs) { - unsigned int order = get_order(bytes); bool cc_shared = attrs & __DMA_ATTR_ALLOC_CC_SHARED; + unsigned int order; struct page *page; phys_addr_t paddr; void *vaddr; + if (cc_shared) + bytes = ALIGN(bytes, cc_shared_granule_size()); + order = get_order(bytes); page = alloc_pages(gfp, order); if (!page) return NULL; @@ -728,13 +734,13 @@ static struct page *alloc_dma_pages(gfp_t gfp, size_t bytes, } vaddr = phys_to_virt(paddr); - if (cc_shared && set_memory_decrypted((unsigned long)vaddr, PFN_UP(bytes))) + if (cc_shared && cc_make_shared(vaddr, bytes)) goto error; return page; error: /* Intentional leak if pages cannot be encrypted again. */ - if (cc_shared && !set_memory_encrypted((unsigned long)vaddr, PFN_UP(bytes))) + if (cc_shared && !cc_make_private(vaddr, bytes)) __free_pages(page, order); return NULL; } @@ -807,9 +813,11 @@ static void swiotlb_free_tlb(void *vaddr, size_t bytes, bool cc_shared) dma_free_from_pool(NULL, vaddr, bytes)) return; + if (cc_shared) + bytes = ALIGN(bytes, cc_shared_granule_size()); /* Intentional leak if pages cannot be encrypted again. */ if (!cc_shared || - !set_memory_encrypted((unsigned long)vaddr, PFN_UP(bytes))) + !cc_make_private(vaddr, bytes)) __free_pages(virt_to_page(vaddr), get_order(bytes)); } -- 2.43.0