From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78E6D2D8DD0 for ; Sat, 28 Feb 2026 22:11:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772316700; cv=none; b=eid7BB3aHxlZy0d2tkDE4cPO25kAhYKB6DJAP0p3x4Aa/MejzsnykK9AFHDOfWqL67SVUfqGjWXIDSnQiAxxvoucsHQ4h2cw4eMGcK4E/yoj6TTzTfnWqOg0DesbJWTSE4P7d/WpOi8/urFORf6tPrlYIzwveLdU1BY7+Uxut7w= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772316700; c=relaxed/simple; bh=9fS3XpJF4I+7zLavVivs358QdKzqf1ld0o55YjNR+4Y=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version:Content-Type; b=WnRt9yxjDbQMa9TvEcRo+oAhR7dFX+W0ULSTZLc3y2+3SJ0tGF8HZ3ZWvTgwoGsOB/J1vFMRexoqU8zM2lbSGVjVOyaPAN9/LjP6z667sbURKD+TVpscw9eCihNQ9txNtY5C0XYN9PTk7C4oXWI38yTVidTxIUJESEE+n7TV2eQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kY9Xet3F; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kY9Xet3F" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2ad21f437eeso24119475ad.0 for ; Sat, 28 Feb 2026 14:11:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1772316699; x=1772921499; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=xviIpIkstRNT9sPfU0zk0oRcwgO9vdYQGOA0hVgSiDQ=; b=kY9Xet3FaUAe57jllodFQyo6ALnb2cFG5tEiyv9onWSNOWI92dFwpIgFrFb3N5KyDL m7SJADXbSb8+iCbUb8t1asFLavXsW86tqI5MvflqDVNUKuTfnaYxN9Xy3Gn9pHnsn+/Q fvIolQVVHXNErHeJt/JQc5V0IkGJWo6GiSGvin5vwaQjK3KIMvaYR+lePdYeUO3eA8I5 pkDKgpzlVC4+qYVXcqBRYwXSKSaHkMYdNeCUvykjU5SJ4K14+qJFw5oz3cnxeME9jtiS pU5UYad0Tgbq/9CCYQ/Rbq1tSJG8JTNlifYe60SVdUbQhMyWHPlyXLLrXs6+TzhQFJyE /unA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772316699; x=1772921499; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=xviIpIkstRNT9sPfU0zk0oRcwgO9vdYQGOA0hVgSiDQ=; b=i+G3sI9tCMXf8u88XteNkJE52jEie8Z0eauPZAuNxXf36xiG3hvvvBTiMDbOW0cESO cR0kZWrE2sb6OYdijeQ68lToc4AlSSc8EwujZgUae0uavvCHEkySxUlf5tk9cHQYMv4U BXIjDLcDcpTGZt+TVVx6yEsVDZzpifspHBtt/BgpA536mFG1rILmr9adVYEUMpUbERdY SIOgOOItGfcjP7S6hRZifFHOGPhlI7nRdP8GVjw3gvz9vOj1UBoC6j6QdtDEjHmRYN/5 Fhi/gHdry7/N5NSHs7JOn2y48lT1fqh1/U2fCvgqgUOS1SGO95Mwcbwn+xuQyMoK5pWJ qUcA== X-Gm-Message-State: AOJu0YwKuW5kkRf5eg7v5k9XPGHYGBHVoPza+4pcGFLumPGcNlNf1sjV mjPtGL5y/Gz7ZtgBZL6RoRqXARE6YPcuamcwZ/QH8jLt4Fh+Qi2/gilx X-Gm-Gg: ATEYQzxQiKa2ic7MWUquoqGoHbnEk6m8AFaPCJszw85cQJUyY3A2XPtpLE9L5C51ucx OllgipEMq6xWc2Y/N6joLN9kk/X3pyPWt6fmit7UfmgCNO2mb2JbceA1+2yibPQWo3CdZwqOOdA OdMlzaDrlRZbbTHA9kHq5meVzeKQ8piAmNcxyqNSsR/ibRrkoG98+3ccFmEwRYnSR91gjsBCuq/ Ulg4zL8JAJPRYOGZFybQTTEyOhS1fKgjN2ea7nWKd+GfNO2QOZXQesHz9nyfUBBiXzSdK6MhxhZ RqoEOO4sQhpTyDfGniH5CDy83HR+F7vaTWjkRM1hG/b5xLp8tUy1ZvACVScioJUbMJ2xppswkbI FXYHRxG4BqoiTI+hbLVnse43Gp2lKy760CL61qqb0UUaLHz7c2Y3FWzZC5BabuENUvmeBLWO+bM ObWBP3Ni9FousnkGlF5TdzOBFZuhSiiQ9U1TWj X-Received: by 2002:a17:903:94e:b0:295:55fc:67a0 with SMTP id d9443c01a7336-2adf78db0ebmr119480805ad.2.1772316698623; Sat, 28 Feb 2026 14:11:38 -0800 (PST) Received: from Barrys-MBP.hub ([47.72.129.29]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2adfb6b9b32sm129494295ad.66.2026.02.28.14.11.31 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sat, 28 Feb 2026 14:11:38 -0800 (PST) From: Barry Song <21cnbao@gmail.com> To: catalin.marinas@arm.com, m.szyprowski@samsung.com, robin.murphy@arm.com, will@kernel.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, Barry Song , Leon Romanovsky , Ada Couprie Diaz , Ard Biesheuvel , Marc Zyngier , Anshuman Khandual , Ryan Roberts , Suren Baghdasaryan , Joerg Roedel , Juergen Gross , Stefano Stabellini , Oleksandr Tyshchenko , Tangquan Zheng , Huacai Zhou , Xueyuan Chen Subject: [PATCH v3 0/5] dma-mapping: arm64: support batched cache sync Date: Sun, 1 Mar 2026 06:11:25 +0800 Message-Id: <20260228221125.59863-1-21cnbao@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From: Barry Song Many embedded ARM64 SoCs still lack hardware cache coherency support, which causes DMA mapping operations to appear as hotspots in on-CPU flame graphs. For an SG list with *nents* entries, the current dma_map/unmap_sg() and DMA sync APIs perform cache maintenance one entry at a time. After each entry, the implementation synchronously waits for the corresponding region’s D-cache operations to complete. On architectures like arm64, efficiency can be improved by issuing all entries’ operations first and then performing a single batched wait for completion. Tangquan's results show that batched synchronization can reduce dma_map_sg() time by 64.61% and dma_unmap_sg() time by 66.60% on an MTK phone platform (MediaTek Dimensity 9500). The tests were performed by pinning the task to CPU7 and fixing the CPU frequency at 2.6 GHz, running dma_map_sg() and dma_unmap_sg() on 10 MB buffers (10 MB / 4 KB sg entries per buffer) for 200 iterations and then averaging the results. Thanks to Xueyuan for volunteering to take on the testing tasks. He put significant effort into validating paths such as IOVA link/unlink and SWIOTLB on RK3588 boards with NVMe. v3: * Fold patches 5/8, 7/8, and 8/8 into patch 4/8 as suggested by Leon, reducing the series from 8 patches to 5; * Fix the SWIOTLB path by ensuring a sync is issued before memcpy; * Add ARCH_HAS_BATCHED_DMA_SYNC Kconfig as suggested by Leon; * Collect Reviewed-by tags from Leon and Juergen. Leon's tag is not added to patch 4 since it has changed significantly since v2 and requires re-review; * Rename some asm macros and functions as suggested by Will; * Add Xueyuan's Tested-by. His help is greatly appreciated! v2 link: https://lore.kernel.org/lkml/20251226225254.46197-1-21cnbao@gmail.com/ v2: * Refine a large amount of arm64 asm code based on feedback from Robin, thanks! * Drop batch_add APIs and always use arch_sync_dma_for_* + flush, even for a single buffer, based on Leon’s suggestion, thanks! * Refine a large amount of code based on feedback from Leon, thanks! * Also add batch support for iommu_dma_sync_sg_for_{cpu,device} v1 link: https://lore.kernel.org/lkml/20251219053658.84978-1-21cnbao@gmail.com/ v1, diff with RFC: * Drop a large number of #ifdef/#else/#endif blocks based on feedback from Catalin and Marek, thanks! * Also add batched iova link/unlink support, marked as RFC since I lack the required hardware. This was suggested by Marek, thanks! RFC link: https://lore.kernel.org/lkml/20251029023115.22809-1-21cnbao@gmail.com/ Barry Song (5): arm64: Provide dcache_by_myline_op_nosync helper arm64: Provide dcache_clean_poc_nosync helper arm64: Provide dcache_inval_poc_nosync helper dma-mapping: Separate DMA sync issuing and completion waiting dma-mapping: Support batch mode for dma_direct_{map,unmap}_sg arch/arm64/Kconfig | 1 + arch/arm64/include/asm/assembler.h | 25 ++++++++++--- arch/arm64/include/asm/cache.h | 5 +++ arch/arm64/include/asm/cacheflush.h | 2 + arch/arm64/kernel/relocate_kernel.S | 3 +- arch/arm64/mm/cache.S | 57 +++++++++++++++++++++++------ arch/arm64/mm/dma-mapping.c | 4 +- drivers/iommu/dma-iommu.c | 35 ++++++++++++++---- drivers/xen/swiotlb-xen.c | 24 ++++++++---- include/linux/dma-map-ops.h | 6 +++ kernel/dma/Kconfig | 3 ++ kernel/dma/direct.c | 23 +++++++++--- kernel/dma/direct.h | 21 ++++++++--- kernel/dma/mapping.c | 6 +-- kernel/dma/swiotlb.c | 7 +++- 15 files changed, 171 insertions(+), 51 deletions(-) Cc: Leon Romanovsky Cc: Marek Szyprowski Cc: Catalin Marinas Cc: Will Deacon Cc: Ada Couprie Diaz Cc: Ard Biesheuvel Cc: Marc Zyngier Cc: Anshuman Khandual Cc: Ryan Roberts Cc: Suren Baghdasaryan Cc: Robin Murphy Cc: Joerg Roedel Cc: Juergen Gross Cc: Stefano Stabellini Cc: Oleksandr Tyshchenko Cc: Tangquan Zheng Cc: Huacai Zhou Cc: Xueyuan Chen -- 2.39.3 (Apple Git-146)