From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 696A9355053 for ; Tue, 6 Jan 2026 19:12:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767726731; cv=none; b=dSYnlND26rqlQFC/X2jpfbv9pk4OoGX9hFORxIKweIeqRHtLqkNfi6prXdnFsqgN3fYtb6FW/6yYqFVcBLKQm59/q+7EXE1T8DCGDAl06JTnXG7qsYZGT22duUk/3gwxlbeUj6/6id5N0eqawh6Pm4Xq07L5v8j+RMIjcpy8bLI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1767726731; c=relaxed/simple; bh=SDzQKGWuO/sdn64pswtkgKtgQjm2NSK1ETlO5wGxiHA=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=qqDSs2uRSwtxY7RxqUhX8/fhrpfu1nXrqaZV86nVVjlZG7R2Rwucg0LKGM60eExObTRtyNKBJJ3D4uRZkk20t2O6+rgUBmN9QLjs20xT/MQ9uLPdakGS1uEwMVC8QvQCW2JQmcnjJNZB5nRLJX2GaQb4MIc2P6zgHzZebhcXFLU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C9CDB497; Tue, 6 Jan 2026 11:12:00 -0800 (PST) Received: from [10.57.46.241] (unknown [10.57.46.241]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1C7853F5A1; Tue, 6 Jan 2026 11:12:04 -0800 (PST) Message-ID: Date: Tue, 6 Jan 2026 19:12:02 +0000 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 5/8] dma-mapping: Support batch mode for dma_direct_sync_sg_for_* To: Barry Song <21cnbao@gmail.com>, Leon Romanovsky Cc: catalin.marinas@arm.com, m.szyprowski@samsung.com, will@kernel.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, xen-devel@lists.xenproject.org, Ada Couprie Diaz , Ard Biesheuvel , Marc Zyngier , Anshuman Khandual , Ryan Roberts , Suren Baghdasaryan , Tangquan Zheng References: <20251226225254.46197-1-21cnbao@gmail.com> <20251226225254.46197-6-21cnbao@gmail.com> <20251227200933.GO11869@unreal> <20251228145041.GS11869@unreal> From: Robin Murphy Content-Language: en-GB In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 2026-01-06 6:41 pm, Barry Song wrote: > On Mon, Dec 29, 2025 at 3:50 AM Leon Romanovsky wrote: >> >> On Sun, Dec 28, 2025 at 09:52:05AM +1300, Barry Song wrote: >>> On Sun, Dec 28, 2025 at 9:09 AM Leon Romanovsky wrote: >>>> >>>> On Sat, Dec 27, 2025 at 11:52:45AM +1300, Barry Song wrote: >>>>> From: Barry Song >>>>> >>>>> Instead of performing a flush per SG entry, issue all cache >>>>> operations first and then flush once. This ultimately benefits >>>>> __dma_sync_sg_for_cpu() and __dma_sync_sg_for_device(). >>>>> >>>>> Cc: Leon Romanovsky >>>>> Cc: Catalin Marinas >>>>> Cc: Will Deacon >>>>> Cc: Marek Szyprowski >>>>> Cc: Robin Murphy >>>>> Cc: Ada Couprie Diaz >>>>> Cc: Ard Biesheuvel >>>>> Cc: Marc Zyngier >>>>> Cc: Anshuman Khandual >>>>> Cc: Ryan Roberts >>>>> Cc: Suren Baghdasaryan >>>>> Cc: Tangquan Zheng >>>>> Signed-off-by: Barry Song >>>>> --- >>>>> kernel/dma/direct.c | 14 +++++++------- >>>>> 1 file changed, 7 insertions(+), 7 deletions(-) >>>> >>>> <...> >>>> >>>>> - if (!dev_is_dma_coherent(dev)) { >>>>> + if (!dev_is_dma_coherent(dev)) >>>>> arch_sync_dma_for_device(paddr, sg->length, >>>>> dir); >>>>> - arch_sync_dma_flush(); >>>>> - } >>>>> } >>>>> + if (!dev_is_dma_coherent(dev)) >>>>> + arch_sync_dma_flush(); >>>> >>>> This patch should be squashed into the previous one. You introduced >>>> arch_sync_dma_flush() there, and now you are placing it elsewhere. >>> >>> Hi Leon, >>> >>> The previous patch replaces all arch_sync_dma_for_* calls with >>> arch_sync_dma_for_* plus arch_sync_dma_flush(), without any >>> functional change. The subsequent patches then implement the >>> actual batching. I feel this is a better approach for reviewing >>> each change independently. Otherwise, the previous patch would >>> be too large. >> >> Don't worry about it. Your patches are small enough. > > My hardware does not require a bounce buffer, but I am concerned that > this patch may be incorrect for systems that do require one. > > Now it is: > > void dma_direct_sync_sg_for_cpu(struct device *dev, > struct scatterlist *sgl, int nents, enum dma_data_direction dir) > { > struct scatterlist *sg; > int i; > > for_each_sg(sgl, sg, nents, i) { > phys_addr_t paddr = dma_to_phys(dev, sg_dma_address(sg)); > > if (!dev_is_dma_coherent(dev)) > arch_sync_dma_for_cpu(paddr, sg->length, dir); > > swiotlb_sync_single_for_cpu(dev, paddr, sg->length, dir); > > if (dir == DMA_FROM_DEVICE) > arch_dma_mark_clean(paddr, sg->length); > } > > if (!dev_is_dma_coherent(dev)) { > arch_sync_dma_flush(); > arch_sync_dma_for_cpu_all(); > } > } > > Should we call swiotlb_sync_single_for_cpu() and > arch_dma_mark_clean() after the flush to ensure the CPU sees the > latest data and that the memcpy is correct? I mean: Yes, this and the equivalents in the later patches are broken for all the sync_for_cpu and unmap paths which may end up bouncing (beware some of them get a bit fiddly) - any cache maintenance *must* be completed before calling SWIOTLB. As for mark_clean, IIRC that was an IA-64 thing, and appears to be entirely dead now. Thanks, Robin. > void dma_direct_sync_sg_for_cpu(struct device *dev, > struct scatterlist *sgl, int nents, enum dma_data_direction dir) > { > struct scatterlist *sg; > int i; > > for_each_sg(sgl, sg, nents, i) { > phys_addr_t paddr = dma_to_phys(dev, sg_dma_address(sg)); > > if (!dev_is_dma_coherent(dev)) > arch_sync_dma_for_cpu(paddr, sg->length, dir); > } > > if (!dev_is_dma_coherent(dev)) { > arch_sync_dma_flush(); > arch_sync_dma_for_cpu_all(); > } > > for_each_sg(sgl, sg, nents, i) { > phys_addr_t paddr = dma_to_phys(dev, sg_dma_address(sg)); > > swiotlb_sync_single_for_cpu(dev, paddr, sg->length, dir); > > if (dir == DMA_FROM_DEVICE) > arch_dma_mark_clean(paddr, sg->length); > } > } > > Could this be the same issue for dma_direct_unmap_sg()? > > Another option is to not support batched synchronization for the bounce > buffer case, since it is rare. In that case, it could be: > > diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c > index 550a1a13148d..a4840f7e8722 100644 > --- a/kernel/dma/direct.c > +++ b/kernel/dma/direct.c > @@ -423,8 +423,11 @@ void dma_direct_sync_sg_for_cpu(struct device *dev, > for_each_sg(sgl, sg, nents, i) { > phys_addr_t paddr = dma_to_phys(dev, sg_dma_address(sg)); > > - if (!dev_is_dma_coherent(dev)) > + if (!dev_is_dma_coherent(dev)) { > arch_sync_dma_for_cpu(paddr, sg->length, dir); > + if (unlikely(dev->dma_io_tlb_mem)) > + arch_sync_dma_flush(); > + } > > swiotlb_sync_single_for_cpu(dev, paddr, sg->length, dir); > > What’s your view on this, Leon? > > Thanks > Barry