From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934864AbbI2Qkt (ORCPT ); Tue, 29 Sep 2015 12:40:49 -0400 Received: from pandora.arm.linux.org.uk ([78.32.30.218]:55560 "EHLO pandora.arm.linux.org.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751508AbbI2Qki (ORCPT ); Tue, 29 Sep 2015 12:40:38 -0400 Date: Tue, 29 Sep 2015 17:40:14 +0100 From: Russell King - ARM Linux To: Robin Murphy Cc: Tomasz Figa , "iommu@lists.linux-foundation.org" , Olav Haugan , Alexandre Courbot , Paul Walmsley , Arnd Bergmann , Tomeu Vizoso , Stephen Warren , Antonios Motakis , Will Deacon , "linux-kernel@vger.kernel.org" , "linux-tegra@vger.kernel.org" , Thierry Reding , Nicolas Iooss , Vince Hsu , Mikko Perttunen Subject: Re: [RFC PATCH 0/3] iommu: Add range flush operation Message-ID: <20150929164014.GL21513@n2100.arm.linux.org.uk> References: <1443504379-31841-1-git-send-email-tfiga@chromium.org> <560A9E36.9030903@arm.com> <20150929143241.GI21513@n2100.arm.linux.org.uk> <560ABBE0.8020805@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <560ABBE0.8020805@arm.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Sep 29, 2015 at 05:27:12PM +0100, Robin Murphy wrote: > Eh, swings and roundabouts. An argument denoting whether the flush is being > called on the map or unmap path would be fine, Sorry, that statement is wrong. It's not about whether you flush before or after the DMA operation. I'm afraid I'm probably going to tell you how to suck eggs here, because I don't think you quite "get it" with non-dma-coherent modern CPUs. Modern CPUs prefetch data into their caches, and they also randomly write back data from their caches to memory. When performing a DMA operation from device to memory, you need to do two things with CPU caches which aren't coherent: 1. Before starting the DMA operation, you need to walk over the memory to be mapped, ensuring that any dirty cache lines are written back. This is to prevent dirty cache lines overwriting data that has already been DMA'd from the device. 2. After the DMA operation has completed, you need to walk over the memory again, invalidating any cache lines which may have been speculatively loaded from that memory while DMA was running. These cache lines may have been loaded prior to the DMA operation placing the new data into memory. So, it's not a before-or-after, you have to always perform write-back cache maintanence prior to any DMA operation, and then invalidate cache maintanence after the DMA operation has completed for any mapping which the DMA may have written to (which means device-to-memory and bidirectional mappings.) -- FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up according to speedtest.net.