From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA609C2F3A0 for ; Mon, 21 Jan 2019 13:57:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 958702087F for ; Mon, 21 Jan 2019 13:57:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731915AbfAUN4X (ORCPT ); Mon, 21 Jan 2019 08:56:23 -0500 Received: from foss.arm.com ([217.140.101.70]:34546 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731888AbfAUN4R (ORCPT ); Mon, 21 Jan 2019 08:56:17 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3FC01EBD; Mon, 21 Jan 2019 05:56:17 -0800 (PST) Received: from [10.1.196.75] (e110467-lin.cambridge.arm.com [10.1.196.75]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F1FD03F614; Mon, 21 Jan 2019 05:56:14 -0800 (PST) Subject: Re: [PATCH 0/3] iommu/arm-smmu: Add support to use Last level cache To: Ard Biesheuvel Cc: Vivek Gautam , Will Deacon , Joerg Roedel , "list@263.net:IOMMU DRIVERS" , pdaly@codeaurora.org, linux-arm-msm , Linux Kernel Mailing List , Tomasz Figa , Jordan Crouse , pratikp@codeaurora.org, linux-arm-kernel References: <20190121055335.15430-1-vivek.gautam@codeaurora.org> From: Robin Murphy Message-ID: <964779d6-c676-3379-bf1e-cde0dd82d63d@arm.com> Date: Mon, 21 Jan 2019 13:56:13 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 21/01/2019 13:36, Ard Biesheuvel wrote: > On Mon, 21 Jan 2019 at 14:25, Robin Murphy wrote: >> >> On 21/01/2019 10:50, Ard Biesheuvel wrote: >>> On Mon, 21 Jan 2019 at 11:17, Vivek Gautam wrote: >>>> >>>> Hi, >>>> >>>> >>>> On Mon, Jan 21, 2019 at 12:56 PM Ard Biesheuvel >>>> wrote: >>>>> >>>>> On Mon, 21 Jan 2019 at 06:54, Vivek Gautam wrote: >>>>>> >>>>>> Qualcomm SoCs have an additional level of cache called as >>>>>> System cache, aka. Last level cache (LLC). This cache sits right >>>>>> before the DDR, and is tightly coupled with the memory controller. >>>>>> The clients using this cache request their slices from this >>>>>> system cache, make it active, and can then start using it. >>>>>> For these clients with smmu, to start using the system cache for >>>>>> buffers and, related page tables [1], memory attributes need to be >>>>>> set accordingly. This series add the required support. >>>>>> >>>>> >>>>> Does this actually improve performance on reads from a device? The >>>>> non-cache coherent DMA routines perform an unconditional D-cache >>>>> invalidate by VA to the PoC before reading from the buffers filled by >>>>> the device, and I would expect the PoC to be defined as lying beyond >>>>> the LLC to still guarantee the architected behavior. >>>> >>>> We have seen performance improvements when running Manhattan >>>> GFXBench benchmarks. >>>> >>> >>> Ah ok, that makes sense, since in that case, the data flow is mostly >>> to the device, not from the device. >>> >>>> As for the PoC, from my knowledge on sdm845 the system cache, aka >>>> Last level cache (LLC) lies beyond the point of coherency. >>>> Non-cache coherent buffers will not be cached to system cache also, and >>>> no additional software cache maintenance ops are required for system cache. >>>> Pratik can add more if I am missing something. >>>> >>>> To take care of the memory attributes from DMA APIs side, we can add a >>>> DMA_ATTR definition to take care of any dma non-coherent APIs calls. >>>> >>> >>> So does the device use the correct inner non-cacheable, outer >>> writeback cacheable attributes if the SMMU is in pass-through? >>> >>> We have been looking into another use case where the fact that the >>> SMMU overrides memory attributes is causing issues (WC mappings used >>> by the radeon and amdgpu driver). So if the SMMU would honour the >>> existing attributes, would you still need the SMMU changes? >> >> Even if we could force a stage 2 mapping with the weakest pagetable >> attributes (such that combining would work), there would still need to >> be a way to set the TCR attributes appropriately if this behaviour is >> wanted for the SMMU's own table walks as well. >> > > Isn't that just a matter of implementing support for SMMUs that lack > the 'dma-coherent' attribute? Not quite - in general they need INC-ONC attributes in case there actually is something in the architectural outer-cacheable domain. The case of the outer cacheablility being not that but a hint to control non-CPU traffic through some not-quite-transparent cache behind the PoC definitely stays wrapped up in qcom-specific magic ;) Robin.