From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15F8FC433EF for ; Thu, 28 Apr 2022 15:08:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348831AbiD1PLO (ORCPT ); Thu, 28 Apr 2022 11:11:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50820 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234234AbiD1PLN (ORCPT ); Thu, 28 Apr 2022 11:11:13 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 8A217522F8; Thu, 28 Apr 2022 08:07:58 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3375A1474; Thu, 28 Apr 2022 08:07:58 -0700 (PDT) Received: from [10.57.80.98] (unknown [10.57.80.98]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9725C3F774; Thu, 28 Apr 2022 08:07:54 -0700 (PDT) Message-ID: Date: Thu, 28 Apr 2022 16:07:49 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: Re: [RFC PATCH 1/2] swiotlb: Split up single swiotlb lock Content-Language: en-GB To: Andi Kleen , Christoph Hellwig Cc: Tianyu Lan , m.szyprowski@samsung.com, michael.h.kelley@microsoft.com, kys@microsoft.com, parri.andrea@gmail.com, thomas.lendacky@amd.com, wei.liu@kernel.org, Tianyu Lan , linux-hyperv@vger.kernel.org, konrad.wilk@oracle.com, linux-kernel@vger.kernel.org, kirill.shutemov@intel.com, iommu@lists.linux-foundation.org, andi.kleen@intel.com, brijesh.singh@amd.com, vkuznets@redhat.com, hch@lst.de References: <20220428141429.1637028-1-ltykernel@gmail.com> <20220428141429.1637028-2-ltykernel@gmail.com> <1517d2f0-08d6-a532-7810-2161b2dff421@linux.intel.com> From: Robin Murphy In-Reply-To: <1517d2f0-08d6-a532-7810-2161b2dff421@linux.intel.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2022-04-28 15:55, Andi Kleen wrote: > > On 4/28/2022 7:45 AM, Christoph Hellwig wrote: >> On Thu, Apr 28, 2022 at 03:44:36PM +0100, Robin Murphy wrote: >>> Rather than introduce this extra level of allocator complexity, how >>> about >>> just dividing up the initial SWIOTLB allocation into multiple io_tlb_mem >>> instances? >> Yeah.  We're almost done removing all knowledge of swiotlb from drivers, >> so the very last thing I want is an interface that allows a driver to >> allocate a per-device buffer. > > At least for TDX need parallelism with a single device for performance. > > So if you split up the io tlb mems for a device then you would need a > new mechanism to load balance the requests for single device over those. > I doubt it would be any simpler. Eh, I think it would be, since the round-robin retry loop can then just sit around the existing io_tlb_mem-based allocator, vs. the churn of inserting it in the middle, plus it's then really easy to statically distribute different starting points across different devices via dev->dma_io_tlb_mem if we wanted to. Admittedly the overall patch probably ends up about the same size, since it likely pushes a bit more complexity into swiotlb_init to compensate, but that's still a trade-off I like. Thanks, Robin.