From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B3DF26FA7A for ; Thu, 14 May 2026 17:52:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.153.30 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778781176; cv=none; b=aw3dZ8017jqoclAts4zVRZaDUJm07xRZD5ywJ7UF2js2wj3YdueX5JCoJtDTPkBflGBI4IDLlc20DSwxcvsfgnChi9M3WcjfZXcS6l7BDsvGZ7/Kjr6rFY803z4hDOBIpAUFEJOUq55PMQr3NlIC8iL4bUve9qrDYBqZ/qpXnYQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778781176; c=relaxed/simple; bh=6Lsiopp+gOlTgIhuJUtI3u2C7QdWhtAwDrbE0I7bAro=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=hAFXmx0uEgoDXsDujH/CTM6tS4qYhbapb18FperEtKUscB0hLUAVRM/YNSIHTw4aS0Gvt2UaRCHHDSZRR8uvcOmEt094xcO0N/6NJ2QFPXJdOw+J/bT/xJDGYaq4tFZJ9riDLRniJzJT5x3+3OvYAQto9Gt5/5wOSezLHDJoYd4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=uID4L/RT; arc=none smtp.client-ip=67.231.153.30 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="uID4L/RT" Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 64EFYuYO2340871 for ; Thu, 14 May 2026 10:52:54 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2025-q2; bh=i7CC6BqBBL9skWarpnUsM1vRDVhAGp/lgIIkSrH6DrI=; b=uID4L/RTFdpM ym5kuxwAZGXnMvm5RC/hAqjahIGR4X97DZNauwiy3+2qdGeLwon1A2giOFCwt250 p2DDOFCJbt9Xw927M+aXcmNFZurvTU6m7SIy/PDM/fiVkf7NqG6TpGzja1cpG9MB oTwuC4fhNj9UVZMTydsFMZmhDJ/rrspbDs+/u3Ko5yH+4iTRpMG6q0zdA+yDi3Md tcjYwnReCHshyKmsj+AJkYVjYTAYGsJnOGhPIjZevNDHjlHO3kIJTYdOvoyyMEyt xMAEBJM041u78SHl6dC35/Cnr/q8/tjegsMQIjHnAXNRAxTU9Av7XNoZtpSdIjkC VOZpBcrPXw== Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 4e3nw295mh-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Thu, 14 May 2026 10:52:53 -0700 (PDT) Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-45d95cba6f1so806698f8f.0 for ; Thu, 14 May 2026 10:52:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778781173; x=1779385973; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=i7CC6BqBBL9skWarpnUsM1vRDVhAGp/lgIIkSrH6DrI=; b=a+LqNeNGnFaGdqCkKyWzF1XStCVyPVSakqQCU98n3V2SoaQ9Rs4T0jeF0NV/1eznfc cSP+IzosnSYbq/nRmOtiXDa9wn0YCiGJXaGghdbmM7+RtDDv1Sp1qBoIWw0Vk+l+0VWg 5kxLP5d7OGOc+z8XchAuhvQt66s9gW2ikdEsyhdofYZepxFbF3Msf1NwP0owbQmWdE1Q ORIoc1S0JzYcMF6NxCC6E6mm8U7q91p6sscRS+pPYx62H6C5hLbYAQhD616Jv8dEjj52 6MCkFpmXN8g2Sextj5ZRPWD9AAjp998XtArrKtD8/xvcMK+cTtOBnGC1ARBfeKQfZEiW kq5A== X-Forwarded-Encrypted: i=1; AFNElJ+Mb8Pcv831IAxEOS1ODW2kdFkLfssRugh6pcL3nUbFZy2Iz5oq6rWsIq3lRPgDoI/rELafZWfDKfQdk3c=@vger.kernel.org X-Gm-Message-State: AOJu0YzNbCE1Be6a17x4Y3IjBCTTR1YI6tjDZSdBZgkOAUig28YHm1M5 m/P9IiWWoHexRvdvV/E3AG0cVYHamvKPV+iKvbV2PGivvUDr7uc6TYmaXUx7Sq8qjAJai4z5EpN r/+td2T7WaWCWbfWPXWqrMxONkWVpmAdjCkFktPAsI77PRl1vVgamPaCrNrHdnEaR X-Gm-Gg: Acq92OFA3SD9rXQ5IgCA7mgDpAKQjIkmELGgeAvzzPJYPqUJE/pJ0UEcJYUAIVD9ZJF IMm6AZJb2PdkZBmmIQfVzRJxPXGkhoIXg2irUvv19G11gTX3g+6Tr1NZhmBgv0zAXKMzgrEnW3s Q4e0TjaINhpGCc6SsB90ES5eP4QZquS9w5UGfbS63GFWBJt6N4FpjCApi/IeuCt9cHPC5L4XZTz oCwinDAMwplXI2QrRN22xNaQKXJo/G9has+GBW5Ql3Me3Ch/I1F2Y/CejYznUNtYKLVMSqBMfpp qQNAGhw1QT7ioObPDyBp6sKVxr7DE6OytmWL8mUxv7Y+F+oR9KXTGQs29I7uAvEhEcOKnORvl23 JWHUtBId2wgIHAqbepVWBQRK7NHigNchtcn9y3gW8/rBNmiLhkDkUoJVAkucruiwvX5s+dTrler Wm+BqFJfQyEiM186Sf1eF9csLvh1dnnzyG3bMT8YkAerR8ryvHfH5pukP0tTznZz2Lfv/9GTOaN UaBkd2euQ5DGfGZ+MXcZek= X-Received: by 2002:a05:6000:2f87:b0:452:c246:ab7a with SMTP id ffacd0b85a97d-45e5c5beb43mr98788f8f.14.1778781172699; Thu, 14 May 2026 10:52:52 -0700 (PDT) X-Received: by 2002:a05:6000:2f87:b0:452:c246:ab7a with SMTP id ffacd0b85a97d-45e5c5beb43mr98748f8f.14.1778781172259; Thu, 14 May 2026 10:52:52 -0700 (PDT) Received: from ?IPV6:2001:8b0:8b6:13d4:102e:f2af:e074:5cde? (e.d.c.5.4.7.0.e.f.a.2.f.e.2.0.1.4.d.3.1.6.b.8.0.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:8b6:13d4:102e:f2af:e074:5cde]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45da15a6449sm7910203f8f.37.2026.05.14.10.52.51 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 14 May 2026 10:52:51 -0700 (PDT) Message-ID: Date: Thu, 14 May 2026 18:52:50 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 4/9] vfio/pci: Convert BAR mmap() to use a DMABUF Content-Language: en-GB To: Leon Romanovsky , Alex Williamson , Jason Gunthorpe Cc: Alex Mastro , =?UTF-8?Q?Christian_K=C3=B6nig?= , Mahmoud Adam , David Matlack , =?UTF-8?B?QmrDtnJuIFTDtnBlbA==?= , Sumit Semwal , Kevin Tian , Ankit Agrawal , Pranjal Shrivastava , Alistair Popple , Vivek Kasireddy , linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, kvm@vger.kernel.org References: <20260416131815.2729131-1-mattev@meta.com> <20260416131815.2729131-5-mattev@meta.com> <20260501161915.75525c15@shazbot.org> <20260505104911.GB11063@unreal> <20260505085058.74c34290@shazbot.org> <20260506053511.GG11063@unreal> From: Matt Evans In-Reply-To: <20260506053511.GG11063@unreal> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTE0MDE3OSBTYWx0ZWRfX6YZi5Yrd18bp E42i5FQJNEA6G/lM37+pa/wkLzbD/nyqNfhIDIgMyp2v17UNneNUKss/yZYWNteSg1pZBg7TBVw 7mjFqIKZwJnaPhmVsL1uSgDPd7CS5SywwzSqaj4Lp57dGfp9g6MYRa9+++tgX09jzNwgdIng4fz dVbBzv80KFcond6J52A9rOb2bnG80SQRcFDOQfpr8/pz7srURPCxdRM8nWOMFa18dMGS5fpRIMK jhUe1Ls0mi2Tppyg+BxsqN0fJkaJwnm0NWXFXTAl/iYL1iP/wdXGDDX0Ouys738+rCmI0GudEtp Av2rlQ6QEkWMrRHqq9lF3HM3omv+iQCmC2wR6T/KjhoWx70Xti1YRVCpnq6G/FIijHGRGyzdVdJ +XxL7mXrugSdeSnnlwkDq6ViMoc+UtH82crcH1Oh7PJAE9mb2jvGNSRFHpyO4irKigwOh60U+yb Tknet1R2IIJjwsAohFQ== X-Proofpoint-ORIG-GUID: UxqUh9FjNbzKDzlRz1NKvb3aobJEhX3d X-Proofpoint-GUID: UxqUh9FjNbzKDzlRz1NKvb3aobJEhX3d X-Authority-Analysis: v=2.4 cv=MspiLWae c=1 sm=1 tr=0 ts=6a060bf5 cx=c_pps a=pX1sPZUoWPGeQgUnKQNIgA==:117 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=7x6HtfJdh03M6CCDgxCd:22 a=JnKecZnUtZousrUlYMGU:22 a=VwQbUJbxAAAA:8 a=QhZvmJRm3SQzdzbRGmYA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 a=XGGHjNwGSpTZ5T66x7Ix:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-14_04,2026-05-13_01,2025-10-01_01 Hi Leon, Alex, Jason, On 06/05/2026 06:35, Leon Romanovsky wrote: > > On Tue, May 05, 2026 at 08:50:58AM -0600, Alex Williamson wrote: >> On Tue, 5 May 2026 13:49:11 +0300 >> Leon Romanovsky wrote: >> >>> On Mon, May 04, 2026 at 04:40:41AM -0300, Jason Gunthorpe wrote: >>>> On Fri, May 01, 2026 at 04:19:15PM -0600, Alex Williamson wrote: >>>> >>>>> Exporting dma-bufs from vfio-pci is a feature, but mmap of MMIO BARs is >>>>> a legacy requirement. That legacy requirement now depends on >>>>> PCI_P2PDMA, which depends on 64BIT and ZONE_DEVICE. >>>> >>>> That should be split up now, Leon missed it when he added the new >>>> APIs that didn't require ZONE_DEVICE.. >>> >>> Sorry, what did I miss here? >>> VFIO_DMABUF is an optional feature and is enabled only when P2P support is >>> available. It does not affect legacy systems where P2P cannot be enabled. >> >> If we look at the long term view of moving exclusively to cdev/iommufd, >> where VFIO_DMABUF becomes the mechanism for implementing P2P DMA >> mappings, VFIO_DMABUF may be optional, but it's highly desirable for >> legacy compatibility. There's an argument though that providing P2P >> compatibility on platforms that support PCI_P2PDMA is probably >> sufficient. >> >> However, in providing mmap of dmabufs as a feature, this series is >> wiring all mmaps through dmabufs and therefore that dependency becomes >> fundamental to the use of vfio-pci. Thus the discussion whether the >> noted config requirements could be lifted. Thanks, > > Right, there was no need to remove ZONE_DEVICE when I added my code, and I > left the task of cleaning it out of is_pci_p2pdma_page() for another day. > Without ZONE_DEVICE, all pages are treated as non‑P2P. Just checking I'm following the deps correctly... If we remove the PCI_P2PDMA dependency on ZONE_DEVICE then (without) it's present with P2P hard-disabled (as Leon says). Already `is_pci_p2pdma_page() == false` because folio_is_zone_device() is always false in that config. The only dependency this series adds is for pcim_p2pdma_provider() (and support). We can select PCI_P2PDMA for provider purposes (VFIO!) but if !ZONE_DEVICE there's no P2P. Nothing in PCI_P2PDMA depends on 64BIT and that dependency can be removed [1]. (ZONE_DEVICE depends on 64BIT.) That seems a path to VFIO operating unchanged on 32-bit platforms and others without ZONE_DEVICE. (PCI_P2PDMA currently also depends on NEED_SG_DMA_FLAGS, which seems to be unnecessary if !ZONE_DEVICE is preventing P2P pages (no sg_dma_mark_bus_address() would happen for example), is that right?) Then VFIO_PCI depends on a subset of PCI_P2PDMA configuation: "Diet" PCI_P2PDMA_CORE available even when !ZONE_DEVICE: - Compatible with 32-bit platforms; doesn't need to depend on 64BIT - Returns a provider, which allows VFIO mmap(), DMABUF mmap(). - Doesn't allow P2P DMA (eg. via DMABUFs). - Doesn't need to select NEED_SG_DMA_FLAGS. "Classic" PCI_P2PDMA depending on ZONE_DEVICE: - Behaves as PCI_P2PDMA does today - If ZONE_DEVICE then PCI_P2PDMA enabled, which enables VFIO_PCI_DMABUF Meaning: 32-bit configs (or 64-bit without ZONE_DEVICE): - No P2P anyway - VFIO mmap() works - VFIO_PCI_DMABUF is disabled (no point enabling DMABUF export since it can't be used for its original P2P purpose) 64-bit configs that support ZONE_DEVICE: - As today, DMABUF P2P, mmap() of BARs via VFIO or DMABUF Just added PCI_P2PDMA_CORE and (functionally) ~no code changes are needed since disabling P2P just falls out of !ZONE_DEVICE. But most of p2pdma.c does nothing in that config (except for pcim_p2pdma_provider() and its support) so benefits from some #ifdef to reduce it. WDYT about a pcip2pdma_core.c containing just the provider management? Thanks, Matt [1]: The drivers/pci/Kconfig's PCI_P2PDMA states: # The need for the scatterlist DMA bus address flag means PCI P2PDMA # requires 64bit I think this isn't quite right as AFAICT the flags are usable on 32b too, and actually it's ZONE_DEVICE that requires 64bit.