From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 215B647ECE2 for ; Wed, 6 May 2026 16:42:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.153.30 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778085768; cv=none; b=VaGovEUXrQWDM+um/pwtkoBVOBtv9uS7swypkVPV5kJY8Whdx0bRNcWuvAR5kmph6Wr/TcW2xBbSTUMvasxp3uPZsJUrrTgNsK3ornzNZRugXynbouWjDZkDRHrphwp97KEEMPVN32JSIkKMfQBuWqMVtWKtiuSi8G8pI88k6Wc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778085768; c=relaxed/simple; bh=PIgsAxwjAni+ibNITbesgAa9yzG9WpIb9zhCZJY95/s=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=DLzWHcJVq9HBXd00HKHACIdPSQK3yzPIJm2bb8BnKI8tMz6KIq28sw8isyirC0xr2flkZmLG01mi3q6765DR6hxvIC/5e7xfs5j0tYvB3E6O7Rab3Q7WykgMAGvIp/eGC7C/SYmNlzZ/jlLkcC1WS9bw3dP/UL6h2ZnAYAWNWOg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=LVK6U0wJ; arc=none smtp.client-ip=67.231.153.30 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="LVK6U0wJ" Received: from pps.filterd (m0109332.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 6466ZkxO3475731 for ; Wed, 6 May 2026 09:42:46 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2025-q2; bh=KvzelPFNcl7MU+TM0miXboxQSsRGRRlAbQLX3lQgau0=; b=LVK6U0wJ9iIb vlscmhtMH1uiKx6GtvkvD3PFd3zgD6F0pJGalLirUzyn7txEkM7B87H6hJ3gBB/R xy6LpmR0MAd+x0ZIqp8r0Hyc0y1/04XFD7dLYOiSF6fGh+E8tNIVi/vB/zavG3F8 8TN42shcv5hTMVXNvb74VYPbCsICbn9zU0M4AXi1sq9f+h9zZCOy9fgdkWE+OWLN jvoNlSSljZ+OfTzxu3364d0Kvicl9FDoxbDiiEVrSaNxol0u2vIUnHyNPLDd0Ix5 f5o4c4x9PCLCIVbSwzFZPTO6tjK7i7qWftyIVCUjJ+tCPZ9EmmMKHs9roWgbBdAX UkgXxMdO6A== Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 4dwf0dhj49-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Wed, 06 May 2026 09:42:45 -0700 (PDT) Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-488dcaf2f2fso50365595e9.0 for ; Wed, 06 May 2026 09:42:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778085764; x=1778690564; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KvzelPFNcl7MU+TM0miXboxQSsRGRRlAbQLX3lQgau0=; b=VlmcU337IidpenO2DSCA7PQO2UhEwdtPSHTb21XB/ZKixEsqTD/QnV5vbLRKeb4+tB QGUPDhWl4HpeXQ2RRKaVeXvj+U/tyI7YkoJHST8srzKkHm+JHU/D9SFlhWV1qgyBMYUw GseVCScTC1epg0fQcGUb2k/fLuX1+p9Geijh+KooGcxifVt4vrnnX+6DYrzz9QZmZa0S avLDp4BeYpzAi57PrS+xydv50y2pFDP7fGXdXzg3cIBbeRGgSAURrKZ6dG42jvzsUQUA a79doZQNTXDAJe1ZpBHizcM3YI/Mz7J1QCLmdJK220mdu3D2KHVOZkhJ5inOv7W8FxTj Zbow== X-Forwarded-Encrypted: i=1; AFNElJ988ryOTuBnq/L9E8kClegMSg5C7yQlAagyJ+gtFkqrrHbpby7XsnoMC9xiKFZdqZdjGd6BamMmdqlIqDM=@vger.kernel.org X-Gm-Message-State: AOJu0Yx4CPmp1cSg94OV9l0H/rGuIr11W7bGNDjUE7cZwnKmcjattHTn 9xTqTWXBFAHSZ3L7EAB8cZZ4IntXtvNPUb42Mo5QfIWvQQ0c6AbUwcru/N2k5XFwseE6EdNsGz3 SDifaUtuiuWaN5sdxrFhqpj9vgbX4gM0r5GnZeD4J0WeYPuPR0FNUNZ0LT1ZJn3Na X-Gm-Gg: AeBDietaQvb840PnfW4boz1MClsavxlf48nx08r3Ux+BSY2FgXkBb4cF/SnUARsjuwZ 3SbHhjfXQiLfdcGeG0Xxn4vPmGm7cfGhSxTuGIrMaMPCU8nx/6QmPFZVh5dWpLFBzfz+ZHvMXML eWOL9busEctVa05E3sSts6WaWQHN1HNVIUB4cgu2W9bVLPoBUS6MIEK/K9+AalZRA+v5Kauw83l lzLOfnoai/8rzMOvA4SBdZlMuvcwdP7LHyP/Xys65WO1KM9dUDl2ZyR3I1JS1/FE69tXNHOITfR Cvp4XcqrgGcNLHqosb/UFhODywsU1kX4bRgK+ua2fyKD7YaiztSEdTZKghDaTVVvrEu59nauwbA mekxvk72653+NJFjLxxURzZq35XYUH6rVh6IzHSvw3L/7hpbTIpCRyw8m999CeXJvyE0sLrN8/2 q1/NmVXvLKF2sRZJ/hsHlwIzHZFySv6GnCh//rxgJw7rHmZKCCSKPsmRrVZ3+JEDGzyvTt6yMIm WNie8ZzR7GIRSdSdsLpTlfI3sec0AzKSQ== X-Received: by 2002:a05:600c:2ed3:b0:488:d6eb:e63c with SMTP id 5b1f17b1804b1-48e51f427b3mr46111905e9.15.1778085764699; Wed, 06 May 2026 09:42:44 -0700 (PDT) X-Received: by 2002:a05:600c:2ed3:b0:488:d6eb:e63c with SMTP id 5b1f17b1804b1-48e51f427b3mr46111525e9.15.1778085764222; Wed, 06 May 2026 09:42:44 -0700 (PDT) Received: from ?IPV6:2001:8b0:8b6:13d4:102e:f2af:e074:5cde? (e.d.c.5.4.7.0.e.f.a.2.f.e.2.0.1.4.d.3.1.6.b.8.0.0.b.8.0.1.0.0.2.ip6.arpa. [2001:8b0:8b6:13d4:102e:f2af:e074:5cde]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e538fb1a7sm113622555e9.9.2026.05.06.09.42.42 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 06 May 2026 09:42:43 -0700 (PDT) Message-ID: <9a7e51bc-351d-4ca0-8080-fd9e0265f3a0@meta.com> Date: Wed, 6 May 2026 17:42:42 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 1/9] vfio/pci: Fix vfio_pci_dma_buf_cleanup() double-put Content-Language: en-GB To: Leon Romanovsky Cc: Alex Williamson , Jason Gunthorpe , Alex Mastro , =?UTF-8?Q?Christian_K=C3=B6nig?= , Mahmoud Adam , David Matlack , =?UTF-8?B?QmrDtnJuIFTDtnBlbA==?= , Sumit Semwal , Kevin Tian , Ankit Agrawal , Pranjal Shrivastava , Alistair Popple , Vivek Kasireddy , linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, kvm@vger.kernel.org, =?UTF-8?Q?Carlos_L=C3=B3pez?= References: <20260416131815.2729131-1-mattev@meta.com> <20260416131815.2729131-2-mattev@meta.com> <20260501131236.278ac431@shazbot.org> <9304aada-ee84-4cf2-a1d7-82313eda07aa@meta.com> <20260506152937.GJ11063@unreal> <20260506161441.GK11063@unreal> From: Matt Evans In-Reply-To: <20260506161441.GK11063@unreal> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Authority-Analysis: v=2.4 cv=Y7LIdBeN c=1 sm=1 tr=0 ts=69fb6f85 cx=c_pps a=ocXdEHcuFBd2kx0v6vcWqw==:117 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=NGcC8JguVDcA:10 a=VkNPw1HP01LnGYTKEx00:22 a=7x6HtfJdh03M6CCDgxCd:22 a=xtH7KyWI9dI7BmFOsl-x:22 a=VwQbUJbxAAAA:8 a=Ikd4Dj_1AAAA:8 a=VabnemYjAAAA:8 a=uIBLic6oc_-Le5kNHd0A:9 a=QEXdDO2ut3YA:10 a=XRP02QYyWapGEjE7oHHB:22 a=gKebqoRLp9LExxC7YDUY:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNTA2MDE2MyBTYWx0ZWRfX5iKKtV9uRWZ5 yE1OoX8ycXPv8Pl47BK2nPrB5IYJvAfrwMyih/94H9J2XmAP8mj8genPmkpSvNLp5FnTa1JXdLW zTY+NhX6CUpWjEZhiL2qWhB0Jip7X/8g9JeBDxbLQf6XnlddwWRYqSoBH5/ZcrkR3gM2+WhBkcd H2nnW4vtpA87V2HpWpjW5OXsRgvoHbZ3UknYgk2g6ntCXNM+WL2x9evRZY4036BV4kFqeTb0tbu 2OVt3m7LcD1+6w+5vJAvbBuepPOYdfFqvysLUw/QqqOxSazN+fWQJpK7DBBrEkjspXEKRqfuCqj AikTdnw+l6tDTV/sXs3Vd8UGYvBBU7cSF5+6Sv/l0TZC4yPeZN+MoLNObZ+ib0MX1eSxnqMqVMu g1SDIzzW49ai3e/Kj9ywjp7wif7Dwu9mm6ZK9A/4961gSN0dpPYLIWz4JkWHAmiBE86StZoq4K2 MgpBmpdTyHIXj93f7+g== X-Proofpoint-ORIG-GUID: tgbCBfXNq12q-BM-U44JlPOOhLEgcrRO X-Proofpoint-GUID: tgbCBfXNq12q-BM-U44JlPOOhLEgcrRO X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-05-06_01,2026-05-06_01,2025-10-01_01 Hi Leon, On 06/05/2026 17:14, Leon Romanovsky wrote: > > On Wed, May 06, 2026 at 04:55:27PM +0100, Matt Evans wrote: >> Hi Leon, >> >> On 06/05/2026 16:29, Leon Romanovsky wrote: >>> >>> On Wed, May 06, 2026 at 02:53:31PM +0100, Matt Evans wrote: >>>> Hi Alex, >>>> >>>> On 01/05/2026 20:12, Alex Williamson wrote: >>>>> >>>>> On Thu, 16 Apr 2026 06:17:44 -0700 >>>>> Matt Evans wrote: >>>>> >>>>>> vfio_pci_dma_buf_cleanup() assumed all VFIO device DMABUFs need to be >>>>>> revoked. However, if vfio_pci_dma_buf_move() revokes DMABUFs before >>>>>> the fd/device closes, then vfio_pci_dma_buf_cleanup() would do a >>>>>> second/underflowing kref_put() then wait_for_completion() on a >>>>>> completion that never fires. Fixed by predicating on revocation >>>>>> status. >>>>>> >>>>>> This could happen if PCI_COMMAND_MEMORY is cleared before closing the >>>>>> device fd (but the scenario is more likely to hit when future commits >>>>>> add more methods to revoke DMABUFs). >>>>>> >>>>>> Fixes: 1a8a5227f2299 ("vfio: Wait for dma-buf invalidation to complete") >>>>>> Signed-off-by: Matt Evans >>>>>> --- >>>>>> >>>>>> (Just a fix, but later "vfio/pci: Convert BAR mmap() to use a DMABUF" >>>>>> and "vfio/pci: Permanently revoke a DMABUF on request" depend on this >>>>>> context, so including in this series.) >>>>> >>>>> We really need a fix for this split out from this series, It's already >>>>> been shown[1] that this is trivially reachable. Carlos proposed[2] a >>>>> similar solution to the one below. I was concurrently working on the >>>>> issued and suggested an alternative[3]. Let's pick a solution for >>>>> 7.1-rc. Thanks, >>>> >>>> It looks like [3] is progressing, so I'll drop this one when I can rebase >>>> onto it. >>>> >>>> I noticed [3] removes the dma_resv_lock(priv->dmabuf->resv) around the >>>> priv->vdev = NULL, and this series' vfio_pci_mmap_huge_fault() relies on >>>> vdev only changing whilst resv is held to resolve a race between a fault and >>>> cleanup (see patch 7 of this series). The handler takes resv so that it can >>>> stably test vdev in order to take memory_lock. >>> >>> I think that you should rely on priv->revoked and not on priv->vdev. >> >> Needs both unfortunately, as the fault handler ultimately needs to take >> vdev->memory_lock. > > One can argue that if priv->revoked == True, all accesses to device > should be denied and treated as priv->vdev == Null. I agree, the handler will early-exit when a buffer is revoked. Though when it _isn't_ revoked, it still needs to go through a careful set of steps to keep vdev around long enough to take the lock (and ensure it still isn't revoked, etc.). I think the sequence in patch 7 still works (with Alex's patch in [3]), since the invariants still hold: - if not-revoked then vdev is still valid (IOW, vdev = NULL only happens after revoked = true) - revoke is only changed when holding priv->dmabuf->resv OK, [3] doesn't seem to break this series (just context/rebase). Sorry for the thinking out loud, it'll be good if someone sees a flaw in my reasoning though. [3] was https://lore.kernel.org/all/20260429142242.70f746b4@nvidia.com/ Matt