From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qv1-f50.google.com (mail-qv1-f50.google.com [209.85.219.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 21CCA3DDDAF for ; Tue, 21 Jul 2026 17:55:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.50 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784656535; cv=none; b=fHaHE/B3KlQ5jM0ZPvfEeyJ2JLDqT7NwOQwPu4ikPhEA5xJiRqTyxJf6UeD6/oyZQ7CfPPzm0c4Vf4Vr9oekLhgxJkRLEQSSRb02A8wDxqg8oJhfangx0UlYpc/uczo45FOPCeBHSRs1ARqwKXr+EtUCjNo/yamy83OR7EednsA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784656535; c=relaxed/simple; bh=fzgF3V5HF54yEqsnjagon7PnrwUxx4c7CuxO/Uie8BY=; h=MIME-Version:Content-Type:Subject:From:To:Cc:In-Reply-To: References:Date:Message-Id; b=m8W2n48GOB6RumEVvC41mfqeATt9BKm6XalU7YU8K0b1sntkCsQG3sDG2ktgXEpl0cxdGSIZIiN1XLuAO7rHZYEL+qtrG0alTAjLwcyA0ZI959FIbYLn12+RnBVmfXDGQg9qfQemtTgT+sBTn4hbzeEuqlOJWxQvE4vy70jaV3Y= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=soleen.com; spf=pass smtp.mailfrom=soleen.com; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b=K0laAEey; arc=none smtp.client-ip=209.85.219.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=soleen.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=soleen.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="K0laAEey" Received: by mail-qv1-f50.google.com with SMTP id 6a1803df08f44-8ee6912d86dso58238276d6.1 for ; Tue, 21 Jul 2026 10:55:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; t=1784656532; x=1785261332; darn=vger.kernel.org; h=message-id:date:references:in-reply-to:cc:to:from:subject :content-transfer-encoding:content-type:mime-version:from:to:cc :subject:date:message-id:reply-to:content-type; bh=ea/F6w9Zt+Er+ldXlrQTi7dDqHaImBI68nyXrbPqNKM=; b=K0laAEeyd7GAJlfJoCpTXx4C85DiDHZC65fsbJtHt7/l/9sRBN4i+dIsna4LdGG6hZ 8zqZ3sN/cfEFExEwH3IRhCm+M9dAymM80S0R8A6ahGMAPCGdV3V4DdAS6EDQ3wsUxFw3 9fCQ84OmA3mNtZZy9hGdcKjXDh52K8UJIUXalbwvwAVC1vxbDuvyBsNMkuC7PKR0UJ1q nVaNdVv5VThivavJmfHIy/TPlcb0JgfFJzn8EXZLMj82KctMFN6eTyLDmwOVTcuVoNSQ YIVGoW52O0C1ghhAxZkdOpLu7kdbEQzS96NDnlYl5jEF1ghlNgB28YBLFS9tOZ0JI7gQ x2yQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1784656532; x=1785261332; h=message-id:date:references:in-reply-to:cc:to:from:subject :content-transfer-encoding:content-type:mime-version:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=ea/F6w9Zt+Er+ldXlrQTi7dDqHaImBI68nyXrbPqNKM=; b=qIFq235N84p8HaZQwLUZVuQcHRA+dN5xxvCJxQ2CPDSFWpcq1D+g1AIDBZP5HR3Zgl D00yeFwdzbB7tAm/KdTk3kBap9dSaKobhS9x9T6DNv7t1+iH1FnMS0r9YRUZdqMuSKDI udg5UeTx0wMDiWRFUjGm5OwoOf+TaWoYZ0BFVfvT/Ya5//C7v11VwZ1V/RceCG5nVS2w passd27A1L6G/rDFzXLMLz0/BQiDi1kz4djcdTkT705iSG2ehEYJODUkyi+91HWFtmkF O2K6MVk5uYfpJp/kJN6lPQLAX+XSKu9Atc4PRQc1PgafOCTEq00dPrmTjvEAu1ZCzxgp 7Y8w== X-Forwarded-Encrypted: i=1; AHgh+RqCOanJCRVfGQUg5OS32CfQq69qC+MXk17uW1GnRwQjM7Ma4VzxakkZKhjeUVI1Kiv/BSx0I+IjD+maCB0=@vger.kernel.org X-Gm-Message-State: AOJu0YwFe9VOsp55LZo3TWjBHls3rGjAvelX6jLr+yHeRAqBhLM+u3Vf Q1SshU59xm80s4KTRCXap82Wr49dA8BmzmwzSZHOXWufqtUXhQxLbowwPmCrEntTLE0= X-Gm-Gg: AR+sD10oSfxoOya3vFZ7r6VU9A2Bi+zfim7nDd4VnQihSDAGb/h9zAcSgHPF/kl8tuU S+jvmXVL4za+fyoCX8tZbABFnX683pO7w3CRqeecUZBhs7nY+NOuZ+waDaN8/dYmmuwlC14fbIN f6eF9U2yy2OWe7vrEYVEZA5RPayacQ8C1EL83z1tHwFvJxxq57WalNXZgX0y8RvbQnSm8IZH2dF cf+NiHcx75q92GtpmY5nRfeYI8/t5tJrsrtSG15Dz0YOFlTCrOD9CPl5t7ebDi+WuQOHnhMLxhU UN7O1u6HmN28ZxjlcKbuyaW61d8ix2HeT66WiLnIxHwW6oDX1nH0s3EXgP8z0fFrguvHu5kzMKO OXgsqSYY4fJ6Lft0kwrdxUr+Uj0JO79WHZr8ptoRpy3ucqoQfoB8YtsoGp/KNYw5aIMkOMVvPkA CpxnbuPVmWbK7FuPOsl0UAuvUy7hRTCywWk0wtsmc= X-Received: by 2002:a05:6214:2269:b0:8f1:8937:5dc7 with SMTP id 6a1803df08f44-907784ddf91mr239817166d6.56.1784656532475; Tue, 21 Jul 2026 10:55:32 -0700 (PDT) Received: from [127.0.1.1] ([71.181.43.54]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-907ba8df5d0sm1663826d6.18.2026.07.21.10.55.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Jul 2026 10:55:31 -0700 (PDT) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Subject: Re: [PATCH v7 03/12] PCI: liveupdate: Track incoming preserved PCI devices From: Pasha Tatashin To: David Matlack Cc: Pasha Tatashin , kexec@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, Adithya Jayachandran , Alexander Graf , Alex Williamson , Bjorn Helgaas , Chris Li , David Rientjes , Jacob Pan , Jason Gunthorpe , Jonathan Corbet , Josh Hilke , Leon Romanovsky , Lukas Wunner , Mike Rapoport , Parav Pandit , Pranjal Shrivastava , Pratyush Yadav , Saeed Mahameed , Samiullah Khawaja , Shuah Khan , Vipin Sharma , William Tu , Yi Liu In-Reply-To: References: <20260710212616.1351130-1-dmatlack@google.com> <20260710212616.1351130-4-dmatlack@google.com> <178432481253.189683.3297727348836286619.b4-review@b4> <178458744511.332171.13770778781241262180.b4-reply@b4> Date: Tue, 21 Jul 2026 17:55:29 +0000 Message-Id: <178465652971.412167.17838319728990505256.b4-reply@b4> X-Mailer: b4 0.15.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=3765; i=pasha.tatashin@soleen.com; h=from:subject:message-id; bh=fzgF3V5HF54yEqsnjagon7PnrwUxx4c7CuxO/Uie8BY=; b=owEBbQKS/ZANAwAKAbt3KEzbc3reAcsmYgBqX7KSG2dHPEQ6Mkut+xegrodYi1A9KSpnUO3pn xL+SSaK1qOJAjMEAAEKAB0WIQRBMaqT7LRvGvB/NmK7dyhM23N63gUCal+ykgAKCRC7dyhM23N6 3sbLD/9VJw+QusBVAbX9+6asMemVQiDftMc/hr+I22YGh5yCipWq3pZjfM7mP8bljemHy6oIbj+ eg+gEXv7pWzzQ+8GV3PpuOBYxkjLRkI0gemShPjl4DIrRF1i5QKtbirG+j8HJShkbA0MUFQYF4w etoSvEkZ5OowXog4ZxpFFJrtp949gwoLonQntZS2YEdepjeuQ6O5+29aDjXTwrqt8dr3iq97QY7 PEegM9HW0HVvWmt7PoJ0dx0EEaKZJ0GqMpr9O8W2GdxDoZXl+Sk9BxKiwyOX4J5Jz7rdQAn/o7I Ns4QUk/VtBQL/1irI528G9H+z0gGxw4nCxP8fANN7VQW2Y68d+e7tNZOgJEXmtgQe6lh2ihna3W AJSXMdazYky84+h1aAgVPom/cLGQhgzwHgZSps6nDG9qMSTLZGGAP5VPvHpjCAt1MsLtajskJ9m Pzp1qdozYbecunkANckh7aN32FTvRvmPFrVjU4a/bjB1bsG9O6ocEd2lvY5VLJoI/7EpNjFQENT gzqd9ygy3xIcDV9ifvMhU0XWWUteFUjC96g3UGsi04IcPbR5Oe4Fe0S0UiiqqUxgncTyddsTlmF IyJ5BW5Xc76J0oYVAhS14CIxQNY+9LXWUbBAgmqx4XEDE427j+I0Ic4eKf2seStQEq8rQZ1xQbi o4SPKFvgFZN6Jyg== X-Developer-Key: i=pasha.tatashin@soleen.com; a=openpgp; fpr=CAAAB722DD22A081F0D49F35633A6A993D43B569 On 2026-07-20 16:07:47-07:00, David Matlack wrote: > On Mon, Jul 20, 2026 at 3:44 PM Pasha Tatashin > wrote: > > > On 2026-07-20 14:54:51-07:00, David Matlack wrote: > > > > Thanks for the explanation. As I understand, the most straightforward > > way to avoid holding the permanent reference is indeed to delete > > dev->liveupdate.incoming entirely and perform an xarray lookup on every > > access, like this: > > > > bool pci_liveupdate_is_incoming(struct pci_dev *dev) > > { > > ... > > incoming = pci_liveupdate_flb_get_incoming(); > > ... > > dev_ser = xa_load(&incoming->xa, key); > > ... > > pci_liveupdate_flb_put_incoming(); > > return dev_ser && dev_ser->refcount > 0; > > } > > > > However, as you note, this is inefficient because it affects every > > single device and adds lookup overhead to every access (not sure about > > the actual cost though, xarray access is pretty fast!). > > > > We can, however, still avoid tinkering with the lifecycle of the FLB, > > and instead treat dev->liveupdate.incoming as a "hint" that we validate > > on access with a fast, liveness check: > > Can you tell me more about your concern about FLB lifetime? > > The lifetime of the FLB will not be affected by this reference unless > there is a bug in the driver where it fails to call > pci_liveupdate_finish() during it's file handler finish callback. >From a design perspective, liveupdate_flb_get/put_incoming() is a logical read-lock/unlock pair on the FLB data. We use a refcount for optimization and sharing, but holding a get over a long asynchronous gap (from boot-time device setup to driver probe) is essentially holding an unbound lock. Unbound locks make it difficult to trace refcount leaks or debug lifecycle issues. > > > 1. At Setup: In pci_liveupdate_setup_device(), we do the xarray lookup > > once, cache the pointer in dev->liveupdate.incoming, and immediately > > call pci_liveupdate_flb_put_incoming(). We do not hold a permanent > > reference. > > > > 2. On Access: When an accessor runs, instead of doing a full xarray > > lookup, it just validates the cached pointer's liveness by temporarily > > securing the FLB: > > > > static struct pci_flb_incoming *pci_liveupdate_get_incoming(struct pci_dev *dev) > > { > > struct pci_flb_incoming *incoming; > > > > incoming = pci_liveupdate_flb_get_incoming(); > > if (!incoming) > > return NULL; > > > > if (dev->liveupdate.incoming) > > return incoming; > > > > pci_liveupdate_flb_put_incoming(); > > return NULL; > > } > > > > * If get_incoming() returns NULL (the FLB has already finished/freed), > > the hint is invalid and the device is no longer incoming. > > This avoids the xarray lookup but still requires taking the incoming > FLB mutex twice (once for get and once for put) on every access. And > if there's no incoming PCI FLB, the LUO will iterate over all incoming > FLBs under the mutex to find it. Can we do a fast-path check first? static struct pci_flb_incoming *pci_liveupdate_get_incoming(struct pci_dev *dev) { struct pci_flb_incoming *incoming; /* Fast-path to avoid unnecessary FLB querying */ if (!dev->liveupdate.incoming) return NULL; incoming = pci_liveupdate_flb_get_incoming(); if (!incoming) return NULL; /* Check again, now that FLB is acquired */ if (dev->liveupdate.incoming) return incoming; pci_liveupdate_flb_put_incoming(); return NULL; } This seems to gives us the best of both worlds: robust refcount hygiene and a sane fast path. What do you think?