From: Vidya Sagar <vidyas@nvidia.com>
To: Bjorn Helgaas <bhelgaas@google.com>,
Dan Williams <djbw@kernel.org>,
"Dave Jiang" <dave.jiang@intel.com>,
Jonathan Cameron <Jonathan.Cameron@huawei.com>,
Joerg Roedel <joerg.roedel@amd.com>,
Jason Gunthorpe <jgg@ziepe.ca>,
"Nicolin Chen" <nicolinc@nvidia.com>,
Kevin Tian <kevin.tian@intel.com>
Cc: <vsethi@nvidia.com>, <kthota@nvidia.com>,
<mmaddireddy@nvidia.com>, <kumarahul@nvidia.com>,
<sagar.tv@gmail.com>, <linux-pci@vger.kernel.org>,
<linux-kernel@vger.kernel.org>, Vidya Sagar <vidyas@nvidia.com>
Subject: [PATCH V1 0/4] PCI: Don't list reset methods that can't reset the device
Date: Thu, 19 Nov 2026 15:36:24 +0530 [thread overview]
Message-ID: <20261119100628.1948990-1-vidyas@nvidia.com> (raw)
On a 4-function ConnectX-8 below a CXL-capable Root Port that has no
hotplug slot, "reset_method" lists methods that can never reset the
device:
- Every function lists "cxl_bus". It can be selected, but every reset
then fails with -ENOTTY, because a Secondary Bus Reset of the Root
Port would also reset the other functions.
- Function 0 also lists "bus", which the kernel then refuses to select
("Unsupported reset method"). Its reset methods are probed while it
is still alone on its bus, before the other functions are scanned.
Separately, probing "bus" or a device-specific reset runs the IOMMU
reset hooks even though no reset follows. Reset methods are probed
again on every write to "reset_method", so on a bound, ATS-capable
device behind an IOMMU each such write blocks the device's DMA for a
moment and, on arm-smmu-v3, sends ATC invalidations.
Patch 1 offers "cxl_bus" only where the slot or parent bus reset that it
relies on can reach the device, i.e., under the same conditions as
"bus" except for the CXL SBR mask. The probe uses the same slot and
parent-bus checks that the "bus" probe has always made, from the same
callers.
Patch 2 skips the IOMMU reset hooks when only probing, as pcie_flr(),
pci_af_flr(), pci_pm_reset() and pci_dev_acpi_reset() already do.
Patch 3 takes the device lock when "reset_method" is written. A reset,
which already holds that lock, could otherwise walk the list while a
write is replacing it.
Patch 4 re-checks function 0's reset methods at the end of
pci_scan_slot(), once the other functions are on the bus, and drops the
ones that no longer work. By then device_add() has already created
function 0's "reset_method" file and sent the "add" uevent, so the
re-check only removes methods and keeps the order of the rest, rather
than overwriting a list that userspace may have set. It holds the
device lock, so a write can't land between reading the list and storing
the result. It needs the first three patches: without patch 1, function
0 still lists "cxl_bus"; without patch 2, the re-check, which runs after
device_add(), would cycle function 0's IOMMU domains during enumeration;
and without patch 3, writes to "reset_method" don't take the lock.
Patches 1 and 2 have Fixes tags (v6.10 and v7.0). Patch 2 uses the
helper that patch 1 adds, so a backport of patch 2 needs patch 1 too.
Patches 3 and 4 address long-standing behaviour and have no Fixes tags.
Not covered: "reset_method" of a device that is already added is not
updated when functions appear on its bus later, e.g., VFs or functions
added by a rescan. Writes to "reset_method" and the reset itself still
check again and refuse a bus reset in that case.
Testing: all four patches were tested on the ConnectX-8 above, applied
incrementally (patch 1, then patches 1-2, then patches 1-4):
- Patch 1: after "echo default", functions 0 and 1 list only "flr".
Function 0 still lists "flr bus cxl_bus" at boot and refuses both
"bus" and "cxl_bus", which is what patch 4 addresses.
- Patches 1-2: writing "reset_method" on a function with its driver
bound no longer runs the IOMMU reset hooks.
- Patches 1-4: function 0 lists only "flr" at boot. A single-function
ConnectX-8 below the same kind of Root Port still lists "bus" and
"cxl_bus".
- Patches 1-4: 100 remove/rescan cycles of the 4-function ConnectX-8,
with a udev rule writing "flr bus" to function 0's "reset_method" on
"add", a loop writing the same while function 0 was alone on the bus
during each rescan, and a loop reading "reset_method" of all
functions. Writes landed before the re-check in all 100 cycles,
function 0 ended every cycle with "flr", and there were no warnings.
Writing "bus" instead left function 0 with an empty list in all 100
cycles, i.e., the re-check kept the written list minus "bus".
- Patches 1-4, plus a temporary 200 ms delay in the re-check between
reading the list and storing the result: 100 remove/rescan cycles,
each writing "" to function 0's "reset_method" as soon as the last
function appeared. Every write waited for the re-check (203-206 ms),
and function 0 ended every cycle with an empty list.
The analysis, the fixes and the changelogs were drafted with an AI
coding assistant (Claude Opus 5.5), working from logs of these resets
and a reading of drivers/pci. I reviewed all of it and did the testing
described above.
Vidya Sagar (4):
PCI/CXL: Don't offer 'cxl_bus' reset where a bus reset can't work
PCI: Don't run IOMMU reset hooks when only probing reset methods
PCI/sysfs: Take the device lock when setting reset methods
PCI: Re-check function 0 reset methods after scanning its slot
drivers/pci/pci-sysfs.c | 3 ++
drivers/pci/pci.c | 65 +++++++++++++++++++++++++++++++++++++----
drivers/pci/pci.h | 1 +
drivers/pci/probe.c | 16 +++++++++-
drivers/pci/quirks.c | 3 ++
5 files changed, 81 insertions(+), 7 deletions(-)
--
2.43.0
next reply other threads:[~2026-11-19 10:14 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-19 10:06 Vidya Sagar [this message]
2026-11-19 10:06 ` [PATCH V1 1/4] PCI/CXL: Don't offer 'cxl_bus' reset where a bus reset can't work Vidya Sagar
2026-11-19 10:06 ` [PATCH V1 2/4] PCI: Don't run IOMMU reset hooks when only probing reset methods Vidya Sagar
2026-11-19 10:06 ` [PATCH V1 3/4] PCI/sysfs: Take the device lock when setting " Vidya Sagar
2026-11-19 10:06 ` [PATCH V1 4/4] PCI: Re-check function 0 reset methods after scanning its slot Vidya Sagar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261119100628.1948990-1-vidyas@nvidia.com \
--to=vidyas@nvidia.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=bhelgaas@google.com \
--cc=dave.jiang@intel.com \
--cc=djbw@kernel.org \
--cc=jgg@ziepe.ca \
--cc=joerg.roedel@amd.com \
--cc=kevin.tian@intel.com \
--cc=kthota@nvidia.com \
--cc=kumarahul@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=mmaddireddy@nvidia.com \
--cc=nicolinc@nvidia.com \
--cc=sagar.tv@gmail.com \
--cc=vsethi@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®