mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH] cxl/port: Do not queue a memdev detach when the memdev unbinds itself
@ 2026-11-22 12:20 Unsung Lee
  0 siblings, 0 replies; only message in thread
From: Unsung Lee @ 2026-11-22 12:20 UTC (permalink / raw)
  To: linux-cxl
  Cc: linux-kernel, Davidlohr Bueso, Jonathan Cameron, Dave Jiang,
	Alison Schofield, Vishal Verma, Dan Williams, Ira Weiny, Li Ming

Binding a memdev to cxl_mem right after unbinding it often fails: the
probe returns -EBUSY, or the bind succeeds and the memdev is unbound
again a few milliseconds later. In the second case the bind write
returns success, so nothing tells userspace that the device ended up
unbound.

cxl_endpoint_port_probe() registers schedule_detach() as a devm action
of the endpoint port, so that a memdev whose endpoint port, or a port
above it, goes away is detached. Unbinding cxl_mem deletes the endpoint
port itself (delete_endpoint()), which runs that action too and queues
cxlmd->detach_work although the memdev is already going away.
detach_memdev() later calls device_release_driver(&cxlmd->dev), which
takes the device lock and releases whatever driver is bound at that
point. A bind that runs before the work makes cxl_mem_probe() fail its
work_pending(&cxlmd->detach_work) check with -EBUSY. A bind that is in
progress when the work runs is released by it as soon as it completes:

  bash     echo mem0 > unbind: cxl_mem removes endpoint2 and its decoders
  kworker  detach_memdev(): device_release_driver(mem0) waits for the lock
  bash     echo mem0 > bind: cxl_mem_probe() returns 0
  kworker  releases mem0 again, endpoint2 and its decoders go away

Mark the memdev while delete_endpoint() removes the endpoint port and
skip the detach in schedule_detach() then, dropping the reference the
work would have dropped. A memdev with ->attach still gets the detach,
since there it releases the parent device, which depends on the
memdev being attached. Losing the endpoint port or a port above it
otherwise still detaches the memdev as before.

The problem was found by an automated test harness driven by an LLM
coding assistant. The analysis and the fix are my own.

Fixes: 8dd2bc0f8e02 ("cxl/mem: Add the cxl_mem driver")
Signed-off-by: Unsung Lee <unsung0317@gmail.com>
---
Question: is the detach meant to be queued on a cxl_mem unbind too? For
memdevs with ->attach the patch keeps it, as there it releases the
parent; I have no such device to test.

Reproducer, as root in a QEMU guest with one type3 device
(-M q35,cxl=on,cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=8G
 -object memory-backend-ram,id=vmem0,share=on,size=8G
 -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1
 -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2
 -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0):

  M=mem0; D=/sys/bus/cxl/drivers/cxl_mem
  for i in $(seq 40); do
          [ -e /sys/bus/cxl/devices/$M/driver ] || { echo $M > $D/bind; sleep 1; }
          echo $M > $D/unbind
          echo $M > $D/bind 2>/dev/null; rc=$?
          sleep 1
          [ -e /sys/bus/cxl/devices/$M/driver ] && s=bound || s=unbound
          echo "bind rc=$rc $s"
  done | sort | uniq -c

Without the patch 7/40 and 11/40 fail on cxl/next 32af781935 (2 of
the 11 with the bind returning 0 and mem0 unbound after), 14/40 on
v7.3.1.

With the patch on cxl/next (release and KASAN/lockdep builds) and on
v7.3.1 (KASAN/lockdep): 120/120 stay bound on each kernel, unbinding
the endpoint or host bridge port still unbinds the memdev, and 3
minutes of racing memdev and port unbind/bind gave no warnings.

Not tested: memdevs with ->attach, switches, real hardware. Regions were
only checked on v7.3.1: cxl/next creates no root decoder for QEMU's
CFMWS (restrictions 0x2f, "cannot have both HDM-H and HDM-DB").

 drivers/cxl/core/port.c |  3 +++
 drivers/cxl/cxlmem.h    |  2 ++
 drivers/cxl/port.c      | 14 +++++++++++++-
 3 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c
index fffcb9954d..4960a288a1 100644
--- a/drivers/cxl/core/port.c
+++ b/drivers/cxl/core/port.c
@@ -1514,9 +1514,12 @@ static void delete_endpoint(void *data)
 
 	scoped_guard(device, host) {
 		if (host->driver && !endpoint->dead) {
+			/* @cxlmd is going away, see schedule_detach() */
+			WRITE_ONCE(cxlmd->unbinding, true);
 			devm_release_action(host, cxl_unlink_parent_dport, endpoint);
 			devm_release_action(host, cxl_unlink_uport, endpoint);
 			devm_release_action(host, unregister_port, endpoint);
+			WRITE_ONCE(cxlmd->unbinding, false);
 		}
 		cxlmd->endpoint = NULL;
 	}
diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
index 4cf8a0f6cf..0d566b8e32 100644
--- a/drivers/cxl/cxlmem.h
+++ b/drivers/cxl/cxlmem.h
@@ -41,6 +41,7 @@
  * @cdev: char dev core object for ioctl operations
  * @cxlds: The device state backing this device
  * @detach_work: active memdev lost a port in its ancestry
+ * @unbinding: the endpoint goes away because this memdev is being unbound
  * @cxl_nvb: coordinate removal of @cxl_nvd if present
  * @cxl_nvd: optional bridge to an nvdimm if the device supports pmem
  * @endpoint: connection to the CXL port topology for this memory device
@@ -58,6 +59,7 @@ struct cxl_memdev {
 	struct cdev cdev;
 	struct cxl_dev_state *cxlds;
 	struct work_struct detach_work;
+	bool unbinding;
 	struct cxl_nvdimm_bridge *cxl_nvb;
 	struct cxl_nvdimm *cxl_nvd;
 	struct cxl_port *endpoint;
diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c
index 96fade0299..092b138827 100644
--- a/drivers/cxl/port.c
+++ b/drivers/cxl/port.c
@@ -26,8 +26,20 @@
  * PCIe topology.
  */
 
-static void schedule_detach(void *cxlmd)
+static void schedule_detach(void *data)
 {
+	struct cxl_memdev *cxlmd = data;
+
+	/*
+	 * A detach queued while @cxlmd unbinds itself would run after it
+	 * and could release the next binding instead. With ->attach the
+	 * detach releases the parent, which still has to happen. Read
+	 * without the host lock when the endpoint port is unbound directly.
+	 */
+	if (READ_ONCE(cxlmd->unbinding) && !cxlmd->attach) {
+		put_device(&cxlmd->dev);
+		return;
+	}
 	schedule_cxl_memdev_detach(cxlmd);
 }
 
-- 
2.34.1


^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2026-11-22 12:20 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-22 12:20 [PATCH] cxl/port: Do not queue a memdev detach when the memdev unbinds itself Unsung Lee

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®