From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D68DA3BE630 for ; Fri, 24 Apr 2026 11:38:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777030701; cv=none; b=D6IXgNtJTeOxOyLlabIcyAZ7J0Dba12C2bikgsrZ/qP0RiysFcgZYxjBAtwggoM/1t6eaPHHERddFs7fZM3P95nBqeZXI1FBW4XNUvwYQzmj392Lt/J7sC6i7UxKLXVUCTGlqiGTx/6ew2VOkl7hC6pF4O+UQNnJKUPp1e4U4fo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777030701; c=relaxed/simple; bh=5BZxNLbiDIZ0F/6EQCU38sByageab1eGWbVMCZM9NBo=; h=From:Subject:Date:Message-Id:MIME-Version:Content-Type:To:Cc; b=b6eZe8rBvhWbhRuvVZ9c6BSU7qeXEEmjIBOxAaM1YsZhLO4DX3pk1CdOHsIm3hPla0UpwxQLaaWVvGDxNrqgfF5D3xwx2RHEbghtrukZ6YUIItErte8a63i9I///gdEkA2p6QssTNvSXRnS3v6NWiuuKtRw0DprZVF+fqESsg+0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=As3JEGpH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="As3JEGpH" Received: by smtp.kernel.org (Postfix) with ESMTPS id 89512C19425; Fri, 24 Apr 2026 11:38:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1777030701; bh=5BZxNLbiDIZ0F/6EQCU38sByageab1eGWbVMCZM9NBo=; h=From:Subject:Date:To:Cc:From; b=As3JEGpH0bYUSzwevabEljZDvEWUB73XEtok+kU6i6ssP6QyifvJFz7KWaW/9Illl Q5zKebzD4g6EdVFbVVJ4bqzfhp8X4KqQlKDt1E02irRQ+ay7D3h8+q/M8uDr3Bg9yT k7g7ajf8s7dPdbyVfIW6Ww2Cc2ZXo6XYMCbi4NvBTTOWR64Tzi03Pz2NFrJTqmxndp oNzKXwLbkpTF65Us3ouhuLaiOPwSJOCTt1IInO9TAzvdEM3Lweyrnbn2Kv7EyQCc34 Q3dK6F1+MFYPRP8V2VacXFPSAY87pChfyKsF3DRk9XoklRpWYFOKIXd0cGOL8evw5d 5JsX0/s2R/JPg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E3A8FE51FE; Fri, 24 Apr 2026 11:38:21 +0000 (UTC) From: Joel Granados Subject: [PATCH RFC 0/5] nvme: Controller Data Queue (CDQ) support Date: Fri, 24 Apr 2026 13:37:50 +0200 Message-Id: <20260424-jag-cdq-lkml-v1-0-d773343a717c@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-B4-Tracking: v=1; b=H4sIAA5W62kC/6tWKk4tykwtVrJSqFYqSi3LLM7MzwNyDHUUlJIzE vPSU3UzU4B8JSMDIzMDEyMT3azEdN3klELdnOzcHCDDMsk82dzS0sI4RQmopaAoNS2zAmxctFK Qm7NSbG0tAPqlo51jAAAA X-Change-ID: 20260424-jag-cdq-lkml-cd9b7c79983d To: Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Jason Gunthorpe Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Joel Granados X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=2969; i=joel.granados@kernel.org; h=from:subject:message-id; bh=5BZxNLbiDIZ0F/6EQCU38sByageab1eGWbVMCZM9NBo=; b=owJ4nAHtARL+kA0DAAoBupfNUreWQU8ByyZiAGnrVij8iqILmh2mISbfp+H+chXX/y9dxWhft vhOvJ/zok/t1YkBswQAAQoAHRYhBK5HCVcl5jElzssnkLqXzVK3lkFPBQJp61YoAAoJELqXzVK3 lkFPfUAL/RHLj4INt7ay2zjkNviCcQAUDSVHKvKv9+DHD4lxKDjhAaOz0KPnKmbLGBBU613vkHW P3V5ZEQF77EeXy4tgJwuZmshh7eX9OG1/EeLBo43QAkrBQ5PhV3q9RNbFO1Uds3O+DYhu+RTj7s g+UuQVlpxU0JKpf9iRPSGZq3oWR8LRk0EsXrqY0lTZ7+wizod4DHHwiBWfmqTt5LwOKq8VN5pKo H7rJhRtSlIQ22/8j62wC1ggLmsjHImP+9qYtoJPNtQRKoSGGxxzwnlzvJZ2kHB/Zov+Xy59CR/A t+idOBvGWR/7qYheGvVBjN2JwxYralYkoboppYOtsxQq8ki3jAMByaFrUJnPK69ZCBHOYutt8qh VOljR0xW3jJJW06jeBG8L9eFOQrhc8gm41xc3fmni6NU3LVcFY2zApTr1LZ05sryFevMIzAiq8A 8Evq5JDbh5GgHE11EpHcaT/jeqlwT3OMzcvzx85expGulXuWEsIOTH9UyCAYq4dMlqQNo12ijM7 Fk= X-Developer-Key: i=joel.granados@kernel.org; a=openpgp; fpr=F1F8E46D30F0F6C4A45FF4465895FAAC338C6E77 X-Endpoint-Received: by B4 Relay for joel.granados@kernel.org/default with auth_id=239 This RFC implements Controller Data Queue (CDQ) support in the NVMe driver, a variation of my original RFC sent last July [2]. It exposes an ioctl interface for userspace to create, configure, and delete CDQs backed by DMA-mapped user memory with eventfd notification. In this version I explore how the CDQ protocol logic might live outside the kernel; the ioctl serves as a testing tool but is not necessarily the final interface. This RFC exists within a broader goal, which is to enable NVMe namespace migration. The timing feels right as hardware with CDQ capability exists, NVMe fully specifies the feature and there is growing interest in Live Migration which by extension includes CDQ. There is however, no clear consensus on how NVMe Live Migration should land in the Linux kernel. The 2022 discussion [1] explored a VFIO-based approach but reached no conclusion, likely because the specification was not yet mature. To move CDQ forward, I would like to understand where the LM logic belongs. I currently see two options (of which I have no particular preference): 1. VFIO: Implement NVMe LM following the VFIO state machine, similar to what was proposed in 2022. 2. VM manager interface: Bypass VFIO and implement LM logic in the interface between the VM manager (e.g., QEMU) and the NVMe driver. One aspect that has not received much attention in previous discussions is namespace migration as prior work focused on migrating state and not the actual data. Migrating potential terabytes is IMO a distinct use case worth considering. LSF/MM/BPF is in a week. I hope this series encourages folks to revisit their positions, give their opinions and set the stage for face2face discussions. Best PS: I'm including the regular NVMe contacts and the folks that seemed to have strong opinions in [2]. I always find it difficult to decide who to include in these so let me know if you want to be removed in the future or if I have missed someone. [1] https://lore.kernel.org/20221206055816.292304-1-lei.rao@intel.com [2] https://lore.kernel.org/20250714-jag-cdq-v1-0-01e027d256d5@kernel.org Signed-off-by: Joel Granados --- Joel Granados (5): nvme: Add CDQ data structures to nvme spec header nvme: Add CDQ data structures to host driver nvme: Add NVME_AER_ONE_SHOT callback handler nvme: Implement CDQ core functionality nvme: Add CDQ ioctl interface drivers/nvme/host/core.c | 312 ++++++++++++++++++++++++++++++++++++++++ drivers/nvme/host/ioctl.c | 53 ++++++- drivers/nvme/host/nvme.h | 20 +++ include/linux/nvme.h | 50 ++++++- include/uapi/linux/nvme_ioctl.h | 29 ++++ 5 files changed, 462 insertions(+), 2 deletions(-) --- base-commit: 028ef9c96e96197026887c0f092424679298aae8 change-id: 20260424-jag-cdq-lkml-cd9b7c79983d Best regards, -- Joel Granados