From: Joel Granados <joel.granados@kernel.org>
To: Keith Busch <kbusch@kernel.org>, Jens Axboe <axboe@kernel.dk>,
Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me>,
Chaitanya Kulkarni <chaitanyak@nvidia.com>,
Jason Gunthorpe <jgg@ziepe.ca>
Cc: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org,
Joel Granados <joel.granados@kernel.org>
Subject: [PATCH RFC 0/5] nvme: Controller Data Queue (CDQ) support
Date: Fri, 24 Apr 2026 13:37:50 +0200 [thread overview]
Message-ID: <20260424-jag-cdq-lkml-v1-0-d773343a717c@kernel.org> (raw)
This RFC implements Controller Data Queue (CDQ) support in the NVMe
driver, a variation of my original RFC sent last July [2]. It exposes an
ioctl interface for userspace to create, configure, and delete CDQs
backed by DMA-mapped user memory with eventfd notification. In this
version I explore how the CDQ protocol logic might live outside the
kernel; the ioctl serves as a testing tool but is not necessarily the
final interface.
This RFC exists within a broader goal, which is to enable NVMe namespace
migration. The timing feels right as hardware with CDQ capability
exists, NVMe fully specifies the feature and there is growing interest
in Live Migration which by extension includes CDQ.
There is however, no clear consensus on how NVMe Live Migration should
land in the Linux kernel. The 2022 discussion [1] explored a VFIO-based
approach but reached no conclusion, likely because the specification was
not yet mature.
To move CDQ forward, I would like to understand where the LM logic belongs. I
currently see two options (of which I have no particular preference):
1. VFIO: Implement NVMe LM following the VFIO state machine, similar to what
was proposed in 2022.
2. VM manager interface: Bypass VFIO and implement LM logic in the interface
between the VM manager (e.g., QEMU) and the NVMe driver.
One aspect that has not received much attention in previous discussions
is namespace migration as prior work focused on migrating state and not
the actual data. Migrating potential terabytes is IMO a distinct use
case worth considering. LSF/MM/BPF is in a week. I hope this series
encourages folks to revisit their positions, give their opinions and set
the stage for face2face discussions.
Best
PS: I'm including the regular NVMe contacts and the folks that seemed to
have strong opinions in [2]. I always find it difficult to decide who to
include in these so let me know if you want to be removed in the future
or if I have missed someone.
[1] https://lore.kernel.org/20221206055816.292304-1-lei.rao@intel.com
[2] https://lore.kernel.org/20250714-jag-cdq-v1-0-01e027d256d5@kernel.org
Signed-off-by: Joel Granados <joel.granados@kernel.org>
---
Joel Granados (5):
nvme: Add CDQ data structures to nvme spec header
nvme: Add CDQ data structures to host driver
nvme: Add NVME_AER_ONE_SHOT callback handler
nvme: Implement CDQ core functionality
nvme: Add CDQ ioctl interface
drivers/nvme/host/core.c | 312 ++++++++++++++++++++++++++++++++++++++++
drivers/nvme/host/ioctl.c | 53 ++++++-
drivers/nvme/host/nvme.h | 20 +++
include/linux/nvme.h | 50 ++++++-
include/uapi/linux/nvme_ioctl.h | 29 ++++
5 files changed, 462 insertions(+), 2 deletions(-)
---
base-commit: 028ef9c96e96197026887c0f092424679298aae8
change-id: 20260424-jag-cdq-lkml-cd9b7c79983d
Best regards,
--
Joel Granados <joel.granados@kernel.org>
next reply other threads:[~2026-04-24 11:38 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-04-24 11:37 Joel Granados [this message]
2026-04-24 11:37 ` [PATCH RFC 1/5] nvme: Add CDQ data structures to nvme spec header Joel Granados
2026-04-24 11:37 ` [PATCH RFC 2/5] nvme: Add CDQ data structures to host driver Joel Granados
2026-04-24 11:37 ` [PATCH RFC 3/5] nvme: Add NVME_AER_ONE_SHOT callback handler Joel Granados
2026-04-24 11:37 ` [PATCH RFC 4/5] nvme: Implement CDQ core functionality Joel Granados
2026-04-24 11:37 ` [PATCH RFC 5/5] nvme: Add CDQ ioctl interface Joel Granados
2026-04-24 13:06 ` [PATCH RFC 0/5] nvme: Controller Data Queue (CDQ) support Jason Gunthorpe
2026-04-24 13:24 ` Christoph Hellwig
2026-04-27 18:24 ` Joel Granados
2026-04-27 18:59 ` Joel Granados
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260424-jag-cdq-lkml-v1-0-d773343a717c@kernel.org \
--to=joel.granados@kernel.org \
--cc=axboe@kernel.dk \
--cc=chaitanyak@nvidia.com \
--cc=hch@lst.de \
--cc=jgg@ziepe.ca \
--cc=kbusch@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®