* [RFC PATCH 1/1] block: charge passthrough requests to the submitter's cgroup
2026-09-21 7:06 [RFC PATCH 0/1] block: charge passthrough requests to the submitter's cgroup Tao Cui
@ 2026-09-21 7:06 ` Tao Cui
2026-09-21 16:51 ` Tejun Heo
2026-09-21 7:27 ` [RFC PATCH 0/1] " Tao Cui
1 sibling, 1 reply; 6+ messages in thread
From: Tao Cui @ 2026-09-21 7:06 UTC (permalink / raw)
To: axboe, kbusch, tj, hch, sagi, yukuai
Cc: linux-block, linux-nvme, cgroups, linux-kernel, cui.tao, cuitao
From: Tao Cui <cuitao@kylinos.cn>
Passthrough requests (SG_IO, bsg, nvme passthrough ioctls and uring
commands) are dispatched via blk_execute_rq{,_nowait}() without ever
passing through submit_bio(), so the bio mapped by blk_rq_map_user()
carries no blkcg association: the transferred bytes never show up in
cgroup io.stat, and every rq_qos policy on the queue (iocost,
iolatency, wbt) is bypassed, as is blk-throttle, which hooks
submit_bio_noacct() directly rather than going through rq_qos.
A quick demonstration on a scsi_debug device with iocost enabled and
vrate pinned to its 1% floor: a direct fio writer was throttled ~10x
while the same cgroup issuing sg_dd writes ran at full device speed
with zero io.stat accounting.
Associate the mapped bio with the submitter's blkcg at dispatch time
and run the regular bio accounting (blk_cgroup_bio_start()) and
rq_qos throttle paths with it. DRV_IN/DRV_OUT commands are mapped to
READ/WRITE so io.stat classifies their bytes normally; request
completion already pairs with the throttle through bio_endio() ->
rq_qos_done_bio().
The charge is gated by opcode (READ/WRITE/DRV_IN/DRV_OUT) and to
queues that already have a gendisk: commands issued during device
probing (SCSI INQUIRY etc.) have no gendisk yet and stay exempt,
following the same probe-exemption reasoning as the passthrough
iostats support.
RFC notes:
- validated on linux-next with scsi_debug + sg_dd (SG_IO) and qemu
emulated nvme + a NVME_IOCTL_IO64_CMD loop: in both cases the
transferred bytes are fully accounted (wbytes/wios) in the issuing
cgroup, and the issuing task is observed waiting on the iocost
waitqueue with vrate clamped;
- iolatency gains the same coverage for free, and so does wbt:
passthrough writes now pass through wbt_wait() like bio-path
writes, a behavior change worth calling out even though wbt
targets buffered writeback and direct passthrough rarely hits it;
blk-throttle does not: it hooks
submit_bio_noacct() directly and is not an rq_qos policy, so
io.max stays unenforced for passthrough (measured). Its
queue-and-resubmit throttling model would also need a synchronous
variant for request-bound bios;
- nvme uring commands share the same blk_execute_rq_nowait()
dispatch as the validated ioctl path;
- the charge runs in the submitter's context and may sleep in
rq_qos throttling; all data-op callers found run in sleepable
context, but this deserves reviewer attention.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
block/blk-mq.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 72 insertions(+)
diff --git a/block/blk-mq.c b/block/blk-mq.c
index a26a11c73ee3..2dd59f6df321 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -10,6 +10,7 @@
#include <linux/backing-dev.h>
#include <linux/bio.h>
#include <linux/blkdev.h>
+#include "blk-cgroup.h"
#include <linux/blk-integrity.h>
#include <linux/kmemleak.h>
#include <linux/mm.h>
@@ -1400,6 +1401,73 @@ static void blk_add_rq_to_plug(struct blk_plug *plug, struct request *rq)
plug->rq_count++;
}
+/*
+ * Passthrough bios are mapped directly onto requests via
+ * blk_rq_map_user() and never pass through submit_bio(), so they carry
+ * no blkcg association and are invisible to cgroup io.stat and to every
+ * rq_qos policy (iocost, blk-throttle, iolatency). Charge the ones that
+ * carry data to the submitter's blkcg at dispatch time and run the
+ * regular bio accounting and rq_qos throttle paths with the associated
+ * bio.
+ *
+ * Gated by opcode (READ/WRITE/DRV_IN/DRV_OUT, the latter two mapped
+ * to READ/WRITE for io.stat classification) and to queues that
+ * already have a gendisk: commands issued during device probing (SCSI
+ * INQUIRY and friends) have no gendisk yet and stay exempt. The
+ * request bios may carry a stale ->bi_blkg from the mempool; the
+ * association helper drops the old reference and re-associates.
+ */
+static void blk_mq_pt_charge(struct request *rq)
+{
+ struct bio *bio = rq->bio;
+ enum req_op op = req_op(rq);
+
+ if (!bio || !rq->q->disk)
+ return;
+
+ switch (op) {
+ case REQ_OP_READ:
+ case REQ_OP_WRITE:
+ case REQ_OP_DRV_IN:
+ case REQ_OP_DRV_OUT:
+ break;
+ default:
+ return;
+ }
+ if (op == REQ_OP_DRV_IN)
+ op = REQ_OP_READ;
+ else if (op == REQ_OP_DRV_OUT)
+ op = REQ_OP_WRITE;
+
+ if (!bio->bi_bdev)
+ bio->bi_bdev = rq->q->disk->part0;
+ bio->bi_opf &= ~REQ_OP_MASK;
+ bio->bi_opf |= op;
+#ifdef CONFIG_BLK_CGROUP
+ /*
+ * Issued from kthreads the css is root and the charge is a
+ * no-op through the root exemptions; data-op issuers that
+ * matter run in the submitter's task context.
+ */
+ {
+ struct cgroup_subsys_state *css;
+
+ rcu_read_lock();
+ css = task_css(current, io_cgrp_id);
+ bio_associate_blkg_from_css(bio, css);
+ rcu_read_unlock();
+ }
+#endif
+ blk_cgroup_bio_start(bio);
+
+ /*
+ * The rq_qos throttle path may sleep on the waitqueues like any
+ * bio submitter; all callers found (ioctl / uring_cmd submit,
+ * target and error handling kthreads) run in sleepable context.
+ */
+ rq_qos_throttle(rq->q, bio);
+}
+
/**
* blk_execute_rq_nowait - insert a request to I/O scheduler for execution
* @rq: request to insert
@@ -1412,6 +1480,7 @@ static void blk_add_rq_to_plug(struct blk_plug *plug, struct request *rq)
* Note:
* This function will invoke @done directly if the queue is dead.
*/
+
void blk_execute_rq_nowait(struct request *rq, bool at_head)
{
struct blk_mq_hw_ctx *hctx = rq->mq_hctx;
@@ -1419,6 +1488,7 @@ void blk_execute_rq_nowait(struct request *rq, bool at_head)
WARN_ON(irqs_disabled());
WARN_ON(!blk_rq_is_passthrough(rq));
+ blk_mq_pt_charge(rq);
blk_account_io_start(rq);
if (current->plug && !at_head) {
@@ -1484,6 +1554,8 @@ blk_status_t blk_execute_rq(struct request *rq, bool at_head)
WARN_ON(irqs_disabled());
WARN_ON(!blk_rq_is_passthrough(rq));
+ blk_mq_pt_charge(rq);
+
rq->end_io_data = &wait;
rq->end_io = blk_end_sync_rq;
--
2.43.0
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [RFC PATCH 0/1] block: charge passthrough requests to the submitter's cgroup
2026-09-21 7:06 [RFC PATCH 0/1] block: charge passthrough requests to the submitter's cgroup Tao Cui
2026-09-21 7:06 ` [RFC PATCH 1/1] " Tao Cui
@ 2026-09-21 7:27 ` Tao Cui
1 sibling, 0 replies; 6+ messages in thread
From: Tao Cui @ 2026-09-21 7:27 UTC (permalink / raw)
To: axboe, kbusch, tj, hch, sagi, yukuai
Cc: cui.tao, linux-block, linux-nvme, cgroups, linux-kernel, cuitao
Hi,
在 2026/9/21 15:06, Tao Cui 写道:
> From: Tao Cui <cuitao@kylinos.cn>
>
> While looking at cgroup IO control coverage, I stumbled into a hole
> that is probably familiar to people who have worked on passthrough:
> SG_IO, bsg, and the NVMe ioctl / uring command paths build requests
> directly and dispatch them through blk_execute_rq{,_nowait}(),
> never passing through submit_bio(). The mapped bio carries no blkcg
> association, so the bytes never show up in cgroup io.stat, and none
> of the rq_qos policies (iocost, iolatency, wbt) see these commands;
> blk-throttle, hooking submit_bio_noacct() directly, is equally
> blind. On a scsi_debug device with iocost enabled and vrate pinned
> to its 1% floor, a direct writer in a cgroup is throttled ~10x while
> the same cgroup issuing sg_dd writes runs at full device speed with
> io.stat staying at zero.
>
> This RFC is a prototype asking for feedback on one possible
> direction, not a proposal I am attached to: is charging passthrough
> at dispatch time the right layer and the right semantics at all?
>
> The patch associates the mapped bio with the submitter's blkcg in
> blk_execute_rq{,_nowait}(), runs the regular bio accounting
> (blk_cgroup_bio_start()) and the rq_qos throttle path with it, and
> gates by opcode (READ/WRITE/DRV_IN/DRV_OUT, mapping the latter two
> to READ/WRITE for io.stat classification) and to queues that already
> have a gendisk so probing stays exempt. On linux-next, with
> scsi_debug + sg_dd and qemu emulated nvme + an NVME_IOCTL_IO64_CMD
> loop, the transferred bytes are fully accounted in the issuing
> cgroup, and with vrate clamped the pacing bites: the same 128MB
> SG_IO run takes 0.025s unthrottled and 31.7s at the 1% floor, with
> the issuing task sleeping on the iocost waitqueue like any bio
> submitter.
>
> Questions I am unsure about:
>
> 1. Is always-on the right default, or should this hide behind the
> queue/iostats_passthrough opt-in from the passthrough iostats
> series, keeping default behavior unchanged?
> 2. The throttle may sleep; all callers I found (ioctl / uring_cmd
> submit, target and error handling kthreads) are sleepable, and
> kthread-issued charges land on root and turn into no-ops - but
> I may have missed paths, and I'd rather ask than assume.
> 3. What should happen on nvme multipath failover, where the bio is
> re-routed to another queue but carries the original queue's
> blkg? Untested; guidance welcome.
> 4. wbt also gains passthrough throttling through the same rq_qos
> walk - probably benign since wbt targets buffered writeback,
> but it is a behavior change I did not set out to make.
> 5. blk-throttle does not benefit (it is not an rq_qos policy, and
> its queue-and-resubmit model does not fit bios already bound to
> a dispatched request); would a synchronous wait variant there
> be welcome as follow-up work, or is leaving it alone preferred?
>
> Reproduction notes in case anyone tries: sg_dd coalesces into very
> large single commands by default (use bpt=1), and a short burst from
> a single cgroup rides the initial budget, so sustained transfers
> are what show the pacing.
>
The cover letter summarized the motivation briefly, so let me add some
background on how this came up and what the prototype is trying to
address.
How this was found
------------------
This came from stress testing blk-iocost on linux-next with mixed IO
workloads and cgroup churn. With the known bio-path accounting gaps
(flush, zone append) already carrying posted fixes, the remaining
missing cases had one property in common: they build requests directly
and enter through blk_execute_rq(), bypassing submit_bio() entirely.
The invariant being violated is simple to state: any IO that consumes
device bandwidth but bypasses submit_bio() currently bypasses blkcg IO
accounting. Passthrough commands, including SG_IO, bsg, nvme ioctls and
uring commands, currently fall into this category: they have no blkcg
association, do not show up in io.stat, and do not participate in
blkcg-aware rq_qos policies such as iocost and iolatency.
The observable difference
-------------------------
On the same scsi_debug device, with the same cgroup and iocost enabled
with vrate pinned at its 1% floor:
bio writer (fio) throttled ~10x
SG_IO writer (sg_dd) full device speed,
io.stat remains zero
The NVMe ioctl path shows the same behavior: passthrough provides a
path that bypasses the existing accounting boundary.
With the prototype patch
------------------------
The mapped bio is associated with the submitting blkcg in
blk_execute_rq{,_nowait}(), and goes through the normal bio accounting
and rq_qos paths.
With the same setup:
- a 128MB SG_IO write (bs=1M, bpt=1) is accounted to the issuing
cgroup's io.stat, with wbytes/wios matching the transfer;
- the same workload takes 0.025s without throttling and 31.7s with
vrate at the 1% floor, with the task sleeping in the iocost throttle
path.
Two reproduction notes: sg_dd combines transfers into large commands by
default, so bpt=1 is needed for scsi_debug. Also, short bursts can be
covered by the initial budget; sustained transfers make the difference
visible.
Relation to passthrough iostats
-------------------------------
This RFC builds on Keith's passthrough iostats work. That series made
disk-level passthrough accounting possible and provided block-device
association for passthrough bios where it is available.
This RFC is about the next layer: whether passthrough IO should also
participate in blkcg accounting and enforcement.
The questions from the cover letter still stand. In particular, I am
not sure whether attaching blkcg/accounting at this layer is the right
abstraction, or whether this should remain an opt-in behavior like
passthrough iostats.
Thanks,
Tao
>
> Tao Cui (1):
> block: charge passthrough requests to the submitter's cgroup
>
> block/blk-mq.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 72 insertions(+)
>
^ permalink raw reply [flat|nested] 6+ messages in thread