[RFC PATCH 0/1] block: charge passthrough requests to the submitter's cgroup

[Tao Cui <cui.tao@linux.dev>]

From: Tao Cui <cuitao@kylinos.cn>

While looking at cgroup IO control coverage, I stumbled into a hole
that is probably familiar to people who have worked on passthrough:
SG_IO, bsg, and the NVMe ioctl / uring command paths build requests
directly and dispatch them through blk_execute_rq{,_nowait}(),
never passing through submit_bio().  The mapped bio carries no blkcg
association, so the bytes never show up in cgroup io.stat, and none
of the rq_qos policies (iocost, iolatency, wbt) see these commands;
blk-throttle, hooking submit_bio_noacct() directly, is equally
blind.  On a scsi_debug device with iocost enabled and vrate pinned
to its 1% floor, a direct writer in a cgroup is throttled ~10x while
the same cgroup issuing sg_dd writes runs at full device speed with
io.stat staying at zero.

This RFC is a prototype asking for feedback on one possible
direction, not a proposal I am attached to: is charging passthrough
at dispatch time the right layer and the right semantics at all?

The patch associates the mapped bio with the submitter's blkcg in
blk_execute_rq{,_nowait}(), runs the regular bio accounting
(blk_cgroup_bio_start()) and the rq_qos throttle path with it, and
gates by opcode (READ/WRITE/DRV_IN/DRV_OUT, mapping the latter two
to READ/WRITE for io.stat classification) and to queues that already
have a gendisk so probing stays exempt.  On linux-next, with
scsi_debug + sg_dd and qemu emulated nvme + an NVME_IOCTL_IO64_CMD
loop, the transferred bytes are fully accounted in the issuing
cgroup, and with vrate clamped the pacing bites: the same 128MB
SG_IO run takes 0.025s unthrottled and 31.7s at the 1% floor, with
the issuing task sleeping on the iocost waitqueue like any bio
submitter.

Questions I am unsure about:

 1. Is always-on the right default, or should this hide behind the
    queue/iostats_passthrough opt-in from the passthrough iostats
    series, keeping default behavior unchanged?
 2. The throttle may sleep; all callers I found (ioctl / uring_cmd
    submit, target and error handling kthreads) are sleepable, and
    kthread-issued charges land on root and turn into no-ops - but
    I may have missed paths, and I'd rather ask than assume.
 3. What should happen on nvme multipath failover, where the bio is
    re-routed to another queue but carries the original queue's
    blkg?  Untested; guidance welcome.
 4. wbt also gains passthrough throttling through the same rq_qos
    walk - probably benign since wbt targets buffered writeback,
    but it is a behavior change I did not set out to make.
 5. blk-throttle does not benefit (it is not an rq_qos policy, and
    its queue-and-resubmit model does not fit bios already bound to
    a dispatched request); would a synchronous wait variant there
    be welcome as follow-up work, or is leaving it alone preferred?

Reproduction notes in case anyone tries: sg_dd coalesces into very
large single commands by default (use bpt=1), and a short burst from
a single cgroup rides the initial budget, so sustained transfers
are what show the pacing.


Tao Cui (1):
  block: charge passthrough requests to the submitter's cgroup

 block/blk-mq.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 72 insertions(+)

-- 
2.43.0