[PATCH] bfq: Transfer allocated request counts when moving queues

[PATCH] bfq: Transfer allocated request counts when moving queues

[sunyangkai]

bfq_entity::allocated counts the requests allocated in an entity's
subtree. Request allocation increments the queue entity and its
ancestors, while request release decrements the queue entity and its
current ancestors.

When bfq_bfqq_move() moves a queue with outstanding requests to another
group, it changes entity->parent without transferring these charges.
The old ancestors retain the charges, while subsequent request releases
decrement the new ancestors even though they were never charged. This
can leave the destination group's allocated count negative and the
source group's count inflated after all requests have completed,
distorting the limits enforced by bfqq_request_over_limit().

Subtract the queue's outstanding request count from its old ancestors
before dropping the old group reference, and add it to the new ancestors
after reparenting. Leave the queue's own count unchanged. Shared
ancestors have no net change, and bfqd->lock serializes the transfer
with request accounting.

Fixes: 98f044999ba1 ("bfq: Track number of allocated requests in bfq_entity")
Cc: stable@vger.kernel.org
Signed-off-by: sunyangkai <sunyangkai@fnnas.com>
---
Validation used a QEMU guest running a 6.18.18-based kernel, with BFQ
on a virtio block device. The workload submitted direct asynchronous
reads in /a/child, moved the task to /b, submitted more reads, and waited
for all completions before repeating. A read-only kretprobe observed
bfq_bfqq_move() entry and return counters.

Without the fix, the drained destination group reached allocated=-1
and then -2. With the fix, outstanding charges were transferred and
the drained group counts were zero. Both runs covered migrations with
outstanding requests, completed all reads successfully, and reported
zero missed probes. This was not a runtime test of upstream master.

 block/bfq-cgroup.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c
index ef9d63e8542e..e3a98244b9f6 100644
--- a/block/bfq-cgroup.c
+++ b/block/bfq-cgroup.c
@@ -645,6 +645,7 @@ void bfq_bfqq_move(struct bfq_data *bfqd, struct bfq_queue *bfqq,
 {
 	struct bfq_entity *entity = &bfqq->entity;
 	struct bfq_group *old_parent = bfqq_group(bfqq);
+	struct bfq_entity *parent;
 	bool has_pending_reqs = false;
 
 	/*
@@ -685,11 +686,16 @@ void bfq_bfqq_move(struct bfq_data *bfqd, struct bfq_queue *bfqq,
 		bfq_deactivate_bfqq(bfqd, bfqq, false, false);
 	else if (entity->on_st_or_in_serv)
 		bfq_put_idle_entity(bfq_entity_service_tree(entity), entity);
+	/* Transfer outstanding request charges before dropping the old group. */
+	for (parent = entity->parent; parent; parent = parent->parent)
+		parent->allocated -= entity->allocated;
 	bfqg_and_blkg_put(old_parent);
 
 	bfq_reassign_last_bfqq(bfqq, NULL);
 	entity->parent = bfqg->my_entity;
 	entity->sched_data = &bfqg->sched_data;
+	for (parent = entity->parent; parent; parent = parent->parent)
+		parent->allocated += entity->allocated;
 	/* pin down bfqg and its associated blkg  */
 	bfqg_and_blkg_get(bfqg);
 
-- 
2.53.0