* [PATCH for-6.12 1/2] blk-throttle: remove last_low_overflow_time
2024-09-03 13:51 [PATCH for-6.12 0/2] blk-throttle: support prioritized processing of metadata Yu Kuai
@ 2024-09-03 13:51 ` Yu Kuai
2024-09-10 21:02 ` Tejun Heo
2024-09-03 13:51 ` [PATCH for-6.12 2/2] blk-throttle: support prioritized processing of metadata Yu Kuai
2024-09-10 22:39 ` [PATCH for-6.12 0/2] " Jens Axboe
2 siblings, 1 reply; 6+ messages in thread
From: Yu Kuai @ 2024-09-03 13:51 UTC (permalink / raw)
To: tj, josef, axboe
Cc: cgroups, linux-block, linux-kernel, yukuai3, yukuai1, yi.zhang,
yangerkun
From: Yu Kuai <yukuai3@huawei.com>
last_low_overflow_time is not used anymore after commit bf20ab538c81
("blk-throttle: remove CONFIG_BLK_DEV_THROTTLING_LOW").
Signed-off-by: Yu Kuai <yukuai3@huawei.com>
---
block/blk-throttle.c | 8 +-------
block/blk-throttle.h | 2 --
2 files changed, 1 insertion(+), 9 deletions(-)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index dc6140fa3de0..eb859c44c9f3 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -1611,17 +1611,13 @@ bool __blk_throtl_bio(struct bio *bio)
sq = &tg->service_queue;
while (true) {
- if (tg->last_low_overflow_time[rw] == 0)
- tg->last_low_overflow_time[rw] = jiffies;
/* throtl is FIFO - if bios are already queued, should queue */
if (sq->nr_queued[rw])
break;
/* if above limits, break to queue */
- if (!tg_may_dispatch(tg, bio, NULL)) {
- tg->last_low_overflow_time[rw] = jiffies;
+ if (!tg_may_dispatch(tg, bio, NULL))
break;
- }
/* within limits, let's charge and dispatch directly */
throtl_charge_bio(tg, bio);
@@ -1661,8 +1657,6 @@ bool __blk_throtl_bio(struct bio *bio)
tg->io_disp[rw], tg_iops_limit(tg, rw),
sq->nr_queued[READ], sq->nr_queued[WRITE]);
- tg->last_low_overflow_time[rw] = jiffies;
-
td->nr_queued[rw]++;
throtl_add_bio_tg(bio, qn, tg);
throttled = true;
diff --git a/block/blk-throttle.h b/block/blk-throttle.h
index 4d9ef5abdf21..1a36d1278eea 100644
--- a/block/blk-throttle.h
+++ b/block/blk-throttle.h
@@ -106,8 +106,6 @@ struct throtl_grp {
/* Number of bio's dispatched in current slice */
unsigned int io_disp[2];
- unsigned long last_low_overflow_time[2];
-
uint64_t last_bytes_disp[2];
unsigned int last_io_disp[2];
--
2.39.2
^ permalink raw reply [flat|nested] 6+ messages in thread* [PATCH for-6.12 2/2] blk-throttle: support prioritized processing of metadata
2024-09-03 13:51 [PATCH for-6.12 0/2] blk-throttle: support prioritized processing of metadata Yu Kuai
2024-09-03 13:51 ` [PATCH for-6.12 1/2] blk-throttle: remove last_low_overflow_time Yu Kuai
@ 2024-09-03 13:51 ` Yu Kuai
2024-09-10 21:19 ` Tejun Heo
2024-09-10 22:39 ` [PATCH for-6.12 0/2] " Jens Axboe
2 siblings, 1 reply; 6+ messages in thread
From: Yu Kuai @ 2024-09-03 13:51 UTC (permalink / raw)
To: tj, josef, axboe
Cc: cgroups, linux-block, linux-kernel, yukuai3, yukuai1, yi.zhang,
yangerkun
From: Yu Kuai <yukuai3@huawei.com>
Currently, blk-throttle handle all IO fifo, hence if data IO is
throttled and then meta IO is dispatched, the meta IO will have to wait
for the data IO, causing priority inversion problems.
This patch support to handle metadata first and then pay debt while
throttling data.
Test script: use cgroup v1 to throttle root cgroup, then create new
dir and file while write back is throttled
test() {
mkdir /mnt/test/xxx
touch /mnt/test/xxx/1
sync /mnt/test/xxx
sync /mnt/test/xxx
}
mkfs.ext4 -F /dev/nvme0n1 -E lazy_itable_init=0,lazy_journal_init=0
mount /dev/nvme0n1 /mnt/test
echo "259:0 $((1024*1024))" > /sys/fs/cgroup/blkio/blkio.throttle.write_bps_device
dd if=/dev/zero of=/mnt/test/foo1 bs=16M count=1 conv=fdatasync status=none &
sleep 4
time test
echo "259:0 0" > /sys/fs/cgroup/blkio/blkio.throttle.write_bps_device
sleep 1
umount /dev/nvme0n1
Test result: time cost for creating new dir and file
before this patch: 14s
after this patch: 0.1s
Signed-off-by: Yu Kuai <yukuai3@huawei.com>
---
block/blk-throttle.c | 65 +++++++++++++++++++++++++++++---------------
1 file changed, 43 insertions(+), 22 deletions(-)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index eb859c44c9f3..9c5bbd261724 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -1595,6 +1595,22 @@ void blk_throtl_cancel_bios(struct gendisk *disk)
spin_unlock_irq(&q->queue_lock);
}
+static bool tg_within_limit(struct throtl_grp *tg, struct bio *bio, bool rw)
+{
+ /* throtl is FIFO - if bios are already queued, should queue */
+ if (tg->service_queue.nr_queued[rw])
+ return false;
+
+ return tg_may_dispatch(tg, bio, NULL);
+}
+
+static void tg_dispatch_in_debt(struct throtl_grp *tg, struct bio *bio, bool rw)
+{
+ if (!bio_flagged(bio, BIO_BPS_THROTTLED))
+ tg->carryover_bytes[rw] -= throtl_bio_data_size(bio);
+ tg->carryover_ios[rw]--;
+}
+
bool __blk_throtl_bio(struct bio *bio)
{
struct request_queue *q = bdev_get_queue(bio->bi_bdev);
@@ -1611,29 +1627,34 @@ bool __blk_throtl_bio(struct bio *bio)
sq = &tg->service_queue;
while (true) {
- /* throtl is FIFO - if bios are already queued, should queue */
- if (sq->nr_queued[rw])
+ if (tg_within_limit(tg, bio, rw)) {
+ /* within limits, let's charge and dispatch directly */
+ throtl_charge_bio(tg, bio);
+
+ /*
+ * We need to trim slice even when bios are not being
+ * queued otherwise it might happen that a bio is not
+ * queued for a long time and slice keeps on extending
+ * and trim is not called for a long time. Now if limits
+ * are reduced suddenly we take into account all the IO
+ * dispatched so far at new low rate and * newly queued
+ * IO gets a really long dispatch time.
+ *
+ * So keep on trimming slice even if bio is not queued.
+ */
+ throtl_trim_slice(tg, rw);
+ } else if (bio_issue_as_root_blkg(bio)) {
+ /*
+ * IOs which may cause priority inversions are
+ * dispatched directly, even if they're over limit.
+ * Debts are handled by carryover_bytes/ios while
+ * calculating wait time.
+ */
+ tg_dispatch_in_debt(tg, bio, rw);
+ } else {
+ /* if above limits, break to queue */
break;
-
- /* if above limits, break to queue */
- if (!tg_may_dispatch(tg, bio, NULL))
- break;
-
- /* within limits, let's charge and dispatch directly */
- throtl_charge_bio(tg, bio);
-
- /*
- * We need to trim slice even when bios are not being queued
- * otherwise it might happen that a bio is not queued for
- * a long time and slice keeps on extending and trim is not
- * called for a long time. Now if limits are reduced suddenly
- * we take into account all the IO dispatched so far at new
- * low rate and * newly queued IO gets a really long dispatch
- * time.
- *
- * So keep on trimming slice even if bio is not queued.
- */
- throtl_trim_slice(tg, rw);
+ }
/*
* @bio passed through this layer without being throttled.
--
2.39.2
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH for-6.12 0/2] blk-throttle: support prioritized processing of metadata
2024-09-03 13:51 [PATCH for-6.12 0/2] blk-throttle: support prioritized processing of metadata Yu Kuai
2024-09-03 13:51 ` [PATCH for-6.12 1/2] blk-throttle: remove last_low_overflow_time Yu Kuai
2024-09-03 13:51 ` [PATCH for-6.12 2/2] blk-throttle: support prioritized processing of metadata Yu Kuai
@ 2024-09-10 22:39 ` Jens Axboe
2 siblings, 0 replies; 6+ messages in thread
From: Jens Axboe @ 2024-09-10 22:39 UTC (permalink / raw)
To: tj, josef, Yu Kuai
Cc: cgroups, linux-block, linux-kernel, yukuai3, yi.zhang, yangerkun
On Tue, 03 Sep 2024 21:51:47 +0800, Yu Kuai wrote:
> Yu Kuai (2):
> blk-throttle: remove last_low_overflow_time
> blk-throttle: support prioritized processing of metadata
>
> block/blk-throttle.c | 69 +++++++++++++++++++++++++++-----------------
> block/blk-throttle.h | 2 --
> 2 files changed, 42 insertions(+), 29 deletions(-)
>
> [...]
Applied, thanks!
[1/2] blk-throttle: remove last_low_overflow_time
commit: 3bf73e6283ef0bae4e27dad62309e50e3bf7ee88
[2/2] blk-throttle: support prioritized processing of metadata
commit: 29390bb5661d49d10424ad8e915230de1f7074c9
Best regards,
--
Jens Axboe
^ permalink raw reply [flat|nested] 6+ messages in thread