mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
* [PATCH v3] block/blk-iolatency: fix always-zero cur_stat mean on non-SSD
@ 2026-09-04  6:54 Tao Cui
  2026-11-20  2:19 ` Guixin Liu
  0 siblings, 1 reply; 2+ messages in thread
From: Tao Cui @ 2026-09-04  6:54 UTC (permalink / raw)
  To: Jens Axboe, Tang Yizhou
  Cc: tj, josef, linux-block, cgroups, linux-kernel, cuitao, cui.tao

From: Tao Cui <cuitao@kylinos.cn>

iolatency_check_latencies() accumulates the per-window latency stat
into iolat->cur_stat via latency_stat_sum(), which for non-SSD devices
calls blk_rq_stat_sum().  But blk_rq_stat_sum() is contracted on a raw
per-cpu src: it folds src->batch into the new mean.  By the time the
per-window stat reaches cur_stat it is already aggregated, with batch
left at 0.  Each window therefore contributes zero latency to
cur_stat->mean, which on non-SSD devices stays at 0; the
latency_sum_ok(&cur_stat) check that gates scaling up is always true,
so the scale-up hysteresis never engages.  SSD devices use the
percentile path and are unaffected.

Add latency_stat_merge(), which merges two already-aggregated stats by
their reconstructed totals (mean * nr_samples) instead of src->batch,
and use it only for the cur_stat accumulation; the earlier per-cpu sum
keeps latency_stat_sum(), whose src is still raw.

Fixes: d70675121546 ("block: introduce blk-iolatency io controller")
Signed-off-by: Tao Cui <cuitao@kylinos.cn>

---
Changes in v3:
- Rebase onto current linux-next head (no code change).
- Verified with a two-kernel QEMU run (virtio-blk forced rotational,
  cgroup v2 io.latency target set below the achievable latency so the
  scale path runs every window): on the unpatched kernel cur_stat->mean
  stayed 0 across all windows while the per-window mean was 120-250us;
  with this patch cur_stat->mean tracked the per-window mean.

Changes in v2:
- Move the fix from blk-stat.c (blk_rq_stat_sum) into blk-iolatency.c,
  per Tang Yizhou's review: blk_rq_stat_sum() is contracted on a raw
  per-cpu src, so the aggregated-stat merge belongs in the iolatency
  caller, not the shared helper.

v1: https://lore.kernel.org/all/20260720133813.45150-1-cui.tao@linux.dev/
v2: https://lore.kernel.org/all/20260721030312.85655-1-cui.tao@linux.dev/
---
 block/blk-iolatency.c | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

diff --git a/block/blk-iolatency.c b/block/blk-iolatency.c
index 2caa79a008ad..853608a4f187 100644
--- a/block/blk-iolatency.c
+++ b/block/blk-iolatency.c
@@ -216,6 +216,31 @@ static inline void latency_stat_sum(struct iolatency_grp *iolat,
 		blk_rq_stat_sum(&sum->rqs, &stat->rqs);
 }
 
+/*
+ * Merge an already-aggregated @stat into @sum by reconstructed totals
+ * (mean * nr_samples); blk_rq_stat_sum() reads src->batch, which is 0
+ * here. The overflow check also rejects the 0 samples case.
+ */
+static inline void latency_stat_merge(struct iolatency_grp *iolat,
+				      struct latency_stat *sum,
+				      struct latency_stat *stat)
+{
+	if (iolat->ssd) {
+		sum->ps.total += stat->ps.total;
+		sum->ps.missed += stat->ps.missed;
+	} else {
+		if (sum->rqs.nr_samples + stat->rqs.nr_samples <=
+		    sum->rqs.nr_samples)
+			return;
+		sum->rqs.mean = div_u64(sum->rqs.mean * sum->rqs.nr_samples +
+					stat->rqs.mean * stat->rqs.nr_samples,
+					sum->rqs.nr_samples + stat->rqs.nr_samples);
+		sum->rqs.min = min(sum->rqs.min, stat->rqs.min);
+		sum->rqs.max = max(sum->rqs.max, stat->rqs.max);
+		sum->rqs.nr_samples += stat->rqs.nr_samples;
+	}
+}
+
 static inline void latency_stat_record_time(struct iolatency_grp *iolat,
 					    u64 req_time)
 {
@@ -547,7 +572,7 @@ static void iolatency_check_latencies(struct iolatency_grp *iolat, u64 now)
 	/* Somebody beat us to the punch, just bail. */
 	spin_lock_irqsave(&lat_info->lock, flags);
 
-	latency_stat_sum(iolat, &iolat->cur_stat, &stat);
+	latency_stat_merge(iolat, &iolat->cur_stat, &stat);
 	lat_info->nr_samples -= iolat->nr_samples;
 	lat_info->nr_samples += latency_stat_samples(iolat, &iolat->cur_stat);
 	iolat->nr_samples = latency_stat_samples(iolat, &iolat->cur_stat);
-- 
2.43.0

^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: [PATCH v3] block/blk-iolatency: fix always-zero cur_stat mean on non-SSD
  2026-09-04  6:54 [PATCH v3] block/blk-iolatency: fix always-zero cur_stat mean on non-SSD Tao Cui
@ 2026-11-20  2:19 ` Guixin Liu
  0 siblings, 0 replies; 2+ messages in thread
From: Guixin Liu @ 2026-11-20  2:19 UTC (permalink / raw)
  To: Tao Cui, Jens Axboe, Tang Yizhou
  Cc: tj, josef, linux-block, cgroups, linux-kernel, cuitao



在 2026/9/4 14:54, Tao Cui 写道:
> From: Tao Cui <cuitao@kylinos.cn>
>
> iolatency_check_latencies() accumulates the per-window latency stat
> into iolat->cur_stat via latency_stat_sum(), which for non-SSD devices
> calls blk_rq_stat_sum().  But blk_rq_stat_sum() is contracted on a raw
> per-cpu src: it folds src->batch into the new mean.  By the time the
> per-window stat reaches cur_stat it is already aggregated, with batch
> left at 0.  Each window therefore contributes zero latency to
> cur_stat->mean, which on non-SSD devices stays at 0; the
> latency_sum_ok(&cur_stat) check that gates scaling up is always true,
> so the scale-up hysteresis never engages.  SSD devices use the
> percentile path and are unaffected.
>
> Add latency_stat_merge(), which merges two already-aggregated stats by
> their reconstructed totals (mean * nr_samples) instead of src->batch,
> and use it only for the cur_stat accumulation; the earlier per-cpu sum
> keeps latency_stat_sum(), whose src is still raw.
>
> Fixes: d70675121546 ("block: introduce blk-iolatency io controller")
Ti Tao:
  Would it be better to change the fix tag to "451bb7c33197 ("blk-iolatency: keep track of previous windows stats")"?

Best Regards,
Guixin Liu
> Signed-off-by: Tao Cui <cuitao@kylinos.cn>
>
> ---
> Changes in v3:
> - Rebase onto current linux-next head (no code change).
> - Verified with a two-kernel QEMU run (virtio-blk forced rotational,
>   cgroup v2 io.latency target set below the achievable latency so the
>   scale path runs every window): on the unpatched kernel cur_stat->mean
>   stayed 0 across all windows while the per-window mean was 120-250us;
>   with this patch cur_stat->mean tracked the per-window mean.
>
> Changes in v2:
> - Move the fix from blk-stat.c (blk_rq_stat_sum) into blk-iolatency.c,
>   per Tang Yizhou's review: blk_rq_stat_sum() is contracted on a raw
>   per-cpu src, so the aggregated-stat merge belongs in the iolatency
>   caller, not the shared helper.
>
> v1: https://lore.kernel.org/all/20260720133813.45150-1-cui.tao@linux.dev/
> v2: https://lore.kernel.org/all/20260721030312.85655-1-cui.tao@linux.dev/
> ---
>  block/blk-iolatency.c | 27 ++++++++++++++++++++++++++-
>  1 file changed, 26 insertions(+), 1 deletion(-)
>
> diff --git a/block/blk-iolatency.c b/block/blk-iolatency.c
> index 2caa79a008ad..853608a4f187 100644
> --- a/block/blk-iolatency.c
> +++ b/block/blk-iolatency.c
> @@ -216,6 +216,31 @@ static inline void latency_stat_sum(struct iolatency_grp *iolat,
>  		blk_rq_stat_sum(&sum->rqs, &stat->rqs);
>  }
>  
> +/*
> + * Merge an already-aggregated @stat into @sum by reconstructed totals
> + * (mean * nr_samples); blk_rq_stat_sum() reads src->batch, which is 0
> + * here. The overflow check also rejects the 0 samples case.
> + */
> +static inline void latency_stat_merge(struct iolatency_grp *iolat,
> +				      struct latency_stat *sum,
> +				      struct latency_stat *stat)
> +{
> +	if (iolat->ssd) {
> +		sum->ps.total += stat->ps.total;
> +		sum->ps.missed += stat->ps.missed;
> +	} else {
> +		if (sum->rqs.nr_samples + stat->rqs.nr_samples <=
> +		    sum->rqs.nr_samples)
> +			return;
> +		sum->rqs.mean = div_u64(sum->rqs.mean * sum->rqs.nr_samples +
> +					stat->rqs.mean * stat->rqs.nr_samples,
> +					sum->rqs.nr_samples + stat->rqs.nr_samples);
> +		sum->rqs.min = min(sum->rqs.min, stat->rqs.min);
> +		sum->rqs.max = max(sum->rqs.max, stat->rqs.max);
> +		sum->rqs.nr_samples += stat->rqs.nr_samples;
> +	}
> +}
> +
>  static inline void latency_stat_record_time(struct iolatency_grp *iolat,
>  					    u64 req_time)
>  {
> @@ -547,7 +572,7 @@ static void iolatency_check_latencies(struct iolatency_grp *iolat, u64 now)
>  	/* Somebody beat us to the punch, just bail. */
>  	spin_lock_irqsave(&lat_info->lock, flags);
>  
> -	latency_stat_sum(iolat, &iolat->cur_stat, &stat);
> +	latency_stat_merge(iolat, &iolat->cur_stat, &stat);
>  	lat_info->nr_samples -= iolat->nr_samples;
>  	lat_info->nr_samples += latency_stat_samples(iolat, &iolat->cur_stat);
>  	iolat->nr_samples = latency_stat_samples(iolat, &iolat->cur_stat);


^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2026-11-20  2:19 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-09-04  6:54 [PATCH v3] block/blk-iolatency: fix always-zero cur_stat mean on non-SSD Tao Cui
2026-11-20  2:19 ` Guixin Liu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®