* [RFC PATCH 1/8] blk-iocost: add iocost_ioc_tick tracepoint for per-period device summary
2026-09-08 10:01 [RFC PATCH 0/8] blk-iocost: BPF struct_ops cost model Tao Cui
@ 2026-09-08 10:01 ` Tao Cui
2026-09-08 10:01 ` [RFC PATCH 2/8] blk-iocost: define iocost_model_ops cost model interface Tao Cui
` (7 subsequent siblings)
8 siblings, 0 replies; 16+ messages in thread
From: Tao Cui @ 2026-09-08 10:01 UTC (permalink / raw)
To: tj, josef, axboe
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, ast, daniel,
linux-kselftest, cui.tao, Tao Cui
From: Tao Cui <cuitao@kylinos.cn>
The existing iocost tracepoints are state-change driven: vrate_adj
fires only when the adjustment logic runs, inuse_* only on surplus
state transitions, activate/idle only on cgroup state changes. In a
steady state none of them fire. The only other way to observe the
controller (period length, vrate, busy level, active cgroup count,
device utilization) is iocost_monitor.py, which reads kernel memory
through drgn and is not usable in most production environments.
Add iocost_ioc_tick, emitted once per period from the tail of
ioc_timer_fn() with the overall controller state: period_us, vrate,
busy_level, active iocg count, usage percentage and running state.
It fires every period the controller runs, including steady states,
plus one final tick before the controller goes idle, which makes
dormancy (e.g. a device saturated entirely by uncharged IO) directly
visible.
At the default period this is a couple of events per second per
device; the cost is zero while the static key is off.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
block/blk-iocost.c | 5 +++++
include/trace/events/iocost.h | 40 +++++++++++++++++++++++++++++++++++
2 files changed, 45 insertions(+)
diff --git a/block/blk-iocost.c b/block/blk-iocost.c
index 2745bffcd5eef..c24daa9d72ee9 100644
--- a/block/blk-iocost.c
+++ b/block/blk-iocost.c
@@ -2238,6 +2238,7 @@ static void ioc_timer_fn(struct timer_list *timer)
struct ioc_now now;
LIST_HEAD(surpluses);
int nr_debtors, nr_shortages = 0, nr_lagging = 0;
+ int nr_active = 0;
u64 usage_us_sum = 0;
u32 ppm_rthr;
u32 ppm_wthr;
@@ -2274,6 +2275,8 @@ static void ioc_timer_fn(struct timer_list *timer)
u64 vdone, vtime, usage_us;
u32 hw_active, hw_inuse;
+ nr_active++;
+
/*
* Collect unused and wind vtime closer to vnow to prevent
* iocgs from accumulating a large amount of budget.
@@ -2460,6 +2463,8 @@ static void ioc_timer_fn(struct timer_list *timer)
ioc_refresh_vrate(ioc, &now);
}
+ trace_iocost_ioc_tick(ioc, nr_active, usage_us_sum);
+
spin_unlock_irq(&ioc->lock);
}
diff --git a/include/trace/events/iocost.h b/include/trace/events/iocost.h
index e772b1bc60d60..2b9ff348a4f51 100644
--- a/include/trace/events/iocost.h
+++ b/include/trace/events/iocost.h
@@ -178,6 +178,46 @@ TRACE_EVENT(iocost_ioc_vrate_adj,
)
);
+/*
+ * Periodic per-device summary, emitted once per period from the tail of
+ * ioc_timer_fn(). Unlike the state-change events above, this fires every
+ * period the controller is running, including steady states, and carries
+ * the overall controller state so basic monitoring doesn't require drgn.
+ */
+TRACE_EVENT(iocost_ioc_tick,
+
+ TP_PROTO(struct ioc *ioc, int nr_active, u64 usage_us_sum),
+
+ TP_ARGS(ioc, nr_active, usage_us_sum),
+
+ TP_STRUCT__entry (
+ __string(devname, ioc_name(ioc))
+ __field(u32, period_us)
+ __field(u64, vrate)
+ __field(int, busy_level)
+ __field(int, nr_active)
+ __field(u32, usage_pct)
+ __field(int, running)
+ ),
+
+ TP_fast_assign(
+ __assign_str(devname);
+ __entry->period_us = ioc->period_us;
+ __entry->vrate = ioc->vtime_base_rate;
+ __entry->busy_level = ioc->busy_level;
+ __entry->nr_active = nr_active;
+ __entry->usage_pct = ioc->period_us ?
+ div_u64(usage_us_sum * 100, ioc->period_us) : 0;
+ __entry->running = ioc->running;
+ ),
+
+ TP_printk("[%s] period=%uus vrate=%llu busy=%d active=%d usage=%u%% running=%d",
+ __get_str(devname), __entry->period_us, __entry->vrate,
+ __entry->busy_level, __entry->nr_active, __entry->usage_pct,
+ __entry->running
+ )
+);
+
TRACE_EVENT(iocost_iocg_forgive_debt,
TP_PROTO(struct ioc_gq *iocg, const char *path, struct ioc_now *now,
--
2.43.0
^ permalink raw reply [flat|nested] 16+ messages in thread* [RFC PATCH 2/8] blk-iocost: define iocost_model_ops cost model interface
2026-09-08 10:01 [RFC PATCH 0/8] blk-iocost: BPF struct_ops cost model Tao Cui
2026-09-08 10:01 ` [RFC PATCH 1/8] blk-iocost: add iocost_ioc_tick tracepoint for per-period device summary Tao Cui
@ 2026-09-08 10:01 ` Tao Cui
2026-09-08 20:31 ` Tejun Heo
2026-09-08 10:01 ` [RFC PATCH 3/8] blk-iocost: implement BPF struct_ops registration Tao Cui
` (6 subsequent siblings)
8 siblings, 1 reply; 16+ messages in thread
From: Tao Cui @ 2026-09-08 10:01 UTC (permalink / raw)
To: tj, josef, axboe
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, ast, daniel,
linux-kselftest, cui.tao, Tao Cui
From: Tao Cui <cuitao@kylinos.cn>
Define the interface for pluggable cost models: a struct_ops with a
single calc_cost(op, nbytes, sector, cursor, iocg_id, flags) callback
taking scalar arguments only. The return value is vtime (2^37 per
second of device time), clamped to 1s per IO by the kernel. A return
value of 0 delegates the IO back to the builtin formula, so a model
which only handles some IO types cannot make the rest free.
iocg_id is the id of the issuing cgroup, so a model can keep
per-cgroup state, e.g. for multi-stream sequentiality detection.
The id is only valid while the cgroup exists and is recycled after
removal, so models must treat it as a transient key; struct_ops
callback signatures are frozen once merged, so the identifier is
part of the initial interface.
The model is called from the IO submission path under RCU and must
not sleep. Only the bio-level charging path consults the model; the
request-level sizing path keeps using the builtin formula.
This is a definition-only patch; the registration infrastructure and
dispatch hook follow in subsequent patches.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
block/Kconfig | 9 ++++++
include/linux/blk-iocost.h | 56 ++++++++++++++++++++++++++++++++++++++
2 files changed, 65 insertions(+)
create mode 100644 include/linux/blk-iocost.h
diff --git a/block/Kconfig b/block/Kconfig
index 70e4a66d941ff..bf43be21e00bb 100644
--- a/block/Kconfig
+++ b/block/Kconfig
@@ -231,4 +231,13 @@ config BLK_ERROR_INJECTION
source "block/Kconfig.iosched"
+config BLK_CGROUP_IOCOST_BPF
+ bool "Enable BPF pluggable cost model support for the cost IO controller"
+ depends on BLK_CGROUP_IOCOST && BPF_SYSCALL && BPF_JIT
+ help
+ Enabling this option registers the "iocost_model_ops" BPF
+ struct_ops type, which allows a BPF program to replace the
+ builtin linear cost model on devices configured with
+ "ctrl=bpf" through io.cost.model.
+
endif # BLOCK
diff --git a/include/linux/blk-iocost.h b/include/linux/blk-iocost.h
new file mode 100644
index 0000000000000..7111b5c03dc36
--- /dev/null
+++ b/include/linux/blk-iocost.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_BLK_IOCOST_H
+#define _LINUX_BLK_IOCOST_H
+
+#include <linux/types.h>
+
+#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF
+
+/*
+ * Pluggable cost model interface for blk-iocost.
+ *
+ * A BPF struct_ops implementation registered against "iocost_model_ops"
+ * replaces the builtin linear cost model on devices configured with
+ * "ctrl=bpf" through io.cost.model. The model is a pure function of the
+ * arguments below and returns the cost of the IO in device time units,
+ * where 1 second of device time equals 2^37 (VTIME_PER_SEC). The
+ * returned value is clamped by the kernel. A cost of 0 delegates the
+ * IO back to the builtin formula, so a model which does not handle a
+ * given IO type cannot make it free.
+ *
+ * The iocg_id argument identifies the issuing cgroup (css id) so the
+ * model can keep per-cgroup state, e.g. for multi-stream sequentiality
+ * detection. The id is only valid while the cgroup exists: css ids are
+ * recycled once the cgroup is removed, so models must treat it as a
+ * transient key and reset state when they observe it reused. Whether
+ * the interface needs a release(iocg_id) callback for state cleanup is
+ * an open question.
+ *
+ * The model is called from the IO submission path under RCU and must
+ * not sleep. Only the bio-level charging path consults the model;
+ * the request-level sizing path (calc_size_vtime_cost()) keeps using
+ * the builtin formula.
+ */
+
+/* flags for calc_cost() */
+#define IOCOST_COST_F_MERGE (1ULL << 0) /* called from merge path */
+
+struct iocost_model_ops {
+ /*
+ * @op: REQ_OP_* value (uapi blk_opf.h)
+ * @nbytes: IO size in bytes
+ * @sector: starting sector
+ * @cursor: iocg cursor sector, 0 if none (sequentiality hint)
+ * @iocg_id: css id of the issuing cgroup
+ * @flags: IOCOST_COST_F_*
+ */
+ u64 (*calc_cost)(u64 op, u64 nbytes, u64 sector, u64 cursor,
+ u64 iocg_id, u64 flags);
+};
+
+bool iocost_bpf_calc_cost(u64 op, u64 nbytes, u64 sector, u64 cursor,
+ u64 iocg_id, u64 flags, u64 *costp);
+bool iocost_bpf_model_registered(void);
+
+#endif /* CONFIG_BLK_CGROUP_IOCOST_BPF */
+#endif /* _LINUX_BLK_IOCOST_H */
--
2.43.0
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [RFC PATCH 2/8] blk-iocost: define iocost_model_ops cost model interface
2026-09-08 10:01 ` [RFC PATCH 2/8] blk-iocost: define iocost_model_ops cost model interface Tao Cui
@ 2026-09-08 20:31 ` Tejun Heo
0 siblings, 0 replies; 16+ messages in thread
From: Tejun Heo @ 2026-09-08 20:31 UTC (permalink / raw)
To: Tao Cui
Cc: tj, josef, axboe, cgroups, linux-block, linux-kernel, bpf,
andrii, ast, daniel, linux-kselftest, Tao Cui
Hello, Tao.
On Tue, 08 Sep 2026 18:01:37 +0800, Tao Cui wrote:
> +config BLK_CGROUP_IOCOST_BPF
> + bool "Enable BPF pluggable cost model support for the cost IO controller"
> + depends on BLK_CGROUP_IOCOST && BPF_SYSCALL && BPF_JIT
This also needs a dependency on DEBUG_INFO_BTF. Otherwise the option can be
enabled without the BTF required to register iocost_model_ops.
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 16+ messages in thread
* [RFC PATCH 3/8] blk-iocost: implement BPF struct_ops registration
2026-09-08 10:01 [RFC PATCH 0/8] blk-iocost: BPF struct_ops cost model Tao Cui
2026-09-08 10:01 ` [RFC PATCH 1/8] blk-iocost: add iocost_ioc_tick tracepoint for per-period device summary Tao Cui
2026-09-08 10:01 ` [RFC PATCH 2/8] blk-iocost: define iocost_model_ops cost model interface Tao Cui
@ 2026-09-08 10:01 ` Tao Cui
2026-09-08 20:31 ` Tejun Heo
2026-09-08 10:01 ` [RFC PATCH 4/8] blk-iocost: dispatch cost calculation to registered BPF model Tao Cui
` (5 subsequent siblings)
8 siblings, 1 reply; 16+ messages in thread
From: Tao Cui @ 2026-09-08 10:01 UTC (permalink / raw)
To: tj, josef, axboe
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, ast, daniel,
linux-kselftest, cui.tao, Tao Cui
From: Tao Cui <cuitao@kylinos.cn>
Register the iocost_model_ops struct_ops type: at most one model
system-wide (EBUSY otherwise), called under RCU from the submit/merge
path. Registration and unregistration are serialized with a mutex
because the caller only holds the per-map lock. Includes the
verifier ops, CFI stubs and late_initcall registration. A model
returning 0 makes the caller fall back to the builtin formula, which
keeps a partial model from making the IO types it does not handle
free.
The verifier allows the base helper set, so models can use maps for
per-cgroup state keyed by iocg_id. The model runs in the submit path
under RCU and must not sleep.
No code calls the registered model yet; the dispatch hook follows.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
block/Makefile | 1 +
block/blk-iocost-bpf.c | 152 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 153 insertions(+)
create mode 100644 block/blk-iocost-bpf.c
diff --git a/block/Makefile b/block/Makefile
index e7bd320e3d697..ee5cebeea006f 100644
--- a/block/Makefile
+++ b/block/Makefile
@@ -39,3 +39,4 @@ obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += blk-crypto.o blk-crypto-profile.o \
blk-crypto-sysfs.o
obj-$(CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK) += blk-crypto-fallback.o
obj-$(CONFIG_BLOCK_HOLDER_DEPRECATED) += holder.o
+obj-$(CONFIG_BLK_CGROUP_IOCOST_BPF) += blk-iocost-bpf.o
diff --git a/block/blk-iocost-bpf.c b/block/blk-iocost-bpf.c
new file mode 100644
index 0000000000000..0c34f56ded71d
--- /dev/null
+++ b/block/blk-iocost-bpf.c
@@ -0,0 +1,152 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Tao Cui */
+
+/*
+ * blk-iocost: BPF struct_ops plumbing for pluggable cost models.
+ *
+ * Registers the "iocost_model_ops" struct_ops type. At most one model
+ * can be registered at a time; devices opt in per-queue with
+ * "echo $DEV ctrl=bpf > io.cost.model". Devices without a registered
+ * model keep using the builtin linear model.
+ */
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/bpf.h>
+#include <linux/bpf_verifier.h>
+#include <linux/btf.h>
+#include <linux/blk-iocost.h>
+
+/* 1s of device time; upper bound on a single IO's chargeable cost */
+#define IOCOST_BPF_MAX_COST (1ULL << 37)
+
+static struct bpf_struct_ops bpf_iocost_model_ops;
+
+static DEFINE_MUTEX(iocost_bpf_reg_lock);
+static struct iocost_model_ops __rcu *iocost_bpf_model;
+static DEFINE_STATIC_KEY_FALSE(iocost_bpf_key);
+
+bool iocost_bpf_model_registered(void)
+{
+ return static_branch_unlikely(&iocost_bpf_key);
+}
+
+bool iocost_bpf_calc_cost(u64 op, u64 nbytes, u64 sector, u64 cursor,
+ u64 iocg_id, u64 flags, u64 *costp)
+{
+ const struct iocost_model_ops *ops;
+ u64 cost;
+
+ if (!static_branch_unlikely(&iocost_bpf_key))
+ return false;
+
+ rcu_read_lock();
+ ops = rcu_dereference(iocost_bpf_model);
+ if (!ops) {
+ rcu_read_unlock();
+ return false;
+ }
+ cost = ops->calc_cost(op, nbytes, sector, cursor, iocg_id, flags);
+ rcu_read_unlock();
+
+ if (!cost)
+ return false;
+
+ *costp = min(cost, IOCOST_BPF_MAX_COST);
+ return true;
+}
+
+static int bpf_iocost_model_init(struct btf *btf)
+{
+ s32 type_id;
+
+ type_id = btf_find_by_name_kind(btf, "iocost_model_ops", BTF_KIND_STRUCT);
+ if (type_id < 0)
+ return -EINVAL;
+ return 0;
+}
+
+static int bpf_iocost_init_member(const struct btf_type *t,
+ const struct btf_member *member,
+ void *kdata, const void *udata)
+{
+ return 0;
+}
+
+static bool bpf_iocost_is_valid_access(int off, int size,
+ enum bpf_access_type type,
+ const struct bpf_prog *prog,
+ struct bpf_insn_access_aux *info)
+{
+ return bpf_tracing_btf_ctx_access(off, size, type, prog, info);
+}
+
+static const struct bpf_func_proto *
+bpf_iocost_get_func_proto(enum bpf_func_id func_id,
+ const struct bpf_prog *prog)
+{
+ return bpf_base_func_proto(func_id, prog);
+}
+
+static const struct bpf_verifier_ops bpf_iocost_verifier_ops = {
+ .get_func_proto = bpf_iocost_get_func_proto,
+ .is_valid_access = bpf_iocost_is_valid_access,
+};
+
+static int bpf_iocost_reg(void *kdata, struct bpf_link *link)
+{
+ struct iocost_model_ops *new_ops = kdata;
+
+ if (!new_ops->calc_cost)
+ return -EINVAL;
+
+ int ret = 0;
+
+ /* the caller only holds the per-map lock, so serialize here */
+ mutex_lock(&iocost_bpf_reg_lock);
+ if (rcu_access_pointer(iocost_bpf_model))
+ ret = -EBUSY;
+ else {
+ static_branch_inc(&iocost_bpf_key);
+ rcu_assign_pointer(iocost_bpf_model, new_ops);
+ }
+ mutex_unlock(&iocost_bpf_reg_lock);
+ return ret;
+}
+
+static void bpf_iocost_unreg(void *kdata, struct bpf_link *link)
+{
+ mutex_lock(&iocost_bpf_reg_lock);
+ if (rcu_access_pointer(iocost_bpf_model) == kdata) {
+ rcu_assign_pointer(iocost_bpf_model, NULL);
+ static_branch_dec(&iocost_bpf_key);
+ }
+ mutex_unlock(&iocost_bpf_reg_lock);
+}
+
+static u64 bpf_iocost_calc_cost_stub(u64 op, u64 nbytes, u64 sector,
+ u64 cursor, u64 iocg_id, u64 flags)
+{
+ return 0;
+}
+
+static struct iocost_model_ops __bpf_ops_iocost_model_ops = {
+ .calc_cost = bpf_iocost_calc_cost_stub,
+};
+
+static struct bpf_struct_ops bpf_iocost_model_ops = {
+ .verifier_ops = &bpf_iocost_verifier_ops,
+ .init = bpf_iocost_model_init,
+ .init_member = bpf_iocost_init_member,
+ .reg = bpf_iocost_reg,
+ .unreg = bpf_iocost_unreg,
+ .name = "iocost_model_ops",
+ .cfi_stubs = &__bpf_ops_iocost_model_ops,
+ .owner = THIS_MODULE,
+};
+
+static int __init bpf_iocost_init(void)
+{
+ return register_bpf_struct_ops(&bpf_iocost_model_ops, iocost_model_ops);
+}
+late_initcall(bpf_iocost_init);
--
2.43.0
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [RFC PATCH 3/8] blk-iocost: implement BPF struct_ops registration
2026-09-08 10:01 ` [RFC PATCH 3/8] blk-iocost: implement BPF struct_ops registration Tao Cui
@ 2026-09-08 20:31 ` Tejun Heo
0 siblings, 0 replies; 16+ messages in thread
From: Tejun Heo @ 2026-09-08 20:31 UTC (permalink / raw)
To: Tao Cui
Cc: tj, josef, axboe, cgroups, linux-block, linux-kernel, bpf,
andrii, ast, daniel, linux-kselftest, Tao Cui
Hello, Tao.
On Tue, 08 Sep 2026 18:01:38 +0800, Tao Cui wrote:
> +static struct bpf_struct_ops bpf_iocost_model_ops = {
> + .verifier_ops = &bpf_iocost_verifier_ops,
> + .init = bpf_iocost_model_init,
> + .init_member = bpf_iocost_init_member,
Please add a .check_member callback rejecting sleepable programs. The
generic verifier permits sleepable struct_ops programs, so a model can
currently call bpf_copy_from_user() while iocost_bpf_calc_cost() holds
rcu_read_lock().
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 16+ messages in thread
* [RFC PATCH 4/8] blk-iocost: dispatch cost calculation to registered BPF model
2026-09-08 10:01 [RFC PATCH 0/8] blk-iocost: BPF struct_ops cost model Tao Cui
` (2 preceding siblings ...)
2026-09-08 10:01 ` [RFC PATCH 3/8] blk-iocost: implement BPF struct_ops registration Tao Cui
@ 2026-09-08 10:01 ` Tao Cui
2026-09-08 20:31 ` Tejun Heo
2026-09-08 10:01 ` [RFC PATCH 5/8] blk-iocost: add ctrl=bpf per-device opt-in Tao Cui
` (4 subsequent siblings)
8 siblings, 1 reply; 16+ messages in thread
From: Tao Cui @ 2026-09-08 10:01 UTC (permalink / raw)
To: tj, josef, axboe
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, ast, daniel,
linux-kselftest, cui.tao, Tao Cui
From: Tao Cui <cuitao@kylinos.cn>
Add the bpf_cost_model field to struct ioc and the dispatch hook in
calc_vtime_cost(): when a BPF model is registered and the device has
opted in (bpf_cost_model set), call the model's calc_cost instead of
the builtin formula. If no model is registered anymore the builtin
formula is used as fallback.
The user interface to set bpf_cost_model follows.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
block/blk-iocost.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/block/blk-iocost.c b/block/blk-iocost.c
index c24daa9d72ee9..bdb4bab86a116 100644
--- a/block/blk-iocost.c
+++ b/block/blk-iocost.c
@@ -177,6 +177,7 @@
#include <linux/timer.h>
#include <linux/time64.h>
#include <linux/parser.h>
+#include <linux/blk-iocost.h>
#include <linux/sched/signal.h>
#include <asm/local.h>
#include <asm/local64.h>
@@ -445,6 +446,7 @@ struct ioc {
int autop_idx;
bool user_qos_params:1;
bool user_cost_model:1;
+ bool bpf_cost_model;
};
struct iocg_pcpu_stat {
@@ -2578,6 +2580,23 @@ static u64 calc_vtime_cost(struct bio *bio, struct ioc_gq *iocg, bool is_merge)
{
u64 cost;
+#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF
+ struct ioc *ioc = iocg->ioc;
+
+ if (READ_ONCE(ioc->bpf_cost_model)) {
+ u64 bpf_cost;
+
+ if (iocost_bpf_calc_cost(bio_op(bio),
+ bio->bi_iter.bi_size,
+ bio->bi_iter.bi_sector,
+ iocg->cursor,
+ iocg_to_blkg(iocg)->blkcg->css.id,
+ is_merge ? IOCOST_COST_F_MERGE : 0,
+ &bpf_cost))
+ return bpf_cost;
+ }
+#endif
+
calc_vtime_cost_builtin(bio, iocg, is_merge, &cost);
return cost;
}
--
2.43.0
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [RFC PATCH 4/8] blk-iocost: dispatch cost calculation to registered BPF model
2026-09-08 10:01 ` [RFC PATCH 4/8] blk-iocost: dispatch cost calculation to registered BPF model Tao Cui
@ 2026-09-08 20:31 ` Tejun Heo
0 siblings, 0 replies; 16+ messages in thread
From: Tejun Heo @ 2026-09-08 20:31 UTC (permalink / raw)
To: Tao Cui
Cc: tj, josef, axboe, cgroups, linux-block, linux-kernel, bpf,
andrii, ast, daniel, linux-kselftest, Tao Cui
Hello, Tao.
On Tue, 08 Sep 2026 18:01:39 +0800, Tao Cui wrote:
> + if (iocost_bpf_calc_cost(bio_op(bio),
> + bio->bi_iter.bi_size,
> + bio->bi_iter.bi_sector,
> + iocg->cursor,
> + iocg_to_blkg(iocg)->blkcg->css.id,
> + is_merge ? IOCOST_COST_F_MERGE : 0,
bio_op() drops REQ_PREFLUSH and REQ_FUA, and the flags argument only carries
merge status. Please preserve the operation flags.
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 16+ messages in thread
* [RFC PATCH 5/8] blk-iocost: add ctrl=bpf per-device opt-in
2026-09-08 10:01 [RFC PATCH 0/8] blk-iocost: BPF struct_ops cost model Tao Cui
` (3 preceding siblings ...)
2026-09-08 10:01 ` [RFC PATCH 4/8] blk-iocost: dispatch cost calculation to registered BPF model Tao Cui
@ 2026-09-08 10:01 ` Tao Cui
2026-09-08 10:01 ` [RFC PATCH 6/8] selftests/bpf: add iocost cost model test Tao Cui
` (3 subsequent siblings)
8 siblings, 0 replies; 16+ messages in thread
From: Tao Cui @ 2026-09-08 10:01 UTC (permalink / raw)
To: tj, josef, axboe
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, ast, daniel,
linux-kselftest, cui.tao, Tao Cui
From: Tao Cui <cuitao@kylinos.cn>
Add "bpf" to the ctrl= options in io.cost.model. Writing
"$dev ctrl=bpf" to io.cost.model switches the device to the
registered BPF cost model (if any is registered); ctrl=auto or
ctrl=user switches back to the builtin model.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
block/blk-iocost.c | 22 +++++++++++++++++-----
1 file changed, 17 insertions(+), 5 deletions(-)
diff --git a/block/blk-iocost.c b/block/blk-iocost.c
index bdb4bab86a116..a58c16fcb29fa 100644
--- a/block/blk-iocost.c
+++ b/block/blk-iocost.c
@@ -3465,7 +3465,8 @@ static u64 ioc_cost_model_prfill(struct seq_file *sf,
seq_printf(sf, "%s ctrl=%s model=linear "
"rbps=%llu rseqiops=%llu rrandiops=%llu "
"wbps=%llu wseqiops=%llu wrandiops=%llu\n",
- dname, ioc->user_cost_model ? "user" : "auto",
+ dname, ioc->bpf_cost_model ? "bpf" :
+ ioc->user_cost_model ? "user" : "auto",
u[I_LCOEF_RBPS], u[I_LCOEF_RSEQIOPS], u[I_LCOEF_RRANDIOPS],
u[I_LCOEF_WBPS], u[I_LCOEF_WSEQIOPS], u[I_LCOEF_WRANDIOPS]);
spin_unlock_irq(&ioc->lock);
@@ -3505,7 +3506,7 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
unsigned int memflags;
struct ioc *ioc;
u64 u[NR_I_LCOEFS];
- bool user;
+ bool user, bpf;
char *body, *p;
int ret;
@@ -3536,6 +3537,7 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
spin_lock_irq(&ioc->lock);
memcpy(u, ioc->params.i_lcoefs, sizeof(u));
user = ioc->user_cost_model;
+ bpf = ioc->bpf_cost_model;
ret = -EINVAL;
@@ -3551,11 +3553,20 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
switch (match_token(p, cost_ctrl_tokens, args)) {
case COST_CTRL:
match_strlcpy(buf, &args[0], sizeof(buf));
- if (!strcmp(buf, "auto"))
+ if (!strcmp(buf, "auto")) {
user = false;
- else if (!strcmp(buf, "user"))
+ bpf = false;
+ } else if (!strcmp(buf, "user")) {
user = true;
- else
+ bpf = false;
+#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF
+ } else if (!strcmp(buf, "bpf")) {
+ if (!iocost_bpf_model_registered())
+ goto unlock;
+ user = false;
+ bpf = true;
+#endif
+ } else
goto unlock;
continue;
case COST_MODEL:
@@ -3580,6 +3591,7 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
} else {
ioc->user_cost_model = false;
}
+ WRITE_ONCE(ioc->bpf_cost_model, bpf);
ioc_refresh_params(ioc, true);
ret = 0;
--
2.43.0
^ permalink raw reply [flat|nested] 16+ messages in thread* [RFC PATCH 6/8] selftests/bpf: add iocost cost model test
2026-09-08 10:01 [RFC PATCH 0/8] blk-iocost: BPF struct_ops cost model Tao Cui
` (4 preceding siblings ...)
2026-09-08 10:01 ` [RFC PATCH 5/8] blk-iocost: add ctrl=bpf per-device opt-in Tao Cui
@ 2026-09-08 10:01 ` Tao Cui
2026-09-08 20:31 ` Tejun Heo
2026-09-08 10:01 ` [RFC PATCH 7/8] selftests/bpf: add multi-stream sequentiality example model Tao Cui
` (2 subsequent siblings)
8 siblings, 1 reply; 16+ messages in thread
From: Tao Cui @ 2026-09-08 10:01 UTC (permalink / raw)
To: tj, josef, axboe
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, ast, daniel,
linux-kselftest, cui.tao, Tao Cui
From: Tao Cui <cuitao@kylinos.cn>
Add an example cost model implementing iocost_model_ops with the
builtin linear HDD formula at double cost, and a runner which
registers it as a struct_ops, switches a device to ctrl=bpf through
io.cost.model and verifies the readback, restoring ctrl=auto
afterwards. The runner first checks the negative case: writing
ctrl=bpf while no model is registered is rejected. Under the same
workload the doubled model should charge exactly twice the builtin
model, which makes it a convenient way to verify that accounting
goes through the BPF path; the accounting itself needs real IO and
is covered by the kernel-side validation described in the cover
letter. The runner is skipped unless $IOCOST_TEST_DEV gives a
major:minor of a device with iocost enabled.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
.../selftests/bpf/prog_tests/iocost_model.c | 124 ++++++++++++++++++
.../selftests/bpf/progs/iocost_model.c | 91 +++++++++++++
2 files changed, 215 insertions(+)
create mode 100644 tools/testing/selftests/bpf/prog_tests/iocost_model.c
create mode 100644 tools/testing/selftests/bpf/progs/iocost_model.c
diff --git a/tools/testing/selftests/bpf/prog_tests/iocost_model.c b/tools/testing/selftests/bpf/prog_tests/iocost_model.c
new file mode 100644
index 0000000000000..386026241d5b4
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/iocost_model.c
@@ -0,0 +1,124 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Tao Cui */
+#include <test_progs.h>
+#include "iocost_model.skel.h"
+
+/*
+ * Switch a device to ctrl=bpf through io.cost.model and verify the
+ * readback, restoring ctrl=auto afterwards. Returns 0 on success.
+ * Shared by the example-model tests below.
+ */
+static int switch_to_bpf(const char *dev)
+{
+ char path[] = "/sys/fs/cgroup/io.cost.model";
+ char word[256], line[256], buf[300];
+ FILE *fp;
+ int err, found = 0;
+
+ fp = fopen(path, "r+");
+ if (!fp) {
+ TH_LOG("open %s: %s", path, strerror(errno));
+ return -1;
+ }
+
+ while (fgets(line, sizeof(line), fp)) {
+ if (sscanf(line, "%255s", word) == 1 && !strcmp(word, dev)) {
+ found = 1;
+ break;
+ }
+ }
+ if (!found) {
+ TH_LOG("%s has no iocost line", dev);
+ fclose(fp);
+ return -1;
+ }
+
+ snprintf(buf, sizeof(buf), "%s ctrl=bpf", dev);
+ err = fprintf(fp, "%s\n", buf);
+ if (err <= 0) {
+ TH_LOG("write ctrl=bpf failed");
+ fclose(fp);
+ return -1;
+ }
+ fflush(fp);
+
+ rewind(fp);
+ found = 0;
+ while (fgets(line, sizeof(line), fp)) {
+ if (sscanf(line, "%255s", word) == 1 && !strcmp(word, dev)) {
+ found = strstr(line, "ctrl=bpf") != NULL;
+ break;
+ }
+ }
+ if (!found)
+ TH_LOG("readback does not show ctrl=bpf");
+
+ fprintf(fp, "%s ctrl=auto\n", dev);
+ fclose(fp);
+ return found ? 0 : -1;
+}
+
+static int dev_has_iocost(void)
+{
+ FILE *fp = fopen("/sys/fs/cgroup/io.cost.qos", "r");
+
+ if (fp)
+ fclose(fp);
+ return fp != NULL;
+}
+
+/*
+ * Register the example cost model and switch a device to ctrl=bpf.
+ * IO accounting itself is not checked here; it needs a device doing
+ * real IO under iocost and is covered by the kernel-side validation
+ * described in the cover letter.
+ *
+ * Requires root, cgroup v2 and a device with iocost support. The
+ * device must be given as major:minor in $IOCOST_TEST_DEV, otherwise
+ * the test is skipped.
+ */
+void serial_test_iocost_model(void)
+{
+ struct iocost_model *skel;
+ char word[256], *dev;
+ int err;
+
+ dev = getenv("IOCOST_TEST_DEV");
+ if (!dev || geteuid() != 0) {
+ test__skip();
+ return;
+ }
+ if (!ASSERT_TRUE(dev_has_iocost(), "iocost_mounted"))
+ return;
+
+ /* negative: ctrl=bpf must be rejected while no model is
+ * registered, so a typo cannot silently disable cost model
+ * updates */
+ if (sscanf(dev, "%255s", word) != 1)
+ return;
+ {
+ char path[] = "/sys/fs/cgroup/io.cost.model";
+ char buf[300];
+ FILE *fp = fopen(path, "w");
+
+ snprintf(buf, sizeof(buf), "%s ctrl=bpf\n", word);
+ err = 0;
+ if (fp) {
+ err = fprintf(fp, "%s", buf) <= 0;
+ fclose(fp);
+ }
+ ASSERT_TRUE(err, "ctrl_bpf_without_model_rejected");
+ }
+
+ skel = iocost_model__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "skel_open_load"))
+ return;
+
+ /* attaching the struct_ops registers the model; a second
+ * registration of the same ops would fail with EBUSY */
+ err = iocost_model__attach(skel);
+ if (ASSERT_OK(err, "attach"))
+ ASSERT_OK(switch_to_bpf(dev), "switch_and_readback");
+
+ iocost_model__destroy(skel);
+}
diff --git a/tools/testing/selftests/bpf/progs/iocost_model.c b/tools/testing/selftests/bpf/progs/iocost_model.c
new file mode 100644
index 0000000000000..70f7987f8eb6a
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/iocost_model.c
@@ -0,0 +1,91 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Tao Cui */
+/*
+ * Example iocost cost model: the builtin linear HDD formula with all
+ * costs doubled.
+ *
+ * The constants mirror what calc_lcoefs() derives from the AUTOP_HDD
+ * defaults (rbps=174019176 rseqiops=41708 rrandiops=370, w-side
+ * analog) in vtime units where 1s == 2^37, expressed with the same
+ * round-up divisions so they cannot drift from the kernel, so a
+ * device switched to ctrl=bpf with this model registered charges
+ * exactly twice the builtin model under the same workload, which
+ * makes it a convenient way to verify that accounting goes through
+ * the BPF path.
+ */
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+/* VTIME_PER_SEC comes from vmlinux.h (a BTF enum constant) */
+#define IOC_PAGE_SIZE 4096
+#define IOC_SECT_TO_PAGE_SHIFT 3 /* 512B sectors to 4kB pages */
+#define LCOEF_RANDIO_PAGES 4096 /* 16MB seek threshold */
+#define IOCOST_COST_F_MERGE (1ULL << 0) /* not in BTF: a plain macro */
+
+/* DIV64_U64_ROUND_UP / DIV_ROUND_UP_ULL equivalents, folded at
+ * compile time */
+#define RU(x, y) ((x) / (y) + (((x) % (y)) ? 1 : 0))
+
+#define RBPS 174019176ULL
+#define RSEQIOPS 41708ULL
+#define RRANDIOPS 370ULL
+#define WBPS 178075866ULL
+#define WSEQIOPS 42705ULL
+#define WRANDIOPS 378ULL
+
+#define RPAGE (RU(VTIME_PER_SEC, RU(RBPS, IOC_PAGE_SIZE)))
+#define RSEQIO (RU(VTIME_PER_SEC, RSEQIOPS) - RPAGE)
+#define RRANDIO (RU(VTIME_PER_SEC, RRANDIOPS) - RPAGE)
+#define WPAGE (RU(VTIME_PER_SEC, RU(WBPS, IOC_PAGE_SIZE)))
+#define WSEQIO (RU(VTIME_PER_SEC, WSEQIOPS) - WPAGE)
+#define WRANDIO (RU(VTIME_PER_SEC, WRANDIOPS) - WPAGE)
+
+static __always_inline u64 builtin_cost(u64 op, u64 nbytes, u64 sector,
+ u64 cursor, u64 flags)
+{
+ u64 pages, seek_pages = 0, base, coef_page;
+
+ if (!nbytes)
+ return 0;
+
+ switch (op) {
+ case REQ_OP_READ:
+ base = RSEQIO;
+ coef_page = RPAGE;
+ break;
+ case REQ_OP_WRITE:
+ base = WSEQIO;
+ coef_page = WPAGE;
+ break;
+ default:
+ return 0;
+ }
+
+ if (cursor) {
+ seek_pages = sector > cursor ? sector - cursor
+ : cursor - sector;
+ seek_pages >>= IOC_SECT_TO_PAGE_SHIFT;
+ if (seek_pages > LCOEF_RANDIO_PAGES)
+ base = (op == REQ_OP_READ) ? RRANDIO : WRANDIO;
+ }
+
+ pages = RU(nbytes, IOC_PAGE_SIZE);
+ if (flags & IOCOST_COST_F_MERGE)
+ base = 0;
+ return base + pages * coef_page;
+}
+
+SEC("struct_ops")
+u64 BPF_PROG(iocost_2x_calc_cost, u64 op, u64 nbytes, u64 sector,
+ u64 cursor, u64 iocg_id, u64 flags)
+{
+ return 2 * builtin_cost(op, nbytes, sector, cursor, flags);
+}
+
+SEC(".struct_ops")
+struct iocost_model_ops iocost_2x = {
+ .calc_cost = (void *)iocost_2x_calc_cost,
+};
+
+char LICENSE[] SEC("license") = "GPL";
--
2.43.0
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [RFC PATCH 6/8] selftests/bpf: add iocost cost model test
2026-09-08 10:01 ` [RFC PATCH 6/8] selftests/bpf: add iocost cost model test Tao Cui
@ 2026-09-08 20:31 ` Tejun Heo
0 siblings, 0 replies; 16+ messages in thread
From: Tejun Heo @ 2026-09-08 20:31 UTC (permalink / raw)
To: Tao Cui
Cc: tj, josef, axboe, cgroups, linux-block, linux-kernel, bpf,
andrii, ast, daniel, linux-kselftest, Tao Cui
Hello, Tao.
On Tue, 08 Sep 2026 18:01:41 +0800, Tao Cui wrote:
> + TH_LOG("open %s: %s", path, strerror(errno));
TH_LOG is not available through test_progs.h. A syntax check of the posted
runner fails here. Please use the test_progs logging facilities.
> +#include "iocost_model.skel.h"
The selftest kernel config also needs CONFIG_BLK_CGROUP_IOCOST and
CONFIG_BLK_CGROUP_IOCOST_BPF so vmlinux.h contains iocost_model_ops. The
runtime skip cannot avoid that build dependency.
> + err = fprintf(fp, "%s", buf) <= 0;
> + fclose(fp);
fprintf() can succeed while the buffered write is rejected at fclose(),
whose result is ignored. Check the actual write result and expected errno.
Please also check the flush and restoration errors in switch_to_bpf().
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 16+ messages in thread
* [RFC PATCH 7/8] selftests/bpf: add multi-stream sequentiality example model
2026-09-08 10:01 [RFC PATCH 0/8] blk-iocost: BPF struct_ops cost model Tao Cui
` (5 preceding siblings ...)
2026-09-08 10:01 ` [RFC PATCH 6/8] selftests/bpf: add iocost cost model test Tao Cui
@ 2026-09-08 10:01 ` Tao Cui
2026-09-08 20:31 ` Tejun Heo
2026-09-08 10:01 ` [RFC PATCH 8/8] docs: cgroup-v2: document io.cost ctrl=bpf option Tao Cui
2026-09-08 20:31 ` [RFC PATCH 0/8] blk-iocost: BPF struct_ops cost model Tejun Heo
8 siblings, 1 reply; 16+ messages in thread
From: Tao Cui @ 2026-09-08 10:01 UTC (permalink / raw)
To: tj, josef, axboe
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, ast, daniel,
linux-kselftest, cui.tao, Tao Cui
From: Tao Cui <cuitao@kylinos.cn>
Add a second example cost model which replaces the builtin
single-cursor sequentiality heuristic with a per-cgroup table of
stream slots keyed by iocg_id: an IO is sequential iff its sector
matches the expected next sector of any tracked stream. Interleaved
sequential readers in one cgroup keep their own slots instead of
ping-ponging a single cursor, and random IO inside a hot window
rarely matches a moving expectation.
Measured (QEMU, virtio-blk with the HDD profile, 4k IOs, w=1000):
two sequential readers in one cgroup are priced 2064us/op by the
builtin model (judged random) and 23us/op by this model (judged
sequential), a throughput recovery from 7.2 to 511 MiB/s; random IO
inside an 8M window is priced 22.9us/op by builtin (undercharge) and
2643us/op by this model; single-stream sequential and whole-disk
random pricing are unchanged.
The runner reuses the switch helper from the first example model.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
.../selftests/bpf/prog_tests/iocost_model.c | 31 +++++
tools/testing/selftests/bpf/progs/iocost_ms.c | 122 ++++++++++++++++++
2 files changed, 153 insertions(+)
create mode 100644 tools/testing/selftests/bpf/progs/iocost_ms.c
diff --git a/tools/testing/selftests/bpf/prog_tests/iocost_model.c b/tools/testing/selftests/bpf/prog_tests/iocost_model.c
index 386026241d5b4..2c149f3cd243a 100644
--- a/tools/testing/selftests/bpf/prog_tests/iocost_model.c
+++ b/tools/testing/selftests/bpf/prog_tests/iocost_model.c
@@ -2,6 +2,7 @@
/* Copyright (c) 2026 Tao Cui */
#include <test_progs.h>
#include "iocost_model.skel.h"
+#include "iocost_ms.skel.h"
/*
* Switch a device to ctrl=bpf through io.cost.model and verify the
@@ -122,3 +123,33 @@ void serial_test_iocost_model(void)
iocost_model__destroy(skel);
}
+
+/*
+ * Same check for the multi-stream example model. Only one model can
+ * be registered at a time, so this test must run separately from
+ * test_iocost_model; both are serial.
+ */
+void serial_test_iocost_model_streams(void)
+{
+ struct iocost_ms *skel;
+ char *dev;
+ int err;
+
+ dev = getenv("IOCOST_TEST_DEV");
+ if (!dev || geteuid() != 0) {
+ test__skip();
+ return;
+ }
+ if (!ASSERT_TRUE(dev_has_iocost(), "iocost_mounted"))
+ return;
+
+ skel = iocost_ms__open_and_load();
+ if (!ASSERT_OK_PTR(skel, "skel_open_load"))
+ return;
+
+ err = iocost_ms__attach(skel);
+ if (ASSERT_OK(err, "attach"))
+ ASSERT_OK(switch_to_bpf(dev), "switch_and_readback");
+
+ iocost_ms__destroy(skel);
+}
diff --git a/tools/testing/selftests/bpf/progs/iocost_ms.c b/tools/testing/selftests/bpf/progs/iocost_ms.c
new file mode 100644
index 0000000000000..465fd5d5c62e0
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/iocost_ms.c
@@ -0,0 +1,122 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 Tao Cui */
+/*
+ * Example multi-stream sequentiality detection cost model.
+ *
+ * The builtin model keeps a single cursor per cgroup, so two
+ * interleaved sequential readers in one cgroup are all priced random
+ * (measured 89x overcharge, 12.9x throughput collapse), while random
+ * IO inside a hot window smaller than the 16MB seek threshold is
+ * priced sequential (measured 107x undercharge). This model replaces
+ * the single cursor with a per-cgroup table of stream slots, keyed by
+ * the iocg_id argument: an IO is sequential iff its sector matches the
+ * expected next sector of any tracked stream. Interleaved streams
+ * keep their own slots, and windowed random IO rarely matches a
+ * moving expectation.
+ */
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+/* VTIME_PER_SEC comes from vmlinux.h (a BTF enum constant) */
+#define IOC_PAGE_SIZE 4096
+#define IOCOST_COST_F_MERGE (1ULL << 0) /* not in BTF: a plain macro */
+
+/* DIV64_U64_ROUND_UP / DIV_ROUND_UP_ULL equivalents, folded at
+ * compile time */
+#define RU(x, y) ((x) / (y) + (((x) % (y)) ? 1 : 0))
+
+#define RBPS 174019176ULL
+#define RSEQIOPS 41708ULL
+#define RRANDIOPS 370ULL
+#define WBPS 178075866ULL
+#define WSEQIOPS 42705ULL
+#define WRANDIOPS 378ULL
+
+#define RPAGE (RU(VTIME_PER_SEC, RU(RBPS, IOC_PAGE_SIZE)))
+#define RSEQIO (RU(VTIME_PER_SEC, RSEQIOPS) - RPAGE)
+#define RRANDIO (RU(VTIME_PER_SEC, RRANDIOPS) - RPAGE)
+#define WPAGE (RU(VTIME_PER_SEC, RU(WBPS, IOC_PAGE_SIZE)))
+#define WSEQIO (RU(VTIME_PER_SEC, WSEQIOPS) - WPAGE)
+#define WRANDIO (RU(VTIME_PER_SEC, WRANDIOPS) - WPAGE)
+
+#define NSLOTS 4
+
+struct streams {
+ __u64 expected[NSLOTS]; /* next expected sector, per stream */
+ __u64 stamp[NSLOTS]; /* LRU stamp, 0 = empty */
+};
+
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __type(key, __u64);
+ __type(value, struct streams);
+ __uint(max_entries, 64);
+} stream_tab SEC(".maps");
+
+SEC("struct_ops")
+u64 BPF_PROG(iocost_ms_calc_cost, u64 op, u64 nbytes, u64 sector,
+ u64 cursor, u64 iocg_id, u64 flags)
+{
+ struct streams *s;
+ u64 page, seqio, randio, base, advance, now;
+ u32 i, victim = 0, found = 0xFFFFFFFF;
+
+ if (op != REQ_OP_READ && op != REQ_OP_WRITE)
+ return 0; /* delegate the rest to builtin */
+ if (!nbytes)
+ return 0;
+
+ if (op == REQ_OP_READ) {
+ page = RPAGE; seqio = RSEQIO; randio = RRANDIO;
+ } else {
+ page = WPAGE; seqio = WSEQIO; randio = WRANDIO;
+ }
+ advance = RU(nbytes, 512); /* sectors */
+
+ s = bpf_map_lookup_elem(&stream_tab, &iocg_id);
+ if (!s) {
+ struct streams zero = {};
+
+ bpf_map_update_elem(&stream_tab, &iocg_id, &zero,
+ BPF_NOEXIST);
+ s = bpf_map_lookup_elem(&stream_tab, &iocg_id);
+ if (!s)
+ return seqio + RU(nbytes, IOC_PAGE_SIZE) * page;
+ }
+
+ if (flags & IOCOST_COST_F_MERGE) {
+ base = 0;
+ goto out;
+ }
+
+ now = bpf_ktime_get_ns();
+ for (i = 0; i < NSLOTS; i++) {
+ if (s->expected[i] == sector && s->stamp[i]) {
+ found = i;
+ break;
+ }
+ }
+ if (found != 0xFFFFFFFF) {
+ base = seqio;
+ s->expected[found] = sector + advance;
+ s->stamp[found] = now;
+ } else {
+ base = randio;
+ for (i = 1; i < NSLOTS; i++) {
+ if (s->stamp[i] < s->stamp[victim])
+ victim = i;
+ }
+ s->expected[victim] = sector + advance;
+ s->stamp[victim] = now;
+ }
+out:
+ return base + RU(nbytes, IOC_PAGE_SIZE) * page;
+}
+
+SEC(".struct_ops")
+struct iocost_model_ops iocost_ms = {
+ .calc_cost = (void *)iocost_ms_calc_cost,
+};
+
+char LICENSE[] SEC("license") = "GPL";
--
2.43.0
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [RFC PATCH 7/8] selftests/bpf: add multi-stream sequentiality example model
2026-09-08 10:01 ` [RFC PATCH 7/8] selftests/bpf: add multi-stream sequentiality example model Tao Cui
@ 2026-09-08 20:31 ` Tejun Heo
0 siblings, 0 replies; 16+ messages in thread
From: Tejun Heo @ 2026-09-08 20:31 UTC (permalink / raw)
To: Tao Cui
Cc: tj, josef, axboe, cgroups, linux-block, linux-kernel, bpf,
andrii, ast, daniel, linux-kselftest, Tao Cui
Hello, Tao.
On Tue, 08 Sep 2026 18:01:42 +0800, Tao Cui wrote:
> + if (flags & IOCOST_COST_F_MERGE) {
> + base = 0;
> + goto out;
Please advance the stream position for merged bios too. With 4K IOs, a new
request at sector S sets expected to S+8. A merged bio at S+8 leaves it
there, so the next new request at S+16 is priced random.
> + s = bpf_map_lookup_elem(&stream_tab, &iocg_id);
> + if (!s)
> + return seqio + RU(nbytes, IOC_PAGE_SIZE) * page;
Once the 64-entry map fills, every new cgroup receives sequential prices,
including for random IO. Please note this limitation in the example.
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 16+ messages in thread
* [RFC PATCH 8/8] docs: cgroup-v2: document io.cost ctrl=bpf option
2026-09-08 10:01 [RFC PATCH 0/8] blk-iocost: BPF struct_ops cost model Tao Cui
` (6 preceding siblings ...)
2026-09-08 10:01 ` [RFC PATCH 7/8] selftests/bpf: add multi-stream sequentiality example model Tao Cui
@ 2026-09-08 10:01 ` Tao Cui
2026-09-08 20:31 ` [RFC PATCH 0/8] blk-iocost: BPF struct_ops cost model Tejun Heo
8 siblings, 0 replies; 16+ messages in thread
From: Tao Cui @ 2026-09-08 10:01 UTC (permalink / raw)
To: tj, josef, axboe
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, ast, daniel,
linux-kselftest, cui.tao, Tao Cui
From: Tao Cui <cuitao@kylinos.cn>
Document the "bpf" value of the ctrl key of io.cost.model and the
iocost_model_ops struct_ops it selects, in the io.cost.model section
of the cgroup v2 documentation: the registration precondition and
single-instance limit, the scalar arguments, the vtime clamp and the
delegation-to-builtin semantics of a zero return value.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
Documentation/admin-guide/cgroup-v2.rst | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 8d2603751c51a..54b0574ce586c 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -2115,10 +2115,24 @@ IO Interface Files
are defined.
===== ================================
- ctrl "auto" or "user"
+ ctrl "auto", "user" or "bpf"
model The cost model in use - "linear"
===== ================================
+ When "ctrl" is "bpf" (only available when
+ CONFIG_BLK_CGROUP_IOCOST_BPF is enabled), cost calculation for
+ the device is dispatched to the BPF program registered against
+ the iocost_model_ops struct_ops. Writing "bpf" fails with -EINVAL
+ until such a program is registered, and at most one program can
+ be registered system-wide. The model is called with scalar
+ arguments (operation, size, sector, cgroup cursor, css id of the
+ issuing cgroup, flags; see include/linux/blk-iocost.h for their
+ meaning) and returns a cost in vtime units, clamped by the kernel
+ to 1 second of device time per IO. A return value of 0 delegates
+ the IO back to the builtin linear formula, so a model which only
+ handles some IO types cannot make the rest free. Devices which
+ don't opt in keep the builtin linear model unchanged.
+
When "ctrl" is "auto", the kernel may change all parameters
dynamically. When "ctrl" is set to "user" or any other
parameters are written to, "ctrl" become "user" and the
--
2.43.0
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [RFC PATCH 0/8] blk-iocost: BPF struct_ops cost model
2026-09-08 10:01 [RFC PATCH 0/8] blk-iocost: BPF struct_ops cost model Tao Cui
` (7 preceding siblings ...)
2026-09-08 10:01 ` [RFC PATCH 8/8] docs: cgroup-v2: document io.cost ctrl=bpf option Tao Cui
@ 2026-09-08 20:31 ` Tejun Heo
2026-09-09 13:12 ` Tao Cui
8 siblings, 1 reply; 16+ messages in thread
From: Tejun Heo @ 2026-09-08 20:31 UTC (permalink / raw)
To: Tao Cui
Cc: tj, josef, axboe, cgroups, linux-block, linux-kernel, bpf,
andrii, ast, daniel, linux-kselftest, Tao Cui
Hello, Tao.
On Tue, 08 Sep 2026 18:01:35 +0800, Tao Cui wrote:
> 4. One model system-wide vs named models selectable per device (as
> tcp-cc names are per-socket)?
The struct_ops instance should be per-device. Loading a model for a device
should switch it to BPF automatically. Switching back to the builtin model
should detach the struct_ops.
> 2. Per-cgroup state lifetime: iocg_id is the css id and is recycled
> once the cgroup is removed, so models must treat it as a
> transient key. Does the interface need a release(iocg_id)
> callback so models can clean up per-cgroup state instead of
> detecting reuse, or is per-cgroup bpf local storage, which
> carries its own lifetime, the better home for model state?
I'd pass blkcg as a kptr and add online/offline callbacks so the model can
manage its per-cgroup state.
> return value of 0 delegates the IO back to the builtin formula, so a
> model which only handles some IO types cannot make the rest free.
I don't think mixing BPF and builtin pricing makes sense, either per IO or
between charging and request sizing. It seems convoluted and error-prone.
Implementing the linear model in BPF is straightforward.
Please combine the interface, registration, dispatch and configuration
changes into one patch. These patches are small and implement one feature.
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [RFC PATCH 0/8] blk-iocost: BPF struct_ops cost model
2026-09-08 20:31 ` [RFC PATCH 0/8] blk-iocost: BPF struct_ops cost model Tejun Heo
@ 2026-09-09 13:12 ` Tao Cui
0 siblings, 0 replies; 16+ messages in thread
From: Tao Cui @ 2026-09-09 13:12 UTC (permalink / raw)
To: Tejun Heo
Cc: cui.tao, josef, axboe, cgroups, linux-block, linux-kernel, bpf,
andrii, ast, daniel, linux-kselftest, Tao Cui
Hello, Tejun
在 2026/9/9 04:31, Tejun Heo 写道:
> Hello, Tao.
>
> On Tue, 08 Sep 2026 18:01:35 +0800, Tao Cui wrote:
>
>> 4. One model system-wide vs named models selectable per device (as
>> tcp-cc names are per-socket)?
>
> The struct_ops instance should be per-device. Loading a model for a device
> should switch it to BPF automatically. Switching back to the builtin model
> should detach the struct_ops.
>
>> 2. Per-cgroup state lifetime: iocg_id is the css id and is recycled
>> once the cgroup is removed, so models must treat it as a
>> transient key. Does the interface need a release(iocg_id)
>> callback so models can clean up per-cgroup state instead of
>> detecting reuse, or is per-cgroup bpf local storage, which
>> carries its own lifetime, the better home for model state?
>
> I'd pass blkcg as a kptr and add online/offline callbacks so the model can
> manage its per-cgroup state.
>
>> return value of 0 delegates the IO back to the builtin formula, so a
>> model which only handles some IO types cannot make the rest free.
>
> I don't think mixing BPF and builtin pricing makes sense, either per IO or
> between charging and request sizing. It seems convoluted and error-prone.
> Implementing the linear model in BPF is straightforward.
>
> Please combine the interface, registration, dispatch and configuration
> changes into one patch. These patches are small and implement one feature.
>
Thanks for the detailed review across the series.
All the comments make sense. I'm reworking the interface for v2
along those lines: the attachment model will be reworked, the
mixed builtin/BPF pricing path will go away so a model fully owns
pricing on a device it's attached to (including flushes), the
per-cgroup state handling is being reworked, operation flags will
be preserved, and sleepable models will be rejected during
verification.
The Kconfig dependency, the example model issues, the selftest
problems, and the patch organization will all be addressed as well.
I'll post v2 once it's ready.
Thanks,
Tao
> Thanks.
>
^ permalink raw reply [flat|nested] 16+ messages in thread