Re: [PATCH RFC v3 2/3] KVM: x86: add KVM_CAP_CSTATE_POLICY for per-VM C-state enforcement

[Sean Christopherson <seanjc@google.com>]

On Mon, Sep 14, 2026, Anthony Harivel wrote:
> Add a new VM-scoped capability that allows userspace to set a maximum
> C-state ceiling for host cpuidle when vCPUs halt.
> 
> When a vCPU enters kvm_vcpu_block(), KVM temporarily disables cpuidle
> states deeper than max_cstate on the current pCPU using the existing
> states_usage[].disable mechanism (CPUIDLE_STATE_DISABLED_BY_DRIVER).
> After wakeup, the original disable flags are restored.
> 
> The capability follows the same pattern as KVM_CAP_HALT_POLL:
>   - VM-scoped ioctl via KVM_ENABLE_CAP
>   - args[0] = max_cstate (-1 to 6, -1 disables the policy)
>   - Re-callable at runtime without VM restart
>   - Memory ordering via smp_wmb/rmb
> 
> This fills an operational gap for NFV and latency-sensitive deployments
> where the host operator needs per-VM control over idle depth without
> requiring guest cooperation. The enforcement is scoped to pinned-core
> configurations where the disable flags do not race with other tasks.

Sorry, NAK, this doesn't belong in KVM.  Given that the only way this can work
is if vCPU are pinned 1:1 to pCPUs, then it should be very doable for the cpuidle
subystem to provide an interface to let (privileged?) userspace restrict the
maximum C-state on a per-CPU basis.

My apologies for not responding to v1 or v2, I am guilty of Jim's axiom that
upstream doesn't respond to RFCs without code.

Pulling in the other options here:

+    The enforcement point is kvm_vcpu_halt() → kvm_vcpu_block() →
+    schedule() → cpuidle. The problem: cpuidle has no per-task
+    C-state constraint. states_usage[].disable is per-CPU, and
+    forced_idle_latency_limit_ns is also per-CPU.

I don't understand why per-CPU controls are a bad thing.  A task-based scheme can
really only work if vCPUs are pinned to pCPUs, i.e. you effectively need per-CPU
controls anyways.  And explicit per-CPU controls would allow for more relaxed
scheduling too, e.g. would allow affining vCPUs to a set of pCPUs without needing
to have strict 1:1 pinning.

+
+    Three options I see:
+
+      Option A: Temporarily toggle states_usage[i].disable on the
+      pinned pCPU before/after kvm_vcpu_block(). Set
+      CPUIDLE_STATE_DISABLED_BY_DRIVER for states > max_cstate,
+      restore after wakeup. Simple, works with existing API, but
+      only correct with dedicated pinning — overcommit with mixed
+      policies would race on the disable flags.
+
+      Option B: Use forced_idle_latency_limit_ns on the pCPU.
+      Same per-CPU limitation, and latency-based rather than
+      state-index-based — less precise.

Conceptually, (b) seems like the right approach.  Per-task will be a mess because
similar to a KVM-based interface, it can probably only work if tasks are pinned
to pCPUs.

And isn't abstracting away the exact C-state via forced_idle_latency_limit_ns a
*good* thing?  Without that, userspace will need to tune its configuration for
each individual uarch based on the properties of various C-states for a given CPU.

+
+      Option C: Propose a new cpuidle API for per-task idle
+      constraints (e.g. a per-task_struct annotation checked by
+      the governor during select()). Correct for all cases, but
+      bigger scope and needs cpuidle maintainer buy-in.