Re: [PATCH 0/6] sched: Expose writable interface for preferred CPUs mask

[Yury Norov <ynorov@nvidia.com>]

On Thu, Nov 05, 2026 at 03:55:37PM +0100, Peter Zijlstra wrote:
> On Tue, Nov 03, 2026 at 11:49:02AM -0500, Yury Norov wrote:
> > On Mon, Nov 02, 2026 at 09:47:04AM +0100, Peter Zijlstra wrote:
> > > On Sun, Nov 01, 2026 at 02:21:56PM -0500, Yury Norov wrote:
> > > > The preferred CPU mask currently lets the steal governor reduce vCPU
> > > > demand when physical CPUs are contended. This series adds manual control,
> > > > opening more use cases.
> > > 
> > > Sceptical, I am.
> > > 
> > > > For example, setting CPUs aside for selected fair tasks at runtime while
> > > > steering other fair tasks toward the remaining CPUs.
> > > 
> > > That's exactly the kind of nonsense we don't need and makes me want to
> > > stop reading right here.
> > 
> > OK, I started with the wrong argument. Please consider this as the argument
> > #3.
> > 
> > #1 is: conceptually, preferred cpumask improves performance because it
> > moves the scheduling decision from host to the guest level. The guest
> > can do it better because it knows more about the workload. But then we'd
> > consider moving steal governor logic to the guest userspace, because it
> > knows the nature of user workload even more.
> 
> I am confused; it is a simple control loop and has nothing to do with
> guest workload.
> 
> The host is over-committed, this results in vcpu steal time. The guests
> observe steal time and reduce vcpu utilization.
> 
> It is a co-operative scheme; a bad actor can game the system. It is
> imperative all guests behave the same or do not play at all.
> 
> If there is no level of trust (our coconut friends) do not use this.

That is correct. I'm just trying to let trusted userspace to be able to
make a final decision, not to cheat the policy.

I believe that the preferred cpumask approach may find more use cases
than just a blind vCPU throttling.

Even in case of throttling, the userspace might know more about the
nature of external pressure, workload and VM priorities, etc., so be
able to make a better decision on the preferred vCPU candidates,
beyond the steal time accounting.

I can imagine a host running two VMs - one for performance critical
workload like AI training, and another for latency critical inference. 
In case of vCPU overcommitment, symmetrical throttling of both VMs
may substantially affect user experience. The better solution would be
allowing both VMs to communicate to each other and decide how many
vCPUs should become non-preferred on each side.

Steal governor is a reasonable default approach to the vCPU overcommit
problem. But I also want to be able to adjust it to my local needs, if
I have to.
 
> > #2: convenience and scope. Some systems don't allow loading modules -
> > either on config level, through selinux, or somehow else. They don't
> > even have compiler installed sometimes. 
> 
> You hurt if you shoot your foot off; not my problem. If you run guests
> like that you don't use this.

I don't configure systems that way - I only request shared access to the
particular hardware, i.e. to run some tests, often for just a few hours.
Sometimes those machines don't allow loading modules, sometimes it takes
few iterations to reach to admins and ask them to update drivers...

> > > we already have cpusets that is more than capable of doing this.
> > 
> > Then why do we need the steal governor at all? We can steer all
> > processes away from a given vCPU without any driver, only by playing
> > with each individual process' affinity. For vCPU throttling, task
> > isolation, or whatever purpose.
> 
> Different level of operation; different semantics. Cpusets is for strong
> bounds on your own workloads, this preferred mask is weak bounds on the
> system due to outside pressure.
> 
> In a way it isn't much different from ACPI idle injection.

The idle-injection indeed has userspace interface. Powerclamp exposes
its actuator through /sys/class/thermal/cooling_deviceXX, and documents
userspace control loop.

By the way, preferred CPUs approach may prepend hotplug in the overheat
defense chain, as a cheap but weaker alternative.

Thanks,
Yury