Re: Real-time scheduling policies and hyper-threading

[Peter Zijlstra <peterz@infradead.org>]

On Fri, Apr 25, 2014 at 03:04:49PM +0400, Roman Gushchin wrote:
> 24.04.2014, 22:59, "Peter Zijlstra" <peterz@infradead.org>:
> > On Thu, Apr 24, 2014 at 10:16:12PM +0400, Roman Gushchin wrote:
> >
> >>  Are there any known solutions of this problem except disabling
> >>  hyper-threading and frequency scaling at all?
> >
> > This is what we generally tell people to do; disable HT in the BIOS,
> > offline the siblings or similar approaches.
> 
> It's a good, but expensive approach, if you have many machines.

Expensive how? If you require the latency, there's really no option.

That said, if only a small part of your workload is RT then you don't
need to disable all SMT siblings, you can only disable the few that are
in your RT partition.

> > You, and only you, know your workload and can devise a correct placement
> > policy. We have cpusets and cpu affinity available to carve up your
> > system for this.
> 
> The problem here is that _all_ userspace tasks should care about cpu placement.
> In reality, there is always a number of monitoring/administration/system tasks,
> which execution increases latencies. Use of nice/SCHED_BATCH/... doesn't solve
> the problem, because of hyper-threading.

The thing is, normal tasks don't care and its perfectly fine to have
heuristics that work more or less most of the time.

But realtime does care, it must absolutely always work in a predictable
fashion.

> Of course, it's possible to divide cpus statically via cpusets/partitioning,
> but it's also not cheap in terms of overall performance utilization.

Well, realtime isn't about getting full utilization from your hardware
to begin with. Its about getting deterministic behaviour from your
hardware.

If your workload allows both, then yay, but its absolutely not true in
general.

> Probably, it's important, that CPU load in our case is never 100% for a period 
> longer than few hundreds milliseconds.

Yeah, that's not a problem. Pegging the CPU for 10s of seconds and
upwards will get funny real quick though.