Re: [PATCH 18/18] x86/resctrl: Separate arch and fs resctrl locks

[James Morse <james.morse@arm.com>]

Hi Peter,

On 31/10/2022 14:21, Peter Newman wrote:
> On Fri, Oct 21, 2022 at 3:13 PM James Morse <james.morse@arm.com> wrote:
>>
>> MPAM's monitors have an overflow interrupt, so it needs to be possible
>> to walk the domains list in irq context. RCU is ideal for this,
>> but some paths need to be able to sleep to allocate memory.

> I'm curious about this requirement. There are already counters which can
> overflow on Intel, but we've been able to detect overflows soon enough
> by checking at a reasonable interval. Are we expecting MSCs to have
> counters that overflow so quickly that the overflows need to be handled
> directly in IRQ context vs being able to run a threaded handler before
> the second overflow?

Ultimately, I don't know. MPAM has three sizes of counter, 31, 44 and 63.
I think its entirely possible someone builds a system with an inconvenient size for the
way they use it - but this wasn't how I anticipated this getting used...


> It seems like MBM would be really intrusive if it could cause the system
> to process overflow IRQs at a high rate.

... I agree ..


> Also is the overflow interrupt handler in one of your MPAM preview
> branches? I was only able to find an error IRQ handler in
> mpam/snapshot/v6.0:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers/platform/mpam/mpam_devices.c?h=mpam/snapshot/v6.0#n1813

Because there probably won't be enough monitors to expose the free-running resctrl files,
I anticipate that most use of the memory bandwidth counters will be via perf, which gives
MPAM the start/stop calls it needs to allocate and free a monitor.

The PMU driver gets asked to read the counters in IRQ context, see __perf_event_read()
called via smp_call_function_single(). (I'm sure there are others).

This is the first reason why the domain list needs to be protected by something like RCU.

Perf also has a sampling mode, where it sets the value to overflow after a specific number
of events, and times how long that takes to occur. (I haven't completely got my head round
it yet) In this mode, the MPAM driver would need to invoke the PMU driver to read the
counters in IRQ context.

I haven't written the overflow  interrupt code yet because I've got no access to a
platform (virtual or otherwise) with working counters, so couldn't possibly test it. (See
also, the 44 and 63 bit counter support!)

I was being lazy with the commit message and only describing the last case. Is the
future-plot-arc important? It would result in a bit of a brain-dump/essay in the commit
message.


Thanks,

James