From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 510A8C433FE for ; Wed, 9 Nov 2022 17:19:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231548AbiKIRTP (ORCPT ); Wed, 9 Nov 2022 12:19:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229590AbiKIRTN (ORCPT ); Wed, 9 Nov 2022 12:19:13 -0500 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 2C2EA25C8 for ; Wed, 9 Nov 2022 09:19:12 -0800 (PST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 004BD1FB; Wed, 9 Nov 2022 09:19:18 -0800 (PST) Received: from [10.1.197.38] (eglon.cambridge.arm.com [10.1.197.38]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F2DDD3F703; Wed, 9 Nov 2022 09:19:08 -0800 (PST) Message-ID: <54bbb5b1-6d04-6d45-d6c9-45e714928cf4@arm.com> Date: Wed, 9 Nov 2022 17:18:59 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux aarch64; rv:91.0) Gecko/20100101 Thunderbird/91.13.0 Subject: Re: [PATCH 18/18] x86/resctrl: Separate arch and fs resctrl locks Content-Language: en-GB To: Peter Newman Cc: x86@kernel.org, linux-kernel@vger.kernel.org, Fenghua Yu , Reinette Chatre , Thomas Gleixner , Ingo Molnar , Borislav Petkov , H Peter Anvin , Babu Moger , shameerali.kolothum.thodi@huawei.com, D Scott Phillips OS , carl@os.amperecomputing.com, lcherian@marvell.com, bobo.shaobowang@huawei.com, tan.shaopeng@fujitsu.com, Jamie Iles , Xin Hao , xingxin.hx@openanolis.org, baolin.wang@linux.alibaba.com References: <20221021131204.5581-1-james.morse@arm.com> <20221021131204.5581-19-james.morse@arm.com> From: James Morse In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Peter, On 31/10/2022 14:21, Peter Newman wrote: > On Fri, Oct 21, 2022 at 3:13 PM James Morse wrote: >> >> MPAM's monitors have an overflow interrupt, so it needs to be possible >> to walk the domains list in irq context. RCU is ideal for this, >> but some paths need to be able to sleep to allocate memory. > I'm curious about this requirement. There are already counters which can > overflow on Intel, but we've been able to detect overflows soon enough > by checking at a reasonable interval. Are we expecting MSCs to have > counters that overflow so quickly that the overflows need to be handled > directly in IRQ context vs being able to run a threaded handler before > the second overflow? Ultimately, I don't know. MPAM has three sizes of counter, 31, 44 and 63. I think its entirely possible someone builds a system with an inconvenient size for the way they use it - but this wasn't how I anticipated this getting used... > It seems like MBM would be really intrusive if it could cause the system > to process overflow IRQs at a high rate. ... I agree .. > Also is the overflow interrupt handler in one of your MPAM preview > branches? I was only able to find an error IRQ handler in > mpam/snapshot/v6.0: > > https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git/tree/drivers/platform/mpam/mpam_devices.c?h=mpam/snapshot/v6.0#n1813 Because there probably won't be enough monitors to expose the free-running resctrl files, I anticipate that most use of the memory bandwidth counters will be via perf, which gives MPAM the start/stop calls it needs to allocate and free a monitor. The PMU driver gets asked to read the counters in IRQ context, see __perf_event_read() called via smp_call_function_single(). (I'm sure there are others). This is the first reason why the domain list needs to be protected by something like RCU. Perf also has a sampling mode, where it sets the value to overflow after a specific number of events, and times how long that takes to occur. (I haven't completely got my head round it yet) In this mode, the MPAM driver would need to invoke the PMU driver to read the counters in IRQ context. I haven't written the overflow interrupt code yet because I've got no access to a platform (virtual or otherwise) with working counters, so couldn't possibly test it. (See also, the 44 and 63 bit counter support!) I was being lazy with the commit message and only describing the last case. Is the future-plot-arc important? It would result in a bit of a brain-dump/essay in the commit message. Thanks, James