From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794822159; b=G9zzJYvM/+jw89HHFTNq2QaFe1IHLiKOJXlVMKpFuuHrXgUy3D2MsuZyHHFiZcENLF3I TnttXgQjtV2L/eK81p7nr2hgZ9InugE1e9VCp/h3YnCSd/B5wa5TvJkltzgZvhozbzCm8 HFhFZRR3spgxvptvASltLlbFeWt6Xc3WUM/P/FzrN3LxzI0Ng6qn7JG9/gsX/c/m/Zuhe 0TTC0h4XV7I4BZUDDo6oq3d7bABAzrj/UOC68PJFS19e0pY9e0TwY4JvE0/S5CqrJATvY wK1qoWMZHzOqXBu/pCniB4w/z+/92jDSqDrVVWQcDiUo+z3GW6S/7kT37XtJoXJTbPA== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794822159; h=Received:DKIM-Signature:Received:Date:Message-ID:From:To:Cc:Subject: In-Reply-To:References:User-Agent:MIME-Version:Content-Type: X-SA-Exim-Connect-IP:X-SA-Exim-Rcpt-To:X-SA-Exim-Mail-From: X-SA-Exim-Scanned; bh=2jMiNtLEwivAxWblU1/xPh4v0Zlz45fkFKl4d1hiyhw=; b=xpivw5l0vht2g7urcZEL7U1Qxm7QK/TUpEd9O0omWlZ6NhBPtIMX6VZLGhhPOhvh7fGK gqUpehg28WiL+3kBQVLSIukJTiNiCwZDyhiErkQecSQrdE9ShHeP3m2zWaMUwk3OnvK6o Q4I8s8HCX0NJPJzQUjkVOjLpu+gHP/7cc3U8UAZOhbffI9lV3PLZxTEFBdib27zI6rZ/8 MCI0Sbag4pN10oxCt0iVNOGlwpodYMnmf4/F3TWvcRRTBFmjr8Xuf9QPWAz34gJCTPweF 15Wf/4rpoYlS7B9RyQrvO1a5wwl+hww5PwWmc4xZTXQhaaFDTril7f4WD44BVEE0WCQ== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=BeOdCh5w; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60184606A323 for ; Mon, 16 Nov 2026 09:42:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key, unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=BeOdCh5w Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3C3051F000C7; Mon, 16 Nov 2026 09:42:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1794822159; bh=2jMiNtLEwivAxWblU1/xPh4v0Zlz45fkFKl4d1hiyhw=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=BeOdCh5wi/T+U1UT1RyK41HnGEiY547WWZs1wrDmMnMFs+lkszYw6NpE49bhMvi8k 8AjDHbH5khCcDsrPZMZhEOTS5F1N0e5QGolzfToIA3+VE3RDil+Bge3+ckcPOsgNUI Z68h5ifBornjk96u9jzuNY/OcYHwv0S9FBAgd//q2pTfy+yVV2tg/JPb240dyERzAz 5U40FSBPQKC+JOJuyu+y0xwiw4awyQD2xSIpU9UlMvxFeNBm5rGt+hZsMT7dThqpS1 JUrHxa6BifZb9MFQCT4QonEh1AV5PPKrMsf989RUg8AUJrKeGkbxsk5Vha+/5Y0TlB EYQ1H+EBD7ycg== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xStES-0000000GLIy-46Om; Mon, 16 Nov 2026 09:42:37 +0000 Date: Mon, 16 Nov 2026 09:42:36 +0000 Message-ID: <86jymd159v.wl-maz@kernel.org> From: Marc Zyngier To: Jinjie Ruan Cc: , , , , Subject: Re: [PATCH] irqchip/gic-v3-its: Size the collection table for all possible CPUs In-Reply-To: <20261116022330.273840-1-ruanjinjie@huawei.com> References: <20261116022330.273840-1-ruanjinjie@huawei.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: ruanjinjie@huawei.com, tglx@kernel.org, radu@rendec.net, jason@lakedaemon.net, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Mon, 16 Nov 2026 02:23:30 +0000, Jinjie Ruan wrote: > > The ITS collection table is sized from the hardware default described by > GITS_BASER: a single ITS page holding entry-size objects. With the usual > 16-byte collection entry and a 4K ITS page that is 256 collections. > > its_cpu_init_collections() installs one collection per CPU using MAPC with > the CPU number as the collection ID, so on a system with more than 256 > possible CPUs the MAPC commands for the remaining CPUs fail with > E_ITS_MAPC_COLLECTION_OOR (0x010903) and their redistributors end up > without a valid collection. LPIs targeted at those CPUs, which is where > managed MSI/MSI-X affinity places a large part of the device interrupts, > are then dropped by the hardware: the DMA completes but the completion > interrupt never arrives, so drivers fall back to polling and time out. > > On a 384-CPU HiSilicon HIP12 system running v7.4-rc2 every ITS reported > > ITS@0x488000000: allocated 256 Interrupt Collections (flat, esz 16, psz 4K, shr 1) > > and all four NVMe controllers logged repeated > > nvme nvme0: I/O tag 256 (1100) QID 182 timeout, completion polled > > while the hinic driver failed to probe with -EIO. With the table sized > for num_possible_cpus() it holds 512 entries and both symptoms are gone. Please drop all these examples, as they serve absolutely no purpose. Everybody understands that losing interrupts is deadly. The whole thing is also filled with implementation details that are not really relevant, and can be described in a more general way. > > Handle GITS_BASER_TYPE_COLLECTION in its_alloc_tables() the same way as > the device and VCPU tables, requesting order_base_2(num_possible_cpus()) > IDs. Just say something like: "The collection table is currently sized at a single ITS page size, and the driver's allocation policy is to allocate a single collection per CPU. On systems with more CPUs that the ITS can fit in a single page, no LPI can be routed to the CPUs that don't have an allocated collection, and the system grinds to a halt. Change the allocation policy to allow two-level tables, similarly to the other tables." > > Fixes: 1ac19ca6bf97 ("irqchip: GICv3: ITS: tables allocators") > Cc: stable@vger.kernel.org This is not a fix. This is new feature support for implementation that are both very wide and have a stupidly small ITS page size. These systems have *never* worked, and therefore there is nothing to fix. > Signed-off-by: Jinjie Ruan > --- > drivers/irqchip/irq-gic-v3-its.c | 15 +++++++++++++++ > 1 file changed, 15 insertions(+) > > diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c > index c952c069f937..319b6edefd37 100644 > --- a/drivers/irqchip/irq-gic-v3-its.c > +++ b/drivers/irqchip/irq-gic-v3-its.c > @@ -2716,6 +2716,21 @@ static int its_alloc_tables(struct its_node *its) > indirect = its_parse_indirect_baser(its, baser, &order, > ITS_MAX_VPEID_BITS); > break; > + > + case GITS_BASER_TYPE_COLLECTION: > + /* > + * Size the collection table for every possible CPU. > + * One collection per CPU is installed via MAPC with > + * col_id == cpu, so a table sized only for the > + * hardware default page (e.g. 256 entries with > + * esz 16 / psz 4K) is too small on systems with more > + * CPUs than that: MAPC for the remaining CPUs fails > + * with E_ITS_MAPC_COLLECTION_OOR and LPIs targeted at > + * those CPUs are never delivered. > + */ This comment serves no purpose, and the code says everything that needs to be known. > + indirect = its_parse_indirect_baser(its, baser, &order, > + order_base_2(num_possible_cpus())); > + break; > } > > err = its_setup_baser(its, baser, cache, shr, order, indirect); Finally, it really looks like we could do better, since *all* table types are now able to use two-level structures. I'd like to see something like the below (untested). M. diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c index 282e4bafb084f..a058cb8373ef1 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -2682,7 +2682,7 @@ static int its_alloc_tables(struct its_node *its) u64 val = its_read_baser(its, baser); u64 type = GITS_BASER_TYPE(val); bool indirect = false; - u32 order; + u32 order, nr_ids; if (type == GITS_BASER_TYPE_NONE) continue; @@ -2696,8 +2696,11 @@ static int its_alloc_tables(struct its_node *its) switch (type) { case GITS_BASER_TYPE_DEVICE: - indirect = its_parse_indirect_baser(its, baser, &order, - device_ids(its)); + nr_ids = device_ids(its); + break; + + case GITS_BASER_TYPE_COLLECTION: + nr_ids = order_base_2(num_possible_cpus()); break; case GITS_BASER_TYPE_VCPU: @@ -2713,11 +2716,11 @@ static int its_alloc_tables(struct its_node *its) } } - indirect = its_parse_indirect_baser(its, baser, &order, - ITS_MAX_VPEID_BITS); + nr_ids = ITS_MAX_VPEID_BITS; break; } + indirect = its_parse_indirect_baser(its, baser, &order, nr_ids); err = its_setup_baser(its, baser, cache, shr, order, indirect); if (err < 0) { its_free_tables(its); -- Without deviation from the norm, progress is not possible.