From: Jinjie Ruan <ruanjinjie@huawei.com>
To: Marc Zyngier <maz@kernel.org>
Cc: <tglx@kernel.org>, <radu@rendec.net>, <jason@lakedaemon.net>,
<linux-arm-kernel@lists.infradead.org>,
<linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] irqchip/gic-v3-its: Size the collection table for all possible CPUs
Date: Mon, 16 Nov 2026 21:07:37 +0800 [thread overview]
Message-ID: <8c02e6f7-cc21-42d2-98e0-d5818d10c083@huawei.com> (raw)
In-Reply-To: <86jymd159v.wl-maz@kernel.org>
在 2026/11/16 17:42, Marc Zyngier 写道:
> On Mon, 16 Nov 2026 02:23:30 +0000,
> Jinjie Ruan <ruanjinjie@huawei.com> wrote:
>>
>> The ITS collection table is sized from the hardware default described by
>> GITS_BASER: a single ITS page holding entry-size objects. With the usual
>> 16-byte collection entry and a 4K ITS page that is 256 collections.
>>
>> its_cpu_init_collections() installs one collection per CPU using MAPC with
>> the CPU number as the collection ID, so on a system with more than 256
>> possible CPUs the MAPC commands for the remaining CPUs fail with
>> E_ITS_MAPC_COLLECTION_OOR (0x010903) and their redistributors end up
>> without a valid collection. LPIs targeted at those CPUs, which is where
>> managed MSI/MSI-X affinity places a large part of the device interrupts,
>> are then dropped by the hardware: the DMA completes but the completion
>> interrupt never arrives, so drivers fall back to polling and time out.
>>
>> On a 384-CPU HiSilicon HIP12 system running v7.4-rc2 every ITS reported
>>
>> ITS@0x488000000: allocated 256 Interrupt Collections (flat, esz 16, psz 4K, shr 1)
>>
>> and all four NVMe controllers logged repeated
>>
>> nvme nvme0: I/O tag 256 (1100) QID 182 timeout, completion polled
>>
>> while the hinic driver failed to probe with -EIO. With the table sized
>> for num_possible_cpus() it holds 512 entries and both symptoms are gone.
>
> Please drop all these examples, as they serve absolutely no purpose.
> Everybody understands that losing interrupts is deadly. The whole
> thing is also filled with implementation details that are not really
> relevant, and can be described in a more general way.
>
>>
>> Handle GITS_BASER_TYPE_COLLECTION in its_alloc_tables() the same way as
>> the device and VCPU tables, requesting order_base_2(num_possible_cpus())
>> IDs.
>
> Just say something like:
>
> "The collection table is currently sized at a single ITS page size,
> and the driver's allocation policy is to allocate a single collection
> per CPU. On systems with more CPUs that the ITS can fit in a single
> page, no LPI can be routed to the CPUs that don't have an allocated
> collection, and the system grinds to a halt.
>
> Change the allocation policy to allow two-level tables, similarly to
> the other tables."
>
>>
>> Fixes: 1ac19ca6bf97 ("irqchip: GICv3: ITS: tables allocators")
>> Cc: stable@vger.kernel.org
>
> This is not a fix. This is new feature support for implementation that
> are both very wide and have a stupidly small ITS page size. These
Hi Marc,
Thanks for the review. It is indeed the case.
> systems have *never* worked, and therefore there is nothing to fix.
Yes, this type machine has indeed not been run before, will remove the
fix tag.
>
>> Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
>> ---
>> drivers/irqchip/irq-gic-v3-its.c | 15 +++++++++++++++
>> 1 file changed, 15 insertions(+)
>>
>> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
>> index c952c069f937..319b6edefd37 100644
>> --- a/drivers/irqchip/irq-gic-v3-its.c
>> +++ b/drivers/irqchip/irq-gic-v3-its.c
>> @@ -2716,6 +2716,21 @@ static int its_alloc_tables(struct its_node *its)
>> indirect = its_parse_indirect_baser(its, baser, &order,
>> ITS_MAX_VPEID_BITS);
>> break;
>> +
>> + case GITS_BASER_TYPE_COLLECTION:
>> + /*
>> + * Size the collection table for every possible CPU.
>> + * One collection per CPU is installed via MAPC with
>> + * col_id == cpu, so a table sized only for the
>> + * hardware default page (e.g. 256 entries with
>> + * esz 16 / psz 4K) is too small on systems with more
>> + * CPUs than that: MAPC for the remaining CPUs fails
>> + * with E_ITS_MAPC_COLLECTION_OOR and LPIs targeted at
>> + * those CPUs are never delivered.
>> + */
>
> This comment serves no purpose, and the code says everything that
> needs to be known.
OK, will delete the comment.
>
>> + indirect = its_parse_indirect_baser(its, baser, &order,
>> + order_base_2(num_possible_cpus()));
>> + break;
>> }
>>
>> err = its_setup_baser(its, baser, cache, shr, order, indirect);
>
> Finally, it really looks like we could do better, since *all* table
> types are now able to use two-level structures. I'd like to see
> something like the below (untested).
The cleanup looks fine to me, will update the code and retest it.
Thanks again!
Best regards,
Jinjie
>
> M.
>
> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
> index 282e4bafb084f..a058cb8373ef1 100644
> --- a/drivers/irqchip/irq-gic-v3-its.c
> +++ b/drivers/irqchip/irq-gic-v3-its.c
> @@ -2682,7 +2682,7 @@ static int its_alloc_tables(struct its_node *its)
> u64 val = its_read_baser(its, baser);
> u64 type = GITS_BASER_TYPE(val);
> bool indirect = false;
> - u32 order;
> + u32 order, nr_ids;
>
> if (type == GITS_BASER_TYPE_NONE)
> continue;
> @@ -2696,8 +2696,11 @@ static int its_alloc_tables(struct its_node *its)
>
> switch (type) {
> case GITS_BASER_TYPE_DEVICE:
> - indirect = its_parse_indirect_baser(its, baser, &order,
> - device_ids(its));
> + nr_ids = device_ids(its);
> + break;
> +
> + case GITS_BASER_TYPE_COLLECTION:
> + nr_ids = order_base_2(num_possible_cpus());
> break;
>
> case GITS_BASER_TYPE_VCPU:
> @@ -2713,11 +2716,11 @@ static int its_alloc_tables(struct its_node *its)
> }
> }
>
> - indirect = its_parse_indirect_baser(its, baser, &order,
> - ITS_MAX_VPEID_BITS);
> + nr_ids = ITS_MAX_VPEID_BITS;
> break;
> }
>
> + indirect = its_parse_indirect_baser(its, baser, &order, nr_ids);
> err = its_setup_baser(its, baser, cache, shr, order, indirect);
> if (err < 0) {
> its_free_tables(its);
>
prev parent reply other threads:[~2026-11-16 13:07 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-16 2:23 Jinjie Ruan
2026-11-16 9:42 ` Marc Zyngier
2026-11-16 13:07 ` Jinjie Ruan [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8c02e6f7-cc21-42d2-98e0-d5818d10c083@huawei.com \
--to=ruanjinjie@huawei.com \
--cc=jason@lakedaemon.net \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=maz@kernel.org \
--cc=radu@rendec.net \
--cc=tglx@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®