mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Jinjie Ruan <ruanjinjie@huawei.com>
To: Marc Zyngier <maz@kernel.org>
Cc: <tglx@kernel.org>, <radu@rendec.net>, <jason@lakedaemon.net>,
	<linux-arm-kernel@lists.infradead.org>,
	<linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] irqchip/gic-v3-its: Size the collection table for all possible CPUs
Date: Mon, 16 Nov 2026 21:07:37 +0800	[thread overview]
Message-ID: <8c02e6f7-cc21-42d2-98e0-d5818d10c083@huawei.com> (raw)
In-Reply-To: <86jymd159v.wl-maz@kernel.org>



在 2026/11/16 17:42, Marc Zyngier 写道:
> On Mon, 16 Nov 2026 02:23:30 +0000,
> Jinjie Ruan <ruanjinjie@huawei.com> wrote:
>>
>> The ITS collection table is sized from the hardware default described by
>> GITS_BASER: a single ITS page holding entry-size objects.  With the usual
>> 16-byte collection entry and a 4K ITS page that is 256 collections.
>>
>> its_cpu_init_collections() installs one collection per CPU using MAPC with
>> the CPU number as the collection ID, so on a system with more than 256
>> possible CPUs the MAPC commands for the remaining CPUs fail with
>> E_ITS_MAPC_COLLECTION_OOR (0x010903) and their redistributors end up
>> without a valid collection.  LPIs targeted at those CPUs, which is where
>> managed MSI/MSI-X affinity places a large part of the device interrupts,
>> are then dropped by the hardware: the DMA completes but the completion
>> interrupt never arrives, so drivers fall back to polling and time out.
>>
>> On a 384-CPU HiSilicon HIP12 system running v7.4-rc2 every ITS reported
>>
>>   ITS@0x488000000: allocated 256 Interrupt Collections (flat, esz 16, psz 4K, shr 1)
>>
>> and all four NVMe controllers logged repeated
>>
>>   nvme nvme0: I/O tag 256 (1100) QID 182 timeout, completion polled
>>
>> while the hinic driver failed to probe with -EIO.  With the table sized
>> for num_possible_cpus() it holds 512 entries and both symptoms are gone.
> 
> Please drop all these examples, as they serve absolutely no purpose.
> Everybody understands that losing interrupts is deadly. The whole
> thing is also filled with implementation details that are not really
> relevant, and can be described in a more general way.
> 
>>
>> Handle GITS_BASER_TYPE_COLLECTION in its_alloc_tables() the same way as
>> the device and VCPU tables, requesting order_base_2(num_possible_cpus())
>> IDs.
> 
> Just say something like:
> 
> "The collection table is currently sized at a single ITS page size,
>  and the driver's allocation policy is to allocate a single collection
>  per CPU. On systems with more CPUs that the ITS can fit in a single
>  page, no LPI can be routed to the CPUs that don't have an allocated
>  collection, and the system grinds to a halt.
> 
>  Change the allocation policy to allow two-level tables, similarly to
>  the other tables."
> 
>>
>> Fixes: 1ac19ca6bf97 ("irqchip: GICv3: ITS: tables allocators")
>> Cc: stable@vger.kernel.org
> 
> This is not a fix. This is new feature support for implementation that
> are both very wide and have a stupidly small ITS page size. These

Hi Marc,

Thanks for the review. It is indeed the case.

> systems have *never* worked, and therefore there is nothing to fix.

Yes, this type machine has indeed not been run before, will remove the
fix tag.

> 
>> Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
>> ---
>>  drivers/irqchip/irq-gic-v3-its.c | 15 +++++++++++++++
>>  1 file changed, 15 insertions(+)
>>
>> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
>> index c952c069f937..319b6edefd37 100644
>> --- a/drivers/irqchip/irq-gic-v3-its.c
>> +++ b/drivers/irqchip/irq-gic-v3-its.c
>> @@ -2716,6 +2716,21 @@ static int its_alloc_tables(struct its_node *its)
>>  			indirect = its_parse_indirect_baser(its, baser, &order,
>>  							    ITS_MAX_VPEID_BITS);
>>  			break;
>> +
>> +		case GITS_BASER_TYPE_COLLECTION:
>> +			/*
>> +			 * Size the collection table for every possible CPU.
>> +			 * One collection per CPU is installed via MAPC with
>> +			 * col_id == cpu, so a table sized only for the
>> +			 * hardware default page (e.g. 256 entries with
>> +			 * esz 16 / psz 4K) is too small on systems with more
>> +			 * CPUs than that: MAPC for the remaining CPUs fails
>> +			 * with E_ITS_MAPC_COLLECTION_OOR and LPIs targeted at
>> +			 * those CPUs are never delivered.
>> +			 */
> 
> This comment serves no purpose, and the code says everything that
> needs to be known.

OK, will delete the comment.

> 
>> +			indirect = its_parse_indirect_baser(its, baser, &order,
>> +							    order_base_2(num_possible_cpus()));
>> +			break;
>>  		}
>>  
>>  		err = its_setup_baser(its, baser, cache, shr, order, indirect);
> 
> Finally, it really looks like we could do better, since *all* table
> types are now able to use two-level structures. I'd like to see
> something like the below (untested).

The cleanup looks fine to me, will update the code and retest it.

Thanks again!

Best regards,
Jinjie

> 
> 	M.
> 
> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
> index 282e4bafb084f..a058cb8373ef1 100644
> --- a/drivers/irqchip/irq-gic-v3-its.c
> +++ b/drivers/irqchip/irq-gic-v3-its.c
> @@ -2682,7 +2682,7 @@ static int its_alloc_tables(struct its_node *its)
>  		u64 val = its_read_baser(its, baser);
>  		u64 type = GITS_BASER_TYPE(val);
>  		bool indirect = false;
> -		u32 order;
> +		u32 order, nr_ids;
>  
>  		if (type == GITS_BASER_TYPE_NONE)
>  			continue;
> @@ -2696,8 +2696,11 @@ static int its_alloc_tables(struct its_node *its)
>  
>  		switch (type) {
>  		case GITS_BASER_TYPE_DEVICE:
> -			indirect = its_parse_indirect_baser(its, baser, &order,
> -							    device_ids(its));
> +			nr_ids = device_ids(its);
> +			break;
> +
> +		case GITS_BASER_TYPE_COLLECTION:
> +			nr_ids = order_base_2(num_possible_cpus());
>  			break;
>  
>  		case GITS_BASER_TYPE_VCPU:
> @@ -2713,11 +2716,11 @@ static int its_alloc_tables(struct its_node *its)
>  				}
>  			}
>  
> -			indirect = its_parse_indirect_baser(its, baser, &order,
> -							    ITS_MAX_VPEID_BITS);
> +			nr_ids = ITS_MAX_VPEID_BITS;
>  			break;
>  		}
>  
> +		indirect = its_parse_indirect_baser(its, baser, &order, nr_ids);
>  		err = its_setup_baser(its, baser, cache, shr, order, indirect);
>  		if (err < 0) {
>  			its_free_tables(its);
> 


      reply	other threads:[~2026-11-16 13:07 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16  2:23 Jinjie Ruan
2026-11-16  9:42 ` Marc Zyngier
2026-11-16 13:07   ` Jinjie Ruan [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=8c02e6f7-cc21-42d2-98e0-d5818d10c083@huawei.com \
    --to=ruanjinjie@huawei.com \
    --cc=jason@lakedaemon.net \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maz@kernel.org \
    --cc=radu@rendec.net \
    --cc=tglx@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®