mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Shanker Donthineni <sdonthineni@nvidia.com>
To: Thomas Gleixner <tglx@linutronix.de>, Marc Zyngier <maz@kernel.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
	Michael Walle <michael@walle.cc>,
	Shanker Donthineni <sdonthineni@nvidia.com>,
	<linux-kernel@vger.kernel.org>, Vikram Sethi <vsethi@nvidia.com>
Subject: [PATCH v3 0/3] Increase the number of IRQ descriptors for SPARSEIRQ
Date: Mon, 10 Apr 2023 10:57:18 -0500	[thread overview]
Message-ID: <20230410155721.3720991-1-sdonthineni@nvidia.com> (raw)

The ARM64 architecture uses SPARSEIRQ with a default value of NR_IRQS,
which is set to 64. This means that only 64+8192 IRQ descriptors are
allowed, which may not be sufficient for modern ARM64 servers that
have a large number of IO devices and GIC hardware that supports
direct vSGI and vLPI injection features.

This limitation has caused issues when attempting to launch multiple
virtual machines with GICv4.1 features, resulting in the error message
'kvm_err("VPE IRQ allocation failure\n")'. The root cause of this issue
is the ~8K IRQ descriptor limit.

To address this issue, an initial proposal was made to define NR_IRQS
to 2^19 for ARM64. However, Marc Zyngier suggested implementing a
generic solution instead of hard-coded values. Thomas Gleixner advised
to use the maple tree data structure and provided most of the necessary
functions.

For more information, refer to the discussion thread at
https://lore.kernel.org/linux-arm-kernel/20230104023738.1258925-1-sdonthineni@nvidia.com/.

This patch series converts the static memory allocation to dynamic using
the maple tree, and increases the maximum number of IRQ descriptors to
INT_MAX from NR_IRQS+8192. This change has been tested on an ARM64 server
with CONFIG_SPARSE_IRQ=y, where 256 virtual machines were launched,
creating a total of 128K+ IRQ descriptors, and IRQ injection was verified.

Changes in v3:
  - Edited commit text
  - Added a helper function irq_resend_init()
  - Rebased to v6.3-rc6

Changes in v2:
  - The patches have been updated to v6.3-rc5.
  - The patches 2/5 and 4/5 have been removed as they are unnecessary.
  - The review comments from Thomas have been addressed.

Shanker Donthineni (3):
  genirq: Use hlist for managing resend handlers
  genirq: Encapsulate sparse bitmap handling
  genirq: Use the maple tree for IRQ descriptors management

 include/linux/irqdesc.h |  3 ++
 kernel/irq/chip.c       |  1 +
 kernel/irq/internals.h  |  6 ++--
 kernel/irq/irqdesc.c    | 76 +++++++++++++++++++++++++----------------
 kernel/irq/resend.c     | 47 ++++++++++++++++---------
 5 files changed, 85 insertions(+), 48 deletions(-)

-- 
2.25.1


             reply	other threads:[~2023-04-10 15:57 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-04-10 15:57 Shanker Donthineni [this message]
2023-04-10 15:57 ` [PATCH v3 1/3] genirq: Use hlist for managing resend handlers Shanker Donthineni
2023-04-10 15:57 ` [PATCH v3 2/3] genirq: Encapsulate sparse bitmap handling Shanker Donthineni
2023-04-10 15:57 ` [PATCH v3 3/3] genirq: Use the maple tree for IRQ descriptors management Shanker Donthineni
2023-04-25  3:16   ` kernel test robot
2023-04-26 12:08     ` Thomas Gleixner
2023-04-28  1:33       ` Yujie Liu
2023-04-28 10:31         ` Thomas Gleixner
2023-05-07  8:05           ` Yujie Liu
2023-05-08  9:36             ` Thomas Gleixner
2023-05-10  7:24               ` Yujie Liu
2023-05-10 14:41                 ` Thomas Gleixner
2023-05-10 14:49                   ` Thomas Gleixner
2023-05-10 15:19                     ` Shanker Donthineni
2023-05-10 17:15                       ` Shanker Donthineni
2023-05-10 19:12                       ` Thomas Gleixner
2023-05-10 16:02   ` Shanker Donthineni
2023-04-15 15:49 ` [PATCH v3 0/3] Increase the number of IRQ descriptors for SPARSEIRQ Shanker Donthineni
2023-04-15 21:50   ` Thomas Gleixner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230410155721.3720991-1-sdonthineni@nvidia.com \
    --to=sdonthineni@nvidia.com \
    --cc=bigeasy@linutronix.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maz@kernel.org \
    --cc=michael@walle.cc \
    --cc=tglx@linutronix.de \
    --cc=vsethi@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®