[PATCH 00/10] irqchip: Proposed GICv4 updates for 4.15

[PATCH 00/10] irqchip: Proposed GICv4 updates for 4.15

[Marc Zyngier]

The core GICv4 code has been merged in 4.14, but there is some
additional nails to be hammered on that coffin, and this is what
this short series tries to achieve.

The first three patches have been posted before, and implement a
workaround for the HiSilicon Hip07 (aka D05) platform.

The following 5 patches change the way we perform VPE mappings.
Instead of always mapping all VPEs on all ITSs, it is much more
optimal to only map them on the ITSs that will dispatch interrupts the
VPEs could be interested in (i.e. the ITS that is directly connected
to a device assigned to that VM). This results in a bit of refcounting
so that we can perform mapping/unmappingas VLPIs get themselves
mapped/unmapped. In turn, VMOVP becomes much cheaper on systems that
have multiple ITSs.

Patch 9 is a small optimization to avoid spamming all the ITSs
with VINVALL when a guest emits an INVALL, since it is enough to emit
a single one on the first v4 ITS we find.

Finally, the last patch prevents irqbalance and co from messing
with the doorbell interrupts by making the affinity of those to
be only managed by the kernel. This reduces the VMOVP traffic by
a substancial amount and avoid unnecessary exits.

This has been tested on a Huawei D05 system, together with the rest of
the KVM GICv4 series[1]. I'd welcome further testing on alternative
GICv4 platforms.

	M.

[1] https://lwn.net/Articles/735882/

Marc Zyngier (10):
  irqchip/gic-v3-its: Add post-mortem info on command timeout
  irqchip/gic-v3-its: Pass its_node pointer to each command builder
  irqchip/gic-v3-its: Workaround HiSilicon Hip07 redistributor
    addressing
  irqchip/gic-v3-its: Track per-ITS list number
  irqchip/gic-v3-its: Make GICv4_ITS_LIST_MAX globally available
  irqchip/gic-v3-its: Make its_send_vinvall operate on a single ITS
  irqchip/gic-v3-its: Make its_send_vmapp operate on a single ITS
  irqchip/gic-v3-its: Limit scope of VPE mapping to be per ITS
  irqchip/gic-v3-its: Only send VINVALL to a single ITS
  irqchip/gic-v4: Make the doorbells managed affinity interrupts

 Documentation/arm64/silicon-errata.txt |   1 +
 arch/arm64/Kconfig                     |   9 ++
 drivers/irqchip/irq-gic-v3-its.c       | 252 +++++++++++++++++++++++++--------
 drivers/irqchip/irq-gic-v4.c           |  35 ++++-
 include/linux/irqchip/arm-gic-v4.h     |   8 ++
 5 files changed, 246 insertions(+), 59 deletions(-)

-- 
2.11.0

[PATCH 01/10] irqchip/gic-v3-its: Add post-mortem info on command timeout

[Marc Zyngier]

If the ITS stops processing commands, we're pretty much toasted
as we cannot update the configuration anymore (and we're not
even sure that the ITS still translates interrups).

If that happens, let's dump some basic information about the
state of affairs before moving on.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3-its.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index e8d89343d613..20545241766e 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -684,9 +684,9 @@ static void its_flush_cmd(struct its_node *its, struct its_cmd_block *cmd)
 		dsb(ishst);
 }
 
-static void its_wait_for_range_completion(struct its_node *its,
-					  struct its_cmd_block *from,
-					  struct its_cmd_block *to)
+static int its_wait_for_range_completion(struct its_node *its,
+					 struct its_cmd_block *from,
+					 struct its_cmd_block *to)
 {
 	u64 rd_idx, from_idx, to_idx;
 	u32 count = 1000000;	/* 1s! */
@@ -707,12 +707,15 @@ static void its_wait_for_range_completion(struct its_node *its,
 
 		count--;
 		if (!count) {
-			pr_err_ratelimited("ITS queue timeout\n");
-			return;
+			pr_err_ratelimited("ITS queue timeout (%llu %llu %llu)\n",
+					   from_idx, to_idx, rd_idx);
+			return -1;
 		}
 		cpu_relax();
 		udelay(1);
 	}
+
+	return 0;
 }
 
 /* Warning, macro hell follows */
@@ -748,7 +751,8 @@ post:									\
 	next_cmd = its_post_commands(its);				\
 	raw_spin_unlock_irqrestore(&its->lock, flags);			\
 									\
-	its_wait_for_range_completion(its, cmd, next_cmd);		\
+	if (its_wait_for_range_completion(its, cmd, next_cmd))		\
+		pr_err_ratelimited("ITS cmd %ps failed\n", builder);	\
 }
 
 static void its_build_sync_cmd(struct its_cmd_block *sync_cmd,
-- 
2.11.0

[PATCH 02/10] irqchip/gic-v3-its: Pass its_node pointer to each command builder

[Marc Zyngier]

In order to be able to issue command variants depending on
how broken an ITS is, let's pass the its pointer to all
command building primitives.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3-its.c | 58 ++++++++++++++++++++++++++--------------
 1 file changed, 38 insertions(+), 20 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 20545241766e..32baec1070b2 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -268,10 +268,12 @@ struct its_cmd_block {
 #define ITS_CMD_QUEUE_SZ		SZ_64K
 #define ITS_CMD_QUEUE_NR_ENTRIES	(ITS_CMD_QUEUE_SZ / sizeof(struct its_cmd_block))
 
-typedef struct its_collection *(*its_cmd_builder_t)(struct its_cmd_block *,
+typedef struct its_collection *(*its_cmd_builder_t)(struct its_node *,
+						    struct its_cmd_block *,
 						    struct its_cmd_desc *);
 
-typedef struct its_vpe *(*its_cmd_vbuilder_t)(struct its_cmd_block *,
+typedef struct its_vpe *(*its_cmd_vbuilder_t)(struct its_node *,
+					      struct its_cmd_block *,
 					      struct its_cmd_desc *);
 
 static void its_mask_encode(u64 *raw_cmd, u64 val, int h, int l)
@@ -375,7 +377,8 @@ static inline void its_fixup_cmd(struct its_cmd_block *cmd)
 	cmd->raw_cmd[3] = cpu_to_le64(cmd->raw_cmd[3]);
 }
 
-static struct its_collection *its_build_mapd_cmd(struct its_cmd_block *cmd,
+static struct its_collection *its_build_mapd_cmd(struct its_node *its,
+						 struct its_cmd_block *cmd,
 						 struct its_cmd_desc *desc)
 {
 	unsigned long itt_addr;
@@ -395,7 +398,8 @@ static struct its_collection *its_build_mapd_cmd(struct its_cmd_block *cmd,
 	return NULL;
 }
 
-static struct its_collection *its_build_mapc_cmd(struct its_cmd_block *cmd,
+static struct its_collection *its_build_mapc_cmd(struct its_node *its,
+						 struct its_cmd_block *cmd,
 						 struct its_cmd_desc *desc)
 {
 	its_encode_cmd(cmd, GITS_CMD_MAPC);
@@ -408,7 +412,8 @@ static struct its_collection *its_build_mapc_cmd(struct its_cmd_block *cmd,
 	return desc->its_mapc_cmd.col;
 }
 
-static struct its_collection *its_build_mapti_cmd(struct its_cmd_block *cmd,
+static struct its_collection *its_build_mapti_cmd(struct its_node *its,
+						  struct its_cmd_block *cmd,
 						  struct its_cmd_desc *desc)
 {
 	struct its_collection *col;
@@ -427,7 +432,8 @@ static struct its_collection *its_build_mapti_cmd(struct its_cmd_block *cmd,
 	return col;
 }
 
-static struct its_collection *its_build_movi_cmd(struct its_cmd_block *cmd,
+static struct its_collection *its_build_movi_cmd(struct its_node *its,
+						 struct its_cmd_block *cmd,
 						 struct its_cmd_desc *desc)
 {
 	struct its_collection *col;
@@ -445,7 +451,8 @@ static struct its_collection *its_build_movi_cmd(struct its_cmd_block *cmd,
 	return col;
 }
 
-static struct its_collection *its_build_discard_cmd(struct its_cmd_block *cmd,
+static struct its_collection *its_build_discard_cmd(struct its_node *its,
+						    struct its_cmd_block *cmd,
 						    struct its_cmd_desc *desc)
 {
 	struct its_collection *col;
@@ -462,7 +469,8 @@ static struct its_collection *its_build_discard_cmd(struct its_cmd_block *cmd,
 	return col;
 }
 
-static struct its_collection *its_build_inv_cmd(struct its_cmd_block *cmd,
+static struct its_collection *its_build_inv_cmd(struct its_node *its,
+						struct its_cmd_block *cmd,
 						struct its_cmd_desc *desc)
 {
 	struct its_collection *col;
@@ -479,7 +487,8 @@ static struct its_collection *its_build_inv_cmd(struct its_cmd_block *cmd,
 	return col;
 }
 
-static struct its_collection *its_build_int_cmd(struct its_cmd_block *cmd,
+static struct its_collection *its_build_int_cmd(struct its_node *its,
+						struct its_cmd_block *cmd,
 						struct its_cmd_desc *desc)
 {
 	struct its_collection *col;
@@ -496,7 +505,8 @@ static struct its_collection *its_build_int_cmd(struct its_cmd_block *cmd,
 	return col;
 }
 
-static struct its_collection *its_build_clear_cmd(struct its_cmd_block *cmd,
+static struct its_collection *its_build_clear_cmd(struct its_node *its,
+						  struct its_cmd_block *cmd,
 						  struct its_cmd_desc *desc)
 {
 	struct its_collection *col;
@@ -513,7 +523,8 @@ static struct its_collection *its_build_clear_cmd(struct its_cmd_block *cmd,
 	return col;
 }
 
-static struct its_collection *its_build_invall_cmd(struct its_cmd_block *cmd,
+static struct its_collection *its_build_invall_cmd(struct its_node *its,
+						   struct its_cmd_block *cmd,
 						   struct its_cmd_desc *desc)
 {
 	its_encode_cmd(cmd, GITS_CMD_INVALL);
@@ -524,7 +535,8 @@ static struct its_collection *its_build_invall_cmd(struct its_cmd_block *cmd,
 	return NULL;
 }
 
-static struct its_vpe *its_build_vinvall_cmd(struct its_cmd_block *cmd,
+static struct its_vpe *its_build_vinvall_cmd(struct its_node *its,
+					     struct its_cmd_block *cmd,
 					     struct its_cmd_desc *desc)
 {
 	its_encode_cmd(cmd, GITS_CMD_VINVALL);
@@ -535,7 +547,8 @@ static struct its_vpe *its_build_vinvall_cmd(struct its_cmd_block *cmd,
 	return desc->its_vinvall_cmd.vpe;
 }
 
-static struct its_vpe *its_build_vmapp_cmd(struct its_cmd_block *cmd,
+static struct its_vpe *its_build_vmapp_cmd(struct its_node *its,
+					   struct its_cmd_block *cmd,
 					   struct its_cmd_desc *desc)
 {
 	unsigned long vpt_addr;
@@ -554,7 +567,8 @@ static struct its_vpe *its_build_vmapp_cmd(struct its_cmd_block *cmd,
 	return desc->its_vmapp_cmd.vpe;
 }
 
-static struct its_vpe *its_build_vmapti_cmd(struct its_cmd_block *cmd,
+static struct its_vpe *its_build_vmapti_cmd(struct its_node *its,
+					    struct its_cmd_block *cmd,
 					    struct its_cmd_desc *desc)
 {
 	u32 db;
@@ -576,7 +590,8 @@ static struct its_vpe *its_build_vmapti_cmd(struct its_cmd_block *cmd,
 	return desc->its_vmapti_cmd.vpe;
 }
 
-static struct its_vpe *its_build_vmovi_cmd(struct its_cmd_block *cmd,
+static struct its_vpe *its_build_vmovi_cmd(struct its_node *its,
+					   struct its_cmd_block *cmd,
 					   struct its_cmd_desc *desc)
 {
 	u32 db;
@@ -598,7 +613,8 @@ static struct its_vpe *its_build_vmovi_cmd(struct its_cmd_block *cmd,
 	return desc->its_vmovi_cmd.vpe;
 }
 
-static struct its_vpe *its_build_vmovp_cmd(struct its_cmd_block *cmd,
+static struct its_vpe *its_build_vmovp_cmd(struct its_node *its,
+					   struct its_cmd_block *cmd,
 					   struct its_cmd_desc *desc)
 {
 	its_encode_cmd(cmd, GITS_CMD_VMOVP);
@@ -735,7 +751,7 @@ void name(struct its_node *its,						\
 		raw_spin_unlock_irqrestore(&its->lock, flags);		\
 		return;							\
 	}								\
-	sync_obj = builder(cmd, desc);					\
+	sync_obj = builder(its, cmd, desc);				\
 	its_flush_cmd(its, cmd);					\
 									\
 	if (sync_obj) {							\
@@ -743,7 +759,7 @@ void name(struct its_node *its,						\
 		if (!sync_cmd)						\
 			goto post;					\
 									\
-		buildfn(sync_cmd, sync_obj);				\
+		buildfn(its, sync_cmd, sync_obj);			\
 		its_flush_cmd(its, sync_cmd);				\
 	}								\
 									\
@@ -755,7 +771,8 @@ post:									\
 		pr_err_ratelimited("ITS cmd %ps failed\n", builder);	\
 }
 
-static void its_build_sync_cmd(struct its_cmd_block *sync_cmd,
+static void its_build_sync_cmd(struct its_node *its,
+			       struct its_cmd_block *sync_cmd,
 			       struct its_collection *sync_col)
 {
 	its_encode_cmd(sync_cmd, GITS_CMD_SYNC);
@@ -767,7 +784,8 @@ static void its_build_sync_cmd(struct its_cmd_block *sync_cmd,
 static BUILD_SINGLE_CMD_FUNC(its_send_single_command, its_cmd_builder_t,
 			     struct its_collection, its_build_sync_cmd)
 
-static void its_build_vsync_cmd(struct its_cmd_block *sync_cmd,
+static void its_build_vsync_cmd(struct its_node *its,
+				struct its_cmd_block *sync_cmd,
 				struct its_vpe *sync_vpe)
 {
 	its_encode_cmd(sync_cmd, GITS_CMD_VSYNC);
-- 
2.11.0

[PATCH 03/10] irqchip/gic-v3-its: Workaround HiSilicon Hip07 redistributor addressing

[Marc Zyngier]

The ITSes on the Hip07 (as present in the Huawei D05) are broken when
it comes to addressing the redistributors, and need to be explicitely
told to address the VLPI page instead of the redistributor base address.

So let's add yet another quirk, fixing up the target address
in the command stream.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 Documentation/arm64/silicon-errata.txt |  1 +
 arch/arm64/Kconfig                     |  9 +++++++++
 drivers/irqchip/irq-gic-v3-its.c       | 29 +++++++++++++++++++++++++++--
 3 files changed, 37 insertions(+), 2 deletions(-)

diff --git a/Documentation/arm64/silicon-errata.txt b/Documentation/arm64/silicon-errata.txt
index 66e8ce14d23d..304bf22bb83c 100644
--- a/Documentation/arm64/silicon-errata.txt
+++ b/Documentation/arm64/silicon-errata.txt
@@ -70,6 +70,7 @@ stable kernels.
 |                |                 |                 |                             |
 | Hisilicon      | Hip0{5,6,7}     | #161010101      | HISILICON_ERRATUM_161010101 |
 | Hisilicon      | Hip0{6,7}       | #161010701      | N/A                         |
+| Hisilicon      | Hip07           | #161600802      | HISILICON_ERRATUM_161600802 |
 |                |                 |                 |                             |
 | Qualcomm Tech. | Falkor v1       | E1003           | QCOM_FALKOR_ERRATUM_1003    |
 | Qualcomm Tech. | Falkor v1       | E1009           | QCOM_FALKOR_ERRATUM_1009    |
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 0df64a6a56d4..4eaf4ea1f4e4 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -539,6 +539,15 @@ config QCOM_QDF2400_ERRATUM_0065
 
 	  If unsure, say Y.
 
+config HISILICON_ERRATUM_161600802
+	bool "Hip07 161600802: Erroneous redistributor VLPI base"
+	default y
+	help
+	  The HiSilicon Hip07 SoC usees the wrong redistributor base
+	  when issued ITS commands such as VMOVP and VMAPP, and requires
+	  a 128kB offset to be applied to the target address in this commands.
+
+	  If unsure, say Y.
 endmenu
 
 
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 32baec1070b2..cc7c8f6cb863 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -103,6 +103,7 @@ struct its_node {
 	u32			device_ids;
 	int			numa_node;
 	bool			is_v4;
+	int			vlpi_redist_offset;
 };
 
 #define ITS_ITT_ALIGN		SZ_256
@@ -552,13 +553,15 @@ static struct its_vpe *its_build_vmapp_cmd(struct its_node *its,
 					   struct its_cmd_desc *desc)
 {
 	unsigned long vpt_addr;
+	u64 target;
 
 	vpt_addr = virt_to_phys(page_address(desc->its_vmapp_cmd.vpe->vpt_page));
+	target = desc->its_vmapp_cmd.col->target_address + its->vlpi_redist_offset;
 
 	its_encode_cmd(cmd, GITS_CMD_VMAPP);
 	its_encode_vpeid(cmd, desc->its_vmapp_cmd.vpe->vpe_id);
 	its_encode_valid(cmd, desc->its_vmapp_cmd.valid);
-	its_encode_target(cmd, desc->its_vmapp_cmd.col->target_address);
+	its_encode_target(cmd, target);
 	its_encode_vpt_addr(cmd, vpt_addr);
 	its_encode_vpt_size(cmd, LPI_NRBITS - 1);
 
@@ -617,11 +620,14 @@ static struct its_vpe *its_build_vmovp_cmd(struct its_node *its,
 					   struct its_cmd_block *cmd,
 					   struct its_cmd_desc *desc)
 {
+	u64 target;
+
+	target = desc->its_vmovp_cmd.col->target_address + its->vlpi_redist_offset;
 	its_encode_cmd(cmd, GITS_CMD_VMOVP);
 	its_encode_seq_num(cmd, desc->its_vmovp_cmd.seq_num);
 	its_encode_its_list(cmd, desc->its_vmovp_cmd.its_list);
 	its_encode_vpeid(cmd, desc->its_vmovp_cmd.vpe->vpe_id);
-	its_encode_target(cmd, desc->its_vmovp_cmd.col->target_address);
+	its_encode_target(cmd, target);
 
 	its_fixup_cmd(cmd);
 
@@ -2781,6 +2787,17 @@ static void __maybe_unused its_enable_quirk_qdf2400_e0065(void *data)
 	its->ite_size = 16;
 }
 
+static void __maybe_unused its_enable_quirk_hip07_161600802(void *data)
+{
+	struct its_node *its = data;
+
+	/*
+	 * Hip07 insists on using the wrong address for the VLPI
+	 * page. Trick it into doing the right thing...
+	 */
+	its->vlpi_redist_offset = SZ_128K;
+}
+
 static const struct gic_quirk its_quirks[] = {
 #ifdef CONFIG_CAVIUM_ERRATUM_22375
 	{
@@ -2806,6 +2823,14 @@ static const struct gic_quirk its_quirks[] = {
 		.init	= its_enable_quirk_qdf2400_e0065,
 	},
 #endif
+#ifdef CONFIG_HISILICON_ERRATUM_161600802
+	{
+		.desc	= "ITS: Hip07 erratum 161600802",
+		.iidr	= 0x00000004,
+		.mask	= 0xffffffff,
+		.init	= its_enable_quirk_hip07_161600802,
+	},
+#endif
 	{
 	}
 };
-- 
2.11.0

[PATCH 04/10] irqchip/gic-v3-its: Track per-ITS list number

[Marc Zyngier]

At boot time, we enumerate all the GICv4-capable ITSs, and build
a mask of the available ITSs. Take this opportunity to store
the ITS number in the its_node structure so that we can use it
at a later time.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3-its.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index cc7c8f6cb863..24ffb19c559d 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -99,6 +99,7 @@ struct its_node {
 	struct its_collection	*collections;
 	struct list_head	its_device_list;
 	u64			flags;
+	unsigned long		list_nr;
 	u32			ite_size;
 	u32			device_ids;
 	int			numa_node;
@@ -2996,6 +2997,8 @@ static int __init its_probe_one(struct resource *res,
 			if (err < 0)
 				goto out_free_its;
 
+			its->list_nr = err;
+
 			pr_info("ITS@%pa: Using ITS number %d\n",
 				&res->start, err);
 		} else {
-- 
2.11.0

[PATCH 05/10] irqchip/gic-v3-its: Make GICv4_ITS_LIST_MAX globally available

[Marc Zyngier]

As we're about to make use of the maximum number of ITSs in
a GICv4 system, let's make this value global (and rename it to
GICv4_ITS_LIST_MAX).

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3-its.c   | 10 ++--------
 include/linux/irqchip/arm-gic-v4.h |  6 ++++++
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 24ffb19c559d..8a1f985c8558 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -150,12 +150,6 @@ static DEFINE_SPINLOCK(its_lock);
 static struct rdists *gic_rdists;
 static struct irq_domain *its_parent;
 
-/*
- * We have a maximum number of 16 ITSs in the whole system if we're
- * using the ITSList mechanism
- */
-#define ITS_LIST_MAX		16
-
 static unsigned long its_list_map;
 static u16 vmovp_seq_num;
 static DEFINE_RAW_SPINLOCK(vmovp_lock);
@@ -2921,8 +2915,8 @@ static int __init its_compute_its_list_map(struct resource *res,
 	 * locking. Should this change, we should address
 	 * this.
 	 */
-	its_number = find_first_zero_bit(&its_list_map, ITS_LIST_MAX);
-	if (its_number >= ITS_LIST_MAX) {
+	its_number = find_first_zero_bit(&its_list_map, GICv4_ITS_LIST_MAX);
+	if (its_number >= GICv4_ITS_LIST_MAX) {
 		pr_err("ITS@%pa: No ITSList entry available!\n",
 		       &res->start);
 		return -EINVAL;
diff --git a/include/linux/irqchip/arm-gic-v4.h b/include/linux/irqchip/arm-gic-v4.h
index 58a4d89aa82c..e26a668826e6 100644
--- a/include/linux/irqchip/arm-gic-v4.h
+++ b/include/linux/irqchip/arm-gic-v4.h
@@ -20,6 +20,12 @@
 
 struct its_vpe;
 
+/*
+ * Maximum number of ITTs when GITS_TYPER.VMOVP == 0, using the
+ * ITSList mechanism to perform inter-ITS synchronization.
+ */
+#define GICv4_ITS_LIST_MAX		16
+
 /* Embedded in kvm.arch */
 struct its_vm {
 	struct fwnode_handle	*fwnode;
-- 
2.11.0

[PATCH 06/10] irqchip/gic-v3-its: Make its_send_vinvall operate on a single ITS

[Marc Zyngier]

Currently, its_send_vinvall operates on all ITSs. As we're about
to try and limit the amount of commands we send to ITSs that are
not involved in dealing with a given VM, let's redefine that
primitive so that it takes a target ITS as a parameter.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3-its.c | 33 +++++++++++++++++++++++----------
 1 file changed, 23 insertions(+), 10 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 8a1f985c8558..202f43b30bd3 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -977,18 +977,12 @@ static void its_send_vmovp(struct its_vpe *vpe)
 	raw_spin_unlock_irqrestore(&vmovp_lock, flags);
 }
 
-static void its_send_vinvall(struct its_vpe *vpe)
+static void its_send_vinvall(struct its_node *its, struct its_vpe *vpe)
 {
 	struct its_cmd_desc desc;
-	struct its_node *its;
 
 	desc.its_vinvall_cmd.vpe = vpe;
-
-	list_for_each_entry(its, &its_nodes, entry) {
-		if (!its->is_v4)
-			continue;
-		its_send_single_vcommand(its, its_build_vinvall_cmd, &desc);
-	}
+	its_send_single_vcommand(its, its_build_vinvall_cmd, &desc);
 }
 
 /*
@@ -2461,6 +2455,18 @@ static void its_vpe_deschedule(struct its_vpe *vpe)
 	}
 }
 
+static void its_vpe_invall(struct its_vpe *vpe)
+{
+	struct its_node *its;
+
+	list_for_each_entry(its, &its_nodes, entry) {
+		if (!its->is_v4)
+			continue;
+
+		its_send_vinvall(its, vpe);
+	}
+}
+
 static int its_vpe_set_vcpu_affinity(struct irq_data *d, void *vcpu_info)
 {
 	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
@@ -2476,7 +2482,7 @@ static int its_vpe_set_vcpu_affinity(struct irq_data *d, void *vcpu_info)
 		return 0;
 
 	case INVALL_VPE:
-		its_send_vinvall(vpe);
+		its_vpe_invall(vpe);
 		return 0;
 
 	default:
@@ -2705,11 +2711,18 @@ static void its_vpe_irq_domain_activate(struct irq_domain *domain,
 					struct irq_data *d)
 {
 	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+	struct its_node *its;
 
 	/* Map the VPE to the first possible CPU */
 	vpe->col_idx = cpumask_first(cpu_online_mask);
 	its_send_vmapp(vpe, true);
-	its_send_vinvall(vpe);
+
+	list_for_each_entry(its, &its_nodes, entry) {
+		if (!its->is_v4)
+			continue;
+
+		its_send_vinvall(its, vpe);
+	}
 }
 
 static void its_vpe_irq_domain_deactivate(struct irq_domain *domain,
-- 
2.11.0

[PATCH 07/10] irqchip/gic-v3-its: Make its_send_vmapp operate on a single ITS

[Marc Zyngier]

Currently, its_send_vmapp operates on all ITSs. As we're about
to try and limit the amount of commands we send to ITSs that are
not involved in dealing with a given VM, let's redefine that
primitive so that it takes a target ITS as a parameter.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3-its.c | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 202f43b30bd3..4583af37785b 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -918,21 +918,16 @@ static void its_send_vmovi(struct its_device *dev, u32 id)
 	its_send_single_vcommand(dev->its, its_build_vmovi_cmd, &desc);
 }
 
-static void its_send_vmapp(struct its_vpe *vpe, bool valid)
+static void its_send_vmapp(struct its_node *its,
+			   struct its_vpe *vpe, bool valid)
 {
 	struct its_cmd_desc desc;
-	struct its_node *its;
 
 	desc.its_vmapp_cmd.vpe = vpe;
 	desc.its_vmapp_cmd.valid = valid;
+	desc.its_vmapp_cmd.col = &its->collections[vpe->col_idx];
 
-	list_for_each_entry(its, &its_nodes, entry) {
-		if (!its->is_v4)
-			continue;
-
-		desc.its_vmapp_cmd.col = &its->collections[vpe->col_idx];
-		its_send_single_vcommand(its, its_build_vmapp_cmd, &desc);
-	}
+	its_send_single_vcommand(its, its_build_vmapp_cmd, &desc);
 }
 
 static void its_send_vmovp(struct its_vpe *vpe)
@@ -2715,12 +2710,12 @@ static void its_vpe_irq_domain_activate(struct irq_domain *domain,
 
 	/* Map the VPE to the first possible CPU */
 	vpe->col_idx = cpumask_first(cpu_online_mask);
-	its_send_vmapp(vpe, true);
 
 	list_for_each_entry(its, &its_nodes, entry) {
 		if (!its->is_v4)
 			continue;
 
+		its_send_vmapp(its, vpe, true);
 		its_send_vinvall(its, vpe);
 	}
 }
@@ -2729,8 +2724,14 @@ static void its_vpe_irq_domain_deactivate(struct irq_domain *domain,
 					  struct irq_data *d)
 {
 	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
+	struct its_node *its;
+
+	list_for_each_entry(its, &its_nodes, entry) {
+		if (!its->is_v4)
+			continue;
 
-	its_send_vmapp(vpe, false);
+		its_send_vmapp(its, vpe, false);
+	}
 }
 
 static const struct irq_domain_ops its_vpe_domain_ops = {
-- 
2.11.0

[PATCH 08/10] irqchip/gic-v3-its: Limit scope of VPE mapping to be per ITS

[Marc Zyngier]

So far, we map all VPEs on all ITSs. While this is not wrong,
this is quite a big hammer, as moving a VPE around requires
all ITSs to be synchronized. Needles to say, this is an
expensive proposition.

Instead, let's switch to a mode where we issue VMAPP commands
only on ITSs that are actually involved in reporting interrupts
to the given VM.

For that purpose, we refcount the number of interrupts are are
mapped for this VM on each ITS, performing the map/unmap
operations as required. It then allows us to use this refcount
to only issue VMOVP to the ITSs that need to know about this
VM.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3-its.c   | 75 ++++++++++++++++++++++++++++++++++++++
 include/linux/irqchip/arm-gic-v4.h |  1 +
 2 files changed, 76 insertions(+)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 4583af37785b..9997e387b936 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -965,6 +965,9 @@ static void its_send_vmovp(struct its_vpe *vpe)
 		if (!its->is_v4)
 			continue;
 
+		if (!vpe->its_vm->vlpi_count[its->list_nr])
+			continue;
+
 		desc.its_vmovp_cmd.col = &its->collections[col_id];
 		its_send_single_vcommand(its, its_build_vmovp_cmd, &desc);
 	}
@@ -1141,6 +1144,58 @@ static int its_irq_set_irqchip_state(struct irq_data *d,
 	return 0;
 }
 
+static void its_map_vm(struct its_node *its, struct its_vm *vm)
+{
+	unsigned long flags;
+
+	/* Not using the ITS list? Everything is always mapped. */
+	if (!its_list_map)
+		return;
+
+	raw_spin_lock_irqsave(&vmovp_lock, flags);
+
+	/*
+	 * If the VM wasn't mapped yet, iterate over the vpes and get
+	 * them mapped now.
+	 */
+	vm->vlpi_count[its->list_nr]++;
+
+	if (vm->vlpi_count[its->list_nr] == 1) {
+		int i;
+
+		for (i = 0; i < vm->nr_vpes; i++) {
+			struct its_vpe *vpe = vm->vpes[i];
+
+			/* Map the VPE to the first possible CPU */
+			vpe->col_idx = cpumask_first(cpu_online_mask);
+			its_send_vmapp(its, vpe, true);
+			its_send_vinvall(its, vpe);
+		}
+	}
+
+	raw_spin_unlock_irqrestore(&vmovp_lock, flags);
+}
+
+static void its_unmap_vm(struct its_node *its, struct its_vm *vm)
+{
+	unsigned long flags;
+
+	/* Not using the ITS list? Everything is always mapped. */
+	if (!its_list_map)
+		return;
+
+	raw_spin_lock_irqsave(&vmovp_lock, flags);
+
+	if (!--vm->vlpi_count[its->list_nr]) {
+		int i;
+
+		for (i = 0; i < vm->nr_vpes; i++)
+			its_send_vmapp(its, vm->vpes[i], false);
+	}
+
+	raw_spin_unlock_irqrestore(&vmovp_lock, flags);
+}
+
 static int its_vlpi_map(struct irq_data *d, struct its_cmd_info *info)
 {
 	struct its_device *its_dev = irq_data_get_irq_chip_data(d);
@@ -1176,6 +1231,9 @@ static int its_vlpi_map(struct irq_data *d, struct its_cmd_info *info)
 		/* Already mapped, move it around */
 		its_send_vmovi(its_dev, event);
 	} else {
+		/* Ensure all the VPEs are mapped on this ITS */
+		its_map_vm(its_dev->its, info->map->vm);
+
 		/* Drop the physical mapping */
 		its_send_discard(its_dev, event);
 
@@ -1237,6 +1295,9 @@ static int its_vlpi_unmap(struct irq_data *d)
 				    LPI_PROP_ENABLED |
 				    LPI_PROP_GROUP1));
 
+	/* Potentially unmap the VM from this ITS */
+	its_unmap_vm(its_dev->its, its_dev->event_map.vm);
+
 	/*
 	 * Drop the refcount and make the device available again if
 	 * this was the last VLPI.
@@ -2458,6 +2519,9 @@ static void its_vpe_invall(struct its_vpe *vpe)
 		if (!its->is_v4)
 			continue;
 
+		if (its_list_map && !vpe->its_vm->vlpi_count[its->list_nr])
+			continue;
+
 		its_send_vinvall(its, vpe);
 	}
 }
@@ -2708,6 +2772,10 @@ static void its_vpe_irq_domain_activate(struct irq_domain *domain,
 	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
 	struct its_node *its;
 
+	/* If we use the list map, we issue VMAPP on demand... */
+	if (its_list_map)
+		return;
+
 	/* Map the VPE to the first possible CPU */
 	vpe->col_idx = cpumask_first(cpu_online_mask);
 
@@ -2726,6 +2794,13 @@ static void its_vpe_irq_domain_deactivate(struct irq_domain *domain,
 	struct its_vpe *vpe = irq_data_get_irq_chip_data(d);
 	struct its_node *its;
 
+	/*
+	 * If we use the list map, we unmap the VPE once no VLPIs are
+	 * associated with the VM.
+	 */
+	if (its_list_map)
+		return;
+
 	list_for_each_entry(its, &its_nodes, entry) {
 		if (!its->is_v4)
 			continue;
diff --git a/include/linux/irqchip/arm-gic-v4.h b/include/linux/irqchip/arm-gic-v4.h
index e26a668826e6..43cde15f221b 100644
--- a/include/linux/irqchip/arm-gic-v4.h
+++ b/include/linux/irqchip/arm-gic-v4.h
@@ -36,6 +36,7 @@ struct its_vm {
 	irq_hw_number_t		db_lpi_base;
 	unsigned long		*db_bitmap;
 	int			nr_db_lpis;
+	u32			vlpi_count[GICv4_ITS_LIST_MAX];
 };
 
 /* Embedded in kvm_vcpu.arch */
-- 
2.11.0

[PATCH 09/10] irqchip/gic-v3-its: Only send VINVALL to a single ITS

[Marc Zyngier]

Sending VINVALL to all ITSs is completely pointless, as all
we're trying to achieve is to tell the redistributor that
the property table for this VPE should be invalidated.

Let's issue the command on the first valid ITS and be done with it.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3-its.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 9997e387b936..f49c5688c2a0 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -2522,7 +2522,12 @@ static void its_vpe_invall(struct its_vpe *vpe)
 		if (its_list_map && !vpe->its_vm->vlpi_count[its->list_nr])
 			continue;
 
+		/*
+		 * Sending a VINVALL to a single ITS is enough, as all
+		 * we need is to reach the redistributors.
+		 */
 		its_send_vinvall(its, vpe);
+		return;
 	}
 }
 
-- 
2.11.0

[PATCH 10/10] irqchip/gic-v4: Make the doorbells managed affinity interrupts

[Marc Zyngier]

We so far allocate the doorbell interrupts without taking any
special measure regarding the affinity of these interrupts. We
simply move them around as required when the vcpu gets scheduled
on a different CPU.

But that's counting without userspace (and the evil irqbalance) that
can try and move the VPE interrupt around, causing the ITS code
to emit VMOVP commands and remap the doorbell to another redistributor.
Worse, this can happen while the vcpu is running, causing all kind
of trouble as the VPE is not resident on this redistributor.

So let's take a definitive action and prevent userspace from messing
with us. This is just a matter of passing an affinity to the IRQ
allocator, and all the VPE interrupts will get the IRQD_AFFINITY_MANAGED
flag, which let the kernel is sole control of the affinity.

Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v4.c       | 35 +++++++++++++++++++++++++++++++++--
 include/linux/irqchip/arm-gic-v4.h |  1 +
 2 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v4.c b/drivers/irqchip/irq-gic-v4.c
index cd0bcc3b7e33..bd5382799788 100644
--- a/drivers/irqchip/irq-gic-v4.c
+++ b/drivers/irqchip/irq-gic-v4.c
@@ -86,7 +86,10 @@
  * - mask/unmask do what is expected on the doorbell interrupt.
  *
  * - irq_set_affinity is used to move a VPE from one redistributor to
- *   another.
+ *   another. Note that we make the doorbells a set of "managed
+ *   affinity" interrupts in order to prevent userspace from messing
+ *   with the affinity (you really don't want irqbalance to bounce
+ *   them around while the VPE is running).
  *
  * - irq_set_vcpu_affinity once again gets hijacked for the purpose of
  *   creating a new sub-API, namely scheduling/descheduling a VPE
@@ -97,10 +100,36 @@
 static struct irq_domain *gic_domain;
 static const struct irq_domain_ops *vpe_domain_ops;
 
+static void its_free_affinity_masks(struct its_vm *vm)
+{
+	int i;
+	
+	if (!vm->affinity_masks)
+		return;
+	
+	for (i = 0; i < vm->nr_vpes; i++)
+		if (vm->affinity_masks[i])
+			free_cpumask_var(vm->affinity_masks[i]);
+
+	kfree(vm->affinity_masks);
+}
+
 int its_alloc_vcpu_irqs(struct its_vm *vm)
 {
 	int vpe_base_irq, i;
 
+	vm->affinity_masks = kzalloc(vm->nr_vpes * sizeof(*vm->affinity_masks),
+				     GFP_KERNEL);
+	if (!vm->affinity_masks)
+		goto err;
+
+	for (i = 0; i < vm->nr_vpes; i++) {
+		if (!alloc_cpumask_var(&vm->affinity_masks[i], GFP_KERNEL))
+			goto err;
+
+		cpumask_copy(vm->affinity_masks[i], cpu_possible_mask);
+	}
+
 	vm->fwnode = irq_domain_alloc_named_id_fwnode("GICv4-vpe",
 						      task_pid_nr(current));
 	if (!vm->fwnode)
@@ -119,7 +148,7 @@ int its_alloc_vcpu_irqs(struct its_vm *vm)
 
 	vpe_base_irq = __irq_domain_alloc_irqs(vm->domain, -1, vm->nr_vpes,
 					       NUMA_NO_NODE, vm,
-					       false, NULL);
+					       false, *vm->affinity_masks);
 	if (vpe_base_irq <= 0)
 		goto err;
 
@@ -133,6 +162,7 @@ int its_alloc_vcpu_irqs(struct its_vm *vm)
 		irq_domain_remove(vm->domain);
 	if (vm->fwnode)
 		irq_domain_free_fwnode(vm->fwnode);
+	its_free_affinity_masks(vm);
 
 	return -ENOMEM;
 }
@@ -142,6 +172,7 @@ void its_free_vcpu_irqs(struct its_vm *vm)
 	irq_domain_free_irqs(vm->vpes[0]->irq, vm->nr_vpes);
 	irq_domain_remove(vm->domain);
 	irq_domain_free_fwnode(vm->fwnode);
+	its_free_affinity_masks(vm);
 }
 
 static int its_send_vpe_cmd(struct its_vpe *vpe, struct its_cmd_info *info)
diff --git a/include/linux/irqchip/arm-gic-v4.h b/include/linux/irqchip/arm-gic-v4.h
index 43cde15f221b..1052d654a863 100644
--- a/include/linux/irqchip/arm-gic-v4.h
+++ b/include/linux/irqchip/arm-gic-v4.h
@@ -32,6 +32,7 @@ struct its_vm {
 	struct irq_domain	*domain;
 	struct page		*vprop_page;
 	struct its_vpe		**vpes;
+	cpumask_var_t		*affinity_masks;
 	int			nr_vpes;
 	irq_hw_number_t		db_lpi_base;
 	unsigned long		*db_bitmap;
-- 
2.11.0

Re: [PATCH 10/10] irqchip/gic-v4: Make the doorbells managed affinity interrupts

[Thomas Gleixner]

On Tue, 10 Oct 2017, Marc Zyngier wrote:
>  int its_alloc_vcpu_irqs(struct its_vm *vm)
>  {
>  	int vpe_base_irq, i;
>  
> +	vm->affinity_masks = kzalloc(vm->nr_vpes * sizeof(*vm->affinity_masks),
> +				     GFP_KERNEL);
> +	if (!vm->affinity_masks)
> +		goto err;
> +
> +	for (i = 0; i < vm->nr_vpes; i++) {
> +		if (!alloc_cpumask_var(&vm->affinity_masks[i], GFP_KERNEL))
> +			goto err;
> +
> +		cpumask_copy(vm->affinity_masks[i], cpu_possible_mask);

That looks a bit like abuse of managed interrupts as you don't use any of
the managing parts, i.e. the managed shutdown/startup on cpu hotplug.

You can prevent user space from fiddling with the affinities by flagging
these interrupts with IRQ_NO_BALANCING.

Thanks,

	tglx

Re: [PATCH 10/10] irqchip/gic-v4: Make the doorbells managed affinity interrupts

[Marc Zyngier]

On 12/10/17 10:22, Thomas Gleixner wrote:
> On Tue, 10 Oct 2017, Marc Zyngier wrote:
>>  int its_alloc_vcpu_irqs(struct its_vm *vm)
>>  {
>>  	int vpe_base_irq, i;
>>  
>> +	vm->affinity_masks = kzalloc(vm->nr_vpes * sizeof(*vm->affinity_masks),
>> +				     GFP_KERNEL);
>> +	if (!vm->affinity_masks)
>> +		goto err;
>> +
>> +	for (i = 0; i < vm->nr_vpes; i++) {
>> +		if (!alloc_cpumask_var(&vm->affinity_masks[i], GFP_KERNEL))
>> +			goto err;
>> +
>> +		cpumask_copy(vm->affinity_masks[i], cpu_possible_mask);
> 
> That looks a bit like abuse of managed interrupts as you don't use any of
> the managing parts, i.e. the managed shutdown/startup on cpu hotplug.
> 
> You can prevent user space from fiddling with the affinities by flagging
> these interrupts with IRQ_NO_BALANCING.

I had a look at that initially, and can't remember what I discounted
it... This is obviously a much simpler approach (just an additional flag
as we initialize the vcpus).

I'll drop that patch and post another one.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...