[PATCH v1] powerpc/pci: restore LSI mappings on card present state change

[PATCH v1] powerpc/pci: restore LSI mappings on card present state change

[Oleksandr Ocheretnyi]

Commit 450be4960a0f ("powerpc/pci: Remove LSI mappings on device
teardown") frees irq descriptors on PCIe hotplug link change event
(Link Down), but the disposed mappings are not restored back on PCIe
hotplug link change event (Card present).

This change restores IRQ mappings disposed earlier when pcieport
link's gone down. So, the call pci_read_irq_line is invoked again
on pcieport's state change (Card present).

Fixes 450be4960a0f ("powerpc/pci: Remove LSI mappings on device teardown")
Signed-off-by: Oleksandr Ocheretnyi <oocheret@cisco.com>
---
 arch/powerpc/kernel/pci-common.c | 30 ++++++++++++++++++++----------
 1 file changed, 20 insertions(+), 10 deletions(-)

diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index eac84d687b53..a0e7cab2baa7 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -390,22 +390,32 @@ static void ppc_pci_intx_release(struct kref *kref)
 	kfree(vi);
 }
 
+static int pci_read_irq_line(struct pci_dev *pci_dev);
+
 static int ppc_pci_unmap_irq_line(struct notifier_block *nb,
 			       unsigned long action, void *data)
 {
 	struct pci_dev *pdev = to_pci_dev(data);
 
-	if (action == BUS_NOTIFY_DEL_DEVICE) {
-		struct pci_intx_virq *vi;
-
-		mutex_lock(&intx_mutex);
-		list_for_each_entry(vi, &intx_list, list_node) {
-			if (vi->virq == pdev->irq) {
-				kref_put(&vi->kref, ppc_pci_intx_release);
-				break;
+	switch (action) {
+		case BUS_NOTIFY_DEL_DEVICE:
+			{
+				struct pci_intx_virq *vi;
+
+				mutex_lock(&intx_mutex);
+				list_for_each_entry(vi, &intx_list, list_node) {
+					if (vi->virq == pdev->irq) {
+						kref_put(&vi->kref, ppc_pci_intx_release);
+						break;
+					}
+				}
+				mutex_unlock(&intx_mutex);
 			}
-		}
-		mutex_unlock(&intx_mutex);
+			break;
+
+		case BUS_NOTIFY_ADD_DEVICE:
+			pci_read_irq_line(pdev);
+			break;
 	}
 
 	return NOTIFY_DONE;
-- 
2.39.3

Re: [PATCH v1] powerpc/pci: restore LSI mappings on card present state change

[krishna kumar]

On 8/27/24 22:27, Oleksandr Ocheretnyi wrote:
> Commit 450be4960a0f ("powerpc/pci: Remove LSI mappings on device
> teardown") frees irq descriptors on PCIe hotplug link change event
> (Link Down), but the disposed mappings are not restored back on PCIe
> hotplug link change event (Card present).
>
> This change restores IRQ mappings disposed earlier when pcieport
> link's gone down. So, the call pci_read_irq_line is invoked again
> on pcieport's state change (Card present).

Few things are important to know regarding these change-sets:

1. The commit (450be4960aof) addressed an issue where the removal
or hot-unplug of an LSI passthroughed IO adapter was not working on
pseries machines. This was due to interrupt resources not getting cleaned
up before removal. Since there were no pcibios_* hooks for the interrupt
cleanup, the interrupt-related resource cleanup has been addressed using
the notifier framework and an explicit call of ppc_pci_intx_release().

2. Does without your current patch and after hot-plug operation, device is
not working (io not happening or interrupt not getting generated) correctly ?

3. There is already a pcibios_* hook available for creating the interrupt
mapping. Here's a snippet -

  /*
   * Called after device has been added to bus and
   * before sysfs has been created.
   */

void (*pcibios_bus_add_device)(struct pci_dev *pdev);

Above function calls - below function to restore the irq mapping.

/* Read default IRQs and fixup if necessary */
   pci_read_irq_line(dev);

4. Does the above pcibios_* hook not sufficient for enabling the interrupt
mapping or does it not getting invoked during hot-plug operation ?

>
> Fixes 450be4960a0f ("powerpc/pci: Remove LSI mappings on device teardown")
> Signed-off-by: Oleksandr Ocheretnyi <oocheret@cisco.com>
> ---
>   arch/powerpc/kernel/pci-common.c | 30 ++++++++++++++++++++----------
>   1 file changed, 20 insertions(+), 10 deletions(-)
>
> diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
> index eac84d687b53..a0e7cab2baa7 100644
> --- a/arch/powerpc/kernel/pci-common.c
> +++ b/arch/powerpc/kernel/pci-common.c
> @@ -390,22 +390,32 @@ static void ppc_pci_intx_release(struct kref *kref)
>   	kfree(vi);
>   }
>   
> +static int pci_read_irq_line(struct pci_dev *pci_dev);
> +
>   static int ppc_pci_unmap_irq_line(struct notifier_block *nb,
>   			       unsigned long action, void *data)
>   {
>   	struct pci_dev *pdev = to_pci_dev(data);
>   
> -	if (action == BUS_NOTIFY_DEL_DEVICE) {
> -		struct pci_intx_virq *vi;
> -
> -		mutex_lock(&intx_mutex);
> -		list_for_each_entry(vi, &intx_list, list_node) {
> -			if (vi->virq == pdev->irq) {
> -				kref_put(&vi->kref, ppc_pci_intx_release);
> -				break;
> +	switch (action) {
> +		case BUS_NOTIFY_DEL_DEVICE:
> +			{
> +				struct pci_intx_virq *vi;
> +
> +				mutex_lock(&intx_mutex);
> +				list_for_each_entry(vi, &intx_list, list_node) {
> +					if (vi->virq == pdev->irq) {
> +						kref_put(&vi->kref, ppc_pci_intx_release);
> +						break;
> +					}
> +				}
> +				mutex_unlock(&intx_mutex);
>   			}
> -		}
> -		mutex_unlock(&intx_mutex);
> +			break;
> +
> +		case BUS_NOTIFY_ADD_DEVICE:
> +			pci_read_irq_line(pdev);
> +			break;

The above code is fine only if my aforementioned points do not hold.

>   	}
>   
>   	return NOTIFY_DONE;


Best Regards,

Krishna