[RFC PATCH] scsi: qla2xxx: Use the real INTx pin for the port number on Hyper-V guests

[Laurence Oberman <loberman@redhat.com>]

qla2x00_set_isp_flags() derives the physical port number from
PCI_INTERRUPT_PIN, and the port number selects the per-port VPD, NVRAM,
NPIV and FCP priority flash regions.

On Hyper-V guests, pci-hyperv returns 0 for byte and word reads of
PCI_INTERRUPT_PIN since commit fea93a3e5d5e ("PCI: hv: Return zero, not
garbage, when reading PCI_INTERRUPT_PIN"). Commit 7268e509b43a ("scsi:
qla2xxx: Handle INTx not connected while passing through") then falls
back to PCI_FUNC(). With Discrete Device Assignment (DDA), each function
of a multi-port HBA is presented as function 0 in its own PCI domain, so
every port gets port_no 0. All ports read port 0's NVRAM, pass the
checksum and register the same WWPN, and nothing is logged.

Without that commit, port_no underflows to 255, no flash region is
selected and every port fails with "Inconsistent NVRAM checksum" and
falls back to the default WWPN.

pci-hyperv only zeroes byte and word accesses within 0x3c-0x3d. A dword
read at PCI_INTERRUPT_LINE reaches the device's config space and returns
the real pin in bits 15:8. On Hyper-V guests, use that when the byte
read returns 0, and warn if the pin is still 0 so that duplicate WWPNs
are not silent. Bare metal, KVM/vfio and other hypervisors are
unchanged.

Tested with an equivalent change on a RHEL 9 (5.14-based) guest with a
dual-port QLE2692 (ISP2261) assigned through DDA (Hyper-V host
10.0.17763):

  before: both ports "Inconsistent NVRAM checksum=0xffffffc0 id=NGKP",
          both WWPN 0x210000e08b1c5586
  after:  port_no 1 and 0, NVRAM valid, WWPN 0x210034800d7065bd and
          0x210034800d7065bc, fabric login and FC-NVMe localport
          registration on both ports

Also tested on bare metal (ISP2031): the new path is not taken and port
numbers and WWPNs are unchanged.

Signed-off-by: Laurence Oberman <loberman@redhat.com>
---
RFC: this relies on pci-hyperv not virtualizing dword reads at 0x3c,
which is an implementation detail rather than an interface. It is posted
as the smallest working fix and to start the discussion. Feedback wanted
on cleaner alternatives:

 - Hyper-V (linux-hyperv): could pci-hyperv expose the real
   PCI_INTERRUPT_PIN, or provide the physical function number, for DDA
   devices? Some drivers use the pin for port identity, not only for
   interrupt routing.

 - Marvell: is there a firmware or register source for the physical
   port that does not depend on PCI config space? CSRX_FUNCTION in
   ctrl_status is a single bit and would not cover quad-port
   27xx/28xx/29xx.

 - The check is x86-only (hypervisor_is_type()). pci-hyperv also
   supports arm64 guests; a better way to detect a pci-hyperv bus is
   welcome.

This has not yet been tested on a 7.3 kernel in the DDA guest; testing
was on a RHEL 9.8 backport. The upstream behaviour without this patch
(duplicate WWPNs with no warning) follows from the code and has not yet
been reproduced on 7.3.

 drivers/scsi/qla2xxx/qla_os.c | 40 ++++++++++++++++++++++++++++++++++-
 1 file changed, 39 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/qla2xxx/qla_os.c b/drivers/scsi/qla2xxx/qla_os.c
index 3c412c7fb6fe..4180d3f15231 100644
--- a/drivers/scsi/qla2xxx/qla_os.c
+++ b/drivers/scsi/qla2xxx/qla_os.c
@@ -24,6 +24,10 @@
 #include <scsi/scsi_transport.h>
 #include <scsi/scsi_transport_fc.h>
 
+#ifdef CONFIG_X86
+#include <asm/hypervisor.h>
+#endif
+
 #include "qla_target.h"
 
 /*
@@ -2704,6 +2708,40 @@ static struct isp_operations qla29xx_isp_ops = {
 	.initialize_adapter	= qla2x00_initialize_adapter,
 };
 
+/*
+ * Read the INTx pin used to derive the physical port number.
+ *
+ * pci-hyperv hard-wires byte and word reads of PCI_INTERRUPT_PIN to zero.
+ * With Hyper-V Discrete Device Assignment every function is also presented
+ * as function 0 in its own PCI domain, so the PCI_FUNC() fallback gives
+ * port 0 for every port and they all report port 0's WWPN. A dword read at
+ * PCI_INTERRUPT_LINE is not virtualized by pci-hyperv and returns the
+ * device's pin in bits 15:8, so use that on Hyper-V guests.
+ */
+static u8
+qla2x00_get_intx_pin(struct qla_hw_data *ha)
+{
+	u8 pin;
+
+	pci_read_config_byte(ha->pdev, PCI_INTERRUPT_PIN, &pin);
+#ifdef CONFIG_X86
+	if (!pin && hypervisor_is_type(X86_HYPER_MS_HYPERV)) {
+		u32 val = 0;
+
+		pci_read_config_dword(ha->pdev, PCI_INTERRUPT_LINE, &val);
+		pin = (val >> 8) & 0xff;
+		if (pin)
+			ql_log_pci(ql_log_info, ha->pdev, 0xffff,
+				   "Hyper-V: INTx pin reads 0, config dword 0x%08x gives pin %u.\n",
+				   val, pin);
+		else
+			ql_log_pci(ql_log_warn, ha->pdev, 0xffff,
+				   "Hyper-V: unable to determine physical port, ports may report duplicate WWPNs.\n");
+	}
+#endif
+	return pin;
+}
+
 static inline void
 qla2x00_set_isp_flags(struct qla_hw_data *ha)
 {
@@ -2887,7 +2925,7 @@ qla2x00_set_isp_flags(struct qla_hw_data *ha)
 		ha->port_no = ha->portnum & 1;
 	else {
 		/* Get adapter physical port no from interrupt pin register. */
-		pci_read_config_byte(ha->pdev, PCI_INTERRUPT_PIN, &ha->port_no);
+		ha->port_no = qla2x00_get_intx_pin(ha);
 		if (ha->port_no == 0) {
 			/* None of INT[A|B|C|D] may be virtualized by vfio */
 			ha->port_no = PCI_FUNC(ha->pdev->devfn);
-- 
2.56.0