[PATCH v2 0/2] alpha: disable DAC for 32-bit PCI cards on Tsunami/Typhoon

[PATCH v2 0/2] alpha: disable DAC for 32-bit PCI cards on Tsunami/Typhoon

[Magnus Lindholm]

This is a follow-up to a patch I posted last year (Sep 2025, "[PATCH
0/1] alpha: disable DAC for 32-bit PCI on tsunami"), which was not
merged.

Changes since v1:
 - Split into two patches: generic bus_dma_limit plumbing (1/2) and
   the Tsunami policy itself (2/2).
 - Rebased onto current mainline (pci_iommu.c's DMA-mapping API moved
   from map_page()/virtual addresses to map_phys()/phys_addr_t 
 - bus_dma_limit is now treated as a numeric address ceiling rather
   than a bitmask: the DAC path in pci_map_single_1()/sg_fill() is
   bound-checked against it directly, instead of folding it into the
   DAC capability test in pci_dac_dma_supported() (which is otherwise
   unchanged). Fixed a NULL-pointer dereference this exposed on
   no-IOMMU machines once that bound check could fail. 
 - Use ST_DEC_TSUNAMI instead of the raw sys_type value 34.

Boot tested on an AlphaServer ES40 (Tsunami) with a QLogic ISP1040B
controller.

Original background, still accurate:

I've spent quite some time trying to make the qla1280 driver work
with 64-bit DMA on Alpha/Tsunami systems with more than 2GB RAM. Many
thanks to Martin, James, Maciej, Thomas and Christoph who took the
time to provide feedback and testing during my attempts.

This is what I've concluded:

* The ISP1040B (32-bit card) works with a 64-bit DMA mask on a 21164
  Rawhide machine - the card itself supports DAC, even though the
  data sheet doesn't officially claim support until rev C (as Thomas
  Bogendoerfer pointed out earlier).

* The ISP1080 (64-bit PCI slot/card) works with a 64-bit DMA mask on
  a 21264 Tsunami machine - so the monster window itself works fine
  on Tsunami.

* Data gets corrupted on Alpha/Tsunami specifically when DAC/monster
  window is used by a 32-bit PCI card. The amount corrupted varies a
  lot between runs, from none at all to several kilobytes out of 20MB
  transferred. When it happens, it's always in 64-byte chunks, which
  coincides with the 21264's cache block size. Manual inspection of
  the corrupted data shows it's memory content from other active
  processes doing DMA on other drives/controllers at the time.

The fix is unchanged from the original posting: limit 32-bit PCI
cards from using DAC/monster window DMA on Tsunami based Alphas, by
setting bus_dma_limit to DMA_BIT_MASK(32) for devices that have no
64-bit memory BAR. There are 64-bit PCI cards that only have 32-bit
memory BARs, like the QLogic ISP1080 and ISP10160 SCSI controllers;
these will be needlessly constrained even though they work correctly
on Tsunami. I believe this is an acceptable trade-off, since those
controllers are not known to be supported by SRM firmware and are
therefore uncommon on Alpha systems.

In practice there are very few 32-bit PCI cards likely to be used on
Alpha with drivers that request 64-bit DMA addressing. The only
example I've found is the qla1280 driver with an ISP1040 controller,
which is supported by most SRM firmware versions and hence fairly
common on Alpha systems.

Magnus Lindholm (2):
  alpha: respect dev->bus_dma_limit as the effective DMA address ceiling
  alpha: disable DAC for 32-bit PCI cards on Tsunami/Typhoon

 arch/alpha/kernel/pci.c       | 22 ++++++++++++++++++++++
 arch/alpha/kernel/pci_iommu.c | 22 ++++++++++++++++------
 2 files changed, 38 insertions(+), 6 deletions(-)

-- 
2.53.0

[PATCH v2 1/2] alpha: respect dev->bus_dma_limit as the effective DMA address ceiling

[Magnus Lindholm]

The Alpha PCI DMA-mapping code has no way for platform code to cap a
device's addressable range below what it claims via dma_set_mask(),
even though dev->bus_dma_limit exists precisely for an upstream bridge
or bus to impose exactly that kind of constraint. Wire it in as the
effective upper bound everywhere an address range gets checked or a
DMA window gets selected:

 - pci_map_single_1() and sg_fill() already bound-check the direct-map
   window against max_dma; make max_dma itself
   min_not_zero(dma_mask, bus_dma_limit), and add the same
   paddr + dac_offset + size - 1 <= max_dma check to their DAC paths,
   which previously had no bound check at all beyond the bitmask test
   in pci_dac_dma_supported().
 - alpha_pci_map_sg()'s no-IOMMU fallback path (no alpha_mv.mv_pci_tbi)
   picks up the same ceiling instead of unconditionally using -1.
 - sg_fill() gains an explicit failure return when there is no IOMMU
   arena and the direct/DAC paths didn't apply. Previously this relied
   on max_dma = -1 making the (unchecked) DAC branch always succeed
   whenever dac_allowed was true; now that DAC has a real bound check,
   that combination is reachable and must not fall through into
   iommu_arena_alloc() with a NULL arena. Mirrors the DMA_MAPPING_ERROR
   return pci_map_single_1() already gives for the same situation.

DAC capability itself is unchanged: pci_dac_dma_supported() still
determines whether a device's DMA mask can address the DAC bit at all
(a capability question). bus_dma_limit is enforced separately, at each
mapping site, against the resulting address.

dev->bus_dma_limit is a numeric upper bound on the end address of a
transfer (see dma_capable() in include/linux/dma-direct.h), not
another bitmask to AND against.

bus_dma_limit defaults to 0 and min_not_zero() ignores a zero operand,
so all of this is a no-op until something actually sets bus_dma_limit
on a device - no behaviour change by itself.

This is infrastructure for a following patch that uses bus_dma_limit to
work around a Tsunami/Typhoon-specific DAC issue; kept separate since
it's a generic, self-contained change with no policy attached.

Suggested-by: Maciej Rozycki <macro@orcam.me.uk>
Signed-off-by: Magnus Lindholm <linmag7@gmail.com>
---
 arch/alpha/kernel/pci_iommu.c | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/arch/alpha/kernel/pci_iommu.c b/arch/alpha/kernel/pci_iommu.c
index 955b6ca61627..6191dcd560bd 100644
--- a/arch/alpha/kernel/pci_iommu.c
+++ b/arch/alpha/kernel/pci_iommu.c
@@ -228,7 +228,8 @@ pci_map_single_1(struct pci_dev *pdev, phys_addr_t paddr, size_t size,
 		 int dac_allowed)
 {
 	struct pci_controller *hose = pdev ? pdev->sysdata : pci_isa_hose;
-	dma_addr_t max_dma = pdev ? pdev->dma_mask : ISA_DMA_MASK;
+	dma_addr_t max_dma = pdev ? min_not_zero(pdev->dma_mask,
+		pdev->dev.bus_dma_limit) : ISA_DMA_MASK;
 	unsigned long offset = offset_in_page(paddr);
 	struct pci_iommu_arena *arena;
 	long npages, dma_ofs, i;
@@ -250,7 +251,8 @@ pci_map_single_1(struct pci_dev *pdev, phys_addr_t paddr, size_t size,
 #endif
 
 	/* Next, use DAC if selected earlier.  */
-	if (dac_allowed) {
+	if (dac_allowed
+	    && paddr + alpha_mv.pci_dac_offset + size - 1 <= max_dma) {
 		ret = paddr + alpha_mv.pci_dac_offset;
 
 		DBGA2("pci_map_single: [%pa,%zx] -> DAC %llx from %ps\n",
@@ -549,7 +551,8 @@ sg_fill(struct device *dev, struct scatterlist *leader, struct scatterlist *end,
 #endif
 
 	/* If physically contiguous and DAC is available, use it.  */
-	if (leader->dma_address == 0 && dac_allowed) {
+	if (leader->dma_address == 0 && dac_allowed
+	    && paddr + alpha_mv.pci_dac_offset + size - 1 <= max_dma) {
 		out->dma_address = paddr + alpha_mv.pci_dac_offset;
 		out->dma_length = size;
 
@@ -559,6 +562,10 @@ sg_fill(struct device *dev, struct scatterlist *leader, struct scatterlist *end,
 		return 0;
 	}
 
+	/* No IOMMU and direct/DAC didn't fit: nothing left to try.  */
+	if (!arena)
+		return -1;
+
 	/* Otherwise, we'll use the iommu to make the pages virtually
 	   contiguous.  */
 
@@ -654,12 +661,14 @@ static int alpha_pci_map_sg(struct device *dev, struct scatterlist *sg,
 	/* Second, figure out where we're going to map things.  */
 	if (alpha_mv.mv_pci_tbi) {
 		hose = pdev ? pdev->sysdata : pci_isa_hose;
-		max_dma = pdev ? pdev->dma_mask : ISA_DMA_MASK;
+		max_dma = pdev ? min_not_zero(pdev->dma_mask,
+			pdev->dev.bus_dma_limit) : ISA_DMA_MASK;
 		arena = hose->sg_pci;
 		if (!arena || arena->dma_base + arena->size - 1 > max_dma)
 			arena = hose->sg_isa;
 	} else {
-		max_dma = -1;
+		max_dma = pdev ? min_not_zero(pdev->dma_mask,
+			pdev->dev.bus_dma_limit) : -1;
 		arena = NULL;
 		hose = NULL;
 	}
@@ -719,7 +728,8 @@ static void alpha_pci_unmap_sg(struct device *dev, struct scatterlist *sg,
 		return;
 
 	hose = pdev ? pdev->sysdata : pci_isa_hose;
-	max_dma = pdev ? pdev->dma_mask : ISA_DMA_MASK;
+	max_dma = pdev ? min_not_zero(pdev->dma_mask,
+		pdev->dev.bus_dma_limit) : ISA_DMA_MASK;
 	arena = hose->sg_pci;
 	if (!arena || arena->dma_base + arena->size - 1 > max_dma)
 		arena = hose->sg_isa;
-- 
2.53.0

[PATCH v2 2/2] alpha: disable DAC for 32-bit PCI cards on Tsunami/Typhoon

[Magnus Lindholm]

The Tsunami/Typhoon Pchip's DAC ("monster window") path corrupts data
when used by 32-bit PCI cards using DAC addresses above 4 GiB, even on
cards whose DAC support is otherwise solid: the same cards work
correctly with DAC on Rawhide (MCPCIA) systems, and native 64-bit PCI
cards are unaffected on Tsunami/Typhoon itself. Corruption shows up as
64-byte chunks (one 21264 cache block) of unrelated data - typically
identifiable content belonging to other processes' concurrent DMA -
substituted into the transfer; the rate varies from none to several
kilobytes per run and has not been tied to any particular alignment.

Work around this by capping affected devices to 32-bit DMA, which
routes them through the existing scatter-gather window instead of DAC.
Conventional PCI provides no status bit to distinguish a 32-bit from a
64-bit option card, so use the presence of a 64-bit memory BAR as a
practical proxy. This covers every affected card seen so far, but is
a proxy rather than a direct test: it will also needlessly restrict a
handful of 64-bit cards that only expose 32-bit BARs (e.g. QLogic
ISP1080, ISP10160). These controllers are not known to be supported by
SRM firmware and are therefore uncommon on Alpha systems, so the
trade-off is accepted.

The only driver currently known to hit this is qla1280 with an
ISP1040 card and a 64-bit DMA mask, which is a common and
SRM-supported configuration on Alpha.

Suggested-by: Maciej Rozycki <macro@orcam.me.uk>
Signed-off-by: Magnus Lindholm <linmag7@gmail.com>
---
 arch/alpha/kernel/pci.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/arch/alpha/kernel/pci.c b/arch/alpha/kernel/pci.c
index 11df411b1d18..7dbf380acd93 100644
--- a/arch/alpha/kernel/pci.c
+++ b/arch/alpha/kernel/pci.c
@@ -23,6 +23,8 @@
 #include <linux/cache.h>
 #include <linux/slab.h>
 #include <linux/syscalls.h>
+#include <linux/dma-mapping.h>
+#include <asm/hwrpb.h>
 #include <asm/machvec.h>
 
 #include "proto.h"
@@ -117,6 +119,26 @@ static void pcibios_fixup_final(struct pci_dev *dev)
 }
 DECLARE_PCI_FIXUP_FINAL(PCI_ANY_ID, PCI_ANY_ID, pcibios_fixup_final);
 
+/*
+ * Tsunami/Typhoon's DAC "monster window" corrupts data on 32-bit PCI
+ * cards; cap them to 32-bit DMA by proxy of having no 64-bit BAR.
+ */
+static void tsunami_dac_quirk(struct pci_dev *pdev)
+{
+	int i;
+
+	if (hwrpb->sys_type != ST_DEC_TSUNAMI)
+		return;
+
+	for (i = 0; i <= PCI_STD_RESOURCE_END; i++)
+		if (pci_resource_flags(pdev, i) & IORESOURCE_MEM_64)
+			return;
+
+	pdev->dev.bus_dma_limit = DMA_BIT_MASK(32);
+	dev_dbg(&pdev->dev, "disabling DAC for device\n");
+}
+DECLARE_PCI_FIXUP_FINAL(PCI_ANY_ID, PCI_ANY_ID, tsunami_dac_quirk);
+
 /* Just declaring that the power-of-ten prefixes are actually the
    power-of-two ones doesn't make it true :) */
 #define KB			1024
-- 
2.53.0