Re: [PATCH v2 4/5] iommu: Add Broadcom BCM2712 IOMMU driver

[Daniel Drake <dan@reactivated.net>]

On 29/07/2026 01:10, Jason Gunthorpe wrote:
> If paging is attached then it sets MMMU_CTRL_OPERATING_FLAGS and
> places the aperture at 40G. Based on the comments about blocking I
> wonder if the "bypass" even works when paging is on?
> 
> If blocking is attached then it sets MMMU_CTRL_OPERATING_FLAGS with
> some 0 address cap which aborts everything?

The hardware lets two regions be configured:

1. Bypass window
Configured as BYPASS_START_OFFSET..BYPASS_END_OFFSET
Any device access to addresses in this range go straight through to the 
same addresses in physical memory. There is no PT translation.

2. IOVA Aperture
Configured as BYPASS_END_OFFSET..ADDR_CAP_OFFSET
Any device access to addresses in this region go through page table 
translation before reaching physical memory.

The bypass must come before the aperture.

Access to any address above ADDR_CAP_OFFSET will abort.

So the blocking domain is implemented by setting both of those ranges to 
0, which means ADDR_CAP_OFFSET=0 as part of that. (I'm going with the 
interpretation that a blocking domain is like a killswitch and should 
abort all memory accesses.)

However, the blocking domain implementation should be removed (see below).

The bypass works fine when paging is on - did you spot something that 
makes you suspect otherwise?

> It looks to me like some of those comments and choices don't reflect
> what the driver actually does. Since there is only ever one
> translation we never need to be worried about where the aperture is,
> it could be anything so long as the HW gives it priority to bypass.
> 
> Could the aperture be placed at 0 with the bypass fully disabled? Then
> it would basically be a normal iommu.
Please let me know which comments/choices you are finding unclear, I 
tried to make this driver readable and would like to fix that.

The aperture could be placed anywhere, but the key idea in the current 
driver structure is that we deliberately place it above physical memory, 
so that we can have the bypass window operational for all regular 
physical addresses, meaning that iommu-unaware devices can operate as 
normal.

Each of the 3-4 IOMMUs has around 5 devices hardwired into it. When the 
IOMMU is switched on (effectively via setting CTRL_OPERATING_FLAGS), 
*all* of the hardwired devices are subject to the IOMMU operation, 
uniformly. There is no gating where you can have one of the devices in 
standard passthrough mode and the rest using the IOMMU. Also there is no 
Stream ID in the transactions, there is no way to configure per-device 
page tables.

Among the devices hardwired to the IOMMUs we might have:
  - Devices already set up by the firmware and relying on regular access
    to physical memory
  - Devices that don't require large contiguous DMA allocations and would
    prefer not to have the translation overhead of the iommu
  - Devices that handle scatter-gather natively and prefer not to have
    the translation overhead of the iommu
  - & devices that want to use the IOMMU :) so that they can work with
    large contiguous allocations which are actually scattered in
    underlying physical memory

So we have multiple needs to tend to, which is why the driver currently 
sets up the bypass in the regular address space (serving the first 3 
above), and the IOMMU aperture in a high, unused part of the address 
space (for the devices that do want to take advantage of the IOMMU).

Are those good enough reasons to set up the driver in this way? Or are 
there other approaches to consider?

Let's take an example where the aperture gets placed at address 0 (and 
hence there is no bypass window). Now it is operating more like a 
"normal" IOMMU (except without per-device configurable behaviour, stream 
IDs, etc). The system boots and the firmware sets up the display 
controller rendering some early boot screens. Then the kernel boots and 
takes over the framebuffer using efifb or simpledrm. Then it loads a 
device driver for, let's say, an image processor. This turns on one of 
the iommus and puts it in paging mode. The aperture gets placed at 
address 0, no bypass window. But, whoops, that iommu happened to *also* 
have the display controller hardwared into it. Now we have a big visual 
glitch and the display output has stopped. And maybe the kernel doesn't 
even have the vc4 display driver available to correct that later, 
because the expectation was just to use the simple fw-configured 
framebuffer.

This is also why I think the blocking domain implementation should be 
removed. It causes collateral damage to all the devices which might not 
have had any idea they were influenced by an iommu.

Thanks for your help!

Daniel