Re: [PATCH 2/2] net: wwan: qcom_bam_dmux: Alloc RX buffers as a single coherent block

[Vishnu Santhosh <vishnu.santhosh@oss.qualcomm.com>]

On 13-08-2026 06:10 pm, Stephan Gerhold wrote:
> On Thu, Aug 13, 2026 at 02:35:01PM +0530, Vishnu Santhosh wrote:
>> On 24-07-2026 03:04 pm, Stephan Gerhold wrote:
>>> On Fri, Jul 24, 2026 at 10:16:31AM +0530, Vishnu Santhosh wrote:
>>>> On 14-07-2026 01:05 pm, Stephan Gerhold wrote:
>>>>> On Tue, Jul 14, 2026 at 11:02:32AM +0530, Vishnu Santhosh wrote:
>>>>>> On Qualcomm SoCs where the modem (e.g. the mDSP on Shikra, VMID 43 /
>>>>>> NAV) is the AXI master for BAM-DMUX RX transfers and the XPU enforces
>>>>>> per-region access control, each individually DMA-mapped RX buffer
>>>>>> requires its own XPU resource group (RG). With ~16 RGs available, the
>>>>>> 32 per-buffer dma_map_single() calls exhaust the table and the first
>>>>>> inbound transfer faults with an XPU violation.
>>>>>>
>>>>>> BAM-DMUX is a singleton (exactly one instance per SoC), so the
>>>>>> destination VMID does not need to be a DT property; it is looked up
>>>>>> from the compatible string's match data instead. Add struct
>>>>>> bam_dmux_data with a single vmid field, and a shikra_data instance
>>>>>> hardcoding QCOM_SCM_VMID_NAV for qcom,shikra-bam-dmux.
>>>>>>
>>>>>> When match data is present, allocate all BAM_DMUX_NUM_SKB RX buffers as
>>>>>> a single contiguous dma_alloc_coherent() block and SCM-assign that
>>>>>> block to HLOS plus the VMID once at probe. This reduces RG consumption
>>>>>> from 32 to 1. The block is never reclaimed across a modem power cycle
>>>>>> (bam_dmux_power_off() does not touch it), so the probe-time assignment
>>>>>> covers every subsequent restart without re-assigning or reclaiming. It
>>>>>> is reclaimed to HLOS only once, at remove or on a probe error, and if
>>>>>> that reclaim fails it is leaked rather than returned to the page
>>>>>> allocator.
>>>>>>
>>>>>> Each rx_skbs[] slot is pre-assigned its virtual and DMA address from
>>>>>> the block, so no per-buffer mapping is needed at power-on. Because the
>>>>>> coherent block is not page-backed, received payload is copied into a
>>>>>> regular netdev skb before handoff to the network stack; this is an
>>>>>> unavoidable extra copy on the XPU-enforced RX path.
>>>>>>
>>>>>> Platforms without match data are unaffected: rx_virt stays NULL, no
>>>>>> coherent memory is allocated, and the per-buffer dma_map_single() path
>>>>>> is unchanged.
>>>>>>
>>>>>> Co-developed-by: Deepak Kumar Singh <deepak.singh@oss.qualcomm.com>
>>>>>> Signed-off-by: Deepak Kumar Singh <deepak.singh@oss.qualcomm.com>
>>>>>> Signed-off-by: Vishnu Santhosh <vishnu.santhosh@oss.qualcomm.com>
>>>>> So how do you handle TX buffers? Right now, they are just passed on from
>>>>> the net subsystem. There can be up to 32 TX buffers in progress as well.
>>>>>
>>>>> Overall, I have mixed feelings about this patch. It looks reasonably
>>>>> simple, but fundamentally I don't understand why we need to go back to
>>>>> the old days of implementing protection using a highly limited MPU (in
>>>>> your case: the xPU).
>>>>>
>>>>> Why does the setup of BAM-DMUX differ e.g. from the setup for the crypto
>>>>> engine? Crypto is also using bam-dma, but it avoids this inflexibility
>>>>> by making use of the &apps_smmu. Is BAM-DMUX not covered by the SMMU? Or
>>>>> did you just decide to bypass the SMMU in this case? (If so: Why?)
>>>> I checked with secure systems team on this. Crypto BAM is
>>>> behind apps_smmu, so protection is enforced through the SMMU's Stage-2
>>>> page tables.
>>>>
>>>> A2 BAM (used by BAM-DMUX) is present in secure domain and does not
>>>> support Stage-2 translation on this SoC, and there is no IOMMU domain
>>>> that can be attached to it. The only protection mechanism available is
>>>> the xPU.
>>>>
>>> Thanks for investigating this!
>>>
>>> So is this a hardware limitation or something you could change with a
>>> firmware update? Could you move the A2 BAM out of the secure domain and
>>> protect it via the IOMMU instead of the xPU mechanism? The other modern
>>> platforms with IPA do not have this limitation, they can use the IOMMU
>>> for this.
>>>
>>> We can try to support the xPU protection mechanism in the BAM-DMUX
>>> driver, but it's pretty bad from a performance and memory usage point of
>>> view if you need to copy buffers around multiple times. So if you have
>>> some way to change this in the firmware (and there is still time to do
>>> so before production boards ship), I would strongly recommend to
>>> investigate that.
>>>
>> Based on what we confirmed with the Secure Systems team, this is a limitation
>> of the current Shikra platform rather than something that can be addressed
>> through a firmware-only update.
>>
>> The A2 BAM used by BAM-DMUX is not connected to an SMMU/IOMMU domain on Shikra,
>> which means Stage-2 translation is not available for this path. As a result,
>> it is not possible to move this path behind an IOMMU.
>>
>> Modern IPA-based platforms differ because their data paths are physically routed
>> through SMMU interfaces and therefore do not rely on VMID/xPU ownership assignment
>> for this type of access control. On Shikra, the A2 BAM path is protected using the
>> xPU3 VM-based access-control model, where DDR memory access is restricted and
>> granted through the request-based Hypervisor VM assignment framework. In contrast,
>> older targets relied on the earlier xPU2 resource-sharing model, in which modem
>> access did not require this type of explicit VM ownership configuration. This
>> architectural difference explains why the issue does not occur on those older platforms.
>>
>> So, for this path on Shikra, SCM-driven VMID assignment remains the only practical solution.
>>
> Ok, thanks for looking into this further.
>
> Can you please try the following as alternative for the implementation?
>
> 1. Make sure that you have CONFIG_DMA_RESTRICTED_POOL=y.
> 2. Define a restricted DMA pool for BAM-DMUX, e.g.:
>
> &{/reserved-memory} {
>          bam_dmux_pool: restricted-dma-pool {
> 	    compatible = "restricted-dma-pool";
>              size = <0x40000>;	/* 32*2K*2 = 128K minimum, but 256K might be safer */
> 	    alignment = <...>;	/* Check xPU alignment requirements */
>          };
> };
>
> 3. Assign to BAM-DMUX together with the qcom,vmid:
>
> &bam_dmux {
> 	memory-region = <&bam_dmux_pool>;
> 	qcom,vmid = <QCOM_SCM_VMID_NAV>;
> };
>
> 4. Extend qcom_bam_dmux.c to look up the DMA pool address and make it
>     accessible using SCM: Call of_reserved_mem_lookup() to get the
>     region, then invoke qcom_scm_assign_mem() with that.
>
> 5. Keep RX/TX DMA code paths in qcom_bam_dmux.c unchanged.
>
> 6. When testing, make sure the kernel log contains
>     "Reserved memory: created restricted DMA pool at %pa, size %ld MiB"
>
> 7. Add memory-region and qcom,vmid as optional in dt-bindings.
>     Add dependency: if qcom,vmid is specified, memory-region must be
>     specified.
>
> In a quick test (without the VMID stuff) this worked quite well for me,
> SWIOTLB should handle the copying behind the scenes without further
> changes to the qcom_bam_dmux driver.
>
> I would prefer that over complicating the driver with two separate ways
> of buffer management. AFAICT, the restricted DMA feature is meant for
> this kind of setup where there is no IOMMU but the firmware can still
> restrict memory accesses to a limited amount of regions, see
> https://lwn.net/Articles/841916/ for a short introduction.
>
> Thanks,
> Stephan

Thanks for the detailed steps. I tested this approach on the Qualcomm Shikra
platform, and it works as expected.

With CONFIG_DMA_RESTRICTED_POOL enabled, following the approach, the kernel reports:

software IO TLB: Reserved memory: created restricted DMA pool at 0x00000000fffc0000, size 0 MiB
OF: reserved mem: initialized node restricted-dma-pool, compatible id restricted-dma-pool
OF: reserved mem: 0x00000000fffc0000..0x00000000ffffffff (256 KiB) map non-reusable restricted-dma-pool


The BAM DMA controller also reports:

bam-dma-engine 6044000.dma-controller: assigned reserved memory node restricted-dma-pool

The BAM-DMUX TX/RX DMA paths are working with this configuration. I will update
v2 to use the restricted DMA pool approach as suggested.


Thanks,
Vishnu