From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F94D3E6394 for ; Sat, 19 Sep 2026 09:54:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789811686; cv=none; b=iYIvvErw8nCchlG7qFZiGOoqsfl2ctIQ130eW0KDxgu4iq3kI6iUjrmsM7VfvPQNyV6LZMONdAHiXfHtifMnQsoK758NvWIi11b0luZSEGCZzXZSFxq/nPsizoWvtp+QXkEFKiE3JzzjPHFJKnoteE6TxuXwxnXaYjmptBxB3oI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789811686; c=relaxed/simple; bh=z/qQL0CPK3fkdmYWyHcS9nuOhJqJmsrxtpGJPsiPrxg=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=tpPXk4x8hBvJ+wgWi40CmqaciFI/f7e994oFx0IObG88wv56QbX59SN00nOuGV3t15oz+G1Ir++B0WtHfh0SAXlBdsflB7LI4SA8p5p/6cG+wtZB4CWXo+jVvYmrpP6k7ZvhtRlRSm4kNDeHXSzB8nbaMOdzIkOSSNLdva14vA8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com; spf=pass smtp.mailfrom=oss.qualcomm.com; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b=kHHgbV8X; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b=U5IUZO7f; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b="kHHgbV8X"; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b="U5IUZO7f" Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 68J6e29e2256184 for ; Sat, 19 Sep 2026 09:54:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= 53ePU/EqYBKQnQJgkF/MFGCIZg32zoI4VYtyTC3M5fU=; b=kHHgbV8XW4BLQhwR zhwf8R2pzBNi0nMPRSWluKnWhOR5SaFGHIaE3NhNxRofWIsnIcqj/efXEKyUwlZu 9fGHhf/1rih7ZrN4SjHVVntrX+tnDM/FLVorSEDFA05otRPX0kRz5G4GB/n++5W5 G2liyMqDBG5BPG1aiOQyrOYi/cjlYmDnXwpfP/RjGVHtBi0HncuwWTLYTgsS9SGJ XhgxwqRp33OfjqFQ/CTb8Ua+vokjppBzyZtDJL0IC346edBPHiPhTEXGCwIoaSmB wkPzOlpqnTGhNMWINX4pH8yuYT6W1PJCV8p+3aOmpDw3gs7yhqbfE8TAYD1EYyOM Oei6Yg== Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4gskp9rft8-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Sat, 19 Sep 2026 09:54:36 +0000 (GMT) Received: by mail-pj1-f71.google.com with SMTP id 98e67ed59e1d1-3823dcc1647so2262823a91.3 for ; Sat, 19 Sep 2026 02:54:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oss.qualcomm.com; s=google; t=1789811676; x=1790416476; darn=vger.kernel.org; h=content-transfer-encoding:content-type:in-reply-to:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:from:to:cc:subject:date:message-id:reply-to :content-type; bh=53ePU/EqYBKQnQJgkF/MFGCIZg32zoI4VYtyTC3M5fU=; b=U5IUZO7fTE1AxCC9rKBOG3furOaEcWddXHITeFe2ayPr9JANW4qEuoOoGy6vQP0Ncs 4bQIj5QPDgYymUpsUF8xbRuc7GYh2FzLqxady7iMQcHXQg0836fptsZPyb7Y4PDf0FTW rp/7m3bvf/6D+oqMk0GCvGSTIlg0pPjlUBUeQ/sTE8cDxB6n3wme1PTa9uA1GhVItJCP niuoklfLi59IJPa9rbZEK0SLK5F/nxupw4NKW5+lyeB6WBMcxUaCxCGg/9k72P27z5yN jlG/dXaY558UJFKyXtG01iOBUUQLNQymUtLa0I1gK9lBTM5ULf/ZCA271UqM83pvOBp/ sHWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789811676; x=1790416476; h=content-transfer-encoding:content-type:in-reply-to:from :content-language:references:cc:to:subject:user-agent:mime-version :date:message-id:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=53ePU/EqYBKQnQJgkF/MFGCIZg32zoI4VYtyTC3M5fU=; b=kq9jt+ERN+xp9uNJIZqw92J5sE9RIjOD57Ku4XbrVS5G/EirArRADN0RXwBXCjFv10 OMqRQC1RHbyIc1+pRwumLhtEG13t3dm7BjJU3PKG1nZ4ciDnMmrjapnVgonaNU2j7VFz iWHScLGGQK1Q3RZ8DwvgSU4dOrW6ZQs7CfsR0sopuJBCR9ye1nVU+pwdAn5XkKzJp5lJ B/tTxPH5Eat191TxrO/1ufZtStaZ2Wx5DmxvU6IcwIznK7mrn2ryY8kANywOu+4vbJc1 HvFsZLYUBqxogEqyY1L37gj4tuO0c7Pc9cmcQi8W4F/hADx+wil2gEk3y17wb3DOuAsd GYGA== X-Forwarded-Encrypted: i=1; AKwUvBx/yY7p/0T1uAiakREABEmBNizpTubUsaD9PG0FvOBbioavdbiY9pRpUHr3muHJEBhG9bOf5lxWzXAnaUY=@vger.kernel.org X-Gm-Message-State: AFuF++lGE9KCfJ7MMpjYYO8EAVcKYbMHvMBVJKir/WVC+9Y8BM2qkDWo qVVBHNYFsQptiQVt4rdAclk/IB8YdKcwdR0kq/JoCP5NzWR48M+4xiMx7L7I6H8AGJh1/1M9hFN wU/Cv5jqdp81NH/+MDerMoE2drpMSSG+MPR85zA3KDfnY6M9DJXtuOFeUbaAu/hSdxLA= X-Gm-Gg: AYBFou0Rv9moueK7W4asUeGzM3As3Zvx2DTVnDiLi30odry1H1E/EAeAbJ1TgD5OGfk Di5mmzTVuxaBxvzKRxt+xaz/csJSnyEPvKZV5w8vIAkt7sTY1iddZdNPhDqIZUtdqRWlJw+5S74 SPbvADRkGQ0MFRoI9h8nrq6c05R8eAT95Q2qJ0cGno6xM8vTbs1iHV+EAptjNAlqbGHm/ItuFVr cTT7LY4k0TXCC/N0pcJAB02Vx+OEHY2V24V27gQUHMI3chHEOSQbfjed5KwJIDSEW0pCnVuYfin UYALWV/BdW9eCAGfgAfdi0LHZbsig9sDpj6yANB96KS1OVlWFCFIJ3YZJfvuF8tZo8ApH64dCpM 5xGsYvFzTQhNctJcILbn3bl4GU+1Pis0teOhm X-Received: by 2002:a17:90a:bd6:b0:39e:6a81:5a97 with SMTP id 98e67ed59e1d1-39e6a815e70mr2667655a91.43.1789811675950; Sat, 19 Sep 2026 02:54:35 -0700 (PDT) X-Received: by 2002:a17:90a:bd6:b0:39e:6a81:5a97 with SMTP id 98e67ed59e1d1-39e6a815e70mr2667631a91.43.1789811675424; Sat, 19 Sep 2026 02:54:35 -0700 (PDT) Received: from [192.168.0.116] ([124.123.146.251]) by smtp.gmail.com with ESMTPSA id a92af1059eb24-144d55d45a6sm4526951c88.9.2026.09.19.02.54.28 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 19 Sep 2026 02:54:34 -0700 (PDT) Message-ID: <697a7c7b-5bf7-4279-898d-a8310f5a0a64@oss.qualcomm.com> Date: Sat, 19 Sep 2026 15:24:26 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 2/2] net: wwan: qcom_bam_dmux: Alloc RX buffers as a single coherent block To: Stephan Gerhold Cc: Stephan Gerhold , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Loic Poulain , Sergey Ryazanov , Johannes Berg , linux-arm-msm@vger.kernel.org, netdev@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, chris.lew@oss.qualcomm.com, Deepak Kumar Singh References: <20260714-qcom-bam-dmux-vmid-ext-v1-0-3f29da7cca76@oss.qualcomm.com> <20260714-qcom-bam-dmux-vmid-ext-v1-2-3f29da7cca76@oss.qualcomm.com> <63bae39b-1256-4e19-a9de-840d8752ec69@oss.qualcomm.com> Content-Language: en-US From: Vishnu Santhosh In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Proofpoint-ORIG-GUID: FERDj-T9VqWEYRba_dm_NooFiHAppbNt X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwOTE5MDE0MSBTYWx0ZWRfXxKYsiBrgHeEe oHdrooijLlefCeO4wLygaf/kg5hHcyqYSaIgtfchdOkxBtIrTT0oxc8KOnd65Ee8Z+HMp3Wf5dd neWDArNdx3senxhRi4dUhFm2aja8jLfWXIC/5DoTQNdML49r6nSYE15EuoEuMhBtuip7B95fHj9 jshiFsRaD1UNeGDJH9bz2ThxlsJJgWvwzz71wIxOuJJ3S1F++7aW7bQ4Yzdwn4OT8EqKIGKhWjg Ygi3qFUweqtb1pOreHY2E4UdxYUuWhR+frooH7Cyi1zKgc52gPkw0/bc/dRHwBsV0CE31zBEOMu zXynNpG2lylh0vL2x8aDTmPJ26GuySOKXwGVXLmzK92gNoS+MVKvgsLa7gKWYMi8DklCcixz2qN ZMloK8puXfOM8nZ8hxBTe9mka4F7FstghWy51kxPhrecRJoML7RKRgi5r3sZR9AwWz/XsmoPwJY crGnTGFYQBhuFcoHBHg== X-Proofpoint-Spam-Info: AW1haW4tMjYwOTE5MDE0MSBTYWx0ZWRfX3Kb5KBwH385j v8l1taTo+l8QzNswXssZ5upkTIEPsdpvm9mrAwxo3bfh+Y/OGo7cVLPtlyh07gyEL4qnHeafYrz UQiBjTrQ+ItBu6KMcgw5b/z0dkSBeV4= X-Authority-Analysis: v=2.4 cv=BKAmP1QG c=1 sm=1 tr=0 ts=6aae5bdc cx=c_pps a=UNFcQwm+pnOIJct1K4W+Mw==:117 a=K/78aEDNEn2Q/Yuv7mVN5Q==:17 a=IkcTkHD0fZMA:10 a=VdqzKS8jKosA:10 a=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=u7WPNUs3qKkmUXheDGA7:22 a=YMgV9FUhrdKAYTUUvYB2:22 a=07d9gI8wAAAA:8 a=EUspDBNiAAAA:8 a=vzY3hVqx5U8Bfi6rXLEA:9 a=QEXdDO2ut3YA:10 a=uKXjsCUrEbL0IQVhDsJ9:22 a=e2CUPOnPG4QKp8I52DXD:22 X-Proofpoint-GUID: FERDj-T9VqWEYRba_dm_NooFiHAppbNt X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-09-19_02,2026-09-16_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 priorityscore=1501 suspectscore=0 impostorscore=0 spamscore=0 malwarescore=0 bulkscore=0 adultscore=0 lowpriorityscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2609040000 definitions=main-2609190141 On 13-08-2026 06:10 pm, Stephan Gerhold wrote: > On Thu, Aug 13, 2026 at 02:35:01PM +0530, Vishnu Santhosh wrote: >> On 24-07-2026 03:04 pm, Stephan Gerhold wrote: >>> On Fri, Jul 24, 2026 at 10:16:31AM +0530, Vishnu Santhosh wrote: >>>> On 14-07-2026 01:05 pm, Stephan Gerhold wrote: >>>>> On Tue, Jul 14, 2026 at 11:02:32AM +0530, Vishnu Santhosh wrote: >>>>>> On Qualcomm SoCs where the modem (e.g. the mDSP on Shikra, VMID 43 / >>>>>> NAV) is the AXI master for BAM-DMUX RX transfers and the XPU enforces >>>>>> per-region access control, each individually DMA-mapped RX buffer >>>>>> requires its own XPU resource group (RG). With ~16 RGs available, the >>>>>> 32 per-buffer dma_map_single() calls exhaust the table and the first >>>>>> inbound transfer faults with an XPU violation. >>>>>> >>>>>> BAM-DMUX is a singleton (exactly one instance per SoC), so the >>>>>> destination VMID does not need to be a DT property; it is looked up >>>>>> from the compatible string's match data instead. Add struct >>>>>> bam_dmux_data with a single vmid field, and a shikra_data instance >>>>>> hardcoding QCOM_SCM_VMID_NAV for qcom,shikra-bam-dmux. >>>>>> >>>>>> When match data is present, allocate all BAM_DMUX_NUM_SKB RX buffers as >>>>>> a single contiguous dma_alloc_coherent() block and SCM-assign that >>>>>> block to HLOS plus the VMID once at probe. This reduces RG consumption >>>>>> from 32 to 1. The block is never reclaimed across a modem power cycle >>>>>> (bam_dmux_power_off() does not touch it), so the probe-time assignment >>>>>> covers every subsequent restart without re-assigning or reclaiming. It >>>>>> is reclaimed to HLOS only once, at remove or on a probe error, and if >>>>>> that reclaim fails it is leaked rather than returned to the page >>>>>> allocator. >>>>>> >>>>>> Each rx_skbs[] slot is pre-assigned its virtual and DMA address from >>>>>> the block, so no per-buffer mapping is needed at power-on. Because the >>>>>> coherent block is not page-backed, received payload is copied into a >>>>>> regular netdev skb before handoff to the network stack; this is an >>>>>> unavoidable extra copy on the XPU-enforced RX path. >>>>>> >>>>>> Platforms without match data are unaffected: rx_virt stays NULL, no >>>>>> coherent memory is allocated, and the per-buffer dma_map_single() path >>>>>> is unchanged. >>>>>> >>>>>> Co-developed-by: Deepak Kumar Singh >>>>>> Signed-off-by: Deepak Kumar Singh >>>>>> Signed-off-by: Vishnu Santhosh >>>>> So how do you handle TX buffers? Right now, they are just passed on from >>>>> the net subsystem. There can be up to 32 TX buffers in progress as well. >>>>> >>>>> Overall, I have mixed feelings about this patch. It looks reasonably >>>>> simple, but fundamentally I don't understand why we need to go back to >>>>> the old days of implementing protection using a highly limited MPU (in >>>>> your case: the xPU). >>>>> >>>>> Why does the setup of BAM-DMUX differ e.g. from the setup for the crypto >>>>> engine? Crypto is also using bam-dma, but it avoids this inflexibility >>>>> by making use of the &apps_smmu. Is BAM-DMUX not covered by the SMMU? Or >>>>> did you just decide to bypass the SMMU in this case? (If so: Why?) >>>> I checked with secure systems team on this. Crypto BAM is >>>> behind apps_smmu, so protection is enforced through the SMMU's Stage-2 >>>> page tables. >>>> >>>> A2 BAM (used by BAM-DMUX) is present in secure domain and does not >>>> support Stage-2 translation on this SoC, and there is no IOMMU domain >>>> that can be attached to it. The only protection mechanism available is >>>> the xPU. >>>> >>> Thanks for investigating this! >>> >>> So is this a hardware limitation or something you could change with a >>> firmware update? Could you move the A2 BAM out of the secure domain and >>> protect it via the IOMMU instead of the xPU mechanism? The other modern >>> platforms with IPA do not have this limitation, they can use the IOMMU >>> for this. >>> >>> We can try to support the xPU protection mechanism in the BAM-DMUX >>> driver, but it's pretty bad from a performance and memory usage point of >>> view if you need to copy buffers around multiple times. So if you have >>> some way to change this in the firmware (and there is still time to do >>> so before production boards ship), I would strongly recommend to >>> investigate that. >>> >> Based on what we confirmed with the Secure Systems team, this is a limitation >> of the current Shikra platform rather than something that can be addressed >> through a firmware-only update. >> >> The A2 BAM used by BAM-DMUX is not connected to an SMMU/IOMMU domain on Shikra, >> which means Stage-2 translation is not available for this path. As a result, >> it is not possible to move this path behind an IOMMU. >> >> Modern IPA-based platforms differ because their data paths are physically routed >> through SMMU interfaces and therefore do not rely on VMID/xPU ownership assignment >> for this type of access control. On Shikra, the A2 BAM path is protected using the >> xPU3 VM-based access-control model, where DDR memory access is restricted and >> granted through the request-based Hypervisor VM assignment framework. In contrast, >> older targets relied on the earlier xPU2 resource-sharing model, in which modem >> access did not require this type of explicit VM ownership configuration. This >> architectural difference explains why the issue does not occur on those older platforms. >> >> So, for this path on Shikra, SCM-driven VMID assignment remains the only practical solution. >> > Ok, thanks for looking into this further. > > Can you please try the following as alternative for the implementation? > > 1. Make sure that you have CONFIG_DMA_RESTRICTED_POOL=y. > 2. Define a restricted DMA pool for BAM-DMUX, e.g.: > > &{/reserved-memory} { > bam_dmux_pool: restricted-dma-pool { > compatible = "restricted-dma-pool"; > size = <0x40000>; /* 32*2K*2 = 128K minimum, but 256K might be safer */ > alignment = <...>; /* Check xPU alignment requirements */ > }; > }; > > 3. Assign to BAM-DMUX together with the qcom,vmid: > > &bam_dmux { > memory-region = <&bam_dmux_pool>; > qcom,vmid = ; > }; > > 4. Extend qcom_bam_dmux.c to look up the DMA pool address and make it > accessible using SCM: Call of_reserved_mem_lookup() to get the > region, then invoke qcom_scm_assign_mem() with that. > > 5. Keep RX/TX DMA code paths in qcom_bam_dmux.c unchanged. > > 6. When testing, make sure the kernel log contains > "Reserved memory: created restricted DMA pool at %pa, size %ld MiB" > > 7. Add memory-region and qcom,vmid as optional in dt-bindings. > Add dependency: if qcom,vmid is specified, memory-region must be > specified. > > In a quick test (without the VMID stuff) this worked quite well for me, > SWIOTLB should handle the copying behind the scenes without further > changes to the qcom_bam_dmux driver. > > I would prefer that over complicating the driver with two separate ways > of buffer management. AFAICT, the restricted DMA feature is meant for > this kind of setup where there is no IOMMU but the firmware can still > restrict memory accesses to a limited amount of regions, see > https://lwn.net/Articles/841916/ for a short introduction. > > Thanks, > Stephan Thanks for the detailed steps. I tested this approach on the Qualcomm Shikra platform, and it works as expected. With CONFIG_DMA_RESTRICTED_POOL enabled, following the approach, the kernel reports: software IO TLB: Reserved memory: created restricted DMA pool at 0x00000000fffc0000, size 0 MiB OF: reserved mem: initialized node restricted-dma-pool, compatible id restricted-dma-pool OF: reserved mem: 0x00000000fffc0000..0x00000000ffffffff (256 KiB) map non-reusable restricted-dma-pool The BAM DMA controller also reports: bam-dma-engine 6044000.dma-controller: assigned reserved memory node restricted-dma-pool The BAM-DMUX TX/RX DMA paths are working with this configuration. I will update v2 to use the restricted DMA pool approach as suggested. Thanks, Vishnu