From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6EDDB3E5EF1 for ; Mon, 15 Jun 2026 13:25:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.180.131 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781529954; cv=none; b=lmVyfCVcKLCF5HLdUHZpaztfWx1tQTg0EZ48XqhQ2sfNsheokcnEs0NLL9cUWvrNJ2g4dOjBIf0dvN+1phXchT3PMcMiHOIAIo1qcbP/uPgM1teb+sQaALYI4HtdFGby0djJx+FqkBo4OdA5atQRgbAjFP9EGrq1BuuuDD59cU0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781529954; c=relaxed/simple; bh=3xhHWlD8RQco2IA1KAQxhlJoKB5DW/j70I2e9jXDPe4=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=qYmbddBuVVvpcUaSg/Su+7xh7k/Aune3iFdOm4tjvb47yXvbRlF5CRYwvkCLkU7kPBG6h2zM88BO/5o8g1cf0a7mtVH+wx8knMqMcOKdHVv70A1NetilUU43RQGywpGB7fWPR7uP/desN7VcDTyo1GaCgt6rHS10FxBcHWZMwMw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com; spf=pass smtp.mailfrom=oss.qualcomm.com; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b=cBOata2h; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b=VCIEG+W5; arc=none smtp.client-ip=205.220.180.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b="cBOata2h"; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b="VCIEG+W5" Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 65FCauiT366583 for ; Mon, 15 Jun 2026 13:25:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= 8CeFE4vffm7VQHEzk2SuyKKfPfRD48OmgDxEnfneNJE=; b=cBOata2ha3mzT2r4 NpDWj8zrLNGSUg7IPgZNw0ZxMh6wdFvQ7Lpe2wW1dtVrVuOcnrGTF5DNORb2qj/j lhPgBcWvfKnVCNo09tuDB+QlOgNwJrQAAzAQGP+g2S/KIwJG0vrCDRdRterxoSce mu6gVgODNhntvCsNyxS7jFeGUw6Nw9UXl38pF5VViGcRYReqGyFK0eVvKw+NXluv Sl2VTpTp9xCRqpeWxRA+wz/w/6qg/RVbsihsQNPoCs00flPzB0mwsP/k4kTCV1Sl D6eV97jg4mA3A9eQGroeGpbgluG/ftmNzsq0XH0JusyxxIvLkAaj1P39+HjaJ5ns NHD68A== Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4etevx0tf7-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Mon, 15 Jun 2026 13:25:52 +0000 (GMT) Received: by mail-pj1-f71.google.com with SMTP id 98e67ed59e1d1-36db995d083so2353116a91.0 for ; Mon, 15 Jun 2026 06:25:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oss.qualcomm.com; s=google; t=1781529951; x=1782134751; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=8CeFE4vffm7VQHEzk2SuyKKfPfRD48OmgDxEnfneNJE=; b=VCIEG+W5dFBU7zILN0BbwUBRubesRG8hW/eStifBc02V+f1glW1/wY3LVHytnivR43 5qmA7EIGVpjMPjBMzlFo4RQVaYyM2WOsw93iaDoPVMfdJKo8EpVxw1MuyhfE0o4Q+1FW C5W/7Moy2tnePNnkTGWiTekDAjgL3TeteQq+0A+Su/0rwTVlLl4VTMqoaTbo5qESXpRu oIQe/qUGp7Y17ehso7FL0p6OSCHMpdBhdqbIoxlXTq16jy1+dXQU/Yv+bWMPec0Yd7eE tCPRVai2AHGJC37pZrA59NLqX8kInfnKR+TxR7FAIA1K1I2i8J+8BHjYjXtlu6PZzqFq w5uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781529951; x=1782134751; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8CeFE4vffm7VQHEzk2SuyKKfPfRD48OmgDxEnfneNJE=; b=imVJXFDiwhvP6JMsPnr+CWA1Gf2VyTcZnQPPhtPe5c78133RaI0IVWrJcY0Nmv6H6e r35nZmm4LBTGDmdR4V7II8NVOnBYr6Qb6wxx/KnM0nX32P89CiYPlQyy8hbFoG47u6gD jeQpOglCqY8CfN9aZCExymBtdk+ZAd/KoVi89zB03Al3zunRdxBHGtB5Gho4/4eAcwXA YlZlDfglq8VffVMCGLTuBQCGuxSAK5QZYmk+btR6WgF6SP/hfDys0F6CbYhW4Thn2esI LiCpNauNJO0Z3BENJVzfJ+FChgACtAPO+B0hK5nZI3+aa3QITyjkzIqTU5rqqFXCYpxZ qFaw== X-Forwarded-Encrypted: i=1; AFNElJ/ZjuKzhlv1DIJt5CiIxr22OUPGzFIiIkAdvud21q81x1FB/nKn+sygxB6ZEmoRiYWxCFixP3wTGD6kqw8=@vger.kernel.org X-Gm-Message-State: AOJu0Yzg8iHnunzi7OnYWPVjHFxCLg6qwNMaPLgZ1qc//LJ8P8/WKl8g mvosQA/0IeTQ3LlTiduBdW1OJVJ4pwdeCUoICjOtrCZDP1bEiTWkISoBQXeS9pvXfW1hVYUr5uH Lx/MUpcvdHHUaZkVvQru8HbESckmMLF+MUC2hXMzcGxH3oLBtlWtTBjyGoeo96VCjF10= X-Gm-Gg: Acq92OEebnOj+/MamNErqKKcMIQJ2Zs88sdbGeINcHDSIAkVePAoGm+4cf6dMD17ifU YDwlgx3ooo9lYSAcvYwkCMqoCej6WQVx4KcC6Rfs5D6Ozo/UMjffKzXPbdQWa3odJLFtZDd+htq m0CpZLSoKOhQB3ZWl1y/P5YpFZEqkqzvEmx0DZAcahD5rSeH6Ns0y0DHNnw/fEKqgJ8YRXnruFL lRLQjZYYqr6x2IUqjQcMPrezniwBVFAQIkNCEnUvmvPrN0DYKAYQCbBTYqpZboN0fs5Ni56N0hu 07opcwmtNKx7DQ2uXPp4YojaBxKr4OndFw8wE58vc273P0XTgs/HwoWeQyK98GN1N/TFeNCit1Q sR4NYTAiB+gRPl7ZNpqTHXu562Ly1oGiBLb2k2Lwo1rSvjnRqhA== X-Received: by 2002:a17:902:d4cf:b0:2bf:77b2:8b2d with SMTP id d9443c01a7336-2c4130100a2mr158666295ad.30.1781529951416; Mon, 15 Jun 2026 06:25:51 -0700 (PDT) X-Received: by 2002:a17:902:d4cf:b0:2bf:77b2:8b2d with SMTP id d9443c01a7336-2c4130100a2mr158665745ad.30.1781529950823; Mon, 15 Jun 2026 06:25:50 -0700 (PDT) Received: from [10.219.57.228] ([202.46.23.19]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2c42f7c58c3sm104350545ad.23.2026.06.15.06.25.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 15 Jun 2026 06:25:50 -0700 (PDT) Message-ID: <8518a085-b8b7-4ee6-b08c-8dde3971a6f1@oss.qualcomm.com> Date: Mon, 15 Jun 2026 18:55:45 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 2/6] iommu/arm-smmu: Add interconnect bandwidth voting support To: Dmitry Baryshkov Cc: Will Deacon , Robin Murphy , Joerg Roedel , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Bjorn Andersson , Konrad Dybcio , linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org References: <20260526-smmu_interconnect_addition-v2-0-2a6d8ca30d63@oss.qualcomm.com> <20260526-smmu_interconnect_addition-v2-2-2a6d8ca30d63@oss.qualcomm.com> <7xfxlxfqjcqdzl6gckaoyy2ioefglc7bgi66yv5khrbl6fi2zc@ivtiukdaj4jv> Content-Language: en-US From: Bibek Kumar Patro In-Reply-To: <7xfxlxfqjcqdzl6gckaoyy2ioefglc7bgi66yv5khrbl6fi2zc@ivtiukdaj4jv> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Proofpoint-ORIG-GUID: A7K5uWWV80UlqJQ4XcC66p6UHEMNok9W X-Proofpoint-GUID: A7K5uWWV80UlqJQ4XcC66p6UHEMNok9W X-Authority-Analysis: v=2.4 cv=f8t4wuyM c=1 sm=1 tr=0 ts=6a2ffd60 cx=c_pps a=UNFcQwm+pnOIJct1K4W+Mw==:117 a=j4ogTh8yFefVWWEFDRgCtg==:17 a=IkcTkHD0fZMA:10 a=FelO9ux0wxsA:10 a=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=u7WPNUs3qKkmUXheDGA7:22 a=yx91gb_oNiZeI1HMLzn7:22 a=EUspDBNiAAAA:8 a=he2KC8_JSkmjwAK-aeQA:9 a=QEXdDO2ut3YA:10 a=uKXjsCUrEbL0IQVhDsJ9:22 X-Proofpoint-Spam-Info: AW1haW4tMjYwNjE1MDE0MCBTYWx0ZWRfX7hvTrdynf+xb 2SqG0xl/gX04k0HPpEa56MaSfOsCIf2I3LMnHqYogMvB40ja3wL+InaYn+uC7Wox0Y2kvbb3u3L t9jnFHNrPRUS7bwQIczh5M6aA7zF64U= X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNjE1MDE0MCBTYWx0ZWRfX8MCSPttyzBpE 7yEi/U77zm5XXMO2hp7H3T2WmmLB7rmGX45BDHaKdkHIjMn9GoObpHzanXM+f6sM/aGK1PclOqv CjjY7Ko+825fGa3C+FAKv1tzR5P+U1S+Pc2PGRbdzSFsQ0DhC62exT3XekqAHInFlhjQAgOKW4f dyzHj4UVamw3fZG7fS8SL8frb9akO8oZvv+XsydE/yHuLFpNccwF910aniTpjh1gcyVHIfsfd/n zNYP6y+fpiUDMmzqfAzNn7TZpUQ4+wTMzoj/DmboVN1qKe/Vdh8YyYX7Nq2lQDrsPucUOp+Y/rB NoQuCmFJY6aNPIQfHWhadreZtdZs1pukp7c3fFJIWfgVaMZaqOkTnMl4AvaY4o7plI70nabxAx/ leXlbM5DLOFALd9XCqaUBWebXvYPsQtj57FAhl93HVIQ6Oe1dND4woYoVvOCasyqUABbdB9ZT14 1XvCqU5D1U01o45GVOg== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.125,FMLib:17.12.100.49 definitions=2026-06-15_03,2026-06-15_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 priorityscore=1501 bulkscore=0 lowpriorityscore=0 phishscore=0 adultscore=0 clxscore=1015 spamscore=0 suspectscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606040000 definitions=main-2606150140 On 6/8/2026 7:25 PM, Dmitry Baryshkov wrote: > On Tue, May 26, 2026 at 08:12:03PM +0530, Bibek Kumar Patro wrote: >> On some SoCs the SMMU registers require an active interconnect >> bandwidth vote to be accessible. While other clients typically >> satisfy this requirement implicitly, certain corner cases (e.g. >> during sleep/wakeup transitions) can leave the SMMU without a >> vote, causing intermittent register access failures. >> >> Add support for an optional interconnect path to the arm-smmu >> driver and vote for bandwidth while the SMMU is active. The path >> is acquired from DT if present and ignored otherwise. >> >> The bandwidth vote is enabled before accessing SMMU registers >> during probe and runtime resume, and released during runtime >> suspend and on error paths. >> >> Generally, from an architectural perspective, GEM_NOC and DDR are >> expected to have an active vote whenever the adreno_smmu block is >> powered on. In most common use cases, this requirement is implicitly >> satisfied because other GPU-related clients (for example, the GMU >> device) already hold a GEM_NOC vote when adreno_smmu is enabled. >> >> However, there are certain corner cases, such as during sleep/wakeup >> transitions, where the GEM_NOC vote can be removed before adreno_smmu >> is powered down. If adreno_smmu is then accessed while the interconnect >> vote is missing, it can lead to the observed failures. Because of the >> precise ordering involved, this scenario is difficult to reproduce >> consistently. >> (also GDSC is involved in adreno usecases can have an independent vote) >> >> Signed-off-by: Bibek Kumar Patro >> --- >> drivers/iommu/arm/arm-smmu/arm-smmu.c | 57 +++++++++++++++++++++++++++++++++-- >> drivers/iommu/arm/arm-smmu/arm-smmu.h | 2 ++ >> 2 files changed, 57 insertions(+), 2 deletions(-) >> >> diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c b/drivers/iommu/arm/arm-smmu/arm-smmu.c >> index 0bd21d206eb3e75c3b9fb1364cdc92e82c5aa499..07c7e44ec6a5bd1488f00f87d859a20495e46601 100644 >> --- a/drivers/iommu/arm/arm-smmu/arm-smmu.c >> +++ b/drivers/iommu/arm/arm-smmu/arm-smmu.c >> @@ -53,6 +53,11 @@ >> #define MSI_IOVA_BASE 0x8000000 >> #define MSI_IOVA_LENGTH 0x100000 >> >> +/* Interconnect bandwidth vote values for the SMMU register access path */ >> +#define ARM_SMMU_ICC_AVG_BW 0 >> +#define ARM_SMMU_ICC_PEAK_BW_HIGH 1000 > > totally random numbers, which might be different for non-Qualcomm platform. > >> +#define ARM_SMMU_ICC_PEAK_BW_LOW 0 >> + >> static int force_stage; >> module_param(force_stage, int, S_IRUGO); >> MODULE_PARM_DESC(force_stage, >> @@ -86,6 +91,36 @@ static inline void arm_smmu_rpm_put(struct arm_smmu_device *smmu) >> } >> } >> >> +static int arm_smmu_icc_get(struct arm_smmu_device *smmu) >> +{ >> + smmu->icc_path = devm_of_icc_get(smmu->dev, NULL); > > Is there always only one bus / path in question? > Yes for TCU, it needs to only have a vote on GEM_NOC interconnect while accessing the DDR in downstream path (client->TCU->DDR), which we are addressing here. Hence it's only one icc path in question here. Thanks & regards, Bibek >> + if (IS_ERR(smmu->icc_path)) { > > if (!IS_ERR(smmu->icc_path)) > return 0; > > int err = PTR_ERR(); > if (err == -ENODEV) { > icc_path = NULL; > return 0; > } > > return dev_err_probe(); > > >> + int err = PTR_ERR(smmu->icc_path); >> + >> + if (err == -ENODEV) { >> + smmu->icc_path = NULL; >> + return 0; >> + } >> + return dev_err_probe(smmu->dev, err, >> + "failed to get interconnect path\n"); >> + } >> + return 0; >> +} >> + >> +static void arm_smmu_icc_enable(struct arm_smmu_device *smmu) >> +{ >> + if (smmu->icc_path) > > Drop the if. > >> + WARN_ON(icc_set_bw(smmu->icc_path, ARM_SMMU_ICC_AVG_BW, >> + ARM_SMMU_ICC_PEAK_BW_HIGH)); > > WARN_ON_ONCE()? > > Pass the error to the caller. > > >> +} >> + >> +static void arm_smmu_icc_disable(struct arm_smmu_device *smmu) >> +{ >> + if (smmu->icc_path) > > Drop the if. > >> + WARN_ON(icc_set_bw(smmu->icc_path, ARM_SMMU_ICC_AVG_BW, >> + ARM_SMMU_ICC_PEAK_BW_LOW)); > > Pass the error to the caller. > >> +} >> + >> static void arm_smmu_rpm_use_autosuspend(struct arm_smmu_device *smmu) >> { >> /* >> @@ -2189,6 +2224,17 @@ static int arm_smmu_device_probe(struct platform_device *pdev) >> if (err) >> return err; >> >> + /* >> + * Acquire and vote the interconnect path before accessing any SMMU >> + * registers (including ARM_SMMU_GR0_ID0 in arm_smmu_device_cfg_probe). >> + */ >> + err = arm_smmu_icc_get(smmu); >> + if (err) { >> + clk_bulk_disable_unprepare(smmu->num_clks, smmu->clks); >> + return err; >> + } >> + arm_smmu_icc_enable(smmu); > > Handle the error. > >> + >> err = arm_smmu_device_cfg_probe(smmu); >> if (err) >> return err; >> @@ -2273,8 +2319,10 @@ static void arm_smmu_device_shutdown(struct platform_device *pdev) >> >> if (pm_runtime_enabled(smmu->dev)) >> pm_runtime_force_suspend(smmu->dev); >> - else >> + else { >> clk_bulk_disable(smmu->num_clks, smmu->clks); >> + arm_smmu_icc_disable(smmu); > > Handle the error. > > etc. > >> + } >> >> clk_bulk_unprepare(smmu->num_clks, smmu->clks); >> } >