From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 387CF1EB2E; Mon, 9 Dec 2024 12:07:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733746064; cv=none; b=QzQJ8wGH1+XWA41qHr5YdmcQ88CkgL2P7Yxr0bRRpi78+fDyx9APUKoHwgCHTvx3IXaYXnw7G6sk4OHkfduqV7INpbbJBO1TYm1QRvXRTrYdEwdlO8XTjGyfmktBb1H0FmzdoMdQcM2/sXpMDKyQd+zFnYbsKZSqZqSXYvPTQ/8= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733746064; c=relaxed/simple; bh=UkRwe0S+AAePSckd9pxmBAifMfyqiHWUtksIZCLtuTk=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=BxjGAyyx6FzxPFwCdqtcVz7eJpwl/pFeMJeOGMod8LUJ0uVXDKabXjRcAgoVWw1xg72K+aNOMpwa0GpG1DxNKTPrPrOCuP7r85ea6VF5DYtRMw1c6JuDwkOrvwfwLvU8buom0Jldo8MPV8N+8WXNuxlG3AIbCAr9C2rH1UvNLUc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com; spf=pass smtp.mailfrom=quicinc.com; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b=Xya126VH; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=quicinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="Xya126VH" Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B9AVuvL020792; Mon, 9 Dec 2024 12:07:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= uzsBsmqo2tei+FselTp4GDT4VT79/TWZH8HwpCsX8ek=; b=Xya126VHC7RORmTr 45E/5fLoXMRDzH1SP37B18GAaYEUihRfS09boHciTXcmJYGn3etto4K2Ca9dmyOv 9f3euq+Wvt0YASi8KTyOTWxWbClODHdua/QFmjEXMsomqBmA195sX7stQjovwRry PxSSgNOZeya4IgTWWndOMbvFkpKjeMBNiCJBns6s67LTBf9gOryIyGiejZ/kG/Br 5FR5jVcD4Fb+2y8LCyOZQ2iI7idz7o4hjUNntsycQVnqym7rSzcsuyzvBJF8pV5I EhBkDTKm+ef2OgrOTp8TLIY7Q86UciRNGiF1hoCqr0zJwUgxGvWWH+5EVHvfBRkw 7RhBUw== Received: from nalasppmta04.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 43cf4e4jfr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 09 Dec 2024 12:07:29 +0000 (GMT) Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA04.qualcomm.com (8.18.1.2/8.18.1.2) with ESMTPS id 4B9C7SWx014540 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 9 Dec 2024 12:07:28 GMT Received: from [10.216.3.14] (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.9; Mon, 9 Dec 2024 04:07:23 -0800 Message-ID: <4bf0acca-bdaf-45be-bc03-d5c1ec23a398@quicinc.com> Date: Mon, 9 Dec 2024 17:37:20 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4 2/7] drm/msm: adreno: add plumbing to generate bandwidth vote table for GMU To: Neil Armstrong , Rob Clark , Sean Paul , Konrad Dybcio , Abhinav Kumar , "Dmitry Baryshkov" , Marijn Suijten , David Airlie , "Simona Vetter" , Bjorn Andersson , Rob Herring , Krzysztof Kozlowski , Conor Dooley CC: , , , , References: <20241205-topic-sm8x50-gpu-bw-vote-v4-0-9650d15dd435@linaro.org> <20241205-topic-sm8x50-gpu-bw-vote-v4-2-9650d15dd435@linaro.org> Content-Language: en-US From: Akhil P Oommen In-Reply-To: <20241205-topic-sm8x50-gpu-bw-vote-v4-2-9650d15dd435@linaro.org> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-ClientProxiedBy: nasanex01b.na.qualcomm.com (10.46.141.250) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: gsbIj9lkGiNEvKpF9HEjzF9zzYXGD5c3 X-Proofpoint-ORIG-GUID: gsbIj9lkGiNEvKpF9HEjzF9zzYXGD5c3 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 impostorscore=0 spamscore=0 priorityscore=1501 mlxscore=0 adultscore=0 phishscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2411120000 definitions=main-2412090095 On 12/5/2024 8:31 PM, Neil Armstrong wrote: > The Adreno GPU Management Unit (GMU) can also scale DDR Bandwidth along > the Frequency and Power Domain level, but by default we leave the > OPP core scale the interconnect ddr path. > > While scaling via the interconnect path was sufficient, newer GPUs > like the A750 requires specific vote paremeters and bandwidth to > achieve full functionality. > > In order to calculate vote values used by the GPU Management > Unit (GMU), we need to parse all the possible OPP Bandwidths and > create a vote value to be sent to the appropriate Bus Control > Modules (BCMs) declared in the GPU info struct. > > This vote value is called IB, while on the other side the GMU also > takes another vote called AB which is a 16bit quantized value > of the floor bandwidth against the maximum supported bandwidth. > The AB vote will be calculated later when setting the frequency. > > The vote array will then be used to dynamically generate the GMU > bw_table sent during the GMU power-up. > > Signed-off-by: Neil Armstrong Reviewed-by: Akhil P Oommen -Akhil > --- > drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 144 ++++++++++++++++++++++++++++++++++ > drivers/gpu/drm/msm/adreno/a6xx_gmu.h | 13 +++ > drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 1 + > 3 files changed, 158 insertions(+) > > diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c > index 14db7376c712d19446b38152e480bd5a1e0a5198..36696d372a42a27b26a018b19e73bc6d8a4a5235 100644 > --- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c > +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c > @@ -9,6 +9,7 @@ > #include > #include > #include > +#include > #include > > #include "a6xx_gpu.h" > @@ -1287,6 +1288,101 @@ static int a6xx_gmu_memory_probe(struct a6xx_gmu *gmu) > return 0; > } > > +/** > + * struct bcm_db - Auxiliary data pertaining to each Bus Clock Manager (BCM) > + * @unit: divisor used to convert bytes/sec bw value to an RPMh msg > + * @width: multiplier used to convert bytes/sec bw value to an RPMh msg > + * @vcd: virtual clock domain that this bcm belongs to > + * @reserved: reserved field > + */ > +struct bcm_db { > + __le32 unit; > + __le16 width; > + u8 vcd; > + u8 reserved; > +}; > + > +static int a6xx_gmu_rpmh_bw_votes_init(const struct a6xx_info *info, > + struct a6xx_gmu *gmu) > +{ > + const struct bcm_db *bcm_data[GMU_MAX_BCMS] = { 0 }; > + unsigned int bcm_index, bw_index, bcm_count = 0; > + > + if (!info->bcms) > + return 0; > + > + /* Retrieve BCM data from cmd-db */ > + for (bcm_index = 0; bcm_index < GMU_MAX_BCMS; bcm_index++) { > + size_t count; > + > + /* Stop at first unconfigured bcm */ > + if (!info->bcms[bcm_index].name) > + break; > + > + bcm_data[bcm_index] = cmd_db_read_aux_data( > + info->bcms[bcm_index].name, > + &count); > + if (IS_ERR(bcm_data[bcm_index])) > + return PTR_ERR(bcm_data[bcm_index]); > + > + if (!count) > + return -EINVAL; > + > + ++bcm_count; > + } > + > + /* Generate BCM votes values for each bandwidth & BCM */ > + for (bw_index = 0; bw_index < gmu->nr_gpu_bws; bw_index++) { > + u32 *data = gmu->gpu_ib_votes[bw_index]; > + u32 bw = gmu->gpu_bw_table[bw_index]; > + > + /* Calculations loosely copied from bcm_aggregate() & tcs_cmd_gen() */ > + for (bcm_index = 0; bcm_index < bcm_count; bcm_index++) { > + bool commit = false; > + u64 peak; > + u32 vote; > + > + /* Skip unconfigured BCM */ > + if (!bcm_data[bcm_index]) > + continue; > + > + if (bcm_index == bcm_count - 1 || > + (bcm_data[bcm_index + 1] && > + bcm_data[bcm_index]->vcd != bcm_data[bcm_index + 1]->vcd)) > + commit = true; > + > + if (!bw) { > + data[bcm_index] = BCM_TCS_CMD(commit, false, 0, 0); > + continue; > + } > + > + if (info->bcms[bcm_index].fixed) { > + u32 perfmode = 0; > + > + if (bw >= info->bcms[bcm_index].perfmode_bw) > + perfmode = info->bcms[bcm_index].perfmode; > + > + data[bcm_index] = BCM_TCS_CMD(commit, true, 0, perfmode); > + continue; > + } > + > + /* Multiply the bandwidth by the width of the connection */ > + peak = (u64)bw * le16_to_cpu(bcm_data[bcm_index]->width); > + do_div(peak, info->bcms[bcm_index].buswidth); > + > + /* Input bandwidth value is in KBps, scale the value to BCM unit */ > + peak *= 1000ULL; > + do_div(peak, le32_to_cpu(bcm_data[bcm_index]->unit)); > + > + vote = clamp(peak, 1, BCM_TCS_CMD_VOTE_MASK); > + > + data[bcm_index] = BCM_TCS_CMD(commit, true, vote, vote); > + } > + } > + > + return 0; > +} > + > /* Return the 'arc-level' for the given frequency */ > static unsigned int a6xx_gmu_get_arc_level(struct device *dev, > unsigned long freq) > @@ -1390,12 +1486,15 @@ static int a6xx_gmu_rpmh_arc_votes_init(struct device *dev, u32 *votes, > * The GMU votes with the RPMh for itself and on behalf of the GPU but we need > * to construct the list of votes on the CPU and send it over. Query the RPMh > * voltage levels and build the votes > + * The GMU can also vote for DDR interconnects, use the OPP bandwidth entries > + * and BCM parameters to build the votes. > */ > > static int a6xx_gmu_rpmh_votes_init(struct a6xx_gmu *gmu) > { > struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu); > struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; > + const struct a6xx_info *info = adreno_gpu->info->a6xx; > struct msm_gpu *gpu = &adreno_gpu->base; > int ret; > > @@ -1407,6 +1506,10 @@ static int a6xx_gmu_rpmh_votes_init(struct a6xx_gmu *gmu) > ret |= a6xx_gmu_rpmh_arc_votes_init(gmu->dev, gmu->cx_arc_votes, > gmu->gmu_freqs, gmu->nr_gmu_freqs, "cx.lvl"); > > + /* Build the interconnect votes */ > + if (info->bcms && gmu->nr_gpu_bws > 1) > + ret |= a6xx_gmu_rpmh_bw_votes_init(info, gmu); > + > return ret; > } > > @@ -1442,10 +1545,43 @@ static int a6xx_gmu_build_freq_table(struct device *dev, unsigned long *freqs, > return index; > } > > +static int a6xx_gmu_build_bw_table(struct device *dev, unsigned long *bandwidths, > + u32 size) > +{ > + int count = dev_pm_opp_get_opp_count(dev); > + struct dev_pm_opp *opp; > + int i, index = 0; > + unsigned int bandwidth = 1; > + > + /* > + * The OPP table doesn't contain the "off" bandwidth level so we need to > + * add 1 to the table size to account for it > + */ > + > + if (WARN(count + 1 > size, > + "The GMU bandwidth table is being truncated\n")) > + count = size - 1; > + > + /* Set the "off" bandwidth */ > + bandwidths[index++] = 0; > + > + for (i = 0; i < count; i++) { > + opp = dev_pm_opp_find_bw_ceil(dev, &bandwidth, 0); > + if (IS_ERR(opp)) > + break; > + > + dev_pm_opp_put(opp); > + bandwidths[index++] = bandwidth++; > + } > + > + return index; > +} > + > static int a6xx_gmu_pwrlevels_probe(struct a6xx_gmu *gmu) > { > struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu); > struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; > + const struct a6xx_info *info = adreno_gpu->info->a6xx; > struct msm_gpu *gpu = &adreno_gpu->base; > > int ret = 0; > @@ -1472,6 +1608,14 @@ static int a6xx_gmu_pwrlevels_probe(struct a6xx_gmu *gmu) > > gmu->current_perf_index = gmu->nr_gpu_freqs - 1; > > + /* > + * The GMU also handles GPU Interconnect Votes so build a list > + * of DDR bandwidths from the GPU OPP table > + */ > + if (info->bcms) > + gmu->nr_gpu_bws = a6xx_gmu_build_bw_table(&gpu->pdev->dev, > + gmu->gpu_bw_table, ARRAY_SIZE(gmu->gpu_bw_table)); > + > /* Build the list of RPMh votes that we'll send to the GMU */ > return a6xx_gmu_rpmh_votes_init(gmu); > } > diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.h b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h > index 88f18ea6a38a08b5b171709e5020010947a5d347..2062a2be224768c1937d7768f7b8439920e9e127 100644 > --- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.h > +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h > @@ -21,6 +21,15 @@ struct a6xx_gmu_bo { > > #define GMU_MAX_GX_FREQS 16 > #define GMU_MAX_CX_FREQS 4 > +#define GMU_MAX_BCMS 3 > + > +struct a6xx_bcm { > + char *name; > + unsigned int buswidth; > + bool fixed; > + unsigned int perfmode; > + unsigned int perfmode_bw; > +}; > > /* > * These define the different GMU wake up options - these define how both the > @@ -85,6 +94,10 @@ struct a6xx_gmu { > unsigned long gpu_freqs[GMU_MAX_GX_FREQS]; > u32 gx_arc_votes[GMU_MAX_GX_FREQS]; > > + int nr_gpu_bws; > + unsigned long gpu_bw_table[GMU_MAX_GX_FREQS]; > + u32 gpu_ib_votes[GMU_MAX_GX_FREQS][GMU_MAX_BCMS]; > + > int nr_gmu_freqs; > unsigned long gmu_freqs[GMU_MAX_CX_FREQS]; > u32 cx_arc_votes[GMU_MAX_CX_FREQS]; > diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h > index 4aceffb6aae89c781facc2a6e4a82b20b341b6cb..9201a53dd341bf432923ffb44947e015208a3d02 100644 > --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h > +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h > @@ -44,6 +44,7 @@ struct a6xx_info { > u32 gmu_chipid; > u32 gmu_cgc_mode; > u32 prim_fifo_threshold; > + const struct a6xx_bcm *bcms; > }; > > struct a6xx_gpu { >