[PATCH 0/4] drm/msm/gpu: add support for ocmem interconnect

[PATCH 0/4] drm/msm/gpu: add support for ocmem interconnect

[Brian Masney]

Some A3xx and all A4xx Adreno GPUs do not have GMEM inside the GPU core
and must use the On Chip MEMory (OCMEM) in order to be functional.
There's a separate interconnect path that needs to be setup to OCMEM.
This patch series adds support for that path, and sets the votes for the
two interconnect paths to the highest speed for a3xx and a4xx-based
platforms.

Brian Masney (4):
  dt-bindings: drm/msm/gpu: document second interconnect
  drm/msm/gpu: add support for ocmem interconnect path
  drm/msm/a3xx: set interconnect bandwidth vote
  drm/msm/a4xx: set interconnect bandwidth vote

 .../devicetree/bindings/display/msm/gpu.txt   |  6 +++++-
 drivers/gpu/drm/msm/adreno/a3xx_gpu.c         |  8 ++++++++
 drivers/gpu/drm/msm/adreno/a4xx_gpu.c         |  8 ++++++++
 drivers/gpu/drm/msm/adreno/a6xx_gmu.c         |  6 +++---
 drivers/gpu/drm/msm/adreno/adreno_gpu.c       | 20 +++++++++++++++----
 drivers/gpu/drm/msm/msm_gpu.h                 |  3 ++-
 6 files changed, 42 insertions(+), 9 deletions(-)

-- 
2.21.0

[PATCH 1/4] dt-bindings: drm/msm/gpu: document second interconnect

[Brian Masney]

Some A3xx and all A4xx Adreno GPUs do not have GMEM inside the GPU core
and must use the On Chip MEMory (OCMEM) in order to be functional.
There's a separate interconnect path that needs to be setup to OCMEM.
Let's document this second interconnect path that's available.

Signed-off-by: Brian Masney <masneyb@onstation.org>
---
 Documentation/devicetree/bindings/display/msm/gpu.txt | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/Documentation/devicetree/bindings/display/msm/gpu.txt b/Documentation/devicetree/bindings/display/msm/gpu.txt
index 2b8fd26c43b0..3e6cd3f64a78 100644
--- a/Documentation/devicetree/bindings/display/msm/gpu.txt
+++ b/Documentation/devicetree/bindings/display/msm/gpu.txt
@@ -23,7 +23,10 @@ Required properties:
 - iommus: optional phandle to an adreno iommu instance
 - operating-points-v2: optional phandle to the OPP operating points
 - interconnects: optional phandle to an interconnect provider.  See
-  ../interconnect/interconnect.txt for details.
+  ../interconnect/interconnect.txt for details. Some A3xx and all A4xx platforms
+  will have two paths; all others will have one path.
+- interconnect-names: The names of the interconnect paths that correspond to the
+  interconnects property. Values must be gfx-mem and ocmem.
 - qcom,gmu: For GMU attached devices a phandle to the GMU device that will
   control the power for the GPU. Applicable targets:
     - qcom,adreno-630.2
@@ -76,6 +79,7 @@ Example a6xx (with GMU):
 		operating-points-v2 = <&gpu_opp_table>;
 
 		interconnects = <&rsc_hlos MASTER_GFX3D &rsc_hlos SLAVE_EBI1>;
+		interconnect-names = "gfx-mem";
 
 		qcom,gmu = <&gmu>;
 
-- 
2.21.0

[PATCH 2/4] drm/msm/gpu: add support for ocmem interconnect path

[Brian Masney]

Some A3xx and all A4xx Adreno GPUs do not have GMEM inside the GPU core
and must use the On Chip MEMory (OCMEM) in order to be functional.
There's a separate interconnect path that needs to be setup to OCMEM.
Add support for this second path to the GPU core.

In the downstream MSM 3.4 sources, the two interconnect paths for the
GPU are between:

  - MSM_BUS_MASTER_GRAPHICS_3D and MSM_BUS_SLAVE_EBI_CH0
  - MSM_BUS_MASTER_V_OCMEM_GFX3D and MSM_BUS_SLAVE_OCMEM

Signed-off-by: Brian Masney <masneyb@onstation.org>
---
 drivers/gpu/drm/msm/adreno/a6xx_gmu.c   |  6 +++---
 drivers/gpu/drm/msm/adreno/adreno_gpu.c | 20 ++++++++++++++++----
 drivers/gpu/drm/msm/msm_gpu.h           |  3 ++-
 3 files changed, 21 insertions(+), 8 deletions(-)

diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
index 85f14feafdec..7885e382fb8f 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
@@ -132,7 +132,7 @@ static void __a6xx_gmu_set_freq(struct a6xx_gmu *gmu, int index)
 	 * Eventually we will want to scale the path vote with the frequency but
 	 * for now leave it at max so that the performance is nominal.
 	 */
-	icc_set_bw(gpu->icc_path, 0, MBps_to_icc(7216));
+	icc_set_bw(gpu->gfx_mem_icc_path, 0, MBps_to_icc(7216));
 }
 
 void a6xx_gmu_set_freq(struct msm_gpu *gpu, unsigned long freq)
@@ -714,7 +714,7 @@ int a6xx_gmu_resume(struct a6xx_gpu *a6xx_gpu)
 	}
 
 	/* Set the bus quota to a reasonable value for boot */
-	icc_set_bw(gpu->icc_path, 0, MBps_to_icc(3072));
+	icc_set_bw(gpu->gfx_mem_icc_path, 0, MBps_to_icc(3072));
 
 	/* Enable the GMU interrupt */
 	gmu_write(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_CLR, ~0);
@@ -858,7 +858,7 @@ int a6xx_gmu_stop(struct a6xx_gpu *a6xx_gpu)
 		a6xx_gmu_shutdown(gmu);
 
 	/* Remove the bus vote */
-	icc_set_bw(gpu->icc_path, 0, 0);
+	icc_set_bw(gpu->gfx_mem_icc_path, 0, 0);
 
 	/*
 	 * Make sure the GX domain is off before turning off the GMU (CX)
diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
index 0783e4b5486a..d1cc021c012c 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
@@ -887,9 +887,20 @@ static int adreno_get_pwrlevels(struct device *dev,
 	DBG("fast_rate=%u, slow_rate=27000000", gpu->fast_rate);
 
 	/* Check for an interconnect path for the bus */
-	gpu->icc_path = of_icc_get(dev, NULL);
-	if (IS_ERR(gpu->icc_path))
-		gpu->icc_path = NULL;
+	gpu->gfx_mem_icc_path = of_icc_get(dev, "gfx-mem");
+	if (!gpu->gfx_mem_icc_path) {
+		/*
+		 * Keep compatbility with device trees that don't have an
+		 * interconnect-names property.
+		 */
+		gpu->gfx_mem_icc_path = of_icc_get(dev, NULL);
+	}
+	if (IS_ERR(gpu->gfx_mem_icc_path))
+		gpu->gfx_mem_icc_path = NULL;
+
+	gpu->ocmem_icc_path = of_icc_get(dev, "ocmem");
+	if (IS_ERR(gpu->ocmem_icc_path))
+		gpu->ocmem_icc_path = NULL;
 
 	return 0;
 }
@@ -976,7 +987,8 @@ void adreno_gpu_cleanup(struct adreno_gpu *adreno_gpu)
 	for (i = 0; i < ARRAY_SIZE(adreno_gpu->info->fw); i++)
 		release_firmware(adreno_gpu->fw[i]);
 
-	icc_put(gpu->icc_path);
+	icc_put(gpu->gfx_mem_icc_path);
+	icc_put(gpu->ocmem_icc_path);
 
 	msm_gpu_cleanup(&adreno_gpu->base);
 }
diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h
index ab8f0f9c9dc8..e72e56f7b0ef 100644
--- a/drivers/gpu/drm/msm/msm_gpu.h
+++ b/drivers/gpu/drm/msm/msm_gpu.h
@@ -111,7 +111,8 @@ struct msm_gpu {
 	struct clk *ebi1_clk, *core_clk, *rbbmtimer_clk;
 	uint32_t fast_rate;
 
-	struct icc_path *icc_path;
+	struct icc_path *gfx_mem_icc_path;
+	struct icc_path *ocmem_icc_path;
 
 	/* Hang and Inactivity Detection:
 	 */
-- 
2.21.0

Re: [Freedreno] [PATCH 2/4] drm/msm/gpu: add support for ocmem interconnect path

[Jordan Crouse]

On Sun, Nov 17, 2019 at 06:48:23AM -0500, Brian Masney wrote:
> Some A3xx and all A4xx Adreno GPUs do not have GMEM inside the GPU core
> and must use the On Chip MEMory (OCMEM) in order to be functional.
> There's a separate interconnect path that needs to be setup to OCMEM.
> Add support for this second path to the GPU core.
> 
> In the downstream MSM 3.4 sources, the two interconnect paths for the
> GPU are between:
> 
>   - MSM_BUS_MASTER_GRAPHICS_3D and MSM_BUS_SLAVE_EBI_CH0
>   - MSM_BUS_MASTER_V_OCMEM_GFX3D and MSM_BUS_SLAVE_OCMEM
> 
> Signed-off-by: Brian Masney <masneyb@onstation.org>
> ---
>  drivers/gpu/drm/msm/adreno/a6xx_gmu.c   |  6 +++---
>  drivers/gpu/drm/msm/adreno/adreno_gpu.c | 20 ++++++++++++++++----
>  drivers/gpu/drm/msm/msm_gpu.h           |  3 ++-
>  3 files changed, 21 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
> index 85f14feafdec..7885e382fb8f 100644
> --- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
> +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
> @@ -132,7 +132,7 @@ static void __a6xx_gmu_set_freq(struct a6xx_gmu *gmu, int index)
>  	 * Eventually we will want to scale the path vote with the frequency but
>  	 * for now leave it at max so that the performance is nominal.
>  	 */
> -	icc_set_bw(gpu->icc_path, 0, MBps_to_icc(7216));
> +	icc_set_bw(gpu->gfx_mem_icc_path, 0, MBps_to_icc(7216));
>  }
>  
>  void a6xx_gmu_set_freq(struct msm_gpu *gpu, unsigned long freq)
> @@ -714,7 +714,7 @@ int a6xx_gmu_resume(struct a6xx_gpu *a6xx_gpu)
>  	}
>  
>  	/* Set the bus quota to a reasonable value for boot */
> -	icc_set_bw(gpu->icc_path, 0, MBps_to_icc(3072));
> +	icc_set_bw(gpu->gfx_mem_icc_path, 0, MBps_to_icc(3072));
>  
>  	/* Enable the GMU interrupt */
>  	gmu_write(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_CLR, ~0);
> @@ -858,7 +858,7 @@ int a6xx_gmu_stop(struct a6xx_gpu *a6xx_gpu)
>  		a6xx_gmu_shutdown(gmu);
>  
>  	/* Remove the bus vote */
> -	icc_set_bw(gpu->icc_path, 0, 0);
> +	icc_set_bw(gpu->gfx_mem_icc_path, 0, 0);
>  
>  	/*
>  	 * Make sure the GX domain is off before turning off the GMU (CX)
> diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
> index 0783e4b5486a..d1cc021c012c 100644
> --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c
> +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
> @@ -887,9 +887,20 @@ static int adreno_get_pwrlevels(struct device *dev,
>  	DBG("fast_rate=%u, slow_rate=27000000", gpu->fast_rate);
>  
>  	/* Check for an interconnect path for the bus */
> -	gpu->icc_path = of_icc_get(dev, NULL);
> -	if (IS_ERR(gpu->icc_path))
> -		gpu->icc_path = NULL;
> +	gpu->gfx_mem_icc_path = of_icc_get(dev, "gfx-mem");
> +	if (!gpu->gfx_mem_icc_path) {
> +		/*
> +		 * Keep compatbility with device trees that don't have an
> +		 * interconnect-names property.
> +		 */
> +		gpu->gfx_mem_icc_path = of_icc_get(dev, NULL);
> +	}
> +	if (IS_ERR(gpu->gfx_mem_icc_path))
> +		gpu->gfx_mem_icc_path = NULL;
> +
> +	gpu->ocmem_icc_path = of_icc_get(dev, "ocmem");
> +	if (IS_ERR(gpu->ocmem_icc_path))
> +		gpu->ocmem_icc_path = NULL;

This is the part where I am reminded that icc_set_bw doesn't check
IS_ERR_OR_NULL and even worse, icc_put warns on IS_ERR and it makes 
me grumble.

>  	return 0;
>  }
> @@ -976,7 +987,8 @@ void adreno_gpu_cleanup(struct adreno_gpu *adreno_gpu)
>  	for (i = 0; i < ARRAY_SIZE(adreno_gpu->info->fw); i++)
>  		release_firmware(adreno_gpu->fw[i]);
>  
> -	icc_put(gpu->icc_path);
> +	icc_put(gpu->gfx_mem_icc_path);
> +	icc_put(gpu->ocmem_icc_path);
>  
>  	msm_gpu_cleanup(&adreno_gpu->base);
>  }
> diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h
> index ab8f0f9c9dc8..e72e56f7b0ef 100644
> --- a/drivers/gpu/drm/msm/msm_gpu.h
> +++ b/drivers/gpu/drm/msm/msm_gpu.h
> @@ -111,7 +111,8 @@ struct msm_gpu {
>  	struct clk *ebi1_clk, *core_clk, *rbbmtimer_clk;
>  	uint32_t fast_rate;
>  
> -	struct icc_path *icc_path;
> +	struct icc_path *gfx_mem_icc_path;
> +	struct icc_path *ocmem_icc_path;

I'm not sure if we want a bulk rename of the main path.  icc_path and
ocmem_icc_path seem to be reasonable names and not overly confusing especially
if we added some documentation to the header).


>  	/* Hang and Inactivity Detection:
>  	 */
> -- 
> 2.21.0
> 
> _______________________________________________
> Freedreno mailing list
> Freedreno@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/freedreno

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project

[PATCH 3/4] drm/msm/a3xx: set interconnect bandwidth vote

[Brian Masney]

Set the two interconnect paths for the GPU to maximum speed for now to
work towards getting the GPU working upstream. We can revisit a later
time to optimize this for battery life.

Signed-off-by: Brian Masney <masneyb@onstation.org>
---
 drivers/gpu/drm/msm/adreno/a3xx_gpu.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/gpu/drm/msm/adreno/a3xx_gpu.c b/drivers/gpu/drm/msm/adreno/a3xx_gpu.c
index 07ddcc529573..f05adf9bc752 100644
--- a/drivers/gpu/drm/msm/adreno/a3xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a3xx_gpu.c
@@ -504,6 +504,14 @@ struct msm_gpu *a3xx_gpu_init(struct drm_device *dev)
 		DRM_DEV_ERROR(dev->dev, "No memory protection without IOMMU\n");
 	}
 
+	/*
+	 * Set the ICC path to maximum speed for now by multiplying the fastest
+	 * frequency by the bus width (8). We'll want to scale this later on to
+	 * improve battery life.
+	 */
+	icc_set_bw(gpu->gfx_mem_icc_path, 0, Bps_to_icc(gpu->fast_rate) * 8);
+	icc_set_bw(gpu->ocmem_icc_path, 0, Bps_to_icc(gpu->fast_rate) * 8);
+
 	return gpu;
 
 fail:
-- 
2.21.0

[PATCH 4/4] drm/msm/a4xx: set interconnect bandwidth vote

[Brian Masney]

Set the two interconnect paths for the GPU to maximum speed for now to
work towards getting the GPU working upstream. We can revisit a later
time to optimize this for battery life.

Signed-off-by: Brian Masney <masneyb@onstation.org>
---
 drivers/gpu/drm/msm/adreno/a4xx_gpu.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
index b01388a9e89e..c631d1df7751 100644
--- a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
@@ -591,6 +591,14 @@ struct msm_gpu *a4xx_gpu_init(struct drm_device *dev)
 		goto fail;
 	}
 
+	/*
+	 * Set the ICC path to maximum speed for now by multiplying the fastest
+	 * frequency by the bus width (8). We'll want to scale this later on to
+	 * improve battery life.
+	 */
+	icc_set_bw(gpu->gfx_mem_icc_path, 0, Bps_to_icc(gpu->fast_rate) * 8);
+	icc_set_bw(gpu->ocmem_icc_path, 0, Bps_to_icc(gpu->fast_rate) * 8);
+
 	return gpu;
 
 fail:
-- 
2.21.0