[PATCH] drm/msm/adreno: keep the GPU timestamp monotonic across power collapse

[PATCH] drm/msm/adreno: keep the GPU timestamp monotonic across power collapse

[Dmitry Baryshkov]

The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from
zero whenever the GPU is powered up again and the timestamp reported to
userspace jumps backwards. On an a702 six reads three seconds apart all
land in the 500..1200 tick range, stepping backwards twice, and on an
a530 the OpenCL device timer conformance test fails because
clGetDeviceAndHostTimer() returns an end time below the start time.

Save the counter in the suspend path of the affected generations, while
the GPU is still powered, and add the accumulated ticks to the value
reported to userspace.

Assisted-by: LLM
Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
---
The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from zero
whenever the GPU is powered up again and the timestamp userspace reads
jumps backwards.  Accumulate what the counter reached before each suspend
and add it to what is reported afterwards.

Measured on an a530 and an a702; the GMU parts keep their own counter alive
and are left alone.
---
 drivers/gpu/drm/msm/adreno/a4xx_gpu.c   |  2 ++
 drivers/gpu/drm/msm/adreno/a5xx_gpu.c   |  2 ++
 drivers/gpu/drm/msm/adreno/a6xx_gpu.c   |  3 +++
 drivers/gpu/drm/msm/adreno/adreno_gpu.c | 10 +++++++++-
 drivers/gpu/drm/msm/adreno/adreno_gpu.h |  3 +++
 5 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
index 6392126f48f2..7d9dd9460f5b 100644
--- a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
@@ -593,6 +593,8 @@ static int a4xx_pm_suspend(struct msm_gpu *gpu) {
 	struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
 	int ret;
 
+	adreno_save_timestamp(gpu);
+
 	ret = msm_gpu_pm_suspend(gpu);
 	if (ret)
 		return ret;
diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
index f1df2514c613..c5552f1085e0 100644
--- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
@@ -1423,6 +1423,8 @@ static int a5xx_pm_suspend(struct msm_gpu *gpu)
 		gpu_write(gpu, REG_A5XX_RBBM_BLOCK_SW_RESET_CMD, 0x00000000);
 	}
 
+	adreno_save_timestamp(gpu);
+
 	ret = msm_gpu_pm_suspend(gpu);
 	if (ret)
 		return ret;
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
index f9de9329dee3..106cc2ac55ff 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
@@ -2252,6 +2252,9 @@ static int a6xx_pm_suspend(struct msm_gpu *gpu)
 
 	trace_msm_gpu_suspend(0);
 
+	/* only the GMU-less parts come here, and their counter is in the GPU */
+	adreno_save_timestamp(gpu);
+
 	a6xx_llc_deactivate(a6xx_gpu);
 
 	msm_devfreq_suspend(gpu);
diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
index 3370cd44382f..f83960b31901 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
@@ -226,6 +226,13 @@ adreno_iommu_create_vm(struct msm_gpu *gpu,
 	return vm;
 }
 
+void adreno_save_timestamp(struct msm_gpu *gpu)
+{
+	struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
+
+	adreno_gpu->timestamp_offset += adreno_gpu->funcs->get_timestamp(gpu);
+}
+
 u64 adreno_private_vm_size(struct msm_gpu *gpu)
 {
 	struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
@@ -398,7 +405,8 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx,
 	case MSM_PARAM_TIMESTAMP:
 		if (adreno_gpu->funcs->get_timestamp) {
 			pm_runtime_get_sync(&gpu->pdev->dev);
-			*value = adreno_gpu->funcs->get_timestamp(gpu);
+			*value = adreno_gpu->timestamp_offset +
+				 adreno_gpu->funcs->get_timestamp(gpu);
 			pm_runtime_put_autosuspend(&gpu->pdev->dev);
 
 			return 0;
diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h b/drivers/gpu/drm/msm/adreno/adreno_gpu.h
index 114a40f79ef3..db080a6d515c 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h
+++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h
@@ -207,6 +207,8 @@ struct adreno_gpu {
 	uint16_t speedbin;
 	const struct adreno_gpu_funcs *funcs;
 
+	u64 timestamp_offset;
+
 	struct completion fault_coredump_done;
 
 	/* interesting register offsets to dump: */
@@ -610,6 +612,7 @@ static inline int adreno_is_a840(struct adreno_gpu *gpu)
 /* Put vm_start above 32b to catch issues with not setting xyz_BASE_HI */
 #define ADRENO_VM_START 0x100000000ULL
 u64 adreno_private_vm_size(struct msm_gpu *gpu);
+void adreno_save_timestamp(struct msm_gpu *gpu);
 int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx,
 		     uint32_t param, uint64_t *value, uint32_t *len);
 int adreno_set_param(struct msm_gpu *gpu, struct msm_context *ctx,

---
base-commit: 140b13475302601368c0cf4e193e66126a49feb3
change-id: 20260828-b4-adreno-timestamp-f4c846391b97

Best regards,
--  
With best wishes
Dmitry

Re: [PATCH] drm/msm/adreno: keep the GPU timestamp monotonic across power collapse

[Connor Abbott]

On Thu, Aug 27, 2026 at 7:13 PM Dmitry Baryshkov
<dmitry.baryshkov@oss.qualcomm.com> wrote:
>
> The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from
> zero whenever the GPU is powered up again and the timestamp reported to
> userspace jumps backwards. On an a702 six reads three seconds apart all
> land in the 500..1200 tick range, stepping backwards twice, and on an
> a530 the OpenCL device timer conformance test fails because
> clGetDeviceAndHostTimer() returns an end time below the start time.
>
> Save the counter in the suspend path of the affected generations, while
> the GPU is still powered, and add the accumulated ticks to the value
> reported to userspace.

This is useless because the entire point of clGetDeviceAndHostTimer()
(and the similar thing in Vulkan) is to match what the GPU itself
returns, and now you've broken that by adding an offset.

Connor

>
> Assisted-by: LLM
> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
> ---
> The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from zero
> whenever the GPU is powered up again and the timestamp userspace reads
> jumps backwards.  Accumulate what the counter reached before each suspend
> and add it to what is reported afterwards.
>
> Measured on an a530 and an a702; the GMU parts keep their own counter alive
> and are left alone.
> ---
>  drivers/gpu/drm/msm/adreno/a4xx_gpu.c   |  2 ++
>  drivers/gpu/drm/msm/adreno/a5xx_gpu.c   |  2 ++
>  drivers/gpu/drm/msm/adreno/a6xx_gpu.c   |  3 +++
>  drivers/gpu/drm/msm/adreno/adreno_gpu.c | 10 +++++++++-
>  drivers/gpu/drm/msm/adreno/adreno_gpu.h |  3 +++
>  5 files changed, 19 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
> index 6392126f48f2..7d9dd9460f5b 100644
> --- a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
> +++ b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
> @@ -593,6 +593,8 @@ static int a4xx_pm_suspend(struct msm_gpu *gpu) {
>         struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
>         int ret;
>
> +       adreno_save_timestamp(gpu);
> +
>         ret = msm_gpu_pm_suspend(gpu);
>         if (ret)
>                 return ret;
> diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
> index f1df2514c613..c5552f1085e0 100644
> --- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
> +++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
> @@ -1423,6 +1423,8 @@ static int a5xx_pm_suspend(struct msm_gpu *gpu)
>                 gpu_write(gpu, REG_A5XX_RBBM_BLOCK_SW_RESET_CMD, 0x00000000);
>         }
>
> +       adreno_save_timestamp(gpu);
> +
>         ret = msm_gpu_pm_suspend(gpu);
>         if (ret)
>                 return ret;
> diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
> index f9de9329dee3..106cc2ac55ff 100644
> --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
> +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
> @@ -2252,6 +2252,9 @@ static int a6xx_pm_suspend(struct msm_gpu *gpu)
>
>         trace_msm_gpu_suspend(0);
>
> +       /* only the GMU-less parts come here, and their counter is in the GPU */
> +       adreno_save_timestamp(gpu);
> +
>         a6xx_llc_deactivate(a6xx_gpu);
>
>         msm_devfreq_suspend(gpu);
> diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
> index 3370cd44382f..f83960b31901 100644
> --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c
> +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
> @@ -226,6 +226,13 @@ adreno_iommu_create_vm(struct msm_gpu *gpu,
>         return vm;
>  }
>
> +void adreno_save_timestamp(struct msm_gpu *gpu)
> +{
> +       struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
> +
> +       adreno_gpu->timestamp_offset += adreno_gpu->funcs->get_timestamp(gpu);
> +}
> +
>  u64 adreno_private_vm_size(struct msm_gpu *gpu)
>  {
>         struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
> @@ -398,7 +405,8 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx,
>         case MSM_PARAM_TIMESTAMP:
>                 if (adreno_gpu->funcs->get_timestamp) {
>                         pm_runtime_get_sync(&gpu->pdev->dev);
> -                       *value = adreno_gpu->funcs->get_timestamp(gpu);
> +                       *value = adreno_gpu->timestamp_offset +
> +                                adreno_gpu->funcs->get_timestamp(gpu);
>                         pm_runtime_put_autosuspend(&gpu->pdev->dev);
>
>                         return 0;
> diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h b/drivers/gpu/drm/msm/adreno/adreno_gpu.h
> index 114a40f79ef3..db080a6d515c 100644
> --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h
> +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h
> @@ -207,6 +207,8 @@ struct adreno_gpu {
>         uint16_t speedbin;
>         const struct adreno_gpu_funcs *funcs;
>
> +       u64 timestamp_offset;
> +
>         struct completion fault_coredump_done;
>
>         /* interesting register offsets to dump: */
> @@ -610,6 +612,7 @@ static inline int adreno_is_a840(struct adreno_gpu *gpu)
>  /* Put vm_start above 32b to catch issues with not setting xyz_BASE_HI */
>  #define ADRENO_VM_START 0x100000000ULL
>  u64 adreno_private_vm_size(struct msm_gpu *gpu);
> +void adreno_save_timestamp(struct msm_gpu *gpu);
>  int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx,
>                      uint32_t param, uint64_t *value, uint32_t *len);
>  int adreno_set_param(struct msm_gpu *gpu, struct msm_context *ctx,
>
> ---
> base-commit: 140b13475302601368c0cf4e193e66126a49feb3
> change-id: 20260828-b4-adreno-timestamp-f4c846391b97
>
> Best regards,
> --
> With best wishes
> Dmitry
>

Re: [PATCH] drm/msm/adreno: keep the GPU timestamp monotonic across power collapse

[Rob Clark]

On Thu, Aug 27, 2026 at 9:07 PM Connor Abbott <cwabbott0@gmail.com> wrote:
>
> On Thu, Aug 27, 2026 at 7:13 PM Dmitry Baryshkov
> <dmitry.baryshkov@oss.qualcomm.com> wrote:
> >
> > The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> > domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from
> > zero whenever the GPU is powered up again and the timestamp reported to
> > userspace jumps backwards. On an a702 six reads three seconds apart all
> > land in the 500..1200 tick range, stepping backwards twice, and on an
> > a530 the OpenCL device timer conformance test fails because
> > clGetDeviceAndHostTimer() returns an end time below the start time.
> >
> > Save the counter in the suspend path of the affected generations, while
> > the GPU is still powered, and add the accumulated ticks to the value
> > reported to userspace.
>
> This is useless because the entire point of clGetDeviceAndHostTimer()
> (and the similar thing in Vulkan) is to match what the GPU itself
> returns, and now you've broken that by adding an offset.

Just to clarify, the timestamp needs to match what UMD reads directly
from hw in various different ways (timestamp queries, shader_clock,
etc)..

If we can _restore_ the value (ie. write the hw reg) on resume, that
would be ok.. but it doesn't look possible.  Otherwise I guess we need
some scheme where an offset value is stored in memory somewhere, and
some coordination with userspace to add the offset in various
different paths.  But the kernel shouldn't be adding the offset to
what it returns to userspace itself.

BR,
-R

> Connor
>
> >
> > Assisted-by: LLM
> > Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
> > ---
> > The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> > domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from zero
> > whenever the GPU is powered up again and the timestamp userspace reads
> > jumps backwards.  Accumulate what the counter reached before each suspend
> > and add it to what is reported afterwards.
> >
> > Measured on an a530 and an a702; the GMU parts keep their own counter alive
> > and are left alone.
> > ---
> >  drivers/gpu/drm/msm/adreno/a4xx_gpu.c   |  2 ++
> >  drivers/gpu/drm/msm/adreno/a5xx_gpu.c   |  2 ++
> >  drivers/gpu/drm/msm/adreno/a6xx_gpu.c   |  3 +++
> >  drivers/gpu/drm/msm/adreno/adreno_gpu.c | 10 +++++++++-
> >  drivers/gpu/drm/msm/adreno/adreno_gpu.h |  3 +++
> >  5 files changed, 19 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
> > index 6392126f48f2..7d9dd9460f5b 100644
> > --- a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
> > +++ b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c
> > @@ -593,6 +593,8 @@ static int a4xx_pm_suspend(struct msm_gpu *gpu) {
> >         struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
> >         int ret;
> >
> > +       adreno_save_timestamp(gpu);
> > +
> >         ret = msm_gpu_pm_suspend(gpu);
> >         if (ret)
> >                 return ret;
> > diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
> > index f1df2514c613..c5552f1085e0 100644
> > --- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
> > +++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
> > @@ -1423,6 +1423,8 @@ static int a5xx_pm_suspend(struct msm_gpu *gpu)
> >                 gpu_write(gpu, REG_A5XX_RBBM_BLOCK_SW_RESET_CMD, 0x00000000);
> >         }
> >
> > +       adreno_save_timestamp(gpu);
> > +
> >         ret = msm_gpu_pm_suspend(gpu);
> >         if (ret)
> >                 return ret;
> > diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
> > index f9de9329dee3..106cc2ac55ff 100644
> > --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
> > +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
> > @@ -2252,6 +2252,9 @@ static int a6xx_pm_suspend(struct msm_gpu *gpu)
> >
> >         trace_msm_gpu_suspend(0);
> >
> > +       /* only the GMU-less parts come here, and their counter is in the GPU */
> > +       adreno_save_timestamp(gpu);
> > +
> >         a6xx_llc_deactivate(a6xx_gpu);
> >
> >         msm_devfreq_suspend(gpu);
> > diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
> > index 3370cd44382f..f83960b31901 100644
> > --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c
> > +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
> > @@ -226,6 +226,13 @@ adreno_iommu_create_vm(struct msm_gpu *gpu,
> >         return vm;
> >  }
> >
> > +void adreno_save_timestamp(struct msm_gpu *gpu)
> > +{
> > +       struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
> > +
> > +       adreno_gpu->timestamp_offset += adreno_gpu->funcs->get_timestamp(gpu);
> > +}
> > +
> >  u64 adreno_private_vm_size(struct msm_gpu *gpu)
> >  {
> >         struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
> > @@ -398,7 +405,8 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx,
> >         case MSM_PARAM_TIMESTAMP:
> >                 if (adreno_gpu->funcs->get_timestamp) {
> >                         pm_runtime_get_sync(&gpu->pdev->dev);
> > -                       *value = adreno_gpu->funcs->get_timestamp(gpu);
> > +                       *value = adreno_gpu->timestamp_offset +
> > +                                adreno_gpu->funcs->get_timestamp(gpu);
> >                         pm_runtime_put_autosuspend(&gpu->pdev->dev);
> >
> >                         return 0;
> > diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h b/drivers/gpu/drm/msm/adreno/adreno_gpu.h
> > index 114a40f79ef3..db080a6d515c 100644
> > --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h
> > +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h
> > @@ -207,6 +207,8 @@ struct adreno_gpu {
> >         uint16_t speedbin;
> >         const struct adreno_gpu_funcs *funcs;
> >
> > +       u64 timestamp_offset;
> > +
> >         struct completion fault_coredump_done;
> >
> >         /* interesting register offsets to dump: */
> > @@ -610,6 +612,7 @@ static inline int adreno_is_a840(struct adreno_gpu *gpu)
> >  /* Put vm_start above 32b to catch issues with not setting xyz_BASE_HI */
> >  #define ADRENO_VM_START 0x100000000ULL
> >  u64 adreno_private_vm_size(struct msm_gpu *gpu);
> > +void adreno_save_timestamp(struct msm_gpu *gpu);
> >  int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx,
> >                      uint32_t param, uint64_t *value, uint32_t *len);
> >  int adreno_set_param(struct msm_gpu *gpu, struct msm_context *ctx,
> >
> > ---
> > base-commit: 140b13475302601368c0cf4e193e66126a49feb3
> > change-id: 20260828-b4-adreno-timestamp-f4c846391b97
> >
> > Best regards,
> > --
> > With best wishes
> > Dmitry
> >

Re: [PATCH] drm/msm/adreno: keep the GPU timestamp monotonic across power collapse

[Konrad Dybcio]

On 8/28/26 1:13 AM, Dmitry Baryshkov wrote:
> The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from
> zero whenever the GPU is powered up again and the timestamp reported to
> userspace jumps backwards. On an a702 six reads three seconds apart all
> land in the 500..1200 tick range, stepping backwards twice, and on an
> a530 the OpenCL device timer conformance test fails because
> clGetDeviceAndHostTimer() returns an end time below the start time.
> 
> Save the counter in the suspend path of the affected generations, while
> the GPU is still powered, and add the accumulated ticks to the value
> reported to userspace.
> 
> Assisted-by: LLM
> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
> ---
> The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from zero
> whenever the GPU is powered up again and the timestamp userspace reads
> jumps backwards.  Accumulate what the counter reached before each suspend
> and add it to what is reported afterwards.

Could (some) a5xx parts use A5XX_GPMU_ALWAYS_ON_COUNTER/0xa879?
msm-3.x defines it as KGSL_PERFCOUNTER_NOT_USED fwiw..

Konrad

Re: [PATCH] drm/msm/adreno: keep the GPU timestamp monotonic across power collapse

[Dmitry Baryshkov]

On Mon, Sep 07, 2026 at 04:20:43PM -0700, Rob Clark wrote:
> On Thu, Aug 27, 2026 at 9:07 PM Connor Abbott <cwabbott0@gmail.com> wrote:
> >
> > On Thu, Aug 27, 2026 at 7:13 PM Dmitry Baryshkov
> > <dmitry.baryshkov@oss.qualcomm.com> wrote:
> > >
> > > The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> > > domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from
> > > zero whenever the GPU is powered up again and the timestamp reported to
> > > userspace jumps backwards. On an a702 six reads three seconds apart all
> > > land in the 500..1200 tick range, stepping backwards twice, and on an
> > > a530 the OpenCL device timer conformance test fails because
> > > clGetDeviceAndHostTimer() returns an end time below the start time.
> > >
> > > Save the counter in the suspend path of the affected generations, while
> > > the GPU is still powered, and add the accumulated ticks to the value
> > > reported to userspace.
> >
> > This is useless because the entire point of clGetDeviceAndHostTimer()
> > (and the similar thing in Vulkan) is to match what the GPU itself
> > returns, and now you've broken that by adding an offset.
> 
> Just to clarify, the timestamp needs to match what UMD reads directly
> from hw in various different ways (timestamp queries, shader_clock,
> etc)..
> 
> If we can _restore_ the value (ie. write the hw reg) on resume, that
> would be ok.. but it doesn't look possible.

Yes, a quick check shows that all relevant registers are write-ignore.

> Otherwise I guess we need
> some scheme where an offset value is stored in memory somewhere, and
> some coordination with userspace to add the offset in various
> different paths.  But the kernel shouldn't be adding the offset to
> what it returns to userspace itself.

Ack, thanks for the explanation. I tried to match what kgsl was doing.
It reads the GPU timers directly from the GPU, but for the CPU timers it
reads the value from KGSL driver which accumulates the offset.

If I understand you correctly, we need to export the offset to the
userspace via some extra page (like vdso) and let Mesa read it. Correct?

-- 
With best wishes
Dmitry

Re: [PATCH] drm/msm/adreno: keep the GPU timestamp monotonic across power collapse

[Dmitry Baryshkov]

On Tue, Sep 08, 2026 at 09:03:55AM +0200, Konrad Dybcio wrote:
> On 8/28/26 1:13 AM, Dmitry Baryshkov wrote:
> > The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> > domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from
> > zero whenever the GPU is powered up again and the timestamp reported to
> > userspace jumps backwards. On an a702 six reads three seconds apart all
> > land in the 500..1200 tick range, stepping backwards twice, and on an
> > a530 the OpenCL device timer conformance test fails because
> > clGetDeviceAndHostTimer() returns an end time below the start time.
> > 
> > Save the counter in the suspend path of the affected generations, while
> > the GPU is still powered, and add the accumulated ticks to the value
> > reported to userspace.
> > 
> > Assisted-by: LLM
> > Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
> > ---
> > The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> > domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from zero
> > whenever the GPU is powered up again and the timestamp userspace reads
> > jumps backwards.  Accumulate what the counter reached before each suspend
> > and add it to what is reported afterwards.
> 
> Could (some) a5xx parts use A5XX_GPMU_ALWAYS_ON_COUNTER/0xa879?
> msm-3.x defines it as KGSL_PERFCOUNTER_NOT_USED fwiw..

It seems it also restarts on power collapse (so always on is not
actually always). Also if it worked, it would have helped only a530 and
a540 (MSM8996 and MSM8998), leaving SDM630/660 and all MSM8956/76/53/37
without the fix.

-- 
With best wishes
Dmitry

Re: [PATCH] drm/msm/adreno: keep the GPU timestamp monotonic across power collapse

[Konrad Dybcio]

On 9/8/26 1:43 PM, Dmitry Baryshkov wrote:
> On Tue, Sep 08, 2026 at 09:03:55AM +0200, Konrad Dybcio wrote:
>> On 8/28/26 1:13 AM, Dmitry Baryshkov wrote:
>>> The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
>>> domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from
>>> zero whenever the GPU is powered up again and the timestamp reported to
>>> userspace jumps backwards. On an a702 six reads three seconds apart all
>>> land in the 500..1200 tick range, stepping backwards twice, and on an
>>> a530 the OpenCL device timer conformance test fails because
>>> clGetDeviceAndHostTimer() returns an end time below the start time.
>>>
>>> Save the counter in the suspend path of the affected generations, while
>>> the GPU is still powered, and add the accumulated ticks to the value
>>> reported to userspace.
>>>
>>> Assisted-by: LLM
>>> Signed-off-by: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com>
>>> ---
>>> The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
>>> domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from zero
>>> whenever the GPU is powered up again and the timestamp userspace reads
>>> jumps backwards.  Accumulate what the counter reached before each suspend
>>> and add it to what is reported afterwards.
>>
>> Could (some) a5xx parts use A5XX_GPMU_ALWAYS_ON_COUNTER/0xa879?
>> msm-3.x defines it as KGSL_PERFCOUNTER_NOT_USED fwiw..
> 
> It seems it also restarts on power collapse (so always on is not
> actually always). Also if it worked, it would have helped only a530 and
> a540 (MSM8996 and MSM8998), leaving SDM630/660 and all MSM8956/76/53/37
> without the fix.

Yeah, that's why I said "some". Is there a chance that's because we
mistreat the cx/gx split on a5xx?

Konrad

Re: [PATCH] drm/msm/adreno: keep the GPU timestamp monotonic across power collapse

[Rob Clark]

On Tue, Sep 8, 2026 at 4:41 AM Dmitry Baryshkov
<dmitry.baryshkov@oss.qualcomm.com> wrote:
>
> On Mon, Sep 07, 2026 at 04:20:43PM -0700, Rob Clark wrote:
> > On Thu, Aug 27, 2026 at 9:07 PM Connor Abbott <cwabbott0@gmail.com> wrote:
> > >
> > > On Thu, Aug 27, 2026 at 7:13 PM Dmitry Baryshkov
> > > <dmitry.baryshkov@oss.qualcomm.com> wrote:
> > > >
> > > > The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> > > > domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from
> > > > zero whenever the GPU is powered up again and the timestamp reported to
> > > > userspace jumps backwards. On an a702 six reads three seconds apart all
> > > > land in the 500..1200 tick range, stepping backwards twice, and on an
> > > > a530 the OpenCL device timer conformance test fails because
> > > > clGetDeviceAndHostTimer() returns an end time below the start time.
> > > >
> > > > Save the counter in the suspend path of the affected generations, while
> > > > the GPU is still powered, and add the accumulated ticks to the value
> > > > reported to userspace.
> > >
> > > This is useless because the entire point of clGetDeviceAndHostTimer()
> > > (and the similar thing in Vulkan) is to match what the GPU itself
> > > returns, and now you've broken that by adding an offset.
> >
> > Just to clarify, the timestamp needs to match what UMD reads directly
> > from hw in various different ways (timestamp queries, shader_clock,
> > etc)..
> >
> > If we can _restore_ the value (ie. write the hw reg) on resume, that
> > would be ok.. but it doesn't look possible.
>
> Yes, a quick check shows that all relevant registers are write-ignore.
>
> > Otherwise I guess we need
> > some scheme where an offset value is stored in memory somewhere, and
> > some coordination with userspace to add the offset in various
> > different paths.  But the kernel shouldn't be adding the offset to
> > what it returns to userspace itself.
>
> Ack, thanks for the explanation. I tried to match what kgsl was doing.
> It reads the GPU timers directly from the GPU, but for the CPU timers it
> reads the value from KGSL driver which accumulates the offset.
>
> If I understand you correctly, we need to export the offset to the
> userspace via some extra page (like vdso) and let Mesa read it. Correct?

Yeah, it doesn't look like there is an existing GPU_READONLY && !APRIV
BO we could re-use, so I think we need to create a new global (TTBR1)
"vdso" BO which is read-only to the GPU but otherwise accessible to
IB1+, along w/ MSM_PARAM_VDSO for UMD to query the address of this BO.
(And update msm_mapping test in igt to validate that it is read-only.)

(I'm not married to the name "vdso" but not creative enough this
morning to think of something better)

I'm a bit curious if kgsl handles this differently.. maybe there is
something I'm missing.  But somehow timestamps need to agree between
what is read on the GPU and what is read on CPU.

BR,
-R


> --
> With best wishes
> Dmitry

Re: [PATCH] drm/msm/adreno: keep the GPU timestamp monotonic across power collapse

[Connor Abbott]

On Tue, Sep 8, 2026 at 10:51 AM Rob Clark <rob.clark@oss.qualcomm.com> wrote:
>
> On Tue, Sep 8, 2026 at 4:41 AM Dmitry Baryshkov
> <dmitry.baryshkov@oss.qualcomm.com> wrote:
> >
> > On Mon, Sep 07, 2026 at 04:20:43PM -0700, Rob Clark wrote:
> > > On Thu, Aug 27, 2026 at 9:07 PM Connor Abbott <cwabbott0@gmail.com> wrote:
> > > >
> > > > On Thu, Aug 27, 2026 at 7:13 PM Dmitry Baryshkov
> > > > <dmitry.baryshkov@oss.qualcomm.com> wrote:
> > > > >
> > > > > The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> > > > > domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from
> > > > > zero whenever the GPU is powered up again and the timestamp reported to
> > > > > userspace jumps backwards. On an a702 six reads three seconds apart all
> > > > > land in the 500..1200 tick range, stepping backwards twice, and on an
> > > > > a530 the OpenCL device timer conformance test fails because
> > > > > clGetDeviceAndHostTimer() returns an end time below the start time.
> > > > >
> > > > > Save the counter in the suspend path of the affected generations, while
> > > > > the GPU is still powered, and add the accumulated ticks to the value
> > > > > reported to userspace.
> > > >
> > > > This is useless because the entire point of clGetDeviceAndHostTimer()
> > > > (and the similar thing in Vulkan) is to match what the GPU itself
> > > > returns, and now you've broken that by adding an offset.
> > >
> > > Just to clarify, the timestamp needs to match what UMD reads directly
> > > from hw in various different ways (timestamp queries, shader_clock,
> > > etc)..
> > >
> > > If we can _restore_ the value (ie. write the hw reg) on resume, that
> > > would be ok.. but it doesn't look possible.
> >
> > Yes, a quick check shows that all relevant registers are write-ignore.
> >
> > > Otherwise I guess we need
> > > some scheme where an offset value is stored in memory somewhere, and
> > > some coordination with userspace to add the offset in various
> > > different paths.  But the kernel shouldn't be adding the offset to
> > > what it returns to userspace itself.
> >
> > Ack, thanks for the explanation. I tried to match what kgsl was doing.
> > It reads the GPU timers directly from the GPU, but for the CPU timers it
> > reads the value from KGSL driver which accumulates the offset.
> >
> > If I understand you correctly, we need to export the offset to the
> > userspace via some extra page (like vdso) and let Mesa read it. Correct?
>
> Yeah, it doesn't look like there is an existing GPU_READONLY && !APRIV
> BO we could re-use, so I think we need to create a new global (TTBR1)
> "vdso" BO which is read-only to the GPU but otherwise accessible to
> IB1+, along w/ MSM_PARAM_VDSO for UMD to query the address of this BO.
> (And update msm_mapping test in igt to validate that it is read-only.)
>
> (I'm not married to the name "vdso" but not creative enough this
> morning to think of something better)
>
> I'm a bit curious if kgsl handles this differently.. maybe there is
> something I'm missing.  But somehow timestamps need to agree between
> what is read on the GPU and what is read on CPU.
>
> BR,
> -R

I think the kgsl timestamp offset stuff was only added for the new
A750+ CX GMU counter which is not read-only. We do need to port that
over to drm/msm. If the offset is applied for older GPUs, it's
probably another case of kgsl not bothering to keep support for older
GPUs in newer branches.

Connor

>
>
> > --
> > With best wishes
> > Dmitry

Re: [PATCH] drm/msm/adreno: keep the GPU timestamp monotonic across power collapse

[Dmitry Baryshkov]

On Tue, Sep 08, 2026 at 11:09:17AM -0400, Connor Abbott wrote:
> On Tue, Sep 8, 2026 at 10:51 AM Rob Clark <rob.clark@oss.qualcomm.com> wrote:
> >
> > On Tue, Sep 8, 2026 at 4:41 AM Dmitry Baryshkov
> > <dmitry.baryshkov@oss.qualcomm.com> wrote:
> > >
> > > On Mon, Sep 07, 2026 at 04:20:43PM -0700, Rob Clark wrote:
> > > > On Thu, Aug 27, 2026 at 9:07 PM Connor Abbott <cwabbott0@gmail.com> wrote:
> > > > >
> > > > > On Thu, Aug 27, 2026 at 7:13 PM Dmitry Baryshkov
> > > > > <dmitry.baryshkov@oss.qualcomm.com> wrote:
> > > > > >
> > > > > > The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> > > > > > domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from
> > > > > > zero whenever the GPU is powered up again and the timestamp reported to
> > > > > > userspace jumps backwards. On an a702 six reads three seconds apart all
> > > > > > land in the 500..1200 tick range, stepping backwards twice, and on an
> > > > > > a530 the OpenCL device timer conformance test fails because
> > > > > > clGetDeviceAndHostTimer() returns an end time below the start time.
> > > > > >
> > > > > > Save the counter in the suspend path of the affected generations, while
> > > > > > the GPU is still powered, and add the accumulated ticks to the value
> > > > > > reported to userspace.
> > > > >
> > > > > This is useless because the entire point of clGetDeviceAndHostTimer()
> > > > > (and the similar thing in Vulkan) is to match what the GPU itself
> > > > > returns, and now you've broken that by adding an offset.
> > > >
> > > > Just to clarify, the timestamp needs to match what UMD reads directly
> > > > from hw in various different ways (timestamp queries, shader_clock,
> > > > etc)..
> > > >
> > > > If we can _restore_ the value (ie. write the hw reg) on resume, that
> > > > would be ok.. but it doesn't look possible.
> > >
> > > Yes, a quick check shows that all relevant registers are write-ignore.
> > >
> > > > Otherwise I guess we need
> > > > some scheme where an offset value is stored in memory somewhere, and
> > > > some coordination with userspace to add the offset in various
> > > > different paths.  But the kernel shouldn't be adding the offset to
> > > > what it returns to userspace itself.
> > >
> > > Ack, thanks for the explanation. I tried to match what kgsl was doing.
> > > It reads the GPU timers directly from the GPU, but for the CPU timers it
> > > reads the value from KGSL driver which accumulates the offset.
> > >
> > > If I understand you correctly, we need to export the offset to the
> > > userspace via some extra page (like vdso) and let Mesa read it. Correct?
> >
> > Yeah, it doesn't look like there is an existing GPU_READONLY && !APRIV
> > BO we could re-use, so I think we need to create a new global (TTBR1)
> > "vdso" BO which is read-only to the GPU but otherwise accessible to
> > IB1+, along w/ MSM_PARAM_VDSO for UMD to query the address of this BO.
> > (And update msm_mapping test in igt to validate that it is read-only.)
> >
> > (I'm not married to the name "vdso" but not creative enough this
> > morning to think of something better)
> >
> > I'm a bit curious if kgsl handles this differently.. maybe there is
> > something I'm missing.  But somehow timestamps need to agree between
> > what is read on the GPU and what is read on CPU.
> >
> > BR,
> > -R
> 
> I think the kgsl timestamp offset stuff was only added for the new
> A750+ CX GMU counter which is not read-only. We do need to port that
> over to drm/msm. If the offset is applied for older GPUs, it's
> probably another case of kgsl not bothering to keep support for older
> GPUs in newer branches.

I've been looking at the kgsl drivers from msm-3.18 / 4.4 as those
kernels targeted MSM8996/98.

-- 
With best wishes
Dmitry

Re: [PATCH] drm/msm/adreno: keep the GPU timestamp monotonic across power collapse

[Dmitry Baryshkov]

On Tue, Sep 08, 2026 at 06:26:22PM +0300, Dmitry Baryshkov wrote:
> On Tue, Sep 08, 2026 at 11:09:17AM -0400, Connor Abbott wrote:
> > On Tue, Sep 8, 2026 at 10:51 AM Rob Clark <rob.clark@oss.qualcomm.com> wrote:
> > >
> > > On Tue, Sep 8, 2026 at 4:41 AM Dmitry Baryshkov
> > > <dmitry.baryshkov@oss.qualcomm.com> wrote:
> > > >
> > > > On Mon, Sep 07, 2026 at 04:20:43PM -0700, Rob Clark wrote:
> > > > > On Thu, Aug 27, 2026 at 9:07 PM Connor Abbott <cwabbott0@gmail.com> wrote:
> > > > > >
> > > > > > On Thu, Aug 27, 2026 at 7:13 PM Dmitry Baryshkov
> > > > > > <dmitry.baryshkov@oss.qualcomm.com> wrote:
> > > > > > >
> > > > > > > The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> > > > > > > domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from
> > > > > > > zero whenever the GPU is powered up again and the timestamp reported to
> > > > > > > userspace jumps backwards. On an a702 six reads three seconds apart all
> > > > > > > land in the 500..1200 tick range, stepping backwards twice, and on an
> > > > > > > a530 the OpenCL device timer conformance test fails because
> > > > > > > clGetDeviceAndHostTimer() returns an end time below the start time.
> > > > > > >
> > > > > > > Save the counter in the suspend path of the affected generations, while
> > > > > > > the GPU is still powered, and add the accumulated ticks to the value
> > > > > > > reported to userspace.
> > > > > >
> > > > > > This is useless because the entire point of clGetDeviceAndHostTimer()
> > > > > > (and the similar thing in Vulkan) is to match what the GPU itself
> > > > > > returns, and now you've broken that by adding an offset.
> > > > >
> > > > > Just to clarify, the timestamp needs to match what UMD reads directly
> > > > > from hw in various different ways (timestamp queries, shader_clock,
> > > > > etc)..
> > > > >
> > > > > If we can _restore_ the value (ie. write the hw reg) on resume, that
> > > > > would be ok.. but it doesn't look possible.
> > > >
> > > > Yes, a quick check shows that all relevant registers are write-ignore.
> > > >
> > > > > Otherwise I guess we need
> > > > > some scheme where an offset value is stored in memory somewhere, and
> > > > > some coordination with userspace to add the offset in various
> > > > > different paths.  But the kernel shouldn't be adding the offset to
> > > > > what it returns to userspace itself.
> > > >
> > > > Ack, thanks for the explanation. I tried to match what kgsl was doing.
> > > > It reads the GPU timers directly from the GPU, but for the CPU timers it
> > > > reads the value from KGSL driver which accumulates the offset.
> > > >
> > > > If I understand you correctly, we need to export the offset to the
> > > > userspace via some extra page (like vdso) and let Mesa read it. Correct?
> > >
> > > Yeah, it doesn't look like there is an existing GPU_READONLY && !APRIV
> > > BO we could re-use, so I think we need to create a new global (TTBR1)
> > > "vdso" BO which is read-only to the GPU but otherwise accessible to
> > > IB1+, along w/ MSM_PARAM_VDSO for UMD to query the address of this BO.
> > > (And update msm_mapping test in igt to validate that it is read-only.)
> > >
> > > (I'm not married to the name "vdso" but not creative enough this
> > > morning to think of something better)
> > >
> > > I'm a bit curious if kgsl handles this differently.. maybe there is
> > > something I'm missing.  But somehow timestamps need to agree between
> > > what is read on the GPU and what is read on CPU.
> > >
> > > BR,
> > > -R
> > 
> > I think the kgsl timestamp offset stuff was only added for the new
> > A750+ CX GMU counter which is not read-only. We do need to port that
> > over to drm/msm. If the offset is applied for older GPUs, it's
> > probably another case of kgsl not bothering to keep support for older
> > GPUs in newer branches.
> 
> I've been looking at the kgsl drivers from msm-3.18 / 4.4 as those
> kernels targeted MSM8996/98.

If I understand correctly, for those GPUs glGetInteger64v(GL_TIMESTAMP)
reads the ALWAYSON perf counter through IOCTL_KGSL_PERFCOUNTER_READ. The
perfcounters are all saved on suspend and summed towards the current
value read from HW.

-- 
With best wishes
Dmitry

Re: [PATCH] drm/msm/adreno: keep the GPU timestamp monotonic across power collapse

[Connor Abbott]

On Tue, Sep 8, 2026 at 11:49 AM Dmitry Baryshkov
<dmitry.baryshkov@oss.qualcomm.com> wrote:
>
> On Tue, Sep 08, 2026 at 06:26:22PM +0300, Dmitry Baryshkov wrote:
> > On Tue, Sep 08, 2026 at 11:09:17AM -0400, Connor Abbott wrote:
> > > On Tue, Sep 8, 2026 at 10:51 AM Rob Clark <rob.clark@oss.qualcomm.com> wrote:
> > > >
> > > > On Tue, Sep 8, 2026 at 4:41 AM Dmitry Baryshkov
> > > > <dmitry.baryshkov@oss.qualcomm.com> wrote:
> > > > >
> > > > > On Mon, Sep 07, 2026 at 04:20:43PM -0700, Rob Clark wrote:
> > > > > > On Thu, Aug 27, 2026 at 9:07 PM Connor Abbott <cwabbott0@gmail.com> wrote:
> > > > > > >
> > > > > > > On Thu, Aug 27, 2026 at 7:13 PM Dmitry Baryshkov
> > > > > > > <dmitry.baryshkov@oss.qualcomm.com> wrote:
> > > > > > > >
> > > > > > > > The always-on counter behind MSM_PARAM_TIMESTAMP sits in the GPU power
> > > > > > > > domain on a4xx, a5xx and the GMU-less a6xx parts, so it restarts from
> > > > > > > > zero whenever the GPU is powered up again and the timestamp reported to
> > > > > > > > userspace jumps backwards. On an a702 six reads three seconds apart all
> > > > > > > > land in the 500..1200 tick range, stepping backwards twice, and on an
> > > > > > > > a530 the OpenCL device timer conformance test fails because
> > > > > > > > clGetDeviceAndHostTimer() returns an end time below the start time.
> > > > > > > >
> > > > > > > > Save the counter in the suspend path of the affected generations, while
> > > > > > > > the GPU is still powered, and add the accumulated ticks to the value
> > > > > > > > reported to userspace.
> > > > > > >
> > > > > > > This is useless because the entire point of clGetDeviceAndHostTimer()
> > > > > > > (and the similar thing in Vulkan) is to match what the GPU itself
> > > > > > > returns, and now you've broken that by adding an offset.
> > > > > >
> > > > > > Just to clarify, the timestamp needs to match what UMD reads directly
> > > > > > from hw in various different ways (timestamp queries, shader_clock,
> > > > > > etc)..
> > > > > >
> > > > > > If we can _restore_ the value (ie. write the hw reg) on resume, that
> > > > > > would be ok.. but it doesn't look possible.
> > > > >
> > > > > Yes, a quick check shows that all relevant registers are write-ignore.
> > > > >
> > > > > > Otherwise I guess we need
> > > > > > some scheme where an offset value is stored in memory somewhere, and
> > > > > > some coordination with userspace to add the offset in various
> > > > > > different paths.  But the kernel shouldn't be adding the offset to
> > > > > > what it returns to userspace itself.
> > > > >
> > > > > Ack, thanks for the explanation. I tried to match what kgsl was doing.
> > > > > It reads the GPU timers directly from the GPU, but for the CPU timers it
> > > > > reads the value from KGSL driver which accumulates the offset.
> > > > >
> > > > > If I understand you correctly, we need to export the offset to the
> > > > > userspace via some extra page (like vdso) and let Mesa read it. Correct?
> > > >
> > > > Yeah, it doesn't look like there is an existing GPU_READONLY && !APRIV
> > > > BO we could re-use, so I think we need to create a new global (TTBR1)
> > > > "vdso" BO which is read-only to the GPU but otherwise accessible to
> > > > IB1+, along w/ MSM_PARAM_VDSO for UMD to query the address of this BO.
> > > > (And update msm_mapping test in igt to validate that it is read-only.)
> > > >
> > > > (I'm not married to the name "vdso" but not creative enough this
> > > > morning to think of something better)
> > > >
> > > > I'm a bit curious if kgsl handles this differently.. maybe there is
> > > > something I'm missing.  But somehow timestamps need to agree between
> > > > what is read on the GPU and what is read on CPU.
> > > >
> > > > BR,
> > > > -R
> > >
> > > I think the kgsl timestamp offset stuff was only added for the new
> > > A750+ CX GMU counter which is not read-only. We do need to port that
> > > over to drm/msm. If the offset is applied for older GPUs, it's
> > > probably another case of kgsl not bothering to keep support for older
> > > GPUs in newer branches.
> >
> > I've been looking at the kgsl drivers from msm-3.18 / 4.4 as those
> > kernels targeted MSM8996/98.
>
> If I understand correctly, for those GPUs glGetInteger64v(GL_TIMESTAMP)
> reads the ALWAYSON perf counter through IOCTL_KGSL_PERFCOUNTER_READ. The
> perfcounters are all saved on suspend and summed towards the current
> value read from HW.
>
> --
> With best wishes
> Dmitry

I think GL_TIMESTAMP just wasn't ever exposed. The equivalent GLES
extension GL_EXT_disjoint_timer_query was written by Qualcomm, and it
explicitly allows zeroing on suspend/resume. IIUC it also doesn't use
the perfcounter, it probably submits a dummy job and reads
kgsl_cmdbatch_profiling_buffer::gpu_ticks_queued.

Connor