[PATCH] drm/gem: Handle shadow plane state allocation failure

[PATCH] drm/gem: Handle shadow plane state allocation failure

[Triet Hoang]

Allocate the new shadow plane state before destroying the existing
state, so that an allocation failure leaves the current state intact.

Signed-off-by: Triet Hoang <triet.hoang.dev@gmail.com>
---
 drivers/gpu/drm/drm_gem_atomic_helper.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/drm_gem_atomic_helper.c b/drivers/gpu/drm/drm_gem_atomic_helper.c
index abef865c5..24a0b9398 100644
--- a/drivers/gpu/drm/drm_gem_atomic_helper.c
+++ b/drivers/gpu/drm/drm_gem_atomic_helper.c
@@ -308,14 +308,16 @@ EXPORT_SYMBOL(__drm_gem_reset_shadow_plane);
  */
 void drm_gem_reset_shadow_plane(struct drm_plane *plane)
 {
-	struct drm_shadow_plane_state *shadow_plane_state;
+	struct drm_shadow_plane_state *shadow_plane_state = kzalloc_obj(*shadow_plane_state);
+
+	if (!shadow_plane_state)
+		return;
 
 	if (plane->state) {
 		drm_gem_destroy_shadow_plane_state(plane, plane->state);
 		plane->state = NULL; /* must be set to NULL here */
 	}
 
-	shadow_plane_state = kzalloc_obj(*shadow_plane_state);
 	__drm_gem_reset_shadow_plane(plane, shadow_plane_state);
 }
 EXPORT_SYMBOL(drm_gem_reset_shadow_plane);
-- 
2.53.0

Re: [PATCH] drm/gem: Handle shadow plane state allocation failure

[Thomas Zimmermann]

Hi,

the whole reset mechanism has never worked well IMHO. Maxime currently 
reworks it entirely. Not sure if we meanwhile want to paper over the 
reset issue.

Best regards
Thomas

Am 16.08.26 um 18:58 schrieb Triet Hoang:
> Allocate the new shadow plane state before destroying the existing
> state, so that an allocation failure leaves the current state intact.
>
> Signed-off-by: Triet Hoang <triet.hoang.dev@gmail.com>
> ---
>   drivers/gpu/drm/drm_gem_atomic_helper.c | 6 ++++--
>   1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/drm_gem_atomic_helper.c b/drivers/gpu/drm/drm_gem_atomic_helper.c
> index abef865c5..24a0b9398 100644
> --- a/drivers/gpu/drm/drm_gem_atomic_helper.c
> +++ b/drivers/gpu/drm/drm_gem_atomic_helper.c
> @@ -308,14 +308,16 @@ EXPORT_SYMBOL(__drm_gem_reset_shadow_plane);
>    */
>   void drm_gem_reset_shadow_plane(struct drm_plane *plane)
>   {
> -	struct drm_shadow_plane_state *shadow_plane_state;
> +	struct drm_shadow_plane_state *shadow_plane_state = kzalloc_obj(*shadow_plane_state);
> +
> +	if (!shadow_plane_state)
> +		return;
>   
>   	if (plane->state) {
>   		drm_gem_destroy_shadow_plane_state(plane, plane->state);
>   		plane->state = NULL; /* must be set to NULL here */
>   	}
>   
> -	shadow_plane_state = kzalloc_obj(*shadow_plane_state);
>   	__drm_gem_reset_shadow_plane(plane, shadow_plane_state);
>   }
>   EXPORT_SYMBOL(drm_gem_reset_shadow_plane);

-- 
--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com
GF: Jochen Jaser, Andrew McDonald, (HRB 36809, AG Nürnberg)