[REGRESSION] drm/vmwgfx shows green Xv instead of actual video (bisected I think)

[REGRESSION] drm/vmwgfx shows green Xv instead of actual video (bisected I think)

[Doug Brown]

Hello,

I have discovered a problem in vmwgfx that caused Xv playback to break
between v6.3 and v6.4-rc1. In v6.3, inside of an Ubuntu 24.04 VMware VM
with 3D acceleration disabled, I can run the following GStreamer command
("sudo apt install gstreamer1.0-tools" if you don't already have it):

gst-launch-1.0 videotestsrc ! 
video/x-raw,format=YV12,width=640,height=480 ! xvimagesink

And it works fine, showing a test pattern.

In v6.4-rc1 and all the way up to a build I just made today of Linus's
latest master branch and also including Zack's latest patchset [1]:

Linux doug-Ubuntu-MATE-2404 6.10.0-rc7+ #55 SMP PREEMPT_DYNAMIC Thu Jul
11 12:46:06 PDT 2024 x86_64 x86_64 x86_64 GNU/Linux

...if I run the same command, the GStreamer window shows up containing
solid green rather than the test pattern.

I believe I have bisected the problem to commit:

39985eea5a6d ("drm/vmwgfx: Abstract placement selection")

However, the bisect process was complicated because two earlier commits
temporarily broke vmwgfx so I had to undo them during bisection, which
also required some conflict resolution:

180253782038 ("drm/ttm: stop allocating dummy resources during BO 
creation") and
f87c1f0b7b79 ("drm/ttm: prevent moving of pinned BOs")

Note that if you have gstreamer1.0-vaapi installed, you should
temporarily remove it first before testing because there is a separate
Xorg bug that causes Xorg to crash when you run the above command if
gstreamer1.0-vaapi is installed.

I'm happy to do any further testing as needed. I've reproduced this
problem across several different machines and distros. A fully-updated
Arch Linux is also affected.

As a side note, that same gst-launch-1.0 command is also currently
broken with 3D acceleration enabled, but that's a separate issue in mesa
that I've submitted a patch for [2].

Thanks,
Doug

[1] 
https://lore.kernel.org/all/20240702021254.1610188-1-zack.rusin@broadcom.com/#t
[2] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30116

#regzbot introduced: v6.3..v6.4-rc1

Re: [REGRESSION] drm/vmwgfx shows green Xv instead of actual video (bisected I think)

[Ian Forbes]

Hi Doug,

Can you try this patch with 6.10.

diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c
b/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c
index c45b4724e414..e20f64b67b26 100644
--- a/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c
+++ b/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c
@@ -92,7 +92,7 @@ static int vmw_overlay_send_put(struct vmw_private *dev_priv,
 {
        struct vmw_escape_video_flush *flush;
        size_t fifo_size;
-       bool have_so = (dev_priv->active_display_unit == vmw_du_screen_object);
+       bool have_so = (dev_priv->active_display_unit != vmw_du_legacy);
        int i, num_items;
        SVGAGuestPtr ptr;


On Thu, Jul 11, 2024 at 3:37 PM Doug Brown <doug@schmorgal.com> wrote:
>
> Hello,
>
> I have discovered a problem in vmwgfx that caused Xv playback to break
> between v6.3 and v6.4-rc1. In v6.3, inside of an Ubuntu 24.04 VMware VM
> with 3D acceleration disabled, I can run the following GStreamer command
> ("sudo apt install gstreamer1.0-tools" if you don't already have it):
>
> gst-launch-1.0 videotestsrc !
> video/x-raw,format=YV12,width=640,height=480 ! xvimagesink
>
> And it works fine, showing a test pattern.
>
> In v6.4-rc1 and all the way up to a build I just made today of Linus's
> latest master branch and also including Zack's latest patchset [1]:
>
> Linux doug-Ubuntu-MATE-2404 6.10.0-rc7+ #55 SMP PREEMPT_DYNAMIC Thu Jul
> 11 12:46:06 PDT 2024 x86_64 x86_64 x86_64 GNU/Linux
>
> ...if I run the same command, the GStreamer window shows up containing
> solid green rather than the test pattern.
>
> I believe I have bisected the problem to commit:
>
> 39985eea5a6d ("drm/vmwgfx: Abstract placement selection")
>
> However, the bisect process was complicated because two earlier commits
> temporarily broke vmwgfx so I had to undo them during bisection, which
> also required some conflict resolution:
>
> 180253782038 ("drm/ttm: stop allocating dummy resources during BO
> creation") and
> f87c1f0b7b79 ("drm/ttm: prevent moving of pinned BOs")
>
> Note that if you have gstreamer1.0-vaapi installed, you should
> temporarily remove it first before testing because there is a separate
> Xorg bug that causes Xorg to crash when you run the above command if
> gstreamer1.0-vaapi is installed.
>
> I'm happy to do any further testing as needed. I've reproduced this
> problem across several different machines and distros. A fully-updated
> Arch Linux is also affected.
>
> As a side note, that same gst-launch-1.0 command is also currently
> broken with 3D acceleration enabled, but that's a separate issue in mesa
> that I've submitted a patch for [2].
>
> Thanks,
> Doug
>
> [1]
> https://lore.kernel.org/all/20240702021254.1610188-1-zack.rusin@broadcom.com/#t
> [2] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30116
>
> #regzbot introduced: v6.3..v6.4-rc1

Re: [REGRESSION] drm/vmwgfx shows green Xv instead of actual video (bisected I think)

[Doug Brown]

Hi Ian,

On 7/15/2024 12:32 PM, Ian Forbes wrote:
> Hi Doug,
> 
> Can you try this patch with 6.10.
> 
> diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c
> b/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c
> index c45b4724e414..e20f64b67b26 100644
> --- a/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c
> +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c
> @@ -92,7 +92,7 @@ static int vmw_overlay_send_put(struct vmw_private *dev_priv,
>  {
>         struct vmw_escape_video_flush *flush;
>         size_t fifo_size;
> -       bool have_so = (dev_priv->active_display_unit == vmw_du_screen_object);
> +       bool have_so = (dev_priv->active_display_unit != vmw_du_legacy);
>         int i, num_items;
>         SVGAGuestPtr ptr;

I can confirm that this patch fixes the problem for me. Nice! I did have
to manually apply it -- looks like email converted the tabs into spaces.

I don't know if I'm supposed to put tags here for you to include in the
final patch or not, but I'll do it just in case:

Tested-by: Doug Brown <doug@schmorgal.com>

Thanks!
Doug