[REGRESSION 7.2] drm/amdgpu: ~275 SDMA jobs per sparse VA bind since 4cdbba5a (RE Requiem 90->4 fps)

[REGRESSION 7.2] drm/amdgpu: ~275 SDMA jobs per sparse VA bind since 4cdbba5a (RE Requiem 90->4 fps)

[Mikhail Gavrilov]

[-- Attachment #1: Type: text/plain, Size: 3017 bytes --]

Hi Christian, Alex,

git bisect points to

  4cdbba5a16aa ("drm/amdgpu: restructure VM state machine v4")

as the first bad commit (its parent tests fine) for a severe
interactivity regression.

It was merged during the current 7.2 merge window; it is not in any
released kernel yet and will first appear in 7.2-rc1.

Symptom: Resident Evil Requiem (re9.exe under VKD3D-Proton, RADV, RX
7900 XTX / Navi31, gfx11) drops from ~90 to 3-4 fps the instant the
camera moves; still scenes are fine. The previous bisect point
d352990bcaab is smooth.

The game streams tiled/sparse resources, so every camera move issues a
burst of vkQueueBindSparse, i.e. a burst of DRM_IOCTL_AMDGPU_GEM_VA
going through amdgpu_gem_va_update_vm. The regression is confined to
that path; the CS submission path is unchanged
(bo_update-per-submission is identical on both sides).

What the restructure changed, measured: a single sparse bind now fans
out into hundreds of individual SDMA PTE-write jobs instead of a
couple. Same workload (active camera panning), identical kernel config
on both bisect points:

  good (d352990bcaab), per second under panning:
    amdgpu_gem_va_update_vm   ~1460
    drm_suballoc_new          ~3070
    amdgpu_job_alloc_with_ib  ~3070
    => ~2.1 SDMA jobs per bind

  bad (4cdbba5a16aa), per second under the same panning:
    amdgpu_gem_va_update_vm   ~158
    drm_suballoc_new          ~43500
    amdgpu_job_alloc_with_ib  ~43500
    => ~275 SDMA jobs per bind

So per-bind job/IB count went from ~2 to ~275 (~130x), and the
absolute job_alloc rate rose ~14x (3k -> 43.5k/s) even though the bad
kernel is starved to 4 fps and therefore issues far fewer binds per
second. The PTE-range update inside one bind appears to have lost its
coalescing and now submits roughly one job per fragment.

Caveat on magnitude. Both kernels are debug builds (KASAN + LOCKDEP +
PREEMPT_FULL), which is my normal environment. The 90->4 collapse is
dominated by KASAN: in perf the amdgpu_gem_va_update_vm subtree is
~19% of cycles on bad, of which ~17 points are
kasan_save_stack/stack-walking on each allocation, so the
uninstrumented CPU cost of building these jobs is only ~3%. But the
per-bind job *count* is config-independent: ~275 jobs per bind
regardless of build. On this debug kernel that already came to ~43.5k
jobs/s (~14x the old rate); a production kernel could see a higher
bind rate, not lower, so the job rate and the corresponding ring/fence
overhead would be at least as bad.

I have a reliable reproducer and both bisect endpoints built; glad to
test a fix and provide Tested-by, and to grab any further traces (the
suballocator churn is drm_suballoc_new/insert/try_free, consistent
with the job flood). Full bisect log and perf/bpftrace captures on
request.

Kernel .config attached.
Probe: https://linux-hardware.org/?probe=e92f6143b2
Mesa/RADV: 26.1.99 (git 99a268c)
vkd3d-proton: 3.1.0 (ee737e324376289)
Proton: Experimental 11.0 (build 20260617)

#regzbot introduced: 4cdbba5a16aa

-- 
Thanks,
Mikhail

[-- Attachment #2: .config.zip --]
[-- Type: application/zip, Size: 71538 bytes --]

Re: [REGRESSION 7.2] drm/amdgpu: ~275 SDMA jobs per sparse VA bind since 4cdbba5a (RE Requiem 90->4 fps)

[Natalie Vock]

On 6/21/26 22:55, Mikhail Gavrilov wrote:
> Hi Christian, Alex,
> 
> git bisect points to
> 
>    4cdbba5a16aa ("drm/amdgpu: restructure VM state machine v4")
> 
> as the first bad commit (its parent tests fine) for a severe
> interactivity regression.
> 
> It was merged during the current 7.2 merge window; it is not in any
> released kernel yet and will first appear in 7.2-rc1.
> 
> Symptom: Resident Evil Requiem (re9.exe under VKD3D-Proton, RADV, RX
> 7900 XTX / Navi31, gfx11) drops from ~90 to 3-4 fps the instant the
> camera moves; still scenes are fine. The previous bisect point
> d352990bcaab is smooth.

I already fixed this in a patchset[1] that should be on its way - seems 
like it didn't make it into rc1 at least?

Christian, Alex, can you make sure that the first patch from that set, 
"drm/amdgpu: Only set bo->moved when the BO was actually moved", is 
included in a -fixes PR for 7.2?

For the record, I intentionally did not include a Fixes: tag in this 
patchset, because the commit was not merged anywhere but 
amd-staging-drm-next at the point of writing, and the same commits in 
amd-staging-drm-next and upstream have different SHAs. AFAIU, this also 
prevents me from telling regzbot that a proper fix is underway for now. 
Not sure if there's a workaround for this?

Best,
Natalie

[1] 
https://lore.kernel.org/amd-gfx/20260529153051.973261-1-natalie.vock@gmx.de/

Re: [REGRESSION 7.2] drm/amdgpu: ~275 SDMA jobs per sparse VA bind since 4cdbba5a (RE Requiem 90->4 fps)

[Mikhail Gavrilov]

On Mon, Jun 22, 2026 at 10:44 AM Natalie Vock <natalie.vock@gmx.de> wrote:
>
> On 6/21/26 22:55, Mikhail Gavrilov wrote:
> > Hi Christian, Alex,
> >
> > git bisect points to
> >
> >    4cdbba5a16aa ("drm/amdgpu: restructure VM state machine v4")
> >
> > as the first bad commit (its parent tests fine) for a severe
> > interactivity regression.
> >
> > It was merged during the current 7.2 merge window; it is not in any
> > released kernel yet and will first appear in 7.2-rc1.
> >
> > Symptom: Resident Evil Requiem (re9.exe under VKD3D-Proton, RADV, RX
> > 7900 XTX / Navi31, gfx11) drops from ~90 to 3-4 fps the instant the
> > camera moves; still scenes are fine. The previous bisect point
> > d352990bcaab is smooth.
>
> I already fixed this in a patchset[1] that should be on its way - seems
> like it didn't make it into rc1 at least?
>
> Christian, Alex, can you make sure that the first patch from that set,
> "drm/amdgpu: Only set bo->moved when the BO was actually moved", is
> included in a -fixes PR for 7.2?
>
> For the record, I intentionally did not include a Fixes: tag in this
> patchset, because the commit was not merged anywhere but
> amd-staging-drm-next at the point of writing, and the same commits in
> amd-staging-drm-next and upstream have different SHAs. AFAIU, this also
> prevents me from telling regzbot that a proper fix is underway for now.
> Not sure if there's a workaround for this?
>
> Best,
> Natalie
>
> [1]
> https://lore.kernel.org/amd-gfx/20260529153051.973261-1-natalie.vock@gmx.de/

Thanks, Natalie. I applied that first patch ("drm/amdgpu: Only set
bo->moved when the BO was actually moved") on top of ef0c9f75a195 and
the regression is gone -- camera movement is back to ~90 fps.

Tested-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>

Re regzbot: since I opened the regression, I can link your fix from
here without a Fixes: tag or a matching SHA -- monitor just takes the
thread URL:

#regzbot monitor:
https://lore.kernel.org/amd-gfx/20260529153051.973261-1-natalie.vock@gmx.de/

Later, once the fix lands upstream, the entry can be closed with a
regzbot "fixed-by" on that commit's SHA.

-- 
Thanks,
Mikhail.