Re: [RFC PATCH v2 1/1] drm/xe: keep VM-bound WC BOs resident during reclaim

[Matthew Brost <matthew.brost@intel.com>]

On Thu, Aug 13, 2026 at 06:23:09PM -0700, Matthew Brost wrote:
> On Wed, Aug 12, 2026 at 04:28:47PM +0800, Neil Zhong wrote:
> > Hi Matt,
> > 
> > > Can you give me instructions on how to recreate this on our end and your
> > > machine, memory details? I have a bunch of various reproducers which I
> > > have been using for shrinker work and the more the better.
> > 
> > Yes. The test system and workload are as follows.
> > 
> >   - Panther Lake iGPU using shared system memory
> >   - 8 GiB installed memory; /proc/meminfo reports 7,723,392 KiB
> >   - 4 KiB base pages
> >   - four zram swap devices, 3,858,416 KiB in total
> >   - Linux 6.18.15, x86_64, PREEMPT_DYNAMIC
> >   - X11 fullscreen output at 3840x2160 and 60 Hz
> >   - a 3840x2160, 60 fps, HEVC HDR video played continuously in a loop
> > 
> > The player uses hardware decoding. Each decoded frame is imported through
> > DMA-BUF and processed by libplacebo/OpenGL for HDR tone mapping before
> > presentation. It calls glFlush() for every rendered frame. During steady
> > playback, the player accounts for about 1.85 GiB of logical Xe BO
> > allocation, including about 672 MiB reported as shared.
> > 
> > The player is not currently public. However, an equivalent pipeline should
> 
> This will become a problem if we introduce a uAPI, as we need an
> open-source consumer of that uAPI. Fortunately, it looks like we are
> already moving toward pinning uAPIs for several other reasons as well.
> 
> > reproduce the condition if it keeps the decoded surfaces, HDR intermediate
> > render targets and presentation surfaces alive, rather than creating a
> > small synthetic BO set. 
> > 
> > No additional memory-pressure tool was used for the first reproduction.
> > I start the video, let its working set reach steady state, and then capture
> > ten minutes while playback continues. On the 8 GiB system, the priority-only
> > run had MemAvailable between 2.89 and 3.15 GiB, while 97.8% of kswapd wakeups
> > were for order-10 allocations. Thus this reproduces without forcing an
> 
> We have upstream fixes for the order-10 allocations to avoid triggering
> reclaim for anything other than order-0 or order-9 allocations. I shared
> that patch in a previous reply, and it is probably worth pulling in as a
> mitigation, plus the core MM series.
> 
> > order-0 shortage.
> > 
> > I collected the trace with:
> > 
> >   sudo ./capture_xe_memory_churn.sh \
> >           -t 600 \
> >           -s 0.2 \
> >           -o ./xe-memory-churn
> > 
> > `-t 600` records ten minutes. `-t 0` can instead be used to record until
> > Ctrl-C. The 0.2 second option is only the /proc and TTM-pool sampling
> > interval; ftrace events are recorded continuously.
> > 
> > The player also logs one line after each frame submission in this form:
> > 
> >   gl_sw_submit_frame timing: ... Flush=123.456 ms
> > 
> > The log prefix contains the wall-clock timestamp. An equivalent reproducer
> > can record the time immediately after glFlush() returns and the measured
> > duration. The trace script inserts a wall-clock epoch marker into a
> > mono_raw ftrace stream so that the two timelines can be aligned.
> > 
> > The figures in my previous email came from the 6.18.15 kernel with [9]-[13]
> > backported, the fragmentation check applied to direct reclaim as well, and
> > the high-plus-min watermark experiment described there. The default-device
> > watermark capture used vm.min_free_kbytes=131072. The follow-up used 50000;
> > the workload and trace procedure were otherwise unchanged.
> > 
> > > Also a quick write up how you extracted these numbers from reproducer so
> > > I can recreate on my end.
> > 
> > The capture script creates temporary entry and return kprobes for:
> > 
> >   ttm_tt_backup()
> >   ttm_tt_restore()
> >   ttm_pool_alloc()
> >   ttm_pool_free()
> >   ttm_pool_shrink()
> > 
> > It also traces the Xe shrinker, TTM restore and cache-attribute functions,
> > kswapd and direct-reclaim events, compaction, and allocation
> > fragmentation.
> > 
> > At ttm_tt_backup() entry, the probe records the ttm_tt pointer and
> > num_pages. At return, it records the positive return value, which is the
> > number of pages actually backed up. I sum those successful return values,
> > not the requested page count, when reporting backup volume. A later
> > ttm_tt_restore() is matched to the most recent successful backup using the
> > same ttm_tt pointer. That provides per-object backup-to-restore latency and
> > repeated-cycle counts.
> > 
> > The byte-volume calculation is:
> > 
> >   backup bytes = sum(successful backup return pages) * PAGE_SIZE
> >   restore bytes = sum(num_pages for matched restores) * PAGE_SIZE
> > 
> > The reported backup-plus-restore volume is the sum of those two values.
> > It is cumulative migration/copy traffic, not resident memory and not net
> > memory freed. A shmem backup remains resident system memory unless those
> > shmem pages are subsequently swapped out.
> > 
> > > This customer, in particular, utilizes priority bands to express this
> > > heuristic (e.g., the compositor is the highest priority, any
> > > non-privileged UI-related content is normal priority, and everything
> > > else is low priority).
> > 
> > I think priority bands are useful for relative reclaim ordering, but they
> > do not by themselves express the guarantee needed here.
> > 
> > Consider a system with one large GPU workload. If nearly all reclaimable
> > BOs belong to that client and are placed in the high-priority band, the low
> > bands will contain few or no candidates. When enough memory is requested,
> > the shrinker must eventually enter the high band. In that case, high
> > priority delays reclaim but does not prevent it. The priority-only test
> > showed exactly this limitation: over ten minutes there were 6,743
> > successful backups and 6,693 restores, with 157 Flush calls over 16.667 ms
> > and a maximum Flush of 494.434 ms.
> > 
> > I am not suggesting that every BO of a high-priority client should be
> > unreclaimable. Whether reclaim is acceptable depends on the workload, and
> > the kernel cannot infer that semantic from WC, VM-bound state, client count
> > or BO size alone. For example, the driver-visible behavior of these two
> > workloads can look very similar:
> > 
> >   1. Foreground 4K60 HDR playback. Its active decoded surfaces, HDR render
> >      targets and presentation surfaces have a 16.667 ms deadline. Backing
> >      them up and restoring them causes an immediate and clearly visible
> >      product failure. These BOs should avoid eviction and shrinking while
> >      they are part of the active visual pipeline.
> > 
> >   2. Background image recognition or classification. Its BOs have no
> >      presentation deadline. Reclaiming them under system memory pressure
> >      is reasonable, even if the job later has to reconstruct its working
> >      set.
> 
> In this case, do you have two VMs sharing BOs? If the foreground and
> background tasks share a single VM, it does not matter which BOs are
> shrunk or restored because, when a VM is validated during an exec IOCTL,
> all BOs mapped within that VM are restored.
> 
> I assume that in this case you are using two VMs that share buffers as
> needed via dma-buf. Is that correct? If not, pinning is not going to
> help unless the pinned set includes everything.
> 
> > 
> > Priority bands cannot distinguish those cases if both clients assign their
> 
> Assuming there are two VMs here, set your foreground queue's priority to
> NORMAL and your background task queue's priority to LOW. (Alternatively,
> if the foreground task has CAP_SYS_ADMIN privileges, you could set it to
> HIGH, etc.) This should cause the background BOs to be shrunk rather
> than the foreground BOs.
>  
> There is another problem in Xe when VMs share BOs. The way a VM is
> locked and validated during exec IOCTLs can cause cross-VM lock
> contention get stuck behing shrinking. This issue exists regardless of
> whether a priority-based solution or pinning is used.
>  
> For example, assume there are two VMs, one for the foreground workload
> and one for the background workload. We correctly evict the background
> BOs when needed, and some set of dma-buf buffers is shared between the
> two VMs. Below a flow that shows a contention problem:
> 
> 1. a private BO from background is shrunk
> 2. background exec IOCTL
> 	2.1. grab all dma-resv locks (including some shared with
> 	     foreground)
> 	2.2. restore shunk BO
> 	2.3. submit GPU job
> 3. foreground exec IOCTL (in parallel with 2)
> 	3.1 grab all dma-resv locks (including some shared with
> 	    background)
> 	3.2. submit GPU job
> 
> In this example 3.1 can get stuck behind 2.2 (an unrelated restore),
> thus 3 can miss a presentation deadline.
> 
> We likely need our Xe IOCTL and GPUVM code to be smart enough to perform
> multiple lock-and-validate passes. For example, we could initially lock
> only the eviction set, validate everything, and then repeat the process
> until the eviction set is empty. Only after that would we lock and
> validate everything else.
> 
> This is an existing problem that really needs to be fixed. I'll probably
> take a look at addressing it.
> 

Here is an attempt at fixing the cross-VM issue:

https://patchwork.freedesktop.org/series/172205/

Matt

> > current working set a high relative priority. This is why I think the
> > business semantic has to come from userspace. Priority bands can remain the
> > general ordering mechanism, while a separate, explicit and bounded
> > NO_SHRINK or latency-critical mark protects only the BOs in an active visual
> > pipeline. The mark should be removed as soon as a surface leaves that
> > working set.
> > 
> > > To be clear - this would be an addition to fixes discussed above, right?
> > 
> > Yes. I see explicit workload-semantic protection as an addition to the
> > core MM fragmentation/shrinker fixes and the Xe/TTM priority bands, not a
> > replacement for either. I agree that testing series 168651 together with
> > the priority bands is still useful for general working-set preservation.
> > It can reduce accidental reclaim, while an accounted NO_SHRINK contract
> > handles the smaller set for which a missed presentation deadline is not an
> > acceptable reclaim tradeoff.
> 
> I don't think anyone is opposed to pinning if we can get the right
> permission control in place (most likely cgroups).
> 
> Matt
> 
> > 
> > Thanks,
> > Neil