From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-oi1-f169.google.com (mail-oi1-f169.google.com [209.85.167.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33D12345ED8 for ; Thu, 20 Aug 2026 03:19:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.169 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787195950; cv=none; b=e5uYjQ7IY/nOEEnE9J1MQSQ8CDl6RLFMnh9ewYkAY6pppsAYtLK8H7+3WTY/WaAUXbbzoLyGDRiIEwrEh+0kNbzM5D68PU39x6znNdQN9b3N1Zr2AWdAQrR9puosfqTCMG5SNbwKsoLCUDd1uH3X9i69/c/owOOSFpCGQOouioA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787195950; c=relaxed/simple; bh=Om9tKy+ndFoGgMqMWjHfskkHsjFr5yP5yS01SJbVSLg=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=Eqxhutdt2S6pv9jq6a3O7shX9EjLSLpL7tMzmVoa+vy6HjRxVzjbNhF3u3+s6qTI6NVlS0DTGQEveL3AO09WqQ7VEqhpcTdaNzV3C5iMQmP9JCNmorZZ23WTSmIzx/2F4C5S7S/PbOEG4ABw+4nQ0D+qsErg6naO9uJqf4iDQoQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=C1wPTKso; arc=none smtp.client-ip=209.85.167.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="C1wPTKso" Received: by mail-oi1-f169.google.com with SMTP id 5614622812f47-4a4cb36ae00so1316937b6e.0 for ; Wed, 19 Aug 2026 20:19:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1787195948; x=1787800748; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to:content-type; bh=MQfNhWjHAIsYvqRXEU3FJgTiTF29iPfV7gMZoO8YVqY=; b=C1wPTKsoG0Jl5yXH7/DAKn+pcxsMkHz27ZLq6uoG9+0A/YlDJHVBx1uNw6qMTQ0tj3 CX3Vd2+04RPkdwmt5BxK0bLoFSEenJiyk1uZeUaDcRpuT/po3FmEpWkTxz8PT0NYsYeX 8ixGohAE/ZBpJPP+OL9T44Y++hHmM4pMkhpiuK86QdtZLLIVSsGnQDzzTLOZYp5S+RMV 5pgefcdUb4DSPYhcxCoWLLC1R6lOJJf2gsi8JsFRz6+H1IvzWV5XTlsHUeQBHjIu+UJ8 kL+k6JADHGh77XP0f2LpN9Zm4bfI/f0WTzz1gRh6y6QglSUJnZf+t2YJdOlIn9/OfceU z+8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787195948; x=1787800748; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=MQfNhWjHAIsYvqRXEU3FJgTiTF29iPfV7gMZoO8YVqY=; b=KVlwuoLe0oLdkhS4ld97CfxMIGm+Nams1c3qy6Yxn+3tTp8bEU/LXmlzLOBzKiTknV x7yVZx0a38kj85RG5syjFp+Gfnc4xYb6sSHt0As5NunXZLJM2vcl3C6BdUP09crmOn+d Git3+hRj/nbFdh8RZ9uaCwzsBALmgDucmZcralEhX9FI5ThObvrjc6cfq7vjkLTXhWQW 9Y2NV5r28harbrTve2IXNxJBHPVFaLAh5+dUVZkJ18rsHVWC0aTkiwmXsC26X44n8BSc YHXmq35ymg+FoqzhstTNdVemJ2utuS2eY6UJKPkBiiYZRsiaVPLlOeqvDY1S8/sp8I/l IdvA== X-Forwarded-Encrypted: i=1; AHgh+RqRj+mI/nsJd8X6s039JriOVRqaWZfn/lofCQDwnW4/1Lyo3Y5eaX08f4PwfFhF57BdEiniDJZINePPzpk=@vger.kernel.org X-Gm-Message-State: AOJu0YwEZvpldx+a67w+VCde+rqTZzePGin8HZ89INFEhcRFfhzCGMS6 rhYK/BgVz9oLs/Ts4MX6C08PRz9kJtrbCwwr/bp8yVtYlZ5ttO155aGPgILyYlEA6RM= X-Gm-Gg: AR+sD121TTyYmNJ1r3fUXibqvYcmoge9MuMdDiouns4MkrauKPzxoSF2z3lQqIl5KTq YX20TY1CJM1PqASLNkOoHnL+MDsY1gYZqaW9rO/uPU2d1Pk7aLZUubbxoMzct6ne34EpfolNnOe CeIWo3ftfv1tj0D46LTi74SAjS6dCz8/Cq2enX8lH7H1sT9atMThcD9Obf2tre8DJ2SE4CeF6E6 dhAaBMZ6/9bSR0HMUw1YMttS3LrQkZ5aLWAjFOyh4BXBF+pGyU8+8a7RHydOat0OVIiK+1skeOI T9YLciBT25LaKCps7X4VA8fO3CEBUwY4tNLZevx7LonPePDegfQuI5dBQx1ivKpIKP5UydhmK1W Ogeade7gPXiNscW3K/gOIu/7otLqIFrUWc1e0Y4GFdIBjMpPCV56yKzEZgiTd/4a/NRi5DCnLje KoOnhgofh2HpBytDlwQbwyvRVTMIT6dZFCLLy1m2MVoHi6yrPsTnt5Q9D5lwRTdfbHhr8QrBZOC eRbwE2fKA8GF3kDpIVyYi0fl+EpIwtAakAZnM5SN6Oh1Wd4ErW+cuE6VM+R9rz2DhyX2A6WYTRY Pt74RF2BT1XhchvVIa0EF85fFg== X-Received: by 2002:a05:6808:5385:b0:4a4:9e18:607a with SMTP id 5614622812f47-4b2bcb6daf0mr9471555b6e.21.1787195947888; Wed, 19 Aug 2026 20:19:07 -0700 (PDT) Received: from craftbox (static-173-249-255-172.cust.tzulo.com. [173.249.255.172]) by smtp.gmail.com with ESMTPSA id 5614622812f47-4b2d68439f5sm619427b6e.5.2026.08.19.20.19.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Aug 2026 20:19:07 -0700 (PDT) From: Nathan Bourgeois To: intel-xe@lists.freedesktop.org Cc: dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org, Matthew Brost , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Rodrigo Vivi , =?UTF-8?q?Christian=20K=C3=B6nig?= , Matthew Auld , David Airlie , Simona Vetter , Nathan Bourgeois Subject: [PATCH] drm/xe: Fix unnecessary host-side population of ttm_tt on non-TT resources Date: Wed, 19 Aug 2026 23:19:01 -0400 Message-ID: <20260820031901.1018324-1-iridescentrosesfall@gmail.com> X-Mailer: git-send-email 2.55.0 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit When a buffer object (BO) has a created but unpopulated ttm_tt and is exported, the default behavior populates the ttm_tt, even if the manager does not require a TT. This unnecessary host-side population reduces host memory available to the user. This occurs when a BO is created with XE_BO_FLAG_DEFER_BACKING and migration places the BO into a resource whose manager does not use the TT backing. The retained ttm_tt is not authoritative for storage, yet the default ttm_bo_setup_export() treats the existence of the ttm_tt as requiring population. The issue was reproduced with vLLM 0.27.1 loading 0xSer0/DeepSeek-V4-Flash-180B (d3c704b) on a system with 128 GB of RAM and six Intel Arc Pro B70 GPUs providing 192 GB of VRAM. Loading the weights (100.61 GB) triggered host-memory exhaustion and OOM events on the baseline kernel. On Ubuntu 24.04 with 7.0-12-generic, the vLLM service cgroup grew by 115.84 GB on the instrumented baseline and by 15.34 GB with this change, a reduction of 100.50 GB. Diagnostic instrumentation counting cumulative TT pages across all six GPUs recorded 109.22 GB on baseline and 8.72 GB with this change. The reduction is 0.11 GB less than the model weight size. The change creates an Xe-local export setup helper which skips ttm_bo_populate() when the BO has a current resource manager and if that manager has use_tt == false. If the manager does not use TT, then population is skipped, while an absent manager conservatively calls ttm_bo_populate(). Behavior for a manager which requires a TT backing remains unchanged. Keep this policy local to Xe so other TTM drivers retain their existing export behavior. The regression test creates both the control case and the VRAM migration case. The control case creates the deferred BO in system RAM and leaves it there, which requires a TT and population when xe_gem_prime_export() is called. The VRAM migration case creates the deferred BO in system RAM and then migrates it to VRAM, which does not require a TT and thus, when xe_gem_prime_export() is called, no population occurs. Fixes: 91494dee1091 ("xe: populate buffers before exporting them.") Cc: stable@vger.kernel.org Assisted-by: Codex:gpt-5.6-sol Signed-off-by: Nathan Bourgeois --- Newer Testing: - Built and booted b4f95affc66ef76342c1f6bf3849f6c8ade6b9d6 plus this patch on EPYC 7352 with six Intel Arc Pro B70. - KUnit xe_live_test: xe_dma_buf_kunit: pass:6 fail:0 skip:0 total:6 PASS - vLLM 0.27.1 loading DeepSeek-V4-Flash-0731 (full model compared to REAP) on six Intel Arc Pro B70 GPUs: No OOM error, 17.82 GB delta. drivers/gpu/drm/xe/tests/xe_dma_buf.c | 105 ++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_dma_buf.c | 28 ++++++- 2 files changed, 132 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/xe/tests/xe_dma_buf.c b/drivers/gpu/drm/xe/tests/xe_dma_buf.c index 0be8440b3976..bff803ab7a57 100644 --- a/drivers/gpu/drm/xe/tests/xe_dma_buf.c +++ b/drivers/gpu/drm/xe/tests/xe_dma_buf.c @@ -260,12 +260,117 @@ static const struct dma_buf_test_params test_params[] = { {} }; +static void xe_test_dmabuf_export_deferred(struct xe_device *xe, u32 bo_flags, + u32 mem_type, bool expect_populated) +{ + struct drm_exec *exec = XE_VALIDATION_OPT_OUT; + struct kunit *test = kunit_get_current_test(); + struct ttm_resource_manager *man; + struct dma_buf *dmabuf; + struct xe_bo *bo; + size_t size = PAGE_SIZE; + int err; + + /* No VRAM on device? */ + if (!ttm_manager_type(&xe->ttm, mem_type)) + return; + + if (mem_type == XE_PL_VRAM0 && + xe->info.vram_flags & XE_VRAM_FLAGS_NEED64K) + size = SZ_64K; + + /* + * DEFER_BACKING places the BO in SYSTEM and creates a ttm_tt which + * is unpopulated. In the case of VRAM, migrating leaves the ttm_tt + * retained and unpopulated while VRAM becomes the real backing. + */ + bo = xe_bo_create_user(xe, NULL, size, DRM_XE_GEM_CPU_CACHING_WC, + bo_flags | XE_BO_FLAG_DEFER_BACKING, NULL); + if (IS_ERR(bo)) { + KUNIT_FAIL(test, "BO creation failed: %pe\n", bo); + return; + } + + err = xe_bo_lock(bo, false); + if (err) { + KUNIT_FAIL(test, "BO lock failed: %d\n", err); + goto out_put_bo; + } + + if (bo->ttm.resource->mem_type != mem_type) + err = xe_bo_migrate(bo, mem_type, NULL, exec); + if (err) { + KUNIT_FAIL(test, "BO migration to %u failed: %d\n", mem_type, + err); + goto out_unlock; + } + + man = ttm_manager_type(bo->ttm.bdev, bo->ttm.resource->mem_type); + if (!man || !bo->ttm.ttm) { + KUNIT_FAIL(test, "Expected a retained unpopulated TT\n"); + goto out_unlock; + } + + /* Precondition: ttm_tt starts unpopulated after migration */ + KUNIT_EXPECT_EQ(test, man->use_tt, expect_populated); + KUNIT_EXPECT_FALSE(test, ttm_tt_is_populated(bo->ttm.ttm)); + + xe_bo_unlock(bo); + + dmabuf = xe_gem_prime_export(&bo->ttm.base, 0); + if (IS_ERR(dmabuf)) { + KUNIT_FAIL(test, "dma-buf export failed: %pe\n", dmabuf); + goto out_put_bo; + } + + err = xe_bo_lock(bo, false); + if (err) { + KUNIT_FAIL(test, "post-export BO lock failed: %d\n", err); + goto out_put_dmabuf; + } + + /* Postcondition: if VRAM, ttm_tt remains unpopulated, if SYSTEM ttm_tt is populated */ + KUNIT_EXPECT_EQ(test, bo->ttm.resource->mem_type, mem_type); + KUNIT_EXPECT_NOT_NULL(test, bo->ttm.ttm); + if (bo->ttm.ttm) + KUNIT_EXPECT_EQ(test, ttm_tt_is_populated(bo->ttm.ttm), + expect_populated); + + xe_bo_unlock(bo); + dma_buf_put(dmabuf); + drm_gem_object_put(&bo->ttm.base); + return; + +out_unlock: + xe_bo_unlock(bo); + goto out_put_bo; +out_put_dmabuf: + dma_buf_put(dmabuf); +out_put_bo: + drm_gem_object_put(&bo->ttm.base); +} + static int dma_buf_run_device(struct xe_device *xe) { const struct dma_buf_test_params *params; struct kunit *test = kunit_get_current_test(); guard(xe_pm_runtime)(xe); + + /* + * A retained TT must not be populated when VRAM is the backing + * resource. + */ + xe_test_dmabuf_export_deferred(xe, XE_BO_FLAG_VRAM0, XE_PL_VRAM0, + false); + + /* + * Control case: deferred SYSTEM backing must still be populated + * before export. + */ + xe_test_dmabuf_export_deferred(xe, XE_BO_FLAG_SYSTEM, XE_PL_SYSTEM, + true); + for (params = test_params; params->mem_mask; ++params) { struct dma_buf_test_params p = *params; diff --git a/drivers/gpu/drm/xe/xe_dma_buf.c b/drivers/gpu/drm/xe/xe_dma_buf.c index bf0728838ead..0c4e4e2e1a81 100644 --- a/drivers/gpu/drm/xe/xe_dma_buf.c +++ b/drivers/gpu/drm/xe/xe_dma_buf.c @@ -219,6 +219,32 @@ static const struct dma_buf_ops xe_dmabuf_ops = { .vunmap = drm_gem_dmabuf_vunmap, }; +static int xe_dma_bo_setup_export(struct ttm_buffer_object *tbo, + struct ttm_operation_ctx *ctx) +{ + struct ttm_resource_manager *man = NULL; + int ret; + + ret = ttm_bo_reserve(tbo, false, false, NULL); + if (ret) + return ret; + + if (tbo->resource) + man = ttm_manager_type(tbo->bdev, tbo->resource->mem_type); + + /* + * Do not populate BO-sized system pages when backed by a non-TT resource. + * This is Xe-specific; the generic ttm_bo_setup_export() always populates. + */ + if (man && !man->use_tt) + ret = 0; + else + ret = ttm_bo_populate(tbo, ctx); + + ttm_bo_unreserve(tbo); + return ret; +} + struct dma_buf *xe_gem_prime_export(struct drm_gem_object *obj, int flags) { struct xe_bo *bo = gem_to_xe_bo(obj); @@ -257,7 +283,7 @@ struct dma_buf *xe_gem_prime_export(struct drm_gem_object *obj, int flags) xe_bo_willneed_get_locked(bo); xe_bo_unlock(bo); - ret = ttm_bo_setup_export(&bo->ttm, &ctx); + ret = xe_dma_bo_setup_export(&bo->ttm, &ctx); if (ret) goto out_put; base-commit: b4f95affc66ef76342c1f6bf3849f6c8ade6b9d6 -- 2.55.0