From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 878ED32470A for ; Sat, 11 Jul 2026 02:56:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783738603; cv=none; b=TxaVEQtdLiBMBHHenMX/ylcMGHzWeO5RuuKl8b1yI8KaS4dsY0M7g+1utOP/LMLQNaVttsGbKUZm+mTnxaC+xS+7N1+MwkXzOSeqFYzd2/d+7nXJVIWTKaSQAjs1rk2+IJSz8CIQlqdHJ8wGfpU7GQ30jwhylEX50KzGu9R5q80= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783738603; c=relaxed/simple; bh=MkDmOHMdUUIRDIcGOStHc6YdQK1SAZZSc3FC6q53LOQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=I+ZukM/Hy2ts65XMJJsFfAE2RCfMolC7Zc3PqaeYpSev0XY8vXcxqQiR18r9Jq9G853fuoJQTGugRVhPS2AICOeWWowhD4K9ztXin1zdyagis4yISu7GQ1hntmJOLImoRft0iNZ7erLvi+Z3WYWksKCeRtP6FVY1cI3tHgLJvMk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=V/Cb4pDI; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="V/Cb4pDI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1783738601; x=1815274601; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=MkDmOHMdUUIRDIcGOStHc6YdQK1SAZZSc3FC6q53LOQ=; b=V/Cb4pDIBzcRZ6wInzs2ni2COH1uP9feH5R8MN1bjUYhAtvNsHEDdyER Kl9NNOMix2sG6LrHGMeneDKPk/L0rNzk78ZsHOJvItHfuvW4RBXdb8oH5 4omtovV5c1DfkbeWq48ufwyNoR7p/gm/g9+XPu15hn4qO+whywP9Xxk1Q EoNBiQ9iCbHAaGCBCYoRGotAgPcqSTAaXVoBXvVXyvhJTFO/8P42s3ZFa uFGtq2OOmaIwkWIAEXCXIDurw/YBWYccPLprK0E/0kAWe/3C2Q5jYgt5E vwl3yNWeQZT94ol5BpEIaGdtZYvXwD4v35SSkoTTr8NctZ7Ssrv3JyVD8 g==; X-CSE-ConnectionGUID: h5lkQ5YrT969hqo17Zk8zQ== X-CSE-MsgGUID: UAg+ixqvSgOcoacApGcN6g== X-IronPort-AV: E=McAfee;i="6800,10657,11841"; a="101986326" X-IronPort-AV: E=Sophos;i="6.25,154,1779174000"; d="scan'208";a="101986326" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jul 2026 19:56:32 -0700 X-CSE-ConnectionGUID: DjfB2bHCRsuh5KDlYFMYOg== X-CSE-MsgGUID: 6iAXW7+RTR2nAszUf8yzMg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.25,154,1779174000"; d="scan'208";a="256987772" Received: from gsse-cloud1.jf.intel.com ([10.54.39.91]) by fmviesa004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jul 2026 19:56:31 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org Cc: Carlos Santa , Ryan Neph , Christian Koenig , Huang Rui , Matthew Auld , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , linux-kernel@vger.kernel.org, =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= Subject: [PATCH v3 23/33] drm/xe: Defrag using out-of-lock page preallocation Date: Fri, 10 Jul 2026 19:56:09 -0700 Message-Id: <20260711025619.2540575-24-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260711025619.2540575-1-matthew.brost@intel.com> References: <20260711025619.2540575-1-matthew.brost@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Convert xe_bo_defrag_one() to a three-phase scheme that moves the expensive beneficial-order allocations out of the BO dma-resv lock: 1. Under the lock, re-check eligibility, capture the tt cpu-caching and read the exact sub-optimal page count (ttm_tt_suboptimal_pages()), bounded by the run budget. 2. Drop the lock and preallocate that many beneficial-order chunks via ttm_pool_prealloc_fill(); reclaim/compaction stalls and the cpu cache mode change both happen here, outside the lock. 3. Re-take the lock, re-check, and validate consuming the prealloc bag. Any shortfall harvests the old backing instead of reclaiming, so the move never stalls under the lock; the BO re-queues for a later pass. With both allocation and caching hoisted out, the BO-lock held time drops to copy-bound (median ~0.27ms, p95 ~2ms); the remaining tail is lock-wait behind active rendering, as intended. Cc: Carlos Santa Cc: Ryan Neph Cc: Christian Koenig Cc: Huang Rui Cc: Matthew Auld Cc: Maarten Lankhorst Cc: Maxime Ripard Cc: Thomas Zimmermann Cc: David Airlie Cc: Simona Vetter Cc: dri-devel@lists.freedesktop.org Cc: linux-kernel@vger.kernel.org Cc: Thomas Hellström Assisted-by: GitHub_Copilot:claude-opus-4.8 Signed-off-by: Matthew Brost --- v3: - Use DIV_ROUND_UP_ULL() for the 64-bit preallocation chunk count (Sashiko) --- drivers/gpu/drm/xe/xe_bo.c | 46 +++++++++++++++++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/xe/xe_bo.c b/drivers/gpu/drm/xe/xe_bo.c index 0bdd1c920404..8b89d34f37d4 100644 --- a/drivers/gpu/drm/xe/xe_bo.c +++ b/drivers/gpu/drm/xe/xe_bo.c @@ -1258,16 +1258,56 @@ static int xe_bo_defrag_one(struct xe_device *xe, struct xe_bo *bo, .defrag_bytes_remaining = budget, }; struct ttm_buffer_object *ttm_bo = &bo->ttm; + struct ttm_pool *pool = &ttm_bo->bdev->pool; + struct ttm_pool_prealloc pp = {}; struct ttm_placement placement; struct ttm_place place; + enum ttm_caching tt_caching; + unsigned int order, want = 0; int ret = 0; *consumed = 0; *needs_more = false; + /* + * Phase 1: under the BO lock, re-check eligibility and estimate how + * many beneficial-order chunks the move will (re)allocate, bounded by + * the run's byte budget. Then drop the lock so the high-order, possibly + * reclaim/compaction stalling allocations happen unlocked. + */ + xe_bo_lock(bo, false); + if (!xe_bo_needs_defrag(bo)) { + xe_bo_defrag_remove(bo); + xe_bo_unlock(bo); + goto out; + } + tt_caching = bo->ttm.ttm->caching; + order = ttm_pool_prealloc_order(pool); + if (order && ttm_bo->ttm) { + u32 suboptimal = ttm_tt_suboptimal_pages(ttm_bo->ttm); + u64 cap = min_t(u64, budget, xe_bo_size(bo)) >> PAGE_SHIFT; + + /* Prealloc only the beneficial-order chunks the move replaces. */ + want = DIV_ROUND_UP_ULL(min_t(u64, suboptimal, cap), 1UL << order); + } + xe_bo_unlock(bo); + + /* Phase 2: preallocate outside the lock; */ + if (want) { + ret = ttm_pool_prealloc_fill(pool, tt_caching, &pp, want); + if (ret || !pp.count) { + ret = ret ?: -ENOMEM; + xe_gt_stats_incr(xe_root_mmio_gt(xe), + XE_GT_STATS_ID_DEFRAG_FAILED_COUNT, 1); + goto out_err; + } + } + + /* + * Phase 3: re-take the lock, re-check, and validate using the prealloc. + */ xe_bo_lock(bo, false); - /* Re-check eligibility under the BO lock. */ if (!xe_bo_needs_defrag(bo)) { xe_bo_defrag_remove(bo); goto unlock; @@ -1277,6 +1317,7 @@ static int xe_bo_defrag_one(struct xe_device *xe, struct xe_bo *bo, place.flags |= TTM_PL_FLAG_TEMPORARY; placement.num_placement = 1; placement.placement = &place; + ctx.prealloc = &pp; /* * On success the move reallocates the backing at beneficial order and @@ -1321,6 +1362,9 @@ static int xe_bo_defrag_one(struct xe_device *xe, struct xe_bo *bo, unlock: xe_bo_unlock(bo); +out_err: + ttm_pool_prealloc_fini(pool, &pp); +out: return ret; } -- 2.34.1