From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 46CE830D3E7 for ; Wed, 22 Jul 2026 04:42:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784695350; cv=none; b=kZZ3/jrA5DbIgxoontgSNg5rLjNqU9REnmTIxyJAMUHNkZsTqUWJLVBs7aRjrOEryR6/aj6LPIn9X5J6TVsZUdkEfkbsh+vAnN8OItqYdJObHLT8YV89g9xgFVA7qJ04CqVQKsUPiPmT3b1bcuLIDDx6oLL/S/WdoCr4liq36RI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784695350; c=relaxed/simple; bh=egqyx0xIHYO6pVjtq79Q7C2oe7YYIs3Mzn5MdOs14pI=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=aGtGu01E1QTFyCBq2WFOspmHDT8NS4fEnzosWA6SUjhLLEMqrL40e9NdQ4Xq2H3gky8BEzQ0ky9126xyH2MXKSbM8CqsL9K3WnmefJUcsPMjqhtoHlbBJ/TaSP38ytPmqvayaXLYerYCwCvun6Hd0iIGuTuOdIV/9+QVrsPIf0A= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=hM6a1C1Q; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="hM6a1C1Q" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1784695347; x=1816231347; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=egqyx0xIHYO6pVjtq79Q7C2oe7YYIs3Mzn5MdOs14pI=; b=hM6a1C1Q8wtbZLfExHMUP45zRDupmYrBErRat7lbPgX1do+6fPgoJ7vj K6SLpaOc4/PjFfF1J3/DRwHcSCd/0uU/fwMMpgk8caIcyZffgV/I9VlnW hNwZY4zaP7lf4AUONbkSY/JlyZqXgtaiiTpSGxxbgGnywrfwkgKWrPNjM /wRajZPxzI8+WweKP7ENRr8ZHyhZlWnwRk6Sp7NHh1R8DPSbnXcVmXHKb 9mkkzGi8iyJrgi1qY1/lHqI5BWu2YD5u9fvi+3IdqNS+ODNVZxv2amImH n4+O69wQ+i0u+LoX2NccjbBd+tPsGSjGsUlCC0IUJ2iPW6KvQxVhwYvUf A==; X-CSE-ConnectionGUID: 2qxVQa9CRPi4TitD24sCPA== X-CSE-MsgGUID: 0CxMOi8mRcOgAG5hfeExZg== X-IronPort-AV: E=McAfee;i="6800,10657,11853"; a="102865779" X-IronPort-AV: E=Sophos;i="6.25,177,1779174000"; d="scan'208";a="102865779" Received: from fmviesa010.fm.intel.com ([10.60.135.150]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2026 21:42:26 -0700 X-CSE-ConnectionGUID: f5jNSLotR9G+hsgBQa+nlQ== X-CSE-MsgGUID: NNhGkjL0QKOHHVPYPaELjA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.25,177,1779174000"; d="scan'208";a="254063555" Received: from gsse-cloud1.jf.intel.com ([10.54.39.91]) by fmviesa010-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Jul 2026 21:42:26 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , Christian Koenig , Huang Rui , Matthew Auld , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Tvrtko Ursulin , Dave Airlie , Matthew Wilcox Subject: [PATCH 1/3] mm/huge_memory: add folio_split_driver_managed() Date: Tue, 21 Jul 2026 21:42:18 -0700 Message-Id: <20260722044220.1110278-1-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Add a lightweight structural split primitive for large (compound) folios that a driver allocated with __GFP_COMP and manages entirely by itself, outside of the core mm's view. The existing split paths - split_folio() and folio_split_unmapped() - are built for folios that the mm owns: they perform a refcount freeze, walk and remap the rmap, and take the anon_vma / i_mmap locks, and folio_split_unmapped() further assumes an anon, pagecache-style refcount model (nr_pages + 1). None of that applies to a folio that is: - singly referenced (the caller holds the only reference), - not mapped through the rmap (folio_mapped() == 0), - not in the page cache or swap cache (folio->mapping == NULL), - not on any LRU or the deferred-split list. For such a folio the split is purely structural: because nothing else in the kernel can reach it, there is no need to freeze the refcount or touch any mapping. folio_split_driver_managed() therefore performs only the compound and split-accounting teardown via __split_unmapped_folio() and then hands each resulting order-@new_order folio its own reference, mirroring split_page() for compound folios. The caller keeps the original reference on the first resulting folio and is responsible for freeing all of them individually. The immediate user is TTM's GPU page pool, which allocates higher-order compound pages, maps them into userspace via VM_PFNMAP (never through the rmap), and needs to split them into order-0 folios under memory pressure so pages can be backed up to shmem and freed one at a time. A CONFIG_TRANSPARENT_HUGEPAGE=n stub is provided so callers can build without the split machinery; it warns and returns -EINVAL. Cc: Maarten Lankhorst Cc: Maxime Ripard Cc: Thomas Zimmermann Cc: David Airlie Cc: Simona Vetter Cc: Christian Koenig Cc: Huang Rui Cc: Matthew Auld Cc: Matthew Brost Cc: Andrew Morton Cc: David Hildenbrand Cc: Lorenzo Stoakes Cc: Zi Yan Cc: Baolin Wang Cc: "Liam R. Howlett" Cc: Nico Pache Cc: Ryan Roberts Cc: Dev Jain Cc: Barry Song Cc: Lance Yang Cc: Tvrtko Ursulin Cc: Dave Airlie Cc: dri-devel@lists.freedesktop.org Cc: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org Suggested-by: Matthew Wilcox Signed-off-by: Matthew Brost Assisted-by: GitHub-Copilot:claude-opus-4.8 --- The patch is based on drm-tip rather than the core MM branches to facilitate Intel CI testing and initial review. It can be rebased onto the core MM branches in a subsequent revision. --- include/linux/huge_mm.h | 8 ++++++ mm/huge_memory.c | 63 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 71 insertions(+) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index ad20f7f8c179..35661d82d54a 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -402,6 +402,7 @@ enum split_type { int __split_huge_page_to_list_to_order(struct page *page, struct list_head *list, unsigned int new_order); int folio_split_unmapped(struct folio *folio, unsigned int new_order); +int folio_split_driver_managed(struct folio *folio, unsigned int new_order); unsigned int min_order_for_split(struct folio *folio); int split_folio_to_list(struct folio *folio, struct list_head *list); int folio_check_splittable(struct folio *folio, unsigned int new_order, @@ -656,6 +657,13 @@ static inline int split_folio_to_list(struct folio *folio, struct list_head *lis return -EINVAL; } +static inline int folio_split_driver_managed(struct folio *folio, + unsigned int new_order) +{ + VM_WARN_ON_ONCE_FOLIO(1, folio); + return -EINVAL; +} + static inline int folio_split(struct folio *folio, unsigned int new_order, struct page *page, struct list_head *list) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 2bccb0a53a0a..06f9a5f35df8 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -4185,6 +4185,69 @@ int folio_split_unmapped(struct folio *folio, unsigned int new_order) return ret; } +/** + * folio_split_driver_managed() - split an exclusively-owned, off-LRU folio + * @folio: folio to split. Must be a large (compound) folio that is owned + * exclusively by the caller and is invisible to the core mm. + * @new_order: the order of the folios after the split. + * + * This is a lightweight structural split for folios that a driver allocated + * and manages itself (for example TTM's GPU page pool, which allocates + * higher-order compound pages with __GFP_COMP and maps them into userspace + * via VM_PFNMAP rather than through the rmap). Such folios are: + * + * - singly referenced (the caller holds the only reference), + * - not mapped through the rmap (folio_mapcount() == 0), + * - not in the page cache or swap cache (folio->mapping == NULL), + * - not on any LRU or the deferred-split list. + * + * Because nothing else in the kernel can reach the folio, this helper does + * not perform the refcount freeze / remap / anon_vma & i_mmap locking dance + * that split_folio() and folio_split_unmapped() require. It performs only + * the compound and split-accounting teardown and then hands each resulting + * folio its own reference, mirroring split_page() for compound folios. + * + * The caller is responsible for freeing the resulting folios individually. + * + * Context: caller holds the only reference and excludes concurrent access. + * Does not sleep. + * + * Return: 0 on success, negative errno on failure. + */ +int folio_split_driver_managed(struct folio *folio, unsigned int new_order) +{ + unsigned int old_order = folio_order(folio); + unsigned int split_nr = 1U << new_order; + unsigned int nr = 1U << old_order; + unsigned int i; + + if (new_order >= old_order) + return -EINVAL; + + VM_WARN_ON_ONCE_FOLIO(folio_ref_count(folio) != 1, folio); + VM_WARN_ON_ONCE_FOLIO(folio_mapped(folio), folio); + VM_WARN_ON_ONCE_FOLIO(folio->mapping, folio); + VM_WARN_ON_ONCE_FOLIO(folio_test_swapcache(folio), folio); + VM_WARN_ON_ONCE_FOLIO(folio_test_lru(folio), folio); + + /* + * Structural + split-accounting teardown only. No mapping/xarray, no + * refcount freeze: the folio is frozen-by-ownership already. + */ + __split_unmapped_folio(folio, new_order, &folio->page, NULL, NULL, + SPLIT_TYPE_UNIFORM); + + /* + * Give every resulting head folio its own reference. The original + * reference stays on the first one, exactly like split_page(). + */ + for (i = split_nr; i < nr; i += split_nr) + set_page_refcounted(folio_page(folio, i)); + + return 0; +} +EXPORT_SYMBOL_GPL(folio_split_driver_managed); + /* * This function splits a large folio into smaller folios of order @new_order. * @page can point to any page of the large folio to split. The split operation -- 2.34.1