From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8FE10435AB5; Wed, 5 Aug 2026 11:33:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.21 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785929632; cv=none; b=h0FNg5ye/SaT+oGpImsYt5nqSdN+fQSSZFNn2vmF2chogrnnIdm4A//6OeLhSyUG6/S7LJI+xYAgDpjdgnH4NNNrEYDCxWEnKYEOhq8hkO4tx3nKkLDUM2fQfsYz7VY+RHbrEeIWKtmLPUtCdzGb9T/XicaITsDT5j1Prx4aLHM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785929632; c=relaxed/simple; bh=1PZI3h2x2Ab0c8hVhz32upUrjeLKnaZ4yhcq7z0XhUo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=bx1VRI5rHRaqErawx6wfSRLxYQDtCW7A3dz1Mm7bm8jHtDnFxnJRehAmDNpcQI0Ou/K8qGT29rMu12uxNyMMTwmty91uWieUn2gAl6CU9BTPDon4aDXS+v+XlmhG1HfDAXreKdv577vfZSOe/kxBoeLu42YeMzxO/UhDyKMyAZc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=UM4dZIU1; arc=none smtp.client-ip=198.175.65.21 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="UM4dZIU1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1785929629; x=1817465629; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1PZI3h2x2Ab0c8hVhz32upUrjeLKnaZ4yhcq7z0XhUo=; b=UM4dZIU1+T6JbUBYnjKQOj8VTM9mA9YjqYPPEDCH9VRT9I42ZNef88Sf wnBTDiv9ZBm5IM6BtrdItmdxwZXuj89TFjmoK3Ojy6OW9lOhRxY+5/KHv TWJilnSJSTgy/tw1KvYyGyBs367noENYENHhWsBMQDzLoS2RFEE8uRjHE Bnb31YIcgSHLuwAUNxbsloAdiBUgWv/UH4s0soxgZAaA2AoSYZZEbIUdL TshZMuztzQD1I7bX0qJJLxTh5MBcw3mIqllSZ2fVqrmaLlXlcyXQeLwmJ TBSPoKpRrYdybziXiugZOrga+jsES223zSmJnqIB8f8u4qXfhtfo6H65E Q==; X-CSE-ConnectionGUID: K3OiiH4FSQC67nV6yBrEzQ== X-CSE-MsgGUID: v2QXv416RMWAG0qXH/6nhQ== X-IronPort-AV: E=McAfee;i="6800,10657,11865"; a="86358204" X-IronPort-AV: E=Sophos;i="6.25,206,1779174000"; d="scan'208";a="86358204" Received: from orviesa003.jf.intel.com ([10.64.159.143]) by orvoesa113.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Aug 2026 04:33:45 -0700 X-CSE-ConnectionGUID: O6nBJ1lDQK2RDwRKOtvgJQ== X-CSE-MsgGUID: GK4wobLNTNuumrCiE9PkbA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.25,206,1779174000"; d="scan'208";a="265289750" Received: from gsse-cloud1.jf.intel.com ([10.54.39.91]) by ORVIESA003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Aug 2026 04:33:45 -0700 From: Matthew Brost To: intel-xe@lists.freedesktop.org, dri-devel@lists.freedesktop.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: stable@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Usama Arif , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Balbir Singh , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , David Airlie , Simona Vetter , =?UTF-8?q?Thomas=20Hellstr=C3=B6m?= , Francois Dugast Subject: [PATCH 3/4] drm/pagemap: Fix folio allocation fallback and use-after-put Date: Wed, 5 Aug 2026 04:33:37 -0700 Message-Id: <20260805113338.3742178-4-matthew.brost@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260805113338.3742178-1-matthew.brost@intel.com> References: <20260805113338.3742178-1-matthew.brost@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit drm_pagemap_migrate_populate_ram_pfn() had two issues when populating RAM PFNs with higher-order folios: 1. The higher-order vma_alloc_folio()/folio_alloc() calls did not pass __GFP_NOWARN, so a THP allocation failure under memory pressure would spam the kernel log, and there was no fallback path despite a TODO comment stating one was needed. Add __GFP_NOWARN to the higher-order allocation and, on failure, fall back to order-0 allocations for the entire range originally covered by the failed higher-order allocation, leaving MIGRATE_PFN_COMPOUND unset for those PFNs. 2. In the free_pages error path, order was computed via folio_order(page_folio(page)) *after* put_page(page) had already dropped the reference, resulting in a use-after-free/put when that was the last reference on the page. Compute order before releasing the page. Introducing the fallback in 1. also requires the source page array handed to ->copy_to_ram() to be built differently. Both callers only populated the entry at the head of each source folio, relying on the copy callback to derive the rest of the folio from the order recorded in the matching drm_pagemap_addr. Once the destination has been demoted to order-0 folios the drm_pagemap_addr entries are per-page, so a source page is needed for every one of them; leaving them NULL makes the copy callback stop after the first page and the remainder of the range is never copied. The source folio is only split later, by migrate_vma_pages() / migrate_device_pages(), so its order cannot be used to detect the demotion - test the destination for MIGRATE_PFN_COMPOUND instead. Factor the array population out into drm_pagemap_migrate_populate_src_pages() and use it from both drm_pagemap_evict_to_ram() and __drm_pagemap_migrate_to_ram(). Fixes: ddeda6136038 ("drm/pagemap: Allocate folios when possible") Cc: stable@vger.kernel.org Cc: Andrew Morton Cc: David Hildenbrand Cc: Lorenzo Stoakes Cc: Zi Yan Cc: Baolin Wang Cc: Liam R. Howlett Cc: Nico Pache Cc: Ryan Roberts Cc: Dev Jain Cc: Barry Song Cc: Lance Yang Cc: Usama Arif Cc: Joshua Hahn Cc: Rakie Kim Cc: Byungchul Park Cc: Gregory Price Cc: Ying Huang Cc: Alistair Popple Cc: Balbir Singh Cc: Maarten Lankhorst Cc: Maxime Ripard Cc: Thomas Zimmermann Cc: David Airlie Cc: Simona Vetter Cc: Thomas Hellström Cc: Francois Dugast Cc: dri-devel@lists.freedesktop.org Cc: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org Cc: stable@vger.kernel.org Assisted-by: GitHub Copilot:claude-opus-5 Signed-off-by: Matthew Brost --- drivers/gpu/drm/drm_pagemap.c | 114 ++++++++++++++++++++++++++-------- 1 file changed, 89 insertions(+), 25 deletions(-) diff --git a/drivers/gpu/drm/drm_pagemap.c b/drivers/gpu/drm/drm_pagemap.c index 892b325fa99b..7610e233d238 100644 --- a/drivers/gpu/drm/drm_pagemap.c +++ b/drivers/gpu/drm/drm_pagemap.c @@ -383,6 +383,58 @@ drm_pagemap_migrate_map_system_pages(struct device *dev, return 0; } +/** + * drm_pagemap_migrate_populate_src_pages() - Populate the source page array + * @pages: Array of source pages to populate + * @src_mpfn: Source array of migrate PFNs + * @dst_mpfn: Destination array of migrate PFNs + * @npages: Number of pages in the arrays + * + * Populate @pages with the device pages the copy callback is to read from. + * + * Entries are normally only populated at the head of each source folio, with + * the copy callback deriving the rest of the folio from the order recorded in + * the corresponding drm_pagemap_addr. That does not work where + * drm_pagemap_migrate_populate_ram_pfn() had to demote a higher-order source + * folio to order-0 destination folios: the drm_pagemap_addr entries are then + * per-page, and the copy callback needs a source page for each of them. + * Populate every entry for those ranges. + * + * Note that the source folio itself is only split later, by + * migrate_vma_pages() / migrate_device_pages(), so its order cannot be used to + * detect the demotion - the destination has to be inspected instead. + */ +static void drm_pagemap_migrate_populate_src_pages(struct page **pages, + unsigned long *src_mpfn, + unsigned long *dst_mpfn, + unsigned long npages) +{ + unsigned long i; + + for (i = 0; i < npages;) { + struct page *page = migrate_pfn_to_page(src_mpfn[i]); + unsigned int order = 0; + unsigned long j, nr; + + if (!page) { + i++; + continue; + } + + order = folio_order(page_folio(page)); + nr = NR_PAGES(order); + + if (order && !(dst_mpfn[i] & MIGRATE_PFN_COMPOUND)) { + for (j = 0; j < nr && i + j < npages; j++) + pages[i + j] = folio_page(page_folio(page), j); + } else { + pages[i] = page; + } + + i += nr; + } +} + /** * drm_pagemap_migrate_unmap_pages() - Unmap pages previously mapped for GPU SVM migration * @dev: The device for which the pages were mapped @@ -875,6 +927,7 @@ static int drm_pagemap_migrate_populate_ram_pfn(struct vm_area_struct *vas, struct page *page = NULL, *src_page; struct folio *folio; unsigned int order = 0; + gfp_t gfp = GFP_HIGHUSER; if (!(src_mpfn[i] & MIGRATE_PFN_MIGRATE)) goto next; @@ -890,12 +943,38 @@ static int drm_pagemap_migrate_populate_ram_pfn(struct vm_area_struct *vas, } order = folio_order(page_folio(src_page)); + if (order) + gfp |= __GFP_NOWARN; - /* TODO: Support fallback to single pages if THP allocation fails */ if (vas) - folio = vma_alloc_folio(GFP_HIGHUSER, order, vas, addr); + folio = vma_alloc_folio(gfp, order, vas, addr); else - folio = folio_alloc(GFP_HIGHUSER, order); + folio = folio_alloc(gfp, order); + + if (!folio && order) { + /* + * Higher-order allocation failed, fall back to + * order-0 allocations for the entire range covered + * by the original higher-order allocation, without + * setting MIGRATE_PFN_COMPOUND, until we move past + * that range. + */ + unsigned long nr = NR_PAGES(order); + unsigned long j; + + gfp &= ~__GFP_NOWARN; + for (j = 0; j < nr && i < npages; j++, i++, addr += PAGE_SIZE) { + folio = vas ? + vma_alloc_folio(gfp, 0, vas, addr) : + folio_alloc(gfp, 0); + if (!folio) + goto free_pages; + + page = folio_page(folio, 0); + mpfn[i] = migrate_pfn(page_to_pfn(page)); + } + continue; + } if (!folio) goto free_pages; @@ -940,11 +1019,11 @@ static int drm_pagemap_migrate_populate_ram_pfn(struct vm_area_struct *vas, if (!page) goto next_put; + order = folio_order(page_folio(page)); + put_page(page); mpfn[i] = 0; - order = folio_order(page_folio(page)); - next_put: i += NR_PAGES(order); } @@ -1120,7 +1199,7 @@ int drm_pagemap_evict_to_ram(struct drm_pagemap_devmem *devmem_allocation) unsigned long *src, *dst; struct drm_pagemap_addr *pagemap_addr; void *buf; - int i, err = 0; + int err = 0; unsigned int retry_count = 2; npages = devmem_allocation->size >> PAGE_SHIFT; @@ -1160,15 +1239,7 @@ int drm_pagemap_evict_to_ram(struct drm_pagemap_devmem *devmem_allocation) if (err) goto err_finalize; - for (i = 0; i < npages;) { - unsigned int order = 0; - - pages[i] = migrate_pfn_to_page(src[i]); - if (pages[i]) - order = folio_order(page_folio(pages[i])); - - i += NR_PAGES(order); - } + drm_pagemap_migrate_populate_src_pages(pages, src, dst, npages); err = ops->copy_to_ram(pages, pagemap_addr, npages, NULL); if (err) @@ -1235,7 +1306,7 @@ static int __drm_pagemap_migrate_to_ram(struct vm_area_struct *vas, struct drm_pagemap_addr *pagemap_addr; unsigned long start, end; void *buf; - int i, err = 0; + int err = 0; zdd = drm_pagemap_page_zone_device_data(page); if (time_before64(get_jiffies_64(), zdd->devmem_allocation->timeslice_expiration)) @@ -1290,15 +1361,8 @@ static int __drm_pagemap_migrate_to_ram(struct vm_area_struct *vas, if (err) goto err_finalize; - for (i = 0; i < npages;) { - unsigned int order = 0; - - pages[i] = migrate_pfn_to_page(migrate.src[i]); - if (pages[i]) - order = folio_order(page_folio(pages[i])); - - i += NR_PAGES(order); - } + drm_pagemap_migrate_populate_src_pages(pages, migrate.src, migrate.dst, + npages); err = ops->copy_to_ram(pages, pagemap_addr, npages, NULL); if (err) -- 2.34.1