[PATCH v1 0/3] drm_buddy regresion fix and minor improvements

[PATCH v1 0/3] drm_buddy regresion fix and minor improvements

[Pierre-Eric Pelloux-Prayer]

Hi,

The first patch fixes a performance issue caused by a change in
drm_buddy ("drm/buddy: Implement tracking clear page feature").
It may be related to https://gitlab.freedesktop.org/drm/amd/-/issues/4260.

The other 2 patches are improvements based on my limited understanding
of the code and that seem to work as expected.


Pierre-Eric Pelloux-Prayer (3):
  drm/buddy: add a flag to disable trimming of non cleared blocks
  drm/buddy: use DRM_BUDDY_CLEAR_ALLOCATION as a hint, not a hard req
  drm/buddy: dont go over the higher orders multiple times

 drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 10 ++-
 drivers/gpu/drm/drm_buddy.c                  | 61 ++++++++++------
 drivers/gpu/drm/tests/drm_buddy_test.c       | 75 +++-----------------
 include/drm/drm_buddy.h                      |  3 +-
 4 files changed, 57 insertions(+), 92 deletions(-)

-- 
2.43.0

[PATCH v1 1/3] drm/buddy: add a flag to disable trimming of non cleared blocks

[Pierre-Eric Pelloux-Prayer]

A vkcts test case is triggering a case where the drm buddy allocator
wastes lots of memory and performs badly:

  dEQP-VK.memory.allocation.basic.size_8KiB.reverse.count_4000

For each memory pool type, the test will allocate 4000 8kB objects,
and then will release them. The alignment request is 256kB.

For each object, the allocator will select a 256kB block (to
match the alignment), and then trim it to 8kB, adding lots of free
entries to the free_lists of order 5 to 1.
On deallocation, none of these objects will be merged with their
buddy because their "clear status" is different: only the block
that was handed over to the driver might come back cleared.
Also since the test don't allocate much memory, the allocator don't
need to force the merge process so it will repeat the same logic
for each run.

As a result, after the first run (which takes about 6sec), the
freelists look like this:

   chunk_size: 4KiB, total: 16368MiB, free: 15354MiB, clear_free: 397MiB
   [...]
   order- 5 free:     1914 MiB, blocks: 15315
   order- 4 free:      957 MiB, blocks: 15325
   order- 3 free:      480 MiB, blocks: 15360
   order- 2 free:      239 MiB, blocks: 15347
   order- 1 free:      238 MiB, blocks: 30489

After the second run (19 sec):

   chunk_size: 4KiB, total: 16368MiB, free: 15374MiB, clear_free: 537MiB
   [...]
   order- 5 free:     3326 MiB, blocks: 26615
   order- 4 free:     1663 MiB, blocks: 26619
   order- 3 free:      833 MiB, blocks: 26659
   order- 2 free:      416 MiB, blocks: 26643
   order- 1 free:      414 MiB, blocks: 53071

list_insert_sorted is part of the problem here since it iterates
over the free_list to figure out where to insert the new blocks.

To fix this while keeping the clear tracking information, a new
bit is exposed to drivers, allowing them to disable trimming for
blocks that aren't "clear". This bit is used by amdgpu because
it always returns cleared memory to drm_buddy.

With this bit set, the "merge buddies on deallocation logic" can
work again, and the free_list are not growing indefinitely anymore.

So after a run we get:

   chunk_size: 4KiB, total: 16368MiB, free: 15306MiB, clear_free: 1734MiB
   [...]
   order- 5 free:        2 MiB, blocks: 17
   order- 4 free:        2 MiB, blocks: 35
   order- 3 free:        1 MiB, blocks: 41
   order- 2 free:      656 KiB, blocks: 41
   order- 1 free:      256 KiB, blocks: 32

The runtime is better (2 sec) and stable across multiple runs, and we
also see that the reported "clear_free" amount is larger than without
the patch.

Fixes: 96950929eb23 ("drm/buddy: Implement tracking clear page feature")
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 8 ++++++++
 drivers/gpu/drm/drm_buddy.c                  | 1 +
 include/drm/drm_buddy.h                      | 1 +
 3 files changed, 10 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
index abdc52b0895a..dbbaa15a973e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
@@ -499,6 +499,14 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
 
 	INIT_LIST_HEAD(&vres->blocks);
 
+	/* Trimming create smaller blocks that may never be given to the driver.
+	 * Such blocks won't be cleared until being seen by the driver, which might
+	 * never occur (for instance UMD might request large alignment) => in such
+	 * case, upon release of the block, the drm_buddy allocator won't merge them
+	 * back, because their clear status is different.
+	 */
+	vres->flags = DRM_BUDDY_TRIM_IF_CLEAR;
+
 	if (place->flags & TTM_PL_FLAG_TOPDOWN)
 		vres->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION;
 
diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c
index a1e652b7631d..555c72abce4c 100644
--- a/drivers/gpu/drm/drm_buddy.c
+++ b/drivers/gpu/drm/drm_buddy.c
@@ -1092,6 +1092,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm,
 
 	/* Trim the allocated block to the required size */
 	if (!(flags & DRM_BUDDY_TRIM_DISABLE) &&
+	    (!(flags & DRM_BUDDY_TRIM_IF_CLEAR) || drm_buddy_block_is_clear(block)) &&
 	    original_size != size) {
 		struct list_head *trim_list;
 		LIST_HEAD(temp);
diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h
index 9689a7c5dd36..c338d03028c3 100644
--- a/include/drm/drm_buddy.h
+++ b/include/drm/drm_buddy.h
@@ -28,6 +28,7 @@
 #define DRM_BUDDY_CLEAR_ALLOCATION		BIT(3)
 #define DRM_BUDDY_CLEARED			BIT(4)
 #define DRM_BUDDY_TRIM_DISABLE			BIT(5)
+#define DRM_BUDDY_TRIM_IF_CLEAR			BIT(6)
 
 struct drm_buddy_block {
 #define DRM_BUDDY_HEADER_OFFSET GENMASK_ULL(63, 12)
-- 
2.43.0

[PATCH v1 2/3] drm/buddy: use DRM_BUDDY_CLEAR_ALLOCATION as a hint, not a hard req

[Pierre-Eric Pelloux-Prayer]

The rationale for this change is that it's preferable to return
non-cleared memory instead of splitting up higher-order blocks as
this leads to more fragmented memory.

The driver will be able to clear the memory by itself if required
and the clear tracking will avoid the need for useless clearing jobs.

This commit renames DRM_BUDDY_CLEAR_ALLOCATION as
DRM_BUDDY_PREFER_CLEAR_ALLOCATION to make the intent clearer, and
delete the tests that expected that passing this flag would return
cleared memory.

Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c |  2 +-
 drivers/gpu/drm/drm_buddy.c                  | 43 ++++++-----
 drivers/gpu/drm/tests/drm_buddy_test.c       | 75 +++-----------------
 include/drm/drm_buddy.h                      |  2 +-
 4 files changed, 35 insertions(+), 87 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
index dbbaa15a973e..24dd094eac84 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
@@ -514,7 +514,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
 		vres->flags |= DRM_BUDDY_CONTIGUOUS_ALLOCATION;
 
 	if (bo->flags & AMDGPU_GEM_CREATE_VRAM_CLEARED)
-		vres->flags |= DRM_BUDDY_CLEAR_ALLOCATION;
+		vres->flags |= DRM_BUDDY_PREFER_CLEAR_ALLOCATION;
 
 	if (fpfn || lpfn != mgr->mm.size)
 		/* Allocate blocks in desired range */
diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c
index 555c72abce4c..fd31322b3d41 100644
--- a/drivers/gpu/drm/drm_buddy.c
+++ b/drivers/gpu/drm/drm_buddy.c
@@ -473,7 +473,7 @@ EXPORT_SYMBOL(drm_buddy_free_list);
 
 static bool block_incompatible(struct drm_buddy_block *block, unsigned int flags)
 {
-	bool needs_clear = flags & DRM_BUDDY_CLEAR_ALLOCATION;
+	bool needs_clear = flags & DRM_BUDDY_PREFER_CLEAR_ALLOCATION;
 
 	return needs_clear != drm_buddy_block_is_clear(block);
 }
@@ -593,21 +593,30 @@ get_maxblock(struct drm_buddy *mm, unsigned int order,
 	     unsigned long flags)
 {
 	struct drm_buddy_block *max_block = NULL, *block = NULL;
+	bool wants_clear;
 	unsigned int i;
 
 	for (i = order; i <= mm->max_order; ++i) {
 		struct drm_buddy_block *tmp_block;
 
+		wants_clear = flags & DRM_BUDDY_PREFER_CLEAR_ALLOCATION;
+
+retry:
 		list_for_each_entry_reverse(tmp_block, &mm->free_list[i], link) {
-			if (block_incompatible(tmp_block, flags))
+			if (wants_clear && !drm_buddy_block_is_clear(tmp_block))
 				continue;
 
 			block = tmp_block;
 			break;
 		}
 
-		if (!block)
+		if (!block) {
+			if (wants_clear) {
+				wants_clear = false;
+				goto retry;
+			}
 			continue;
+		}
 
 		if (!max_block) {
 			max_block = block;
@@ -630,6 +639,7 @@ alloc_from_freelist(struct drm_buddy *mm,
 {
 	struct drm_buddy_block *block = NULL;
 	unsigned int tmp;
+	bool wants_clear;
 	int err;
 
 	if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) {
@@ -640,9 +650,11 @@ alloc_from_freelist(struct drm_buddy *mm,
 	} else {
 		for (tmp = order; tmp <= mm->max_order; ++tmp) {
 			struct drm_buddy_block *tmp_block;
+			wants_clear = flags & DRM_BUDDY_PREFER_CLEAR_ALLOCATION;
 
+retry:
 			list_for_each_entry_reverse(tmp_block, &mm->free_list[tmp], link) {
-				if (block_incompatible(tmp_block, flags))
+				if (wants_clear && !drm_buddy_block_is_clear(tmp_block))
 					continue;
 
 				block = tmp_block;
@@ -651,25 +663,20 @@ alloc_from_freelist(struct drm_buddy *mm,
 
 			if (block)
 				break;
-		}
-	}
 
-	if (!block) {
-		/* Fallback method */
-		for (tmp = order; tmp <= mm->max_order; ++tmp) {
-			if (!list_empty(&mm->free_list[tmp])) {
-				block = list_last_entry(&mm->free_list[tmp],
-							struct drm_buddy_block,
-							link);
-				if (block)
-					break;
+			if (wants_clear) {
+				/* Relax this requirement to avoid splitting up higher order
+				 * blocks.
+				 */
+				wants_clear = false;
+				goto retry;
 			}
 		}
-
-		if (!block)
-			return ERR_PTR(-ENOSPC);
 	}
 
+	if (!block)
+		return ERR_PTR(-ENOSPC);
+
 	BUG_ON(!drm_buddy_block_is_free(block));
 
 	while (tmp != order) {
diff --git a/drivers/gpu/drm/tests/drm_buddy_test.c b/drivers/gpu/drm/tests/drm_buddy_test.c
index 7a0e523651f0..7ae65d93adb0 100644
--- a/drivers/gpu/drm/tests/drm_buddy_test.c
+++ b/drivers/gpu/drm/tests/drm_buddy_test.c
@@ -240,7 +240,7 @@ static void drm_test_buddy_alloc_range_bias(struct kunit *test)
 	bias_end = max(bias_end, bias_start + ps);
 	bias_rem = bias_end - bias_start;
 
-	flags = DRM_BUDDY_CLEAR_ALLOCATION | DRM_BUDDY_RANGE_ALLOCATION;
+	flags = DRM_BUDDY_PREFER_CLEAR_ALLOCATION | DRM_BUDDY_RANGE_ALLOCATION;
 	size = max(round_up(prandom_u32_state(&prng) % bias_rem, ps), ps);
 
 	KUNIT_ASSERT_FALSE_MSG(test,
@@ -272,67 +272,9 @@ static void drm_test_buddy_alloc_clear(struct kunit *test)
 	LIST_HEAD(clean);
 
 	mm_size = SZ_4K << max_order;
-	KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps));
-
-	KUNIT_EXPECT_EQ(test, mm.max_order, max_order);
-
-	/*
-	 * Idea is to allocate and free some random portion of the address space,
-	 * returning those pages as non-dirty and randomly alternate between
-	 * requesting dirty and non-dirty pages (not going over the limit
-	 * we freed as non-dirty), putting that into two separate lists.
-	 * Loop over both lists at the end checking that the dirty list
-	 * is indeed all dirty pages and vice versa. Free it all again,
-	 * keeping the dirty/clear status.
-	 */
-	KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
-							    5 * ps, ps, &allocated,
-							    DRM_BUDDY_TOPDOWN_ALLOCATION),
-				"buddy_alloc hit an error size=%lu\n", 5 * ps);
-	drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED);
-
-	n_pages = 10;
-	do {
-		unsigned long flags;
-		struct list_head *list;
-		int slot = i % 2;
-
-		if (slot == 0) {
-			list = &dirty;
-			flags = 0;
-		} else {
-			list = &clean;
-			flags = DRM_BUDDY_CLEAR_ALLOCATION;
-		}
-
-		KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
-								    ps, ps, list,
-								    flags),
-					"buddy_alloc hit an error size=%lu\n", ps);
-	} while (++i < n_pages);
-
-	list_for_each_entry(block, &clean, link)
-		KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), true);
-
-	list_for_each_entry(block, &dirty, link)
-		KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false);
-
-	drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED);
-
-	/*
-	 * Trying to go over the clear limit for some allocation.
-	 * The allocation should never fail with reasonable page-size.
-	 */
-	KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
-							    10 * ps, ps, &clean,
-							    DRM_BUDDY_CLEAR_ALLOCATION),
-				"buddy_alloc hit an error size=%lu\n", 10 * ps);
-
-	drm_buddy_free_list(&mm, &clean, DRM_BUDDY_CLEARED);
-	drm_buddy_free_list(&mm, &dirty, 0);
-	drm_buddy_fini(&mm);
 
 	KUNIT_EXPECT_FALSE(test, drm_buddy_init(&mm, mm_size, ps));
+	KUNIT_EXPECT_EQ(test, mm.max_order, max_order);
 
 	/*
 	 * Create a new mm. Intentionally fragment the address space by creating
@@ -366,14 +308,13 @@ static void drm_test_buddy_alloc_clear(struct kunit *test)
 	do {
 		size = SZ_4K << order;
 
-		KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
-								    size, size, &allocated,
-								    DRM_BUDDY_CLEAR_ALLOCATION),
-					"buddy_alloc hit an error size=%u\n", size);
+		KUNIT_ASSERT_FALSE_MSG(
+			test, drm_buddy_alloc_blocks(&mm, 0, mm_size,
+						     size, size, &allocated,
+						     DRM_BUDDY_PREFER_CLEAR_ALLOCATION),
+			"buddy_alloc hit an error size=%u\n", size);
 		total = 0;
 		list_for_each_entry(block, &allocated, link) {
-			if (size != mm_size)
-				KUNIT_EXPECT_EQ(test, drm_buddy_block_is_clear(block), false);
 			total += drm_buddy_block_size(&mm, block);
 		}
 		KUNIT_EXPECT_EQ(test, total, size);
@@ -399,7 +340,7 @@ static void drm_test_buddy_alloc_clear(struct kunit *test)
 	drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED);
 	KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, 0, SZ_4K << max_order,
 							    2 * ps, ps, &allocated,
-							    DRM_BUDDY_CLEAR_ALLOCATION),
+							    DRM_BUDDY_PREFER_CLEAR_ALLOCATION),
 				"buddy_alloc hit an error size=%lu\n", 2 * ps);
 	drm_buddy_free_list(&mm, &allocated, DRM_BUDDY_CLEARED);
 	KUNIT_ASSERT_FALSE_MSG(test, drm_buddy_alloc_blocks(&mm, SZ_4K << max_order, mm_size,
diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h
index c338d03028c3..ed06be63a770 100644
--- a/include/drm/drm_buddy.h
+++ b/include/drm/drm_buddy.h
@@ -25,7 +25,7 @@
 #define DRM_BUDDY_RANGE_ALLOCATION		BIT(0)
 #define DRM_BUDDY_TOPDOWN_ALLOCATION		BIT(1)
 #define DRM_BUDDY_CONTIGUOUS_ALLOCATION		BIT(2)
-#define DRM_BUDDY_CLEAR_ALLOCATION		BIT(3)
+#define DRM_BUDDY_PREFER_CLEAR_ALLOCATION	BIT(3)
 #define DRM_BUDDY_CLEARED			BIT(4)
 #define DRM_BUDDY_TRIM_DISABLE			BIT(5)
 #define DRM_BUDDY_TRIM_IF_CLEAR			BIT(6)
-- 
2.43.0

[PATCH v1 3/3] drm/buddy: dont go over the higher orders multiple times

[Pierre-Eric Pelloux-Prayer]

AFAICT the rationale for the loop is to:
1) try to allocate from the preferred order
2) if it fails, try higher orders (order + 1 -> max order)
3) if it fails, try smaller orders (order - 1 -> min order)

Steps 1 and 2 are covered by the loop going through [order, max_order].
Currently step 3 tries again [order, max_order] but with decreasing
values of order.

This is wasteful, so change it to evaluate only order.

Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
---
 drivers/gpu/drm/drm_buddy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c
index fd31322b3d41..9d3723f2cff9 100644
--- a/drivers/gpu/drm/drm_buddy.c
+++ b/drivers/gpu/drm/drm_buddy.c
@@ -590,13 +590,14 @@ __drm_buddy_alloc_range_bias(struct drm_buddy *mm,
 
 static struct drm_buddy_block *
 get_maxblock(struct drm_buddy *mm, unsigned int order,
+	     unsigned int max_order,
 	     unsigned long flags)
 {
 	struct drm_buddy_block *max_block = NULL, *block = NULL;
 	bool wants_clear;
 	unsigned int i;
 
-	for (i = order; i <= mm->max_order; ++i) {
+	for (i = order; i <= max_order; ++i) {
 		struct drm_buddy_block *tmp_block;
 
 		wants_clear = flags & DRM_BUDDY_PREFER_CLEAR_ALLOCATION;
@@ -635,6 +636,7 @@ get_maxblock(struct drm_buddy *mm, unsigned int order,
 static struct drm_buddy_block *
 alloc_from_freelist(struct drm_buddy *mm,
 		    unsigned int order,
+		    unsigned int max_order,
 		    unsigned long flags)
 {
 	struct drm_buddy_block *block = NULL;
@@ -643,12 +645,12 @@ alloc_from_freelist(struct drm_buddy *mm,
 	int err;
 
 	if (flags & DRM_BUDDY_TOPDOWN_ALLOCATION) {
-		block = get_maxblock(mm, order, flags);
+		block = get_maxblock(mm, order, max_order, flags);
 		if (block)
 			/* Store the obtained block order */
 			tmp = drm_buddy_block_order(block);
 	} else {
-		for (tmp = order; tmp <= mm->max_order; ++tmp) {
+		for (tmp = order; tmp <= max_order; ++tmp) {
 			struct drm_buddy_block *tmp_block;
 			wants_clear = flags & DRM_BUDDY_PREFER_CLEAR_ALLOCATION;
 
@@ -956,6 +958,7 @@ static struct drm_buddy_block *
 __drm_buddy_alloc_blocks(struct drm_buddy *mm,
 			 u64 start, u64 end,
 			 unsigned int order,
+			 unsigned int max_order,
 			 unsigned long flags)
 {
 	if (flags & DRM_BUDDY_RANGE_ALLOCATION)
@@ -964,7 +967,7 @@ __drm_buddy_alloc_blocks(struct drm_buddy *mm,
 						     order, flags);
 	else
 		/* Allocate from freelist */
-		return alloc_from_freelist(mm, order, flags);
+		return alloc_from_freelist(mm, order, max_order, flags);
 }
 
 /**
@@ -995,7 +998,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm,
 {
 	struct drm_buddy_block *block = NULL;
 	u64 original_size, original_min_size;
-	unsigned int min_order, order;
+	unsigned int min_order, max_order, order;
 	LIST_HEAD(allocated);
 	unsigned long pages;
 	int err;
@@ -1044,6 +1047,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm,
 
 	do {
 		order = min(order, (unsigned int)fls(pages) - 1);
+		max_order = mm->max_order;
 		BUG_ON(order > mm->max_order);
 		BUG_ON(order < min_order);
 
@@ -1051,6 +1055,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm,
 			block = __drm_buddy_alloc_blocks(mm, start,
 							 end,
 							 order,
+							 max_order,
 							 flags);
 			if (!IS_ERR(block))
 				break;
@@ -1062,6 +1067,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm,
 					block = __drm_buddy_alloc_blocks(mm, start,
 									 end,
 									 min_order,
+									 mm->max_order,
 									 flags);
 					if (!IS_ERR(block)) {
 						order = min_order;
@@ -1082,6 +1088,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm,
 				err = -ENOSPC;
 				goto err_free;
 			}
+			max_order = order;
 		} while (1);
 
 		mark_allocated(block);
-- 
2.43.0

Re: [PATCH v1 1/3] drm/buddy: add a flag to disable trimming of non cleared blocks

[Christian König]

On 02.07.25 18:12, Pierre-Eric Pelloux-Prayer wrote:
> A vkcts test case is triggering a case where the drm buddy allocator
> wastes lots of memory and performs badly:
> 
>   dEQP-VK.memory.allocation.basic.size_8KiB.reverse.count_4000
> 
> For each memory pool type, the test will allocate 4000 8kB objects,
> and then will release them. The alignment request is 256kB.
> 
> For each object, the allocator will select a 256kB block (to
> match the alignment), and then trim it to 8kB, adding lots of free
> entries to the free_lists of order 5 to 1.
> On deallocation, none of these objects will be merged with their
> buddy because their "clear status" is different: only the block
> that was handed over to the driver might come back cleared.
> Also since the test don't allocate much memory, the allocator don't
> need to force the merge process so it will repeat the same logic
> for each run.
> 
> As a result, after the first run (which takes about 6sec), the
> freelists look like this:
> 
>    chunk_size: 4KiB, total: 16368MiB, free: 15354MiB, clear_free: 397MiB
>    [...]
>    order- 5 free:     1914 MiB, blocks: 15315
>    order- 4 free:      957 MiB, blocks: 15325
>    order- 3 free:      480 MiB, blocks: 15360
>    order- 2 free:      239 MiB, blocks: 15347
>    order- 1 free:      238 MiB, blocks: 30489
> 
> After the second run (19 sec):
> 
>    chunk_size: 4KiB, total: 16368MiB, free: 15374MiB, clear_free: 537MiB
>    [...]
>    order- 5 free:     3326 MiB, blocks: 26615
>    order- 4 free:     1663 MiB, blocks: 26619
>    order- 3 free:      833 MiB, blocks: 26659
>    order- 2 free:      416 MiB, blocks: 26643
>    order- 1 free:      414 MiB, blocks: 53071
> 
> list_insert_sorted is part of the problem here since it iterates
> over the free_list to figure out where to insert the new blocks.
> 
> To fix this while keeping the clear tracking information, a new
> bit is exposed to drivers, allowing them to disable trimming for
> blocks that aren't "clear". This bit is used by amdgpu because
> it always returns cleared memory to drm_buddy.

That's an extremely good catch, but I don't think this is the right solution.

If I understood it correctly we now give back 256k instead of 8k to the caller and that is a really big no-go.

Regards,
Christian.


> 
> With this bit set, the "merge buddies on deallocation logic" can
> work again, and the free_list are not growing indefinitely anymore.
> 
> So after a run we get:
> 
>    chunk_size: 4KiB, total: 16368MiB, free: 15306MiB, clear_free: 1734MiB
>    [...]
>    order- 5 free:        2 MiB, blocks: 17
>    order- 4 free:        2 MiB, blocks: 35
>    order- 3 free:        1 MiB, blocks: 41
>    order- 2 free:      656 KiB, blocks: 41
>    order- 1 free:      256 KiB, blocks: 32
> 
> The runtime is better (2 sec) and stable across multiple runs, and we
> also see that the reported "clear_free" amount is larger than without
> the patch.
> 
> Fixes: 96950929eb23 ("drm/buddy: Implement tracking clear page feature")
> Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 8 ++++++++
>  drivers/gpu/drm/drm_buddy.c                  | 1 +
>  include/drm/drm_buddy.h                      | 1 +
>  3 files changed, 10 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
> index abdc52b0895a..dbbaa15a973e 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
> @@ -499,6 +499,14 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man,
>  
>  	INIT_LIST_HEAD(&vres->blocks);
>  
> +	/* Trimming create smaller blocks that may never be given to the driver.
> +	 * Such blocks won't be cleared until being seen by the driver, which might
> +	 * never occur (for instance UMD might request large alignment) => in such
> +	 * case, upon release of the block, the drm_buddy allocator won't merge them
> +	 * back, because their clear status is different.
> +	 */
> +	vres->flags = DRM_BUDDY_TRIM_IF_CLEAR;
> +
>  	if (place->flags & TTM_PL_FLAG_TOPDOWN)
>  		vres->flags |= DRM_BUDDY_TOPDOWN_ALLOCATION;
>  
> diff --git a/drivers/gpu/drm/drm_buddy.c b/drivers/gpu/drm/drm_buddy.c
> index a1e652b7631d..555c72abce4c 100644
> --- a/drivers/gpu/drm/drm_buddy.c
> +++ b/drivers/gpu/drm/drm_buddy.c
> @@ -1092,6 +1092,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm,
>  
>  	/* Trim the allocated block to the required size */
>  	if (!(flags & DRM_BUDDY_TRIM_DISABLE) &&
> +	    (!(flags & DRM_BUDDY_TRIM_IF_CLEAR) || drm_buddy_block_is_clear(block)) &&
>  	    original_size != size) {
>  		struct list_head *trim_list;
>  		LIST_HEAD(temp);
> diff --git a/include/drm/drm_buddy.h b/include/drm/drm_buddy.h
> index 9689a7c5dd36..c338d03028c3 100644
> --- a/include/drm/drm_buddy.h
> +++ b/include/drm/drm_buddy.h
> @@ -28,6 +28,7 @@
>  #define DRM_BUDDY_CLEAR_ALLOCATION		BIT(3)
>  #define DRM_BUDDY_CLEARED			BIT(4)
>  #define DRM_BUDDY_TRIM_DISABLE			BIT(5)
> +#define DRM_BUDDY_TRIM_IF_CLEAR			BIT(6)
>  
>  struct drm_buddy_block {
>  #define DRM_BUDDY_HEADER_OFFSET GENMASK_ULL(63, 12)