[PATCH] mm: drop lruvec->lru_lock if contended when skipping folio

[PATCH] mm: drop lruvec->lru_lock if contended when skipping folio

[Usama Arif]

From: Yu Zhao <yuzhao@google.com>

lruvec->lru_lock is highly contended and is held when calling
isolate_lru_folios. If the lru has a large number of CMA folios
consecutively, while the allocation type requested is not MIGRATE_MOVABLE,
isolate_lru_folios can hold the lock for a very long time while it
skips those. vmscan_lru_isolate tracepoint showed that skipped can go
above 70k in production.
This can cause lockups [1] and high memory pressure for extended periods of
time [2]. Hence release the lock if its contended when skipping a folio to
give other tasks a chance to acquire it and not stall.

[1] https://lore.kernel.org/all/CAOUHufbkhMZYz20aM_3rHZ3OcK4m2puji2FGpUpn_-DevGk3Kg@mail.gmail.com/
[2] https://lore.kernel.org/all/ZrssOrcJIDy8hacI@gmail.com/

Signed-off-by: Yu Zhao <yuzhao@google.com>
Signed-off-by: Usama Arif <usamaarif642@gmail.com>
---
 mm/vmscan.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 25f4e8403f41..4e817d78abbb 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1696,8 +1696,14 @@ static unsigned long isolate_lru_folios(unsigned long nr_to_scan,
 		if (folio_zonenum(folio) > sc->reclaim_idx ||
 				skip_cma(folio, sc)) {
 			nr_skipped[folio_zonenum(folio)] += nr_pages;
-			move_to = &folios_skipped;
-			goto move;
+			list_move(&folio->lru, &folios_skipped);
+			if (!spin_is_contended(&lruvec->lru_lock))
+				continue;
+			if (!list_empty(dst))
+				break;
+			spin_unlock_irq(&lruvec->lru_lock);
+			cond_resched();
+			spin_lock_irq(&lruvec->lru_lock);
 		}
 
 		/*
-- 
2.43.5

Re: [PATCH] mm: drop lruvec->lru_lock if contended when skipping folio

[Yu Zhao]

On Wed, Aug 14, 2024 at 8:26 AM Usama Arif <usamaarif642@gmail.com> wrote:
>
> From: Yu Zhao <yuzhao@google.com>

I think in case the patch should be "From" you, "Suggested-by" me, and
"Reported-by" and "Tested-by" Bharata.

Re: [PATCH] mm: drop lruvec->lru_lock if contended when skipping folio

[Usama Arif]

On 14/08/2024 17:01, Yu Zhao wrote:
> On Wed, Aug 14, 2024 at 8:26 AM Usama Arif <usamaarif642@gmail.com> wrote:
>>
>> From: Yu Zhao <yuzhao@google.com>
> 
> I think in case the patch should be "From" you, "Suggested-by" me, and
> "Reported-by" and "Tested-by" Bharata.

Sounds good, will change it in next revision. I will wait for a couple of days to see if there is feedback on the patch and resend with what you have suggested.

Thanks!