From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A5A043A4F32 for ; Sat, 28 Mar 2026 17:30:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774719049; cv=none; b=UNs5keWT+6msq3/RTX+NAeP3TD6UUpKNvnc9Nb+UAfcQx+UDJKAT1X6Gw0mYEPCKVjb1JrCFwjhSt7xxyXX817HwFvzY63HhpkMTLNyDELTJq6ul/TjD3d2n7FZy7HhVcf2HKxjjaPTFpWK2hd5H3Fyr1WsIqePHCpECj24aJSA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774719049; c=relaxed/simple; bh=iuYapQ4kBsV++ECYkF2zvMVWkp9lBJ5qt/55dZMMHUw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=TpkG/BrdW9x0pUCzhY4IWLS5n3yHl6qd/k0XFVGV6i+OXqdfJ/HytSOEnN4GPQyEogCAkxs70Y+gUHrXfq+Az54Jik7xXyAIa6xOzJCbN8le9DOD4plsTDdY3KL3Ol6qi1YX32kzHQAIEj+X2WaF40illoirBFAJsvVGsKFn/FI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gzxXTDRp; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gzxXTDRp" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-82c4b5dfe6cso1508976b3a.2 for ; Sat, 28 Mar 2026 10:30:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1774719047; x=1775323847; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=U6wpH0E/1pqdSBKibo+awwee7+Dq2N19KcOYS5YYS6Q=; b=gzxXTDRpwOaA4jvvSbZfdNtMN2TEHFL7j0dMCwBhKgzzABPm4ju4Dsf/AGq/4EQzGa eJh6iSL/nPElTo90gy2uQ5rUOstzuBhwKfbjTRupQh1OyhxQWOiauMb4f22879rRCaZ2 CvZNNA5eN7/xEwOvmKNCQQRT7n/2oYuoIbNeoG/xpSxdGt6iObJ6yRf3qae68IbcPn+q pIdEu0W+bRY+nTBox0gDrfIKOqCqgaIuNdO4Q84fFmU1JsXZ7oK8Y7BhLp/B2IslcP6+ vY9jwsdjB+ZAOkrkGdN9JQCwJQ7B3UeJ1yXubv95KwpQBZbjDY/zO7ZrU3sBvYx67bmQ 7Lmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774719047; x=1775323847; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=U6wpH0E/1pqdSBKibo+awwee7+Dq2N19KcOYS5YYS6Q=; b=qticCDOzXp+WgU6pwYdTrBAx8ls136wPunSZ7Acs5YVJoZXuF9Vx8fmeo8ZQWPY7Rk kWmBHl3BFNCFOv714qwQxZV7nUXpKA41ye05NX670SvUJDRjgmDBREx5yffmY+xXzpfm BurwsOqRyhc805URonS4SqSeJP5+qXCeY1enPBd00lPeMIUh1phIaHbTWkfwrWJbRPfW eEvtrx007DvktCJpXzRKkL2RPTJRdP8jj1pjuEVcNIQKQqidUpeFhr2sWyKgEuhNGyws FA6amGFrMzUsEd7aQ/zzEkKwEdnVOHISFttN4EdFfMngxQcYPEkYWDB4clRYtwhyZx88 FXHA== X-Forwarded-Encrypted: i=1; AJvYcCUGpKrVWQCG4IaBe9UEUbo+/sfHm9d+bTeKFMR7oIE4xn+ioNk7RWlQyO9/b2FofyAGAZkTXWUBfsao0xY=@vger.kernel.org X-Gm-Message-State: AOJu0YyVV0YlC0sr3P8GR3FZ0mvHtmY8GeW8IK56kq6FFl2pHi0NEMG6 2k7T/4Eju5w1hhEgFln6ZP1rHCkEZaMuQl1UwZgabBTFU9MwXNNy5W/Q X-Gm-Gg: ATEYQzwPLsXr7ye8UzxStn+TCggVGfVX1LLm6FqAWfsWfZwIUK12b/sF2Sv8odU8xrZ D20DbAbRDACQuz10w/se34zUZTFCM5X0CBl+vtEMjgiSeAcTt0sAog0b6t0OZycz53rn4HKNWdY QCt5useD3lTP2ioCzu4V9/QlZqGc1+JGy8E7WkU8Qd2vjz9Agp8eTNKcv6aXv8RS312b2V7GrAz sazEwr4oUHhfv3zrBUcjh/mHP8DEOxYCAMyox3FlYsw17kiJDfP1ndRQXnX6n3WGNhZzgWKJM9X JYYtpUNobeRZCl0KRCYMAGo3WGPM5isMyVJNSpYOFJDJiFexRXJi8yzMu3ho7FCtlRKqpmdki5Y qXiwHYbJfB5km14U0vy15/w283et1xVzERUv+Bom9/Nue1QS5V+2McqlQEYURC5ziiLqg7iDLUX cy5RvcLxmb0J3d+LtRnxNxdptVNu3QVPNLZgvYamUzQqcP5l4nO5KCaUVNj4c= X-Received: by 2002:a05:6a00:1c91:b0:82a:ea3:c172 with SMTP id d2e1a72fcca58-82c960a3070mr6069835b3a.46.1774719046762; Sat, 28 Mar 2026 10:30:46 -0700 (PDT) Received: from KASONG-MC4 ([101.32.222.185]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-82ca85ef1bfsm2388218b3a.44.2026.03.28.10.30.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 28 Mar 2026 10:30:45 -0700 (PDT) Date: Sun, 29 Mar 2026 01:30:38 +0800 From: Kairui Song To: linux-mm@kvack.org Cc: Eric Naim , Andrew Morton , Axel Rasmussen , Yuanchu Xie , Wei Xu , Johannes Weiner , David Hildenbrand , Michal Hocko , Qi Zheng , Shakeel Butt , Lorenzo Stoakes , Barry Song , David Stevens , Chen Ridong , Leno Hou , Yafang Shao , Yu Zhao , Zicheng Wang , Kalesh Singh , Suren Baghdasaryan , Chris Li , Vernon Yang , linux-kernel@vger.kernel.org Subject: Re: [PATCH 0/8] mm/mglru: improve reclaim loop and dirty folio handling Message-ID: References: <20260318-mglru-reclaim-v1-0-2c46f9eb0508@tencent.com> <7ab8edd7-381f-4db2-9560-b58718669208@cachyos.org> <85b4be3c-09a3-4a28-924d-71a20db3fd62@cachyos.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: On Wed, Mar 25, 2026 at 05:47:41PM +0800, Kairui Song wrote: > On Wed, Mar 25, 2026 at 5:27 PM Eric Naim wrote: > > > > On 3/25/26 1:47 PM, Kairui Song wrote: > > > On Wed, Mar 25, 2026 at 1:04 PM Eric Naim wrote: > > >> > > >> Hi Kairui, > > >> > > >> On 3/18/26 3:08 AM, Kairui Song via B4 Relay wrote: > > >>> This series cleans up and slightly improves MGLRU's reclaim loop and > > >>> dirty flush logic. As a result, we can see an up to ~50% reduce of file > > >>> faults and 30% increase in MongoDB throughput with YCSB and no swap > > >>> involved, other common benchmarks have no regression, and LOC is > > >>> reduced, with less unexpected OOM in our production environment. > > >>> > > > > > > ... > > > > > >> > > >> I applied this patch set to 7.0-rc5 and noticed the system locking up when performing the below test. > > >> > > >> fallocate -l 5G 5G > > >> while true; do tail /dev/zero; done > > >> while true; do time cat 5G > /dev/null; sleep $(($(cat /sys/kernel/mm/lru_gen/min_ttl_ms)/1000+1)); done > > >> > > >> After reading [1], I suspect that this was because the system was using zram as swap, and yes if zram is disabled then the lock up does not occur. > > > > > > Hi Eric, > > > > > > Thanks for the report, I was about to send V2 but noticing your report > > > I'll try to reproduce your issue first. > > > > > > So far I didn't notice any regression, is this an issue caused by this > > > patch or is it an existing issue? I don't have any context about how > > > you are doing the test. BTW the calculation in patch "mm/mglru: > > > restructure the reclaim loop" needs to have a lowest bar > > > "max(nr_to_scan, SWAP_CLUSTER_MAX)" for small machines, not sure if > > > related but will add to V2. > > > > > > > As of writing this, I got some new information that makes this a bit more confusing. The kernel that doesn't have the issue was patched with [1] as a means of protecting the working set (similar to lru_gen_min_ttl_ms). > > > > So this time on an unpatched kernel, the system still freezes but quickly recovers itself after about 2 seconds. With this patchset applied, the system freezes but it doesn't quickly recover (if at all). > > > > Curiously, I had the user test again but this time with lru_gen_min_ttl_ms = 100. With this set, the system doesn't freeze at all with or without this patchset. > > Ah thanks, that makes sense now, the downstream patch you mentioned > limits the reclaim of file pages to avoid thrashing, and your test > cases exhaust the memory on purpose which forces the kernel to reclaim > all reclaimable folios including page cache. > > A thrashing page cache causes desktop hangs easily, using TTL is an > effective way to avoid thrashing and trigger OOM early. That's why the > problem is gone with lru_gen_min_ttl_ms = 100 or le9. > > > > And about the test you posted: > > > while true; do tail /dev/zero; done > > > > > > I believe this will just consume all memory with zero pages and then > > > get OOM killed, that's exactly what the test is meant to do. By lockup > > > I'm not sure you mean since you mentioned OOM kill. The system > > > actually hung or the desktop is dead? > > > > The system actually hung. They needed a hard reset to recover the system. (pure speculation: given a few minutes the system would likely recover itself as this seems to be a common scenario) > > Yeah I believe so. > > Thrashing prevention is why MGLRU's TTL is introduced, so I do suggest > using that. It can be further improved too. > > Will keep that in mind and try to make some test cases to cover your > case too and make some adjustments. > > BTW how does the kernel behave with MGLRU disabled for your case? Hi all, I tested it multiple times on my Fedora, comparing MGLRU to classic LRU (using v2 of this series also also includes some minor improvements). I modified the reproduce a bit just to test the OOM behavior: - Running following command in console A: fallocate -l 5G 5G while true; do time cat 5G > /dev/null; done - Then run following command in console B: while true; do tail /dev/zero; done The console A output is below: With MGLRU disabled: ... real 0m4.925s user 0m0.016s sys 0m4.904s # Under pressure real 0m5.544s user 0m0.015s sys 0m5.521s real 0m5.444s user 0m0.012s sys 0m5.425s real 0m7.607s user 0m0.016s sys 0m7.561s real 0m7.268s user 0m0.017s sys 0m7.240s real 0m6.686s user 0m0.016s sys 0m6.656s real 0m9.919s user 0m0.014s sys 0m9.831s # <- OOM in B triggers real 0m4.559s user 0m0.012s sys 0m4.539s real 0m1.381s user 0m0.009s sys 0m1.362s real 0m11.816s user 0m0.010s sys 0m11.795s real 0m6.797s user 0m0.021s sys 0m6.753s real 0m0.944s user 0m0.013s sys 0m0.931s # <- OOM kill in B ends real 0m0.285s user 0m0.013s sys 0m0.272s MGLRU enabled, before this series: ... real 0m0.355s user 0m0.009s sys 0m0.346s # Under pressure real 0m0.352s user 0m0.008s sys 0m0.344s real 0m0.549s user 0m0.014s sys 0m0.535s real 0m0.628s user 0m0.009s sys 0m0.619s real 0m0.651s user 0m0.009s sys 0m0.642s real 0m5.294s user 0m0.010s sys 0m5.280s # <- OOM in B triggers real 0m1.041s user 0m0.014s sys 0m1.026s real 0m0.837s user 0m0.011s sys 0m0.826s real 0m2.450s user 0m0.013s sys 0m2.435s real 0m2.499s user 0m0.012s sys 0m2.485s real 0m1.857s user 0m0.015s sys 0m1.841s real 0m0.512s user 0m0.015s sys 0m0.497s real 0m0.418s user 0m0.011s sys 0m0.407s # <- OOM kill in B ends real 0m0.282s user 0m0.010s sys 0m0.272s MGLRU enabled, after this series: ... real 0m0.280s user 0m0.015s sys 0m0.265s # Under pressure real 0m0.283s user 0m0.010s sys 0m0.273s real 0m0.278s user 0m0.012s sys 0m0.266s real 0m0.315s user 0m0.018s sys 0m0.297s real 0m0.679s user 0m0.014s sys 0m0.663s real 0m0.716s user 0m0.011s sys 0m0.705s real 0m0.657s user 0m0.009s sys 0m0.648s real 0m6.615s user 0m0.007s sys 0m6.453s # <- OOM in B triggers real 0m1.244s user 0m0.018s sys 0m1.226s real 0m1.290s user 0m0.014s sys 0m1.276s real 0m1.119s user 0m0.011s sys 0m1.108s real 0m0.882s user 0m0.010s sys 0m0.872s real 0m0.855s user 0m0.007s sys 0m0.848s real 0m0.933s user 0m0.005s sys 0m0.928s real 0m0.833s user 0m0.009s sys 0m0.823s real 0m0.279s user 0m0.012s sys 0m0.267s # <- OOM killed in B real 0m0.273s user 0m0.010s sys 0m0.263s It seems with MGLRU enabled, both performance and OOM jitter seem better. As for this series, it now has no significant effect or slightly changed the jitter pattern, which I can't say is better or worse. The peak latency seems slightly higher, but the system seems to recover faster. Or maybe that's just noise. The OOM behavior is not really perfect in any case, but with MGLRU's TTL enabled, I got confirmation that the jitter is gone completely (only a few frames).