From: Shakeel Butt <shakeel.butt@linux.dev>
To: Tejun Heo <tj@kernel.org>, Andrew Morton <akpm@linux-foundation.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>,
Michal Koutny <mkoutny@suse.com>,
Christian Brauner <brauner@kernel.org>,
Michal Hocko <mhocko@kernel.org>,
Roman Gushchin <roman.gushchin@linux.dev>,
Muchun Song <muchun.song@linux.dev>, Jan Kara <jack@suse.cz>,
Matthew Wilcox <willy@infradead.org>,
David Hildenbrand <david@kernel.org>,
Vlastimil Babka <vbabka@kernel.org>, Zi Yan <ziy@nvidia.com>,
Baolin Wang <baolin.wang@linux.alibaba.com>,
Nico Pache <nico.pache@linux.dev>,
Usama Arif <usama.arif@linux.dev>,
Kiryl Shutsemau <kas@kernel.org>,
Lorenzo Stoakes <ljs@kernel.org>,
Suren Baghdasaryan <surenb@google.com>,
Meta kernel team <kernel-team@meta.com>,
cgroups@vger.kernel.org, linux-mm@kvack.org,
linux-fsdevel@vger.kernel.org, linux-btrfs@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: [RFC PATCH 3/7] mm, fs: handle memory.high overage in buffered reads and writes
Date: Sat, 21 Nov 2026 18:47:33 -0800 [thread overview]
Message-ID: <20261122024737.2741569-4-shakeel.butt@linux.dev> (raw)
In-Reply-To: <20261122024737.2741569-1-shakeel.butt@linux.dev>
A buffered read or write can copy almost 2GiB in one call. A write
charges the page cache it fills. Dirty throttling limits only the dirty
part, and does nothing on tmpfs and memfd, whose bdi has no
BDI_CAP_WRITEBACK. A read charges the page cache it reads, and the
anonymous pages it faults in when the user buffer is untouched, which
can't be reclaimed without swap. Today the charge path throttles these
calls for memory.high, except huge tmpfs pages and THP faults on the
user buffer, whose charges can't block under the default defrag
setting.
A later patch stops enforcing memory.high in the charge path, where
the task may hold locks other tasks need. Handle the overage in the
copy loops instead:
- generic_perform_write() and iomap_write_iter(): next to the dirty
throttling call.
- btrfs_buffered_write(): between ranges. btrfs has its own loop, which
the other two hooks don't cover.
- filemap_read(): after each folio is copied. Once per batch isn't
enough, as a batch of large folios can hold tens of MiB.
Writers hold the inode's i_rwsem there, and dirty throttling already
sleeps under it. Two cases hold more: splice() from a pipe also holds
the pipe's lock, and gfs2 holds the resource group and quota glocks in
writes that allocate, which other writers to that resource group wait
for, on any node. The charge path sleeps with all of these locks today,
plus those taken in write_begin(). In btrfs, copy_one_range() has
already released its space reservation or handed it to delalloc, and
dropped the snapshot lock of a NOCOW write.
Most readers hold no locks. XFS, NFS and netfs hold the file's lock
shared, and gfs2 and ocfs2 a cluster lock, so a writer to that file
waits while a reader is throttled, as it does today. filemap_read()
keeps its folio batch referenced while the task is throttled. If it
dropped the batch, reclaim could free the folios not copied yet, and
they would be read again: in a VM, a cold 256M read under
memory.high=64M then read five to nine times as much from disk for the
same progress. After the sleep, it reads i_size and
mapping_writably_mapped() again.
IOCB_NOWAIT writes and IOCB_NOWAIT or IOCB_WAITQ reads skip the hook:
io_uring issues them with ctx->uring_lock held. Reads into kernel
buffers skip it too: kernel_read() callers may hold other locks, like
cred_guard_mutex in execve(). Reclaim uses the gfp mask of the
allocation that follows, so filesystems that clear __GFP_FS get
GFP_NOFS reclaim.
Without these hooks, once the synchronous step is gone, a 256M write
under memory.high=64M in a VM reaches 257-264M on ext4, XFS and btrfs
instead of 64-67M.
Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
---
fs/btrfs/file.c | 9 +++++++++
fs/iomap/buffered-io.c | 9 +++++++++
mm/filemap.c | 31 +++++++++++++++++++++++++++++++
3 files changed, 49 insertions(+)
diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c
index f978c6524aa0..d5a00a26fba6 100644
--- a/fs/btrfs/file.c
+++ b/fs/btrfs/file.c
@@ -18,6 +18,7 @@
#include <linux/uio.h>
#include <linux/iversion.h>
#include <linux/fsverity.h>
+#include <linux/memcontrol.h>
#include "ctree.h"
#include "direct-io.h"
#include "disk-io.h"
@@ -1320,6 +1321,7 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *iter)
struct file *file = iocb->ki_filp;
loff_t pos;
struct inode *inode = file_inode(file);
+ const gfp_t high_gfp = btrfs_alloc_write_mask(inode->i_mapping);
struct extent_changeset *data_reserved = NULL;
size_t num_written = 0;
ssize_t ret;
@@ -1356,6 +1358,13 @@ ssize_t btrfs_buffered_write(struct kiocb *iocb, struct iov_iter *iter)
break;
pos += ret;
num_written += ret;
+ /*
+ * A large write can charge a lot before the task returns to
+ * userspace. Handle memory.high overage between ranges, where
+ * btrfs holds only the inode lock.
+ */
+ if (!nowait)
+ mem_cgroup_over_high_throttle(high_gfp);
cond_resched();
}
diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c
index 6d843b168d5f..861aa600b3af 100644
--- a/fs/iomap/buffered-io.c
+++ b/fs/iomap/buffered-io.c
@@ -7,6 +7,7 @@
#include <linux/buffer_head.h>
#include <linux/writeback.h>
#include <linux/swap.h>
+#include <linux/memcontrol.h>
#include <linux/migrate.h>
#include <linux/fserror.h>
#include <linux/fsverity.h>
@@ -1193,6 +1194,7 @@ static int iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i,
struct address_space *mapping = iter->inode->i_mapping;
size_t chunk = mapping_max_folio_size(mapping);
unsigned int bdp_flags = (iter->flags & IOMAP_NOWAIT) ? BDP_ASYNC : 0;
+ gfp_t high_gfp = mapping_gfp_constraint(mapping, GFP_KERNEL);
do {
struct folio *folio;
@@ -1212,6 +1214,13 @@ static int iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i,
if (unlikely(status))
break;
+ /*
+ * A large write can charge a lot before the task returns to
+ * userspace. Handle memory.high overage here too.
+ */
+ if (!(iter->flags & IOMAP_NOWAIT))
+ mem_cgroup_over_high_throttle(high_gfp);
+
if (bytes > iomap_length(iter))
bytes = iomap_length(iter);
diff --git a/mm/filemap.c b/mm/filemap.c
index dc0ceaa58a0f..b0cefe71c1fd 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -2822,6 +2822,7 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter,
struct file_ra_state *ra = &filp->f_ra;
struct address_space *mapping = filp->f_mapping;
struct inode *inode = mapping->host;
+ gfp_t high_gfp = mapping_gfp_constraint(mapping, GFP_KERNEL);
struct folio_batch fbatch;
int i, error = 0;
bool writably_mapped;
@@ -2913,6 +2914,28 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter,
error = -EFAULT;
break;
}
+
+ /*
+ * A large read can charge a lot, for the page cache and
+ * for faulting in the buffer. Handle memory.high
+ * overage after each folio, as a batch of large folios
+ * can be big. Keep the batch, or reclaim could free
+ * folios not copied yet and they would be read again.
+ * The file may change meanwhile, so check its size and
+ * mappings again. Reads into kernel buffers skip this:
+ * their callers may hold other locks.
+ */
+ if (user_backed_iter(iter) &&
+ !(iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) &&
+ mem_cgroup_over_high_pending()) {
+ mem_cgroup_handle_over_high(high_gfp);
+ isize = i_size_read(inode);
+ if (iocb->ki_pos >= isize)
+ break;
+ end_offset = min_t(loff_t, isize,
+ iocb->ki_pos + iter->count);
+ writably_mapped = mapping_writably_mapped(mapping);
+ }
}
put_folios:
for (i = 0; i < folio_batch_count(&fbatch); i++) {
@@ -4385,6 +4408,7 @@ ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i)
struct address_space *mapping = file->f_mapping;
const struct address_space_operations *a_ops = mapping->a_ops;
size_t chunk = mapping_max_folio_size(mapping);
+ gfp_t high_gfp = mapping_gfp_constraint(mapping, GFP_KERNEL);
long status = 0;
ssize_t written = 0;
@@ -4401,6 +4425,13 @@ ssize_t generic_perform_write(struct kiocb *iocb, struct iov_iter *i)
bytes = min(chunk - offset, bytes);
balance_dirty_pages_ratelimited(mapping);
+ /*
+ * A large write can charge a lot before the task returns to
+ * userspace. Handle memory.high overage here too.
+ */
+ if (!(iocb->ki_flags & IOCB_NOWAIT))
+ mem_cgroup_over_high_throttle(high_gfp);
+
if (fatal_signal_pending(current)) {
status = -EINTR;
break;
--
2.53.1-Meta
next prev parent reply other threads:[~2026-11-22 2:48 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-22 2:47 [RFC PATCH 0/7] memcg: effective high limit that avoids isolation issues Shakeel Butt
2026-11-22 2:47 ` [RFC PATCH 1/7] memcg: add helpers to handle memory.high overage in long operations Shakeel Butt
2026-11-22 2:47 ` [RFC PATCH 2/7] mm/gup: handle memory.high overage while populating Shakeel Butt
2026-11-22 2:47 ` Shakeel Butt [this message]
2026-11-22 2:47 ` [RFC PATCH 4/7] mm/shmem: handle memory.high overage in fallocate and reads Shakeel Butt
2026-11-22 2:47 ` [RFC PATCH 5/7] mm/khugepaged: handle memory.high overage in MADV_COLLAPSE Shakeel Butt
2026-11-22 2:47 ` [RFC PATCH 6/7] memcg: remove synchronous memory.high enforcement Shakeel Butt
2026-11-22 2:47 ` [RFC PATCH 7/7] selftests/cgroup: make test_memcg_high_sync check that throttling holds Shakeel Butt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261122024737.2741569-4-shakeel.butt@linux.dev \
--to=shakeel.butt@linux.dev \
--cc=akpm@linux-foundation.org \
--cc=baolin.wang@linux.alibaba.com \
--cc=brauner@kernel.org \
--cc=cgroups@vger.kernel.org \
--cc=david@kernel.org \
--cc=hannes@cmpxchg.org \
--cc=jack@suse.cz \
--cc=kas@kernel.org \
--cc=kernel-team@meta.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=ljs@kernel.org \
--cc=mhocko@kernel.org \
--cc=mkoutny@suse.com \
--cc=muchun.song@linux.dev \
--cc=nico.pache@linux.dev \
--cc=roman.gushchin@linux.dev \
--cc=surenb@google.com \
--cc=tj@kernel.org \
--cc=usama.arif@linux.dev \
--cc=vbabka@kernel.org \
--cc=willy@infradead.org \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®