From: Joshua Hahn <joshua.hahnjy@gmail.com>
To: Johannes Weiner <hannes@cmpxchg.org>, Michal Hocko <mhocko@kernel.org>
Cc: Roman Gushchin <roman.gushchin@linux.dev>,
Shakeel Butt <shakeel.butt@linux.dev>,
Muchun Song <muchun.song@linux.dev>,
Andrew Morton <akpm@linux-foundation.org>,
David Hildenbrand <david@kernel.org>,
Lorenzo Stoakes <ljs@kernel.org>,
"Liam R . Howlett" <liam.howlett@oracle.com>,
Vlastimil Babka <vbabka@kernel.org>,
Mike Rapoport <rppt@kernel.org>,
Suren Baghdasaryan <surenb@google.com>,
cgroups@vger.kernel.org, linux-mm@kvack.org,
linux-kernel@vger.kernel.org, kernel-team@meta.com
Subject: [PATCH v4 3/5] mm/page_counter: introduce page_counter_try_charge_stock()
Date: Tue, 23 Jun 2026 11:01:21 -0700 [thread overview]
Message-ID: <20260623180124.868655-4-joshua.hahnjy@gmail.com> (raw)
In-Reply-To: <20260623180124.868655-1-joshua.hahnjy@gmail.com>
Add a stock-aware variant of page_counter_try_charge.
As before with try_charge_memcg, it first tries to satisfy the charge by
consuming the per-cpu stock (and skipping the hierarchical charge). On a
miss, it tries to greedily overcharge up to counter->batch pages to
refill the stock. Finally, if this fails, it tries to charge exactly the
requested number of pages.
The number of pages that were charged to the page_counter is reported
back to the caller, so that stock hits don't trigger memory limit
checks.
The variant is unused for now; memcg is converted in a later patch.
Signed-off-by: Joshua Hahn <joshua.hahnjy@gmail.com>
---
include/linux/page_counter.h | 4 +++
mm/page_counter.c | 48 ++++++++++++++++++++++++++++++++++++
2 files changed, 52 insertions(+)
diff --git a/include/linux/page_counter.h b/include/linux/page_counter.h
index 4abc7fe7c3494..b97b5491447e4 100644
--- a/include/linux/page_counter.h
+++ b/include/linux/page_counter.h
@@ -84,6 +84,10 @@ void page_counter_charge(struct page_counter *counter, unsigned long nr_pages);
bool page_counter_try_charge(struct page_counter *counter,
unsigned long nr_pages,
struct page_counter **fail);
+bool page_counter_try_charge_stock(struct page_counter *counter,
+ unsigned long nr_pages,
+ struct page_counter **fail,
+ unsigned long *nr_charged);
void page_counter_uncharge(struct page_counter *counter, unsigned long nr_pages);
void page_counter_set_min(struct page_counter *counter, unsigned long nr_pages);
void page_counter_set_low(struct page_counter *counter, unsigned long nr_pages);
diff --git a/mm/page_counter.c b/mm/page_counter.c
index 6bb48a913a90d..cce3af3f19e03 100644
--- a/mm/page_counter.c
+++ b/mm/page_counter.c
@@ -172,6 +172,54 @@ bool page_counter_try_charge(struct page_counter *counter,
return false;
}
+bool page_counter_try_charge_stock(struct page_counter *counter,
+ unsigned long nr_pages,
+ struct page_counter **fail,
+ unsigned long *nr_charged)
+{
+ struct page_counter_stock *stock;
+ unsigned long charge = 0;
+ int old;
+
+ if (!counter->stock)
+ goto charge_exact;
+
+ preempt_disable();
+ stock = this_cpu_ptr(counter->stock);
+ old = atomic_read(&stock->nr_pages);
+ while ((unsigned long)old >= nr_pages) {
+ if (atomic_try_cmpxchg(&stock->nr_pages, &old,
+ old - (int)nr_pages)) {
+ preempt_enable();
+ goto out_success;
+ }
+ }
+ preempt_enable();
+
+ charge = max_t(unsigned long, READ_ONCE(counter->batch), nr_pages);
+ if (charge <= nr_pages)
+ goto charge_exact;
+
+ if (page_counter_try_charge(counter, charge, fail)) {
+ preempt_disable();
+ stock = this_cpu_ptr(counter->stock);
+ atomic_add((int)(charge - nr_pages), &stock->nr_pages);
+ preempt_enable();
+ goto out_success;
+ }
+
+charge_exact:
+ /* stock is not enabled, no need for surplus, or greedy charge failed */
+ charge = nr_pages;
+ if (!page_counter_try_charge(counter, charge, fail))
+ return false;
+
+out_success:
+ if (nr_charged)
+ *nr_charged = charge;
+ return true;
+}
+
/**
* page_counter_uncharge - hierarchically uncharge pages
* @counter: counter
--
2.53.0-Meta
next prev parent reply other threads:[~2026-06-23 18:02 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-23 18:01 [PATCH v4 0/5] mm/memcontrol, page_counter: move stock from mem_cgroup to page_counter Joshua Hahn
2026-06-23 18:01 ` [PATCH v4 1/5] mm/page_counter: introduce per-page_counter stock Joshua Hahn
2026-06-23 18:01 ` [PATCH v4 2/5] mm/memcontrol: flatten try_charge_memcg control flow Joshua Hahn
2026-06-23 18:01 ` Joshua Hahn [this message]
2026-06-23 18:01 ` [PATCH v4 4/5] mm/memcontrol: convert memcg to use page_counter_stock Joshua Hahn
2026-06-24 14:43 ` Usama Arif
2026-06-24 15:23 ` Joshua Hahn
2026-06-24 16:43 ` Usama Arif
2026-06-24 18:24 ` Joshua Hahn
2026-07-09 8:23 ` kernel test robot
2026-07-30 6:56 ` Oliver Sang
2026-07-30 14:30 ` Joshua Hahn
2026-06-23 18:01 ` [PATCH v4 5/5] mm/memcontrol: remove unused memcg_stock code Joshua Hahn
2026-06-26 20:18 ` [PATCH v4 0/5] mm/memcontrol, page_counter: move stock from mem_cgroup to page_counter Joshua Hahn
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260623180124.868655-4-joshua.hahnjy@gmail.com \
--to=joshua.hahnjy@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=cgroups@vger.kernel.org \
--cc=david@kernel.org \
--cc=hannes@cmpxchg.org \
--cc=kernel-team@meta.com \
--cc=liam.howlett@oracle.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=ljs@kernel.org \
--cc=mhocko@kernel.org \
--cc=muchun.song@linux.dev \
--cc=roman.gushchin@linux.dev \
--cc=rppt@kernel.org \
--cc=shakeel.butt@linux.dev \
--cc=surenb@google.com \
--cc=vbabka@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®