From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-oo1-f47.google.com (mail-oo1-f47.google.com [209.85.161.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D76938C421 for ; Tue, 23 Jun 2026 18:02:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.161.47 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782237732; cv=none; b=HJs0oVFIECNDbcCRfNVCSdq5PkElXjT6rcMsmhe6p6dcnAeiavsI0RCfayJ8ct63ddBiQrJlt3gtlvKpjhwKSlbvplIEF2+xCCNSbqdherbWT4xm6AccVFRqpkwPxAqsTav2+e6g2RFD8k3kciIovjgfFgsTfyHwRcJqsRBysl4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1782237732; c=relaxed/simple; bh=Jh+o5uMNVOhQ+P6EcPC3ygD+Ch1h+HVJp9Xn5lLIfd4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EPej+RQK8nq6TThcqGsV72TcaSwCFJ36vjlvnqGVHH448eHM+axX3A6rdrl034fw5T1KkeZ4E1NZeikkg9n2sDo/LaVsAa/WwDBIqmXtZvl5xbijYU4XQMqDJKyNjNx35f5iLzkKnwn1vAmwLM4FzMCLs4ugErq9KW8McERhm68= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=nUgmPwA8; arc=none smtp.client-ip=209.85.161.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nUgmPwA8" Received: by mail-oo1-f47.google.com with SMTP id 006d021491bc7-6a0ddedcd00so92374eaf.1 for ; Tue, 23 Jun 2026 11:02:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1782237729; x=1782842529; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=trCYdPhQh348weekSBC7Ceo7XQeHFRdxOAyYAHYqFjo=; b=nUgmPwA88+PS/WBOjlCzs5+7HoE1jX0UpgOorJKU5uNM7pl5lH0O1nfok1xgS9eQzf bzJqOk+vGUHR6SlOx+xy8+YpOjbI5DWsylBOJSYWwKt2nEeYkbhAVvGkLarqu+wQi4X+ ju/rzuIGuK09rp/10UxhWmMapPZ1uT5zu1qZV+/wPGSjbLXTCIPdy6xgRoy5051gpd51 o1UERCh8XEr2H33h+1jd26raO97HOVWnnfyid5rfN26PO9OWE1HXUpd7PL8hRhAEFxaQ Wa0InS0RPXSo09d/2QgQfoYWWgkwOB+wqnEBxDTkGwcWRxPjCI1TSKojUKEHAJX5KBuA MzPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782237729; x=1782842529; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=trCYdPhQh348weekSBC7Ceo7XQeHFRdxOAyYAHYqFjo=; b=Hn04PM5tt2g2vUCBfLaxQUyOBDPEAHD/choUwgX07KkSiZP2ucg/ZBEy3LOxPYbB/e X1Po+M5OdlmYAE9TMInCSbabOd+iAH5FO36s8nNhWEyp/rsccv41xpQzoRKw0rKQgDlu Ha9T99oi1yVdGfLqerJzsnGttzAeculZNjY6B5mtiktOqOiT5vAyE3ckzUOcH39+aMs9 BgGUOlXcY8gHLKgi5MzOQdczI3NQ7ERP6Lm7fs8EzcYXskbAd+jZ5Ns5R58EyHdZhGEr o5/vXvaXinugX2pB5+pvTqMC0Jw59qj6askLi4hexzKMD6g4Ed4aPQRKj1NJ4aa8U2sN M2Cg== X-Forwarded-Encrypted: i=1; AFNElJ8WZQpCpJHcfa4dNZBmvg5bt+//IsDFthn4YBJC5qfK78hQpav4V9WU5mCQMyi3jfufy27Gav8HL1ZJLDY=@vger.kernel.org X-Gm-Message-State: AOJu0Yyzun6OTO8c6PBHZ4iVwleDm6+nsOnqL53y2u2o3Hi9qauXLuVM HxCU6Boe9LOxWIQiyex6RDuJgSh5hOrqKsM5EfZtjfAD4YhKYJsxV2BV X-Gm-Gg: AfdE7cnky09brm59FAF21cK/DKWgYhgdDf37+cVhpQ7/WFujb0GQV+HjHHpLTDbdmN6 C/A2zAoTT6MuvD7Rk5h11FWgoeGSS4UJSOpQ7vsbU9GUoYetHtzL1PLppPBM6sbOsaKPU++Jbot xnWWwNM1P6suRtWDCQmYzmxAZW7/fC48McYctYErwA+dGcUKKlXobOhyBmK4UovikpWuC63Pi0z 6Q+V0xK+PPFjZb+fLm5E3Z1JZk+fKdvKWEQtpFLsPFO+JMr4/8hxiyvVpju0KRU3Rdux7skS5Ta m2uf2VyTPEnwDm15Kln4cYR+WshgssbgMBJ07FxpYMYhTC02yDbYdou4NsVCFq0h15Ck9/8kwSG DPUgglNHzC1ru9/p7kKpfldosXKdtb080//QesQiVZyn0mh6zn2bjumUu4DALZSvoUijP8Y/lqR 6IMJEMb/cmfiXhF0/VAcrNn/s9E0lWdAnByAcxwl1Bffk= X-Received: by 2002:a05:6820:3082:b0:69d:f855:b748 with SMTP id 006d021491bc7-6a11623b266mr2355735eaf.34.1782237690173; Tue, 23 Jun 2026 11:01:30 -0700 (PDT) Received: from localhost ([2a03:2880:10ff:45::]) by smtp.gmail.com with ESMTPSA id 006d021491bc7-6a0e9f2a583sm7583532eaf.1.2026.06.23.11.01.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Jun 2026 11:01:29 -0700 (PDT) From: Joshua Hahn To: Johannes Weiner , Michal Hocko Cc: Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R . Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com Subject: [PATCH v4 3/5] mm/page_counter: introduce page_counter_try_charge_stock() Date: Tue, 23 Jun 2026 11:01:21 -0700 Message-ID: <20260623180124.868655-4-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260623180124.868655-1-joshua.hahnjy@gmail.com> References: <20260623180124.868655-1-joshua.hahnjy@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Add a stock-aware variant of page_counter_try_charge. As before with try_charge_memcg, it first tries to satisfy the charge by consuming the per-cpu stock (and skipping the hierarchical charge). On a miss, it tries to greedily overcharge up to counter->batch pages to refill the stock. Finally, if this fails, it tries to charge exactly the requested number of pages. The number of pages that were charged to the page_counter is reported back to the caller, so that stock hits don't trigger memory limit checks. The variant is unused for now; memcg is converted in a later patch. Signed-off-by: Joshua Hahn --- include/linux/page_counter.h | 4 +++ mm/page_counter.c | 48 ++++++++++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+) diff --git a/include/linux/page_counter.h b/include/linux/page_counter.h index 4abc7fe7c3494..b97b5491447e4 100644 --- a/include/linux/page_counter.h +++ b/include/linux/page_counter.h @@ -84,6 +84,10 @@ void page_counter_charge(struct page_counter *counter, unsigned long nr_pages); bool page_counter_try_charge(struct page_counter *counter, unsigned long nr_pages, struct page_counter **fail); +bool page_counter_try_charge_stock(struct page_counter *counter, + unsigned long nr_pages, + struct page_counter **fail, + unsigned long *nr_charged); void page_counter_uncharge(struct page_counter *counter, unsigned long nr_pages); void page_counter_set_min(struct page_counter *counter, unsigned long nr_pages); void page_counter_set_low(struct page_counter *counter, unsigned long nr_pages); diff --git a/mm/page_counter.c b/mm/page_counter.c index 6bb48a913a90d..cce3af3f19e03 100644 --- a/mm/page_counter.c +++ b/mm/page_counter.c @@ -172,6 +172,54 @@ bool page_counter_try_charge(struct page_counter *counter, return false; } +bool page_counter_try_charge_stock(struct page_counter *counter, + unsigned long nr_pages, + struct page_counter **fail, + unsigned long *nr_charged) +{ + struct page_counter_stock *stock; + unsigned long charge = 0; + int old; + + if (!counter->stock) + goto charge_exact; + + preempt_disable(); + stock = this_cpu_ptr(counter->stock); + old = atomic_read(&stock->nr_pages); + while ((unsigned long)old >= nr_pages) { + if (atomic_try_cmpxchg(&stock->nr_pages, &old, + old - (int)nr_pages)) { + preempt_enable(); + goto out_success; + } + } + preempt_enable(); + + charge = max_t(unsigned long, READ_ONCE(counter->batch), nr_pages); + if (charge <= nr_pages) + goto charge_exact; + + if (page_counter_try_charge(counter, charge, fail)) { + preempt_disable(); + stock = this_cpu_ptr(counter->stock); + atomic_add((int)(charge - nr_pages), &stock->nr_pages); + preempt_enable(); + goto out_success; + } + +charge_exact: + /* stock is not enabled, no need for surplus, or greedy charge failed */ + charge = nr_pages; + if (!page_counter_try_charge(counter, charge, fail)) + return false; + +out_success: + if (nr_charged) + *nr_charged = charge; + return true; +} + /** * page_counter_uncharge - hierarchically uncharge pages * @counter: counter -- 2.53.0-Meta