From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ot1-f42.google.com (mail-ot1-f42.google.com [209.85.210.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C33A43A8FE1 for ; Fri, 10 Apr 2026 21:07:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.42 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775855270; cv=none; b=tWqxe7qq2PtrHd7CDTfJClFbTT/RgLRuprr9eP6+CLGRAm6XiSZzHyDqocjX6TaAWTyIk6fff/WfZVp++0lrg+7jf/xm4iYfKX6UWVaS0v05oqj5VMtpQNgu5a6Gyd44kK5dJEmtGqdgltrqL9NL6jLMOlDdXwBqDVPnwHVo0/A= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775855270; c=relaxed/simple; bh=cUF+TCsFr+schEecVTCKTIfvNZvIK6t/KRC4PIJ9c3A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EBj0I8TUNPK2LZ8CPluJOUxN+vuSvXM4sPCp8RdQa2JVeBt4LKUoFRemUD1339gmgFPUVXPYv+2UPlh/80l5o4rVRD5q7XbroCpHDIAxscn9WEqtRJ+3X9QOoaaS9LAXcHHJybrRLOGieCWPKh5EhsT5nH0xuA0Aam0KyiLcBto= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=o4Ga4Ul+; arc=none smtp.client-ip=209.85.210.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="o4Ga4Ul+" Received: by mail-ot1-f42.google.com with SMTP id 46e09a7af769-7dbd2a0211bso1859026a34.0 for ; Fri, 10 Apr 2026 14:07:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775855268; x=1776460068; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Uc/5SAW/hlzwTRYBBK07KZ6zfX2jO27FY+LfQ7kQsVY=; b=o4Ga4Ul+vHznqkwIZ0R9+pHO3B9As+aY4C4IRAECwWjG9lxhXZWJvxIFEEX1p9fNfv A+ydKpJspKEFfBGVDGKE/DYr1EhFuxZRowe/7W4lG/zz0mn36bWrmySyBifhYz8alsGA CsueoW93b0qAcggHL8BAnFpNyLZaT7zz9U2NO77ld84d940MjY4sjG+fL+/yeKody2d7 Gj2U5Ka6OqnC93DfurJ9zaqIRbSpMl9nhP+qqJ0xMXl36tLUDmVFOQFZcfxWQeyrJiD0 UuGUOejUwasiUM9JxDjzIwYKlU9PUz2yAVZJ8i2Wsw/i0Ypnu9QpaVUktLKsdR/d8n+J LAhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775855268; x=1776460068; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Uc/5SAW/hlzwTRYBBK07KZ6zfX2jO27FY+LfQ7kQsVY=; b=BoFlVEQmZuwoFtOFkOsGGnob+PKrBx6ezDp7nridBQvwAJrWLf8K09Oy613ASuIyjH z4U1WhwPbIz0KugeQL2jGZrCSCtUC1KapURH20F6NPhogTxB/c+D+1qh8N8X5d8ui4Ru taLuGUmvjmtMb+SAW447Mtq8MI3IALaOZitR/bploZnkH4PTxDjZIUzCOMZtanNLuMu4 rvT1JjhAzWlwCbLoLb7IK1DJdkUx9U/SK3zbF7Gpixo6fjjR6M5tNFnVCH7b6FHpX3vs ApNFMqLwjnN6swSPYybZ0f35TaCFur3uOmoaoPkd119o9/QtTR7JTSWXJWiTTEmejmqM ghgA== X-Forwarded-Encrypted: i=1; AJvYcCWnj+z4CJFJj9MQ9/sXR3AA7S+YQPY1uBd+gqJaUX5z5NS7Cbt3JFthK/+anDhaRWoCS5c3nRg+LtNCukM=@vger.kernel.org X-Gm-Message-State: AOJu0Yw0+x2cR7jMLqpgHAH30I6ucP7ra0hrolNJSyEXtxNNCWOJF9R7 Ahlcr5NuMzpUPq5atoL+C+2cK/HFBb9Aa4u1mCPaJxIyp8AAifyyV7L9uZXq1Q== X-Gm-Gg: AeBDietv7Lce4VjiSlyubp/kXC3++990KM68zZjXT2BHW/R+KHuuKdq6bQmHWhMulgs ntv/dpqaOmkIwt8VEDiJqCE9/5tfvORyFh92ko48eiV5MMwb4W9jtOiSNw6CHLM70FbhPfkTW0L Cja5wIdp8JebBg4mUMHhiK0PUZgFpCBpT3ABIKHCwV+EiuOmXR7WKF4TcxPk8b/GzsS11bLiejC HUeO5XEzd8NEqY8MJM070m6dTcRkVwHFyInVQbLs2QexVGm+Kr9I/1N4085QiIr2i8VxwadauRM gwYjo74V4pQbGQm5WA5wLb+okXhL15vISBQU8n3Mdm4D6KrFwFX6gvyXjnZreGO7A0LFhD+hB/2 4pLXC61MCePBEUjoMnKXO47wyOuUM/tY0Ycgc2ttJSUXSBtToC6RX6uuHxdycfNFWAQxolJz58i 982uuVIOiSIRoGYJ59xCun X-Received: by 2002:a05:6830:81f7:b0:7d7:e844:7f4e with SMTP id 46e09a7af769-7dc27f1ca2emr2973075a34.22.1775855267769; Fri, 10 Apr 2026 14:07:47 -0700 (PDT) Received: from localhost ([2a03:2880:10ff:7::]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7dc3959099fsm214769a34.9.2026.04.10.14.07.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Apr 2026 14:07:47 -0700 (PDT) From: Joshua Hahn To: Johannes Weiner Cc: Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com Subject: [PATCH 2/8 RFC] mm/page_counter: use page_counter_stock in page_counter_try_charge Date: Fri, 10 Apr 2026 14:06:56 -0700 Message-ID: <20260410210742.550489-3-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260410210742.550489-1-joshua.hahnjy@gmail.com> References: <20260410210742.550489-1-joshua.hahnjy@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Make page_counter_try_charge() stock-aware. We preserve the same semantics as the existing stock handling logic in try_charge_memcg: 1. Limit-check against the stock. If there is enough, charge to the stock (non-hierarchical) and return immediately. 2. Greedily attempt to fulfill the charge request and fill the stock up at the same time via a hierarchical charge. 3. If we fail with this charge, retry again (once) with the exact number of pages requested. 4. If we succeed with the greedy attempt, then try to add those extra pages to the stock. If that fails (trylock), then uncharge those surplus pages hierarchically. As of this patch, the page_counter_stock is unused, as it has not been enabled on any memcg yet. No functional changes intended. Suggested-by: Johannes Weiner Signed-off-by: Joshua Hahn --- mm/page_counter.c | 41 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 38 insertions(+), 3 deletions(-) diff --git a/mm/page_counter.c b/mm/page_counter.c index 965021993e161..7a921872079b8 100644 --- a/mm/page_counter.c +++ b/mm/page_counter.c @@ -121,9 +121,24 @@ bool page_counter_try_charge(struct page_counter *counter, struct page_counter **fail) { struct page_counter *c; + unsigned long charge = nr_pages; bool protection = track_protection(counter); bool track_failcnt = counter->track_failcnt; + if (counter->stock && local_trylock(&counter->stock->lock)) { + struct page_counter_stock *stock = this_cpu_ptr(counter->stock); + + if (stock->nr_pages >= charge) { + stock->nr_pages -= charge; + local_unlock(&counter->stock->lock); + return true; + } + local_unlock(&counter->stock->lock); + } + + charge = max_t(unsigned long, counter->batch, nr_pages); + +retry: for (c = counter; c; c = c->parent) { long new; /* @@ -140,9 +155,9 @@ bool page_counter_try_charge(struct page_counter *counter, * we either see the new limit or the setter sees the * counter has changed and retries. */ - new = atomic_long_add_return(nr_pages, &c->usage); + new = atomic_long_add_return(charge, &c->usage); if (new > c->max) { - atomic_long_sub(nr_pages, &c->usage); + atomic_long_sub(charge, &c->usage); /* * This is racy, but we can live with some * inaccuracy in the failcnt which is only used @@ -163,11 +178,31 @@ bool page_counter_try_charge(struct page_counter *counter, WRITE_ONCE(c->watermark, new); } } + + /* charge > nr_pages implies this page_counter has stock enabled */ + if (charge > nr_pages) { + if (local_trylock(&counter->stock->lock)) { + struct page_counter_stock *stock; + + stock = this_cpu_ptr(counter->stock); + stock->nr_pages += charge - nr_pages; + local_unlock(&counter->stock->lock); + } else { + page_counter_uncharge(counter, charge - nr_pages); + } + } + return true; failed: for (c = counter; c != *fail; c = c->parent) - page_counter_cancel(c, nr_pages); + page_counter_cancel(c, charge); + + if (charge > nr_pages) { + /* Retry without trying to grab extra pages to refill stock */ + charge = nr_pages; + goto retry; + } return false; } -- 2.52.0