From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ot1-f41.google.com (mail-ot1-f41.google.com [209.85.210.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 068DA175A8F for ; Thu, 21 May 2026 03:46:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.41 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779335169; cv=none; b=fE7IFaSVRlobMgg47U5j20HM6r1JM2LCofZq6OtT53XE8r5Et7X1S368elN/dyNCeuUgXgYQKcenzxYKfHNGEpNDc+zzVOMGl9ARH7xZ+/eHWYbsmysNtKS1LMquWQIxD/TmewwQSXhUOekp65UDo1CPmnbQP03N0s0WO8CmH4Y= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779335169; c=relaxed/simple; bh=bopuXWm9olVJHXpLu1DG1Qyl7MHDKEjinXkMh4Vre9g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=nyK2KtPm+1zHw8cuHGLdtp2y5KARbps3YRkXi/3LIwGNC39r1OBKinpv/2DqwtXGhqjoAEDN65yCVypxwkOyGASETDQCgFdN34n0vbdoMbNPB2RuOjxopH1G51LVwh7XpfuxCzHREUac+sfVHb4K3Q+ULrQrckxgf2mQamOyUlo= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=B4g4Mxuu; arc=none smtp.client-ip=209.85.210.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="B4g4Mxuu" Received: by mail-ot1-f41.google.com with SMTP id 46e09a7af769-7dcdd1b492eso5709839a34.1 for ; Wed, 20 May 2026 20:46:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1779335167; x=1779939967; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zEBQggBlYt3BpKFY9Kg3q0PYXokZZ4ieOXENn/+MaCo=; b=B4g4Mxuuk5NY6oQG5OPetJ7rN985SsEfZSaMmagjS2eGXzN9Q2jtot1vGbUTR/OLnM A5PmHm4rCxFoOXRGuhTNwDeUT7nebohWLAdV1zPhKF0Y20t7ZLZS8z2P/SOhoYLchoQE INCcOmbT6D7zie9ukmxkCwH8wN2MeUO9OmXz/6xZwLMOBkLmpCeek8aH0Da6n1YfxkTQ 45VV7wguGHzRXIzs96EZM1tWP3SFo2ORhgfpb/Cmkjn8E7wcyuOHccnGjr964sIaSxf3 BtaM7YkiM5L85xYOFkFDbwdAjQKQBAnzBL+bbRwawI1cRUvDY1D+0g3QAeO5J616FUXp Qz7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779335167; x=1779939967; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=zEBQggBlYt3BpKFY9Kg3q0PYXokZZ4ieOXENn/+MaCo=; b=BzVAaRmXbaHgBDjpig83WUx8Fz+XrMRTQzPIFkpSm7rVBBq7lrJq2bZiQnvKtxtsoK eRh+VoR3f1eu/pZOMR/jewTcn0Yspdbvrc9a+jDPqeIM6deikKVFrmk/eTs0o7EghoSX YOsLbfUSonpmKeBSCEdtBIzVO1tFAkQoFMM040QkZbPZ6OiPPLc02mde5/UB2MIiFyc6 IiLH+Sl98+72KddNzHDvMsQlcKwvbygGFk60YpAFCeax05jplWO9d7+IhKWfNlwWLE09 b7ewMSYn2YLD8fdE2pIIgJkuq3kGpfPXjvo5l/tvcHVlBWyn0cq42g6KU/roAOychZYF jj1g== X-Forwarded-Encrypted: i=1; AFNElJ9PEKTgw+uGwubz+hG737135YoN3XdPanpL4CICdih93egh3rUSN+FC5f2qj78EYT2kiMTSQWq70EZL5jE=@vger.kernel.org X-Gm-Message-State: AOJu0YwzW4MulFMpjGvRMIPrFytb2YluqesrfAVzTdDJ2STe2+z62vKe rtahcRxe2qy50cA0deXqP3vxobdylHAGpFOx0hULu7X+1nFCuvJEDBQ4 X-Gm-Gg: Acq92OFnzCZUtFSUg6yT3D/1JY8nUgX19Nyt6fTe5kG1dG721wQnXAXFfpt12Yy53yi Fd9efcFdGV75C4E5tqMeM/bJx74zVBHFWw9aqFuBmP3Sx+oqgAvwfl7yABGkhBExWUHRr2dTBEL GM/OO/+7DGRClYHibSY1ypAeTxNlJRUPgO+YNCnKGPjDSTGtVmYH/gHe0vwkf2UTNqpr/UepoFG WaZU5QxlVydlgLNEOSauB4eDo4a+qvJGjVYBj/Va1NbZBgCwufHsE/x3CZi/iM33N+VIhrtlUrr s398buAKZP4hySraYbhoLbAwnQuKCLmXJPikiodwW+GUa1SDfCfr2kt+R74XArl8kweYLS6GijN aSmh1s5/OO8cDgK+JQ0TC3kohnuN+vE501p2liA5GYebgabngN04YZzlyPfV9MW/vo6Sf04k+zS AJhr7B1qKlT1aTyGQ7PZvkJeXyn9WBREOVTstckoD44JJ/zR4vFuCmJQ== X-Received: by 2002:a05:6830:18fa:b0:7dc:e059:c25d with SMTP id 46e09a7af769-7e5ea0e7ddcmr512971a34.0.1779335166967; Wed, 20 May 2026 20:46:06 -0700 (PDT) Received: from localhost ([2a03:2880:10ff:58::]) by smtp.gmail.com with ESMTPSA id 46e09a7af769-7e55bc4749esm14591959a34.23.2026.05.20.20.46.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 20:46:06 -0700 (PDT) From: Joshua Hahn To: Alexandre Ghiti Cc: Andrew Morton , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Dennis Zhou , Tejun Heo , Christoph Lameter , Vlastimil Babka , Yosry Ahmed , Nhat Pham , Sergey Senozhatsky , Chengming Zhou , Suren Baghdasaryan , Qi Zheng , David Hildenbrand , Lorenzo Stoakes , Minchan Kim , Mike Rapoport , Axel Rasmussen , Barry Song , Kairui Song , Wei Xu , Yuanchu Xie , "Liam R . Howlett" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org Subject: Re: [PATCH 0/8] per-memcg-per-node kmem accounting Date: Wed, 20 May 2026 20:46:03 -0700 Message-ID: <20260521034604.4126295-1-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <7e28d0c9-80f2-420b-87c4-55e571071059@ghiti.fr> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On Wed, 20 May 2026 10:39:59 +0200 Alexandre Ghiti wrote: > Hi Joshua, > > On 5/18/26 16:57, Joshua Hahn wrote: > > On Mon, 11 May 2026 22:20:35 +0200 Alexandre Ghiti wrote: > > > >> This series pursues the work initiated by Joshua [1]. We need kernel > >> memory to be accounted on a per-node basis in order to be able to > >> know the memcg and physical memory association. > >> > >> This series takes advantage of the recent introduction of per-node > >> obj_cgroup [2] and makes those obj_cgroup tied to their numa node. > >> > >> The bulk of the series is percpu per-node accounting: percpu > >> "precharges" the memcg before we know the actual location of the pages > >> it uses, so charging and accounting had to be split. All other kmem > >> users (slab, zswap, __memcg_kmem_charge_page) are straightforward > >> conversions (zswap support is limited in this series because Joshua > >> is working on it in parallel [3]). > >> > >> Thanks Joshua for your early feedbacks! > > Hello Alex, > > > > Thank you for your work! > > > > Overall I think the direction makes sense to me. Pre-overcharging makes sense to > > me as an approach, we would much rather overaccount than underaccount and > > later have to breach limits. > > > > I do have some concerns on performance, though. Namely, I think there are > > some expensive operations that I think would benefit from some performane > > benchmarking with this patch added (maybe some simple microbenchmarks that > > demonstrates kernel allocation overhead could be useful). > > > > From what I can tell, there is some additional performance overhead that has > > to do with iterating over num_possible_cpus() x pages_per_alloc, which > > doesn't seem trivial to me. > > Indeed, let me microbenchmark the overhead on a large system. Hi Alex, That sounds great with me : -) Looking forward to the numbers! > > Another concern that I see is the stock credit system. Maybe we could be > > bypassing the stock check leading to more time spent doing the atomic > > operations. > > I'm not following on this one, which atomic operations do you see that > could be bypassed? So in my initial scan of the patch 7 I had a concern that if we have a nested stock system (obj_cgroup stock and local credit "stock"), then we could incur more work if these are out of sync; do extra work in the stock refill path in obj_cgroup_precharge, and then do extra work on top in the loop within the pcpu_memcg_post_alloc_hook (obj_cgroup_account_kmem does the charging atomically I think). So what I mean is, I'm not sure what the "size" is typically for pcpu_memcg_post_alloc_hook. But it might be a worthwhile optimization to do precharge all the pages, then for each cpu iterate over the pages to figure out how many pages are used per nid (doing just math, not actually doing the atomic adds), and then outside both of these loops just iterate over every nid_objcg once to perform the atomic operation. Maybe this is needed or not (depending on how big "size" typically is and whether we go from doing O(1000) atomic adds --> O(10) or some big reduction, but I just wanted to toss it out there as something that could potentially be expensive. > > obj_stock caches a single obj_cgroup, which means that if we split the objcg > > to be per-node (in patch 6), then the obj_stock basically gets invalidated > > every operation since we iterate over more objcgs (even though we are in > > the same logical objcg). Maybe I'm missing something? > > > The objcg split comes from commit 01b9da291c49 ("mm: memcontrol: convert > objcg to be per-memcg per-node type") and the problem you describe is > exactly what Shakeel is trying to fix [1]. Whoops O_o I completely missed that one. Sorry for flagging it again! > But I remember trying a microbenchmark and noticed a +5% regression (on > top of the 67% then...), I'll rebase this series on top of Shakeel's and > re-run. Sounds like a great idea! Thanks again Alex, have a great day! : -) Joshua