From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-oi1-f179.google.com (mail-oi1-f179.google.com [209.85.167.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 991023164B5 for ; Thu, 26 Feb 2026 19:29:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.179 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772134181; cv=none; b=mYPCbXX3nNdIsTHYKBli0YImtJCedfX1YZ+VSY7JJDvigCAyuKlMumyauDJpIKJe47NFB80KbxHbs2jeu6KqcO6QNImwnlOF+klD0lp/AAOb4P4CtKY/RaS3iVV0swv3mnkqwpXKkULSIEr53MXYutRBNxleVk1T9gs4EvTIomE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772134181; c=relaxed/simple; bh=ZdlDkTKjnXNgim5fvDAs8eRO7tVH7SG/4iGWsBd1uts=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=SNce37DY32YCrgx3CnKgR3yMS2nfPROHmQrYTezDo0xXbIALPovmLgcZG/Xr2ja+n79zf7g3lkl1iSyVhTY5A8CSmN0OYNH+ypDo+AtuOOdbRtCKhkaYF98UgmMPhKrqjtnJAwIDv2jS9CPZGWmuZkgHEuI7sMDrGuHw3vGyXac= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=UcnLBdpr; arc=none smtp.client-ip=209.85.167.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UcnLBdpr" Received: by mail-oi1-f179.google.com with SMTP id 5614622812f47-463a94f8475so957132b6e.0 for ; Thu, 26 Feb 2026 11:29:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1772134179; x=1772738979; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=AADVg9qc0LmRgS4Sxkhna0+iWY0MK1VRsyPEGxGIkaA=; b=UcnLBdprFaEXwODqHMAKGvKC2E3y/FeEp42toSIxAwrto1HNlAuQZjj/0ar5+j7wRE PwZFTfODuOHEWUtv9h6/HQPfNu8cKv5Qv7nW04QnuNngFKWUtAYiwYHKmMB/xSagBny0 xJp5NlWvK50RQLfMiT0vA5XA2aQQkLEj9/Xjf3iL5X4TeszebU2rW3CRpqIoNuyv6CVv 29OnmqQOZGKUB/tZXQ8gA5tkaVje2gjkFJlWg8wPWKZms0TdAgIIXYjIWD4Sy7961gTl 27GwfUMNI3oGbey7DitDJjem8GPB3V5+hL19fLfQpsFJflCjxeUq+f/9RVKSicVOCjbc H3iA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772134179; x=1772738979; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=AADVg9qc0LmRgS4Sxkhna0+iWY0MK1VRsyPEGxGIkaA=; b=ptWWp1QYB7YYfMDRyjloK2Js/hYQ+aDrUuZaDg4CSWin/Dg8AY/tggnJMAGErhqriF 9nJpgONphCti3clDTaM/k0+GpZxtdaPWB6hNn9KiyeEUjG1pukHf3QdAhCI5xI3nSqm/ E1dbhlvqVT9AvHmBIORPdYrXpkJOeVnBom1AC8sbVFKWDD4fU0l02HhaEaeBU5swJPYz 2gEzne0DExI03eX0J93bc/CDuQbrYv/VJdJJPvN2lCwGG6q4U31/blgAjWWZCSzVDLaI kTueqdvPbnRQN+uf7fD5w6FRBWpZrSM0lLyPbB+DOHLoQkSH5R7DsNgRHDYLQnlMr7LK ng7w== X-Forwarded-Encrypted: i=1; AJvYcCW0RZdPm47/F8D4xrtEpg1czVJd41Dd+iowW7tErxsok2F1J2yCTip+nbSfbBj659qTngb2ZERkteY2/H8=@vger.kernel.org X-Gm-Message-State: AOJu0YwckmgABz8WUymdYHZyiAaarzX4Sw8pbkfWMa14foHcPKFO5md8 O05zw9otPRpyagdkSL8+UBybeS4Z+Y45CW7wf2meyewq8wlr01th271B X-Gm-Gg: ATEYQzxl9yQr8TiXLvAn8+t8ZRq69oQQhqfBnTVyCTfHgx6FNosCH1kEnxlmYGAbhY1 t0SNLhDg2wxUsqdPT8j0NN4BaJSFden6fGpFyZ0NsVxG+s737YbieGV+HcOhyzvP04bFnw1keoQ CgMa3bA8Q6lhiYzcX+4x55vjtdV/mRtclwBlJBUsYuLt8Xazs70HGzK/zcbkQD3mTczh3Db6awV +GFFpgxByDhHuLcOuMx0QtBnMKMlrorR9TO+YqK893EbOaGifHJO263KFcav+LbHB8eCERv+eT/ dR1bCGb6oXIPb4p3QxFAGwbV1BmDrENVRLZFRY+Anr8jQJIF1+NjuKR06VfDZ92ABaBh0pCaq6H eYG7ovEazzkevmCyBCpOEtLj3HrWs1p1HRTLonchpIojOG+wItaRHEFI2fH31lB8VL9DhouHAOS ozJv3kYAkM1RZ7nppiLtVtjg== X-Received: by 2002:a05:6808:5143:b0:45f:ea8:4184 with SMTP id 5614622812f47-464bed8fce7mr134294b6e.13.1772134179339; Thu, 26 Feb 2026 11:29:39 -0800 (PST) Received: from localhost ([2a03:2880:10ff:43::]) by smtp.gmail.com with ESMTPSA id 5614622812f47-464bb5e417fsm450688b6e.16.2026.02.26.11.29.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Feb 2026 11:29:37 -0800 (PST) From: Joshua Hahn To: Minchan Kim , Sergey Senozhatsky Cc: Johannes Weiner , Yosry Ahmed , Nhat Pham , Nhat Pham , Chengming Zhou , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com Subject: [PATCH 0/8] mm/zswap, zsmalloc: Per-memcg-lruvec zswap accounting Date: Thu, 26 Feb 2026 11:29:23 -0800 Message-ID: <20260226192936.3190275-1-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.47.3 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit INTRODUCTION ============ The current design for zswap and zsmalloc leaves a clean divide between layers of the memory stack. At the higher level, we have zswap, which interacts directly with memory consumers, compression algorithms, and handles memory usage accounting via memcg limits. At the lower level, we have zsmalloc, which handles the page allocation and migration of physical pages. While this logical separation simplifies the codebase, it leaves problems for accounting that requires both memory cgroup awareness and physical memory location. To name a few: - On tiered systems, it is impossible to understand how much toptier memory a cgroup is using, since zswap has no understanding of where the compressed memory is physically stored. + With SeongJae Park's work to store incompressible pages as-is in zswap [1], the size of compressed memory can become non-trivial, and easily consume a meaningful portion of memory. - cgroups that restrict memory nodes have no control over which nodes their zswapped objects live on. This can lead to unexpectedly high fault times for workloads, who must eat the remote access latency cost of retrieving the compressed object from a remote node. + Nhat Pham addressed this issue via a best-effort attempt to place compressed objects in the same page as the original page, but this cannot guarantee complete isolation [2]. - On the flip side, zsmalloc's ignorance of cgroup also makes its shrinker memcg-unaware, which can lead to ineffective reclaim when pressure is localized to a single cgroup. Until recently, zpool acted as another layer of indirection between zswap and zsmalloc, which made bridging memcg and physical location difficult. Now that zsmalloc is the only allocator backend for zswap and zram [3], it is possible to move memory-cgroup accounting to the zsmalloc layer. Introduce a new per-zpdesc array of objcg pointers to track per-memcg-lruvec memory usage by zswap, while leaving zram users unaffected. This creates one source of truth for NR_ZSWAP, and more accurate accounting for NR_ZSWAPPED. This brings sizeof(struct zpdesc) from 56 bytes to 64 bytes, but this increase in size is unseen by the rest of the system because zpdesc overlays struct page. Implementation details and care taken to handle the page->memcg_data field can be found in patch 3. In addition, move the accounting of memcg charges to the zsmalloc layer, whose only user is zswap at the moment. PATCH OUTLINE ============= Patches 1 and 2 are small cleanups that make the codebase consistent and easier to digest. Patches 3, 4, and 5 allocate and populate the new zpdesc->objcgs field with compressed objects' obj_cgroups. zswap_entry->objcgs is removed, and redirected to look at the zspage for memcg information. Patch 6 moves the charging and lifetime management of obj_cgroups to the zsmalloc layer, which leaves zswap only as a plumbing layer to hand cgroup information to zsmalloc. Patches 7 and 8 introduce node counters and memcg-lruvec counters for zswap. Special care is taken for compressed objects that span multiple nodes. [1] https://lore.kernel.org/linux-mm/20250822190817.49287-1-sj@kernel.org/ [2] https://lore.kernel.org/linux-mm/20250402204416.3435994-1-nphamcs@gmail.com/#t3 [3] https://lore.kernel.org/linux-mm/20250829162212.208258-1-hannes@cmpxchg.org/ [4] https://lore.kernel.org/linux-mm/c8bc2dce-d4ec-c16e-8df4-2624c48cfc06@google.com/ Joshua Hahn (8): mm/zsmalloc: Rename zs_object_copy to zs_obj_copy mm/zsmalloc: Make all obj_idx unsigned ints mm/zsmalloc: Introduce objcgs pointer in struct zpdesc mm/zsmalloc: Store obj_cgroup pointer in zpdesc mm/zsmalloc,zswap: Redirect zswap_entry->obcg to zpdesc mm/zsmalloc, zswap: Handle objcg charging and lifetime in zsmalloc mm/memcontrol: Track MEMCG_ZSWAPPED in bytes mm/vmstat, memcontrol: Track ZSWAP_B, ZSWAPPED_B per-memcg-lruvec drivers/block/zram/zram_drv.c | 17 +- include/linux/memcontrol.h | 15 +- include/linux/mmzone.h | 2 + include/linux/zsmalloc.h | 6 +- mm/memcontrol.c | 68 ++------ mm/vmstat.c | 2 + mm/zpdesc.h | 25 ++- mm/zsmalloc.c | 282 ++++++++++++++++++++++++++++++++-- mm/zswap.c | 67 ++++---- 9 files changed, 345 insertions(+), 139 deletions(-) -- 2.47.3