From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 497373EAC8A for ; Fri, 29 May 2026 14:44:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=209.85.218.41 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780065853; cv=pass; b=e97u52jLUmgxt1LSnofao8EwidUNLM4XYEUyXKaILj+kie6KpXaBL7xDfPfi1l16z506yvwEVoNGO78gG5jNZBJVIIV9m8yDiogt64NokV1tRwntcd8op8CUQK9mVrYTEadDfFG7eDyeAjmIduPUIGA2MGkSOzprqDB82qEMltE= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780065853; c=relaxed/simple; bh=ZMlUlOXe9c7G5c0G6bckWvx3txdC2cASx0GyRz7LbHo=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=GxKfj7MinwesyoYbk9dQIQUVlc++W02igxoNZywpZOSWgMMB6MWA2ee2D8l+/m7x1wEmQQPZXKBI/7V3Qm4ufUv+V215bYCFSOdLsZHXveNXPjaYRlLrCUmzHXo4B0vjFd9JW3xbrfsuFNKu+crT9rvJQ+iCNMGNRarMOz/uo98= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=h4fnmHOX; arc=pass smtp.client-ip=209.85.218.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="h4fnmHOX" Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-bd4f7f05e90so2401076666b.2 for ; Fri, 29 May 2026 07:44:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1780065850; cv=none; d=google.com; s=arc-20240605; b=k7hPAZp8MvRiDuxDQFx7tMDBBVmviPWeQmAT/sSxuYrGBaEhFU7PUHx40Vv63BO27k lwYTIcDD+yLUzliRhEiSd3sePwSWQdP1UxMD5M3cKGneEE1Bl+8R9yacT30M4Dm3DQd5 5u1UaigHXB3xstOYRPxDHSIJl3gapTJEU+xyHcufNqAcu5pHFaGPv6EekuahYODPMXNH N5w1iYYrD8iJ4Cc5qnOq5/11MJB44SmHMNHXIdKuDXaMBnog1HHxXbb6KnkqwZw9mynl jnEUymT0xNGIzaWlW9xZ0xT40lUmZpP3AseZ6ue9EbPDflzYvmIpZqOidPQT663T+JQH s0pg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=xee0neRMxkEGWjOCgf0A+ajeM60T1isEWix9DcnYtQ4=; fh=P/EObqX6Y/5zkzwmwa7NU59WfxAtq6WB2rjrbmoEJfk=; b=Kvmf+tS1MTknCo2056im7MVzS+bCFES6WHpCBuu4o//AL2/jIM51bjfAl8p1h0ooOL 7/EvBUrdk7I00kRN8k/jFMhbONB+kuo5oxlUAXg3vDzxoZ5IJ8gt8a7040Wn+G0JVTbD AOcP4u1z4cQ7mXGRj78AYxzpsiotdtFz/0ToL1KwMKgZCt+DBXbmUo4V1edrV5w28hEq JLqwVN8FFLjETUy9bgmFikUUh5hJH4/naE5HvSM4U6SCQaMGCVl+NdiObWrxvzY3NIcZ FZM6z0EyCGfFooW8VLTt/ihCYIk7mqdZzOTS0aXzxBCBpJfOiug4Rr24EVTcwgKT7VZi ldVg==; darn=vger.kernel.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1780065850; x=1780670650; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=xee0neRMxkEGWjOCgf0A+ajeM60T1isEWix9DcnYtQ4=; b=h4fnmHOXugXcPqKH+FhPRfa0ee5LyB/9TrGUSfMf8aUzwe3n2Z0auCqEcao+8iB+wU TYq/l8LPcn0LqW1XGuMwzi/RO3xw4viqpsgGDfGJifCXxfknSJUEDVwIBDzA3FKAN5Em HWsG4/vr/R6BYV+LCkKcfqR5V6TgoLjuDMdivo0I8vP3Ys3mESBChE4ySxF4Q8wax1FH uhLU3kfhpu3ZLXVfpUOMe7CbJZo4saC3fPY5S8se2RQxvkk05Ou9jaZW/cj5VzuyuxDt neTbp8M/MhZMTRJpLe2X9IX8rTzWGtfaSsa1yXGdB7w9yBuGRThJzQOa3PI10mC1mYnD RfMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780065850; x=1780670650; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=xee0neRMxkEGWjOCgf0A+ajeM60T1isEWix9DcnYtQ4=; b=WGC+190YAU6me5x98Yboiqtb3+i3ck0oZOUOnkCsuqT6+Q5rtaHIWRu+mCIHwPsiW+ 92HYLHeP3bpPzYCTxbUzmkMQbXLTQHEAiKk215/n3n+2tGhQ2JaC/5LM5UPEhCPmWGK+ ALdMekMwB4otdnSLk0YcZ1rOKU9v5ot7GnpVZhKRHfJXLYmGPODihdm4KMGrHv55+YlB UDQMxvmxioMhcMQj+WMyXUipVwecr3TFs3+j4zhjzh8W4Zne48pRBRX2ftooQPltEFE8 4ArSHxjXLzLHC7+JUZogwLTLt1u9VoN62XqKJXl1BbuE2Zx+9JC956wHn6RoZHaEd94E j9jQ== X-Forwarded-Encrypted: i=1; AFNElJ+GOP7SgcGN47GuEVJUZBp1C7xbQ6fBlu6lYyEWd/ZHzRfhNCX8bh5LhYod5TeEVoeeDfGEuLYPRqSnjoA=@vger.kernel.org X-Gm-Message-State: AOJu0Yyf/l3uoiR5dzJvK2keQaZg/VS0m9VapFnl0B1fEsFSiaql58xg zDjMXN9bXMUFteOKADn628E9o6hCSb6ici78WK94TBwGbqSIb18Upe3/4HZfrSN0tJ9qiFTR5sx 9Vm/SQFnP4ikvmvjFAC7Jdd3MhXcp/3tpKFHBv8rnwQ== X-Gm-Gg: Acq92OHzFff/mXVAJP4eV65JW0RC78DwrODjjnCs3VBt0/VtOSqRyhM6NBIITKE1OC3 SvcvAhTH3AuFBr0/GS2MS5wuM1xA/wUnhzmm4aeCEdoecT07BYEJZbklolexbtJzhfY7eOJdgHc eocZyt6fyQA0kwz+CmeqqLgZD87w9nxvYZUPT6IgIO73P1xUm3RJRwZpoUvZCJ+SG/1gPLqaiV8 xHOJdDjyTwtphFq5iP5PPoRwXMtg0y4JMZH9OvszQLbHfs42a9/9ZrR5cu0e0SI9QvC228OgRd+ 6NX784yufTGSZhZJVRhwZzlWngz0sMMflDDsDv1caD6xgppi7r0= X-Received: by 2002:a17:907:a68a:b0:bd5:2a43:b471 with SMTP id a640c23a62f3a-be9ccb8b34cmr202848866b.48.1780065849252; Fri, 29 May 2026 07:44:09 -0700 (PDT) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: In-Reply-To: From: Kairui Song Date: Fri, 29 May 2026 22:43:32 +0800 X-Gm-Features: AVHnY4KQ3LS9xvgeQfV41IMJNm14XsvQyrU_7jFjtxnCiD6FZQ0cqNPWPLweRic Message-ID: Subject: Re: [RFC PATCH v2 4/9] mm: admit large swapin by backend range in swapin_sync() To: fujunjie Cc: Andrew Morton , linux-mm@kvack.org, Alexandre Ghiti , Usama Arif , Chris Li , Johannes Weiner , Yosry Ahmed , Nhat Pham , David Hildenbrand , Hugh Dickins , Roman Gushchin , Shakeel Butt , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, May 29, 2026 at 8:26=E2=80=AFPM fujunjie wrote: > > A large swapin can only read one folio when the whole range has compatibl= e > backing. Mixed zswap/disk ranges must not reach large-folio IO, and zswap > range probes are only snapshots. > > Filter the orders passed to swap_cache_alloc_folio() in swapin_sync(). > Uniform zeromap ranges and all-disk ranges keep the existing large swapin > path. Fully zswap-backed ranges may be tried. Mixed zswap/disk ranges fal= l > back before allocation. > > After a large swapcache folio is installed, recheck the zswap range and > drop the fresh folio if it became mixed. Also consume -EAGAIN from > swap_read_folio() the same way. Both cases retry order-0, where each slot > can resolve its current backend independently. > > Signed-off-by: fujunjie > --- > mm/memcontrol-v1.c | 8 ++- > mm/memory.c | 31 ++++++++- > mm/swap_state.c | 169 ++++++++++++++++++++++++++++++++++++++++++--- > 3 files changed, 194 insertions(+), 14 deletions(-) > > diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c > index 765069211567..5b11b8055c66 100644 > --- a/mm/memcontrol-v1.c > +++ b/mm/memcontrol-v1.c > @@ -682,8 +682,8 @@ void __memcg1_swapout(struct folio *folio, struct swa= p_cluster_info *ci) > * memcg1_swapin - uncharge swap slot on swapin > * @folio: folio being swapped in > * > - * Call this function after successfully adding the charged > - * folio to swapcache. > + * Call this after the charged folio has been added to swapcache and the= caller > + * is no longer going to drop it back to swapped-out state. > * > * Context: The folio has to be in swap cache and locked. > */ > @@ -721,7 +721,9 @@ void memcg1_swapin(struct folio *folio) > id =3D __swap_cgroup_clear(ci, swp_cluster_offset(folio->swap), > nr_pages); > swap_cluster_unlock(ci); > - mem_cgroup_uncharge_swap(id, nr_pages); > + > + if (id) > + mem_cgroup_uncharge_swap(id, nr_pages); > } > #endif > > diff --git a/mm/memory.c b/mm/memory.c > index 5a365492a9a2..d73a19692dea 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -4538,6 +4538,24 @@ static inline bool should_try_to_free_swap(struct = swap_info_struct *si, > folio_ref_count(folio) =3D=3D (extra_refs + folio_nr_page= s(folio)); > } > > +static void memcg1_swapin_retry_folio(struct folio *folio, > + struct vm_fault *vmf) > +{ > + if (!folio_test_large(folio) || !folio_test_swapcache(folio)) > + return; > + > + if (vmf->flags & FAULT_FLAG_RETRY_NOWAIT) { > + if (!folio_trylock(folio)) > + return; > + } else { > + folio_lock(folio); > + } > + > + if (folio_test_large(folio) && folio_test_swapcache(folio)) > + memcg1_swapin(folio); > + folio_unlock(folio); > +} > + > static vm_fault_t pte_marker_clear(struct vm_fault *vmf) > { > vmf->pte =3D pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, > @@ -4857,8 +4875,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) > > swapcache =3D folio; > ret |=3D folio_lock_or_retry(folio, vmf); > - if (ret & VM_FAULT_RETRY) > + if (ret & VM_FAULT_RETRY) { > + memcg1_swapin_retry_folio(folio, vmf); > goto out_release; > + } > > page =3D folio_file_page(folio, swp_offset(entry)); > /* > @@ -5067,6 +5087,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) > if (unlikely(folio !=3D swapcache)) { > folio_add_new_anon_rmap(folio, vma, address, RMAP_EXCLUSI= VE); > folio_add_lru_vma(folio, vma); > + if (folio_test_large(swapcache)) > + memcg1_swapin(swapcache); > folio_put_swap(swapcache, NULL); > } else if (!folio_test_anon(folio)) { > /* > @@ -5076,6 +5098,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) > VM_WARN_ON_ONCE_FOLIO(folio_nr_pages(folio) !=3D nr_pages= , folio); > VM_WARN_ON_ONCE_FOLIO(folio_mapped(folio), folio); > folio_add_new_anon_rmap(folio, vma, address, rmap_flags); > + if (folio_test_large(folio)) > + memcg1_swapin(folio); > folio_put_swap(folio, NULL); > } else { > VM_WARN_ON_ONCE(nr_pages !=3D 1 && nr_pages !=3D folio_nr= _pages(folio)); > @@ -5132,8 +5156,11 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) > if (vmf->pte) > pte_unmap_unlock(vmf->pte, vmf->ptl); > out_page: > - if (folio_test_swapcache(folio)) > + if (folio_test_swapcache(folio)) { > + if (folio_test_large(folio)) > + memcg1_swapin(folio); > folio_free_swap(folio); > + } > folio_unlock(folio); > out_release: > folio_put(folio); > diff --git a/mm/swap_state.c b/mm/swap_state.c > index d37097913b30..f03ad4832f16 100644 > --- a/mm/swap_state.c > +++ b/mm/swap_state.c > @@ -21,6 +21,7 @@ > #include > #include > #include > +#include > #include > #include "internal.h" > #include "swap_table.h" > @@ -403,7 +404,8 @@ void __swap_cache_replace_folio(struct swap_cluster_i= nfo *ci, > static struct folio *__swap_cache_alloc(struct swap_cluster_info *ci, > swp_entry_t targ_entry, gfp_t gfp= , > unsigned int order, struct vm_fau= lt *vmf, > - struct mempolicy *mpol, pgoff_t i= lx) > + struct mempolicy *mpol, pgoff_t i= lx, > + bool defer_memcg1_swapin) Hi Fujunjie, Thanks for the update, but this whole defer_memcg1_swapin thing is so ugly I don't think this is the right way at all. If you really need this, maybe you can always defer the memcg1 uncharge, I don't see why we need to treat large folio differently. This charge doesn't effect the memory pressure, the reason we uncharge memcg1's swap counter is to avoid long pinning swap cache holding the swap cache of a cgroup so the cgroup will no longer be able to swap out more folios. Deferring it won't hurt. > { > int err; > swp_entry_t entry; > @@ -466,7 +468,8 @@ static struct folio *__swap_cache_alloc(struct swap_c= luster_info *ci, > } > > /* memsw uncharges swap when folio is added to swap cache */ > - memcg1_swapin(folio); > + if (!defer_memcg1_swapin || !order) > + memcg1_swapin(folio); > if (shadow) > workingset_refault(folio, shadow); > > @@ -495,9 +498,12 @@ static struct folio *__swap_cache_alloc(struct swap_= cluster_info *ci, > * Return: Returns the folio if allocation succeeded and folio is in the= swap > * cache. Returns error code if failed due to race, OOM or invalid argum= ents. > */ > -struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp, > - unsigned long orders, struct vm_faul= t *vmf, > - struct mempolicy *mpol, pgoff_t ilx) > +static struct folio *__swap_cache_alloc_folio(swp_entry_t targ_entry, > + gfp_t gfp, unsigned long or= ders, > + struct vm_fault *vmf, > + struct mempolicy *mpol, > + pgoff_t ilx, > + bool defer_memcg1_swapin) > { > int order, err; > struct folio *ret; > @@ -512,7 +518,8 @@ struct folio *swap_cache_alloc_folio(swp_entry_t targ= _entry, gfp_t gfp, > > do { > ret =3D __swap_cache_alloc(ci, targ_entry, gfp, order, > - vmf, mpol, ilx); > + vmf, mpol, ilx, > + defer_memcg1_swapin); > if (!IS_ERR(ret)) > break; > err =3D PTR_ERR(ret); > @@ -525,6 +532,124 @@ struct folio *swap_cache_alloc_folio(swp_entry_t ta= rg_entry, gfp_t gfp, > return ret; > } > > +struct folio *swap_cache_alloc_folio(swp_entry_t targ_entry, gfp_t gfp, > + unsigned long orders, struct vm_faul= t *vmf, > + struct mempolicy *mpol, pgoff_t ilx) > +{ > + return __swap_cache_alloc_folio(targ_entry, gfp, orders, vmf, > + mpol, ilx, false); > +} > + > +static struct folio *swap_cache_alloc_speculative_folio(swp_entry_t targ= _entry, > + gfp_t gfp, > + unsigned long ord= ers, > + struct vm_fault *= vmf, > + struct mempolicy = *mpol, > + pgoff_t ilx) > +{ > + /* > + * Speculative large swapin may drop this fresh swapcache folio a= nd > + * retry order-0 after backend or page-table revalidation. Keep t= he > + * cgroup v1 memsw swap owner until the caller commits the folio. > + */ > + return __swap_cache_alloc_folio(targ_entry, gfp, orders, vmf, > + mpol, ilx, true); > +} > + > +static bool swapin_zeromap_same(swp_entry_t entry, unsigned int nr_pages= ) > +{ > + unsigned int ci_start =3D swp_cluster_offset(entry); > + struct swap_cluster_info *ci =3D __swap_entry_to_cluster(entry); > + bool is_zero; > + unsigned int i; > + > + if (ci_start + nr_pages > SWAPFILE_CLUSTER) { > + VM_WARN_ON_ONCE(1); > + return false; > + } > + > + rcu_read_lock(); > + if (!rcu_dereference(ci->table)) { > + rcu_read_unlock(); > + return true; > + } > + > + is_zero =3D __swap_table_test_zero(ci, ci_start); > + for (i =3D 1; i < nr_pages; i++) { > + if (is_zero !=3D __swap_table_test_zero(ci, ci_start + i)= ) { > + rcu_read_unlock(); > + return false; > + } > + } > + rcu_read_unlock(); > + > + return true; > +} > + > +static unsigned long swapin_admit_orders(swp_entry_t entry, > + unsigned long orders) And this swapin_admit_orders chunk doesn't look good either... > +{ > + unsigned long candidates =3D orders & ~BIT(0); > + unsigned long admitted =3D orders & BIT(0); > + int order; > + > + if (!candidates) > + return orders; > + > + while (candidates) { > + enum zswap_range_state state; > + unsigned int nr_pages; > + swp_entry_t range_entry; > + bool admit =3D false; > + > + order =3D fls_long(candidates) - 1; > + if (order > MAX_PAGE_ORDER) { > + candidates &=3D ~BIT(order); > + continue; > + } > + > + nr_pages =3D 1U << order; > + range_entry =3D swp_entry(swp_type(entry), > + round_down(swp_offset(entry), nr_= pages)); > + if (!swapin_zeromap_same(range_entry, nr_pages)) > + goto next; I think you don't need to test zeromap at all? __swap_cache_alloc handles that already. > + > + state =3D zswap_probe_range(range_entry, nr_pages); If you just move the zswap_probe_range into __swap_cache_alloc and do fallback there (or maybe you can shrink the order faster), then this two new helpers are all redundant. > + switch (state) { > + case ZSWAP_RANGE_MIXED: > + break; > + case ZSWAP_RANGE_ALL_ZSWAP: > + case ZSWAP_RANGE_NEVER_ENABLED: > + case ZSWAP_RANGE_NO_ZSWAP: > + admit =3D true; > + break; > + } > + > +next: > + if (admit) > + admitted |=3D BIT(order); > + else > + count_mthp_stat(order, MTHP_STAT_SWPIN_FALLBACK); > + candidates &=3D ~BIT(order); > + } > + > + return admitted ? admitted : BIT(0); > +} > + > +static bool zswap_needs_order0_retry(struct folio *folio) > +{ > + if (!folio_test_large(folio)) > + return false; > + > + /* > + * Admission sees only an advisory zswap snapshot. Recheck after = the > + * large swapcache folio is installed; if the range became mixed,= drop > + * the fresh folio before IO and let order-0 handle each slot. > + */ > + return zswap_probe_range(folio->swap, folio_nr_pages(folio)) =3D= =3D > + ZSWAP_RANGE_MIXED; > +} > + Again, I think you can just probe the suitable size in __swap_cache_alloc directly, that way, we avoid the diverge of sync / non-sync device, and avoid the whole chunk making the code much simplier too, just like what we are alreadying doing for zero map in __swap_cache_alloc, or am I over simpliying it?