From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ed1-f51.google.com (mail-ed1-f51.google.com [209.85.208.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6C97A4611C2 for ; Tue, 4 Aug 2026 16:51:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=209.85.208.51 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785862328; cv=pass; b=WmvZHX4GZ5+HmjBrukFpiVq/Fr6IudksHozIX3ucqsT/g3jfcWozshurZmzrDBJU/Mn+kJPPnXpcFdc8HE1MbPmSDQ11ZU55YwBrXgPGHLKztqBElBE5ZDfxwdrHWoc/ClXzBrHdSc6gFt87q/ouT9/ezm0iSIToRKxGaFyHdU8= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785862328; c=relaxed/simple; bh=cuTYq4f+yjmc8EH2mpYMB3dT9yNAv5nLRDVolq8Y8KA=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=eKdSqpWjhcKT4FLnb6bOYXyGhCb7tOKPElr7Z831lIY/wj3cyzvv29LmH7EyguQlTXnOuAdodpa86y4dBhsczgefgc3hUUj3ya3XUUR4a3tRhp9EfY/2z11v7q/yLeKmp18SYw+01kznf//ml677PQzl1CaGE49tUHcjQvKFL10= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=AaHGfVoe; arc=pass smtp.client-ip=209.85.208.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AaHGfVoe" Received: by mail-ed1-f51.google.com with SMTP id 4fb4d7f45d1cf-6a14f118a46so88783a12.2 for ; Tue, 04 Aug 2026 09:51:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1785862313; cv=none; d=google.com; s=arc-20260327; b=CcEU1PXWuSISV97qfVsdQJheJr9eULoN0CkbhBfdjLLfj3hA7LjCgtPtX3AVV9MkfD hvZBxIm3N8a6eXU3pgDUs7PIB/w+lzJ2MUtXZx+DBmEQbNyeHtM2FEll1cKcIdh7JoB8 nTU/rtySUvMsDGF78lkM5t8O08SlNqxMO4nA/9NI4VnY/5hdvrCp5+x61J7oIA8LITVd kfXdbwk/druUTSrIjIIS9ZyF0WF4DNHUzrXGXG7tkH0J4xxgOYwCINODj4Z3SrzOk9pk 1IypS8dY8kjwS70u3IqKNLUwPydqBj8WJ9LF/XNz4pAJ/brAZSyCRLYqKpMvRWjM8VIq Gelw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=p2iVYbrotNhHUhqNiMpzDfrt6U5uyhbZkO3b8XSb8+4=; fh=svzfcCdL26kLss6apeohXOWYEeMKEM240AWcOQ0VEJw=; b=qIpDBJ9A/OELm09mIYYkVarRTA55yfv3a/LQy5S8n2G2+H+a2p9KuHt4pF+IcPBPk+ bn9uNp789LoESi6Ov8zD5Hj1/Lk7/JMveagSo5GZOgoSuanZVPzHd1msgLvwBrrS82Ie EYNASci4MdJVbVR96p1fIqype1mnpioWWuUweOEYRGMxOHujqZaZPVyUE7+HCF9MfZ8t 8Oq0faF3Z34K9e2KcNqZ03y70AYcbaloQHoQgx1pc57BORsO/NxcchEMB5T60bAsZfnQ 4ohZ3QlDfl2qBkuLLrraEqhdnBD42JJ8DhOVp+pffurmymBJQtRvgm7A4QMbq4J7vHo+ CC6A==; darn=vger.kernel.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785862313; x=1786467113; darn=vger.kernel.org; h=content-transfer-encoding:content-type:cc:to:subject:message-id :date:from:in-reply-to:references:mime-version:from:to:cc:subject :date:message-id:reply-to:content-type; bh=p2iVYbrotNhHUhqNiMpzDfrt6U5uyhbZkO3b8XSb8+4=; b=AaHGfVoeLdeGZBC99jqzlF1ugZqi/L8sAvcqKOnMetdELomtVEFrt3g3Lrh1RLcz0T Up89MxWPx5Xyr8SeD3LYUxQXjf0gIWMQen7iB8v4eilIY15E0cX2qHaB7cXEiqYZDTRg sRXH76Z/4v2IJAnMRiMOUbDXxPp/h7Eoq7eyqgP/pEs/C2wYh/D1ZWcE5MAk7oRSz8Yr 3eODaNkC3HrNst1s8y/OByMKZLTg+AbVX26ddebDH38TIOBSPUTPvK8UFxtjYDdWn2gc nK8djCkyfhEqbCYoOOczEudOB0wEHUPZK7BdCPKSqtP1d/80JYXCFiMy3Gvejsd1sz91 ssrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785862313; x=1786467113; h=content-transfer-encoding:content-type:cc:to:subject:message-id :date:from:in-reply-to:references:mime-version:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=p2iVYbrotNhHUhqNiMpzDfrt6U5uyhbZkO3b8XSb8+4=; b=jNR5TMartiVWdHTWzl6VqGxzOvEvCvPZ6wKuX9jP8oZVBBVAFZTU0zdd9lYwTsE22G 3QfRxNAS3k3Crk14QSygfIPFJH3IBQow98FfkovkFw51rdTPwtUlpon93lfxs8myoqnh 9ADt0vaeamYnITgfeS6Z2pfJkIBQrvK+SsQBvntuB48eGJzapeF6Xwv86THPIbOMlH/M ZdQc/nfKeD8SlKJ7siEVAW6vqQgpieIzeeo4eN+MjMKMEcq+5VQi9Cn0xOqXvBvjQ71/ QIvkeYkfA3Dgp6SehWeGYHoUxzZDVRRBrGOdqkzfVfDwmFF0RsUVj8xavkkXumwb9Mtu fnTg== X-Forwarded-Encrypted: i=1; AHgh+Rpd3PJmqIhv+cVi6Tm1g473f9fCsE2vDyxe/HdihFiLyEQq5+RuDDSM3Y9q9XWzt47c3YFzba/fJl10TqQ=@vger.kernel.org X-Gm-Message-State: AOJu0YyxR8+hhi7jT1cyU562xFqC9W+ZQyvvi1uJEcuN4KxyHxaZqgbI ab8hqrAqAebZv6WClDzm/Hu4BwrKJbVz30zlT+oK4NUH1bVN2FipaqH3dmUdZ10MwOwrf5XF6z/ vbLB+ckUe8ANEUSpal991ZwuUPCrjJ6c= X-Gm-Gg: AR+sD13MnMhRjbjd237Z3KYMyW6twJ6oYsPBYYa5WOjnnnKEk62OU8NH/4vcZCZQVk4 a1EgB9slE8ShKLIZjumL59uKoOLaNFqviGk515dK1ePcGwfBaeQvajOrgrhdJsmhuiTtP/AGqaH ZPo2nt3hzviu5DruASbkRDAWCni5UF2vpT80qPD157F/Hsw+w9aPTVW6gFzZGkJCc4lNqzWFN+v dDoEyjU40nnFv7NzRLSZDL5uE5vukqwKmtVe8sUKMd4wn/S2X2vdsA2laYLuj4if4nmwCDPQ8qr t1JLWHn8khZ37uustXV36K3iLKyQQgUR/Eee3rtiiiXWDhL4tzsXp0H+YFMkLb/vAwHXEM73YFy dfg== X-Received: by 2002:a17:907:9d04:b0:c16:67d8:7a0f with SMTP id a640c23a62f3a-c1fe80ff391mr1558163766b.28.1785862313102; Tue, 04 Aug 2026 09:51:53 -0700 (PDT) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <20260728155907.391820-1-youngjun.park@lge.com> <20260728155907.391820-3-youngjun.park@lge.com> In-Reply-To: <20260728155907.391820-3-youngjun.park@lge.com> From: Kairui Song Date: Wed, 5 Aug 2026 00:51:15 +0800 X-Gm-Features: AUfX_mx7jDWceoRxuaXwjHB4HW7vdXzWdZB72Y4eyTnnl4TDnwBQ1wa02h1r8aY Message-ID: Subject: Re: [PATCH 2/2] mm/swap: scan by cluster in find_next_to_unuse() To: Youngjun Park Cc: Andrew Morton , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, Jul 29, 2026 at 12:52=E2=80=AFAM Youngjun Park wrote: > > From: Youngjun Park > > From: Youngjun Park > > find_next_to_unuse() walks every offset from 0 to si->max, and swapoff > restarts that walk on each retry, so the cost scales with the size of > the device rather than with the few slots the shmem and mmlist passes > could not free. It has caused stalls before. > > The flat walk predates the swap table. Slot state now lives in a per > cluster table, and wait_for_allocation() stops all allocation before > try_to_unuse() runs, so a cluster that holds no slot in use stays that > way. Skip such a cluster with cluster_is_empty() instead of reading all > of its entries. > > Commit dc644a073769 ("mm: add three more cond_resched() in swapoff") > answered those stalls with a cond_resched() every 256 offsets. A walk > bounded by one cluster no longer needs that counter. The loop now runs > at most SWAPFILE_CLUSTER times before it returns or reschedules, the > same bound swap_reclaim_full_clusters() already scans between > cond_resched() calls. > > cluster_is_empty() reads ci->count without ci->lock, like the rest of > this scan. A slot stops being counted only after its folio has left the > swap cache, so an empty cluster holds nothing for try_to_unuse() to act > on and skipping it loses no work. A really nice optimization, thanks! > > Signed-off-by: Youngjun Park > --- > mm/swapfile.c | 39 ++++++++++++++++++++++++++------------- > 1 file changed, 26 insertions(+), 13 deletions(-) > > diff --git a/mm/swapfile.c b/mm/swapfile.c > index 5d15913dcf86..230abb276ceb 100644 > --- a/mm/swapfile.c > +++ b/mm/swapfile.c > @@ -370,8 +370,6 @@ static void discard_swap_cluster(struct swap_info_str= uct *si, > } > } > > -#define LATENCY_LIMIT 256 > - > static inline bool cluster_is_empty(struct swap_cluster_info *info) > { > return info->count =3D=3D 0; > @@ -2763,7 +2761,9 @@ static int unuse_mm(struct mm_struct *mm, unsigned = int type) > static unsigned int find_next_to_unuse(struct swap_info_struct *si, > unsigned int prev) > { > - unsigned int i; > + struct swap_cluster_info *ci; > + unsigned long i, end; > + unsigned int ci_off; > unsigned long swp_tb; > > /* > @@ -2772,19 +2772,32 @@ static unsigned int find_next_to_unuse(struct swa= p_info_struct *si, > * hits are okay, and sys_swapoff() has already prevented new > * allocations from this area (while holding swap_lock). > */ > - for (i =3D prev + 1; i < si->max; i++) { > - swp_tb =3D swap_table_get(__swap_offset_to_cluster(si, i)= , > - i % SWAPFILE_CLUSTER); > - if (!swp_tb_is_null(swp_tb) && !swp_tb_is_bad(swp_tb)) > - break; > - if ((i % LATENCY_LIMIT) =3D=3D 0) > + i =3D prev + 1; > + while (i < si->max) { > + ci =3D __swap_offset_to_cluster(si, i); > + ci_off =3D i % SWAPFILE_CLUSTER; > + end =3D min(si->max, i - ci_off + SWAPFILE_CLUSTER); Do we need the min here? Table size is always SWAPFILE_CLUSTER aligned. > + > + /* > + * An empty cluster has no slot in use, so skip it whole. > + * A slot is uncounted only after its folio left the swap > + * cache, so there is nothing here for try_to_unuse() to = act on. > + */ > + if (cluster_is_empty(ci)) { Hmm, it's not wrong, but this is indeed the only user calling cluster_is_empty without holding a lock, and not in initilization path, perhaps we should at least make it READ_ONCE? Maybe KCSAN will not be happy, I guess? Just nitpick.