From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 52FB53D9541 for ; Tue, 28 Jul 2026 15:59:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785254363; cv=none; b=BdcX1ZzRgD+7gSPjyvnPCT4U3I13XHo+Lvj545u6Zc8kZb61WCsLNmAzRShD+Zu96KEZuCtNsg8hOe1P8v1NYEm36fOWtHsN98ISx4EFI0Q0gGuUGgfPT7xZooqTBJ5IrUTOJG2mFOoiMS/6CZ5Er5x36aRq2gD1JY+kkPBRz8k= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785254363; c=relaxed/simple; bh=WOlaw+0U2auLpezgW4PZ5llhEZ7IYFm9TX71rW6wexI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=b4rD/ZQNeUIiVTJjRXA8eHbJkULgZJRWKkJahEJVGkyw0JJYYb7LD5zj0FWuDSeszgmBMqQnMA9MsYowIDn+p7IB1Cyuhje97qbDHzV3F2DPcIhZ8il0IrAH2z1gmz5w4Arnf8XjpdLhoSMD6IA5JmfIb4700wPs14iQdciQtkc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=DXW5OxvG; arc=none smtp.client-ip=209.85.210.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DXW5OxvG" Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-84874b52eabso14768b3a.0 for ; Tue, 28 Jul 2026 08:59:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785254362; x=1785859162; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=jdc520hStMu+f+3jSqbQBLlztrWg7wz8IxJRE68s3LU=; b=DXW5OxvGgd9QPJveik7847FBHTwvwiFzZ4857cwXxk1xmQyz7Xd0r9rZ0Nn4s5cnoG Rqx6DrwYN2/BPBTKuLRTjuTyjdtsR7NAl0D+dvHxDsU4lMKhBRXIloOOoXnSNy9njdx9 VtYKelBvRICzXRpd6dCZpW4AVqtn6qaI59+TYJ3QZ0GGOz/fQEPo4NQ75GkQY6bdoXxM B0d2uUI8nOdNPxdQsywkAyiCEwBO//TAAGrBOFM040qXi+pXY8ro7MbNk0X86L9R41bn GV0F9Kbh1UTudMECRE/8H/W1xGCfQU03yocKaCD7OsJg9wtBIfSdVy5TNLcVk1bUBh9G TGxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785254362; x=1785859162; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=jdc520hStMu+f+3jSqbQBLlztrWg7wz8IxJRE68s3LU=; b=VZhQ+MrRpvDVedLDvAtIr55/66d9GHhO7m5UuLVw+oy6879dAs2oAlQ5E0SXC7u1+N zzYpNqG34UowjP5c1lIorrGm2ZZY3OSqiprLQlEGu0FkeFkSuutpDYX5ECvw/fKokcij 37w7sFrG9sdhLGv/FPNg9usdqtD6YpwUhPTxH8LMSu0T7BRhJe9kWNLuIDVW58VHH2NL GMc7+U8njVyqZdrmnnvr7NrxiDmv7JRheO/zmFFEh5HAEJ8ZSzq7S++21dH+2hTPlCTR Kw5ylFFmp9lIeaO33izQI/MN0YVwWtxgDd4TbRSzGdy0mlF9ArM6NVoHRb7EFhjpmxHj KoIw== X-Forwarded-Encrypted: i=1; AHgh+RqGxeSakbcNKLWn76ruDMjY8qtsrsR6cgUZPkgAGG8U9N0M9Uzp0hddk2eLe4knwGOzna2C1srlXWS2fsw=@vger.kernel.org X-Gm-Message-State: AOJu0YxDlSVZe5cfNdh1dV5zeXtg4KHIKtk7EnVL5GAIo1WVPqX6cVr8 n6LjARq5Eh9RrXZxWdi/l35vqvyFle9MPooIVyRYDZdlEu+bmRJucq+T X-Gm-Gg: AR+sD12OgouJMf5llZfxRNaKAdbtdMnPVx0ox4vVD7I8QjJm+whjj/KWrqBvsoOWvA3 ih7WRMlgXIkgkYWEr9k3E0/R36IVl13M7+6FCaHkC0BW6v6gBQa5jfO1t519gvZDwhw84W/uB/t gPDCajWZ/WnDDusxH3tscnBASN42lFlW9dZjZv8ErNv2POxSW/nR9KEC6CPo4b7vHP4XtVmTkIT MG7PhHR9HskcZzZ+Wk6koGjVkz9l+luTd2qeUC1WkfzdPpV4bQi6+mSNiYDMVq7Y8+g8gXUkUz9 9C/4v46cNOJdfDdqHK3KcY2nh/YerbbPajXKWi41/SBUiNTwA8J38K0PBzhgmRz+HSZDp1VXPFE mp+GS2GbbwP06oKdEAgt6oka3DXLc/fwjIxHCNF1G81WqZ+1PwDN8NDKs/VNbGmD0aDVKos5fgF ft6V63KmkV72OpMUtH8mo1/wPInw25SEhYFjBovVhm6kkNcFBNLKFdVM2zZcjiy5k= X-Received: by 2002:a05:6a00:1d83:b0:845:4d71:8d15 with SMTP id d2e1a72fcca58-84e9331e137mr3522058b3a.37.1785254361511; Tue, 28 Jul 2026 08:59:21 -0700 (PDT) Received: from localhost.localdomain ([220.85.166.190]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-84e9fe2393csm156441b3a.10.2026.07.28.08.59.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jul 2026 08:59:20 -0700 (PDT) From: Youngjun Park X-Google-Original-From: Youngjun Park To: Andrew Morton Cc: Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , her0gyugyu@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/2] mm/swap: scan by cluster in find_next_to_unuse() Date: Wed, 29 Jul 2026 00:59:07 +0900 Message-ID: <20260728155907.391820-3-youngjun.park@lge.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20260728155907.391820-1-youngjun.park@lge.com> References: <20260728155907.391820-1-youngjun.park@lge.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Youngjun Park From: Youngjun Park find_next_to_unuse() walks every offset from 0 to si->max, and swapoff restarts that walk on each retry, so the cost scales with the size of the device rather than with the few slots the shmem and mmlist passes could not free. It has caused stalls before. The flat walk predates the swap table. Slot state now lives in a per cluster table, and wait_for_allocation() stops all allocation before try_to_unuse() runs, so a cluster that holds no slot in use stays that way. Skip such a cluster with cluster_is_empty() instead of reading all of its entries. Commit dc644a073769 ("mm: add three more cond_resched() in swapoff") answered those stalls with a cond_resched() every 256 offsets. A walk bounded by one cluster no longer needs that counter. The loop now runs at most SWAPFILE_CLUSTER times before it returns or reschedules, the same bound swap_reclaim_full_clusters() already scans between cond_resched() calls. cluster_is_empty() reads ci->count without ci->lock, like the rest of this scan. A slot stops being counted only after its folio has left the swap cache, so an empty cluster holds nothing for try_to_unuse() to act on and skipping it loses no work. Signed-off-by: Youngjun Park --- mm/swapfile.c | 39 ++++++++++++++++++++++++++------------- 1 file changed, 26 insertions(+), 13 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index 5d15913dcf86..230abb276ceb 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -370,8 +370,6 @@ static void discard_swap_cluster(struct swap_info_struct *si, } } -#define LATENCY_LIMIT 256 - static inline bool cluster_is_empty(struct swap_cluster_info *info) { return info->count == 0; @@ -2763,7 +2761,9 @@ static int unuse_mm(struct mm_struct *mm, unsigned int type) static unsigned int find_next_to_unuse(struct swap_info_struct *si, unsigned int prev) { - unsigned int i; + struct swap_cluster_info *ci; + unsigned long i, end; + unsigned int ci_off; unsigned long swp_tb; /* @@ -2772,19 +2772,32 @@ static unsigned int find_next_to_unuse(struct swap_info_struct *si, * hits are okay, and sys_swapoff() has already prevented new * allocations from this area (while holding swap_lock). */ - for (i = prev + 1; i < si->max; i++) { - swp_tb = swap_table_get(__swap_offset_to_cluster(si, i), - i % SWAPFILE_CLUSTER); - if (!swp_tb_is_null(swp_tb) && !swp_tb_is_bad(swp_tb)) - break; - if ((i % LATENCY_LIMIT) == 0) + i = prev + 1; + while (i < si->max) { + ci = __swap_offset_to_cluster(si, i); + ci_off = i % SWAPFILE_CLUSTER; + end = min(si->max, i - ci_off + SWAPFILE_CLUSTER); + + /* + * An empty cluster has no slot in use, so skip it whole. + * A slot is uncounted only after its folio left the swap + * cache, so there is nothing here for try_to_unuse() to act on. + */ + if (cluster_is_empty(ci)) { + i = end; cond_resched(); - } + continue; + } - if (i == si->max) - i = 0; + for (; i < end; ci_off++, i++) { + swp_tb = swap_table_get(ci, ci_off); + if (!swp_tb_is_null(swp_tb) && !swp_tb_is_bad(swp_tb)) + return i; + } + cond_resched(); + } - return i; + return 0; } static int try_to_unuse(unsigned int type) -- 2.48.1