From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-qk1-f174.google.com (mail-qk1-f174.google.com [209.85.222.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CEB7330FF2F for ; Sun, 16 Nov 2025 18:12:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.174 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763316758; cv=none; b=Xuihe4SLcsF5fjfu7ir2vmWJUdfm7W8C1L/cTJjcZphm0rB7PD9oWWTUEZGKZcInfOGS5QfBoKw5FBf6VfZhBBnR5WeNyZWX5AKiMLLNKbfVt0obBsLzNRQ0bxnuTGTIbWUQk8ds4t/iGqTUgNOZbGm4Jpk1ZRqaSuGViMlmLmk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763316758; c=relaxed/simple; bh=+zAChd3NkmTmvRBMsJuw/5gv+Pku3XixBwSNFCX4tkU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=qpazlBSQDDLthDttk7ktpaIm1XtM6MTFAJkftMJbO4hMv6pGaytE02bnx6mLVWhGGm/K5ypfO5uKrIF61Id0Ezuvqbvip96hkRhVQUUXYL0PP0JxWd+I7o9V7CWsG9GroH9mqt5TYLcPL4qMR4t0D5gxfLILsHLwFwDBjCh1V6I= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=m/0Pthc/; arc=none smtp.client-ip=209.85.222.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="m/0Pthc/" Received: by mail-qk1-f174.google.com with SMTP id af79cd13be357-8b23b6d9f11so359125485a.3 for ; Sun, 16 Nov 2025 10:12:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1763316756; x=1763921556; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=NNujllloQjbVXvXsSLn+SsAVbWylG5jsrz1jM3eZHak=; b=m/0Pthc/kaIiHURiYv52Pxftk7ryxa/QaEm2Y5ObksFLRCkB73tOXPQBe6Hs7c26IX ktJf+KdNO20wUHrcUaUuJLMzpPeQWOLrDJJJJ87aI/J4cAbAmrmNqhVYpLs+9NriPOcZ Tu0CRmhZK2cQh7ZuLaKSJ7R0pVjZL554CrrU0wQ7oA7SB4txVeqpD8SVeTDCWOB20Eva UygqwI3ZRRIQc55jVdy7/VWXcLP0sU66Y6sITyUlxaFZ83NTfBDcE6d8XYMNuUWaooKw oVqQdjzP1qTqfHGA03RzvW+GfpoiBLwb2b2CMseqd6ogbWj06jQmhZV5Fnq069BLrS+V kV1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763316756; x=1763921556; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=NNujllloQjbVXvXsSLn+SsAVbWylG5jsrz1jM3eZHak=; b=KeZFkinVb8eq9ya4S5gbtJxebUNDsKtZPrnudaZOLPQ5alhBxABtazTcjNVuVGMx2h cJNa02A4ijkrgHUEnsJ9Qo1YDIRWCnVW0xNKKga4si45msZGHXC47Yq/hkm9xqTQar4Y b7+LY/r4IoGlKi0ZQXxFXtn/rcly/zsDO+cFt6pvQBlYtEAk8QilRMTm3GrscXNG5T7m nwFSk6r+MPSlCS2RMSITh0NAzHPFcN9cGQjM5TkonaAI7fkRkwpM+aN8F0em5Z1PXeHd e6xGZUfxdoW+4AAbuKPvmfZ+Qh3GAslDTyKL/6mUImtyEbFvaetxmlGlhUnvFVM81ui8 8lIA== X-Forwarded-Encrypted: i=1; AJvYcCVdQ5L/V8Lj4KLoMH0tLHxvT6XAwqRMavrLG34VFUme36bbNlydw9OZA3NKjGmxRTiUKTP9ksvZlYGfFO4=@vger.kernel.org X-Gm-Message-State: AOJu0Yw93sGMksJH8wQJ4qszyazeWMvr/umpa/Qhy6VF0ro8ztBn8eGH G/V9tN6ZxhMUTJqFrp2yWW2bPjO07KOZkGNtlSlNwWOk4p5XALXnbQJk X-Gm-Gg: ASbGncuX62+Mfur5XMXD4FQ3CqZRI96B5hj0LleZjSPdNRPsKRrhroZJJjR4KZXUCJO NxjOt6GkqbSET1X5AH3iZ21b27WFEKHu19j7moGX43YzlkL0d7AREi5LKmTYWVRAfht1vNBBlkB CmK4eAXRhaiOzm4tB3eGNaB0r8sm2PmNYUh9aDpYzrpfmzz4wVdWkLAU+EwRCNJlX8AtJIgGytX 7pwnPGUfVgDzo8Wm2tK099AIpkzNVyW9jmH74sY1vw0Mr8Mjh6OIqaDTsI9bM9G6LZyByV8M4Ou uMG8Dzmh47KOjcMVZ5T+vpKouKpEMuzsXYC23lSnmezC9Bv6P3BncKgxjP8RTdIKlQls3U+v5r/ NeAyqDtalPBIHXYq6zPE9TL8dAdUY2N+gU4S6SY4+o318UNeJFa9QJCBJsXDu4s1rDcitdcxhjM ajkAK5+iSZ6BkQrsfJeof2i0hlKbtswcS5BEelabvuDcnK3v7RQWc9 X-Google-Smtp-Source: AGHT+IFBHwKki63O9ZEooguIYvPxGgh6LHO5HKXPWrs1r7j2lrhxsDaVOFKFEdD+jVmLK9V7yU1m4g== X-Received: by 2002:a05:620a:4003:b0:7fd:50bd:193b with SMTP id af79cd13be357-8b2c31306c7mr1217791885a.14.1763316755663; Sun, 16 Nov 2025 10:12:35 -0800 (PST) Received: from [127.0.0.1] ([101.32.222.185]) by smtp.gmail.com with ESMTPSA id af79cd13be357-8b2e089b436sm305447785a.45.2025.11.16.10.12.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 16 Nov 2025 10:12:35 -0800 (PST) From: Kairui Song Date: Mon, 17 Nov 2025 02:11:47 +0800 Subject: [PATCH v2 06/19] mm, swap: free the swap cache after folio is mapped Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20251117-swap-table-p2-v2-6-37730e6ea6d5@tencent.com> References: <20251117-swap-table-p2-v2-0-37730e6ea6d5@tencent.com> In-Reply-To: <20251117-swap-table-p2-v2-0-37730e6ea6d5@tencent.com> To: linux-mm@kvack.org Cc: Andrew Morton , Baoquan He , Barry Song , Chris Li , Nhat Pham , Yosry Ahmed , David Hildenbrand , Johannes Weiner , Youngjun Park , Hugh Dickins , Baolin Wang , Ying Huang , Kemeng Shi , Lorenzo Stoakes , "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, Kairui Song X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1763316712; l=2699; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=HEaEGHmPSVwIYafbKFzg3TdTU/AWpN5SLIcfoADgF+k=; b=rOTAx2cwYpJO5/jLxllc/HUaQOchTua/fMcbMU4PSzUf/z0sYlbdLmmKo57P+f48bdG+uzgae MIysZK9BagGBmrvlbWD6/S76/u62ShcYaULdXWDmf8sUKpLLz7U2u95 X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= From: Kairui Song To reduce repeated faults due to parallel swapins of the same PTE, remove the folio from the swap cache after it is mapped. So new faults from the swap PTE will be much more likely to see the folio in the swap cache and wait on it. This does not eliminate all swapin races: an ongoing swapin fault may still see an empty swap cache. That's harmless, as the PTE is changed before the swap cache is cleared, so it will just return and not trigger any repeated faults. Signed-off-by: Kairui Song --- mm/memory.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 5bd417ae7f26..022ba4596fd5 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4360,6 +4360,7 @@ static vm_fault_t remove_device_exclusive_entry(struct vm_fault *vmf) static inline bool should_try_to_free_swap(struct swap_info_struct *si, struct folio *folio, struct vm_area_struct *vma, + unsigned int extra_refs, unsigned int fault_flags) { if (!folio_test_swapcache(folio)) @@ -4382,7 +4383,7 @@ static inline bool should_try_to_free_swap(struct swap_info_struct *si, * reference only in case it's likely that we'll be the exclusive user. */ return (fault_flags & FAULT_FLAG_WRITE) && !folio_test_ksm(folio) && - folio_ref_count(folio) == (1 + folio_nr_pages(folio)); + folio_ref_count(folio) == (extra_refs + folio_nr_pages(folio)); } static vm_fault_t pte_marker_clear(struct vm_fault *vmf) @@ -4933,15 +4934,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) */ arch_swap_restore(folio_swap(entry, folio), folio); - /* - * Remove the swap entry and conditionally try to free up the swapcache. - * We're already holding a reference on the page but haven't mapped it - * yet. - */ - swap_free_nr(entry, nr_pages); - if (should_try_to_free_swap(si, folio, vma, vmf->flags)) - folio_free_swap(folio); - add_mm_counter(vma->vm_mm, MM_ANONPAGES, nr_pages); add_mm_counter(vma->vm_mm, MM_SWAPENTS, -nr_pages); pte = mk_pte(page, vma->vm_page_prot); @@ -4995,6 +4987,15 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) arch_do_swap_page_nr(vma->vm_mm, vma, address, pte, pte, nr_pages); + /* + * Remove the swap entry and conditionally try to free up the swapcache. + * Do it after mapping, so raced page faults will likely see the folio + * in swap cache and wait on the folio lock. + */ + swap_free_nr(entry, nr_pages); + if (should_try_to_free_swap(si, folio, vma, nr_pages, vmf->flags)) + folio_free_swap(folio); + folio_unlock(folio); if (unlikely(folio != swapcache)) { /* -- 2.51.2