From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from SA9PR02CU001.outbound.protection.outlook.com (mail-southcentralusazon11013055.outbound.protection.outlook.com [40.93.196.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A5903A9639 for ; Wed, 15 Jul 2026 05:14:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.196.55 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784092494; cv=fail; b=kGUihKhZgpy70IV5NTyp5bwk5ckCsgIJ7yQqm24CaKYBRyD6Sbu9zpcBhQfafrZa9ZSlmF4xnIG1T8TWw14LbTBTZRuQC/NzNd38ZnN2CKX32NM7HeLfjjaUEjm8ZNoxpMFJcmLeSguo4yNKBzHXUNNRcMLLQRR/xxctkFOfw90= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784092494; c=relaxed/simple; bh=nNdKRwpNsfppWRzCU3nh7yTWKy+wrdeyk03B8tHJ4Ss=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=XzTWR9scWc+xjWIXinnmhyFetbQi+LEfkHoknorWSylYGsb4HlcgOfM8GQjQ+WRsdBtMdiaW16DVduSXF2TA2RkKn/gDsHBJfzqZNN7unTouajbI3977u8IOi/QzerlnsXI4ckJ4rT40oRJErK7Cwxih8Tg87GRHz+UIBDFyyNk= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=hz44F1+7; arc=fail smtp.client-ip=40.93.196.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="hz44F1+7" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=g9gJKcNmlkL32XX2v3mpoMq8ynSPQLJgF85ckds+VYj7bE7P236NEkaYBNuOMSCclrVJLi0izSBu4h3801T6Z1Uel6k3PlRm1ir/U9wr0t8wv5OLfuQQkm85rBe3Ryhdssa9JwOb5t2+aRIoNEW/jzgTH4cODlaisMtkQRQxlMrd7q4u56BGe4pXOl/iJiVpY+FhKTelmwVCKmIMO8iGcQ3ZGhezC5hxCBeQXtDaDJ6bkREMgH3cRQzqykzFyvv5f2i8KikHa93JisqKH/OyAiUbh0ppcIiop6c9FngZfzS/eDl+sC97V4am39QZ8uTe6rPM82LgUMn8c6bl6U7ikQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Or/ESv/wXRCFuXMTCdXCBrHtMHZFfdgahWzegeBl+7U=; b=MnojE1ERQOvQxy+FuMNHmidK/Kc21zAt8ogPmDalfPRSvUTsC2tHgP65J5kSh1IlpBPkwSnCX3ZVLJEGuj0Er91jkiUcaeO0Uij+nYSgUg8PvDOJ3D87Wvq5lw8iHbdjsiw2p4gm/+UqFl1R4jQ8FUsW0Dv+n91X1me13uefN3NVxoCw20lCIqHzGBH8A7+BnQYUWHwYhoPQbF5bqm1zBmbOcQHFEk/x7JQNdsayeZ0RQwQUirw89RnYRVKrfFmf1bMkZMeLu17vVwohKAt3eQdPRURKl+cyHF0R5zaRLCrMddgmMEg7YpXazkezKfZ0hc4wIZb0O/hd9AiTdQsvQw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Or/ESv/wXRCFuXMTCdXCBrHtMHZFfdgahWzegeBl+7U=; b=hz44F1+7NLueQppkDKpbOHWMDAIix7g6LPVwoQMkOx/Hi0jqSFCCdAi6D1UZi4QDmYJbFRUUG/6+Hf8qIYpWKXpnpaBFMpM2J0xBBE4e4oAPTxJoMbv8YaPghi29gfwqA5UDVNcchRDd5qJ1qvDZk5sR+m8sohUpdel1vKSlCxw= Received: from CH2PR18CA0011.namprd18.prod.outlook.com (2603:10b6:610:4f::21) by SJ0PR12MB8116.namprd12.prod.outlook.com (2603:10b6:a03:4ec::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.202.18; Wed, 15 Jul 2026 05:14:47 +0000 Received: from DS2PEPF00003447.namprd04.prod.outlook.com (2603:10b6:610:4f:cafe::91) by CH2PR18CA0011.outlook.office365.com (2603:10b6:610:4f::21) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.223.10 via Frontend Transport; Wed, 15 Jul 2026 05:14:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by DS2PEPF00003447.mail.protection.outlook.com (10.167.17.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.223.9 via Frontend Transport; Wed, 15 Jul 2026 05:14:46 +0000 Received: from [10.128.124.134] (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Wed, 15 Jul 2026 00:14:39 -0500 Message-ID: Date: Wed, 15 Jul 2026 10:44:30 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 0/6] mm: batch rmap walks during large folio migration To: "David Hildenbrand (Arm)" , Andrew Morton , Lorenzo Stoakes , Rik van Riel , "Liam R. Howlett" , Vlastimil Babka , Harry Yoo , Jann Horn , Lance Yang , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Zi Yan , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , "Alistair Popple" CC: Karim Manaouil , Frank van der Linden , Kinsey Ho , Wei Xu , Bharata B Rao , David Rientjes , Dev Jain , , References: <20260712-migrate-rmap-batch-v1-0-872a734431d1@amd.com> Content-Language: en-US From: "Garg, Shivank" In-Reply-To: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS2PEPF00003447:EE_|SJ0PR12MB8116:EE_ X-MS-Office365-Filtering-Correlation-Id: 2a22b5cf-86f5-4077-5b0e-08dee22ffd11 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|7416014|376014|1800799024|82310400026|36860700016|18002099003|4143699003|11063799006|56012099006|22082099003|921020; X-Microsoft-Antispam-Message-Info: JTMV7BjsgkFLRABbqaIrD0aRO/MTtFecgf2Sw8toqg/WmWUYwQKqlu/yMpt3AVmprIROeZZCXfK+K61yh1TgpBUcRX9WHDIFlz0IslNz3nWBs27VmcbfSsFDTSDcB+GdAMzJJl+43r8HoD4j8FxL/ULTnhPnFUquwChBkUkYlGA42W/V+wUwHT9neu3nfWv/sPaoYq6UN00vzH+o03VAIto7L1IfEIkyO0cIZsAV0QpZkRsUGn87Z0wofAi8sOMr93bMvqI4U6V6IT+JcGf9W23UZigb8RnmUxOOFasMUNZ87PdrEVG7FKMxO4C3oz5F9TKLLiG5o3DnOQJWigQZCw0+ELvpxniSw/X5FXZniSaUVOuYmVzuQcgFZQLaiSxZqkFliDS/w1RpKg25ZWMJ/E3T8HqN9K5fwF06dmVIeARAGVDsMTJZM5K1yoKYLMogwDXbXqjR4cAxj76ycRPIxgrznrHd3btbpyM49SibxD21FrZ9Hqajr6GDkIpIoJ8IiIPVnzwliQXJHs6Zl2GKwofAx4Hhi2tGFrnUDyirkspEEO3YTDOVYUhwwtnCg1FAsIg6EBZU9oSgC4LOQFNVGs2CCkfAc4KuT9MOdPb0hIZBXTya5uFA+WBoOq037dopkRT3QCVn6tY0YLUL0p7qoPe2BHVUN8mTX6cnBbgxHuK6htP6DjgxECC/Hnh6i34d1zzcxKyMG22+FIWXXFaMYunfeaQgt1oWAYEB5FoVzcCAv/Q7DxeCHNjjAKIMFLfi X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(23010399003)(7416014)(376014)(1800799024)(82310400026)(36860700016)(18002099003)(4143699003)(11063799006)(56012099006)(22082099003)(921020);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ILVRjR8FTpE5v+TessQKh/nFbr5LIXzO3E9hEGr8Ei6/NJJTxR/iBX261E4xSfRyDdkwrzSrFYA9a+5a/OlTA9E5byzy9DRY9OLQFmfECRZby1/szyFaZJGo5AxkAoZEQFsUP07zphiOA1Xjt3EiV9VAjE4oTg0nGqbMcGkqxqh0aztW70gOIW66gfPyuGH+wtRIUkJPh1KD4Sdh1S5+XEW5N+AmJtKX5ljzOQuHsgP+CpTjZiznMxmowF6tU7LlKTWZZCD7tybe1cbqteMknJY6zoe5+RO1fCh9zpJ/YWi8Qrb6ggUBTCaBUFakTaDZaBZVu7GLcuhSJjA2rabtkR9FjVbhpRO+gQe/lVjdeXi2la60h4JHJnLZfckVfEdiUT3p1SuRhqo71nx2uqc20PIy3e7VNtT6oneRFL0EFiCv/EoRPPSbEGnW2tH17ovF X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2026 05:14:46.9496 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2a22b5cf-86f5-4077-5b0e-08dee22ffd11 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DS2PEPF00003447.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB8116 On 7/14/2026 4:25 PM, David Hildenbrand (Arm) wrote: > On 7/12/26 09:38, Shivank Garg wrote: >> Migrating a PTE-mapped large folio currently walks the folio's rmap >> one page at a time. It does so twice per migration: >> 1. try_to_migrate_one(): Replace present PTEs with migration entries >> 2. remove_migration_pte(): Restore migration entries to working PTEs. >> >> For a folio with N contiguous PTEs, each walk repeats page_vma_mapped_walk() >> and the associated operation per subpage of the folio. (256 iterations >> for 1M folio). >> >> This series enables both walks to process a batch of consecutive PTEs >> that map the same large folio and do some refactoring and clean up on >> the way. >> >> Performance: >> >> AMD EPYC ZEN 3, 2-socket system (1 node per socket). >> move_pages() migrating 1 GB from node 0 -> node 1, throughput >> in GB/s: >> batch- batch-restore + >> folio vanilla restore unmap restore% restore+unmap% >> 4K 2.98±0.09 2.95±0.07 3.06±0.05 -1.0% +2.6% >> 64K 6.16±0.09 6.24±0.08 7.80±0.08 +1.3% +26.6% >> 256K 6.85±0.06 6.85±0.09 9.54±0.19 +0.0% +39.3% >> 1M 7.06±0.09 7.17±0.09 9.89±0.06 +1.5% +40.1% >> 2M 10.97±0.17 10.67±0.13 10.69±0.15 -2.7% -2.6% >> >> In plain migration, the folio copy itself dominate the cost, so these rmap walk >> savings are only partly visible end-to-end. >> When the folio copy is batched-copy and offloaded to DMA engine [1], the copy is >> no longer the bottlneck and rmap-walk batching translates into much larger gains: >> >> This series + [1], DMA Offload on PTDMA (DCBM), 16 channels: >> >> batch-copy >> folio vanilla -offload(dcbm) dcbm+restore dcbm+restore+unmap >> 4K 3.36 ± 0.10 4.16 ± 0.18 4.10 ± 0.14 4.12 ± 0.13 >> 64K 6.16 ± 0.47 9.63 ± 0.23 10.00 ± 0.33 14.63 ± 0.14 >> 256K 6.70 ± 0.09 12.98 ± 0.18 13.52 ± 0.15 32.30 ± 0.65 (2.38x) >> 1M 6.99 ± 0.02 14.32 ± 0.67 14.84 ± 0.17 38.97 ± 1.03 (2.62x) >> 2M 10.74 ± 0.29 64.25 ± 1.48 65.36 ± 1.00 64.97 ± 1.75 >> >> [1] https://lore.kernel.org/linux-mm/20260630-shivank-batch-migrate-offload-v6-0-da95d7e8b8a2@amd.com >> >> Applies cleanly on mm-new 10-07-26 (61cccb8363f). >> >> Signed-off-by: Shivank Garg >> --- >> Dev Jain (1): >> mm/rmap: Add batched version of folio_try_share_anon_rmap_pte > > This would ideally be based on Dev's series instead if including a patch from > his series. > Sure, make sense. I will rebase on this. I initially thought that including this patch will make series complete to easily apply and test. Thanks, Shivank