From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from MW6PR02CU001.outbound.protection.outlook.com (mail-westus2azon11012026.outbound.protection.outlook.com [52.101.48.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A1FB387355 for ; Mon, 23 Mar 2026 09:59:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.48.26 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774259997; cv=fail; b=kdnRBETEsnwH6P3/Kr4Jv9GvNlM6rph4uE6o+lqrKVTYxPen+LJuqhYZcXtJGyzZoM1iNGZ6GlTFCC1c8bU5x8SJMzdqY5AFSdgMhB0H0u/2dhg13PGdV06wmSZAMVMoxJUAofXCpnfBrWdqf7B97DuTbyCFmujD6Mh29WmTta0= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774259997; c=relaxed/simple; bh=IZNZmBrBOP8IphkxOC1ECcCLRy00dSCmm8OCbrl4Sns=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=ZblT/xG1fF0MNnpZhDnzGKWcue4QjMpny13rYn2uo9fv0Lowsh5/KUhmV/CQZiMfdPizoYT+LDB48XtA2aTqIci/8u5CJ8aqt6W8AXa0DsubA3uJvXLJ385dCoI82mfPm2Z1InMPQXK+ppMuvzhKQxcEb145KBWVSZ5dfrA+K+8= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Dn06zf3W; arc=fail smtp.client-ip=52.101.48.26 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Dn06zf3W" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jnO9+r7scsx+syBkl4LY+5aBcvUj5zyATtOg+SrBqh+wh9xBr4LJyCx1hssFbNLJYQRZ5T4YQXa1y+h0WoIzimweaJho32xjohkS7u/9LzzWzE3rZ1ryvd4p4zg2SP2auoPo+Z5Ux4As9bdBP7jkUDpv/6C5A1sbrd0Vi687GDE9/hUQQAzkQxuFxr//OjflG377T8I6i0gQPk0rWFm23Kah/TjsLVemVQHcYM4unGhGd50WhaWrX0wIalLT3V6ZQBL5/n4hN9mhr3z5sXsW4KdlfBYyL4nc59ulHWa4TnUa1nMtEqtsF39TYr6dnhIc/jm5WQmvV5Dp2VL2VjLVsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=q+t6u7HmXxcXn/fSNHIDeUa3r0UOm6bGHyLQcDTMFvg=; b=DbWDl4ZMQ3lAucy3tzjHL9dW9TvUKBpPtvCwKRrkkYGsy6CEAWsqjasUlmSG2Z09K2b1q7yDM7wLZ3Jz0BDbKwPNBbh4aBaL+tng273qs1zgYo2qxqEdNJXpeKjMJuIaGpwOj9I26jpOMR6QzDk62jDeZ4rhbpueRfvatGZYrg6NBI52wmpdfzkvZ10iX7jQMfEKwI2vLGt3qrAbcAd1he+tkV1DsAN2+1INAMxmislOMnSrpBfFPwP14lsRUW1ZpQf0b+gATaXDiypAA6XKNxww1yxLAQtE5HpodsKMyDmAfGZJftLFFLfhYh90gygLwnmX46ngjWaxBUsrVCx0iQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=q+t6u7HmXxcXn/fSNHIDeUa3r0UOm6bGHyLQcDTMFvg=; b=Dn06zf3WjwDhIlt7KWicYAeJ/L4X4e57y/j4u62a/FokrXpK2ND+rLvcNBpbqDmKqCAhlATM4us5VCSHQ269r/AOAuKCjywHBF0uZ7IlmDpWJsuFMeMlMS76nnyVreE6GAU37FcSFG+e5/kqw3S6jwdlT/M9+m9xcwyRkKzGjVA= Received: from SA9PR13CA0134.namprd13.prod.outlook.com (2603:10b6:806:27::19) by MW4PR12MB5625.namprd12.prod.outlook.com (2603:10b6:303:168::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9745.20; Mon, 23 Mar 2026 09:59:52 +0000 Received: from SA2PEPF00003F66.namprd04.prod.outlook.com (2603:10b6:806:27:cafe::9e) by SA9PR13CA0134.outlook.office365.com (2603:10b6:806:27::19) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9723.23 via Frontend Transport; Mon, 23 Mar 2026 09:59:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb08.amd.com; pr=C Received: from satlexmb08.amd.com (165.204.84.17) by SA2PEPF00003F66.mail.protection.outlook.com (10.167.248.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9723.19 via Frontend Transport; Mon, 23 Mar 2026 09:59:52 +0000 Received: from satlexmb08.amd.com (10.181.42.217) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Mon, 23 Mar 2026 04:59:51 -0500 Received: from [10.252.223.214] (10.180.168.240) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server id 15.2.2562.17 via Frontend Transport; Mon, 23 Mar 2026 04:59:44 -0500 Message-ID: <7deb1c5b-49f6-41a4-b8c5-1cfba14488f0@amd.com> Date: Mon, 23 Mar 2026 15:29:38 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH v6 0/5] mm: Hot page tracking and promotion infrastructure To: , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , References: <20260323095104.238982-1-bharata@amd.com> Content-Language: en-US From: Bharata B Rao In-Reply-To: <20260323095104.238982-1-bharata@amd.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00003F66:EE_|MW4PR12MB5625:EE_ X-MS-Office365-Filtering-Correlation-Id: 4a95ff5b-651c-4345-1a16-08de88c2ed86 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|1800799024|82310400026|7416014|376014|18002099003|22082099003|56012099003; X-Microsoft-Antispam-Message-Info: zePQY9qgtHUqyPMkHu9fH2nn04rnbS+m+leOUe96bWa1Id4pF7fwYkGpwuG0jD7yhdVcVwnqKe4XJ0dAtX4DN665cVxMp0Bjg9wPJq3Mz/FW7en5cDdRT5w9eMQPj9dkulDQ8D01V2a6QXI2Btxoh4CKUe9zQVT+fkriFLUmkdSCOHdQQU1kuG8BcqEoPP6Y9vSI5K0DMZ3F5eJM3uv+cGHw8KpRlPP6LwB12ld8u4FeKkweDxmfyXrKa/tUyQwS2b/QZ3mfkRpQOrHn8NbAFc8VxohwPyYuz8ERKYFx60L67er9H8BHzK7ZnCWt1+qdtrcsbCd7lPjPKTtVf0u1rrTkqxoAEdl4afC+Pga5o2pwMsIUbzWC08wqh9F5+urLXnuNy/LLHfDivYw1sVzt7Np4W1XyRVCdMlPQ4dvnVuHjDNW/aGDyNzxGZtcZV7nrQmcL8Uc6dQEavO+XHURyjxU8zelzqWzjLjDHNFvTQjdjbR7aN8cwolih/oK22/XEAm/kpunStck+lIbHTBo+s5PlqMwrNjhPdnUrtwLomA42bLg16g5ldnDRhwuqmHQEhTpPp6JqiVGTo79HHJU+haFLSLcW2AaOLgkOdTip2lv4SgZHXn/PXAyzLQzwIn7vzd5RKYparhlipcPevd4bb5DMbYERFzOGGrPLhxi7Jxmqbq/zpUC4KUhZ7t5ugvm1j7CiNyzVTSW+TKmlVCkf7HXNxObembSBuFgjOD057Rp5JQcxZWBuACgHxJ4NgQbo0Fi1MjrKkyF9IfGlMIpTHQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb08.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700016)(1800799024)(82310400026)(7416014)(376014)(18002099003)(22082099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: kO+vJhx7jlYSBrjlI7XlMP/shgUvBPyHCjF4iHBkPMb9fg351D0LcFpDYusQb6gyR5uedFXqidYkDW+kifGhxnspmFkop6Oa25Miwym8Vi/m+tpGj4yoOObeYR6964/w1ryGGNRVXVQFz7p2k6/eV51KrcgK6okfJKS17gGVZKTSiYhYBtL3QidvVPE1GJzxj0+nB2eH5Xrdgtf8P16VF2Vp3wYBuOM07dONg4M1pIe55NEQo2j+PAkX6nQFvYxlfs9rcumtcOO82cJEZvkrneMKmIEJ9Al+WhM17AYYmaWibeGzCQq/Dzjihvf936TElfQoZ7XltnT013uDs5B5bfKoyCOx8kCyj8sWmZowDqVdbLQAJ6Ob/Lq+aT3CHg8KqPGjTSeHVbz0POjg1VdmcwIkfOT3W4a4pSE3iB+K4Qy2gN5NKDPUMOfRRTzdEGvu X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2026 09:59:52.2600 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4a95ff5b-651c-4345-1a16-08de88c2ed86 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb08.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00003F66.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB5625 Graph500 results Test system details ------------------- 3 node AMD Zen5 system with 2 regular NUMA nodes (0, 1) and a CXL node (2) $ numactl -H available: 3 nodes (0-2) node 0 cpus: 0-95,192-287 node 0 size: 128460 MB node 1 cpus: 96-191,288-383 node 1 size: 128893 MB node 2 cpus: node 2 size: 257993 MB node distances: node 0 1 2 0: 10 32 50 1: 32 10 60 2: 255 255 10 Hotness sources --------------- NUMAB0 - Without NUMA Balancing in base case and with no source enabled in the pghot case. No migrations occur. NUMAB2 - Existing hot page promotion for the base case and use of hint faults as source in the pghot case. NUMAB3 - Enabled both regular and tiering mode of NUMA Balancing (kernel.numa_balancing=3) Pghot by default promotes after two accesses but for NUMAB2 source, promotion is done after one access to match the base behaviour. (/sys/kernel/debug/pghot/freq_threshold=1) Graph500 details ---------------- Command: mpirun -n 128 --bind-to core --map-by core graph500/src/graph500_reference_bfs 28 16 After the graph creation, the processes are stopped and data is migrated to CXL node 2 before continuing so that BFS phase starts accessing lower tier memory. Total memory usage is slightly over 100GB and will fit within Node 0 and 1. Hence there is no memory pressure to induce demotions. harmonic_mean_TEPS - Higher is better ===================================================================================== Base Base pghot-default pghot-precise NUMAB0 NUMAB2 NUMAB2 NUMAB2 ===================================================================================== harmonic_mean_TEPS 5.07693e+08 7.08679e+08 5.56854e+08 7.39417e+08 mean_time 8.45968 6.06046 7.71283 5.80853 median_TEPS 5.08914e+08 7.23181e+08 5.51614e+08 7.58993e+08 max_TEPS 5.15226e+08 1.01654e+09 7.75233e+08 9.69136e+08 pgpromote_success 0 13797978 13746431 13752523 numa_pte_updates 0 26727341 39998363 48374479 numa_hint_faults 0 13798301 24459996 32728927 ===================================================================================== pghot-default NUMAB3 ===================================================================================== harmonic_mean_TEPS 7.18678e+08 mean_time 5.97614 median_TEPS 7.376e+08 max_TEPS 7.47337e+08 pgpromote_success 13821625 numa_pte_updates 93534398 numa_hint_faults 69164048 ===================================================================================== - The base case shows a good improvement with NUMAB2 in harmonic_mean_TEPS. - The same improvement gets maintained with pghot-precise too. - pghot-default mode doesn't show benefit even when achieving similar page promotion numbers. This mode doesn't track accessing NID and by default promotes to NID=0 which probably isn't all that beneficial as processes are running on both Node 0 and Node 1. - pghot-default recovers the performance when balancing between toptier nodes 0 and 1 is enabled in addition to hot page promotion.