From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from PH8PR06CU001.outbound.protection.outlook.com (mail-westus3azon11012030.outbound.protection.outlook.com [40.107.209.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C42637C0F3 for ; Tue, 28 Jul 2026 06:18:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.209.30 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785219493; cv=fail; b=Pnav6eDIdd83BAMQzgcKxqRjBACSAKaelM64AFBvhzQorZk0skSslhj7E0ISO4+1+2zTn7WvFslf2YYwdkaduAXu2IU97JNNfyrtrxwj0lyg0rTP1veTvpSsZuw8tm/rSzSGu43E9rpY4TFJNGqYd531w1yN4XOcvP8cx/M75fo= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785219493; c=relaxed/simple; bh=n0GSVKdwBumQmal0NuMBXWkdEB5rjMRzKWMlyBo92Gc=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=KwfSf9qGn2FcXVU9x6JKeWmkBjmW75tgvIUdNRYbQIc43vLY76d/yUmC3qiu6xAPp+MbONu3CqNQDFo7cVof/pCW6JpmH/z7FXd/HOtlzanMYbCmZgiHXIjiipNxgfdpRyz5cxtGBpwEZA93NniQsHoSq5WwOqaMGjpsLvI/8q8= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=sbckCMQS; arc=fail smtp.client-ip=40.107.209.30 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="sbckCMQS" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=MYHLIK5FbfQrZsT4xGS6cCUXHQKuy9LOH+Yae+/iJnno1NwMEZEuyQXd4Q9pJL+2sD+u8uSQm+1AEHr/TjWATKdG9HWqPPqvwD+nIMZ2zIP3S62a2VFY0JdP6IcJzcBt5vwA1GSbFgOSNY6Dw9te9Dq+CSlr2sjzVggymnAHRI4yyjECjHT2qHvesaR5hkCyKoyTRUuI7grQiqJvNp2J3aSROJiR/0RaPvX0tRkplA65UbugRHxHLZPg8C03Mfc/yMfXt42Nj5HTuZNuOrWVBBjfLZO7VBUg7ExrbdqT6IGcuXMajdYZjN3jwNeCXxkoUJcApGhoi6QRnayLhA7jdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4dOAev5n88S1WhaRTFhj4TpkWLL6vSuVFHQ3peCrY8k=; b=xtXaajNqDuB3wXeXglKAPKmpTAZqdk2NinmAZ6qqlJOd5v8EnAPKcgw9zFzH1lKjWHgi/1on4Vk4FjFIsGVMEd4c9G9Hd3rYNfoBVN38nQbYn0qEZgobKiehg2oUwDtIidV8TGw03tIk+QEyz4P1h+aGTkV9XRTjBVwOj50qlQA1JwGiaYGMwnc9Hrye9RkHCujiIjMYJFWWywxJ/kVXORH+NWa8itspQxqy74dQ9iIsKpGuTHWOjdVgmjTupWr/ZiLhhP7DYV3fYQyjYq1VMtjZ8i4w7LrBoNatninbws8sUlXv0OM4EktI1Wgg1yJ81jWdZPVKUYenirWOWbyoQw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4dOAev5n88S1WhaRTFhj4TpkWLL6vSuVFHQ3peCrY8k=; b=sbckCMQSHibWXBiz2LNdbbB5w5crA2D3NFgGhDdPDCq0qNcrfwOWJAs5y787MbOGC4Kkd8OR6EygQT76+aHxuXSgdZycqlk7WSFvivumg9C2GR6Mn0Frxg8Ey5sd02vqjYx5deRkqF9ueQ1ZyWCg+niO/aiXz13S6nopzf8wNCE= Received: from BLAPR05CA0045.namprd05.prod.outlook.com (2603:10b6:208:335::25) by IA1PR12MB6162.namprd12.prod.outlook.com (2603:10b6:208:3ea::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.245.13; Tue, 28 Jul 2026 06:18:01 +0000 Received: from MN1PEPF0000F0E3.namprd04.prod.outlook.com (2603:10b6:208:335:cafe::81) by BLAPR05CA0045.outlook.office365.com (2603:10b6:208:335::25) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.270.12 via Frontend Transport; Tue, 28 Jul 2026 06:18:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MN1PEPF0000F0E3.mail.protection.outlook.com (10.167.242.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.270.5 via Frontend Transport; Tue, 28 Jul 2026 06:18:01 +0000 Received: from satlexmb10.amd.com (10.181.42.219) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Tue, 28 Jul 2026 01:18:01 -0500 Received: from satlexmb08.amd.com (10.181.42.217) by satlexmb10.amd.com (10.181.42.219) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.41; Tue, 28 Jul 2026 01:18:01 -0500 Received: from [10.136.47.225] (10.180.168.240) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server id 15.2.2562.41 via Frontend Transport; Tue, 28 Jul 2026 01:17:54 -0500 Message-ID: Date: Tue, 28 Jul 2026 11:47:48 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v8 0/8] mm: Hot page tracking and promotion infrastructure - llama-bench To: , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , References: <20260728054356.291998-1-bharata@amd.com> Content-Language: en-US From: Bharata B Rao In-Reply-To: <20260728054356.291998-1-bharata@amd.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E3:EE_|IA1PR12MB6162:EE_ X-MS-Office365-Filtering-Correlation-Id: 478e9ce8-e996-41b8-05c3-08deec6ffa44 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700016|23010399003|376014|7416014|82310400026|11063799006|56012099006|10067099003|18002099003|22082099003; X-Microsoft-Antispam-Message-Info: X7zBqdxLoNOeMsPRX4RO3YTzYi8ckfzx5bau69K0m533WRmlcwzYbAplK2KJlpVuSr1QCuk0OUqo4UK5GtSJt6m2zdAFD48dxdT5jlJqBLiSfAQGLBI6xjiehWS9xcuGrC4ih/1mdZ5nULevxf6c9N1WFXdj7pXY7Bxdu94i+7d/5h0naOn31w9Er5LNpZ+ZNw9fVmWJqXAe1e57V+5ojYTqLON2PhtT5BljhUo7El0ErZfaD8NZpDg6/jCQGq7/g/CeO25CogGeidPn3o+IcDvN+hOeSX0zg4DvXOFEVokEr/8kB/EYImZFKiBiwVsZ8FixMqZ+HVA72GcLN4OXwYJfwNKGr+PutP4ZlR6BzqlfxESc2nirbT+soAzGsii9whYeVFQYX76PhnyxYTU1/rV1hpj/CprFZKqcJZdbeYLEHkuyeUvWaQBWX2S7/YENYymCfDgdwbcGAU15jv+IRFUSh9HMA2azUlIt8fXxIc/VlGE3srJVhymY5c+eG8xv/LW+j2Dnh9A/v0baHIusJnq9bAjZ5E1FubFy157bN6auttVsItNes8VF01qPfgNeRkKuy6EoU/ibIKQMsalwVi2JB4fezFpq5U4WZGBxU15KWM2RN4Sgug28x3Ja5rY0HFJvVi3mZgJ6EMLu5xbAfPU8sL26ymslRlqIDvMcgrKvC1TcOztGvJz8M6wTkNISx06fd2Ww31xBDE9iz6qqww== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700016)(23010399003)(376014)(7416014)(82310400026)(11063799006)(56012099006)(10067099003)(18002099003)(22082099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: FS+Fi1iAan4u1B+wYNKtjHyML0OVIg6YfaXpBrJ1ztbc54O/vcWzySeUF2sg1vYu/pfLi74rKy0wcaTwYjyGTe0W53a3nqXdX+E0t+IsnxIsFEXCXfvEZLMzipdUHOaS6jEphtpHDmM4hW80A9u4bz3o0srrl2R6W14cJZp7R7m+lcAhV+VrGhwoX8Ka29IkIGINZDc2NnmY7qXPlOFzm06daePnqS4K8LZwwujscneTJvLQDxS+vo0ypgLkpnT8e2AGW54iH4vEPmD9mWnUpuCYtOvttEZgDdW1lRa0PWIBeqVtJhYauSuyW8AgtwwxCC5MA1T9JLUz1HUv6NCzdClFBTrWpk8KFymqmhX26hUcX2jXzVLdXXnG8A4s2g3MULGnBV4ggLBtckg0q6YlQTqIAcju8A+tlDVx0KmH5ZYuWZAfLeW8JYsNAHGWM0X8 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jul 2026 06:18:01.7038 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 478e9ce8-e996-41b8-05c3-08deec6ffa44 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E3.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6162 llama-bench is the micro-benchmark shipped with llama.cpp. It loads a GGUF model through the production inference path and reports two throughput numbers per run: pp512 (one batched 512-token prefill, compute-bound) and tg128 (128 sequential autoregressive decode steps, bandwidth-bound). Workload: llama-bench (llama.cpp), Mixtral-8x22B-Instruct Q4_K_M (140.6B params, 79.7 GiB), -t 64 -p 512 -n 128 -r 5 --mmap 0. pp512 = prefill (compute-bound); tg128 = decode (memory-bandwidth-bound). Setup: 3-node box, N0/N1 DRAM (256 GiB each), N2 CXL (distances 10/12/50). Bench pinned to N1 CPUs with MPOL_PREFERRED_MANY({1}); a 200 GiB hot hog on N1 forces kswapd to naturally demote ~1/3 of the model to CXL; each mode is then measured while the hog keeps N1 under pressure. pghot target_nid=1, freq_threshold=1, freq_window=3000ms, rate_limit=65536 MBps, kmigrated 100ms/512. hwhints runs arm AMD IBS (l3miss-only=1, period=10000). Legend (columns) ---------------- r1 = base / notier 7.2.0-rc2-base nb=0 src=- r2 = base / tier (NUMAB2) 7.2.0-rc2-base nb=2 src=- r3 = pghot / hintfaults 7.2.0-rc2-pghot nb=2 src=0x1 r4 = pghot / hwhints(IBS) 7.2.0-rc2-pghot nb=0 src=0x2 r5 = pghot / both 7.2.0-rc2-pghot nb=2 src=0x3 (nb = kernel.numa_balancing; src = pghot_enabled_sources) Table 1 - Throughput (llama-bench, tokens/s over 5 reps) ------------------------------------------------------- metric r1 r2 r3 r4 r5 ------------------------------------------------------------------- pp512 t/s 69.31 62.46 54.53 53.95 51.49 stddev 2.70 6.65 10.41 8.12 4.79 tg128 t/s 4.262 4.625 5.993 5.161 6.851 stddev 0.004 0.313 1.255 0.535 1.807 tg128 vs r1 1.00x 1.09x 1.41x 1.21x 1.61x pp512 vs r1 1.00x 0.90x 0.79x 0.78x 0.74x Table 1b - tg128 per-iteration (t/s), shows convergence ------------------------------------------------------- iter r1 r2 r3 r4 r5 ------------------------------------------------------------------- iter1 4.268 4.183 4.393 4.462 4.415 iter2 4.261 4.457 5.336 4.820 5.602 iter3 4.260 4.678 5.995 5.218 7.322 iter4 4.261 4.841 6.501 5.494 8.152 iter5 4.259 4.969 7.740 5.813 8.763 Table 2 - Key vmstat counters, run-phase deltas (millions of pages/events) -------------------------------------------------------------------------- metric r1 r2 r3 r4 r5 ------------------------------------------------------------------ pgpromote_success 0.00 3.05 6.95 2.17 6.53 pgpromote_candidate 0.00 14.34 24.12 13.83 21.09 pgdemote_kswapd 0.69 5.02 10.84 3.89 8.05 pgmigrate_success 0.69 8.07 17.80 6.06 14.59 numa_pte_updates 0.00 28.77 24.56 0.00 12.04 numa_hint_faults 0.00 27.77 24.12 0.00 11.39 pghot_recorded_accesses 0.00 0.00 24.12 13.83 21.28 pghot_recorded_hintfaults 0.00 0.00 24.12 0.00 11.39 pghot_recorded_hwhints 0.00 0.00 0.00 34.03 32.42 hwhint_total_events 0.00 0.00 0.00 34.03 32.42 hwhint_dram_accesses 0.00 0.00 0.00 20.03 22.34 hwhint_extmem_accesses 0.00 0.00 0.00 13.83 9.89 Key observations ---------------- 1. pghot-both (both hintfaults and hwhints enabled) improves decode over both base baselines. 2. Prefill (pp512) suffers from tiering promotion (0.79x hintfaults, 0.78x hwhints, 0.74x both vs notier); pghot/hf and /hw pp512 are noisy (stddev ~8-10).