From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from CO1PR03CU002.outbound.protection.outlook.com (mail-westus2azon11010069.outbound.protection.outlook.com [52.101.46.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 843E0328634 for ; Wed, 28 Jan 2026 07:50:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.46.69 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769586655; cv=fail; b=YLDC4jfn1G70/qp4Eubf9mTPSmJjwhsfmfkIxyUSgoemmoFKK4LwufeH4GBmX8VVRWYaUqv808PpXp2Px5yaX4oBunzNhLlEa74zW7d01Kk1vJ64j92a4m+m8TS3dJTpc9G1uGby9KOaZrBXYNVf0GgLz+57IWjh8vxZNKtAkI8= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769586655; c=relaxed/simple; bh=S3/dQ9qN9z7j4RKJNuHbi1W/Po32hYOol8DjIohvECA=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type; b=K8FeQq1e6VrRHNpIjUQYqggWkMT9H8Sn1/v9qHchAtkkM2Tw6pQqf3CYRFM5INh8t7X2Kpu4AvvKn9c5ndzCKBgYXfUrxdq8UifcXUJ6u6DtEGAG6Cfqi97zatIISQRCidI82YLAbvFH5l4CIVKHh931wFNn6uvoVy6+tLpwfmY= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Eq6Zpam6; arc=fail smtp.client-ip=52.101.46.69 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Eq6Zpam6" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=iDgU2iLUHGUNHu0PO//zuYs9XXNgdh+w39StIiYsvZYZJvJteuszRZRB4O52bU1lVCakCNcU8WjTln23FBKXe0V/lg6pLOtgGzxhy+ivqA7HAhbL3W2XiZoFtTE0xajpYfsJYdNMCYm3mD4YGnauPbBttuEddShzoQzJ0/aiaZK46kt7v5gWo7kjTaRtQ0dwZBB8VDzn0ORaeA2Spg5qXCLQVwjsrvoE/xUxwbtPFyuVrXOXzMsivUhm67FvUN9N0UT7q7G+Y229d2DDH6OrU6Mhe9hH8+PONc8kqmLLmrjlmY+Aro7MSHZOWdgl9tF7ZNxDsyUxrtPZgtn2n/eWpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cgtkEnnsPAf2xSsf7JBRb9e1uIRJDMPkZB1stLLchpI=; b=h7X4cYY/L+9YHJLXrauK92ZcxMgqKf0qPbnpRHLUqwh13QlUw044J0ncBC4hvMK4ZMyf50ltsNePc3hnddbpp0Qd7fzWCERfbQSCcJ/6MbwXAJBbReYUZ2PGm0ZPG20SVKkwgDWpAgwFcaT0+ntwt0vHpAwORdrIkI/eC4dpc/lxN3k9+OYQd8dbIj1seDAgNVh9IF0jtsZLGZjGF4J8mgsEKJegh/G+avFwx8Czb5Hqge1noJX6mkty7NBgen9Noaa+r+S7n2b2UutLKDDCDQ5vawNCsq3cU2cPLFXU0o/kPAmVWY81ymtxLzVu16aNAycI9Ft9Nh+NLAB80/Ai5g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linux.ibm.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cgtkEnnsPAf2xSsf7JBRb9e1uIRJDMPkZB1stLLchpI=; b=Eq6Zpam6m/qsBdytXwyolH1crg/l9HALQ59wxCXYeTlt30QTlRoAd2EsTr1Y2skR6f3AONYLM6bupJAYfh8RkVM7dARtHcl2agupeG7WN/ykIlENmhNfnYuxk0DsEXTjsR/ZtHXyu5zcRzW+M8qnn6V4ER6BIvaCY9c+4br16lg= Received: from DS7PR05CA0101.namprd05.prod.outlook.com (2603:10b6:8:56::21) by CH1PPFDB1826343.namprd12.prod.outlook.com (2603:10b6:61f:fc00::628) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9564.7; Wed, 28 Jan 2026 07:50:49 +0000 Received: from DS3PEPF000099DE.namprd04.prod.outlook.com (2603:10b6:8:56:cafe::d4) by DS7PR05CA0101.outlook.office365.com (2603:10b6:8:56::21) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9564.7 via Frontend Transport; Wed, 28 Jan 2026 07:50:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb08.amd.com; pr=C Received: from satlexmb08.amd.com (165.204.84.17) by DS3PEPF000099DE.mail.protection.outlook.com (10.167.17.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9564.3 via Frontend Transport; Wed, 28 Jan 2026 07:50:49 +0000 Received: from Satlexmb09.amd.com (10.181.42.218) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Wed, 28 Jan 2026 01:50:48 -0600 Received: from satlexmb08.amd.com (10.181.42.217) by satlexmb09.amd.com (10.181.42.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Tue, 27 Jan 2026 23:50:48 -0800 Received: from [10.136.38.16] (10.180.168.240) by satlexmb08.amd.com (10.181.42.217) with Microsoft SMTP Server id 15.2.2562.17 via Frontend Transport; Wed, 28 Jan 2026 01:50:44 -0600 Message-ID: Date: Wed, 28 Jan 2026 13:20:44 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v7] sched/clock: Avoid false sharing for sched_clock_irqtime To: Shrikanth Hegde , "Guo, Wangyang" CC: , Benjamin Lei , "Tim Chen" , Tianyou Li , "Ingo Molnar" , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider References: <20260127072509.2627346-1-wangyang.guo@intel.com> <7c6236a5-82b7-4d1b-9bb2-3f0d2ac186f8@linux.ibm.com> <3b69443c-f82c-4b73-8384-2a039d730213@intel.com> <5e001aeb-50a8-475f-bff2-6d101e15d078@amd.com> <868b9993-34a2-4b33-b47a-989b2c680689@linux.ibm.com> <2a7521d9-e4d8-4dc6-8e5a-122796f0d1ab@linux.ibm.com> Content-Language: en-US From: K Prateek Nayak In-Reply-To: <2a7521d9-e4d8-4dc6-8e5a-122796f0d1ab@linux.ibm.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099DE:EE_|CH1PPFDB1826343:EE_ X-MS-Office365-Filtering-Correlation-Id: af90ac6a-2f23-4e59-7363-08de5e41f3f7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?B?czBGNFRudENuOVh6THJpSDBhODF3aFRReTVXMjBpRXZBdmRxamVuN2Zmc0Jo?= =?utf-8?B?UWozd0J2Y0R5OVR2S2RGMW9sQkxQMVgvendJZjFaR1pTdTBjR1hDek93bmVq?= =?utf-8?B?aVI5L2pwNmlUT3dXV1d5YWlmaUxHMEF5bStKMWNQRFZndzVtaWZGcXRIZzRY?= =?utf-8?B?OXErSEEwOTlJZVcwcWs3YUJCcXhoV0k5QmpPaG1BL3RpRkd2TUZ6bTdDeGVs?= =?utf-8?B?Y1Z1aURrdWtCWTVKU0Z6MU5xR2xJWElhNnExUXlQNE9TdW1Yd3J6UmQwbjIz?= =?utf-8?B?anhkc285QUtiVkg4MkhOUDlJNVEwOUllTjkxNEVreGlhWlh2M2ROYk9MNTlk?= =?utf-8?B?N3ZuMnJ3YzA0bEs1T1h6MWd5aDRqc05QTUFyUGUwS0tuakRYNS9rU2hKYUh4?= =?utf-8?B?WTF3VGtYSUhZeXJaVEVOS3hha2VlcFlLdkZ3cjZlWHB5YnB0OW9obytxaDlE?= =?utf-8?B?T1FZMWRuUzhaamdNdXN0ODkyNWhNMHhzdWk2cW14bEM2aE9wYXd2aFZ5Y0ZJ?= =?utf-8?B?anRPcUJEK0RkeENZYU1tNVZMVkhCYk5SWU82SmNhTmk4STlYSzcyQUY1WHUy?= =?utf-8?B?bGpLSmhNUXhISUdSVWNLbGNLK29tUzRCazBuQTRTeFhwMFJuUnpMbnNnN3ZX?= =?utf-8?B?WkNnZVB5WXNPdlM5M2pvV3ZOQXdlZVh1UDU3RzlkU2dXSHJpdlJIYjhBUjE1?= =?utf-8?B?c2ZJQkNqdkRrKzdSMlJzU1VFOXhoY2pKWFd5Nmk4Z3VidGFXZURrQ3RhMVlC?= =?utf-8?B?OTdLMGxBWkRMYUcxK3J1VWlNNXBMeklVQTMyUFZEeEdXZXBQK2ZSb0Z1YlRY?= =?utf-8?B?R0RGR2N0K0JOUzJsb3JMNlZZNGRIak95U3ZTUmdJVlpQZHR5eEZQRFYzbDF0?= =?utf-8?B?OVltTkErL0dtWmtFb3o4ZDJSZjZjd210b0RxK3dEdWdXcFp0c2NtOW9mYnFY?= =?utf-8?B?UlRkdkNxYk5RMGNlSU1URmw0ak0xUnNJYjF0cWZidjFnZTJELzA0VUZKbDIw?= =?utf-8?B?MGhIdTg3MSt3cEEzcTMwK0tLaHV3Y29FbmNlbHQ3c3A0NmZGejM5T05XNzdD?= =?utf-8?B?bk5seCs1L3JoYlNGR3RCMWFWRGFiM3ZRNTkyZWN4b3lDaWRrcjZGZUhLN3cx?= =?utf-8?B?eUx3bVd1RWI3cFBrZDhuS0RHdUNaQ2JKc0FBZkxFT3g5bXQwelp1K1JpZm1v?= =?utf-8?B?ZldDdHYzWERmU2dJaEZaOExXSmIyeFdXcUhpRVl5UjdBVkpjRHhXeFNHaERh?= =?utf-8?B?aVFzWFJLWXVKaUxQZU45MkxTOUgxZThwdHhvNmRQMFpzZ1Zka0VpY3FBd1VZ?= =?utf-8?B?Y3pkUW1oY2h5NlE2MjNyaFBLaFlIMVE2SHRtSk12WWlISzVpeng3Mm04WWUw?= =?utf-8?B?dGdVK1RpSmppV3VVTEV4TzhSTk85TzZOS0JNTS9IbGRydmFUWE44QzZBTXpv?= =?utf-8?B?Wjl1ZUxmZ1RvUGo1ZkJLQStrR3lDNHFvVFl2N1lKSHlsNkhoQ0FvcjMwS0Zn?= =?utf-8?B?akQ2NUk2SHpBdjVuMjM2c2RWenozVDJRamhrZHdkdkloL21XajM2dGFpa2tq?= =?utf-8?B?RjJoNFBRcWFzZXJ6cG5DVDBlc1BoRWU1eXIyNktxbWs2cTZhTDNoOHZ6VEFo?= =?utf-8?B?MDBuV3lIZVpmZWl1WUU2WW1YVXJlRmRtVUc5RUQwTjBKMUE2c1hKVVExRUpW?= =?utf-8?B?NHVJN2plbGNoZ3VSYWxxdDRkQXQzdS9wOXYyNkZUOHNpMm1pWi9pZ3phWkkz?= =?utf-8?B?TFFuL2kxbnV0bVdmZmF3ZzNFbnk5YmNKaVVsU0duOWMzcTREWnVMN3NENmRC?= =?utf-8?B?ZEgwQkMrejM3MENZZzlOTlpCajcvcE8wQnpxT3BNMVRUS2pGeDlvYXMvM3ZI?= =?utf-8?B?T1cvN3FXYVp2OXVEQWFCL0NQUnFZNHRyYmZZbG82dnJDc0g3emNVVFNSQk5L?= =?utf-8?B?NWFSeEFDbjFMVTZON2JLaEczVTd6ZzlLRmhEazFCcFRuOFJaVkZEZDhuOFFR?= =?utf-8?B?MmVaZnpDWnFqNlFkRFhYa0R1a0hMV3Z4cUJyQi9Xb0psWXpWeVZIaXMzdlB1?= =?utf-8?B?eVlVMmdSOVJnNmxoNHQyM2VHZVBCM0pqMlFvSHhVKzU2SUZ4bEUyaWR0OVVX?= =?utf-8?B?OExEQjlJcEgyYlpwYlNGdzZ1VVVzRWJZd25OM0RYU0JxdmNPaUp4eUJXb2Nr?= =?utf-8?B?bnhwdFkzSE0wa29kcVExeDQ1R1BjSFF2OHM2TzhySTNaMExVaW1CUUFYTno1?= =?utf-8?B?TkxmeVMyTEFFeE1XTXZ1cmtTOXdBPT0=?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb08.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jan 2026 07:50:49.1357 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: af90ac6a-2f23-4e59-7363-08de5e41f3f7 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb08.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099DE.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH1PPFDB1826343 On 1/28/2026 1:02 PM, Shrikanth Hegde wrote: > > > On 1/28/26 12:48 PM, K Prateek Nayak wrote: >> On 1/28/2026 11:56 AM, Shrikanth Hegde wrote: >>> >>> >>> On 1/28/26 8:35 AM, K Prateek Nayak wrote: >>>> On 1/28/2026 7:49 AM, Guo, Wangyang wrote: >>>>> Yes, when clock mark unstable through tsc_.*mark_unstable() with non-native_sched_clock, clear_sched_clock_stable won't be called, thus sched_clock_irqtime still keep enabled. >>>>> >>>>> Maybe the dedicated workqueue for sched_clock_irqtime is still needed considering this case. >>>> >>>> In that case, shouldn't tsc_init() only enable irqtime when >>>> using_native_sched_clock()? How can tsc_init() make a call on irqtime if >>>> TSC isn't being used as the sched_clock() ultimately? >>>> >>>> For kvmclock, if PVCLOCK_TSC_STABLE_BIT is not set, it'll call >>>> clear_sched_clock_stable() at kvm_sched_clock_init() but none of the >>>> other clocksources do so we can assume once we override the sched_clock() >>>> it is up to the sched_clock() provider to deal with the clock stability. >>>> >>> >>> I think this would depend if mark_tsc_unstable happens after system boot, >>> specially while running kvm guest? >> >> I don't see anything on the guest side that would mark the kvmclock as >> unstable if host's TSC turns unstable post init and since kvmclock >> doesn't set CLOCK_SOURCE_MUST_VERIFY, I doubt if a watchdog runs to >> verify it in the guest. >> >> I have the following in the guest: >> >>      $ sudo dmesg | grep -i clock >>      [    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 >>      [    0.000000] kvm-clock: using sched offset of 423259259 cycles > > This means pv_sched_clock is kvm_sched_clock_read from now. and > irqtime is enabled in the guest. right? So within the guest today ... $ sudo dmesg | grep -i "clock\|tsc" [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 504626078 cycles # kvm_sched_clock_init() happens here so it can potentially do # clear_sched_clock_stable() here if !PVCLOCK_TSC_STABLE_BIT. [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000004] tsc: Detected 1996.251 MHz processor # We enable irqtime here once TSC frequency has been determined # without considering using_native_sched_clock() After that TSC is never selected so we don't care if it is stable or not since it is not the clocksource - the guest continues on with unstable sched_clock() but also irqtime enabled since TSC was calibrated successfully. > >>      [    0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns >>      [    0.071675] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns >>      [    0.378467] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns >>      [    0.388678] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x398cb1e4d56, max_idle_ns: 881590790753 ns >>      [    0.679262] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns >>      [    0.903121] PTP clock support registered >>      [    0.927243] clocksource: Switched to clocksource kvm-clock >>      [    0.944986] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns >>      [    0.993198] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x398cb1e4d56, max_idle_ns: 881590790753 ns >>      [    1.123796] rtc_cmos 00:05: setting system clock to 2026-01-28T07:03:45 UTC (1769583825) >>      [    1.155755] sched_clock: Marking stable (940009972, 212965288)->(1171254846, -18279586) >>      [    1.712598] clk: Disabling unused clocks >> >> Then I mark TSC unstable on the host >> >>      tsc: Marking TSC unstable due to Faking unreliable TSC! >>      TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'. >>      clocksource: Checking clocksource tsc synchronization from CPU 93 to CPUs 0,2,26,75,101,114,118,195. >>      sched_clock: Marking unstable (945948313746, 69389667)<-(947618130068, -1600430832) >>      clocksource:         CPU 93 check durations 3436ns - 25277ns for clocksource tsc. >>      clocksource: Switched to clocksource hpet >> > > so now, using_native_sched_clock should fail in guest? If so, with the patch, > irqtime won't be disabled no? Ideally yes, but the guest continues using kvmclock without any hitch. I think the x86 KVM layer has something to ensure stability but I'm not 100% sure. Since I don't see "tsc: Marking TSC unstable ..." or "sched_clock: Marking unstable ..." in the guest, we don't hit the mark_tsc_unstable() path within the guest which would disable irqtime today so essentially host's TSC turning changing doesn't seem to affect the guest. > >> And nothing happens in the guest. >> >>      cat /sys/devices/system/clocksource/clocksource0/current_clocksource >>      kvm-clock >> >> >> If I launch the guest after marking host TSC unstable, I see: >> >>      Unstable clock detected, switching default tracing clock to "global" >> >> and I don't get any "sched_clock: Marking stable" messages. >> > > Maybe kvm_para_has_feature(KVM_FEATURE_CLOCKSOURCE_STABLE_BIT) won't be set. True but that is only for new guests launches past marking TSC unstable on the host. -- Thanks and Regards, Prateek