From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CEB3932B9BC for ; Wed, 28 Jan 2026 07:57:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769587027; cv=none; b=P4MsktgldV7E3FA2uOQy/1zevhm6fs1ypSg4X4q3+wJyHTOfqvE1WMOKiy2z+lyJlnDIibZxm8VS/zPMHchBFKJQjHGdIfWAdS2v8aVwvI+1gjb+391F4IRGK/azY0g42Dj/yrVcgFURKuOH2lW8QDsl308tGZpHzDPMi9vPffw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769587027; c=relaxed/simple; bh=r108l2Q1KDNgZqLDNJZ8goMqvc6IIhhSdxJRDEB7LzA=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=qPEc51Dw0VSFbHwo6Q7B2gfSa+IRbt/OrsNxDkj1dzpKhHZM0klMgemM5EWAXY5U0rpi8oXlM37OINV9pm30UsSt7iTXuhiyjyaOqgSuF9FKGKR0Juy4wSYJlsH9Borb9qOQVBXKjrDiIAwFTU1GGCpr1/h3l9PZ+3NYq7Edq60= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=mQpizmHI; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="mQpizmHI" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 60RHhRei011204; Wed, 28 Jan 2026 07:56:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=pI7W20 29GZI+TNTYzNBLnrRy/y6+esG7Uqy2twojjCI=; b=mQpizmHItE2XnZvobmXg0Z ZgWYlwr9t9j6LFiKhcwpdk2c3oiD/i5YdVtvi3PS70B8L/5IQyk4/P3/ZuKuSuwP 2AdCBgRknRKZ4Ot/Aj7RST75TcqxDuWYbowhLkV8PNAZLuwfJMygGJn0in7UdlaQ QBlcsCx8jMQ89OuhLVHlFPyeIK8/aH9ofFtq+QXMbvnl7woo/v+hZ0Sb67sjPipu mHWUlq1e+iJolSd3MbdpZkfwMhxJxrig61/7Jr6VraqQrzeWJl0BVa+92VQD/G0B eEq2/ES9Edyke1SbBO0TRjK/3Z4WynLrmQtzg79u9hWH2i8fr2uSWj29URZhg6oA == Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnrthyus-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 28 Jan 2026 07:56:49 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.1.12/8.18.0.8) with ESMTP id 60S7pUMm005594; Wed, 28 Jan 2026 07:56:48 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4bvnrthyug-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 28 Jan 2026 07:56:48 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 60S6RawY031067; Wed, 28 Jan 2026 07:56:47 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4bw8dsmj4q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 28 Jan 2026 07:56:47 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 60S7ukR741877800 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Jan 2026 07:56:46 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F319820043; Wed, 28 Jan 2026 07:56:45 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CC75220040; Wed, 28 Jan 2026 07:56:41 +0000 (GMT) Received: from [9.111.60.95] (unknown [9.111.60.95]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 28 Jan 2026 07:56:41 +0000 (GMT) Message-ID: <54247242-3eef-4b99-a235-81dc6637d725@linux.ibm.com> Date: Wed, 28 Jan 2026 13:26:40 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v7] sched/clock: Avoid false sharing for sched_clock_irqtime To: K Prateek Nayak , "Guo, Wangyang" Cc: linux-kernel@vger.kernel.org, Benjamin Lei , Tim Chen , Tianyou Li , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Valentin Schneider References: <20260127072509.2627346-1-wangyang.guo@intel.com> <7c6236a5-82b7-4d1b-9bb2-3f0d2ac186f8@linux.ibm.com> <3b69443c-f82c-4b73-8384-2a039d730213@intel.com> <5e001aeb-50a8-475f-bff2-6d101e15d078@amd.com> <868b9993-34a2-4b33-b47a-989b2c680689@linux.ibm.com> <2a7521d9-e4d8-4dc6-8e5a-122796f0d1ab@linux.ibm.com> Content-Language: en-US From: Shrikanth Hegde In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-GUID: vz_lWbtbWA36F68h89JUU7m_hjXEJY3N X-Authority-Analysis: v=2.4 cv=Uptu9uwB c=1 sm=1 tr=0 ts=6979c141 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=IkcTkHD0fZMA:10 a=vUbySO9Y5rIA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VnNF1IyMAAAA:8 a=Bt4jt8Zslq1rXZOMBLwA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-ORIG-GUID: cH8dO-6pKVjWSyz8wdOhFP6jikDSiNFM X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMTI4MDA2MiBTYWx0ZWRfX5n8B6sLNI3wP yXwZGFCm9/ec7TwsJ/Jhg0i0SBl4h4l/8t427PlHmo3WNw/1qAvAvPGqugUlY91XixG2sdhwTw6 45wIhPMH5IA1I7vQ7LuNQQ+WyYDcbLaD72EE05PBl88fDQQ5AV5GdD9FgYjVkB+4uy2lBmdeaiu k3ry9jWt1GyJW0bvaiqRCBRofpwDtbXBouUT4rQa1znEFNPBKN1UDFAzmdg5Cb2NeCYNMKtzC/w tbDz8Yq4J9JnaM80rDLW+QS+TVoaKNFy6Fvic4aJbRAXs6VNfUJ5FKBlox7IkrATmwadacZ64/Y S6OEa89M8lMkmf3RqOmJa38w+O6QMSby6SspDh64/xPqRBmauhI8/qvQ127M3BB01lvtvuZld3O R3RRyJoE73I95Vds09kkkKjQDU992bH4g6rMmjYF6MqPnenFZO8Phw3QGsv0RjjaFifmp9ojtiU Pxx4L/6nPoC/WlTDfNA== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-01-28_01,2026-01-27_03,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 priorityscore=1501 clxscore=1015 lowpriorityscore=0 phishscore=0 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2601150000 definitions=main-2601280062 On 1/28/26 1:20 PM, K Prateek Nayak wrote: > On 1/28/2026 1:02 PM, Shrikanth Hegde wrote: >> >> >> On 1/28/26 12:48 PM, K Prateek Nayak wrote: >>> On 1/28/2026 11:56 AM, Shrikanth Hegde wrote: >>>> >>>> >>>> On 1/28/26 8:35 AM, K Prateek Nayak wrote: >>>>> On 1/28/2026 7:49 AM, Guo, Wangyang wrote: >>>>>> Yes, when clock mark unstable through tsc_.*mark_unstable() with non-native_sched_clock, clear_sched_clock_stable won't be called, thus sched_clock_irqtime still keep enabled. >>>>>> >>>>>> Maybe the dedicated workqueue for sched_clock_irqtime is still needed considering this case. >>>>> >>>>> In that case, shouldn't tsc_init() only enable irqtime when >>>>> using_native_sched_clock()? How can tsc_init() make a call on irqtime if >>>>> TSC isn't being used as the sched_clock() ultimately? >>>>> >>>>> For kvmclock, if PVCLOCK_TSC_STABLE_BIT is not set, it'll call >>>>> clear_sched_clock_stable() at kvm_sched_clock_init() but none of the >>>>> other clocksources do so we can assume once we override the sched_clock() >>>>> it is up to the sched_clock() provider to deal with the clock stability. >>>>> >>>> >>>> I think this would depend if mark_tsc_unstable happens after system boot, >>>> specially while running kvm guest? >>> >>> I don't see anything on the guest side that would mark the kvmclock as >>> unstable if host's TSC turns unstable post init and since kvmclock >>> doesn't set CLOCK_SOURCE_MUST_VERIFY, I doubt if a watchdog runs to >>> verify it in the guest. >>> >>> I have the following in the guest: >>> >>>      $ sudo dmesg | grep -i clock >>>      [    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 >>>      [    0.000000] kvm-clock: using sched offset of 423259259 cycles >> >> This means pv_sched_clock is kvm_sched_clock_read from now. and >> irqtime is enabled in the guest. right? > > So within the guest today ... > > $ sudo dmesg | grep -i "clock\|tsc" > [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 > [ 0.000000] kvm-clock: using sched offset of 504626078 cycles > > # kvm_sched_clock_init() happens here so it can potentially do > # clear_sched_clock_stable() here if !PVCLOCK_TSC_STABLE_BIT. > > [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns > [ 0.000004] tsc: Detected 1996.251 MHz processor > > # We enable irqtime here once TSC frequency has been determined > # without considering using_native_sched_clock() > > > After that TSC is never selected so we don't care if it is stable > or not since it is not the clocksource - the guest continues on > with unstable sched_clock() but also irqtime enabled since TSC > was calibrated successfully. > >> >>>      [    0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns >>>      [    0.071675] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns >>>      [    0.378467] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns >>>      [    0.388678] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x398cb1e4d56, max_idle_ns: 881590790753 ns >>>      [    0.679262] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns >>>      [    0.903121] PTP clock support registered >>>      [    0.927243] clocksource: Switched to clocksource kvm-clock >>>      [    0.944986] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns >>>      [    0.993198] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x398cb1e4d56, max_idle_ns: 881590790753 ns >>>      [    1.123796] rtc_cmos 00:05: setting system clock to 2026-01-28T07:03:45 UTC (1769583825) >>>      [    1.155755] sched_clock: Marking stable (940009972, 212965288)->(1171254846, -18279586) >>>      [    1.712598] clk: Disabling unused clocks >>> >>> Then I mark TSC unstable on the host >>> >>>      tsc: Marking TSC unstable due to Faking unreliable TSC! >>>      TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'. >>>      clocksource: Checking clocksource tsc synchronization from CPU 93 to CPUs 0,2,26,75,101,114,118,195. >>>      sched_clock: Marking unstable (945948313746, 69389667)<-(947618130068, -1600430832) >>>      clocksource:         CPU 93 check durations 3436ns - 25277ns for clocksource tsc. >>>      clocksource: Switched to clocksource hpet >>> >> >> so now, using_native_sched_clock should fail in guest? If so, with the patch, >> irqtime won't be disabled no? > > Ideally yes, but the guest continues using kvmclock without any hitch. > I think the x86 KVM layer has something to ensure stability but I'm > not 100% sure. > > Since I don't see "tsc: Marking TSC unstable ..." or "sched_clock: > Marking unstable ..." in the guest, we don't hit the mark_tsc_unstable() > path within the guest which would disable irqtime today so essentially > host's TSC turning changing doesn't seem to affect the guest. > >> Okay. Fair enough. Then v7 should cover all scenarios i think. with that, Reviewed-by: Shrikanth Hegde