From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7861F3E1CFA for ; Wed, 25 Mar 2026 15:34:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774452854; cv=none; b=QZGIq8UvBPzh73CqDT1W5JCFtTpBjeqln7q3aesvVSKpcAxlOifEIPE7eZP+2/Drpf7eiDN4V7ghQ0XmkDHspstPFqvqQ312Lsn7zHgGxdtESESf1cXlZi3Y1izaYP5ycfvoHkG/fD634RH3d/cxS5f0O2X+kdtM1RCyxrY3CCE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774452854; c=relaxed/simple; bh=vPGIzMiyO5TeElcR4phkh0O4Z3y4C+7N0wLVWgHYFZk=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=CySCv+f66fwbmemkmzq+4cWmVSGX5/CXesoZeA/HIAO7nhHedwEwsgdsN3nLZFnWDFgybu33j8lguLBzeYFL4foSOje/BJNfME1Nd/cqawJxWnCd0g+vKnb4z4ygfSGRCFN+frybusmn8/3G/3E39uiMviYituZl0Hcm5/6Ru0w= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com; spf=pass smtp.mailfrom=oss.qualcomm.com; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b=hazzhJ4/; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b=hiqnWxRm; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b="hazzhJ4/"; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b="hiqnWxRm" Received: from pps.filterd (m0279862.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62PFH9ho2432203 for ; Wed, 25 Mar 2026 15:34:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= JJbISd+NYRXiouLIvxTEhljHuABzyURDFTMQeKuO6e0=; b=hazzhJ4/WcyRYWXM X0WTj0cuikqgggof07qbKtXmlB5xRBQ7g5pcoxOMdz4P/do9TK1qF3OAB3ty7Osu 4rmoOaVwtoxWktm0pwI0V6vCTc39XSml0d1L184pV7vbbcyRBYkDKaRHAuUQKGLd f/sYS61TVXURYhMt/A+KN5a5kUjMVAoyz4X9ccDAjqyGygPIR0Tbb3IxSD9MMMeh WvvPd3Nngq+lFfuXyUqOT1VQreyKB0pzG+qmTRZkO7n0FNte0iRJj3IClck/SqvK GYfvrrATvcq09P6wX0y2lXzYjSIRdv5MuUWCjmL10BKZ+jSU5K4If9/lqMczuXI7 eYSz2Q== Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4d489mjf6t-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Wed, 25 Mar 2026 15:34:11 +0000 (GMT) Received: by mail-pl1-f199.google.com with SMTP id d9443c01a7336-2b0b339b8dbso13839765ad.0 for ; Wed, 25 Mar 2026 08:34:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oss.qualcomm.com; s=google; t=1774452851; x=1775057651; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=JJbISd+NYRXiouLIvxTEhljHuABzyURDFTMQeKuO6e0=; b=hiqnWxRmMDmGDeFGOPXXO4xCorOD69e4anW65L3cOOKZ5m31zYns2InMwyzTInUdb0 kKrbEZqA7tzo2K18XXzULaPa+xFMfQmeC3znb1H4q7DTELSF/Ms9YiLrTXanHACSrMvD GdeepvmQF0FBw0wPs29mY4n+kGkVR68oADr8CBXFycoJ1a76mQaZYIuOIm33v89NJKB1 dJ397X3TR2I1VTczefQ03BLl37VMeTEpogOGBy+LdQ+jOXpAsnWVFtE9HmDmuYqcssYa W4dlZ2ofWxeyX3dCbVMBlBvHhrltQDgnZG6DcZ3ee7mSWbeGUdNM2PRuPlED/0UZx+r3 +2iA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774452851; x=1775057651; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=JJbISd+NYRXiouLIvxTEhljHuABzyURDFTMQeKuO6e0=; b=s4kg9JDo2cmIxKpR8BIryvPZh8rPmcUR1LcxKxAOUaMo5VPJPZcpOfIhzFZ9EmWZ7O vYPF8IaAyIuZsd0fNQ0ic50I/dpaDCTa+THmNUENml2o6t0BkT8znul7CBwXwJReYR5A x8dY1QsezkqG9kiZiJOGHY2hxMKhLK+bl3LYXShmi4tTOAmUp4Pn/g5YHFdWzsKuam8/ OxGRSF3KTAFdXBb+kLsMnnVewth5EtFimsKzD5Pt0Q83V8S2eBXc27RXERKBwRb33SLm 0krq+AYh2MjwA5Ol4Z+EaYw8iEiWGWb2smMVgffUKsYMQaIkn1AGpkEjHK4ia6MlI2Us WqhQ== X-Forwarded-Encrypted: i=1; AJvYcCWsbZIxtDa25hlxx0tyWteQSpcG2pKwiFhKnFYt+q22wtqk/S6Xmd/H5zqHYFUgTK0FF7l+ifQanf0OvWA=@vger.kernel.org X-Gm-Message-State: AOJu0Ywzp20YT4g7VefApvxWmEtB6ogiUA9o5CvDvDu6kf+rj4qGsy1V 3BdjMEaS82diH0igamEtoDa9W8/tN2ufziyVtKe31CsMv7KDGf8Sfuqub4M1LrFnH+3KivmKBDe SWN8lmoJaMapRp7G+WnFzkHTMa+3WESD16XpGAtQTF4YWLJWDUFi0ussSPMEwboCqlIA= X-Gm-Gg: ATEYQzz2IiUZOY4xYZONZlfhDA4E1hhj64hXVLlDl+Df8RhEPDSnws5LckNMDJPV5ge BYOvGOdPzI3fcqRZxC2M1N52bJvVdpVmLVGk4NKkU0sB3PzlG5NWDCKKcxY83mW5OP5eSpBPRR8 mjNx3t7/YUgD7+BXL4ePy66Bo/mBtpUsZp1S1QfoZ7NgG3Uwu7vciHK20xejRHDPmbh8VZfnrTI wsxlWsBqc+vKu5GxE4//a+z1POka5669jCs5e+IntN/j33zH3aqe5AOK8sVVFcR8tqv2DvootWq LoMKQInZ7YomnAzSSfUjlTIHfq6e8F1Ix9Or4n9w4roFMzLRQHG36jFUKW/7JlFmHuc1iR+E1lm MK1uM4NrCAiFiT8g+Hu2TaOYPGJc5dghivbi4IRWk4K93jqL7HcIz X-Received: by 2002:a17:903:943:b0:2ae:4445:f397 with SMTP id d9443c01a7336-2b0b09e0913mr45260725ad.16.1774452850478; Wed, 25 Mar 2026 08:34:10 -0700 (PDT) X-Received: by 2002:a17:903:943:b0:2ae:4445:f397 with SMTP id d9443c01a7336-2b0b09e0913mr45260405ad.16.1774452849948; Wed, 25 Mar 2026 08:34:09 -0700 (PDT) Received: from [192.168.29.179] ([49.43.224.151]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2b0bc87e4e4sm1852915ad.38.2026.03.25.08.34.06 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 25 Mar 2026 08:34:09 -0700 (PDT) Message-ID: Date: Wed, 25 Mar 2026 21:04:04 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending To: Ulf Hansson Cc: "Rafael J. Wysocki" , Daniel Lezcano , Christian Loehle , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org References: <20260316-cpuidle_ipi-v1-1-d0ff6350f4e2@oss.qualcomm.com> Content-Language: en-US From: "Maulik Shah (mkshah)" In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzI1MDExMiBTYWx0ZWRfX0t0kn8Xg9Mcm gZt3AUV8fvb22OpIt+rR8ZqcibGsZDCaO4FicuQQ7jhxVBBDYMF0W8kn+k7x/h1R94+YWar4O7J 83YZH1qb583sAGO21jnjiHUf9FdulvDTwIQ+k1d6Wk/jkz4fPzGaoq0dRmMkalp1lPmtPRmTT8o aJfoxMpqbK1Dv57ubecmh06JognzONQGBbR6MqypFkSKjefZACNJrLbWsNSgTu3aGtdb3fJu0r/ aeRp9KzpfZjTk+86jKJU65jswfa27NZDcb7GfIvO53r+GgTKexyXC+RkiH6zTWawSj1MDnVXgdi Q+9vOif5fh0PzhTOgno0Ao6NcFCjfxcfb6uZPy6FaDgGXlT70nYewKhx+gMtgXt1tzOGsUSWV1J 4nc70cou9cMPyfrVzF80c8nM+C6UXlGNG5zggxD7i2VwYipIM6jSj3ddUmX2ZEXOQ7aMkkEh0tA eiNACtbKe8lnMwWzNWg== X-Proofpoint-GUID: 8K1CbtZrULCokWfimyLKZEcosoXV1oWC X-Proofpoint-ORIG-GUID: 8K1CbtZrULCokWfimyLKZEcosoXV1oWC X-Authority-Analysis: v=2.4 cv=AKSYvs3t c=1 sm=1 tr=0 ts=69c40073 cx=c_pps a=JL+w9abYAAE89/QcEU+0QA==:117 a=j/PqwQOCDgRQPIT2BnD5zw==:17 a=IkcTkHD0fZMA:10 a=Yq5XynenixoA:10 a=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=u7WPNUs3qKkmUXheDGA7:22 a=_K5XuSEh1TEqbUxoQ0s3:22 a=NEAV23lmAAAA:8 a=EUspDBNiAAAA:8 a=okY-DNlmIvmWAd7kz_EA:9 a=QEXdDO2ut3YA:10 a=324X-CrmTo6CU4MGRt3R:22 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-25_04,2026-03-24_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 phishscore=0 adultscore=0 lowpriorityscore=0 malwarescore=0 suspectscore=0 bulkscore=0 clxscore=1015 impostorscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603250112 On 3/24/2026 9:16 PM, Ulf Hansson wrote: > On Mon, 16 Mar 2026 at 08:38, Maulik Shah wrote: >> >> CPU can get IPI interrupt from another CPU while it is executing >> cpuidle_select() or about to execute same. The selection do not account >> for pending interrupts and may continue to enter selected idle state only >> to exit immediately. >> >> Example trace collected when there is cross CPU IPI. >> >> [000] 154.892148: sched_waking: comm=sugov:4 pid=491 prio=-1 target_cpu=007 >> [000] 154.892148: ipi_raise: target_mask=00000000,00000080 (Function call interrupts) >> [007] 154.892162: cpu_idle: state=2 cpu_id=7 >> [007] 154.892208: cpu_idle: state=4294967295 cpu_id=7 >> [007] 154.892211: irq_handler_entry: irq=2 name=IPI >> [007] 154.892211: ipi_entry: (Function call interrupts) >> [007] 154.892213: sched_wakeup: comm=sugov:4 pid=491 prio=-1 target_cpu=007 >> [007] 154.892214: ipi_exit: (Function call interrupts) >> >> This impacts performance and the above count increments. >> >> commit ccde6525183c ("smp: Introduce a helper function to check for pending >> IPIs") already introduced a helper function to check the pending IPIs and >> it is used in pmdomain governor to deny the cluster level idle state when >> there is a pending IPI on any of cluster CPUs. >> >> This however does not stop CPU to enter CPU level idle state. Make use of >> same at CPUidle to deny the idle entry when there is already IPI pending. >> >> With change observing glmark2 [1] off screen scores improving in the range >> of 25% to 30% on Qualcomm lemans-evk board which is arm64 based having two >> clusters each with 4 CPUs. >> >> [1] https://github.com/glmark2/glmark2 >> >> Signed-off-by: Maulik Shah >> --- >> drivers/cpuidle/cpuidle.c | 3 +++ >> 1 file changed, 3 insertions(+) >> >> diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c >> index c7876e9e024f9076663063ad21cfc69343fdbbe7..c88c0cbf910d6c2c09697e6a3ac78c081868c2ad 100644 >> --- a/drivers/cpuidle/cpuidle.c >> +++ b/drivers/cpuidle/cpuidle.c >> @@ -224,6 +224,9 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev, >> bool broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP); >> ktime_t time_start, time_end; >> >> + if (cpus_peek_for_pending_ipi(drv->cpumask)) >> + return -EBUSY; > > As other reviews already pointed out, this must be called only for the > current CPU. Yes, addressing in v2. > > That said, did you play with bailing out just before the call to the > target_state->enter()? It would be interesting to know if that changes > the "stats" somehow. Yes, i did play moving this and "stats" do change with differences in next idle state selection. below is high level scenario happening when the IPI deny change is placed inside target_state->enter() or any place which will update the rejected count. Using a menu governor, entered_state = target_state->enter(dev, drv, index); - entered_state will be negative error when IPI is pending. - This makes rejected count increment (which is okay) but it also makes dev->last_residency_ns = 0; - For next idle entry, menu governor will log this last interval as failed, via menu_select() -> menu_update_intervals(data, UINT_MAX); this is because menu_reflect() is not invoked for failed entry. - This makes 1 of last 8 intervals invalid, which don't get accounted from get_typical_interval(), hence divisor value in this API is never reaching 8, the goto again, loop inside get_typical_interval() will get aborted "faster". This interval logging have good influence on next 8 idle entries until it will get replaced with meaningful value. - In ideal case get_typical_interval() would go from divisor value reaching 8, 7 and then 6 and once its down to 6, it gets aborted if not predicted yet, so maximum 3 trials, but with any interval invalid it finishes faster, often without prediction. - sometimes IPI bailouts may happen frequently, in such cases we have 3 to 4 intervals as invalid in history, effectively making get_typical_interval() not predicting since divisor value in first loop would be less than 6. - Not predicting via get_typical_interval() can make deeper idle selection more often by only going with adjusted sleep lengths. In summary, The benefits seen by aborting the idle entry with IPI pending getting nullified when it is logged as "rejected" due to next idle entries can go deeper. If we ignore setting dev->last_residency_ns = 0 for rejected cases or make menu_select() ignore the last rejected iteration to update in intervals history, this would improve the chances of get_typical_interval() to predict a meaningful sleep length (in a separate change). For this change, IMO its better to bail out early without logging, since CPU did not make entry to idle driver yet. Thanks, Maulik > >> + >> instrumentation_begin(); >> >> /* >> >> --- >> base-commit: b84a0ebe421ca56995ff78b66307667b62b3a900 >> change-id: 20260316-cpuidle_ipi-4c64036f9a48 >> >> Best regards, >> -- >> Maulik Shah >> > > Kind regards > Uffe