From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27995397E97 for ; Mon, 23 Mar 2026 12:13:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774268006; cv=none; b=jtXcM6iq16q/MbN7BWOQHE2Yj3wGsCD3TichSNWLfbR6wdsgyYBzleXnx7Ej3+DC8ppO+JPjvPTD6/PPGZveqLiR/zMnW7QQIlmL5wULYfqE3Rl5szkExyUH9jQe42e11G8k2SMc1QJYdXjUYiA4uk08BrLLI+jqkEqE3SXt3e4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774268006; c=relaxed/simple; bh=o+JFJtu6y7N5kdyo3DdaubQZRMksn2QckUlkaFBAxlI=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=q2svueH7CPZ3S3YcoudPkXsl9sxSh1ewhtRZ4ewPIGlDMD5EHu9Ebg3hbCiTN2SzRxBzoDFaggrcw6W8Wd9lOG5op5n7CvnuBB72xOw4tqHmZtnHuGxdozIXnJwd8t9NoZdbI3KTrtITfpky7MGN4nkqCsPaozcQg4lIuXUYnko= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com; spf=pass smtp.mailfrom=oss.qualcomm.com; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b=bmb7UHgU; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b=Qh3kq8EH; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b="bmb7UHgU"; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b="Qh3kq8EH" Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62N9YXaA3737936 for ; Mon, 23 Mar 2026 12:13:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= 9kZ8mRHoal7TjebVoiJ3QW1/Vt28tdmOa/xyLh0UN7o=; b=bmb7UHgUYndyvqdS 38xJGwuRiN4KyoMiZDow2PhbsOtlhFY7sCSntZgoPsWuiOxbZv6NRZvdBVc2OXfO RAw7fk4CE7BGiBOu82MYlYrCkO/3BVsJhNoSR6Ewob8jYYdoziZKQO94cvh10oaX /LU/jZksG6VhREqdsHgnOEJSK+JLc/xQqpHG09wVjDO4KmW6ZPmVLnMe7nrJ8qYY DqydSs/j+nr1+DLSpd20p6u74aYHiJo7QGnS+alQwbh4MluoVz4e5vN4kHjzlHVE cS5KKMy8dibv1oBy9mxtxO2Sy+DCeiDsNJ5aMHHk/9gsCzz1pBztpvK7gOeIvVJ1 oDzX+A== Received: from mail-pj1-f69.google.com (mail-pj1-f69.google.com [209.85.216.69]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4d1mghd5mw-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Mon, 23 Mar 2026 12:13:23 +0000 (GMT) Received: by mail-pj1-f69.google.com with SMTP id 98e67ed59e1d1-35b94e2caf9so15048024a91.0 for ; Mon, 23 Mar 2026 05:13:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oss.qualcomm.com; s=google; t=1774268002; x=1774872802; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=9kZ8mRHoal7TjebVoiJ3QW1/Vt28tdmOa/xyLh0UN7o=; b=Qh3kq8EHXj3F4QjbCb89HPd7KRRjyXeSpj7tRZ0h3QojPSsX5WN2tsBkFJB3zaQfXo QaYZuYQa8UNULgbBnsR6sr2b7kITqTnJa1NW00me0GXds1DDwy0zZIkNgkVon2Lf6J3t 7N71soSeOgPsdrNHnh2iy9WXoeiQmUQZe8w9Kr5n1SAILHHqrsO5w9LjwVAF3x1WtsDo j9L1xnbe9I81IAOHyNm1HOuPbOXZaNJIlsEPAV+rFjXgMieZvQ1Mf3yDMFrw6BKZ883d ZoBtpukm6iP+OrsogkaRJJb+563cYWECbzCe9VWzy2KvAMW1XjdJzQxPlV2k5qIn8jf9 Bwgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774268002; x=1774872802; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9kZ8mRHoal7TjebVoiJ3QW1/Vt28tdmOa/xyLh0UN7o=; b=tMwVeGbrFu98FdpDBF0gOOZwsbzUq0dOcCy0KS5Bp/1jCRsSdnIzHZt0gXGmSZYbzR I3rK8vYL6hOiBublCLGwc/Qc5haxL2o1TNdilqQOB0go7UilPJLJDDy9v2LwD7IKlmhv QzUzMDNp8P8Rx3O9OJ++o3gLH/7E5Oc7UWlLaf0ChGj3tBOwqa0QM2MIx1YC6yaIal/0 a0BODTtmWx0J2gN1C7LORJkOs/HHCVsbvQ89rWi1KW1HL1zjnAzT54Mw32MQ9CREstsI lJP7PphWkcWd0j6sbhG9oAijdUblwNMm6Gixq9ed6hTUvFJ6uxnqNNpU0fdnCv4ZTwl5 +R3Q== X-Forwarded-Encrypted: i=1; AJvYcCUkNjut/MnAfr8woznSdOcEtcTIzXiuosgANJqq+PcfRj/s3hMobTXyLw7A/1+rGhW184jHcGNlCG6e8To=@vger.kernel.org X-Gm-Message-State: AOJu0Ywfg+q3lLEFWd1gvTEyOkDdiyYjWc8F/YdvCvmwyWY8i+7XsoAJ E2NAqFhSq1i/uGgDiGY+g9QtmD5JaDgN2TmxTN39T7YnK3xBL9MvNJc2TdhoNOVobP5sOf3Dq+I i64LfHOHiHXYwN7PAJFM2/48O6k6bqX3Ss7ozkqxmvjVkqFw6mwNpBU7grbmeiUXhiPw= X-Gm-Gg: ATEYQzwVa+QoHbea5sG+P+sAwbNu1/sMJ83Vy04NzN1y8fHwPTyn5+yghlsiv2HQkkS rz8TjJftZUwxt/PfMGkB1I6gw+SEobVvb08mBne86a6A+mq9O5UgSBeE7YUMNbkTYl9JUabrJXE fzUAFimyPOoC7/5C3UdsCtGRy84xd9qWJEGvQ8Zn7kP2ltgYOP4fa/5CWf6N+0kxAl/6TDU0W9T RoXRdlRepdzOBWqybqQN/qrlUiCIBpB3YltRAHQj+kjrfjVRgvUlBPhjvA3f2MbQwj/eHA0C67U kRdy6yuFGurBP47IxCCrZGl96Y7cMG6S30ZjSl+xh8082OGRXwGlLIe/2rip+QKtuy+2e7fgDaf 7vmi5/GSzY/Jiy8hujhtgHbTqvf+lm9yTXwXzmnqwXM31hREJec4= X-Received: by 2002:a17:90b:5203:b0:359:8e1c:542 with SMTP id 98e67ed59e1d1-35bd2d217b9mr9330692a91.18.1774268002450; Mon, 23 Mar 2026 05:13:22 -0700 (PDT) X-Received: by 2002:a17:90b:5203:b0:359:8e1c:542 with SMTP id 98e67ed59e1d1-35bd2d217b9mr9330655a91.18.1774268001912; Mon, 23 Mar 2026 05:13:21 -0700 (PDT) Received: from [10.217.198.242] ([202.46.22.19]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-35bc6017492sm12976530a91.5.2026.03.23.05.13.19 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 23 Mar 2026 05:13:21 -0700 (PDT) Message-ID: Date: Mon, 23 Mar 2026 17:43:18 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] cpuidle: Deny idle entry when CPU already have IPI interrupt pending To: "Rafael J. Wysocki" Cc: Daniel Lezcano , Christian Loehle , Ulf Hansson , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org References: <20260316-cpuidle_ipi-v1-1-d0ff6350f4e2@oss.qualcomm.com> Content-Language: en-US From: "Maulik Shah (mkshah)" In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Authority-Analysis: v=2.4 cv=HI7O14tv c=1 sm=1 tr=0 ts=69c12e63 cx=c_pps a=vVfyC5vLCtgYJKYeQD43oA==:117 a=fChuTYTh2wq5r3m49p7fHw==:17 a=IkcTkHD0fZMA:10 a=Yq5XynenixoA:10 a=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=u7WPNUs3qKkmUXheDGA7:22 a=DJpcGTmdVt4CTyJn9g5Z:22 a=NEAV23lmAAAA:8 a=EUspDBNiAAAA:8 a=lggpn7pwzXPsZ0vCzx8A:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 a=rl5im9kqc5Lf4LNbBjHf:22 X-Proofpoint-ORIG-GUID: lJPWv61qZfJuCAaRvu8bMZAKXOt4o5lJ X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzIzMDA5NSBTYWx0ZWRfX0a4QUgdRN04V jiRqO7gyeOSblky9L9n6jwSi+EbOgXxNw22lr4gIt+lcF9ZY5e9NoP/x13B44z4/gcS7Agr8t7y bllOaXbSyjtrR/RGH6pAshxwkEnPkKMpeqN2LYWXgvK7c173Wq8w4MoH3d/PjBEq9/FKSlf/JGP 49jqP8bZ01XF9KUgseYChUTnCQez2CeHh+nZujeUz/kWTB7tTgrC/yxtseOmnD6n6EmOnSOlphq 9PW7sa8I4y9sFizMOyFKIYd/6Mwqvo1M/eDQxPCUTJ2+QI0VIdlrsVaMyLQKUEr2z1a9NrxxGFu eRnnIRCJFr7n3EFM78K/48uVHdvu+vtPIanrxdb9j70/unusEg800iSjr4lweWnG9VfY/c69WHI 7GAq2k5CWzUGZvD3o0UYYFlBsC15wyV9n2fJAefXO7HnLzM5itpdGoNIj5VP6rDQ6X+fih8Q6rc 3IDSm7FjzsJxyx33/Qw== X-Proofpoint-GUID: lJPWv61qZfJuCAaRvu8bMZAKXOt4o5lJ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-23_03,2026-03-20_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 clxscore=1015 phishscore=0 malwarescore=0 bulkscore=0 impostorscore=0 suspectscore=0 spamscore=0 adultscore=0 lowpriorityscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603230095 On 3/20/2026 11:59 PM, Rafael J. Wysocki wrote: > On Mon, Mar 16, 2026 at 8:38 AM Maulik Shah > wrote: >> >> CPU can get IPI interrupt from another CPU while it is executing >> cpuidle_select() or about to execute same. The selection do not account >> for pending interrupts and may continue to enter selected idle state only >> to exit immediately. >> >> Example trace collected when there is cross CPU IPI. >> >> [000] 154.892148: sched_waking: comm=sugov:4 pid=491 prio=-1 target_cpu=007 >> [000] 154.892148: ipi_raise: target_mask=00000000,00000080 (Function call interrupts) >> [007] 154.892162: cpu_idle: state=2 cpu_id=7 >> [007] 154.892208: cpu_idle: state=4294967295 cpu_id=7 >> [007] 154.892211: irq_handler_entry: irq=2 name=IPI >> [007] 154.892211: ipi_entry: (Function call interrupts) >> [007] 154.892213: sched_wakeup: comm=sugov:4 pid=491 prio=-1 target_cpu=007 >> [007] 154.892214: ipi_exit: (Function call interrupts) >> >> This impacts performance and the above count increments. >> >> commit ccde6525183c ("smp: Introduce a helper function to check for pending >> IPIs") already introduced a helper function to check the pending IPIs and >> it is used in pmdomain governor to deny the cluster level idle state when >> there is a pending IPI on any of cluster CPUs. > > You seem to be overlooking the fact that resched wakeups need not be > signaled via IPIs, but they may be updates of a monitored cache line. > >> This however does not stop CPU to enter CPU level idle state. Make use of >> same at CPUidle to deny the idle entry when there is already IPI pending. >> >> With change observing glmark2 [1] off screen scores improving in the range >> of 25% to 30% on Qualcomm lemans-evk board which is arm64 based having two >> clusters each with 4 CPUs. >> >> [1] https://github.com/glmark2/glmark2 >> >> Signed-off-by: Maulik Shah >> --- >> drivers/cpuidle/cpuidle.c | 3 +++ >> 1 file changed, 3 insertions(+) >> >> diff --git a/drivers/cpuidle/cpuidle.c b/drivers/cpuidle/cpuidle.c >> index c7876e9e024f9076663063ad21cfc69343fdbbe7..c88c0cbf910d6c2c09697e6a3ac78c081868c2ad 100644 >> --- a/drivers/cpuidle/cpuidle.c >> +++ b/drivers/cpuidle/cpuidle.c >> @@ -224,6 +224,9 @@ noinstr int cpuidle_enter_state(struct cpuidle_device *dev, >> bool broadcast = !!(target_state->flags & CPUIDLE_FLAG_TIMER_STOP); >> ktime_t time_start, time_end; >> >> + if (cpus_peek_for_pending_ipi(drv->cpumask)) >> + return -EBUSY; >> + > > So what if the driver handles all CPUs in the system and there are > many of them (say ~500) and if IPIs occur rarely (because resched > events are not IPIs)? Missed the case of driver handling multiple CPUs, In v2 would fix this as below, which checks pending IPI on single CPU trying to enter idle. if (cpus_peek_for_pending_ipi(cpumask_of(dev->cpu))) I see IPIs do occur often, in the glmark2 offscreen case mentioned in commit text, out of total ~12.2k IPIs across all 8 CPUs, ~9.6k are function call IPIs, ~2k are IRQ work IPIs, ~560 Timer broadcast IPIs while rescheduling IPIs are only 82. Thanks, Maulik > >> instrumentation_begin(); >> >> /* >> >> ---