From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F85822A7E9 for ; Tue, 2 Dec 2025 07:13:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.168.131 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764659600; cv=none; b=DOeNkWiBcLoqQfDf9yWfK6ReACpiR8emCJzbpJb3GFs2Y7XLmqBegV0LHK41aoBF/yfaZAWjcUF/26PJxbQ48VelNyoUQMw/ebqYYackWSl5lzoJsaOcvEotqVmh30vuyZVX2PnlBTYIRZ0/Apl9uoZVzXZ6w0CFDatC228TU30= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764659600; c=relaxed/simple; bh=5x0U6TJ/uk/Fa7eQkXCOkVXa1W7ieyVH0Pco5C2egN4=; h=Message-ID:Date:MIME-Version:To:Cc:From:Subject:Content-Type; b=pnMwjQaJFaL8ebTim+ZIPFZco09LmhcJkUafViBYB6M/ikkyvTc8HQgJ4a5eVuqpq6R9JL5IP5aiQPjV8yJ6FGeaNjcVPDQlopDLBk/Qv3Df8/xPoirKG6Y2J9QT2Ker6L0hugCZHY91QFD084cHWlJMLnk7we5F7jyDx+H9naU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com; spf=pass smtp.mailfrom=oss.qualcomm.com; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b=ZD9IfqIZ; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b=T5ypth/H; arc=none smtp.client-ip=205.220.168.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oss.qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=qualcomm.com header.i=@qualcomm.com header.b="ZD9IfqIZ"; dkim=pass (2048-bit key) header.d=oss.qualcomm.com header.i=@oss.qualcomm.com header.b="T5ypth/H" Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 5B26l5Tk1147219 for ; Tue, 2 Dec 2025 07:13:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qualcomm.com; h= cc:content-transfer-encoding:content-type:date:from:message-id :mime-version:subject:to; s=qcppdkim1; bh=hewxWIB3R5yRqjpdL8Ute0 t9X5WK4bF7onrLh9soNKE=; b=ZD9IfqIZEe2dmAZzchYfrlineXHxdnSZmcxv9z 5kHN7OYt8dvXM8pnWhXSTcd6Hj6OmPSyGrrJSz2IdLU+ycGlpInAr+pDlo0TAA+v 1P9S5ycscaVXME8Uh8HEBEeSwen03uhsMvpiCiZLQMH6nMbBKFZFWyg682DcDEqx OGMdK+s8vVr/w/61c1fuxoWk3028wjAS40S5c8ig64yBCK/U4FDenI98Y0vRrSbO 3BC1cJvpBZTlUTyrtlLTPE9aiia9BsU6OJojeLV6zuJg9xCK1H1p0rh6tWWwADMn VgHDdyyeADsgiNpQpOiD2Pd+2nc8f3h7liZO4strvqWwvDIw== Received: from mail-pj1-f71.google.com (mail-pj1-f71.google.com [209.85.216.71]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 4asj5e9mgd-1 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Tue, 02 Dec 2025 07:13:18 +0000 (GMT) Received: by mail-pj1-f71.google.com with SMTP id 98e67ed59e1d1-34188ba5990so12677103a91.0 for ; Mon, 01 Dec 2025 23:13:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oss.qualcomm.com; s=google; t=1764659598; x=1765264398; darn=vger.kernel.org; h=content-transfer-encoding:subject:from:cc:to:content-language :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=hewxWIB3R5yRqjpdL8Ute0t9X5WK4bF7onrLh9soNKE=; b=T5ypth/H7dR1oWKSd4E2PbvjKJrYEKKqfHKI5plfBcwSHlwP4FHt7m8S2b3xi5Cwhj GT7qqNxPtNUMPRTASeXREZGIUvk7N1oaLJv8kPeHdZfN0c6Mx1lcyCjblM11nPMmKWEm Wxrg+/I68x/LithgRj9/AIxDr73TJ6vaBJjG+/6pgkgiwLilpIyd3C/8etT6NW8dQaQj izfwCKyj4vStYZsFz4RY49pTe9n0VV/ZH5IVXLHxG8zBTzUQauWjIxPfifPwTSEcMBNs MhmrPS7ZzcbMjxZHs5Gtk4QGSB5KUZdjiupltqlJVzzHCRyHWYoMBL6FTFoiAEORJoir Cgpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1764659598; x=1765264398; h=content-transfer-encoding:subject:from:cc:to:content-language :user-agent:mime-version:date:message-id:x-gm-gg:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=hewxWIB3R5yRqjpdL8Ute0t9X5WK4bF7onrLh9soNKE=; b=HdHSJ3w/g4rkWi+X5RcbPrcdqVqqz8i0TKCAwW/IiXF+EriA+wa/zaa794XRCQwLTi NPZz8Sf7I94KFBhxqkYzYtkTrIHPG7YVUWFAJoHH68duo9p7g3xn3AEyDXPRxTCLE/Zp WtTZkkvbSEidlx2IuKwkpclqdW1c0ZXmP8IIySkCJL9eaRUnZvCvEJ7OZpO39l8fLOhI Hpg1t+8VclJFFLPjBnjjymig3DgZausP4aczYW9SwFsPd/PLgx48RYerXqbD08w/41bP 1hHZ8NmRgIRQ08JmiKR+6Hz0ZXD+BZ24heqA3GNxeiL/FL4Lqe8kJQVDpHSEJ9x0R2xX ejVg== X-Forwarded-Encrypted: i=1; AJvYcCVS3P4FasbHybyHafxFi6v/pODh4mB6joVZgSTCSuSLzyisVQ6tg8plQBc8gM5W2VEQvlYoyFGbnkRudSI=@vger.kernel.org X-Gm-Message-State: AOJu0Yw5WBDlWa5+5A3o+45KcXRg0ZqKGwdsrAbHt1hc/ck8qlPkrtkm +cygyXVBsiYmynGTfwF0EAKlUYqkA40/5Gigk84m3Pd5Zx2jsyYyC6fZavoS2rVw/9d1ZXGYc3F CvAyrZHk/53QDLT/iV5lPjll5ld1JZ3bq51pMfLW8OS8suiFMvDzgXQisXWleWLvwo/M= X-Gm-Gg: ASbGnctLTth7cPC2/nG0lJqIenljh0CRJ5EthnB9NNBo0iYrncbX5bkVifFHmIvnwMO Fts8YIpsBbV+TUTT7g0bAh5X4/4mOtlXp7dsxD5ECsZj/OiRmDVesaOa8IbyVUNV6RvQxNodbq+ Bhf0qFR94O1GTAIUF/UrzIPVRNfnYurEqaSkmcwZmjCmGOPV0n1CTqz7yHgp7TZVVW1ysRPQiMh 4BZBbMwRPJX99T1+NESVofJkLnla2ICDgwx/kODQ2OeGAg0ySld+YSV4SPATdL+BLiHjLyExNrC JkFSTlyTtcUjd5Y2mnTZhbhReoMUNbjZRfCNi9tEvSUg8zrnzKGTuBoYouulhdeiwo4XtAAS5xQ RKw3aF0epYFbpEvcOvNhfWTLh1Z7qkYAH6jW5yTe1y7lykxbDk2awfNdMUdideeL5Q4a2tbOUag == X-Received: by 2002:a17:90b:388a:b0:340:cb18:922 with SMTP id 98e67ed59e1d1-3475ebe8306mr30703945a91.14.1764659597713; Mon, 01 Dec 2025 23:13:17 -0800 (PST) X-Google-Smtp-Source: AGHT+IHDuozONxBMeTBW+jcOSVoCypye7xFKYbz8aNGS3md16rvW5c1nNMygoRo4d7+TmHsfTDKKVw== X-Received: by 2002:a17:90b:388a:b0:340:cb18:922 with SMTP id 98e67ed59e1d1-3475ebe8306mr30703919a91.14.1764659597240; Mon, 01 Dec 2025 23:13:17 -0800 (PST) Received: from [10.133.33.100] (tpe-colo-wan-fw-bordernet.qualcomm.com. [103.229.16.4]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3477b20c00asm15183228a91.8.2025.12.01.23.13.15 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 01 Dec 2025 23:13:16 -0800 (PST) Message-ID: <62eb142e-329b-4faa-8750-2d92d4a37d3c@oss.qualcomm.com> Date: Tue, 2 Dec 2025 15:13:12 +0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: yin.li@oss.qualcomm.com Cc: linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, quic_okukatla@quicinc.com From: Yin Li Subject: test-by-test Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Proofpoint-GUID: 5VI5Eu6AiU5x2jH4066Qy_GC4dU-PSfM X-Authority-Analysis: v=2.4 cv=GMsF0+NK c=1 sm=1 tr=0 ts=692e918e cx=c_pps a=UNFcQwm+pnOIJct1K4W+Mw==:117 a=nuhDOHQX5FNHPW3J6Bj6AA==:17 a=IkcTkHD0fZMA:10 a=wP3pNCr1ah4A:10 a=s4-Qcg_JpJYA:10 a=VkNPw1HP01LnGYTKEx00:22 a=WJEcxuyqiWSRgSB5JEoA:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 a=uKXjsCUrEbL0IQVhDsJ9:22 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMjAyMDA1NiBTYWx0ZWRfX4oMlk83z+/N9 hhTyjPPZm6NBAewkxm5ac4+/mHKIXaiRx/R8NFRTsuzjgsgpHhDZgWjeo19+SdlfO9xHMxW3eyQ 8bC8R1MPbu/oZsSazdAoUz8rC25bFeNR/xArMCxhZQuJV4+gKGqbDSmKLZf/xdA6bt9G8pqQw9o y/FF7SWxJGf6yJQicxa3U83077bW/ZdPwRwGISzKoMVr9T0CY5sFJ+NjgVgFpChRzcgyq/0pP7F SvK9QvBzKCF+tr/t6ynwdoCpM0u+ge9CQrdU8WgrXfv+OHYtLvaGgJoXxE7oEaxdxBlwVLrWapl RLqrLAU3Z0Wv5dxiFFLF7wZ4NnTMwQ+BOROaHw1nnhvRf4GHl3hc8FymW+AsZU+Dyg03Pc+GUrP 5IHHjpwDBNoV0a+R2ND9bR/nQknYIw== X-Proofpoint-ORIG-GUID: 5VI5Eu6AiU5x2jH4066Qy_GC4dU-PSfM X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-28_08,2025-11-27_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 lowpriorityscore=0 bulkscore=0 malwarescore=0 impostorscore=0 suspectscore=0 clxscore=1011 adultscore=0 phishscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2510240001 definitions=main-2512020056 Hi Georgi, on 16 May 2025 18:50:15 +0300, Georgi Djakov wrote: >Hi Mike, >... > >> To prevent this priority inversion, switch to using rt_mutex for >> icc_bw_lock. This isn't needed for icc_lock since that's not used in the >> critical, latency-sensitive voting paths. > >If the issue does not occur anymore with this patch, then this is a good >sign, but we still need to get some numbers and put them in the commit >message. We constructed a priority inversion test scenario, which included multiple real-time threads with different priorities and CFS threads with different nice values ​​competing for a mutex, to verify the overhead of the RT thread acquiring the lock mutex. The maximum, minimum, and average of overhead were determined through 100 iterations of testing. Then replace the mutex with an rt-mutex and perform the same test, obtaining the overhead's max, min, and average through 100 loops. Calculate the change in average. Finally we can draw the conclusion: 1) In a scenario where the overhead of threads competing for a mutex is set to 5ms, using a mutex will result in an average overhead of 4127687ns for the tested rt threads to acquire the mutex. 2) After replacing the mutex with rt-mutex, the latency can be reduced to 2010555ns, which greatly improves the mutex overhead brought by priority inversion and reduces latency by about 50%. 3) Furthermore, to align with the user's given overhead of 40ms, the test case was modified to have a competing mutex thread overhead of 40ms, and the experiment was repeated, yielding similar results. >The RT mutexes add some overhead and complexity that could >increase latency for both uncontended and contended paths. I am curious >if there is any regression for the non-priority scenarios. Also if >there >are many threads, the mutex cost itself could become a bottleneck. After testing, the overhead of a single rt-mutex is approximately 937ns, and the overhead of a single mutex is approximately 520ns. The overhead of a single rt-mutex does indeed lead to more latency. However, in scenarios where multiple clients frequently access the interconnect API, the latency of using mutexes far outweighs the overhead added by rt-mutexes themselves. Compared to the performance improvement of rt-mutex in a thread-contention environment, the latency itself is perfectly acceptable. >This pulls in unconditionally all the RT-mutex stuff, which some people >might not want (although today it's also selected by the I2C subsystem >for example). I am wondering if we should make it configurable with the >normal mutex being the default or just follow the i2c example... but >maybe we can decide this when we have some numbers. Making locks configurable is not a common practice. We do not intend to make changes in this patch. on 7 Sep 2022 08:15:21 +0000, David Laigh wrote: >From: Georgi Djakov > ... >I can't see why the RT kernel doesn't have exactly the same issues. >Given how long a process switch takes I really suspect that most >spinlocks should remain spinlocks. It was proposed that serializing with a spinlock might be a simpler solution, but we cannot do that as holding the lock we call wait_for_completion_timeout in the RPM/RPMh which takes a mutex and could sleep in the atomic context. -- Thx and BRs, Yin