From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-dy1-f201.google.com (mail-dy1-f201.google.com [74.125.82.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D09DE3DB319 for ; Tue, 16 Jun 2026 06:14:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781590473; cv=none; b=f20OsYNSLaFWdfg2jB9TYDhf5MAx1iHhrs9tIsJEvDEcsrUkKrriVaV83NV/9ZtxvzrOsO5tnJ4W39z1SOdXobvXFqq9pyex3RRUTeIiJ8TAHnz/Pju2gWhgNWq0KXWVbBnyklefC4AiJDhT9WWHIvKlYAjko2JWK7usGfdCPwI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781590473; c=relaxed/simple; bh=rEPXRyX0K/GCuSjm0kaZnjRIwLDhYV7ih8X/3JF6rIU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=beG+ieUaKURdMJO9Ih45ILe/8DMbJOUHtBsEWsMTpdU32iOagHDnya+uNugpgZ3eeduS2VTIUGJ3Rp/a01yC06fmFYoi8uBjTvig1kOfFk/1ZZsKqgLmEZN9S/gf730RipLuTN8r9KgZuTBpkFzv97cR+17+R7rbBeUyiVDuIbg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Cs5ImRdj; arc=none smtp.client-ip=74.125.82.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Cs5ImRdj" Received: by mail-dy1-f201.google.com with SMTP id 5a478bee46e88-308004a2c49so1141668eec.0 for ; Mon, 15 Jun 2026 23:14:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1781590471; x=1782195271; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=PssKuP4CKaena/eiTOkn4uj7AlehBZrS5ZJr/3Okq+4=; b=Cs5ImRdjAJ/wDL9fzRSmSZZzzAe9MXJ5jbi2Pe9GDJKY9A3cu/5cbYdaWUoW2j/OJX XtAVPmkH+tXSREnNawZSmF0LnN16Llru9BMVaVYmaLofbC27qp36oiSYlI0DrxygFTu9 k2FTCcsmR3pw+8SCuJmHvhEr0FP56wbDEzwq9KYIwOAl3qsftAgMj2oumnN4GHe77Lct 4waYxSVl6cIaVgd65yBBNlSvqhDqLjgNby6yhoi9wCXorisL9Jb5UV3blkFx16h5O8u2 2ioBpCHYwbFUf5KWCyJ0A0jY/sd1GnkEizEoFGczig1mU5sWgaVpz5IUwcaOUVjfkAH6 UKmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1781590471; x=1782195271; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=PssKuP4CKaena/eiTOkn4uj7AlehBZrS5ZJr/3Okq+4=; b=KAJy1OU07CEApM5h8mTnTGjU+vhyuFO3csukuX01nNH2xcfyo+igyMzwfNq0LP77uz bkYKSriAZ7cAglcr1/tpMEIIszN6E76pSyLCHI1DUjztbkN4iuiEQkGvwFrP1uESf2vb lB/aWLFSjbIihdOelf4kqW51wK3k8DB6uWPe8a2w81JNfvq3LsmRyiQcuvYhMiYakKHd Iv1NwyNuTJ0eXc9621INIECvz/brqpAUx8fuTCTyJVj39LZa16ViFV7Uoxx2AuMMAzhj Vck6vnOZ3fmHPy56UUx2cTObFqnVKeRRIPbBYeOK1SHkBhAow+L2uV2o4guFsoJghn32 G/ag== X-Forwarded-Encrypted: i=1; AFNElJ+3lDWOxcA4rLExWvujMRY4sKQSW5LvAistRiESn4GDuZog78QTIF/ue8s5YTK3JKo46VVGI+qea5y6DSs=@vger.kernel.org X-Gm-Message-State: AOJu0Yz/Ostp+KITwEemMSBT0pQMb2ZE0R1PMUFvIOvKvCXZ6WKs1DJe h9mKoBHNeo9mfj4AZxk5Yw8YSMk3qPSqJveJNTeFlIn1Z+9bZAG5g51paO//CWXsTYx6yfrQmef 3m561YbtYcw== X-Received: from dycpz7.prod.google.com ([2002:a05:7300:fe07:b0:304:e502:fe2d]) (user=irogers job=prod-delivery.src-stubby-dispatcher) by 2002:a05:7301:19a2:b0:2df:919f:ce59 with SMTP id 5a478bee46e88-30ba5ec7bb1mr1493016eec.19.1781590470759; Mon, 15 Jun 2026 23:14:30 -0700 (PDT) Date: Mon, 15 Jun 2026 23:14:03 -0700 In-Reply-To: <20260616061404.41929-1-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260616012744.4049193-1-irogers@google.com> <20260616061404.41929-1-irogers@google.com> X-Mailer: git-send-email 2.54.0.1136.gdb2ca164c4-goog Message-ID: <20260616061404.41929-12-irogers@google.com> Subject: [PATCH v2 11/12] perf tests: Speed up lock contention analysis shell test From: Ian Rogers To: irogers@google.com, acme@kernel.org, namhyung@kernel.org Cc: adrian.hunter@intel.com, james.clark@linaro.org, jolsa@kernel.org, leo.yan@arm.com, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, thomas.falcon@intel.com, tmricht@linux.ibm.com Content-Type: text/plain; charset="UTF-8" The lock contention analysis test suite (`lock_contention.sh`) performs a series of 13 separate profiling checks to verify various aggregation and filtering parameters of `perf lock contention`. Each of these checks runs the `perf bench sched messaging` messaging benchmark as its workload. By default, `sched messaging` runs 10 groups of 40 processes (400 processes total) generating substantial task scheduling, context switching, and IPC message passing. When traced system-wide for lock events, the tracing overhead (handling millions of lock acquisitions and releases) slows execution down significantly, causing the test suite to take over 80 seconds. Optimize this by introducing a scaled-down messaging benchmark workload: `perf bench sched messaging -g 1 -p`. Running 1 group (40 processes) takes only 0.01 seconds natively (instead of 0.08 seconds), drastically reduces the sheer volume of lock acquire/release trace events, and reduces CPU context switching during tracing while still generating sufficient lock events to fully exercise the BPF/record filters. Signed-off-by: Ian Rogers --- tools/perf/tests/shell/lock_contention.sh | 30 +++++++++++++---------- 1 file changed, 17 insertions(+), 13 deletions(-) diff --git a/tools/perf/tests/shell/lock_contention.sh b/tools/perf/tests/shell/lock_contention.sh index 52e8b9db9fbd..f589d387f6b3 100755 --- a/tools/perf/tests/shell/lock_contention.sh +++ b/tools/perf/tests/shell/lock_contention.sh @@ -9,6 +9,10 @@ perfdata=$(mktemp /tmp/__perf_test.perf.data.XXXXX) result=$(mktemp /tmp/__perf_test.result.XXXXX) errout=$(mktemp /tmp/__perf_test.errout.XXXXX) +# Workload to generate lock contention. +# Using 1 group (-g 1) keeps runtime low while generating sufficient lock events. +msg_workload="perf bench sched messaging -g 1 -p" + cleanup() { rm -f ${perfdata} rm -f ${result} @@ -50,7 +54,7 @@ check() { test_record() { echo "Testing perf lock record and perf lock contention" - perf lock record -o ${perfdata} -- perf bench sched messaging -p > /dev/null 2>&1 + perf lock record -o ${perfdata} -- ${msg_workload} > /dev/null 2>&1 # the output goes to the stderr and we expect only 1 output (-E 1) perf lock contention -i ${perfdata} -E 1 -q 2> ${result} if [ "$(cat "${result}" | wc -l)" != "1" ]; then @@ -70,7 +74,7 @@ test_bpf() fi # the perf lock contention output goes to the stderr - perf lock con -a -b -E 1 -q -- perf bench sched messaging -p > /dev/null 2> ${result} + perf lock con -a -b -E 1 -q -- ${msg_workload} > /dev/null 2> ${result} if [ "$(cat "${result}" | wc -l)" != "1" ]; then echo "[Fail] BPF result count is not 1:" "$(cat "${result}" | wc -l)" err=1 @@ -81,7 +85,7 @@ test_bpf() test_record_concurrent() { echo "Testing perf lock record and perf lock contention at the same time" - perf lock record -o- -- perf bench sched messaging -p 2> ${errout} | \ + perf lock record -o- -- ${msg_workload} 2> ${errout} | \ perf lock contention -i- -E 1 -q 2> ${result} if [ "$(cat "${result}" | wc -l)" != "1" ]; then echo "[Fail] Recorded result count is not 1:" "$(cat "${result}" | wc -l)" @@ -107,7 +111,7 @@ test_aggr_task() fi # the perf lock contention output goes to the stderr - perf lock con -a -b -t -E 1 -q -- perf bench sched messaging -p > /dev/null 2> ${result} + perf lock con -a -b -t -E 1 -q -- ${msg_workload} > /dev/null 2> ${result} if [ "$(cat "${result}" | wc -l)" != "1" ]; then echo "[Fail] BPF result count is not 1:" "$(cat "${result}" | wc -l)" err=1 @@ -130,7 +134,7 @@ test_aggr_addr() fi # the perf lock contention output goes to the stderr - perf lock con -a -b -l -E 1 -q -- perf bench sched messaging -p > /dev/null 2> ${result} + perf lock con -a -b -l -E 1 -q -- ${msg_workload} > /dev/null 2> ${result} if [ "$(cat "${result}" | wc -l)" != "1" ]; then echo "[Fail] BPF result count is not 1:" "$(cat "${result}" | wc -l)" err=1 @@ -148,7 +152,7 @@ test_aggr_cgroup() fi # the perf lock contention output goes to the stderr - perf lock con -a -b --lock-cgroup -E 1 -q -- perf bench sched messaging -p > /dev/null 2> ${result} + perf lock con -a -b --lock-cgroup -E 1 -q -- ${msg_workload} > /dev/null 2> ${result} if [ "$(cat "${result}" | wc -l)" != "1" ]; then echo "[Fail] BPF result count is not 1:" "$(cat "${result}" | wc -l)" err=1 @@ -170,7 +174,7 @@ test_type_filter() return fi - perf lock con -a -b -Y spinlock -q -- perf bench sched messaging -p > /dev/null 2> ${result} + perf lock con -a -b -Y spinlock -q -- ${msg_workload} > /dev/null 2> ${result} if [ "$(grep -c -v spinlock "${result}")" != "0" ]; then echo "[Fail] BPF result should not have non-spinlocks:" "$(cat "${result}")" err=1 @@ -202,7 +206,7 @@ test_lock_filter() return fi - perf lock con -a -b -L tasklist_lock -q -- perf bench sched messaging -p > /dev/null 2> ${result} + perf lock con -a -b -L tasklist_lock -q -- ${msg_workload} > /dev/null 2> ${result} if [ "$(grep -c -v "${test_lock_filter_type}" "${result}")" != "0" ]; then echo "[Fail] BPF result should not have non-${test_lock_filter_type} locks:" "$(cat "${result}")" err=1 @@ -241,7 +245,7 @@ test_stack_filter() return fi - perf lock con -a -b -S unix_stream -E 1 -q -- perf bench sched messaging -p > /dev/null 2> ${result} + perf lock con -a -b -S unix_stream -E 1 -q -- ${msg_workload} > /dev/null 2> ${result} if [ "$(cat "${result}" | wc -l)" != "1" ]; then echo "[Fail] BPF result should have a lock from unix_stream:" "$(cat "${result}")" err=1 @@ -269,7 +273,7 @@ test_aggr_task_stack_filter() return fi - perf lock con -a -b -t -S unix_stream -E 1 -q -- perf bench sched messaging -p > /dev/null 2> ${result} + perf lock con -a -b -t -S unix_stream -E 1 -q -- ${msg_workload} > /dev/null 2> ${result} if [ "$(cat "${result}" | wc -l)" != "1" ]; then echo "[Fail] BPF result should have a task from unix_stream:" "$(cat "${result}")" err=1 @@ -285,7 +289,7 @@ test_cgroup_filter() return fi - perf lock con -a -b --lock-cgroup -E 1 -F wait_total -q -- perf bench sched messaging -p > /dev/null 2> ${result} + perf lock con -a -b --lock-cgroup -E 1 -F wait_total -q -- ${msg_workload} > /dev/null 2> ${result} if [ "$(cat "${result}" | wc -l)" != "1" ]; then echo "[Fail] BPF result should have a cgroup result:" "$(cat "${result}")" err=1 @@ -293,7 +297,7 @@ test_cgroup_filter() fi cgroup=$(cat "${result}" | awk '{ print $3 }') - perf lock con -a -b --lock-cgroup -E 1 -G "${cgroup}" -q -- perf bench sched messaging -p > /dev/null 2> ${result} + perf lock con -a -b --lock-cgroup -E 1 -G "${cgroup}" -q -- ${msg_workload} > /dev/null 2> ${result} if [ "$(cat "${result}" | wc -l)" != "1" ]; then echo "[Fail] BPF result should have a result with cgroup filter:" "$(cat "${cgroup}")" err=1 @@ -328,7 +332,7 @@ test_csv_output() fi # the perf lock contention output goes to the stderr - perf lock con -a -b -E 1 -x , --output ${result} -- perf bench sched messaging -p > /dev/null 2>&1 + perf lock con -a -b -E 1 -x , --output ${result} -- ${msg_workload} > /dev/null 2>&1 output=$(grep -v "^#" ${result} | tr -d -c , | wc -c) if [ "${header}" != "${output}" ]; then echo "[Fail] BPF result does not match the number of commas: ${header} != ${output}" -- 2.54.0.1136.gdb2ca164c4-goog