From: Qiliang Yuan <qiliang.yuan@linux.dev>
To: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Namhyung Kim <namhyung@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
Adrian Hunter <adrian.hunter@intel.com>,
James Clark <james.clark@linaro.org>
Cc: Tejun Heo <tj@kernel.org>,
cgroups@vger.kernel.org, linux-perf-users@vger.kernel.org,
linux-kernel@vger.kernel.org,
Qiliang Yuan <qiliang.yuan@linux.dev>
Subject: [PATCH] perf/core: Skip cgroup attach IPIs while perf_sched_events is disabled
Date: Sat, 21 Nov 2026 21:03:24 -0500 [thread overview]
Message-ID: <20261121-feat-perf-cgroup-attach-ipi-v1-1-a0e6d3cce8d1@linux.dev> (raw)
The perf_event controller is implicitly enabled on cgroup v2, so moving
tasks to another cgroup calls perf_cgroup_attach(). For every migrated
thread, it calls task_function_call(), which sends a synchronous IPI to
the CPU the thread last ran on, so that a CPU counting cgroup events
switches to the new cgroup if the thread is running there.
Without cgroup events, no CPU tracks a cgroup and perf_cgroup_switch()
returns right away, yet every thread still costs an IPI. Moving a
process with hundreds of threads, such as an inference server, then
takes milliseconds, all with cgroup_threadgroup_rwsem held for writing,
which blocks fork() and exit() on the whole system.
Return early when perf_sched_events is disabled. A cgroup event enables
it and waits for a grace period before it is installed. With a full
barrier before the key check, the grace period orders the cgroup changes
of the migrated tasks before the installation, so the event picks up the
new cgroup of the task running on its CPU.
Moving a process with sleeping threads between two cgroups 100 times,
20 ms apart, while another process forks and reaps children in a loop,
base and patched kernels alternating over 2 rounds of 3 runs, median of
the 6 runs and range in parentheses. Each move also waits about 7 ms for
a grace period to take cgroup_threadgroup_rwsem. Median time per move:
threads before (ms) after (ms) change
1 7.8 (7.3-7.9) 7.9 (7.8-7.9) +1%
16 8.8 (8.6-9.1) 7.9 (7.8-7.9) -11%
128 14.0 (13.7-14.7) 7.9 (7.8-8.0) -44%
512 25.1 (24.5-26.1) 9.9 (9.8-10.0) -61%
99th percentile of fork() and waitpid() in the other process:
threads before (ms) after (ms) change
1 3.8 (3.5-5.3) 3.6 (3.5-3.7) -5%
16 4.1 (3.8-5.0) 3.7 (3.6-3.8) -11%
128 9.8 (9.5-10.1) 3.7 (3.6-3.8) -63%
512 21.6 (20.8-22.7) 4.8 (4.7-4.9) -78%
Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
kernel/events/core.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/kernel/events/core.c b/kernel/events/core.c
index e870f12c3ad81..a5a9161bab31c 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -15617,6 +15617,17 @@ static void perf_cgroup_attach(struct cgroup_taskset *tset)
struct task_struct *task;
struct cgroup_subsys_state *css;
+ /*
+ * A cgroup event enables perf_sched_events and waits for a grace
+ * period in account_event() before it is installed, so with the key
+ * disabled no CPU tracks a cgroup yet. Order the cgroup changes of the
+ * migrated tasks before the key check, pairing with that grace period,
+ * so that the event picks up the new cgroup of the running task.
+ */
+ smp_mb();
+ if (!static_branch_unlikely(&perf_sched_events))
+ return;
+
cgroup_taskset_for_each(task, css, tset)
task_function_call(task, __perf_cgroup_move, task);
}
---
base-commit: 5f22fb20497f4983094131f00a734362300d22d2
change-id: 20261120-feat-perf-cgroup-attach-ipi-2824375d4d93
Best regards,
--
Qiliang Yuan <qiliang.yuan@linux.dev>
reply other threads:[~2026-11-22 2:03 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261121-feat-perf-cgroup-attach-ipi-v1-1-a0e6d3cce8d1@linux.dev \
--to=qiliang.yuan@linux.dev \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=cgroups@vger.kernel.org \
--cc=irogers@google.com \
--cc=james.clark@linaro.org \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®