mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Qiliang Yuan <qiliang.yuan@linux.dev>
To: Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	 Arnaldo Carvalho de Melo <acme@kernel.org>,
	 Namhyung Kim <namhyung@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	 Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	 Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
	 Adrian Hunter <adrian.hunter@intel.com>,
	 James Clark <james.clark@linaro.org>
Cc: Tejun Heo <tj@kernel.org>,
	cgroups@vger.kernel.org,  linux-perf-users@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	 Qiliang Yuan <qiliang.yuan@linux.dev>
Subject: [PATCH] perf/core: Skip cgroup attach IPIs while perf_sched_events is disabled
Date: Sat, 21 Nov 2026 21:03:24 -0500	[thread overview]
Message-ID: <20261121-feat-perf-cgroup-attach-ipi-v1-1-a0e6d3cce8d1@linux.dev> (raw)

The perf_event controller is implicitly enabled on cgroup v2, so moving
tasks to another cgroup calls perf_cgroup_attach(). For every migrated
thread, it calls task_function_call(), which sends a synchronous IPI to
the CPU the thread last ran on, so that a CPU counting cgroup events
switches to the new cgroup if the thread is running there.

Without cgroup events, no CPU tracks a cgroup and perf_cgroup_switch()
returns right away, yet every thread still costs an IPI. Moving a
process with hundreds of threads, such as an inference server, then
takes milliseconds, all with cgroup_threadgroup_rwsem held for writing,
which blocks fork() and exit() on the whole system.

Return early when perf_sched_events is disabled. A cgroup event enables
it and waits for a grace period before it is installed. With a full
barrier before the key check, the grace period orders the cgroup changes
of the migrated tasks before the installation, so the event picks up the
new cgroup of the task running on its CPU.

Moving a process with sleeping threads between two cgroups 100 times,
20 ms apart, while another process forks and reaps children in a loop,
base and patched kernels alternating over 2 rounds of 3 runs, median of
the 6 runs and range in parentheses. Each move also waits about 7 ms for
a grace period to take cgroup_threadgroup_rwsem. Median time per move:

  threads        before (ms)       after (ms)   change
        1      7.8 (7.3-7.9)    7.9 (7.8-7.9)      +1%
       16      8.8 (8.6-9.1)    7.9 (7.8-7.9)     -11%
      128   14.0 (13.7-14.7)    7.9 (7.8-8.0)     -44%
      512   25.1 (24.5-26.1)   9.9 (9.8-10.0)     -61%

99th percentile of fork() and waitpid() in the other process:

  threads        before (ms)      after (ms)   change
        1      3.8 (3.5-5.3)   3.6 (3.5-3.7)      -5%
       16      4.1 (3.8-5.0)   3.7 (3.6-3.8)     -11%
      128     9.8 (9.5-10.1)   3.7 (3.6-3.8)     -63%
      512   21.6 (20.8-22.7)   4.8 (4.7-4.9)     -78%

Signed-off-by: Qiliang Yuan <qiliang.yuan@linux.dev>
---
 kernel/events/core.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/kernel/events/core.c b/kernel/events/core.c
index e870f12c3ad81..a5a9161bab31c 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -15617,6 +15617,17 @@ static void perf_cgroup_attach(struct cgroup_taskset *tset)
 	struct task_struct *task;
 	struct cgroup_subsys_state *css;
 
+	/*
+	 * A cgroup event enables perf_sched_events and waits for a grace
+	 * period in account_event() before it is installed, so with the key
+	 * disabled no CPU tracks a cgroup yet. Order the cgroup changes of the
+	 * migrated tasks before the key check, pairing with that grace period,
+	 * so that the event picks up the new cgroup of the running task.
+	 */
+	smp_mb();
+	if (!static_branch_unlikely(&perf_sched_events))
+		return;
+
 	cgroup_taskset_for_each(task, css, tset)
 		task_function_call(task, __perf_cgroup_move, task);
 }

---
base-commit: 5f22fb20497f4983094131f00a734362300d22d2
change-id: 20261120-feat-perf-cgroup-attach-ipi-2824375d4d93

Best regards,
-- 
Qiliang Yuan <qiliang.yuan@linux.dev>


                 reply	other threads:[~2026-11-22  2:03 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261121-feat-perf-cgroup-attach-ipi-v1-1-a0e6d3cce8d1@linux.dev \
    --to=qiliang.yuan@linux.dev \
    --cc=acme@kernel.org \
    --cc=adrian.hunter@intel.com \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=cgroups@vger.kernel.org \
    --cc=irogers@google.com \
    --cc=james.clark@linaro.org \
    --cc=jolsa@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=peterz@infradead.org \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®