[PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions

[PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions

[Ian Rogers]

Under system-wide (-a) parallel streaming mode (--threads=cpu),
background recording threads can be inundated by a continuous
firehose of hardware samples generated by the OS. In this state,
a background thread's local hit count remains unequal to its
sample count, causing it to bypass the blocking fdarray__poll()
call entirely on each iteration of its recording loop.

Because the termination check relies on the POLLHUP event status
populated specifically by fdarray__poll(), bypassing it prevents
the background thread from ever recognizing that its control pipe
was closed by the main thread. This traps the background thread
in an infinite recording loop, hanging the main thread indefinitely
as it awaits a termination acknowledgment that never arrives.

Ensure teardown completion by adding explicit evlist__disable()
calls in the main thread's cleanup paths at out_child: and
out_child_no_flush:. Additionally, patch fdarray__filter() to
respect the fdarray_flag__nonfilterable flag, preventing it
from incorrectly setting the background thread's control pipe
file descriptor to -1 and clearing its revents mask upon
processing termination POLLHUP signals.

Fixes: f94563fac269 ("perf record: fix poll storm when monitored threads exit")
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/lib/api/fd/array.c    | 6 ++++--
 tools/perf/builtin-record.c | 2 ++
 2 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/tools/lib/api/fd/array.c b/tools/lib/api/fd/array.c
index ffe8272af59b..16a047f1906e 100644
--- a/tools/lib/api/fd/array.c
+++ b/tools/lib/api/fd/array.c
@@ -115,6 +115,9 @@ int fdarray__filter(struct fdarray *fda, short revents,
 		return 0;
 
 	for (fd = 0; fd < fda->nr; ++fd) {
+		if (fda->priv[fd].flags & fdarray_flag__nonfilterable)
+			continue;
+
 		if (!fda->entries[fd].events)
 			continue;
 
@@ -132,8 +135,7 @@ int fdarray__filter(struct fdarray *fda, short revents,
 			continue;
 		}
 
-		if (!(fda->priv[fd].flags & fdarray_flag__nonfilterable))
-			++nr;
+		++nr;
 	}
 
 	return nr;
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index ebd3ed0c9b3e..d1276382b77a 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -2890,11 +2890,13 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 		record__synthesize_workload(rec, true);
 
 out_child:
+	evlist__disable(rec->evlist);
 	record__stop_threads(rec);
 	record__mmap_read_all(rec, true);
 	goto out_free_threads;
 out_child_no_flush:
 	/* mmap read already failed — retrying would just fail again */
+	evlist__disable(rec->evlist);
 	record__stop_threads(rec);
 out_free_threads:
 	record__free_thread_data(rec);
-- 
2.55.0.795.g602f6c329a-goog

[PATCH v1 2/2] perf cap: If capability is missing still perform root test

[Ian Rogers]

Refine the capability checking logic in perf_cap__capable() to in the
event the capability is missing do the fallback used_root
test. Previously this was only performed when the capability syscall
failed.

Fixes: e25ebda78e23 ("perf cap: Tidy up and improve capability testing")
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/cap.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/cap.c b/tools/perf/util/cap.c
index ac6d1d9a523d..a2baecbc3145 100644
--- a/tools/perf/util/cap.c
+++ b/tools/perf/util/cap.c
@@ -20,6 +20,7 @@ bool perf_cap__capable(int cap, bool *used_root)
 	};
 	struct __user_cap_data_struct data[MAX_LINUX_CAPABILITY_U32S] = {};
 	__u32 cap_val;
+	bool has_cap = false;
 
 	*used_root = false;
 	while (syscall(SYS_capget, &header, &data[0]) == -1) {
@@ -37,12 +38,21 @@ bool perf_cap__capable(int cap, bool *used_root)
 	if (cap >= 32) {
 		if (header.version == _LINUX_CAPABILITY_VERSION_3) {
 			cap_val = data[1].effective;
+			has_cap = (cap_val & (1U << (cap & 0x1f))) != 0;
 		} else {
 			/* Capability beyond 32 is requested but only 32 are supported. */
-			return false;
+			has_cap = false;
 		}
 	} else {
 		cap_val = data[0].effective;
+		has_cap = (cap_val & (1U << (cap & 0x1f))) != 0;
 	}
-	return (cap_val & (1 << (cap & 0x1f))) != 0;
+
+	if (has_cap) {
+		*used_root = false;
+		return true;
+	}
+
+	*used_root = true;
+	return geteuid() == 0;
 }
-- 
2.55.0.795.g602f6c329a-goog

Re: [PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions

[Namhyung Kim]

Hi Ian,

On Thu, Jul 09, 2026 at 10:39:06PM -0700, Ian Rogers wrote:
> Under system-wide (-a) parallel streaming mode (--threads=cpu),
> background recording threads can be inundated by a continuous
> firehose of hardware samples generated by the OS. In this state,
> a background thread's local hit count remains unequal to its
> sample count, causing it to bypass the blocking fdarray__poll()
> call entirely on each iteration of its recording loop.
> 
> Because the termination check relies on the POLLHUP event status
> populated specifically by fdarray__poll(), bypassing it prevents
> the background thread from ever recognizing that its control pipe
> was closed by the main thread. This traps the background thread
> in an infinite recording loop, hanging the main thread indefinitely
> as it awaits a termination acknowledgment that never arrives.
> 
> Ensure teardown completion by adding explicit evlist__disable()
> calls in the main thread's cleanup paths at out_child: and
> out_child_no_flush:. Additionally, patch fdarray__filter() to
> respect the fdarray_flag__nonfilterable flag, preventing it
> from incorrectly setting the background thread's control pipe
> file descriptor to -1 and clearing its revents mask upon
> processing termination POLLHUP signals.
> 
> Fixes: f94563fac269 ("perf record: fix poll storm when monitored threads exit")
> Assisted-by: Antigravity:gemini-3.5-flash
> Signed-off-by: Ian Rogers <irogers@google.com>

I didn't know you already sent the fix so I was doing the same.  It
seems your fix is better as it added evlist__disable().  I'll test this
and pick up if it's ok.

Thanks,
Namhyung

> ---
>  tools/lib/api/fd/array.c    | 6 ++++--
>  tools/perf/builtin-record.c | 2 ++
>  2 files changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/lib/api/fd/array.c b/tools/lib/api/fd/array.c
> index ffe8272af59b..16a047f1906e 100644
> --- a/tools/lib/api/fd/array.c
> +++ b/tools/lib/api/fd/array.c
> @@ -115,6 +115,9 @@ int fdarray__filter(struct fdarray *fda, short revents,
>  		return 0;
>  
>  	for (fd = 0; fd < fda->nr; ++fd) {
> +		if (fda->priv[fd].flags & fdarray_flag__nonfilterable)
> +			continue;
> +
>  		if (!fda->entries[fd].events)
>  			continue;
>  
> @@ -132,8 +135,7 @@ int fdarray__filter(struct fdarray *fda, short revents,
>  			continue;
>  		}
>  
> -		if (!(fda->priv[fd].flags & fdarray_flag__nonfilterable))
> -			++nr;
> +		++nr;
>  	}
>  
>  	return nr;
> diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
> index ebd3ed0c9b3e..d1276382b77a 100644
> --- a/tools/perf/builtin-record.c
> +++ b/tools/perf/builtin-record.c
> @@ -2890,11 +2890,13 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
>  		record__synthesize_workload(rec, true);
>  
>  out_child:
> +	evlist__disable(rec->evlist);
>  	record__stop_threads(rec);
>  	record__mmap_read_all(rec, true);
>  	goto out_free_threads;
>  out_child_no_flush:
>  	/* mmap read already failed — retrying would just fail again */
> +	evlist__disable(rec->evlist);
>  	record__stop_threads(rec);
>  out_free_threads:
>  	record__free_thread_data(rec);
> -- 
> 2.55.0.795.g602f6c329a-goog
> 

Re: (subset) [PATCH v1 1/2] perf record: Fix teardown hang on system-wide multi-threaded sessions

[Namhyung Kim]

On Thu, 09 Jul 2026 22:39:06 -0700, Ian Rogers wrote:
> Under system-wide (-a) parallel streaming mode (--threads=cpu),
> background recording threads can be inundated by a continuous
> firehose of hardware samples generated by the OS. In this state,
> a background thread's local hit count remains unequal to its
> sample count, causing it to bypass the blocking fdarray__poll()
> call entirely on each iteration of its recording loop.
> 
> [...]
Applied to perf-tools-next, thanks!

Best regards,
Namhyung

[PATCH v2] perf record: Fix destructor invocation and event counting in fdarray__filter

[Ian Rogers]

Fix the API-level fdarray filtering logic in fdarray__filter() following
the integration of the system-wide recording teardown fix. Rather than
bypassing the evaluation loop entirely for nonfilterable file
descriptors (which accidentally skips calling registered entry
destructors for control pipes and wakeup FDs, while creating poll storms
and reference leaks for system-wide events that also have nonfilterable
set), separate the lifecycle and masking logic.

Reorder the filtering sequence so that registered entry destructors are
safely executed first for all file descriptors. Then, utilize the
fdarray_flag__non_perf_event flag to perfectly distinguish and bypass
destructive file descriptor zeroing (.fd = -1) and revents mask
clearing for background control pipes and timers. This allows system-wide
event file descriptors to be correctly destroyed, zeroed, and filtered out upon
receiving POLLHUP/POLLERR signals without triggering busy loops, while safely
preserving control pipe teardown signaling and destructors.

Fixes: fb4751e79c45 ("perf record: Fix teardown hang on system-wide multi-threaded sessions")
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/lib/api/fd/array.c    | 6 +++---
 tools/perf/builtin-record.c | 4 +++-
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/tools/lib/api/fd/array.c b/tools/lib/api/fd/array.c
index 16a047f1906e..92d9a3fb652c 100644
--- a/tools/lib/api/fd/array.c
+++ b/tools/lib/api/fd/array.c
@@ -115,9 +115,6 @@ int fdarray__filter(struct fdarray *fda, short revents,
 		return 0;
 
 	for (fd = 0; fd < fda->nr; ++fd) {
-		if (fda->priv[fd].flags & fdarray_flag__nonfilterable)
-			continue;
-
 		if (!fda->entries[fd].events)
 			continue;
 
@@ -125,6 +122,9 @@ int fdarray__filter(struct fdarray *fda, short revents,
 			if (entry_destructor)
 				entry_destructor(fda, fd, arg);
 
+			if (fda->priv[fd].flags & fdarray_flag__non_perf_event)
+				continue;
+
 			/*
 			 * Set fd to -1 so poll() ignores this entry; otherwise
 			 * POLLHUP/POLLERR are still reported for events=0 fds
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index f58d7e3c7879..c6644dab1cfb 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -1242,7 +1242,9 @@ static int record__alloc_thread_data(struct record *rec, struct evlist *evlist)
 				goto out_free;
 			}
 			ret = fdarray__add(&thread_data[t].pollfd, thread_data[t].pipes.msg[0],
-					   POLLIN | POLLERR | POLLHUP, fdarray_flag__nonfilterable);
+					   POLLIN | POLLERR | POLLHUP,
+					   fdarray_flag__nonfilterable |
+					   fdarray_flag__non_perf_event);
 			if (ret < 0) {
 				pr_err("Failed to add descriptor to thread[%d] pollfd\n", t);
 				goto out_free;
-- 
2.55.0.141.g00534a21ce-goog