* [REGRESSION] apparmor: AF_UNIX datagram send slowdown after 6456ccbd2ff7
@ 2026-08-05 17:33 Chengfeng Lin
2026-08-05 23:45 ` John Johansen
0 siblings, 1 reply; 3+ messages in thread
From: Chengfeng Lin @ 2026-08-05 17:33 UTC (permalink / raw)
To: john.johansen, georgia.garcia
Cc: paul, jmorris, serge, apparmor, linux-security-module,
linux-kernel, regressions
Hi John and Georgia,
I found a repeatable AF_UNIX datagram send slowdown caused by the exact source
delta of 6456ccbd2ff7 ("apparmor: fix regression in fs based unix sockets when
using old abi"). In a bare-metal parent/child/parent test, a standalone
sendmmsg() workload was 15.30% slower after the change. An independent larger
runner reproduced the slowdown with both sendmmsg() and io_uring SEND, so this
is not specific to io_uring.
#regzbot introduced: 6456ccbd2ff7
#regzbot title: apparmor AF_UNIX datagram send slowdown
The standalone workload uses socketpair(AF_UNIX, SOCK_DGRAM). Each sendmmsg()
call sends 32 messages of 128 bytes. Peer draining and payload validation are
outside the timed region. Each point had 3 warm-up rounds followed by 15
measured rounds of 65,536 messages.
Because 6456ccbd2ff7 was part of an AppArmor series, I isolated its source
delta on one controlled baseline:
parent 0bfa1c2da7a8: AppArmor topic prefix through 50d56a1a366a
child 30cb02a874b4: parent plus only the exact two-file 6456ccbd2ff7 diff
The child tree matches the same baseline with the AppArmor prefix advanced
through 6456ccbd2ff7. I used a fresh boot for each point:
point mean ns/message
parent A 343.382
child 395.831
parent B 343.259
The child was 15.295% slower than the parent midpoint. Dropping the first
measured round gave 15.289%; parent drift was -0.036%, and the maximum CV was
0.137%. All 45 measured rows passed the payload, length, count, and empty-queue
checks.
The machine was an Intel Core i7-12700KF system with 32 GiB RAM. The workload
was pinned to P-core CPU 2, with governor and EPP set to performance and Turbo
disabled. All three runs used actual preempt=none and reported
/proc/self/attr/current as unconfined.
As a separate check, the larger original runner used matched actual
preempt=full. Its direct sendmmsg() profile was 14.841% slower, and its
io_uring SEND profile was 17.953% slower. I am keeping these results separate
from the standalone percentage.
A separate v6.16.12 -> v6.17.13 perf comparison was directionally consistent.
The children overhead reported for security_unix_may_send was 0.27% in both
v6.16.12 controls and 6.77% at v6.17.13. The latter included the path through
apparmor_unix_may_send(), aa_unix_peer_perm(), and unix_peer_perm(). This was
a release-endpoint comparison rather than the exact source pair above, so I
use it only as supporting path evidence, not as an internal decomposition of
the 15.295% slowdown.
I understand that 6456ccbd2ff7 fixes a real old AppArmor policy ABI issue,
so I am not suggesting a revert. Would it be possible to retain that fix while
avoiding the added cost on this unconfined AF_UNIX datagram send path?
Evidence bundle:
https://github.com/lcf0399/linux-regression-evidence/tree/7b9862c7ac2d5f9622725912d7f0df95011bcbfd/apparmor-af-unix-send-old-abi-6456cc
Thanks,
Chengfeng
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [REGRESSION] apparmor: AF_UNIX datagram send slowdown after 6456ccbd2ff7
2026-08-05 17:33 [REGRESSION] apparmor: AF_UNIX datagram send slowdown after 6456ccbd2ff7 Chengfeng Lin
@ 2026-08-05 23:45 ` John Johansen
2026-09-22 16:27 ` Chengfeng Lin
0 siblings, 1 reply; 3+ messages in thread
From: John Johansen @ 2026-08-05 23:45 UTC (permalink / raw)
To: Chengfeng Lin, georgia.garcia
Cc: paul, jmorris, serge, apparmor, linux-security-module,
linux-kernel, regressions
On 8/5/26 10:33, Chengfeng Lin wrote:
> Hi John and Georgia,
>
> I found a repeatable AF_UNIX datagram send slowdown caused by the exact source
> delta of 6456ccbd2ff7 ("apparmor: fix regression in fs based unix sockets when
> using old abi"). In a bare-metal parent/child/parent test, a standalone
> sendmmsg() workload was 15.30% slower after the change. An independent larger
> runner reproduced the slowdown with both sendmmsg() and io_uring SEND, so this
> is not specific to io_uring.
>
> #regzbot introduced: 6456ccbd2ff7
> #regzbot title: apparmor AF_UNIX datagram send slowdown
>
> The standalone workload uses socketpair(AF_UNIX, SOCK_DGRAM). Each sendmmsg()
> call sends 32 messages of 128 bytes. Peer draining and payload validation are
> outside the timed region. Each point had 3 warm-up rounds followed by 15
> measured rounds of 65,536 messages.
>
> Because 6456ccbd2ff7 was part of an AppArmor series, I isolated its source
> delta on one controlled baseline:
>
> parent 0bfa1c2da7a8: AppArmor topic prefix through 50d56a1a366a
> child 30cb02a874b4: parent plus only the exact two-file 6456ccbd2ff7 diff
>
> The child tree matches the same baseline with the AppArmor prefix advanced
> through 6456ccbd2ff7. I used a fresh boot for each point:
>
> point mean ns/message
> parent A 343.382
> child 395.831
> parent B 343.259
>
> The child was 15.295% slower than the parent midpoint. Dropping the first
> measured round gave 15.289%; parent drift was -0.036%, and the maximum CV was
> 0.137%. All 45 measured rows passed the payload, length, count, and empty-queue
> checks.
>
> The machine was an Intel Core i7-12700KF system with 32 GiB RAM. The workload
> was pinned to P-core CPU 2, with governor and EPP set to performance and Turbo
> disabled. All three runs used actual preempt=none and reported
> /proc/self/attr/current as unconfined.
>
> As a separate check, the larger original runner used matched actual
> preempt=full. Its direct sendmmsg() profile was 14.841% slower, and its
> io_uring SEND profile was 17.953% slower. I am keeping these results separate
> from the standalone percentage.
>
> A separate v6.16.12 -> v6.17.13 perf comparison was directionally consistent.
> The children overhead reported for security_unix_may_send was 0.27% in both
> v6.16.12 controls and 6.77% at v6.17.13. The latter included the path through
> apparmor_unix_may_send(), aa_unix_peer_perm(), and unix_peer_perm(). This was
> a release-endpoint comparison rather than the exact source pair above, so I
> use it only as supporting path evidence, not as an internal decomposition of
> the 15.295% slowdown.
>
> I understand that 6456ccbd2ff7 fixes a real old AppArmor policy ABI issue,
> so I am not suggesting a revert. Would it be possible to retain that fix while
> avoiding the added cost on this unconfined AF_UNIX datagram send path?
>
thanks for the report, we will have to investigate and get back to you on this,
generally speaking we try to keep the mediation overhead for unconfined tasks
to a minimum using early bailout and some caching. The referenced commit does
look to be doing early bailout, so we are going to have to do some analysis, on
where exactly the overhead is coming from.
I can't promise we can get it back to what it was before but we should be able
to do better than the current regression.
> Evidence bundle:
>
> https://github.com/lcf0399/linux-regression-evidence/tree/7b9862c7ac2d5f9622725912d7f0df95011bcbfd/apparmor-af-unix-send-old-abi-6456cc
>
> Thanks,
> Chengfeng
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [REGRESSION] apparmor: AF_UNIX datagram send slowdown after 6456ccbd2ff7
2026-08-05 23:45 ` John Johansen
@ 2026-09-22 16:27 ` Chengfeng Lin
0 siblings, 0 replies; 3+ messages in thread
From: Chengfeng Lin @ 2026-09-22 16:27 UTC (permalink / raw)
To: John Johansen
Cc: georgia.garcia, paul, jmorris, serge, apparmor,
linux-security-module, linux-kernel, regressions
[-- Attachment #1: Type: text/plain, Size: 4284 bytes --]
Hi John,
I tested a small early-return prototype on v7.3-rc4 (93f51579e7df).
It reduced send syscall time by 12.84% in the original unconfined
socketpair benchmark. A separate two-process test found a 1.25% cost
when both peers were confined but allowed to communicate.
The attached patch returns early from aa_unix_peer_perm() only when the
whole label is unconfined. It skips peer-address and audit preparation.
The sender and receiver are still checked separately. The patch adds no
per-object fields and does not change label updates or locking.
For each prototype comparison, I used original -> patched -> original,
with a separate boot for each point. Both kernels used the same baseline,
GCC 15.2.0 and config, except for the kernel release suffix
(CONFIG_LOCALVERSION). The machine was the same bare-metal i7-12700KF
with 32 GiB RAM, full preemption, performance governor/EPP and Turbo off.
Each condition had three invocations per boot. Each invocation had
3 warm-up and 15 measured rounds of 65,536 messages, sent in batches of
32 messages of 128 bytes. Only sendmmsg() was timed; peer draining and
payload checks were outside the timed interval.
The original socketpair test ran on CPU 2. Results in ns/message were:
original A patched original B
416.073 359.448 408.719
The patch saved 52.95 ns/message, or 12.84%, against the original midpoint.
Maximum within-invocation CV was 0.114%; original-kernel drift was 1.78%.
The two-process test used connected abstract AF_UNIX datagram sockets,
with the sender on CPU 2 and receiver on CPU 4. Both ran as UID 1000.
The confined policies allowed communication. Results in ns/message were:
peers original midpoint patched change
both unconfined 455.564 412.052 -9.55%
sender confined only 569.902 561.786 -1.42%
receiver confined only 578.244 562.807 -2.67%
both confined 741.385 750.620 +1.25%
The socketpair and two-process tests have different workload shapes.
Their absolute timings should not be compared directly.
The both-confined case added 9.24 ns/message, with maximum CV 0.51%
and original-kernel drift 0.16%. Across all four cases, these maxima
were 1.80% and 0.60%. These are send-only microbenchmark results,
not application timings.
I tested confined/unconfined peer combinations, stacked labels, datagram
and stream sockets, old and new policy ABIs, and serial policy replacement
followed by socket recreation. Allow and deny results matched expectations
on both the original and patched kernels. Fine-grained UNIX permission
tests used ABI 5.0 with network_v9.
Separate untimed probes confirmed that the outer checks still ran for
both sender and receiver. In the unconfined socketpair probe, internal
unix_peer_perm() calls were 2,112 -> 0 -> 2,112 across
original -> patched -> original. The two-process probes also confirmed
that only unconfined sides skipped this internal check.
The follow-up kept sockets open across policy and mode changes. Permission
results matched between the original and patched kernels. Each boot also
had two concurrent tests, each lasting two seconds with 16 policy
replacements during the send loop. A fixed sender deny remained effective
while the receiver policy changed.
One limit is worth noting: receiver-side deny updates did not revoke
access through the existing socket, on either kernel. That path uses the
socket's stored label, so matching the original behavior does not prove
immediate revocation. The live-update tests did not cover stacked labels
and were not exhaustive.
Separately, the original controlled source delta still reproduced a
13.61% slowdown. v7.3-rc4 was 20.35% slower than the same old baseline.
The latter includes other source and config changes; I do not attribute
that whole gap to 6456ccbd2ff7.
Results and test sources are at [1].
Would this whole-label early return be worth pursuing, given the small
cost on the both-confined path? This is a prototype for review.
Thanks,
Chengfeng
[1] https://github.com/lcf0399/linux-regression-evidence/tree/23b61ebab6aff0d8316334da7a988811082d204d/apparmor-af-unix-send-old-abi-6456cc/bare-metal/early-unconfined-20260922
[-- Attachment #2: 0002-early-unconfined.patch --]
[-- Type: text/plain, Size: 596 bytes --]
--- a/security/apparmor/af_unix.c
+++ b/security/apparmor/af_unix.c
@@ -634,12 +634,17 @@
struct unix_sock *peeru = unix_sk(peer_sk);
struct unix_sock *u = unix_sk(sk);
int plen;
- struct sockaddr_un *paddr = aa_sunaddr(unix_sk(peer_sk), &plen);
+ struct sockaddr_un *paddr;
AA_BUG(!label);
AA_BUG(!sk);
AA_BUG(!peer_sk);
AA_BUG(!peer_label);
+
+ if (unconfined(label))
+ return 0;
+
+ paddr = aa_sunaddr(unix_sk(peer_sk), &plen);
return unix_peer_perm(subj_cred, label, op, request, sk,
is_unix_fs(sk) ? &u->path : NULL,
@@ -809,4 +814,3 @@
return error;
}
-
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2026-09-22 16:28 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-08-05 17:33 [REGRESSION] apparmor: AF_UNIX datagram send slowdown after 6456ccbd2ff7 Chengfeng Lin
2026-08-05 23:45 ` John Johansen
2026-09-22 16:27 ` Chengfeng Lin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®