[BUG] bpf, lsm: new first argument in 7.4 inode hooks silently breaks BPF LSMs

[BUG] bpf, lsm: new first argument in 7.4 inode hooks silently breaks BPF LSMs

[roblabla]

Resubmitting as plain text. Sorry for the noise.

Hello,
First of all, I'm sorry if I'm sending this email to the wrong mailing lists, it's my first time reporting a linux issue ^^'.

In commit 16959c469f232f5c76cfb628297cafbf6dbdba79 (lsm: expose mount idmaps to inode hooks), merged in 7.4-rc1, a new argument was added in position 0 of various inode-related `security_` functions. Specifically, it added this argument to inode_create, inode_link, inode_symlink, inode_mkdir, inode_mknod and inode_permission.

This has the immediate effect of breaking eBPF LSM programs relying on these hooks that were compiled with BPF CO-RE and using BPF_CORE_READ. Those programs don't simply stop loading (which would be fine enough) - instead, they will misinterpret the argument from one type to another, which can cause them to behave erratically. This has already bitten me in the past, when inode_setattr got a similar argument change in Linux 6.9. This caused our eBPF LSM to misbehave and deny actions that should have been ordinarily accepted, leading to huge production outages on our customers' machines.

Here's a simple reproducer that shows the problem (full repro with the loader, makefile and run.sh is at https://github.com/roblabla/kernel-7.4-bpf-bug-reproducer):

```
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include <bpf/bpf_core_read.h>

char LICENSE[] SEC("license") = "GPL";

#define EPERM 1
#define NAME_LEN 32

static const char forbidden[] = "forbidden";

SEC("lsm/inode_create")
int BPF_PROG(deny_forbidden, struct inode *dir, struct dentry *dentry)
{
    char name[NAME_LEN] = {};
    const unsigned char *name_ptr;
    unsigned long dir_ino;

    dir_ino = BPF_CORE_READ(dir, i_ino);
    name_ptr = BPF_CORE_READ(dentry, d_name.name);
    bpf_probe_read_kernel_str(name, sizeof(name), name_ptr);

    bpf_printk("inode_create: dir ino=%lu name='%s'", dir_ino, name);

    for (int i = 0; i < sizeof(forbidden); i++)
        if (name[i] != forbidden[i])
            return 0;

    bpf_printk("inode_create: denied '%s'", name);
    return -EPERM;
}
```

Compile it under a kernel before 7.4 release, then run it on mainline. Here's the output:

- run on 7.1-rc7:  touch forbidden: DENIED   (dir ino=59 name='forbidden')
- run on 7.4-rc1:  touch forbidden: created  (dir ino=0 name='')

Before Linux 7.4, this program will properly deny the creation of any file named "forbidden". But in 7.4, the name_ptr BPF_CORE_READ will read garbage, and the check will fail. This is a bit of a contrived example, of course, but should show that there's a real problem in the way we handle eBPF loading in the face of changing signatures, that is prone to causing big compatibility issues.

Now, I'm not sure how the Linux Kernel treats eBPF LSM hooks - if they're considered part of the userspace that shouldn't be broken. My understanding is that kprobes are not part of the Linux Kernel stability guarantees, for rather obvious reasons (they allow watching the internals of the kernel) - but I'm not sure if this extends to the LSM Hooks. I assume not. Still, this kind of breakage is really painful for us, and I hope we can find a solution.

Could we fix this type of breakage by making sure new arguments are only added at the _end_ of the LSM hooks signatures, such that eBPF written for an older version of the kernel do not start misinterpreting arguments? eBPFs don't care about having "too few" arguments in the hook, so this would solve the situation.

Separately, I think the current lack of error handling in case of argument mismatch should really be improved. Could we perhaps have a mechanism such that the kernel (or maybe libbpf?) could check the arguments of the eBPF program, and refuse loading if it mismatches what the kernel expects?

Thanks,
roblabla

Re: [BUG] bpf, lsm: new first argument in 7.4 inode hooks silently breaks BPF LSMs

[Paul Moore]

On Fri, Nov 6, 2026 at 8:39 AM roblabla <linux@roblab.la> wrote:
>
> Hello,
> First of all, I'm sorry if I'm sending this email to the wrong mailing lists, it's my first time reporting a linux issue ^^'.

Hi,

It looks like you sent this email to the right people/lists, well done
on the first hurdle :)

> In commit 16959c469f232f5c76cfb628297cafbf6dbdba79 (lsm: expose mount idmaps to inode hooks), merged in 7.4-rc1, a new argument was added in position 0 of various inode-related `security_` functions. Specifically, it added this argument to inode_create, inode_link, inode_symlink, inode_mkdir, inode_mknod and inode_permission.
>
> This has the immediate effect of breaking eBPF LSM programs relying on these hooks that were compiled with BPF CO-RE and using BPF_CORE_READ.

...

> Now, I'm not sure how the Linux Kernel treats eBPF LSM hooks - if they're considered part of the userspace that shouldn't be broken. My understanding is that kprobes are not part of the Linux Kernel stability guarantees, for rather obvious reasons (they allow watching the internals of the kernel) - but I'm not sure if this extends to the LSM Hooks. I assume not. Still, this kind of breakage is really painful for us, and I hope we can find a solution.

Like kprobes, the LSM hooks are intentionally not part of any stable
API promise.  While we try not to needlessly churn the LSM hooks, we
reserve the right to modify parameters, the LSM callbacks associated
with a hook, and even the basic hook semantics from one kernel release
to another.  The reasoning is very similar to that of kprobes: the LSM
hooks are linked to kernel internals and thus need the ability to
change as the kernel changes.

LSMs that need to provide a stable API must provide an abstraction
layer between the LSM hook callbacks and their stable API.  The LSM
framework is not responsible for the stability of individual LSM APIs.

> Could we fix this type of breakage by making sure new arguments are only added at the _end_ of the LSM hooks signatures,

I'm sorry, but I can't make that promise at the LSM framework layer.
It is impossible to predict how the kernel internals will continue to
evolve and thus I can't promise how the LSM hooks and callbacks will
need to change as a result.

Regarding the specifics of how the BPF LSM has dealt with this in the
past, and any plans for the future, I'll let others speak to that;
hopefully they will have some tricks you can use to make these
transitions easier.

--
paul-moore.com

Re: [BUG] bpf, lsm: new first argument in 7.4 inode hooks silently breaks BPF LSMs

[Andrii Nakryiko]

On Fri, Nov 6, 2026 at 7:41 AM Paul Moore <paul@paul-moore.com> wrote:
>
> On Fri, Nov 6, 2026 at 8:39 AM roblabla <linux@roblab.la> wrote:
> >
> > Hello,
> > First of all, I'm sorry if I'm sending this email to the wrong mailing lists, it's my first time reporting a linux issue ^^'.
>
> Hi,
>
> It looks like you sent this email to the right people/lists, well done
> on the first hurdle :)
>
> > In commit 16959c469f232f5c76cfb628297cafbf6dbdba79 (lsm: expose mount idmaps to inode hooks), merged in 7.4-rc1, a new argument was added in position 0 of various inode-related `security_` functions. Specifically, it added this argument to inode_create, inode_link, inode_symlink, inode_mkdir, inode_mknod and inode_permission.
> >
> > This has the immediate effect of breaking eBPF LSM programs relying on these hooks that were compiled with BPF CO-RE and using BPF_CORE_READ.
>
> ...
>
> > Now, I'm not sure how the Linux Kernel treats eBPF LSM hooks - if they're considered part of the userspace that shouldn't be broken. My understanding is that kprobes are not part of the Linux Kernel stability guarantees, for rather obvious reasons (they allow watching the internals of the kernel) - but I'm not sure if this extends to the LSM Hooks. I assume not. Still, this kind of breakage is really painful for us, and I hope we can find a solution.
>
> Like kprobes, the LSM hooks are intentionally not part of any stable
> API promise.  While we try not to needlessly churn the LSM hooks, we
> reserve the right to modify parameters, the LSM callbacks associated
> with a hook, and even the basic hook semantics from one kernel release
> to another.  The reasoning is very similar to that of kprobes: the LSM
> hooks are linked to kernel internals and thus need the ability to
> change as the kernel changes.
>
> LSMs that need to provide a stable API must provide an abstraction
> layer between the LSM hook callbacks and their stable API.  The LSM
> framework is not responsible for the stability of individual LSM APIs.
>
> > Could we fix this type of breakage by making sure new arguments are only added at the _end_ of the LSM hooks signatures,
>
> I'm sorry, but I can't make that promise at the LSM framework layer.
> It is impossible to predict how the kernel internals will continue to
> evolve and thus I can't promise how the LSM hooks and callbacks will
> need to change as a result.

Yeah, I think that's a reasonable stance and we went through similar
discussion about tracepoints not being stable APIs, no questions about
that. But I'm wondering if it's not too late (as far as I understand,
this hasn't gotten into final release just yet), would it be ok to
just move newly added argument to be the last ones? We had similar
problem in not too distant past with one of scheduler tracepoints,
where new argument was added in the middle of argument list, and that
causes just unnecessary friction when using BPF with old and new
kernels. We did move the argument to the end and resolved this
(without really mandating any stability guarantees).

So, in short, would it be possible to send a follow up to Daan's
commit and move `struct mnt_idmap *idmap` to the end of LSM argument
list?

>
> Regarding the specifics of how the BPF LSM has dealt with this in the
> past, and any plans for the future, I'll let others speak to that;
> hopefully they will have some tricks you can use to make these
> transitions easier.
>

(this is answer to roblabla, I don't think you, Paul, should care)

The answer here is nuanced. On one hand, libbpf and kernel (BPF
verifier) don't really know exactly what BPF program assumes argument
types are. libbpf-provided BPF_PROG() macro that everyone is using
with BPF LSM just assumes user specifies correct sequence and types of
arguments, and libbpf just provides a bit of syntactic sugar with
casting untyped input arguments (they are just "u64" types) into
expected types. So it's not easy to detect this automatically neither
in libbpf nor in BPF verifier.

Having said that, there is a way to handle this, although ugly and
*everyone* using specific LSMs would need to do this on their own.
Instead of using convenience BPF_PROG() macro to define BPF program,
you just use raw `u64 ctx[]` access, and then cast ctx[0] to either
`struct mnt_idmap *` on new kernel, or `struct dentry *` on old
kernel. And you do that for all arguments. E.g., something like this

SEC("...")
int my_prog(u64 ctx[])
{
    bool new_kernel = ...;
    struct mnt_idmap *idmap = new_kernel ? bpf_core_cast(ctx[0],
struct mnt_idmap) : NULL;
    struct dentry *old_dentry = bpf_core_cast(new_kernel ? ctx[1] : ctx[0]);
    ... and so on ...
}

How you determine "new_kernel" can be done in various ways as well, I
don't want to overburden this email with details, but it's doable as
well.


so, tl;dr, it's possible to handle this LSM signature change, but it's
painful and requires knowing about breakage and doing a bunch of extra
plumbing.

ideally, if it's just a new argument addition, it should go to the end
of arg list and then all these issues go away

> --
> paul-moore.com

Re: [BUG] bpf, lsm: new first argument in 7.4 inode hooks silently breaks BPF LSMs

[roblabla]

Hi,

Thanks for the replies!

> LSMs that need to provide a stable API must provide an abstraction
> layer between the LSM hook callbacks and their stable API.  The LSM
> framework is not responsible for the stability of individual LSM APIs.

Understood, that lines up with my expectations. I think the biggest
thing here would not necessarily be stability guarantees, but rather to
make sure that the eBPFs fail to load when the compatibility is broken.
But that's probably a problem for the eBPF side, and not LSM, I assume.

> So, in short, would it be possible to send a follow up to Daan's
> commit and move `struct mnt_idmap *idmap` to the end of LSM argument
> list?

If this is possible, that'd be nice so that we avoid this breakage in
the short term, since the change isn't in a released kernel yet.

> Having said that, there is a way to handle this, although ugly and
> *everyone* using specific LSMs would need to do this on their own.
> Instead of using convenience BPF_PROG() macro to define BPF program,
> you just use raw `u64 ctx[]` access, and then cast ctx[0] to either
> `struct mnt_idmap *` on new kernel, or `struct dentry *` on old
> kernel.

So, we already do similar tricks in the software we ship (which uses
various eBPF program types, not just LSM) to cover kernels 4.11 to 7.3,
using a mixture of CO-RE tricks similar to what you describe and
per-kernel builds selected at runtime.

Here's the problem though: We ship to customers, and we don't control
when they upgrade their kernel relative to our agent. When the kernel
changes a hook signature, a program built before we could handle that
new version still loads fine and just reads the wrong arguments, which
causes some really gnarly bugs that can cause really bad breakage (such
as accidentally denying events that should have been allowed, often
resulting in machines left broken).

So what would be really nice is a way to make eBPF code robust against
kernel changes such that it either works, or fails to load, but doesn't
do the halfway "loads but silently misbehaves". Here are two ideas that
have been in the back of my mind that could help solve this issue (though
I understand that they may be rather large changes, I'm putting them out
there mostly as a conversation starter):

1. Could libbpf/the kernel have a mismatch check on hook arguments? I
   understand that libbpf and the verifier don't know the types a
   program expects, since BPF_PROG() only casts u64s. But the macro
   does know the types at compile time. Could it (or an opt-in variant)
   record the expected prototype in the program's BTF, so that libbpf
   can compare it against the LSM hook's prototype in vmlinux BTF and
   refuse to load on a mismatch?

2. Since kernel BTF has parameter names for the bpf_lsm_* functions,
   could we maybe have CO-RE-style relocation to resolve "the argument
   named dentry" to the argument index at load time? This way, adding a
   new argument could happen anywhere in the argument list without
   breaking eBPFs, and removing an argument will (correctly) cause our
   eBPF to stop loading. It does mean the eBPF programs must provide the
   same argument names, but maybe that's an acceptable tradeoff?

Thanks,
roblabla