mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Andrii Nakryiko <andrii.nakryiko@gmail.com>
To: Samuel Wu <wusamuel@google.com>
Cc: Jonathan Corbet <corbet@lwn.net>,
	Shuah Khan <skhan@linuxfoundation.org>,
	 Randy Dunlap <rdunlap@infradead.org>,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	 Christian Brauner <brauner@kernel.org>, Jan Kara <jack@suse.cz>,
	Andrii Nakryiko <andrii@kernel.org>,
	 Eduard Zingerman <eddyz87@gmail.com>,
	Ihor Solodrai <ihor.solodrai@linux.dev>,
	 Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	 Kumar Kartikeya Dwivedi <memxor@gmail.com>,
	Martin KaFai Lau <martin.lau@linux.dev>,
	Song Liu <song@kernel.org>,
	 Yonghong Song <yonghong.song@linux.dev>,
	Jiri Olsa <jolsa@kernel.org>,
	 Emil Tsalapatis <emil@etsalapatis.com>,
	Nathan Chancellor <nathan@kernel.org>,
	 Nick Desaulniers <ndesaulniers@google.com>,
	Bill Wendling <morbo@google.com>,
	 Justin Stitt <justinstitt@google.com>,
	kernel-team@android.com,  linux-kernel@vger.kernel.org,
	linux-fsdevel@vger.kernel.org,  linux-doc@vger.kernel.org,
	bpf@vger.kernel.org,  linux-kselftest@vger.kernel.org,
	llvm@lists.linux.dev
Subject: Re: [PATCH bpf-next v2 1/6] kallsyms: Add KALLSYMS_QUERY ioctl to /proc/kallsyms
Date: Thu, 29 Oct 2026 15:09:32 -0700	[thread overview]
Message-ID: <CAEf4Bzbs8F1DPBz=WXVTExrrNjzvaFsi4L+HEj+zbUTpPSTX5g@mail.gmail.com> (raw)
In-Reply-To: <CAG2Kctojb7xiOqTmwWqoAKFaNUECjOzXVc_t6KbG13Pqm2dbng@mail.gmail.com>

On Thu, Oct 29, 2026 at 2:58 PM Samuel Wu <wusamuel@google.com> wrote:
>
> On Tue, Oct 27, 2026 at 3:25 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Oct 22, 2026 at 5:17 PM Samuel Wu <wusamuel@google.com> wrote:
> > >
> > > Add an extensible KALLSYMS_QUERY ioctl() interface to /proc/kallsyms
> > > to support fast point queries without scanning the entire text file.
> > > Exactly one query mode must be set in query_flags.
> > >
> > > When KALLSYMS_QUERY_BY_SYMBOL is set in query_flags, KALLSYMS_QUERY
> > > resolves a NUL-terminated symbol name in vmlinux or loaded modules and
> > > then populates sym_addr:
> > > - Searches vmlinux first via kallsyms_lookup_names() (binary search),
> > >   which takes precedence over loaded modules, and falls back to
> > >   module_kallsyms_on_each_symbol() under module_mutex for module
> > >   symbols
> > > - Matches names exactly, like kallsyms_lookup_name(), disregarding the
> > >   .llvm.<hash> matching, following precedent set by commit fb6a421fb615
> > >   ("kallsyms: Match symbols exactly with CONFIG_LTO_CLANG")
> > > - Resolves 0x0 symbol addresses in both vmlinux and modules
> > > - Rejects ambiguous symbols (multiple entries with the same name at
> > >   different addresses within vmlinux, or across loaded modules) with
> > >   -EADDRNOTAVAIL, while allowing same-address aliases
> > >
> > > Userspace symbolizers resolve kernel addresses to symbol names with
> > > /proc/kallsyms. This is typically done by reading the entire
> > > /proc/kallsyms text file, and keeping a compressed sorted copy of it in
> > > order to resolve addresses at runtime.
> > >
> > > When KALLSYMS_QUERY_BY_ADDR is set in query_flags, KALLSYMS_QUERY
> > > resolves one address to the symbol that contains it:
> > > - sym_addr holds the address to resolve on input.
> > > - sym_name_addr must point to a user buffer of sym_name_size bytes
> > >   that receives the NUL-terminated symbol name. On output,
> > >   sym_name_size holds the number of bytes written (including the
> > >   terminating NUL). -ENAMETOOLONG is returned if the buffer is too
> > >   small; a buffer of KSYM_NAME_LEN (512 at the time of writing) bytes
> > >   is always large enough.
> > > - Symbols are resolved via kallsyms_lookup(), the same lookup as %pS,
> > >   covering vmlinux, loaded modules, BPF JIT images, and freed module
> > >   init functions still tracked by ftrace.
> > >
> > > On an 8 CPU Cuttlefish (x86-64 VM), using this patchseries to load a BPF
> > > program with 11 typeless ksyms has ~220x speedup (157ms -> 0.7ms).
> > >
> > > On an 8 CPU x86-64 VM with a 175k line kallsyms, there are broad
> > > performance and memory savings for the symbolization and callstack
> > > sampling tool, Perfetto:
> > > - Init time: ~350x speedup (155.7ms -> 0.44ms)
> > > - 50k symbolizations: ~12x speedup (25.2ms -> 2.2ms)
> > > - VmRSS: -68% (15.1MB -> 4.8MB)
> > >
> > > Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> > > Signed-off-by: Samuel Wu <wusamuel@google.com>
> > > ---
> > >  include/uapi/linux/fs.h |  41 +++++++++++++
> > >  kernel/kallsyms.c       | 132 ++++++++++++++++++++++++++++++++++++++++
> > >  2 files changed, 173 insertions(+)
> > >
> > > diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h
> > > index 34c6f219462a..b43f227dad0e 100644
> > > --- a/include/uapi/linux/fs.h
> > > +++ b/include/uapi/linux/fs.h
> > > @@ -674,6 +674,47 @@ struct procmap_query {
> > >         __u64 build_id_addr;            /* in */
> > >  };
> > >
> > > +/* /proc/kallsyms ioctl */
> > > +#define KALLSYMS_QUERY _IOWR(PROCFS_IOCTL_MAGIC, 18, struct kallsyms_query)
> > > +
> > > +/* Exactly one query mode must be set in query_flags. */
> > > +enum kallsyms_query_flags {
> > > +       /*
> > > +        * Look up a symbol in vmlinux or loaded modules (vmlinux first; BPF
> > > +        * and ftrace symbols are not searched) by exact NUL-terminated name
> > > +        * at sym_name_addr and return sym_addr. sym_addr and sym_name_size
> > > +        * must be 0 on input.
> > > +        */
> > > +       KALLSYMS_QUERY_BY_SYMBOL = 0x01,
> > > +       /*
> > > +        * Look up the symbol covering sym_addr and write its NUL-terminated
> > > +        * name to the sym_name_addr buffer of sym_name_size bytes. The same
> > > +        * symbols as %pS are searched: vmlinux, loaded modules, BPF JIT
> > > +        * images, and freed module init functions still tracked by ftrace.
> > > +        * A buffer of KSYM_NAME_LEN (512 at the time of writing) bytes is
> > > +        * always enough. On success, sym_name_size is set to the number of
> > > +        * bytes written (including the terminating NUL).
> > > +        */
> > > +       KALLSYMS_QUERY_BY_ADDR = 0x02,
> > > +};
> > > +
> > > +/*
> > > + * Returns -ENOENT if no symbol matches, -EADDRNOTAVAIL if the name maps to
> > > + * more than one address, -ENAMETOOLONG if the name does not fit in
> > > + * KSYM_NAME_LEN (512 at the time of writing) or sym_name_size bytes, and
> > > + * -EPERM if kallsyms addresses are hidden from the opener of /proc/kallsyms
> > > + * (e.g. kptr_restrict).
> > > + */
> > > +struct kallsyms_query {
> > > +       __u64 size;             /* in - struct size for compatibility */
> > > +       __u64 query_flags;      /* in - query mode (enum kallsyms_query_flags) */
> > > +       __u64 sym_name_addr;    /* in - address of symbol name buffer */
> > > +       __u64 sym_addr;         /* out (BY_SYMBOL) / in (BY_ADDR) */
> > > +       __u64 sym_name_size;    /* in/out - name buffer size / bytes written */
> > > +};
> >
> > it might be smart to have QUERY_BY_ADDR and QUERY_BY_NAME to be two
> > separate ioctl commands with its own query in/out struct. This would
> > remove addr vs name ambiguity in sym_name_addr field, and will allow
> > us to evolve APIs independently, if necessary. They should still
> > probably reuse the same type to represent symbol information, and have
> > kallsyms_query struct provide a pointer to that struct (and its
> > maximum size), which the kernel will fill out and set the size it
> > actually knows about.
> >
> > struct kallsyms_query_by_addr {
> >     __u64 size;
> >     __u64 query_flags;
> >     __u64 sym_addr; /* in */
> >     struct kallsyms_sym *sym; /* out */
> >     __u64 sym_len;
> > };
> >
> > struct kallsyms_sym {
> >     __u64 sym_addr;
> >     __u64 sym_name;
> >     __u64 sym_name_len;
> >     __u64 sym_module;
> >     __u64 sym_module_len; /* if zero, no module requested/available */
> > };
> >
> > e.g. for symbol you'd want to return its name, address, but also
> > module name. and we should leave a way to extend it further with more
> > optional information (that's the kallsyms extensibility bit I was
> > talking about).
>
> Agree, two ioctls with a shared symbol type is cleaner. v3 plan is to
> have QUERY_BY_NAME and QUERY_BY_ADDR both fill a shared struct
> kallsyms_sym (address, size, name, module), passed by pointer so it
> can grow later.
>
> > aslo, we should think about adding ways to paginate (or at least leave
> > a possibility of that in the future) if there are multiple matches
> > (e.g., for static functions with the same name resolving to multiple
> > addresses); not sure how it should look like API wise, but it would be
> > good to still return "any" or "first" match, but let user know there
> > are multiple, so user can decide how to proceed...
>
> Makes sense, but I'm thinking v3 doesn't need pagination or batching.
> I didn't see any performance wins when profiling libbpf, and it can be
> added later without breaking UAPI.
>
> > on input I'd also add module name as optional input argument, with
> > "vmlinux" limiting to just vmlinux, set module name to just that name,
> > and NULL/zero meaning "we don't care"
>
> Makes sense, so here is a possible plan for QUERY_BY_NAME:
> - Optional module filter: NULL = any, "vmlinux", or a module name
> - Returns the first match by default (vmlinux first, then modules). An
> ALL_MATCHES flag searches everything and reports the number of matches

searches everything as in all modules, or you mean just full linear scan?

>
> And an open question I'm unsure about: Should libbpf keep the light
> skeleton semantics (take the first match, no ambiguity check), or set
> ALL_MATCHES to keep today's check that fails on names with multiple
> addresses?

I'd say fail on ambiguity

>
> > for QUERY_BY_ADDR we should have a flag saying whether an exact match
> > by address is requested or we should find a symbol that covers the
> > address (that's most useful for symbolization). maybe even "at or
> > after" address mode, so you can paginate (though probably explicitly
> > designed batch API would be even more useful later)
>
> For v3 I think we can keep BY_ADDR to the covering lookup only. The
> result can fill in the covering symbol's (start address, size, name,
> module), and callers can check for an exact match themselves by
> comparing the start address with the queried address. A batch API can
> be added later when there is a user for it.

There is a user for batch API, that's blazesym, wprof, retsnoop and
other applications that use kallsyms for symbolization. I didn't
insist on adding it from the get go just to reduce the scope, but
there is definitely a use case for having fast way to dump all or
parts of kallsyms

>
> > tl;dr, this is the right direction, but I think we should polish API a
> > bit more to be more universally useful and extensible
> > > +
> > > +#define KALLSYMS_QUERY_SIZE_VER0       40 /* sizeof first published struct */
> > > +

[...]

> >
> > PROCMAP_QUERY uses offsetofend() for the last defined field, any
> > reason to have this #define and BUILD_BUG_ON here?
>
> David pointed out that the size field in struct kallsyms_query is
> redundant. In v3, I can drop the size field and use _IOC_SIZE(cmd),
> keep the *_SIZE_VER0 define (since we need minimum size). Perfectly
> fine to switch to the PROCMAP_QUERY style though if preferred.

I'm not sure how size is redundant, I don't see ioctl subsystem
accepting argument size as a separate parameter, so the size has to
come from the user and be embedded in user-provided structure, if we
want any sort of backward/forward compatibility. At least looking at
PROCMAP_QUERY I don't see how we could have avoided it, so not sure
how we can do that here.

>
> Thanks Andrii for all the thoughtful feedback!
>
> > > +               return -EINVAL;
> > > +
> > > +       err = copy_struct_from_user(&karg, sizeof(karg), uarg, usize);
> > > +       if (err)
> > > +               return err;
> > > +
> >
> > [...]

  reply	other threads:[~2026-10-29 22:09 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-23  0:16 [PATCH bpf-next v2 0/6] " Samuel Wu
2026-10-23  0:16 ` [PATCH bpf-next v2 1/6] " Samuel Wu
2026-10-23  0:51   ` bot+bpf-ci
2026-10-27 21:31     ` Samuel Wu
2026-10-25 23:29   ` Randy Dunlap
2026-10-27 21:33     ` Samuel Wu
2026-10-27 22:24   ` Andrii Nakryiko
2026-10-29 21:58     ` Samuel Wu
2026-10-29 22:09       ` Andrii Nakryiko [this message]
2026-10-29 22:35         ` Samuel Wu
2026-10-29 22:54           ` Andrii Nakryiko
2026-10-30  0:23             ` Samuel Wu
2026-10-30 15:35             ` David Laight
2026-10-30 18:42               ` Andrii Nakryiko
2026-11-04  1:05                 ` Samuel Wu
2026-10-28 13:40   ` David Laight
2026-10-29 22:05     ` Samuel Wu
2026-10-29 22:16       ` Andrii Nakryiko
2026-10-30  0:28         ` Samuel Wu
2026-10-30 10:13           ` David Laight
2026-10-30 10:02       ` David Laight
2026-11-04  1:14         ` Samuel Wu
2026-11-04 14:23           ` David Laight
2026-11-05  0:05             ` Samuel Wu
2026-11-05  8:37               ` David Laight
2026-11-05 21:36                 ` Samuel Wu
2026-11-05 22:31               ` Andrii Nakryiko
2026-11-06  2:00                 ` Samuel Wu
2026-10-23  0:16 ` [PATCH bpf-next v2 2/6] tools: Copy KALLSYMS_QUERY definitions to linux/fs.h Samuel Wu
2026-10-23  0:16 ` [PATCH bpf-next v2 3/6] docs/procfs: Document KALLSYMS_QUERY ioctl on /proc/kallsyms Samuel Wu
2026-10-23  0:16 ` [PATCH bpf-next v2 4/6] libbpf: Resolve typeless ksyms via KALLSYMS_QUERY ioctl Samuel Wu
2026-10-23  0:51   ` bot+bpf-ci
2026-10-27 21:45     ` Samuel Wu
2026-10-26  9:28   ` Jiri Olsa
2026-10-27 22:07     ` Samuel Wu
2026-10-28 14:01       ` David Laight
2026-10-29 21:35         ` Samuel Wu
2026-10-29 22:33           ` David Laight
2026-10-23  0:16 ` [PATCH bpf-next v2 5/6] selftests/bpf: Test typeless module ksym resolution Samuel Wu
2026-10-23  0:16 ` [PATCH bpf-next v2 6/6] selftests/proc: Add KALLSYMS_QUERY tests Samuel Wu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAEf4Bzbs8F1DPBz=WXVTExrrNjzvaFsi4L+HEj+zbUTpPSTX5g@mail.gmail.com' \
    --to=andrii.nakryiko@gmail.com \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=brauner@kernel.org \
    --cc=corbet@lwn.net \
    --cc=daniel@iogearbox.net \
    --cc=eddyz87@gmail.com \
    --cc=emil@etsalapatis.com \
    --cc=ihor.solodrai@linux.dev \
    --cc=jack@suse.cz \
    --cc=jolsa@kernel.org \
    --cc=justinstitt@google.com \
    --cc=kernel-team@android.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=llvm@lists.linux.dev \
    --cc=martin.lau@linux.dev \
    --cc=memxor@gmail.com \
    --cc=morbo@google.com \
    --cc=nathan@kernel.org \
    --cc=ndesaulniers@google.com \
    --cc=rdunlap@infradead.org \
    --cc=skhan@linuxfoundation.org \
    --cc=song@kernel.org \
    --cc=viro@zeniv.linux.org.uk \
    --cc=wusamuel@google.com \
    --cc=yonghong.song@linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®