Re: [PATCH bpf-next 5/6] bpf: defer registrations until the vmlinux BTF is available

[Jay Wang <wanjay@amazon.com>]

Addressed since v2; the current version is v3 (patches 5-7/9):
https://lore.kernel.org/bpf/20260925224229.1850-1-wanjay@amazon.com/

Most of these are real, and btf_parse_deferred_modules() is
restructured around them:

- A module BTF is now published (btf_mod->btf set, id allocated) only
  after its queued registrations are applied, so no program can see a
  module BTF without its kfuncs, same as for vmlinux.

- Only modules that have reached MODULE_STATE_LIVE are replayed there,
  with the module pinned, so nothing registers concurrently on the same
  BTF and the module cannot go away underneath.  A module still in its
  init when the vmlinux BTF arrives is only published; the LIVE notifier
  applies what its init queued once init has returned, in the loader's
  own thread, and a failed init frees the queue at GOING.  This covers
  both the concurrent krealloc() on kfunc_set_tab and the
  try_module_get()-on-a-COMING-module lifetime issue.

- On btf_alloc_id() failure the buffer the sysfs file points at is no
  longer freed: it is handed back to the entry, which stays on the list
  as a dead entry until the module goes, and only GOING removes the file
  and then frees the data.

- Modules with a .BTF.base section: their sysfs file is still created at
  load time with the final size, but its reader loads the vmlinux BTF
  and waits until this module's BTF has been relocated and published
  before serving anything, so no unrelocated or half-relocated data is
  ever visible; modules without one are served as before, their data is
  final.  Because that reader may be the thread doing the deferred
  parsing, sysfs files are no longer removed from that path (a failed
  entry stays dead until the module goes) and MODULE_STATE_GOING removes
  the file after dropping btf_module_mutex.  Tested with an out-of-tree
  module (distilled .BTF.base, kfunc registered from init) loaded before
  the vmlinux BTF, with its own sysfs file as the first user.

- sysfs file creation failure is non-fatal in the deferred path too.

- The btf_struct_ops_register() stub is only defined when
  BTF_MODULE_NOTIFIER is; checked with BPF_JIT=n.

- Lock ordering: the vmlinux queue has its own mutex, so nothing takes
  btf_module_mutex under btf_vmlinux_lock any more (the kfunc name
  check returns early for the vmlinux BTF), and CO-RE fetches the
  vmlinux BTF before taking cand_cache_mutex, bpf_core_find_cands()
  only peeks.  That removes both new edges of the cycle.  lockdep is
  clean with CO-RE programs, module BTF and rmmod in the same boot.

- The vmlinux queue on parse failure: with =m the failure is no longer
  cached (see 3/6), so the queue is intentionally kept for the retry;
  bpf_ctx_convert.t is reset so nothing dangles.

Jay