[PATCH v4 0/2] rust: take advantage of newer rust-analyzer features

[PATCH v4 0/2] rust: take advantage of newer rust-analyzer features

[Jesung Yang via B4 Relay]

As discussed in [1], we need to support multiple versions of
rust-analyzer to take advantage of newer features without breaking
compatibility for users on older toolchains.

In this specific patch series addressing IDE support for inherent methods
of primitive types, the main compatibility issue arises from using
`sysroot_src`, which brings `std` as a dependency for crates in
`drivers/` and `samples/` (please see PATCH [2/2] for more details). This
causes rust-analyzer to incorrectly resolve symbols from `std` in those
crates.

It turns out that rust-analyzer v0.3.1877 (2024-03-11), which
corresponds to our current MSRV of 1.78, has the same `std` resolution
issue regardless of whether `sysroot_src` is used. However, as far as I
know, we're likely to bump the MSRV to 1.85, where the corresponding
rust-analyzer release indeed needs the versioning infrastructure to fix
the `std` issue.

Hence, this series revises the approach taken in [2] by first adding
multi-version support for rust-analyzer. Specifically, it enables
support for the v0.3.2727 (2025-12-22) release and newer, which is
required to resolve inherent method resolution issues for primitive
types found in recent versions of rust-analyzer.

As Eliot mentioned in [3], we might also want to do the `include_dirs`
trick on top of this series to support rust-analyzer releases older than
v0.3.2727, but it should be handled in a dedicated patch series.

[1] https://lore.kernel.org/rust-for-linux/20260101-rust-project-reduce-size-v1-1-4cd66e9e02d9@gmail.com/
[2] https://lore.kernel.org/r/20260101-ra-fix-primitive-v1-1-def809357b4e@gmail.com/
[3] https://lore.kernel.org/rust-for-linux/DFVQBFD54CJO.2D3VQ091URH2B@nvidia.com/

Signed-off-by: Jesung Yang <y.j3ms.n@gmail.com>
---
Changes in v4:
- Use `dataclass` for internal data structures.
- Change `RaVersionInfo` to an enum.
- Move `RaVersionInfo` closer to its point of use.
- Statically check if all `RaVersionInfo` variants are properly handled
  (using mypy).
- Relocate `ctx.manual_sysroot_crates` check in `append_sysroot_crate`.
- Move `crate_attrs=["no_std"]` addition to `scripts:
  generate_rust_analyzer.py: fix IDE support for primitive types`.
- Move `typing.NotRequired` closer to the relevant field.
- Link to v3: https://lore.kernel.org/r/20260308-ra-fix-primitive-v3-0-598017bcefd8@gmail.com

Changes in v3:
- Remove extra `crate_attrs=["no_std"]` for crates that specify
  `#![no_std]` by themselves.
- Fix rust-analyzer version for Rust 1.78.
- Tweak `map_ra_version_baseline` to distinguish between Rust version
  and rust-analyzer version.
- Simplify overall structure.
- Rebase on 6c02871d258 ("scripts: generate_rust_analyzer.py: reduce cfg
  plumbing")
- Link to v2: https://lore.kernel.org/r/20260109-ra-fix-primitive-v2-0-249852a4145a@gmail.com

Changes in v2:
- Implement multiple rust-analyzer version support.
- Rebase on 9ace4753a520 (Linux 6.19-rc4).
- Remove an unnecessary new line between tags.
- Link to v1: https://lore.kernel.org/r/20260101-ra-fix-primitive-v1-1-def809357b4e@gmail.com

---
Jesung Yang (2):
      scripts: generate_rust_analyzer.py: add versioning infrastructure
      scripts: generate_rust_analyzer.py: fix IDE support for primitive types

 scripts/generate_rust_analyzer.py | 220 ++++++++++++++++++++++++++++++++++----
 1 file changed, 198 insertions(+), 22 deletions(-)
---
base-commit: 6c02871d2585bca9d43cea239f908c05ef55a89d
change-id: 20260101-ra-fix-primitive-78154fe8173f

Best regards,
-- 
Jesung Yang <y.j3ms.n@gmail.com>

[PATCH v4 1/2] scripts: generate_rust_analyzer.py: add versioning infrastructure

[Jesung Yang via B4 Relay]

From: Jesung Yang <y.j3ms.n@gmail.com>

Introduce multi-version support for rust-analyzer. The script now
executes `rust-analyzer --version` to query the version string and
generates a `rust-project.json` file compatible with the detected
version.

This is a preparatory patch to address inherent method resolution
failures for primitive types occurring in rust-analyzer v0.3.2693
(2025-11-24) or later when used with our current `rust-project.json`
generation logic. Since the actual fix requires using the `sysroot_src`
field with a feature only available in rust-analyzer v0.3.2727
(2025-12-22) or later, this infrastructure is necessary to maintain
compatibility with older rust-analyzer releases.

Signed-off-by: Jesung Yang <y.j3ms.n@gmail.com>
---
 scripts/generate_rust_analyzer.py | 205 ++++++++++++++++++++++++++++++++++----
 1 file changed, 183 insertions(+), 22 deletions(-)

diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
index b4a55344688d..21832763c5be 100755
--- a/scripts/generate_rust_analyzer.py
+++ b/scripts/generate_rust_analyzer.py
@@ -4,10 +4,14 @@
 """
 
 import argparse
+from dataclasses import dataclass
+from datetime import datetime, date
+import enum
 import json
 import logging
 import os
 import pathlib
+import re
 import subprocess
 import sys
 from typing import Dict, Iterable, List, Literal, Optional, TypedDict
@@ -36,6 +40,7 @@ class Crate(TypedDict):
     is_workspace_member: bool
     deps: List[Dependency]
     cfg: List[str]
+    crate_attrs: List[str]
     edition: str
     env: Dict[str, str]
 
@@ -49,7 +54,19 @@ class CrateWithGenerated(Crate):
     source: Source
 
 
+class RustProject(TypedDict):
+    crates: List[Crate]
+    sysroot: str
+
+
+@dataclass(frozen=True)
+class RaVersionCtx:
+    manual_sysroot_crates: bool
+    use_crate_attrs: bool
+
+
 def generate_crates(
+    ctx: RaVersionCtx,
     srctree: pathlib.Path,
     objtree: pathlib.Path,
     sysroot_src: pathlib.Path,
@@ -75,10 +92,14 @@ def generate_crates(
         deps: List[Dependency],
         *,
         cfg: Optional[List[str]],
+        crate_attrs: Optional[List[str]],
         is_workspace_member: Optional[bool],
         edition: Optional[str],
     ) -> Crate:
         cfg = cfg if cfg is not None else crates_cfgs.get(display_name, [])
+        crate_attrs = (
+            crate_attrs if ctx.use_crate_attrs and crate_attrs is not None else []
+        )
         is_workspace_member = (
             is_workspace_member if is_workspace_member is not None else True
         )
@@ -89,6 +110,7 @@ def generate_crates(
             "is_workspace_member": is_workspace_member,
             "deps": deps,
             "cfg": cfg,
+            "crate_attrs": crate_attrs,
             "edition": edition,
             "env": {
                 "RUST_MODFILE": "This is only for rust-analyzer"
@@ -109,6 +131,7 @@ def generate_crates(
             root_module,
             deps,
             cfg=cfg,
+            crate_attrs=None,
             is_workspace_member=is_workspace_member,
             edition=edition,
         )
@@ -147,6 +170,7 @@ def generate_crates(
         deps: List[Dependency],
         *,
         cfg: Optional[List[str]] = None,
+        crate_attrs: Optional[List[str]] = None,
         is_workspace_member: Optional[bool] = None,
         edition: Optional[str] = None,
     ) -> Dependency:
@@ -156,6 +180,7 @@ def generate_crates(
                 root_module,
                 deps,
                 cfg=cfg,
+                crate_attrs=crate_attrs,
                 is_workspace_member=is_workspace_member,
                 edition=edition,
             )
@@ -166,7 +191,9 @@ def generate_crates(
         deps: List[Dependency],
         *,
         cfg: Optional[List[str]] = None,
-    ) -> Dependency:
+    ) -> Optional[Dependency]:
+        if not ctx.manual_sysroot_crates:
+            return None
         return append_crate(
             display_name,
             sysroot_src / display_name / "src" / "lib.rs",
@@ -200,67 +227,70 @@ def generate_crates(
             edition=core_edition,
         )
 
+    def sysroot_deps(*deps: Optional[Dependency]) -> List[Dependency]:
+        return [dep for dep in deps if dep is not None]
+
     # NB: sysroot crates reexport items from one another so setting up our transitive dependencies
     # here is important for ensuring that rust-analyzer can resolve symbols. The sources of truth
     # for this dependency graph are `(sysroot_src / crate / "Cargo.toml" for crate in crates)`.
     core = append_sysroot_crate("core", [])
-    alloc = append_sysroot_crate("alloc", [core])
-    std = append_sysroot_crate("std", [alloc, core])
-    proc_macro = append_sysroot_crate("proc_macro", [core, std])
+    alloc = append_sysroot_crate("alloc", sysroot_deps(core))
+    std = append_sysroot_crate("std", sysroot_deps(alloc, core))
+    proc_macro = append_sysroot_crate("proc_macro", sysroot_deps(core, std))
 
     compiler_builtins = append_crate(
         "compiler_builtins",
         srctree / "rust" / "compiler_builtins.rs",
-        [core],
+        sysroot_deps(core),
     )
 
     proc_macro2 = append_crate(
         "proc_macro2",
         srctree / "rust" / "proc-macro2" / "lib.rs",
-        [core, alloc, std, proc_macro],
+        sysroot_deps(core, alloc, std, proc_macro),
     )
 
     quote = append_crate(
         "quote",
         srctree / "rust" / "quote" / "lib.rs",
-        [core, alloc, std, proc_macro, proc_macro2],
+        sysroot_deps(core, alloc, std, proc_macro) + [proc_macro2],
         edition="2018",
     )
 
     syn = append_crate(
         "syn",
         srctree / "rust" / "syn" / "lib.rs",
-        [std, proc_macro, proc_macro2, quote],
+        sysroot_deps(std, proc_macro) + [proc_macro2, quote],
     )
 
     macros = append_proc_macro_crate(
         "macros",
         srctree / "rust" / "macros" / "lib.rs",
-        [std, proc_macro, proc_macro2, quote, syn],
+        sysroot_deps(std, proc_macro) + [proc_macro2, quote, syn],
     )
 
     build_error = append_crate(
         "build_error",
         srctree / "rust" / "build_error.rs",
-        [core, compiler_builtins],
+        sysroot_deps(core) + [compiler_builtins],
     )
 
     pin_init_internal = append_proc_macro_crate(
         "pin_init_internal",
         srctree / "rust" / "pin-init" / "internal" / "src" / "lib.rs",
-        [std, proc_macro, proc_macro2, quote, syn],
+        sysroot_deps(std, proc_macro) + [proc_macro2, quote, syn],
     )
 
     pin_init = append_crate(
         "pin_init",
         srctree / "rust" / "pin-init" / "src" / "lib.rs",
-        [core, compiler_builtins, pin_init_internal, macros],
+        sysroot_deps(core) + [compiler_builtins, pin_init_internal, macros],
     )
 
     ffi = append_crate(
         "ffi",
         srctree / "rust" / "ffi.rs",
-        [core, compiler_builtins],
+        sysroot_deps(core) + [compiler_builtins],
     )
 
     def append_crate_with_generated(
@@ -272,6 +302,7 @@ def generate_crates(
             srctree / "rust"/ display_name / "lib.rs",
             deps,
             cfg=generated_cfg,
+            crate_attrs=None,
             is_workspace_member=True,
             edition=None,
         )
@@ -288,10 +319,14 @@ def generate_crates(
         }
         return register_crate(crate_with_generated)
 
-    bindings = append_crate_with_generated("bindings", [core, ffi, pin_init])
-    uapi = append_crate_with_generated("uapi", [core, ffi, pin_init])
+    bindings = append_crate_with_generated(
+        "bindings", sysroot_deps(core) + [ffi, pin_init]
+    )
+    uapi = append_crate_with_generated(
+        "uapi", sysroot_deps(core) + [ffi, pin_init]
+    )
     kernel = append_crate_with_generated(
-        "kernel", [core, macros, build_error, pin_init, ffi, bindings, uapi]
+        "kernel", sysroot_deps(core) + [macros, build_error, pin_init, ffi, bindings, uapi]
     )
 
     scripts = srctree / "scripts"
@@ -303,7 +338,7 @@ def generate_crates(
         append_crate(
             name,
             path,
-            [std],
+            sysroot_deps(std),
         )
 
     def is_root_crate(build_file: pathlib.Path, target: str) -> bool:
@@ -335,12 +370,120 @@ def generate_crates(
             append_crate(
                 name,
                 path,
-                [core, kernel, pin_init],
+                sysroot_deps(core) + [kernel, pin_init],
                 cfg=generated_cfg,
             )
 
     return crates
 
+
+Version = tuple[int, int, int]
+
+
+@enum.unique
+class RaVersionInfo(enum.Enum):
+    """
+    Represents rust-analyzer compatibility baselines. Concrete versions are mapped to the most
+    recent baseline they have reached. Must be in release order.
+    """
+
+    # v0.3.1877, released on 2024-03-11; shipped with the rustup 1.78 toolchain.
+    DEFAULT = (
+        datetime.strptime("2024-03-11", "%Y-%m-%d"),
+        (0, 3, 1877),
+        (1, 78, 0),
+    )
+
+    def __init__(
+        self, release_date: date, ra_version: Version, rust_version: Version
+    ) -> None:
+        self.release_date = release_date
+        self.ra_version = ra_version
+        self.rust_version = rust_version
+
+
+def generate_rust_project(
+    version_info: RaVersionInfo,
+    srctree: pathlib.Path,
+    objtree: pathlib.Path,
+    sysroot: pathlib.Path,
+    sysroot_src: pathlib.Path,
+    external_src: Optional[pathlib.Path],
+    cfgs: List[str],
+    core_edition: str,
+) -> RustProject:
+    from typing import NoReturn
+
+    # TODO: Switch to `typing.assert_never` when Python 3.11 is adopted.
+    def assert_never(arg: NoReturn, /) -> NoReturn:
+        # Adapted from:
+        # https://github.com/python/cpython/blob/1b118353bb0a/Lib/typing.py#L2629-L2651
+        value = repr(arg)
+        raise AssertionError(f"Expected code to be unreachable, but got: {value}")
+
+    if version_info == RaVersionInfo.DEFAULT:
+        ctx = RaVersionCtx(
+            use_crate_attrs=False,
+            manual_sysroot_crates=True,
+        )
+        return {
+            "crates": generate_crates(
+                ctx, srctree, objtree, sysroot_src, external_src, cfgs, core_edition
+            ),
+            "sysroot": str(sysroot),
+        }
+    else:
+        assert_never(version_info)
+
+def query_ra_version() -> Optional[str]:
+    try:
+        # Use the rust-analyzer binary found in $PATH.
+        ra_version_output = (
+            subprocess.check_output(
+                ["rust-analyzer", "--version"],
+                stdin=subprocess.DEVNULL,
+            )
+            .decode("utf-8")
+            .strip()
+        )
+        return ra_version_output
+    except FileNotFoundError:
+        logging.warning("Failed to find rust-analyzer in $PATH")
+        return None
+
+def map_ra_version_baseline(ra_version_output: str) -> RaVersionInfo:
+    baselines = reversed(RaVersionInfo)
+
+    version_match = re.search(r"\d+\.\d+\.\d+", ra_version_output)
+    if version_match:
+        version_string = version_match.group()
+        found_version = tuple(map(int, version_string.split(".")))
+
+        # `rust-analyzer --version` shows different version string depending on how the binary
+        # is built: it may print either the Rust version or the rust-analyzer version itself.
+        # To distinguish between them, we leverage rust-analyzer's versioning convention.
+        #
+        # See:
+        # - https://github.com/rust-lang/rust-analyzer/blob/fad5c3d2d642/xtask/src/dist.rs#L19-L21
+        is_ra_version = version_string.startswith(("0.3", "0.4", "0.5"))
+        if is_ra_version:
+            for info in baselines:
+                if found_version >= info.ra_version:
+                    return info
+        else:
+            for info in baselines:
+                if found_version >= info.rust_version:
+                    return info
+
+    date_match = re.search(r"\d{4}-\d{2}-\d{2}", ra_version_output)
+    if date_match:
+        found_date = datetime.strptime(date_match.group(), "%Y-%m-%d")
+        for info in baselines:
+            if found_date >= info.release_date:
+                return info
+
+    return RaVersionInfo.DEFAULT
+
 def main() -> None:
     parser = argparse.ArgumentParser()
     parser.add_argument('--verbose', '-v', action='store_true')
@@ -369,10 +512,28 @@ def main() -> None:
         level=logging.INFO if args.verbose else logging.WARNING
     )
 
-    rust_project = {
-        "crates": generate_crates(args.srctree, args.objtree, args.sysroot_src, args.exttree, args.cfgs, args.core_edition),
-        "sysroot": str(args.sysroot),
-    }
+    ra_version_output = query_ra_version()
+    if ra_version_output:
+        compatible_ra_version = map_ra_version_baseline(ra_version_output)
+    else:
+        logging.warning(
+            "Falling back to `rust-project.json` for rust-analyzer %s, %s (shipped with Rust %s)",
+            ".".join(map(str, RaVersionInfo.DEFAULT.ra_version)),
+            datetime.strftime(RaVersionInfo.DEFAULT.release_date, "%Y-%m-%d"),
+            ".".join(map(str, RaVersionInfo.DEFAULT.rust_version)),
+        )
+        compatible_ra_version = RaVersionInfo.DEFAULT
+
+    rust_project = generate_rust_project(
+        compatible_ra_version,
+        args.srctree,
+        args.objtree,
+        args.sysroot,
+        args.sysroot_src,
+        args.exttree,
+        args.cfgs,
+        args.core_edition,
+    )
 
     json.dump(rust_project, sys.stdout, sort_keys=True, indent=4)
 

-- 
2.52.0

Re: [PATCH v4 1/2] scripts: generate_rust_analyzer.py: add versioning infrastructure

[Tamir Duberstein]

On Tue, 17 Mar 2026 18:29:54 +0900, Jesung Yang <y.j3ms.n@gmail.com> wrote:
> diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
> index b4a55344688d..21832763c5be 100755
> --- a/scripts/generate_rust_analyzer.py
> +++ b/scripts/generate_rust_analyzer.py
> @@ -49,7 +54,19 @@ class CrateWithGenerated(Crate):
>      source: Source
>  
>  
> +class RustProject(TypedDict):
> +    crates: List[Crate]
> +    sysroot: str
> +

Could we move this type down so it's closer to its site of use? It's ~350 linues down.

> @@ -200,67 +227,70 @@ def generate_crates(
>              edition=core_edition,
>          )
>  
> +    def sysroot_deps(*deps: Optional[Dependency]) -> List[Dependency]:
> +        return [dep for dep in deps if dep is not None]
> +
>      # NB: sysroot crates reexport items from one another so setting up our transitive dependencies

It would be great to avoid having to sprinkle this everywhere, though I admit
it's not immediately obvious to me how to achieve it. Maybe we can live with it
until the MSRV bump. Again, would be great to structure this in a way that
makes that cleanup simple when the time comes.

> @@ -335,12 +370,120 @@ def generate_crates(
> [ ... skip 17 lines ... ]
> +    Represents rust-analyzer compatibility baselines. Concrete versions are mapped to the most
> +    recent baseline they have reached. Must be in release order.
> +    """
> +
> +    # v0.3.1877, released on 2024-03-11; shipped with the rustup 1.78 toolchain.
> +    DEFAULT = (

There should be a note here explaining that this should be bumped when MSRV is
bumped, and how to obtain the new values that should go here. It's really
important that the string MSRV appears as it's a likely grep target for when
that pointer is updated.

> [ ... skip 40 lines ... ]
> +            ),
> +            "sysroot": str(sysroot),
> +        }
> +    else:
> +        assert_never(version_info)
> +

The call to generate_crates should probably be below the block that synthesizes RaVersionInfo.

> [ ... skip 24 lines ... ]
> +        # `rust-analyzer --version` shows different version string depending on how the binary
> +        # is built: it may print either the Rust version or the rust-analyzer version itself.
> +        # To distinguish between them, we leverage rust-analyzer's versioning convention.
> +        #
> +        # See:
> +        # - https://github.com/rust-lang/rust-analyzer/blob/fad5c3d2d642/xtask/src/dist.rs#L19-L21

either "a different version string" or "different version strings"


-- 
Tamir Duberstein <tamird@kernel.org>

Re: [PATCH v4 1/2] scripts: generate_rust_analyzer.py: add versioning infrastructure

[Jesung Yang]

On Thu, Mar 19, 2026 at 3:02 PM Tamir Duberstein <tamird@kernel.org> wrote:
>
> On Tue, 17 Mar 2026 18:29:54 +0900, Jesung Yang <y.j3ms.n@gmail.com> wrote:
> > diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
> > index b4a55344688d..21832763c5be 100755
> > --- a/scripts/generate_rust_analyzer.py
> > +++ b/scripts/generate_rust_analyzer.py
> > @@ -49,7 +54,19 @@ class CrateWithGenerated(Crate):
> >      source: Source
> >
> >
> > +class RustProject(TypedDict):
> > +    crates: List[Crate]
> > +    sysroot: str
> > +
>
> Could we move this type down so it's closer to its site of use? It's ~350 linues down.

Sure, will do.

> > @@ -200,67 +227,70 @@ def generate_crates(
> >              edition=core_edition,
> >          )
> >
> > +    def sysroot_deps(*deps: Optional[Dependency]) -> List[Dependency]:
> > +        return [dep for dep in deps if dep is not None]
> > +
> >      # NB: sysroot crates reexport items from one another so setting up our transitive dependencies
>
> It would be great to avoid having to sprinkle this everywhere, though I admit
> it's not immediately obvious to me how to achieve it. Maybe we can live with it
> until the MSRV bump. Again, would be great to structure this in a way that
> makes that cleanup simple when the time comes.

I considered using function decorators, but on second thought, that
would just be another way of sprinkling functions around.

In my view, the cleanup should still be fairly straightforward even with
the current `sysroot_deps` approach. We could just remove the function
and let mypy (or an LSP) flag the undefined function error/warnings.
Does that sound reasonable to you?

> > @@ -335,12 +370,120 @@ def generate_crates(
> > [ ... skip 17 lines ... ]
> > +    Represents rust-analyzer compatibility baselines. Concrete versions are mapped to the most
> > +    recent baseline they have reached. Must be in release order.
> > +    """
> > +
> > +    # v0.3.1877, released on 2024-03-11; shipped with the rustup 1.78 toolchain.
> > +    DEFAULT = (
>
> There should be a note here explaining that this should be bumped when MSRV is
> bumped, and how to obtain the new values that should go here. It's really
> important that the string MSRV appears as it's a likely grep target for when
> that pointer is updated.

I'll add a comment about the MSRV bump. Unfortunately, there isn't a
reliable way to programmatically retrieve the matching rust-analyzer
release AFAICT. I typically follow these steps:

1. Look for the commit history of `src/tools/rust-analyzer` in the
   upstream Rust repository (either locally or via GitHub [1]).
2. Check the latest merge commit from on the rust-analyzer side, which
   usually starts with "Merge pull request #<PR_ID>".
3. Search for that <PR_ID> on the rust-analyzer release page [2].
4. Grab the version string and release date.

Perhaps the better approach is to perform these manual checks upfront
for versions 1.86 through 1.94, at which point this version
infrastructure can finally be removed.

[1] https://github.com/rust-lang/rust/commits/<TAG>/src/tools/rust-analyzer
[2] https://github.com/rust-lang/rust-analyzer/releases

> > [ ... skip 40 lines ... ]
> > +            ),
> > +            "sysroot": str(sysroot),
> > +        }
> > +    else:
> > +        assert_never(version_info)
> > +
>
> The call to generate_crates should probably be below the block that synthesizes RaVersionInfo.

Sorry, could you elaborate on this? Not sure what you meant by
"synthesizes RaVersionInfo".

> > [ ... skip 24 lines ... ]
> > +        # `rust-analyzer --version` shows different version string depending on how the binary
> > +        # is built: it may print either the Rust version or the rust-analyzer version itself.
> > +        # To distinguish between them, we leverage rust-analyzer's versioning convention.
> > +        #
> > +        # See:
> > +        # - https://github.com/rust-lang/rust-analyzer/blob/fad5c3d2d642/xtask/src/dist.rs#L19-L21
>
> either "a different version string" or "different version strings"

Nice catch, thanks.

Best regards,
Jesung

Re: [PATCH v4 1/2] scripts: generate_rust_analyzer.py: add versioning infrastructure

[Tamir Duberstein]

On 2026-04-25 12:15:03+00:00, Jesung Yang wrote:
> On Thu, Mar 19, 2026 at 3:02 PM Tamir Duberstein <tamird@kernel.org> wrote:
> 
> > On Tue, 17 Mar 2026 18:29:54 +0900, Jesung Yang <y.j3ms.n@gmail.com> wrote:
> >
> > Could we move this type down so it's closer to its site of use? It's ~350 linues down.
> 
> Sure, will do.
> 
> > > @@ -200,67 +227,70 @@ def generate_crates(
> >
> > It would be great to avoid having to sprinkle this everywhere, though I admit
> > it's not immediately obvious to me how to achieve it. Maybe we can live with it
> > until the MSRV bump. Again, would be great to structure this in a way that
> > makes that cleanup simple when the time comes.
> 
> I considered using function decorators, but on second thought, that
> would just be another way of sprinkling functions around.

Can you explain how function decorators would help? Not entirely clear
to me.

> 
> In my view, the cleanup should still be fairly straightforward even with
> the current `sysroot_deps` approach. We could just remove the function
> and let mypy (or an LSP) flag the undefined function error/warnings.
> Does that sound reasonable to you?

Mechanically that would work of course, but it would result in a
flip-flop in the git history. I was hoping to avoid changes at the
caller for this reason.

> 
> > > @@ -335,12 +370,120 @@ def generate_crates(
> >
> > There should be a note here explaining that this should be bumped when MSRV is
> > bumped, and how to obtain the new values that should go here. It's really
> > important that the string MSRV appears as it's a likely grep target for when
> > that pointer is updated.
> 
> I'll add a comment about the MSRV bump. Unfortunately, there isn't a
> reliable way to programmatically retrieve the matching rust-analyzer
> release AFAICT. I typically follow these steps:
> 
> 1. Look for the commit history of `src/tools/rust-analyzer` in the
>    upstream Rust repository (either locally or via GitHub [1]).
> 2. Check the latest merge commit from on the rust-analyzer side, which
>    usually starts with "Merge pull request #<PR_ID>".
> 3. Search for that <PR_ID> on the rust-analyzer release page [2].
> 4. Grab the version string and release date.
> 
> Perhaps the better approach is to perform these manual checks upfront
> for versions 1.86 through 1.94, at which point this version
> infrastructure can finally be removed.
> 
> [1] https://github.com/rust-lang/rust/commits/<TAG>/src/tools/rust-analyzer
> [2] https://github.com/rust-lang/rust-analyzer/releases

I defer to you on whether or not to do this up front, but I'd like us to
document the procedure either way. If possible, prefer to describe it in
terms of the git CLI rather than GitHub.

> 
> > > [ ... skip 40 lines ... ]
> >
> > The call to generate_crates should probably be below the block that synthesizes RaVersionInfo.
> 
> Sorry, could you elaborate on this? Not sure what you meant by
> "synthesizes RaVersionInfo".

Sorry, it's been a while and now I'm not sure. I likely meant the block
in which the calls to `query_ra_version` and `map_ra_version_baseline`
appear.

> 
> > > [ ... skip 24 lines ... ]
> >
> > either "a different version string" or "different version strings"
> 
> Nice catch, thanks.
> 
> Best regards,
> Jesung

Re: [PATCH v4 1/2] scripts: generate_rust_analyzer.py: add versioning infrastructure

[Jesung Yang]

On Sat, Apr 25, 2026 at 3:12 PM Tamir Duberstein <tamird@kernel.org> wrote:
> On 2026-04-25 12:15:03+00:00, Jesung Yang wrote:
> > On Thu, Mar 19, 2026 at 3:02 PM Tamir Duberstein <tamird@kernel.org> wrote:
> > >
> > > It would be great to avoid having to sprinkle this everywhere, though I admit
> > > it's not immediately obvious to me how to achieve it. Maybe we can live with it
> > > until the MSRV bump. Again, would be great to structure this in a way that
> > > makes that cleanup simple when the time comes.
> >
> > I considered using function decorators, but on second thought, that
> > would just be another way of sprinkling functions around.
>
> Can you explain how function decorators would help? Not entirely clear
> to me.

The idea was to augment all `append_*` functions with a decorator that
contains the `sysroot_deps` logic so that we don't have to call
`sysroot_deps` multiple times at the call site. Something like the
following (not tested, just for demonstration):

-------------------------------------------------
def filter_deps(f):
    def wrapper(*args, **kwargs):
        args = list(args)
        args[2] = [dep for dep in args[2] if dep in not None]
        return f(*args, **kwargs)
    return wrapper

@filter_deps
def append_proc_macro_crate(
    display_name: str,
    root_module: pathlib.Path,
    deps: List[Dependency],
    *,
    ...,
) -> Dependency:
    # ...

@filter_deps
def append_crate(
    display_name: str,
    root_module: pathlib.Path,
    deps: List[Dependency],
    *,
    ...,
) -> Dependency:
    # ...
-------------------------------------------------

Since this only touches the function definitions, we have fewer
things to clean up.

Note that this isn't a clean solution: we'd need to keep mypy happy,
`deps` isn't always the third parameter, and there's a discrepancy
between the written signature and the wrapped behavior (the latter
effectively changes the type of `deps` from `List[Dependency]` to
`List[Optional[Dependency]]`).

> > In my view, the cleanup should still be fairly straightforward even with
> > the current `sysroot_deps` approach. We could just remove the function
> > and let mypy (or an LSP) flag the undefined function error/warnings.
> > Does that sound reasonable to you?
>
> Mechanically that would work of course, but it would result in a
> flip-flop in the git history. I was hoping to avoid changes at the
> caller for this reason.

That makes sense. I don't have a clean solution to avoid this right now;
I'll keep mulling it over in the background while we finalize the other
points.

> > > There should be a note here explaining that this should be bumped when MSRV is
> > > bumped, and how to obtain the new values that should go here. It's really
> > > important that the string MSRV appears as it's a likely grep target for when
> > > that pointer is updated.
> >
> > I'll add a comment about the MSRV bump. Unfortunately, there isn't a
> > reliable way to programmatically retrieve the matching rust-analyzer
> > release AFAICT. I typically follow these steps:
> >
> > 1. Look for the commit history of `src/tools/rust-analyzer` in the
> >    upstream Rust repository (either locally or via GitHub [1]).
> > 2. Check the latest merge commit from on the rust-analyzer side, which
> >    usually starts with "Merge pull request #<PR_ID>".
> > 3. Search for that <PR_ID> on the rust-analyzer release page [2].
> > 4. Grab the version string and release date.
> >
> > Perhaps the better approach is to perform these manual checks upfront
> > for versions 1.86 through 1.94, at which point this version
> > infrastructure can finally be removed.
> >
> > [1] https://github.com/rust-lang/rust/commits/<TAG>/src/tools/rust-analyzer
> > [2] https://github.com/rust-lang/rust-analyzer/releases
>
> I defer to you on whether or not to do this up front, but I'd like us to
> document the procedure either way. If possible, prefer to describe it in
> terms of the git CLI rather than GitHub.

As a best effort, we can do something like this:

```bash
git clone https://github.com/rust-lang/rust
git clone https://github.com/rust-lang/rust-analyzer

RUST_VERSION=1.85.0
LOOKAHEAD=100

subject_string=$(
  git -C ./rust log -n $LOOKAHEAD --format='%s' $RUST_VERSION \
    -- src/tools/rust-analyzer | grep 'Merge pull request #'
)
readarray -t subject_array <<< "$subject_string"

git -C ./rust-analyzer log --oneline --fixed-strings
"${subject_array[@]/#/--grep=}"
```

This provides merge commit candidates that can be searched on the
rust-analyzer GitHub release page. Since rust-analyzer version strings
are only available on GitHub, a plain local git CLI cannot retrieve them
directly. The GitHub CLI (`gh`) could likely automate the final step,
but I'm unsure if suggesting `gh` as a requirement is the right choice
here.

Best regards,
Jesung

Re: [PATCH v4 1/2] scripts: generate_rust_analyzer.py: add versioning infrastructure

[Tamir Duberstein]

On Sun, Apr 26, 2026 at 8:42 AM Jesung Yang <y.j3ms.n@gmail.com> wrote:
>
> On Sat, Apr 25, 2026 at 3:12 PM Tamir Duberstein <tamird@kernel.org> wrote:
> > On 2026-04-25 12:15:03+00:00, Jesung Yang wrote:
> > > On Thu, Mar 19, 2026 at 3:02 PM Tamir Duberstein <tamird@kernel.org> wrote:
> > > >
> > > > It would be great to avoid having to sprinkle this everywhere, though I admit
> > > > it's not immediately obvious to me how to achieve it. Maybe we can live with it
> > > > until the MSRV bump. Again, would be great to structure this in a way that
> > > > makes that cleanup simple when the time comes.
> > >
> > > I considered using function decorators, but on second thought, that
> > > would just be another way of sprinkling functions around.
> >
> > Can you explain how function decorators would help? Not entirely clear
> > to me.
>
> The idea was to augment all `append_*` functions with a decorator that
> contains the `sysroot_deps` logic so that we don't have to call
> `sysroot_deps` multiple times at the call site. Something like the
> following (not tested, just for demonstration):
>
> -------------------------------------------------
> def filter_deps(f):
>     def wrapper(*args, **kwargs):
>         args = list(args)
>         args[2] = [dep for dep in args[2] if dep in not None]
>         return f(*args, **kwargs)
>     return wrapper
>
> @filter_deps
> def append_proc_macro_crate(
>     display_name: str,
>     root_module: pathlib.Path,
>     deps: List[Dependency],
>     *,
>     ...,
> ) -> Dependency:
>     # ...
>
> @filter_deps
> def append_crate(
>     display_name: str,
>     root_module: pathlib.Path,
>     deps: List[Dependency],
>     *,
>     ...,
> ) -> Dependency:
>     # ...
> -------------------------------------------------
>
> Since this only touches the function definitions, we have fewer
> things to clean up.
>
> Note that this isn't a clean solution: we'd need to keep mypy happy,
> `deps` isn't always the third parameter, and there's a discrepancy
> between the written signature and the wrapped behavior (the latter
> effectively changes the type of `deps` from `List[Dependency]` to
> `List[Optional[Dependency]]`).

Right, and also the way you've written the decorator won't appease
mypy; you'll need type annotations. If the goal is to change the type
of `deps` to a list of optionals, a decorator is a nice way to do that
without polluting the git history.

I believe the proper way to annotate this kind of decorator is using
ParamSpec (https://docs.python.org/3/library/typing.html#typing.ParamSpec)
which might not be supported in Python 3.9. That might be OK though -
our minimum Python for type checking need not equal our minimum Python
at runtime.

>
> > > In my view, the cleanup should still be fairly straightforward even with
> > > the current `sysroot_deps` approach. We could just remove the function
> > > and let mypy (or an LSP) flag the undefined function error/warnings.
> > > Does that sound reasonable to you?
> >
> > Mechanically that would work of course, but it would result in a
> > flip-flop in the git history. I was hoping to avoid changes at the
> > caller for this reason.
>
> That makes sense. I don't have a clean solution to avoid this right now;
> I'll keep mulling it over in the background while we finalize the other
> points.
>
> > > > There should be a note here explaining that this should be bumped when MSRV is
> > > > bumped, and how to obtain the new values that should go here. It's really
> > > > important that the string MSRV appears as it's a likely grep target for when
> > > > that pointer is updated.
> > >
> > > I'll add a comment about the MSRV bump. Unfortunately, there isn't a
> > > reliable way to programmatically retrieve the matching rust-analyzer
> > > release AFAICT. I typically follow these steps:
> > >
> > > 1. Look for the commit history of `src/tools/rust-analyzer` in the
> > >    upstream Rust repository (either locally or via GitHub [1]).
> > > 2. Check the latest merge commit from on the rust-analyzer side, which
> > >    usually starts with "Merge pull request #<PR_ID>".
> > > 3. Search for that <PR_ID> on the rust-analyzer release page [2].
> > > 4. Grab the version string and release date.
> > >
> > > Perhaps the better approach is to perform these manual checks upfront
> > > for versions 1.86 through 1.94, at which point this version
> > > infrastructure can finally be removed.
> > >
> > > [1] https://github.com/rust-lang/rust/commits/<TAG>/src/tools/rust-analyzer
> > > [2] https://github.com/rust-lang/rust-analyzer/releases
> >
> > I defer to you on whether or not to do this up front, but I'd like us to
> > document the procedure either way. If possible, prefer to describe it in
> > terms of the git CLI rather than GitHub.
>
> As a best effort, we can do something like this:
>
> ```bash
> git clone https://github.com/rust-lang/rust
> git clone https://github.com/rust-lang/rust-analyzer
>
> RUST_VERSION=1.85.0
> LOOKAHEAD=100
>
> subject_string=$(
>   git -C ./rust log -n $LOOKAHEAD --format='%s' $RUST_VERSION \
>     -- src/tools/rust-analyzer | grep 'Merge pull request #'
> )
> readarray -t subject_array <<< "$subject_string"
>
> git -C ./rust-analyzer log --oneline --fixed-strings
> "${subject_array[@]/#/--grep=}"
> ```
>
> This provides merge commit candidates that can be searched on the
> rust-analyzer GitHub release page. Since rust-analyzer version strings
> are only available on GitHub, a plain local git CLI cannot retrieve them
> directly. The GitHub CLI (`gh`) could likely automate the final step,
> but I'm unsure if suggesting `gh` as a requirement is the right choice
> here.

The rust-analyzer releases are all git tags; once you figure out the
right tag, you can navigate to
https://github.com/rust-lang/rust-analyzer/releases/tag/<tag> to get
the version number.

>
> Best regards,
> Jesung
>

Re: [PATCH v4 1/2] scripts: generate_rust_analyzer.py: add versioning infrastructure

[Jesung Yang]

On Sun, Apr 26, 2026 at 1:50 PM Tamir Duberstein <tamird@kernel.org> wrote:
> On Sun, Apr 26, 2026 at 8:42 AM Jesung Yang <y.j3ms.n@gmail.com> wrote:
> >
> > The idea was to augment all `append_*` functions with a decorator that
> > contains the `sysroot_deps` logic so that we don't have to call
> > `sysroot_deps` multiple times at the call site. Something like the
> > following (not tested, just for demonstration):
> >
> > -------------------------------------------------
> > def filter_deps(f):
> >     def wrapper(*args, **kwargs):
> >         args = list(args)
> >         args[2] = [dep for dep in args[2] if dep in not None]
> >         return f(*args, **kwargs)
> >     return wrapper
> >
> > @filter_deps
> > def append_proc_macro_crate(
> >     display_name: str,
> >     root_module: pathlib.Path,
> >     deps: List[Dependency],
> >     *,
> >     ...,
> > ) -> Dependency:
> >     # ...
> >
> > @filter_deps
> > def append_crate(
> >     display_name: str,
> >     root_module: pathlib.Path,
> >     deps: List[Dependency],
> >     *,
> >     ...,
> > ) -> Dependency:
> >     # ...
> > -------------------------------------------------
> >
> > Since this only touches the function definitions, we have fewer
> > things to clean up.
> >
> > Note that this isn't a clean solution: we'd need to keep mypy happy,
> > `deps` isn't always the third parameter, and there's a discrepancy
> > between the written signature and the wrapped behavior (the latter
> > effectively changes the type of `deps` from `List[Dependency]` to
> > `List[Optional[Dependency]]`).
>
> Right, and also the way you've written the decorator won't appease
> mypy; you'll need type annotations. If the goal is to change the type
> of `deps` to a list of optionals, a decorator is a nice way to do that
> without polluting the git history.
>
> I believe the proper way to annotate this kind of decorator is using
> ParamSpec (https://docs.python.org/3/library/typing.html#typing.ParamSpec)
> which might not be supported in Python 3.9. That might be OK though -
> our minimum Python for type checking need not equal our minimum Python
> at runtime.

I've experimented with `ParamSpec`, but it seems we can't use it in 3.9.
If I understand correctly, neither `from typing import ParamSpec` nor
the `def func[**P]` syntax is available in Python 3.9, and both cause
runtime crashes. Could you elaborate on what you have in mind?

Best regards,
Jesung

Re: [PATCH v4 1/2] scripts: generate_rust_analyzer.py: add versioning infrastructure

[Tamir Duberstein]

On Mon, Apr 27, 2026 at 7:14 AM Jesung Yang <y.j3ms.n@gmail.com> wrote:
>
> On Sun, Apr 26, 2026 at 1:50 PM Tamir Duberstein <tamird@kernel.org> wrote:
> > On Sun, Apr 26, 2026 at 8:42 AM Jesung Yang <y.j3ms.n@gmail.com> wrote:
> [...]
>
> I've experimented with `ParamSpec`, but it seems we can't use it in 3.9.
> If I understand correctly, neither `from typing import ParamSpec` nor
> the `def func[**P]` syntax is available in Python 3.9, and both cause
> runtime crashes. Could you elaborate on what you have in mind?

Some options:

```
T = t.TypeVar('T')
if sys.version_info < (3, 10):
    import typing_extensions as te

    P = te.ParamSpec('P')
else:
    P = t.ParamSpec('P')


def wrapper(func: t.Callable[P, T]) -> t.Callable[P, T]:
    def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
        return func(*args, **kwargs)
    return wrapper
```

This requires typing_extensions, so is not ideal, but mypy passes with
both --python 3.9 and --python 3.10.

```
T = t.TypeVar('T')
if sys.version_info < (3, 10):
    def wrapper(func: t.Callable[..., T]) -> t.Callable[..., T]:
        def wrapper(*args: t.Any, **kwargs: t.Any) -> T:
            return func(*args, **kwargs)
        return wrapper
else:
    P = t.ParamSpec('P')

    def wrapper(func: t.Callable[P, T]) -> t.Callable[P, T]:
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
            return func(*args, **kwargs)
        return wrapper
```

This also works and type checks in both versions. Of course, the type
information is discarded on 3.9, but as long as we run mypy on 3.10 we
are OK.

Re: [PATCH v4 1/2] scripts: generate_rust_analyzer.py: add versioning infrastructure

[Jesung Yang]

On Mon, Apr 27, 2026 at 2:23 PM Tamir Duberstein <tamird@kernel.org> wrote:
>
> Some options:
>
> ```
> T = t.TypeVar('T')
> if sys.version_info < (3, 10):
>     import typing_extensions as te
>
>     P = te.ParamSpec('P')
> else:
>     P = t.ParamSpec('P')
>
>
> def wrapper(func: t.Callable[P, T]) -> t.Callable[P, T]:
>     def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
>         return func(*args, **kwargs)
>     return wrapper
> ```
>
> This requires typing_extensions, so is not ideal, but mypy passes with
> both --python 3.9 and --python 3.10.
>
> ```
> T = t.TypeVar('T')
> if sys.version_info < (3, 10):
>     def wrapper(func: t.Callable[..., T]) -> t.Callable[..., T]:
>         def wrapper(*args: t.Any, **kwargs: t.Any) -> T:
>             return func(*args, **kwargs)
>         return wrapper
> else:
>     P = t.ParamSpec('P')
>
>     def wrapper(func: t.Callable[P, T]) -> t.Callable[P, T]:
>         def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
>             return func(*args, **kwargs)
>         return wrapper
> ```
>
> This also works and type checks in both versions. Of course, the type
> information is discarded on 3.9, but as long as we run mypy on 3.10 we
> are OK.

I'd go for the latter, since the former would require Python 3.9 users
to install an extra dependency to run `make LLVM=1 rust-analyzer`.
Thanks for the suggestions!

Best regards,
Jesung

Re: [PATCH v4 1/2] scripts: generate_rust_analyzer.py: add versioning infrastructure

[Jesung Yang]

On Tue, Apr 28, 2026 at 10:35 PM Jesung Yang <y.j3ms.n@gmail.com> wrote:
> On Mon, Apr 27, 2026 at 2:23 PM Tamir Duberstein <tamird@kernel.org> wrote:
[...]
> >
> > ```
> > T = t.TypeVar('T')
> > if sys.version_info < (3, 10):
> >     def wrapper(func: t.Callable[..., T]) -> t.Callable[..., T]:
> >         def wrapper(*args: t.Any, **kwargs: t.Any) -> T:
> >             return func(*args, **kwargs)
> >         return wrapper
> > else:
> >     P = t.ParamSpec('P')
> >
> >     def wrapper(func: t.Callable[P, T]) -> t.Callable[P, T]:
> >         def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
> >             return func(*args, **kwargs)
> >         return wrapper
> > ```
> >
> > This also works and type checks in both versions. Of course, the type
> > information is discarded on 3.9, but as long as we run mypy on 3.10 we
> > are OK.
>
> I'd go for the latter, since the former would require Python 3.9 users
> to install an extra dependency to run `make LLVM=1 rust-analyzer`.

OK, I did some more investigation and it appears that `ParamSpec` is
generally intended for "preserve and forward" behavior; IIUC, we can't
modify an arbitrary parameter type with it. To quote the docs:

  They are used to forward the parameter types of one callable to
  another callable -- a pattern commonly found in higher order functions
  and decorators [1].

`Concatenate` seems to be one possible way to transform parameter types:

  `Concatenate` can be used in conjunction with `Callable` and
  `ParamSpec` to annotate a higher-order callable which adds, removes,
  or transforms parameters of another callable [2].

However, it requires the parameter whose type we want to modify to
occupy a fixed position. In our case, we would need to, for example,
move the `deps` parameter to the first positional argument in each
`append_*` function to ensure its placement is consistent across all of
them.

Given the above, I now think the simpler approach is to 1) modify
the type of `deps` to `List[Optional[Dependency]]`, and 2) directly
embed the filtering logic inside each `append_*` function:

-------------------------------------------------
def filter_deps(deps: List[Optional[Dependency]]) -> List[Dependency]:
    return [dep for dep in deps if dep is not None]

def append_crate(
    ...,
    deps: List[Optional[Dependency]],
    ...,
) -> Dependency:
    return register_crate(
        build_crate(
            display_name,
            root_module,
            deps=filter_deps(deps),
            cfg=cfg,
            crate_attrs=crate_attrs,
            is_workspace_member=is_workspace_member,
            edition=edition,
        )
    )

# ...
-------------------------------------------------

(The commit using the above approach can be found at [3].)

Similar to the decorator approach, this requires less cleanup compared
to the original `sysroot_deps` version, since we're only touching the
function definitions rather than call sites.

Of course we will need to revert the parameter type when the time comes,
but in my view, this is much more straightforward than the decorator
approach, which needs extra setup to work generally.

What do you think?

[1] https://docs.python.org/3/library/typing.html#typing.ParamSpec
[2] https://docs.python.org/3/library/typing.html#typing.Concatenate
[3] https://github.com/J3m3/linux/commit/ada5d5af9348

Best regards,
Jesung

Re: [PATCH v4 1/2] scripts: generate_rust_analyzer.py: add versioning infrastructure

[Tamir Duberstein]

On Wed, Apr 29, 2026 at 10:38 AM Jesung Yang <y.j3ms.n@gmail.com> wrote:
>
> On Tue, Apr 28, 2026 at 10:35 PM Jesung Yang <y.j3ms.n@gmail.com> wrote:
> > On Mon, Apr 27, 2026 at 2:23 PM Tamir Duberstein <tamird@kernel.org> wrote:
> [...]
> > >
> > > ```
> > > T = t.TypeVar('T')
> > > if sys.version_info < (3, 10):
> > >     def wrapper(func: t.Callable[..., T]) -> t.Callable[..., T]:
> > >         def wrapper(*args: t.Any, **kwargs: t.Any) -> T:
> > >             return func(*args, **kwargs)
> > >         return wrapper
> > > else:
> > >     P = t.ParamSpec('P')
> > >
> > >     def wrapper(func: t.Callable[P, T]) -> t.Callable[P, T]:
> > >         def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
> > >             return func(*args, **kwargs)
> > >         return wrapper
> > > ```
> > >
> > > This also works and type checks in both versions. Of course, the type
> > > information is discarded on 3.9, but as long as we run mypy on 3.10 we
> > > are OK.
> >
> > I'd go for the latter, since the former would require Python 3.9 users
> > to install an extra dependency to run `make LLVM=1 rust-analyzer`.
>
> OK, I did some more investigation and it appears that `ParamSpec` is
> generally intended for "preserve and forward" behavior; IIUC, we can't
> modify an arbitrary parameter type with it. To quote the docs:
>
>   They are used to forward the parameter types of one callable to
>   another callable -- a pattern commonly found in higher order functions
>   and decorators [1].
>
> `Concatenate` seems to be one possible way to transform parameter types:
>
>   `Concatenate` can be used in conjunction with `Callable` and
>   `ParamSpec` to annotate a higher-order callable which adds, removes,
>   or transforms parameters of another callable [2].
>
> However, it requires the parameter whose type we want to modify to
> occupy a fixed position. In our case, we would need to, for example,
> move the `deps` parameter to the first positional argument in each
> `append_*` function to ensure its placement is consistent across all of
> them.
>
> Given the above, I now think the simpler approach is to 1) modify
> the type of `deps` to `List[Optional[Dependency]]`, and 2) directly
> embed the filtering logic inside each `append_*` function:
>
> -------------------------------------------------
> def filter_deps(deps: List[Optional[Dependency]]) -> List[Dependency]:
>     return [dep for dep in deps if dep is not None]
>
> def append_crate(
>     ...,
>     deps: List[Optional[Dependency]],
>     ...,
> ) -> Dependency:
>     return register_crate(
>         build_crate(
>             display_name,
>             root_module,
>             deps=filter_deps(deps),
>             cfg=cfg,
>             crate_attrs=crate_attrs,
>             is_workspace_member=is_workspace_member,
>             edition=edition,
>         )
>     )
>
> # ...
> -------------------------------------------------
>
> (The commit using the above approach can be found at [3].)
>
> Similar to the decorator approach, this requires less cleanup compared
> to the original `sysroot_deps` version, since we're only touching the
> function definitions rather than call sites.
>
> Of course we will need to revert the parameter type when the time comes,
> but in my view, this is much more straightforward than the decorator
> approach, which needs extra setup to work generally.
>
> What do you think?
>
> [1] https://docs.python.org/3/library/typing.html#typing.ParamSpec
> [2] https://docs.python.org/3/library/typing.html#typing.Concatenate
> [3] https://github.com/J3m3/linux/commit/ada5d5af9348

Seems ok to me. I would push the logic into build_crate to avoid
needing the `filter_deps` helper.

Another thought: where you have `# TODO: use `typing.NotRequired` when
Python 3.11 is adopted.` we should split the definition of the type
for < 3.11 so that we can get the correct (strict) type checking when
a more recent version of Python is used. We should get in the habit of
running type checking against all Python versions we support.

Cheers.
Tamir

[PATCH v4 2/2] scripts: generate_rust_analyzer.py: fix IDE support for primitive types

[Jesung Yang via B4 Relay]

From: Jesung Yang <y.j3ms.n@gmail.com>

Update `generate_rust_analyzer.py` so that the generated
`rust-project.json` contains the `sysroot_src` field with
`"crate_attrs": ["no_std"]` specified for relevant crates. This ensures
that rust-analyzer provides proper IDE support for inherent methods of
primitive types.

Since commit 50384460c68f ("Rewrite method resolution to follow rustc
more closely") to rust-analyzer, it no longer provides language server
features like code completion and go-to-definition for inherent methods
of primitive types when sysroot crates (e.g., `core`, `std`) are inlined
in `rust-project.json` [1]. As `generate_rust_analyzer.py` currently
inlines these crates, our setup is affected by this change.

Specifying the `sysroot_src` field restores this functionality by
allowing rust-analyzer to locate sysroot crates by itself. However, this
causes `std` to be treated as a dependency for all local crates by
default. To align with our compilation settings, provide the `no_std`
attribute via the `crate_attrs` field, as the `-Zcrate-attr=no_std`
compiler flag is not visible to rust-analyzer. This combined approach
removes manual manipulation of sysroot dependencies while preventing
incorrect symbol resolution against the standard library.

Note that this configuration requires rust-analyzer release 2025-12-22
(v0.3.2727) or later, which introduced support for the `crate_attrs`
field.

Link: https://rust-lang.zulipchat.com/#narrow/channel/x/topic/x/near/561607963 [1]
Link: https://rust-for-linux.zulipchat.com/#narrow/channel/x/topic/x/near/561607753
Signed-off-by: Jesung Yang <y.j3ms.n@gmail.com>
---
 scripts/generate_rust_analyzer.py | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
index 21832763c5be..bf5a0f16995a 100755
--- a/scripts/generate_rust_analyzer.py
+++ b/scripts/generate_rust_analyzer.py
@@ -54,9 +54,11 @@ class CrateWithGenerated(Crate):
     source: Source
 
 
-class RustProject(TypedDict):
+class RustProject(TypedDict, total=False):
     crates: List[Crate]
     sysroot: str
+    # TODO: use `typing.NotRequired` when Python 3.11 is adopted.
+    sysroot_src: str
 
 
 @dataclass(frozen=True)
@@ -372,6 +374,7 @@ def generate_crates(
                 path,
                 sysroot_deps(core) + [kernel, pin_init],
                 cfg=generated_cfg,
+                crate_attrs=["no_std"],
             )
 
     return crates
@@ -432,6 +435,18 @@ def generate_rust_project(
             ),
             "sysroot": str(sysroot),
         }
+    elif version_info == RaVersionInfo.SUPPORTS_CRATE_ATTRS:
+        ctx = RaVersionCtx(
+            use_crate_attrs=True,
+            manual_sysroot_crates=False,
+        )
+        return {
+            "crates": generate_crates(
+                ctx, srctree, objtree, sysroot_src, external_src, cfgs, core_edition
+            ),
+            "sysroot": str(sysroot),
+            "sysroot_src": str(sysroot_src),
+        }
     else:
         assert_never(version_info)
 

-- 
2.52.0

Re: [PATCH v4 2/2] scripts: generate_rust_analyzer.py: fix IDE support for primitive types

[Tamir Duberstein]

On Tue, 17 Mar 2026 18:29:55 +0900, Jesung Yang <y.j3ms.n@gmail.com> wrote:
> diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
> index 21832763c5be..bf5a0f16995a 100755
> --- a/scripts/generate_rust_analyzer.py
> +++ b/scripts/generate_rust_analyzer.py
> @@ -372,6 +374,7 @@ def generate_crates(
>                  path,
>                  sysroot_deps(core) + [kernel, pin_init],
>                  cfg=generated_cfg,
> +                crate_attrs=["no_std"],
>              )

This seems to be independent of the RA version. Is this broken today? Should it
be a separate patch? Maybe that patch need not be in this series.

> @@ -432,6 +435,18 @@ def generate_rust_project(
>              ),
>              "sysroot": str(sysroot),
>          }
> +    elif version_info == RaVersionInfo.SUPPORTS_CRATE_ATTRS:
> +        ctx = RaVersionCtx(

I think you forgot to define RaVersionInfo.SUPPORTS_CRATE_ATTRS?

> [ ... skip 7 lines ... ]
> +            "sysroot": str(sysroot),
> +            "sysroot_src": str(sysroot_src),
> +        }
>      else:
>          assert_never(version_info)
>  

This bifurcates the handling of RA version differences: half of it is encoded
in RaVersionCtx, and the other half is here in the presence or absence of
"sysroot_src". Can we keep it somehow more contained?


-- 
Tamir Duberstein <tamird@kernel.org>

Re: [PATCH v4 2/2] scripts: generate_rust_analyzer.py: fix IDE support for primitive types

[Jesung Yang]

On Thu, Mar 19, 2026 at 3:02 PM Tamir Duberstein <tamird@kernel.org> wrote:
>
> On Tue, 17 Mar 2026 18:29:55 +0900, Jesung Yang <y.j3ms.n@gmail.com> wrote:
> > diff --git a/scripts/generate_rust_analyzer.py b/scripts/generate_rust_analyzer.py
> > index 21832763c5be..bf5a0f16995a 100755
> > --- a/scripts/generate_rust_analyzer.py
> > +++ b/scripts/generate_rust_analyzer.py
> > @@ -372,6 +374,7 @@ def generate_crates(
> >                  path,
> >                  sysroot_deps(core) + [kernel, pin_init],
> >                  cfg=generated_cfg,
> > +                crate_attrs=["no_std"],
> >              )
>
> This seems to be independent of the RA version. Is this broken today? Should it
> be a separate patch? Maybe that patch need not be in this series.

The primary motivation for the versioning infrastructure was to
incorporate the `crate_attrs` feature introduced in the recent
rust-analyzer release (denoted as `RaVersionInfo.SUPPORTS_CRATE_ATTRS`
in our case).
This solves the side effect of enabling `sysroot_src`: `sysroot_src`
implicitly brings in `std` as a dependency, which is undesirable for
driver crates. Setting `crate_attrs=["no_std"]` is a direct fix for
this.

I could split this series into two separate patches, but I prefer to
keep them in the same series. I think a standalone versioning
infrastructure patch would be unjustified in isolation without an actual
consumer of that logic.

> Is this broken today?

If we enable `sysroot_src` without `crate_attrs=["no_std"]`, users with
a fairly new rust-analyzer release would see, for example, auto-complete
suggestions for `std`.

To provide full context, here is the sequence of issues:

  1) Inherent impls for primitive type have been broken in recent
     rust-analyzer releases.
  2) To fix 1), we need to enable `sysroot_src`. There are also other
     reasons reported by Eliot to enable it, as we know.
  3) A new problem emerges: rust-analyzer thinks we have `std` in the
     kernel driver crates.
  4) To fix 3), we need to use `crate_attrs=["no_std"]` alongside
     `sysroot_src`.
  5) However, `crate_attrs` is only available in rust-analyzer>=0.3.2727.
  6) This means rust-analyzer<0.3.2727 would still suffer from 3) if we
     just use `sysroot_src` + `crate_attrs=["no_std"]`.
  7) We need an infrastructure to conditionally include both
     `sysroot_src` and `crate_attrs["no_std"]` based on the user's
     rust-analyzer version.

I hope this clarifies the intent.

> > @@ -432,6 +435,18 @@ def generate_rust_project(
> >              ),
> >              "sysroot": str(sysroot),
> >          }
> > +    elif version_info == RaVersionInfo.SUPPORTS_CRATE_ATTRS:
> > +        ctx = RaVersionCtx(
>
> I think you forgot to define RaVersionInfo.SUPPORTS_CRATE_ATTRS?

Yeah, interestingly, mypy didn't complain about this. I'll fix it in
v5.

> > [ ... skip 7 lines ... ]
> > +            "sysroot": str(sysroot),
> > +            "sysroot_src": str(sysroot_src),
> > +        }
> >      else:
> >          assert_never(version_info)
> >
>
> This bifurcates the handling of RA version differences: half of it is encoded
> in RaVersionCtx, and the other half is here in the presence or absence of
> "sysroot_src". Can we keep it somehow more contained?

Fair point. Actually, the presence or absence of `sysroot_src` should
depend on `manual_sysroot_crates`. To keep things more self-contained,
I think it would be better to embed `RaVersionCtx` directly into
`RaVersionInfo`, since the values in `RaVersionCtx` are solely
determined by the version.

I'm thinking of something like this:

-------------------------------------------------
@enum.unique
class RaVersionInfo(enum.Enum):
    DEFAULT = (
        datetime.strptime("2024-03-11", "%Y-%m-%d"),
        (0, 3, 1877),
        (1, 78, 0),
        RaVersionCtx(
            use_crate_attrs=False,
            manual_sysroot_crates=True,
        ),
    )
    SUPPORTS_CRATE_ATTRS = (
        datetime.strptime("2025-12-22", "%Y-%m-%d"),
        (0, 3, 2727),
        (1, 94, 0),
        RaVersionCtx(
            use_crate_attrs=True,
            manual_sysroot_crates=False,
        ),
    )

    def __init__(
        self,
        release_date: date,
        ra_version: Version,
        rust_version: Version,
        ctx: RaVersionCtx,
    ) -> None:
        self.release_date = release_date
        self.ra_version = ra_version
        self.rust_version = rust_version
        self.ctx = ctx

# ...

def generate_rust_project(version_info: RaVersionInfo, ...) -> RustProject:
    ctx = version_info
    rust_project = {
        "crates": generate_crates(ctx, ...),
        "sysroot": str(sysroot),
    }

    if not ctx.manual_sysroot_crates:
        rust_project["sysroot_src"] = str(sysroot_src)

    return rust_project

-------------------------------------------------

This allows us to remove the if-else branches and the `assert_never`
check altogether.

Best regards,
Jesung

Re: [PATCH v4 2/2] scripts: generate_rust_analyzer.py: fix IDE support for primitive types

[Tamir Duberstein]

On 2026-04-25 12:17:04+00:00, Jesung Yang wrote:
> On Thu, Mar 19, 2026 at 3:02 PM Tamir Duberstein <tamird@kernel.org> wrote:
> 
> > On Tue, 17 Mar 2026 18:29:55 +0900, Jesung Yang <y.j3ms.n@gmail.com> wrote:
> >
> > This seems to be independent of the RA version. Is this broken today? Should it
> > be a separate patch? Maybe that patch need not be in this series.
> 
> The primary motivation for the versioning infrastructure was to
> incorporate the `crate_attrs` feature introduced in the recent
> rust-analyzer release (denoted as `RaVersionInfo.SUPPORTS_CRATE_ATTRS`
> in our case).
> This solves the side effect of enabling `sysroot_src`: `sysroot_src`
> implicitly brings in `std` as a dependency, which is undesirable for
> driver crates. Setting `crate_attrs=["no_std"]` is a direct fix for
> this.

I think I understand. Setting `crate_attrs=["no_std"]` is a no-op in the
absence of `sysroot_src` but required for correct behavior in the
presenve of `sysroot_src`. Do I have that right?

> 
> I could split this series into two separate patches, but I prefer to
> keep them in the same series. I think a standalone versioning
> infrastructure patch would be unjustified in isolation without an actual
> consumer of that logic.
> 
> > Is this broken today?
> 
> If we enable `sysroot_src` without `crate_attrs=["no_std"]`, users with
> a fairly new rust-analyzer release would see, for example, auto-complete
> suggestions for `std`.
> 
> To provide full context, here is the sequence of issues:
> 
>   1) Inherent impls for primitive type have been broken in recent
>      rust-analyzer releases.
>   2) To fix 1), we need to enable `sysroot_src`. There are also other
>      reasons reported by Eliot to enable it, as we know.
>   3) A new problem emerges: rust-analyzer thinks we have `std` in the
>      kernel driver crates.
>   4) To fix 3), we need to use `crate_attrs=["no_std"]` alongside
>      `sysroot_src`.
>   5) However, `crate_attrs` is only available in rust-analyzer>=0.3.2727.
>   6) This means rust-analyzer<0.3.2727 would still suffer from 3) if we
>      just use `sysroot_src` + `crate_attrs=["no_std"]`.
>   7) We need an infrastructure to conditionally include both
>      `sysroot_src` and `crate_attrs["no_std"]` based on the user's
>      rust-analyzer version.
> 
> I hope this clarifies the intent.

Oh, this contradicts my understanding. Forgive me if I misunderstood,
but aren't you populating `crate_attrs` above unconditionally? Unless by
"only availble" you mean it is silently ignored by earlier versions? In
that case this all makes sense.

> 
> > > @@ -432,6 +435,18 @@ def generate_rust_project(
> >
> > I think you forgot to define RaVersionInfo.SUPPORTS_CRATE_ATTRS?
> 
> Yeah, interestingly, mypy didn't complain about this. I'll fix it in
> v5.
> 
> > > [ ... skip 7 lines ... ]
> >
> > This bifurcates the handling of RA version differences: half of it is encoded
> > in RaVersionCtx, and the other half is here in the presence or absence of
> > "sysroot_src". Can we keep it somehow more contained?
> 
> Fair point. Actually, the presence or absence of `sysroot_src` should
> depend on `manual_sysroot_crates`. To keep things more self-contained,
> I think it would be better to embed `RaVersionCtx` directly into
> `RaVersionInfo`, since the values in `RaVersionCtx` are solely
> determined by the version.
> 
> I'm thinking of something like this:
> 
> -------------------------------------------------
> @enum.unique
> class RaVersionInfo(enum.Enum):
>     DEFAULT = (
>         datetime.strptime("2024-03-11", "%Y-%m-%d"),
>         (0, 3, 1877),
>         (1, 78, 0),
>         RaVersionCtx(
>             use_crate_attrs=False,
>             manual_sysroot_crates=True,
>         ),
>     )
>     SUPPORTS_CRATE_ATTRS = (
>         datetime.strptime("2025-12-22", "%Y-%m-%d"),
>         (0, 3, 2727),
>         (1, 94, 0),
>         RaVersionCtx(
>             use_crate_attrs=True,
>             manual_sysroot_crates=False,
>         ),
>     )
> 
>     def __init__(
>         self,
>         release_date: date,
>         ra_version: Version,
>         rust_version: Version,
>         ctx: RaVersionCtx,
>     ) -> None:
>         self.release_date = release_date
>         self.ra_version = ra_version
>         self.rust_version = rust_version
>         self.ctx = ctx
> 
> # ...
> 
> def generate_rust_project(version_info: RaVersionInfo, ...) -> RustProject:
>     ctx = version_info
>     rust_project = {
>         "crates": generate_crates(ctx, ...),
>         "sysroot": str(sysroot),
>     }
> 
>     if not ctx.manual_sysroot_crates:
>         rust_project["sysroot_src"] = str(sysroot_src)
> 
>     return rust_project
> 
> -------------------------------------------------
> 
> This allows us to remove the if-else branches and the `assert_never`
> check altogether.

Looks reasonable to me. I'd rename DEFAULT to MSRV. Its value should be
the new MSRV which is 1.85.0.

> Best regards,
> Jesung

Re: [PATCH v4 2/2] scripts: generate_rust_analyzer.py: fix IDE support for primitive types

[Jesung Yang]

On Sat, Apr 25, 2026 at 3:27 PM Tamir Duberstein <tamird@kernel.org> wrote:
> On 2026-04-25 12:17:04+00:00, Jesung Yang wrote:
> > On Thu, Mar 19, 2026 at 3:02 PM Tamir Duberstein <tamird@kernel.org> wrote:
> > > On Tue, 17 Mar 2026 18:29:55 +0900, Jesung Yang <y.j3ms.n@gmail.com> wrote:
> > >
> > > This seems to be independent of the RA version. Is this broken today? Should it
> > > be a separate patch? Maybe that patch need not be in this series.
> >
> > The primary motivation for the versioning infrastructure was to
> > incorporate the `crate_attrs` feature introduced in the recent
> > rust-analyzer release (denoted as `RaVersionInfo.SUPPORTS_CRATE_ATTRS`
> > in our case).
> > This solves the side effect of enabling `sysroot_src`: `sysroot_src`
> > implicitly brings in `std` as a dependency, which is undesirable for
> > driver crates. Setting `crate_attrs=["no_std"]` is a direct fix for
> > this.
>
> I think I understand. Setting `crate_attrs=["no_std"]` is a no-op in the
> absence of `sysroot_src` but required for correct behavior in the
> presenve of `sysroot_src`. Do I have that right?

That's correct.

(Technically, the standalone `crate_attrs=["no_std"]` isn't a true
no-op, but you've got the idea; in our case it's effectively a no-op.)

> > I could split this series into two separate patches, but I prefer to
> > keep them in the same series. I think a standalone versioning
> > infrastructure patch would be unjustified in isolation without an actual
> > consumer of that logic.
> >
> > > Is this broken today?
> >
> > If we enable `sysroot_src` without `crate_attrs=["no_std"]`, users with
> > a fairly new rust-analyzer release would see, for example, auto-complete
> > suggestions for `std`.
> >
> > To provide full context, here is the sequence of issues:
> >
> >   1) Inherent impls for primitive type have been broken in recent
> >      rust-analyzer releases.
> >   2) To fix 1), we need to enable `sysroot_src`. There are also other
> >      reasons reported by Eliot to enable it, as we know.
> >   3) A new problem emerges: rust-analyzer thinks we have `std` in the
> >      kernel driver crates.
> >   4) To fix 3), we need to use `crate_attrs=["no_std"]` alongside
> >      `sysroot_src`.
> >   5) However, `crate_attrs` is only available in rust-analyzer>=0.3.2727.
> >   6) This means rust-analyzer<0.3.2727 would still suffer from 3) if we
> >      just use `sysroot_src` + `crate_attrs=["no_std"]`.
> >   7) We need an infrastructure to conditionally include both
> >      `sysroot_src` and `crate_attrs["no_std"]` based on the user's
> >      rust-analyzer version.
> >
> > I hope this clarifies the intent.
>
> Oh, this contradicts my understanding. Forgive me if I misunderstood,
> but aren't you populating `crate_attrs` above unconditionally? Unless by
> "only availble" you mean it is silently ignored by earlier versions? In
> that case this all makes sense.

The conditional guard actually lives in the first patch:

+        crate_attrs = (
+            crate_attrs if ctx.use_crate_attrs and crate_attrs is not
None else []
+        )

... and I don't understand why I bundled this into the first patch.

Apologies for the confusion, I'll move the logic related to
`use_crate_attrs` to this patch (i.e., [PATCH v4 2/2] scripts:
generate_rust_analyzer.py: fix IDE support for primitive types).

(As a side note: `crate_attrs` is indeed ignored by earlier versions,
but since this behavior is not publicly documented and is more of an
implementation detail, I'd prefer not to rely on it.)

> > > This bifurcates the handling of RA version differences: half of it is encoded
> > > in RaVersionCtx, and the other half is here in the presence or absence of
> > > "sysroot_src". Can we keep it somehow more contained?
> >
> > Fair point. Actually, the presence or absence of `sysroot_src` should
> > depend on `manual_sysroot_crates`. To keep things more self-contained,
> > I think it would be better to embed `RaVersionCtx` directly into
> > `RaVersionInfo`, since the values in `RaVersionCtx` are solely
> > determined by the version.
> >
> > I'm thinking of something like this:
> >
> > -------------------------------------------------
> > [...]
> > -------------------------------------------------
> >
> > This allows us to remove the if-else branches and the `assert_never`
> > check altogether.
>
> Looks reasonable to me. I'd rename DEFAULT to MSRV. Its value should be
> the new MSRV which is 1.85.0.

Thanks for the reminder!

Best regards,
Jesung

Re: [PATCH v4 2/2] scripts: generate_rust_analyzer.py: fix IDE support for primitive types

[Jesung Yang]

On Sun, Apr 26, 2026 at 12:43 PM Jesung Yang <y.j3ms.n@gmail.com> wrote:
>
> The conditional guard actually lives in the first patch:
>
> +        crate_attrs = (
> +            crate_attrs if ctx.use_crate_attrs and crate_attrs is not
> None else []
> +        )
>
> ... and I don't understand why I bundled this into the first patch.
>
> Apologies for the confusion, I'll move the logic related to
> `use_crate_attrs` to this patch (i.e., [PATCH v4 2/2] scripts:
> generate_rust_analyzer.py: fix IDE support for primitive types).

I think it'd be more natural to locate the conditional logic based on
`RaVersionCtx.manual_sysroot_crates` and `RaVersionCtx.use_crate_attrs`
in the same patch, since both need to be grouped together to properly
fix IDE support for primitive types.

I'll move the introduction of the entire `RaVersionCtx` to the second
patch.

Best regards,
Jesung