[PATCH v6 0/2] docs: Integrate rustdoc into Rust documentation

[PATCH v6 0/2] docs: Integrate rustdoc into Rust documentation

[Carlos Bilbao]

Include HTML output generated with rustdoc into the Linux kernel
documentation on Rust.

Carlos Bilbao:
 docs: Move rustdoc output, cross-reference it
 docs: Integrate rustdoc generation into htmldocs

---
Changes since V5:
 - Continue executing htmldocs even if rustdoc fails.

Changes since V4:
 - Limit rustdoc note only to html outputs.

Changes since V3:
 - Added Reviewed-Bys from Akira Yokosawa.
 - PATCH 1/2: Avoid error 404 adding tag `rustdoc` for Sphinx.
 - PATCH 1/2: Don't use "here" as link text, describe destination instead.
 - PATCH 2/2: Check CONFIG_RUST in a way that allows us to skip generation.
 - PATCH 2/2: Reoder Sphinx runs so they complete even if rustdoc fails.

Changes since V2:
 - Split v2 into two-patch series.
 - Add "only:: html" directive in Documentation/rust/index.rst reference

Changes since V1:
 - Work on top of v6.1-rc1.
 - Don't use rustdoc.rst, instead add link to Documentation/rust/index.rst.
 - In Documentation/Makefile, replace @make rustdoc for $(Q)$(MAKE) rustdoc.
 - Don't do LLVM=1 for all rustdoc generation within `make htmldocs`.
 - Add spaces on definition of RUSTDOC_OUTPUT, for consistency.

---
 Documentation/Makefile       | 10 ++++++++++
 Documentation/rust/index.rst |  8 ++++++++
 rust/Makefile                | 15 +++++++++------
 3 files changed, 27 insertions(+), 6 deletions(-)

[PATCH v6 1/2] docs: Move rustdoc output, cross-reference it

[Carlos Bilbao]

Generate rustdoc documentation with the rest of subsystem's documentation
in Documentation/output. Add a cross reference to the generated rustdoc in
Documentation/rust/index.rst if Sphinx target rustdoc is set.

Reviewed-by: Akira Yokosawa <akiyks@gmail.com>
Signed-off-by: Carlos Bilbao <carlos.bilbao@amd.com>
---
 Documentation/rust/index.rst |  8 ++++++++
 rust/Makefile                | 15 +++++++++------
 2 files changed, 17 insertions(+), 6 deletions(-)

diff --git a/Documentation/rust/index.rst b/Documentation/rust/index.rst
index 4ae8c66b94fa..e599be2cec9b 100644
--- a/Documentation/rust/index.rst
+++ b/Documentation/rust/index.rst
@@ -6,6 +6,14 @@ Rust
 Documentation related to Rust within the kernel. To start using Rust
 in the kernel, please read the quick-start.rst guide.
 
+.. only:: rustdoc and html
+
+	You can also browse `rustdoc documentation <rustdoc/kernel/index.html>`_.
+
+.. only:: not rustdoc and html
+
+	This documentation does not include rustdoc generated information.
+
 .. toctree::
     :maxdepth: 1
 
diff --git a/rust/Makefile b/rust/Makefile
index ff70c4c916f8..04f4ee3a193e 100644
--- a/rust/Makefile
+++ b/rust/Makefile
@@ -1,5 +1,8 @@
 # SPDX-License-Identifier: GPL-2.0
 
+# Where to place rustdoc generated documentation
+RUSTDOC_OUTPUT = $(objtree)/Documentation/output/rust/rustdoc
+
 always-$(CONFIG_RUST) += target.json
 no-clean-files += target.json
 
@@ -64,7 +67,7 @@ quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
 	OBJTREE=$(abspath $(objtree)) \
 	$(RUSTDOC) $(if $(rustdoc_host),$(rust_common_flags),$(rust_flags)) \
 		$(rustc_target_flags) -L$(objtree)/$(obj) \
-		--output $(objtree)/$(obj)/doc \
+		--output $(RUSTDOC_OUTPUT) \
 		--crate-name $(subst rustdoc-,,$@) \
 		@$(objtree)/include/generated/rustc_cfg $<
 
@@ -81,15 +84,15 @@ quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $<
 # and then retouch the generated files.
 rustdoc: rustdoc-core rustdoc-macros rustdoc-compiler_builtins \
     rustdoc-alloc rustdoc-kernel
-	$(Q)cp $(srctree)/Documentation/images/logo.svg $(objtree)/$(obj)/doc
-	$(Q)cp $(srctree)/Documentation/images/COPYING-logo $(objtree)/$(obj)/doc
-	$(Q)find $(objtree)/$(obj)/doc -name '*.html' -type f -print0 | xargs -0 sed -Ei \
+	$(Q)cp $(srctree)/Documentation/images/logo.svg $(RUSTDOC_OUTPUT)
+	$(Q)cp $(srctree)/Documentation/images/COPYING-logo $(RUSTDOC_OUTPUT)
+	$(Q)find $(RUSTDOC_OUTPUT) -name '*.html' -type f -print0 | xargs -0 sed -Ei \
 		-e 's:rust-logo\.svg:logo.svg:g' \
 		-e 's:rust-logo\.png:logo.svg:g' \
 		-e 's:favicon\.svg:logo.svg:g' \
 		-e 's:<link rel="alternate icon" type="image/png" href="[./]*favicon-(16x16|32x32)\.png">::g'
 	$(Q)echo '.logo-container > img { object-fit: contain; }' \
-		>> $(objtree)/$(obj)/doc/rustdoc.css
+		>> $(RUSTDOC_OUTPUT)/rustdoc.css
 
 rustdoc-macros: private rustdoc_host = yes
 rustdoc-macros: private rustc_target_flags = --crate-type proc-macro \
@@ -150,7 +153,7 @@ quiet_cmd_rustdoc_test = RUSTDOC T $<
 		@$(objtree)/include/generated/rustc_cfg \
 		$(rustc_target_flags) $(rustdoc_test_target_flags) \
 		--sysroot $(objtree)/$(obj)/test/sysroot $(rustdoc_test_quiet) \
-		-L$(objtree)/$(obj)/test --output $(objtree)/$(obj)/doc \
+		-L$(objtree)/$(obj)/test --output $(RUSTDOC_OUTPUT) \
 		--crate-name $(subst rusttest-,,$@) $<
 
 # We cannot use `-Zpanic-abort-tests` because some tests are dynamic,
-- 
2.34.1

[PATCH v6 2/2] docs: Integrate rustdoc generation into htmldocs

[Carlos Bilbao]

Change target `make htmldocs` to combine RST Sphinx and the generation of
Rust documentation, when support is available.

Reviewed-by: Akira Yokosawa <akiyks@gmail.com>
Signed-off-by: Carlos Bilbao <carlos.bilbao@amd.com>
---
 Documentation/Makefile | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/Documentation/Makefile b/Documentation/Makefile
index bb73dcb5ed05..4111d0dd54d0 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -59,6 +59,10 @@ PAPEROPT_letter = -D latex_paper_size=letter
 KERNELDOC       = $(srctree)/scripts/kernel-doc
 KERNELDOC_CONF  = -D kerneldoc_srctree=$(srctree) -D kerneldoc_bin=$(KERNELDOC)
 ALLSPHINXOPTS   =  $(KERNELDOC_CONF) $(PAPEROPT_$(PAPER)) $(SPHINXOPTS)
+ifeq ($(CONFIG_RUST),y)
+# Let Sphinx know we will include rustdoc
+ALLSPHINXOPTS   +=  -t rustdoc
+endif
 # the i18n builder cannot share the environment and doctrees with the others
 I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
 
@@ -94,6 +98,12 @@ quiet_cmd_sphinx = SPHINX  $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
 htmldocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
 	@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))
+# If Rust support is available, add rustdoc generated contents.
+# If there are any, the errors from this make rustdoc will be displayed but
+# won't stop the execution of htmldocs
+ifeq ($(CONFIG_RUST),y)
+	$(Q)$(MAKE) rustdoc || true
+endif
 
 texinfodocs:
 	@$(srctree)/scripts/sphinx-pre-install --version-check
-- 
2.34.1

Re: [PATCH v6 0/2] docs: Integrate rustdoc into Rust documentation

[Jonathan Corbet]

Carlos Bilbao <carlos.bilbao@amd.com> writes:

> Include HTML output generated with rustdoc into the Linux kernel
> documentation on Rust.
>
> Carlos Bilbao:
>  docs: Move rustdoc output, cross-reference it
>  docs: Integrate rustdoc generation into htmldocs
>
So I spent a fair while messing with this, really hoping to get it
merged for 6.3, but I think it's still just too fragile.

- With a too-new version of bindgen it kills the htmldocs build
  entirely.  If we can't build the Rust docs successfully with the
  toolchain that the user has installed then so be it, but it shouldn't
  wreck the docs build entirely.

- It still throws the user into the "make oldconfig" process, which is
  unexpected for a docs build.  This happens even on a system with no
  Rust installed at all.

- I dumped my Fedora toolchain and did the curl|bash ritual to set up
  the proper versions with rustup.  It would be nice to have some better
  documentation on how to do that ... *how* to install bindgen may not
  be obvious to all users.  It also failed until I installed lld
  separately.

On a system with the right tool versions, and with a kernel already
configured to build the Rust support, it worked nicely, though.

I'd really like to see a solution that (1) doesn't kill the docs build
process if the tool versions are wrong, and (2) doesn't force the
creation of a kernel configuration.  What are our chances of getting
there?

Thanks,

jon

Re: [PATCH v6 0/2] docs: Integrate rustdoc into Rust documentation

[Miguel Ojeda]

On Fri, Feb 3, 2023 at 6:55 PM Jonathan Corbet <corbet@lwn.net> wrote:
>
> - I dumped my Fedora toolchain and did the curl|bash ritual to set up
>   the proper versions with rustup.  It would be nice to have some better
>   documentation on how to do that ... *how* to install bindgen may not
>   be obvious to all users.  It also failed until I installed lld
>   separately.

Are you referring to the Quick Start guide? I am asking because the
guide has a one-line command ready for copy-paste to install
`bindgen`, so I guess you mean something else (?). It also notes that
there are standalone installers available (if you don't want `rustup`)
-- I will change the URL to point directly to the page with the
installers, though, since that should help.

The script behind `make rustavailable` is getting a bunch of
improvements (and tests) in an ongoing patch series -- I will check
the `lld` issue for v2 too. If you don't mind it, I will Cc you then,
in case you could give it a try and confirm it helped.

Thanks!

> I'd really like to see a solution that (1) doesn't kill the docs build
> process if the tool versions are wrong, and (2) doesn't force the
> creation of a kernel configuration.  What are our chances of getting
> there?

It may be quite involved, since a few artifacts from the normal build
would be needed [1]. Since some things in the build system will likely
change anyway, it is probably best to come back to this after that.
What do you think, Carlos? Or would you like to give it a try?

[1] https://lore.kernel.org/linux-doc/CANiq72mC+WzOxhZVtEvnsFYzuBPkd51=TYXK01ztcTZ-CAcUiw@mail.gmail.com/

Cheers,
Miguel