Re: [PATCH v4 00/19] Split sphinx call logic from docs Makefile

[Mauro Carvalho Chehab <mchehab+huawei@kernel.org>]

Em Fri, 05 Sep 2025 10:07:51 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> writes:
> 
> > This series does a major cleanup at docs Makefile by moving the
> > actual doc build logic to a helper script (scripts/sphinx-build-wrapper).
> >
> > Such script was written in a way that it can be called either
> > directly or via a makefile. When running via makefile, it will
> > use GNU jobserver to ensure that, when sphinx-build is
> > called, the number of jobs will match at most what it is
> > specified by the "-j" parameter.  
> 
> I will try to make another pass over this stuff later today.  I would
> really appreciate some more eyes on it, though, and perhaps even some
> reports of testing.  This is a significant change, and the presence of
> surprised would not be ... surprising ...

Take your time.

Yeah, tests are very welcomed. The change itself is not significant
in the sense that we just moved a complex logic with lots of magic
from Makefile to Python (*) without changing the implementation, but 
yeah, it requires testing. From my side, I've been testing cleandocs,
htmldocs and pdfdocs for 3-4 weeks now on over ~20 different distros,
all without O=, but didn't make any burn-in test for the other targets.

I've got (and fixed) some corner cases, but I won't doubt that other
corner cases might be there.

(*) It actually solved one issue: with current implementation, at least for me,
    using V=1 doesn't show the sphinx-build command line, as this is hidden
    inside the complex makefile foreach macro:

	loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit;

	quiet_cmd_sphinx = SPHINX  $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
	      cmd_sphinx = \
	       PYTHONPYCACHEPREFIX="$(PYTHONPYCACHEPREFIX)" \
	       BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(src)/$5/$(SPHINX_CONF)) \
	       $(PYTHON3) $(srctree)/scripts/jobserver-exec \
	       $(CONFIG_SHELL) $(srctree)/Documentation/sphinx/parallel-wrapper.sh \
	       $(SPHINXBUILD) \
	       -b $2 \
	       -c $(abspath $(src)) \
	       -d $(abspath $(BUILDDIR)/.doctrees/$3) \
	       -D version=$(KERNELVERSION) -D release=$(KERNELRELEASE) \
	       $(ALLSPHINXOPTS) \
	       $(abspath $(src)/$5) \
	       $(abspath $(BUILDDIR)/$3/$4) && \
	       if [ "x$(DOCS_CSS)" != "x" ]; then \
	               cp $(if $(patsubst /%,,$(DOCS_CSS)),$(abspath $(srctree)/$(DOCS_CSS)),$(DOCS_CSS)) $(BUILDDIR)/$3/_static/; \
	       fi

	htmldocs:
	...
	       @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))

    IMHO, this is problematic, as it makes harder to debug corner
    cases. While doing this changeset, I actually had to add an echo
    line to show the command line, to ensure that sphinx-build was
    called with the same arguments.

    On a side note, the above macro is complex and hard to maintain.

    One of the reasons why the Python script is bigger is that it
    has one statement per line instead of trying to do lots of
    calls inside a single statement like above (there, cmd_sphinx
    has 8 function calls; htmldocs foreach line has 3).

    The core of cmd_sphinx grew from 14 LOC on Makefile to 64 lines
    in Python, not counting:

	- comments
	- CSS logic
	- path handling logic
	- jobserver logic (in Makefile, this is a single "+" character,
  	  at the foreach line)

    I remember I had to touch on the foreach/call logic there a couple
    of times in the past. It is not the easiest thing to do.

Thanks,
Mauro