[PATCH 0/4] docs: test_doc_build: Test up to Sphinx 9.1.0

[PATCH 0/4] docs: test_doc_build: Test up to Sphinx 9.1.0

[Mauro Carvalho Chehab]

Hi Jon,

This small patch series extend test_doc_build to cover up to
Sphinx version 9.1.0.

I didn't use AI to write code changes on this series. All manual
work. I did use it to help me identifying what versions are
relevant. After such "AI browsing", I double-checked references
and ended not adding version 9.0.0 at the DEFAULT_VERSIONS_TO_TEST,
as I didn't find a major distro using it.

With that, I kept there version 8.1.3, as Debian trixie uses it
(I removed Fedora 42 from comments, as this is already EOL).

Version 8.2.3 is on an Ubuntu LTS version (being the one on
my machine which has Fedora 44).

Version 9.1.0 is the current latest version, so it is used at
the rolling version distributions.

Also pointed by LLM, version 9.0.0 comes with a new dependency.
I could have simply ommitted that, since pip should automatically
add it, but I opted to map it there with its minimal requirement,
as this way it would be easier to track changes on future
Sphinx versions.

On this series:

patch 1 improves python version detection, as there are version
deprecations on newer Sphinx versions. It also makes the logic
more generic, as the current logic was limited up to Python 3.14.

patch 2 adds the new entries at +PYTHON_VER_CHANGES, SPHINX_REQUIREMENTS
and DEFAULT_VERSIONS_TO_TEST.

patch 3 solves pylint issues.

patch 4 addresses a minor issue related to urllib3 2.0.

After running the new script, it now reports with
tools/docs/test_doc_build.py -b on its summary:

        Sphinx 3.4.3 elapsed time: 00:10:23
        Sphinx 5.3.0 elapsed time: 00:09:27
        Sphinx 6.1.1 elapsed time: 00:33:15
        Sphinx 7.2.1 elapsed time: 00:09:22
        Sphinx 7.2.6 elapsed time: 00:09:21
        Sphinx 7.3.0 elapsed time: 00:09:36
        Sphinx 7.4.7 elapsed time: 00:04:44
        Sphinx 8.1.3 elapsed time: 00:04:59
        Sphinx 8.2.3 elapsed time: 00:05:00
        Sphinx 9.1.0 elapsed time: 00:05:01

And the logic properly used different python versions starting
from python3.10 (which is the minimal one I have installed on
my local machine):

	Sphinx 3.4.3 with /usr/bin/python3.9
	...
	Sphinx 7.4.7 with /usr/bin/python3.9
	Sphinx 8.0.0 with /usr/bin/python3.10
	Sphinx 8.1.0 with /usr/bin/python3.11
	...
	Sphinx 9.0.0 with /usr/bin/python3.11
	Sphinx 9.1.0 with /usr/bin/python3.12

Mauro Carvalho Chehab (4):
  docs: test_doc_build: better handle python version
  docs: test_doc_build: update it to support newer Sphinx versions
  docs: test_doc_build: address pylint warnings
  docs: test_doc_build: don't select urllib3 yanked version

 tools/docs/test_doc_build.py | 120 +++++++++++++++++++++++------------
 1 file changed, 81 insertions(+), 39 deletions(-)

-- 
2.56.0

[PATCH 1/4] docs: test_doc_build: better handle python version

[Mauro Carvalho Chehab]

Make the python version logic more generic to allow adding more
python degradation changes on more recent Sphinx tools.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/docs/test_doc_build.py | 70 ++++++++++++++++++++++++++----------
 1 file changed, 52 insertions(+), 18 deletions(-)

diff --git a/tools/docs/test_doc_build.py b/tools/docs/test_doc_build.py
index 47b4606569f9..4082317d581d 100755
--- a/tools/docs/test_doc_build.py
+++ b/tools/docs/test_doc_build.py
@@ -11,29 +11,47 @@ and optionally test the build.
 
 import argparse
 import asyncio
+import glob
 import os.path
-import shutil
+import re
 import sys
 import time
 import subprocess
 
-# Minimal python version supported by the building system.
+MIN_PYTHON_SUBVER = 9  # Currently, Python 3.9 is the minimal supported version
 
-PYTHON = os.path.basename(sys.executable)
+RE_VERSION = re.compile(r"python3\.\d+$")
 
-min_python_bin = None
 
-for i in range(9, 13):
-    p = f"python3.{i}"
-    if shutil.which(p):
-        min_python_bin = p
-        break
+def get_min_python_bin(min_subver):
+    """Get minimal python binary since 3.MIN_PYTHON_SUBVER"""
+    bins = set()
 
-if not min_python_bin:
-    min_python_bin = PYTHON
+    paths = os.environ.get("PATH", "").split(os.pathsep)
+    for d in paths:
+        for p in glob.glob(os.path.join(d, "python3.[0-9]*")):
+            if not RE_VERSION.search(p):
+                continue
 
-# Starting from 8.0, Python 3.9 is not supported anymore.
-PYTHON_VER_CHANGES = {(8, 0, 0): PYTHON}
+            ver = int(p.split(".")[1])
+            if ver < min_subver:
+                continue
+
+            if os.path.isfile(p) and os.access(p, os.X_OK):
+                bins.add(p)
+
+    if not bins:
+        return None
+
+    return sorted(bins, key=lambda x: (3, int(x.split(".")[1])))[0]
+
+# Minimal python versions. Please notice that for 3.4.3, older python
+# versions would work with Sphinx, but kernel doc build is not compatible
+# with them anymore, due to Sphinx extensions and kernel-doc.
+PYTHON_VER_CHANGES = {
+    (3, 4, 3): (3, MIN_PYTHON_SUBVER),
+    (8, 0, 0): (3, 10),
+}
 
 DEFAULT_VERSIONS_TO_TEST = [
     (3, 4, 3),   # Minimal supported version
@@ -173,7 +191,7 @@ class AsyncCommands:
         self.output = None
         self.fp = fp
 
-    def log(self, out, verbose, is_info=True):
+    def log(self, out, verbose=True, is_info=True):
         out = out.removesuffix('\n')
 
         if verbose:
@@ -362,8 +380,19 @@ class SphinxVenv:
             if not args.verbose:
                 args.verbose = True
 
+        cmd = AsyncCommands(fp)
+
         cur_requirements = {}
-        python_bin = min_python_bin
+        python_bin = get_min_python_bin(MIN_PYTHON_SUBVER)
+        if not python_bin:
+            python_bin = os.path.basename(sys.executable)
+            python_ver = tuple(sys.version_info[:2])
+
+            if python_ver < (3, MIN_PYTHON_SUBVER):
+                sys.exit(f"Python {python_ver[0]}.{python_ver[1]} is too old.")
+        else:
+            ver = int(python_bin.split(".")[1])
+            python_ver = tuple([3, ver])
 
         vers = set(SPHINX_REQUIREMENTS.keys()) | set(args.versions)
 
@@ -372,8 +401,14 @@ class SphinxVenv:
                 new_reqs = SPHINX_REQUIREMENTS[cur_ver]
                 cur_requirements.update(new_reqs)
 
-            if cur_ver in PYTHON_VER_CHANGES:          # pylint: disable=R1715
-                python_bin = PYTHON_VER_CHANGES[cur_ver]
+            if cur_ver in PYTHON_VER_CHANGES:
+                new_ver = PYTHON_VER_CHANGES[cur_ver]
+                if python_ver < new_ver:
+                    python_bin = get_min_python_bin(new_ver[1])
+
+                    if not python_bin:
+                        cmd.log(f"Can't check {cur_ver}: {python_bin} not found.")
+                        break
 
             if cur_ver not in args.versions:
                 continue
@@ -390,7 +425,6 @@ class SphinxVenv:
                                        python_bin)
 
         if args.build:
-            cmd = AsyncCommands(fp)
             cmd.log("\nSummary:", verbose=True)
             for ver, elapsed_time in sorted(self.built_time.items()):
                 cmd.log(f"\tSphinx {ver} elapsed time: {elapsed_time}",
-- 
2.56.0

[PATCH 2/4] docs: test_doc_build: update it to support newer Sphinx versions

[Mauro Carvalho Chehab]

Current version is 9.1.0. Update the script to handle it. As there
were python deprecations on newer versions, add support for those.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/docs/test_doc_build.py | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/tools/docs/test_doc_build.py b/tools/docs/test_doc_build.py
index 4082317d581d..8b57d896c666 100755
--- a/tools/docs/test_doc_build.py
+++ b/tools/docs/test_doc_build.py
@@ -51,6 +51,8 @@ def get_min_python_bin(min_subver):
 PYTHON_VER_CHANGES = {
     (3, 4, 3): (3, MIN_PYTHON_SUBVER),
     (8, 0, 0): (3, 10),
+    (8, 1, 0): (3, 11),
+    (9, 1, 0): (3, 12),
 }
 
 DEFAULT_VERSIONS_TO_TEST = [
@@ -61,8 +63,9 @@ DEFAULT_VERSIONS_TO_TEST = [
     (7, 2, 6),   # Ubuntu 24.04 LTS
     (7, 4, 7),   # Ubuntu 24.10
     (7, 3, 0),   # openSUSE Tumbleweed
-    (8, 1, 3),   # Fedora 42
-    (8, 2, 3)    # Latest version - covers rolling distros
+    (8, 1, 3),   # Debian 13
+    (8, 2, 3),   # Ubuntu 26.04 LTS / Fedora 44
+    (9, 1, 0),   # Latest version
 ]
 
 # Sphinx versions to be installed and their incremental requirements
@@ -178,6 +181,10 @@ SPHINX_REQUIREMENTS = {
         "PyYAML": "6.0.1",
         "sphinxcontrib-serializinghtml": "1.1.9",
     },
+    # from https://raw.githubusercontent.com/sphinx-doc/sphinx/v9.0.0/pyproject.toml
+    (9, 0, 0): {
+        "roman-numerals": "1.0.0",
+    },
 }
 
 
-- 
2.56.0

[PATCH 3/4] docs: test_doc_build: address pylint warnings

[Mauro Carvalho Chehab]

Address existing pylint issues.

While here, adjust some line breaks using black suggestions.

No functional changes.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/docs/test_doc_build.py | 43 ++++++++++++++++++------------------
 1 file changed, 22 insertions(+), 21 deletions(-)

diff --git a/tools/docs/test_doc_build.py b/tools/docs/test_doc_build.py
index 8b57d896c666..9db77ba9b6e2 100755
--- a/tools/docs/test_doc_build.py
+++ b/tools/docs/test_doc_build.py
@@ -2,7 +2,7 @@
 # SPDX-License-Identifier: GPL-2.0
 # Copyright(c) 2025: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
 #
-# pylint: disable=R0903,R0912,R0913,R0914,R0917,C0301
+# pylint: disable=R0903,R0912,R0913,R0914,R0915,R0917,C0301
 
 """
 Install minimal supported requirements for different Sphinx versions
@@ -18,6 +18,8 @@ import sys
 import time
 import subprocess
 
+from textwrap import indent
+
 MIN_PYTHON_SUBVER = 9  # Currently, Python 3.9 is the minimal supported version
 
 RE_VERSION = re.compile(r"python3\.\d+$")
@@ -45,6 +47,7 @@ def get_min_python_bin(min_subver):
 
     return sorted(bins, key=lambda x: (3, int(x.split(".")[1])))[0]
 
+
 # Minimal python versions. Please notice that for 3.4.3, older python
 # versions would work with Sphinx, but kernel doc build is not compatible
 # with them anymore, due to Sphinx extensions and kernel-doc.
@@ -56,16 +59,16 @@ PYTHON_VER_CHANGES = {
 }
 
 DEFAULT_VERSIONS_TO_TEST = [
-    (3, 4, 3),   # Minimal supported version
-    (5, 3, 0),   # CentOS Stream 9 / AlmaLinux 9
-    (6, 1, 1),   # Debian 12
-    (7, 2, 1),   # openSUSE Leap 15.6
-    (7, 2, 6),   # Ubuntu 24.04 LTS
-    (7, 4, 7),   # Ubuntu 24.10
-    (7, 3, 0),   # openSUSE Tumbleweed
-    (8, 1, 3),   # Debian 13
-    (8, 2, 3),   # Ubuntu 26.04 LTS / Fedora 44
-    (9, 1, 0),   # Latest version
+    (3, 4, 3),  # Minimal supported version
+    (5, 3, 0),  # CentOS Stream 9 / AlmaLinux 9
+    (6, 1, 1),  # Debian 12
+    (7, 2, 1),  # openSUSE Leap 15.6
+    (7, 2, 6),  # Ubuntu 24.04 LTS
+    (7, 4, 7),  # Ubuntu 24.10
+    (7, 3, 0),  # openSUSE Tumbleweed
+    (8, 1, 3),  # Debian 13
+    (8, 2, 3),  # Ubuntu 26.04 LTS / Fedora 44
+    (9, 1, 0),  # Latest version
 ]
 
 # Sphinx versions to be installed and their incremental requirements
@@ -76,7 +79,6 @@ SPHINX_REQUIREMENTS = {
         "alabaster": "0.7.12",
         "babel": "2.8.0",
         "certifi": "2020.6.20",
-        "docutils": "0.16",
         "idna": "2.10",
         "imagesize": "1.2.0",
         "Jinja2": "2.11.2",
@@ -199,6 +201,7 @@ class AsyncCommands:
         self.fp = fp
 
     def log(self, out, verbose=True, is_info=True):
+        """Ancillary routine to output a message"""
         out = out.removesuffix('\n')
 
         if verbose:
@@ -227,7 +230,6 @@ class AsyncCommands:
 
     async def run(self, cmd, capture_output=False, check=False,
                   env=None, verbose=True):
-
         """
         Execute an arbitrary command, handling errors.
 
@@ -326,8 +328,8 @@ class SphinxVenv:
 
         # Pip install succeeded. Write requirements file
         if args.req_file:
-            with open(req_file, "w", encoding="utf-8") as fp:
-                fp.write(result.stdout)
+            with open(req_file, "w", encoding="utf-8") as fout:
+                fout.write(result.stdout)
 
         if args.build:
             start_time = time.time()
@@ -379,7 +381,7 @@ class SphinxVenv:
         """
 
         if args.log:
-            fp = open(args.log, "w", encoding="utf-8")
+            fp = open(args.log, "w", encoding="utf-8") # pylint: disable=R1732
             if not args.verbose:
                 args.verbose = False
         else:
@@ -440,15 +442,15 @@ class SphinxVenv:
         if fp:
             fp.close()
 
+
 def parse_version(ver_str):
     """Convert a version string into a tuple."""
 
     return tuple(map(int, ver_str.split(".")))
 
 
-DEFAULT_VERS = "    - "
-DEFAULT_VERS += "\n    - ".join(map(lambda v: f"{v[0]}.{v[1]}.{v[2]}",
-                                    DEFAULT_VERSIONS_TO_TEST))
+DEFAULT_VERS = indent("\n".join(map(lambda v: f"{v[0]}.{v[1]}.{v[2]}",
+                                    DEFAULT_VERSIONS_TO_TEST)), "    - ")
 
 SCRIPT = os.path.relpath(__file__)
 
@@ -493,6 +495,7 @@ MAKE_TARGETS = [
     "xmldocs",
 ]
 
+
 async def main():
     """Main program"""
 
@@ -540,8 +543,6 @@ async def main():
     if not args.make_args:
         args.make_args = []
 
-    sphinx_versions = sorted(list(SPHINX_REQUIREMENTS.keys()))
-
     if args.full:
         args.versions += list(SPHINX_REQUIREMENTS.keys())
 
-- 
2.56.0

[PATCH 4/4] docs: test_doc_build: don't select urllib3 yanked version

[Mauro Carvalho Chehab]

Trying to install urllib3 gives a warning about a yanked version:

	WARNING: The candidate selected for download or install is a yanked version: 'urllib3' candidate (version 2.0.0 at https://files.pythonhosted.org/packages/ca/25/fe81738a115a2f1005b19bd69b6253b7b5cd6c9119164f13f02ec627fc41/urllib3-2.0.0-py3-none-any.whl (from https://pypi.org/simple/urllib3/) (requires-python:>=3.7))

To avoid that, use the recommended replacement (2.0.2) which
should be enough to avoid the warning while keep using the same
minor.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 tools/docs/test_doc_build.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/docs/test_doc_build.py b/tools/docs/test_doc_build.py
index 9db77ba9b6e2..ace3c6212132 100755
--- a/tools/docs/test_doc_build.py
+++ b/tools/docs/test_doc_build.py
@@ -112,7 +112,7 @@ SPHINX_REQUIREMENTS = {
         "sphinxcontrib-applehelp": "1.0.4",
         "sphinxcontrib-htmlhelp": "2.0.1",
         "sphinxcontrib-serializinghtml": "1.1.5",
-        "urllib3": "2.0.0",
+        "urllib3": "2.0.2",
     },
 
     # Starting from here, ensure all docutils versions are covered with
-- 
2.56.0