[PATCH v3] docs: kerneldoc.py: preserve headings in kernel-doc output

[PATCH v3] docs: kerneldoc.py: preserve headings in kernel-doc output

[Karl Mehltretter]

kerneldoc.py parses extracted ReST into a detached section node while
retaining the surrounding title hierarchy. Docutils 0.21.2 silently drops
affected sections; 0.22.4 reports:

  Documentation/networking/kapi:107:
  ../include/net/net_shaper.h:75:
  ERROR: A level 3 section cannot be used here.

The same issue drops three Xe DRM RAS headings already in mainline.

Use nested_parse_with_titles() to give kernel-doc output an independent
title hierarchy.

Fixes: 2404dad1f67f ("doc: Cope with the deprecation of AutoReporter")
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Assisted-by: LLM
Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
---
Reposted as a standalone documentation patch at Jakub's suggestion:
https://lore.kernel.org/r/20260925120152.09ef180c@kernel.org/

Changes since v2:
- Route the patch to the documentation maintainers without a net-next tag.
- Add Randy's Acked-by and Tested-by tags. No code changes.

 Documentation/sphinx/kerneldoc.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py
index c1cadb4eb0997..9986df33e37ea 100644
--- a/Documentation/sphinx/kerneldoc.py
+++ b/Documentation/sphinx/kerneldoc.py
@@ -38,6 +38,7 @@ from docutils.statemachine import ViewList
 from docutils.parsers.rst import directives, Directive
 import sphinx
 from sphinx.util.docutils import switch_source_input
+from sphinx.util.nodes import nested_parse_with_titles
 from sphinx.util import logging
 from pprint import pformat
 
@@ -252,7 +253,7 @@ class KernelDocDirective(Directive):
 
     def do_parse(self, result, node):
         with switch_source_input(self.state, result):
-            self.state.nested_parse(result, 0, node, match_titles=1)
+            nested_parse_with_titles(self.state, result, node)
 
 def setup_kfiles(app):
     global kfiles
-- 
2.53.0

Re: [PATCH v3] docs: kerneldoc.py: preserve headings in kernel-doc output

[Jonathan Corbet]

Karl Mehltretter <kmehltretter@gmail.com> writes:

> kerneldoc.py parses extracted ReST into a detached section node while
> retaining the surrounding title hierarchy. Docutils 0.21.2 silently drops
> affected sections; 0.22.4 reports:
>
>   Documentation/networking/kapi:107:
>   ../include/net/net_shaper.h:75:
>   ERROR: A level 3 section cannot be used here.
>
> The same issue drops three Xe DRM RAS headings already in mainline.
>
> Use nested_parse_with_titles() to give kernel-doc output an independent
> title hierarchy.
>
> Fixes: 2404dad1f67f ("doc: Cope with the deprecation of AutoReporter")
> Acked-by: Randy Dunlap <rdunlap@infradead.org>
> Tested-by: Randy Dunlap <rdunlap@infradead.org>
> Assisted-by: LLM
> Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
> ---
> Reposted as a standalone documentation patch at Jakub's suggestion:
> https://lore.kernel.org/r/20260925120152.09ef180c@kernel.org/
>
> Changes since v2:
> - Route the patch to the documentation maintainers without a net-next tag.
> - Add Randy's Acked-by and Tested-by tags. No code changes.

Which versions of Sphinx and docutils have you tested this with?  I
needs to work across the full range of supported versions...

Thanks,

jon

Re: [PATCH v3] docs: kerneldoc.py: preserve headings in kernel-doc output

[Mauro Carvalho Chehab]

On Wed, 14 Oct 2026 11:08:39 -0600
Jonathan Corbet <corbet@lwn.net> wrote:

> Karl Mehltretter <kmehltretter@gmail.com> writes:
> 
> > kerneldoc.py parses extracted ReST into a detached section node while
> > retaining the surrounding title hierarchy. Docutils 0.21.2 silently drops
> > affected sections; 0.22.4 reports:
> >
> >   Documentation/networking/kapi:107:
> >   ../include/net/net_shaper.h:75:
> >   ERROR: A level 3 section cannot be used here.

The backward-compatible fix is to ensure that the title markups will
be used on a consistent way, e.g., if:

- level 1 uses "====\n" + title + "\n===="
- level 2 uses "===="
- level 3 uses "----"
- level 4 uses "^^^^"

at the main doc, the kernel-doc tags shall follow the same schema,
e.g. if they are included at level 3, it has to use the level 4
markup.

So, in the specific case of kapi.rst, you can't use "====" nor
"----" inside net_shaper.h, as those are level 2 and level 3.
so, it has to use, for instance, "^^^^".

Yet, it is hard to make this on a consistent way and keep it
maintainable. The best is not use titles inside kernel-doc.

> >
> > The same issue drops three Xe DRM RAS headings already in mainline.
> >
> > Use nested_parse_with_titles() to give kernel-doc output an independent
> > title hierarchy.
> >
> > Fixes: 2404dad1f67f ("doc: Cope with the deprecation of AutoReporter")
> > Acked-by: Randy Dunlap <rdunlap@infradead.org>
> > Tested-by: Randy Dunlap <rdunlap@infradead.org>
> > Assisted-by: LLM
> > Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
> > ---
> > Reposted as a standalone documentation patch at Jakub's suggestion:
> > https://lore.kernel.org/r/20260925120152.09ef180c@kernel.org/
> >
> > Changes since v2:
> > - Route the patch to the documentation maintainers without a net-next tag.
> > - Add Randy's Acked-by and Tested-by tags. No code changes.  
> 
> Which versions of Sphinx and docutils have you tested this with?  I
> needs to work across the full range of supported versions...

The minimal docutils supported is 0.16, and the minimal Sphinx
supported version is 3.4.4.

You can use /tools/docs/test_doc_build.py to double check if your change
is compatible with the minimal supported Sphinx/docutils versions.

Just run the script:

	$ tools/docs/test_doc_build.py -b

and wait (*) for it to download supported versions and test if this breaks 
something. Notice that "-b" picks only the most significant versions:

  DEFAULT_VERSIONS_TO_TEST = [
    (3, 4, 3),   # Minimal supported version
    (5, 3, 0),   # CentOS Stream 9 / AlmaLinux 9
    (6, 1, 1),   # Debian 12
    (7, 2, 1),   # openSUSE Leap 15.6
    (7, 2, 6),   # Ubuntu 24.04 LTS
    (7, 4, 7),   # Ubuntu 24.10
    (7, 3, 0),   # openSUSE Tumbleweed
    (8, 1, 3),   # Fedora 42
    (8, 2, 3)    # Latest version - covers rolling distros
  ]

e.g. the ones found on distros up to Fedora 42 and the latest version
on that time.

to be really sure, use "-f" to test all possible versions.

(*) even with the simplified version checks, ti runs for quite a while: 
some versions take ~20 mins to build docs.

-

Jon,

I'll likely submit a patch soon to add 9.0 and 9.1 to the above list,
after finish running the tests here to see if the scripts need other
updates.

Thanks,
Mauro

Re: [PATCH v3] docs: kerneldoc.py: preserve headings in kernel-doc output

[Karl Mehltretter]

On Wed, Oct 14, 2026 at 11:46:13PM +0100, Mauro Carvalho Chehab wrote:
> e.g. the ones found on distros up to Fedora 42 and the latest version
> on that time.
> 
> to be really sure, use "-f" to test all possible versions.
> 
> (*) even with the simplified version checks, ti runs for quite a while: 
> some versions take ~20 mins to build docs.
> 

Thanks. To be really sure, I am running the full A/B matrix from
tools/docs/test_doc_build.py -f, also newer Sphinx versions and the
Sphinx 9.1.0/Docutils 0.21.2 combination mentioned earlier.

The endpoint tests have passed:

  - Sphinx 3.4.3 / Docutils 0.16
  - Sphinx 9.1.0 / Docutils 0.22.4

Both builds complete successfully. There are no unexpected diagnostic or
output differences. Changes are limited to the intended heading
restoration and warning removal.

The remaining matrix is running. I will report the full results.

Karl

Re: [PATCH v3] docs: kerneldoc.py: preserve headings in kernel-doc output

[Karl Mehltretter]

On Wed, Oct 14, 2026 at 11:46:13PM +0100, Mauro Carvalho Chehab wrote:
> On Wed, 14 Oct 2026 11:08:39 -0600
> Jonathan Corbet <corbet@lwn.net> wrote:
> 
> > Which versions of Sphinx and docutils have you tested this with?  I
> > needs to work across the full range of supported versions...
> 
> The minimal docutils supported is 0.16, and the minimal Sphinx
> supported version is 3.4.4.
> 
> You can use /tools/docs/test_doc_build.py to double check if your change
> is compatible with the minimal supported Sphinx/docutils versions.
> 

I completed the A/B matrix using tools/docs/test_doc_build.py -f. The
full matrix shows the improvement without a regression. 

I also tested Sphinx 9.0.0, 9.0.4, and 9.1.0, plus the previously
discussed Sphinx 9.1.0 / Docutils 0.21.2 pair.

In total, this covered 32 combinations and 64 full htmldocs builds: A
unpatched and B patched.

Successful, with identical A/B output and normalized diagnostics (26):

  Docutils  Sphinx
  --------  --------------------------------------------------------
  0.16      3.4.3, 3.5.0, 4.0.0
  0.17      4.1.0, 4.3.0, 4.4.0
  0.17.1    4.5.0, 5.0.0, 5.1.0
  0.18      5.2.0
  0.18.1    5.3.0, 6.0.0, 6.1.0, 6.1.1, 6.2.0, 7.0.0, 7.1.0
  0.20      7.2.6, 7.3.0
  0.20.1    7.4.0, 7.4.7
  0.21      8.0.0
  0.21.1    8.1.0, 8.1.3
  0.21.2    8.2.0, 9.1.0

Successful, with only the intended A/B output change (2):

  Sphinx  Docutils  Normalized diagnostics  Changed output
  ------  --------  ----------------------  --------------------------
  8.2.3   0.21.2    62 -> 62, identical     networking/kapi.html,
                                             searchindex.js
  9.1.0   0.22.4    74 -> 60, removals only networking/kapi.html,
                                             searchindex.js

Identical pre-existing failure before and after the patch (4):

  Sphinx         Docutils  Status A/B  Normalized diagnostics A/B
  -------------  --------  ----------  --------------------------
  7.2.0, 7.2.1   0.19      2 / 2       191 / 191, identical
  9.0.0, 9.0.4   0.22      2 / 2        53 /  53, identical

No combination gained a diagnostic or changed its build result. The
only generated-output differences were the intended restored headings
and derived anchors. 

Karl

Re: [PATCH v3] docs: kerneldoc.py: preserve headings in kernel-doc output

[Jonathan Corbet]

Karl Mehltretter <kmehltretter@gmail.com> writes:

> On Wed, Oct 14, 2026 at 11:46:13PM +0100, Mauro Carvalho Chehab wrote:
>> On Wed, 14 Oct 2026 11:08:39 -0600
>> Jonathan Corbet <corbet@lwn.net> wrote:
>> 
>> > Which versions of Sphinx and docutils have you tested this with?  I
>> > needs to work across the full range of supported versions...
>> 
>> The minimal docutils supported is 0.16, and the minimal Sphinx
>> supported version is 3.4.4.
>> 
>> You can use /tools/docs/test_doc_build.py to double check if your change
>> is compatible with the minimal supported Sphinx/docutils versions.
>> 
>
> I completed the A/B matrix using tools/docs/test_doc_build.py -f. The
> full matrix shows the improvement without a regression. 
>
> I also tested Sphinx 9.0.0, 9.0.4, and 9.1.0, plus the previously
> discussed Sphinx 9.1.0 / Docutils 0.21.2 pair.
>
> In total, this covered 32 combinations and 64 full htmldocs builds: A
> unpatched and B patched.
>
> Successful, with identical A/B output and normalized diagnostics (26):
>
>   Docutils  Sphinx
>   --------  --------------------------------------------------------
>   0.16      3.4.3, 3.5.0, 4.0.0
>   0.17      4.1.0, 4.3.0, 4.4.0
>   0.17.1    4.5.0, 5.0.0, 5.1.0
>   0.18      5.2.0
>   0.18.1    5.3.0, 6.0.0, 6.1.0, 6.1.1, 6.2.0, 7.0.0, 7.1.0
>   0.20      7.2.6, 7.3.0
>   0.20.1    7.4.0, 7.4.7
>   0.21      8.0.0
>   0.21.1    8.1.0, 8.1.3
>   0.21.2    8.2.0, 9.1.0

That seems exhaustive, thanks :)

I have buttoned things up for the first 7.4 pull, but will consider this
for later in the merge window.

Thanks,

jon