mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
To: Linux Doc Mailing List <linux-doc@vger.kernel.org>,
	Jonathan Corbet <corbet@lwn.net>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>,
	"Mauro Carvalho Chehab" <mchehab+huawei@kernel.org>,
	linux-kernel@vger.kernel.org
Subject: [RFC v2 23/38] docs: sphinx/kernel_abi: parse ABI files only once
Date: Tue, 28 Jan 2025 01:06:12 +0100	[thread overview]
Message-ID: <52b1fc2f56d6a2eee0d8da2ec08e133de8207402.1738020236.git.mchehab+huawei@kernel.org> (raw)
In-Reply-To: <cover.1738020236.git.mchehab+huawei@kernel.org>

Right now, the logic parses ABI files on 4 steps, one for each
directory. While this is fine in principle, by doing that, not
all symbol cross-references will be created.

Change the logic to do the parsing only once in order to get
a global dictionary to be used when creating ABI cross-references.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
 Documentation/admin-guide/abi-obsolete.rst |   2 +-
 Documentation/admin-guide/abi-removed.rst  |   2 +-
 Documentation/admin-guide/abi-stable.rst   |   2 +-
 Documentation/admin-guide/abi-testing.rst  |   2 +-
 Documentation/sphinx/kernel_abi.py         | 115 ++++++++++++---------
 scripts/get_abi.py                         |  22 ++--
 6 files changed, 81 insertions(+), 64 deletions(-)

diff --git a/Documentation/admin-guide/abi-obsolete.rst b/Documentation/admin-guide/abi-obsolete.rst
index 6d4d9ab7b8c3..bdef91d2cea4 100644
--- a/Documentation/admin-guide/abi-obsolete.rst
+++ b/Documentation/admin-guide/abi-obsolete.rst
@@ -9,4 +9,4 @@ marked to be removed at some later point in time.
 The description of the interface will document the reason why it is
 obsolete and when it can be expected to be removed.
 
-.. kernel-abi:: ABI/obsolete
+.. kernel-abi:: obsolete
diff --git a/Documentation/admin-guide/abi-removed.rst b/Documentation/admin-guide/abi-removed.rst
index 9fc78af6f077..bea0608b8442 100644
--- a/Documentation/admin-guide/abi-removed.rst
+++ b/Documentation/admin-guide/abi-removed.rst
@@ -3,4 +3,4 @@
 ABI removed symbols
 ===================
 
-.. kernel-abi:: ABI/removed
+.. kernel-abi:: removed
diff --git a/Documentation/admin-guide/abi-stable.rst b/Documentation/admin-guide/abi-stable.rst
index c47c2a295865..33637c0d4fd5 100644
--- a/Documentation/admin-guide/abi-stable.rst
+++ b/Documentation/admin-guide/abi-stable.rst
@@ -12,4 +12,4 @@ for at least 2 years.
 Most interfaces (like syscalls) are expected to never change and always
 be available.
 
-.. kernel-abi:: ABI/stable
+.. kernel-abi:: stable
diff --git a/Documentation/admin-guide/abi-testing.rst b/Documentation/admin-guide/abi-testing.rst
index 40b31985e587..55054985a8ff 100644
--- a/Documentation/admin-guide/abi-testing.rst
+++ b/Documentation/admin-guide/abi-testing.rst
@@ -18,4 +18,4 @@ Programs that use these interfaces are strongly encouraged to add their
 name to the description of these interfaces, so that the kernel
 developers can easily notify them if any changes occur.
 
-.. kernel-abi:: ABI/testing
+.. kernel-abi:: testing
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index 2408ba7a3981..6d1b34e44d46 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -49,6 +49,13 @@ from get_abi import AbiParser
 
 __version__ = "1.0"
 
+logger = logging.getLogger('kernel_abi')
+path = os.path.join(srctree, "Documentation/ABI")
+
+# Parse ABI symbols only once
+kernel_abi = AbiParser(path, logger=logger)
+kernel_abi.parse_abi()
+kernel_abi.check_issues()
 
 def setup(app):
 
@@ -64,14 +71,15 @@ class KernelCmd(Directive):
     u"""KernelABI (``kernel-abi``) directive"""
 
     required_arguments = 1
-    optional_arguments = 2
+    optional_arguments = 3
     has_content = False
     final_argument_whitespace = True
-    logger = logging.getLogger('kernel_abi')
     parser = None
 
     option_spec = {
         "debug": directives.flag,
+        "no-symbols": directives.flag,
+        "no-files":  directives.flag,
     }
 
     def run(self):
@@ -79,62 +87,67 @@ class KernelCmd(Directive):
         if not doc.settings.file_insertion_enabled:
             raise self.warning("docutils: file insertion disabled")
 
-        path = os.path.join(srctree, "Documentation", self.arguments[0])
-        self.parser = AbiParser(path, logger=self.logger)
-        self.parser.parse_abi()
-        self.parser.check_issues()
-
-        node = self.nested_parse(None, self.arguments[0])
-        return node
-
-    def nested_parse(self, data, fname):
         env = self.state.document.settings.env
         content = ViewList()
         node = nodes.section()
 
-        if data is not None:
-            # Handles the .rst file
-            for line in data.split("\n"):
-                content.append(line, fname, 0)
+        abi_type = self.arguments[0]
 
-            self.do_parse(content, node)
+        if "no-symbols" in self.options:
+            show_symbols = False
+        else:
+            show_symbols = True
 
+        if "no-files" in self.options:
+            show_file = False
+        else:
+            show_file = True
+
+        tab_width = self.options.get('tab-width',
+                                     self.state.document.settings.tab_width)
+
+        old_f = None
+        n = 0
+        n_sym = 0
+        for msg, f, ln in kernel_abi.doc(show_file=show_file,
+                                            show_symbols=show_symbols,
+                                            filter_path=abi_type):
+            n_sym += 1
+            msg_list = statemachine.string2lines(msg, tab_width,
+                                                 convert_whitespace=True)
+            if "debug" in self.options:
+                lines = [
+                    "", "",  ".. code-block:: rst",
+                    "    :linenos:", ""
+                ]
+                for m in msg_list:
+                    lines.append("    " + m)
+            else:
+                lines = msg_list
+
+            for line in lines:
+                # sphinx counts lines from 0
+                content.append(line, f, ln - 1)
+                n += 1
+
+            if f != old_f:
+                # Add the file to Sphinx build dependencies
+                env.note_dependency(os.path.abspath(f))
+
+                old_f = f
+
+            # Sphinx doesn't like to parse big messages. So, let's
+            # add content symbol by symbol
+            if content:
+                self.do_parse(content, node)
+                content = ViewList()
+
+        if show_symbols and not show_file:
+            logger.verbose("%s ABI: %i symbols (%i ReST lines)" % (abi_type, n_sym, n))
+        elif not show_symbols and show_file:
+            logger.verbose("%s ABI: %i files (%i ReST lines)" % (abi_type, n_sym, n))
         else:
-            # Handles the ABI parser content, symbol by symbol
-
-            old_f = fname
-            n = 0
-            for msg, f, ln in self.parser.doc():
-                msg_list = statemachine.string2lines(msg, tab_width,
-                                                     convert_whitespace=True)
-                if "debug" in self.options:
-                    lines = [
-                        "", "",  ".. code-block:: rst",
-                        "    :linenos:", ""
-                    ]
-                    for m in msg_list:
-                        lines.append("    " + m)
-                else:
-                    lines = msg_list
-
-                for line in lines:
-                    # sphinx counts lines from 0
-                    content.append(line, f, ln - 1)
-                    n += 1
-
-                if f != old_f:
-                    # Add the file to Sphinx build dependencies
-                    env.note_dependency(os.path.abspath(f))
-
-                    old_f = f
-
-                # Sphinx doesn't like to parse big messages. So, let's
-                # add content symbol by symbol
-                if content:
-                    self.do_parse(content, node)
-                    content = ViewList()
-
-            self.logger.info("%s: parsed %i lines" % (fname, n))
+            logger.verbose("%s ABI: %i data (%i ReST lines)" % (abi_type, n_sym, n))
 
         return node.children
 
diff --git a/scripts/get_abi.py b/scripts/get_abi.py
index 3cde278b8e4c..7435ed1071b8 100755
--- a/scripts/get_abi.py
+++ b/scripts/get_abi.py
@@ -279,12 +279,20 @@ class AbiParser:
     def parse_readme(self, nametag, fname):
         """Parse ABI README file"""
 
+        nametag["what"] = ["ABI file contents"]
+        nametag["path"] = "README"
         with open(fname, "r", encoding="utf8", errors="backslashreplace") as fp:
-            nametag["description"] = "```\n"
             for line in fp:
-                nametag["description"] += "  " + line
+                match = self.re_tag.match(line)
+                if match:
+                    new = match.group(1).lower()
 
-            nametag["description"] += "```\n"
+                    match = self.re_valid.search(new)
+                    if match:
+                        nametag["description"] += "\n:" + line
+                        continue
+
+                nametag["description"] += line
 
     def parse_file(self, fname, path, basename):
         """Parse a single file"""
@@ -473,12 +481,8 @@ class AbiParser:
                     continue
 
             if filter_path:
-                if filter_path == "README":
-                    if not names[0].endswith("README"):
-                        continue
-                else:
-                    if v.get("path") != filter_path:
-                        continue
+                if v.get("path") != filter_path:
+                    continue
 
             msg = ""
 
-- 
2.48.1


  parent reply	other threads:[~2025-01-28  0:06 UTC|newest]

Thread overview: 61+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-01-28  0:05 [RFC v2 00/38] Improve ABI documentation generation Mauro Carvalho Chehab
2025-01-28  0:05 ` [RFC v2 01/38] docs: power: video.rst: fix a footnote reference Mauro Carvalho Chehab
2025-01-28 22:10   ` Jonathan Corbet
2025-01-28  0:05 ` [RFC v2 02/38] docs: media: ipu3: fix two footnote references Mauro Carvalho Chehab
2025-01-28  0:05 ` [RFC v2 03/38] docs: block: ublk.rst: remove a reference from a dropped text Mauro Carvalho Chehab
2025-01-28  0:05 ` [RFC v2 04/38] docs: sphinx: remove kernellog.py file Mauro Carvalho Chehab
2025-01-28  0:05 ` [RFC v2 05/38] docs: sphinx/kernel_abi: adjust coding style Mauro Carvalho Chehab
2025-02-04 17:19   ` Jonathan Corbet
2025-01-28  0:05 ` [RFC v2 06/38] docs: admin-guide: abi: add SPDX tags to ABI files Mauro Carvalho Chehab
2025-01-28  0:05 ` [RFC v2 07/38] ABI: sysfs-class-rfkill: fix kernelversion tags Mauro Carvalho Chehab
2025-01-28  0:05 ` [RFC v2 08/38] ABI: sysfs-bus-coresight-*: " Mauro Carvalho Chehab
2025-02-04 10:30   ` Suzuki K Poulose
2025-01-28  0:05 ` [RFC v2 09/38] ABI: sysfs-driver-dma-idxd: fix date tags Mauro Carvalho Chehab
2025-01-28  0:05 ` [RFC v2 10/38] ABI: sysfs-fs-f2fs: " Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 11/38] ABI: sysfs-power: fix a what tag Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 12/38] scripts/documentation-file-ref-check: don't check perl/python scripts Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 13/38] scripts/get_abi.py: add a Python tool to generate ReST output Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 14/38] scripts/get_abi.py: add support for symbol search Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 15/38] docs: use get_abi.py for ABI generation Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 16/38] scripts/get_abi.py: optimize parse_abi() function Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 17/38] scripts/get_abi.py: use an interactor for ReST output Mauro Carvalho Chehab
2025-01-29 11:04   ` Akira Yokosawa
2025-01-29 14:06     ` Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 18/38] docs: sphinx/kernel_abi: use AbiParser directly Mauro Carvalho Chehab
2025-01-28 22:37   ` Jonathan Corbet
2025-01-29  0:43     ` Mauro Carvalho Chehab
2025-02-02 14:56       ` Mauro Carvalho Chehab
2025-02-04 17:12         ` Jonathan Corbet
2025-02-10  7:27           ` Mauro Carvalho Chehab
2025-02-10 14:40             ` Jonathan Corbet
2025-02-10 16:03               ` Mauro Carvalho Chehab
2025-02-16 12:53                 ` Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 19/38] docs: sphinx/kernel_abi: reduce buffer usage for ABI messages Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 20/38] docs: sphinx/kernel_abi: properly split lines Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 21/38] scripts/get_abi.pl: Add filtering capabilities to rest output Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 22/38] scripts/get_abi.pl: add support to parse ABI README file Mauro Carvalho Chehab
2025-01-28  0:06 ` Mauro Carvalho Chehab [this message]
2025-01-28  0:06 ` [RFC v2 24/38] docs: admin-guide/abi: split files from symbols Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 25/38] docs: sphinx/automarkup: add cross-references for ABI Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 26/38] docs: sphinx/kernel_abi: avoid warnings during Sphinx module init Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 27/38] scripts/get_abi.py: Rename title name for ABI files Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 28/38] docs: media: Allow creating cross-references for RC ABI Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 29/38] docs: thunderbolt: Allow creating cross-references for ABI Mauro Carvalho Chehab
2025-01-28 11:27   ` Mika Westerberg
2025-01-28  0:06 ` [RFC v2 30/38] docs: arm: asymmetric-32bit: " Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 31/38] docs: arm: generic-counter: " Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 32/38] docs: iio: Allow creating cross-references ABI Mauro Carvalho Chehab
2025-01-28 18:27   ` Jonathan Cameron
2025-01-28  0:06 ` [RFC v2 33/38] docs: networking: Allow creating cross-references statistics ABI Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 34/38] docs: submit-checklist: Allow creating cross-references for ABI README Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 35/38] docs: translations: " Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 36/38] docs: ABI: drop two duplicate symbols Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 37/38] scripts/get_abi.py: add support for undefined ABIs Mauro Carvalho Chehab
2025-01-28  0:06 ` [RFC v2 38/38] scripts/get_abi.pl: drop now obsoleted script Mauro Carvalho Chehab
2025-01-28 22:42 ` [RFC v2 00/38] Improve ABI documentation generation Jonathan Corbet
2025-01-29  1:45   ` Mauro Carvalho Chehab
2025-01-29 14:22     ` Mauro Carvalho Chehab
2025-01-29 15:41   ` Mauro Carvalho Chehab
2025-01-29 15:58     ` Jonathan Corbet
2025-01-29 16:19       ` Mauro Carvalho Chehab
2025-03-27 21:58 ` [f2fs-dev] " patchwork-bot+f2fs

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=52b1fc2f56d6a2eee0d8da2ec08e133de8207402.1738020236.git.mchehab+huawei@kernel.org \
    --to=mchehab+huawei@kernel.org \
    --cc=corbet@lwn.net \
    --cc=gregkh@linuxfoundation.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®