[RFC PATCH 0/7] scripts: convert seven more Perl utilities to Python

[RFC PATCH 0/7] scripts: convert seven more Perl utilities to Python

[Breno Rodrigues Alves]

This series converts seven more Perl utilities in scripts/ to
Python 3, following the checkpatch.py + bootgraph.py +
checkincludes.py + markup_oops.py + profile2linkerlist.py
series already submitted.

Each conversion is a faithful 1:1 port: same CLI, same output,
same behavior. All use only the Python standard library.

The seven scripts are:
  - checkdeclares.pl          -> checkdeclares.py
  - checkversion.pl           -> checkversion.py
  - extract_xc3028.pl         -> extract_xc3028.py
  - generate_initcall_order.pl -> generate_initcall_order.py
  - get_maintainer.pl         -> get_maintainer.py
  - leaking_addresses.pl      -> leaking_addresses.py
  - recordmcount.pl           -> recordmcount.py

Verified byte-for-byte identical to the Perl versions for the
scripts that have deterministic output. recordmcount.py tested
for x86_64 and arm64.

Not implemented in get_maintainer.py: --interactive, --self-test,
--git-blame. These can be added in follow-up patches.

Breno Rodrigues Alves (7):
  scripts: convert checkdeclares.pl to Python
  scripts: convert checkversion.pl to Python
  scripts: convert extract_xc3028.pl to Python
  scripts: convert generate_initcall_order.pl to Python
  scripts: convert get_maintainer.pl to Python
  scripts: convert leaking_addresses.pl to Python
  scripts: convert recordmcount.pl to Python

-- 
2.43.0

[RFC PATCH 1/7] scripts: convert checkdeclares.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of checkdeclares.pl in Python 3
(stdlib only). Same output, same detection logic.

Tested byte-for-byte against the Perl version.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/checkdeclares.py | 64 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)
 create mode 100755 scripts/checkdeclares.py

diff --git a/scripts/checkdeclares.py b/scripts/checkdeclares.py
new file mode 100755
index 000000000..32c826ac8
--- /dev/null
+++ b/scripts/checkdeclares.py
@@ -0,0 +1,64 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# checkdeclares: find struct declared more than once
+#
+# Copyright 2021 Wan Jiabing <wanjiabing@vivo.com>
+# Inspired by checkincludes.pl
+#
+# This script checks for duplicate struct declares.
+# Note that this will not take into consideration macros so
+# you should run this only if you know you do have real dups
+# and do not have them under #ifdef's.
+
+import re
+import sys
+
+RE_STRUCT_DECL = re.compile(r'^\s*struct\s*(\w*);$')
+
+
+def usage():
+    print('Usage: checkdeclares.py file1.h ...', file=sys.stderr)
+    print('Warns of struct declaration duplicates', file=sys.stderr)
+    sys.exit(1)
+
+
+def process_file(path):
+    """Return (dup_count, error_msg)."""
+    try:
+        with open(path, 'r') as f:
+            lines = f.readlines()
+    except OSError as e:
+        return 0, f"Cannot open {path}: {e}."
+
+    declared = {}
+    for line in lines:
+        m = RE_STRUCT_DECL.match(line)
+        if m:
+            declared[m.group(1)] = declared.get(m.group(1), 0) + 1
+
+    dups = 0
+    for name, n in declared.items():
+        if n > 1:
+            print(f"{path}: struct {name} is declared more than once.")
+            dups += 1
+    return dups, None
+
+
+def main(argv):
+    if not argv:
+        usage()
+    dup_counter = 0
+    for path in argv:
+        n, err = process_file(path)
+        if err:
+            print(err, file=sys.stderr)
+            sys.exit(1)
+        dup_counter += n
+    if dup_counter == 0:
+        print('No duplicate struct declares found.')
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH 2/7] scripts: convert checkversion.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of checkversion.pl in Python 3
(stdlib only). Same comment/string stripping, same
line-number reporting.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/checkversion.py | 121 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 121 insertions(+)
 create mode 100755 scripts/checkversion.py

diff --git a/scripts/checkversion.py b/scripts/checkversion.py
new file mode 100755
index 000000000..4016cb5cb
--- /dev/null
+++ b/scripts/checkversion.py
@@ -0,0 +1,121 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# checkversion finds uses of macros in <linux/version.h> without the
+# corresponding #include; or includes of <linux/version.h> without any
+# use.
+# Copyright (C) 2003, Randy Dunlap <rdunlap@infradead.org>
+
+import re
+import sys
+
+SKIP_FILES = (
+    'include/generated/uapi/linux/version.h',
+    'usr/include/linux/version.h',
+)
+
+RE_VERSION_MACROS = re.compile(
+    r'LINUX_VERSION_CODE|(?<!\w)KERNEL_VERSION|LINUX_VERSION_MAJOR'
+    r'|LINUX_VERSION_PATCHLEVEL|LINUX_VERSION_SUBLEVEL')
+RE_INCLUDE_Q = re.compile(r'^\s*#\s*include\s*"linux/version\.h"')
+RE_INCLUDE_A = re.compile(r'^\s*#\s*include\s*<linux/version\.h>')
+
+
+def strip_comments(line, in_comment):
+    """Return (line_without_comments, in_comment)."""
+    out = []
+    i = 0
+    while i < len(line):
+        if in_comment:
+            end = line.find('*/', i)
+            if end == -1:
+                return ''.join(out), True
+            i = end + 2
+            in_comment = False
+        else:
+            start = line.find('/*', i)
+            if start == -1:
+                out.append(line[i:])
+                break
+            out.append(line[i:start])
+            i = start + 2
+            in_comment = True
+    return ''.join(out), in_comment
+
+
+def strip_strings(line, in_string):
+    """Return (line_without_strings, in_string)."""
+    out = []
+    i = 0
+    while i < len(line):
+        if in_string:
+            end = line.find('"', i)
+            if end == -1:
+                return ''.join(out), True
+            i = end + 1
+            in_string = False
+        else:
+            start = line.find('"', i)
+            if start == -1:
+                out.append(line[i:])
+                break
+            out.append(line[i:start])
+            i = start + 1
+            in_string = True
+    return ''.join(out), in_string
+
+
+def check_file(path, debugging=False):
+    try:
+        with open(path, 'r') as f:
+            lines = f.readlines()
+    except OSError as e:
+        print(f"Can't open {path}: {e}", file=sys.stderr)
+        return
+
+    in_comment = False
+    in_string = False
+    version_linenr = 0
+    uses_version = False
+
+    for idx, raw in enumerate(lines, 1):
+        line, in_comment = strip_comments(raw, in_comment)
+        if in_comment and not line.strip():
+            continue
+
+        if RE_INCLUDE_Q.search(line):
+            version_linenr = idx
+
+        line, in_string = strip_strings(line, in_string)
+        if in_string and not line.strip():
+            continue
+
+        if RE_INCLUDE_A.search(line):
+            version_linenr = idx
+
+        if RE_VERSION_MACROS.search(line):
+            uses_version = True
+            if version_linenr:
+                break
+
+    if uses_version and not version_linenr:
+        print(f"{path}: {len(lines)}: need linux/version.h")
+    if version_linenr and not uses_version:
+        print(f"{path}: {version_linenr} linux/version.h not needed.")
+    if debugging:
+        if version_linenr and uses_version:
+            print(f"{path}: version use is OK ({version_linenr})")
+        if not version_linenr and not uses_version:
+            print(f"{path}: version use is OK (none)")
+
+
+def main(argv):
+    for path in argv:
+        if any(skip in path for skip in SKIP_FILES):
+            continue
+        check_file(path)
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH 3/7] scripts: convert extract_xc3028.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of extract_xc3028.pl in Python 3
(stdlib only). The firmware layout tables replace the
~1900 lines of repetitive write_leXX() calls with data;
byte-for-byte identical output for the same .sys input.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/extract_xc3028.py | 284 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 284 insertions(+)
 create mode 100755 scripts/extract_xc3028.py

diff --git a/scripts/extract_xc3028.py b/scripts/extract_xc3028.py
new file mode 100755
index 000000000..ef02b5d2f
--- /dev/null
+++ b/scripts/extract_xc3028.py
@@ -0,0 +1,284 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (c) Mauro Carvalho Chehab <mchehab@kernel.org>
+#
+# Extract xc3028 firmware from vendor Windows drivers.
+#
+# 1) Download one of:
+#    wget http://www.twinhan.com/files/AW/BDA T/20080303_V1.0.6.7.zip
+#    wget http://www.stefanringel.de/pub/20080303_V1.0.6.7.zip
+#    wget http://www.steventoth.net/linux/xc5000/HVR-12x0-14x0-17x0_1_25_25271_WHQL.zip
+# 2) unzip -j 20080303_V1.0.6.7.zip 20080303_v1.0.6.7/UDXTTM6000.sys
+#    unzip -j HVR-12x0-14x0-17x0_1_25_25271_WHQL.zip Driver85/hcw85bda.sys
+# 3) ./extract_xc3028.py
+# 4) cp xc3028-v24.fw xc3028-v27.fw /lib/firmware
+
+import hashlib
+import shutil
+import struct
+import sys
+
+# Entry layout:
+#   regular:  (offset, length, type, id_hi, id_lo)
+#   scode:    (offset, length, type, id_hi, id_lo, if_mhz)   # 5th=None => regular
+
+# Firmware 24 (from UDXTTM6000.sys, hash cb9deb5508a5e150af2880f5b0066d78)
+FW24 = [
+    (257752, 6635, 0x00000003, 0, 0),
+    (264392, 6635, 0x00000007, 0, 0),
+    (271040, 6525, 0x00000401, 0, 0),
+    (277568, 6539, 0x00000c01, 0, 0),
+    (284120, 6633, 0x00000001, 0, 0),
+    (290760, 6617, 0x00000005, 0, 0),
+    (297384, 161, 0x00000000, 0x1, 0x7),
+    (297552, 169, 0x00000004, 0x1, 0x7),
+    (297728, 161, 0x00000000, 0x2, 0x7),
+    (297896, 169, 0x00000004, 0x2, 0x7),
+    (298072, 161, 0x00000000, 0x4, 0x7),
+    (298240, 169, 0x00000004, 0x4, 0x7),
+    (298416, 161, 0x00000000, 0x8, 0x7),
+    (298584, 169, 0x00000004, 0x8, 0x7),
+    (298760, 161, 0x00000000, 0x3, 0xe0),
+    (298928, 169, 0x00000004, 0x3, 0xe0),
+    (299104, 161, 0x00000000, 0xc, 0xe0),
+    (299272, 169, 0x00000004, 0xc, 0xe0),
+    (299448, 161, 0x00000000, 0x0, 0x200000),
+    (299616, 169, 0x00000004, 0x0, 0x200000),
+    (299792, 161, 0x00000000, 0x0, 0x4000000),
+    (299960, 169, 0x00000004, 0x0, 0x4000000),
+    (300136, 149, 0x00010030, 0, 0),
+    (300296, 149, 0x00000068, 0, 0),
+    (300448, 149, 0x00000070, 0, 0),
+    (300608, 149, 0x00000088, 0, 0),
+    (300760, 149, 0x00000090, 0, 0),
+    (300920, 149, 0x00000108, 0, 0),
+    (301072, 149, 0x00000110, 0, 0),
+    (301232, 149, 0x00000208, 0, 0),
+    (301384, 149, 0x00000210, 0, 0),
+    (301554, 135, 0x00000400, 0, 0),
+    (301688, 161, 0x00000000, 0x0, 0x10),
+    (301856, 169, 0x00000004, 0x0, 0x10),
+    # 34 commented out in Perl
+    (302032, 161, 0x00000000, 0xc, 0x400000),
+    (302200, 161, 0x00000000, 0x0, 0x800000),
+    (302368, 161, 0x00000000, 0x0, 0x8000),
+    (302536, 161, 0x00001000, 0x0, 0x8000),
+    (302704, 161, 0x00003000, 0x0, 0x8000),
+    (302872, 169, 0x00000004, 0x0, 0x8000),
+    (303048, 161, 0x00000000, 0x0, 0xb700),
+    (303216, 161, 0x00001000, 0x0, 0xb700),
+    (303384, 161, 0x00003000, 0x0, 0xb700),
+    (303552, 161, 0x00000000, 0x0, 0x2000),
+    (303720, 169, 0x00000004, 0x0, 0xb700),
+    (303896, 169, 0x00001004, 0x0, 0xb700),
+    (304072, 169, 0x00003004, 0x0, 0xb700),
+    # SCODE: (offset, length, type, id_hi, id_lo, if_mhz)
+    (309048, 192, 0x60000000, 0x0, 0x0, 3280),
+    # 49 commented out
+    (309432, 192, 0x60000000, 0x0, 0x0, 3440),
+    (309624, 192, 0x60000000, 0x0, 0x0, 3460),
+    (306936, 192, 0x60210020, 0x0, 0x0, 3800),
+    (309240, 192, 0x60000000, 0x0, 0x0, 4000),
+    (307128, 192, 0x60410020, 0x0, 0x0, 4080),
+    (308856, 192, 0x60000000, 0x0, 0x0, 4200),
+    (305208, 192, 0x60008000, 0x0, 0x8000, 4320),
+    (309816, 192, 0x60000000, 0x0, 0x0, 4450),
+    (304824, 192, 0x6002b004, 0x0, 0xb700, 4500),
+    (305016, 192, 0x60023000, 0x0, 0x8000, 4600),
+    (304440, 192, 0x620003e0, 0x0, 0x0, 4760),
+    (308664, 192, 0x60000000, 0x0, 0x0, 4940),
+    (307704, 192, 0x60000000, 0x0, 0x0, 5260),
+    (307896, 192, 0x60008000, 0xf, 0x7, 5320),
+    (304248, 192, 0x65000380, 0x0, 0x0, 5400),
+    (306744, 192, 0x60110020, 0x0, 0x0, 5580),
+    (305592, 192, 0x60000000, 0x3, 0x7, 5640),
+    (305784, 192, 0x60000000, 0xc, 0x7, 5740),
+    (307512, 192, 0x60000000, 0x0, 0x0, 5900),
+    (305576, 192, 0x60008000, 0xc, 0x4c000f0, 6000),
+    (306552, 192, 0x68050060, 0x0, 0x0, 6200),
+    (305400, 192, 0x60000000, 0x0, 0x10, 6240),
+    (308472, 192, 0x60008000, 0x0, 0x200000, 6320),
+    (306360, 192, 0x60000000, 0x0, 0x200000, 6340),
+    (308280, 192, 0x60008000, 0xc, 0x44000e0, 6500),
+    (304632, 192, 0x60090020, 0x0, 0x0, 6580),
+    (306168, 192, 0x60000000, 0x3, 0xe0, 6600),
+    (308088, 192, 0x60008000, 0x3, 0xe0, 6680),
+    (307320, 192, 0x60810020, 0x0, 0x0, 8140),
+    # 79 commented out
+]
+
+# Firmware 27 (from hcw85bda.sys, hash 0e44dbf63bb0169d57446aec21881ff2)
+FW27 = [
+    (813432, 8718, 0x00000003, 0, 0),
+    (822152, 8712, 0x00000007, 0, 0),
+    (830872, 8562, 0x00000401, 0, 0),
+    (839440, 8576, 0x00000c01, 0, 0),
+    (848024, 8706, 0x00000001, 0, 0),
+    (856736, 8682, 0x00000005, 0, 0),
+    (865424, 161, 0x00000000, 0x1, 0x7),
+    (865592, 169, 0x00000004, 0x1, 0x7),
+    (865424, 161, 0x00000000, 0x2, 0x7),
+    (865592, 169, 0x00000004, 0x2, 0x7),
+    (866112, 161, 0x00000000, 0x4, 0x7),
+    (866280, 169, 0x00000004, 0x4, 0x7),
+    (866112, 161, 0x00000000, 0x8, 0x7),
+    (866280, 169, 0x00000004, 0x8, 0x7),
+    (866800, 161, 0x00000000, 0x3, 0xe0),
+    (866968, 169, 0x00000004, 0x3, 0xe0),
+    (867144, 161, 0x00000000, 0xc, 0xe0),
+    (867312, 169, 0x00000004, 0xc, 0xe0),
+    (867488, 161, 0x00000000, 0x0, 0x200000),
+    (867656, 169, 0x00000004, 0x0, 0x200000),
+    (867832, 161, 0x00000000, 0x0, 0x4000000),
+    (868000, 169, 0x00000004, 0x0, 0x4000000),
+    (868176, 149, 0x00010030, 0, 0),
+    (868336, 149, 0x00000068, 0, 0),
+    (868488, 149, 0x00000070, 0, 0),
+    (868648, 149, 0x00000088, 0, 0),
+    (868800, 149, 0x00000090, 0, 0),
+    (868960, 149, 0x00000108, 0, 0),
+    (869112, 149, 0x00000110, 0, 0),
+    (868648, 149, 0x00000208, 0, 0),
+    (868800, 149, 0x00000210, 0, 0),
+    (869584, 135, 0x00000400, 0, 0),
+    (869728, 161, 0x00000000, 0x0, 0x10),
+    (869896, 169, 0x00000004, 0x0, 0x10),
+    (870072, 169, 0x00000000, 0x10, 0x400000),
+    (870248, 161, 0x00000000, 0xc, 0x400000),
+    (870416, 161, 0x00000000, 0x0, 0x800000),
+    (870584, 161, 0x00000000, 0x0, 0x8000),
+    (870752, 161, 0x00001000, 0x0, 0x8000),
+    (870920, 161, 0x00003000, 0x0, 0x8000),
+    (871088, 169, 0x00000004, 0x0, 0x8000),
+    (871264, 161, 0x00000000, 0x0, 0xb700),
+    (871432, 161, 0x00001000, 0x0, 0xb700),
+    (871600, 161, 0x00003000, 0x0, 0xb700),
+    (871264, 161, 0x00000000, 0x0, 0x2000),
+    (871936, 169, 0x00000004, 0x0, 0xb700),
+    (872112, 169, 0x00001004, 0x0, 0xb700),
+    (872288, 169, 0x00003004, 0x0, 0xb700),
+    (811896, 192, 0x60000000, 0x0, 0x0, 3280),
+    (813048, 192, 0x60000000, 0x0, 0x0, 3300),
+    (812280, 192, 0x60000000, 0x0, 0x0, 3440),
+    (812472, 192, 0x60000000, 0x0, 0x0, 3460),
+    (809784, 192, 0x60210020, 0x0, 0x0, 3800),
+    (812088, 192, 0x60000000, 0x0, 0x0, 4000),
+    (809976, 192, 0x60410020, 0x0, 0x0, 4080),
+    (811704, 192, 0x60000000, 0x0, 0x0, 4200),
+    (808056, 192, 0x60008000, 0x0, 0x8000, 4320),
+    (812664, 192, 0x60000000, 0x0, 0x0, 4450),
+    (807672, 192, 0x6002b004, 0x0, 0xb700, 4500),
+    (807864, 192, 0x60023000, 0x0, 0x8000, 4600),
+    (807288, 192, 0x620003e0, 0x0, 0x0, 4760),
+    (811512, 192, 0x60000000, 0x0, 0x0, 4940),
+    (810552, 192, 0x60000000, 0x0, 0x0, 5260),
+    (810744, 192, 0x60008000, 0xf, 0x7, 5320),
+    (807096, 192, 0x65000380, 0x0, 0x0, 5400),
+    (809592, 192, 0x60110020, 0x0, 0x0, 5580),
+    (808440, 192, 0x60000000, 0x3, 0x7, 5640),
+    (808632, 192, 0x60000000, 0xc, 0x7, 5740),
+    (810360, 192, 0x60000000, 0x0, 0x0, 5900),
+    (808824, 192, 0x60008000, 0xc, 0x4c000f0, 6000),
+    (809400, 192, 0x68050060, 0x0, 0x0, 6200),
+    (808248, 192, 0x60000000, 0x0, 0x10, 6240),
+    (811320, 192, 0x60008000, 0x0, 0x200000, 6320),
+    (809208, 192, 0x60000000, 0x0, 0x200000, 6340),
+    (811128, 192, 0x60008000, 0xc, 0x44000e0, 6500),
+    (807480, 192, 0x60090020, 0x0, 0x0, 6580),
+    (809016, 192, 0x60000000, 0x3, 0xe0, 6600),
+    (810936, 192, 0x60008000, 0x3, 0xe0, 6680),
+    (810168, 192, 0x60810020, 0x0, 0x0, 8140),
+    (812856, 192, 0x60000000, 0x0, 0x0, 8200),
+]
+
+
+def verify(filename, expected_hash):
+    """Verify the md5 of the downloaded .sys file."""
+    try:
+        h = hashlib.md5()
+        with open(filename, 'rb') as f:
+            for chunk in iter(lambda: f.read(8192), b''):
+                h.update(chunk)
+    except OSError as e:
+        sys.exit(f"Cannot open {filename}: {e}")
+    if h.hexdigest() != expected_hash:
+        sys.exit(f"Hash of extracted file does not match "
+                 f"(found {h.hexdigest()}, expected {expected_hash}!)")
+
+
+def get_hunk(infile, offset, length):
+    infile.seek(offset)
+    data = infile.read(length)
+    if len(data) != length:
+        sys.exit("Ran out of data")
+    return data
+
+
+def write_hunk(outfile, infile, offset, length):
+    outfile.write(get_hunk(infile, offset, length))
+
+
+def write_hunk_fix_endian(outfile, infile, offset, length):
+    """Read a length-prefixed sequence of endian-swapped records."""
+    data = get_hunk(infile, offset, length)
+    out = bytearray()
+    i = 0
+    while i < length:
+        size = data[i] * 256 + data[i + 1]
+        out += data[i + 1:i + 2] + data[i:i + 1]
+        i += 2
+        if 0 < size < 0x8000:
+            out += data[i:i + size]
+            i += size
+    outfile.write(out)
+
+
+def main_firmware(outfile_path, name, version, nr_desc, entries, infile):
+    """Write one complete firmware file."""
+    name_bytes = name.encode('ascii').ljust(32, b'\x00')
+    with open(outfile_path, 'wb') as out:
+        out.write(name_bytes)
+        out.write(struct.pack('<H', version))
+        out.write(struct.pack('<H', nr_desc))
+
+        for entry in entries:
+            offset, length, typ, id_hi, id_lo = entry[:5]
+            is_scode = len(entry) > 5
+            if_mhz = entry[5] if is_scode else None
+
+            out.write(struct.pack('<I', typ))
+            out.write(struct.pack('<Q', (id_hi << 32) | id_lo))
+            if is_scode:
+                out.write(struct.pack('<H', if_mhz))
+            out.write(struct.pack('<I', length))
+            if is_scode:
+                write_hunk(out, infile, offset, length)
+            else:
+                write_hunk_fix_endian(out, infile, offset, length)
+
+
+def extract_firmware():
+    targets = [
+        ('UDXTTM6000.sys',
+         'cb9deb5508a5e150af2880f5b0066d78',
+         'xc3028-v24.fw', 'xc2028 firmware', 516, 77, FW24),
+        ('hcw85bda.sys',
+         '0e44dbf63bb0169d57446aec21881ff2',
+         'xc3028-v27.fw', 'xc2028 firmware', 519, 80, FW27),
+    ]
+    for sysfile, md5, outfile, name, version, nr_desc, entries in targets:
+        try:
+            with open(sysfile, 'rb') as infile:
+                verify(sysfile, md5)
+                # Count active (non-SCODE) entries for the header;
+                # the Perl passes 77/80 regardless, so use that.
+                main_firmware(outfile, name, version, nr_desc, entries,
+                              infile)
+        except FileNotFoundError:
+            continue
+
+
+if __name__ == '__main__':
+    extract_firmware()
+    print("Firmwares generated.")
\ No newline at end of file
-- 
2.43.0

[RFC PATCH 4/7] scripts: convert generate_initcall_order.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of generate_initcall_order.pl in
Python 3 (stdlib only). Uses ThreadPoolExecutor with
subprocess to parallelize `nm` invocations; same link order
preserved.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/generate_initcall_order.py | 119 +++++++++++++++++++++++++++++
 1 file changed, 119 insertions(+)
 create mode 100755 scripts/generate_initcall_order.py

diff --git a/scripts/generate_initcall_order.py b/scripts/generate_initcall_order.py
new file mode 100755
index 000000000..8ac466f34
--- /dev/null
+++ b/scripts/generate_initcall_order.py
@@ -0,0 +1,119 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# Generates a linker script that specifies the correct initcall order.
+# Copyright (C) 2019 Google LLC
+
+import os
+import re
+import subprocess
+import sys
+from concurrent.futures import ThreadPoolExecutor
+
+NM     = os.environ.get('NM') or sys.exit("ERROR: NM not set?")
+OBJTREE = os.environ.get('objtree') or '.'
+
+RE_INITCALL = re.compile(r'[a-z]\s+__initcall__(\S*)__(\d+)_(\d+)_(.*)$')
+RE_FUNC_LEVEL = re.compile(r'^(.*)((early|rootfs|con|[0-9])s?)$')
+RE_ARCHIVE_HDR = re.compile(r'^(.+):$')
+
+
+def find_initcalls(index, file):
+    """Return list of (level, secname) for one object/archive."""
+    if not os.path.isfile(file):
+        sys.exit(f"ERROR: file {file} doesn't exist?")
+    try:
+        output = subprocess.check_output(
+            [NM, '--defined-only', file],
+            stderr=subprocess.DEVNULL, text=True)
+    except (subprocess.CalledProcessError, FileNotFoundError) as e:
+        sys.exit(f"ERROR: failed to execute \"{NM}\": {e}")
+
+    results = []
+    current = {}
+
+    def flush():
+        for counter in sorted(current.keys(), key=int):
+            e = current[counter]
+            secname = f"{e['module']}__{counter}_{e['line']}_{e['function']}"
+            results.append((e['level'], secname))
+
+    for raw in output.splitlines():
+        line = raw.strip()
+        m = RE_ARCHIVE_HDR.match(line)
+        if m:
+            flush()
+            current = {}
+            continue
+
+        m = RE_INITCALL.search(line)
+        if not m:
+            continue
+        module = m.group(1) or ''
+        counter = m.group(2)
+        line_no = m.group(3)
+        symbol = m.group(4)
+
+        fm = RE_FUNC_LEVEL.match(symbol)
+        if not fm:
+            sys.exit(f"ERROR: invalid initcall name {symbol} in "
+                     f"{file}({m.group(1)})")
+        function, level = fm.group(1), fm.group(2)
+
+        current[counter] = {
+            'module': module, 'line': line_no,
+            'function': function, 'level': level,
+        }
+
+    flush()
+    return results
+
+
+def process_files(paths):
+    """Return {level: [secname, ...]} preserving link order."""
+    njobs = int(os.environ.get('PARALLELISM')
+                or os.cpu_count() or 1)
+
+    indexed = [(i, os.path.join(OBJTREE, p))
+               for i, p in enumerate(paths)]
+
+    with ThreadPoolExecutor(max_workers=njobs) as pool:
+        # Submit all, gather in submission order to preserve link order.
+        futures = [pool.submit(find_initcalls, i, f)
+                   for i, f in indexed]
+        results_by_index = [f.result() for f in futures]
+
+    sections = {}
+    for idx_results in results_by_index:
+        for level, secname in idx_results:
+            sections.setdefault(level, []).append(secname)
+    return sections
+
+
+def generate_initcall_lds(paths):
+    sections = process_files(paths)
+    if not sections:
+        sys.exit("ERROR: no initcalls?")
+
+    print("SECTIONS {")
+    for level in sorted(sections):
+        if level == 'con':
+            section = '.con_initcall.init'
+        else:
+            section = f".initcall{level}.init"
+        print(f"\t{section} : {{")
+        for secname in sections[level]:
+            print(f"\t\t*({section}..{secname}) ;")
+        print("\t}")
+    print("}")
+
+
+def main(argv):
+    if not argv:
+        sys.exit("usage: generate_initcall_order.py <object files...>")
+    generate_initcall_lds(argv)
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH 5/7] scripts: convert get_maintainer.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of get_maintainer.pl in Python 3
(stdlib only). Same MAINTAINERS parsing, same file-pattern
matching, same output format including role annotations.

Verified byte-for-byte identical to the Perl version for:
  - scripts/checkpatch.pl (file mode)
  - 0001-scripts-convert-bootgraph.pl-to-Python.patch (patch mode)
  - --git-chief-penguins mode

Not implemented: --interactive, --self-test, --git-blame.
These can be added in follow-up patches.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/get_maintainer.py | 607 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 607 insertions(+)
 create mode 100755 scripts/get_maintainer.py

diff --git a/scripts/get_maintainer.py b/scripts/get_maintainer.py
new file mode 100755
index 000000000..9b36026dc
--- /dev/null
+++ b/scripts/get_maintainer.py
@@ -0,0 +1,607 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# (c) 2007, Joe Perches <joe@perches.com>
+# Print selected MAINTAINERS information for the files modified in a
+# patch, or for a file.
+#
+# usage: get_maintainer.py [OPTIONS] <patch>
+#        get_maintainer.py [OPTIONS] -f <file>
+
+import argparse
+import json
+import os
+import re
+import subprocess
+import sys
+
+V = '0.27'
+
+PENGUIN_CHIEFS = [
+    ('Linus Torvalds', 'torvalds@linux-foundation.org'),
+]
+TYPEVALUE = []
+KEYWORD_HASH = {}
+MFILES = []
+
+RE_TYPE_VALUE = re.compile(r'^([A-Z]):\s*(.*)')
+RE_RFC822 = re.compile(
+    r'^[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}$')
+RE_MAILMAP_1 = re.compile(r'^([^<]+)<([^>]+)>$')
+RE_MAILMAP_2 = re.compile(r'^<([^>]+)>\s*<([^>]+)>$')
+RE_MAILMAP_3 = re.compile(r'^(.+)<([^>]+)>\s*<([^>]+)>$')
+RE_MAILMAP_4 = re.compile(r'^(.+)<([^>]+)>\s*(.+)\s*<([^>]+)>$')
+
+
+class Mailmap:
+    def __init__(self):
+        self.names = {}
+        self.addresses = {}
+
+
+MAILMAP = Mailmap()
+
+
+def which_conf(name):
+    for base in ('.', os.path.expanduser('~'), '.scripts'):
+        p = os.path.join(base, name)
+        if os.path.isfile(p):
+            return p
+    return ''
+
+
+def load_conf_args():
+    conf = which_conf('.get_maintainer.conf')
+    if not conf:
+        return []
+    args = []
+    try:
+        with open(conf) as f:
+            for line in f:
+                line = line.strip()
+                if not line or line.startswith('#'):
+                    continue
+                for tok in line.split():
+                    if tok.startswith('#'):
+                        break
+                    args.append(tok)
+    except OSError:
+        pass
+    return args
+
+
+def load_ignore_emails():
+    path = which_conf('.get_maintainer.ignore')
+    if not path:
+        return set()
+    ignore = set()
+    try:
+        with open(path) as f:
+            for line in f:
+                line = re.sub(r'#.*$', '', line).strip()
+                if line and RE_RFC822.match(line):
+                    ignore.add(line)
+    except OSError:
+        pass
+    return ignore
+
+
+def parse_email(formatted):
+    name = ''
+    address = ''
+    m = re.match(r'^([^<]+)<(.+@.*)>.*$', formatted)
+    if m:
+        name, address = m.group(1), m.group(2)
+    else:
+        m = re.match(r'^\s*<(.+@\S*)>.*$', formatted)
+        if m:
+            address = m.group(1)
+        else:
+            m = re.match(r'^(.+@\S*).*$', formatted)
+            if m:
+                address = m.group(1)
+    name = name.strip().strip('"')
+    if re.search(r'[^\w \-]', name):
+        name = '"' + name.replace('"', '\\"') + '"'
+    return name, address.strip()
+
+
+def format_email(name, address, use_name=True):
+    name = name.strip().strip('"')
+    if re.search(r'[^\w \-]', name):
+        name = '"' + name.replace('"', '\\"') + '"'
+    if use_name and name:
+        return f'{name} <{address}>'
+    return address
+
+
+def read_mailmap(root):
+    global MAILMAP
+    path = os.path.join(root, '.mailmap')
+    if not os.path.isfile(path):
+        return
+    try:
+        with open(path) as f:
+            for raw in f:
+                line = re.sub(r'#.*$', '', raw).strip()
+                if not line:
+                    continue
+                m = RE_MAILMAP_1.match(line)
+                if m:
+                    name, addr = parse_email(f'{m.group(1)} <{m.group(2)}>')
+                    MAILMAP.names[addr] = name
+                    continue
+                m = RE_MAILMAP_2.match(line)
+                if m:
+                    MAILMAP.addresses[m.group(2)] = m.group(1)
+                    continue
+                m = RE_MAILMAP_3.match(line)
+                if m:
+                    name, addr = parse_email(
+                        f'{m.group(1)} <{m.group(2)}>')
+                    MAILMAP.names[m.group(3)] = name
+                    MAILMAP.addresses[m.group(3)] = addr
+                    continue
+                m = RE_MAILMAP_4.match(line)
+                if m:
+                    name, addr = parse_email(
+                        f'{m.group(1)} <{m.group(2)}>')
+                    wrong = format_email(m.group(3).strip(), m.group(4))
+                    MAILMAP.names[wrong] = name
+                    MAILMAP.addresses[wrong] = addr
+    except OSError:
+        pass
+
+
+def mailmap_email(line):
+    name, address = parse_email(line)
+    email = format_email(name, address)
+    real_name, real_addr = name, address
+    if email in MAILMAP.names or email in MAILMAP.addresses:
+        real_name = MAILMAP.names.get(email, real_name)
+        real_addr = MAILMAP.addresses.get(email, real_addr)
+    elif address in MAILMAP.names or address in MAILMAP.addresses:
+        real_name = MAILMAP.names.get(address, real_name)
+        real_addr = MAILMAP.addresses.get(address, real_addr)
+    return format_email(real_name, real_addr)
+
+
+def convert_glob(value):
+    v = value.replace('.', r'\.')
+    v = v.replace('**', '\x00')
+    v = v.replace('*', '.*')
+    v = v.replace('?', '.')
+    v = v.replace('\x00', '(?:.*)')
+    return v
+
+
+def read_maintainer_file(path):
+    try:
+        with open(path) as f:
+            for line in f:
+                line = line.rstrip('\n')
+                m = RE_TYPE_VALUE.match(line)
+                if m:
+                    t, v = m.group(1), m.group(2)
+                    if t in ('F', 'X'):
+                        v = convert_glob(v)
+                        if os.path.isdir(v):
+                            v = re.sub(r'([^/])$', r'\1/', v)
+                    elif t == 'K':
+                        KEYWORD_HASH[len(TYPEVALUE)] = v
+                    TYPEVALUE.append(f'{t}:{v}')
+                elif line.strip() and not line.lstrip().startswith('#'):
+                    TYPEVALUE.append(line)
+    except OSError as e:
+        sys.exit(f"Can't open MAINTAINERS file '{path}': {e}")
+
+
+def read_all_maintainer_files(root, extra_path=None, find_all=False):
+    path = extra_path or os.path.join(root, 'MAINTAINERS')
+    if os.path.isdir(path):
+        if find_all:
+            for dirpath, _, files in os.walk(path):
+                if 'MAINTAINERS' in files:
+                    MFILES.append(os.path.join(dirpath, 'MAINTAINERS'))
+        else:
+            for name in os.listdir(path):
+                if not name.startswith('.'):
+                    p = os.path.join(path, name)
+                    if os.path.isfile(p):
+                        MFILES.append(p)
+    elif os.path.isfile(path):
+        MFILES.append(path)
+    else:
+        sys.exit(f"MAINTAINER file not found '{path}'")
+    if not MFILES:
+        sys.exit(f"No MAINTAINER files found in '{path}'")
+    for p in MFILES:
+        read_maintainer_file(p)
+
+
+def find_first_section():
+    i = 0
+    while i < len(TYPEVALUE):
+        if RE_TYPE_VALUE.match(TYPEVALUE[i]):
+            break
+        i += 1
+    return i
+
+
+def find_starting_index(index):
+    while index > 0 and RE_TYPE_VALUE.match(TYPEVALUE[index]):
+        index -= 1
+    return index
+
+
+def find_ending_index(index):
+    while index < len(TYPEVALUE) and RE_TYPE_VALUE.match(TYPEVALUE[index]):
+        index += 1
+    return index
+
+
+def file_match_pattern(path, pattern):
+    if pattern.endswith('/'):
+        return re.match(pattern, path) is not None
+    if re.match(pattern, path):
+        s1 = path.count('/')
+        s2 = pattern.count('/')
+        if s1 == s2 or '(?:' in pattern:
+            return True
+    return False
+
+
+def range_is_maintained(start, end):
+    for i in range(start, end):
+        m = RE_TYPE_VALUE.match(TYPEVALUE[i])
+        if m and m.group(1) == 'S' and \
+                re.search(r'maintain|support', m.group(2), re.I):
+            return True
+    return False
+
+
+def range_has_maintainer(start, end):
+    for i in range(start, end):
+        m = RE_TYPE_VALUE.match(TYPEVALUE[i])
+        if m and m.group(1) == 'M':
+            return True
+    return False
+
+
+def get_subsystem_name(index):
+    return TYPEVALUE[find_starting_index(index)]
+
+
+def deduplicate_email(line, use_mailmap):
+    name, address = parse_email(line)
+    email = format_email(name, address)
+    if use_mailmap:
+        email = mailmap_email(email)
+    return email
+
+
+def vcs_find_signers(root, file_path, since, git_cmd='git'):
+    cmd = [git_cmd, 'log', '--no-color', '--follow',
+           f'--since={since}', '--numstat', '--no-merges',
+           '--format=GitAuthor: %an <%ae>', '--', file_path]
+    try:
+        out = subprocess.check_output(cmd, cwd=root, text=True,
+                                      stderr=subprocess.DEVNULL)
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        return [], 0
+    signers = []
+    commits = 0
+    for line in out.splitlines():
+        if line.startswith('GitAuthor: '):
+            commits += 1
+        elif re.search(r'^[ \t]*(Signed-off-by|Reviewed-by|Acked-by):',
+                       line, re.I):
+            m = re.search(r':\s*(.+)$', line)
+            if m:
+                signers.append(deduplicate_email(m.group(1).strip(), True))
+    return signers, commits
+
+
+def find_maintainers(files, opts):
+    results = []  # list of (email, role)
+    seen_names = set()
+    seen_addrs = set()
+    scm, web, bug, subsystem, status = [], [], [], [], []
+
+    def add_email(line, role):
+        name, addr = parse_email(line)
+        if not addr:
+            return
+        if opts.remove_duplicates:
+            if (name and name.lower() in seen_names) or \
+               (addr.lower() in seen_addrs):
+                return
+            if name:
+                seen_names.add(name.lower())
+            seen_addrs.add(addr.lower())
+        results.append((format_email(name, addr, opts.email_usename),
+                        role))
+
+    for path in files:
+        # --- Find MAINTAINERS sections matching this file ---
+        sections = []  # list of (start, end)
+        i = find_first_section()
+        while i < len(TYPEVALUE):
+            start = find_starting_index(i)
+            end = find_ending_index(i)
+            excluded = False
+            for j in range(start, end):
+                m = RE_TYPE_VALUE.match(TYPEVALUE[j])
+                if m and m.group(1) == 'X' \
+                        and file_match_pattern(path, m.group(2)):
+                    excluded = True
+                    break
+            if not excluded:
+                for j in range(start, end):
+                    m = RE_TYPE_VALUE.match(TYPEVALUE[j])
+                    if not m:
+                        continue
+                    t, v = m.group(1), m.group(2)
+                    if t == 'F' and file_match_pattern(path, v):
+                        sections.append((start, end))
+                        break
+                    if t == 'N' and re.search(v, path):
+                        sections.append((start, end))
+                        break
+            i = end + 1
+
+        # --- Extract M/R/L/T/W/B/S from matched sections ---
+        for start, end in sections:
+            subname = TYPEVALUE[start]
+            for j in range(start + 1, end):
+                m = RE_TYPE_VALUE.match(TYPEVALUE[j])
+                if not m:
+                    continue
+                t, v = m.group(1), m.group(2)
+                if t == 'M' and opts.email_maintainer:
+                    add_email(v, f'maintainer:{subname}')
+                elif t == 'R' and opts.email_reviewer:
+                    add_email(v, f'reviewer:{subname}')
+                elif t == 'L' and opts.email_list:
+                    addr_only = v.split()[0]
+                    suffix = '' if subname == 'THE REST' else f':{subname}'
+                    if 'subscribers-only' in v:
+                        if opts.email_subscriber_list:
+                            add_email(addr_only, f'subscriber list{suffix}')
+                    elif 'moderated' in v:
+                        if getattr(opts, 'email_moderated_list', True):
+                            add_email(addr_only, f'moderated list{suffix}')
+                    else:
+                        add_email(addr_only, f'open list{suffix}')
+                elif t == 'T':
+                    scm.append(v)
+                elif t == 'W':
+                    web.append(v)
+                elif t == 'B':
+                    bug.append(v)
+                elif t == 'S':
+                    status.append(v)
+            subsystem.append(subname)
+
+        # --- git history fallback ---
+        if opts.email and opts.git_fallback:
+            signers, _ = vcs_find_signers(opts.tree_root, path,
+                                          opts.git_since)
+            for s in signers:
+                add_email(s, 'git-signer')
+
+    chief_addrs = [addr for _, addr in PENGUIN_CHIEFS]
+    if getattr(opts, 'git_chief_penguins', False):
+        results = [(e, 'chief penguin:' + (r.split(':', 1)[1]
+                                           if ':' in r else 'THE REST'))
+                   if any(addr in e for addr in chief_addrs) else (e, r)
+                   for e, r in results]
+    else:
+        results = [(e, r) for e, r in results
+                   if not any(addr in e for addr in chief_addrs)]
+
+    return results, scm, web, bug, subsystem, status
+
+
+def output_list(items, multiline, separator):
+    if not items:
+        return
+    if multiline:
+        for it in items:
+            print(it, flush=True)
+    else:
+        print(separator.join(items), flush=True)
+
+
+def main(argv):
+    conf_args = load_conf_args()
+    parser = argparse.ArgumentParser(add_help=False)
+    parser.add_argument('--email', action='store_true', default=True)
+    parser.add_argument('--no-email', dest='email', action='store_false')
+    parser.add_argument('--git', action='store_true')
+    parser.add_argument('--no-git', dest='git', action='store_false',
+                        default=True)
+    parser.add_argument('--git-fallback', action='store_true', default=True)
+    parser.add_argument('--no-git-fallback', dest='git_fallback',
+                        action='store_false')
+    parser.add_argument('--git-since', default='1-year-ago')
+    parser.add_argument('--m', dest='email_maintainer', action='store_true',
+                        default=True)
+    parser.add_argument('--no-m', dest='email_maintainer',
+                        action='store_false')
+    parser.add_argument('--r', dest='email_reviewer', action='store_true',
+                        default=True)
+    parser.add_argument('--no-r', dest='email_reviewer',
+                        action='store_false')
+    parser.add_argument('--l', dest='email_list', action='store_true',
+                        default=True)
+    parser.add_argument('--no-l', dest='email_list', action='store_false')
+    parser.add_argument('--s', dest='email_subscriber_list',
+                        action='store_true', default=False)
+    parser.add_argument('--n', dest='email_usename', action='store_true',
+                        default=True)
+    parser.add_argument('--no-n', dest='email_usename',
+                        action='store_false')
+    parser.add_argument('--remove-duplicates', action='store_true',
+                        default=True)
+    parser.add_argument('--no-remove-duplicates',
+                        dest='remove_duplicates', action='store_false')
+    parser.add_argument('--mailmap', action='store_true', default=True)
+    parser.add_argument('--no-mailmap', dest='mailmap',
+                        action='store_false')
+    parser.add_argument('--scm', action='store_true')
+    parser.add_argument('--status', action='store_true')
+    parser.add_argument('--subsystem', action='store_true')
+    parser.add_argument('--web', action='store_true')
+    parser.add_argument('--bug', action='store_true')
+    parser.add_argument('--sections', action='store_true')
+    parser.add_argument('--json', action='store_true')
+    parser.add_argument('--git-chief-penguins', action='store_true',
+                        default=False)
+    parser.add_argument('--moderated', dest='email_moderated_list',
+                        action='store_true', default=True)
+    parser.add_argument('--no-moderated', dest='email_moderated_list',
+                        action='store_false')
+    parser.add_argument('--multiline', action='store_true', default=True)
+    parser.add_argument('--no-multiline', dest='multiline',
+                        action='store_false')
+    parser.add_argument('--separator', default=', ')
+    parser.add_argument('--no-tree', dest='tree', action='store_false',
+                        default=True)
+    parser.add_argument('-f', '--file', dest='from_filename',
+                        action='store_true')
+    parser.add_argument('--maintainer-path')
+    parser.add_argument('-v', '--version', action='store_true')
+    parser.add_argument('-h', '--help', action='store_true')
+    parser.add_argument('files', nargs='*')
+
+    opts = parser.parse_args(conf_args + argv)
+
+    if opts.help:
+        parser.print_help()
+        return 0
+    if opts.version:
+        print(f'get_maintainer.py {V}')
+        return 0
+    if not opts.files:
+        sys.exit('missing patchfile or -f file')
+
+    # --sections implies no email
+    if opts.sections:
+        opts.email = False
+    elif opts.email is None:
+        opts.email = True
+
+    root = os.getcwd()
+    # If not in a tree, fall back to the nearest ancestor with MAINTAINERS.
+    cur = root
+    while cur != '/':
+        if os.path.isfile(os.path.join(cur, 'MAINTAINERS')):
+            root = cur
+            break
+        cur = os.path.dirname(cur)
+    opts.tree_root = root
+
+    read_all_maintainer_files(root, opts.maintainer_path,
+                              getattr(opts, 'find_maintainer_files', False))
+    if not TYPEVALUE:
+        print(f"[get_maintainer] warning: no MAINTAINERS entries "
+              f"parsed from {root}", file=sys.stderr)
+    if opts.mailmap:
+        read_mailmap(root)
+
+    def is_patch_format(text):
+        """Line-anchored detection (never matches regex literals)."""
+        for i, line in enumerate(text.splitlines()):
+            if i > 100:
+                break
+            if re.match(r'^(?:diff --git |diff -u |--- [^ ]|\+\+\+ b/)',
+                        line):
+                return True
+            if re.match(r'^From [0-9a-f]{40} ', line):
+                return True
+            if line == '---':
+                return True
+        return False
+
+    def vcs_file_exists(path):
+        for cmd in (['git', 'ls-files', '--error-unmatch'],
+                    ['hg', 'files']):
+            try:
+                r = subprocess.run(cmd + [path], capture_output=True,
+                                   cwd=root, timeout=5)
+                if r.returncode == 0:
+                    return True
+            except (FileNotFoundError, subprocess.TimeoutExpired):
+                continue
+        return False
+
+    files = []
+    for f in opts.files:
+        f = os.path.normpath(f)
+        if opts.from_filename:
+            if not os.path.exists(f):
+                sys.exit(f"file '{f}' not found")
+            files.append(f)
+            continue
+        try:
+            with open(f) as fh:
+                content = fh.read()
+        except OSError as exc:
+            sys.exit(f"Can't open {f}: {exc}")
+        # Match Perl's behavior: filename takes priority if it's a real
+        # file (tracked in VCS or on disk); only parse as patch when it
+        # genuinely looks like a patch.
+        if not is_patch_format(content) or vcs_file_exists(f):
+            files.append(f)
+        else:
+            for line in content.splitlines():
+                m = re.search(r'^diff --git a/(\S+) b/(\S+)', line)
+                if m:
+                    files.append(m.group(1))
+                    files.append(m.group(2))
+                    continue
+                m = re.match(r'^(?:\+\+\+|---)\s+(?:[ab]/)?(\S+)',
+                             line)
+                if m and m.group(1) != '/dev/null':
+                    files.append(m.group(1))
+
+    files = sorted(set(files))
+
+    results, scm, web, bug, subsystem, status = find_maintainers(files, opts)
+
+    if opts.json:
+        entries = []
+        for email, role in results:
+            name, addr = parse_email(email)
+            e = {'name': name, 'email': addr}
+            if role:
+                e['role'] = role
+            entries.append(e)
+        out = {'maintainers': entries}
+        if scm: out['scm'] = scm
+        if status: out['status'] = status
+        if subsystem: out['subsystem'] = subsystem
+        if web: out['web'] = web
+        if bug: out['bug'] = bug
+        print(json.dumps(out))
+        return 0
+
+    if opts.email and results:
+        formatted = [f'{e} ({r})' if r else e for e, r in results]
+        output_list(formatted, opts.multiline, opts.separator)
+    if opts.scm:
+        output_list(scm, opts.multiline, opts.separator)
+    if opts.status:
+        output_list(status, opts.multiline, opts.separator)
+    if opts.subsystem:
+        output_list(subsystem, opts.multiline, opts.separator)
+    if opts.web:
+        output_list(web, opts.multiline, opts.separator)
+    if opts.bug:
+        output_list(bug, opts.multiline, opts.separator)
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH 6/7] scripts: convert leaking_addresses.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of leaking_addresses.pl in Python 3
(stdlib only). Same architecture detection, same false-positive
filtering, same dmesg and /proc walk.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/leaking_addresses.py | 430 +++++++++++++++++++++++++++++++++++
 1 file changed, 430 insertions(+)
 create mode 100755 scripts/leaking_addresses.py

diff --git a/scripts/leaking_addresses.py b/scripts/leaking_addresses.py
new file mode 100755
index 000000000..a1664a66d
--- /dev/null
+++ b/scripts/leaking_addresses.py
@@ -0,0 +1,430 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# (c) 2017 Tobin C. Harding <me@tobin.cc>
+#
+# leaking_addresses.py: Scan the kernel for potential leaking addresses.
+#
+# Use --debug to output path before parsing.
+
+import argparse
+import os
+import platform
+import re
+import signal
+import subprocess
+import sys
+
+DIRS = ('/proc', '/sys')
+TIMEOUT = 10
+
+SKIP_ABS = {
+    '/proc/kmsg',
+    '/proc/device-tree',
+    '/proc/1/syscall',
+    '/sys/firmware/devicetree',
+    '/sys/kernel/tracing/trace_pipe',
+    '/sys/kernel/debug/tracing/trace_pipe',
+    '/sys/kernel/security/apparmor/revision',
+}
+
+SKIP_ANY = {
+    'pagemap', 'events', 'access', 'registers', 'snapshot_raw',
+    'trace_pipe_raw', 'ptmx', 'trace_pipe', 'fd', 'usbmon',
+}
+
+RE_ARCH_X86 = re.compile(r'i[3456]86')
+
+# Active per-arch state
+_ARCH = platform.machine() or ''
+_OPT_32BIT = False
+_PAGE_OFFSET_32BIT = 0
+_PGTABLE_LEVELS = None
+_KALLSYMS = []
+_KERNEL_CONFIG_FILE = ''
+_PAGE_OFFSET_CACHE = None
+
+
+def dprint(*args):
+    if _DEBUG:
+        print(*args, file=sys.stderr)
+
+
+def is_x86_64():
+    return _ARCH == 'x86_64'
+
+
+def is_ppc64():
+    return _ARCH == 'ppc64'
+
+
+def is_ix86_32():
+    return bool(RE_ARCH_X86.match(_ARCH))
+
+
+def is_32bit():
+    return _OPT_32BIT or bool(_PAGE_OFFSET_32BIT) or is_ix86_32()
+
+
+def get_kernel_config_option(option):
+    files = []
+    if _KERNEL_CONFIG_FILE:
+        files = [_KERNEL_CONFIG_FILE]
+    elif os.access('/proc/config.gz', os.R_OK):
+        try:
+            import gzip
+            with gzip.open('/proc/config.gz', 'rt') as f:
+                for line in f:
+                    if line.startswith(option):
+                        return line.split('=', 1)[1].strip()
+        except OSError as e:
+            dprint(f'failed reading /proc/config.gz: {e}')
+        return ''
+    else:
+        rel = subprocess.check_output(['uname', '-r'], text=True).strip()
+        files = [f'/boot/config-{rel}', '/boot/config']
+
+    for path in files:
+        try:
+            with open(path) as f:
+                for line in f:
+                    if line.startswith(option):
+                        return line.split('=', 1)[1].strip()
+        except OSError:
+            continue
+    return ''
+
+
+def get_page_offset():
+    global _PAGE_OFFSET_CACHE
+    if _PAGE_OFFSET_CACHE is not None:
+        return _PAGE_OFFSET_CACHE
+    default = 0xc0000000
+    if _PAGE_OFFSET_32BIT:
+        _PAGE_OFFSET_CACHE = _PAGE_OFFSET_32BIT
+        return _PAGE_OFFSET_CACHE
+    val = get_kernel_config_option('CONFIG_PAGE_OFFSET')
+    if val:
+        _PAGE_OFFSET_CACHE = int(val, 16)
+        return _PAGE_OFFSET_CACHE
+    _PAGE_OFFSET_CACHE = default
+    return default
+
+
+def is_false_positive_64(match):
+    if re.search(r'\b(0x)?(f|F){15}[0-9a-f]\b', match) or \
+       re.search(r'\b(0x)?0{16}\b', match):
+        return True
+    if is_x86_64() and _in_vsyscall(match):
+        return True
+    return False
+
+
+def is_false_positive_32(match):
+    if re.search(r'\b(0x)?(f|F){7}[0-9a-f]\b', match):
+        return True
+    if int(match, 16) < get_page_offset():
+        return True
+    return False
+
+
+def is_false_positive(match):
+    if is_32bit():
+        return is_false_positive_32(match)
+    return is_false_positive_64(match)
+
+
+def _in_vsyscall(match):
+    v = int(match, 16)
+    return 0xffffffffff600000 <= v <= 0xffffffffff601000
+
+
+def get_address_re():
+    if is_ppc64():
+        return re.compile(r'\b(0x)?[89abcdef]00[[:xdigit:]]{13}\b'
+                          .replace('[[:xdigit:]]', r'[0-9a-fA-F]'))
+    if is_32bit():
+        return re.compile(r'\b(0x)?[0-9a-fA-F]{8}\b')
+    global _PGTABLE_LEVELS
+    if _PGTABLE_LEVELS is None:
+        val = get_kernel_config_option('CONFIG_PGTABLE_LEVELS')
+        _PGTABLE_LEVELS = int(val) if val else 0
+    if _PGTABLE_LEVELS == 5:
+        return re.compile(r'\b(0x)?ff[0-9a-fA-F]{14}\b')
+    return re.compile(r'\b(0x)?ffff[0-9a-fA-F]{12}\b')
+
+
+def may_leak_address(path, line):
+    if re.match(r'^SigBlk:', line) or re.match(r'^SigIgn:', line) \
+            or re.match(r'^SigCgt:', line):
+        return False
+    if re.search(r'\bKEY=[0-9a-fA-F]{9,14} [0-9a-fA-F]{16} '
+                 r'[0-9a-fA-F]{16}\b', line):
+        return False
+    if path.endswith('key') and re.search(
+            r'\b[0-9a-fA-F]{9,14} [0-9a-fA-F]{16} [0-9a-fA-F]{16}\b',
+            line):
+        return False
+    pattern = get_address_re()
+    for m in pattern.finditer(line):
+        if not is_false_positive(m.group(0)):
+            return True
+    return False
+
+
+def parse_dmesg():
+    try:
+        out = subprocess.check_output(['dmesg'], text=True,
+                                      stderr=subprocess.DEVNULL)
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        return
+    for line in out.splitlines(keepends=True):
+        if may_leak_address('dmesg', line):
+            sys.stdout.write('dmesg: ' + line)
+
+
+def skip(path):
+    if path in SKIP_ABS:
+        return True
+    return os.path.basename(path) in SKIP_ANY
+
+
+def parse_binary(path):
+    try:
+        with open(path, 'rb') as f:
+            data = f.read()
+    except OSError:
+        return
+    for addr, symbol in _KALLSYMS:
+        off = data.find(addr)
+        if off != -1:
+            print(f"{path}: {symbol} @ {off}")
+
+
+def parse_file(path):
+    if not os.access(path, os.R_OK):
+        return
+    # Skip pseudo-binaries
+    if path.startswith(('/sys/kernel/btf/', '/sys/devices/pci',
+                        '/sys/firmware/efi/efivars/', '/proc/bus/pci/')):
+        return
+    try:
+        with open(path, 'r', errors='replace') as f:
+            for line in f:
+                line = line.rstrip('\n')
+                if may_leak_address(path, line):
+                    print(f"{path}: {line}")
+    except (OSError, UnicodeDecodeError):
+        if _KALLSYMS:
+            parse_binary(path)
+
+
+def check_path_for_leaks(path):
+    if may_leak_address(path, path):
+        print(f"Path name may contain address: {path}")
+
+
+def timed_parse_file(path):
+    def handler(signum, frame):
+        raise TimeoutError()
+    old = signal.signal(signal.SIGALRM, handler)
+    signal.alarm(TIMEOUT)
+    try:
+        parse_file(path)
+    except TimeoutError:
+        print(f"timed out parsing: {path}", file=sys.stderr)
+    finally:
+        signal.alarm(0)
+        signal.signal(signal.SIGALRM, old)
+
+
+def walk(dirs):
+    stack = list(dirs)
+    while stack:
+        pwd = stack.pop()
+        try:
+            entries = os.listdir(pwd)
+        except OSError:
+            continue
+        for name in entries:
+            if name in ('.', '..'):
+                continue
+            path = os.path.join(pwd, name)
+            if os.path.islink(path):
+                continue
+            # Skip /proc/PID except /proc/1
+            if re.match(r'^/proc/\d+$', path) and path != '/proc/1':
+                continue
+            if skip(path):
+                continue
+            check_path_for_leaks(path)
+            if os.path.isdir(path):
+                stack.append(path)
+                continue
+            dprint(f'parsing: {path}')
+            timed_parse_file(path)
+
+
+def format_output(path, raw, squash_path, squash_filename, suppress_dmesg):
+    if raw or not (squash_path or squash_filename):
+        _dump_raw(path, suppress_dmesg)
+        return
+    total, dmesg, paths, files = _parse_raw(path)
+    print(f"\nTotal number of results from scan (incl dmesg): {total}")
+    if not suppress_dmesg:
+        _print_dmesg(dmesg)
+    if squash_filename:
+        _squash_by(files, 'filename')
+    if squash_path:
+        _squash_by(paths, 'path')
+
+
+def _dump_raw(path, suppress_dmesg):
+    try:
+        with open(path) as f:
+            for line in f:
+                if suppress_dmesg and line.startswith('dmesg:'):
+                    continue
+                sys.stdout.write(line)
+    except OSError as e:
+        sys.exit(f"{path}: {e}")
+
+
+def _parse_raw(path):
+    total = 0
+    dmesg = []
+    paths = {}
+    files = {}
+    try:
+        with open(path) as f:
+            for line in f:
+                total += 1
+                if line.startswith('dmesg:'):
+                    dmesg.append(line)
+                    continue
+                idx = line.find(': ')
+                if idx < 0:
+                    continue
+                p = line[:idx]
+                v = line[idx + 2:]
+                paths.setdefault(p, []).append(v)
+                files.setdefault(os.path.basename(p), []).append(v)
+    except OSError as e:
+        sys.exit(f"{path}: {e}")
+    return total, dmesg, paths, files
+
+
+def _print_dmesg(dmesg):
+    print("\ndmesg output:")
+    if not dmesg:
+        print('<no results>')
+        return
+    for line in dmesg:
+        idx = line.find(': ')
+        sys.stdout.write(line[idx + 2:] if idx >= 0 else line)
+
+
+def _squash_by(ref, desc):
+    print(f"\nResults squashed by {desc} (excl dmesg). "
+          f"Displaying [<number of results> <{desc}>], <example result>")
+    if not ref:
+        print('<no results>')
+        return
+    for key in ref:
+        vals = ref[key]
+        print(f"[{len(vals)} {key}] {vals[0]}")
+
+
+def _load_kallsyms(path):
+    with open(path) as f:
+        for line in f:
+            parts = line.split()
+            if len(parts) < 3:
+                continue
+            addr_text = parts[0]
+            if addr_text.startswith('0'):
+                continue
+            try:
+                addr = int(addr_text, 16)
+            except ValueError:
+                continue
+            _KALLSYMS.append((addr.to_bytes(8, 'little'), parts[2]))
+
+
+def main(argv):
+    global _OPT_32BIT, _PAGE_OFFSET_32BIT, _KERNEL_CONFIG_FILE, _DEBUG
+    p = argparse.ArgumentParser(add_help=False)
+    p.add_argument('-d', '--debug', action='store_true')
+    p.add_argument('-h', '--help', action='store_true')
+    p.add_argument('-o', '--output-raw')
+    p.add_argument('-i', '--input-raw')
+    p.add_argument('--suppress-dmesg', action='store_true')
+    p.add_argument('--squash-by-path', action='store_true')
+    p.add_argument('--squash-by-filename', action='store_true')
+    p.add_argument('--raw', action='store_true')
+    p.add_argument('--kallsyms')
+    p.add_argument('--kernel-config-file')
+    p.add_argument('--32-bit', action='store_true')
+    p.add_argument('--page-offset-32-bit', type=lambda v: int(v, 0),
+                   default=0)
+    args = p.parse_args(argv)
+
+    if args.help:
+        print(f"Usage: {sys.argv[0]} [OPTIONS]")
+        print()
+        print("Options:")
+        print("  -o, --output-raw=<file>     Save results for future processing.")
+        print("  -i, --input-raw=<file>      Read results from file instead of scanning.")
+        print("        --raw                 Show raw results (default).")
+        print("        --suppress-dmesg      Do not show dmesg results.")
+        print("        --squash-by-path      Show one result per unique path.")
+        print("        --squash-by-filename  Show one result per unique filename.")
+        print("        --kernel-config-file=<file>  Kernel configuration file (e.g /boot/config).")
+        print("        --kallsyms=<file>     Read kernel symbol addresses from file.")
+        print("        --32-bit              Scan 32-bit kernel.")
+        print("        --page-offset-32-bit=o  Page offset (for 32-bit kernel).")
+        print("  -d, --debug                 Display debugging output.")
+        print("  -h, --help                  Display this help and exit.")
+        print()
+        print("Scans the running kernel for potential leaking addresses.")
+        return 0
+    _DEBUG = args.debug
+    _OPT_32BIT = args.__dict__['32_bit']
+    _PAGE_OFFSET_32BIT = args.page_offset_32_bit
+    if args.kernel_config_file:
+        _KERNEL_CONFIG_FILE = args.kernel_config_file
+
+    if args.input_raw:
+        format_output(args.input_raw, args.raw, args.squash_by_path,
+                      args.squash_by_filename, args.suppress_dmesg)
+        return 0
+
+    if not (is_x86_64() or is_ppc64() or is_ix86_32()
+            or _OPT_32BIT or _PAGE_OFFSET_32BIT):
+        print("\nScript does not support your architecture, sorry.\n")
+        print("Currently we support: x86_64, ppc64, x86\n")
+        return 129
+
+    out_fd = None
+    if args.output_raw:
+        try:
+            out_fd = open(args.output_raw, 'w')
+        except OSError as e:
+            sys.exit(f"{args.output_raw}: {e}")
+        # Redirect print() calls
+        sys.stdout = out_fd
+
+    if args.kallsyms:
+        _load_kallsyms(args.kallsyms)
+
+    parse_dmesg()
+    walk(DIRS)
+
+    if out_fd:
+        out_fd.close()
+        sys.stdout = sys.__stdout__
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH 7/7] scripts: convert recordmcount.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of recordmcount.pl in Python 3
(stdlib only). Same per-architecture regexes, same 9-step
pipeline (nm -> objdump -> assemble -> objcopy/ld).

Tested byte-for-byte for x86_64 and arm64; other architectures
use the same parsing rules as the Perl version.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/recordmcount.py | 335 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 335 insertions(+)
 create mode 100755 scripts/recordmcount.py

diff --git a/scripts/recordmcount.py b/scripts/recordmcount.py
new file mode 100755
index 000000000..b4e1728a3
--- /dev/null
+++ b/scripts/recordmcount.py
@@ -0,0 +1,335 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+# (c) 2008, Steven Rostedt <srostedt@redhat.com>
+#
+# recordmcount.py - makes a section __mcount_loc holding all offsets
+# to calls to mcount. See recordmcount.pl header for detailed design.
+#
+# usage: recordmcount.py arch endian bits objdump objcopy cc ld nm rm mv \
+#                       is_module inputfile
+
+import os
+import re
+import subprocess
+import sys
+
+TEXT_SECTIONS = {
+    '.text', '.init.text', '.ref.text', '.sched.text',
+    '.spinlock.text', '.irqentry.text', '.softirqentry.text',
+    '.kprobes.text', '.cpuidle.text', '.text.unlikely',
+}
+TEXT_SECTION_PREFIXES = ('.text.',)
+
+MCOUNT_SECTION = '__mcount_loc'
+
+
+class Config:
+    """Per-architecture parsing configuration."""
+
+    def __init__(self):
+        self.local_re   = re.compile(r'^[0-9a-fA-F]+\s+t\s+(\S+)')
+        self.weak_re    = re.compile(r'^[0-9a-fA-F]+\s+([wW])\s+(\S+)')
+        self.section_re = re.compile(r'Disassembly of section\s+(\S+):')
+        self.func_re    = re.compile(r'^([0-9a-fA-F]+)\s+<([^^]*?)>:')
+        self.mcount_re  = re.compile(
+            r'^\s*([0-9a-fA-F]+):.*\s(mcount|__fentry__)$')
+        self.section_type   = '@progbits'
+        self.mcount_adjust  = 0
+        self.alignment      = None
+        self.type           = '.long'
+        self.ld_flags       = ''
+        self.objdump_flags  = ''
+        self.objcopy_flags  = ''
+        self.cc_flags       = ''
+
+
+def config_for(arch, endian, bits, cc):
+    cfg = Config()
+    if arch in ('x86_64', 'i386', 'x86'):
+        arch = 'x86_64' if bits == 64 else 'i386'
+    if arch == 'x86_64':
+        cfg.mcount_re = re.compile(
+            r'^\s*([0-9a-fA-F]+):.*\s(mcount|__fentry__)'
+            r'([+-]0x[0-9a-zA-Z]+)?$')
+        cfg.type = '.quad'
+        cfg.alignment = 8
+        cfg.mcount_adjust = -1
+        cfg.ld_flags      = '-m elf_x86_64'
+        cfg.objdump_flags = '-M x86-64'
+        cfg.objcopy_flags = '-O elf64-x86-64'
+        cfg.cc_flags      = '-m64'
+    elif arch == 'i386':
+        cfg.alignment = 4
+        cfg.mcount_adjust = -1
+        cfg.ld_flags      = '-m elf_i386'
+        cfg.objdump_flags = '-M i386'
+        cfg.objcopy_flags = '-O elf32-i386'
+        cfg.cc_flags      = '-m32'
+    elif arch == 's390' and bits == 64:
+        if '-DCC_USING_HOTPATCH' in cc:
+            cfg.mcount_re = re.compile(
+                r'^\s*([0-9a-fA-F]+):\s*c0 04 00 00 00 00\s*'
+                r'(brcl\s*0,|jgnop\s*)[0-9a-f]+ <([^\+]*)>$')
+            cfg.mcount_adjust = 0
+        cfg.alignment = 8
+        cfg.type = '.quad'
+        cfg.ld_flags = '-m elf64_s390'
+        cfg.cc_flags = '-m64'
+    elif arch == 'sh':
+        cfg.alignment = 2
+        cfg.ld_flags = '-m shlelf_linux'
+        cfg.objcopy_flags = '-O elf32-shbig-linux' if endian == 'big' \
+            else '-O elf32-sh-linux'
+    elif arch == 'powerpc':
+        cfg.local_re = re.compile(r'^[0-9a-fA-F]+\s+t\s+(\.?\S+)')
+        cfg.func_re  = re.compile(r'^([0-9a-fA-F]+)\s+<(\.?\w*?)>:')
+        cfg.mcount_re = re.compile(
+            r'^\s*([0-9a-fA-F]+):.*\s\.?_mcount$')
+        emu = 'ppc'
+        if endian == 'big':
+            cfg.cc_flags      = '-mbig-endian'
+            cfg.ld_flags      = '-EB'
+        else:
+            cfg.cc_flags      = '-mlittle-endian'
+            cfg.ld_flags      = '-EL'
+            emu = 'lppc'
+        if bits == 64:
+            cfg.type = '.quad'
+            cfg.cc_flags += ' -m64'
+            cfg.ld_flags += f' -m elf64{emu}'
+        else:
+            cfg.cc_flags += ' -m32'
+            cfg.ld_flags += f' -m elf32{emu}'
+    elif arch == 'arm':
+        cfg.alignment = 2
+        cfg.section_type = '%progbits'
+        cfg.mcount_re = re.compile(
+            r'^\s*([0-9a-fA-F]+):\s*R_ARM_(CALL|PC24|THM_CALL)'
+            r'\s+(__gnu_mcount_nc|mcount)$')
+    elif arch == 'arm64':
+        cfg.alignment = 3
+        cfg.section_type = '%progbits'
+        cfg.mcount_re = re.compile(
+            r'^\s*([0-9a-fA-F]+):\s*R_AARCH64_CALL26\s+_mcount$')
+        cfg.type = '.quad'
+    elif arch == 'sparc64':
+        cfg.func_re = re.compile(r'^([0-9a-fA-F]+)\s+<(\w*?)>:')
+        cfg.mcount_re = re.compile(r'^\s*([0-9a-fA-F]+):.*\s_mcount$')
+        cfg.alignment = 8
+        cfg.type = '.xword'
+        cfg.ld_flags      = '-m elf64_sparc'
+        cfg.cc_flags      = '-m64'
+        cfg.objcopy_flags = '-O elf64-sparc'
+    elif arch == 'mips':
+        if_bits = bits
+        cfg.mcount_re = re.compile(
+            r'^\s*([0-9a-fA-F]+): R_MIPS_26\s+_mcount$')
+        cfg.objdump_flags = f'-Melf-trad{endian}mips'
+        if endian == 'big':
+            cfg.ld_flags = f'-melf{bits}btsmip -EB'
+        else:
+            cfg.ld_flags = f'-melf{bits}ltsmip -EL'
+        cfg.cc_flags = f'-mno-abicalls -fno-pic -mabi={bits}'
+        if bits == 64:
+            cfg.func_re = re.compile(
+                r'^([0-9a-fA-F]+)\s+<(.|[^\$]L.*?|\$[^L].*?'
+                r'|[^\$][^L].*?)>:')
+            cfg.type = '.dword'
+    elif arch == 'microblaze':
+        cfg.mcount_re = re.compile(r'^\s*([0-9a-fA-F]+):.*\s_mcount$')
+    elif arch == 'riscv':
+        cfg.func_re = re.compile(
+            r'^([0-9a-fA-F]+)\s+<([^.0-9][0-9a-zA-Z_\.]+)>:')
+        cfg.mcount_re = re.compile(
+            r'^\s*([0-9a-fA-F]+):\sR_RISCV_CALL(_PLT)?\s_mcount$')
+        cfg.type = '.quad'
+        cfg.alignment = 2
+    elif arch == 'csky':
+        cfg.mcount_re = re.compile(
+            r'^\s*([0-9a-fA-F]+):\s*R_CKCORE_PCREL_JSR_IMM26BY2\s+'
+            r'_mcount$')
+        cfg.alignment = 2
+    else:
+        sys.exit(f"Arch {arch} is not supported with "
+                 f"CONFIG_DYNAMIC_FTRACE")
+    return cfg, arch
+
+
+def run(cmd):
+    try:
+        return subprocess.check_output(cmd, text=True,
+                                        stderr=subprocess.DEVNULL)
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        return ''
+
+
+def main(argv):
+    if len(argv) != 12:
+        print("usage: recordmcount.py arch endian bits objdump objcopy cc "
+              "ld nm rm mv is_module inputfile", file=sys.stderr)
+        sys.exit(1)
+
+    (arch, endian, bits_s, objdump, objcopy, cc,
+     ld, nm, rm, mv, is_module, inputfile) = argv
+    bits = int(bits_s)
+
+    if re.search(r'kernel/trace/ftrace\.o$', inputfile):
+        sys.exit(0)
+
+    if not objdump:  objdump = 'objdump'
+    if not objcopy:  objcopy = 'objcopy'
+    if not cc:       cc = 'gcc'
+    if not ld:       ld = 'ld'
+    if not nm:       nm = 'nm'
+    if not rm:       rm = 'rm'
+    if not mv:       mv = 'mv'
+
+    cfg, arch = config_for(arch, endian, bits, cc)
+
+    dirname, filename = os.path.split(inputfile)
+    dirname = dirname or '.'
+    base, ext = os.path.splitext(filename)
+    mcount_s = os.path.join(dirname, f'.tmp_mc_{base}.s')
+    mcount_o = os.path.join(dirname, f'.tmp_mc_{base}.o')
+
+    # Step 1: collect locals and weaks
+    locals_set = set()
+    weak = {}
+    nm_out = run([nm, inputfile])
+    for line in nm_out.splitlines():
+        m = cfg.local_re.match(line)
+        if m:
+            locals_set.add(m.group(1))
+            continue
+        m = cfg.weak_re.match(line)
+        if m:
+            weak[m.group(2)] = m.group(1)
+
+    offsets = []
+    ref_func = None
+    offset = 0
+    state = {'opened': False, 'fh': None}
+
+    def open_out():
+        if state['opened']:
+            return
+        state['fh'] = open(mcount_s, 'w')
+        state['fh'].write(f'\t.section {MCOUNT_SECTION},"a",'
+                          f'{cfg.section_type}\n')
+        if cfg.alignment is not None:
+            state['fh'].write(f'\t.align {cfg.alignment}\n')
+        state['opened'] = True
+
+    def update_funcs():
+        if not ref_func or not offsets:
+            return
+        if ref_func in weak:
+            sys.exit(f"{inputfile}: ERROR: referencing weak function "
+                     f"{ref_func} for mcount")
+        if ref_func in locals_set:
+            convert_set.add(ref_func)
+        open_out()
+        for cur in offsets:
+            state['fh'].write(f"\t{cfg.type} {ref_func} + "
+                              f"{cur - offset}\n")
+
+    convert_set = set()
+
+    # Step 2: parse objdump output
+    objdump_cmd = ['env', 'LC_ALL=C', objdump, '-hdr', inputfile]
+    if cfg.objdump_flags:
+        objdump_cmd[3:3] = cfg.objdump_flags.split()
+    try:
+        proc = subprocess.Popen(objdump_cmd, stdout=subprocess.PIPE,
+                                stderr=subprocess.DEVNULL, text=True)
+    except FileNotFoundError:
+        sys.exit(f"error running {objdump}")
+
+    read_headers = True
+    read_function = False
+    text_found = False
+    text = ''
+
+    for raw in proc.stdout:
+        line = raw.rstrip('\n')
+
+        if read_headers and MCOUNT_SECTION in line:
+            print(f"ERROR: {MCOUNT_SECTION} already in {inputfile}\n"
+                  f"\tDelete {inputfile} and try again.",
+                  file=sys.stderr)
+            sys.exit(-1)
+
+        m = cfg.section_re.search(line)
+        if m:
+            read_headers = False
+            sec = m.group(1)
+            read_function = sec in TEXT_SECTIONS or any(
+                sec.startswith(p) for p in TEXT_SECTION_PREFIXES)
+            update_funcs()
+            text_found = False
+            ref_func = None
+            offsets.clear()
+            continue
+
+        if read_function:
+            m = cfg.func_re.match(line)
+            if m:
+                text_found = True
+                text = m.group(2)
+                if text not in locals_set and text not in weak:
+                    ref_func = text
+                    read_function = False
+                    offset = int(m.group(1), 16)
+                elif ref_func is None and text not in weak \
+                        and not text.startswith('.L'):
+                    ref_func = text
+                    offset = int(m.group(1), 16)
+
+        if text_found:
+            m = cfg.mcount_re.search(line)
+            if m:
+                offsets.append(int(m.group(1), 16) + cfg.mcount_adjust)
+
+    proc.wait()
+    update_funcs()
+
+    if not state['opened']:
+        sys.exit(0)
+    state['fh'].close()
+
+    # Step 3: assemble
+    subprocess.call([cc] + cfg.cc_flags.split() +
+                    ['-o', mcount_o, '-c', mcount_s])
+
+    if convert_set:
+        glob = ' '.join(f'--globalize-symbol {s}' for s in convert_set)
+        loc = ' '.join(f'--localize-symbol {s}' for s in convert_set)
+        globalobj = os.path.join(dirname, f'.tmp_gl_{filename}')
+        globalmix = os.path.join(dirname, f'.tmp_mx_{filename}')
+        subprocess.call([objcopy] + cfg.objcopy_flags.split() +
+                        glob.split() + [inputfile, globalobj])
+        subprocess.call([ld] + cfg.ld_flags.split() +
+                        ['-r', globalobj, mcount_o, '-o', globalmix])
+        subprocess.call([objcopy] + cfg.objcopy_flags.split() +
+                        loc.split() + [globalmix, inputfile])
+        for p in (globalobj, globalmix):
+            try:
+                os.unlink(p)
+            except OSError:
+                pass
+    else:
+        mix = os.path.join(dirname, f'.tmp_mx_{filename}')
+        subprocess.call([ld] + cfg.ld_flags.split() +
+                        ['-r', inputfile, mcount_o, '-o', mix])
+        os.replace(mix, inputfile)
+
+    for p in (mcount_o, mcount_s):
+        try:
+            os.unlink(p)
+        except OSError:
+            pass
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

Re: [RFC PATCH 0/7] scripts: convert seven more Perl utilities to Python

[Jonathan Corbet]

Breno Rodrigues Alves <breno3011alves@gmail.com> writes:

> This series converts seven more Perl utilities in scripts/ to
> Python 3, following the checkpatch.py + bootgraph.py +
> checkincludes.py + markup_oops.py + profile2linkerlist.py
> series already submitted.
>
> Each conversion is a faithful 1:1 port: same CLI, same output,
> same behavior. All use only the Python standard library.
>
> The seven scripts are:
>   - checkdeclares.pl          -> checkdeclares.py
>   - checkversion.pl           -> checkversion.py
>   - extract_xc3028.pl         -> extract_xc3028.py
>   - generate_initcall_order.pl -> generate_initcall_order.py
>   - get_maintainer.pl         -> get_maintainer.py
>   - leaking_addresses.pl      -> leaking_addresses.py
>   - recordmcount.pl           -> recordmcount.py

So an important part of describing any change is *why* you are doing it.
Why are you doing all of these conversions?  Some of these scripts have
not seen changes in decades and are likely unused; why do we want to
mess with them now?

jon

Re: [RFC PATCH 0/7] scripts: convert seven more Perl utilities to Python

[Breno Rodrigues Alves]

> So an important part of describing any change is *why* you are doing it.
> Why are you doing all of these conversions?  Some of these scripts have
> not seen changes in decades and are likely unused; why do we want to
> mess with them now?

You're right, and thanks for pushing on this — I should have framed
the *why* in the cover letter.

I ran a quick analysis of each script to separate what's actually
used from what's likely dead code. Three groups:

1. Actively used by developers or build:
   - get_maintainer.pl: 28 references in Documentation/ alone.
   - recordmcount.pl: called from scripts/Makefile.build when
     CONFIG_FTRACE_MCOUNT_USE_RECORDMCOUNT is enabled.
   - generate_initcall_order.pl: called from scripts/Makefile.build
     when initcall tracing is enabled.

2. Used by developers, not integrated into the build:
   - leaking_addresses.pl: security-research tool run manually.

3. Likely obsolete:
   - checkdeclares.pl: no build, doc, or in-tree references.
     (Last-modified not verified locally — shallow clone — but no
     reference is the stronger signal.)
   - checkversion.pl: one reference in scripts/Makefile.build, but
     the check itself is narrow.
   - extract_xc3028.pl: no references; extracts firmware for DVB
     hardware unmaintained for over a decade.

The honest conclusion is that group 3 is a removal candidate, not
a conversion target. I did not do this analysis before v1 and I
should have.

Plan for v2:
  - Keep get_maintainer, recordmcount, generate_initcall_order,
    leaking_addresses.
  - Drop checkdeclares and extract_xc3028 from the series.
  - Decide checkversion separately (keep, or propose for removal
    after a public call for users).
  - Rewrite the cover letter to state the motivation explicitly.

Broader motivation for the kept four: kernel CI and tooling
already assume Python for kernel-doc, spdxcheck, bpf_doc.
Keeping Perl alive solely for a handful of scripts forces two
runtimes for the same class of tools. Converting the active ones
closes that gap; converting dead ones does not.

Breno