[RFC PATCH 0/7] scripts: convert remaining Perl utilities in scripts/

[RFC PATCH 0/7] scripts: convert remaining Perl utilities in scripts/

[Breno Rodrigues Alves]

This series converts the last seven Perl utilities in
scripts/ to Python 3. After this series, scripts/ contains
no Perl scripts left: all 17 utilities have been converted
(12 in the previous series, 7 more here).

Each conversion is a faithful 1:1 port: same CLI, same output,
same behavior. All use only the Python standard library.

The seven scripts are:
  - checkstack.pl             -> checkstack.py
  - extract-module-sig.pl     -> extract-module-sig.py
  - extract-sys-certs.pl      -> extract-sys-certs.py
  - headerdep.pl              -> headerdep.py
  - kconfig-sym-check.pl      -> kconfig-sym-check.py
  - parse-maintainers.pl      -> parse-maintainers.py
  - streamline_config.pl      -> streamline_config.py

Five of the seven were verified byte-for-byte identical to
their Perl counterparts. The remaining two (extract-sys-certs,
streamline_config) depend on build artifacts (vmlinux, .config)
and are 1:1 ports of the Perl logic.

Breno Rodrigues Alves (7):
  scripts: convert checkstack.pl to Python
  scripts: convert extract-module-sig.pl to Python
  scripts: convert extract-sys-certs.pl to Python
  scripts: convert headerdep.pl to Python
  scripts: convert kconfig-sym-check.pl to Python
  scripts: convert parse-maintainers.pl to Python
  scripts: convert streamline_config.pl to Python

Breno Rodrigues Alves (7):
  scripts: convert checkstack.pl to Python
  scripts: convert extract-module-sig.pl to Python
  scripts: convert extract-sys-certs.pl to Python
  scripts: convert headerdep.pl to Python
  scripts: convert kconfig-sym-check.pl to Python
  scripts: convert parse-maintainers.pl to Python
  scripts: convert streamline_config.pl to Python

 scripts/checkstack.py         | 143 ++++++++++
 scripts/extract-module-sig.py | 103 ++++++++
 scripts/extract-sys-certs.py  | 132 ++++++++++
 scripts/headerdep.py          | 137 ++++++++++
 scripts/kconfig-sym-check.py  | 177 +++++++++++++
 scripts/parse-maintainers.py  | 136 ++++++++++
 scripts/streamline_config.py  | 475 ++++++++++++++++++++++++++++++++++
 7 files changed, 1303 insertions(+)
 create mode 100755 scripts/checkstack.py
 create mode 100755 scripts/extract-module-sig.py
 create mode 100755 scripts/extract-sys-certs.py
 create mode 100755 scripts/headerdep.py
 create mode 100755 scripts/kconfig-sym-check.py
 create mode 100755 scripts/parse-maintainers.py
 create mode 100755 scripts/streamline_config.py

-- 
2.43.0

[RFC PATCH 1/7] scripts: convert checkstack.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of checkstack.pl in Python 3
(stdlib only). Same per-architecture regexes for x86/x86_64,
arm, arm64, m68k, mips, nios2, openrisc, parisc, powerpc,
s390, sparc, riscv and loongarch.

Verified byte-for-byte identical to the Perl version on a
synthetic objdump stream for x86_64.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/checkstack.py | 143 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 143 insertions(+)
 create mode 100755 scripts/checkstack.py

diff --git a/scripts/checkstack.py b/scripts/checkstack.py
new file mode 100755
index 000000000..8be3861b3
--- /dev/null
+++ b/scripts/checkstack.py
@@ -0,0 +1,143 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# Check the stack usage of functions.
+# Usage: objdump -d vmlinux | checkstack.py [arch] [min_stack]
+
+import platform
+import re
+import sys
+
+X = '[0-9a-f]'
+RE_FUNC = re.compile(rf'^({X}*) <(.*)>:$')
+RE_FILE = re.compile(r'(.*):\s*file format')
+
+
+def arm_push_handler(line):
+    if re.search(r'.*push.*fp, ip, lr, pc}', line):
+        return (line.count(',') + 1) * 4
+    return 0
+
+
+def config_for(arch):
+    """Return (main_re, dynamic_re, sub_fn)."""
+    if re.match(r'^(?:aarch|arm)64$', arch):
+        return (
+            re.compile(rf'^.*stp.*sp, ?#-([0-9]{{1,8}})\]!'),
+            re.compile(rf'^.*sub.*sp, sp, #(0x{X}{{1,8}})'),
+            None)
+    if arch == 'arm':
+        return (re.compile(r'.*sub.*sp, sp, #([0-9]{1,4})'), None,
+                arm_push_handler)
+    if re.match(r'^x86(?:_64)?$', arch) or re.match(r'^i[3456]86$', arch):
+        return (
+            re.compile(rf'^.*[as][du][db]    \$(0x{X}{{1,8}}),%(?:e|r)sp$'),
+            re.compile(r'^.*[as][du][db]    (%.*),%(?:e|r)sp$'),
+            None)
+    if arch == 'm68k':
+        return (re.compile(r'.*(?:linkw %fp,|addaw )#-([0-9]{1,4})'
+                           r'(?:,%sp)?$'), None, None)
+    if arch == 'mips64':
+        return (re.compile(r'.*daddiu.*sp,sp,-([0-9]{1,8})'), None, None)
+    if arch == 'mips':
+        return (re.compile(r'.*addiu.*sp,sp,-([0-9]{1,8})'), None, None)
+    if arch == 'nios2':
+        return (re.compile(r'.*addi.*sp,sp,-([0-9]{1,8})'), None, None)
+    if arch == 'openrisc':
+        return (re.compile(r'.*l\.addi.*r1,r1,-([0-9]{1,8})'), None, None)
+    if arch in ('parisc', 'parisc64'):
+        return (re.compile(rf'.*ldo ({X}{{1,8}})\(sp\),sp'), None, None)
+    if arch == 'powerpc' or re.match(r'^ppc(?:64)?(?:le)?$', arch):
+        return (re.compile(rf'.*st[dw]u.*r1,-({X}{{1,8}})\(r1\)'),
+                None, None)
+    if re.match(r'^s390x?$', arch):
+        return (re.compile(r'.*(?:lay|ag?hi).*\%r15,-([0-9]+)'
+                           r'(?:\(\%r15\))?$'), None, None)
+    if arch in ('sparc', 'sparc64'):
+        return (re.compile(r'.*save.*%sp, -([0-9]{1,8}), %sp'), None, None)
+    if re.match(r'^riscv(?:64)?$', arch):
+        return (re.compile(r'.*addi.*sp,sp,-([0-9]{1,8})'), None, None)
+    if re.match(r'^loongarch(?:32|64)?$', arch):
+        return (re.compile(r'.*addi\..*sp, .*sp, -([0-9]{1,8}).*'),
+                None, None)
+    print(f'wrong or unknown architecture "{arch}"')
+    sys.exit(1)
+
+
+def parse_size(s):
+    return int(s, 16) if s.startswith('0x') else int(s)
+
+
+def main(argv):
+    arch = argv[0] if argv else platform.machine()
+    min_stack = int(argv[1]) if len(argv) > 1 and argv[1].isdigit() else 512
+
+    main_re, dyn_re, sub_fn = config_for(arch)
+
+    stack = []
+    func = None
+    file = ''
+    intro = ''
+    total = 0
+
+    for raw in sys.stdin:
+        line = raw.rstrip('\n')
+
+        m = RE_FUNC.match(line)
+        if m:
+            func = m.group(2)
+            if total > min_stack:
+                stack.append(f'{intro}{total}\n')
+            addr = m.group(1)
+            intro = f'0x{addr} {func} [{file}]:'
+            pad = 56 - len(intro)
+            while pad > 0:
+                intro += '\t'
+                pad -= 8
+            total = 0
+            continue
+
+        m = RE_FILE.match(line)
+        if m:
+            file = m.group(1).replace('.ko', '')
+            idx = file.rfind('/')
+            if idx != -1:
+                file = file[idx + 1:]
+            continue
+
+        m = main_re.search(line)
+        if m:
+            size = parse_size(m.group(1))
+            if size > 0xf0000000:
+                size = -size + 0x80000000 + 0x80000000
+            if size > 0x10000000:
+                continue
+            total += size
+            continue
+
+        if dyn_re is not None:
+            m = dyn_re.search(line)
+            if m:
+                total += parse_size(m.group(1))
+                continue
+
+        if sub_fn is not None:
+            total += sub_fn(line)
+
+    if total > min_stack:
+        stack.append(f'{intro}{total}\n')
+
+    def key(entry):
+        m = re.search(r':\t*(\d+)$', entry)
+        size = int(m.group(1)) if m else 0
+        m = re.search(r' (.*):', entry)
+        fname = m.group(1) if m else ''
+        return (-size, fname)
+
+    for entry in sorted(stack, key=key):
+        sys.stdout.write(entry)
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH 2/7] scripts: convert extract-module-sig.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of extract-module-sig.pl in Python 3
(stdlib only). Same magic-number handling, same descriptor
extraction, same -0/-a/-d/-n/-k/-s output modes.

Verified byte-for-byte identical to the Perl version on a
synthetic signed module.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/extract-module-sig.py | 103 ++++++++++++++++++++++++++++++++++
 1 file changed, 103 insertions(+)
 create mode 100755 scripts/extract-module-sig.py

diff --git a/scripts/extract-module-sig.py b/scripts/extract-module-sig.py
new file mode 100755
index 000000000..e9a43e72f
--- /dev/null
+++ b/scripts/extract-module-sig.py
@@ -0,0 +1,103 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# extract-module-sig.py <part> <module-file>
+#   -0: unsigned module  -a: all sig data  -d: descriptor
+#   -n: signer name      -k: key ID        -s: signature
+
+import struct
+import sys
+
+MAGIC = b'~Module signature appended~\n'
+
+
+def main(argv):
+    if len(argv) != 2:
+        sys.exit(f'Format: {sys.argv[0]} -[0adnks] module-file >out')
+
+    part, modfile = argv
+
+    try:
+        with open(modfile, 'rb') as f:
+            buf = f.read()
+    except OSError as e:
+        sys.exit(f'{modfile}: {e}')
+
+    print(f'Read {len(buf)} bytes from module file', file=sys.stderr)
+
+    if len(buf) < 12 + len(MAGIC):
+        sys.exit('The file is too short to have a sig magic number and descriptor')
+
+    p = len(buf) - len(MAGIC)
+    if buf[p:] != MAGIC:
+        sys.exit(f'Magic number not found at {len(buf)}')
+    print(f'Found magic number at {len(buf)}', file=sys.stderr)
+
+    p -= 12
+    info_raw = buf[p:p + 12]
+    algo = info_raw[0]
+    hash_ = info_raw[1]
+    id_type = info_raw[2]
+    name_len = info_raw[3]
+    kid_len = info_raw[4]
+    sig_len = struct.unpack('>I', info_raw[8:12])[0]
+    info = [algo, hash_, id_type, name_len, kid_len, sig_len]
+
+    type_msg = {
+        0: 'Found PGP key identifier',
+        1: 'Found X.509 cert identifier',
+        2: 'Found PKCS#7/CMS encapsulation',
+    }.get(id_type, f'Found unsupported identifier type {id_type}')
+    print(type_msg, file=sys.stderr)
+
+    if p < name_len + kid_len + sig_len:
+        sys.exit('Insufficient name+kid+sig data in file')
+
+    p -= sig_len
+    raw_sig = buf[p:p + sig_len]
+    p -= kid_len
+    raw_kid = buf[p:p + kid_len]
+    p -= name_len
+    raw_name = buf[p:p + name_len]
+    module_len = p
+
+    if sig_len > 0:
+        n = min(16, sig_len)
+        hs = ''.join(f'{b:02x}' for b in raw_sig[:n])
+        print(f'Found {sig_len} bytes of signature [{hs}]', file=sys.stderr)
+
+    if kid_len > 0:
+        n = min(16, kid_len)
+        hs = ''.join(f'{b:02x}' for b in raw_kid[:n])
+        print(f'Found {kid_len} bytes of key identifier [{hs}]',
+              file=sys.stderr)
+
+    if name_len > 0:
+        sys.stderr.buffer.write(
+            f"Found {name_len} bytes of signer's name [".encode() +
+            raw_name + b']\n')
+
+    if part == '-0':
+        sys.stdout.buffer.write(buf[:module_len])
+    elif part == '-a':
+        sys.stdout.buffer.write(buf[module_len:])
+    elif part == '-d':
+        print(' '.join(str(x) for x in info))
+    elif part == '-n':
+        if id_type == 2:
+            print("No signer's name for PKCS#7 message type sig",
+                  file=sys.stderr)
+        sys.stdout.buffer.write(raw_name)
+    elif part == '-k':
+        if id_type == 2:
+            print('No key ID for PKCS#7 message type sig', file=sys.stderr)
+        sys.stdout.buffer.write(raw_kid)
+    elif part == '-s':
+        sys.stdout.buffer.write(raw_sig)
+    else:
+        sys.exit(f'Unknown part: {part}')
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH 3/7] scripts: convert extract-sys-certs.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of extract-sys-certs.pl in Python 3
(stdlib only). Same vmlinux section-table parsing, same symbol
lookup (nm with System.map fallback), same VMA-to-file-offset
translation.

Requires a vmlinux with the system certificate list to fully
test; logic is 1:1 with the Perl version.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/extract-sys-certs.py | 132 +++++++++++++++++++++++++++++++++++
 1 file changed, 132 insertions(+)
 create mode 100755 scripts/extract-sys-certs.py

diff --git a/scripts/extract-sys-certs.py b/scripts/extract-sys-certs.py
new file mode 100755
index 000000000..92a414eb3
--- /dev/null
+++ b/scripts/extract-sys-certs.py
@@ -0,0 +1,132 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# extract-sys-certs.py [-s <systemmap-file>] <vmlinux-file> <keyring-file>
+
+import re
+import subprocess
+import sys
+
+RE_SECTION = re.compile(
+    r'\s*([0-9]+)\s+(\S+)\s+([0-9a-f]+)\s+([0-9a-f]+)\s+'
+    r'([0-9a-f]+)\s+([0-9a-f]+)\s+2\*\*([0-9]+)')
+RE_SYMBOL = re.compile(r'([0-9a-f]+)\s([a-zA-Z])\s(\S+)')
+
+
+def parse_symbols(lines):
+    syms = {}
+    n = 0
+    for line in lines:
+        m = RE_SYMBOL.search(line)
+        if m:
+            syms[m.group(3)] = int(m.group(1), 16)
+            n += 1
+    return syms, n
+
+
+def main(argv):
+    args = list(argv)
+    sysmap = ''
+    if len(args) == 4 and args[0] == '-s':
+        sysmap = args[1]
+        args = args[2:]
+    elif len(args) != 2:
+        sys.exit(f'Format: {sys.argv[0]} [-s <systemmap-file>] '
+                 f'<vmlinux-file> <keyring-file>')
+
+    vmlinux, keyring = args
+
+    try:
+        objdump_out = subprocess.check_output(
+            ['objdump', '-h', vmlinux], text=True)
+    except (subprocess.CalledProcessError, FileNotFoundError) as e:
+        sys.exit(f'{vmlinux}: {e}')
+
+    sections = []
+    for line in objdump_out.splitlines():
+        m = RE_SECTION.search(line)
+        if not m:
+            continue
+        sections.append({
+            'name': m.group(2),
+            'len': int(m.group(3), 16),
+            'vma': int(m.group(4), 16),
+            'foff': int(m.group(6), 16),
+        })
+
+    print(f'Have {len(sections)} sections')
+
+    try:
+        nm_out = subprocess.check_output(
+            ['nm', vmlinux], text=True, stderr=subprocess.DEVNULL)
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        nm_out = ''
+
+    symbols, nr = parse_symbols(nm_out.splitlines())
+
+    if nr == 0 and sysmap:
+        print(f'No symbols in vmlinux, trying {sysmap}')
+        with open(sysmap) as f:
+            symbols, nr = parse_symbols(f.readlines())
+
+    if nr == 0:
+        sys.exit('No symbols available')
+    print(f'Have {nr} symbols')
+
+    if '__cert_list_start' not in symbols or \
+            'system_certificate_list_size' not in symbols:
+        sys.exit("Can't find system certificate list")
+
+    start = symbols['__cert_list_start']
+    size_sym = symbols['system_certificate_list_size']
+
+    found = None
+    size = 0
+    buf = b''
+    with open(vmlinux, 'rb') as f:
+        for sec in sections:
+            s_vma = sec['vma']
+            s_len = sec['len']
+            s_vend = s_vma + s_len
+
+            if start < s_vma or start >= s_vend:
+                continue
+
+            if size_sym < s_vma or size_sym > s_vend:
+                sys.exit('Certificate list size was not found on the same section')
+
+            if found is not None:
+                sys.exit(f"Cert object in multiple sections: "
+                         f"{sec['name']} and {found['name']}")
+
+            size_off = size_sym - s_vma + sec['foff']
+            f.seek(size_off)
+            packed = f.read(8)
+            if len(packed) != 8:
+                sys.exit(vmlinux)
+            size = int.from_bytes(packed, sys.byteorder)
+            end = start + size
+
+            print(f'Have {size} bytes of certs at VMA 0x{start:x}')
+            if end > s_vend:
+                sys.exit(f"Cert object partially overflows section {sec['name']}")
+            found = sec
+
+        if found is None:
+            sys.exit('Cert object not inside a section')
+
+        print(f"Certificate list in section {found['name']}")
+        foff = start - found['vma'] + found['foff']
+        print(f'Certificate list at file offset 0x{foff:x}')
+        f.seek(foff)
+        buf = f.read(size)
+        if len(buf) != size:
+            sys.exit(f'Short read on {vmlinux}')
+
+    with open(keyring, 'wb') as f:
+        f.write(buf)
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH 4/7] scripts: convert headerdep.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of headerdep.pl in Python 3
(stdlib only). Same include graph construction, same cycle
detection, same GraphViz output mode (--graph).

Verified byte-for-byte identical to the Perl version on a
synthetic cyclic include chain.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/headerdep.py | 137 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 137 insertions(+)
 create mode 100755 scripts/headerdep.py

diff --git a/scripts/headerdep.py b/scripts/headerdep.py
new file mode 100755
index 000000000..4b9a57ab1
--- /dev/null
+++ b/scripts/headerdep.py
@@ -0,0 +1,137 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# Detect cycles in the header file dependency graph.
+# Vegard Nossum <vegardno@ifi.uio.no>
+
+import argparse
+import os
+import re
+import sys
+
+deps = {}
+opt_include = []
+
+
+def strip(filename):
+    for inc in opt_include:
+        prefix = inc + '/'
+        if filename.startswith(prefix):
+            return filename[len(prefix):]
+    return filename
+
+
+def search(filename):
+    if os.path.isfile(filename):
+        return filename
+    for inc in opt_include:
+        path = f'{inc}/{filename}'
+        if os.path.isfile(path):
+            return path
+    return None
+
+
+def parse_all(headers):
+    queue = list(headers)
+    while queue:
+        h = queue.pop()
+        if h in deps:
+            continue
+        deps[h] = []
+        path = search(h)
+        if path is None:
+            continue
+        with open(path) as f:
+            lines = f.readlines()
+        for i, line in enumerate(lines):
+            m = re.match(r'^#\s*include\s*<(.*?)>', line)
+            if m:
+                dep = m.group(1)
+                queue.append(dep)
+                deps[h].append((i + 1, dep))
+
+
+def print_cycle(cycle):
+    # Shift line numbers to align with the "In file included" message
+    cycle = list(cycle)
+    for i in range(len(cycle) - 1):
+        cycle[i] = (cycle[i + 1][0], cycle[i][1])
+    cycle[-1] = (0, cycle[-1][1])
+
+    first = cycle.pop(0)
+    last = cycle.pop() if cycle else None
+
+    msg = 'In file included'
+    if last is not None:
+        print(f'{msg} from {last[1]},')
+    for hdr in reversed(cycle):
+        marker = ' <-- here' if last is not None and hdr[1] == last[1] else ''
+        print(f'{" " * len(msg)} from {hdr[1]}:{hdr[0]}{marker}')
+    print(f'{first[1]}:{first[0]}: warning: recursive header inclusion')
+
+
+def detect_cycles(headers, opt_all):
+    queue = [[(0, h)] for h in headers]
+    while queue:
+        top = queue.pop()
+        name = top[-1][1]
+        for dep in deps.get(name, []):
+            chain = top + [(dep[0], dep[1])]
+            if any(c[1] == dep[1] for c in top):
+                print_cycle(chain)
+                if not opt_all:
+                    return
+                continue
+            queue.append(chain)
+
+
+def mangle(name):
+    return name.replace('/', '__').replace('.', '_').replace('-', '_')
+
+
+def graph():
+    print('digraph {')
+    print()
+    print('\t/* vertices */')
+    for h in deps:
+        print(f'\t{mangle(h)} [label="{h}"];')
+    print()
+    print('\t/* edges */')
+    for h in deps:
+        for dep in deps[h]:
+            print(f'\t{mangle(h)} -> {mangle(dep[1])};')
+    print('}')
+
+
+def main(argv):
+    global opt_include
+
+    p = argparse.ArgumentParser(add_help=False)
+    p.add_argument('--all', action='store_true')
+    p.add_argument('--graph', action='store_true')
+    p.add_argument('-I', dest='includes', action='append', default=[])
+    p.add_argument('files', nargs='*')
+    p.add_argument('-h', '--help', '--version', action='store_true')
+    args = p.parse_args(argv)
+
+    if args.help:
+        print(f'Usage: {sys.argv[0]} [options] file...')
+        return 0
+
+    opt_include = list(args.includes) + ['include']
+    # Match Perl: grep { strip($_) } keeps the ORIGINAL path, using
+    # strip() only as a truthiness filter. This makes deps keys
+    # consistent with the Perl version (e.g. 'include/x.h', not 'x.h').
+    headers = [h for h in args.files if strip(h)]
+
+    parse_all(headers)
+
+    if args.graph:
+        graph()
+    else:
+        detect_cycles(headers, args.all)
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH 5/7] scripts: convert kconfig-sym-check.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of kconfig-sym-check.pl in Python 3
(stdlib only). Same Kconfig parsing, same symbol-reference
detection, same excludes-file handling, same exit code.

Verified byte-for-byte identical to the Perl version on the
full kernel tree (~/linux).

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/kconfig-sym-check.py | 177 +++++++++++++++++++++++++++++++++++
 1 file changed, 177 insertions(+)
 create mode 100755 scripts/kconfig-sym-check.py

diff --git a/scripts/kconfig-sym-check.py b/scripts/kconfig-sym-check.py
new file mode 100755
index 000000000..f15b25535
--- /dev/null
+++ b/scripts/kconfig-sym-check.py
@@ -0,0 +1,177 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# Detect unknown symbols referenced in Kconfig files.
+# Usage: kconfig-sym-check.py <srctree> [excludes file]
+
+import os
+import re
+import subprocess
+import sys
+
+VALID = re.compile(r'^[a-zA-Z0-9_]+$')
+RE_DEF = re.compile(r'^\s*(?:config|menuconfig)\s+([a-zA-Z0-9_]+)\s*(?:#.*)?$')
+RE_HELP = re.compile(r'^\s*(?:help|---help---)$')
+RE_REF = re.compile(
+    r'^\s*(?:default|def_bool|def_tristate|select|depends\s+on|imply|'
+    r'visible\s+if|range|if|bool|tristate|int|hex|string|prompt)\s+(.+)\s*$')
+RE_STRING = re.compile(r'"(?:[^"\\]|\\.)*"|\'(?:[^\'\\]|\\.)*\'')
+RE_SKIP = re.compile(r'^\s*(?:#.*)?$')
+
+
+def indent_depth(ws):
+    col = 0
+    for c in ws:
+        if c == '\t':
+            col = (col // 8) * 8 + 8
+        else:
+            col += 1
+    return col
+
+
+def find_kconfig_files(srctree):
+    try:
+        out = subprocess.check_output(
+            ['git', '-C', srctree, 'ls-files', '*Kconfig*'],
+            text=True, stderr=subprocess.DEVNULL)
+        files = [ln.strip() for ln in out.splitlines() if ln.strip()]
+        if files:
+            return [os.path.join(srctree, f) for f in files]
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        pass
+    try:
+        out = subprocess.check_output(
+            ['find', srctree, '-name', '*Kconfig*'], text=True)
+        return [ln.strip() for ln in out.splitlines() if ln.strip()]
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        return []
+
+
+def parse_kconfig(path, configs, refs):
+    with open(path) as f:
+        raw = f.readlines()
+
+    merged = []
+    i = 0
+    while i < len(raw):
+        line = raw[i].rstrip('\n')
+        while line.endswith('\\') and i + 1 < len(raw):
+            line = line[:-1] + ' ' + raw[i + 1].rstrip('\n')
+            i += 1
+        merged.append(line)
+        i += 1
+
+    help_mode = False
+    help_level = 0
+    i = 0
+    while i < len(merged):
+        line = merged[i]
+
+        if not line.strip():
+            i += 1
+            continue
+        if line.lstrip().startswith('#'):
+            i += 1
+            continue
+
+        level = indent_depth(re.match(r'^(\s*)', line).group(1))
+
+        if help_mode and level < help_level:
+            help_mode = False
+        if help_mode:
+            i += 1
+            continue
+
+        if RE_HELP.match(line):
+            help_mode = True
+            j = i + 1
+            while j < len(merged) and RE_SKIP.match(merged[j]):
+                j += 1
+            if j >= len(merged):
+                break
+            next_level = indent_depth(
+                re.match(r'^(\s*)', merged[j]).group(1))
+            if next_level >= level:
+                help_level = next_level
+            else:
+                help_mode = False
+            i = j
+            continue
+
+        m = RE_DEF.match(line)
+        if m:
+            configs[m.group(1)] = configs.get(m.group(1), 0) + 1
+            i += 1
+            continue
+
+        m = RE_REF.match(line)
+        if m:
+            s = m.group(1)
+            s = RE_STRING.sub('', s)
+            s = re.sub(r'#.*', '', s)
+            s = re.sub(r'\$\((?:[^()]*|\((?:[^()]*|\([^()]*\))*\))*\)', '', s)
+            s = re.sub(r'%%[^%]*%%', '', s)
+            for sym in re.split(r'[^a-zA-Z0-9_]+', s):
+                if not sym or not sym[0].isalpha():
+                    continue
+                if sym in ('if', 'y', 'n', 'm'):
+                    continue
+                if re.match(r'^0[xX][0-9a-fA-F]+$', sym):
+                    continue
+                refs[sym] = refs.get(sym, 0) + 1
+        i += 1
+
+
+def main(argv):
+    if not argv:
+        s = ''
+        try:
+            s = subprocess.check_output(
+                ['git', 'rev-parse', '--show-toplevel'],
+                text=True, stderr=subprocess.DEVNULL).strip()
+        except (subprocess.CalledProcessError, FileNotFoundError):
+            pass
+        msg = f'Usage: {sys.argv[0]} <srctree> [excludes file]\n'
+        msg += 'Please provide <srctree>.'
+        if s:
+            msg += f" Is it '{s}'?"
+        sys.exit(msg + '\n')
+
+    srctree = argv[0]
+    excludes = argv[1] if len(argv) > 1 else None
+
+    files = [f for f in find_kconfig_files(srctree)
+             if '/scripts/kconfig/tests/' not in f]
+
+    configs = {}
+    refs = {}
+    for f in files:
+        parse_kconfig(f, configs, refs)
+
+    known = {}
+    if excludes:
+        with open(excludes) as f:
+            for line in f:
+                line = line.rstrip('\n')
+                if not line.strip() or line.lstrip().startswith('#'):
+                    continue
+                m = re.match(r'^\s*([a-zA-Z0-9_]+)\s*(?:#.*)?$', line)
+                if m:
+                    known[m.group(1)] = 1
+
+    ret = 0
+    for k in sorted(refs):
+        if k in configs or k in known:
+            continue
+        print(k, end='')
+        if k in ('N', 'Y', 'M'):
+            print(f" - warning: '{k}' is probably not what you want; "
+                  f"Kconfig tristate literals are always lowercase "
+                  f"('n', 'y', 'm')", end='')
+        print()
+        ret = 1
+    return ret
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH 6/7] scripts: convert parse-maintainers.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of parse-maintainers.pl in Python 3
(stdlib only). Same section parsing, same --order preferred
ordering, same --input/--output/--section behavior.

Verified byte-for-byte identical to the Perl version on the
kernel MAINTAINERS file (30,222 lines).

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/parse-maintainers.py | 136 +++++++++++++++++++++++++++++++++++
 1 file changed, 136 insertions(+)
 create mode 100755 scripts/parse-maintainers.py

diff --git a/scripts/parse-maintainers.py b/scripts/parse-maintainers.py
new file mode 100755
index 000000000..520f66ab8
--- /dev/null
+++ b/scripts/parse-maintainers.py
@@ -0,0 +1,136 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# Sort MAINTAINERS file sections alphabetically.
+
+import argparse
+import re
+import sys
+from functools import cmp_to_key
+
+PREFERRED_ORDER = 'MRLSWQBCPTFXNK'
+
+
+def by_category_cmp(a, b):
+    aa = a.upper().replace('THE REST', 'ZZZZZZ')
+    bb = b.upper().replace('THE REST', 'ZZZZZZ')
+    return (aa > bb) - (aa < bb)
+
+
+def by_pattern_cmp(a, b):
+    a1 = a[0].upper() if a else ''
+    b1 = b[0].upper() if b else ''
+    ai = PREFERRED_ORDER.find(a1)
+    bi = PREFERRED_ORDER.find(b1)
+    if ai < 0:
+        ai = 1000
+    if bi < 0:
+        bi = 1000
+    if (a1 == 'F' and b1 == 'F') or (a1 == 'X' and b1 == 'X'):
+        return (a > b) - (a < b)
+    if ai < bi:
+        return -1
+    if ai == bi:
+        return 0
+    return 1
+
+
+def trim(s):
+    return s.strip()
+
+
+def alpha_output(ref, filename, order):
+    if not ref:
+        return
+    with open(filename, 'w') as f:
+        sep = None
+        for key in sorted(ref.keys(), key=cmp_to_key(by_category_cmp)):
+            if key == ' ':
+                f.write(ref[key])
+            else:
+                if sep is None:
+                    sep = '\n'
+                else:
+                    f.write(sep)
+                f.write(key + '\n')
+                patterns = ref[key].split('\n')
+                if order:
+                    patterns = sorted(patterns, key=cmp_to_key(by_pattern_cmp))
+                for p in patterns:
+                    if p:
+                        f.write(p + '\n')
+
+
+def file_input(ref, filename):
+    lastline = ''
+    case = ' '
+    ref[case] = ''
+
+    with open(filename) as f:
+        lines = f.readlines()
+
+    for line in lines:
+        m = re.match(r'^([A-Z]):\s*(.*)', line)
+        if m:
+            line = m.group(1) + ':\t' + trim(m.group(2)) + '\n'
+            if lastline == '':
+                ref[case] += line
+                continue
+            case = trim(lastline)
+            if case in ref:
+                sys.exit(f"Header '{case}' already exists")
+            ref[case] = line
+            lastline = ''
+            continue
+
+        if case == ' ':
+            ref[case] += lastline
+            lastline = line
+            continue
+        if trim(lastline) != '':
+            sys.exit(f"Odd non-pattern line '{lastline}' for '{case}'")
+        lastline = line
+
+    ref[case] += lastline
+
+
+def main(argv):
+    p = argparse.ArgumentParser(add_help=False)
+    p.add_argument('--input', default='MAINTAINERS')
+    p.add_argument('--output', default='MAINTAINERS.new')
+    p.add_argument('--section', default='SECTION.new')
+    p.add_argument('--order', action='store_true', default=False)
+    p.add_argument('-h', '--help', '--usage', action='store_true')
+    p.add_argument('patterns', nargs='*')
+    args = p.parse_args(argv)
+
+    if args.help:
+        print_usage()
+        return 0
+
+    hash_ = {}
+    file_input(hash_, args.input)
+
+    new_hash = {}
+    for pattern in args.patterns:
+        for key in list(hash_.keys()):
+            if re.search(pattern, key) or re.search(pattern, hash_[key]):
+                new_hash[key] = hash_[key]
+                del hash_[key]
+
+    alpha_output(hash_, args.output, args.order)
+    alpha_output(new_hash, args.section, args.order)
+    return 0
+
+
+def print_usage():
+    print(f'''usage: {sys.argv[0]} [options] <pattern matching regexes>
+
+  --input => MAINTAINERS file to read (default: MAINTAINERS)
+  --output => sorted MAINTAINERS file to write (default: MAINTAINERS.new)
+  --section => new sorted MAINTAINERS file to write (default: SECTION.new)
+  --order => Use the preferred section content output ordering (default: 0)''')
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH 7/7] scripts: convert streamline_config.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of streamline_config.pl in Python 3
(stdlib only). Same Kconfig recursion, same Makefile parsing,
same dependency/select resolution, same LMC_KEEP preservation.

Requires a .config and loaded modules to fully test; logic is
1:1 with the Perl version.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/streamline_config.py | 475 +++++++++++++++++++++++++++++++++++
 1 file changed, 475 insertions(+)
 create mode 100755 scripts/streamline_config.py

diff --git a/scripts/streamline_config.py b/scripts/streamline_config.py
new file mode 100755
index 000000000..f2e53975f
--- /dev/null
+++ b/scripts/streamline_config.py
@@ -0,0 +1,475 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright 2005-2009 - Steven Rostedt
+#
+# Simplify the kernel config to only the modules loaded on this system.
+
+import os
+import re
+import subprocess
+import sys
+
+DEBUG = 'LOCALMODCONFIG_DEBUG' in os.environ
+
+# Module state populated by read_kconfig()
+depends = {}
+selects = {}
+prompts = {}
+objects = {}
+config2kfile = {}
+defaults = {}
+read_kconfigs = {}
+ksource = '.'
+
+
+def dprint(*args):
+    if DEBUG:
+        print(*args, file=sys.stderr)
+
+
+def read_config():
+    uname = subprocess.check_output(['uname', '-r'], text=True).strip()
+
+    searchconfigs = [
+        ('.config', 'cat', None),
+        ('/proc/config.gz', 'zcat', None),
+        (f'/boot/config-{uname}', 'cat', None),
+        (f'/boot/vmlinuz-{uname}', 'scripts/extract-ikconfig',
+         'scripts/extract-ikconfig'),
+        ('vmlinux', 'scripts/extract-ikconfig', 'scripts/extract-ikconfig'),
+        (f'/lib/modules/{uname}/kernel/kernel/configs.ko',
+         'scripts/extract-ikconfig', 'scripts/extract-ikconfig'),
+        ('kernel/configs.ko', 'scripts/extract-ikconfig',
+         'scripts/extract-ikconfig'),
+        ('kernel/configs.o', 'scripts/extract-ikconfig',
+         'scripts/extract-ikconfig'),
+    ]
+
+    for file, exec_cmd, test_cmd in searchconfigs:
+        if not os.path.isfile(file):
+            continue
+        if test_cmd is not None:
+            r = subprocess.run([test_cmd, file], capture_output=True)
+            if r.returncode:
+                continue
+        print(f"using config: '{file}'", file=sys.stderr)
+        try:
+            out = subprocess.check_output(
+                f'{exec_cmd} {file}', shell=True, text=True,
+                stderr=subprocess.DEVNULL)
+        except subprocess.CalledProcessError:
+            continue
+        return out.splitlines(keepends=True)
+
+    sys.exit('No config file found')
+
+
+def read_kconfig(kconfig, if_state):
+    """Recursively read a Kconfig file, populate globals.
+
+    if_state is a [level, [deps...]] list for mutation.
+    """
+    state = 'NONE'
+    config = ''
+    cont = False
+    line = ''
+
+    source = f'{ksource}/{kconfig}'
+    last_source = ''
+    while re.search(r'\$\((\w+)\)', source) and source != last_source:
+        last_source = source
+        m = re.search(r'\$\((\w+)\)', source)
+        source = source.replace(m.group(0), os.environ.get(m.group(1), ''))
+
+    try:
+        with open(source) as f:
+            raw_lines = f.readlines()
+    except OSError:
+        sys.exit(f"Can't open {source}")
+
+    iflevel = if_state[0]
+    ifdeps = if_state[1]
+
+    for raw in raw_lines:
+        line_text = raw.rstrip('\n')
+
+        if cont:
+            line_text = line + ' ' + line_text
+
+        if line_text.endswith('\\'):
+            cont = True
+            line = line_text[:-1]
+            continue
+        cont = False
+
+        m = re.match(r'^source\s+"?([^"]+)', line_text)
+        if m:
+            sub = m.group(1)
+            if sub not in read_kconfigs:
+                read_kconfigs[sub] = 1
+                read_kconfig(sub, if_state)
+            continue
+
+        m = re.match(r'^\s*(?:menu)?config\s+(\S+)\s*$', line_text)
+        if m:
+            state = 'NEW'
+            config = m.group(1)
+            config2kfile[f'CONFIG_{config}'] = kconfig
+            for i in range(if_state[0]):
+                if i:
+                    depends[config] = depends.get(config, '') + ' ' + ifdeps[i]
+                else:
+                    depends[config] = ifdeps[i]
+                state = 'DEP'
+            continue
+
+        m = re.match(r'^\s*depends\s+on\s+(.*)$', line_text)
+        if m and state in ('NEW', 'DEP'):
+            if state == 'NEW':
+                depends[config] = m.group(1)
+                state = 'DEP'
+            else:
+                depends[config] = depends.get(config, '') + ' ' + m.group(1)
+            continue
+
+        m = re.match(r'^\s*def(?:_(?:bool|tristate)|ault)\s+(\S.*)$',
+                     line_text)
+        if m and state != 'NONE':
+            dep = m.group(1)
+            defaults[config] = 1
+            if not re.match(r'^\s*(?:y|m|n)\s*$', dep):
+                dep = re.sub(r'.*\sif\s+', '', dep)
+                depends[config] = depends.get(config, '') + ' ' + dep
+                dprint(f'Added default depends {dep} to {config}')
+            continue
+
+        m = re.match(r'^\s*select\s+(\S+)', line_text)
+        if m and state != 'NONE':
+            conf = m.group(1)
+            if conf in selects:
+                selects[conf] += ' ' + config
+            else:
+                selects[conf] = config
+            continue
+
+        if state != 'NONE' and \
+                re.match(r'^\s*(?:tristate\s+\S|prompt\b)', line_text):
+            prompts[config] = 1
+            continue
+
+        m = re.match(r'^if\s+(.*\S)\s*$', line_text)
+        if m:
+            deps_str = re.sub(r'^[^a-zA-Z0-9_]*', '', m.group(1))
+            deps_str = re.sub(r'[^a-zA-Z0-9_]*$', '', deps_str)
+            parts = re.split(r'[^a-zA-Z0-9_]+', deps_str)
+            if if_state[0] < len(ifdeps):
+                ifdeps[if_state[0]] = ':'.join(parts)
+            else:
+                ifdeps.append(':'.join(parts))
+            if_state[0] += 1
+            continue
+
+        if re.match(r'^endif', line_text):
+            if if_state[0]:
+                if_state[0] -= 1
+            continue
+
+        if re.match(r'^\s*(?:---)?help(?:---)?\s*$', line_text) or \
+                re.match(r'^(?:comment|choice|menu)\b', line_text):
+            state = 'NONE'
+
+
+def convert_vars(line, make_vars):
+    """Expand $(VAR) references in a Makefile line."""
+    out = ''
+    while True:
+        m = re.match(r'^(.*?)(\$\(([^()]+)\))(.*)$', line)
+        if not m:
+            out += line
+            break
+        out += m.group(1)
+        var = m.group(3)
+        if var in make_vars:
+            out += make_vars[var]
+        else:
+            out += m.group(2)
+        line = m.group(4)
+    return out
+
+
+def main(argv):
+    global ksource
+
+    config_file = read_config()
+
+    localmodconfig = False
+    localyesconfig = False
+    args = []
+    for a in argv:
+        if a == '--localmodconfig':
+            localmodconfig = True
+        elif a == '--localyesconfig':
+            localyesconfig = True
+        else:
+            args.append(a)
+
+    ksource = args[0] if args else '.'
+    kconfig = args[1] if len(args) > 1 else None
+    lsmod_file = os.environ.get('LSMOD')
+
+    try:
+        makefiles = subprocess.check_output(
+            ['find', ksource, '-name', 'Makefile', '-or', '-name', 'Kbuild'],
+            text=True, stderr=subprocess.DEVNULL).splitlines()
+    except subprocess.CalledProcessError:
+        makefiles = []
+
+    if kconfig:
+        read_kconfig(kconfig, [0, []])
+
+    # Parse Makefiles for obj-$(CONFIG_FOO) += obj.o
+    for makefile in makefiles:
+        line = ''
+        make_vars = {}
+        try:
+            with open(makefile) as f:
+                for raw in f:
+                    line_text = raw.rstrip('\n')
+                    if line_text.endswith('\\'):
+                        line += line_text[:-1]
+                        continue
+                    line += line_text
+                    expanded = convert_vars(line, make_vars)
+                    line = ''
+
+                    m = re.match(
+                        r'obj-\$[({](CONFIG_[^})]*)[)}]\s*[+:]?=\s*(.*)',
+                        expanded)
+                    objs = None
+                    var = None
+                    if m:
+                        var = m.group(1)
+                        objs = m.group(2)
+                    else:
+                        m = re.match(r'^\s*(\S+)\s*[:]?=\s*(.*\S)',
+                                     expanded)
+                        if m:
+                            make_vars[m.group(1)] = m.group(2)
+                    if objs is not None:
+                        for o in re.split(r'\s+', objs):
+                            o = o.replace('-', '_')
+                            m = re.match(r'(.*)\.o$', o)
+                            if m:
+                                objects.setdefault(m.group(1), []).append(var)
+        except OSError:
+            continue
+
+    # Get loaded modules
+    modules = {}
+    if lsmod_file:
+        if not os.path.isfile(lsmod_file):
+            alt = os.path.join(os.environ.get('objtree', ''), lsmod_file)
+            if os.path.isfile(alt):
+                lsmod_file = alt
+            else:
+                sys.exit(f'{lsmod_file} not found')
+        with open(lsmod_file) as f:
+            lines = f.readlines()
+    else:
+        lsmod = 'lsmod'
+        for d in ('/sbin', '/bin', '/usr/sbin', '/usr/bin'):
+            if os.path.isfile(f'{d}/lsmod') and os.access(
+                    f'{d}/lsmod', os.X_OK):
+                lsmod = f'{d}/lsmod'
+                break
+        try:
+            out = subprocess.check_output([lsmod], text=True,
+                                          stderr=subprocess.DEVNULL)
+            lines = out.splitlines(keepends=True)
+        except (subprocess.CalledProcessError, FileNotFoundError):
+            sys.exit(f'Can not call lsmod with {lsmod}')
+
+    for line in lines:
+        if line.startswith('Module'):
+            continue
+        m = re.match(r'^(\S+)', line)
+        if m:
+            modules[m.group(1)] = 1
+
+    # Build config set from objects
+    configs = {}
+    for mod in modules:
+        if mod in objects:
+            for conf in objects[mod]:
+                configs[conf] = mod
+                dprint(f'{conf} added by direct ({mod})')
+        else:
+            print(f'{mod} config not found!', file=sys.stderr)
+
+    # Read current configs
+    orig_configs = {}
+    for line in config_file:
+        m = re.search(r'(CONFIG_[A-Za-z_0-9]*)=(m|y)', line)
+        if m:
+            orig_configs[m.group(1)] = m.group(2)
+
+    valid = 'A-Za-z_0-9'
+
+    def parse_config_depends(p, depconfig):
+        nonlocal repeat
+        while re.search(rf'[{valid}]', p):
+            m = re.match(rf'^[^{valid}]*([{valid}]+)', p)
+            if not m:
+                sys.exit('parse_config_depends: unexpected')
+            conf = 'CONFIG_' + m.group(1)
+            p = re.sub(rf'^[^{valid}]*[{valid}]+', '', p, count=1)
+
+            if conf not in orig_configs or orig_configs[conf] == 'y':
+                continue
+            if conf not in configs:
+                repeat = 1
+                dprint(f'{conf} selected by depend {depconfig}')
+                configs[conf] = 1
+
+    def parse_config_selects(config, p):
+        nonlocal repeat
+        next_config = None
+        while re.search(rf'[{valid}]', p):
+            m = re.match(rf'^[^{valid}]*([{valid}]+)', p)
+            if not m:
+                sys.exit('parse_config_selects: unexpected')
+            conf = 'CONFIG_' + m.group(1)
+            p = re.sub(rf'^[^{valid}]*[{valid}]+', '', p, count=1)
+
+            if conf not in orig_configs:
+                dprint(f'{conf} not set for {config} select')
+                continue
+            if orig_configs[conf] != 'm':
+                dprint(f'{conf} (non module) selects {config}, we are good')
+                return
+            if conf in configs:
+                dprint(f'{conf} selects {config} so we are good')
+                return
+            if next_config is None:
+                next_config = conf
+
+        if next_config is None:
+            if config not in defaults:
+                print(f'WARNING: {config} is required, but nothing in the',
+                      file=sys.stderr)
+                print(f'  current config selects it.', file=sys.stderr)
+            return
+
+        repeat = 1
+        configs[next_config] = 1
+        dprint(f'{next_config} selected by select {config}')
+
+    repeat = 1
+    while repeat:
+        repeat = 0
+        process_selects = {}
+        for config in list(configs.keys()):
+            if config in orig_configs and orig_configs[config] != 'm':
+                continue
+            short = config.replace('CONFIG_', '', 1)
+            if short in depends:
+                parse_config_depends(depends[short], short)
+            if short not in prompts and short in selects:
+                process_selects[short] = 1
+
+        for config in process_selects:
+            dprint(f'Process select {config}')
+            parse_config_selects(config, selects[config])
+
+    # Preserve list
+    preserved = []
+    if 'LMC_KEEP' in os.environ:
+        preserved = os.environ['LMC_KEEP'].split(':')
+
+    def in_preserved(conf):
+        kconf = config2kfile.get(conf)
+        if kconf is None:
+            return False
+        for excl in preserved:
+            if re.match(f'^{excl}', kconf):
+                return True
+        return False
+
+    setconfigs = {}
+
+    for line in config_file:
+        # IKCONFIG special case
+        m = re.search(r'CONFIG_IKCONFIG', line)
+        if m:
+            if '# CONFIG_IKCONFIG is not set' in line:
+                print('CONFIG_IKCONFIG=m')
+                print('# CONFIG_IKCONFIG_PROC is not set')
+            else:
+                sys.stdout.write(line)
+            continue
+
+        m = re.match(r'CONFIG_MODULE_SIG_KEY="(.+)"', line)
+        if m:
+            orig_cert = m.group(1)
+            default_cert = 'certs/signing_key.pem'
+            if 'MODULE_SIG_KEY' not in depends or \
+                    f'"{default_cert}"' not in depends['MODULE_SIG_KEY']:
+                print(f'WARNING: MODULE_SIG_KEY assertion failure, '
+                      f'update needed to {sys.argv[0]}', file=sys.stderr)
+                sys.stdout.write(line)
+            elif orig_cert != default_cert and not os.path.isfile(orig_cert):
+                print(f'Module signature verification enabled but module '
+                      f'signing key "{orig_cert}" not found. Resetting '
+                      f'signing key to default value.', file=sys.stderr)
+                print(f'CONFIG_MODULE_SIG_KEY="{default_cert}"')
+            else:
+                sys.stdout.write(line)
+            continue
+
+        m = re.match(r'CONFIG_SYSTEM_TRUSTED_KEYS="(.+)"', line)
+        if m:
+            orig_keys = m.group(1)
+            if not os.path.isfile(orig_keys):
+                print(f'System keyring enabled but keys "{orig_keys}" not '
+                      f'found. Resetting keys to default value.',
+                      file=sys.stderr)
+                print('CONFIG_SYSTEM_TRUSTED_KEYS=""')
+            else:
+                sys.stdout.write(line)
+            continue
+
+        m = re.match(r'^(CONFIG.*)=(m|y)', line)
+        if m:
+            conf, val = m.group(1), m.group(2)
+            if in_preserved(conf):
+                dprint(f'Preserve config {conf}')
+                sys.stdout.write(line)
+                continue
+            if conf in configs:
+                if localyesconfig:
+                    setconfigs[conf] = 'y'
+                    print(f'{conf}=y')
+                    continue
+                setconfigs[conf] = val
+            elif val == 'm':
+                print(f'# {conf} is not set')
+                continue
+        sys.stdout.write(line)
+
+    # Integrity check
+    for mod in modules:
+        if mod not in objects:
+            continue
+        if not any(c in setconfigs for c in objects[mod]):
+            print(f'module {mod} did not have configs', end='',
+                  file=sys.stderr)
+            for c in objects[mod]:
+                print(f' {c}', end='', file=sys.stderr)
+            print(file=sys.stderr)
+
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH v2 0/7] scripts: convert remaining Perl utilities in scripts/

[Breno Rodrigues Alves]

v2 changes since v1:
  - Fix the location of two converted scripts: kconfig-sym-check.py
    and streamline_config.py belong in scripts/kconfig/, not in
    scripts/ root, to match the path of their original .pl files.
    Both are now added at the correct path from the start.

This series converts the last seven Perl utilities in scripts/ to
Python 3. After this series, scripts/ contains no remaining Perl
utilities pending conversion (the original .pl files are kept for
coexistence, matching the pattern of the earlier conversions).

Each conversion is a faithful 1:1 port: same CLI, same output,
same behavior. All use only the Python standard library.

The seven scripts are:
  - checkstack.pl             -> checkstack.py
  - extract-module-sig.pl     -> extract-module-sig.py
  - extract-sys-certs.pl      -> extract-sys-certs.py
  - headerdep.pl              -> headerdep.py
  - kconfig-sym-check.pl      -> scripts/kconfig/kconfig-sym-check.py
  - parse-maintainers.pl      -> parse-maintainers.py
  - streamline_config.pl      -> scripts/kconfig/streamline_config.py

Five of the seven were verified byte-for-byte identical to their
Perl counterparts. The remaining two (extract-sys-certs,
streamline_config) depend on build artifacts (vmlinux, .config)
and are 1:1 ports of the Perl logic.

Breno Rodrigues Alves (7):
  scripts: convert checkstack.pl to Python
  scripts: convert extract-module-sig.pl to Python
  scripts: convert extract-sys-certs.pl to Python
  scripts: convert headerdep.pl to Python
  scripts: convert kconfig-sym-check.pl to Python
  scripts: convert parse-maintainers.pl to Python
  scripts: convert streamline_config.pl to Python

Breno Rodrigues Alves (7):
  scripts: convert checkstack.pl to Python
  scripts: convert extract-module-sig.pl to Python
  scripts: convert extract-sys-certs.pl to Python
  scripts: convert headerdep.pl to Python
  scripts: convert kconfig-sym-check.pl to Python
  scripts: convert parse-maintainers.pl to Python
  scripts: convert streamline_config.pl to Python

 scripts/checkstack.py                | 143 ++++++++
 scripts/extract-module-sig.py        | 103 ++++++
 scripts/extract-sys-certs.py         | 132 ++++++++
 scripts/headerdep.py                 | 137 ++++++++
 scripts/kconfig/kconfig-sym-check.py | 177 ++++++++++
 scripts/kconfig/streamline_config.py | 475 +++++++++++++++++++++++++++
 scripts/parse-maintainers.py         | 136 ++++++++
 7 files changed, 1303 insertions(+)
 create mode 100755 scripts/checkstack.py
 create mode 100755 scripts/extract-module-sig.py
 create mode 100755 scripts/extract-sys-certs.py
 create mode 100755 scripts/headerdep.py
 create mode 100755 scripts/kconfig/kconfig-sym-check.py
 create mode 100755 scripts/kconfig/streamline_config.py
 create mode 100755 scripts/parse-maintainers.py

-- 
2.43.0

[RFC PATCH v2 1/7] scripts: convert checkstack.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of checkstack.pl in Python 3
(stdlib only). Same per-architecture regexes for x86/x86_64,
arm, arm64, m68k, mips, nios2, openrisc, parisc, powerpc,
s390, sparc, riscv and loongarch.

Verified byte-for-byte identical to the Perl version on a
synthetic objdump stream for x86_64.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/checkstack.py | 143 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 143 insertions(+)
 create mode 100755 scripts/checkstack.py

diff --git a/scripts/checkstack.py b/scripts/checkstack.py
new file mode 100755
index 000000000..8be3861b3
--- /dev/null
+++ b/scripts/checkstack.py
@@ -0,0 +1,143 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# Check the stack usage of functions.
+# Usage: objdump -d vmlinux | checkstack.py [arch] [min_stack]
+
+import platform
+import re
+import sys
+
+X = '[0-9a-f]'
+RE_FUNC = re.compile(rf'^({X}*) <(.*)>:$')
+RE_FILE = re.compile(r'(.*):\s*file format')
+
+
+def arm_push_handler(line):
+    if re.search(r'.*push.*fp, ip, lr, pc}', line):
+        return (line.count(',') + 1) * 4
+    return 0
+
+
+def config_for(arch):
+    """Return (main_re, dynamic_re, sub_fn)."""
+    if re.match(r'^(?:aarch|arm)64$', arch):
+        return (
+            re.compile(rf'^.*stp.*sp, ?#-([0-9]{{1,8}})\]!'),
+            re.compile(rf'^.*sub.*sp, sp, #(0x{X}{{1,8}})'),
+            None)
+    if arch == 'arm':
+        return (re.compile(r'.*sub.*sp, sp, #([0-9]{1,4})'), None,
+                arm_push_handler)
+    if re.match(r'^x86(?:_64)?$', arch) or re.match(r'^i[3456]86$', arch):
+        return (
+            re.compile(rf'^.*[as][du][db]    \$(0x{X}{{1,8}}),%(?:e|r)sp$'),
+            re.compile(r'^.*[as][du][db]    (%.*),%(?:e|r)sp$'),
+            None)
+    if arch == 'm68k':
+        return (re.compile(r'.*(?:linkw %fp,|addaw )#-([0-9]{1,4})'
+                           r'(?:,%sp)?$'), None, None)
+    if arch == 'mips64':
+        return (re.compile(r'.*daddiu.*sp,sp,-([0-9]{1,8})'), None, None)
+    if arch == 'mips':
+        return (re.compile(r'.*addiu.*sp,sp,-([0-9]{1,8})'), None, None)
+    if arch == 'nios2':
+        return (re.compile(r'.*addi.*sp,sp,-([0-9]{1,8})'), None, None)
+    if arch == 'openrisc':
+        return (re.compile(r'.*l\.addi.*r1,r1,-([0-9]{1,8})'), None, None)
+    if arch in ('parisc', 'parisc64'):
+        return (re.compile(rf'.*ldo ({X}{{1,8}})\(sp\),sp'), None, None)
+    if arch == 'powerpc' or re.match(r'^ppc(?:64)?(?:le)?$', arch):
+        return (re.compile(rf'.*st[dw]u.*r1,-({X}{{1,8}})\(r1\)'),
+                None, None)
+    if re.match(r'^s390x?$', arch):
+        return (re.compile(r'.*(?:lay|ag?hi).*\%r15,-([0-9]+)'
+                           r'(?:\(\%r15\))?$'), None, None)
+    if arch in ('sparc', 'sparc64'):
+        return (re.compile(r'.*save.*%sp, -([0-9]{1,8}), %sp'), None, None)
+    if re.match(r'^riscv(?:64)?$', arch):
+        return (re.compile(r'.*addi.*sp,sp,-([0-9]{1,8})'), None, None)
+    if re.match(r'^loongarch(?:32|64)?$', arch):
+        return (re.compile(r'.*addi\..*sp, .*sp, -([0-9]{1,8}).*'),
+                None, None)
+    print(f'wrong or unknown architecture "{arch}"')
+    sys.exit(1)
+
+
+def parse_size(s):
+    return int(s, 16) if s.startswith('0x') else int(s)
+
+
+def main(argv):
+    arch = argv[0] if argv else platform.machine()
+    min_stack = int(argv[1]) if len(argv) > 1 and argv[1].isdigit() else 512
+
+    main_re, dyn_re, sub_fn = config_for(arch)
+
+    stack = []
+    func = None
+    file = ''
+    intro = ''
+    total = 0
+
+    for raw in sys.stdin:
+        line = raw.rstrip('\n')
+
+        m = RE_FUNC.match(line)
+        if m:
+            func = m.group(2)
+            if total > min_stack:
+                stack.append(f'{intro}{total}\n')
+            addr = m.group(1)
+            intro = f'0x{addr} {func} [{file}]:'
+            pad = 56 - len(intro)
+            while pad > 0:
+                intro += '\t'
+                pad -= 8
+            total = 0
+            continue
+
+        m = RE_FILE.match(line)
+        if m:
+            file = m.group(1).replace('.ko', '')
+            idx = file.rfind('/')
+            if idx != -1:
+                file = file[idx + 1:]
+            continue
+
+        m = main_re.search(line)
+        if m:
+            size = parse_size(m.group(1))
+            if size > 0xf0000000:
+                size = -size + 0x80000000 + 0x80000000
+            if size > 0x10000000:
+                continue
+            total += size
+            continue
+
+        if dyn_re is not None:
+            m = dyn_re.search(line)
+            if m:
+                total += parse_size(m.group(1))
+                continue
+
+        if sub_fn is not None:
+            total += sub_fn(line)
+
+    if total > min_stack:
+        stack.append(f'{intro}{total}\n')
+
+    def key(entry):
+        m = re.search(r':\t*(\d+)$', entry)
+        size = int(m.group(1)) if m else 0
+        m = re.search(r' (.*):', entry)
+        fname = m.group(1) if m else ''
+        return (-size, fname)
+
+    for entry in sorted(stack, key=key):
+        sys.stdout.write(entry)
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH v2 2/7] scripts: convert extract-module-sig.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of extract-module-sig.pl in Python 3
(stdlib only). Same magic-number handling, same descriptor
extraction, same -0/-a/-d/-n/-k/-s output modes.

Verified byte-for-byte identical to the Perl version on a
synthetic signed module.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/extract-module-sig.py | 103 ++++++++++++++++++++++++++++++++++
 1 file changed, 103 insertions(+)
 create mode 100755 scripts/extract-module-sig.py

diff --git a/scripts/extract-module-sig.py b/scripts/extract-module-sig.py
new file mode 100755
index 000000000..e9a43e72f
--- /dev/null
+++ b/scripts/extract-module-sig.py
@@ -0,0 +1,103 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# extract-module-sig.py <part> <module-file>
+#   -0: unsigned module  -a: all sig data  -d: descriptor
+#   -n: signer name      -k: key ID        -s: signature
+
+import struct
+import sys
+
+MAGIC = b'~Module signature appended~\n'
+
+
+def main(argv):
+    if len(argv) != 2:
+        sys.exit(f'Format: {sys.argv[0]} -[0adnks] module-file >out')
+
+    part, modfile = argv
+
+    try:
+        with open(modfile, 'rb') as f:
+            buf = f.read()
+    except OSError as e:
+        sys.exit(f'{modfile}: {e}')
+
+    print(f'Read {len(buf)} bytes from module file', file=sys.stderr)
+
+    if len(buf) < 12 + len(MAGIC):
+        sys.exit('The file is too short to have a sig magic number and descriptor')
+
+    p = len(buf) - len(MAGIC)
+    if buf[p:] != MAGIC:
+        sys.exit(f'Magic number not found at {len(buf)}')
+    print(f'Found magic number at {len(buf)}', file=sys.stderr)
+
+    p -= 12
+    info_raw = buf[p:p + 12]
+    algo = info_raw[0]
+    hash_ = info_raw[1]
+    id_type = info_raw[2]
+    name_len = info_raw[3]
+    kid_len = info_raw[4]
+    sig_len = struct.unpack('>I', info_raw[8:12])[0]
+    info = [algo, hash_, id_type, name_len, kid_len, sig_len]
+
+    type_msg = {
+        0: 'Found PGP key identifier',
+        1: 'Found X.509 cert identifier',
+        2: 'Found PKCS#7/CMS encapsulation',
+    }.get(id_type, f'Found unsupported identifier type {id_type}')
+    print(type_msg, file=sys.stderr)
+
+    if p < name_len + kid_len + sig_len:
+        sys.exit('Insufficient name+kid+sig data in file')
+
+    p -= sig_len
+    raw_sig = buf[p:p + sig_len]
+    p -= kid_len
+    raw_kid = buf[p:p + kid_len]
+    p -= name_len
+    raw_name = buf[p:p + name_len]
+    module_len = p
+
+    if sig_len > 0:
+        n = min(16, sig_len)
+        hs = ''.join(f'{b:02x}' for b in raw_sig[:n])
+        print(f'Found {sig_len} bytes of signature [{hs}]', file=sys.stderr)
+
+    if kid_len > 0:
+        n = min(16, kid_len)
+        hs = ''.join(f'{b:02x}' for b in raw_kid[:n])
+        print(f'Found {kid_len} bytes of key identifier [{hs}]',
+              file=sys.stderr)
+
+    if name_len > 0:
+        sys.stderr.buffer.write(
+            f"Found {name_len} bytes of signer's name [".encode() +
+            raw_name + b']\n')
+
+    if part == '-0':
+        sys.stdout.buffer.write(buf[:module_len])
+    elif part == '-a':
+        sys.stdout.buffer.write(buf[module_len:])
+    elif part == '-d':
+        print(' '.join(str(x) for x in info))
+    elif part == '-n':
+        if id_type == 2:
+            print("No signer's name for PKCS#7 message type sig",
+                  file=sys.stderr)
+        sys.stdout.buffer.write(raw_name)
+    elif part == '-k':
+        if id_type == 2:
+            print('No key ID for PKCS#7 message type sig', file=sys.stderr)
+        sys.stdout.buffer.write(raw_kid)
+    elif part == '-s':
+        sys.stdout.buffer.write(raw_sig)
+    else:
+        sys.exit(f'Unknown part: {part}')
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH v2 3/7] scripts: convert extract-sys-certs.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of extract-sys-certs.pl in Python 3
(stdlib only). Same vmlinux section-table parsing, same symbol
lookup (nm with System.map fallback), same VMA-to-file-offset
translation.

Requires a vmlinux with the system certificate list to fully
test; logic is 1:1 with the Perl version.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/extract-sys-certs.py | 132 +++++++++++++++++++++++++++++++++++
 1 file changed, 132 insertions(+)
 create mode 100755 scripts/extract-sys-certs.py

diff --git a/scripts/extract-sys-certs.py b/scripts/extract-sys-certs.py
new file mode 100755
index 000000000..92a414eb3
--- /dev/null
+++ b/scripts/extract-sys-certs.py
@@ -0,0 +1,132 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# extract-sys-certs.py [-s <systemmap-file>] <vmlinux-file> <keyring-file>
+
+import re
+import subprocess
+import sys
+
+RE_SECTION = re.compile(
+    r'\s*([0-9]+)\s+(\S+)\s+([0-9a-f]+)\s+([0-9a-f]+)\s+'
+    r'([0-9a-f]+)\s+([0-9a-f]+)\s+2\*\*([0-9]+)')
+RE_SYMBOL = re.compile(r'([0-9a-f]+)\s([a-zA-Z])\s(\S+)')
+
+
+def parse_symbols(lines):
+    syms = {}
+    n = 0
+    for line in lines:
+        m = RE_SYMBOL.search(line)
+        if m:
+            syms[m.group(3)] = int(m.group(1), 16)
+            n += 1
+    return syms, n
+
+
+def main(argv):
+    args = list(argv)
+    sysmap = ''
+    if len(args) == 4 and args[0] == '-s':
+        sysmap = args[1]
+        args = args[2:]
+    elif len(args) != 2:
+        sys.exit(f'Format: {sys.argv[0]} [-s <systemmap-file>] '
+                 f'<vmlinux-file> <keyring-file>')
+
+    vmlinux, keyring = args
+
+    try:
+        objdump_out = subprocess.check_output(
+            ['objdump', '-h', vmlinux], text=True)
+    except (subprocess.CalledProcessError, FileNotFoundError) as e:
+        sys.exit(f'{vmlinux}: {e}')
+
+    sections = []
+    for line in objdump_out.splitlines():
+        m = RE_SECTION.search(line)
+        if not m:
+            continue
+        sections.append({
+            'name': m.group(2),
+            'len': int(m.group(3), 16),
+            'vma': int(m.group(4), 16),
+            'foff': int(m.group(6), 16),
+        })
+
+    print(f'Have {len(sections)} sections')
+
+    try:
+        nm_out = subprocess.check_output(
+            ['nm', vmlinux], text=True, stderr=subprocess.DEVNULL)
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        nm_out = ''
+
+    symbols, nr = parse_symbols(nm_out.splitlines())
+
+    if nr == 0 and sysmap:
+        print(f'No symbols in vmlinux, trying {sysmap}')
+        with open(sysmap) as f:
+            symbols, nr = parse_symbols(f.readlines())
+
+    if nr == 0:
+        sys.exit('No symbols available')
+    print(f'Have {nr} symbols')
+
+    if '__cert_list_start' not in symbols or \
+            'system_certificate_list_size' not in symbols:
+        sys.exit("Can't find system certificate list")
+
+    start = symbols['__cert_list_start']
+    size_sym = symbols['system_certificate_list_size']
+
+    found = None
+    size = 0
+    buf = b''
+    with open(vmlinux, 'rb') as f:
+        for sec in sections:
+            s_vma = sec['vma']
+            s_len = sec['len']
+            s_vend = s_vma + s_len
+
+            if start < s_vma or start >= s_vend:
+                continue
+
+            if size_sym < s_vma or size_sym > s_vend:
+                sys.exit('Certificate list size was not found on the same section')
+
+            if found is not None:
+                sys.exit(f"Cert object in multiple sections: "
+                         f"{sec['name']} and {found['name']}")
+
+            size_off = size_sym - s_vma + sec['foff']
+            f.seek(size_off)
+            packed = f.read(8)
+            if len(packed) != 8:
+                sys.exit(vmlinux)
+            size = int.from_bytes(packed, sys.byteorder)
+            end = start + size
+
+            print(f'Have {size} bytes of certs at VMA 0x{start:x}')
+            if end > s_vend:
+                sys.exit(f"Cert object partially overflows section {sec['name']}")
+            found = sec
+
+        if found is None:
+            sys.exit('Cert object not inside a section')
+
+        print(f"Certificate list in section {found['name']}")
+        foff = start - found['vma'] + found['foff']
+        print(f'Certificate list at file offset 0x{foff:x}')
+        f.seek(foff)
+        buf = f.read(size)
+        if len(buf) != size:
+            sys.exit(f'Short read on {vmlinux}')
+
+    with open(keyring, 'wb') as f:
+        f.write(buf)
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH v2 4/7] scripts: convert headerdep.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of headerdep.pl in Python 3
(stdlib only). Same include graph construction, same cycle
detection, same GraphViz output mode (--graph).

Verified byte-for-byte identical to the Perl version on a
synthetic cyclic include chain.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/headerdep.py | 137 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 137 insertions(+)
 create mode 100755 scripts/headerdep.py

diff --git a/scripts/headerdep.py b/scripts/headerdep.py
new file mode 100755
index 000000000..4b9a57ab1
--- /dev/null
+++ b/scripts/headerdep.py
@@ -0,0 +1,137 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# Detect cycles in the header file dependency graph.
+# Vegard Nossum <vegardno@ifi.uio.no>
+
+import argparse
+import os
+import re
+import sys
+
+deps = {}
+opt_include = []
+
+
+def strip(filename):
+    for inc in opt_include:
+        prefix = inc + '/'
+        if filename.startswith(prefix):
+            return filename[len(prefix):]
+    return filename
+
+
+def search(filename):
+    if os.path.isfile(filename):
+        return filename
+    for inc in opt_include:
+        path = f'{inc}/{filename}'
+        if os.path.isfile(path):
+            return path
+    return None
+
+
+def parse_all(headers):
+    queue = list(headers)
+    while queue:
+        h = queue.pop()
+        if h in deps:
+            continue
+        deps[h] = []
+        path = search(h)
+        if path is None:
+            continue
+        with open(path) as f:
+            lines = f.readlines()
+        for i, line in enumerate(lines):
+            m = re.match(r'^#\s*include\s*<(.*?)>', line)
+            if m:
+                dep = m.group(1)
+                queue.append(dep)
+                deps[h].append((i + 1, dep))
+
+
+def print_cycle(cycle):
+    # Shift line numbers to align with the "In file included" message
+    cycle = list(cycle)
+    for i in range(len(cycle) - 1):
+        cycle[i] = (cycle[i + 1][0], cycle[i][1])
+    cycle[-1] = (0, cycle[-1][1])
+
+    first = cycle.pop(0)
+    last = cycle.pop() if cycle else None
+
+    msg = 'In file included'
+    if last is not None:
+        print(f'{msg} from {last[1]},')
+    for hdr in reversed(cycle):
+        marker = ' <-- here' if last is not None and hdr[1] == last[1] else ''
+        print(f'{" " * len(msg)} from {hdr[1]}:{hdr[0]}{marker}')
+    print(f'{first[1]}:{first[0]}: warning: recursive header inclusion')
+
+
+def detect_cycles(headers, opt_all):
+    queue = [[(0, h)] for h in headers]
+    while queue:
+        top = queue.pop()
+        name = top[-1][1]
+        for dep in deps.get(name, []):
+            chain = top + [(dep[0], dep[1])]
+            if any(c[1] == dep[1] for c in top):
+                print_cycle(chain)
+                if not opt_all:
+                    return
+                continue
+            queue.append(chain)
+
+
+def mangle(name):
+    return name.replace('/', '__').replace('.', '_').replace('-', '_')
+
+
+def graph():
+    print('digraph {')
+    print()
+    print('\t/* vertices */')
+    for h in deps:
+        print(f'\t{mangle(h)} [label="{h}"];')
+    print()
+    print('\t/* edges */')
+    for h in deps:
+        for dep in deps[h]:
+            print(f'\t{mangle(h)} -> {mangle(dep[1])};')
+    print('}')
+
+
+def main(argv):
+    global opt_include
+
+    p = argparse.ArgumentParser(add_help=False)
+    p.add_argument('--all', action='store_true')
+    p.add_argument('--graph', action='store_true')
+    p.add_argument('-I', dest='includes', action='append', default=[])
+    p.add_argument('files', nargs='*')
+    p.add_argument('-h', '--help', '--version', action='store_true')
+    args = p.parse_args(argv)
+
+    if args.help:
+        print(f'Usage: {sys.argv[0]} [options] file...')
+        return 0
+
+    opt_include = list(args.includes) + ['include']
+    # Match Perl: grep { strip($_) } keeps the ORIGINAL path, using
+    # strip() only as a truthiness filter. This makes deps keys
+    # consistent with the Perl version (e.g. 'include/x.h', not 'x.h').
+    headers = [h for h in args.files if strip(h)]
+
+    parse_all(headers)
+
+    if args.graph:
+        graph()
+    else:
+        detect_cycles(headers, args.all)
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH v2 5/7] scripts: convert kconfig-sym-check.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of kconfig-sym-check.pl in Python 3
(stdlib only). Same Kconfig parsing, same symbol-reference
detection, same excludes-file handling, same exit code.

Verified byte-for-byte identical to the Perl version on the
full kernel tree (~/linux).

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/kconfig/kconfig-sym-check.py | 177 +++++++++++++++++++++++++++
 1 file changed, 177 insertions(+)
 create mode 100755 scripts/kconfig/kconfig-sym-check.py

diff --git a/scripts/kconfig/kconfig-sym-check.py b/scripts/kconfig/kconfig-sym-check.py
new file mode 100755
index 000000000..f15b25535
--- /dev/null
+++ b/scripts/kconfig/kconfig-sym-check.py
@@ -0,0 +1,177 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# Detect unknown symbols referenced in Kconfig files.
+# Usage: kconfig-sym-check.py <srctree> [excludes file]
+
+import os
+import re
+import subprocess
+import sys
+
+VALID = re.compile(r'^[a-zA-Z0-9_]+$')
+RE_DEF = re.compile(r'^\s*(?:config|menuconfig)\s+([a-zA-Z0-9_]+)\s*(?:#.*)?$')
+RE_HELP = re.compile(r'^\s*(?:help|---help---)$')
+RE_REF = re.compile(
+    r'^\s*(?:default|def_bool|def_tristate|select|depends\s+on|imply|'
+    r'visible\s+if|range|if|bool|tristate|int|hex|string|prompt)\s+(.+)\s*$')
+RE_STRING = re.compile(r'"(?:[^"\\]|\\.)*"|\'(?:[^\'\\]|\\.)*\'')
+RE_SKIP = re.compile(r'^\s*(?:#.*)?$')
+
+
+def indent_depth(ws):
+    col = 0
+    for c in ws:
+        if c == '\t':
+            col = (col // 8) * 8 + 8
+        else:
+            col += 1
+    return col
+
+
+def find_kconfig_files(srctree):
+    try:
+        out = subprocess.check_output(
+            ['git', '-C', srctree, 'ls-files', '*Kconfig*'],
+            text=True, stderr=subprocess.DEVNULL)
+        files = [ln.strip() for ln in out.splitlines() if ln.strip()]
+        if files:
+            return [os.path.join(srctree, f) for f in files]
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        pass
+    try:
+        out = subprocess.check_output(
+            ['find', srctree, '-name', '*Kconfig*'], text=True)
+        return [ln.strip() for ln in out.splitlines() if ln.strip()]
+    except (subprocess.CalledProcessError, FileNotFoundError):
+        return []
+
+
+def parse_kconfig(path, configs, refs):
+    with open(path) as f:
+        raw = f.readlines()
+
+    merged = []
+    i = 0
+    while i < len(raw):
+        line = raw[i].rstrip('\n')
+        while line.endswith('\\') and i + 1 < len(raw):
+            line = line[:-1] + ' ' + raw[i + 1].rstrip('\n')
+            i += 1
+        merged.append(line)
+        i += 1
+
+    help_mode = False
+    help_level = 0
+    i = 0
+    while i < len(merged):
+        line = merged[i]
+
+        if not line.strip():
+            i += 1
+            continue
+        if line.lstrip().startswith('#'):
+            i += 1
+            continue
+
+        level = indent_depth(re.match(r'^(\s*)', line).group(1))
+
+        if help_mode and level < help_level:
+            help_mode = False
+        if help_mode:
+            i += 1
+            continue
+
+        if RE_HELP.match(line):
+            help_mode = True
+            j = i + 1
+            while j < len(merged) and RE_SKIP.match(merged[j]):
+                j += 1
+            if j >= len(merged):
+                break
+            next_level = indent_depth(
+                re.match(r'^(\s*)', merged[j]).group(1))
+            if next_level >= level:
+                help_level = next_level
+            else:
+                help_mode = False
+            i = j
+            continue
+
+        m = RE_DEF.match(line)
+        if m:
+            configs[m.group(1)] = configs.get(m.group(1), 0) + 1
+            i += 1
+            continue
+
+        m = RE_REF.match(line)
+        if m:
+            s = m.group(1)
+            s = RE_STRING.sub('', s)
+            s = re.sub(r'#.*', '', s)
+            s = re.sub(r'\$\((?:[^()]*|\((?:[^()]*|\([^()]*\))*\))*\)', '', s)
+            s = re.sub(r'%%[^%]*%%', '', s)
+            for sym in re.split(r'[^a-zA-Z0-9_]+', s):
+                if not sym or not sym[0].isalpha():
+                    continue
+                if sym in ('if', 'y', 'n', 'm'):
+                    continue
+                if re.match(r'^0[xX][0-9a-fA-F]+$', sym):
+                    continue
+                refs[sym] = refs.get(sym, 0) + 1
+        i += 1
+
+
+def main(argv):
+    if not argv:
+        s = ''
+        try:
+            s = subprocess.check_output(
+                ['git', 'rev-parse', '--show-toplevel'],
+                text=True, stderr=subprocess.DEVNULL).strip()
+        except (subprocess.CalledProcessError, FileNotFoundError):
+            pass
+        msg = f'Usage: {sys.argv[0]} <srctree> [excludes file]\n'
+        msg += 'Please provide <srctree>.'
+        if s:
+            msg += f" Is it '{s}'?"
+        sys.exit(msg + '\n')
+
+    srctree = argv[0]
+    excludes = argv[1] if len(argv) > 1 else None
+
+    files = [f for f in find_kconfig_files(srctree)
+             if '/scripts/kconfig/tests/' not in f]
+
+    configs = {}
+    refs = {}
+    for f in files:
+        parse_kconfig(f, configs, refs)
+
+    known = {}
+    if excludes:
+        with open(excludes) as f:
+            for line in f:
+                line = line.rstrip('\n')
+                if not line.strip() or line.lstrip().startswith('#'):
+                    continue
+                m = re.match(r'^\s*([a-zA-Z0-9_]+)\s*(?:#.*)?$', line)
+                if m:
+                    known[m.group(1)] = 1
+
+    ret = 0
+    for k in sorted(refs):
+        if k in configs or k in known:
+            continue
+        print(k, end='')
+        if k in ('N', 'Y', 'M'):
+            print(f" - warning: '{k}' is probably not what you want; "
+                  f"Kconfig tristate literals are always lowercase "
+                  f"('n', 'y', 'm')", end='')
+        print()
+        ret = 1
+    return ret
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH v2 6/7] scripts: convert parse-maintainers.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of parse-maintainers.pl in Python 3
(stdlib only). Same section parsing, same --order preferred
ordering, same --input/--output/--section behavior.

Verified byte-for-byte identical to the Perl version on the
kernel MAINTAINERS file (30,222 lines).

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/parse-maintainers.py | 136 +++++++++++++++++++++++++++++++++++
 1 file changed, 136 insertions(+)
 create mode 100755 scripts/parse-maintainers.py

diff --git a/scripts/parse-maintainers.py b/scripts/parse-maintainers.py
new file mode 100755
index 000000000..520f66ab8
--- /dev/null
+++ b/scripts/parse-maintainers.py
@@ -0,0 +1,136 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# Sort MAINTAINERS file sections alphabetically.
+
+import argparse
+import re
+import sys
+from functools import cmp_to_key
+
+PREFERRED_ORDER = 'MRLSWQBCPTFXNK'
+
+
+def by_category_cmp(a, b):
+    aa = a.upper().replace('THE REST', 'ZZZZZZ')
+    bb = b.upper().replace('THE REST', 'ZZZZZZ')
+    return (aa > bb) - (aa < bb)
+
+
+def by_pattern_cmp(a, b):
+    a1 = a[0].upper() if a else ''
+    b1 = b[0].upper() if b else ''
+    ai = PREFERRED_ORDER.find(a1)
+    bi = PREFERRED_ORDER.find(b1)
+    if ai < 0:
+        ai = 1000
+    if bi < 0:
+        bi = 1000
+    if (a1 == 'F' and b1 == 'F') or (a1 == 'X' and b1 == 'X'):
+        return (a > b) - (a < b)
+    if ai < bi:
+        return -1
+    if ai == bi:
+        return 0
+    return 1
+
+
+def trim(s):
+    return s.strip()
+
+
+def alpha_output(ref, filename, order):
+    if not ref:
+        return
+    with open(filename, 'w') as f:
+        sep = None
+        for key in sorted(ref.keys(), key=cmp_to_key(by_category_cmp)):
+            if key == ' ':
+                f.write(ref[key])
+            else:
+                if sep is None:
+                    sep = '\n'
+                else:
+                    f.write(sep)
+                f.write(key + '\n')
+                patterns = ref[key].split('\n')
+                if order:
+                    patterns = sorted(patterns, key=cmp_to_key(by_pattern_cmp))
+                for p in patterns:
+                    if p:
+                        f.write(p + '\n')
+
+
+def file_input(ref, filename):
+    lastline = ''
+    case = ' '
+    ref[case] = ''
+
+    with open(filename) as f:
+        lines = f.readlines()
+
+    for line in lines:
+        m = re.match(r'^([A-Z]):\s*(.*)', line)
+        if m:
+            line = m.group(1) + ':\t' + trim(m.group(2)) + '\n'
+            if lastline == '':
+                ref[case] += line
+                continue
+            case = trim(lastline)
+            if case in ref:
+                sys.exit(f"Header '{case}' already exists")
+            ref[case] = line
+            lastline = ''
+            continue
+
+        if case == ' ':
+            ref[case] += lastline
+            lastline = line
+            continue
+        if trim(lastline) != '':
+            sys.exit(f"Odd non-pattern line '{lastline}' for '{case}'")
+        lastline = line
+
+    ref[case] += lastline
+
+
+def main(argv):
+    p = argparse.ArgumentParser(add_help=False)
+    p.add_argument('--input', default='MAINTAINERS')
+    p.add_argument('--output', default='MAINTAINERS.new')
+    p.add_argument('--section', default='SECTION.new')
+    p.add_argument('--order', action='store_true', default=False)
+    p.add_argument('-h', '--help', '--usage', action='store_true')
+    p.add_argument('patterns', nargs='*')
+    args = p.parse_args(argv)
+
+    if args.help:
+        print_usage()
+        return 0
+
+    hash_ = {}
+    file_input(hash_, args.input)
+
+    new_hash = {}
+    for pattern in args.patterns:
+        for key in list(hash_.keys()):
+            if re.search(pattern, key) or re.search(pattern, hash_[key]):
+                new_hash[key] = hash_[key]
+                del hash_[key]
+
+    alpha_output(hash_, args.output, args.order)
+    alpha_output(new_hash, args.section, args.order)
+    return 0
+
+
+def print_usage():
+    print(f'''usage: {sys.argv[0]} [options] <pattern matching regexes>
+
+  --input => MAINTAINERS file to read (default: MAINTAINERS)
+  --output => sorted MAINTAINERS file to write (default: MAINTAINERS.new)
+  --section => new sorted MAINTAINERS file to write (default: SECTION.new)
+  --order => Use the preferred section content output ordering (default: 0)''')
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

[RFC PATCH v2 7/7] scripts: convert streamline_config.pl to Python

[Breno Rodrigues Alves]

Faithful re-implementation of streamline_config.pl in Python 3
(stdlib only). Same Kconfig recursion, same Makefile parsing,
same dependency/select resolution, same LMC_KEEP preservation.

Requires a .config and loaded modules to fully test; logic is
1:1 with the Perl version.

Signed-off-by: Breno Rodrigues Alves <breno3011alves@gmail.com>
---
 scripts/kconfig/streamline_config.py | 475 +++++++++++++++++++++++++++
 1 file changed, 475 insertions(+)
 create mode 100755 scripts/kconfig/streamline_config.py

diff --git a/scripts/kconfig/streamline_config.py b/scripts/kconfig/streamline_config.py
new file mode 100755
index 000000000..f2e53975f
--- /dev/null
+++ b/scripts/kconfig/streamline_config.py
@@ -0,0 +1,475 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright 2005-2009 - Steven Rostedt
+#
+# Simplify the kernel config to only the modules loaded on this system.
+
+import os
+import re
+import subprocess
+import sys
+
+DEBUG = 'LOCALMODCONFIG_DEBUG' in os.environ
+
+# Module state populated by read_kconfig()
+depends = {}
+selects = {}
+prompts = {}
+objects = {}
+config2kfile = {}
+defaults = {}
+read_kconfigs = {}
+ksource = '.'
+
+
+def dprint(*args):
+    if DEBUG:
+        print(*args, file=sys.stderr)
+
+
+def read_config():
+    uname = subprocess.check_output(['uname', '-r'], text=True).strip()
+
+    searchconfigs = [
+        ('.config', 'cat', None),
+        ('/proc/config.gz', 'zcat', None),
+        (f'/boot/config-{uname}', 'cat', None),
+        (f'/boot/vmlinuz-{uname}', 'scripts/extract-ikconfig',
+         'scripts/extract-ikconfig'),
+        ('vmlinux', 'scripts/extract-ikconfig', 'scripts/extract-ikconfig'),
+        (f'/lib/modules/{uname}/kernel/kernel/configs.ko',
+         'scripts/extract-ikconfig', 'scripts/extract-ikconfig'),
+        ('kernel/configs.ko', 'scripts/extract-ikconfig',
+         'scripts/extract-ikconfig'),
+        ('kernel/configs.o', 'scripts/extract-ikconfig',
+         'scripts/extract-ikconfig'),
+    ]
+
+    for file, exec_cmd, test_cmd in searchconfigs:
+        if not os.path.isfile(file):
+            continue
+        if test_cmd is not None:
+            r = subprocess.run([test_cmd, file], capture_output=True)
+            if r.returncode:
+                continue
+        print(f"using config: '{file}'", file=sys.stderr)
+        try:
+            out = subprocess.check_output(
+                f'{exec_cmd} {file}', shell=True, text=True,
+                stderr=subprocess.DEVNULL)
+        except subprocess.CalledProcessError:
+            continue
+        return out.splitlines(keepends=True)
+
+    sys.exit('No config file found')
+
+
+def read_kconfig(kconfig, if_state):
+    """Recursively read a Kconfig file, populate globals.
+
+    if_state is a [level, [deps...]] list for mutation.
+    """
+    state = 'NONE'
+    config = ''
+    cont = False
+    line = ''
+
+    source = f'{ksource}/{kconfig}'
+    last_source = ''
+    while re.search(r'\$\((\w+)\)', source) and source != last_source:
+        last_source = source
+        m = re.search(r'\$\((\w+)\)', source)
+        source = source.replace(m.group(0), os.environ.get(m.group(1), ''))
+
+    try:
+        with open(source) as f:
+            raw_lines = f.readlines()
+    except OSError:
+        sys.exit(f"Can't open {source}")
+
+    iflevel = if_state[0]
+    ifdeps = if_state[1]
+
+    for raw in raw_lines:
+        line_text = raw.rstrip('\n')
+
+        if cont:
+            line_text = line + ' ' + line_text
+
+        if line_text.endswith('\\'):
+            cont = True
+            line = line_text[:-1]
+            continue
+        cont = False
+
+        m = re.match(r'^source\s+"?([^"]+)', line_text)
+        if m:
+            sub = m.group(1)
+            if sub not in read_kconfigs:
+                read_kconfigs[sub] = 1
+                read_kconfig(sub, if_state)
+            continue
+
+        m = re.match(r'^\s*(?:menu)?config\s+(\S+)\s*$', line_text)
+        if m:
+            state = 'NEW'
+            config = m.group(1)
+            config2kfile[f'CONFIG_{config}'] = kconfig
+            for i in range(if_state[0]):
+                if i:
+                    depends[config] = depends.get(config, '') + ' ' + ifdeps[i]
+                else:
+                    depends[config] = ifdeps[i]
+                state = 'DEP'
+            continue
+
+        m = re.match(r'^\s*depends\s+on\s+(.*)$', line_text)
+        if m and state in ('NEW', 'DEP'):
+            if state == 'NEW':
+                depends[config] = m.group(1)
+                state = 'DEP'
+            else:
+                depends[config] = depends.get(config, '') + ' ' + m.group(1)
+            continue
+
+        m = re.match(r'^\s*def(?:_(?:bool|tristate)|ault)\s+(\S.*)$',
+                     line_text)
+        if m and state != 'NONE':
+            dep = m.group(1)
+            defaults[config] = 1
+            if not re.match(r'^\s*(?:y|m|n)\s*$', dep):
+                dep = re.sub(r'.*\sif\s+', '', dep)
+                depends[config] = depends.get(config, '') + ' ' + dep
+                dprint(f'Added default depends {dep} to {config}')
+            continue
+
+        m = re.match(r'^\s*select\s+(\S+)', line_text)
+        if m and state != 'NONE':
+            conf = m.group(1)
+            if conf in selects:
+                selects[conf] += ' ' + config
+            else:
+                selects[conf] = config
+            continue
+
+        if state != 'NONE' and \
+                re.match(r'^\s*(?:tristate\s+\S|prompt\b)', line_text):
+            prompts[config] = 1
+            continue
+
+        m = re.match(r'^if\s+(.*\S)\s*$', line_text)
+        if m:
+            deps_str = re.sub(r'^[^a-zA-Z0-9_]*', '', m.group(1))
+            deps_str = re.sub(r'[^a-zA-Z0-9_]*$', '', deps_str)
+            parts = re.split(r'[^a-zA-Z0-9_]+', deps_str)
+            if if_state[0] < len(ifdeps):
+                ifdeps[if_state[0]] = ':'.join(parts)
+            else:
+                ifdeps.append(':'.join(parts))
+            if_state[0] += 1
+            continue
+
+        if re.match(r'^endif', line_text):
+            if if_state[0]:
+                if_state[0] -= 1
+            continue
+
+        if re.match(r'^\s*(?:---)?help(?:---)?\s*$', line_text) or \
+                re.match(r'^(?:comment|choice|menu)\b', line_text):
+            state = 'NONE'
+
+
+def convert_vars(line, make_vars):
+    """Expand $(VAR) references in a Makefile line."""
+    out = ''
+    while True:
+        m = re.match(r'^(.*?)(\$\(([^()]+)\))(.*)$', line)
+        if not m:
+            out += line
+            break
+        out += m.group(1)
+        var = m.group(3)
+        if var in make_vars:
+            out += make_vars[var]
+        else:
+            out += m.group(2)
+        line = m.group(4)
+    return out
+
+
+def main(argv):
+    global ksource
+
+    config_file = read_config()
+
+    localmodconfig = False
+    localyesconfig = False
+    args = []
+    for a in argv:
+        if a == '--localmodconfig':
+            localmodconfig = True
+        elif a == '--localyesconfig':
+            localyesconfig = True
+        else:
+            args.append(a)
+
+    ksource = args[0] if args else '.'
+    kconfig = args[1] if len(args) > 1 else None
+    lsmod_file = os.environ.get('LSMOD')
+
+    try:
+        makefiles = subprocess.check_output(
+            ['find', ksource, '-name', 'Makefile', '-or', '-name', 'Kbuild'],
+            text=True, stderr=subprocess.DEVNULL).splitlines()
+    except subprocess.CalledProcessError:
+        makefiles = []
+
+    if kconfig:
+        read_kconfig(kconfig, [0, []])
+
+    # Parse Makefiles for obj-$(CONFIG_FOO) += obj.o
+    for makefile in makefiles:
+        line = ''
+        make_vars = {}
+        try:
+            with open(makefile) as f:
+                for raw in f:
+                    line_text = raw.rstrip('\n')
+                    if line_text.endswith('\\'):
+                        line += line_text[:-1]
+                        continue
+                    line += line_text
+                    expanded = convert_vars(line, make_vars)
+                    line = ''
+
+                    m = re.match(
+                        r'obj-\$[({](CONFIG_[^})]*)[)}]\s*[+:]?=\s*(.*)',
+                        expanded)
+                    objs = None
+                    var = None
+                    if m:
+                        var = m.group(1)
+                        objs = m.group(2)
+                    else:
+                        m = re.match(r'^\s*(\S+)\s*[:]?=\s*(.*\S)',
+                                     expanded)
+                        if m:
+                            make_vars[m.group(1)] = m.group(2)
+                    if objs is not None:
+                        for o in re.split(r'\s+', objs):
+                            o = o.replace('-', '_')
+                            m = re.match(r'(.*)\.o$', o)
+                            if m:
+                                objects.setdefault(m.group(1), []).append(var)
+        except OSError:
+            continue
+
+    # Get loaded modules
+    modules = {}
+    if lsmod_file:
+        if not os.path.isfile(lsmod_file):
+            alt = os.path.join(os.environ.get('objtree', ''), lsmod_file)
+            if os.path.isfile(alt):
+                lsmod_file = alt
+            else:
+                sys.exit(f'{lsmod_file} not found')
+        with open(lsmod_file) as f:
+            lines = f.readlines()
+    else:
+        lsmod = 'lsmod'
+        for d in ('/sbin', '/bin', '/usr/sbin', '/usr/bin'):
+            if os.path.isfile(f'{d}/lsmod') and os.access(
+                    f'{d}/lsmod', os.X_OK):
+                lsmod = f'{d}/lsmod'
+                break
+        try:
+            out = subprocess.check_output([lsmod], text=True,
+                                          stderr=subprocess.DEVNULL)
+            lines = out.splitlines(keepends=True)
+        except (subprocess.CalledProcessError, FileNotFoundError):
+            sys.exit(f'Can not call lsmod with {lsmod}')
+
+    for line in lines:
+        if line.startswith('Module'):
+            continue
+        m = re.match(r'^(\S+)', line)
+        if m:
+            modules[m.group(1)] = 1
+
+    # Build config set from objects
+    configs = {}
+    for mod in modules:
+        if mod in objects:
+            for conf in objects[mod]:
+                configs[conf] = mod
+                dprint(f'{conf} added by direct ({mod})')
+        else:
+            print(f'{mod} config not found!', file=sys.stderr)
+
+    # Read current configs
+    orig_configs = {}
+    for line in config_file:
+        m = re.search(r'(CONFIG_[A-Za-z_0-9]*)=(m|y)', line)
+        if m:
+            orig_configs[m.group(1)] = m.group(2)
+
+    valid = 'A-Za-z_0-9'
+
+    def parse_config_depends(p, depconfig):
+        nonlocal repeat
+        while re.search(rf'[{valid}]', p):
+            m = re.match(rf'^[^{valid}]*([{valid}]+)', p)
+            if not m:
+                sys.exit('parse_config_depends: unexpected')
+            conf = 'CONFIG_' + m.group(1)
+            p = re.sub(rf'^[^{valid}]*[{valid}]+', '', p, count=1)
+
+            if conf not in orig_configs or orig_configs[conf] == 'y':
+                continue
+            if conf not in configs:
+                repeat = 1
+                dprint(f'{conf} selected by depend {depconfig}')
+                configs[conf] = 1
+
+    def parse_config_selects(config, p):
+        nonlocal repeat
+        next_config = None
+        while re.search(rf'[{valid}]', p):
+            m = re.match(rf'^[^{valid}]*([{valid}]+)', p)
+            if not m:
+                sys.exit('parse_config_selects: unexpected')
+            conf = 'CONFIG_' + m.group(1)
+            p = re.sub(rf'^[^{valid}]*[{valid}]+', '', p, count=1)
+
+            if conf not in orig_configs:
+                dprint(f'{conf} not set for {config} select')
+                continue
+            if orig_configs[conf] != 'm':
+                dprint(f'{conf} (non module) selects {config}, we are good')
+                return
+            if conf in configs:
+                dprint(f'{conf} selects {config} so we are good')
+                return
+            if next_config is None:
+                next_config = conf
+
+        if next_config is None:
+            if config not in defaults:
+                print(f'WARNING: {config} is required, but nothing in the',
+                      file=sys.stderr)
+                print(f'  current config selects it.', file=sys.stderr)
+            return
+
+        repeat = 1
+        configs[next_config] = 1
+        dprint(f'{next_config} selected by select {config}')
+
+    repeat = 1
+    while repeat:
+        repeat = 0
+        process_selects = {}
+        for config in list(configs.keys()):
+            if config in orig_configs and orig_configs[config] != 'm':
+                continue
+            short = config.replace('CONFIG_', '', 1)
+            if short in depends:
+                parse_config_depends(depends[short], short)
+            if short not in prompts and short in selects:
+                process_selects[short] = 1
+
+        for config in process_selects:
+            dprint(f'Process select {config}')
+            parse_config_selects(config, selects[config])
+
+    # Preserve list
+    preserved = []
+    if 'LMC_KEEP' in os.environ:
+        preserved = os.environ['LMC_KEEP'].split(':')
+
+    def in_preserved(conf):
+        kconf = config2kfile.get(conf)
+        if kconf is None:
+            return False
+        for excl in preserved:
+            if re.match(f'^{excl}', kconf):
+                return True
+        return False
+
+    setconfigs = {}
+
+    for line in config_file:
+        # IKCONFIG special case
+        m = re.search(r'CONFIG_IKCONFIG', line)
+        if m:
+            if '# CONFIG_IKCONFIG is not set' in line:
+                print('CONFIG_IKCONFIG=m')
+                print('# CONFIG_IKCONFIG_PROC is not set')
+            else:
+                sys.stdout.write(line)
+            continue
+
+        m = re.match(r'CONFIG_MODULE_SIG_KEY="(.+)"', line)
+        if m:
+            orig_cert = m.group(1)
+            default_cert = 'certs/signing_key.pem'
+            if 'MODULE_SIG_KEY' not in depends or \
+                    f'"{default_cert}"' not in depends['MODULE_SIG_KEY']:
+                print(f'WARNING: MODULE_SIG_KEY assertion failure, '
+                      f'update needed to {sys.argv[0]}', file=sys.stderr)
+                sys.stdout.write(line)
+            elif orig_cert != default_cert and not os.path.isfile(orig_cert):
+                print(f'Module signature verification enabled but module '
+                      f'signing key "{orig_cert}" not found. Resetting '
+                      f'signing key to default value.', file=sys.stderr)
+                print(f'CONFIG_MODULE_SIG_KEY="{default_cert}"')
+            else:
+                sys.stdout.write(line)
+            continue
+
+        m = re.match(r'CONFIG_SYSTEM_TRUSTED_KEYS="(.+)"', line)
+        if m:
+            orig_keys = m.group(1)
+            if not os.path.isfile(orig_keys):
+                print(f'System keyring enabled but keys "{orig_keys}" not '
+                      f'found. Resetting keys to default value.',
+                      file=sys.stderr)
+                print('CONFIG_SYSTEM_TRUSTED_KEYS=""')
+            else:
+                sys.stdout.write(line)
+            continue
+
+        m = re.match(r'^(CONFIG.*)=(m|y)', line)
+        if m:
+            conf, val = m.group(1), m.group(2)
+            if in_preserved(conf):
+                dprint(f'Preserve config {conf}')
+                sys.stdout.write(line)
+                continue
+            if conf in configs:
+                if localyesconfig:
+                    setconfigs[conf] = 'y'
+                    print(f'{conf}=y')
+                    continue
+                setconfigs[conf] = val
+            elif val == 'm':
+                print(f'# {conf} is not set')
+                continue
+        sys.stdout.write(line)
+
+    # Integrity check
+    for mod in modules:
+        if mod not in objects:
+            continue
+        if not any(c in setconfigs for c in objects[mod]):
+            print(f'module {mod} did not have configs', end='',
+                  file=sys.stderr)
+            for c in objects[mod]:
+                print(f' {c}', end='', file=sys.stderr)
+            print(file=sys.stderr)
+
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
\ No newline at end of file
-- 
2.43.0

Re: [RFC PATCH 0/7] scripts: convert remaining Perl utilities in scripts/

[Breno Rodrigues Alves]

Hi Jon,

Thank you for the detailed and direct feedback. You are absolutely
right, and I appreciate you taking the time to correct my
misunderstandings.

Regarding get_maintainer and checkpatch: I was completely mistaken. I
conflated the ongoing discussions and RFC proposals with the existing
state of the kernel. I apologize for the inaccuracy.

Your point about the maintenance burden and the "consistency" argument
is very well taken. I see now that "consistency" is not a valid
technical justification if there are no active bugs or integration
requirements (like Sphinx for kerneldoc). As you noted, people running
the scripts do not care about the implementation language, and I do not
have the track record yet to offer myself as a replacement maintainer
for these tools.

I realize that my v2 patch for checkstack.py falls into the "change for
its own sake" category you described. Since the Perl script works and
has a stable maintenance status, submitting this patch was premature.

I will withdraw the checkstack patch. Please disregard the [PATCH v2]
sent earlier. I want to contribute meaningfully to the kernel, but
clearly, I need to focus on fixing actual bugs or making changes with a
clear, undeniable technical necessity.

Thank you again for the guidance. It is a lesson I will not forget.

Best regards,
Breno

Re: [RFC PATCH 0/7] scripts: convert remaining Perl utilities in scripts/

[Jonathan Corbet]

Breno Rodrigues Alves <breno3011alves@gmail.com> writes:

> Hi Heiko,
>
>> That is an odd reason - if I'm not wrong then you have zero upstream
>> commits, and now are proposing fundamental changes to tools maintained
>> and used by others? I guess this is 100% LLM work(?).
>
> You are right to suspect that I used an LLM as an aid for this series.
> As required by the kernel policy, I will include an Assisted-by: tag in
> every commit to make this fully transparent.

The tag is no longer required, but openness about how you did this work
is. 

> Regarding my lack of upstream history: that is true, I am a new
> contributor. However, I fully understand the code I am submitting. The
> LLM was used to speed up the mechanical translation, but I reviewed every
> line, checked the logic, and tested the scripts' behavior to ensure 1:1
> parity. I am aware that final responsibility for the code is mine.
>
>> So an important part of describing any change is *why* you are doing it.
>> Why are you doing all of these conversions? Some of these scripts have
>> not seen changes in decades and are likely unused; why do we want to
>> mess with them now?
>
> I would like to clarify the motivation behind the conversion, because I
> do not think I presented it convincingly in the cover letter. The goal
> is not simply to "remove Perl", but to reduce the long-term maintenance
> burden.

What is the maintenance burden of scripts that have not been touched in
decades and, perhaps, are not used at all?

To the extent that these scripts *are* maintained, they have maintainers
who understand them; that may no longer be the case after you replace
them.  Who will maintain them going forward?  You can surely understand
that there might be some reluctance to entrust that role to a new
contributor with no track record.

> The kernel already has a clear track record of migrating critical
> tooling to Python (e.g. get_maintainer and checkpatch).

Huh?  You are *trying* to convert those scripts, that certainly has not
been done.  The current maintainer of checkpatch, I believe, was not
enthusiastic about your proposed change.

> Keeping a
> residual set of Perl scripts forces maintainers and contributors to deal
> with two environments and two sets of idioms for the same class of
> tasks. Unifying that code base in Python simplifies review, testing, and
> future contributions. Even scripts that have not changed in decades are
> still part of the tooling ecosystem and may be used by developers;
> removing them from the Perl equation is a consistency gain.

People running them do not care what language they are written in.

> To reduce the risk and demonstrate my commitment, I would like to
> propose a different path: can we start with a single script? I could
> prepare a patch focused only on checkstack.pl (or another script you
> consider a priority), with a detailed justification and a test plan, so
> you can evaluate the process and quality before we consider the rest of
> the series.

You really need to justify such changes beyond "consistency"; the
existing scripts work and the ones that are in active use have at least
some sort of maintenance that you are attempting to sweep aside.

If you want examples, look at the kerneldoc conversion.  There was a
clear reason for that change (it has to integrate with Sphinx, which is
a Python application) and developers who spent several cycles working on
and evaluating the change.  We knew what the maintenance picture would
be going forward.

This work looks more like change for its own sake.

Thanks,

jon

Re: [RFC PATCH 0/7] scripts: convert remaining Perl utilities in scripts/

[Breno Rodrigues Alves]

Hi Heiko,

> That is an odd reason - if I'm not wrong then you have zero upstream
> commits, and now are proposing fundamental changes to tools maintained
> and used by others? I guess this is 100% LLM work(?).

You are right to suspect that I used an LLM as an aid for this series.
As required by the kernel policy, I will include an Assisted-by: tag in
every commit to make this fully transparent.

Regarding my lack of upstream history: that is true, I am a new
contributor. However, I fully understand the code I am submitting. The
LLM was used to speed up the mechanical translation, but I reviewed every
line, checked the logic, and tested the scripts' behavior to ensure 1:1
parity. I am aware that final responsibility for the code is mine.

> So an important part of describing any change is *why* you are doing it.
> Why are you doing all of these conversions? Some of these scripts have
> not seen changes in decades and are likely unused; why do we want to
> mess with them now?

I would like to clarify the motivation behind the conversion, because I
do not think I presented it convincingly in the cover letter. The goal
is not simply to "remove Perl", but to reduce the long-term maintenance
burden.

The kernel already has a clear track record of migrating critical
tooling to Python (e.g. get_maintainer and checkpatch). Keeping a
residual set of Perl scripts forces maintainers and contributors to deal
with two environments and two sets of idioms for the same class of
tasks. Unifying that code base in Python simplifies review, testing, and
future contributions. Even scripts that have not changed in decades are
still part of the tooling ecosystem and may be used by developers;
removing them from the Perl equation is a consistency gain.

To reduce the risk and demonstrate my commitment, I would like to
propose a different path: can we start with a single script? I could
prepare a patch focused only on checkstack.pl (or another script you
consider a priority), with a detailed justification and a test plan, so
you can evaluate the process and quality before we consider the rest of
the series.

I am happy to adjust the approach in whatever way is most productive for
you and the community.

Best regards,
Breno