[PATCH v2 0/4] perf build: Fix pylint build failures

[PATCH v2 0/4] perf build: Fix pylint build failures

[Leo Yan]

Recent changes to perf's Python module, including enabling pylint by
default, exposed build failures in our CI. For example:

  tests/shell/lib/perf_brstack_max.py:8:0: E0401: Unable to import 'perf' (import-error)

This series builds the perf Python extension before running pylint on
its consumers without delaying C builds. It also prepends the output
Python directory to PYTHONPATH for out-of-tree builds.

The series also mutes the optparse deprecation warning from older pylint
versions. Python has reversed this deprecation, and newer pylint
versions no longer report it.

A final patch renames an exception variable in the PostgreSQL exporter
to fix pylint warning.

Tested on an x86 Intel Xeon(R) w7-3465X (28 cores, 56 threads), the
average real build time over three runs was 20.104s for v1 and 18.998s
for v2, about 1.1s (5.5%) faster.

Signed-off-by: Leo Yan <leo.yan@arm.com>
---
Changes in v2:
- Run pylint after the Python extension without delaying C builds
  (Sashiko).
- Fix the PostgreSQL warning.
- Link to v1: https://lore.kernel.org/r/20260930-perf_fix_pylint_errors-v1-0-f72151d8b255@arm.com

---
Leo Yan (4):
      perf build: Build the Python extension before pylint consumers
      perf build: Add the output Python directory to pylint's PYTHONPATH
      perf test: Suppress optparse deprecation warning with older pylint
      perf python: Avoid shadowing exception name in PostgreSQL exporter

 tools/perf/Build                          |  4 +++-
 tools/perf/Makefile.perf                  | 17 ++++++++++++++++-
 tools/perf/python/export-to-postgresql.py |  4 ++--
 tools/perf/tests/Build                    |  4 +++-
 tools/perf/tests/shell/lib/attr.py        |  3 ++-
 5 files changed, 26 insertions(+), 6 deletions(-)
---
base-commit: 45d15e89a783a0a279b7a54f9018c230127380d7
change-id: 20260930-perf_fix_pylint_errors-3cef583b1e47

Best regards,
-- 
Leo Yan <leo.yan@arm.com>

[PATCH v2 1/4] perf build: Build the Python extension before pylint consumers

[Leo Yan]

Pylint checks for tests and standalone Python scripts can run before the
perf Python extension is available. On a clean parallel build, an import
error can stop make before the extension is built.

Run the script and test pylint checks in separate sub-makes after the
language bindings are built. Keep their rules in the respective Build
files so the perf and perf-test C sub-makes can compile concurrently with
util and the Python extension. An isolated make perf does not need to
build the extension just for these checks.

Fixes: 8a54784e708b ("perf build: Add pylint build tests")
Reported-by: Aishwarya TCV <aishwarya.tcv@arm.com>
Assisted-by: Codex:gpt-6
Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 tools/perf/Build         |  4 +++-
 tools/perf/Makefile.perf | 15 +++++++++++++++
 tools/perf/tests/Build   |  4 +++-
 3 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/tools/perf/Build b/tools/perf/Build
index 8a0b5592a973384deafc5bb5cc7ed0cc91e0257e..09fd88f5aca54118a9b089e9c20124e23de77517 100644
--- a/tools/perf/Build
+++ b/tools/perf/Build
@@ -102,4 +102,6 @@ $(OUTPUT)%.pylint_log: %
 	$(call rule_mkdir)
 	$(Q)$(call echo-cmd,test)$(PYLINT) "$<" > $@ || (cat $@ && rm $@ && false)
 
-perf-y += $(PYLINT_TEST_LOGS)
+.PHONY: pylint
+pylint: $(addprefix $(OUTPUT),$(PYLINT_TEST_LOGS))
+	@:
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 0102c3468a0b79fb22966d8cbd8fa8525d2ff3ae..0eb4e7c29c400021907d3da716bc4871f0df8d88 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -578,6 +578,7 @@ strip: $(PROGRAMS) $(OUTPUT)perf
 	$(STRIP) $(STRIP_OPTS) $(PROGRAMS) $(OUTPUT)perf
 
 PERF_IN := $(OUTPUT)perf-in.o
+
 export NO_JEVENTS
 
 build := -f $(srctree)/tools/build/Makefile.build dir=. obj
@@ -633,6 +634,20 @@ $(SCRIPTS) : $(OUTPUT)% : %.sh
 $(OUTPUT)PERF-VERSION-FILE: .FORCE-PERF-VERSION-FILE
 	$(Q)$(SHELL_PATH) util/PERF-VERSION-GEN $(OUTPUT)
 
+ifdef PYLINT
+# Run Python checks after the extension is ready, independently of C builds.
+all: pylint-perf pylint-tests
+.PHONY: pylint-perf pylint-tests
+
+pylint-perf pylint-tests: | $(LANG_BINDINGS)
+
+pylint-perf:
+	$(Q)$(MAKE) $(build)=perf pylint
+
+pylint-tests:
+	$(Q)$(MAKE) $(build)=perf-test dir=tests pylint
+endif
+
 # These can record PERF_VERSION
 perf.spec $(SCRIPTS) \
 	: $(OUTPUT)PERF-VERSION-FILE
diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index 8fecd539012429c0f517d95f145a91137333322b..05c545aac73232a322652a6119c3178f39b83145 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -119,4 +119,6 @@ $(OUTPUT)%.pylint_log: %
 	$(call rule_mkdir)
 	$(Q)$(call echo-cmd,test)$(PYLINT) "$<" > $@ || (cat $@ && rm $@ && false)
 
-perf-test-y += $(PYLINT_TEST_LOGS)
+.PHONY: pylint
+pylint: $(addprefix $(OUTPUT)tests/,$(PYLINT_TEST_LOGS))
+	@:

-- 
2.34.1

[PATCH v2 2/4] perf build: Add the output Python directory to pylint's PYTHONPATH

[Leo Yan]

Prepend the output Python directory to pylint's PYTHONPATH so builds
using O= can also find the generated extension.

Fixes: 5fe83fb352c8 ("perf build: Make pylint build test opt-out (NO_PYLINT=1)")
Reported-by: Aishwarya TCV <aishwarya.tcv@arm.com>
Assisted-by: Codex:gpt-6
Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 tools/perf/Makefile.perf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 0eb4e7c29c400021907d3da716bc4871f0df8d88..7ceefc1da774625d9f48d3bb0498ceb819b84d26 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -310,7 +310,7 @@ ifneq ($(force_fixdep),1)
       $(warning Warning: pylint version is older than 2.16.0, skipping pylint checks.)
       override PYLINT :=
     else
-      override PYLINT := PYTHONPATH=$(srctree)/tools/perf/python:$(srctree)/tools/perf/pmu-events \
+      override PYLINT := PYTHONPATH=$(abspath $(OUTPUT)python):$(srctree)/tools/perf/python:$(srctree)/tools/perf/pmu-events \
         $(PYLINT) --disable=all --enable=E,W --disable=W0123,W0311,W0511,W0603,W0622,W0718 \
         --ignored-modules=textual,setproctitle
     endif

-- 
2.34.1

[PATCH v2 3/4] perf test: Suppress optparse deprecation warning with older pylint

[Leo Yan]

Python 3.13 reversed optparse's deprecation, and pylint 3.3.5 removed it
from the deprecated-module list. Perf supports older pylint versions,
so suppress the warning on this import and document the reason. Keep
other deprecated-module diagnostics enabled.

Link: https://docs.python.org/3.13/whatsnew/3.13.html#optparse
Link: https://pylint.readthedocs.io/en/latest/whatsnew/3/3.3/index.html#what-s-new-in-pylint-3-3-5
Fixes: 8a54784e708b ("perf build: Add pylint build tests")
Reported-by: Aishwarya TCV <aishwarya.tcv@arm.com>
Assisted-by: Codex:gpt-6
Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 tools/perf/tests/shell/lib/attr.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/perf/tests/shell/lib/attr.py b/tools/perf/tests/shell/lib/attr.py
index 7f3d5b64b00dd7de45baff4f7d9baff89e003147..523257bcf8e126cb662af589727144bb26cff4d5 100644
--- a/tools/perf/tests/shell/lib/attr.py
+++ b/tools/perf/tests/shell/lib/attr.py
@@ -4,7 +4,8 @@ import configparser
 import os
 import sys
 import glob
-import optparse
+# Pylint before 3.3.5 still treats optparse as deprecated.
+import optparse  # pylint: disable=deprecated-module
 import platform
 import tempfile
 import logging

-- 
2.34.1

[PATCH v2 4/4] perf python: Avoid shadowing exception name in PostgreSQL exporter

[Leo Yan]

During the perf build, pylint reports:

  ************* Module export-to-postgresql
  python/export-to-postgresql.py:319:8: W0621: Redefining name 'e' from outer scope (line 100) (redefined-outer-name)

Rename 'e' to 'db_error' to clear the warning.

Fixes: b1f968c9656a ("perf python: Port export-to-postgresql to perf module")
Assisted-by: Codex:gpt-6
Signed-off-by: Leo Yan <leo.yan@arm.com>
---
 tools/perf/python/export-to-postgresql.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/perf/python/export-to-postgresql.py b/tools/perf/python/export-to-postgresql.py
index b6782129354ed3b1bd750af41e7a002c7134bf12..9cac64dee361214de3d1f9907c5e6849327c8ed7 100755
--- a/tools/perf/python/export-to-postgresql.py
+++ b/tools/perf/python/export-to-postgresql.py
@@ -316,9 +316,9 @@ class PostgresExporter:
             # SQL identifiers are double-quoted; escape embedded double quotes by doubling them.
             db_name = self.dbname.replace('"', '""')
             self.do_query(f'CREATE DATABASE "{db_name}"')
-        except Exception as e:
+        except Exception as db_error:
             shutil.rmtree(self.output_dir_name, ignore_errors=True)
-            raise e
+            raise db_error
         self.disconnect()
 
         self.connect(self.dbname)

-- 
2.34.1

Re: [PATCH v2 3/4] perf test: Suppress optparse deprecation warning with older pylint

[Ian Rogers]

On Wed, Sep 30, 2026 at 6:11 AM Leo Yan <leo.yan@arm.com> wrote:
>
> Python 3.13 reversed optparse's deprecation, and pylint 3.3.5 removed it
> from the deprecated-module list. Perf supports older pylint versions,
> so suppress the warning on this import and document the reason. Keep
> other deprecated-module diagnostics enabled.
>
> Link: https://docs.python.org/3.13/whatsnew/3.13.html#optparse
> Link: https://pylint.readthedocs.io/en/latest/whatsnew/3/3.3/index.html#what-s-new-in-pylint-3-3-5
> Fixes: 8a54784e708b ("perf build: Add pylint build tests")
> Reported-by: Aishwarya TCV <aishwarya.tcv@arm.com>
> Assisted-by: Codex:gpt-6
> Signed-off-by: Leo Yan <leo.yan@arm.com>
> ---
>  tools/perf/tests/shell/lib/attr.py | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/tools/perf/tests/shell/lib/attr.py b/tools/perf/tests/shell/lib/attr.py
> index 7f3d5b64b00dd7de45baff4f7d9baff89e003147..523257bcf8e126cb662af589727144bb26cff4d5 100644
> --- a/tools/perf/tests/shell/lib/attr.py
> +++ b/tools/perf/tests/shell/lib/attr.py
> @@ -4,7 +4,8 @@ import configparser
>  import os
>  import sys
>  import glob
> -import optparse
> +# Pylint before 3.3.5 still treats optparse as deprecated.
> +import optparse  # pylint: disable=deprecated-module

Thanks Leo! Could we just update to argparse here?
```
diff --git a/tools/perf/tests/shell/lib/attr.py
b/tools/perf/tests/shell/lib/attr.py
index 7f3d5b64b00d..68f06df9565a 100644
--- a/tools/perf/tests/shell/lib/attr.py
+++ b/tools/perf/tests/shell/lib/attr.py
@@ -4,7 +4,7 @@ import configparser
 import os
 import sys
 import glob
-import optparse
+import argparse
 import platform
 import tempfile
 import logging
@@ -426,29 +426,20 @@ def setup_log(verbose):
     ch.setFormatter(formatter)
     log.addHandler(ch)

-USAGE = '''%s [OPTIONS]
-  -d dir  # tests dir
-  -p path # perf binary
-  -t test # single test
-  -v      # verbose level
-''' % sys.argv[0]
-
 def main():
-    parser = optparse.OptionParser(usage=USAGE)
-
-    parser.add_option("-t", "--test",
-                      action="store", type="string", dest="test")
-    parser.add_option("-d", "--test-dir",
-                      action="store", type="string", dest="test_dir")
-    parser.add_option("-p", "--perf",
-                      action="store", type="string", dest="perf")
-    parser.add_option("-v", "--verbose",
-                      default=0, action="count", dest="verbose")
-
-    options, args = parser.parse_args()
-    if args:
-        parser.error('FAILED wrong arguments %s' %  ' '.join(args))
-        return -1
+    parser = argparse.ArgumentParser()
+
+    parser.add_argument("-t", "--test",
+                        action="store", dest="test", help="single test")
+    parser.add_argument("-d", "--test-dir",
+                        action="store", dest="test_dir", help="tests dir")
+    parser.add_argument("-p", "--perf",
+                        action="store", dest="perf", help="perf binary")
+    parser.add_argument("-v", "--verbose",
+                        default=0, action="count", dest="verbose",
+                        help="verbose level")
+
+    options = parser.parse_args()

     setup_log(options.verbose)
```

Thanks,
Ian

>  import platform
>  import tempfile
>  import logging
>
> --
> 2.34.1
>

Re: [PATCH v2 1/4] perf build: Build the Python extension before pylint consumers

[Ian Rogers]

On Wed, Sep 30, 2026 at 6:11 AM Leo Yan <leo.yan@arm.com> wrote:
>
> Pylint checks for tests and standalone Python scripts can run before the
> perf Python extension is available. On a clean parallel build, an import
> error can stop make before the extension is built.
>
> Run the script and test pylint checks in separate sub-makes after the
> language bindings are built. Keep their rules in the respective Build
> files so the perf and perf-test C sub-makes can compile concurrently with
> util and the Python extension. An isolated make perf does not need to
> build the extension just for these checks.
>
> Fixes: 8a54784e708b ("perf build: Add pylint build tests")
> Reported-by: Aishwarya TCV <aishwarya.tcv@arm.com>
> Assisted-by: Codex:gpt-6
> Signed-off-by: Leo Yan <leo.yan@arm.com>

Reviewed-by: Ian Rogers <irogers@google.com>

Thanks Leo! Should we do the same thing for mypy and shellcheck? This
should help overall build performance.

Ian

> ---
>  tools/perf/Build         |  4 +++-
>  tools/perf/Makefile.perf | 15 +++++++++++++++
>  tools/perf/tests/Build   |  4 +++-
>  3 files changed, 21 insertions(+), 2 deletions(-)
>
> diff --git a/tools/perf/Build b/tools/perf/Build
> index 8a0b5592a973384deafc5bb5cc7ed0cc91e0257e..09fd88f5aca54118a9b089e9c20124e23de77517 100644
> --- a/tools/perf/Build
> +++ b/tools/perf/Build
> @@ -102,4 +102,6 @@ $(OUTPUT)%.pylint_log: %
>         $(call rule_mkdir)
>         $(Q)$(call echo-cmd,test)$(PYLINT) "$<" > $@ || (cat $@ && rm $@ && false)
>
> -perf-y += $(PYLINT_TEST_LOGS)
> +.PHONY: pylint
> +pylint: $(addprefix $(OUTPUT),$(PYLINT_TEST_LOGS))
> +       @:
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index 0102c3468a0b79fb22966d8cbd8fa8525d2ff3ae..0eb4e7c29c400021907d3da716bc4871f0df8d88 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -578,6 +578,7 @@ strip: $(PROGRAMS) $(OUTPUT)perf
>         $(STRIP) $(STRIP_OPTS) $(PROGRAMS) $(OUTPUT)perf
>
>  PERF_IN := $(OUTPUT)perf-in.o
> +
>  export NO_JEVENTS
>
>  build := -f $(srctree)/tools/build/Makefile.build dir=. obj
> @@ -633,6 +634,20 @@ $(SCRIPTS) : $(OUTPUT)% : %.sh
>  $(OUTPUT)PERF-VERSION-FILE: .FORCE-PERF-VERSION-FILE
>         $(Q)$(SHELL_PATH) util/PERF-VERSION-GEN $(OUTPUT)
>
> +ifdef PYLINT
> +# Run Python checks after the extension is ready, independently of C builds.
> +all: pylint-perf pylint-tests
> +.PHONY: pylint-perf pylint-tests
> +
> +pylint-perf pylint-tests: | $(LANG_BINDINGS)
> +
> +pylint-perf:
> +       $(Q)$(MAKE) $(build)=perf pylint
> +
> +pylint-tests:
> +       $(Q)$(MAKE) $(build)=perf-test dir=tests pylint
> +endif
> +
>  # These can record PERF_VERSION
>  perf.spec $(SCRIPTS) \
>         : $(OUTPUT)PERF-VERSION-FILE
> diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
> index 8fecd539012429c0f517d95f145a91137333322b..05c545aac73232a322652a6119c3178f39b83145 100644
> --- a/tools/perf/tests/Build
> +++ b/tools/perf/tests/Build
> @@ -119,4 +119,6 @@ $(OUTPUT)%.pylint_log: %
>         $(call rule_mkdir)
>         $(Q)$(call echo-cmd,test)$(PYLINT) "$<" > $@ || (cat $@ && rm $@ && false)
>
> -perf-test-y += $(PYLINT_TEST_LOGS)
> +.PHONY: pylint
> +pylint: $(addprefix $(OUTPUT)tests/,$(PYLINT_TEST_LOGS))
> +       @:
>
> --
> 2.34.1
>

Re: [PATCH v2 2/4] perf build: Add the output Python directory to pylint's PYTHONPATH

[Ian Rogers]

On Wed, Sep 30, 2026 at 6:11 AM Leo Yan <leo.yan@arm.com> wrote:
>
> Prepend the output Python directory to pylint's PYTHONPATH so builds
> using O= can also find the generated extension.
>
> Fixes: 5fe83fb352c8 ("perf build: Make pylint build test opt-out (NO_PYLINT=1)")
> Reported-by: Aishwarya TCV <aishwarya.tcv@arm.com>
> Assisted-by: Codex:gpt-6
> Signed-off-by: Leo Yan <leo.yan@arm.com>

Reviewed-by: Ian Rogers <irogers@google.com>

Thanks!
Ian

> ---
>  tools/perf/Makefile.perf | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
> index 0eb4e7c29c400021907d3da716bc4871f0df8d88..7ceefc1da774625d9f48d3bb0498ceb819b84d26 100644
> --- a/tools/perf/Makefile.perf
> +++ b/tools/perf/Makefile.perf
> @@ -310,7 +310,7 @@ ifneq ($(force_fixdep),1)
>        $(warning Warning: pylint version is older than 2.16.0, skipping pylint checks.)
>        override PYLINT :=
>      else
> -      override PYLINT := PYTHONPATH=$(srctree)/tools/perf/python:$(srctree)/tools/perf/pmu-events \
> +      override PYLINT := PYTHONPATH=$(abspath $(OUTPUT)python):$(srctree)/tools/perf/python:$(srctree)/tools/perf/pmu-events \
>          $(PYLINT) --disable=all --enable=E,W --disable=W0123,W0311,W0511,W0603,W0622,W0718 \
>          --ignored-modules=textual,setproctitle
>      endif
>
> --
> 2.34.1
>

Re: [PATCH v2 4/4] perf python: Avoid shadowing exception name in PostgreSQL exporter

[Ian Rogers]

On Wed, Sep 30, 2026 at 6:11 AM Leo Yan <leo.yan@arm.com> wrote:
>
> During the perf build, pylint reports:
>
>   ************* Module export-to-postgresql
>   python/export-to-postgresql.py:319:8: W0621: Redefining name 'e' from outer scope (line 100) (redefined-outer-name)
>
> Rename 'e' to 'db_error' to clear the warning.
>
> Fixes: b1f968c9656a ("perf python: Port export-to-postgresql to perf module")
> Assisted-by: Codex:gpt-6
> Signed-off-by: Leo Yan <leo.yan@arm.com>

Reviewed-by: Ian Rogers <irogers@google.com>

Thanks!
Ian

> ---
>  tools/perf/python/export-to-postgresql.py | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/tools/perf/python/export-to-postgresql.py b/tools/perf/python/export-to-postgresql.py
> index b6782129354ed3b1bd750af41e7a002c7134bf12..9cac64dee361214de3d1f9907c5e6849327c8ed7 100755
> --- a/tools/perf/python/export-to-postgresql.py
> +++ b/tools/perf/python/export-to-postgresql.py
> @@ -316,9 +316,9 @@ class PostgresExporter:
>              # SQL identifiers are double-quoted; escape embedded double quotes by doubling them.
>              db_name = self.dbname.replace('"', '""')
>              self.do_query(f'CREATE DATABASE "{db_name}"')
> -        except Exception as e:
> +        except Exception as db_error:
>              shutil.rmtree(self.output_dir_name, ignore_errors=True)
> -            raise e
> +            raise db_error
>          self.disconnect()
>
>          self.connect(self.dbname)
>
> --
> 2.34.1
>

Re: [PATCH v2 0/4] perf build: Fix pylint build failures

[Arnaldo Carvalho de Melo]

On Wed, Sep 30, 2026 at 02:11:35PM +0100, Leo Yan wrote:
> Recent changes to perf's Python module, including enabling pylint by
> default, exposed build failures in our CI. For example:
> 
>   tests/shell/lib/perf_brstack_max.py:8:0: E0401: Unable to import 'perf' (import-error)
> 
> This series builds the perf Python extension before running pylint on
> its consumers without delaying C builds. It also prepends the output
> Python directory to PYTHONPATH for out-of-tree builds.

Thanks, applied patches 1-2 and 4, waiting for the suggestion from Ian
to be considered for patch 3.

- Arnaldo