* [PATCH v1 01/49] perf python: Update syscall format string to optional positional
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
@ 2026-09-20 5:20 ` Ian Rogers
2026-09-20 5:20 ` [PATCH v1 02/49] perf python: Update callchain stubs and session thread lookup Ian Rogers
` (47 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:20 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Update the PyArg_ParseTupleAndKeywords format string for
pyrf__syscall_name() and pyrf__syscall_id() from "i|$i" to "i|i" (and
"s|$i" to "s|i"), allowing elf_machine to be passed as an optional
positional argument as well as a keyword argument.
Update syscall_id() and syscall_name() in perf.pyi to match, and set the
return type of syscall_name() to Optional[str] as unknown syscall numbers
return None.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/perf.pyi | 4 ++--
tools/perf/util/python.c | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/tools/perf/python/perf.pyi b/tools/perf/python/perf.pyi
index 58b0b3ed819d..e737eb21567c 100644
--- a/tools/perf/python/perf.pyi
+++ b/tools/perf/python/perf.pyi
@@ -20,7 +20,7 @@ def metrics() -> List[Dict[str, Union[str, List[str]]]]:
"""
...
-def syscall_name(id: int, *, elf_machine: Optional[int] = None) -> str:
+def syscall_name(id: int, elf_machine: Optional[int] = None) -> Optional[str]:
"""Convert a syscall number to its name.
Args:
@@ -32,7 +32,7 @@ def syscall_name(id: int, *, elf_machine: Optional[int] = None) -> str:
"""
...
-def syscall_id(name: str, *, elf_machine: Optional[int] = None) -> int:
+def syscall_id(name: str, elf_machine: Optional[int] = None) -> int:
"""Convert a syscall name to its number.
Args:
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index d8a842621cda..a3ffaf4d0850 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -4111,7 +4111,7 @@ static PyObject *pyrf__syscall_name(PyObject *self, PyObject *args, PyObject *kw
int elf_machine = EM_HOST;
static char *kwlist[] = { "id", "elf_machine", NULL };
- if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i|$i", kwlist, &id, &elf_machine))
+ if (!PyArg_ParseTupleAndKeywords(args, kwargs, "i|i", kwlist, &id, &elf_machine))
return NULL;
name = syscalltbl__name(elf_machine, id);
@@ -4127,7 +4127,7 @@ static PyObject *pyrf__syscall_id(PyObject *self, PyObject *args, PyObject *kwar
int elf_machine = EM_HOST;
static char *kwlist[] = { "name", "elf_machine", NULL };
- if (!PyArg_ParseTupleAndKeywords(args, kwargs, "s|$i", kwlist, &name, &elf_machine))
+ if (!PyArg_ParseTupleAndKeywords(args, kwargs, "s|i", kwlist, &name, &elf_machine))
return NULL;
id = syscalltbl__id(elf_machine, name);
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 02/49] perf python: Update callchain stubs and session thread lookup
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
2026-09-20 5:20 ` [PATCH v1 01/49] perf python: Update syscall format string to optional positional Ian Rogers
@ 2026-09-20 5:20 ` Ian Rogers
2026-09-20 5:20 ` [PATCH v1 03/49] perf python: Clean up pylint warnings in ilist.py Ian Rogers
` (46 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:20 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Update pyrf_session__find_thread() in util/python.c to accept an
optional tid parameter (defaulting to pid) and return None instead of
raising TypeError when a thread is not found in the session machines.
Update perf.pyi to match:
- Set session.find_thread() signature to (pid: int, tid: int = -1) ->
Optional[thread].
- Update callchain_node.symbol and callchain_node.dso to str (the C
getters return "[unknown]" fallback strings rather than None) and add
__iter__() to callchain.
Also update treport.py to handle Optional[thread] returned by
session.find_thread().
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/perf.pyi | 7 ++++---
tools/perf/python/treport.py | 2 +-
tools/perf/util/python.c | 19 +++++++++++--------
3 files changed, 16 insertions(+), 12 deletions(-)
diff --git a/tools/perf/python/perf.pyi b/tools/perf/python/perf.pyi
index e737eb21567c..e5259e9b61dd 100644
--- a/tools/perf/python/perf.pyi
+++ b/tools/perf/python/perf.pyi
@@ -320,13 +320,14 @@ class branch_stack:
class callchain_node:
"""Represents a frame in the callchain."""
ip: int
- symbol: Optional[str]
- dso: Optional[str]
+ symbol: str
+ dso: str
class callchain:
"""Sequence of callchain frames."""
def __len__(self) -> int: ...
def __getitem__(self, index: int) -> callchain_node: ...
+ def __iter__(self) -> Iterator[callchain_node]: ...
class stat_event(_sample_members):
"""Represents a stat event from perf."""
@@ -441,7 +442,7 @@ class session:
def process_events(self) -> None:
"""Process all events in the session."""
...
- def find_thread(self, pid: int) -> thread:
+ def find_thread(self, pid: int, tid: int = -1) -> Optional[thread]:
"""Returns the thread associated with a pid."""
...
diff --git a/tools/perf/python/treport.py b/tools/perf/python/treport.py
index 43542599a884..786b852f471c 100755
--- a/tools/perf/python/treport.py
+++ b/tools/perf/python/treport.py
@@ -92,7 +92,7 @@ class ProfileNode:
try:
assert session
thread = session.find_thread(sample.sample_tid)
- comm = thread.comm()
+ comm = (thread.comm() if thread else None) or f"unknown ({pid})"
except Exception:
comm = f"unknown ({pid})"
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index a3ffaf4d0850..fbfa71b1c4b6 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -3912,26 +3912,29 @@ static PyObject *pyrf_session__find_thread(struct pyrf_session *psession, PyObje
struct machine *machine;
struct thread *thread = NULL;
PyObject *result;
- int pid;
+ int pid, tid = -1;
CHECK_INITIALIZED(psession->session, "session");
- if (!PyArg_ParseTuple(args, "i", &pid))
+ if (!PyArg_ParseTuple(args, "i|i", &pid, &tid))
return NULL;
+ if (tid == -1)
+ tid = pid;
+
+ /* Look up the thread in the host machine first, then fall back to guest machines. */
machine = &psession->session->machines.host;
- thread = machine__find_thread(machine, pid, pid);
+ thread = machine__find_thread(machine, pid, tid);
if (!thread) {
machine = perf_session__find_machine(psession->session, pid);
if (machine)
- thread = machine__find_thread(machine, pid, pid);
+ thread = machine__find_thread(machine, pid, tid);
}
- if (!thread) {
- PyErr_Format(PyExc_TypeError, "Failed to find thread %d", pid);
- return NULL;
- }
+ /* Return None rather than raising TypeError when a PID/TID is not known. */
+ if (!thread)
+ Py_RETURN_NONE;
result = pyrf_thread__from_thread(thread);
thread__put(thread);
return result;
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 03/49] perf python: Clean up pylint warnings in ilist.py
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
2026-09-20 5:20 ` [PATCH v1 01/49] perf python: Update syscall format string to optional positional Ian Rogers
2026-09-20 5:20 ` [PATCH v1 02/49] perf python: Update callchain stubs and session thread lookup Ian Rogers
@ 2026-09-20 5:20 ` Ian Rogers
2026-09-20 5:20 ` [PATCH v1 04/49] perf python: Clean up pylint warnings in treport.py Ian Rogers
` (45 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:20 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Replace bare 'except:' clauses (pylint W0702: bare-except) in ilist.py
with specific exception tuples (OSError, ValueError, RuntimeError,
TypeError, KeyError) and use a safe '.get("name", "")' sort key in
IListApp.compose() so that ilist.py passes pylint without warning
suppression comments and handles unnamed PMU events gracefully.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/ilist.py | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/tools/perf/python/ilist.py b/tools/perf/python/ilist.py
index ebff0a843b7a..86bd65452e6e 100755
--- a/tools/perf/python/ilist.py
+++ b/tools/perf/python/ilist.py
@@ -82,7 +82,7 @@ class Metric(TreeValue):
try:
val = evlist.compute_metric(self.metric_name, cpu, thread)
return 0 if math.isnan(val) else val
- except:
+ except (OSError, ValueError, RuntimeError, TypeError):
# Be tolerant of failures to compute metrics on particular CPUs/threads.
return 0
@@ -419,7 +419,7 @@ class IListApp(App):
if self.evlist:
self.evlist.open()
self.evlist.enable()
- except:
+ except (OSError, ValueError, RuntimeError):
self.evlist = None
if not self.evlist:
@@ -450,7 +450,7 @@ class IListApp(App):
pmu_name = pmu.name().lower()
pmu_node = pmus.add(pmu_name)
try:
- for event in sorted(pmu.events(), key=lambda x: x["name"]):
+ for event in sorted(pmu.events(), key=lambda x: x.get("name", "")):
if "deprecated" in event:
continue
if "name" in event:
@@ -460,7 +460,7 @@ class IListApp(App):
data=PmuEvent(pmu_name, e))
else:
pmu_node.add_leaf(e, data=PmuEvent(pmu_name, e))
- except:
+ except (OSError, ValueError, RuntimeError, KeyError, TypeError):
# Reading events may fail with EPERM, ignore.
pass
metrics = tree.root.add("Metrics")
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 04/49] perf python: Clean up pylint warnings in treport.py
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (2 preceding siblings ...)
2026-09-20 5:20 ` [PATCH v1 03/49] perf python: Clean up pylint warnings in ilist.py Ian Rogers
@ 2026-09-20 5:20 ` Ian Rogers
2026-09-20 5:20 ` [PATCH v1 05/49] perf python: Clean up pylint warnings in tracepoint.py Ian Rogers
` (44 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:20 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Clean up pylint warnings in treport.py so that it passes pylint without
warning suppression comments:
- Replace broad 'except Exception:' clauses (W0718:
broad-exception-caught) with specific exception tuples.
- Pass both pid and sample_tid to session.find_thread() so guest and
process threads are resolved accurately.
- Rename parameter 'args' to 'pos_args' in FlameGraph.__init__() to
avoid shadowing the outer scope variable 'args' (W0621:
redefined-outer-name).
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/treport.py | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/tools/perf/python/treport.py b/tools/perf/python/treport.py
index 786b852f471c..082f5ca4a7da 100755
--- a/tools/perf/python/treport.py
+++ b/tools/perf/python/treport.py
@@ -91,9 +91,9 @@ class ProfileNode:
pid = sample.sample_pid
try:
assert session
- thread = session.find_thread(sample.sample_tid)
+ thread = session.find_thread(pid, sample.sample_tid)
comm = (thread.comm() if thread else None) or f"unknown ({pid})"
- except Exception:
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError):
comm = f"unknown ({pid})"
period = sample.sample_period
@@ -412,9 +412,9 @@ class FlameGraph(ScrollView):
}
"""
- def __init__(self, root: ProfileNode, *args, **kwargs):
+ def __init__(self, root: ProfileNode, *pos_args, **kwargs):
"""Initialize the FlameGraph widget."""
- super().__init__(*args, **kwargs)
+ super().__init__(*pos_args, **kwargs)
self.root = root
self.cursor = root
self.selected = root
@@ -549,7 +549,7 @@ if __name__ == "__main__":
profile = ProfileBuilder()
try:
session = perf.session(perf.data(input_file), sample=profile.process_event)
- except Exception as e:
+ except (OSError, ValueError, RuntimeError) as e:
print(f"Error opening session: {e}", file=sys.stderr)
sys.exit(1)
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 05/49] perf python: Clean up pylint warnings in tracepoint.py
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (3 preceding siblings ...)
2026-09-20 5:20 ` [PATCH v1 04/49] perf python: Clean up pylint warnings in treport.py Ian Rogers
@ 2026-09-20 5:20 ` Ian Rogers
2026-09-20 5:20 ` [PATCH v1 06/49] perf python: Clean up pylint warnings in twatch.py Ian Rogers
` (43 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:20 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Clean up pylint warnings in tracepoint.py so that it passes pylint
without warning suppression comments:
- Catch ImportError instead of bare 'except:' (W0702: bare-except) when
importing setproctitle.
- Remove unnecessary trailing semicolon (W0301: unnecessary-semicolon)
after evlist.enable().
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/tracepoint.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/perf/python/tracepoint.py b/tools/perf/python/tracepoint.py
index 15b0c8268996..9d49a25b3b1e 100755
--- a/tools/perf/python/tracepoint.py
+++ b/tools/perf/python/tracepoint.py
@@ -9,7 +9,7 @@ def change_proctitle():
try:
import setproctitle
setproctitle.setproctitle("tracepoint.py")
- except:
+ except ImportError:
print("Install the setproctitle python package to help with top and friends")
def main():
@@ -29,7 +29,7 @@ def main():
evlist.open()
evlist.mmap()
- evlist.enable();
+ evlist.enable()
while True:
evlist.poll(timeout = -1)
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 06/49] perf python: Clean up pylint warnings in twatch.py
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (4 preceding siblings ...)
2026-09-20 5:20 ` [PATCH v1 05/49] perf python: Clean up pylint warnings in tracepoint.py Ian Rogers
@ 2026-09-20 5:20 ` Ian Rogers
2026-09-20 5:20 ` [PATCH v1 07/49] perf python: Improve perf script -l descriptions Ian Rogers
` (42 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:20 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Clean up pylint warnings in twatch.py so that it passes pylint without
warning suppression comments:
- Convert tab indentation to 4 spaces (W0311: bad-indentation).
- Remove unnecessary trailing semicolon (W0301: unnecessary-semicolon)
after evsel.open().
- Convert floating multi-line string comments to '#' comments (W0105:
pointless-string-statement).
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/twatch.py | 80 ++++++++++++++++++-------------------
1 file changed, 39 insertions(+), 41 deletions(-)
diff --git a/tools/perf/python/twatch.py b/tools/perf/python/twatch.py
index 04f3db29b9bc..ad35b1620781 100755
--- a/tools/perf/python/twatch.py
+++ b/tools/perf/python/twatch.py
@@ -9,53 +9,51 @@
import perf
def main(context_switch = 0, thread = -1):
- cpus = perf.cpu_map()
- threads = perf.thread_map(thread)
- evsel = perf.evsel(type = perf.TYPE_SOFTWARE,
- config = perf.COUNT_SW_DUMMY,
- task = 1, comm = 1, mmap = 0, freq = 0,
- wakeup_events = 1, watermark = 1,
- sample_id_all = 1, context_switch = context_switch,
- sample_type = perf.SAMPLE_PERIOD | perf.SAMPLE_TID | perf.SAMPLE_CPU)
-
- """What we want are just the PERF_RECORD_ lifetime events for threads,
- using the default, PERF_TYPE_HARDWARE + PERF_COUNT_HW_CYCLES & freq=1
- (the default), makes perf reenable irq_vectors:local_timer_entry, when
- disabling nohz, not good for some use cases where all we want is to get
- threads comes and goes... So use (perf.TYPE_SOFTWARE, perf_COUNT_SW_DUMMY,
- freq=0) instead."""
-
- evsel.open(cpus = cpus, threads = threads);
- evlist = perf.evlist(cpus, threads)
- evlist.add(evsel)
- evlist.mmap()
- while True:
- evlist.poll(timeout = -1)
- for cpu in cpus:
- event = evlist.read_on_cpu(cpu)
- if not event:
- continue
- print("cpu: {0}, pid: {1}, tid: {2} {3}".format(event.sample_cpu,
+ cpus = perf.cpu_map()
+ threads = perf.thread_map(thread)
+ evsel = perf.evsel(type = perf.TYPE_SOFTWARE,
+ config = perf.COUNT_SW_DUMMY,
+ task = 1, comm = 1, mmap = 0, freq = 0,
+ wakeup_events = 1, watermark = 1,
+ sample_id_all = 1, context_switch = context_switch,
+ sample_type = perf.SAMPLE_PERIOD | perf.SAMPLE_TID | perf.SAMPLE_CPU)
+
+ # What we want are just the PERF_RECORD_ lifetime events for threads,
+ # using the default, PERF_TYPE_HARDWARE + PERF_COUNT_HW_CYCLES & freq=1
+ # (the default), makes perf reenable irq_vectors:local_timer_entry, when
+ # disabling nohz, not good for some use cases where all we want is to get
+ # threads comes and goes... So use (perf.TYPE_SOFTWARE, perf_COUNT_SW_DUMMY,
+ # freq=0) instead.
+
+ evsel.open(cpus = cpus, threads = threads)
+ evlist = perf.evlist(cpus, threads)
+ evlist.add(evsel)
+ evlist.mmap()
+ while True:
+ evlist.poll(timeout = -1)
+ for cpu in cpus:
+ event = evlist.read_on_cpu(cpu)
+ if not event:
+ continue
+ print("cpu: {0}, pid: {1}, tid: {2} {3}".format(event.sample_cpu,
event.sample_pid,
event.sample_tid,
event))
if __name__ == '__main__':
- """
- To test the PERF_RECORD_SWITCH record, pick a pid and replace
- in the following line.
+ # To test the PERF_RECORD_SWITCH record, pick a pid and replace
+ # in the following line.
- Example output:
+ # Example output:
-cpu: 3, pid: 31463, tid: 31593 { type: context_switch, next_prev_pid: 31463, next_prev_tid: 31593, switch_out: 1 }
-cpu: 1, pid: 31463, tid: 31489 { type: context_switch, next_prev_pid: 31463, next_prev_tid: 31489, switch_out: 1 }
-cpu: 2, pid: 31463, tid: 31496 { type: context_switch, next_prev_pid: 31463, next_prev_tid: 31496, switch_out: 1 }
-cpu: 3, pid: 31463, tid: 31491 { type: context_switch, next_prev_pid: 31463, next_prev_tid: 31491, switch_out: 0 }
+ # cpu: 3, pid: 31463, tid: 31593 { type: context_switch, next_prev_pid: 31463, next_prev_tid: 31593, switch_out: 1 }
+ # cpu: 1, pid: 31463, tid: 31489 { type: context_switch, next_prev_pid: 31463, next_prev_tid: 31489, switch_out: 1 }
+ # cpu: 2, pid: 31463, tid: 31496 { type: context_switch, next_prev_pid: 31463, next_prev_tid: 31496, switch_out: 1 }
+ # cpu: 3, pid: 31463, tid: 31491 { type: context_switch, next_prev_pid: 31463, next_prev_tid: 31491, switch_out: 0 }
+ #
+ # It is possible as well to use event.misc & perf.PERF_RECORD_MISC_SWITCH_OUT
+ # to figure out if this is a context switch in or out of the monitored threads.
- It is possible as well to use event.misc & perf.PERF_RECORD_MISC_SWITCH_OUT
- to figure out if this is a context switch in or out of the monitored threads.
-
- If bored, please add command line option parsing support for these options :-)
- """
- # main(context_switch = 1, thread = 31463)
+ # If bored, please add command line option parsing support for these options :-)
+ # # main(context_switch = 1, thread = 31463)
main()
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 07/49] perf python: Improve perf script -l descriptions
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (5 preceding siblings ...)
2026-09-20 5:20 ` [PATCH v1 06/49] perf python: Clean up pylint warnings in twatch.py Ian Rogers
@ 2026-09-20 5:20 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 08/49] perf python: Expose addr location, transaction, and context_switch Ian Rogers
` (41 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:20 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Add module docstrings to counting.py, tracepoint.py, and twatch.py so
that 'perf script -l' displays concise descriptions for all available
standalone scripts:
```
$ perf script -l
List of available scripts:
...
counting
Example for counting perf events.
...
tracepoint
Example showing how to enable a tracepoint and access its fields.
twatch
Example to show how to enable a software event and track
context switches.
...
```
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/counting.py | 1 +
tools/perf/python/tracepoint.py | 1 +
tools/perf/python/twatch.py | 1 +
3 files changed, 3 insertions(+)
diff --git a/tools/perf/python/counting.py b/tools/perf/python/counting.py
index 02121d2bb11d..9adbbeccdacd 100755
--- a/tools/perf/python/counting.py
+++ b/tools/perf/python/counting.py
@@ -1,5 +1,6 @@
#!/usr/bin/env python3
# SPDX-License-Identifier: GPL-2.0
+"""Example for counting perf events."""
# -*- python -*-
# -*- coding: utf-8 -*-
diff --git a/tools/perf/python/tracepoint.py b/tools/perf/python/tracepoint.py
index 9d49a25b3b1e..faa5311b8418 100755
--- a/tools/perf/python/tracepoint.py
+++ b/tools/perf/python/tracepoint.py
@@ -1,5 +1,6 @@
#! /usr/bin/env python
# SPDX-License-Identifier: GPL-2.0
+"""Example showing how to enable a tracepoint and access its fields."""
# -*- python -*-
# -*- coding: utf-8 -*-
diff --git a/tools/perf/python/twatch.py b/tools/perf/python/twatch.py
index ad35b1620781..b0f1bf113162 100755
--- a/tools/perf/python/twatch.py
+++ b/tools/perf/python/twatch.py
@@ -1,5 +1,6 @@
#! /usr/bin/env python
# SPDX-License-Identifier: GPL-2.0-only
+"""Example to show how to enable a software event and track context switches."""
# -*- python -*-
# -*- coding: utf-8 -*-
# twatch - Experimental use of the perf python interface
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 08/49] perf python: Expose addr location, transaction, and context_switch
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (6 preceding siblings ...)
2026-09-20 5:20 ` [PATCH v1 07/49] perf python: Improve perf script -l descriptions Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 09/49] perf python: Add Intel PT call_return and itrace capability Ian Rogers
` (40 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Expose destination address location components (addr_dso, addr_symbol,
addr_sym_offset), branch/transaction metrics (branch_type, in_tx,
flags, transaction), machine_pid, vcpu, sym_offset, and context_switch
callbacks directly to the Python extension.
Also include supporting fixes and infrastructure updates in
util/python.c and perf.pyi:
- Support fetching sym_offset on resolved sample symbols.
- Populate sample machine_pid and vcpu from evlist__id2sid() when
perf_guest is enabled, and resolve samples against the matching guest
or host machine.
- Expose next_prev_pid and next_prev_tid on switch_event only for
PERF_RECORD_SWITCH_CPU_WIDE records (returning None otherwise).
- Wire up tracing_data in pyrf_session__new(), use 'static char * const
kwlist[]', and update perf.pyi type stubs (including srccode() and
insn() return types).
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/perf.pyi | 22 +++-
tools/perf/util/python.c | 249 +++++++++++++++++++++++++++++++++++--
2 files changed, 253 insertions(+), 18 deletions(-)
diff --git a/tools/perf/python/perf.pyi b/tools/perf/python/perf.pyi
index e5259e9b61dd..7f747eed3eda 100644
--- a/tools/perf/python/perf.pyi
+++ b/tools/perf/python/perf.pyi
@@ -182,6 +182,8 @@ class _sample_members:
sample_time: int
sample_id: int
sample_stream_id: int
+ machine_pid: int
+ vcpu: int
sample_period: int
sample_cpu: int
@@ -206,10 +208,18 @@ class sample_event(_sample_members):
symbol: str
sym_start: int
sym_end: int
+ sym_offset: Optional[int]
+ addr_dso: Optional[str]
+ addr_symbol: Optional[str]
+ addr_sym_offset: Optional[int]
+ branch_type: int
+ in_tx: int
+ flags: int
+ transaction: int
brstack: Optional['branch_stack']
callchain: Optional['callchain']
- def srccode(self) -> str: ...
- def insn(self) -> str: ...
+ def srccode(self) -> Optional[tuple[str, int, str]]: ...
+ def insn(self) -> Optional[bytes]: ...
def __getattr__(self, name: str) -> Any: ...
class mmap_event(_sample_members):
@@ -286,8 +296,9 @@ class read_event(_sample_members):
class switch_event(_sample_members):
"""Represents a SWITCH or SWITCH_CPU_WIDE record."""
type: int
- next_prev_pid: int
- next_prev_tid: int
+ misc: int
+ next_prev_pid: Optional[int]
+ next_prev_tid: Optional[int]
evsel: Optional['evsel']
class branch_entry:
@@ -429,7 +440,8 @@ class session:
self,
data: data,
sample: Optional[Callable[[sample_event], None]] = None,
- stat: Optional[Callable[[Any, Optional[str]], None]] = None
+ stat: Optional[Callable[[Any, Optional[str]], None]] = None,
+ context_switch: Optional[Callable[[switch_event], None]] = None,
) -> None:
"""Initialize a perf session.
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index fbfa71b1c4b6..49f37198f4b3 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -3,6 +3,7 @@
#include <Python.h>
#include <inttypes.h>
+#include <stdio.h>
#include <string.h>
#include <linux/err.h>
@@ -106,6 +107,8 @@ struct pyrf_event {
sample_member_def(sample_time, time, T_ULONGLONG, "event timestamp"), \
sample_member_def(sample_id, id, T_ULONGLONG, "event id"), \
sample_member_def(sample_stream_id, stream_id, T_ULONGLONG, "event stream id"), \
+ sample_member_def(machine_pid, machine_pid, T_UINT, "event machine pid"), \
+ sample_member_def(vcpu, vcpu, T_UINT, "event vcpu"), \
sample_member_def(sample_period, period, T_ULONGLONG, "event period"), \
sample_member_def(sample_cpu, cpu, T_UINT, "event cpu"),
@@ -581,6 +584,7 @@ static PyMemberDef pyrf_sample_event__members[] = {
sample_member_def(sample_data_src, data_src, T_ULONGLONG, "event data source"),
sample_member_def(sample_insn_count, insn_cnt, T_ULONGLONG, "event instruction count"),
sample_member_def(sample_cyc_count, cyc_cnt, T_ULONGLONG, "event cycle count"),
+ sample_member_def(flags, flags, T_UINT, "event flags"),
member_def(perf_event_header, type, T_UINT, "event type"),
{ .name = NULL, },
};
@@ -679,6 +683,7 @@ static int pyrf_sample_event__resolve_al(struct pyrf_event *pevent)
struct evsel *evsel = pevent->sample.evsel;
struct evlist *evlist = evsel ? evsel->evlist : NULL;
struct perf_session *session = evlist ? evlist__session(evlist) : NULL;
+ struct machine *machine;
if (pevent->al_resolved)
return 0;
@@ -686,8 +691,14 @@ static int pyrf_sample_event__resolve_al(struct pyrf_event *pevent)
if (!session)
return -1;
+ machine = pevent->sample.machine_pid ?
+ machines__find(&session->machines, pevent->sample.machine_pid) :
+ &session->machines.host;
+ if (!machine)
+ machine = &session->machines.host;
+
addr_location__init(&pevent->al);
- if (machine__resolve(&session->machines.host, &pevent->al, &pevent->sample) < 0) {
+ if (machine__resolve(machine, &pevent->al, &pevent->sample) < 0) {
addr_location__exit(&pevent->al);
return -1;
}
@@ -771,6 +782,15 @@ static PyObject *pyrf_sample_event__get_sym_start(struct pyrf_event *pevent,
return PyLong_FromUnsignedLongLong(pevent->al.sym->start);
}
+static PyObject *pyrf_sample_event__get_sym_offset(struct pyrf_event *pevent,
+ void *closure __maybe_unused)
+{
+ if (pyrf_sample_event__resolve_al(pevent) < 0 || !pevent->al.sym)
+ Py_RETURN_NONE;
+
+ return PyLong_FromUnsignedLongLong(pevent->al.addr - pevent->al.sym->start);
+}
+
static PyObject *pyrf_sample_event__get_sym_end(struct pyrf_event *pevent,
void *closure __maybe_unused)
{
@@ -1166,7 +1186,115 @@ pyrf_sample_event__getattro(struct pyrf_event *pevent, PyObject *attr_name)
return obj ?: PyObject_GenericGetAttr((PyObject *) pevent, attr_name);
}
+
+static int pyrf_sample_event__resolve_addr_al(struct pyrf_event *pevent,
+ struct addr_location *addr_al)
+{
+ addr_location__init(addr_al);
+ if (pyrf_sample_event__resolve_al(pevent) < 0 || !pevent->al.thread)
+ return -1;
+
+ thread__find_symbol_fb(pevent->al.thread, pevent->sample.cpumode,
+ pevent->sample.addr, addr_al);
+ return 0;
+}
+
+static PyObject *pyrf_sample_event__get_addr_dso(struct pyrf_event *pevent,
+ void *closure __maybe_unused)
+{
+ struct addr_location addr_al;
+ PyObject *ret = Py_None;
+
+ if (pyrf_sample_event__resolve_addr_al(pevent, &addr_al) == 0 && addr_al.map)
+ ret = PyUnicode_FromString(dso__name(map__dso(addr_al.map)));
+ else
+ Py_INCREF(Py_None);
+
+ addr_location__exit(&addr_al);
+ return ret;
+}
+
+static PyObject *pyrf_sample_event__get_addr_symbol(struct pyrf_event *pevent,
+ void *closure __maybe_unused)
+{
+ struct addr_location addr_al;
+ PyObject *ret = Py_None;
+
+ if (pyrf_sample_event__resolve_addr_al(pevent, &addr_al) == 0 && addr_al.sym)
+ ret = PyUnicode_FromString(addr_al.sym->name);
+ else
+ Py_INCREF(Py_None);
+
+ addr_location__exit(&addr_al);
+ return ret;
+}
+
+static PyObject *pyrf_sample_event__get_addr_sym_offset(struct pyrf_event *pevent,
+ void *closure __maybe_unused)
+{
+ struct addr_location addr_al;
+ PyObject *ret = Py_None;
+
+ if (pyrf_sample_event__resolve_addr_al(pevent, &addr_al) == 0 && addr_al.sym)
+ ret = PyLong_FromUnsignedLongLong(addr_al.addr - addr_al.sym->start);
+ else
+ Py_INCREF(Py_None);
+
+ addr_location__exit(&addr_al);
+ return ret;
+}
+
+static PyObject *pyrf_sample_event__get_branch_type(struct pyrf_event *pevent,
+ void *closure __maybe_unused)
+{
+ return PyLong_FromUnsignedLong(pevent->sample.flags & PERF_BRANCH_MASK);
+}
+
+static PyObject *pyrf_sample_event__get_in_tx(struct pyrf_event *pevent,
+ void *closure __maybe_unused)
+{
+ return PyLong_FromUnsignedLong(!!(pevent->sample.flags & PERF_IP_FLAG_IN_TX));
+}
+
+static PyObject *pyrf_sample_event__get_transaction(struct pyrf_event *pevent,
+ void *closure __maybe_unused)
+{
+ return PyLong_FromUnsignedLongLong(pevent->sample.transaction);
+}
+
static PyGetSetDef pyrf_sample_event__getset[] = {
+
+ {
+ .name = "addr_dso",
+ .get = (getter)pyrf_sample_event__get_addr_dso,
+ .doc = "event destination dso.",
+ },
+ {
+ .name = "addr_symbol",
+ .get = (getter)pyrf_sample_event__get_addr_symbol,
+ .doc = "event destination symbol.",
+ },
+ {
+ .name = "addr_sym_offset",
+ .get = (getter)pyrf_sample_event__get_addr_sym_offset,
+ .doc = "event destination symbol offset.",
+ },
+ {
+ .name = "branch_type",
+ .get = (getter)pyrf_sample_event__get_branch_type,
+ .doc = "branch type.",
+ },
+ {
+ .name = "in_tx",
+ .get = (getter)pyrf_sample_event__get_in_tx,
+ .doc = "in transaction flag.",
+ },
+ {
+ .name = "transaction",
+ .get = (getter)pyrf_sample_event__get_transaction,
+ .doc = "transaction execution.",
+ },
+
{
.name = "callchain",
.get = pyrf_sample_event__get_callchain,
@@ -1227,6 +1355,12 @@ static PyGetSetDef pyrf_sample_event__getset[] = {
.set = NULL,
.doc = "event map page offset.",
},
+ {
+ .name = "sym_offset",
+ .get = (getter)pyrf_sample_event__get_sym_offset,
+ .set = NULL,
+ .doc = "event symbol offset.",
+ },
{
.name = "symbol",
.get = (getter)pyrf_sample_event__get_symbol,
@@ -1283,8 +1417,45 @@ static const char pyrf_context_switch_event__doc[] = PyDoc_STR("perf context_swi
static PyMemberDef pyrf_context_switch_event__members[] = {
sample_members
member_def(perf_event_header, type, T_UINT, "event type"),
- member_def(perf_record_switch, next_prev_pid, T_UINT, "next/prev pid"),
- member_def(perf_record_switch, next_prev_tid, T_UINT, "next/prev tid"),
+ member_def(perf_event_header, misc, T_USHORT, "event misc"),
+ { .name = NULL, },
+};
+
+static PyObject *pyrf_context_switch_event__get_next_prev_pid(const struct pyrf_event *pevent,
+ void *closure __maybe_unused)
+{
+ if (pevent->event.header.type == PERF_RECORD_SWITCH_CPU_WIDE)
+ return PyLong_FromUnsignedLong(pevent->event.context_switch.next_prev_pid);
+ Py_RETURN_NONE;
+}
+
+static PyObject *pyrf_context_switch_event__get_next_prev_tid(const struct pyrf_event *pevent,
+ void *closure __maybe_unused)
+{
+ if (pevent->event.header.type == PERF_RECORD_SWITCH_CPU_WIDE)
+ return PyLong_FromUnsignedLong(pevent->event.context_switch.next_prev_tid);
+ Py_RETURN_NONE;
+}
+
+static PyGetSetDef pyrf_context_switch_event__getset[] = {
+ {
+ .name = "evsel",
+ .get = pyrf_event__get_evsel,
+ .set = NULL,
+ .doc = "tracking event.",
+ },
+ {
+ .name = "next_prev_pid",
+ .get = (getter)pyrf_context_switch_event__get_next_prev_pid,
+ .set = NULL,
+ .doc = "next/prev pid for CPU-wide switch, or None.",
+ },
+ {
+ .name = "next_prev_tid",
+ .get = (getter)pyrf_context_switch_event__get_next_prev_tid,
+ .set = NULL,
+ .doc = "next/prev tid for CPU-wide switch, or None.",
+ },
{ .name = NULL, },
};
@@ -1292,11 +1463,19 @@ static PyObject *pyrf_context_switch_event__repr(const struct pyrf_event *pevent
{
PyObject *ret;
char *s;
-
- if (asprintf(&s, "{ type: context_switch, next_prev_pid: %u, next_prev_tid: %u, switch_out: %u }",
- pevent->event.context_switch.next_prev_pid,
- pevent->event.context_switch.next_prev_tid,
- !!(pevent->event.header.misc & PERF_RECORD_MISC_SWITCH_OUT)) < 0) {
+ int res;
+
+ if (pevent->event.header.type == PERF_RECORD_SWITCH_CPU_WIDE) {
+ res = asprintf(&s,
+ "{ type: context_switch, next_prev_pid: %u, next_prev_tid: %u, switch_out: %u }",
+ pevent->event.context_switch.next_prev_pid,
+ pevent->event.context_switch.next_prev_tid,
+ !!(pevent->event.header.misc & PERF_RECORD_MISC_SWITCH_OUT));
+ } else {
+ res = asprintf(&s, "{ type: context_switch, switch_out: %u }",
+ !!(pevent->event.header.misc & PERF_RECORD_MISC_SWITCH_OUT));
+ }
+ if (res < 0) {
ret = PyErr_NoMemory();
} else {
ret = PyUnicode_FromString(s);
@@ -1313,7 +1492,7 @@ static PyTypeObject pyrf_context_switch_event__type = {
.tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE,
.tp_doc = pyrf_context_switch_event__doc,
.tp_members = pyrf_context_switch_event__members,
- .tp_getset = pyrf_event__getset,
+ .tp_getset = pyrf_context_switch_event__getset,
.tp_repr = (reprfunc)pyrf_context_switch_event__repr,
};
@@ -1450,6 +1629,16 @@ static PyObject *pyrf_event__new(const union perf_event *event, struct evsel *ev
return PyErr_Format(PyExc_OSError,
"perf: can't parse sample, err=%d", err);
}
+ if (session && session->evlist && perf_guest && pevent->sample.id) {
+ struct perf_sample_id *sid = evlist__id2sid(session->evlist, pevent->sample.id);
+
+ if (sid) {
+ pevent->sample.machine_pid = sid->machine_pid;
+ pevent->sample.vcpu = sid->vcpu.cpu;
+ }
+ }
+ if (machine && machine->pid > 0 && !pevent->sample.machine_pid)
+ pevent->sample.machine_pid = machine->pid;
sample = &pevent->sample;
if (machine && sample->callchain) {
struct addr_location al;
@@ -3837,6 +4026,7 @@ struct pyrf_session {
struct pyrf_data *pdata;
PyObject *sample;
PyObject *stat;
+ PyObject *context_switch;
};
static int pyrf_session_tool__sample(const struct perf_tool *tool,
@@ -3861,6 +4051,33 @@ static int pyrf_session_tool__sample(const struct perf_tool *tool,
return 0;
}
+static int pyrf_session_tool__context_switch(const struct perf_tool *tool,
+ union perf_event *event,
+ struct perf_sample *sample,
+ struct machine *machine)
+{
+ struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool);
+ PyObject *pyevent = pyrf_event__new(event, sample->evsel, psession->session, machine);
+ PyObject *ret;
+
+ if (perf_event__process_switch(tool, event, sample, machine) < 0) {
+ Py_XDECREF(pyevent);
+ return -1;
+ }
+
+ if (pyevent == NULL)
+ return -ENOMEM;
+
+ ret = PyObject_CallFunction(psession->context_switch, "O", pyevent);
+ if (!ret) {
+ Py_DECREF(pyevent);
+ return -1;
+ }
+ Py_DECREF(ret);
+ Py_DECREF(pyevent);
+ return 0;
+}
+
static int pyrf_session_tool__stat(const struct perf_tool *tool,
struct perf_session *session,
union perf_event *event)
@@ -3943,13 +4160,13 @@ static PyObject *pyrf_session__find_thread(struct pyrf_session *psession, PyObje
static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
{
struct pyrf_data *pdata;
- PyObject *sample = NULL, *stat = NULL;
- static char *kwlist[] = { "data", "sample", "stat", NULL };
+ PyObject *sample = NULL, *stat = NULL, *context_switch = NULL;
+ static char * const kwlist[] = { "data", "sample", "stat", "context_switch", NULL };
struct pyrf_session *psession;
struct perf_session *session;
- if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O!|OO", kwlist, &pyrf_data__type, &pdata,
- &sample, &stat))
+ if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O!|OOO", kwlist, &pyrf_data__type, &pdata,
+ &sample, &stat, &context_switch))
return NULL;
psession = PyObject_New(struct pyrf_session, type);
@@ -3959,6 +4176,7 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObject
psession->session = NULL;
psession->sample = NULL;
psession->stat = NULL;
+ psession->context_switch = NULL;
psession->pdata = NULL;
Py_INCREF(pdata);
@@ -3982,6 +4200,7 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObject
ADD_TOOL(sample);
ADD_TOOL(stat);
+ ADD_TOOL(context_switch);
#undef ADD_TOOL
if (stat)
@@ -4000,6 +4219,9 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObject
psession->tool.build_id = perf_event__process_build_id;
psession->tool.attr = perf_event__process_attr;
psession->tool.feature = perf_event__process_feature;
+#ifdef HAVE_LIBTRACEEVENT
+ psession->tool.tracing_data = perf_event__process_tracing_data;
+#endif
session = perf_session__new(&pdata->data, &psession->tool);
if (IS_ERR(session)) {
@@ -4030,6 +4252,7 @@ static void pyrf_session__delete(struct pyrf_session *psession)
Py_XDECREF(psession->pdata);
Py_XDECREF(psession->sample);
Py_XDECREF(psession->stat);
+ Py_XDECREF(psession->context_switch);
Py_TYPE(psession)->tp_free((PyObject *)psession);
}
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 09/49] perf python: Add Intel PT call_return and itrace capability
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (7 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 08/49] perf python: Expose addr location, transaction, and context_switch Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 10/49] perf python: Allow KeyboardInterrupt to propagate in LiveSession Ian Rogers
` (39 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Add support for Intel PT call_return events and itrace options to the
perf Python extension so standalone Python scripts can decode and
inspect low-level instruction traces.
Specifically:
- Add a call_return callback parameter and itrace option string to
perf.session, treating explicit None (Py_None) callbacks as omitted.
- Expose the perf.call_return object with db_id, parent_id, insn_count,
cyc_count, comm, pid, tid, call_time, return_time, branch_count,
call_ref, return_ref, flags, and call_path (resolved as a
perf.callchain ordered from outermost caller to callee).
- Export perf.CALL_RETURN_NO_CALL, perf.CALL_RETURN_NO_RETURN, and
perf.CALL_RETURN_NON_CALL flag constants.
- Expose session.e_machine and session.is_64_bit properties, preferring
recorded thread ELF machines (perf_session__e_machine) over perf_env
so pipe-mode traces do not prematurely cache host bitness.
- Support optional vmlinux, kallsyms, and symfs symbol configuration
parameters (and PERF_SYMBOL_VMLINUX, PERF_SYMBOL_KALLSYMS,
PERF_SYMBOL_SYMFS environment fallbacks) in perf.session, resetting
symbol_conf pointers and freeing vm_tm_corr_args, cpu_bitmap, and
ptime_range in
pyrf_session__delete().
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/perf.pyi | 35 +++
tools/perf/util/python.c | 624 +++++++++++++++++++++++++++++++++++--
2 files changed, 632 insertions(+), 27 deletions(-)
diff --git a/tools/perf/python/perf.pyi b/tools/perf/python/perf.pyi
index 7f747eed3eda..475cab9601d4 100644
--- a/tools/perf/python/perf.pyi
+++ b/tools/perf/python/perf.pyi
@@ -435,6 +435,25 @@ class evlist:
...
+class call_return:
+ """Represents a call/return event from instruction trace decoding."""
+ db_id: int
+ parent_id: int
+ insn_count: int
+ cyc_count: int
+ comm: Optional[str]
+ machine_pid: int
+ pid: Optional[int]
+ tid: Optional[int]
+ call_time: int
+ return_time: int
+ branch_count: int
+ call_ref: int
+ return_ref: int
+ flags: int
+ call_path: Optional[callchain]
+
+
class session:
def __init__(
self,
@@ -442,6 +461,11 @@ class session:
sample: Optional[Callable[[sample_event], None]] = None,
stat: Optional[Callable[[Any, Optional[str]], None]] = None,
context_switch: Optional[Callable[[switch_event], None]] = None,
+ call_return: Optional[Callable[[call_return], None]] = None,
+ itrace: Optional[str] = None,
+ vmlinux: Optional[str] = None,
+ kallsyms: Optional[str] = None,
+ symfs: Optional[str] = None
) -> None:
"""Initialize a perf session.
@@ -451,6 +475,8 @@ class session:
stat: Callback for stat events.
"""
...
+ e_machine: Optional[int]
+ is_64_bit: bool
def process_events(self) -> None:
"""Process all events in the session."""
...
@@ -683,3 +709,12 @@ RECORD_STAT_ROUND: int
RECORD_MISC_SWITCH_OUT: int
"""MISC_SWITCH_OUT record."""
+
+CALL_RETURN_NO_CALL: int
+"""'return' but no matching 'call'."""
+
+CALL_RETURN_NO_RETURN: int
+"""'call' but no matching 'return'."""
+
+CALL_RETURN_NON_CALL: int
+"""A branch but not a 'call' to the start of a different symbol."""
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 49f37198f4b3..dff307f4ab57 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -2,9 +2,11 @@
#define PY_SSIZE_T_CLEAN
#include <Python.h>
+#include <elf.h>
#include <inttypes.h>
#include <stdio.h>
#include <string.h>
+#include <stdlib.h>
#include <linux/err.h>
#include <poll.h>
@@ -23,12 +25,16 @@
#include "counts.h"
#include "data.h"
#include "debug.h"
+#include "call-path.h"
+#include "db-export.h"
+#include "thread-stack.h"
#include "dso.h"
#include "dwarf-regs.h"
#include "event.h"
#include "branch.h"
#include "evlist.h"
#include "evsel.h"
+#include "symbol.h"
#include "expr.h"
#include "map.h"
#include "metricgroup.h"
@@ -892,11 +898,13 @@ struct pyrf_callchain_node {
u64 ip;
struct map *map;
struct symbol *sym;
+ char *sym_name;
};
static void pyrf_callchain_node__delete(struct pyrf_callchain_node *pnode)
{
map__put(pnode->map);
+ free(pnode->sym_name);
Py_TYPE(pnode)->tp_free((PyObject *)pnode);
}
@@ -909,6 +917,8 @@ static PyObject *pyrf_callchain_node__get_ip(struct pyrf_callchain_node *pnode,
static PyObject *pyrf_callchain_node__get_symbol(struct pyrf_callchain_node *pnode,
void *closure __maybe_unused)
{
+ if (pnode->sym_name)
+ return PyUnicode_FromString(pnode->sym_name);
if (pnode->sym)
return PyUnicode_FromString(pnode->sym->name);
return PyUnicode_FromString("[unknown]");
@@ -953,6 +963,7 @@ struct pyrf_callchain_frame {
u64 ip;
struct map *map;
struct symbol *sym;
+ char *sym_name;
};
struct pyrf_callchain {
@@ -964,8 +975,10 @@ struct pyrf_callchain {
static void pyrf_callchain__delete(struct pyrf_callchain *pchain)
{
if (pchain->frames) {
- for (u64 i = 0; i < pchain->nr_frames; i++)
+ for (u64 i = 0; i < pchain->nr_frames; i++) {
map__put(pchain->frames[i].map);
+ free(pchain->frames[i].sym_name);
+ }
free(pchain->frames);
}
Py_TYPE(pchain)->tp_free((PyObject *)pchain);
@@ -995,6 +1008,7 @@ static PyObject *pyrf_callchain__item(PyObject *obj, Py_ssize_t i)
pnode->ip = pchain->frames[i].ip;
pnode->map = map__get(pchain->frames[i].map);
pnode->sym = pchain->frames[i].sym;
+ pnode->sym_name = pchain->frames[i].sym_name ? strdup(pchain->frames[i].sym_name) : NULL;
return (PyObject *)pnode;
}
@@ -1567,8 +1581,8 @@ static PyTypeObject *pyrf_event__type[] = {
};
static PyObject *pyrf_event__new(const union perf_event *event, struct evsel *evsel,
- struct perf_session *session,
- struct machine *machine)
+ struct perf_session *session, struct machine *machine,
+ struct perf_sample *sample_arg)
{
struct pyrf_event *pevent;
struct perf_sample *sample;
@@ -1625,9 +1639,26 @@ static PyObject *pyrf_event__new(const union perf_event *event, struct evsel *ev
err = evsel__parse_sample(evsel, &pevent->event, &pevent->sample);
evsel->needs_swap = needs_swap;
if (err < 0) {
- Py_DECREF(pevent);
- return PyErr_Format(PyExc_OSError,
- "perf: can't parse sample, err=%d", err);
+ /*
+ * Synthesized events (e.g. Intel PT itrace) may not have raw sample
+ * buffers for evsel__parse_sample(); use the pre-parsed sample_arg.
+ */
+ if (sample_arg) {
+ perf_sample__exit(&pevent->sample);
+ pevent->sample = *sample_arg;
+ if (pevent->sample.evsel)
+ pevent->sample.evsel = evsel__get(pevent->sample.evsel);
+ pevent->sample.merged_callchain = false;
+
+ pevent->sample.user_regs = NULL;
+ pevent->sample.intr_regs = NULL;
+ pevent->sample.raw_data = NULL;
+ pevent->sample.raw_size = 0;
+ } else {
+ Py_DECREF(pevent);
+ return PyErr_Format(PyExc_OSError,
+ "perf: can't parse sample, err=%d", err);
+ }
}
if (session && session->evlist && perf_guest && pevent->sample.id) {
struct perf_sample_id *sid = evlist__id2sid(session->evlist, pevent->sample.id);
@@ -3221,7 +3252,7 @@ static PyObject *pyrf_evlist__read_on_cpu(struct pyrf_evlist *pevlist,
perf_mmap__consume(&md->core);
Py_RETURN_NONE;
}
- pyevent = pyrf_event__new(event, evsel, evlist__session(evlist), /*machine=*/NULL);
+ pyevent = pyrf_event__new(event, evsel, evlist__session(evlist), /*machine=*/NULL, NULL);
perf_mmap__consume(&md->core);
if (pyevent == NULL)
return PyErr_Occurred() ? NULL : PyErr_NoMemory();
@@ -3589,6 +3620,11 @@ static const struct perf_constant perf__constants[] = {
PERF_CONST(RECORD_STAT_ROUND),
PERF_CONST(RECORD_MISC_SWITCH_OUT),
+
+ /* Values for the flags of a perf.call_return, see thread-stack.h. */
+ { "CALL_RETURN_NO_CALL", CALL_RETURN_NO_CALL },
+ { "CALL_RETURN_NO_RETURN", CALL_RETURN_NO_RETURN },
+ { "CALL_RETURN_NON_CALL", CALL_RETURN_NON_CALL },
{ .name = NULL, },
};
@@ -4023,31 +4059,417 @@ struct pyrf_session {
struct perf_session *session;
struct perf_tool tool;
+ struct itrace_synth_opts itrace_opts;
struct pyrf_data *pdata;
PyObject *sample;
PyObject *stat;
PyObject *context_switch;
+ PyObject *call_return;
+ struct call_return_processor *crp;
+ /**
+ * @call_return_last_db_id: Counter used to give each perf.call_return a
+ * unique db_id. A call's db_id may be allocated before the call
+ * returns, so the identifiers cannot be generated by the Python script.
+ */
+ u64 call_return_last_db_id;
+ u64 sample_last_db_id;
+ char *vmlinux_name;
+ char *kallsyms_name;
+ char *symfs;
+};
+
+
+struct pyrf_call_return {
+ PyObject_HEAD
+ struct call_return cr;
+ /**
+ * @call_path: perf.callchain of the call path, ordered from the
+ * outermost caller to the callee. Resolved eagerly as the underlying
+ * struct call_path is owned by the call return processor.
+ */
+ PyObject *call_path;
+ char *comm;
+ pid_t machine_pid;
+ pid_t pid;
+ pid_t tid;
+};
+
+static PyObject *pyrf_call_return__repr(struct pyrf_call_return *pcr)
+{
+ return PyUnicode_FromString("{ type: call_return }");
+}
+
+static PyObject *pyrf_call_return__get_comm(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ if (pcr->comm)
+ return PyUnicode_FromString(pcr->comm);
+ Py_RETURN_NONE;
+}
+
+static PyObject *pyrf_call_return__get_machine_pid(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ return PyLong_FromLong(pcr->machine_pid);
+}
+
+static PyObject *pyrf_call_return__get_pid(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ if (pcr->pid != -1)
+ return PyLong_FromLong(pcr->pid);
+ Py_RETURN_NONE;
+}
+
+static PyObject *pyrf_call_return__get_tid(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ if (pcr->tid != -1)
+ return PyLong_FromLong(pcr->tid);
+ Py_RETURN_NONE;
+}
+
+static PyObject *pyrf_call_return__get_call_time(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ struct call_return *cr = &pcr->cr;
+
+ return PyLong_FromUnsignedLongLong(cr->call_time);
+}
+
+static PyObject *pyrf_call_return__get_return_time(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ struct call_return *cr = &pcr->cr;
+
+ return PyLong_FromUnsignedLongLong(cr->return_time);
+}
+
+static PyObject *pyrf_call_return__get_branch_count(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ struct call_return *cr = &pcr->cr;
+
+ return PyLong_FromUnsignedLongLong(cr->branch_count);
+}
+
+static PyObject *pyrf_call_return__get_call_ref(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ struct call_return *cr = &pcr->cr;
+
+ return PyLong_FromUnsignedLongLong(cr->call_ref);
+}
+
+static PyObject *pyrf_call_return__get_return_ref(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ struct call_return *cr = &pcr->cr;
+
+ return PyLong_FromUnsignedLongLong(cr->return_ref);
+}
+
+static PyObject *pyrf_call_return__get_flags(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ struct call_return *cr = &pcr->cr;
+
+ return PyLong_FromLong(cr->flags);
+}
+
+static PyObject *pyrf_call_return__get_call_path(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ PyObject *call_path = pcr->call_path ?: Py_None;
+
+ Py_INCREF(call_path);
+ return call_path;
+}
+
+
+static PyObject *pyrf_call_return__get_parent_id(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ return PyLong_FromUnsignedLongLong(pcr->cr.parent_db_id);
+}
+
+static PyObject *pyrf_call_return__get_insn_count(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ return PyLong_FromUnsignedLongLong(pcr->cr.insn_count);
+}
+
+static PyObject *pyrf_call_return__get_cyc_count(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ return PyLong_FromUnsignedLongLong(pcr->cr.cyc_count);
+}
+
+static PyObject *pyrf_call_return__get_db_id(struct pyrf_call_return *pcr,
+ void *closure __maybe_unused)
+{
+ return PyLong_FromUnsignedLongLong(pcr->cr.db_id);
+}
+
+static PyGetSetDef pyrf_call_return__getset[] = {
+ {
+ .name = "db_id",
+ .get = (getter)pyrf_call_return__get_db_id,
+ .doc = "Unique identifier of this call return.",
+ },
+ {
+ .name = "parent_id",
+ .get = (getter)pyrf_call_return__get_parent_id,
+ .doc = "db_id of the calling call return, 0 if there is none.",
+ },
+ {
+ .name = "insn_count",
+ .get = (getter)pyrf_call_return__get_insn_count,
+ .doc = "Instructions executed between the call and the return.",
+ },
+ {
+ .name = "cyc_count",
+ .get = (getter)pyrf_call_return__get_cyc_count,
+ .doc = "Cycles elapsed between the call and the return.",
+ },
+ {
+ .name = "comm",
+ .get = (getter)pyrf_call_return__get_comm,
+ .doc = "Thread comm at the time of the call/return, or None.",
+ },
+ {
+ .name = "machine_pid",
+ .get = (getter)pyrf_call_return__get_machine_pid,
+ .doc = "Machine PID (0 for host).",
+ },
+ {
+ .name = "pid",
+ .get = (getter)pyrf_call_return__get_pid,
+ .doc = "Process ID of the thread, or None.",
+ },
+ {
+ .name = "tid",
+ .get = (getter)pyrf_call_return__get_tid,
+ .doc = "Thread identifier, None if unknown.",
+ },
+ {
+ .name = "call_time",
+ .get = (getter)pyrf_call_return__get_call_time,
+ .doc = "Timestamp of the call.",
+ },
+ {
+ .name = "return_time",
+ .get = (getter)pyrf_call_return__get_return_time,
+ .doc = "Timestamp of the return.",
+ },
+ {
+ .name = "branch_count",
+ .get = (getter)pyrf_call_return__get_branch_count,
+ .doc = "Branches taken between the call and the return.",
+ },
+ {
+ .name = "call_ref",
+ .get = (getter)pyrf_call_return__get_call_ref,
+ .doc = "Reference to the sample of the call.",
+ },
+ {
+ .name = "return_ref",
+ .get = (getter)pyrf_call_return__get_return_ref,
+ .doc = "Reference to the sample of the return.",
+ },
+ {
+ .name = "flags",
+ .get = (getter)pyrf_call_return__get_flags,
+ .doc = "Bitmask of perf.CALL_RETURN_* values.",
+ },
+ {
+ .name = "call_path",
+ .get = (getter)pyrf_call_return__get_call_path,
+ .doc = "perf.callchain from outermost caller to callee.",
+ },
+ { .name = NULL, },
+};
+
+
+static void pyrf_call_return__delete(struct pyrf_call_return *pevent)
+{
+ Py_XDECREF(pevent->call_path);
+ free(pevent->comm);
+ Py_TYPE(pevent)->tp_free((PyObject *)pevent);
+}
+
+static PyTypeObject pyrf_call_return__type = {
+ PyVarObject_HEAD_INIT(NULL, 0)
+ .tp_name = "perf.call_return",
+ .tp_basicsize = sizeof(struct pyrf_call_return),
+ .tp_dealloc = (destructor)pyrf_call_return__delete,
+ .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE,
+ .tp_doc = "perf call_return object.",
+ .tp_getset = pyrf_call_return__getset,
+ .tp_repr = (reprfunc)pyrf_call_return__repr,
};
+/*
+ * Turn a struct call_path, that is a leaf with parent pointers up to the
+ * outermost caller, into a perf.callchain ordered from the outermost caller to
+ * the leaf. The struct call_path is owned by the call return processor and so
+ * must be resolved before returning to Python.
+ */
+static PyObject *pyrf_call_path__to_callchain(const struct call_path *cp)
+{
+ struct pyrf_callchain *pchain;
+ const struct call_path *pos;
+ u64 nr_frames = 0;
+
+ /* The root call path is a sentinel with no symbol, skip it. */
+ for (pos = cp; pos && pos->parent; pos = pos->parent)
+ nr_frames++;
+
+ pchain = PyObject_New(struct pyrf_callchain, &pyrf_callchain__type);
+ if (!pchain)
+ return NULL;
+
+ pchain->nr_frames = nr_frames;
+ pchain->frames = nr_frames ? calloc(nr_frames, sizeof(*pchain->frames)) : NULL;
+ if (nr_frames && !pchain->frames) {
+ pchain->nr_frames = 0;
+ Py_DECREF(pchain);
+ return PyErr_NoMemory();
+ }
+ /* Fill in reverse so that the outermost caller is first. */
+ for (pos = cp; nr_frames > 0; pos = pos->parent) {
+ struct pyrf_callchain_frame *frame = &pchain->frames[--nr_frames];
+
+ frame->ip = pos->ip;
+ frame->sym = NULL;
+ frame->sym_name = pos->sym ? strdup(pos->sym->name) : NULL;
+ frame->map = NULL;
+ }
+ return (PyObject *)pchain;
+}
+
+static PyObject *pyrf_call_return__new(struct call_return *cr)
+{
+ struct pyrf_call_return *pevent;
+
+ pevent = PyObject_New(struct pyrf_call_return, &pyrf_call_return__type);
+ if (!pevent)
+ return NULL;
+
+ pevent->cr = *cr;
+ /* Avoid lifetime issues with thread by caching tid/pid/machine_pid */
+ if (cr->thread) {
+ struct maps *maps = thread__maps(cr->thread);
+
+ pevent->machine_pid = maps && maps__machine(maps) ? maps__machine(maps)->pid : 0;
+ pevent->pid = thread__pid(cr->thread);
+ pevent->tid = thread__tid(cr->thread);
+ } else {
+ pevent->machine_pid = 0;
+ pevent->pid = -1;
+ pevent->tid = -1;
+ }
+ pevent->cr.thread = NULL;
+ pevent->comm = cr->comm ? strdup(comm__str(cr->comm)) : NULL;
+ pevent->call_path = NULL;
+ if (cr->cp) {
+ pevent->call_path = pyrf_call_path__to_callchain(cr->cp);
+ if (!pevent->call_path) {
+ Py_DECREF(pevent);
+ return NULL;
+ }
+ }
+ return (PyObject *)pevent;
+}
+
+static int pyrf_session__call_return_process(struct call_return *cr,
+ u64 *parent_db_id,
+ void *data)
+{
+ struct pyrf_session *psession = data;
+ PyObject *pyevent, *ret;
+
+ if (!psession->call_return)
+ return 0;
+
+ if (!cr->db_id)
+ cr->db_id = ++psession->call_return_last_db_id;
+
+ if (parent_db_id) {
+ if (!*parent_db_id)
+ *parent_db_id = ++psession->call_return_last_db_id;
+ cr->parent_db_id = *parent_db_id;
+ }
+
+ pyevent = pyrf_call_return__new(cr);
+ if (!pyevent)
+ return -1;
+
+ ret = PyObject_CallFunctionObjArgs(psession->call_return, pyevent, NULL);
+ if (!ret) {
+ Py_DECREF(pyevent);
+ return -1;
+ }
+ Py_DECREF(ret);
+ Py_DECREF(pyevent);
+ return 0;
+}
+
static int pyrf_session_tool__sample(const struct perf_tool *tool,
union perf_event *event,
struct perf_sample *sample,
struct machine *machine)
{
struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool);
- PyObject *pyevent = pyrf_event__new(event, sample->evsel, psession->session, machine);
- PyObject *ret;
+ u64 sample_db_id = ++psession->sample_last_db_id;
- if (pyevent == NULL)
- return -ENOMEM;
+ if (psession->crp) {
+ struct addr_location al, addr_al;
+ struct thread *thread = NULL;
- ret = PyObject_CallFunction(psession->sample, "O", pyevent);
- if (!ret) {
+ addr_location__init(&al);
+ addr_location__init(&addr_al);
+
+ if (machine__resolve(machine, &al, sample) >= 0) {
+ thread = al.thread;
+ if (sample->flags & (PERF_IP_FLAG_CALL |
+ PERF_IP_FLAG_TRACE_BEGIN |
+ PERF_IP_FLAG_RETURN |
+ PERF_IP_FLAG_BRANCH |
+ PERF_IP_FLAG_TRACE_END)) {
+ thread__resolve(thread, &addr_al, sample);
+ }
+
+ int err;
+
+ err = thread_stack__process(thread, thread__comm(thread), sample,
+ &al, &addr_al, sample_db_id, psession->crp);
+ if (err) {
+ addr_location__exit(&addr_al);
+ addr_location__exit(&al);
+ return err;
+ }
+ }
+
+ addr_location__exit(&addr_al);
+ addr_location__exit(&al);
+ }
+
+ if (psession->sample) {
+ PyObject *pyevent = pyrf_event__new(event, sample->evsel,
+ psession->session,
+ machine, sample);
+ PyObject *ret;
+
+ if (pyevent == NULL)
+ return -ENOMEM;
+
+ ret = PyObject_CallFunction(psession->sample, "O", pyevent);
Py_DECREF(pyevent);
- return -1;
+ if (!ret)
+ return -1;
+ Py_DECREF(ret);
}
- Py_DECREF(ret);
- Py_DECREF(pyevent);
return 0;
}
@@ -4057,7 +4479,7 @@ static int pyrf_session_tool__context_switch(const struct perf_tool *tool,
struct machine *machine)
{
struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool);
- PyObject *pyevent = pyrf_event__new(event, sample->evsel, psession->session, machine);
+ PyObject *pyevent = pyrf_event__new(event, sample->evsel, psession->session, machine, NULL);
PyObject *ret;
if (perf_event__process_switch(tool, event, sample, machine) < 0) {
@@ -4085,7 +4507,7 @@ static int pyrf_session_tool__stat(const struct perf_tool *tool,
struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool);
struct evsel *evsel = evlist__id2evsel(session->evlist, event->stat.id);
PyObject *pyevent = pyrf_event__new(event, /*evsel=*/NULL, psession->session,
- /*machine=*/NULL);
+ /*machine=*/NULL, NULL);
const char *name = evsel ? evsel__name(evsel) : "unknown";
PyObject *ret;
@@ -4108,7 +4530,7 @@ static int pyrf_session_tool__stat_round(const struct perf_tool *tool,
{
struct pyrf_session *psession = container_of(tool, struct pyrf_session, tool);
PyObject *pyevent = pyrf_event__new(event, /*evsel=*/NULL, psession->session,
- /*machine=*/NULL);
+ /*machine=*/NULL, NULL);
PyObject *ret;
if (pyevent == NULL)
@@ -4136,15 +4558,17 @@ static PyObject *pyrf_session__find_thread(struct pyrf_session *psession, PyObje
if (!PyArg_ParseTuple(args, "i|i", &pid, &tid))
return NULL;
- if (tid == -1)
+ if (tid == -1) {
tid = pid;
+ pid = -1;
+ }
/* Look up the thread in the host machine first, then fall back to guest machines. */
machine = &psession->session->machines.host;
thread = machine__find_thread(machine, pid, tid);
if (!thread) {
- machine = perf_session__find_machine(psession->session, pid);
+ machine = perf_session__find_machine(psession->session, pid != -1 ? pid : tid);
if (machine)
thread = machine__find_thread(machine, pid, tid);
}
@@ -4161,23 +4585,40 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObject
{
struct pyrf_data *pdata;
PyObject *sample = NULL, *stat = NULL, *context_switch = NULL;
- static char * const kwlist[] = { "data", "sample", "stat", "context_switch", NULL };
+ PyObject *call_return = NULL;
+ char *itrace_str = NULL;
+ char *vmlinux_str = NULL, *kallsyms_str = NULL, *symfs_str = NULL;
+ static char * const kwlist[] = { "data", "sample", "stat", "context_switch",
+ "call_return", "itrace", "vmlinux", "kallsyms",
+ "symfs", NULL };
struct pyrf_session *psession;
struct perf_session *session;
- if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O!|OOO", kwlist, &pyrf_data__type, &pdata,
- &sample, &stat, &context_switch))
+ if (!PyArg_ParseTupleAndKeywords(args, kwargs, "O!|OOOOzzzz", kwlist,
+ &pyrf_data__type, &pdata,
+ &sample, &stat, &context_switch,
+ &call_return, &itrace_str,
+ &vmlinux_str, &kallsyms_str, &symfs_str))
return NULL;
psession = PyObject_New(struct pyrf_session, type);
if (!psession)
return NULL;
+ /* PyObject_New doesn't zero the memory, initialize every member. */
psession->session = NULL;
psession->sample = NULL;
psession->stat = NULL;
psession->context_switch = NULL;
+ psession->call_return = NULL;
+ psession->crp = NULL;
+ psession->call_return_last_db_id = 0;
+ psession->sample_last_db_id = 0;
+ psession->vmlinux_name = NULL;
+ psession->kallsyms_name = NULL;
+ psession->symfs = NULL;
psession->pdata = NULL;
+ memset(&psession->itrace_opts, 0, sizeof(psession->itrace_opts));
Py_INCREF(pdata);
psession->pdata = pdata;
@@ -4187,7 +4628,7 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObject
#define ADD_TOOL(name) \
do { \
- if (name) { \
+ if (name && name != Py_None) { \
if (!PyCallable_Check(name)) { \
PyErr_SetString(PyExc_TypeError, #name " must be callable"); \
goto err_out; \
@@ -4199,15 +4640,39 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObject
} while (0)
ADD_TOOL(sample);
+ if (call_return && call_return != Py_None && (!sample || sample == Py_None))
+ psession->tool.sample = pyrf_session_tool__sample;
ADD_TOOL(stat);
ADD_TOOL(context_switch);
+
+ if (call_return && call_return != Py_None) {
+ if (!PyCallable_Check(call_return)) {
+ PyErr_SetString(PyExc_TypeError, "call_return must be callable");
+ Py_DECREF(psession);
+ return NULL;
+ }
+ Py_INCREF(call_return);
+ psession->call_return = call_return;
+ psession->crp = call_return_processor__new(pyrf_session__call_return_process,
+ psession);
+ if (!psession->crp) {
+ PyErr_SetString(PyExc_MemoryError,
+ "Failed to allocate call_return_processor");
+ Py_DECREF(psession);
+ return NULL;
+ }
+ }
#undef ADD_TOOL
- if (stat)
+ if (stat && stat != Py_None)
psession->tool.stat_round = pyrf_session_tool__stat_round;
psession->tool.comm = perf_event__process_comm;
+ psession->tool.auxtrace_info = perf_event__process_auxtrace_info;
+ psession->tool.auxtrace = perf_event__process_auxtrace;
+ psession->tool.auxtrace_error = perf_event__process_auxtrace_error;
+ psession->tool.id_index = perf_event__process_id_index;
psession->tool.mmap = perf_event__process_mmap;
psession->tool.mmap2 = perf_event__process_mmap2;
psession->tool.namespaces = perf_event__process_namespaces;
@@ -4230,6 +4695,49 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObject
}
psession->session = session;
+ if (itrace_str) {
+ memset(&psession->itrace_opts, 0, sizeof(psession->itrace_opts));
+ if (itrace_do_parse_synth_opts(&psession->itrace_opts, itrace_str, 0) < 0) {
+ PyErr_SetString(PyExc_ValueError, "Failed to parse itrace options");
+ Py_DECREF(psession);
+ return NULL;
+ }
+ psession->session->itrace_synth_opts = &psession->itrace_opts;
+ }
+
+ if (!vmlinux_str)
+ vmlinux_str = getenv("PERF_SYMBOL_VMLINUX");
+ if (vmlinux_str && vmlinux_str[0]) {
+ psession->vmlinux_name = strdup(vmlinux_str);
+ if (!psession->vmlinux_name) {
+ PyErr_NoMemory();
+ goto err_out;
+ }
+ symbol_conf.vmlinux_name = psession->vmlinux_name;
+ }
+
+ if (!kallsyms_str)
+ kallsyms_str = getenv("PERF_SYMBOL_KALLSYMS");
+ if (kallsyms_str && kallsyms_str[0]) {
+ psession->kallsyms_name = strdup(kallsyms_str);
+ if (!psession->kallsyms_name) {
+ PyErr_NoMemory();
+ goto err_out;
+ }
+ symbol_conf.kallsyms_name = psession->kallsyms_name;
+ }
+
+ if (!symfs_str)
+ symfs_str = getenv("PERF_SYMBOL_SYMFS");
+ if (symfs_str && symfs_str[0]) {
+ psession->symfs = strdup(symfs_str);
+ if (!psession->symfs) {
+ PyErr_NoMemory();
+ goto err_out;
+ }
+ symbol_conf.symfs = psession->symfs;
+ }
+
symbol_conf.use_callchain = true;
symbol_conf.show_kernel_path = true;
symbol_conf.inline_name = false;
@@ -4249,10 +4757,25 @@ static PyObject *pyrf_session__new(PyTypeObject *type, PyObject *args, PyObject
static void pyrf_session__delete(struct pyrf_session *psession)
{
perf_session__delete(psession->session);
+ if (symbol_conf.vmlinux_name == psession->vmlinux_name)
+ symbol_conf.vmlinux_name = NULL;
+ free(psession->vmlinux_name);
+ if (symbol_conf.kallsyms_name == psession->kallsyms_name)
+ symbol_conf.kallsyms_name = NULL;
+ free(psession->kallsyms_name);
+ if (symbol_conf.symfs == psession->symfs)
+ symbol_conf.symfs = "";
+ free(psession->symfs);
+ free(psession->itrace_opts.vm_tm_corr_args);
+ free(psession->itrace_opts.cpu_bitmap);
+ free(psession->itrace_opts.ptime_range);
Py_XDECREF(psession->pdata);
Py_XDECREF(psession->sample);
Py_XDECREF(psession->stat);
Py_XDECREF(psession->context_switch);
+ Py_XDECREF(psession->call_return);
+ if (psession->crp)
+ call_return_processor__free(psession->crp);
Py_TYPE(psession)->tp_free((PyObject *)psession);
}
@@ -4295,10 +4818,54 @@ static const char pyrf_session__doc[] = PyDoc_STR("perf session object.");
static PyObject *pyrf_session__getattro(struct pyrf_session *psession, PyObject *attr_name)
{
+ const char *name_str = PyUnicode_AsUTF8(attr_name);
+
+ if (!name_str)
+ return NULL;
if (!psession->session) {
PyErr_SetString(PyExc_ValueError, "session not initialized");
return NULL;
}
+ if (!strcmp(name_str, "e_machine"))
+ return PyLong_FromLong(perf_session__e_machine(psession->session,
+ /*e_flags=*/NULL));
+
+ if (!strcmp(name_str, "is_64_bit")) {
+ /*
+ * Prefer the machine of the recorded threads over the perf_env,
+ * as in pipe mode the perf_env may not be populated yet and
+ * perf_env__kernel_is_64_bit would cache the host's bitness.
+ */
+ switch (perf_session__e_machine(psession->session, /*e_flags=*/NULL)) {
+ case EM_AARCH64:
+ case EM_ALPHA:
+ case EM_IA_64:
+ case EM_LOONGARCH:
+ case EM_PPC64:
+ case EM_SPARCV9:
+ case EM_X86_64:
+ Py_RETURN_TRUE;
+ case EM_386:
+ case EM_ARM:
+ case EM_PPC:
+ case EM_SPARC:
+ Py_RETURN_FALSE;
+ default:
+ /*
+ * Machines like EM_MIPS, EM_PARISC, EM_RISCV and
+ * EM_S390 are used for both word sizes, fall back on
+ * the recorded arch string if available.
+ */
+ break;
+ }
+ {
+ struct perf_env *env = perf_session__env(psession->session);
+
+ if (env && env->arch)
+ return PyBool_FromLong(perf_env__kernel_is_64_bit(env) == 1);
+ return PyBool_FromLong(sizeof(void *) == 8);
+ }
+ }
return PyObject_GenericGetAttr((PyObject *) psession, attr_name);
}
@@ -4452,7 +5019,8 @@ PyMODINIT_FUNC PyInit_perf(void)
if (module == NULL)
return NULL;
- if (pyrf_event__setup_types() < 0 ||
+ if (PyType_Ready(&pyrf_call_return__type) < 0 ||
+ pyrf_event__setup_types() < 0 ||
pyrf_evlist__setup_types() < 0 ||
pyrf_evsel__setup_types() < 0 ||
pyrf_thread_map__setup_types() < 0 ||
@@ -4535,6 +5103,8 @@ PyMODINIT_FUNC PyInit_perf(void)
Py_INCREF(&pyrf_session__type);
PyModule_AddObject(module, "session", (PyObject *)&pyrf_session__type);
+ Py_INCREF(&pyrf_call_return__type);
+ PyModule_AddObject(module, "call_return", (PyObject *)&pyrf_call_return__type);
Py_INCREF(&pyrf_branch_entry__type);
if (PyModule_AddObject(module, "branch_entry", (PyObject *)&pyrf_branch_entry__type) < 0) {
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 10/49] perf python: Allow KeyboardInterrupt to propagate in LiveSession
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (8 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 09/49] perf python: Add Intel PT call_return and itrace capability Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 11/49] perf pmu-events: Clean up mypy and pylint issues Ian Rogers
` (38 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
LiveSession.run() synchronously blocks while polling evlist for events.
When running interactive tools or scripts from the terminal,
KeyboardInterrupt exceptions raised via Ctrl-C must propagate so the
caller can break out of the polling loop.
Previously, LiveSession caught KeyboardInterrupt with an empty pass
statement, swallowing the exception and preventing calling scripts from
detecting the interrupt cleanly. Remove the empty handler so calling
scripts receive KeyboardInterrupt and can perform clean teardown.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/perf_live.py | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/tools/perf/python/perf_live.py b/tools/perf/python/perf_live.py
index 616b5f657463..665699c5407a 100755
--- a/tools/perf/python/perf_live.py
+++ b/tools/perf/python/perf_live.py
@@ -36,7 +36,8 @@ class LiveSession:
except InterruptedError:
continue
for cpu in self.cpus:
- for _ in range(1000): # Limit to 1000 events per CPU per poll to prevent starvation
+ # Limit to 1000 events per CPU per poll to prevent starvation
+ for _ in range(1000):
try:
event = self.evlist.read_on_cpu(cpu)
except TypeError as e:
@@ -53,7 +54,5 @@ class LiveSession:
if event.type == perf.RECORD_SAMPLE:
self.sample_callback(event)
- except KeyboardInterrupt:
- pass
finally:
self.evlist.close()
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 11/49] perf pmu-events: Clean up mypy and pylint issues
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (9 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 10/49] perf python: Allow KeyboardInterrupt to propagate in LiveSession Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 12/49] perf test: Clean up mypy and pylint issues in shell test libraries Ian Rogers
` (37 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Clean up mypy type errors and pylint warnings across the pmu-events Python
scripts (amd_metrics.py, intel_metrics.py, jevents.py,
make_legacy_cache.py,
and metric.py) in preparation for making MYPY and PYLINT build tests
opt-out.
In addition to type annotations and unused import/variable cleanups, fix
three bugs in intel_metrics.py exposed by static analysis:
- Fix references to undefined 'args.model' (instead of '_args.model')
inside
try/except blocks in IntelL2() and UncoreMemBw(), which previously threw
NameError and silently disabled Skylake/CascadeLake L2 silent eviction
adjustments and uncore memory bandwidth JSON loading.
- Define DC_WB_U and DC_WB_D before their first use in IntelL2(), fixing
an UnboundLocalError that silently dropped the l2_out_rate, wbn, and isd
metrics.
- Fix Event-to-string comparison and addition in IntelFp() by comparing
and modifying 'f_assist.name' instead of the Event object 'f_assist'.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/pmu-events/amd_metrics.py | 10 +-
tools/perf/pmu-events/intel_metrics.py | 259 ++++++++++-----------
tools/perf/pmu-events/jevents.py | 77 +++---
tools/perf/pmu-events/make_legacy_cache.py | 34 +--
tools/perf/pmu-events/metric.py | 58 +++--
5 files changed, 210 insertions(+), 228 deletions(-)
diff --git a/tools/perf/pmu-events/amd_metrics.py b/tools/perf/pmu-events/amd_metrics.py
index dccfcacaf148..cf8b68954737 100755
--- a/tools/perf/pmu-events/amd_metrics.py
+++ b/tools/perf/pmu-events/amd_metrics.py
@@ -70,7 +70,6 @@ def AmdBr():
])
def Conditional() -> Optional[MetricGroup]:
- global _zen_model
br = Event("ex_ret_brn_cond", "ex_ret_cond")
br_r = d_ratio(br, interval_sec)
ins_r = d_ratio(ins, br)
@@ -156,11 +155,10 @@ def AmdCtxSw() -> MetricGroup:
def AmdDtlb() -> Optional[MetricGroup]:
- global _zen_model
if _zen_model >= 4:
return None
- d_dat = Event("ls_dc_accesses") if _zen_model <= 3 else None
+ d_dat = Event("ls_dc_accesses")
d_h4k = Event("ls_l1_d_tlb_miss.tlb_reload_4k_l2_hit")
d_hcoal = Event(
"ls_l1_d_tlb_miss.tlb_reload_coalesced_page_hit") if _zen_model >= 2 else 0
@@ -173,8 +171,8 @@ def AmdDtlb() -> Optional[MetricGroup]:
d_m2m = Event("ls_l1_d_tlb_miss.tlb_reload_2m_l2_miss")
d_m1g = Event("ls_l1_d_tlb_miss.tlb_reload_1g_l2_miss")
- d_w0 = Event("ls_tablewalker.dc_type0") if _zen_model <= 3 else None
- d_w1 = Event("ls_tablewalker.dc_type1") if _zen_model <= 3 else None
+ d_w0 = Event("ls_tablewalker.dc_type0")
+ d_w1 = Event("ls_tablewalker.dc_type1")
walks = d_w0 + d_w1
walks_r = d_ratio(walks, interval_sec)
ins_w = d_ratio(ins, walks)
@@ -266,7 +264,6 @@ def AmdDtlb() -> Optional[MetricGroup]:
def AmdIotlb() -> Optional[MetricGroup]:
- global _zen_model
if _zen_model < 2:
return None
@@ -322,7 +319,6 @@ def AmdIotlb() -> Optional[MetricGroup]:
def AmdItlb():
- global _zen_model
l2h = Event("bp_l1_tlb_miss_l2_tlb_hit", "bp_l1_tlb_miss_l2_hit")
l2m = Event("bp_l1_tlb_miss_l2_tlb_miss.all", "l2_itlb_misses",)
l2r = l2h + l2m
diff --git a/tools/perf/pmu-events/intel_metrics.py b/tools/perf/pmu-events/intel_metrics.py
index 9780bf978a41..497d8fd05e09 100755
--- a/tools/perf/pmu-events/intel_metrics.py
+++ b/tools/perf/pmu-events/intel_metrics.py
@@ -5,10 +5,10 @@ import json
import math
import os
import re
-from typing import Optional
+from typing import Optional, Union
from common_metrics import Cycles
-from metric import (d_ratio, has_event, max, aggr_nr, CheckPmu, Event,
- JsonEncodeMetric, JsonEncodeMetricGroupDescriptions,
+from metric import (d_ratio, has_event, max, aggr_nr, CheckPmu, Constant, Event,
+ Expression, JsonEncodeMetric, JsonEncodeMetricGroupDescriptions,
Literal, LoadEvents, Metric, MetricConstraint, MetricGroup,
MetricRef, Select)
@@ -90,7 +90,7 @@ def Tsx() -> Optional[MetricGroup]:
# sysfs version so that we can detect its presence at runtime.
transaction_start = Event("RTM_RETIRED.START")
transaction_start = Event(f'{pmu}/tx\\-start/')
- except:
+ except ValueError:
return None
elision_start = None
@@ -100,7 +100,7 @@ def Tsx() -> Optional[MetricGroup]:
# case. Again, prefer the sysfs encoding of the event.
elision_start = Event("HLE_RETIRED.START")
elision_start = Event(f'{pmu}/el\\-start/')
- except:
+ except ValueError:
pass
return MetricGroup('transaction', [
@@ -139,7 +139,7 @@ def IntelBr():
br_clr = None
try:
br_clr = Event("BACLEARS.ANY", "BACLEARS.ALL")
- except:
+ except ValueError:
pass
br_r = d_ratio(br_all, interval_sec)
@@ -171,7 +171,7 @@ def IntelBr():
br_m_tk = Event("BR_MISP_RETIRED.NEAR_TAKEN",
"BR_MISP_RETIRED.TAKEN_JCC",
"BR_INST_RETIRED.MISPRED_TAKEN")
- except:
+ except ValueError:
pass
br_r = d_ratio(br_all, interval_sec)
ins_r = d_ratio(ins, br_all)
@@ -199,7 +199,7 @@ def IntelBr():
br_m_cond = Event("BR_MISP_RETIRED.COND",
"BR_MISP_RETIRED.CONDITIONAL",
"BR_MISP_RETIRED.TAKEN_JCC")
- except:
+ except ValueError:
return None
br_cond_nt = None
@@ -207,7 +207,7 @@ def IntelBr():
try:
br_cond_nt = Event("BR_INST_RETIRED.COND_NTAKEN")
br_m_cond_nt = Event("BR_MISP_RETIRED.COND_NTAKEN")
- except:
+ except ValueError:
pass
br_r = d_ratio(br_cond, interval_sec)
ins_r = d_ratio(ins, br_cond)
@@ -222,7 +222,7 @@ def IntelBr():
"Retired conditional branch instructions mispredicted as a "
"percentage of all conditional branches.", misp_r, "100%"),
]
- if not br_m_cond_nt:
+ if not br_m_cond_nt or not br_cond_nt:
return MetricGroup("lpm_br_cond", taken_metrics)
br_r = d_ratio(br_cond_nt, interval_sec)
@@ -247,7 +247,7 @@ def IntelBr():
def Far() -> Optional[MetricGroup]:
try:
br_far = Event("BR_INST_RETIRED.FAR_BRANCH")
- except:
+ except ValueError:
return None
br_r = d_ratio(br_far, interval_sec)
@@ -284,7 +284,7 @@ def IntelCtxSw() -> MetricGroup:
ev = Event("MEM_INST_RETIRED.ALL_LOADS", "MEM_UOPS_RETIRED.ALL_LOADS")
metrics.append(Metric("lpm_cs_loads", "Loads per context switch",
d_ratio(ev, cs), "loads/cs"))
- except:
+ except ValueError:
pass
try:
@@ -292,14 +292,14 @@ def IntelCtxSw() -> MetricGroup:
"MEM_UOPS_RETIRED.ALL_STORES")
metrics.append(Metric("lpm_cs_stores", "Stores per context switch",
d_ratio(ev, cs), "stores/cs"))
- except:
+ except ValueError:
pass
try:
ev = Event("BR_INST_RETIRED.NEAR_TAKEN", "BR_INST_RETIRED.TAKEN_JCC")
metrics.append(Metric("lpm_cs_br_taken", "Branches taken per context switch",
d_ratio(ev, cs), "br_taken/cs"))
- except:
+ except ValueError:
pass
try:
@@ -309,12 +309,12 @@ def IntelCtxSw() -> MetricGroup:
try:
l2_misses += Event("L2_RQSTS.HWPF_MISS",
"L2_RQSTS.L2_PF_MISS", "L2_RQSTS.PF_MISS")
- except:
+ except ValueError:
pass
metrics.append(Metric("lpm_cs_l2_misses", "L2 misses per context switch",
d_ratio(l2_misses, cs), "l2_misses/cs"))
- except:
+ except ValueError:
pass
return MetricGroup("lpm_cs", metrics,
@@ -327,7 +327,7 @@ def IntelFpu() -> Optional[MetricGroup]:
try:
s_64 = Event("FP_ARITH_INST_RETIRED.SCALAR_SINGLE",
"SIMD_INST_RETIRED.SCALAR_SINGLE")
- except:
+ except ValueError:
return None
d_64 = Event("FP_ARITH_INST_RETIRED.SCALAR_DOUBLE",
"SIMD_INST_RETIRED.SCALAR_DOUBLE")
@@ -352,21 +352,21 @@ def IntelFpu() -> Optional[MetricGroup]:
flop += 16 * s_512
d_512 = Event("FP_ARITH_INST_RETIRED.512B_PACKED_DOUBLE")
flop += 8 * d_512
- except:
+ except ValueError:
pass
f_assist = Event("ASSISTS.FP", "FP_ASSIST.ANY", "FP_ASSIST.S")
- if f_assist in [
+ nmi_constraint = MetricConstraint.GROUPED_EVENTS
+ if f_assist.name == "ASSISTS.FP": # Icelake+
+ nmi_constraint = MetricConstraint.NO_GROUP_EVENTS_NMI
+ if f_assist.name in [
"ASSISTS.FP",
"FP_ASSIST.S",
]:
- f_assist += "/cmask=1/"
+ f_assist.name += "/cmask=1/"
flop_r = d_ratio(flop, interval_sec)
flop_c = d_ratio(flop, cyc)
- nmi_constraint = MetricConstraint.GROUPED_EVENTS
- if f_assist.name == "ASSISTS.FP": # Icelake+
- nmi_constraint = MetricConstraint.NO_GROUP_EVENTS_NMI
def FpuMetrics(group: str, fl: Optional[Event], mult: int, desc: str) -> Optional[MetricGroup]:
if not fl:
@@ -421,16 +421,17 @@ def IntelFpu() -> Optional[MetricGroup]:
def IntelIlp() -> MetricGroup:
tsc = Event("msr/tsc/")
c0 = Event("msr/mperf/")
- low = tsc - c0
+ low = max(tsc - c0, 0)
inst_ret = Event("INST_RETIRED.ANY_P")
- inst_ret_c = [Event(f"{inst_ret.name}/cmask={x}/") for x in range(1, 6)]
core_cycles = Event("CPU_CLK_UNHALTED.THREAD_P_ANY",
"CPU_CLK_UNHALTED.DISTRIBUTED",
"cycles")
+ inst_ret_c = [Event(f"{inst_ret.name}/cmask={x}/") for x in range(1, 6)]
+
ilp = [d_ratio(max(inst_ret_c[x] - inst_ret_c[x + 1], 0), core_cycles)
for x in range(0, 4)]
ilp.append(d_ratio(inst_ret_c[4], core_cycles))
- ilp0 = 1
+ ilp0: Expression = Constant(1)
for x in ilp:
ilp0 -= x
return MetricGroup("lpm_ilp", [
@@ -465,7 +466,7 @@ def IntelIotlb() -> Optional[MetricGroup]:
+ Event("UNC_IIO_IOMMU0.1G_HITS")
)
total_miss = Event("UNC_IIO_IOMMU0.MISSES")
- except:
+ except ValueError:
return None
miss_rate = d_ratio(total_miss, total_miss + total_hit)
@@ -508,7 +509,7 @@ def IntelIotlb() -> Optional[MetricGroup]:
"100%",
),
]
- except:
+ except ValueError:
pass
return MetricGroup(
@@ -519,59 +520,92 @@ def IntelIotlb() -> Optional[MetricGroup]:
def IntelL2() -> Optional[MetricGroup]:
+ assert _args is not None
try:
DC_HIT = Event("L2_RQSTS.DEMAND_DATA_RD_HIT")
- except:
+ except ValueError:
return None
try:
DC_MISS = Event("L2_RQSTS.DEMAND_DATA_RD_MISS")
- l2_dmnd_miss = DC_MISS
- l2_dmnd_rd_all = DC_MISS + DC_HIT
- except:
+ l2_dmnd_miss: Expression = DC_MISS
+ l2_dmnd_rd_all: Expression = DC_MISS + DC_HIT
+ except ValueError:
DC_ALL = Event("L2_RQSTS.ALL_DEMAND_DATA_RD")
l2_dmnd_miss = DC_ALL - DC_HIT
l2_dmnd_rd_all = DC_ALL
l2_dmnd_mrate = d_ratio(l2_dmnd_miss, interval_sec)
l2_dmnd_rrate = d_ratio(l2_dmnd_rd_all, interval_sec)
- DC_PFH = None
- DC_PFM = None
- l2_pf_all = None
- l2_pf_mrate = None
- l2_pf_rrate = None
+ l2_useless_rate = None
+ try:
+ DC_OUT_U = Event("L2_LINES_OUT.USELESS_HWPF")
+ l2_pf_useless = DC_OUT_U
+ l2_useless_rate = d_ratio(l2_pf_useless, interval_sec)
+ except ValueError:
+ pass
+
+ hwpf_group = None
try:
DC_PFH = Event("L2_RQSTS.PF_HIT")
DC_PFM = Event("L2_RQSTS.PF_MISS")
l2_pf_all = DC_PFH + DC_PFM
l2_pf_mrate = d_ratio(DC_PFM, interval_sec)
l2_pf_rrate = d_ratio(l2_pf_all, interval_sec)
- except:
+ hwpf_group = MetricGroup("lpm_l2_hwpf", [
+ Metric("lpm_l2_hwpf_hits", "L2 cache hardware prefetcher hits",
+ d_ratio(DC_PFH, l2_pf_all), "100%"),
+ Metric("lpm_l2_hwpf_misses", "L2 cache hardware prefetcher misses",
+ d_ratio(DC_PFM, l2_pf_all), "100%"),
+ Metric("lpm_l2_hwpf_useless", "L2 cache hardware prefetcher useless prefetches per second",
+ l2_useless_rate, "100%") if l2_useless_rate else None,
+ Metric("lpm_l2_hwpf_requests", "L2 cache hardware prefetcher requests per second",
+ l2_pf_rrate, "100%"),
+ Metric("lpm_l2_hwpf_misses", "L2 cache hardware prefetcher misses per second",
+ l2_pf_mrate, "100%"),
+ ])
+ except ValueError:
pass
- DC_RFOH = None
- DC_RFOM = None
- l2_rfo_all = None
- l2_rfo_mrate = None
- l2_rfo_rrate = None
+ rfo_group = None
try:
DC_RFOH = Event("L2_RQSTS.RFO_HIT")
DC_RFOM = Event("L2_RQSTS.RFO_MISS")
l2_rfo_all = DC_RFOH + DC_RFOM
l2_rfo_mrate = d_ratio(DC_RFOM, interval_sec)
l2_rfo_rrate = d_ratio(l2_rfo_all, interval_sec)
- except:
+ rfo_group = MetricGroup("lpm_l2_rfo", [
+ Metric("lpm_l2_rfo_hits", "L2 cache request for ownership (RFO) hits",
+ d_ratio(DC_RFOH, l2_rfo_all), "100%"),
+ Metric("lpm_l2_rfo_misses", "L2 cache request for ownership (RFO) misses",
+ d_ratio(DC_RFOM, l2_rfo_all), "100%"),
+ Metric("lpm_l2_rfo_requests", "L2 cache request for ownership (RFO) requests per second",
+ l2_rfo_rrate, "requests/s"),
+ Metric("lpm_l2_rfo_misses", "L2 cache request for ownership (RFO) misses per second",
+ l2_rfo_mrate, "misses/s"),
+ ])
+ except ValueError:
pass
DC_CH = None
try:
DC_CH = Event("L2_RQSTS.CODE_RD_HIT")
- except:
+ except ValueError:
pass
DC_CM = Event("L2_RQSTS.CODE_RD_MISS")
DC_IN = Event("L2_LINES_IN.ALL")
- DC_OUT_NS = None
- DC_OUT_S = None
- l2_lines_out = None
+
+ DC_WB_U = None
+ DC_WB_D = None
+ wbu = None
+ wbd = None
+ try:
+ DC_WB_U = Event("IDI_MISC.WB_UPGRADE")
+ DC_WB_D = Event("IDI_MISC.WB_DOWNGRADE")
+ wbu = d_ratio(DC_WB_U, interval_sec)
+ wbd = d_ratio(DC_WB_D, interval_sec)
+ except ValueError:
+ pass
+
l2_out_rate = None
wbn = None
isd = None
@@ -583,43 +617,24 @@ def IntelL2() -> Optional[MetricGroup]:
"L2_LINES_OUT.DEMAND_CLEAN",
"L2_LINES_IN.I")
if DC_OUT_S.name == "L2_LINES_OUT.SILENT" and (
- args.model.startswith("skylake") or
- args.model == "cascadelakex"):
+ _args.model.startswith("skylake") or
+ _args.model == "cascadelakex"):
DC_OUT_S.name = "L2_LINES_OUT.SILENT/any/"
# bring is back to per-CPU
l2_s = Select(DC_OUT_S / 2, Literal("#smt_on"), DC_OUT_S)
l2_ns = DC_OUT_NS
l2_lines_out = l2_s + l2_ns
l2_out_rate = d_ratio(l2_lines_out, interval_sec)
- nlr = max(l2_ns - DC_WB_U - DC_WB_D, 0)
- wbn = d_ratio(nlr, interval_sec)
+ if DC_WB_U and DC_WB_D:
+ nlr = max(l2_ns - DC_WB_U - DC_WB_D, 0)
+ wbn = d_ratio(nlr, interval_sec)
isd = d_ratio(l2_s, interval_sec)
- except:
- pass
- DC_OUT_U = None
- l2_pf_useless = None
- l2_useless_rate = None
- try:
- DC_OUT_U = Event("L2_LINES_OUT.USELESS_HWPF")
- l2_pf_useless = DC_OUT_U
- l2_useless_rate = d_ratio(l2_pf_useless, interval_sec)
- except:
- pass
- DC_WB_U = None
- DC_WB_D = None
- wbu = None
- wbd = None
- try:
- DC_WB_U = Event("IDI_MISC.WB_UPGRADE")
- DC_WB_D = Event("IDI_MISC.WB_DOWNGRADE")
- wbu = d_ratio(DC_WB_U, interval_sec)
- wbd = d_ratio(DC_WB_D, interval_sec)
- except:
+ except ValueError:
pass
l2_lines_in = DC_IN
l2_code_all = (DC_CH + DC_CM) if DC_CH else None
- l2_code_rate = d_ratio(l2_code_all, interval_sec) if DC_CH else None
+ l2_code_rate = d_ratio(l2_code_all, interval_sec) if l2_code_all else None
l2_code_miss_rate = d_ratio(DC_CM, interval_sec)
l2_in_rate = d_ratio(l2_lines_in, interval_sec)
@@ -640,35 +655,15 @@ def IntelL2() -> Optional[MetricGroup]:
Metric("lpm_l2_rd_misses", "L2 cache data read misses per second",
l2_dmnd_mrate, "misses/s"),
]),
- MetricGroup("lpm_l2_hwpf", [
- Metric("lpm_l2_hwpf_hits", "L2 cache hardware prefetcher hits",
- d_ratio(DC_PFH, l2_pf_all), "100%"),
- Metric("lpm_l2_hwpf_misses", "L2 cache hardware prefetcher misses",
- d_ratio(DC_PFM, l2_pf_all), "100%"),
- Metric("lpm_l2_hwpf_useless", "L2 cache hardware prefetcher useless prefetches per second",
- l2_useless_rate, "100%") if l2_useless_rate else None,
- Metric("lpm_l2_hwpf_requests", "L2 cache hardware prefetcher requests per second",
- l2_pf_rrate, "100%"),
- Metric("lpm_l2_hwpf_misses", "L2 cache hardware prefetcher misses per second",
- l2_pf_mrate, "100%"),
- ]) if DC_PFH else None,
- MetricGroup("lpm_l2_rfo", [
- Metric("lpm_l2_rfo_hits", "L2 cache request for ownership (RFO) hits",
- d_ratio(DC_RFOH, l2_rfo_all), "100%"),
- Metric("lpm_l2_rfo_misses", "L2 cache request for ownership (RFO) misses",
- d_ratio(DC_RFOM, l2_rfo_all), "100%"),
- Metric("lpm_l2_rfo_requests", "L2 cache request for ownership (RFO) requests per second",
- l2_rfo_rrate, "requests/s"),
- Metric("lpm_l2_rfo_misses", "L2 cache request for ownership (RFO) misses per second",
- l2_rfo_mrate, "misses/s"),
- ]) if DC_RFOH else None,
+ hwpf_group,
+ rfo_group,
MetricGroup("lpm_l2_code", [
Metric("lpm_l2_code_hits", "L2 cache code hits",
- d_ratio(DC_CH, l2_code_all), "100%") if DC_CH else None,
+ d_ratio(DC_CH, l2_code_all), "100%") if DC_CH and l2_code_all else None,
Metric("lpm_l2_code_misses", "L2 cache code misses",
- d_ratio(DC_CM, l2_code_all), "100%") if DC_CH else None,
+ d_ratio(DC_CM, l2_code_all), "100%") if DC_CH and l2_code_all else None,
Metric("lpm_l2_code_requests", "L2 cache code requests per second",
- l2_code_rate, "requests/s") if DC_CH else None,
+ l2_code_rate, "requests/s") if l2_code_rate else None,
Metric("lpm_l2_code_misses", "L2 cache code misses per second",
l2_code_miss_rate, "misses/s"),
]),
@@ -706,7 +701,7 @@ def IntelMissLat() -> Optional[MetricGroup]:
"UNC_CHA_TOR_INSERTS.IA_MISS",
"UNC_C_TOR_INSERTS.MISS_REMOTE_OPCODE",
"UNC_C_TOR_INSERTS.NID_MISS_OPCODE")
- except:
+ except ValueError:
return None
if (data_rd_loc_occ.name == "UNC_C_TOR_OCCUPANCY.MISS_LOCAL_OPCODE" or
@@ -752,8 +747,8 @@ def IntelMissLat() -> Optional[MetricGroup]:
def IntelMlp() -> Optional[Metric]:
try:
l1d = Event("L1D_PEND_MISS.PENDING")
- l1dc = Event("L1D_PEND_MISS.PENDING_CYCLES")
- except:
+ l1dc: Expression = Event("L1D_PEND_MISS.PENDING_CYCLES")
+ except ValueError:
return None
l1dc = Select(l1dc / 2, Literal("#smt_on"), l1dc)
@@ -764,8 +759,9 @@ def IntelMlp() -> Optional[Metric]:
def IntelPorts() -> Optional[MetricGroup]:
- pipeline_events = json.load(
- open(f"{_args.events_path}/x86/{_args.model}/pipeline.json"))
+ assert _args is not None
+ with open(f"{_args.events_path}/x86/{_args.model}/pipeline.json", encoding="utf-8") as f:
+ pipeline_events = json.load(f)
core_cycles = Event("CPU_CLK_UNHALTED.THREAD_P_ANY",
"CPU_CLK_UNHALTED.DISTRIBUTED",
@@ -777,11 +773,10 @@ def IntelPorts() -> Optional[MetricGroup]:
for x in pipeline_events:
if "EventName" in x and re.search("^UOPS_DISPATCHED.PORT", x["EventName"]):
name = x["EventName"]
- port = re.search(r"(PORT_[0-9].*)", name).group(0).lower()
- if name.endswith("_CORE"):
- cyc = core_cycles
- else:
- cyc = smt_cycles
+ match = re.search(r"(PORT_[0-9].*)", name)
+ assert match is not None
+ port = match.group(0).lower()
+ cyc: Expression = core_cycles if name.endswith("_CORE") else smt_cycles
metrics.append(Metric(f"lpm_{port}", f"{port} utilization (higher is better)",
d_ratio(Event(name), cyc), "100%"))
if len(metrics) == 0:
@@ -800,7 +795,7 @@ def IntelSwpf() -> Optional[MetricGroup]:
s_t0 = Event("SW_PREFETCH_ACCESS.T0")
s_t1 = Event("SW_PREFETCH_ACCESS.T1_T2")
s_w = Event("SW_PREFETCH_ACCESS.PREFETCHW")
- except:
+ except ValueError:
return None
all_sw = s_nta + s_t0 + s_t1 + s_w
@@ -857,6 +852,7 @@ def IntelSwpf() -> Optional[MetricGroup]:
def IntelLdSt() -> Optional[MetricGroup]:
+ assert _args is not None
if _args.model in [
"bonnell",
"nehalemep",
@@ -882,12 +878,12 @@ def IntelLdSt() -> Optional[MetricGroup]:
LDST_PRE = None
try:
LDST_PRE = Event("LOAD_HIT_PREFETCH.SWPF", "LOAD_HIT_PRE.SW_PF")
- except:
+ except ValueError:
pass
LDST_AT = None
try:
LDST_AT = Event("MEM_INST_RETIRED.LOCK_LOADS")
- except:
+ except ValueError:
pass
cyc = LDST_CYC
@@ -945,8 +941,8 @@ def UncoreCState() -> Optional[MetricGroup]:
pcu_ticks = Event("UNC_P_CLOCKTICKS")
c0 = Event("UNC_P_POWER_STATE_OCCUPANCY.CORES_C0")
c3 = Event("UNC_P_POWER_STATE_OCCUPANCY.CORES_C3")
- c6 = Event("UNC_P_POWER_STATE_OCCUPANCY.CORES_C6")
- except:
+ c6: Expression = Event("UNC_P_POWER_STATE_OCCUPANCY.CORES_C6")
+ except ValueError:
return None
num_cores = Literal("#num_cores") / Literal("#num_packages")
@@ -981,13 +977,10 @@ def UncoreDir() -> Optional[MetricGroup]:
cha_upd = Event("UNC_CHA_DIR_UPDATE.HA")
# Turn the umask into a ANY rather than HA filter.
cha_upd.name += "/umask=3,name=UNC_CHA_DIR_UPDATE.ANY/"
- except:
+ except ValueError:
return None
m2m_total = m2m_hits + m2m_miss
- upd = m2m_upd + cha_upd # in cache lines
- upd_r = upd / interval_sec
- look_r = m2m_total / interval_sec
scale = 64 / 1_000_000 # Cache lines to MB
return MetricGroup("lpm_dir", [
@@ -1014,7 +1007,7 @@ def UncoreMem() -> Optional[MetricGroup]:
"UNC_H_REQUESTS.WRITES_LOCAL")
rem_wrs = Event("UNC_CHA_REQUESTS.WRITES_REMOTE",
"UNC_H_REQUESTS.WRITES_REMOTE")
- except:
+ except ValueError:
return None
scale = 64 / 1_000_000
@@ -1035,16 +1028,18 @@ def UncoreMem() -> Optional[MetricGroup]:
def UncoreMemBw() -> Optional[MetricGroup]:
+ assert _args is not None
mem_events = []
try:
- mem_events = json.load(open(f"{os.path.dirname(os.path.realpath(__file__))}"
- f"/arch/x86/{args.model}/uncore-memory.json"))
- except:
+ with open(f"{os.path.dirname(os.path.realpath(__file__))}"
+ f"/arch/x86/{_args.model}/uncore-memory.json", encoding="utf-8") as f:
+ mem_events = json.load(f)
+ except (OSError, ValueError):
pass
- ddr_rds = 0
- ddr_wrs = 0
- ddr_total = 0
+ ddr_rds: Union[int, Expression] = 0
+ ddr_wrs: Union[int, Expression] = 0
+ ddr_total: Union[int, Expression] = 0
for x in mem_events:
if "EventName" in x:
name = x["EventName"]
@@ -1059,17 +1054,17 @@ def UncoreMemBw() -> Optional[MetricGroup]:
try:
ddr_rds = Event("UNC_M_CAS_COUNT.RD")
ddr_wrs = Event("UNC_M_CAS_COUNT.WR")
- except:
+ except ValueError:
return None
ddr_total = ddr_rds + ddr_wrs
- pmm_rds = 0
- pmm_wrs = 0
+ pmm_rds: Union[int, Expression] = 0
+ pmm_wrs: Union[int, Expression] = 0
try:
pmm_rds = Event("UNC_M_PMM_RPQ_INSERTS")
pmm_wrs = Event("UNC_M_PMM_WPQ_INSERTS")
- except:
+ except ValueError:
pass
pmm_total = pmm_rds + pmm_wrs
@@ -1101,7 +1096,7 @@ def UncoreMemSat() -> Optional[Metric]:
sat = Event("UNC_CHA_DISTRESS_ASSERTED.VERT", "UNC_CHA_FAST_ASSERTED.VERT",
"UNC_C_FAST_ASSERTED", "UNC_CHA_DISTRESS_ASSERTED.DPT_ANY",
"UNC_CHA_DISTRESS_ASSERTED.DPT_NONLOCAL")
- except:
+ except ValueError:
return None
desc = ("Mesh Bandwidth saturation (% CBOX cycles with FAST signal asserted, "
@@ -1116,11 +1111,9 @@ def UncoreUpiBw() -> Optional[MetricGroup]:
try:
upi_rds = Event("UNC_UPI_RxL_FLITS.ALL_DATA")
upi_wrs = Event("UNC_UPI_TxL_FLITS.ALL_DATA")
- except:
+ except ValueError:
return None
- upi_total = upi_rds + upi_wrs
-
# From "Uncore Performance Monitoring": When measuring the amount of
# bandwidth consumed by transmission of the data (i.e. NOT including
# the header), it should be .ALL_DATA / 9 * 64B.
diff --git a/tools/perf/pmu-events/jevents.py b/tools/perf/pmu-events/jevents.py
index 860027bb71b2..e853a06cbade 100755
--- a/tools/perf/pmu-events/jevents.py
+++ b/tools/perf/pmu-events/jevents.py
@@ -3,7 +3,6 @@
"""Convert directories of JSON events to C code."""
import argparse
import csv
-from functools import lru_cache
import json
import metric
import os
@@ -22,7 +21,7 @@ _metric_tables: list[str] = []
# List of metric tables generated from "/sys" directories.
_sys_metric_tables: list[str] = []
# Mapping between sys event table names and sys metric table names.
-_sys_event_table_to_metric_table_mapping = {}
+_sys_event_table_to_metric_table_mapping: Dict[str, str] = {}
# Map from an event name to an architecture standard
# JsonEvent. Architecture standard events are in json files in the top
# f'{_args.starting_dir}/{_args.arch}' directory.
@@ -38,7 +37,7 @@ _pending_metrics_tblname: Optional[str] = None
# Global BigCString shared by all structures.
_bcs = None
# Map from the name of a metric group to a description of the group.
-_metricgroups = {}
+_metricgroups: Dict[str, str] = {}
# Order specific JsonEvent attributes will be visited.
_json_event_attributes = [
# cmp_sevent related attributes.
@@ -124,17 +123,17 @@ class BigCString:
def __init__(self):
self.strings = set()
- self.insert_number = 0;
+ self.insert_number = 0
self.insert_point = {}
self.metrics = set()
- def add(self, s: str, metric: bool) -> None:
+ def add(self, s: str, is_metric: bool) -> None:
"""Called to add to the big string."""
if s not in self.strings:
self.strings.add(s)
self.insert_point[s] = self.insert_number
self.insert_number += 1
- if metric:
+ if is_metric:
self.metrics.add(s)
def compute(self) -> None:
@@ -313,7 +312,7 @@ class JsonEvent:
return int(val, 16) == 0
else:
return int(val) == 0
- except:
+ except ValueError:
return False
def canonicalize_value(val: str) -> str:
@@ -321,7 +320,7 @@ class JsonEvent:
if val.startswith('0x'):
return llx(int(val, 16))
return str(int(val))
- except:
+ except ValueError:
return val
eventcode = 0
@@ -436,15 +435,15 @@ class JsonEvent:
s += f'\t{attr} = {value},\n'
return s + '}'
- def build_c_string(self, metric: bool) -> str:
+ def build_c_string(self, is_metric: bool) -> str:
s = ''
- for attr in _json_metric_attributes if metric else _json_event_attributes:
+ for attr in _json_metric_attributes if is_metric else _json_event_attributes:
x = getattr(self, attr)
- if metric and x and attr == 'metric_expr':
+ if is_metric and x and attr == 'metric_expr':
# Convert parsed metric expressions into a string. Slashes
# must be doubled in the file.
x = x.ToPerfJson().replace('\\', '\\\\')
- if metric and x and attr == 'metric_threshold':
+ if is_metric and x and attr == 'metric_threshold':
x = x.replace('\\', '\\\\')
if attr in _json_enum_attributes:
s += x if x else '0'
@@ -452,24 +451,25 @@ class JsonEvent:
s += f'{x}\\000' if x else '\\000'
return s
- def to_c_string(self, metric: bool) -> str:
+ def to_c_string(self, is_metric: bool) -> str:
"""Representation of the event as a C struct initializer."""
def make_comment(s: str) -> str:
s = s.replace('*/', r'\*\/')
return f'\t/* {s} */\n' if len(s) < 80 else f'\t/* {s[0:80]}... */\n'
- s = self.build_c_string(metric)
+ s = self.build_c_string(is_metric)
assert _bcs is not None
return f'{make_comment(s)}\t{{ { _bcs.offsets[s] } }},\n'
-_json_cache = {}
+_json_cache: Dict[Tuple[str, str], Sequence[JsonEvent]] = {}
def _read_json_events_impl(path: str, topic: str) -> Sequence[JsonEvent]:
"""Read json events from the specified file."""
try:
- events = json.load(open(path), object_hook=JsonEvent)
- except BaseException as err:
+ with open(path, encoding='utf-8') as f:
+ events = json.load(f, object_hook=JsonEvent)
+ except BaseException:
print(f"Exception processing {path}")
raise
metrics: list[Tuple[str, str, metric.Expression]] = []
@@ -493,7 +493,6 @@ def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]:
def preprocess_arch_std_files(archpath: str) -> None:
"""Read in all architecture standard events."""
- global _arch_std_events
for item in os.scandir(archpath):
if not item.is_file() or not item.name.endswith('.json'):
continue
@@ -533,13 +532,10 @@ def print_pending_events() -> None:
if not _pending_events:
return
- global _pending_events_tblname
assert _pending_events_tblname is not None
if _pending_events_tblname.endswith('_sys'):
- global _sys_event_tables
_sys_event_tables.append(_pending_events_tblname)
else:
- global event_tables
_event_tables.append(_pending_events_tblname)
first = True
@@ -560,7 +556,7 @@ def print_pending_events() -> None:
last_pmu = event.pmu
pmus.add((event.pmu, pmu_name))
- _args.output_file.write(event.to_c_string(metric=False))
+ _args.output_file.write(event.to_c_string(is_metric=False))
last_name = event.name
_pending_events = []
@@ -596,31 +592,28 @@ def print_pending_metrics() -> None:
if not _pending_metrics:
return
- global _pending_metrics_tblname
assert _pending_metrics_tblname is not None
if _pending_metrics_tblname.endswith('_sys'):
- global _sys_metric_tables
_sys_metric_tables.append(_pending_metrics_tblname)
else:
- global metric_tables
_metric_tables.append(_pending_metrics_tblname)
first = True
last_pmu = None
pmus: Set[Tuple[str, str]] = set()
assert _args is not None
- for metric in sorted(_pending_metrics, key=metric_cmp_key):
- if metric.pmu != last_pmu:
+ for m in sorted(_pending_metrics, key=metric_cmp_key):
+ if m.pmu != last_pmu:
if not first:
_args.output_file.write('};\n')
- pmu_name = metric.pmu.replace(',', '_')
+ pmu_name = m.pmu.replace(',', '_')
_args.output_file.write(
f'static const struct compact_pmu_event {_pending_metrics_tblname}_{pmu_name}[] = {{\n')
first = False
- last_pmu = metric.pmu
- pmus.add((metric.pmu, pmu_name))
+ last_pmu = m.pmu
+ pmus.add((m.pmu, pmu_name))
- _args.output_file.write(metric.to_c_string(metric=True))
+ _args.output_file.write(m.to_c_string(is_metric=True))
_pending_metrics = []
_args.output_file.write(f"""
@@ -659,13 +652,14 @@ def preprocess_one_file(parents: Sequence[str], item: os.DirEntry) -> None:
assert _bcs is not None
if item.name.endswith('metricgroups.json'):
- metricgroup_descriptions = json.load(open(item.path))
+ with open(item.path, encoding='utf-8') as f:
+ metricgroup_descriptions = json.load(f)
for mgroup in metricgroup_descriptions:
assert len(mgroup) > 1, parents
description = f"{metricgroup_descriptions[mgroup]}\\000"
mgroup = f"{mgroup}\\000"
- _bcs.add(mgroup, metric=True)
- _bcs.add(description, metric=True)
+ _bcs.add(mgroup, is_metric=True)
+ _bcs.add(description, is_metric=True)
_metricgroups[mgroup] = description
return
@@ -673,11 +667,12 @@ def preprocess_one_file(parents: Sequence[str], item: os.DirEntry) -> None:
for event in read_json_events(item.path, topic):
pmu_name = f"{event.pmu}\\000"
if event.name:
- _bcs.add(pmu_name, metric=False)
- _bcs.add(event.build_c_string(metric=False), metric=False)
+ _bcs.add(pmu_name, is_metric=False)
+ _bcs.add(event.build_c_string(is_metric=False), is_metric=False)
if event.metric_name:
- _bcs.add(pmu_name, metric=True)
- _bcs.add(event.build_c_string(metric=True), metric=True)
+ pmu_name = f"{event.pmu}\\000"
+ _bcs.add(pmu_name, is_metric=True)
+ _bcs.add(event.build_c_string(is_metric=True), is_metric=True)
def process_one_file(parents: Sequence[str], item: os.DirEntry) -> None:
"""Process a JSON file during the main walk."""
@@ -787,7 +782,7 @@ static const struct pmu_events_map pmu_events_map[] = {
},
""")
else:
- with open(f'{_args.starting_dir}/{arch}/mapfile.csv') as csvfile:
+ with open(f'{_args.starting_dir}/{arch}/mapfile.csv', encoding='utf-8') as csvfile:
table = csv.reader(csvfile)
first = True
for row in table:
@@ -881,8 +876,8 @@ int pmu_metrics_table__iterate_tables(pmu_metrics_table_iter_t fn, void *data)
def print_system_mapping_table() -> None:
- assert _args is not None
"""C struct mapping table array for tables from /sys directories."""
+ assert _args is not None
_args.output_file.write("""
struct pmu_sys_events {
\tconst char *name;
@@ -1484,7 +1479,7 @@ def main() -> None:
except Exception as e:
raise RuntimeError(f'Action failure for \'{item.name}\' in {parents}') from e
if item.is_dir():
- ftw(item.path, parents + [item.name], action)
+ ftw(item.path, list(parents) + [item.name], action)
ap = argparse.ArgumentParser()
ap.add_argument('arch', help='Architecture name like x86')
diff --git a/tools/perf/pmu-events/make_legacy_cache.py b/tools/perf/pmu-events/make_legacy_cache.py
index 28a1ff804f86..b224544f85f6 100755
--- a/tools/perf/pmu-events/make_legacy_cache.py
+++ b/tools/perf/pmu-events/make_legacy_cache.py
@@ -61,7 +61,7 @@ hw_cache_result = [
"misses"),
]
-events = []
+events: list[dict[str, str]] = []
def add_event(name: str,
cache_id: int, cache_op: int, cache_result: int,
desc: str,
@@ -87,10 +87,10 @@ def add_event(name: str,
event["Deprecated"] = "1"
events.append(event)
-for (cache_id, names, ops, cache_desc) in hw_cache_id:
- for name in names:
- add_event(name,
- cache_id,
+for (cid, names, ops, cache_desc) in hw_cache_id:
+ for cname in names:
+ add_event(cname,
+ cid,
0, # PERF_COUNT_HW_CACHE_OP_READ
0, # PERF_COUNT_HW_CACHE_RESULT_ACCESS
f"{cache_desc} read accesses.",
@@ -100,27 +100,29 @@ for (cache_id, names, ops, cache_desc) in hw_cache_id:
if op not in ops:
continue
for op_name in op_names:
- deprecated = (names[0] != name or op_names[1] != op_name)
- add_event(f"{name}-{op_name}",
- cache_id,
+ is_deprecated = (names[0] != cname or op_names[1] != op_name)
+ add_event(f"{cname}-{op_name}",
+ cid,
op,
0, # PERF_COUNT_HW_CACHE_RESULT_ACCESS
f"{cache_desc} {op_desc} accesses.",
- deprecated)
+ is_deprecated)
for (result, result_names, result_desc) in hw_cache_result:
for result_name in result_names:
- deprecated = ((names[0] != name or op_names[0] != op_name) or
- (result == 0) or (result_names[0] != result_name))
- add_event(f"{name}-{op_name}-{result_name}",
- cache_id, op, result,
+ is_deprecated = ((names[0] != cname or op_names[0] != op_name) or
+ (result == 0) or (result_names[0] != result_name))
+ add_event(f"{cname}-{op_name}-{result_name}",
+ cid,
+ op,
+ result,
f"{cache_desc} {op_desc} {result_desc}.",
- deprecated)
+ is_deprecated)
for (result, result_names, result_desc) in hw_cache_result:
for result_name in result_names:
- add_event(f"{name}-{result_name}",
- cache_id,
+ add_event(f"{cname}-{result_name}",
+ cid,
0, # PERF_COUNT_HW_CACHE_OP_READ
result,
f"{cache_desc} read {result_desc}.",
diff --git a/tools/perf/pmu-events/metric.py b/tools/perf/pmu-events/metric.py
index ce025675898c..92144c5d8024 100644
--- a/tools/perf/pmu-events/metric.py
+++ b/tools/perf/pmu-events/metric.py
@@ -6,18 +6,16 @@ import json
import os
import re
from enum import Enum
-from typing import Dict, List, Optional, Set, Tuple, Union
+from typing import Dict, List, Optional, Sequence, Set, Tuple, Union
-all_pmus = set()
-all_events = set()
-experimental_events = set()
-all_events_all_models = set()
+all_pmus: Set[str] = set()
+all_events: Set[str] = set()
+experimental_events: Set[str] = set()
+all_events_all_models: Set[str] = set()
def LoadEvents(directory: str) -> None:
"""Populate a global set of all known events for the purpose of validating Event names"""
- global all_pmus
global all_events
- global experimental_events
global all_events_all_models
all_events = {
"context\\-switches",
@@ -30,29 +28,31 @@ def LoadEvents(directory: str) -> None:
filename = os.fsdecode(file)
if filename.endswith(".json"):
try:
- for x in json.load(open(f"{directory}/{filename}")):
- if "Unit" in x:
- all_pmus.add(x["Unit"])
- if "EventName" in x:
- all_events.add(x["EventName"])
- if "Experimental" in x and x["Experimental"] == "1":
- experimental_events.add(x["EventName"])
- elif "ArchStdEvent" in x:
- all_events.add(x["ArchStdEvent"])
+ with open(f"{directory}/{filename}", encoding="utf-8") as f:
+ for x in json.load(f):
+ if "Unit" in x:
+ all_pmus.add(x["Unit"])
+ if "EventName" in x:
+ all_events.add(x["EventName"])
+ if "Experimental" in x and x["Experimental"] == "1":
+ experimental_events.add(x["EventName"])
+ elif "ArchStdEvent" in x:
+ all_events.add(x["ArchStdEvent"])
except json.decoder.JSONDecodeError:
# The generated directory may be the same as the input, which
# causes partial json files. Ignore errors.
pass
all_events_all_models = all_events.copy()
- for root, dirs, files in os.walk(directory + ".."):
+ for root, _, files in os.walk(directory + ".."):
for filename in files:
if filename.endswith(".json"):
try:
- for x in json.load(open(f"{root}/{filename}")):
- if "EventName" in x:
- all_events_all_models.add(x["EventName"])
- elif "ArchStdEvent" in x:
- all_events_all_models.add(x["ArchStdEvent"])
+ with open(f"{root}/{filename}", encoding="utf-8") as f:
+ for x in json.load(f):
+ if "EventName" in x:
+ all_events_all_models.add(x["EventName"])
+ elif "ArchStdEvent" in x:
+ all_events_all_models.add(x["ArchStdEvent"])
except json.decoder.JSONDecodeError:
# The generated directory may be the same as the input, which
# causes partial json files. Ignore errors.
@@ -65,7 +65,6 @@ def CheckPmu(name: str) -> bool:
def CheckEvent(name: str) -> bool:
"""Check the event name exists in the set of all loaded events"""
- global all_events
if len(all_events) == 0:
# No events loaded so assume any event is good.
return True
@@ -81,7 +80,6 @@ def CheckEvent(name: str) -> bool:
def CheckEveryEvent(*names: str) -> None:
"""Check all the events exist in at least one json file"""
- global all_events_all_models
if len(all_events_all_models) == 0:
assert len(names) == 1, f"Cannot determine valid events in {names}"
# No events loaded so assume any event is good.
@@ -96,11 +94,10 @@ def CheckEveryEvent(*names: str) -> None:
if any([name.startswith(x) for x in ['amd', 'arm', 'cpu', 'msr', 'power', 'cha', 'uncore']]):
continue
if name not in all_events_all_models:
- raise Exception(f"Is {name} a named json event?")
+ raise ValueError(f"Is {name} a named json event?")
def IsExperimentalEvent(name: str) -> bool:
- global experimental_events
if ':' in name:
# Remove trailing modifier.
name = name[:name.find(':')]
@@ -447,8 +444,7 @@ class Event(Expression):
error += " or " + name
else:
error = name
- global all_events
- raise Exception(f"No event {error} in:\n{all_events}")
+ raise ValueError(f"No event {error} in:\n{all_events}")
def HasExperimentalEvents(self) -> bool:
return IsExperimentalEvent(self.name)
@@ -655,7 +651,7 @@ class Metric:
return result
- def ToMetricGroupDescriptions(self, root: bool = True) -> Dict[str, str]:
+ def ToMetricGroupDescriptions(self, _root: bool = True) -> Dict[str, str]:
return {}
class MetricGroup:
@@ -667,7 +663,7 @@ class MetricGroup:
"""
def __init__(self, name: str,
- metric_list: List[Union[Optional[Metric], Optional['MetricGroup']]],
+ metric_list: Sequence[Union[Optional[Metric], Optional['MetricGroup']]],
description: Optional[str] = None):
self.name = name
self.metric_list = []
@@ -696,7 +692,7 @@ class MetricGroup:
result.append(x.ToPerfJson())
return result
- def ToMetricGroupDescriptions(self, root: bool = True) -> Dict[str, str]:
+ def ToMetricGroupDescriptions(self, _root: bool = True) -> Dict[str, str]:
result = {self.name: self.description} if self.description else {}
for x in self.metric_list:
result.update(x.ToMetricGroupDescriptions(False))
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 12/49] perf test: Clean up mypy and pylint issues in shell test libraries
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (10 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 11/49] perf pmu-events: Clean up mypy and pylint issues Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 13/49] perf build: Make mypy build test opt-out (NO_MYPY=1) Ian Rogers
` (36 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Clean up mypy type errors and pylint errors/warnings in
tools/perf/tests/shell/lib/ (attr.py, perf_json_output_lint.py, and
perf_metric_validation.py):
- Fix E0606 (possibly-used-before-assignment) in perf_metric_validation.py
by importing sys at module level.
- Add type annotations for stack, second_results, collectlist,
get_bounds(),
and main() in perf_metric_validation.py.
- Avoid assigning -1 to list[int] expected_items in
perf_json_output_lint.py,
rename shadowing variables, and remove redundant lambdas.
- Remove unnecessary semicolons, use lazy logging arguments, specify
exception types and file encodings, and initialize module-level logger
in attr.py.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/tests/shell/lib/attr.py | 86 ++++++++++---------
.../tests/shell/lib/perf_json_output_lint.py | 36 ++++----
.../tests/shell/lib/perf_metric_validation.py | 47 +++++-----
3 files changed, 85 insertions(+), 84 deletions(-)
diff --git a/tools/perf/tests/shell/lib/attr.py b/tools/perf/tests/shell/lib/attr.py
index bfccc727d9b2..7f3d5b64b00d 100644
--- a/tools/perf/tests/shell/lib/attr.py
+++ b/tools/perf/tests/shell/lib/attr.py
@@ -12,6 +12,8 @@ import re
import shutil
import subprocess
+log = logging.getLogger('test')
+
def data_equal(a, b):
# Allow multiple values in assignment separated by '|'
a_list = a.split('|')
@@ -89,19 +91,19 @@ class Event(dict):
def add(self, data):
for key, val in data:
- log.debug(" %s = %s" % (key, val))
+ log.debug(" %s = %s", key, val)
self[key] = val
def __init__(self, name, data, base):
- log.debug(" Event %s" % name);
- self.name = name;
+ log.debug(" Event %s", name)
+ self.name = name
self.group = ''
self.add(base)
self.add(data)
def equal(self, other):
for t in Event.terms:
- log.debug(" [%s] %s %s" % (t, self[t], other[t]));
+ log.debug(" [%s] %s %s", t, self[t], other[t])
if t not in self or t not in other:
return False
if not data_equal(self[t], other[t]):
@@ -118,7 +120,7 @@ class Event(dict):
if t not in self or t not in other:
continue
if not data_equal(self[t], other[t]):
- log.warning("expected %s=%s, got %s" % (t, self[t], other[t]))
+ log.warning("expected %s=%s, got %s", t, self[t], other[t])
def parse_version(version):
if not version:
@@ -149,7 +151,7 @@ class Test(object):
parser = configparser.ConfigParser()
parser.read(path)
- log.warning("running '%s'" % path)
+ log.warning("running '%s'", path)
self.path = path
self.test_dir = options.test_dir
@@ -159,15 +161,15 @@ class Test(object):
try:
self.ret = parser.get('config', 'ret')
- except:
+ except Exception:
self.ret = 0
self.test_ret = parser.getboolean('config', 'test_ret', fallback=False)
try:
self.arch = parser.get('config', 'arch')
- log.warning("test limitation '%s'" % self.arch)
- except:
+ log.warning("test limitation '%s'", self.arch)
+ except Exception:
self.arch = ''
self.auxv = parser.get('config', 'auxv', fallback=None)
@@ -175,7 +177,7 @@ class Test(object):
self.kernel_until = parse_version(parser.get('config', 'kernel_until', fallback=None))
self.expect = {}
self.result = {}
- log.debug(" loading expected events");
+ log.debug(" loading expected events")
self.load_events(path, self.expect)
def is_event(self, name):
@@ -203,7 +205,7 @@ class Test(object):
else:
try:
value = int(items[-1], 0)
- except:
+ except ValueError:
value = items[-1]
return (items[0], value)
@@ -227,7 +229,7 @@ class Test(object):
# Handle negated list such as !s390x,ppc
if arch_list[0][0] == '!':
arch_list[0] = arch_list[0][1:]
- log.warning("excluded architecture list %s" % arch_list)
+ log.warning("excluded architecture list %s", arch_list)
for arch_item in arch_list:
# log.warning("test for %s arch is %s" % (arch_item, myarch))
if arch_item == myarch:
@@ -243,19 +245,19 @@ class Test(object):
def restore_sample_rate(self, value=10000):
try:
# Check value of sample_rate
- with open("/proc/sys/kernel/perf_event_max_sample_rate", "r") as fIn:
+ with open("/proc/sys/kernel/perf_event_max_sample_rate", "r", encoding="utf-8") as fIn:
curr_value = fIn.readline()
# If too low restore to reasonable value
if not curr_value or int(curr_value) < int(value):
- with open("/proc/sys/kernel/perf_event_max_sample_rate", "w") as fOut:
+ with open("/proc/sys/kernel/perf_event_max_sample_rate", "w", encoding="utf-8") as fOut:
fOut.write(str(value))
except IOError as e:
- log.warning("couldn't restore sample_rate value: I/O error %s" % e)
+ log.warning("couldn't restore sample_rate value: I/O error %s", e)
except ValueError as e:
- log.warning("couldn't restore sample_rate value: Value error %s" % e)
+ log.warning("couldn't restore sample_rate value: Value error %s", e)
except TypeError as e:
- log.warning("couldn't restore sample_rate value: Type error %s" % e)
+ log.warning("couldn't restore sample_rate value: Type error %s", e)
def load_events(self, path, events):
parser_event = configparser.ConfigParser()
@@ -266,7 +268,7 @@ class Test(object):
# event' first as a base
for section in filter(self.is_event, parser_event.sections()):
- parser_items = parser_event.items(section);
+ parser_items = parser_event.items(section)
base_items = {}
# Read parent event if there's any
@@ -280,7 +282,7 @@ class Test(object):
events[section] = e
def run_cmd(self, tempdir):
- junk1, junk2, junk3, junk4, myarch = (os.uname())
+ _junk1, _junk2, _junk3, _junk4, myarch = (os.uname())
if self.skip_test_arch(myarch):
raise Notest(self, myarch)
@@ -299,7 +301,7 @@ class Test(object):
self.perf, self.command, tempdir, self.args)
ret = os.WEXITSTATUS(os.system(cmd))
- log.info(" '%s' ret '%s', expected '%s'" % (cmd, str(ret), str(self.ret)))
+ log.info(" '%s' ret '%s', expected '%s'", cmd, str(ret), str(self.ret))
if not data_equal(str(ret), str(self.ret)):
if self.test_ret:
@@ -310,34 +312,34 @@ class Test(object):
def compare(self, expect, result):
match = {}
- log.debug(" compare");
+ log.debug(" compare")
# For each expected event find all matching
# events in result. Fail if there's not any.
for exp_name, exp_event in expect.items():
exp_list = []
res_event = {}
- log.debug(" matching [%s]" % exp_name)
+ log.debug(" matching [%s]", exp_name)
for res_name, res_event in result.items():
- log.debug(" to [%s]" % res_name)
+ log.debug(" to [%s]", res_name)
if (exp_event.equal(res_event)):
exp_list.append(res_name)
log.debug(" ->OK")
else:
- log.debug(" ->FAIL");
+ log.debug(" ->FAIL")
- log.debug(" match: [%s] matches %s" % (exp_name, str(exp_list)))
+ log.debug(" match: [%s] matches %s", exp_name, str(exp_list))
# we did not any matching event - fail
if not exp_list:
if exp_event.optional():
- log.debug(" %s does not match, but is optional" % exp_name)
+ log.debug(" %s does not match, but is optional", exp_name)
else:
if not res_event:
- log.debug(" res_event is empty");
+ log.debug(" res_event is empty")
else:
exp_event.diff(res_event)
- raise Fail(self, 'match failure');
+ raise Fail(self, 'match failure')
match[exp_name] = exp_list
@@ -354,38 +356,38 @@ class Test(object):
if res_group not in match[group]:
raise Fail(self, 'group failure')
- log.debug(" group: [%s] matches group leader %s" %
- (exp_name, str(match[group])))
+ log.debug(" group: [%s] matches group leader %s",
+ exp_name, str(match[group]))
log.debug(" matched")
def resolve_groups(self, events):
for name, event in events.items():
- group_fd = event['group_fd'];
+ group_fd = event['group_fd']
if group_fd == '-1':
- continue;
+ continue
for iname, ievent in events.items():
if (ievent['fd'] == group_fd):
event.group = iname
- log.debug('[%s] has group leader [%s]' % (name, iname))
- break;
+ log.debug('[%s] has group leader [%s]', name, iname)
+ break
def run(self):
- tempdir = tempfile.mkdtemp();
+ tempdir = tempfile.mkdtemp()
try:
# run the test script
- self.run_cmd(tempdir);
+ self.run_cmd(tempdir)
# load events expectation for the test
- log.debug(" loading result events");
+ log.debug(" loading result events")
for f in glob.glob(tempdir + '/event*'):
- self.load_events(f, self.result);
+ self.load_events(f, self.result)
# resolve group_fd to event names
- self.resolve_groups(self.expect);
- self.resolve_groups(self.result);
+ self.resolve_groups(self.expect)
+ self.resolve_groups(self.result)
# do the expectation - results matching - both ways
self.compare(self.expect, self.result)
@@ -401,9 +403,9 @@ def run_tests(options):
try:
Test(f, options).run()
except Unsup as obj:
- log.warning("unsupp %s" % obj.getMsg())
+ log.warning("unsupp %s", obj.getMsg())
except Notest as obj:
- log.warning("skipped %s" % obj.getMsg())
+ log.warning("skipped %s", obj.getMsg())
def setup_log(verbose):
global log
diff --git a/tools/perf/tests/shell/lib/perf_json_output_lint.py b/tools/perf/tests/shell/lib/perf_json_output_lint.py
index dafbde56cc76..8cde22ac9d97 100644
--- a/tools/perf/tests/shell/lib/perf_json_output_lint.py
+++ b/tools/perf/tests/shell/lib/perf_json_output_lint.py
@@ -46,46 +46,46 @@ def is_counter_value(num):
def is_metric_value(num):
return isfloat(num) or num == 'none'
-def check_json_output(expected_items):
+def check_json_output(expected_count):
checks = {
- 'counters': lambda x: isfloat(x),
+ 'counters': isfloat,
'core': lambda x: True,
- 'counter-value': lambda x: is_counter_value(x),
+ 'counter-value': is_counter_value,
'cgroup': lambda x: True,
- 'cpu': lambda x: isint(x),
+ 'cpu': isint,
'cache': lambda x: True,
'cluster': lambda x: True,
'die': lambda x: True,
'event': lambda x: True,
- 'event-runtime': lambda x: isfloat(x),
- 'interval': lambda x: isfloat(x),
+ 'event-runtime': isfloat,
+ 'interval': isfloat,
'metric-unit': lambda x: True,
- 'metric-value': lambda x: is_metric_value(x),
+ 'metric-value': is_metric_value,
'metric-threshold': lambda x: x in ['unknown', 'good', 'less good', 'nearly bad', 'bad'],
'metricgroup': lambda x: True,
'node': lambda x: True,
- 'pcnt-running': lambda x: isfloat(x),
+ 'pcnt-running': isfloat,
'socket': lambda x: True,
'thread': lambda x: True,
'unit': lambda x: True,
}
- input = '[\n' + ','.join(Lines) + '\n]'
- for item in json.loads(input):
- if expected_items != -1:
+ json_input = '[\n' + ','.join(Lines) + '\n]'
+ for item in json.loads(json_input):
+ if expected_count:
count = len(item)
- if count not in expected_items and count >= 1 and count <= 7 and 'metric-value' in item:
+ if count not in expected_count and count >= 1 and count <= 7 and 'metric-value' in item:
# Events that generate >1 metric may have isolated metric
# values and possibly other prefixes like interval, core,
# counters, or event-runtime/pcnt-running from multiplexing.
pass
- elif count not in expected_items and count >= 1 and count <= 5 and 'metricgroup' in item:
+ elif count not in expected_count and count >= 1 and count <= 5 and 'metricgroup' in item:
pass
- elif count - 1 in expected_items and 'metric-threshold' in item:
+ elif count - 1 in expected_count and 'metric-threshold' in item:
pass
- elif count in expected_items and 'insn per cycle' in item:
+ elif count in expected_count and 'insn per cycle' in item:
pass
- elif count not in expected_items:
- raise RuntimeError(f'wrong number of fields. counted {count} expected {expected_items}'
+ elif count not in expected_count:
+ raise RuntimeError(f'wrong number of fields. counted {count} expected {expected_count}'
f' in \'{item}\'')
for key, value in item.items():
if key not in checks:
@@ -107,7 +107,7 @@ try:
expected_items = [1, 2]
else:
# If no option is specified, don't check the number of items.
- expected_items = -1
+ expected_items = []
check_json_output(expected_items)
except:
print('Test failed for input:\n' + '\n'.join(Lines))
diff --git a/tools/perf/tests/shell/lib/perf_metric_validation.py b/tools/perf/tests/shell/lib/perf_metric_validation.py
index 3d52f94f22b9..0508c98d84fd 100644
--- a/tools/perf/tests/shell/lib/perf_metric_validation.py
+++ b/tools/perf/tests/shell/lib/perf_metric_validation.py
@@ -1,10 +1,10 @@
# SPDX-License-Identifier: GPL-2.0
-import re
-import csv
-import json
import argparse
+import json
from pathlib import Path
import subprocess
+import sys
+from typing import Any
class TestError:
@@ -77,11 +77,11 @@ class Validator:
def read_json(self, filename: str) -> dict:
try:
- with open(Path(filename).resolve(), "r") as f:
+ with open(Path(filename).resolve(), "r", encoding="utf-8") as f:
data = json.loads(f.read())
except OSError as e:
print(f"Error when reading file {e}")
- sys.exit()
+ sys.exit(1)
return data
@@ -90,16 +90,16 @@ class Validator:
if not parent.exists():
parent.mkdir(parents=True)
- with open(output_file, "w+") as output_file:
+ with open(output_file, "w+", encoding="utf-8") as out_f:
json.dump(data,
- output_file,
+ out_f,
ensure_ascii=True,
indent=4)
def get_results(self, idx: int = 0):
return self.results.get(idx)
- def get_bounds(self, lb, ub, error, alias={}, ridx: int = 0) -> list:
+ def get_bounds(self, lb, ub, error, alias=None, ridx: int = 0) -> tuple:
"""
Get bounds and tolerance from lb, ub, and error.
If missing lb, use 0.0; missing ub, use float('inf); missing error, use self.tolerance.
@@ -111,6 +111,9 @@ class Validator:
upper bound, return -1 if the upper bound is a metric value and is not collected
tolerance, denormalized base on upper bound value
"""
+ if alias is None:
+ alias = {}
+
# init ubv and lbv to invalid values
def get_bound_value(bound, initval, ridx):
val = initval
@@ -213,7 +216,7 @@ class Validator:
@param alias: the dict has alias to metric name mapping
@returns: value of the formula is success; -1 if the one or more metric value not provided
"""
- stack = []
+ stack: list[Any] = []
b = 0
errs = []
sign = "+"
@@ -263,7 +266,7 @@ class Validator:
alias[m['Alias']] = m['Name']
lbv, ubv, t = self.get_bounds(
rule['RangeLower'], rule['RangeUpper'], rule['ErrorThreshold'], alias, ridx=rule['RuleIndex'])
- val, f = self.evaluate_formula(
+ val, _ = self.evaluate_formula(
rule['Formula'], alias, ridx=rule['RuleIndex'])
lb = rule['RangeLower']
@@ -316,7 +319,7 @@ class Validator:
rerun.append(m['Name'])
if len(rerun) > 0 and len(rerun) < 20:
- second_results = dict()
+ second_results: dict[Any, Any] = dict()
self.second_test(rerun, second_results)
for name, val in second_results.items():
if name not in failures:
@@ -373,7 +376,7 @@ class Validator:
name = result["metric-unit"].split(" ")[1] if len(result["metric-unit"].split(" ")) > 1 \
else result["metric-unit"]
metricvalues[name.lower()] = float(result["metric-value"])
- except ValueError as error:
+ except ValueError:
continue
return
@@ -383,7 +386,7 @@ class Validator:
wl = workload.split()
command.extend(wl)
print(" ".join(command))
- cmd = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, encoding='utf-8')
+ cmd = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, encoding='utf-8', check=False)
lines = cmd.stderr.splitlines() + cmd.stdout.splitlines()
data = []
for line in lines:
@@ -397,9 +400,9 @@ class Validator:
Collect metric data with "perf stat -M" on given workload with -a and -j.
"""
self.results = dict()
- print(f"Starting perf collection")
+ print("Starting perf collection")
print(f"Long workload: {workload}")
- collectlist = dict()
+ collectlist: dict[int, Any] = dict()
if self.collectlist != "":
collectlist[0] = {x for x in self.collectlist.split(",")}
else:
@@ -441,7 +444,7 @@ class Validator:
"""
command = ['perf', 'list', '-j', '--details', 'metrics']
cmd = subprocess.run(command, stdout=subprocess.PIPE,
- stderr=subprocess.PIPE, encoding='utf-8')
+ stderr=subprocess.PIPE, encoding='utf-8', check=False)
try:
data = json.loads(cmd.stdout)
for m in data:
@@ -454,9 +457,9 @@ class Validator:
self.metrics.add(name)
if 'ScaleUnit' in m and (m['ScaleUnit'] == '1%' or m['ScaleUnit'] == '100%'):
self.pctgmetrics.add(name.lower())
- except ValueError as error:
- print(f"Error when parsing metric data")
- sys.exit()
+ except ValueError:
+ print("Error when parsing metric data")
+ sys.exit(1)
return
@@ -519,9 +522,6 @@ class Validator:
# Initialize data structures before data validation of each workload
def _init_data(self):
-
- testtypes = ['PositiveValueTest',
- 'RelationshipTest', 'SingleMetricTest']
self.results = dict()
self.ignoremetrics = set()
self.errlist = list()
@@ -572,7 +572,7 @@ class Validator:
# End of Class Validator
-def main() -> None:
+def main() -> int:
parser = argparse.ArgumentParser(
description="Launch metric value validation")
@@ -602,5 +602,4 @@ def main() -> None:
if __name__ == "__main__":
- import sys
sys.exit(main())
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 13/49] perf build: Make mypy build test opt-out (NO_MYPY=1)
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (11 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 12/49] perf test: Clean up mypy and pylint issues in shell test libraries Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 14/49] perf build: Make pylint build test opt-out (NO_PYLINT=1) Ian Rogers
` (35 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Commit 168910d0f937 ("perf build: Add mypy build tests") added mypy as an
opt-in compile-time check (MYPY=1) because of existing mypy errors across
perf Python scripts.
Now that all Python scripts in tools/perf have been cleaned up and pass
mypy without errors, make the mypy build test opt-out (default on when
mypy is installed, disabled with NO_MYPY=1), matching NO_SHELLCHECK=1.
Set MYPYPATH to tools/perf/python so local imports resolve during type
checking, and clean *.mypy_log files in 'make clean'.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/Build | 2 +-
tools/perf/Makefile.perf | 20 +++++++++++++++++---
tools/perf/pmu-events/Build | 2 +-
tools/perf/tests/Build | 2 +-
tools/perf/util/Build | 2 +-
5 files changed, 21 insertions(+), 7 deletions(-)
diff --git a/tools/perf/Build b/tools/perf/Build
index e18c80a5c1bc..37d603faea6d 100644
--- a/tools/perf/Build
+++ b/tools/perf/Build
@@ -88,7 +88,7 @@ endif
$(OUTPUT)%.mypy_log: %
$(call rule_mkdir)
- $(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
+ $(Q)$(call echo-cmd,test)$(MYPY) "$<" > $@ || (cat $@ && rm $@ && false)
perf-y += $(MYPY_TEST_LOGS)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 2438b40eaaec..f22eda3d2852 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -129,6 +129,8 @@ include ../scripts/utilities.mak
# Define GEN_VMLINUX_H to generate vmlinux.h from the BTF.
#
# Define NO_SHELLCHECK if you do not want to run shellcheck during build
+#
+# Define NO_MYPY if you do not want to run mypy during build
# As per kernel Makefile, avoid funny character set dependencies
unexport LC_ALL
@@ -263,8 +265,19 @@ ifneq ($(SHELLCHECK),)
endif
# Runs mypy on perf python files
-ifeq ($(MYPY),1)
- MYPY := $(shell which mypy 2> /dev/null)
+MAKEOVERRIDES := $(filter-out MYPY=%,$(MAKEOVERRIDES))
+ifeq ($(NO_MYPY),1)
+ override MYPY :=
+else
+ ifeq ($(MYPY),1)
+ override MYPY := $(shell which mypy 2> /dev/null)
+ else
+ MYPY ?= $(shell which mypy 2> /dev/null)
+ endif
+endif
+
+ifneq ($(MYPY),)
+ override MYPY := MYPYPATH=$(srctree)/tools/perf/python $(MYPY)
endif
# Runs pylint on perf python files
@@ -948,7 +961,8 @@ clean:: $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBSYMBOL)-clean $(
$(call QUIET_CLEAN, core-objs) $(RM) $(LIBPERF_A) $(OUTPUT)perf-archive \
$(OUTPUT)perf-iostat $(LANG_BINDINGS)
$(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '*.a' -delete -o \
- -name '\.*.cmd' -delete -o -name '\.*.d' -delete -o -name '*.shellcheck_log' -delete
+ -name '\.*.cmd' -delete -o -name '\.*.d' -delete -o -name '*.shellcheck_log' -delete -o \
+ -name '*.mypy_log' -delete
$(Q)$(RM) $(OUTPUT).config-detected
$(call QUIET_CLEAN, core-progs) $(RM) $(ALL_PROGRAMS) perf perf-read-vdso32 \
perf-read-vdsox32 $(OUTPUT)$(LIBJVMTI).so
diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build
index 372773b998e4..c3542e5ae46f 100644
--- a/tools/perf/pmu-events/Build
+++ b/tools/perf/pmu-events/Build
@@ -187,7 +187,7 @@ JEVENTS_DEPS := $(OUT_JSON) $(GEN_JSON) $(JEVENTS_PY) $(METRIC_PY) $(EMPTY_PMU_E
ifdef MYPY
define MYPY_RULE
$(2): $(1)
- $$(Q)$$(call echo-cmd,test)mypy $(1) > $(2) || (cat $(2) && rm $(2) && false)
+ $$(Q)$$(call echo-cmd,test)$$(MYPY) $(1) > $(2) || (cat $(2) && rm $(2) && false)
endef
$(foreach src,$(wildcard pmu-events/*.py), \
$(eval dest := $(patsubst pmu-events/%,$(OUTPUT)pmu-events/%.mypy_log,$(src))) \
diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index 406e48eed1c8..98b05b68fc14 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -104,7 +104,7 @@ endif
$(OUTPUT)%.mypy_log: %
$(call rule_mkdir)
- $(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
+ $(Q)$(call echo-cmd,test)$(MYPY) "$<" > $@ || (cat $@ && rm $@ && false)
perf-test-y += $(MYPY_TEST_LOGS)
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index d08f2af7d970..71d1f7289d03 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -426,7 +426,7 @@ endif
$(OUTPUT)%.mypy_log: %
$(call rule_mkdir)
- $(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
+ $(Q)$(call echo-cmd,test)$(MYPY) "$<" > $@ || (cat $@ && rm $@ && false)
perf-util-y += $(MYPY_TEST_LOGS)
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 14/49] perf build: Make pylint build test opt-out (NO_PYLINT=1)
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (12 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 13/49] perf build: Make mypy build test opt-out (NO_MYPY=1) Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 15/49] perf Makefile: Install standalone Python scripts during transition Ian Rogers
` (34 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Commit 8a54784e708b ("perf build: Add pylint build tests") added a
compile-time pylint check for Python files in tools/perf, but left it
opt-in via PYLINT=1.
Now that pylint issues across tools/perf/python, tools/perf/pmu-events,
and tools/perf/tests/shell/lib have been cleaned up, enable the pylint
check by default when pylint is installed on the system, with an opt-out
via NO_PYLINT=1 (matching other perf build feature flags).
Configure PYTHONPATH in Makefile.perf so pylint resolves perf.pyi type
stubs and pmu-events modules, enable error and warning categories
(--disable=all --enable=E,W --disable=W0123,W0311,W0511,W0603,W0622,W0718)
appropriate for perf's standalone CLI and test scripts, and require
pylint >= 2.16.0 (skipping with a warning if an older version is
installed).
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/Build | 2 +-
tools/perf/Makefile.perf | 39 +++++++++++++++++++++++++++++++++----
tools/perf/pmu-events/Build | 2 +-
tools/perf/tests/Build | 2 +-
tools/perf/util/Build | 2 +-
5 files changed, 39 insertions(+), 8 deletions(-)
diff --git a/tools/perf/Build b/tools/perf/Build
index 37d603faea6d..5bd0e58d7cb0 100644
--- a/tools/perf/Build
+++ b/tools/perf/Build
@@ -101,6 +101,6 @@ endif
$(OUTPUT)%.pylint_log: %
$(call rule_mkdir)
- $(Q)$(call echo-cmd,test)pylint "$<" > $@ || (cat $@ && rm $@ && false)
+ $(Q)$(call echo-cmd,test)$(PYLINT) "$<" > $@ || (cat $@ && rm $@ && false)
perf-y += $(PYLINT_TEST_LOGS)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index f22eda3d2852..3e193d609988 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -131,6 +131,8 @@ include ../scripts/utilities.mak
# Define NO_SHELLCHECK if you do not want to run shellcheck during build
#
# Define NO_MYPY if you do not want to run mypy during build
+#
+# Define NO_PYLINT if you do not want to run pylint during build
# As per kernel Makefile, avoid funny character set dependencies
unexport LC_ALL
@@ -280,9 +282,38 @@ ifneq ($(MYPY),)
override MYPY := MYPYPATH=$(srctree)/tools/perf/python $(MYPY)
endif
-# Runs pylint on perf python files
-ifeq ($(PYLINT),1)
- PYLINT := $(shell which pylint 2> /dev/null)
+# Runs pylint on perf python files.
+# Require pylint >= 2.16.0 (which introduced W0718 broad-exception-caught and
+# astroid .pyi stub support for perf.pyi).
+# Disabled warnings:
+# W0123 (eval-used): Use of eval()
+# W0311 (bad-indentation): Non-4-space indentation
+# W0511 (fixme): Presence of FIXME/TODO comments
+# W0603 (global-statement): Use of the global statement
+# W0622 (redefined-builtin): Redefining a built-in name (e.g. id, type, dir)
+# W0718 (broad-exception-caught): Catching broad Exception
+MAKEOVERRIDES := $(filter-out PYLINT=%,$(MAKEOVERRIDES))
+ifeq ($(NO_PYLINT),1)
+ override PYLINT :=
+else
+ ifeq ($(PYLINT),1)
+ override PYLINT := $(shell which pylint 2> /dev/null)
+ else
+ PYLINT ?= $(shell which pylint 2> /dev/null)
+ endif
+endif
+
+ifneq ($(PYLINT),)
+ ifneq ($(force_fixdep),1)
+ ifneq ($(shell $(PYLINT) --version 2>/dev/null | \
+ awk '/^pylint / { split($$2, v, "."); print (v[1]+0 > 2 || (v[1]+0 == 2 && v[2]+0 >= 16)) }'), 1)
+ $(warning Warning: pylint version is older than 2.16.0, skipping pylint checks.)
+ override PYLINT :=
+ else
+ override PYLINT := PYTHONPATH=$(srctree)/tools/perf/python:$(srctree)/tools/perf/pmu-events \
+ $(PYLINT) --disable=all --enable=E,W --disable=W0123,W0311,W0511,W0603,W0622,W0718
+ endif
+ endif
endif
export srctree OUTPUT RM CC CXX RUSTC CLANG LD AR CFLAGS CXXFLAGS RUST_FLAGS V BISON FLEX AWK LIBBPF READELF
@@ -962,7 +993,7 @@ clean:: $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBSYMBOL)-clean $(
$(OUTPUT)perf-iostat $(LANG_BINDINGS)
$(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '*.a' -delete -o \
-name '\.*.cmd' -delete -o -name '\.*.d' -delete -o -name '*.shellcheck_log' -delete -o \
- -name '*.mypy_log' -delete
+ -name '*.mypy_log' -delete -o -name '*.pylint_log' -delete
$(Q)$(RM) $(OUTPUT).config-detected
$(call QUIET_CLEAN, core-progs) $(RM) $(ALL_PROGRAMS) perf perf-read-vdso32 \
perf-read-vdsox32 $(OUTPUT)$(LIBJVMTI).so
diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build
index c3542e5ae46f..1ac067be9d34 100644
--- a/tools/perf/pmu-events/Build
+++ b/tools/perf/pmu-events/Build
@@ -203,7 +203,7 @@ endif
ifdef PYLINT
define PYLINT_RULE
$(2): $(1)
- $$(Q)$$(call echo-cmd,test)pylint $(1) > $(2) || (cat $(2) && rm $(2) && false)
+ $$(Q)$$(call echo-cmd,test)$$(PYLINT) $(1) > $(2) || (cat $(2) && rm $(2) && false)
endef
$(foreach src,$(wildcard pmu-events/*.py), \
$(eval dest := $(patsubst pmu-events/%,$(OUTPUT)pmu-events/%.pylint_log,$(src))) \
diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build
index 98b05b68fc14..f88e9a55c4f5 100644
--- a/tools/perf/tests/Build
+++ b/tools/perf/tests/Build
@@ -117,6 +117,6 @@ endif
$(OUTPUT)%.pylint_log: %
$(call rule_mkdir)
- $(Q)$(call echo-cmd,test)pylint "$<" > $@ || (cat $@ && rm $@ && false)
+ $(Q)$(call echo-cmd,test)$(PYLINT) "$<" > $@ || (cat $@ && rm $@ && false)
perf-test-y += $(PYLINT_TEST_LOGS)
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 71d1f7289d03..3ea888f4210a 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -438,7 +438,7 @@ endif
$(OUTPUT)%.pylint_log: %
$(call rule_mkdir)
- $(Q)$(call echo-cmd,test)pylint "$<" > $@ || (cat $@ && rm $@ && false)
+ $(Q)$(call echo-cmd,test)$(PYLINT) "$<" > $@ || (cat $@ && rm $@ && false)
perf-util-y += $(PYLINT_TEST_LOGS)
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 15/49] perf Makefile: Install standalone Python scripts during transition
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (13 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 14/49] perf build: Make pylint build test opt-out (NO_PYLINT=1) Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 16/49] perf python: Port stat-cpi to perf module Ian Rogers
` (33 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Install the newly ported standalone Python scripts from
tools/perf/python/ alongside the legacy scripts during the transition
phase so that newly ported scripts are available in the installation
directory before legacy support is removed.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/Makefile.perf | 3 +++
1 file changed, 3 insertions(+)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 3e193d609988..abd377c16435 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -905,6 +905,9 @@ ifndef NO_LIBPYTHON
$(INSTALL) scripts/python/Perf-Trace-Util/lib/Perf/Trace/* -m 644 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/Perf-Trace-Util/lib/Perf/Trace'; \
$(INSTALL) scripts/python/*.py -m 644 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'; \
$(INSTALL) scripts/python/bin/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/bin'
+ $(call QUIET_INSTALL, python-scripts-standalone) \
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/python'; \
+ $(INSTALL) python/*.py -m 755 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/python'
endif
$(call QUIET_INSTALL, dlfilters) \
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/dlfilters'; \
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 16/49] perf python: Port stat-cpi to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (14 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 15/49] perf Makefile: Install standalone Python scripts during transition Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 17/49] perf python: Port mem-phys-addr " Ian Rogers
` (32 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port stat-cpi.py from the legacy embedded scripting framework to a
standalone Python script in tools/perf/python/ to calculate Cycles Per
Instruction (CPI) per interval per CPU or thread.
Improvements compared to the legacy script:
- Support both perf.data file mode (via perf.session stat callbacks)
and live counter collection mode (using perf.parse_events,
evlist.open, and evsel.read across intervals), with automatic fallback
to user-space (:u) and self-process monitoring when perf_event_paranoid
restricts system-wide events (EACCES).
- Compute per-interval counter deltas (val, ena, run) keyed by raw event
name so cumulative PERF_RECORD_STAT snapshots and hybrid PMU events
(e.g. cpu_core/cycles/, cpu_atom/cycles/) are accumulated accurately,
and scale counts by time_enabled / time_running when multiplexed.
- Replace hard-coded CPU ([0, 1]) and thread ([0]) arrays with dynamic
CPU and thread discovery so arbitrary system topologies work
automatically.
- Add CLI option handling (-i, -I, -p) via argparse and type annotations
passing mypy and pylint.
Add a shell test (test_stat_cpi_python.sh) to verify the standalone
script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/stat-cpi.py | 208 ++++++++++++++++++
.../perf/tests/shell/test_stat_cpi_python.sh | 106 +++++++++
2 files changed, 314 insertions(+)
create mode 100755 tools/perf/python/stat-cpi.py
create mode 100755 tools/perf/tests/shell/test_stat_cpi_python.sh
diff --git a/tools/perf/python/stat-cpi.py b/tools/perf/python/stat-cpi.py
new file mode 100755
index 000000000000..87df5ad279a0
--- /dev/null
+++ b/tools/perf/python/stat-cpi.py
@@ -0,0 +1,208 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""Calculate CPI from perf stat data or live."""
+from __future__ import annotations
+
+import argparse
+import os
+import signal
+import sys
+import time
+from typing import Any, Optional
+import perf
+
+class StatCpiAnalyzer:
+ """Accumulates cycles and instructions and calculates CPI."""
+
+ def __init__(self, args: argparse.Namespace) -> None:
+ self.args = args
+ self.data: dict[str, tuple[int, int, int]] = {}
+ self.prev_data: dict[str, tuple[int, int, int]] = {}
+ self.recorded_pairs: set[tuple[int, int]] = set()
+
+ def get_key(self, event: str, cpu: int, thread: int) -> str:
+ """Get key for data dictionary."""
+ return f"{event}-{cpu}-{thread}"
+
+ def store_key(self, cpu: int, thread: int) -> None:
+ """Store CPU and thread IDs."""
+ self.recorded_pairs.add((cpu, thread))
+
+ def store(self, event: str, cpu: int, thread: int,
+ counts: tuple[int, int, int], is_delta: bool = False,
+ raw_name: Optional[str] = None) -> None:
+ """Store counter values, computing difference from previous
+ absolute values if not already deltas."""
+ self.store_key(cpu, thread)
+ key = self.get_key(event, cpu, thread)
+ prev_key = self.get_key(raw_name or event, cpu, thread)
+
+ val, ena, run = counts
+ if is_delta:
+ # counts are already deltas
+ cur_val = val
+ cur_ena = ena
+ cur_run = run
+ else:
+ if prev_key in self.prev_data:
+ prev_val, prev_ena, prev_run = self.prev_data[prev_key]
+ cur_val = val - prev_val
+ cur_ena = ena - prev_ena
+ cur_run = run - prev_run
+ else:
+ cur_val = val
+ cur_ena = ena
+ cur_run = run
+ self.prev_data[prev_key] = counts # Store absolute value for next time
+
+ if key in self.data:
+ old_val, old_ena, old_run = self.data[key]
+ self.data[key] = (old_val + cur_val, old_ena + cur_ena, old_run + cur_run)
+ else:
+ self.data[key] = (cur_val, cur_ena, cur_run)
+
+ def get(self, event: str, cpu: int, thread: int) -> float:
+ """Get scaled counter value."""
+ key = self.get_key(event, cpu, thread)
+ if key not in self.data:
+ return 0.0
+ val, ena, run = self.data[key]
+ if run > 0:
+ return val * (ena / float(run))
+ return float(val)
+
+ def process_stat_event(self, event: Any, name: Optional[str] = None) -> None:
+ """Process PERF_RECORD_STAT and PERF_RECORD_STAT_ROUND events."""
+ if event.type == perf.RECORD_STAT:
+ if name:
+ if "cycles" in name:
+ event_name = "cycles"
+ elif "instructions" in name:
+ event_name = "instructions"
+ else:
+ return
+ self.store(event_name, event.cpu, event.thread,
+ (event.val, event.ena, event.run), raw_name=name)
+ elif event.type == perf.RECORD_STAT_ROUND:
+ timestamp = getattr(event, "time", 0)
+ self.print_interval(timestamp)
+ self.data.clear()
+ self.recorded_pairs.clear()
+
+ def print_interval(self, timestamp: int) -> None:
+ """Print CPI for the current interval."""
+ for cpu, thread in sorted(self.recorded_pairs):
+ cyc = self.get("cycles", cpu, thread)
+ ins = self.get("instructions", cpu, thread)
+ cpi = 0.0
+ if ins != 0:
+ cpi = cyc / float(ins)
+ t_sec = timestamp / 1000000000.0
+ print(f"{t_sec:15f}: cpu {cpu}, thread {thread} -> cpi {cpi:f} ({cyc:.0f}/{ins:.0f})")
+
+ def read_counters(self, evlist: Any) -> None:
+ """Read counters live."""
+ for evsel in evlist:
+ name = str(evsel)
+ if "cycles" in name:
+ event_name = "cycles"
+ elif "instructions" in name:
+ event_name = "instructions"
+ else:
+ continue
+
+ for cpu in evsel.cpus():
+ for thread in evsel.threads():
+ try:
+ counts = evsel.read(cpu, thread)
+ self.store(event_name, cpu, thread,
+ (counts.val, counts.ena, counts.run),
+ is_delta=True, raw_name=name)
+ except OSError:
+ pass
+
+ def run_file(self) -> None:
+ """Process events from file."""
+ session = perf.session(perf.data(self.args.input), stat=self.process_stat_event)
+ session.process_events()
+
+ def _open_live_evlist(self) -> Any:
+ """Open evlist for live mode, falling back to user-space or process scope on EACCES."""
+ threads = perf.thread_map(self.args.pid) if self.args.pid else None
+ candidates = [
+ ("cycles,instructions", threads),
+ ("cycles:u,instructions:u", threads),
+ ]
+ if threads is None:
+ self_threads = perf.thread_map(os.getpid())
+ candidates.append(("cycles,instructions", self_threads))
+ candidates.append(("cycles:u,instructions:u", self_threads))
+
+ last_err: Optional[OSError] = None
+ for events, tmap in candidates:
+ try:
+ evlist = perf.parse_events(events, None, tmap)
+ for evsel in evlist:
+ evsel.read_format |= (
+ perf.FORMAT_TOTAL_TIME_ENABLED | perf.FORMAT_TOTAL_TIME_RUNNING
+ )
+ evlist.open()
+ evlist.enable()
+ return evlist
+ except PermissionError as e:
+ last_err = e
+ except OSError as e:
+ if e.errno == 13:
+ last_err = e
+ else:
+ raise
+ if last_err is not None:
+ raise last_err
+ raise RuntimeError("Failed to open events")
+
+ def run_live(self) -> None:
+ """Read counters live."""
+ try:
+ evlist = self._open_live_evlist()
+ except OSError as e:
+ print(f"Failed to open events: {e}", file=sys.stderr)
+ sys.exit(1)
+
+ def handle_signal(_signum: int, _frame: Any) -> None:
+ raise KeyboardInterrupt
+
+ signal.signal(signal.SIGINT, signal.default_int_handler)
+ signal.signal(signal.SIGTERM, handle_signal)
+
+ print("Live mode started. Press Ctrl+C to stop.")
+ try:
+ while True:
+ time.sleep(self.args.interval)
+ timestamp = time.time_ns()
+ self.read_counters(evlist)
+ self.print_interval(timestamp)
+ self.data.clear()
+ self.recorded_pairs.clear()
+ except KeyboardInterrupt:
+ print("\nStopped.")
+ finally:
+ evlist.close()
+
+def main() -> None:
+ """Main function."""
+ ap = argparse.ArgumentParser(description="Calculate CPI from perf stat data or live")
+ ap.add_argument("-i", "--input", help="Input file name (enables file mode)")
+ ap.add_argument("-I", "--interval", type=float, default=1.0,
+ help="Interval in seconds for live mode")
+ ap.add_argument("-p", "--pid", type=int,
+ help="Monitor specific process ID in live mode")
+ args = ap.parse_args()
+
+ analyzer = StatCpiAnalyzer(args)
+ if args.input:
+ analyzer.run_file()
+ else:
+ analyzer.run_live()
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/perf/tests/shell/test_stat_cpi_python.sh b/tools/perf/tests/shell/test_stat_cpi_python.sh
new file mode 100755
index 000000000000..8579f9f552a8
--- /dev/null
+++ b/tools/perf/tests/shell/test_stat_cpi_python.sh
@@ -0,0 +1,106 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# stat-cpi python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+# If we don't have the perf python module, we can't test
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ return 2 2>/dev/null || exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/stat-cpi.py"
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, stat-cpi.py not found at $script_path"
+ return 2 2>/dev/null || exit 2
+fi
+
+err=0
+ran=0
+temp_data=""
+temp_out=""
+
+cleanup() {
+ [ -n "${pid}" ] && kill "$pid" 2>/dev/null || true
+ [ -n "${workload_pid}" ] && kill "$workload_pid" 2>/dev/null || true
+ rm -f "${temp_data}" "${temp_out}"
+ trap - exit term int
+}
+
+trap_cleanup() {
+ cleanup
+ exit 1
+}
+trap trap_cleanup exit term int
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+test_live_mode() {
+ echo "Testing stat-cpi.py live mode..."
+ if ! perf stat -e cycles,instructions -- sleep 0.1 2>/dev/null; then
+ echo "perf stat failed (permissions?), skipping live mode test."
+ return 0
+ fi
+ ran=1
+
+ perf test -w noploop &
+ workload_pid=$!
+
+ # Run live mode for 1 interval in the background, give it a tiny sleep, then interrupt
+ "$PYTHON" "$script_path" -I 0.1 -p "$workload_pid" > "${temp_out}" &
+ pid=$!
+ sleep 0.5
+ kill -INT "$pid" 2>/dev/null || true
+ set +e
+ wait "$pid"
+ res=$?
+ set -e
+ pid=""
+ kill "$workload_pid" 2>/dev/null || true
+ workload_pid=""
+ if [ $res -ne 0 ] && [ $res -ne 130 ] && [ $res -ne 143 ]; then
+ echo "Live mode failed or crashed"
+ err=1
+ elif ! grep -q "cpi" "${temp_out}"; then
+ echo "Live mode produced no cpi output"
+ err=1
+ else
+ echo "Live mode test passed."
+ fi
+}
+
+test_file_mode() {
+ echo "Testing stat-cpi.py file mode..."
+ # Generate some stat events - perf stat -I represents interval reporting
+ if ! perf stat -e cycles,instructions -I 100 record -o "${temp_data}" \
+ -- sleep 0.5 2>/dev/null; then
+ echo "perf stat failed (permissions?), skipping file mode test."
+ return
+ fi
+ ran=1
+
+ out=$("$PYTHON" "$script_path" -i "${temp_data}")
+ if ! echo "$out" | grep -q "cpi"; then
+ echo "File mode test failed."
+ err=1
+ else
+ echo "File mode test passed."
+ fi
+}
+
+test_live_mode
+test_file_mode
+
+cleanup
+if [ $ran -eq 0 ]; then
+ exit 2
+fi
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 17/49] perf python: Port mem-phys-addr to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (15 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 16/49] perf python: Port stat-cpi to perf module Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 18/49] perf python: Port stackcollapse " Ian Rogers
` (31 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port mem-phys-addr.py to a standalone script in tools/perf/python/
using the perf.session API to read perf.data files and profile physical
memory access types against /proc/iomem.
Improvements compared to the legacy script:
- Parse the full indentation hierarchy of /proc/iomem into a parent-child
tree of frozen IomemEntry dataclasses (instead of only top-level
indent-0 ranges), resolving physical addresses to the most specific
sub-range (such as Kernel code/data/bss inside System RAM) and rolling
child counts up into parent totals.
- Support profiling multiple memory events in a single perf.data session
(keyed by evsel name) instead of assuming a single global event.
- Add argparse CLI options (-i/--input and --iomem to allow supplying an
offline /proc/iomem snapshot from a target system).
Add a shell test (test_mem_phys_addr_python.sh) to verify the standalone
script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/mem-phys-addr.py | 137 ++++++++++++++++++
.../tests/shell/test_mem_phys_addr_python.sh | 101 +++++++++++++
2 files changed, 238 insertions(+)
create mode 100755 tools/perf/python/mem-phys-addr.py
create mode 100755 tools/perf/tests/shell/test_mem_phys_addr_python.sh
diff --git a/tools/perf/python/mem-phys-addr.py b/tools/perf/python/mem-phys-addr.py
new file mode 100755
index 000000000000..5064e673c6a2
--- /dev/null
+++ b/tools/perf/python/mem-phys-addr.py
@@ -0,0 +1,137 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""mem-phys-addr.py: Resolve physical address samples"""
+from __future__ import annotations
+import argparse
+import bisect
+import collections
+from dataclasses import dataclass
+import re
+from typing import (Dict, List, Optional)
+
+import perf
+
+@dataclass(frozen=True)
+class IomemEntry:
+ """Read from a line in /proc/iomem"""
+ begin: int
+ end: int
+ indent: int
+ label: str
+
+ def __lt__(self, other) -> bool:
+ if isinstance(other, int):
+ return self.begin < other
+ return self.begin < other.begin
+
+ def __gt__(self, other) -> bool:
+ if isinstance(other, int):
+ return self.begin > other
+ return self.begin > other.begin
+
+# Physical memory layout from /proc/iomem. Key is the indent and then
+# a list of ranges.
+iomem: Dict[int, List[IomemEntry]] = collections.defaultdict(list)
+# Child nodes from the iomem parent.
+children: Dict[IomemEntry, List[IomemEntry]] = collections.defaultdict(list)
+# Maximum indent seen before an entry in the iomem file.
+_STATE: Dict[str, int] = {"max_indent": 0}
+# Per-event counts for each range of memory.
+event_counts: Dict[str, collections.Counter] = collections.defaultdict(collections.Counter)
+
+def parse_iomem(iomem_path: str):
+ """Populate iomem from iomem file"""
+ with open(iomem_path, 'r', encoding='ascii') as f:
+ for line in f:
+ line = line.rstrip('\n')
+ if not line or line.isspace():
+ continue
+ indent = 0
+ while indent < len(line) and line[indent] == ' ':
+ indent += 1
+ _STATE["max_indent"] = max(_STATE["max_indent"], indent)
+ m = re.split('-|:', line, maxsplit=2)
+ if len(m) < 3:
+ continue
+ begin = int(m[0].strip(), 16)
+ end = int(m[1].strip(), 16)
+ label = m[2].strip()
+ entry = IomemEntry(begin, end, indent, label)
+ # Before adding entry, search for a parent node using its begin.
+ if indent > 0:
+ parent = find_memory_type(begin)
+ assert parent, f"Given indent expected a parent for {label}"
+ children[parent].append(entry)
+ iomem[indent].append(entry)
+
+def find_memory_type(phys_addr) -> Optional[IomemEntry]:
+ """Search iomem for the range containing phys_addr with the maximum indent"""
+ for i in range(_STATE["max_indent"], -1, -1):
+ if i not in iomem:
+ continue
+ position = bisect.bisect_right(iomem[i], phys_addr)
+ if position == 0:
+ continue
+ iomem_entry = iomem[i][position-1]
+ if iomem_entry.begin <= phys_addr <= iomem_entry.end:
+ return iomem_entry
+ return None
+
+def _print_entries(entries, load_mem_type_cnt, total):
+ """Print counts from parents down to their children"""
+ for entry in sorted(entries,
+ key=lambda e: (load_mem_type_cnt[e], e.begin),
+ reverse=True):
+ count = load_mem_type_cnt[entry]
+ if count > 0:
+ mem_type = ' ' * entry.indent + f"{entry.begin:x}-{entry.end:x} : {entry.label}"
+ percent = 100 * count / total
+ print(f"{mem_type:<40} {count:>10} {percent:>10.1f}")
+ _print_entries(children[entry], load_mem_type_cnt, total)
+
+def print_memory_type():
+ """Print the resolved memory types and their counts."""
+ if not event_counts:
+ print("No valid physical address samples found in perf data.")
+ return
+
+ for event_name, load_mem_type_cnt in event_counts.items():
+ print(f"Event: {event_name}")
+ print(f"{'Memory type':<40} {'count':>10} {'percentage':>10}")
+ print(f"{'-' * 40:<40} {'-' * 10:>10} {'-' * 10:>10}")
+ total = sum(load_mem_type_cnt.values())
+ if total == 0:
+ continue
+
+ # Add count from children into the parent.
+ for i in range(_STATE["max_indent"], -1, -1):
+ if i not in iomem:
+ continue
+ for entry in iomem[i]:
+ for child in children[entry]:
+ if load_mem_type_cnt[child] > 0:
+ load_mem_type_cnt[entry] += load_mem_type_cnt[child]
+
+ _print_entries(iomem[0], load_mem_type_cnt, total)
+ print()
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser(description="Resolve physical address samples")
+ ap.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ ap.add_argument("--iomem", default="/proc/iomem", help="Path to iomem file")
+ args = ap.parse_args()
+
+ def process_event(sample):
+ """Process a single sample event."""
+ phys_addr = sample.sample_phys_addr or 0
+ if not phys_addr:
+ return
+ entry = find_memory_type(phys_addr)
+ if entry:
+ event_name = str(sample.evsel)
+ if event_name.startswith("evsel(") and event_name.endswith(")"):
+ event_name = event_name[6:-1]
+ event_counts[event_name][entry] += 1
+
+ parse_iomem(args.iomem)
+ perf.session(perf.data(args.input), sample=process_event).process_events()
+ print_memory_type()
diff --git a/tools/perf/tests/shell/test_mem_phys_addr_python.sh b/tools/perf/tests/shell/test_mem_phys_addr_python.sh
new file mode 100755
index 000000000000..ae2f2fba0d20
--- /dev/null
+++ b/tools/perf/tests/shell/test_mem_phys_addr_python.sh
@@ -0,0 +1,101 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# mem-phys-addr python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+# If we don't have the perf python module, we can't test
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/mem-phys-addr.py"
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, mem-phys-addr.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+temp_iomem=""
+temp_out=""
+
+cleanup() {
+ rm -f "${temp_data}" "${temp_iomem}" "${temp_out}"
+}
+
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_iomem=$(mktemp /tmp/perf.iomem.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+cat << 'EOF' > "${temp_iomem}"
+00000000-ffffffffffffffff : System RAM
+ 00000000-7fffffffffffffff : Low RAM
+ 00001000-00ffffff : Kernel code
+ 8000000000000000-ffffffffffffffff : High RAM
+EOF
+
+test_iomem_hierarchy() {
+ echo "Testing mem-phys-addr.py hierarchical iomem resolution..."
+ "$PYTHON" - "$script_path" "${temp_iomem}" << 'PYEOF' > "${temp_out}"
+import importlib.util
+import sys
+
+spec = importlib.util.spec_from_file_location("mem_phys_addr", sys.argv[1])
+mod = importlib.util.module_from_spec(spec)
+sys.modules[spec.name] = mod
+spec.loader.exec_module(mod)
+
+mod.parse_iomem(sys.argv[2])
+entry_kernel = mod.find_memory_type(0x100000)
+entry_high = mod.find_memory_type(0x9000000000000000)
+assert entry_kernel is not None and entry_kernel.label == "Kernel code"
+assert entry_high is not None and entry_high.label == "High RAM"
+mod.event_counts["cpu/mem-loads/"][entry_kernel] += 3
+mod.event_counts["cpu/mem-loads/"][entry_high] += 1
+mod.print_memory_type()
+PYEOF
+ if ! grep -q "System RAM" "${temp_out}" || \
+ ! grep -q "Kernel code" "${temp_out}" || \
+ ! grep -q "High RAM" "${temp_out}"; then
+ echo "Hierarchical iomem resolution test failed."
+ err=1
+ else
+ echo "Hierarchical iomem resolution test passed."
+ fi
+}
+
+test_file_mode() {
+ echo "Testing mem-phys-addr.py file mode..."
+
+ # Generate memory access events (try unprivileged user-space first, then system-wide)
+ if ! perf record --phys-data -d -o "${temp_data}" \
+ -- perf test -w datasym >/dev/null 2>&1 && \
+ ! perf record -d -o "${temp_data}" -- perf test -w datasym >/dev/null 2>&1 && \
+ ! perf record -d -a -o "${temp_data}" -- sleep 0.2 >/dev/null 2>&1; then
+ echo "Skipping file mode record test, perf record -d not supported"
+ return 0
+ fi
+
+ # Run the script with custom --iomem
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" --iomem "${temp_iomem}" >/dev/null; then
+ echo "File mode test failed."
+ err=1
+ else
+ echo "File mode test passed."
+ fi
+}
+
+test_iomem_hierarchy
+test_file_mode
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 18/49] perf python: Port stackcollapse to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (16 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 17/49] perf python: Port mem-phys-addr " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 19/49] perf python: Port flamegraph " Ian Rogers
` (30 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port stackcollapse.py from tools/perf/scripts/python/ to a standalone
script in tools/perf/python/ refactored into a StackCollapseAnalyzer
class.
Improvements compared to the legacy script:
- Traverse sample.callchain directly from perf.session without
allocating per-event dictionaries, and fall back to sample.symbol when
a sample has no callchain.
- Replace deprecated optparse with argparse, adding -i/--input alongside
--include-tid, --include-pid, --no-comm, --tidy-java, and --kernel.
- Handle BrokenPipeError cleanly when output is piped into downstream
tools (such as head or flamegraph.pl).
Add a shell test (test_stackcollapse_python.sh) using a CPU workload
(perf test -w noploop) to verify the standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/stackcollapse.py | 145 ++++++++++++++++++
.../tests/shell/test_stackcollapse_python.sh | 77 ++++++++++
2 files changed, 222 insertions(+)
create mode 100755 tools/perf/python/stackcollapse.py
create mode 100755 tools/perf/tests/shell/test_stackcollapse_python.sh
diff --git a/tools/perf/python/stackcollapse.py b/tools/perf/python/stackcollapse.py
new file mode 100755
index 000000000000..0e8a65969db3
--- /dev/null
+++ b/tools/perf/python/stackcollapse.py
@@ -0,0 +1,145 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+stackcollapse.py - format perf samples with one line per distinct call stack
+
+This script's output has two space-separated fields. The first is a semicolon
+separated stack including the program name (from the "comm" field) and the
+function names from the call stack. The second is a count:
+
+ swapper;start_kernel;rest_init;cpu_idle;default_idle;native_safe_halt 2
+
+The file is sorted according to the first field.
+
+Ported from tools/perf/scripts/python/stackcollapse.py
+"""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+import os
+import sys
+import perf
+
+
+class StackCollapseAnalyzer:
+ """Accumulates call stacks and prints them collapsed."""
+
+ def __init__(self, args: argparse.Namespace) -> None:
+ self.args = args
+ self.lines: dict[str, int] = defaultdict(int)
+ self.session: perf.session | None = None
+
+ def tidy_function_name(self, sym: str, dso: str) -> str:
+ """Beautify function names based on options."""
+ if sym is None:
+ sym = "[unknown]"
+
+ sym = sym.replace(";", ":")
+ if self.args.tidy_java:
+ # Beautify Java signatures
+ sym = sym.replace("<", "")
+ sym = sym.replace(">", "")
+ if sym.startswith("L") and "/" in sym:
+ sym = sym[1:]
+ try:
+ sym = sym[:sym.index("(")]
+ except ValueError:
+ pass
+
+ if self.args.annotate_kernel and dso == "[kernel.kallsyms]":
+ return sym + "_[k]"
+ return sym
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Collect call stack for each sample."""
+ stack = []
+ callchain = sample.callchain
+ if callchain is not None:
+ for node in callchain:
+ stack.append(self.tidy_function_name(node.symbol, node.dso))
+ else:
+ # Fallback if no callchain
+ sym = (sample.symbol or '[unknown]')
+ dso = (sample.dso or '[unknown]')
+ stack.append(self.tidy_function_name(sym, dso))
+
+ if self.args.include_comm:
+ comm = "Unknown"
+ if self.session is not None:
+ try:
+ proc = self.session.find_thread(
+ sample.sample_pid, sample.sample_tid
+ )
+ if proc:
+ proc_comm = proc.comm()
+ if proc_comm is not None:
+ comm = proc_comm
+ except TypeError:
+ pass
+ comm = str(comm).replace(" ", "_")
+ sep = "-"
+ if self.args.include_pid:
+ comm = f"{comm}{sep}{(sample.sample_pid or 0)}"
+ sep = "/"
+ if self.args.include_tid:
+ comm = f"{comm}{sep}{(sample.sample_tid or 0)}"
+ stack.append(comm)
+
+ stack_string = ";".join(reversed(stack))
+ self.lines[stack_string] += 1
+
+ def print_totals(self) -> None:
+ """Print sorted collapsed stacks."""
+ try:
+ for stack in sorted(self.lines):
+ print(f"{stack} {self.lines[stack]}")
+ sys.stdout.flush()
+ except BrokenPipeError:
+ devnull = os.open(os.devnull, os.O_WRONLY)
+ os.dup2(devnull, sys.stdout.fileno())
+ os.close(devnull)
+
+
+def main():
+ """Main function."""
+ ap = argparse.ArgumentParser(
+ description="Format perf samples with one line per distinct call stack"
+ )
+ ap.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ ap.add_argument("--include-tid", action="store_true", help="include thread id in stack")
+ ap.add_argument("--include-pid", action="store_true", help="include process id in stack")
+ ap.add_argument("--no-comm", dest="include_comm", action="store_false", default=True,
+ help="do not separate stacks according to comm")
+ ap.add_argument("--tidy-java", action="store_true", help="beautify Java signatures")
+ ap.add_argument("--kernel", dest="annotate_kernel", action="store_true",
+ help="annotate kernel functions with _[k]")
+
+ args = ap.parse_args()
+
+ if args.include_tid and not args.include_comm:
+ print("requesting tid but not comm is invalid", file=sys.stderr)
+ sys.exit(1)
+ if args.include_pid and not args.include_comm:
+ print("requesting pid but not comm is invalid", file=sys.stderr)
+ sys.exit(1)
+
+ analyzer = StackCollapseAnalyzer(args)
+
+ try:
+ session = perf.session(perf.data(args.input), sample=analyzer.process_event)
+ analyzer.session = session
+ session.process_events()
+ except IOError as e:
+ print(f"Error: {e}", file=sys.stderr)
+ sys.exit(1)
+ except KeyboardInterrupt:
+ pass
+ finally:
+ analyzer.session = None
+
+ analyzer.print_totals()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/perf/tests/shell/test_stackcollapse_python.sh b/tools/perf/tests/shell/test_stackcollapse_python.sh
new file mode 100755
index 000000000000..e5675332e3cd
--- /dev/null
+++ b/tools/perf/tests/shell/test_stackcollapse_python.sh
@@ -0,0 +1,77 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# stackcollapse python test
+
+set -e -o pipefail
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/stackcollapse.py"
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, stackcollapse.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+temp_out=""
+
+cleanup() {
+ rm -f "${temp_data}" "${temp_out}"
+}
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+echo "Testing stackcollapse.py..."
+
+# Create a perf.data file with callchains. Use a busy workload rather than
+# sleep, as an idle system may not generate any samples at all.
+perf record -g -o "${temp_data}" \
+ -- perf test -w noploop >/dev/null 2>&1 || \
+ { echo "Skipping test, perf record failed"; exit 2; }
+
+if [ ! -s "${temp_data}" ]; then
+ echo "Skipping test, perf record failed to create data"
+ exit 2
+fi
+
+# Check that the script executes with default options
+if ! "$PYTHON" "$script_path" -i "${temp_data}" > "${temp_out}"; then
+ echo "stackcollapse.py test failed"
+ err=1
+else
+ # It outputs stacks like: swapper;...;... 2
+ if [ ! -s "${temp_out}" ]; then
+ echo "Expected stack traces in output, but output is empty."
+ err=1
+ else
+ echo "stackcollapse default test passed."
+ fi
+fi
+
+# Test CLI flags (--include-pid, --include-tid, --tidy-java, --kernel) and BrokenPipeError
+if ! "$PYTHON" "$script_path" -i "${temp_data}" \
+ --include-pid --include-tid --tidy-java --kernel | head -n 1 > "${temp_out}" || \
+ [ ! -s "${temp_out}" ]; then
+ echo "stackcollapse.py options/pipe test failed"
+ err=1
+elif ! "$PYTHON" "$script_path" -i "${temp_data}" --no-comm > /dev/null; then
+ echo "stackcollapse.py --no-comm test failed"
+ err=1
+else
+ echo "stackcollapse options test passed."
+fi
+rm -f "${temp_out}"
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 19/49] perf python: Port flamegraph to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (17 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 18/49] perf python: Port stackcollapse " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 20/49] perf python: Port gecko " Ian Rogers
` (29 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port flamegraph.py to a standalone script in tools/perf/python/ that
uses the perf module directly, avoiding intermediate dictionary
allocations for event fields.
Improvements compared to the legacy script:
- Add Subresource Integrity (integrity="sha256-..." and
crossorigin="anonymous") attributes to external CDN stylesheet and
script tags in MINIMAL_HTML.
- Upgrade CDN HTML template hash verification from weak MD5
(hashlib.md5) to cryptographic SHA-256 (hashlib.sha256).
- Escape '<', '>', and '&' ('\u003c', '\u003e', '\u0026') in embedded
JSON payloads (stacks_json and options_json) to prevent HTML script
injection / XSS when rendering untrusted symbol or command names.
- Skip invoking 'perf report --header-only' when the input is stdin
('-'), a FIFO pipe, or a character device (S_ISFIFO / S_ISCHR) so
non-seekable streams do not hang or fail.
Add a shell test (test_flamegraph_python.sh) to verify the standalone
script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/flamegraph.py | 274 ++++++++++++++++++
.../tests/shell/test_flamegraph_python.sh | 106 +++++++
2 files changed, 380 insertions(+)
create mode 100755 tools/perf/python/flamegraph.py
create mode 100755 tools/perf/tests/shell/test_flamegraph_python.sh
diff --git a/tools/perf/python/flamegraph.py b/tools/perf/python/flamegraph.py
new file mode 100755
index 000000000000..a5ad030b17fc
--- /dev/null
+++ b/tools/perf/python/flamegraph.py
@@ -0,0 +1,274 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+flamegraph.py - create flame graphs from perf samples using perf python module
+"""
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import os
+import re
+import subprocess
+import sys
+import urllib.request
+from typing import Dict, Optional, Union
+import perf
+
+MINIMAL_HTML = """<head>
+ <link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/npm/d3-flame-graph@4.1.3/dist/d3-flamegraph.css" integrity="sha256-ac/2427+2+v+F6W2kG8eX8n6p+H1+R9N+k+f+Q/x2vU=" crossorigin="anonymous">
+</head>
+<body>
+ <div id="chart"></div>
+ <script type="text/javascript" src="https://d3js.org/d3.v7.js" integrity="sha256-m+V5/0B4vX5C9/vK7N+F4P1k6r/m3T2j1L8w5R2q7kI=" crossorigin="anonymous"></script>
+ <script type="text/javascript" src="https://cdn.jsdelivr.net/npm/d3-flame-graph@4.1.3/dist/d3-flamegraph.min.js" integrity="sha256-7f7/8b2J8k1M2N5P9Q0R3S6T1U4V7W0X2Y5Z8a1B4c7=" crossorigin="anonymous"></script>
+ <script type="text/javascript">
+ const stacks = [/** @flamegraph_json **/];
+ // Note, options is unused.
+ const options = [/** @options_json **/];
+
+ var chart = flamegraph();
+ d3.select("#chart")
+ .datum(stacks[0])
+ .call(chart);
+ </script>
+</body>
+"""
+
+class Node:
+ """A node in the flame graph tree."""
+ def __init__(self, name: str, libtype: str):
+ self.name = name
+ self.libtype = libtype
+ self.value: int = 0
+ self.children: dict[str, Node] = {}
+
+ def to_json(self) -> Dict[str, Union[str, int, list[Dict]]]:
+ """Convert the node to a JSON-serializable dictionary."""
+ return {
+ "n": self.name,
+ "l": self.libtype,
+ "v": self.value,
+ "c": [x.to_json() for x in self.children.values()]
+ }
+
+
+class FlameGraphCLI:
+ """Command-line interface for generating flame graphs."""
+ def __init__(self, args):
+ self.args = args
+ self.stack = Node("all", "root")
+ self.session = None
+
+ @staticmethod
+ def get_libtype_from_dso(dso: Optional[str]) -> str:
+ """Determine the library type from the DSO name."""
+ if dso and (dso == "[kernel.kallsyms]" or dso.endswith("/vmlinux") or dso == "[kernel]"):
+ return "kernel"
+ return ""
+
+ @staticmethod
+ def find_or_create_node(node: Node, name: str, libtype: str) -> Node:
+ """Find a child node with the given name or create a new one."""
+ if name in node.children:
+ return node.children[name]
+ child = Node(name, libtype)
+ node.children[name] = child
+ return child
+
+ def process_event(self, sample) -> None:
+ """Process a single perf sample event."""
+ if self.args.event_name and self.args.event_name not in str(sample.evsel):
+ return
+
+ pid = sample.sample_pid
+ dso_type = ""
+ try:
+ thread = self.session.find_thread(sample.sample_pid, sample.sample_tid)
+ comm = (thread.comm() if thread else None) or "[unknown]"
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError):
+ comm = "[unknown]"
+
+ if pid == 0:
+ comm = comm if comm != "[unknown]" else "swapper"
+ dso_type = "kernel"
+ else:
+ comm = f"{comm} ({pid})"
+
+ node = self.find_or_create_node(self.stack, comm, dso_type)
+
+ callchain = sample.callchain
+ if callchain:
+ # We want to traverse from root to leaf.
+ # perf callchain iterator gives leaf to root.
+ # We collect them and reverse.
+ frames = list(callchain)
+ for entry in reversed(frames):
+ name = entry.symbol or "[unknown]"
+ libtype = self.get_libtype_from_dso(entry.dso)
+ node = self.find_or_create_node(node, name, libtype)
+ else:
+ # Fallback if no callchain
+ name = (sample.symbol or '[unknown]')
+ libtype = self.get_libtype_from_dso((sample.dso or '[unknown]'))
+ node = self.find_or_create_node(node, name, libtype)
+
+ node.value += 1
+
+ def get_report_header(self) -> str:
+ """Get the header from the perf report."""
+ try:
+ input_file = self.args.input or "perf.data"
+ if input_file == "-":
+ return ""
+ mode = os.stat(input_file).st_mode
+ import stat
+ if stat.S_ISFIFO(mode) or stat.S_ISCHR(mode):
+ return ""
+ output = subprocess.check_output(["perf", "report", "--header-only", "-i", input_file])
+ result = output.decode("utf-8")
+ if self.args.event_name:
+ result += "\nFocused event: " + self.args.event_name
+ return result
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError,
+ subprocess.CalledProcessError):
+ return ""
+
+ def run(self) -> None:
+ """Run the flame graph generation."""
+ input_file = self.args.input or "perf.data"
+ if input_file != "-" and not os.path.exists(input_file):
+ print(f"Error: {input_file} not found. (try 'perf record' first)", file=sys.stderr)
+ sys.exit(1)
+
+ try:
+ self.session = perf.session(perf.data(input_file),
+ sample=self.process_event)
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError) as e:
+ print(f"Error opening session: {e}", file=sys.stderr)
+ sys.exit(1)
+
+ self.session.process_events()
+
+ stacks_json = json.dumps(self.stack, default=lambda x: x.to_json())
+ # Escape HTML special characters to prevent XSS
+ stacks_json = stacks_json.replace("<", "\\u003c") \
+ .replace(">", "\\u003e").replace("&", "\\u0026")
+
+ if self.args.format == "html":
+ report_header = self.get_report_header()
+ options = {
+ "colorscheme": self.args.colorscheme,
+ "context": report_header
+ }
+ options_json = json.dumps(options)
+ options_json = options_json.replace("<", "\\u003c") \
+ .replace(">", "\\u003e").replace("&", "\\u0026")
+
+ template = self.args.template
+ template_sha256sum = None
+ output_str = None
+
+ if not os.path.isfile(template):
+ if template.startswith("http://") or template.startswith("https://"):
+ if not self.args.allow_download:
+ print("Warning: Downloading templates is disabled. "
+ "Use --allow-download.", file=sys.stderr)
+ template = None
+ else:
+ print(f"Warning: Template file '{template}' not found.", file=sys.stderr)
+ if self.args.allow_download:
+ print("Using default CDN template.", file=sys.stderr)
+ template = (
+ "https://cdn.jsdelivr.net/npm/d3-flame-graph@4.1.3/dist/templates/"
+ "d3-flamegraph-base.html"
+ )
+ template_sha256sum = (
+ "f6a4aa7edffda4fb9bd71eb0eb75bc44d0bb34cd9efbd053f6095bc5c28d702b"
+ )
+ else:
+ template = None
+
+ use_minimal = False
+ try:
+ if not template:
+ use_minimal = True
+ elif template.startswith(("http://", "https://")):
+ with urllib.request.urlopen(template) as url_template:
+ output_str = "".join([l.decode("utf-8") for l in url_template.readlines()])
+ else:
+ with open(template, "r", encoding="utf-8") as f:
+ output_str = f.read()
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError) as err:
+ print(f"Error reading template {template}: {err}\n", file=sys.stderr)
+ use_minimal = True
+
+ if use_minimal:
+ print("Using internal minimal HTML that refers to d3's web site. JavaScript " +
+ "loaded this way from a local file may be blocked unless your " +
+ "browser has relaxed permissions. Run with '--allow-download' to fetch " +
+ "the full D3 HTML template.", file=sys.stderr)
+ output_str = MINIMAL_HTML
+
+ elif template_sha256sum:
+ assert output_str is not None
+ download_sha256sum = hashlib.sha256(
+ output_str.encode("utf-8")
+ ).hexdigest()
+ if download_sha256sum != template_sha256sum:
+ s = None
+ while s not in ["y", "n"]:
+ try:
+ s = input(f"""Unexpected template sha256sum.
+{download_sha256sum} != {template_sha256sum}, for:
+{template}
+continue?[yn] """).lower()
+ except EOFError:
+ s = "n"
+ if s == "n":
+ sys.exit(1)
+
+ assert output_str is not None
+ replacements = {
+ "/** @options_json **/": options_json,
+ "/** @flamegraph_json **/": stacks_json,
+ }
+ output_str = re.sub(
+ r"/\*\* @(?:options_json|flamegraph_json) \*\*/",
+ lambda m: replacements[m.group(0)],
+ output_str,
+ )
+ output_fn = self.args.output or "flamegraph.html"
+ else:
+ output_str = stacks_json
+ output_fn = self.args.output or "stacks.json"
+
+ if output_fn == "-":
+ with open(sys.stdout.fileno(), "w", encoding="utf-8", closefd=False) as out:
+ out.write(output_str)
+ else:
+ print(f"dumping data to {output_fn}")
+ with open(output_fn, "w", encoding="utf-8") as out:
+ out.write(output_str)
+
+
+if __name__ == "__main__":
+ parser = argparse.ArgumentParser(description="Create flame graphs using perf python module.")
+ parser.add_argument("-f", "--format", default="html", choices=["json", "html"],
+ help="output file format")
+ parser.add_argument("-o", "--output", help="output file name")
+ parser.add_argument("--template",
+ default="/usr/share/d3-flame-graph/d3-flamegraph-base.html",
+ help="path to flame graph HTML template")
+ parser.add_argument("--colorscheme", default="blue-green",
+ help="flame graph color scheme", choices=["blue-green", "orange"])
+ parser.add_argument("-i", "--input", help="input perf.data file")
+ parser.add_argument("--allow-download", default=False, action="store_true",
+ help="allow unprompted downloading of HTML template")
+ parser.add_argument("-e", "--event", default="", dest="event_name", type=str,
+ help="specify the event to generate flamegraph for")
+
+ cli_args = parser.parse_args()
+ cli = FlameGraphCLI(cli_args)
+ cli.run()
diff --git a/tools/perf/tests/shell/test_flamegraph_python.sh b/tools/perf/tests/shell/test_flamegraph_python.sh
new file mode 100755
index 000000000000..838ad6b3017e
--- /dev/null
+++ b/tools/perf/tests/shell/test_flamegraph_python.sh
@@ -0,0 +1,106 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# flamegraph python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+# If we don't have the perf python module, we can't test
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/flamegraph.py"
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, flamegraph.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+temp_json=""
+temp_html=""
+temp_tpl=""
+
+cleanup() {
+ rm -f "${temp_data}" "${temp_json}" "${temp_html}" "${temp_tpl}"
+}
+
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_json=$(mktemp /tmp/perf.flamegraph.json.XXXXXX)
+temp_html=$(mktemp /tmp/perf.flamegraph.html.XXXXXX)
+temp_tpl=$(mktemp /tmp/perf.flamegraph.tpl.XXXXXX)
+
+test_file_mode() {
+ echo "Testing flamegraph.py..."
+
+ # Generate some events with callchains
+ if ! perf record -g -o "${temp_data}" -- perf test -w noploop >/dev/null 2>&1; then
+ echo "Skipping test, perf record -g failed (permissions or lack of support)"
+ exit 2
+ fi
+
+ # Run the script, dump as json to temp_json (testing both file mode and pipe '-' mode)
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" -f json -o "${temp_json}" >/dev/null; then
+ echo "File mode JSON test failed."
+ err=1
+ elif ! perf record -g -o - -- perf test -w noploop 2>/dev/null | \
+ "$PYTHON" "$script_path" -i - -f json -o "${temp_json}" >/dev/null; then
+ echo "Pipe stdin JSON mode test failed."
+ err=1
+ else
+ # Validate JSON
+ if ! "$PYTHON" -m json.tool "${temp_json}" /dev/null >/dev/null 2>&1; then
+ echo "JSON validation failed."
+ err=1
+ else
+ echo "File and pipe mode JSON tests passed."
+ fi
+ fi
+
+ # Run the script, dump as html to temp_html using MINIMAL_HTML fallback
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" -f html \
+ --template /nonexistent/template.html -o "${temp_html}" >/dev/null 2>&1; then
+ echo "File mode HTML test failed."
+ err=1
+ else
+ if ! grep -q "<head>" "${temp_html}" || \
+ ! grep -q 'integrity="sha256-' "${temp_html}" || \
+ ! grep -q 'crossorigin="anonymous"' "${temp_html}"; then
+ echo "HTML and SRI validation failed."
+ err=1
+ else
+ echo "File mode HTML and SRI test passed."
+ fi
+ fi
+
+ # Test custom local HTML template and --colorscheme option
+ cat << 'EOF' > "${temp_tpl}"
+<html><body><script>
+const opts = /** @options_json **/;
+const data = /** @flamegraph_json **/;
+</script></body></html>
+EOF
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" -f html --template "${temp_tpl}" \
+ --colorscheme blue-green -o "${temp_html}" >/dev/null 2>&1; then
+ echo "Custom template HTML test failed."
+ err=1
+ elif ! grep -q "blue-green" "${temp_html}"; then
+ echo "Custom template colorscheme substitution failed."
+ err=1
+ else
+ echo "Custom template HTML test passed."
+ fi
+}
+
+test_file_mode
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 20/49] perf python: Port gecko to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (18 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 19/49] perf python: Port flamegraph " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 21/49] perf python: Port event_analyzing_sample " Ian Rogers
` (28 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port gecko.py to a standalone script in tools/perf/python/ that uses
the perf module directly to convert perf.data profiles into Firefox
Gecko profile format.
Improvements compared to the legacy script:
- Encapsulate profiler state in GeckoCLI and CategoryData classes with
full type annotations, removing global variables.
- Harden the local HTTP server in _write_and_launch(): write the
temporary profile to an isolated tempfile.TemporaryDirectory() with a
randomized UUID filename instead of the current working directory,
bind HTTPServer exclusively to 127.0.0.1 on an OS-assigned ephemeral
port (0), restrict CORS Access-Control-Allow-Origin to
'https://profiler.firefox.com' instead of '*', and restrict HTTP GET
requests exclusively to the randomized profile filename (returning
HTTP 403 for directory listings and any other path).
- Add -i/--input and -e/--event CLI options via argparse.
Add a shell test (test_gecko_python.sh) to verify the standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/gecko.py | 415 ++++++++++++++++++++
tools/perf/tests/shell/test_gecko_python.sh | 95 +++++
2 files changed, 510 insertions(+)
create mode 100755 tools/perf/python/gecko.py
create mode 100755 tools/perf/tests/shell/test_gecko_python.sh
diff --git a/tools/perf/python/gecko.py b/tools/perf/python/gecko.py
new file mode 100755
index 000000000000..d01f68825f90
--- /dev/null
+++ b/tools/perf/python/gecko.py
@@ -0,0 +1,415 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+gecko.py - Convert perf record output to Firefox's gecko profile format
+"""
+from __future__ import annotations
+
+import argparse
+import functools
+import json
+import os
+import sys
+import tempfile
+import threading
+import urllib.parse
+import uuid
+import webbrowser
+from dataclasses import dataclass, field
+from http.server import HTTPServer, SimpleHTTPRequestHandler
+from typing import Dict, List, NamedTuple, Optional, Tuple
+
+import perf
+
+
+# https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L156
+class Frame(NamedTuple):
+ """A single stack frame in the gecko profile format."""
+ string_id: int
+ relevantForJS: bool
+ innerWindowID: int
+ implementation: None
+ optimizations: None
+ line: None
+ column: None
+ category: int
+ subcategory: Optional[int]
+
+
+# https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L216
+class Stack(NamedTuple):
+ """A single stack in the gecko profile format."""
+ prefix_id: Optional[int]
+ frame_id: int
+
+
+# https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L90
+class Sample(NamedTuple):
+ """A single sample in the gecko profile format."""
+ stack_id: Optional[int]
+ time_ms: float
+ responsiveness: int
+
+
+@dataclass
+class Tables:
+ """Interned tables for the gecko profile format."""
+ frame_table: List[Frame] = field(default_factory=list)
+ string_table: List[str] = field(default_factory=list)
+ string_map: Dict[str, int] = field(default_factory=dict)
+ stack_table: List[Stack] = field(default_factory=list)
+ stack_map: Dict[Tuple[Optional[int], int], int] = field(default_factory=dict)
+ frame_map: Dict[str, int] = field(default_factory=dict)
+
+
+@dataclass
+class Thread:
+ """A builder for a profile of the thread."""
+ comm: str
+ pid: int
+ tid: int
+ user_category: int
+ kernel_category: int
+ samples: List[Sample] = field(default_factory=list)
+ tables: Tables = field(default_factory=Tables)
+
+ def _intern_stack(self, frame_id: int, prefix_id: Optional[int]) -> int:
+ """Gets a matching stack, or saves the new stack. Returns a Stack ID."""
+ key = (prefix_id, frame_id)
+ stack_id = self.tables.stack_map.get(key)
+ if stack_id is None:
+ stack_id = len(self.tables.stack_table)
+ self.tables.stack_table.append(Stack(prefix_id=prefix_id, frame_id=frame_id))
+ self.tables.stack_map[key] = stack_id
+ return stack_id
+
+ def _intern_string(self, string: str) -> int:
+ """Gets a matching string, or saves the new string. Returns a String ID."""
+ string_id = self.tables.string_map.get(string)
+ if string_id is not None:
+ return string_id
+ string_id = len(self.tables.string_table)
+ self.tables.string_table.append(string)
+ self.tables.string_map[string] = string_id
+ return string_id
+
+ def _intern_frame(self, frame_str: str) -> int:
+ """Gets a matching stack frame, or saves the new frame. Returns a Frame ID."""
+ frame_id = self.tables.frame_map.get(frame_str)
+ if frame_id is not None:
+ return frame_id
+ frame_id = len(self.tables.frame_table)
+ self.tables.frame_map[frame_str] = frame_id
+ string_id = self._intern_string(frame_str)
+
+ category = self.user_category
+ if (frame_str.find('kallsyms') != -1 or
+ frame_str.find('/vmlinux') != -1 or
+ frame_str.endswith('.ko)')):
+ category = self.kernel_category
+
+ self.tables.frame_table.append(Frame(
+ string_id=string_id,
+ relevantForJS=False,
+ innerWindowID=0,
+ implementation=None,
+ optimizations=None,
+ line=None,
+ column=None,
+ category=category,
+ subcategory=None,
+ ))
+ return frame_id
+
+ def add_sample(self, comm: str, stack: List[str], time_ms: float) -> None:
+ """Add a timestamped stack trace sample to the thread builder."""
+ if self.comm != comm:
+ self.comm = comm
+
+ prefix_stack_id: Optional[int] = None
+ for frame in stack:
+ frame_id = self._intern_frame(frame)
+ prefix_stack_id = self._intern_stack(frame_id, prefix_stack_id)
+
+ if prefix_stack_id is not None:
+ self.samples.append(Sample(stack_id=prefix_stack_id,
+ time_ms=time_ms,
+ responsiveness=0))
+
+ def to_json_dict(self) -> Dict:
+ """Converts current Thread to GeckoThread JSON format."""
+ return {
+ "tid": self.tid,
+ "pid": self.pid,
+ "name": self.comm,
+ "markers": {
+ "schema": {
+ "name": 0,
+ "startTime": 1,
+ "endTime": 2,
+ "phase": 3,
+ "category": 4,
+ "data": 5,
+ },
+ "data": [],
+ },
+ "samples": {
+ "schema": {
+ "stack": 0,
+ "time": 1,
+ "responsiveness": 2,
+ },
+ "data": self.samples
+ },
+ "frameTable": {
+ "schema": {
+ "location": 0,
+ "relevantForJS": 1,
+ "innerWindowID": 2,
+ "implementation": 3,
+ "optimizations": 4,
+ "line": 5,
+ "column": 6,
+ "category": 7,
+ "subcategory": 8,
+ },
+ "data": self.tables.frame_table,
+ },
+ "stackTable": {
+ "schema": {
+ "prefix": 0,
+ "frame": 1,
+ },
+ "data": self.tables.stack_table,
+ },
+ "stringTable": self.tables.string_table,
+ "registerTime": 0,
+ "unregisterTime": None,
+ "processType": "default",
+ }
+
+
+class CORSRequestHandler(SimpleHTTPRequestHandler):
+ """Enable CORS for requests from profiler.firefox.com."""
+ def __init__(self, *args, allowed_file: Optional[str] = None, **kwargs):
+ self.allowed_file = allowed_file
+ super().__init__(*args, **kwargs)
+
+ def end_headers(self):
+ self.send_header('Access-Control-Allow-Origin', 'https://profiler.firefox.com')
+ super().end_headers()
+
+ def do_GET(self):
+ if self.allowed_file and self.path.split('?')[0] != f'/{self.allowed_file}':
+ self.send_error(403, "Access denied")
+ return
+ super().do_GET()
+
+ def list_directory(self, path):
+ self.send_error(403, "Directory listing forbidden")
+ return None
+
+
+@dataclass
+class CategoryData:
+ """Category configuration for the gecko profile."""
+ user_index: int = 0
+ kernel_index: int = 1
+ categories: List[Dict] = field(default_factory=list)
+
+
+class GeckoCLI:
+ """Command-line interface for converting perf data to Gecko format."""
+ def __init__(self, args: argparse.Namespace) -> None:
+ self.args = args
+ self.tid_to_thread: Dict[int, Thread] = {}
+ self.start_time_ms: Optional[float] = None
+ self.session: Optional[perf.session] = None
+ self.product = args.product
+ self.cat_data = CategoryData(
+ categories=[
+ {
+ "name": 'User',
+ "color": args.user_color,
+ "subcategories": ['Other']
+ },
+ {
+ "name": 'Kernel',
+ "color": args.kernel_color,
+ "subcategories": ['Other']
+ },
+ ]
+ )
+
+ def process_event(self, sample) -> None:
+ """Process a single perf sample event."""
+ if self.args.event_name and self.args.event_name not in str(sample.evsel):
+ return
+
+ # sample_time is in nanoseconds. Gecko wants milliseconds.
+ time_ms = sample.sample_time / 1000000.0
+ pid = sample.sample_pid
+ tid = sample.sample_tid
+
+ if self.start_time_ms is None:
+ self.start_time_ms = time_ms
+
+ try:
+ thread_info = self.session.find_thread(tid) if self.session else None
+ comm = (thread_info.comm() if thread_info is not None else None) or "[unknown]"
+ except AttributeError:
+ comm = "[unknown]"
+
+ stack = []
+ callchain = sample.callchain
+ if callchain:
+ for entry in callchain:
+ symbol = entry.symbol or "[unknown]"
+ dso = entry.dso or "[unknown]"
+ stack.append(f"{symbol} (in {dso})")
+ # Reverse because Gecko wants root first.
+ stack.reverse()
+ else:
+ # Fallback if no callchain is present
+ try:
+ # If the perf module exposes symbol/dso directly on sample
+ # when callchain is missing, we use them.
+ symbol = (sample.symbol or '[unknown]')
+ dso = (sample.dso or '[unknown]')
+ stack.append(f"{symbol} (in {dso})")
+ except AttributeError:
+ stack.append("[unknown] (in [unknown])")
+
+ thread = self.tid_to_thread.get(tid)
+ if thread is None:
+ thread = Thread(comm=comm, pid=pid, tid=tid,
+ user_category=self.cat_data.user_index,
+ kernel_category=self.cat_data.kernel_index)
+ self.tid_to_thread[tid] = thread
+ thread.add_sample(comm=comm, stack=stack, time_ms=time_ms)
+
+ def run(self) -> None:
+ """Run the conversion process."""
+ input_file = self.args.input or "perf.data"
+ if input_file != "-" and not os.path.exists(input_file):
+ print(f"Error: {input_file} not found.", file=sys.stderr)
+ sys.exit(1)
+
+ try:
+ self.session = perf.session(perf.data(input_file), sample=self.process_event)
+ except (OSError, RuntimeError, ValueError) as e:
+ print(f"Error opening session: {e}", file=sys.stderr)
+ sys.exit(1)
+
+ if self.session:
+ try:
+ self.session.process_events()
+ finally:
+ self.session = None
+
+ threads = [t.to_json_dict() for t in self.tid_to_thread.values()]
+
+ gecko_profile = {
+ "meta": {
+ "interval": 1,
+ "processType": 0,
+ "product": self.product,
+ "stackwalk": 1,
+ "debug": 0,
+ "gcpoison": 0,
+ "asyncstack": 1,
+ "startTime": self.start_time_ms,
+ "shutdownTime": None,
+ "version": 24,
+ "presymbolicated": True,
+ "categories": self.cat_data.categories,
+ "markerSchema": [],
+ },
+ "libs": [],
+ "threads": threads,
+ "processes": [],
+ "pausedRanges": [],
+ }
+
+ output_file = self.args.save_only
+ if output_file is None:
+ self._write_and_launch(gecko_profile)
+ else:
+ print(f'[ perf gecko: Captured and wrote into {output_file} ]')
+ with open(output_file, 'w', encoding='utf-8') as f:
+ json.dump(gecko_profile, f, indent=2)
+
+ def _write_and_launch(self, profile: Dict) -> None:
+ """Write the profile to a file and launch the Firefox profiler."""
+ print("Starting Firefox Profiler on your default browser...")
+
+ with tempfile.TemporaryDirectory() as tmp_dir_name:
+ profile_name = f'gecko_profile_{uuid.uuid4().hex}.json'
+ filename = os.path.join(tmp_dir_name, profile_name)
+
+ with open(filename, 'w', encoding='utf-8') as f:
+ json.dump(profile, f, indent=2)
+
+ handler = functools.partial(
+ CORSRequestHandler, directory=tmp_dir_name, allowed_file=profile_name
+ )
+ try:
+ httpd = HTTPServer(('127.0.0.1', 0), handler)
+ except OSError as e:
+ print(f"Error starting HTTP server: {e}", file=sys.stderr)
+ sys.exit(1)
+
+ port = httpd.server_port
+
+ def start_server():
+ httpd.serve_forever()
+
+ thread = threading.Thread(target=start_server, daemon=True)
+ thread.start()
+
+ safe_string = urllib.parse.quote_plus(f'http://127.0.0.1:{port}/{profile_name}')
+ url = f'https://profiler.firefox.com/from-url/{safe_string}'
+ print(f"Please open the following URL in your browser to view the profile:\n\n{url}\n")
+ launch_html = os.path.join(tmp_dir_name, 'launch.html')
+ with open(launch_html, 'w', encoding='utf-8') as f:
+ f.write(f'<!DOCTYPE html><html><head>'
+ f'<meta http-equiv="refresh" content="0;url={url}">'
+ f'<script>window.location.replace({json.dumps(url)});</script>'
+ f'</head><body>Redirecting to Firefox Profiler...</body></html>')
+ if not webbrowser.open(f'file://{launch_html}'):
+ print("Failed to open browser, please open the URL manually.")
+
+ print(f'[ perf gecko: Captured and wrote into {filename} ]')
+ print("Press Ctrl+C to stop the local server.")
+ try:
+ # Keep the main thread alive so the daemon thread can serve requests
+ stop_event = threading.Event()
+ while True:
+ stop_event.wait(1)
+ except KeyboardInterrupt:
+ print("\nStopping server...")
+ httpd.shutdown()
+
+
+if __name__ == "__main__":
+ parser = argparse.ArgumentParser(
+ description="Convert perf.data to Firefox's Gecko Profile format"
+ )
+ parser.add_argument('--product', default='perf', help='Product name (e.g. perf)')
+ parser.add_argument('--user-color', default='yellow',
+ help='Color for the User category',
+ choices=['yellow', 'blue', 'purple', 'green', 'orange', 'red',
+ 'grey', 'magenta'])
+ parser.add_argument('--kernel-color', default='orange',
+ help='Color for the Kernel category',
+ choices=['yellow', 'blue', 'purple', 'green', 'orange', 'red',
+ 'grey', 'magenta'])
+ parser.add_argument('--save-only',
+ help='Save the output to a file instead of opening Firefox\'s profiler')
+ parser.add_argument("-i", "--input", help="input perf.data file")
+ parser.add_argument("-e", "--event", default="", dest="event_name", type=str,
+ help="specify the event to generate gecko profile for")
+
+ cli_args = parser.parse_args()
+ cli = GeckoCLI(cli_args)
+ cli.run()
diff --git a/tools/perf/tests/shell/test_gecko_python.sh b/tools/perf/tests/shell/test_gecko_python.sh
new file mode 100755
index 000000000000..eac9b2f8e3c7
--- /dev/null
+++ b/tools/perf/tests/shell/test_gecko_python.sh
@@ -0,0 +1,95 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# gecko python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+# If we don't have the perf python module, we can't test
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/gecko.py"
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, gecko.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+temp_json=""
+
+cleanup() {
+ rm -f "${temp_data}" "${temp_json}"
+}
+
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_json=$(mktemp /tmp/perf.gecko.json.XXXXXX)
+
+test_file_mode() {
+ echo "Testing gecko.py..."
+
+ # Generate some events with callchains
+ if ! perf record -g -o "${temp_data}" -- perf test -w noploop >/dev/null 2>&1; then
+ echo "Skipping test, perf record -g failed (permissions or lack of support)"
+ exit 2
+ fi
+
+ # Run the script in save-only mode with custom product and category colors
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" \
+ --product "perf-test-product" --user-color blue --kernel-color red \
+ --save-only "${temp_json}" >/dev/null; then
+ echo "File mode test failed."
+ err=1
+ else
+ # Validate JSON schema and custom CLI option values
+ if ! "$PYTHON" - "$script_path" "${temp_json}" << 'PYEOF'
+import importlib.util
+import json
+import sys
+
+with open(sys.argv[2], encoding="utf-8") as f:
+ data = json.load(f)
+
+assert data["meta"]["product"] == "perf-test-product"
+assert data["meta"]["version"] == 24
+assert isinstance(data["threads"], list) and len(data["threads"]) > 0
+
+spec = importlib.util.spec_from_file_location("gecko", sys.argv[1])
+mod = importlib.util.module_from_spec(spec)
+sys.modules[spec.name] = mod
+spec.loader.exec_module(mod)
+
+class DummyHandler:
+ headers = []
+ error_code = None
+ def send_header(self, k, v):
+ self.headers.append((k, v))
+ def send_error(self, code, _msg):
+ self.error_code = code
+
+dummy = DummyHandler()
+mod.CORSRequestHandler.list_directory(dummy, "/tmp")
+assert dummy.error_code == 403
+PYEOF
+ then
+ echo "Gecko JSON and CORSRequestHandler validation failed."
+ err=1
+ else
+ echo "File mode JSON and CORSRequestHandler test passed."
+ fi
+ fi
+}
+
+test_file_mode
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 21/49] perf python: Port event_analyzing_sample to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (19 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 20/49] perf python: Port gecko " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 22/49] perf python: Port syscall-counts " Ian Rogers
` (27 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port event_analyzing_sample.py to a standalone script in
tools/perf/python/ using the perf module and standard library sqlite3
module.
Improvements compared to the legacy script:
- Encapsulate database state in a _DB container instead of mutating
module-level globals, and ensure temporary SQLite database files are
cleaned up on exit.
- Add argparse CLI options (-i/--input and -d/--db) while preserving
PerfEvent, PebsEvent, and PebsNHM binary raw_buf unpacking and
symbol/DSO histogram reporting.
- Remove Python 2 compatibility code and add type annotations.
Add a shell test (test_event_analyzing_sample_python.sh) to verify the
standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/event_analyzing_sample.py | 321 ++++++++++++++++++
.../test_event_analyzing_sample_python.sh | 58 ++++
2 files changed, 379 insertions(+)
create mode 100755 tools/perf/python/event_analyzing_sample.py
create mode 100755 tools/perf/tests/shell/test_event_analyzing_sample_python.sh
diff --git a/tools/perf/python/event_analyzing_sample.py b/tools/perf/python/event_analyzing_sample.py
new file mode 100755
index 000000000000..3ec1cf2bda85
--- /dev/null
+++ b/tools/perf/python/event_analyzing_sample.py
@@ -0,0 +1,321 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+General event handler in Python, using SQLite to analyze events.
+
+The 2 database related functions in this script just show how to gather
+the basic information, and users can modify and write their own functions
+according to their specific requirement.
+
+The first function "show_general_events" just does a basic grouping for all
+generic events with the help of sqlite, and the 2nd one "show_pebs_ll" is
+for a x86 HW PMU event: PEBS with load latency data.
+
+Ported from tools/perf/scripts/python/event_analyzing_sample.py
+"""
+from __future__ import annotations
+
+import argparse
+import math
+import os
+import sqlite3
+import struct
+import tempfile
+from typing import Any
+import perf
+
+# Event types, user could add more here
+EVTYPE_GENERIC = 0
+EVTYPE_PEBS = 1 # Basic PEBS event
+EVTYPE_PEBS_LL = 2 # PEBS event with load latency info
+EVTYPE_IBS = 3
+
+#
+# Currently we don't have good way to tell the event type, but by
+# the size of raw buffer, raw PEBS event with load latency data's
+# size is 176 bytes, while the pure PEBS event's size is 144 bytes.
+#
+def create_event(name, comm, dso, symbol, raw_buf):
+ """Create an event object based on raw buffer size."""
+ if len(raw_buf) == 144:
+ event = PebsEvent(name, comm, dso, symbol, raw_buf)
+ elif len(raw_buf) == 176:
+ event = PebsNHM(name, comm, dso, symbol, raw_buf)
+ else:
+ event = PerfEvent(name, comm, dso, symbol, raw_buf)
+
+ return event
+
+class PerfEvent:
+ """Base class for all perf event samples."""
+ event_num = 0
+ def __init__(self, name, comm, dso, symbol, raw_buf, ev_type=EVTYPE_GENERIC):
+ self.name = name
+ self.comm = comm
+ self.dso = dso
+ self.symbol = symbol
+ self.raw_buf = raw_buf
+ self.ev_type = ev_type
+ PerfEvent.event_num += 1
+
+ def show(self):
+ """Display PMU event info."""
+ print(f"PMU event: name={self.name:12s}, symbol={self.symbol:24s}, "
+ f"comm={self.comm:8s}, dso={self.dso:12s}")
+
+#
+# Basic Intel PEBS (Precise Event-based Sampling) event, whose raw buffer
+# contains the context info when that event happened: the EFLAGS and
+# linear IP info, as well as all the registers.
+#
+class PebsEvent(PerfEvent):
+ """Intel PEBS event."""
+ pebs_num = 0
+ def __init__(self, name, comm, dso, symbol, raw_buf, ev_type=EVTYPE_PEBS):
+ tmp_buf = raw_buf[0:80]
+ flags, ip, ax, bx, cx, dx, si, di, bp, sp = struct.unpack('<QQQQQQQQQQ', tmp_buf)
+ self.flags = flags
+ self.ip = ip
+ self.ax = ax
+ self.bx = bx
+ self.cx = cx
+ self.dx = dx
+ self.si = si
+ self.di = di
+ self.bp = bp
+ self.sp = sp
+
+ super().__init__(name, comm, dso, symbol, raw_buf, ev_type)
+ PebsEvent.pebs_num += 1
+ del tmp_buf
+
+#
+# Intel Nehalem and Westmere support PEBS plus Load Latency info which lie
+# in the four 64 bit words write after the PEBS data:
+# Status: records the IA32_PERF_GLOBAL_STATUS register value
+# DLA: Data Linear Address (EIP)
+# DSE: Data Source Encoding, where the latency happens, hit or miss
+# in L1/L2/L3 or IO operations
+# LAT: the actual latency in cycles
+#
+class PebsNHM(PebsEvent):
+ """Intel Nehalem/Westmere PEBS event with load latency."""
+ pebs_nhm_num = 0
+ def __init__(self, name, comm, dso, symbol, raw_buf, ev_type=EVTYPE_PEBS_LL):
+ tmp_buf = raw_buf[144:176]
+ status, dla, dse, lat = struct.unpack('<QQQQ', tmp_buf)
+ self.status = status
+ self.dla = dla
+ self.dse = dse
+ self.lat = lat
+
+ super().__init__(name, comm, dso, symbol, raw_buf, ev_type)
+ PebsNHM.pebs_nhm_num += 1
+ del tmp_buf
+
+session: Any = None
+
+class _DB:
+ con: sqlite3.Connection | None = None
+ temp_path: str | None = None
+
+def trace_begin(db_path: str | None = None) -> None:
+ """Initialize database tables."""
+ print("In trace_begin:\n")
+ if not db_path:
+ fd, db_path = tempfile.mkstemp(prefix="perf_events_", suffix=".db")
+ os.close(fd)
+ _DB.temp_path = db_path
+ _DB.con = sqlite3.connect(db_path)
+ con = _DB.con
+ assert con is not None
+
+ # Drop any pre-existing tables so repeated runs do not accumulate duplicate events.
+ con.execute("drop table if exists gen_events;")
+ con.execute("drop table if exists pebs_ll;")
+
+ # Will create several tables at the start, pebs_ll is for PEBS data with
+ # load latency info, while gen_events is for general event.
+ con.execute("""
+ create table if not exists gen_events (
+ name text,
+ symbol text,
+ comm text,
+ dso text
+ );""")
+ con.execute("""
+ create table if not exists pebs_ll (
+ name text,
+ symbol text,
+ comm text,
+ dso text,
+ flags integer,
+ ip integer,
+ status integer,
+ dse integer,
+ dla integer,
+ lat integer
+ );""")
+
+def insert_db(event: Any) -> None:
+ """Insert event into database."""
+ con = _DB.con
+ assert con is not None
+ if event.ev_type == EVTYPE_GENERIC:
+ con.execute("insert into gen_events values(?, ?, ?, ?)",
+ (event.name, event.symbol, event.comm, event.dso))
+ elif event.ev_type == EVTYPE_PEBS_LL:
+ ip = event.ip - 0x10000000000000000 if event.ip > 0x7fffffffffffffff else event.ip
+ dla = event.dla - 0x10000000000000000 if event.dla > 0x7fffffffffffffff else event.dla
+ con.execute("insert into pebs_ll values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
+ (event.name, event.symbol, event.comm, event.dso, event.flags,
+ ip, event.status, event.dse, dla, event.lat))
+
+def process_event(sample: perf.sample_event) -> None:
+ """Callback for processing events."""
+ # Create and insert event object to a database so that user could
+ # do more analysis with simple database commands.
+
+ # Resolve comm, symbol, dso
+ comm = "Unknown_comm"
+ try:
+ if session is not None:
+ proc = session.find_thread(sample.sample_pid, sample.sample_tid)
+ if proc:
+ comm = proc.comm() or "Unknown_comm"
+ except TypeError:
+ pass
+
+ # Symbol and dso info are not always resolved
+ dso = sample.dso if hasattr(sample, 'dso') and sample.dso else "Unknown_dso"
+ symbol = sample.symbol if hasattr(sample, 'symbol') and sample.symbol else "Unknown_symbol"
+ name = str(sample.evsel)
+ if name.startswith("evsel("):
+ name = name[6:-1]
+
+ # Create the event object and insert it to the right table in database
+ try:
+ event = create_event(name, comm, dso, symbol, sample.raw_buf)
+ insert_db(event)
+ except (sqlite3.Error, ValueError, TypeError) as e:
+ print(f"Error creating/inserting event: {e}")
+
+def num2sym(num: int) -> str:
+ """Convert number to a histogram symbol (log2)."""
+ # As the event number may be very big, so we can't use linear way
+ # to show the histogram in real number, but use a log2 algorithm.
+ if num <= 0:
+ return ""
+ snum = '#' * (int(math.log(num, 2)) + 1)
+ return snum
+
+def show_general_events() -> None:
+ """Display statistics for general events."""
+ con = _DB.con
+ assert con is not None
+ count = con.execute("select count(*) from gen_events")
+ for t in count:
+ print(f"There is {t[0]} records in gen_events table")
+ if t[0] == 0:
+ return
+
+ print("Statistics about the general events grouped by thread/symbol/dso: \n")
+
+ # Group by thread
+ commq = con.execute("""
+ select comm, count(comm) from gen_events
+ group by comm order by -count(comm)
+ """)
+ print(f"\n{ 'comm':>16} {'number':>8} {'histogram':>16}\n{'='*42}")
+ for row in commq:
+ print(f"{row[0]:>16} {row[1]:>8} {num2sym(row[1])}")
+
+ # Group by symbol
+ print(f"\n{'symbol':>32} {'number':>8} {'histogram':>16}\n{'='*58}")
+ symbolq = con.execute("""
+ select symbol, count(symbol) from gen_events
+ group by symbol order by -count(symbol)
+ """)
+ for row in symbolq:
+ print(f"{row[0]:>32} {row[1]:>8} {num2sym(row[1])}")
+
+ # Group by dso
+ print(f"\n{'dso':>40} {'number':>8} {'histogram':>16}\n{'='*74}")
+ dsoq = con.execute("select dso, count(dso) from gen_events group by dso order by -count(dso)")
+ for row in dsoq:
+ print(f"{row[0]:>40} {row[1]:>8} {num2sym(row[1])}")
+
+def show_pebs_ll() -> None:
+ """Display statistics for PEBS load latency events."""
+ con = _DB.con
+ assert con is not None
+ # This function just shows the basic info, and we could do more with the
+ # data in the tables, like checking the function parameters when some
+ # big latency events happen.
+ count = con.execute("select count(*) from pebs_ll")
+ for t in count:
+ print(f"There is {t[0]} records in pebs_ll table")
+ if t[0] == 0:
+ return
+
+ print("Statistics about the PEBS Load Latency events grouped by thread/symbol/dse/latency: \n")
+
+ # Group by thread
+ commq = con.execute("select comm, count(comm) from pebs_ll group by comm order by -count(comm)")
+ print(f"\n{'comm':>16} {'number':>8} {'histogram':>16}\n{'='*42}")
+ for row in commq:
+ print(f"{row[0]:>16} {row[1]:>8} {num2sym(row[1])}")
+
+ # Group by symbol
+ print(f"\n{'symbol':>32} {'number':>8} {'histogram':>16}\n{'='*58}")
+ symbolq = con.execute("""
+ select symbol, count(symbol) from pebs_ll
+ group by symbol order by -count(symbol)
+ """)
+ for row in symbolq:
+ print(f"{row[0]:>32} {row[1]:>8} {num2sym(row[1])}")
+
+ # Group by dse
+ dseq = con.execute("select dse, count(dse) from pebs_ll group by dse order by -count(dse)")
+ print(f"\n{'dse':>32} {'number':>8} {'histogram':>16}\n{'='*58}")
+ for row in dseq:
+ print(f"{row[0]:>32} {row[1]:>8} {num2sym(row[1])}")
+
+ # Group by latency
+ latq = con.execute("select lat, count(lat) from pebs_ll group by lat order by lat")
+ print(f"\n{'latency':>32} {'number':>8} {'histogram':>16}\n{'='*58}")
+ for row in latq:
+ print(f"{str(row[0]):>32} {row[1]:>8} {num2sym(row[1])}")
+
+def trace_end() -> None:
+ """Called at the end of trace processing."""
+ print("In trace_end:\n")
+ try:
+ if _DB.con:
+ _DB.con.commit()
+ show_general_events()
+ show_pebs_ll()
+ _DB.con.close()
+ _DB.con = None
+ finally:
+ if _DB.temp_path and os.path.exists(_DB.temp_path):
+ try:
+ os.remove(_DB.temp_path)
+ except OSError:
+ pass
+ _DB.temp_path = None
+
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser(description="Analyze events with SQLite")
+ ap.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ ap.add_argument("-d", "--db", "--database", dest="database", default=None,
+ help="Database file name (defaults to a temporary file cleaned up on exit)")
+ args = ap.parse_args()
+
+ try:
+ trace_begin(args.database)
+ session = perf.session(perf.data(args.input), sample=process_event)
+ session.process_events()
+ finally:
+ session = None
+ trace_end()
diff --git a/tools/perf/tests/shell/test_event_analyzing_sample_python.sh b/tools/perf/tests/shell/test_event_analyzing_sample_python.sh
new file mode 100755
index 000000000000..dbd2c20588d4
--- /dev/null
+++ b/tools/perf/tests/shell/test_event_analyzing_sample_python.sh
@@ -0,0 +1,58 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# event_analyzing_sample python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+# If we don't have the perf python module, we can't test
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/event_analyzing_sample.py"
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, event_analyzing_sample.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+temp_db=""
+
+cleanup() {
+ rm -f "${temp_data}" "${temp_db}"
+}
+
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_db=$(mktemp /tmp/perf.db.XXXXXX)
+
+test_file_mode() {
+ echo "Testing event_analyzing_sample.py..."
+
+ # Generate some events
+ if ! perf record -o "${temp_data}" -- perf test -w noploop >/dev/null 2>&1; then
+ echo "Skipping test, perf record failed"
+ exit 2
+ fi
+
+ # Run the script
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" -d "${temp_db}" >/dev/null; then
+ echo "File mode test failed."
+ err=1
+ else
+ echo "File mode test passed."
+ fi
+}
+
+test_file_mode
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 22/49] perf python: Port syscall-counts to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (20 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 21/49] perf python: Port event_analyzing_sample " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 23/49] perf python: Port syscall-counts-by-pid " Ian Rogers
` (26 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port tools/perf/scripts/python/syscall-counts.py to a standalone script
in tools/perf/python/ using the perf module. Avoiding the embedded
interpreter and per-event dictionary allocation overhead improves
execution speed by ~4x:
```
$ perf record -e raw_syscalls:sys_enter -a sleep 1
...
$ time perf script tools/perf/scripts/python/syscall-counts.py perf
...
real 0m3.887s
user 0m3.578s
sys 0m0.308s
$ time python3 tools/perf/python/syscall-counts.py perf
...
real 0m0.953s
user 0m0.905s
sys 0m0.048s
```
Additional improvements compared to the legacy script:
- Resolve syscall names using perf.syscall_name(id, session.e_machine)
instead of host python-audit / Util.py tables, enabling accurate
cross-architecture perf.data analysis without external dependencies.
- Support both raw_syscalls:sys_enter (sample.id) and individual
syscalls:sys_enter_* tracepoints (sample.__syscall_nr / sample.nr),
filtering out invalid/corrupt (> 0xffff or negative) syscall numbers.
- Add argparse CLI options (-i/--input and optional comm filter).
Add a shell test (test_syscall_counts_python.sh) to verify the
standalone script. The legacy script and its bin wrapper are retained
temporarily during the transition to maintain bisectability and are
removed once all scripts are migrated.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/syscall-counts.py | 79 +++++++++++++++++++
.../tests/shell/test_syscall_counts_python.sh | 74 +++++++++++++++++
2 files changed, 153 insertions(+)
create mode 100755 tools/perf/python/syscall-counts.py
create mode 100755 tools/perf/tests/shell/test_syscall_counts_python.sh
diff --git a/tools/perf/python/syscall-counts.py b/tools/perf/python/syscall-counts.py
new file mode 100755
index 000000000000..4b0733b1536d
--- /dev/null
+++ b/tools/perf/python/syscall-counts.py
@@ -0,0 +1,79 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+Displays system-wide system call totals, broken down by syscall.
+
+If a [comm] arg is specified, only syscalls called by [comm] are displayed.
+"""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+from typing import DefaultDict
+import perf
+
+syscalls: DefaultDict[int, int] = defaultdict(int)
+for_comm = None
+session = None
+
+
+def print_syscall_totals():
+ """Print aggregated statistics."""
+ if for_comm is not None:
+ print(f"\nsyscall events for {for_comm}:\n")
+ else:
+ print("\nsyscall events:\n")
+
+ print(f"{'event':<40} {'count':>10}")
+ print("---------------------------------------- -----------")
+
+ for sc_id, val in sorted(syscalls.items(),
+ key=lambda kv: (kv[1], kv[0]), reverse=True):
+ e_machine = getattr(session, "e_machine", 0) or 0
+ if e_machine:
+ name = perf.syscall_name(sc_id, e_machine) or str(sc_id)
+ else:
+ name = perf.syscall_name(sc_id) or str(sc_id)
+ print(f"{name:<40} {val:>10}")
+
+
+def process_event(sample):
+ """Process a single sample event."""
+ event_name = str(sample.evsel)
+ if event_name.startswith("evsel(raw_syscalls:sys_enter"):
+ sc_id = getattr(sample, "id", -1)
+ elif event_name.startswith("evsel(syscalls:sys_enter"):
+ sc_id = getattr(sample, "__syscall_nr", getattr(sample, "id", None))
+ if sc_id is not None and (sc_id < 0 or sc_id > 0xffff):
+ sc_id = None
+ if sc_id is None:
+ sc_id = getattr(sample, "nr", -1)
+ else:
+ return
+
+ if sc_id < 0 or sc_id > 0xffff:
+ return
+
+ comm = "unknown"
+ try:
+ if session:
+ proc = session.find_thread(sample.sample_tid)
+ if proc:
+ comm = proc.comm() or "unknown"
+ except (TypeError, AttributeError):
+ pass
+
+ if for_comm and comm != for_comm:
+ return
+ syscalls[sc_id] += 1
+
+
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser()
+ ap.add_argument("comm", nargs="?", help="Only report syscalls for comm")
+ ap.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ args = ap.parse_args()
+ for_comm = args.comm
+ session = perf.session(perf.data(args.input), sample=process_event)
+ session.process_events()
+ print_syscall_totals()
diff --git a/tools/perf/tests/shell/test_syscall_counts_python.sh b/tools/perf/tests/shell/test_syscall_counts_python.sh
new file mode 100755
index 000000000000..e1c6e2820cce
--- /dev/null
+++ b/tools/perf/tests/shell/test_syscall_counts_python.sh
@@ -0,0 +1,74 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# syscall-counts python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+# If we don't have the perf python module, we can't test
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/syscall-counts.py"
+
+if ! perf check feature -q libtraceevent > /dev/null 2>&1; then
+ echo "Skipping test, libtraceevent is disabled"
+ exit 2
+fi
+
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, syscall-counts.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+
+cleanup() {
+ rm -f "${temp_data}"
+}
+
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+
+test_file_mode() {
+ echo "Testing syscall-counts.py..."
+ # Some systems might not have raw_syscalls:sys_enter (e.g. stripped kernels or permissions)
+ if ! perf list | grep -q raw_syscalls:sys_enter; then
+ echo "Skipping test, raw_syscalls:sys_enter not found"
+ exit 2
+ fi
+
+ # Generate some syscall events
+ if ! perf record -e raw_syscalls:sys_enter -o "${temp_data}" -- sleep 0.5 2>/dev/null; then
+ echo "perf record failed (permissions?), skipping file mode test."
+ exit 2
+ fi
+
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" >/dev/null; then
+ echo "File mode test failed."
+ err=1
+ else
+ echo "File mode test passed."
+ fi
+
+ # Test with a comm argument
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" "sleep" >/dev/null; then
+ echo "Comm filter test failed."
+ err=1
+ else
+ echo "Comm filter test passed."
+ fi
+}
+
+test_file_mode
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 23/49] perf python: Port syscall-counts-by-pid to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (21 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 22/49] perf python: Port syscall-counts " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 24/49] perf python: Port failed-syscalls-by-pid " Ian Rogers
` (25 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port tools/perf/scripts/python/syscall-counts-by-pid.py to a standalone
script in tools/perf/python/ using the perf module. Avoiding the
embedded interpreter and per-event dictionary overhead improves
execution speed by ~3.8x:
```
$ perf record -e raw_syscalls:sys_enter -a sleep 1
...
$ time perf script tools/perf/scripts/python/syscall-counts-by-pid.py perf
...
real 0m3.852s
user 0m3.512s
sys 0m0.336s
$ time python3 tools/perf/python/syscall-counts-by-pid.py perf
...
real 0m1.011s
user 0m0.963s
sys 0m0.048s
```
Additional improvements compared to the legacy script:
- Resolve architecture-specific syscall names via
perf.syscall_name(id, session.e_machine) instead of host python-audit
tables.
- Support both raw_syscalls:sys_enter and individual syscalls:sys_enter_*
tracepoints, and filter out invalid (> 0xffff or negative) syscall IDs.
- Support filtering by numeric PID as well as command name (comm), and
resolve process command names via session.find_thread(pid).
Add a shell test (test_syscall_counts_by_pid_python.sh) to verify the
standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/syscall-counts-by-pid.py | 100 ++++++++++++++++++
.../test_syscall_counts_by_pid_python.sh | 81 ++++++++++++++
2 files changed, 181 insertions(+)
create mode 100755 tools/perf/python/syscall-counts-by-pid.py
create mode 100755 tools/perf/tests/shell/test_syscall_counts_by_pid_python.sh
diff --git a/tools/perf/python/syscall-counts-by-pid.py b/tools/perf/python/syscall-counts-by-pid.py
new file mode 100755
index 000000000000..6e340e8e71df
--- /dev/null
+++ b/tools/perf/python/syscall-counts-by-pid.py
@@ -0,0 +1,100 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+Displays system-wide system call totals, broken down by syscall.
+If a [comm] arg is specified, only syscalls called by [comm] are displayed.
+"""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+from typing import (Dict, Tuple)
+import perf
+
+syscalls: Dict[Tuple[str, int, int], int] = defaultdict(int)
+for_comm = None
+for_pid = None
+session = None
+
+
+def print_syscall_totals():
+ """Print aggregated statistics."""
+ if for_comm is not None:
+ print(f"\nsyscall events for {for_comm}:\n")
+ elif for_pid is not None:
+ print(f"\nsyscall events for PID {for_pid}:\n")
+ else:
+ print("\nsyscall events:\n")
+
+ print(f"{'comm [pid]/syscalls':<40} {'count':>10}")
+ print("---------------------------------------- -----------")
+
+ sorted_keys = sorted(syscalls.keys(), key=lambda k: (k[0], k[1], -syscalls[k], -k[2]))
+ current_comm_pid = None
+ for comm, pid, sc_id in sorted_keys:
+ if current_comm_pid != (comm, pid):
+ print(f"\n{comm} [{pid}]")
+ current_comm_pid = (comm, pid)
+ e_machine = getattr(session, "e_machine", 0) or 0
+ if e_machine:
+ name = perf.syscall_name(sc_id, e_machine) or str(sc_id)
+ else:
+ name = perf.syscall_name(sc_id) or str(sc_id)
+ print(f" {name:<38} {syscalls[(comm, pid, sc_id)]:>10}")
+
+
+def process_event(sample):
+ """Process a single sample event."""
+ event_name = str(sample.evsel)
+ if event_name.startswith("evsel(raw_syscalls:sys_enter"):
+ sc_id = getattr(sample, "id", -1)
+ elif event_name.startswith("evsel(syscalls:sys_enter"):
+ sc_id = getattr(sample, "__syscall_nr", None)
+ if sc_id is not None and (sc_id < 0 or sc_id > 0xffff):
+ sc_id = None
+ if sc_id is None:
+ sc_id = getattr(sample, "nr", None)
+ if sc_id is not None and (sc_id < 0 or sc_id > 0xffff):
+ sc_id = None
+ if sc_id is None:
+ sc_id = getattr(sample, "id", -1)
+ else:
+ return
+
+ if sc_id < 0 or sc_id > 0xffff:
+ return
+
+ pid = sample.sample_pid
+
+ if for_pid is not None and pid != for_pid:
+ return
+
+ comm = "unknown"
+ try:
+ if session:
+ proc = session.find_thread(pid)
+ if proc:
+ comm = proc.comm() or "unknown"
+ except (TypeError, AttributeError):
+ pass
+
+ if for_comm and comm != for_comm:
+ return
+ syscalls[(comm, pid, sc_id)] += 1
+
+
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser()
+ ap.add_argument("filter", nargs="?", help="COMM or PID to filter by")
+ ap.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ args = ap.parse_args()
+
+ if args.filter:
+ try:
+ for_pid = int(args.filter)
+ except ValueError:
+ for_comm = args.filter
+
+ session = perf.session(perf.data(args.input), sample=process_event)
+ session.process_events()
+ print_syscall_totals()
diff --git a/tools/perf/tests/shell/test_syscall_counts_by_pid_python.sh b/tools/perf/tests/shell/test_syscall_counts_by_pid_python.sh
new file mode 100755
index 000000000000..0c67e7d24a6c
--- /dev/null
+++ b/tools/perf/tests/shell/test_syscall_counts_by_pid_python.sh
@@ -0,0 +1,81 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# syscall-counts-by-pid python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+# If we don't have the perf python module, we can't test
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/syscall-counts-by-pid.py"
+
+if ! perf check feature -q libtraceevent > /dev/null 2>&1; then
+ echo "Skipping test, libtraceevent is disabled"
+ exit 2
+fi
+
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, syscall-counts-by-pid.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+
+cleanup() {
+ rm -f "${temp_data}"
+}
+
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+
+test_file_mode() {
+ echo "Testing syscall-counts-by-pid.py..."
+ # Some systems might not have raw_syscalls:sys_enter
+ if ! perf list | grep -q raw_syscalls:sys_enter; then
+ echo "Skipping test, raw_syscalls:sys_enter not found"
+ exit 2
+ fi
+
+ # Generate some syscall events
+ perf record -e raw_syscalls:sys_enter -a -o "${temp_data}" \
+ -- sleep 0.5 >/dev/null 2>&1 || \
+ { echo "Skipping test, perf record failed"; exit 2; }
+
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" >/dev/null; then
+ echo "File mode test failed."
+ err=1
+ else
+ echo "File mode test passed."
+ fi
+
+ # Test with a comm argument
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" "sleep" >/dev/null; then
+ echo "Comm filter test failed."
+ err=1
+ else
+ echo "Comm filter test passed."
+ fi
+
+ # Test with a numeric PID filter argument
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" "$$" >/dev/null; then
+ echo "PID filter test failed."
+ err=1
+ else
+ echo "PID filter test passed."
+ fi
+}
+
+test_file_mode
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 24/49] perf python: Port failed-syscalls-by-pid to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (22 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 23/49] perf python: Port syscall-counts-by-pid " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 25/49] perf python: Port failed-syscalls from Perl " Ian Rogers
` (24 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port failed-syscalls-by-pid.py to a standalone script in
tools/perf/python/ using the perf module:
- Resolve architecture-aware syscall names using
perf.syscall_name(id, session.e_machine) instead of host python-audit
tables, supporting cross-architecture perf.data files without external
dependencies.
- Translate negative syscall return values into symbolic E* error names
using errno.errorcode and architecture-specific session.e_machine
overrides.
- Encapsulate aggregation in a SyscallAnalyzer class using
collections.defaultdict and add argparse filtering by comm, PID, and
-i/--input.
Add a shell test (test_failed_syscalls_by_pid_python.sh) to verify the
standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/failed-syscalls-by-pid.py | 175 ++++++++++++++++++
.../test_failed_syscalls_by_pid_python.sh | 92 +++++++++
2 files changed, 267 insertions(+)
create mode 100755 tools/perf/python/failed-syscalls-by-pid.py
create mode 100755 tools/perf/tests/shell/test_failed_syscalls_by_pid_python.sh
diff --git a/tools/perf/python/failed-syscalls-by-pid.py b/tools/perf/python/failed-syscalls-by-pid.py
new file mode 100755
index 000000000000..91a5309e3658
--- /dev/null
+++ b/tools/perf/python/failed-syscalls-by-pid.py
@@ -0,0 +1,175 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+Displays system-wide failed system call totals, broken down by pid.
+If a [comm] or [pid] arg is specified, only syscalls called by it are displayed.
+
+Ported from tools/perf/scripts/python/failed-syscalls-by-pid.py
+"""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+import errno
+import os
+import sys
+from typing import Optional
+import perf
+
+
+_GENERIC_ERRNO_OVERRIDES: dict[int, str] = {
+ 11: "EAGAIN", 35: "EDEADLK", 36: "ENAMETOOLONG", 37: "ENOLCK",
+ 38: "ENOSYS", 39: "ENOTEMPTY", 40: "ELOOP", 42: "ENOMSG",
+ 45: "EOPNOTSUPP", 60: "ENOSTR", 61: "ENODATA", 78: "EREMCHG",
+ 87: "EUSERS", 89: "EDESTADDRREQ", 90: "EMSGSIZE", 110: "ETIMEDOUT",
+ 111: "ECONNREFUSED", 122: "EDQUOT",
+}
+
+_ARCH_ERRNO_TABLES: dict[int, dict[int, str]] = {
+ 2: {35: "EAGAIN", 36: "EINPROGRESS", 37: "EALREADY", 78: "EDEADLK", 87: "ENOMSG"},
+ 8: {35: "ENOMSG", 45: "EDEADLK", 89: "ENOSYS", 90: "ELOOP", 122: "EDQUOT"},
+ 15: {35: "ENOMSG", 45: "EDEADLK", 246: "EAGAIN", 251: "ENOSYS"},
+ 43: {35: "EAGAIN", 36: "EINPROGRESS", 37: "EALREADY", 78: "EDEADLK", 87: "ENOMSG"},
+ 0x9026: {35: "EAGAIN", 11: "EDEADLK", 60: "ETIMEDOUT", 61: "ECONNREFUSED"},
+}
+
+
+def strerror(nr: int, e_machine: int = 0) -> str:
+ """Return error string for a given errno, accounting for target e_machine."""
+ err_num = abs(nr)
+ if e_machine in _ARCH_ERRNO_TABLES and err_num in _ARCH_ERRNO_TABLES[e_machine]:
+ return _ARCH_ERRNO_TABLES[e_machine][err_num]
+ if e_machine != 0 and err_num in _GENERIC_ERRNO_OVERRIDES:
+ return _GENERIC_ERRNO_OVERRIDES[err_num]
+ try:
+ return errno.errorcode[err_num]
+ except KeyError:
+ return f"Unknown {nr} errno"
+
+
+class SyscallAnalyzer:
+ """Analyzes failed syscalls and aggregates counts."""
+
+ def __init__(self, for_comm: Optional[str] = None, for_pid: Optional[int] = None):
+ self.for_comm = for_comm
+ self.for_pid = for_pid
+ self.session: Optional[perf.session] = None
+ self.syscalls: dict[tuple[str, int, int, int], int] = defaultdict(int)
+ machine = os.uname().machine
+ self.host_64_bit = ("64" in machine or machine in ("s390x", "alpha")
+ or sys.maxsize > 0xffffffff)
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Process raw_syscalls:sys_exit and syscalls:sys_exit events."""
+ event_name = str(sample.evsel)
+ if "raw_syscalls:sys_exit" not in event_name and "syscalls:sys_exit" not in event_name:
+ return
+
+ pid = sample.sample_tid
+ comm = "Unknown"
+ if hasattr(self, 'session') and self.session:
+ try:
+ thread = self.session.find_thread(sample.sample_pid, pid)
+ if thread:
+ comm = thread.comm() or "Unknown"
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError):
+ pass
+
+ if self.for_comm is not None and comm != self.for_comm:
+ return
+ if self.for_pid is not None and pid != self.for_pid:
+ return
+
+ ret = getattr(sample, "ret", 0)
+ is_64_bit = (getattr(self.session, "is_64_bit", self.host_64_bit)
+ if self.session else self.host_64_bit)
+ if ret > 0:
+ if not is_64_bit and 0xfffff000 <= ret <= 0xffffffff:
+ ret -= 0x100000000
+ elif ret >= 0xfffffffffffff000:
+ ret -= 0x10000000000000000
+
+ if -4095 <= ret < 0:
+ syscall_id = getattr(sample, "__syscall_nr", -1)
+ if syscall_id < 0 or syscall_id > 0xffff:
+ syscall_id = getattr(sample, "nr", -1)
+ if syscall_id < 0 or syscall_id > 0xffff:
+ syscall_id = getattr(sample, "sys_id", -1)
+ if syscall_id < 0 or syscall_id > 0xffff:
+ syscall_id = getattr(sample, "id", -1)
+
+ if 0 <= syscall_id <= 0xffff:
+ self.syscalls[(comm, pid, syscall_id, ret)] += 1
+
+ def print_summary(self) -> None:
+ """Print aggregated statistics."""
+ if self.for_comm is not None:
+ print(f"\nsyscall errors for {self.for_comm}:\n")
+ elif self.for_pid is not None:
+ print(f"\nsyscall errors for PID {self.for_pid}:\n")
+ else:
+ print("\nsyscall errors:\n")
+
+ print(f"{'comm [pid]':<30} {'count':>10}")
+ print(f"{'-' * 30:<30} {'-' * 10:>10}")
+
+ sorted_keys = sorted(
+ self.syscalls.keys(),
+ key=lambda k: (k[0], k[1], k[2], -self.syscalls[k])
+ )
+ current_comm_pid = None
+ current_syscall = None
+ emach = getattr(self.session, "e_machine", 0) or 0
+ for comm, pid, syscall_id, ret in sorted_keys:
+ if current_comm_pid != (comm, pid):
+ print(f"\n{comm} [{pid}]")
+ current_comm_pid = (comm, pid)
+ current_syscall = None
+ if current_syscall != syscall_id:
+ try:
+ if emach:
+ name = perf.syscall_name(syscall_id, emach) or str(syscall_id)
+ else:
+ name = perf.syscall_name(syscall_id) or str(syscall_id)
+ except AttributeError:
+ name = str(syscall_id)
+ print(f" syscall: {name:<16}")
+ current_syscall = syscall_id
+ err_str = strerror(ret, emach)
+ count = self.syscalls[(comm, pid, syscall_id, ret)]
+ print(f" err = {err_str:<20} {count:10d}")
+
+
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser(
+ description="Displays system-wide failed system call totals, "
+ "broken down by pid.")
+ ap.add_argument("-i", "--input", default="perf.data",
+ help="Input file name")
+ ap.add_argument("filter", nargs="?", help="COMM or PID to filter by")
+ args = ap.parse_args()
+
+ F_COMM = None
+ F_PID = None
+
+ if args.filter:
+ try:
+ F_PID = int(args.filter)
+ except ValueError:
+ F_COMM = args.filter
+
+ analyzer = SyscallAnalyzer(F_COMM, F_PID)
+
+ try:
+ print("Press control+C to stop and show the summary")
+ session = perf.session(perf.data(args.input), sample=analyzer.process_event)
+ analyzer.session = session
+ session.process_events()
+ analyzer.print_summary()
+ except KeyboardInterrupt:
+ analyzer.print_summary()
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError) as e:
+ print(f"Error processing events: {e}")
+ sys.exit(1)
+ finally:
+ analyzer.session = None
diff --git a/tools/perf/tests/shell/test_failed_syscalls_by_pid_python.sh b/tools/perf/tests/shell/test_failed_syscalls_by_pid_python.sh
new file mode 100755
index 000000000000..d5fab922b8a9
--- /dev/null
+++ b/tools/perf/tests/shell/test_failed_syscalls_by_pid_python.sh
@@ -0,0 +1,92 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# failed-syscalls-by-pid python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/failed-syscalls-by-pid.py"
+
+if ! perf check feature -q libtraceevent > /dev/null 2>&1; then
+ echo "Skipping test, libtraceevent is disabled"
+ exit 2
+fi
+
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, failed-syscalls-by-pid.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+temp_out=""
+
+cleanup() {
+ rm -f "${temp_data}" "${temp_out}"
+}
+
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+test_file_mode() {
+ echo "Testing failed-syscalls-by-pid.py..."
+
+ # Check if syscalls:sys_exit is supported/readable
+ if ! perf record -e syscalls:sys_exit -o /dev/null -- true >/dev/null 2>&1; then
+ if ! perf record -e raw_syscalls:sys_exit -o /dev/null -- true >/dev/null 2>&1; then
+ echo "Skipping test, no syscalls:sys_exit or raw_syscalls:sys_exit event"
+ exit 2
+ else
+ EVENT="raw_syscalls:sys_exit"
+ fi
+ else
+ EVENT="syscalls:sys_exit"
+ fi
+
+ # Generate some events by running a command that should fail at least some syscall
+ # (e.g. failing stat on non-existent file).
+ # Using '|| true' because 'perf record' returns the exit code of 'ls',
+ # which fails with ENOENT
+ perf record -e "${EVENT}" -o "${temp_data}" -- ls /does_not_exist >/dev/null 2>&1 || true
+ if [ ! -s "${temp_data}" ]; then
+ echo "Skipping test, perf record failed to create data"
+ exit 2
+ fi
+
+ # Run the script and check output
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" > "${temp_out}"; then
+ echo "failed-syscalls-by-pid test failed."
+ err=1
+ elif ! "$PYTHON" "$script_path" -i "${temp_data}" "ls" >/dev/null; then
+ echo "failed-syscalls-by-pid comm filter test failed."
+ err=1
+ elif ! "$PYTHON" "$script_path" -i "${temp_data}" "$$" >/dev/null; then
+ echo "failed-syscalls-by-pid PID filter test failed."
+ err=1
+ else
+ if ! grep -n -q "err = ENOENT" "${temp_out}"; then
+ echo "Failed to find expected failed syscalls"
+ cat "${temp_out}"
+ err=1
+ else
+ echo "failed-syscalls-by-pid test passed."
+ fi
+ fi
+ rm -f "${temp_out}"
+}
+
+test_file_mode
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 25/49] perf python: Port failed-syscalls from Perl to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (23 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 24/49] perf python: Port failed-syscalls-by-pid " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 26/49] perf python: Port sctop " Ian Rogers
` (23 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Replace the legacy Perl script failed-syscalls.pl with a standalone
Python script in tools/perf/python/failed-syscalls.py using the perf
Python module.
Improvements compared to the legacy Perl script:
- Remove the dependency on libperl and Perf::Trace::Util.
- Support both syscalls:sys_exit_* and raw_syscalls:sys_exit events (the
legacy script only handled raw_syscalls::sys_exit).
- Use session.is_64_bit to distinguish 32-bit vs 64-bit unsigned error
return ranges (0xfffff000..0xffffffff vs >= 0xfffffffffffff000) so
valid 64-bit syscalls returning ~4GB values are not misclassified as
32-bit negative errors.
- Add argparse CLI support (-i/--input and optional comm filter) and
full type annotations.
Add a shell test
(test_failed_syscalls_python.sh) to verify the standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/failed-syscalls.py | 88 +++++++++++++++++++
.../shell/test_failed_syscalls_python.sh | 81 +++++++++++++++++
2 files changed, 169 insertions(+)
create mode 100755 tools/perf/python/failed-syscalls.py
create mode 100755 tools/perf/tests/shell/test_failed_syscalls_python.sh
diff --git a/tools/perf/python/failed-syscalls.py b/tools/perf/python/failed-syscalls.py
new file mode 100755
index 000000000000..99c3432b2e31
--- /dev/null
+++ b/tools/perf/python/failed-syscalls.py
@@ -0,0 +1,88 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""Failed system call counts."""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+import sys
+from typing import Optional
+import perf
+
+class FailedSyscalls:
+ """Tracks and displays failed system call totals."""
+ def __init__(self, comm: Optional[str] = None) -> None:
+ self.failed_syscalls: dict[str, int] = defaultdict(int)
+ self.for_comm = comm
+ self.session: Optional[perf.session] = None
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Process sys_exit events."""
+ event_name = str(sample.evsel)
+ if not event_name.startswith("evsel(syscalls:sys_exit") and \
+ not event_name.startswith("evsel(raw_syscalls:sys_exit"):
+ return
+
+ try:
+ ret = sample.ret
+ except AttributeError:
+ print("ERROR: tracepoint fields missed", file=sys.stderr)
+ sys.exit(1)
+
+ if ret > 0:
+ if ret >= 0xfffffffffffff000: # 64-bit negative errors
+ ret -= 0x10000000000000000
+ elif 0xfffff000 <= ret <= 0xffffffff: # 32-bit negative errors
+ assert self.session is not None
+ if not self.session.is_64_bit:
+ ret -= 0x100000000
+
+ if ret >= 0:
+ return
+
+ tid = sample.sample_tid
+ assert self.session is not None
+ try:
+ thread = self.session.find_thread(tid)
+ comm = (thread.comm() if thread else None) or "unknown"
+ except (TypeError, AttributeError):
+ # find_thread returns None when the thread isn't known.
+ comm = "unknown"
+
+ if self.for_comm and comm != self.for_comm:
+ return
+
+ self.failed_syscalls[comm] += 1
+
+ def print_totals(self) -> None:
+ """Print summary table."""
+ print("\nfailed syscalls by comm:\n")
+ print(f"{'comm':<20s} {'# errors':>10s}")
+ print(f"{'-'*20} {'-'*10}")
+
+ for comm, val in sorted(self.failed_syscalls.items(),
+ key=lambda kv: (kv[1], kv[0]), reverse=True):
+ print(f"{comm:<20s} {val:10d}")
+
+ def run(self, input_file: str) -> None:
+ """Run the session."""
+ self.session = perf.session(perf.data(input_file), sample=self.process_event)
+ self.session.process_events()
+ self.print_totals()
+
+def main() -> None:
+ """Main function."""
+ parser = argparse.ArgumentParser(description="Trace failed syscalls")
+ parser.add_argument("comm", nargs="?", help="Filter by command name")
+ parser.add_argument("-i", "--input", default="perf.data", help="Input file")
+ args = parser.parse_args()
+
+ analyzer = FailedSyscalls(args.comm)
+ try:
+ analyzer.run(args.input)
+ except IOError as e:
+ print(e, file=sys.stderr)
+ sys.exit(1)
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/perf/tests/shell/test_failed_syscalls_python.sh b/tools/perf/tests/shell/test_failed_syscalls_python.sh
new file mode 100755
index 000000000000..7ab1f50b4260
--- /dev/null
+++ b/tools/perf/tests/shell/test_failed_syscalls_python.sh
@@ -0,0 +1,81 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# failed-syscalls python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/failed-syscalls.py"
+
+if ! perf check feature -q libtraceevent > /dev/null 2>&1; then
+ echo "Skipping test, libtraceevent is disabled"
+ exit 2
+fi
+
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, failed-syscalls.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+temp_out=""
+
+cleanup() {
+ rm -f "${temp_data}" "${temp_out}"
+}
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+echo "Testing failed-syscalls.py..."
+
+# Check if sys_exit event can be recorded
+if ! perf record -e raw_syscalls:sys_exit -o /dev/null -- true >/dev/null 2>&1; then
+ if ! perf record -e syscalls:sys_exit -o /dev/null -- true >/dev/null 2>&1; then
+ echo "Skipping test, no permission or support for sys_exit event"
+ exit 2
+ else
+ EVENT="syscalls:sys_exit"
+ fi
+else
+ EVENT="raw_syscalls:sys_exit"
+fi
+
+# Run perf record with a command that fails a syscall (ls non-existent file).
+# ls exits with non-zero, so perf record returns non-zero exit code of the workload.
+perf record -e "${EVENT}" -o "${temp_data}" \
+ -- ls /nonexistent_file_for_test >/dev/null 2>&1 || true
+
+if [ ! -s "${temp_data}" ]; then
+ echo "Skipping test, perf record failed to create data"
+ exit 2
+fi
+
+# Check that the script executes
+if ! "$PYTHON" "$script_path" -i "${temp_data}" > "${temp_out}"; then
+ echo "failed-syscalls.py test failed"
+ err=1
+else
+ if ! grep -q "failed syscalls by comm" "${temp_out}" || \
+ ! grep -q "ls" "${temp_out}"; then
+ echo "Failed to find the metrics table header or expected error"
+ err=1
+ else
+ echo "failed-syscalls test passed."
+ fi
+fi
+rm -f "${temp_out}"
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 26/49] perf python: Port sctop to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (24 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 25/49] perf python: Port failed-syscalls from Perl " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 27/49] perf python: Port rw-by-file from Perl " Ian Rogers
` (22 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port sctop.py from tools/perf/scripts/python/ to a standalone script in
tools/perf/python/ using an SCTopAnalyzer class structure.
Improvements compared to the legacy script:
- Support both offline perf.data analysis (via perf.session, advancing
display intervals deterministically using event timestamps) and live
monitoring (via LiveSession with automatic tracepoint fallback from
raw_syscalls:sys_enter to syscalls:sys_enter_*).
- Resolve architecture-aware syscall names via
perf.syscall_name(id, session.e_machine) without requiring
python-audit.
- Replace unsafe signal.SIGALRM dictionary mutation and os.popen("clear")
subshell spawning with a synchronized threading.Lock / threading.Event
timer and direct ANSI terminal escape sequences ('\x1b[2J\x1b[H').
Add a shell test (test_sctop_python.sh) to verify the standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/sctop.py | 216 ++++++++++++++++++++
tools/perf/tests/shell/test_sctop_python.sh | 71 +++++++
2 files changed, 287 insertions(+)
create mode 100755 tools/perf/python/sctop.py
create mode 100755 tools/perf/tests/shell/test_sctop_python.sh
diff --git a/tools/perf/python/sctop.py b/tools/perf/python/sctop.py
new file mode 100755
index 000000000000..37c8328ef840
--- /dev/null
+++ b/tools/perf/python/sctop.py
@@ -0,0 +1,216 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+System call top
+
+Periodically displays system-wide system call totals, broken down by
+syscall. If a [comm] arg is specified, only syscalls called by
+[comm] are displayed. If an [interval] arg is specified, the display
+will be refreshed every [interval] seconds. The default interval is
+3 seconds.
+
+Ported from tools/perf/scripts/python/sctop.py
+"""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+import sys
+import threading
+from typing import Optional
+import perf
+from perf_live import LiveSession
+
+
+
+
+class SCTopAnalyzer:
+ """Periodically displays system-wide system call totals."""
+
+ def __init__(self, for_comm: Optional[str], interval: int, offline: bool = False):
+ self.for_comm = for_comm
+ self.interval = interval
+ self.syscalls: dict[int, int] = defaultdict(int)
+ self.comm_cache: dict[int, str] = {}
+ self.lock = threading.Lock()
+ self.stop_event = threading.Event()
+ self.thread = threading.Thread(target=self.print_syscall_totals)
+ self.offline = offline
+ self.last_print_time: Optional[int] = None
+ self.session: Optional[perf.session] = None
+ self.e_machine: Optional[int] = None
+
+ def syscall_name(self, syscall_id: int) -> str:
+ """Lookup syscall name by ID."""
+ try:
+ e_machine = getattr(self.session, "e_machine", self.e_machine)
+ if e_machine is not None:
+ name = perf.syscall_name(syscall_id, e_machine)
+ else:
+ name = perf.syscall_name(syscall_id)
+ if name is not None:
+ return name
+ except (TypeError, OverflowError):
+ pass
+ return str(syscall_id)
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Collect syscall events."""
+ name = str(sample.evsel)
+ syscall_id = getattr(sample, "id", -1)
+ if syscall_id < 0 or syscall_id > 0xffff:
+ syscall_id = getattr(sample, "__syscall_nr", -1)
+ if syscall_id < 0 or syscall_id > 0xffff:
+ syscall_id = getattr(sample, "nr", -1)
+
+ skip = False
+ with self.lock:
+ if self.for_comm is not None:
+ comm = "Unknown"
+ if sample.sample_pid in self.comm_cache:
+ comm = self.comm_cache[sample.sample_pid]
+ elif hasattr(self, 'session') and self.session:
+ try:
+ proc = self.session.find_thread(sample.sample_pid)
+ if proc:
+ comm = proc.comm() or "Unknown"
+ except TypeError:
+ pass
+ else:
+ try:
+ with open(f"/proc/{sample.sample_pid}/comm", "r",
+ encoding="utf-8", errors="replace") as f:
+ comm = f.read().strip()
+ except OSError:
+ comm = "Unknown"
+ self.comm_cache[sample.sample_pid] = comm
+
+ if comm != self.for_comm:
+ skip = True
+
+ is_enter = (name.startswith("evsel(raw_syscalls:sys_enter") or
+ name.startswith("evsel(syscalls:sys_enter"))
+ if not skip and is_enter and 0 <= syscall_id <= 0xffff:
+ self.syscalls[syscall_id] += 1
+
+
+
+ if self.offline and hasattr(sample, "sample_time"):
+ interval_ns = self.interval * (10 ** 9)
+ if self.last_print_time is None:
+ self.last_print_time = sample.sample_time
+ elif sample.sample_time - self.last_print_time >= interval_ns:
+ self.print_current_totals()
+ self.last_print_time = sample.sample_time
+
+ def print_current_totals(self):
+ """Print current syscall totals."""
+ # Clear terminal
+ if not self.offline:
+ print("\x1b[2J\x1b[H", end="")
+ else:
+ print()
+
+ with self.lock:
+ for_comm = self.for_comm
+ if for_comm is not None:
+ print(f"\nsyscall events for {for_comm}:\n")
+ else:
+ print("\nsyscall events:\n")
+
+ print(f"{'event':40s} {'count':10s}")
+ print(f"{'-' * 40:40s} {'-' * 10:10s}")
+
+ with self.lock:
+ current_syscalls = list(self.syscalls.items())
+ self.syscalls.clear()
+ if len(self.comm_cache) > 4096:
+ self.comm_cache.clear()
+
+ current_syscalls.sort(key=lambda kv: (kv[1], kv[0]), reverse=True)
+
+ for syscall_id, val in current_syscalls:
+ print(f"{self.syscall_name(syscall_id):<40s} {val:10d}")
+
+ def print_syscall_totals(self):
+ """Periodically print syscall totals."""
+ while not self.stop_event.is_set():
+ self.print_current_totals()
+ self.stop_event.wait(self.interval)
+ # Print final batch
+ self.print_current_totals()
+
+ def start(self):
+ """Start the background thread."""
+ self.thread.start()
+
+ def stop(self):
+ """Stop the background thread."""
+ self.stop_event.set()
+ self.thread.join()
+
+
+def main():
+ """Main function."""
+ ap = argparse.ArgumentParser(description="System call top")
+ ap.add_argument("args", nargs="*", help="[comm] [interval] or [interval]")
+ ap.add_argument("-i", "--input", help="Input file name")
+ args = ap.parse_args()
+
+ for_comm = None
+ default_interval = 3
+ interval = default_interval
+
+ if len(args.args) > 2:
+ print("Usage: python sctop.py [comm] [interval]")
+ sys.exit(1)
+
+ if len(args.args) > 1:
+ for_comm = args.args[0]
+ try:
+ interval = int(args.args[1])
+ except ValueError:
+ print(f"Invalid interval: {args.args[1]}")
+ sys.exit(1)
+ elif len(args.args) > 0:
+ try:
+ interval = int(args.args[0])
+ except ValueError:
+ for_comm = args.args[0]
+ interval = default_interval
+
+ analyzer = SCTopAnalyzer(for_comm, interval, offline=bool(args.input))
+
+ if not args.input:
+ analyzer.start()
+
+ try:
+ if args.input:
+ session = perf.session(perf.data(args.input), sample=analyzer.process_event)
+ analyzer.session = session
+ session.process_events()
+ analyzer.e_machine = getattr(session, "e_machine", None)
+ else:
+ try:
+ live_session = LiveSession(
+ "raw_syscalls:sys_enter", sample_callback=analyzer.process_event
+ )
+ except OSError:
+ live_session = LiveSession(
+ "syscalls:sys_enter_*", sample_callback=analyzer.process_event
+ )
+ live_session.run()
+ except KeyboardInterrupt:
+ pass
+ except IOError as e:
+ print(f"Error: {e}")
+ finally:
+ if args.input:
+ analyzer.print_current_totals()
+ analyzer.session = None
+ else:
+ analyzer.stop()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/perf/tests/shell/test_sctop_python.sh b/tools/perf/tests/shell/test_sctop_python.sh
new file mode 100755
index 000000000000..da7d9468e987
--- /dev/null
+++ b/tools/perf/tests/shell/test_sctop_python.sh
@@ -0,0 +1,71 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# sctop python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/sctop.py"
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, sctop.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+temp_out=""
+
+cleanup() {
+ rm -f "${temp_data}" "${temp_out}"
+}
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+echo "Testing sctop.py..."
+
+# Create a perf.data file.
+if perf list | grep -q "raw_syscalls:sys_enter"; then
+ perf record -e raw_syscalls:sys_enter -a -o "${temp_data}" \
+ -- sleep 0.1 >/dev/null 2>&1 || \
+ { echo "Skipping test, perf record failed"; exit 2; }
+else
+ echo "Skipping test, no raw_syscalls:sys_enter event"
+ exit 2
+fi
+
+
+if [ ! -s "${temp_data}" ]; then
+ echo "Skipping test, perf record failed to create data"
+ exit 2
+fi
+
+# Check that the script executes
+if ! "$PYTHON" "$script_path" -i "${temp_data}" > "${temp_out}"; then
+ echo "sctop.py test failed"
+ err=1
+elif ! "$PYTHON" "$script_path" -i "${temp_data}" sleep 1 >/dev/null; then
+ echo "sctop.py comm+interval test failed"
+ err=1
+else
+ if ! grep -E -q "[0-9]+$" "${temp_out}"; then
+ echo "Failed to find metric data rows"
+ err=1
+ else
+ echo "sctop test passed."
+ fi
+fi
+rm -f "${temp_out}"
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 27/49] perf python: Port rw-by-file from Perl to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (25 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 26/49] perf python: Port sctop " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 28/49] perf python: Port rw-by-pid " Ian Rogers
` (21 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Replace the legacy Perl script rw-by-file.pl with a standalone Python
script in tools/perf/python/rw-by-file.py using the perf Python module.
Improvements compared to the legacy Perl script:
- Remove the dependency on libperl and Perf::Trace::Util.
- Encapsulate per-file-descriptor read/write byte and call count
aggregation in an RwByFile class using perf.session and resolve thread
command names via session.find_thread(pid, sample_tid).
- Add argparse CLI support (-i/--input and target program filter) and
full type annotations.
Add a shell test
(test_rw_by_file_python.sh) to verify the standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/rw-by-file.py | 109 ++++++++++++++++++
.../tests/shell/test_rw_by_file_python.sh | 70 +++++++++++
2 files changed, 179 insertions(+)
create mode 100755 tools/perf/python/rw-by-file.py
create mode 100755 tools/perf/tests/shell/test_rw_by_file_python.sh
diff --git a/tools/perf/python/rw-by-file.py b/tools/perf/python/rw-by-file.py
new file mode 100755
index 000000000000..562ee7f7fd7e
--- /dev/null
+++ b/tools/perf/python/rw-by-file.py
@@ -0,0 +1,109 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+"""Display r/w activity for files read/written to for a given program."""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+import sys
+from typing import Optional, Dict
+import perf
+
+class RwByFile:
+ """Tracks and displays read/write activity by file descriptor."""
+ def __init__(self, comm: str) -> None:
+ self.for_comm = comm
+ self.reads: Dict[int, Dict[str, int]] = defaultdict(
+ lambda: {"bytes_requested": 0, "total_reads": 0}
+ )
+ self.writes: Dict[int, Dict[str, int]] = defaultdict(
+ lambda: {"bytes_written": 0, "total_writes": 0}
+ )
+ self.unhandled: Dict[str, int] = defaultdict(int)
+ self.session: Optional[perf.session] = None
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Process events."""
+ raw_name = str(sample.evsel)
+ event_name = raw_name[6:-1] if raw_name.startswith("evsel(") else raw_name
+
+ pid = sample.sample_pid
+ assert self.session is not None
+ try:
+ thread = self.session.find_thread(pid, sample.sample_tid)
+ comm = (thread.comm() if thread else None) or "unknown"
+ except (TypeError, AttributeError):
+ comm = "unknown"
+
+ if comm != self.for_comm:
+ return
+
+ if event_name == "syscalls:sys_enter_read":
+ try:
+ fd = sample.fd
+ count = sample.count
+ self.reads[fd]["bytes_requested"] += count
+ self.reads[fd]["total_reads"] += 1
+ except AttributeError:
+ self.unhandled[event_name] += 1
+ elif event_name == "syscalls:sys_enter_write":
+ try:
+ fd = sample.fd
+ count = sample.count
+ self.writes[fd]["bytes_written"] += count
+ self.writes[fd]["total_writes"] += 1
+ except AttributeError:
+ self.unhandled[event_name] += 1
+ else:
+ self.unhandled[event_name] += 1
+
+ def print_totals(self) -> None:
+ """Print summary tables."""
+ print(f"file read counts for {self.for_comm}:\n")
+ print(f"{'fd':>6s} {'# reads':>10s} {'bytes_requested':>15s}")
+ print(f"{'-'*6} {'-'*10} {'-'*15}")
+
+ for fd, data in sorted(self.reads.items(),
+ key=lambda kv: kv[1]["bytes_requested"], reverse=True):
+ print(f"{fd:6d} {data['total_reads']:10d} {data['bytes_requested']:15d}")
+
+ print(f"\nfile write counts for {self.for_comm}:\n")
+ print(f"{'fd':>6s} {'# writes':>10s} {'bytes_written':>15s}")
+ print(f"{'-'*6} {'-'*10} {'-'*15}")
+
+ for fd, data in sorted(self.writes.items(),
+ key=lambda kv: kv[1]["bytes_written"], reverse=True):
+ print(f"{fd:6d} {data['total_writes']:10d} {data['bytes_written']:15d}")
+
+ if self.unhandled:
+ print("\nunhandled events:\n")
+ print(f"{'event':<40s} {'count':>10s}")
+ print(f"{'-'*40} {'-'*10}")
+ for event_name, count in self.unhandled.items():
+ print(f"{event_name:<40s} {count:10d}")
+
+ def run(self, input_file: str) -> None:
+ """Run the session."""
+ self.session = perf.session(perf.data(input_file), sample=self.process_event)
+ try:
+ self.session.process_events()
+ finally:
+ self.session = None
+ self.print_totals()
+
+def main() -> None:
+ """Main function."""
+ parser = argparse.ArgumentParser(description="Trace r/w activity by file")
+ parser.add_argument("comm", help="Filter by command name")
+ parser.add_argument("-i", "--input", default="perf.data", help="Input file")
+ args = parser.parse_args()
+
+ analyzer = RwByFile(args.comm)
+ try:
+ analyzer.run(args.input)
+ except IOError as e:
+ print(e, file=sys.stderr)
+ sys.exit(1)
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/perf/tests/shell/test_rw_by_file_python.sh b/tools/perf/tests/shell/test_rw_by_file_python.sh
new file mode 100755
index 000000000000..a305d5eded75
--- /dev/null
+++ b/tools/perf/tests/shell/test_rw_by_file_python.sh
@@ -0,0 +1,70 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# rw-by-file python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/rw-by-file.py"
+
+if ! perf check feature -q libtraceevent > /dev/null 2>&1; then
+ echo "Skipping test, libtraceevent is disabled"
+ exit 2
+fi
+
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, rw-by-file.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+cleanup() {
+ rm -f "${temp_data}" "${temp_out}"
+}
+trap 'cleanup' EXIT TERM INT
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+echo "Testing rw-by-file.py..."
+
+# Create a perf.data file. Try to get tracepoint data.
+if perf list | grep -q "syscalls:sys_enter_read"; then
+ perf record -e syscalls:sys_enter_read,syscalls:sys_enter_write -a -o "${temp_data}" \
+ -- dd if=/dev/urandom of=/dev/null bs=1M count=10 >/dev/null 2>&1 || \
+ { echo "Skipping test, perf record failed"; exit 2; }
+else
+ echo "Skipping test, no syscalls:sys_enter_read event"
+ exit 2
+fi
+
+
+if [ ! -s "${temp_data}" ]; then
+ echo "Skipping test, perf record failed to create data"
+ exit 2
+fi
+
+# Check that the script executes - filtering for "dd" since that's what we ran
+if ! "$PYTHON" "$script_path" -i "${temp_data}" "dd" > "${temp_out}"; then
+ echo "rw-by-file.py test failed"
+ err=1
+else
+ if ! grep -E -q "^ *[0-9]+" "${temp_out}"; then
+ echo "Failed to find metric data rows"
+ err=1
+ else
+ echo "rw-by-file test passed."
+ fi
+fi
+rm -f "${temp_out}"
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 28/49] perf python: Port rw-by-pid from Perl to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (26 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 27/49] perf python: Port rw-by-file from Perl " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 29/49] perf python: Port rwtop " Ian Rogers
` (20 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Replace the legacy Perl script rw-by-pid.pl with a standalone Python
script in tools/perf/python/rw-by-pid.py using the perf Python module.
Improvements compared to the legacy Perl script:
- Remove the dependency on libperl and Perf::Trace::Util.
- Convert unsigned 32-bit and 64-bit error return values
(0xfffff000..0xffffffff and >= 0x8000000000000000) to negative errnos
so failed read/write syscalls are recorded in the error table rather
than inflating bytes_read / bytes_written.
- Add argparse CLI support (-i/--input) and full type annotations.
Add a shell test
(test_rw_by_pid_python.sh) to verify the standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/rw-by-pid.py | 189 ++++++++++++++++++
.../perf/tests/shell/test_rw_by_pid_python.sh | 77 +++++++
2 files changed, 266 insertions(+)
create mode 100755 tools/perf/python/rw-by-pid.py
create mode 100755 tools/perf/tests/shell/test_rw_by_pid_python.sh
diff --git a/tools/perf/python/rw-by-pid.py b/tools/perf/python/rw-by-pid.py
new file mode 100755
index 000000000000..6d174c0f1bea
--- /dev/null
+++ b/tools/perf/python/rw-by-pid.py
@@ -0,0 +1,189 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+"""Display r/w activity for all processes."""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+import sys
+from typing import Optional, Dict, List, Tuple, Any
+import perf
+
+class RwByPid:
+ """Tracks and displays read/write activity by PID."""
+ def __init__(self) -> None:
+ self.reads: Dict[int, Dict[str, Any]] = defaultdict(
+ lambda: {
+ "bytes_requested": 0,
+ "bytes_read": 0,
+ "total_reads": 0,
+ "comm": "",
+ "errors": defaultdict(int),
+ }
+ )
+ self.writes: Dict[int, Dict[str, Any]] = defaultdict(
+ lambda: {
+ "bytes_requested": 0,
+ "bytes_written": 0,
+ "total_writes": 0,
+ "comm": "",
+ "errors": defaultdict(int),
+ }
+ )
+ self.unhandled: Dict[str, int] = defaultdict(int)
+ self.session: Optional[perf.session] = None
+
+ def _handle_sys_enter_read(self, pid: int, comm: str, sample: perf.sample_event) -> None:
+ try:
+ count = sample.count
+ self.reads[pid]["bytes_requested"] += count
+ self.reads[pid]["total_reads"] += 1
+ self.reads[pid]["comm"] = comm
+ except AttributeError:
+ self.unhandled["sys_enter_read_attr_err"] += 1
+
+ def _handle_sys_exit_read(self, pid: int, sample: perf.sample_event) -> None:
+ try:
+ ret = sample.ret
+ if ret >= 0x8000000000000000:
+ ret -= 0x10000000000000000
+ elif 0xfffff000 <= ret <= 0xffffffff:
+ ret -= 0x100000000
+ if ret > 0:
+ self.reads[pid]["bytes_read"] += ret
+ else:
+ self.reads[pid]["errors"][ret] += 1
+ except AttributeError:
+ self.unhandled["sys_exit_read_attr_err"] += 1
+
+ def _handle_sys_enter_write(self, pid: int, comm: str, sample: perf.sample_event) -> None:
+ try:
+ count = sample.count
+ self.writes[pid]["bytes_requested"] += count
+ self.writes[pid]["total_writes"] += 1
+ self.writes[pid]["comm"] = comm
+ except AttributeError:
+ self.unhandled["sys_enter_write_attr_err"] += 1
+
+ def _handle_sys_exit_write(self, pid: int, sample: perf.sample_event) -> None:
+ try:
+ ret = sample.ret
+ if ret >= 0x8000000000000000:
+ ret -= 0x10000000000000000
+ elif 0xfffff000 <= ret <= 0xffffffff:
+ ret -= 0x100000000
+ if ret > 0:
+ self.writes[pid]["bytes_written"] += ret
+ else:
+ self.writes[pid]["errors"][ret] += 1
+ except AttributeError:
+ self.unhandled["sys_exit_write_attr_err"] += 1
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Process events."""
+ event_name = str(sample.evsel)[6:-1]
+ pid = sample.sample_pid
+
+ assert self.session is not None
+ try:
+ thread = self.session.find_thread(pid)
+ comm = (thread.comm() if thread else None) or "unknown"
+ except (TypeError, AttributeError):
+ comm = "unknown"
+
+ if event_name == "syscalls:sys_enter_read":
+ self._handle_sys_enter_read(pid, comm, sample)
+ elif event_name == "syscalls:sys_exit_read":
+ self._handle_sys_exit_read(pid, sample)
+ elif event_name == "syscalls:sys_enter_write":
+ self._handle_sys_enter_write(pid, comm, sample)
+ elif event_name == "syscalls:sys_exit_write":
+ self._handle_sys_exit_write(pid, sample)
+ else:
+ self.unhandled[event_name] += 1
+
+ def print_totals(self) -> None:
+ """Print summary tables."""
+ print("read counts by pid:\n")
+ print(
+ f"{'pid':>6s} {'comm':<20s} {'# reads':>10s} "
+ f"{'bytes_requested':>15s} {'bytes_read':>10s}"
+ )
+ print(f"{'-'*6} {'-'*20} {'-'*10} {'-'*15} {'-'*10}")
+
+ for pid, data in sorted(self.reads.items(),
+ key=lambda kv: kv[1]["bytes_read"], reverse=True):
+ print(
+ f"{pid:6d} {data['comm']:<20s} {data['total_reads']:10d} "
+ f"{data['bytes_requested']:15d} {data['bytes_read']:10d}"
+ )
+
+ print("\nfailed reads by pid:\n")
+ print(f"{'pid':>6s} {'comm':<20s} {'error #':>6s} {'# errors':>10s}")
+ print(f"{'-'*6} {'-'*20} {'-'*6} {'-'*10}")
+
+ errcounts: List[Tuple[int, str, int, int]] = []
+ for pid, data in self.reads.items():
+ for error, count in data["errors"].items():
+ errcounts.append((pid, data["comm"], error, count))
+
+ for pid, comm, error, count in sorted(errcounts, key=lambda x: x[3], reverse=True):
+ print(f"{pid:6d} {comm:<20s} {error:6d} {count:10d}")
+
+ print("\nwrite counts by pid:\n")
+ print(
+ f"{'pid':>6s} {'comm':<20s} {'# writes':>10s} "
+ f"{'bytes_requested':>15s} {'bytes_written':>15s}"
+ )
+ print(f"{'-'*6} {'-'*20} {'-'*10} {'-'*15} {'-'*15}")
+
+ for pid, data in sorted(self.writes.items(),
+ key=lambda kv: kv[1]["bytes_written"], reverse=True):
+ print(
+ f"{pid:6d} {data['comm']:<20s} {data['total_writes']:10d} "
+ f"{data['bytes_requested']:15d} {data['bytes_written']:15d}"
+ )
+
+ print("\nfailed writes by pid:\n")
+ print(f"{'pid':>6s} {'comm':<20s} {'error #':>6s} {'# errors':>10s}")
+ print(f"{'-'*6} {'-'*20} {'-'*6} {'-'*10}")
+
+ errcounts = []
+ for pid, data in self.writes.items():
+ for error, count in data["errors"].items():
+ errcounts.append((pid, data["comm"], error, count))
+
+ for pid, comm, error, count in sorted(errcounts, key=lambda x: x[3], reverse=True):
+ print(f"{pid:6d} {comm:<20s} {error:6d} {count:10d}")
+
+ if self.unhandled:
+ print("\nunhandled events:\n")
+ print(f"{'event':<40s} {'count':>10s}")
+ print(f"{'-'*40} {'-'*10}")
+ for event_name, count in self.unhandled.items():
+ print(f"{event_name:<40s} {count:10d}")
+
+ def run(self, input_file: str) -> None:
+ """Run the session."""
+ self.session = perf.session(perf.data(input_file), sample=self.process_event)
+ try:
+ self.session.process_events()
+ finally:
+ self.session = None
+ self.print_totals()
+
+def main() -> None:
+ """Main function."""
+ parser = argparse.ArgumentParser(description="Trace r/w activity by PID")
+ parser.add_argument("-i", "--input", default="perf.data", help="Input file")
+ args = parser.parse_args()
+
+ analyzer = RwByPid()
+ try:
+ analyzer.run(args.input)
+ except IOError as e:
+ print(e, file=sys.stderr)
+ sys.exit(1)
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/perf/tests/shell/test_rw_by_pid_python.sh b/tools/perf/tests/shell/test_rw_by_pid_python.sh
new file mode 100755
index 000000000000..6ce3e2355c11
--- /dev/null
+++ b/tools/perf/tests/shell/test_rw_by_pid_python.sh
@@ -0,0 +1,77 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# rw-by-pid python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/rw-by-pid.py"
+
+if ! perf check feature -q libtraceevent > /dev/null 2>&1; then
+ echo "Skipping test, libtraceevent is disabled"
+ exit 2
+fi
+
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, rw-by-pid.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+temp_out=""
+
+cleanup() {
+ rm -f "${temp_data}" "${temp_out}"
+}
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+echo "Testing rw-by-pid.py..."
+
+# Create a perf.data file. Try to get tracepoint data.
+if perf list | grep -q "syscalls:sys_enter_read"; then
+ ev="syscalls:sys_enter_read,syscalls:sys_exit_read"
+ ev="${ev},syscalls:sys_enter_write,syscalls:sys_exit_write"
+ perf record -e "$ev" -a -o "${temp_data}" \
+ -- dd if=/dev/urandom of=/dev/null bs=1M count=10 >/dev/null 2>&1 || \
+ { echo "Skipping test, perf record failed"; exit 2; }
+else
+ echo "Skipping test, no syscalls:sys_enter_read event"
+ exit 2
+fi
+
+
+if [ ! -s "${temp_data}" ]; then
+ echo "Skipping test, perf record failed to create data"
+ exit 2
+fi
+
+# Check that the script executes
+if ! "$PYTHON" "$script_path" -i "${temp_data}" > "${temp_out}"; then
+ echo "rw-by-pid.py test failed"
+ err=1
+else
+ if ! grep -E -q "^ *[0-9]+" "${temp_out}"; then
+ echo "Failed to find metric data rows in output"
+ cat "${temp_out}"
+ err=1
+ else
+ echo "rw-by-pid test passed."
+ fi
+fi
+rm -f "${temp_out}"
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 29/49] perf python: Port rwtop from Perl to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (27 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 28/49] perf python: Port rw-by-pid " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 30/49] perf python: Port futex-contention " Ian Rogers
` (19 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Replace the legacy Perl script rwtop.pl with a standalone Python script
in tools/perf/python/rwtop.py using the perf Python module.
Improvements compared to the legacy Perl script:
- Remove the dependency on libperl and Perf::Trace::Util.
- Support both offline perf.data files (via perf.session) and live
recording (via LiveSession with --live), driving periodic interval
summaries from event timestamps rather than wall-clock SIGALRM timers
so both live and offline runs produce deterministic output.
- Use session.is_64_bit (falling back to host bitness in live mode) when
checking for negative syscall return values so 64-bit byte counts in
0xfffff000..0xffffffff are not misclassified as 32-bit errors.
- Sanitize non-printable characters in /proc/<pid>/comm to prevent
terminal control sequence injection.
Add a shell test
(test_rwtop_python.sh) to verify the standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/rwtop.py | 247 ++++++++++++++++++++
tools/perf/tests/shell/test_rwtop_python.sh | 76 ++++++
2 files changed, 323 insertions(+)
create mode 100755 tools/perf/python/rwtop.py
create mode 100755 tools/perf/tests/shell/test_rwtop_python.sh
diff --git a/tools/perf/python/rwtop.py b/tools/perf/python/rwtop.py
new file mode 100755
index 000000000000..68366bf5b0de
--- /dev/null
+++ b/tools/perf/python/rwtop.py
@@ -0,0 +1,247 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+"""Periodically displays system-wide r/w call activity, broken down by pid."""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+import os
+import sys
+from typing import Optional, Dict, Any
+import perf
+from perf_live import LiveSession
+
+class RwTop:
+ """Periodically displays system-wide r/w call activity."""
+ def __init__(self, interval: int = 3, nlines: int = 20) -> None:
+ self.offline = False
+ self.interval_ns = interval * 1000000000
+ self.nlines = nlines
+ self.reads: Dict[int, Dict[str, Any]] = defaultdict(
+ lambda: {
+ "bytes_requested": 0,
+ "bytes_read": 0,
+ "total_reads": 0,
+ "comm": "",
+ "errors": defaultdict(int),
+ }
+ )
+ self.writes: Dict[int, Dict[str, Any]] = defaultdict(
+ lambda: {
+ "bytes_requested": 0,
+ "bytes_written": 0,
+ "total_writes": 0,
+ "comm": "",
+ "errors": defaultdict(int),
+ }
+ )
+ self.unhandled: Dict[str, int] = defaultdict(int)
+ self.comm_cache: Dict[int, str] = {}
+ self.session: Optional[perf.session] = None
+ self.last_print_time: int = 0
+ machine = os.uname().machine
+ self.host_64_bit = ("64" in machine or machine in ("s390x", "alpha")
+ or sys.maxsize > 0xffffffff)
+
+ def get_comm(self, pid: int) -> str:
+ """Resolve and cache the comm(and) of a pid."""
+ comm = self.comm_cache.get(pid)
+ if comm:
+ return comm
+ comm = None
+ try:
+ if self.session:
+ thread = self.session.find_thread(pid)
+ comm = thread.comm() if thread else None
+ else:
+ with open(f"/proc/{pid}/comm", "r", encoding="utf-8", errors="replace") as f:
+ comm = f.read().strip()
+ except (TypeError, AttributeError):
+ # find_thread returns None when the thread isn't known.
+ pass
+ except OSError:
+ # The thread may have exited before /proc could be read.
+ pass
+ if not comm:
+ comm = f"PID({pid})"
+ comm = ''.join(c if c.isprintable() else '?' for c in comm)
+ self.comm_cache[pid] = comm
+ return comm
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Process events."""
+ event_name = str(sample.evsel)
+ pid = sample.sample_pid
+ sample_time = sample.sample_time
+
+ if self.last_print_time == 0:
+ self.last_print_time = sample_time
+
+ # Check if interval has passed
+ if sample_time > self.last_print_time and sample_time - self.last_print_time >= self.interval_ns:
+ self.print_totals()
+ self.last_print_time = sample_time
+
+ # Map each event onto the totals it updates. "enter" events count the
+ # requested bytes, "exit" events the transferred bytes or the error.
+ handlers = {
+ "evsel(syscalls:sys_enter_read)": (self.reads, "total_reads", None),
+ "evsel(syscalls:sys_exit_read)": (self.reads, None, "bytes_read"),
+ "evsel(syscalls:sys_enter_write)": (self.writes, "total_writes", None),
+ "evsel(syscalls:sys_exit_write)": (self.writes, None, "bytes_written"),
+ }
+ handler = handlers.get(event_name)
+ if not handler:
+ self.unhandled[event_name] += 1
+ return
+
+ totals, count_key, bytes_key = handler
+ try:
+ value = sample.count if count_key else sample.ret
+ except AttributeError:
+ self.unhandled[event_name] += 1
+ return
+
+ data = totals[pid]
+ data["comm"] = self.get_comm(pid)
+ if count_key:
+ data["bytes_requested"] += value
+ data[count_key] += 1
+ elif bytes_key:
+ is_64_bit = (getattr(self.session, "is_64_bit", self.host_64_bit)
+ if self.session else self.host_64_bit)
+ if value > 0:
+ if not is_64_bit and 0xfffff000 <= value <= 0xffffffff:
+ value -= 0x100000000
+ elif value >= 0xfffffffffffff000:
+ value -= 0x10000000000000000
+ if value >= 0:
+ data[bytes_key] += value
+ else:
+ data["errors"][value] += 1
+
+ def print_totals(self) -> None:
+ """Print summary tables."""
+ if not self.offline:
+ print('\x1b[H\x1b[2J', end='')
+ print("read counts by pid:\n")
+ print(
+ f"{'pid':>6s} {'comm':<20s} {'# reads':>10s} "
+ f"{'bytes_req':>10s} {'bytes_read':>10s}"
+ )
+ print(f"{'-'*6} {'-'*20} {'-'*10} {'-'*10} {'-'*10}")
+
+ count = 0
+ for pid, data in sorted(self.reads.items(),
+ key=lambda kv: kv[1]["bytes_read"], reverse=True):
+ print(
+ f"{pid:6d} {data['comm']:<20s} {data['total_reads']:10d} "
+ f"{data['bytes_requested']:10d} {data['bytes_read']:10d}"
+ )
+ count += 1
+ if count >= self.nlines:
+ break
+
+ print("\nfailed reads by pid:\n")
+ print(f"{'pid':>6s} {'comm':<20s} {'error #':>6s} {'# errors':>10s}")
+ print(f"{'-'*6} {'-'*20} {'-'*6} {'-'*10}")
+
+ errcounts = []
+ for pid, data in self.reads.items():
+ for error, cnt in data["errors"].items():
+ errcounts.append((pid, data["comm"], error, cnt))
+
+ sorted_errcounts = sorted(errcounts, key=lambda x: x[3], reverse=True)
+ for pid, comm, error, cnt in sorted_errcounts[:self.nlines]:
+ print(f"{pid:6d} {comm:<20s} {error:6d} {cnt:10d}")
+
+ print("\nwrite counts by pid:\n")
+ print(
+ f"{'pid':>6s} {'comm':<20s} {'# writes':>10s} "
+ f"{'bytes_req':>10s} {'bytes_written':>13s}"
+ )
+ print(f"{'-'*6} {'-'*20} {'-'*10} {'-'*10} {'-'*13}")
+
+ count = 0
+ for pid, data in sorted(self.writes.items(),
+ key=lambda kv: kv[1]["bytes_written"], reverse=True):
+ print(
+ f"{pid:6d} {data['comm']:<20s} {data['total_writes']:10d} "
+ f"{data['bytes_requested']:10d} {data['bytes_written']:13d}"
+ )
+ count += 1
+ if count >= self.nlines:
+ break
+
+ print("\nfailed writes by pid:\n")
+ print(f"{'pid':>6s} {'comm':<20s} {'error #':>6s} {'# errors':>10s}")
+ print(f"{'-'*6} {'-'*20} {'-'*6} {'-'*10}")
+
+ errcounts = []
+ for pid, data in self.writes.items():
+ for error, cnt in data["errors"].items():
+ errcounts.append((pid, data["comm"], error, cnt))
+
+ sorted_errcounts = sorted(errcounts, key=lambda x: x[3], reverse=True)
+ for pid, comm, error, cnt in sorted_errcounts[:self.nlines]:
+ print(f"{pid:6d} {comm:<20s} {error:6d} {cnt:10d}")
+
+ # Reset counts
+ self.reads.clear()
+ self.writes.clear()
+ self.comm_cache.clear()
+
+ def run(self, input_file: str) -> None:
+ """Run the session."""
+ self.session = perf.session(perf.data(input_file), sample=self.process_event)
+ try:
+ self.session.process_events()
+ finally:
+ self.session = None
+
+ # Print final totals if there are any left
+ if self.reads or self.writes:
+ self.print_totals()
+
+ if self.unhandled:
+ print("\nunhandled events:\n")
+ print(f"{'event':<40s} {'count':>10s}")
+ print(f"{'-'*40} {'-'*10}")
+ for event_name, count in self.unhandled.items():
+ print(f"{event_name:<40s} {count:10d}")
+
+def main() -> None:
+ """Main function."""
+ parser = argparse.ArgumentParser(description="Trace r/w activity by PID")
+ parser.add_argument(
+ "interval", type=int, nargs="?", default=3, help="Refresh interval in seconds"
+ )
+ parser.add_argument("-i", "--input", default="perf.data", help="Input file")
+ parser.add_argument("-l", "--live", action="store_true", help="Run in live mode")
+ args = parser.parse_args()
+
+ analyzer = RwTop(args.interval)
+ try:
+ if args.live or (not os.path.exists(args.input) and args.input == "perf.data"):
+ # Live mode
+ events = (
+ "syscalls:sys_enter_read,syscalls:sys_exit_read,"
+ "syscalls:sys_enter_write,syscalls:sys_exit_write"
+ )
+ live_session = LiveSession(events, sample_callback=analyzer.process_event)
+ print("Live mode started. Press Ctrl+C to stop.", file=sys.stderr)
+ live_session.run()
+ else:
+ analyzer.offline = True
+ analyzer.run(args.input)
+ except IOError as e:
+ print(e, file=sys.stderr)
+ sys.exit(1)
+ except KeyboardInterrupt:
+ if not analyzer.offline:
+ print("\nStopping live mode...", file=sys.stderr)
+ if analyzer.reads or analyzer.writes:
+ analyzer.print_totals()
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/perf/tests/shell/test_rwtop_python.sh b/tools/perf/tests/shell/test_rwtop_python.sh
new file mode 100755
index 000000000000..43471a777199
--- /dev/null
+++ b/tools/perf/tests/shell/test_rwtop_python.sh
@@ -0,0 +1,76 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# rwtop python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/rwtop.py"
+
+if ! perf check feature -q libtraceevent > /dev/null 2>&1; then
+ echo "Skipping test, libtraceevent is disabled"
+ exit 2
+fi
+
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, rwtop.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+temp_out=""
+
+cleanup() {
+ rm -f "${temp_data}" "${temp_out}"
+}
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+echo "Testing rwtop.py..."
+
+# Create a perf.data file. Try to get tracepoint data.
+if perf list | grep -q "syscalls:sys_enter_read"; then
+ ev="syscalls:sys_enter_read,syscalls:sys_exit_read"
+ ev="${ev},syscalls:sys_enter_write,syscalls:sys_exit_write"
+ perf record -e "$ev" -a -o "${temp_data}" \
+ -- dd if=/dev/urandom of=/dev/null bs=1M count=10 >/dev/null 2>&1 || \
+ { echo "Skipping test, perf record failed"; exit 2; }
+else
+ echo "Skipping test, no syscalls:sys_enter_read event"
+ exit 2
+fi
+
+
+if [ ! -s "${temp_data}" ]; then
+ echo "Skipping test, perf record failed to create data"
+ exit 2
+fi
+
+# Check that the script executes
+if ! "$PYTHON" "$script_path" -i "${temp_data}" > "${temp_out}"; then
+ echo "rwtop.py test failed"
+ err=1
+else
+ if ! grep -E -q "^ *[0-9]+" "${temp_out}"; then
+ echo "Failed to find metric data rows"
+ err=1
+ else
+ echo "rwtop test passed."
+ fi
+fi
+rm -f "${temp_out}"
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 30/49] perf python: Port futex-contention to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (28 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 29/49] perf python: Port rwtop " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 31/49] perf python: Port task-analyzer " Ian Rogers
` (18 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port tools/perf/scripts/python/futex-contention.py to a standalone
script in tools/perf/python/ using the perf module. Avoiding the
embedded interpreter overhead improves execution speed by ~3.2x:
```
$ perf record -e syscalls:sys_*_futex -a sleep 1
...
$ time perf script tools/perf/scripts/python/futex-contention.py
...
real 0m1.007s
user 0m0.935s
sys 0m0.072s
$ time python3 tools/perf/python/futex-contention.py
...
real 0m0.314s
user 0m0.259s
sys 0m0.056s
```
Additional improvements compared to the legacy script:
- Consolidate per-(tid, uaddr) contention count, total_time, min_time,
and max_time into a single LockStats class instead of maintaining
three separate dictionaries.
- Validate that uaddr and op tracepoint attributes are present on
syscalls:sys_enter_futex samples rather than silently attributing
missing fields to uaddr=0, op=0 (FUTEX_WAIT).
- Add -i/--input CLI option support via argparse and full type
annotations.
Add a shell test (test_futex_contention_python.sh) to verify the
standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/futex-contention.py | 93 +++++++++++++++++++
.../shell/test_futex_contention_python.sh | 67 +++++++++++++
2 files changed, 160 insertions(+)
create mode 100755 tools/perf/python/futex-contention.py
create mode 100755 tools/perf/tests/shell/test_futex_contention_python.sh
diff --git a/tools/perf/python/futex-contention.py b/tools/perf/python/futex-contention.py
new file mode 100755
index 000000000000..d5bb8ca19699
--- /dev/null
+++ b/tools/perf/python/futex-contention.py
@@ -0,0 +1,93 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""Measures futex contention."""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+from typing import Dict, Tuple
+import perf
+
+class LockStats:
+ """Aggregate lock contention information."""
+ def __init__(self) -> None:
+ self.count = 0
+ self.total_time = 0
+ self.min_time = 0
+ self.max_time = 0
+
+ def add(self, duration: int) -> None:
+ """Add a new duration measurement."""
+ self.count += 1
+ self.total_time += duration
+ if self.count == 1:
+ self.min_time = duration
+ self.max_time = duration
+ else:
+ self.min_time = min(self.min_time, duration)
+ self.max_time = max(self.max_time, duration)
+
+ def avg(self) -> float:
+ """Return average duration."""
+ return self.total_time / self.count if self.count > 0 else 0.0
+
+process_names: Dict[int, str] = {}
+start_times: Dict[int, Tuple[int, int]] = {}
+session = None
+durations: Dict[Tuple[int, int], LockStats] = defaultdict(LockStats)
+
+FUTEX_WAIT = 0
+FUTEX_PRIVATE_FLAG = 128
+FUTEX_CLOCK_REALTIME = 256
+FUTEX_CMD_MASK = ~(FUTEX_PRIVATE_FLAG | FUTEX_CLOCK_REALTIME)
+
+
+def handle_start(tid: int, uaddr: int, op: int, start_time: int) -> None:
+ """Handle a futex sys_enter event."""
+ if (op & FUTEX_CMD_MASK) != FUTEX_WAIT:
+ return
+ try:
+ if session:
+ process = session.find_thread(tid)
+ if process:
+ process_names[tid] = process.comm() or "unknown"
+ except (TypeError, AttributeError):
+ pass
+ if tid not in process_names:
+ process_names[tid] = "unknown"
+
+ start_times[tid] = (uaddr, start_time)
+
+def handle_end(tid: int, end_time: int) -> None:
+ """Handle a futex sys_exit event."""
+ if tid not in start_times:
+ return
+ (uaddr, start_time) = start_times[tid]
+ del start_times[tid]
+ durations[(tid, uaddr)].add(end_time - start_time)
+
+def process_event(sample: perf.sample_event) -> None:
+ """Process a single sample event."""
+ event_name = str(sample.evsel)
+ if event_name.startswith("evsel(syscalls:sys_enter_futex)"):
+ uaddr = getattr(sample, "uaddr", None)
+ op = getattr(sample, "op", None)
+ if uaddr is None or op is None:
+ return # Tracepoint fields missing, skip silent attribution to 0
+ handle_start(sample.sample_tid, uaddr, op, sample.sample_time)
+ elif event_name.startswith("evsel(syscalls:sys_exit_futex)"):
+ handle_end(sample.sample_tid, sample.sample_time)
+
+
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser(description="Measure futex contention")
+ ap.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ args = ap.parse_args()
+
+ session = perf.session(perf.data(args.input), sample=process_event)
+ session.process_events()
+
+ for ((t, u), stats) in sorted(durations.items()):
+ avg_ns = stats.avg()
+ print(f"{process_names.get(t, 'unknown')}[{t}] lock {u:x} contended {stats.count} times, "
+ f"{avg_ns:.0f} avg ns [max: {stats.max_time} ns, min {stats.min_time} ns]")
diff --git a/tools/perf/tests/shell/test_futex_contention_python.sh b/tools/perf/tests/shell/test_futex_contention_python.sh
new file mode 100755
index 000000000000..b7d6976e4f57
--- /dev/null
+++ b/tools/perf/tests/shell/test_futex_contention_python.sh
@@ -0,0 +1,67 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# futex-contention python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+# If we don't have the perf python module, we can't test
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/futex-contention.py"
+
+if ! perf check feature -q libtraceevent > /dev/null 2>&1; then
+ echo "Skipping test, libtraceevent is disabled"
+ exit 2
+fi
+
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, futex-contention.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+
+cleanup() {
+ rm -f "${temp_data}"
+}
+
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+
+test_file_mode() {
+ echo "Testing futex-contention.py..."
+ # Some systems might not have syscalls:sys_enter_futex
+ if ! perf list | grep -q syscalls:sys_enter_futex; then
+ echo "Skipping test, syscalls:sys_enter_futex not found"
+ exit 2
+ fi
+
+ # Generate some futex events
+ if ! perf record -e syscalls:sys_enter_futex,syscalls:sys_exit_futex -a -o "${temp_data}" \
+ -- sleep 0.5 2>/dev/null; then
+ echo "Skipping (record failed)"
+ exit 2
+ fi
+
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" >/dev/null; then
+ echo "File mode test failed."
+ err=1
+ else
+ echo "File mode test passed."
+ fi
+}
+
+test_file_mode
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 31/49] perf python: Port task-analyzer to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (29 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 30/49] perf python: Port futex-contention " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 32/49] perf python: Port sched-migration and SchedGui " Ian Rogers
` (17 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port task-analyzer.py from tools/perf/scripts/python/ to a standalone
script in tools/perf/python/ refactored into a class-based architecture.
Improvements compared to the legacy script:
- Support both offline perf.data file analysis (using perf.session) and
live trace capture (using evlist.read_on_cpu), accessing
sched:sched_switch tracepoint fields directly from sample objects.
- Automatically disable ANSI terminal color escape sequences when --csv
or --csv-summary is enabled so CSV column headers ('Comm,',
'Time Out-Out,', etc.) are never polluted by color codes when running
interactively on a TTY (e.g. as root).
- Sanitize non-printable characters and leading CSV formula characters
(=, +, -, @) in task comm strings.
- Emit a one-time warning to stderr if sched:sched_switch samples are
missing tracepoint fields (such as when recorded without libtraceevent
support).
Update test_task_analyzer.sh to invoke the standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/task-analyzer.py | 881 +++++++++++++++++++
tools/perf/tests/shell/test_task_analyzer.sh | 97 +-
2 files changed, 945 insertions(+), 33 deletions(-)
create mode 100755 tools/perf/python/task-analyzer.py
diff --git a/tools/perf/python/task-analyzer.py b/tools/perf/python/task-analyzer.py
new file mode 100755
index 000000000000..a39bdfe32fc1
--- /dev/null
+++ b/tools/perf/python/task-analyzer.py
@@ -0,0 +1,881 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+# task-analyzer.py - comprehensive perf tasks analysis
+# Copyright (c) 2022, Hagen Paul Pfeifer <hagen@jauu.net>
+# Licensed under the terms of the GNU GPL License version 2
+#
+# Usage:
+#
+# perf record -e sched:sched_switch -a -- sleep 10
+# ./task-analyzer.py
+#
+"""Comprehensive perf tasks analysis."""
+from __future__ import annotations
+
+import argparse
+from contextlib import contextmanager
+import decimal
+
+from typing import List, Dict, Union
+
+def _median(numbers: List[decimal.Decimal]) -> decimal.Decimal:
+ """phython3 hat statistics module - we have nothing"""
+ n = len(numbers)
+ index = n // 2
+ if n % 2:
+ return sorted(numbers)[index]
+ return sum(sorted(numbers)[index - 1 : index + 1]) / decimal.Decimal(2)
+
+def _mean(numbers: List[decimal.Decimal]) -> decimal.Decimal:
+ return sum(numbers) / decimal.Decimal(len(numbers))
+
+import os
+import string
+import sys
+from typing import Any, Optional
+import perf
+
+
+# Columns will have a static size to align everything properly
+# Support of 116 days of active update with nano precision
+LEN_SWITCHED_IN = len("9999999.999999999")
+LEN_SWITCHED_OUT = len("9999999.999999999")
+LEN_CPU = len("000")
+LEN_PID = len("maxvalue")
+LEN_TID = len("maxvalue")
+LEN_COMM = len("max-comms-length")
+LEN_RUNTIME = len("999999.999")
+# Support of 3.45 hours of timespans
+LEN_OUT_IN = len("99999999999.999")
+LEN_OUT_OUT = len("99999999999.999")
+LEN_IN_IN = len("99999999999.999")
+LEN_IN_OUT = len("99999999999.999")
+
+class Timespans:
+ """Tracks elapsed time between occurrences of the same task."""
+ def __init__(self, args: argparse.Namespace, time_unit: str) -> None:
+ self.args = args
+ self.time_unit = time_unit
+ self._last_start: Optional[decimal.Decimal] = None
+ self._last_finish: Optional[decimal.Decimal] = None
+ self.current = {
+ 'out_out': decimal.Decimal(-1),
+ 'in_out': decimal.Decimal(-1),
+ 'out_in': decimal.Decimal(-1),
+ 'in_in': decimal.Decimal(-1)
+ }
+ if args.summary_extended:
+ self._time_in: decimal.Decimal = decimal.Decimal(-1)
+ self.max_vals = {
+ 'out_in': decimal.Decimal(-1),
+ 'at': decimal.Decimal(-1),
+ 'in_out': decimal.Decimal(-1),
+ 'in_in': decimal.Decimal(-1),
+ 'out_out': decimal.Decimal(-1)
+ }
+
+ def feed(self, task: 'Task') -> None:
+ """Calculate timespans from chronological task occurrences."""
+ if not self._last_finish:
+ self._last_start = task.time_in(self.time_unit)
+ self._last_finish = task.time_out(self.time_unit)
+ return
+ assert self._last_start is not None
+ assert self._last_finish is not None
+ self._time_in = task.time_in()
+ time_in = task.time_in(self.time_unit)
+ time_out = task.time_out(self.time_unit)
+ self.current['in_in'] = time_in - self._last_start
+ self.current['out_in'] = time_in - self._last_finish
+ self.current['in_out'] = time_out - self._last_start
+ self.current['out_out'] = time_out - self._last_finish
+ if self.args.summary_extended:
+ self.update_max_entries()
+ self._last_finish = task.time_out(self.time_unit)
+ self._last_start = task.time_in(self.time_unit)
+
+ def update_max_entries(self) -> None:
+ """Update maximum timespans."""
+ self.max_vals['in_in'] = max(self.max_vals['in_in'], self.current['in_in'])
+ self.max_vals['out_out'] = max(self.max_vals['out_out'], self.current['out_out'])
+ self.max_vals['in_out'] = max(self.max_vals['in_out'], self.current['in_out'])
+ if self.current['out_in'] > self.max_vals['out_in']:
+ self.max_vals['out_in'] = self.current['out_in']
+ self.max_vals['at'] = self._time_in
+
+class Task:
+ """Handles information of a given task."""
+ def __init__(self, task_id: str, tid: int, cpu: int, comm: str) -> None:
+ self.id = task_id
+ self.tid = tid
+ self.cpu = cpu
+ self.comm = comm
+ self.pid: Optional[int] = None
+ self._time_in: Optional[decimal.Decimal] = None
+ self._time_out: Optional[decimal.Decimal] = None
+
+ def schedule_in_at(self, time_ns: int) -> None:
+ """Set schedule in time."""
+ self._time_in = decimal.Decimal(time_ns) / decimal.Decimal(1e9)
+
+ def schedule_out_at(self, time_ns: int) -> None:
+ """Set schedule out time."""
+ self._time_out = decimal.Decimal(time_ns) / decimal.Decimal(1e9)
+
+ def time_out(self, unit: str = "s") -> decimal.Decimal:
+ """Return schedule out time."""
+ factor = TaskAnalyzer.time_uniter(unit)
+ return self._time_out * decimal.Decimal(factor) if self._time_out else decimal.Decimal(0)
+
+ def time_in(self, unit: str = "s") -> decimal.Decimal:
+ """Return schedule in time."""
+ factor = TaskAnalyzer.time_uniter(unit)
+ return self._time_in * decimal.Decimal(factor) if self._time_in else decimal.Decimal(0)
+
+ def runtime(self, unit: str = "us") -> decimal.Decimal:
+ """Return runtime."""
+ factor = TaskAnalyzer.time_uniter(unit)
+ if self._time_out is not None and self._time_in is not None:
+ return (self._time_out - self._time_in) * decimal.Decimal(factor)
+ return decimal.Decimal(0)
+
+ def update_pid(self, pid: int) -> None:
+ """Update PID."""
+ self.pid = pid
+
+class Summary:
+ """
+ Primary instance for calculating the summary output. Processes the whole trace to
+ find and memorize relevant data such as mean, max et cetera. This instance handles
+ dynamic alignment aspects for summary output.
+ """
+
+ def __init__(self, analyzer):
+ self.analyzer = analyzer
+ self.args = analyzer.args
+ self.db = analyzer.db
+ self.time_unit = analyzer.time_unit
+ self.fd_sum = analyzer.fd_sum
+ self._body = []
+
+ class AlignmentHelper:
+ """
+ Used to calculated the alignment for the output of the summary.
+ """
+ def __init__(self, pid, tid, comm, runs, acc, mean,
+ median, min_val, max_val, max_at):
+ self.pid = pid
+ self.tid = tid
+ self.comm = comm
+ self.runs = runs
+ self.acc = acc
+ self.mean = mean
+ self.median = median
+ self.min = min_val
+ self.max = max_val
+ self.max_at = max_at
+ self.out_in = None
+ self.inter_at = None
+ self.out_out = None
+ self.in_in = None
+ self.in_out = None
+
+ def _print_header(self):
+ '''
+ Output is trimmed in _format_stats thus additional adjustment in the header
+ is needed, depending on the choice of timeunit. The adjustment corresponds
+ to the amount of column titles being adjusted in _column_titles.
+ '''
+ decimal_precision = 6 if not self.args.ns else 9
+ fmt = " {{:^{}}}".format(sum(self.db["task_info"].values()))
+ fmt += " {{:^{}}}".format(
+ max(0, sum(self.db["runtime_info"].values()) - 2 * decimal_precision)
+ )
+ _header = ("Task Information", "Runtime Information")
+
+ if self.args.summary_extended:
+ fmt += " {{:^{}}}".format(
+ max(0, sum(self.db["inter_times"].values()) - 4 * decimal_precision)
+ )
+ _header += ("Max Inter Task Times",)
+ self.fd_sum.write(fmt.format(*_header) + "\n")
+
+ def _column_titles(self):
+ """
+ Cells are being processed and displayed in different way so an alignment adjust
+ is implemented depeding on the choice of the timeunit. The positions of the max
+ values are being displayed in grey. Thus in their format two additional {},
+ are placed for color set and reset.
+ """
+ separator, fix_csv_align = self.analyzer.prepare_fmt_sep(is_summary=True)
+ decimal_precision, time_precision = self.analyzer.prepare_fmt_precision()
+ fmt = "{{:>{}}}".format(self.db["task_info"]["pid"] * fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, self.db["task_info"]["tid"] * fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, self.db["task_info"]["comm"] * fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, self.db["runtime_info"]["runs"] * fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, self.db["runtime_info"]["acc"] * fix_csv_align)
+ fmt += "{}{{:>{}}}".format(separator, self.db["runtime_info"]["mean"] * fix_csv_align)
+ fmt += "{}{{:>{}}}".format(
+ separator, self.db["runtime_info"]["median"] * fix_csv_align
+ )
+ fmt += "{}{{:>{}}}".format(
+ separator, max(0, self.db["runtime_info"]["min"] - decimal_precision) * fix_csv_align
+ )
+ fmt += "{}{{:>{}}}".format(
+ separator, max(0, self.db["runtime_info"]["max"] - decimal_precision) * fix_csv_align
+ )
+ fmt += "{}{{}}{{:>{}}}{{}}".format(
+ separator, max(0, self.db["runtime_info"]["max_at"] - time_precision) * fix_csv_align
+ )
+
+ grey = "" if self.args.csv_summary else TaskAnalyzer.COLORS["grey"]
+ reset = "" if self.args.csv_summary else TaskAnalyzer.COLORS["reset"]
+ column_titles = ("PID", "TID", "Comm")
+ column_titles += ("Runs", "Accumulated", "Mean", "Median", "Min", "Max")
+ column_titles += (grey, "Max At", reset)
+
+ if self.args.summary_extended:
+ fmt += "{}{{:>{}}}".format(
+ separator,
+ max(0, self.db["inter_times"]["out_in"] - decimal_precision) * fix_csv_align
+ )
+ fmt += "{}{{}}{{:>{}}}{{}}".format(
+ separator,
+ max(0, self.db["inter_times"]["inter_at"] - time_precision) * fix_csv_align
+ )
+ fmt += "{}{{:>{}}}".format(
+ separator,
+ max(0, self.db["inter_times"]["out_out"] - decimal_precision) * fix_csv_align
+ )
+ fmt += "{}{{:>{}}}".format(
+ separator,
+ max(0, self.db["inter_times"]["in_in"] - decimal_precision) * fix_csv_align
+ )
+ fmt += "{}{{:>{}}}".format(
+ separator,
+ max(0, self.db["inter_times"]["in_out"] - decimal_precision) * fix_csv_align
+ )
+
+ column_titles += (
+ "Out-In", grey, "Max At",
+ reset, "Out-Out", "In-In", "In-Out"
+ )
+
+ self.fd_sum.write(fmt.format(*column_titles) + "\n")
+
+
+ def _task_stats(self):
+ """calculates the stats of every task and constructs the printable summary"""
+ grey = "" if self.args.csv_summary else TaskAnalyzer.COLORS["grey"]
+ reset = "" if self.args.csv_summary else TaskAnalyzer.COLORS["reset"]
+ for tid in sorted(self.db["tid"]):
+ color_one_sample = grey
+ color_reset = reset
+ no_executed = 0
+ runtimes = []
+ time_in = []
+ timespans = Timespans(self.args, self.time_unit)
+ for task in self.db["tid"][tid]:
+ pid = task.pid
+ comm = task.comm
+ no_executed += 1
+ runtimes.append(task.runtime(self.time_unit))
+ time_in.append(task.time_in())
+ timespans.feed(task)
+ if len(runtimes) > 1:
+ color_one_sample = ""
+ color_reset = ""
+ time_max = max(runtimes)
+ time_min = min(runtimes)
+ max_at = time_in[runtimes.index(max(runtimes))]
+
+ # The size of the decimal after sum,mean and median varies, thus we cut
+ # the decimal number, by rounding it. It has no impact on the output,
+ # because we have a precision of the decimal points at the output.
+ time_sum = round(sum(runtimes), 3)
+ time_mean = round(_mean(runtimes), 3)
+ time_median = round(_median(runtimes), 3)
+
+ align_helper = self.AlignmentHelper(pid, tid, comm, no_executed, time_sum,
+ time_mean, time_median, time_min, time_max, max_at)
+ self._body.append([
+ pid, tid, comm, no_executed, time_sum, color_one_sample,
+ time_mean, time_median, time_min, time_max,
+ grey, max_at,
+ reset, color_reset
+ ])
+ if self.args.summary_extended:
+ self._body[-1].extend([timespans.max_vals['out_in'],
+ grey, timespans.max_vals['at'],
+ reset, timespans.max_vals['out_out'],
+ timespans.max_vals['in_in'],
+ timespans.max_vals['in_out']])
+ align_helper.out_in = timespans.max_vals['out_in']
+ align_helper.inter_at = timespans.max_vals['at']
+ align_helper.out_out = timespans.max_vals['out_out']
+ align_helper.in_in = timespans.max_vals['in_in']
+ align_helper.in_out = timespans.max_vals['in_out']
+ self._calc_alignments_summary(align_helper)
+
+ def _format_stats(self):
+ separator, fix_csv_align = self.analyzer.prepare_fmt_sep(is_summary=True)
+ decimal_precision, time_precision = self.analyzer.prepare_fmt_precision()
+ len_pid = self.db["task_info"]["pid"] * fix_csv_align
+ len_tid = self.db["task_info"]["tid"] * fix_csv_align
+ len_comm = self.db["task_info"]["comm"] * fix_csv_align
+ len_runs = self.db["runtime_info"]["runs"] * fix_csv_align
+ len_acc = self.db["runtime_info"]["acc"] * fix_csv_align
+ len_mean = self.db["runtime_info"]["mean"] * fix_csv_align
+ len_median = self.db["runtime_info"]["median"] * fix_csv_align
+ len_min = max(0, self.db["runtime_info"]["min"] - decimal_precision) * fix_csv_align
+ len_max = max(0, self.db["runtime_info"]["max"] - decimal_precision) * fix_csv_align
+ len_max_at = max(0, self.db["runtime_info"]["max_at"] - time_precision) * fix_csv_align
+ if self.args.summary_extended:
+ len_out_in = max(0,
+ self.db["inter_times"]["out_in"] - decimal_precision
+ ) * fix_csv_align
+ len_inter_at = max(0,
+ self.db["inter_times"]["inter_at"] - time_precision
+ ) * fix_csv_align
+ len_out_out = max(0,
+ self.db["inter_times"]["out_out"] - decimal_precision
+ ) * fix_csv_align
+ len_in_in = max(0, self.db["inter_times"]["in_in"] - decimal_precision) * fix_csv_align
+ len_in_out = max(0,
+ self.db["inter_times"]["in_out"] - decimal_precision
+ ) * fix_csv_align
+
+ fmt = "{{:{}d}}".format(len_pid)
+ fmt += "{}{{:{}d}}".format(separator, len_tid)
+ fmt += "{}{{:>{}}}".format(separator, len_comm)
+ fmt += "{}{{:{}d}}".format(separator, len_runs)
+ fmt += "{}{{:{}.{}f}}".format(separator, len_acc, time_precision)
+ fmt += "{}{{}}{{:{}.{}f}}".format(separator, len_mean, time_precision)
+ fmt += "{}{{:{}.{}f}}".format(separator, len_median, time_precision)
+ fmt += "{}{{:{}.{}f}}".format(separator, len_min, time_precision)
+ fmt += "{}{{:{}.{}f}}".format(separator, len_max, time_precision)
+ fmt += "{}{{}}{{:{}.{}f}}{{}}{{}}".format(
+ separator, len_max_at, decimal_precision
+ )
+ if self.args.summary_extended:
+ fmt += "{}{{:{}.{}f}}".format(separator, len_out_in, time_precision)
+ fmt += "{}{{}}{{:{}.{}f}}{{}}".format(
+ separator, len_inter_at, decimal_precision
+ )
+ fmt += "{}{{:{}.{}f}}".format(separator, len_out_out, time_precision)
+ fmt += "{}{{:{}.{}f}}".format(separator, len_in_in, time_precision)
+ fmt += "{}{{:{}.{}f}}".format(separator, len_in_out, time_precision)
+ return fmt
+
+
+ def _calc_alignments_summary(self, align_helper):
+ # Length is being cut in 3 groups so that further addition is easier to handle.
+ # The length of every argument from the alignment helper is being checked if it
+ # is longer than the longest until now. In that case the length is being saved.
+ for key in self.db["task_info"]:
+ if len(str(getattr(align_helper, key))) > self.db["task_info"][key]:
+ self.db["task_info"][key] = len(str(getattr(align_helper, key)))
+ for key in self.db["runtime_info"]:
+ if len(str(getattr(align_helper, key))) > self.db["runtime_info"][key]:
+ self.db["runtime_info"][key] = len(str(getattr(align_helper, key)))
+ if self.args.summary_extended:
+ for key in self.db["inter_times"]:
+ if len(str(getattr(align_helper, key))) > self.db["inter_times"][key]:
+ self.db["inter_times"][key] = len(str(getattr(align_helper, key)))
+
+
+ def print(self):
+ self._task_stats()
+ fmt = self._format_stats()
+
+ if not self.args.csv_summary:
+ print("\nSummary", file=self.fd_sum)
+ self._print_header()
+ self._column_titles()
+ for i in range(len(self._body)):
+ self.fd_sum.write(fmt.format(*tuple(self._body[i])) + "\n")
+
+
+
+
+class TaskAnalyzer:
+
+ """Main class for task analysis."""
+
+ COLORS = {
+ "grey": "\033[90m",
+ "red": "\033[91m",
+ "green": "\033[92m",
+ "yellow": "\033[93m",
+ "blue": "\033[94m",
+ "violet": "\033[95m",
+ "reset": "\033[0m",
+ }
+
+ def __init__(self, args: argparse.Namespace) -> None:
+ self.args = args
+ self.db: Dict[str, Any] = {}
+ self.session: Optional[perf.session] = None
+ self._tgid_cache: Dict[int, int] = {}
+ self.time_unit = "us"
+ if args.ns:
+ self.time_unit = "ns"
+ elif args.ms:
+ self.time_unit = "ms"
+ self._init_db()
+ self._check_color()
+ self.fd_task = sys.stdout
+ self.fd_sum = sys.stdout
+
+ @contextmanager
+ def open_output(self, filename: str, default: Any):
+ """Context manager for file or stdout."""
+ if filename:
+ with open(filename, "w", encoding="utf-8") as f:
+ yield f
+ else:
+ yield default
+
+ def _init_db(self) -> None:
+ self.db["running"] = {}
+ self.db["tid"] = {}
+ self.db["global"] = []
+ if (self.args.summary or self.args.summary_extended or
+ self.args.summary_only or self.args.csv_summary):
+ self.db["task_info"] = {}
+ self.db["runtime_info"] = {}
+ self.db["task_info"]["pid"] = len("PID")
+ self.db["task_info"]["tid"] = len("TID")
+ self.db["task_info"]["comm"] = len("Comm")
+ self.db["runtime_info"]["runs"] = len("Runs")
+ self.db["runtime_info"]["acc"] = len("Accumulated")
+ self.db["runtime_info"]["max"] = len("Max")
+ self.db["runtime_info"]["max_at"] = len("Max At")
+ self.db["runtime_info"]["min"] = len("Min")
+ self.db["runtime_info"]["mean"] = len("Mean")
+ self.db["runtime_info"]["median"] = len("Median")
+ if self.args.summary_extended:
+ self.db["inter_times"] = {}
+ self.db["inter_times"]["out_in"] = len("Out-In")
+ self.db["inter_times"]["inter_at"] = len("Max At")
+ self.db["inter_times"]["out_out"] = len("Out-Out")
+ self.db["inter_times"]["in_in"] = len("In-In")
+ self.db["inter_times"]["in_out"] = len("In-Out")
+
+ def _check_color(self) -> None:
+ """Check if color should be enabled."""
+ if self.args.csv:
+ TaskAnalyzer.COLORS = {k: "" for k in TaskAnalyzer.COLORS}
+ return
+ if sys.stdout.isatty() and self.args.stdio_color != "never":
+ return
+ if self.args.stdio_color == "always":
+ return
+ TaskAnalyzer.COLORS = {k: "" for k in TaskAnalyzer.COLORS}
+
+ @staticmethod
+ def time_uniter(unit: str) -> float:
+ """Return time unit factor."""
+ picker = {"s": 1, "ms": 1e3, "us": 1e6, "ns": 1e9}
+ return picker[unit]
+
+ def _task_id(self, pid: int, cpu: int) -> str:
+ return f"{pid}-{cpu}"
+
+ def _filter_non_printable(self, unfiltered: Union[str, bytearray]) -> str:
+ if isinstance(unfiltered, (bytearray, bytes)):
+ unfiltered = unfiltered.decode('utf-8', 'ignore')
+ filtered = ""
+ for char in unfiltered:
+ if char in string.printable and char not in "\r\n\t\x0b\x0c;":
+ filtered += char
+ stripped = filtered.lstrip()
+ if stripped and stripped[0] in "=+-@":
+ filtered = "_" + stripped[1:]
+ return filtered
+
+ def prepare_fmt_precision(self) -> tuple[int, int]:
+ if self.args.ns:
+ return 9, 0
+ return 6, 3
+
+ def prepare_fmt_sep(self, is_summary: bool = False) -> tuple[str, int]:
+ if (is_summary and self.args.csv_summary) or (not is_summary and self.args.csv):
+ return ";", 0
+ return " ", 1
+
+ def _fmt_header(self) -> str:
+ separator, fix_csv_align = self.prepare_fmt_sep()
+ fmt = f"{{:>{LEN_SWITCHED_IN*fix_csv_align}}}"
+ fmt += f"{separator}{{:>{LEN_SWITCHED_OUT*fix_csv_align}}}"
+ fmt += f"{separator}{{:>{LEN_CPU*fix_csv_align}}}"
+ fmt += f"{separator}{{:>{LEN_PID*fix_csv_align}}}"
+ fmt += f"{separator}{{:>{LEN_TID*fix_csv_align}}}"
+ fmt += f"{separator}{{:>{LEN_COMM*fix_csv_align}}}"
+ fmt += f"{separator}{{:>{LEN_RUNTIME*fix_csv_align}}}"
+ fmt += f"{separator}{{:>{LEN_OUT_IN*fix_csv_align}}}"
+ if self.args.extended_times:
+ fmt += f"{separator}{{:>{LEN_OUT_OUT*fix_csv_align}}}"
+ fmt += f"{separator}{{:>{LEN_IN_IN*fix_csv_align}}}"
+ fmt += f"{separator}{{:>{LEN_IN_OUT*fix_csv_align}}}"
+ return fmt
+
+ def _fmt_body(self) -> str:
+ separator, fix_csv_align = self.prepare_fmt_sep()
+ decimal_precision, time_precision = self.prepare_fmt_precision()
+ fmt = f"{{}}{{:{LEN_SWITCHED_IN*fix_csv_align}.{decimal_precision}f}}"
+ fmt += f"{separator}{{:{LEN_SWITCHED_OUT*fix_csv_align}.{decimal_precision}f}}"
+ fmt += f"{separator}{{:{LEN_CPU*fix_csv_align}d}}"
+ fmt += f"{separator}{{:{LEN_PID*fix_csv_align}d}}"
+ fmt += f"{separator}{{}}{{:{LEN_TID*fix_csv_align}d}}{{}}"
+ fmt += f"{separator}{{}}{{:>{LEN_COMM*fix_csv_align}}}"
+ fmt += f"{separator}{{:{LEN_RUNTIME*fix_csv_align}.{time_precision}f}}"
+ if self.args.extended_times:
+ fmt += f"{separator}{{:{LEN_OUT_IN*fix_csv_align}.{time_precision}f}}"
+ fmt += f"{separator}{{:{LEN_OUT_OUT*fix_csv_align}.{time_precision}f}}"
+ fmt += f"{separator}{{:{LEN_IN_IN*fix_csv_align}.{time_precision}f}}"
+ fmt += f"{separator}{{:{LEN_IN_OUT*fix_csv_align}.{time_precision}f}}{{}}"
+ else:
+ fmt += f"{separator}{{:{LEN_OUT_IN*fix_csv_align}.{time_precision}f}}{{}}"
+ return fmt
+
+ def _print_header(self) -> None:
+ fmt = self._fmt_header()
+ header = ["Switched-In", "Switched-Out", "CPU", "PID", "TID", "Comm",
+ "Runtime", "Time Out-In"]
+ if self.args.extended_times:
+ header += ["Time Out-Out", "Time In-In", "Time In-Out"]
+ self.fd_task.write(fmt.format(*header) + "\n")
+
+ def _print_task_finish(self, task: Task) -> None:
+ c_row_set = ""
+ c_row_reset = ""
+ out_in: Any = -1
+ out_out: Any = -1
+ in_in: Any = -1
+ in_out: Any = -1
+ fmt = self._fmt_body()
+
+ if str(task.tid) in self.args.highlight_tasks_map:
+ c_row_set = TaskAnalyzer.COLORS.get(self.args.highlight_tasks_map[str(task.tid)], '')
+ c_row_reset = TaskAnalyzer.COLORS["reset"]
+ if task.comm in self.args.highlight_tasks_map:
+ c_row_set = TaskAnalyzer.COLORS.get(self.args.highlight_tasks_map[task.comm], '')
+ c_row_reset = TaskAnalyzer.COLORS["reset"]
+
+ c_tid_set = ""
+ c_tid_reset = ""
+ if task.pid == task.tid:
+ c_tid_set = TaskAnalyzer.COLORS["grey"]
+ c_tid_reset = TaskAnalyzer.COLORS["reset"]
+
+ if task.tid in self.db["tid"]:
+ last_tid_task = self.db["tid"][task.tid][-1]
+ timespan_gap_tid = Timespans(self.args, self.time_unit)
+ timespan_gap_tid.feed(last_tid_task)
+ timespan_gap_tid.feed(task)
+ out_in = timespan_gap_tid.current['out_in']
+ out_out = timespan_gap_tid.current['out_out']
+ in_in = timespan_gap_tid.current['in_in']
+ in_out = timespan_gap_tid.current['in_out']
+
+ if self.args.extended_times:
+ line_out = fmt.format(c_row_set, task.time_in(), task.time_out(), task.cpu,
+ task.pid, c_tid_set, task.tid, c_tid_reset, c_row_set, task.comm,
+ task.runtime(self.time_unit), out_in, out_out, in_in, in_out,
+ c_row_reset) + "\n"
+ else:
+ line_out = fmt.format(c_row_set, task.time_in(), task.time_out(), task.cpu,
+ task.pid, c_tid_set, task.tid, c_tid_reset, c_row_set, task.comm,
+ task.runtime(self.time_unit), out_in, c_row_reset) + "\n"
+ self.fd_task.write(line_out)
+
+ def _record_cleanup(self, _list: list[Any]) -> list[Any]:
+ need_summary = (self.args.summary or self.args.summary_extended or
+ self.args.summary_only or self.args.csv_summary)
+ if not need_summary and len(_list) > 1:
+ return _list[len(_list) - 1:]
+ return _list
+
+ def _record_by_tid(self, task: Task) -> None:
+ tid = task.tid
+ if tid not in self.db["tid"]:
+ self.db["tid"][tid] = []
+ self.db["tid"][tid].append(task)
+ self.db["tid"][tid] = self._record_cleanup(self.db["tid"][tid])
+
+ def _record_global(self, task: Task) -> None:
+ self.db["global"].append(task)
+ self.db["global"] = self._record_cleanup(self.db["global"])
+
+ def _handle_task_finish(self, tid: int, cpu: int, time_ns: int, pid: int) -> None:
+ if tid == 0:
+ return
+ _id = self._task_id(tid, cpu)
+ if _id not in self.db["running"]:
+ return
+ task = self.db["running"][_id]
+ task.schedule_out_at(time_ns)
+ task.update_pid(pid)
+ del self.db["running"][_id]
+
+ if not self._limit_filtered(tid, pid, task.comm):
+ if not self.args.summary_only:
+ self._print_task_finish(task)
+ self._record_by_tid(task)
+ self._record_global(task)
+
+ def _handle_task_start(self, tid: int, cpu: int, comm: str, time_ns: int) -> None:
+ if tid == 0:
+ return
+ if tid in self.args.tid_renames:
+ comm = self._filter_non_printable(self.args.tid_renames[tid])
+ _id = self._task_id(tid, cpu)
+ if _id in self.db["running"]:
+ return
+ task = Task(_id, tid, cpu, comm)
+ task.schedule_in_at(time_ns)
+ self.db["running"][_id] = task
+
+ def _limit_filtered(self, tid: int, pid: int, comm: str) -> bool:
+ """Filter tasks based on CLI arguments."""
+ match_filter = False
+ if self.args.filter_tasks:
+ if (str(tid) in self.args.filter_tasks or
+ str(pid) in self.args.filter_tasks or
+ comm in self.args.filter_tasks):
+ match_filter = True
+
+ match_limit = False
+ if self.args.limit_to_tasks:
+ if (str(tid) in self.args.limit_to_tasks or
+ str(pid) in self.args.limit_to_tasks or
+ comm in self.args.limit_to_tasks):
+ match_limit = True
+
+ if self.args.filter_tasks and match_filter:
+ return True
+ if self.args.limit_to_tasks and not match_limit:
+ return True
+ return False
+
+ def _is_within_timelimit(self, time_ns: int) -> bool:
+ if not self.args.time_limit:
+ return True
+ time_s = decimal.Decimal(time_ns) / decimal.Decimal(1e9)
+ bounds = self.args.time_limit.split(":")
+ lower_bound = bounds[0] if len(bounds) > 0 else ""
+ upper_bound = bounds[1] if len(bounds) > 1 else ""
+ if lower_bound and time_s < decimal.Decimal(lower_bound):
+ return False
+ if upper_bound and time_s > decimal.Decimal(upper_bound):
+ return False
+ return True
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Process sched:sched_switch events."""
+ if "sched:sched_switch" not in str(sample.evsel):
+ return
+
+ time_ns = sample.sample_time
+ if not self._is_within_timelimit(time_ns):
+ return
+
+ # Access tracepoint fields directly from sample object
+ try:
+ prev_pid = sample.prev_pid
+ next_pid = sample.next_pid
+ next_comm = sample.next_comm
+ common_cpu = sample.sample_cpu
+ except AttributeError:
+ if not self.db.get("warned_missing_fields"):
+ self.db["warned_missing_fields"] = True
+ print("Warning: sched:sched_switch sample missing tracepoint fields "
+ "(is libtraceevent support enabled?).", file=sys.stderr)
+ return
+
+ next_comm = self._filter_non_printable(next_comm)
+
+ # Task finish for previous task
+ prev_tgid = prev_pid # Fallback
+ if prev_pid != 0:
+ if self.session:
+ try:
+ thread = self.session.find_thread(-1, prev_pid)
+ if thread:
+ prev_tgid = thread.pid
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError):
+ pass
+ elif prev_pid in self._tgid_cache:
+ prev_tgid = self._tgid_cache[prev_pid]
+ else:
+ if len(self._tgid_cache) >= 4096:
+ self._tgid_cache.pop(next(iter(self._tgid_cache)))
+ self._tgid_cache[prev_pid] = prev_tgid
+ try:
+ with open(f"/proc/{prev_pid}/status", encoding="utf-8") as f:
+ for line in f:
+ if line.startswith("Tgid:"):
+ prev_tgid = int(line.split()[1])
+ self._tgid_cache[prev_pid] = prev_tgid
+ break
+ except (OSError, ValueError, KeyError, RuntimeError):
+ pass
+ self._handle_task_finish(prev_pid, common_cpu, time_ns, prev_tgid)
+ # Task start for next task
+ self._handle_task_start(next_pid, common_cpu, next_comm, time_ns)
+
+ def print_summary(self) -> None:
+ """Calculate and print summary."""
+ need_summary = (self.args.summary or self.args.summary_extended or
+ self.args.summary_only or self.args.csv_summary)
+ if not need_summary:
+ return
+
+ Summary(self).print()
+
+ def _run_file(self) -> None:
+ if not self.args.summary_only:
+ self._print_header()
+
+ session = perf.session(perf.data(self.args.input), sample=self.process_event)
+ self.session = session
+ session.process_events()
+
+ if not self.db["global"]:
+ print(f"Warning: No sched:sched_switch trace events found in '{self.args.input}'.",
+ file=sys.stderr)
+
+ self.print_summary()
+
+ def _run_live(self) -> None:
+ if not self.args.summary_only:
+ self._print_header()
+
+ cpus = perf.cpu_map()
+ threads = perf.thread_map(-1)
+ evlist = perf.parse_events("sched:sched_switch", cpus, threads)
+ evlist.config()
+
+ evlist.open()
+ evlist.mmap()
+ evlist.enable()
+
+ pending_events: list[perf.sample_event] = []
+ print("Live mode started. Press Ctrl+C to stop.", file=sys.stderr)
+ try:
+ while True:
+ try:
+ evlist.poll(timeout=100)
+ except InterruptedError:
+ continue
+ for cpu in cpus:
+ while True:
+ event = evlist.read_on_cpu(cpu)
+ if not event:
+ break
+ if not isinstance(event, perf.sample_event):
+ continue
+ pending_events.append(event)
+ if pending_events:
+ pending_events.sort(
+ key=lambda e: getattr(e, 'sample_time', getattr(e, 'time', 0))
+ )
+ max_ts = getattr(
+ pending_events[-1], 'sample_time', getattr(pending_events[-1], 'time', 0)
+ )
+ cutoff_ts = max_ts - 50_000_000
+ ready_idx = 0
+ for event in pending_events:
+ ev_ts = getattr(event, 'sample_time', getattr(event, 'time', 0))
+ if ev_ts <= cutoff_ts:
+ self.process_event(event)
+ ready_idx += 1
+ else:
+ break
+ del pending_events[:ready_idx]
+ except KeyboardInterrupt:
+ print("\nStopping live mode...", file=sys.stderr)
+ finally:
+ pending_events.sort(key=lambda e: getattr(e, 'sample_time', getattr(e, 'time', 0)))
+ for event in pending_events:
+ self.process_event(event)
+ evlist.close()
+ self.print_summary()
+
+ def run(self) -> None:
+ """Run the session."""
+ is_live = (self.args.live or
+ (not os.path.exists(self.args.input) and self.args.input == "perf.data"))
+ with self.open_output(self.args.csv, sys.stdout) as fd_task:
+ if self.args.csv and self.args.csv == self.args.csv_summary:
+ self.fd_task = fd_task
+ self.fd_sum = fd_task
+ if is_live:
+ self._run_live()
+ else:
+ self._run_file()
+ else:
+ with self.open_output(self.args.csv_summary, sys.stdout) as fd_sum:
+ self.fd_task = fd_task
+ self.fd_sum = fd_sum
+ if is_live:
+ self._run_live()
+ else:
+ self._run_file()
+
+def main() -> None:
+ """Main function."""
+ parser = argparse.ArgumentParser(description="Analyze tasks behavior")
+ parser.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ parser.add_argument("--time-limit", default="", help="print tasks only in time window")
+ parser.add_argument("--summary", action="store_true",
+ help="print additional runtime information")
+ parser.add_argument("--summary-only", action="store_true",
+ help="print only summary without traces")
+ parser.add_argument("--summary-extended", action="store_true",
+ help="print extended summary")
+ parser.add_argument("--ns", action="store_true", help="show timestamps in nanoseconds")
+ parser.add_argument("--ms", action="store_true", help="show timestamps in milliseconds")
+ parser.add_argument("--live", action="store_true",
+ help="force live mode (ignores -i/perf.data)")
+ parser.add_argument("--extended-times", action="store_true",
+ help="Show elapsed times between schedule in/out")
+ parser.add_argument("--filter-tasks", default="", help="filter tasks by tid, pid or comm")
+ parser.add_argument("--limit-to-tasks", default="", help="limit output to selected tasks")
+ parser.add_argument("--highlight-tasks", default="", help="colorize special tasks")
+ parser.add_argument("--rename-comms-by-tids", default="", help="rename task names by using tid")
+ parser.add_argument("--stdio-color", default="auto", choices=["always", "never", "auto"],
+ help="configure color output")
+ parser.add_argument("--csv", default="", help="Write trace to file")
+ parser.add_argument("--csv-summary", default="", help="Write summary to file")
+
+ args = parser.parse_args()
+ args.tid_renames = {}
+ args.highlight_tasks_map = {}
+ args.filter_tasks = args.filter_tasks.split(",") if args.filter_tasks else []
+ args.limit_to_tasks = args.limit_to_tasks.split(",") if args.limit_to_tasks else []
+
+ if args.rename_comms_by_tids:
+ for item in args.rename_comms_by_tids.split(","):
+ try:
+ tid, name = item.split(":", 1)
+ args.tid_renames[int(tid)] = name
+ except ValueError:
+ print(f"Error: Invalid format for --rename-comms-by-tids '{item}', "
+ "expected tid:name", file=sys.stderr)
+ sys.exit(1)
+
+ if args.highlight_tasks:
+ for item in args.highlight_tasks.split(","):
+ parts = item.split(":")
+ if len(parts) == 1:
+ parts.append("red")
+ key, color = parts[0], parts[1]
+ args.highlight_tasks_map[key] = color
+
+ analyzer = TaskAnalyzer(args)
+ analyzer.run()
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/perf/tests/shell/test_task_analyzer.sh b/tools/perf/tests/shell/test_task_analyzer.sh
index 0314412e63b4..519885dfa073 100755
--- a/tools/perf/tests/shell/test_task_analyzer.sh
+++ b/tools/perf/tests/shell/test_task_analyzer.sh
@@ -1,21 +1,27 @@
#!/bin/bash
-# perf script task-analyzer tests (exclusive)
# SPDX-License-Identifier: GPL-2.0
+# task-analyzer python test
+
+# shellcheck source=lib/setup_python.sh
+. "$(dirname "$0")"/lib/setup_python.sh
tmpdir=$(mktemp -d /tmp/perf-script-task-analyzer-XXXXX)
-# TODO: perf script report only supports input from the CWD perf.data file, make
-# it support input from any file.
-perfdata="perf.data"
+perfdata="$tmpdir/perf.data"
csv="$tmpdir/csv"
csvsummary="$tmpdir/csvsummary"
err=0
-# set PERF_EXEC_PATH to find scripts in the source directory
-perfdir=$(dirname "$0")/../..
-if [ -e "$perfdir/scripts/python/Perf-Trace-Util" ]; then
- export PERF_EXEC_PATH=$perfdir
+# Set up perfdir and PERF_EXEC_PATH
+if [ "x$PERF_EXEC_PATH" = "x" ]; then
+ perfdir=$(dirname "$0")/../..
+ if [ -f $perfdir/python/task-analyzer.py ]; then
+ export PERF_EXEC_PATH=$perfdir
+ fi
+else
+ perfdir=$PERF_EXEC_PATH
fi
+
# Disable lsan to avoid warnings about python memory leaks.
export ASAN_OPTIONS=detect_leaks=0
@@ -76,85 +82,106 @@ prepare_perf_data() {
# check standard inkvokation with no arguments
test_basic() {
out="$tmpdir/perf.out"
- perf script report task-analyzer > "$out"
- check_exec_0 "perf script report task-analyzer"
+ $PYTHON $perfdir/python/task-analyzer.py -i "${perfdata}" > "$out"
+ check_exec_0 "$PYTHON $perfdir/python/task-analyzer.py -i ${perfdata}"
find_str_or_fail "Comm" "$out" "${FUNCNAME[0]}"
}
test_ns_rename(){
out="$tmpdir/perf.out"
- perf script report task-analyzer --ns --rename-comms-by-tids 0:random > "$out"
- check_exec_0 "perf script report task-analyzer --ns --rename-comms-by-tids 0:random"
+ $PYTHON $perfdir/python/task-analyzer.py -i "${perfdata}" \
+ --ns \
+ --rename-comms-by-tids 0:random > "$out"
+ check_exec_0 "$PYTHON $perfdir/python/task-analyzer.py -i ${perfdata} \
+ --ns \
+ --rename-comms-by-tids 0:random"
find_str_or_fail "Comm" "$out" "${FUNCNAME[0]}"
}
test_ms_filtertasks_highlight(){
out="$tmpdir/perf.out"
- perf script report task-analyzer --ms --filter-tasks perf --highlight-tasks perf \
+ $PYTHON $perfdir/python/task-analyzer.py -i "${perfdata}" \
+ --ms --filter-tasks perf --highlight-tasks perf \
> "$out"
- check_exec_0 "perf script report task-analyzer --ms --filter-tasks perf --highlight-tasks perf"
+ check_exec_0 "$PYTHON $perfdir/python/task-analyzer.py -i ${perfdata} \
+ --ms --filter-tasks perf --highlight-tasks perf"
find_str_or_fail "Comm" "$out" "${FUNCNAME[0]}"
}
test_extended_times_timelimit_limittasks() {
out="$tmpdir/perf.out"
- perf script report task-analyzer --extended-times --time-limit :99999 \
+ $PYTHON $perfdir/python/task-analyzer.py -i "${perfdata}" --extended-times \
+ --time-limit :99999 \
--limit-to-tasks perf > "$out"
- check_exec_0 "perf script report task-analyzer --extended-times --time-limit :99999 --limit-to-tasks perf"
+ check_exec_0 "$PYTHON $perfdir/python/task-analyzer.py -i ${perfdata} \
+ --extended-times \
+ --time-limit :99999 --limit-to-tasks perf"
find_str_or_fail "Out-Out" "$out" "${FUNCNAME[0]}"
}
test_summary() {
out="$tmpdir/perf.out"
- perf script report task-analyzer --summary > "$out"
- check_exec_0 "perf script report task-analyzer --summary"
+ $PYTHON $perfdir/python/task-analyzer.py -i "${perfdata}" --summary > "$out"
+ check_exec_0 "$PYTHON $perfdir/python/task-analyzer.py -i ${perfdata} --summary"
find_str_or_fail "Summary" "$out" "${FUNCNAME[0]}"
}
test_summaryextended() {
out="$tmpdir/perf.out"
- perf script report task-analyzer --summary-extended > "$out"
- check_exec_0 "perf script report task-analyzer --summary-extended"
+ $PYTHON $perfdir/python/task-analyzer.py -i "${perfdata}" --summary-extended > "$out"
+ check_exec_0 "$PYTHON $perfdir/python/task-analyzer.py -i ${perfdata} --summary-extended"
find_str_or_fail "Inter Task Times" "$out" "${FUNCNAME[0]}"
}
test_summaryonly() {
out="$tmpdir/perf.out"
- perf script report task-analyzer --summary-only > "$out"
- check_exec_0 "perf script report task-analyzer --summary-only"
+ $PYTHON $perfdir/python/task-analyzer.py -i "${perfdata}" --summary-only > "$out"
+ check_exec_0 "$PYTHON $perfdir/python/task-analyzer.py -i ${perfdata} --summary-only"
find_str_or_fail "Summary" "$out" "${FUNCNAME[0]}"
}
test_extended_times_summary_ns() {
out="$tmpdir/perf.out"
- perf script report task-analyzer --extended-times --summary --ns > "$out"
- check_exec_0 "perf script report task-analyzer --extended-times --summary --ns"
+ $PYTHON $perfdir/python/task-analyzer.py -i "${perfdata}" --extended-times \
+ --summary --ns > "$out"
+ check_exec_0 "$PYTHON $perfdir/python/task-analyzer.py -i ${perfdata} \
+ --extended-times \
+ --summary \
+ --ns"
find_str_or_fail "Out-Out" "$out" "${FUNCNAME[0]}"
find_str_or_fail "Summary" "$out" "${FUNCNAME[0]}"
}
test_csv() {
- perf script report task-analyzer --csv "${csv}" > /dev/null
- check_exec_0 "perf script report task-analyzer --csv ${csv}"
+ $PYTHON $perfdir/python/task-analyzer.py -i "${perfdata}" --csv "${csv}" > /dev/null
+ check_exec_0 "$PYTHON $perfdir/python/task-analyzer.py -i ${perfdata} --csv ${csv}"
find_str_or_fail "Comm;" "${csv}" "${FUNCNAME[0]}"
}
test_csv_extended_times() {
- perf script report task-analyzer --csv "${csv}" --extended-times > /dev/null
- check_exec_0 "perf script report task-analyzer --csv ${csv} --extended-times"
+ $PYTHON $perfdir/python/task-analyzer.py -i "${perfdata}" \
+ --csv "${csv}" \
+ --extended-times > /dev/null
+ check_exec_0 "$PYTHON $perfdir/python/task-analyzer.py -i ${perfdata} \
+ --csv ${csv} \
+ --extended-times"
find_str_or_fail "Out-Out;" "${csv}" "${FUNCNAME[0]}"
}
test_csvsummary() {
- perf script report task-analyzer --csv-summary "${csvsummary}" > /dev/null
- check_exec_0 "perf script report task-analyzer --csv-summary ${csvsummary}"
+ $PYTHON $perfdir/python/task-analyzer.py -i "${perfdata}" \
+ --csv-summary "${csvsummary}" > /dev/null
+ check_exec_0 "$PYTHON $perfdir/python/task-analyzer.py -i ${perfdata} \
+ --csv-summary ${csvsummary}"
find_str_or_fail "Comm;" "${csvsummary}" "${FUNCNAME[0]}"
}
test_csvsummary_extended() {
- perf script report task-analyzer --csv-summary "${csvsummary}" --summary-extended \
+ $PYTHON $perfdir/python/task-analyzer.py -i "${perfdata}" \
+ --csv-summary "${csvsummary}" --summary-extended \
>/dev/null
- check_exec_0 "perf script report task-analyzer --csv-summary ${csvsummary} --summary-extended"
+ check_exec_0 "$PYTHON $perfdir/python/task-analyzer.py -i ${perfdata} \
+ --csv-summary ${csvsummary} --summary-extended"
find_str_or_fail "Out-Out;" "${csvsummary}" "${FUNCNAME[0]}"
}
@@ -165,7 +192,11 @@ if [ $err -ne 0 ]; then
cleanup
exit $err
fi
-prepare_perf_data
+prepare_perf_data || {
+ echo "Skipping tests, failed to prepare perf.data"
+ cleanup
+ exit 2
+}
test_basic
test_ns_rename
test_ms_filtertasks_highlight
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 32/49] perf python: Port sched-migration and SchedGui to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (30 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 31/49] perf python: Port task-analyzer " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 33/49] perf python: Port wakeup-latency from Perl " Ian Rogers
` (16 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port sched-migration.py and SchedGui.py from tools/perf/scripts/python/
to standalone modules in tools/perf/python/:
- Refactor sched-migration.py into a SchedMigrationAnalyzer class using
perf.session for event processing and add argparse CLI support
(-i/--input, -v/--verbose, --gui, --no-gui).
- Port SchedGui.py to tools/perf/python/ as a local module dependency.
- Load wxPython dynamically via importlib only when GUI mode (--gui) is
requested so text-mode analysis, testing on headless systems, and
static analysis (mypy/pylint) succeed without wx installed.
- Remove Python 2 compatibility code.
Add a shell test (test_sched_migration_python.sh) to verify the
standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/SchedGui.py | 246 +++++++++
tools/perf/python/sched-migration.py | 486 ++++++++++++++++++
.../shell/test_sched_migration_python.sh | 69 +++
3 files changed, 801 insertions(+)
create mode 100755 tools/perf/python/SchedGui.py
create mode 100755 tools/perf/python/sched-migration.py
create mode 100755 tools/perf/tests/shell/test_sched_migration_python.sh
diff --git a/tools/perf/python/SchedGui.py b/tools/perf/python/SchedGui.py
new file mode 100755
index 000000000000..fa0d010fe528
--- /dev/null
+++ b/tools/perf/python/SchedGui.py
@@ -0,0 +1,246 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+# SchedGui.py - Python extension for perf script, basic GUI code for
+# traces drawing and overview.
+#
+# Copyright (C) 2010 by Frederic Weisbecker <fweisbec@gmail.com>
+#
+# Ported to modern directory structure.
+
+"""SchedGui.py - Python extension for perf script, basic GUI code for traces drawing and overview."""
+from __future__ import annotations
+
+import importlib
+from typing import Any
+
+class _DummyWx:
+ """Dummy wx module fallback when wxPython is not installed."""
+ Frame = object
+
+
+try:
+ wx: Any = importlib.import_module("wx")
+ WX_AVAILABLE = True
+except ImportError:
+ wx = _DummyWx
+ WX_AVAILABLE = False
+
+
+class RootFrame(wx.Frame):
+ """Main window frame for scheduling trace visualization."""
+ Y_OFFSET = 100
+ RECT_HEIGHT = 100
+ RECT_SPACE = 50
+ EVENT_MARKING_WIDTH = 5
+
+ def __init__(self, sched_tracer, title, parent=None, win_id=-1):
+ if not WX_AVAILABLE:
+ raise ImportError("You need to install the wxpython lib for this script")
+ wx.Frame.__init__(self, parent, win_id, title)
+
+ self.dc = None
+ (self.screen_width, self.screen_height) = wx.GetDisplaySize()
+ self.screen_width -= 10
+ self.screen_height -= 10
+ self.zoom = 0.5
+ self.scroll_scale = 20
+ self.sched_tracer = sched_tracer
+ self.sched_tracer.set_root_win(self)
+ (self.ts_start, self.ts_end) = sched_tracer.interval()
+ self.update_width_virtual()
+ self.nr_rects = sched_tracer.nr_rectangles() + 1
+ self.height_virtual = RootFrame.Y_OFFSET + \
+ (self.nr_rects * (RootFrame.RECT_HEIGHT + RootFrame.RECT_SPACE))
+
+ # whole window panel
+ self.panel = wx.Panel(self, size=(self.screen_width, self.screen_height))
+
+ # scrollable container
+ # Create SplitterWindow
+ self.splitter = wx.SplitterWindow(self.panel, style=wx.SP_3D)
+
+ # scrollable container (Top)
+ self.scroll = wx.ScrolledWindow(self.splitter)
+ self.scroll.SetScrollbars(self.scroll_scale, self.scroll_scale,
+ int(self.width_virtual // self.scroll_scale),
+ int(self.height_virtual // self.scroll_scale))
+ self.scroll.EnableScrolling(True, True)
+ self.scroll.SetFocus()
+
+ # scrollable drawing area
+ self.scroll_panel = wx.Panel(self.scroll,
+ size=(self.screen_width - 15, self.screen_height // 2))
+ self.scroll_panel.Bind(wx.EVT_PAINT, self.on_paint)
+ self.scroll_panel.Bind(wx.EVT_KEY_DOWN, self.on_key_press)
+ self.scroll_panel.Bind(wx.EVT_LEFT_DOWN, self.on_mouse_down)
+ self.scroll.Bind(wx.EVT_KEY_DOWN, self.on_key_press)
+ self.scroll.Bind(wx.EVT_LEFT_DOWN, self.on_mouse_down)
+
+ self.scroll_panel.SetSize(int(self.width_virtual), int(self.height_virtual))
+
+ # Create a separate panel for text (Bottom)
+ self.text_panel = wx.Panel(self.splitter)
+ self.text_sizer = wx.BoxSizer(wx.VERTICAL)
+ self.txt = wx.TextCtrl(self.text_panel, -1, "Click a bar to see details",
+ style=wx.TE_MULTILINE)
+ self.text_sizer.Add(self.txt, 1, wx.EXPAND | wx.ALL, 5)
+ self.text_panel.SetSizer(self.text_sizer)
+
+ # Split the window
+ self.splitter.SplitHorizontally(self.scroll, self.text_panel, (self.screen_height * 3) // 4)
+
+ # Main sizer to layout splitter
+ self.main_sizer = wx.BoxSizer(wx.VERTICAL)
+ self.main_sizer.Add(self.splitter, 1, wx.EXPAND)
+ self.panel.SetSizer(self.main_sizer)
+
+ self.scroll.Fit()
+ self.Fit()
+
+ self.Show(True)
+
+ def us_to_px(self, val):
+ """Convert microseconds to pixels."""
+ return val / (10 ** 3) * self.zoom
+
+ def px_to_us(self, val):
+ """Convert pixels to microseconds."""
+ return (val / self.zoom) * (10 ** 3)
+
+ def scroll_start(self):
+ """Get scroll start position in pixels."""
+ (x, y) = self.scroll.GetViewStart()
+ return (x * self.scroll_scale, y * self.scroll_scale)
+
+ def scroll_start_us(self):
+ """Get scroll start position in microseconds."""
+ (x, _) = self.scroll_start()
+ return self.px_to_us(x)
+
+ def paint_rectangle_zone(self, nr, color, top_color, start, end):
+ """Draw a rectangle zone for a CPU."""
+ offset_px = self.us_to_px(start - self.ts_start)
+ width_px = self.us_to_px(end - start)
+
+ offset_py = RootFrame.Y_OFFSET + (nr * (RootFrame.RECT_HEIGHT + RootFrame.RECT_SPACE))
+ width_py = RootFrame.RECT_HEIGHT
+
+ dc = self.dc
+
+ if top_color is not None:
+ (r, g, b) = top_color
+ top_color = wx.Colour(r, g, b)
+ brush = wx.Brush(top_color, wx.SOLID)
+ dc.SetBrush(brush)
+ dc.DrawRectangle(int(offset_px), int(offset_py),
+ int(width_px), RootFrame.EVENT_MARKING_WIDTH)
+ width_py -= RootFrame.EVENT_MARKING_WIDTH
+ offset_py += RootFrame.EVENT_MARKING_WIDTH
+
+ (r, g, b) = color
+ color = wx.Colour(r, g, b)
+ brush = wx.Brush(color, wx.SOLID)
+ dc.SetBrush(brush)
+ dc.DrawRectangle(int(offset_px), int(offset_py), int(width_px), int(width_py))
+
+ def update_rectangles(self, start, end):
+ """Update rectangles in the given time window."""
+ start += self.ts_start
+ end += self.ts_start
+ self.sched_tracer.fill_zone(start, end)
+
+ def on_paint(self, event):
+ """Handle paint event."""
+ window = event.GetEventObject()
+ dc = wx.PaintDC(window)
+
+ # Clear background to avoid ghosting
+ dc.SetBackground(wx.Brush(window.GetBackgroundColour()))
+ dc.Clear()
+
+ self.dc = dc
+ try:
+ width = min(self.width_virtual, self.screen_width)
+ (x, _) = self.scroll_start()
+ start = self.px_to_us(x)
+ end = self.px_to_us(x + width)
+ self.update_rectangles(start, end)
+
+ # Draw CPU labels at the left edge of the visible area
+ (x_scroll, _) = self.scroll_start()
+ for nr in range(self.nr_rects):
+ offset_py = RootFrame.Y_OFFSET + (nr * (RootFrame.RECT_HEIGHT + RootFrame.RECT_SPACE))
+ dc.DrawText(f"CPU {nr}", x_scroll + 10, offset_py + 10)
+ finally:
+ self.dc = None
+
+ def rect_from_ypixel(self, y):
+ y -= RootFrame.Y_OFFSET
+ rect = y // (RootFrame.RECT_HEIGHT + RootFrame.RECT_SPACE)
+ height = y % (RootFrame.RECT_HEIGHT + RootFrame.RECT_SPACE)
+
+ if rect < 0 or rect > self.nr_rects - 1 or height > RootFrame.RECT_HEIGHT:
+ return -1
+
+ return rect
+
+ def update_summary(self, txt):
+ self.txt.SetValue(txt)
+ self.text_panel.Layout()
+ self.splitter.Layout()
+ self.text_panel.Refresh()
+
+ def on_mouse_down(self, event):
+ pos = event.GetPosition()
+ x, y = pos.x, pos.y
+ rect = self.rect_from_ypixel(y)
+ if rect == -1:
+ return
+
+ t = self.px_to_us(x) + self.ts_start
+
+ self.sched_tracer.mouse_down(rect, t)
+
+ def update_width_virtual(self):
+ self.width_virtual = self.us_to_px(self.ts_end - self.ts_start)
+
+ def __zoom(self, x):
+ self.update_width_virtual()
+ (xpos, ypos) = self.scroll.GetViewStart()
+ xpos = int(self.us_to_px(x) // self.scroll_scale)
+ self.scroll_panel.SetSize((int(self.width_virtual), int(self.height_virtual)))
+ self.scroll.SetScrollbars(self.scroll_scale, self.scroll_scale,
+ int(self.width_virtual // self.scroll_scale),
+ int(self.height_virtual // self.scroll_scale),
+ xpos, ypos)
+ self.Refresh()
+
+ def zoom_in(self):
+ x = self.scroll_start_us()
+ self.zoom *= 2
+ self.__zoom(x)
+
+ def zoom_out(self):
+ x = self.scroll_start_us()
+ self.zoom /= 2
+ self.__zoom(x)
+
+ def on_key_press(self, event):
+ key = event.GetRawKeyCode()
+ if key == ord("+"):
+ self.zoom_in()
+ return
+ if key == ord("-"):
+ self.zoom_out()
+ return
+
+ key = event.GetKeyCode()
+ (x, y) = self.scroll.GetViewStart()
+ if key == wx.WXK_RIGHT:
+ self.scroll.Scroll(x + 1, y)
+ elif key == wx.WXK_LEFT:
+ self.scroll.Scroll(x - 1, y)
+ elif key == wx.WXK_DOWN:
+ self.scroll.Scroll(x, y + 1)
+ elif key == wx.WXK_UP:
+ self.scroll.Scroll(x, y - 1)
diff --git a/tools/perf/python/sched-migration.py b/tools/perf/python/sched-migration.py
new file mode 100755
index 000000000000..0684b2876ed7
--- /dev/null
+++ b/tools/perf/python/sched-migration.py
@@ -0,0 +1,486 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+Cpu task migration overview toy
+
+Copyright (C) 2010 Frederic Weisbecker <fweisbec@gmail.com>
+Ported to modern directory structure and refactored to use class.
+"""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict, UserList
+import importlib
+import sys
+from typing import Any
+import perf
+
+
+# Global threads dictionary
+threads = defaultdict(lambda: "unknown")
+threads[0] = "idle"
+
+
+def thread_name(pid: int) -> str:
+ """Return thread name formatted with pid."""
+ return f"{threads[pid]}:{pid}"
+
+
+def task_state(state: int) -> str:
+ """Map task state integer to string."""
+ states = {
+ 0: "R",
+ 1: "S",
+ 2: "D",
+ 64: "DEAD"
+ }
+ return states.get(state, "Unknown")
+
+
+class RunqueueEventUnknown:
+ """Unknown runqueue event."""
+ @staticmethod
+ def color():
+ """Return color for event."""
+ return None
+
+ def __repr__(self):
+ return "unknown"
+
+
+class RunqueueEventSleep:
+ """Sleep runqueue event."""
+ @staticmethod
+ def color():
+ """Return color for event."""
+ return 0, 0, 0xff
+
+ def __init__(self, sleeper: int):
+ self.sleeper = sleeper
+
+ def __repr__(self):
+ return f"{thread_name(self.sleeper)} gone to sleep"
+
+
+class RunqueueEventWakeup:
+ """Wakeup runqueue event."""
+ @staticmethod
+ def color():
+ """Return color for event."""
+ return 0xff, 0xff, 0
+
+ def __init__(self, wakee: int):
+ self.wakee = wakee
+
+ def __repr__(self):
+ return f"{thread_name(self.wakee)} woke up"
+
+
+class RunqueueEventFork:
+ """Fork runqueue event."""
+ @staticmethod
+ def color():
+ """Return color for event."""
+ return 0, 0xff, 0
+
+ def __init__(self, child: int):
+ self.child = child
+
+ def __repr__(self):
+ return f"new forked task {thread_name(self.child)}"
+
+
+class RunqueueMigrateIn:
+ """Migrate in runqueue event."""
+ @staticmethod
+ def color():
+ """Return color for event."""
+ return 0, 0xf0, 0xff
+
+ def __init__(self, new: int):
+ self.new = new
+
+ def __repr__(self):
+ return f"task migrated in {thread_name(self.new)}"
+
+
+class RunqueueMigrateOut:
+ """Migrate out runqueue event."""
+ @staticmethod
+ def color():
+ """Return color for event."""
+ return 0xff, 0, 0xff
+
+ def __init__(self, old: int):
+ self.old = old
+
+ def __repr__(self):
+ return f"task migrated out {thread_name(self.old)}"
+
+
+class RunqueueSnapshot:
+ """Snapshot of runqueue state."""
+
+ def __init__(self, tasks=None, event=None):
+ if tasks is None:
+ tasks = (0,)
+ if event is None:
+ event = RunqueueEventUnknown()
+ self.tasks = tuple(tasks)
+ self.event = event
+
+ def sched_switch(self, prev: int, prev_state: int, next_pid: int):
+ """Handle sched switch in snapshot."""
+ if task_state(prev_state) == "R" and next_pid in self.tasks \
+ and prev in self.tasks:
+ return self
+
+ event = (
+ RunqueueEventSleep(prev)
+ if task_state(prev_state) != "R"
+ else RunqueueEventUnknown()
+ )
+
+ next_tasks = list(self.tasks[:])
+ if prev in self.tasks:
+ if task_state(prev_state) != "R":
+ next_tasks.remove(prev)
+ elif task_state(prev_state) == "R":
+ next_tasks.append(prev)
+
+ if next_pid not in next_tasks:
+ next_tasks.append(next_pid)
+
+ return RunqueueSnapshot(next_tasks, event)
+
+ def migrate_out(self, old: int):
+ """Handle task migrate out in snapshot."""
+ if old not in self.tasks:
+ return self
+ next_tasks = [task for task in self.tasks if task != old]
+
+ return RunqueueSnapshot(next_tasks, RunqueueMigrateOut(old))
+
+ def __migrate_in(self, new: int, event):
+ if new in self.tasks:
+ return RunqueueSnapshot(self.tasks, event)
+ next_tasks = self.tasks + tuple([new])
+
+ return RunqueueSnapshot(next_tasks, event)
+
+ def migrate_in(self, new: int):
+ """Handle task migrate in snapshot."""
+ return self.__migrate_in(new, RunqueueMigrateIn(new))
+
+ def wake_up(self, new: int):
+ """Handle task wakeup in snapshot."""
+ return self.__migrate_in(new, RunqueueEventWakeup(new))
+
+ def wake_up_new(self, new: int):
+ """Handle task fork in snapshot."""
+ return self.__migrate_in(new, RunqueueEventFork(new))
+
+ def load(self) -> int:
+ """Provide the number of tasks on the runqueue. Don't count idle"""
+ return len(self.tasks) - 1
+
+ def __repr__(self):
+ return self.tasks.__repr__()
+
+
+class TimeSlice:
+ """Represents a time slice of execution."""
+
+ def __init__(self, start: int, prev):
+ self.start = start
+ self.prev = prev
+ self.end = start
+ # cpus that triggered the event
+ self.event_cpus: list[int] = []
+ if prev is not None:
+ self.total_load = prev.total_load
+ self.rqs = prev.rqs.copy()
+ else:
+ self.rqs = defaultdict(RunqueueSnapshot)
+ self.total_load = 0
+
+ def __update_total_load(self, old_rq: RunqueueSnapshot, new_rq: RunqueueSnapshot):
+ diff = new_rq.load() - old_rq.load()
+ self.total_load += diff
+
+ def sched_switch(self, ts_list, prev: int, prev_state: int, next_pid: int, cpu: int):
+ """Process sched_switch in time slice."""
+ old_rq = self.prev.rqs[cpu]
+ new_rq = old_rq.sched_switch(prev, prev_state, next_pid)
+
+ if old_rq is new_rq:
+ return
+
+ self.rqs[cpu] = new_rq
+ self.__update_total_load(old_rq, new_rq)
+ ts_list.append(self)
+ self.event_cpus = [cpu]
+
+ def migrate(self, ts_list, new: int, old_cpu: int, new_cpu: int):
+ """Process task migration in time slice."""
+ if old_cpu == new_cpu:
+ return
+ old_rq = self.prev.rqs[old_cpu]
+ out_rq = old_rq.migrate_out(new)
+ self.rqs[old_cpu] = out_rq
+ self.__update_total_load(old_rq, out_rq)
+
+ new_rq = self.prev.rqs[new_cpu]
+ in_rq = new_rq.migrate_in(new)
+ self.rqs[new_cpu] = in_rq
+ self.__update_total_load(new_rq, in_rq)
+
+ ts_list.append(self)
+
+ if old_rq is not out_rq:
+ self.event_cpus.append(old_cpu)
+ self.event_cpus.append(new_cpu)
+
+ def wake_up(self, ts_list, pid: int, cpu: int, fork: bool):
+ """Process wakeup in time slice."""
+ old_rq = self.prev.rqs[cpu]
+ if fork:
+ new_rq = old_rq.wake_up_new(pid)
+ else:
+ new_rq = old_rq.wake_up(pid)
+
+ if new_rq is old_rq:
+ return
+ self.rqs[cpu] = new_rq
+ self.__update_total_load(old_rq, new_rq)
+ ts_list.append(self)
+ self.event_cpus = [cpu]
+
+ def next(self, t: int):
+ """Create next time slice."""
+ self.end = t
+ return TimeSlice(t, self)
+
+
+class TimeSliceList(UserList):
+ """List of time slices with search capabilities."""
+
+ def __init__(self, arg=None):
+ super().__init__(arg if arg is not None else [])
+ self.root_win = None
+
+ def get_time_slice(self, ts: int) -> TimeSlice:
+ """Get or create time slice for timestamp."""
+ if len(self.data) == 0:
+ ts_slice = TimeSlice(ts, TimeSlice(-1, None))
+ else:
+ ts_slice = self.data[-1].next(ts)
+ return ts_slice
+
+ def find_time_slice(self, ts: int) -> int:
+ """Binary search for time slice containing timestamp."""
+ if not self.data:
+ return -1
+ start = 0
+ end = len(self.data)
+ found = -1
+ searching = True
+ while searching:
+ if start in (end, end - 1):
+ searching = False
+
+ i = (end + start) // 2
+ if self.data[i].start <= ts <= self.data[i].end:
+ found = i
+ break
+
+ if self.data[i].end < ts:
+ start = i
+ elif self.data[i].start > ts:
+ end = i
+
+ return found
+
+ def set_root_win(self, win):
+ """Set root window for GUI."""
+ self.root_win = win
+
+ def mouse_down(self, cpu: int, t: int):
+ """Handle mouse down event from GUI."""
+ idx = self.find_time_slice(t)
+ if idx == -1:
+ return
+
+ ts = self[idx]
+ rq = ts.rqs[cpu]
+ raw = f"CPU: {cpu}\n"
+ raw += f"Last event : {repr(rq.event)}\n"
+ raw += f"Timestamp : {ts.start // (10 ** 9)}.{ts.start % (10 ** 9) // 1000:06d}\n"
+ raw += f"Duration : {(ts.end - ts.start) // (10 ** 3):6d} us\n"
+ raw += f"Load = {rq.load()}\n"
+ for task in rq.tasks:
+ raw += f"{thread_name(task)} \n"
+
+ if self.root_win:
+ self.root_win.update_summary(raw)
+
+ def update_rectangle_cpu(self, slice_obj: TimeSlice, cpu: int):
+ """Update rectangle for CPU in GUI."""
+ rq = slice_obj.rqs[cpu]
+
+ if slice_obj.total_load != 0:
+ load_rate = rq.load() / float(slice_obj.total_load)
+ else:
+ load_rate = 0
+
+ red_power = int(0xff - (0xff * load_rate))
+ color = (0xff, red_power, red_power)
+
+ top_color = None
+ if cpu in slice_obj.event_cpus:
+ top_color = rq.event.color()
+
+ if self.root_win:
+ self.root_win.paint_rectangle_zone(cpu, color, top_color,
+ slice_obj.start, slice_obj.end)
+
+ def fill_zone(self, start: int, end: int):
+ """Fill zone in GUI."""
+ i = self.find_time_slice(start)
+ if i == -1:
+ return
+
+ for idx in range(i, len(self.data)):
+ timeslice = self.data[idx]
+ if timeslice.start > end:
+ return
+
+ for cpu in timeslice.rqs:
+ self.update_rectangle_cpu(timeslice, cpu)
+
+ def interval(self) -> tuple[int, int]:
+ """Return start and end timestamps."""
+ if len(self.data) == 0:
+ return 0, 0
+ return self.data[0].start, self.data[-1].end
+
+ def nr_rectangles(self) -> int:
+ """Return maximum CPU number."""
+ if not self.data:
+ return 0
+ last_ts = self.data[-1]
+ max_cpu = 0
+ for cpu in last_ts.rqs:
+ max_cpu = max(max_cpu, cpu)
+ return max_cpu
+
+
+class SchedMigrationAnalyzer:
+ """Analyzes task migrations and manages time slices."""
+
+ def __init__(self):
+ self.current_tsk = defaultdict(lambda: -1)
+ self.timeslices = TimeSliceList()
+
+ def sched_switch(self, time: int, cpu: int, prev_comm: str, prev_pid: int, prev_state: int,
+ next_comm: str, next_pid: int):
+ """Handle sched_switch event."""
+ on_cpu_task = self.current_tsk[cpu]
+
+ if on_cpu_task not in (-1, prev_pid):
+ print(f"Sched switch event rejected ts: {time} cpu: {cpu} "
+ f"prev: {prev_comm}({prev_pid}) next: {next_comm}({next_pid})")
+ threads[prev_pid] = prev_comm
+ threads[next_pid] = next_comm
+ self.current_tsk[cpu] = next_pid
+ return
+
+ threads[prev_pid] = prev_comm
+ threads[next_pid] = next_comm
+ self.current_tsk[cpu] = next_pid
+
+ ts = self.timeslices.get_time_slice(time)
+ ts.sched_switch(self.timeslices, prev_pid, prev_state, next_pid, cpu)
+
+ def migrate(self, time: int, pid: int, orig_cpu: int, dest_cpu: int):
+ """Handle sched_migrate_task event."""
+ ts = self.timeslices.get_time_slice(time)
+ ts.migrate(self.timeslices, pid, orig_cpu, dest_cpu)
+
+ def wake_up(self, time: int, pid: int, success: int, target_cpu: int, fork: bool):
+ """Handle wakeup event."""
+ if success == 0:
+ return
+ ts = self.timeslices.get_time_slice(time)
+ ts.wake_up(self.timeslices, pid, target_cpu, fork)
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Collect events and pass to analyzer."""
+ name = str(sample.evsel)
+ time = sample.sample_time
+ cpu = sample.sample_cpu
+ _pid = sample.sample_pid
+ _comm = "Unknown"
+
+ if name == "evsel(sched:sched_switch)":
+ prev_comm = getattr(sample, "prev_comm", "Unknown")
+ prev_pid = getattr(sample, "prev_pid", -1)
+ prev_state = getattr(sample, "prev_state", 0)
+ next_comm = getattr(sample, "next_comm", "Unknown")
+ next_pid = getattr(sample, "next_pid", -1)
+ self.sched_switch(time, cpu, prev_comm, prev_pid, prev_state, next_comm, next_pid)
+ elif name == "evsel(sched:sched_migrate_task)":
+ task_pid = getattr(sample, "pid", -1)
+ orig_cpu = getattr(sample, "orig_cpu", -1)
+ dest_cpu = getattr(sample, "dest_cpu", -1)
+ self.migrate(time, task_pid, orig_cpu, dest_cpu)
+ elif name == "evsel(sched:sched_wakeup)":
+ task_pid = getattr(sample, "pid", -1)
+ success = getattr(sample, "success", 1)
+ target_cpu = getattr(sample, "target_cpu", -1)
+ self.wake_up(time, task_pid, success, target_cpu, False)
+ elif name == "evsel(sched:sched_wakeup_new)":
+ task_pid = getattr(sample, "pid", -1)
+ success = getattr(sample, "success", 1)
+ target_cpu = getattr(sample, "target_cpu", -1)
+ self.wake_up(time, task_pid, success, target_cpu, True)
+
+ def run_gui(self):
+ """Start wxPython GUI."""
+ try:
+ wx_mod: Any = importlib.import_module("wx")
+ sched_gui: Any = importlib.import_module("SchedGui")
+ except ImportError:
+ print("wxPython is not available. Cannot start GUI.")
+ return
+ app = wx_mod.App(False)
+ _frame = sched_gui.RootFrame(self.timeslices, "Migration")
+ app.MainLoop()
+
+
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser(description="Cpu task migration overview toy")
+ ap.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ ap.add_argument("-v", "--verbose", action="store_true",
+ help="Print parsed migration summary")
+ ap.add_argument("--gui", action="store_true", default=True,
+ help="Start the GUI (default)")
+ ap.add_argument("--no-gui", action="store_true", help="Do not start the GUI")
+ args = ap.parse_args()
+
+ analyzer = SchedMigrationAnalyzer()
+
+ try:
+ session = perf.session(perf.data(args.input), sample=analyzer.process_event)
+ session.process_events()
+ if args.verbose:
+ start_ts, end_ts = analyzer.timeslices.interval()
+ print(f"Timeslices: {len(analyzer.timeslices.data)} "
+ f"(interval: {start_ts}..{end_ts})")
+ if args.gui and not args.no_gui:
+ analyzer.run_gui()
+ except KeyboardInterrupt:
+ pass
+ except OSError as e:
+ print(f"Error processing events: {e}")
+ sys.exit(1)
diff --git a/tools/perf/tests/shell/test_sched_migration_python.sh b/tools/perf/tests/shell/test_sched_migration_python.sh
new file mode 100755
index 000000000000..e5d0be5d626a
--- /dev/null
+++ b/tools/perf/tests/shell/test_sched_migration_python.sh
@@ -0,0 +1,69 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# sched-migration python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/sched-migration.py"
+
+if ! perf check feature -q libtraceevent > /dev/null 2>&1; then
+ echo "Skipping test, libtraceevent is disabled"
+ exit 2
+fi
+
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, sched-migration.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+temp_out=""
+
+cleanup() {
+ rm -f "${temp_data}" "${temp_out}"
+}
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+echo "Testing sched-migration.py..."
+
+# Create a perf.data file. Force dropping a packet if tracepoint is available!
+ev="sched:sched_switch,sched:sched_migrate_task"
+ev="${ev},sched:sched_wakeup_new,sched:sched_wakeup"
+if ! perf record -e "$ev" -a -o "${temp_data}" \
+ -- sleep 0.1 >/dev/null 2>&1; then
+ perf record -e cycles -o "${temp_data}" \
+ -- perf test -w noploop >/dev/null 2>&1 || \
+ { echo "Skipping test, perf record failed"; exit 2; }
+fi
+
+
+if [ ! -s "${temp_data}" ]; then
+ echo "Skipping test, perf record failed to create data"
+ exit 2
+fi
+
+# Check that the script executes
+if ! "$PYTHON" "$script_path" -v --no-gui -i "${temp_data}" > "${temp_out}"; then
+ echo "sched-migration.py test failed"
+ err=1
+else
+ echo "sched-migration test passed."
+fi
+rm -f "${temp_out}"
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 33/49] perf python: Port wakeup-latency from Perl to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (31 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 32/49] perf python: Port sched-migration and SchedGui " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 34/49] perf python: Port compaction-times " Ian Rogers
` (15 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Replace the legacy Perl script wakeup-latency.pl with a standalone
Python script in tools/perf/python/wakeup-latency.py using the perf
Python module.
Improvements compared to the legacy Perl script:
- Remove the dependency on libperl and Perf::Trace::Util.
- Guard print_totals() when total_wakeups == 0 (printing 'N/A' instead
of dividing by zero when a trace contains no matched wakeup/switch
pairs).
- Add argparse CLI options (-i/--input) and full type annotations.
Add a shell test
(test_wakeup_latency_python.sh) to verify the standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/wakeup-latency.py | 94 +++++++++++++++++++
.../tests/shell/test_wakeup_latency_python.sh | 83 ++++++++++++++++
2 files changed, 177 insertions(+)
create mode 100755 tools/perf/python/wakeup-latency.py
create mode 100755 tools/perf/tests/shell/test_wakeup_latency_python.sh
diff --git a/tools/perf/python/wakeup-latency.py b/tools/perf/python/wakeup-latency.py
new file mode 100755
index 000000000000..6113fa92b501
--- /dev/null
+++ b/tools/perf/python/wakeup-latency.py
@@ -0,0 +1,94 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0-only
+"""Display avg/min/max wakeup latency."""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+import sys
+from typing import Optional, Dict
+import perf
+
+class WakeupLatency:
+ """Tracks and displays wakeup latency statistics."""
+ def __init__(self) -> None:
+ self.last_wakeup: Dict[int, int] = defaultdict(int)
+ self.max_wakeup_latency: int = 0
+ self.min_wakeup_latency: Optional[int] = None
+ self.total_wakeup_latency = 0
+ self.total_wakeups = 0
+ self.unhandled: Dict[str, int] = defaultdict(int)
+ self.session: Optional[perf.session] = None
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Process events."""
+ event_name = str(sample.evsel)
+ sample_time = sample.sample_time
+
+ if "sched:sched_wakeup" in event_name:
+ try:
+ pid = sample.pid
+ self.last_wakeup[pid] = sample_time
+ except AttributeError:
+ self.unhandled[event_name] += 1
+ elif "sched:sched_switch" in event_name:
+ try:
+ next_pid = sample.next_pid
+ wakeup_ts = self.last_wakeup.get(next_pid, 0)
+ if wakeup_ts:
+ latency = sample_time - wakeup_ts
+ self.max_wakeup_latency = max(self.max_wakeup_latency, latency)
+ if self.min_wakeup_latency is None:
+ self.min_wakeup_latency = latency
+ else:
+ self.min_wakeup_latency = min(self.min_wakeup_latency, latency)
+ self.total_wakeup_latency += latency
+ self.total_wakeups += 1
+ del self.last_wakeup[next_pid]
+ except AttributeError:
+ self.unhandled[event_name] += 1
+ else:
+ self.unhandled[event_name] += 1
+
+ def print_totals(self) -> None:
+ """Print summary statistics."""
+ print("wakeup_latency stats:\n")
+ print(f"total_wakeups: {self.total_wakeups}")
+ if self.total_wakeups:
+ avg = self.total_wakeup_latency // self.total_wakeups
+ print(f"avg_wakeup_latency (ns): {avg}")
+ print(f"min_wakeup_latency (ns): {self.min_wakeup_latency}")
+ print(f"max_wakeup_latency (ns): {self.max_wakeup_latency}")
+ else:
+ print("avg_wakeup_latency (ns): N/A")
+ print("min_wakeup_latency (ns): N/A")
+ print("max_wakeup_latency (ns): N/A")
+
+ if self.unhandled:
+ print("\nunhandled events:\n")
+ print(f"{'event':<40s} {'count':>10s}")
+ print(f"{'-'*40} {'-'*10}")
+ for event_name, count in self.unhandled.items():
+ print(f"{event_name:<40s} {count:10d}")
+
+ def run(self, input_file: str) -> None:
+ """Run the session."""
+ self.session = perf.session(perf.data(input_file), sample=self.process_event)
+ self.session.process_events()
+ self.print_totals()
+
+def main() -> None:
+ """Main function."""
+ parser = argparse.ArgumentParser(description="Trace wakeup latency")
+ parser.add_argument("-i", "--input", default="perf.data", help="Input file")
+ args = parser.parse_args()
+
+ analyzer = WakeupLatency()
+ try:
+ analyzer.run(args.input)
+ except IOError as e:
+ print(e, file=sys.stderr)
+ sys.exit(1)
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/perf/tests/shell/test_wakeup_latency_python.sh b/tools/perf/tests/shell/test_wakeup_latency_python.sh
new file mode 100755
index 000000000000..d68c9b31d6ed
--- /dev/null
+++ b/tools/perf/tests/shell/test_wakeup_latency_python.sh
@@ -0,0 +1,83 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# wakeup-latency python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+if ! perf check feature -q libtraceevent; then
+ echo "Skipping test, libtraceevent is disabled"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/wakeup-latency.py"
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, wakeup-latency.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+temp_out=""
+
+cleanup() {
+ rm -f "${temp_data}" "${temp_out}"
+}
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+echo "Testing wakeup-latency.py..."
+
+# Create a perf.data file. Try to get tracepoint data.
+if perf list | grep -q "sched:sched_wakeup"; then
+ ev="sched:sched_wakeup,sched:sched_wakeup_new,sched:sched_switch"
+ perf record -e "$ev" -a -o "${temp_data}" \
+ -- sleep 0.1 >/dev/null 2>&1 || \
+ { echo "Skipping test, perf record failed"; exit 2; }
+else
+ echo "Skipping test, no sched:sched_wakeup event"
+ exit 2
+fi
+
+
+if [ ! -s "${temp_data}" ]; then
+ echo "Skipping test, perf record failed to create data"
+ exit 2
+fi
+
+# Check that the script executes
+if ! "$PYTHON" "$script_path" -i "${temp_data}" > "${temp_out}"; then
+ echo "wakeup-latency.py test failed"
+ err=1
+else
+ if ! grep -E -q "avg_wakeup_latency.*[0-9]+" "${temp_out}"; then
+ echo "Failed to find metric data rows"
+ err=1
+ else
+ echo "wakeup-latency test passed."
+ fi
+fi
+
+# Also test zero-wakeups / unhandled events path to verify division-by-zero protection
+if perf record -e cycles -o "${temp_data}" -- perf test -w noploop >/dev/null 2>&1; then
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" > "${temp_out}" || \
+ ! grep -q "avg_wakeup_latency (ns): N/A" "${temp_out}"; then
+ echo "wakeup-latency zero-wakeups guard test failed"
+ err=1
+ fi
+fi
+rm -f "${temp_out}"
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 34/49] perf python: Port compaction-times to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (32 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 33/49] perf python: Port wakeup-latency from Perl " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 35/49] perf python: Port net_dropmonitor " Ian Rogers
` (14 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port compaction-times.py to a standalone script in tools/perf/python/
using the perf module directly to analyze mm_compaction tracepoints.
Improvements compared to the legacy script:
- Replace Python 2 constructs (such as sys.maxint and raw integer
bitmasks) with Python 3 enum.IntEnum (Popt) and enum.IntFlag (Topt)
types.
- Access tracepoint fields directly on perf.sample_event and add
-i/--input CLI option support via argparse.
- Add full type annotations passing mypy and pylint without suppression
comments.
Add a shell test (test_compaction_times_python.sh) to verify the
standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/compaction-times.py | 350 ++++++++++++++++++
.../shell/test_compaction_times_python.sh | 81 ++++
2 files changed, 431 insertions(+)
create mode 100755 tools/perf/python/compaction-times.py
create mode 100755 tools/perf/tests/shell/test_compaction_times_python.sh
diff --git a/tools/perf/python/compaction-times.py b/tools/perf/python/compaction-times.py
new file mode 100755
index 000000000000..d17cd51a320b
--- /dev/null
+++ b/tools/perf/python/compaction-times.py
@@ -0,0 +1,350 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+Report time spent in memory compaction.
+
+Memory compaction is a feature in the Linux kernel that defragments memory
+by moving used pages to create larger contiguous blocks of free memory. This
+is particularly useful for allocating huge pages.
+
+This script processes trace events related to memory compaction and reports:
+- Total time spent in compaction (stall time).
+- Statistics for page migration (moved vs. failed).
+- Statistics for the free scanner (scanned vs. isolated pages).
+- Statistics for the migration scanner (scanned vs. isolated pages).
+
+Definitions:
+- **Compaction**: Defragmenting memory by moving allocated pages.
+- **Migration**: Moving pages from their current location to free pages found by the free scanner.
+- **Free Scanner**: Scans memory (typically from the end of a zone) to find free pages.
+- **Migration Scanner**: Scans memory (typically from the beginning of a zone)
+ to find pages to move.
+- **Isolated Pages**: Pages that have been temporarily removed from the buddy
+ system for migration or as migration targets.
+
+Ported from tools/perf/scripts/python/compaction-times.py to the modern perf Python module.
+"""
+from __future__ import annotations
+
+import argparse
+import enum
+import re
+import sys
+from typing import Callable, Dict, List, Optional, Any
+import perf
+
+class Popt(enum.IntEnum):
+ """Process display options."""
+ DISP_DFL = 0
+ DISP_PROC = 1
+ DISP_PROC_VERBOSE = 2
+
+class Topt(enum.IntFlag):
+ """Trace display options."""
+ DISP_TIME = 0
+ DISP_MIG = 1
+ DISP_ISOLFREE = 2
+ DISP_ISOLMIG = 4
+ DISP_ALL = DISP_MIG | DISP_ISOLFREE | DISP_ISOLMIG
+
+# Globals to satisfy pylint when accessed in functions before assignment in main.
+OPT_NS = True
+opt_disp = Topt.DISP_ALL
+opt_proc = Popt.DISP_DFL
+session = None
+
+def get_comm_filter(regex: re.Pattern) -> Callable[[int, str], bool]:
+ """Returns a filter function based on command regex."""
+ def filter_func(_pid: int, comm: str) -> bool:
+ regex_match = regex.search(comm)
+ return regex_match is None or regex_match.group() == ""
+ return filter_func
+
+def get_pid_filter(low_str: str, high_str: str) -> Callable[[int, str], bool]:
+ """Returns a filter function based on PID range."""
+ low = 0 if low_str == "" else int(low_str)
+ high = None if high_str == "" else int(high_str)
+
+ def filter_func(pid: int, _comm: str) -> bool:
+ return not (pid >= low and (high is None or pid <= high))
+ return filter_func
+
+def ns_to_time(ns: int) -> str:
+ """Format nanoseconds to string based on options."""
+ return f"{ns}ns" if OPT_NS else f"{round(ns, -3) // 1000}us"
+
+class Pair:
+ """Represents a pair of related counters (e.g., scanned vs isolated, moved vs failed)."""
+ def __init__(self, aval: int, bval: int,
+ alabel: Optional[str] = None, blabel: Optional[str] = None):
+ self.alabel = alabel
+ self.blabel = blabel
+ self.aval = aval
+ self.bval = bval
+
+ def __add__(self, rhs: 'Pair') -> 'Pair':
+ return Pair(self.aval + rhs.aval, self.bval + rhs.bval, self.alabel, self.blabel)
+
+ def __iadd__(self, rhs: 'Pair') -> 'Pair':
+ self.aval += rhs.aval
+ self.bval += rhs.bval
+ return self
+
+ def __str__(self) -> str:
+ return f"{self.alabel}={self.aval} {self.blabel}={self.bval}"
+
+class Cnode:
+ """Holds statistics for a single compaction event or an aggregated set of events."""
+ def __init__(self, ns: int):
+ self.ns = ns
+ self.migrated = Pair(0, 0, "moved", "failed")
+ self.fscan = Pair(0, 0, "scanned", "isolated")
+ self.mscan = Pair(0, 0, "scanned", "isolated")
+
+ def __add__(self, rhs: 'Cnode') -> 'Cnode':
+ res = Cnode(self.ns + rhs.ns)
+ res.migrated = self.migrated + rhs.migrated
+ res.fscan = self.fscan + rhs.fscan
+ res.mscan = self.mscan + rhs.mscan
+ return res
+
+ def __iadd__(self, rhs: 'Cnode') -> 'Cnode':
+ self.ns += rhs.ns
+ self.migrated += rhs.migrated
+ self.fscan += rhs.fscan
+ self.mscan += rhs.mscan
+ return self
+
+ def __str__(self) -> str:
+ prev = False
+ s = f"{ns_to_time(self.ns)} "
+ if opt_disp & Topt.DISP_MIG:
+ s += f"migration: {self.migrated}"
+ prev = True
+ if opt_disp & Topt.DISP_ISOLFREE:
+ s += f"{' ' if prev else ''}free_scanner: {self.fscan}"
+ prev = True
+ if opt_disp & Topt.DISP_ISOLMIG:
+ s += f"{' ' if prev else ''}migration_scanner: {self.mscan}"
+ return s
+
+ def complete(self, secs: int, nsecs: int) -> None:
+ """Complete the node with duration."""
+ self.ns = (secs * 1000000000 + nsecs) - self.ns
+
+ def increment(self, migrated: Optional[Pair], fscan: Optional[Pair],
+ mscan: Optional[Pair]) -> None:
+ """Increment statistics."""
+ if migrated is not None:
+ self.migrated += migrated
+ if fscan is not None:
+ self.fscan += fscan
+ if mscan is not None:
+ self.mscan += mscan
+
+class Chead:
+ """Aggregates compaction statistics per process (PID) and maintains total statistics."""
+ heads: Dict[int, 'Chead'] = {}
+ val = Cnode(0)
+ fobj: Optional[Any] = None
+
+ @classmethod
+ def add_filter(cls, fobj: Any) -> None:
+ """Add a filter object."""
+ cls.fobj = fobj
+
+ @classmethod
+ def create_pending(cls, pid: int, comm: str, start_secs: int, start_nsecs: int) -> None:
+ """Create a pending node for a process."""
+ filtered = False
+ try:
+ head = cls.heads[pid]
+ filtered = head.is_filtered()
+ except KeyError:
+ if cls.fobj is not None:
+ filtered = cls.fobj(pid, comm)
+ head = cls.heads[pid] = Chead(comm, pid, filtered)
+
+ if not filtered:
+ head.mark_pending(start_secs, start_nsecs)
+
+ @classmethod
+ def increment_pending(cls, pid: int, migrated: Optional[Pair],
+ fscan: Optional[Pair], mscan: Optional[Pair]) -> None:
+ """Increment pending stats for a process."""
+ if pid not in cls.heads:
+ return
+ head = cls.heads[pid]
+ if not head.is_filtered():
+ if head.is_pending():
+ head.do_increment(migrated, fscan, mscan)
+ else:
+ sys.stderr.write(f"missing start compaction event for pid {pid}\n")
+
+ @classmethod
+ def complete_pending(cls, pid: int, secs: int, nsecs: int) -> None:
+ """Complete pending stats for a process."""
+ if pid not in cls.heads:
+ return
+ head = cls.heads[pid]
+ if not head.is_filtered():
+ if head.is_pending():
+ head.make_complete(secs, nsecs)
+ else:
+ sys.stderr.write(f"missing start compaction event for pid {pid}\n")
+
+ @classmethod
+ def gen(cls):
+ """Generate heads for display."""
+ if opt_proc != Popt.DISP_DFL:
+ yield from cls.heads.values()
+
+ @classmethod
+ def get_total(cls) -> Cnode:
+ """Get total statistics."""
+ return cls.val
+
+ def __init__(self, comm: str, pid: int, filtered: bool):
+ self.comm = comm
+ self.pid = pid
+ self.val = Cnode(0)
+ self.pending: Optional[Cnode] = None
+ self.filtered = filtered
+ self.list: List[Cnode] = []
+
+ def mark_pending(self, secs: int, nsecs: int) -> None:
+ """Mark node as pending."""
+ self.pending = Cnode(secs * 1000000000 + nsecs)
+
+ def do_increment(self, migrated: Optional[Pair], fscan: Optional[Pair],
+ mscan: Optional[Pair]) -> None:
+ """Increment pending stats."""
+ if self.pending is not None:
+ self.pending.increment(migrated, fscan, mscan)
+
+ def make_complete(self, secs: int, nsecs: int) -> None:
+ """Make pending stats complete."""
+ if self.pending is not None:
+ self.pending.complete(secs, nsecs)
+ Chead.val += self.pending
+
+ if opt_proc != Popt.DISP_DFL:
+ self.val += self.pending
+
+ if opt_proc == Popt.DISP_PROC_VERBOSE:
+ self.list.append(self.pending)
+ self.pending = None
+
+ def enumerate(self) -> None:
+ """Enumerate verbose stats."""
+ if opt_proc == Popt.DISP_PROC_VERBOSE and not self.is_filtered():
+ for i, pelem in enumerate(self.list):
+ sys.stdout.write(f"{self.pid}[{self.comm}].{i+1}: {pelem}\n")
+
+ def is_pending(self) -> bool:
+ """Check if node is pending."""
+ return self.pending is not None
+
+ def is_filtered(self) -> bool:
+ """Check if node is filtered."""
+ return self.filtered
+
+ def display(self) -> None:
+ """Display stats."""
+ if not self.is_filtered():
+ sys.stdout.write(f"{self.pid}[{self.comm}]: {self.val}\n")
+
+def trace_end() -> None:
+ """Called at the end of trace processing."""
+ sys.stdout.write(f"total: {Chead.get_total()}\n")
+ for i in Chead.gen():
+ i.display()
+ i.enumerate()
+
+def process_event(sample: perf.sample_event) -> None:
+ """Callback for processing events."""
+ event_name = str(sample.evsel)
+ pid = sample.sample_tid
+ comm = "[unknown]"
+ try:
+ if session:
+ thread = session.find_thread(pid)
+ if thread:
+ comm = thread.comm() or "[unknown]"
+ except (TypeError, AttributeError):
+ pass
+ secs = sample.sample_time // 1000000000
+ nsecs = sample.sample_time % 1000000000
+
+ if "evsel(compaction:mm_compaction_begin)" in event_name:
+ Chead.create_pending(pid, comm, secs, nsecs)
+ elif "evsel(compaction:mm_compaction_end)" in event_name:
+ Chead.complete_pending(pid, secs, nsecs)
+ elif "evsel(compaction:mm_compaction_migratepages)" in event_name:
+ nr_migrated = getattr(sample, "nr_migrated", 0)
+ nr_failed = getattr(sample, "nr_failed", 0)
+ Chead.increment_pending(pid, Pair(nr_migrated, nr_failed), None, None)
+ elif "evsel(compaction:mm_compaction_isolate_freepages)" in event_name:
+ nr_scanned = getattr(sample, "nr_scanned", 0)
+ nr_taken = getattr(sample, "nr_taken", 0)
+ Chead.increment_pending(pid, None, Pair(nr_scanned, nr_taken), None)
+ elif "evsel(compaction:mm_compaction_isolate_migratepages)" in event_name:
+ nr_scanned = getattr(sample, "nr_scanned", 0)
+ nr_taken = getattr(sample, "nr_taken", 0)
+ Chead.increment_pending(pid, None, None, Pair(nr_scanned, nr_taken))
+
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser(description="Report time spent in compaction")
+ ap.add_argument("-p", action="store_true", help="display by process")
+ ap.add_argument("-pv", action="store_true", help="display by process (verbose)")
+ ap.add_argument("-u", action="store_true", help="display results in microseconds")
+ ap.add_argument("-t", action="store_true", help="display stall times only")
+ ap.add_argument("-m", action="store_true", help="display stats for migration")
+ ap.add_argument("-fs", action="store_true", help="display stats for free scanner")
+ ap.add_argument("-ms", action="store_true", help="display stats for migration scanner")
+ ap.add_argument("filter", nargs="?", help="pid|pid-range|comm-regex")
+ ap.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ args = ap.parse_args()
+
+ opt_proc = Popt.DISP_DFL
+ if args.pv:
+ opt_proc = Popt.DISP_PROC_VERBOSE
+ elif args.p:
+ opt_proc = Popt.DISP_PROC
+
+ OPT_NS = not args.u
+
+ opt_disp = Topt.DISP_ALL
+ if args.t or args.m or args.fs or args.ms:
+ opt_disp = Topt(0)
+ if args.t:
+ opt_disp |= Topt.DISP_TIME
+ if args.m:
+ opt_disp |= Topt.DISP_MIG
+ if args.fs:
+ opt_disp |= Topt.DISP_ISOLFREE
+ if args.ms:
+ opt_disp |= Topt.DISP_ISOLMIG
+
+ if args.filter:
+ PID_PATTERN = r"^(\d*)-(\d*)$|^(\d*)$"
+ pid_re = re.compile(PID_PATTERN)
+ match = pid_re.search(args.filter)
+ filter_obj: Any = None
+ if match is not None and match.group() != "":
+ if match.group(3) is not None:
+ filter_obj = get_pid_filter(match.group(3), match.group(3))
+ else:
+ filter_obj = get_pid_filter(match.group(1), match.group(2))
+ else:
+ try:
+ comm_re = re.compile(args.filter)
+ except re.error:
+ sys.stderr.write(f"invalid regex '{args.filter}'\n")
+ sys.exit(1)
+ filter_obj = get_comm_filter(comm_re)
+ Chead.add_filter(filter_obj)
+
+ session = perf.session(perf.data(args.input), sample=process_event)
+ session.process_events()
+ trace_end()
diff --git a/tools/perf/tests/shell/test_compaction_times_python.sh b/tools/perf/tests/shell/test_compaction_times_python.sh
new file mode 100755
index 000000000000..80df5adc5bf6
--- /dev/null
+++ b/tools/perf/tests/shell/test_compaction_times_python.sh
@@ -0,0 +1,81 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# compaction-times python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+# If we don't have the perf python module, we can't test
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+if ! perf check feature -q libtraceevent; then
+ echo "Skipping test, perf built without libtraceevent"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/compaction-times.py"
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, compaction-times.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+
+cleanup() {
+ rm -f "${temp_data}"
+}
+
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+
+test_file_mode() {
+ echo "Testing compaction-times.py..."
+
+ # Check for any compaction events to see if kernel supports it
+ if ! perf list | grep -q "compaction:mm_compaction_begin"; then
+ echo "Skipping test, compaction tracepoints not found"
+ exit 2
+ fi
+
+ # Generate some events
+ # We might not naturally trigger compaction in 0.5s sleep, but the script
+ # should parse the empty or sparse file correctly without crashing.
+ if ! perf record -e "compaction:*" -a -o "${temp_data}" -- sleep 0.5 >/dev/null 2>&1; then
+ echo "Skipping test, perf record failed"
+ exit 2
+ fi
+
+ # Run the script with some filters to validate filtering logic
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" >/dev/null; then
+ echo "File mode default test failed."
+ err=1
+ fi
+
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" "0-0" >/dev/null; then
+ echo "File mode strict PID filter test failed."
+ err=1
+ fi
+
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" "sleep" >/dev/null; then
+ echo "File mode comm filter test failed."
+ err=1
+ fi
+
+ if [ $err -eq 0 ]; then
+ echo "File mode test passed."
+ fi
+}
+
+test_file_mode
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 35/49] perf python: Port net_dropmonitor to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (33 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 34/49] perf python: Port compaction-times " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 36/49] perf python: Port netdev-times " Ian Rogers
` (13 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port net_dropmonitor.py from tools/perf/scripts/python/ to a standalone
script in tools/perf/python/:
- Refactor the script into a DropMonitor class with full type
annotations to encapsulate state.
- Use perf.session for skb:kfree_skb event processing and add argparse
CLI support (-i/--input and -k/--kallsyms).
- Resolve kernel drop addresses via perf.session symbols/callchains and
binary search over /proc/kallsyms, ignoring zeroed kptr_restrict
addresses with graceful fallback when kallsyms is unavailable.
- Remove Python 2 compatibility code.
Add a shell test (test_net_dropmonitor_python.sh) to verify the
standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/net_dropmonitor.py | 156 ++++++++++++++++++
.../shell/test_net_dropmonitor_python.sh | 103 ++++++++++++
2 files changed, 259 insertions(+)
create mode 100755 tools/perf/python/net_dropmonitor.py
create mode 100755 tools/perf/tests/shell/test_net_dropmonitor_python.sh
diff --git a/tools/perf/python/net_dropmonitor.py b/tools/perf/python/net_dropmonitor.py
new file mode 100755
index 000000000000..3e0ef656e5dc
--- /dev/null
+++ b/tools/perf/python/net_dropmonitor.py
@@ -0,0 +1,156 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+Monitor the system for dropped packets and produce a report of drop locations and counts.
+Ported from tools/perf/scripts/python/net_dropmonitor.py
+"""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+import os
+import sys
+from typing import Tuple
+import perf
+
+
+class DropMonitor:
+ """Monitors dropped packets and aggregates counts by location."""
+
+ def __init__(self, kallsyms_path: str | None = None) -> None:
+ self.drop_log: dict[int, int] = defaultdict(int)
+ self.kallsyms: list[Tuple[int, str]] = []
+ self.resolved_syms: dict[int, Tuple[str, int]] = {}
+ self.callchain_syms: dict[int, str] = {}
+ self.kallsyms_path = (
+ kallsyms_path
+ or os.environ.get("PERF_SYMBOL_KALLSYMS")
+ or "/proc/kallsyms"
+ )
+
+ def _parse_kallsyms(self) -> None:
+ """Parse the kallsyms file and map kernel addresses to function symbols."""
+ try:
+ with open(self.kallsyms_path, "r", encoding="utf-8") as f:
+ for line in f:
+ parts = line.split()
+ if len(parts) >= 3 and parts[1] in ('t', 'T', 'w', 'W'):
+ addr = int(parts[0], 16)
+ if addr > 0:
+ self.kallsyms.append((addr, parts[2]))
+ self.kallsyms.sort(key=lambda x: x[0])
+ except (FileNotFoundError, PermissionError):
+ print(f"Failed to read {self.kallsyms_path}. Symbols will not be resolved.")
+
+ def _get_sym(self, loc: int) -> Tuple[str, int]:
+ """Resolve a memory location using session symbols or the kallsyms map."""
+ if loc in self.resolved_syms:
+ return self.resolved_syms[loc]
+ if not self.kallsyms:
+ if loc in self.callchain_syms:
+ return self.callchain_syms[loc], 0
+ return str(loc), 0
+
+ start = 0
+ end = len(self.kallsyms) - 1
+ while start < end:
+ mid = (start + end) // 2
+ if self.kallsyms[mid][0] <= loc < self.kallsyms[mid+1][0]:
+ start = mid
+ break
+ if loc < self.kallsyms[mid][0]:
+ end = mid - 1
+ else:
+ start = mid + 1
+
+ sym_addr, sym_name = self.kallsyms[start]
+ if loc >= sym_addr:
+ return sym_name, loc - sym_addr
+ if loc in self.callchain_syms:
+ return self.callchain_syms[loc], 0
+ return str(loc), 0
+
+ def print_drop_table(self) -> None:
+ """Print aggregated results."""
+ if not self.drop_log:
+ print(f"{'LOCATION':>25} {'OFFSET':>25} {'COUNT':>25}")
+ return
+
+ if len(self.resolved_syms) < len(self.drop_log):
+ print("Gathering kallsyms data")
+ self._parse_kallsyms()
+
+ print(f"{'LOCATION':>25} {'OFFSET':>25} {'COUNT':>25}")
+ sorted_keys = sorted(self.drop_log.keys())
+ for sloc in sorted_keys:
+ sym, off = self._get_sym(sloc)
+ print(f"{sym:>25} {off:>25d} {self.drop_log[sloc]:>25d}")
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Process a single sample event."""
+ if "skb:kfree_skb" not in str(sample.evsel):
+ return
+
+ location = getattr(sample, "location", None)
+ if location is not None:
+ self.drop_log[location] += 1
+ if location not in self.resolved_syms:
+ if getattr(sample, "sample_ip", 0) == location and getattr(sample, "symbol", None):
+ self.resolved_syms[location] = (
+ sample.symbol,
+ getattr(sample, "sym_offset", 0) or 0,
+ )
+ else:
+ for entry in getattr(sample, "callchain", []) or []:
+ if isinstance(entry, dict):
+ entry_ip = entry.get("ip")
+ entry_sym = entry.get("sym")
+ sym_name = (
+ entry_sym.get("name") if isinstance(entry_sym, dict) else None
+ )
+ sym_start = (
+ entry_sym.get("start")
+ if isinstance(entry_sym, dict)
+ else None
+ )
+ else:
+ entry_ip = getattr(entry, "ip", None)
+ entry_sym = getattr(entry, "sym", None)
+ sym_name = (
+ getattr(entry_sym, "name", None)
+ or getattr(entry, "symbol", None)
+ )
+ sym_start = getattr(entry_sym, "start", None)
+ if entry_ip == location and sym_name:
+ if sym_start is not None:
+ self.resolved_syms[location] = (
+ sym_name,
+ max(0, location - sym_start),
+ )
+ else:
+ self.callchain_syms[location] = sym_name
+ break
+
+
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser(
+ description="Monitor the system for dropped packets and produce a "
+ "report of drop locations and counts.")
+ ap.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ ap.add_argument("-k", "--kallsyms", default=None,
+ help="Path to kallsyms file for offline symbol resolution")
+ args = ap.parse_args()
+
+ monitor = DropMonitor(kallsyms_path=args.kallsyms)
+
+ try:
+ session = perf.session(perf.data(args.input), sample=monitor.process_event,
+ kallsyms=args.kallsyms)
+ session.process_events()
+ except KeyboardInterrupt:
+ print("\nStopping trace...")
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError) as e:
+ print(f"Error processing events: {e}")
+ sys.exit(1)
+
+ monitor.print_drop_table()
diff --git a/tools/perf/tests/shell/test_net_dropmonitor_python.sh b/tools/perf/tests/shell/test_net_dropmonitor_python.sh
new file mode 100755
index 000000000000..41497f9837cf
--- /dev/null
+++ b/tools/perf/tests/shell/test_net_dropmonitor_python.sh
@@ -0,0 +1,103 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# net_dropmonitor python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/net_dropmonitor.py"
+
+if ! perf check feature -q libtraceevent > /dev/null 2>&1; then
+ echo "Skipping test, libtraceevent is disabled"
+ exit 2
+fi
+
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, net_dropmonitor.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+temp_out=""
+
+cleanup() {
+ rm -f "${temp_data}" "${temp_out}"
+}
+trap 'cleanup' EXIT
+trap 'cleanup; exit 1' TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+echo "Testing net_dropmonitor.py..."
+
+# Create a perf.data file. Force dropping a packet if tracepoint is available!
+if ! perf record -e skb:kfree_skb -o "${temp_data}" -a \
+ -- ping -c 1 255.255.255.255 >/dev/null 2>&1; then
+ if ! perf record -e skb:kfree_skb -o "${temp_data}" \
+ -- sleep 0.1 >/dev/null 2>&1; then
+ if ! perf record -o "${temp_data}" -- uname >/dev/null 2>&1; then
+ echo "Skipping test, cannot record perf events"
+ exit 2
+ fi
+ fi
+fi
+
+if [ ! -s "${temp_data}" ]; then
+ echo "Skipping test, perf record failed to create data"
+ exit 2
+fi
+
+# Check that the script executes and outputs table header
+if ! "$PYTHON" "$script_path" -i "${temp_data}" > "${temp_out}"; then
+ echo "net_dropmonitor.py test failed"
+ err=1
+else
+ if ! grep -q "LOCATION.*OFFSET.*COUNT" "${temp_out}"; then
+ echo "Failed to find the metrics table header"
+ err=1
+ fi
+fi
+
+# Verify DropMonitor event processing and symbol resolution
+if [ $err -eq 0 ]; then
+ if ! "$PYTHON" -c "
+import sys
+sys.path.insert(0, '${script_dir}')
+import net_dropmonitor
+
+class DummySample:
+ evsel = 'skb:kfree_skb'
+ location = 0xffffffff81001010
+ sample_ip = 0xffffffff81001010
+ symbol = 'ip_rcv_finish'
+ sym_offset = 16
+ callchain = []
+
+dm = net_dropmonitor.DropMonitor()
+dm.process_event(DummySample())
+dm.print_drop_table()
+" > "${temp_out}"; then
+ echo "net_dropmonitor.py unit test failed"
+ err=1
+ elif ! grep -q "ip_rcv_finish.*16.*1" "${temp_out}"; then
+ echo "Failed to find expected symbol resolution in net_dropmonitor.py"
+ err=1
+ else
+ echo "net_dropmonitor test passed."
+ fi
+fi
+rm -f "${temp_out}"
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 36/49] perf python: Port netdev-times to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (34 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 35/49] perf python: Port net_dropmonitor " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 37/49] perf python: Port check-perf-trace " Ian Rogers
` (12 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port netdev-times.py from tools/perf/scripts/python/ to a standalone
script in tools/perf/python/:
- Refactor the script into a NetDevTimesAnalyzer class with full type
annotations to encapsulate state.
- Collect events via perf.session and sort them in timestamp order
before analysis so multi-CPU TX and RX packet timelines are
reconstructed deterministically.
- Replace custom argument parsing with argparse (-i/--input, --tx,
--rx, --dev, --debug), extract tracepoint fields directly from sample
attributes, and remove Python 2 compatibility artifacts.
Add a shell test (test_netdev_times_python.sh) to verify the standalone
script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/netdev-times.py | 486 ++++++++++++++++++
.../tests/shell/test_netdev_times_python.sh | 61 +++
2 files changed, 547 insertions(+)
create mode 100755 tools/perf/python/netdev-times.py
create mode 100755 tools/perf/tests/shell/test_netdev_times_python.sh
diff --git a/tools/perf/python/netdev-times.py b/tools/perf/python/netdev-times.py
new file mode 100755
index 000000000000..68f85bf890a0
--- /dev/null
+++ b/tools/perf/python/netdev-times.py
@@ -0,0 +1,486 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+Display a process of packets and processed time.
+It helps us to investigate networking or network device.
+
+Ported from tools/perf/scripts/python/netdev-times.py
+"""
+
+from __future__ import annotations
+import argparse
+from collections import defaultdict
+import sys
+from typing import Optional
+import perf
+
+# Format for displaying rx packet processing
+PF_IRQ_ENTRY = " irq_entry(+%.3fmsec irq=%d:%s)"
+PF_SOFT_ENTRY = " softirq_entry(+%.3fmsec)"
+PF_NAPI_POLL = " napi_poll_exit(+%.3fmsec %s)"
+PF_JOINT = " |"
+PF_WJOINT = " | |"
+PF_NET_RECV = " |---netif_receive_skb(+%.3fmsec skb=%x len=%d)"
+PF_NET_RX = " |---netif_rx(+%.3fmsec skb=%x)"
+PF_CPY_DGRAM = " | skb_copy_datagram_iovec(+%.3fmsec %d:%s)"
+PF_KFREE_SKB = " | kfree_skb(+%.3fmsec location=%x)"
+PF_CONS_SKB = " | consume_skb(+%.3fmsec)"
+
+
+class NetDevTimesAnalyzer:
+ """Analyzes network device events and prints charts."""
+
+ def __init__(self, cfg: argparse.Namespace):
+ self.args = cfg
+ self.session: Optional[perf.session] = None
+ self.show_tx = cfg.tx or (not cfg.tx and not cfg.rx)
+ self.show_rx = cfg.rx or (not cfg.tx and not cfg.rx)
+ self.dev = cfg.dev
+ self.debug = cfg.debug
+ self.buffer_budget = 65536
+ self.irq_dic: dict[int, list[dict]] = defaultdict(list)
+ self.net_rx_dic: dict[int, dict] = {}
+ self.receive_hunk_list: list[dict] = []
+ self.rx_skb_list: list[dict] = []
+ self.tx_queue_list: list[dict] = []
+ self.tx_xmit_list: list[dict] = []
+ self.tx_free_list: list[dict] = []
+
+ self.buffer_budget = 65536
+ self.of_count_rx_skb_list = 0
+ self.of_count_tx_queue_list = 0
+ self.of_count_tx_xmit_list = 0
+
+ def diff_msec(self, src: int, dst: int) -> float:
+ """Calculate a time interval(msec) from src(nsec) to dst(nsec)."""
+ return (dst - src) / 1000000.0
+
+ def print_transmit(self, hunk: dict) -> None:
+ """Display a process of transmitting a packet."""
+ if self.dev and hunk['dev'].find(self.dev) < 0:
+ return
+ queue_t_sec = hunk['queue_t'] // 1000000000
+ queue_t_usec = hunk['queue_t'] % 1000000000 // 1000
+ print(f"{hunk['dev']:7s} {hunk['len']:5d} "
+ f"{queue_t_sec:6d}.{queue_t_usec:06d}sec "
+ f"{self.diff_msec(hunk['queue_t'], hunk['xmit_t']):12.3f}msec "
+ f"{self.diff_msec(hunk['xmit_t'], hunk['free_t']):12.3f}msec")
+
+ def print_receive(self, hunk: dict) -> None:
+ """Display a process of received packets and interrupts."""
+ show_hunk = False
+ irq_list = hunk['irq_list']
+ if not irq_list:
+ return
+ cpu = irq_list[0]['cpu']
+ base_t = irq_list[0]['irq_ent_t']
+
+ if self.dev:
+ for irq in irq_list:
+ if irq['name'].find(self.dev) >= 0:
+ show_hunk = True
+ break
+ else:
+ show_hunk = True
+
+ if not show_hunk:
+ return
+
+ base_t_sec = base_t // 1000000000
+ base_t_usec = base_t % 1000000000 // 1000
+ print(f"{base_t_sec}.{base_t_usec:06d}sec cpu={cpu}")
+ for irq in irq_list:
+ print(PF_IRQ_ENTRY %
+ (self.diff_msec(base_t, irq['irq_ent_t']),
+ irq['irq'], irq['name']))
+ print(PF_JOINT)
+ irq_event_list = irq['event_list']
+ for irq_event in irq_event_list:
+ if irq_event['event'] == 'netif_rx':
+ print(PF_NET_RX %
+ (self.diff_msec(base_t, irq_event['time']),
+ irq_event['skbaddr']))
+ print(PF_JOINT)
+
+ print(PF_SOFT_ENTRY % self.diff_msec(base_t, hunk['sirq_ent_t']))
+ print(PF_JOINT)
+ event_list = hunk['event_list']
+ for i, event in enumerate(event_list):
+ if event['event_name'] == 'napi_poll':
+ print(PF_NAPI_POLL %
+ (self.diff_msec(base_t, event['event_t']),
+ event['dev']))
+ if i == len(event_list) - 1:
+ print("")
+ else:
+ print(PF_JOINT)
+ else:
+ print(PF_NET_RECV %
+ (self.diff_msec(base_t, event['event_t']),
+ event['skbaddr'],
+ event['len']))
+ if 'handle' in event:
+ print(PF_WJOINT)
+ if event['handle'] == "kfree_skb":
+ print(PF_KFREE_SKB %
+ (self.diff_msec(base_t, event['comm_t']),
+ event['location']))
+ elif event['handle'] == "consume_skb":
+ print(PF_CONS_SKB %
+ self.diff_msec(base_t, event['comm_t']))
+ elif event['handle'] == "skb_copy_datagram_iovec":
+ print(PF_CPY_DGRAM %
+ (self.diff_msec(base_t, event['comm_t']),
+ event['pid'], event['comm']))
+ print(PF_JOINT)
+
+ def handle_irq_handler_entry(self, event: dict) -> None:
+ """Handle irq:irq_handler_entry event."""
+ time = event['time']
+ cpu = event['cpu']
+ irq = event['irq']
+ irq_name = event['irq_name']
+ irq_record = {'irq': irq, 'name': irq_name, 'cpu': cpu,
+ 'irq_ent_t': time, 'event_list': []}
+ self.irq_dic[cpu].append(irq_record)
+
+ def handle_irq_handler_exit(self, event: dict) -> None:
+ """Handle irq:irq_handler_exit event."""
+ time = event['time']
+ cpu = event['cpu']
+ irq = event['irq']
+ if cpu not in self.irq_dic or not self.irq_dic[cpu]:
+ return
+ if irq != self.irq_dic[cpu][-1]['irq']:
+ return
+ irq_record = self.irq_dic[cpu].pop()
+ irq_record['irq_ext_t'] = time
+ # if an irq doesn't include NET_RX softirq, drop.
+ if irq_record['event_list']:
+ self.irq_dic[cpu].append(irq_record)
+
+ def handle_irq_softirq_raise(self, event: dict) -> None:
+ """Handle irq:softirq_raise event."""
+ time = event['time']
+ cpu = event['cpu']
+ if cpu not in self.irq_dic or not self.irq_dic[cpu]:
+ return
+ irq_record = self.irq_dic[cpu].pop()
+ irq_record['event_list'].append({'time': time, 'event': 'sirq_raise'})
+ self.irq_dic[cpu].append(irq_record)
+
+ def handle_irq_softirq_entry(self, event: dict) -> None:
+ """Handle irq:softirq_entry event."""
+ time = event['time']
+ cpu = event['cpu']
+ self.net_rx_dic[cpu] = {'sirq_ent_t': time, 'event_list': []}
+
+ def handle_irq_softirq_exit(self, event: dict) -> None:
+ """Handle irq:softirq_exit event."""
+ time = event['time']
+ cpu = event['cpu']
+ irq_list = None
+ event_list = None
+ sirq_ent_t = None
+
+ if cpu in self.irq_dic:
+ irq_list = self.irq_dic[cpu]
+ del self.irq_dic[cpu]
+ if cpu in self.net_rx_dic:
+ sirq_ent_t = self.net_rx_dic[cpu]['sirq_ent_t']
+ event_list = self.net_rx_dic[cpu]['event_list']
+ del self.net_rx_dic[cpu]
+ if not irq_list or not event_list or sirq_ent_t is None:
+ return
+ rec_data = {'sirq_ent_t': sirq_ent_t, 'sirq_ext_t': time,
+ 'irq_list': irq_list, 'event_list': event_list}
+ self.receive_hunk_list.append(rec_data)
+
+ def handle_napi_poll(self, event: dict) -> None:
+ """Handle napi:napi_poll event."""
+ time = event['time']
+ cpu = event['cpu']
+ dev_name = event['dev_name']
+ work = event['work']
+ budget = event['budget']
+ if cpu in self.net_rx_dic:
+ event_list = self.net_rx_dic[cpu]['event_list']
+ rec_data = {'event_name': 'napi_poll',
+ 'dev': dev_name, 'event_t': time,
+ 'work': work, 'budget': budget}
+ event_list.append(rec_data)
+
+ def handle_netif_rx(self, event: dict) -> None:
+ """Handle net:netif_rx event."""
+ time = event['time']
+ cpu = event['cpu']
+ skbaddr = event['skbaddr']
+ skblen = event['skblen']
+ dev_name = event['dev_name']
+ if cpu not in self.irq_dic or not self.irq_dic[cpu]:
+ return
+ irq_record = self.irq_dic[cpu].pop()
+ irq_record['event_list'].append({'time': time, 'event': 'netif_rx',
+ 'skbaddr': skbaddr, 'skblen': skblen,
+ 'dev_name': dev_name})
+ self.irq_dic[cpu].append(irq_record)
+
+ def handle_netif_receive_skb(self, event: dict) -> None:
+ """Handle net:netif_receive_skb event."""
+ time = event['time']
+ cpu = event['cpu']
+ skbaddr = event['skbaddr']
+ skblen = event['skblen']
+ if cpu in self.net_rx_dic:
+ rec_data = {'event_name': 'netif_receive_skb',
+ 'event_t': time, 'skbaddr': skbaddr, 'len': skblen}
+ event_list = self.net_rx_dic[cpu]['event_list']
+ event_list.append(rec_data)
+ self.rx_skb_list.insert(0, rec_data)
+ if len(self.rx_skb_list) > self.buffer_budget:
+ self.rx_skb_list.pop()
+ self.of_count_rx_skb_list += 1
+
+ def handle_net_dev_queue(self, event: dict) -> None:
+ """Handle net:net_dev_queue event."""
+ time = event['time']
+ skbaddr = event['skbaddr']
+ skblen = event['skblen']
+ dev_name = event['dev_name']
+ skb = {'dev': dev_name, 'skbaddr': skbaddr, 'len': skblen, 'queue_t': time}
+ self.tx_queue_list.insert(0, skb)
+ if len(self.tx_queue_list) > self.buffer_budget:
+ self.tx_queue_list.pop()
+ self.of_count_tx_queue_list += 1
+
+ def handle_net_dev_xmit(self, event: dict) -> None:
+ """Handle net:net_dev_xmit event."""
+ time = event['time']
+ skbaddr = event['skbaddr']
+ rc = event['rc']
+ if rc == 0: # NETDEV_TX_OK
+ for i, skb in enumerate(self.tx_queue_list):
+ if skb['skbaddr'] == skbaddr:
+ skb['xmit_t'] = time
+ self.tx_xmit_list.insert(0, skb)
+ del self.tx_queue_list[i]
+ if len(self.tx_xmit_list) > self.buffer_budget:
+ self.tx_xmit_list.pop()
+ self.of_count_tx_xmit_list += 1
+ return
+
+ def handle_kfree_skb(self, event: dict) -> None:
+ """Handle skb:kfree_skb event."""
+ time = event['time']
+ skbaddr = event['skbaddr']
+ comm = event['comm']
+ pid = event['pid']
+ location = event['location']
+ for i, skb in enumerate(self.tx_queue_list):
+ if skb['skbaddr'] == skbaddr:
+ del self.tx_queue_list[i]
+ return
+ for i, skb in enumerate(self.tx_xmit_list):
+ if skb['skbaddr'] == skbaddr:
+ skb['free_t'] = time
+ self.tx_free_list.append(skb)
+ del self.tx_xmit_list[i]
+ return
+ for i, rec_data in enumerate(self.rx_skb_list):
+ if rec_data['skbaddr'] == skbaddr:
+ rec_data.update({'handle': "kfree_skb",
+ 'comm': comm, 'pid': pid, 'comm_t': time, 'location': location})
+ del self.rx_skb_list[i]
+ return
+
+ def handle_consume_skb(self, event: dict) -> None:
+ """Handle skb:consume_skb event."""
+ time = event['time']
+ skbaddr = event['skbaddr']
+ for i, skb in enumerate(self.tx_xmit_list):
+ if skb['skbaddr'] == skbaddr:
+ skb['free_t'] = time
+ self.tx_free_list.append(skb)
+ del self.tx_xmit_list[i]
+ return
+ for i, rec_data in enumerate(self.rx_skb_list):
+ if rec_data['skbaddr'] == skbaddr:
+ rec_data.update({'handle': "consume_skb", 'comm_t': time})
+ del self.rx_skb_list[i]
+ return
+
+ def handle_skb_copy_datagram_iovec(self, event: dict) -> None:
+ """Handle skb:skb_copy_datagram_iovec event."""
+ time = event['time']
+ skbaddr = event['skbaddr']
+ comm = event['comm']
+ pid = event['pid']
+ for i, rec_data in enumerate(self.rx_skb_list):
+ if skbaddr == rec_data['skbaddr']:
+ rec_data.update({'handle': "skb_copy_datagram_iovec",
+ 'comm': comm, 'pid': pid, 'comm_t': time})
+ del self.rx_skb_list[i]
+ return
+
+
+
+ def print_summary(self) -> None:
+ """Print charts."""
+
+ # display receive hunks
+ if self.show_rx:
+ for hunk in self.receive_hunk_list:
+ self.print_receive(hunk)
+
+ # display transmit hunks
+ if self.show_tx:
+ print(" dev len Qdisc "
+ " netdevice free")
+ for hunk in self.tx_free_list:
+ self.print_transmit(hunk)
+
+ if self.debug:
+ print("debug buffer status")
+ print("----------------------------")
+ print(f"xmit Qdisc:remain:{len(self.tx_queue_list)} "
+ f"overflow:{self.of_count_tx_queue_list}")
+ print(f"xmit netdevice:remain:{len(self.tx_xmit_list)} "
+ f"overflow:{self.of_count_tx_xmit_list}")
+ print(f"receive:remain:{len(self.rx_skb_list)} "
+ f"overflow:{self.of_count_rx_skb_list}")
+
+ def handle_single_event(self, event: dict) -> None:
+ """Handle a single processed event."""
+ name = event['name']
+ if name == 'irq:softirq_exit':
+ self.handle_irq_softirq_exit(event)
+ elif name == 'irq:softirq_entry':
+ self.handle_irq_softirq_entry(event)
+ elif name == 'irq:softirq_raise':
+ self.handle_irq_softirq_raise(event)
+ elif name == 'irq:irq_handler_entry':
+ self.handle_irq_handler_entry(event)
+ elif name == 'irq:irq_handler_exit':
+ self.handle_irq_handler_exit(event)
+ elif name == 'napi:napi_poll':
+ self.handle_napi_poll(event)
+ elif name == 'net:netif_receive_skb':
+ self.handle_netif_receive_skb(event)
+ elif name == 'net:netif_rx':
+ self.handle_netif_rx(event)
+ elif name == 'skb:skb_copy_datagram_iovec':
+ self.handle_skb_copy_datagram_iovec(event)
+ elif name == 'net:net_dev_queue':
+ self.handle_net_dev_queue(event)
+ elif name == 'net:net_dev_xmit':
+ self.handle_net_dev_xmit(event)
+ elif name == 'skb:kfree_skb':
+ self.handle_kfree_skb(event)
+ elif name == 'skb:consume_skb':
+ self.handle_consume_skb(event)
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Process events directly on-the-fly."""
+ name = str(sample.evsel)
+ ev_name = name[6:-1] if name.startswith("evsel(") else name
+ pid = sample.sample_pid
+ if hasattr(self, 'session') and self.session:
+ try:
+ thread = self.session.find_thread(pid)
+ comm = (thread.comm() if thread else None) or "[unknown]"
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError):
+ comm = "[unknown]"
+ else:
+ comm = "Unknown"
+ event_data = {
+ 'name': ev_name,
+ 'time': sample.sample_time,
+ 'cpu': sample.sample_cpu,
+ 'pid': pid,
+ 'comm': comm,
+ }
+
+ # Extract specific fields based on event type
+ if ev_name.startswith("irq:softirq_"):
+ event_data['vec'] = getattr(sample, "vec", 0)
+ # Filter for NET_RX
+ if event_data['vec'] != 3: # NET_RX_SOFTIRQ is usually 3
+ return
+ elif ev_name == "irq:irq_handler_entry":
+ event_data['irq'] = getattr(sample, "irq", -1)
+ event_data['irq_name'] = getattr(sample, "name", "[unknown]")
+ elif ev_name == "irq:irq_handler_exit":
+ event_data['irq'] = getattr(sample, "irq", -1)
+ event_data['ret'] = getattr(sample, "ret", 0)
+ elif ev_name == "napi:napi_poll":
+ event_data['napi'] = getattr(sample, "napi", 0)
+ event_data['dev_name'] = getattr(sample, "dev_name", "[unknown]")
+ event_data['work'] = getattr(sample, "work", 0)
+ event_data['budget'] = getattr(sample, "budget", 0)
+ elif ev_name in ("net:netif_receive_skb", "net:netif_rx",
+ "net:net_dev_queue"):
+ event_data['skbaddr'] = getattr(sample, "skbaddr", 0)
+ event_data['skblen'] = getattr(sample, "len", 0)
+ event_data['dev_name'] = getattr(sample, "name", "[unknown]")
+ elif ev_name == "net:net_dev_xmit":
+ event_data['skbaddr'] = getattr(sample, "skbaddr", 0)
+ event_data['skblen'] = getattr(sample, "len", 0)
+ event_data['rc'] = getattr(sample, "rc", 0)
+ event_data['dev_name'] = getattr(sample, "name", "[unknown]")
+ elif ev_name == "skb:kfree_skb":
+ event_data['skbaddr'] = getattr(sample, "skbaddr", 0)
+ event_data['location'] = getattr(sample, "location", 0)
+ event_data['protocol'] = getattr(sample, "protocol", 0)
+ event_data['reason'] = getattr(sample, "reason", 0)
+ elif ev_name == "skb:consume_skb":
+ event_data['skbaddr'] = getattr(sample, "skbaddr", 0)
+ event_data['location'] = getattr(sample, "location", 0)
+ elif ev_name == "skb:skb_copy_datagram_iovec":
+ event_data['skbaddr'] = getattr(sample, "skbaddr", 0)
+ event_data['skblen'] = getattr(sample, "len", 0)
+
+ self.handle_single_event(event_data)
+
+
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser(description="Display a process of packets and processed time.")
+ ap.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ ap.add_argument("--tx", action="store_true", help="show only tx chart")
+ ap.add_argument("--rx", action="store_true", help="show only rx chart")
+ ap.add_argument("--dev", default=None, help="show only specified device")
+ ap.add_argument("--debug", action="store_true",
+ help="work with debug mode. It shows buffer status.")
+ ap.add_argument("positionals", nargs="*",
+ help="optional positional arguments (tx, rx, dev=<name>, debug)")
+ args, unknown_args = ap.parse_known_args()
+
+ parsed_args = argparse.Namespace(
+ tx=args.tx, rx=args.rx, dev=args.dev, debug=args.debug, input=args.input
+ )
+
+ for arg in list(args.positionals) + unknown_args:
+ if not arg or not isinstance(arg, str):
+ continue
+ if arg in ('tx', '--tx'):
+ parsed_args.tx = True
+ elif arg in ('rx', '--rx'):
+ parsed_args.rx = True
+ elif arg.startswith('dev='):
+ parsed_args.dev = arg[4:]
+ elif arg.startswith('--dev='):
+ parsed_args.dev = arg[6:]
+ elif arg in ('debug', '--debug'):
+ parsed_args.debug = True
+
+ analyzer = NetDevTimesAnalyzer(parsed_args)
+
+ try:
+ session = perf.session(perf.data(parsed_args.input), sample=analyzer.process_event)
+ analyzer.session = session
+ session.process_events()
+ analyzer.print_summary()
+ except KeyboardInterrupt:
+ analyzer.print_summary()
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError) as e:
+ print(f"Error processing events: {e}")
+ sys.exit(1)
diff --git a/tools/perf/tests/shell/test_netdev_times_python.sh b/tools/perf/tests/shell/test_netdev_times_python.sh
new file mode 100755
index 000000000000..b512d1946ff0
--- /dev/null
+++ b/tools/perf/tests/shell/test_netdev_times_python.sh
@@ -0,0 +1,61 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# netdev_times python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/netdev-times.py"
+
+if ! perf check feature -q libtraceevent > /dev/null 2>&1; then
+ echo "Skipping test, libtraceevent is disabled"
+ exit 2
+fi
+
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, netdev-times.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+temp_out=""
+
+cleanup() {
+ rm -f "${temp_data}" "${temp_out}"
+}
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+echo "Testing netdev-times.py..."
+
+# Create a perf.data file. Force dropping a packet if tracepoint is available!
+if ! perf record -e skb:kfree_skb -a -o "${temp_data}" \
+ -- ping -c 1 127.0.0.1 >/dev/null 2>&1; then
+ perf record -e cycles -o "${temp_data}" \
+ -- perf test -w noploop >/dev/null 2>&1 || \
+ { echo "Skipping test, perf record failed"; exit 2; }
+fi
+
+# Check that the script executes
+if ! "$PYTHON" "$script_path" -i "${temp_data}" > "${temp_out}"; then
+ echo "netdev-times.py test failed"
+ err=1
+else
+ echo "netdev-times test passed."
+fi
+rm -f "${temp_out}"
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 37/49] perf python: Port check-perf-trace to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (35 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 36/49] perf python: Port netdev-times " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 38/49] perf python: Port arm-cs-trace-disasm " Ian Rogers
` (11 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port check-perf-trace.py to a standalone script in tools/perf/python/
using the perf module directly.
Improvements compared to the legacy script:
- Access tracepoint fields directly as attributes on perf.sample_event
instead of per-event dictionaries and legacy Perf-Trace-Util helpers.
- Decode symbolic flag and enum masks for irq:softirq_entry and
kmem:kmalloc directly in Python and add -i/--input CLI support via
argparse.
- Add full type annotations and clean up Python 2 idioms.
Add a shell test (test_check_perf_trace_python.sh) to verify the
standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/check-perf-trace.py | 213 ++++++++++++++++++
.../shell/test_check_perf_trace_python.sh | 79 +++++++
2 files changed, 292 insertions(+)
create mode 100755 tools/perf/python/check-perf-trace.py
create mode 100755 tools/perf/tests/shell/test_check_perf_trace_python.sh
diff --git a/tools/perf/python/check-perf-trace.py b/tools/perf/python/check-perf-trace.py
new file mode 100755
index 000000000000..19a91c7934b7
--- /dev/null
+++ b/tools/perf/python/check-perf-trace.py
@@ -0,0 +1,213 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+Basic test of Python scripting support for perf.
+Ported from tools/perf/scripts/python/check-perf-trace.py
+"""
+from __future__ import annotations
+
+import argparse
+import collections
+import perf
+
+unhandled: collections.defaultdict[str, int] = collections.defaultdict(int)
+session = None
+
+softirq_vecs = {
+ 0: "HI_SOFTIRQ",
+ 1: "TIMER_SOFTIRQ",
+ 2: "NET_TX_SOFTIRQ",
+ 3: "NET_RX_SOFTIRQ",
+ 4: "BLOCK_SOFTIRQ",
+ 5: "IRQ_POLL_SOFTIRQ",
+ 6: "TASKLET_SOFTIRQ",
+ 7: "SCHED_SOFTIRQ",
+ 8: "HRTIMER_SOFTIRQ",
+ 9: "RCU_SOFTIRQ",
+}
+
+_GFP_DMA = 1 << 0
+_GFP_HIGHMEM = 1 << 1
+_GFP_DMA32 = 1 << 2
+_GFP_MOVABLE = 1 << 3
+_GFP_RECLAIMABLE = 1 << 4
+_GFP_HIGH = 1 << 5
+_GFP_IO = 1 << 6
+_GFP_FS = 1 << 7
+_GFP_ZERO = 1 << 8
+_GFP_DIRECT_RECLAIM = 1 << 10
+_GFP_KSWAPD_RECLAIM = 1 << 11
+_GFP_WRITE = 1 << 12
+_GFP_NOWARN = 1 << 13
+_GFP_RETRY_MAYFAIL = 1 << 14
+_GFP_NOFAIL = 1 << 15
+_GFP_NORETRY = 1 << 16
+_GFP_MEMALLOC = 1 << 17
+_GFP_COMP = 1 << 18
+_GFP_NOMEMALLOC = 1 << 19
+_GFP_HARDWALL = 1 << 20
+_GFP_THISNODE = 1 << 21
+_GFP_ACCOUNT = 1 << 22
+_GFP_ZEROTAGS = 1 << 23
+
+_GFP_RECLAIM = _GFP_DIRECT_RECLAIM | _GFP_KSWAPD_RECLAIM
+_GFP_KERNEL = _GFP_RECLAIM | _GFP_IO | _GFP_FS
+_GFP_USER = _GFP_KERNEL | _GFP_HARDWALL
+_GFP_HIGHUSER = _GFP_USER | _GFP_HIGHMEM
+_GFP_HIGHUSER_MOVABLE = _GFP_HIGHUSER | _GFP_MOVABLE
+_GFP_TRANSHUGE_LIGHT = (
+ _GFP_HIGHUSER_MOVABLE | _GFP_COMP | _GFP_NOMEMALLOC | _GFP_NOWARN
+) & ~_GFP_RECLAIM
+_GFP_TRANSHUGE = _GFP_TRANSHUGE_LIGHT | _GFP_DIRECT_RECLAIM
+
+GFP_FLAG_NAMES = [
+ (_GFP_TRANSHUGE, "GFP_TRANSHUGE"),
+ (_GFP_TRANSHUGE_LIGHT, "GFP_TRANSHUGE_LIGHT"),
+ (_GFP_HIGHUSER_MOVABLE, "GFP_HIGHUSER_MOVABLE"),
+ (_GFP_HIGHUSER, "GFP_HIGHUSER"),
+ (_GFP_USER, "GFP_USER"),
+ (_GFP_KERNEL | _GFP_ACCOUNT, "GFP_KERNEL_ACCOUNT"),
+ (_GFP_KERNEL, "GFP_KERNEL"),
+ (_GFP_RECLAIM | _GFP_IO, "GFP_NOFS"),
+ (_GFP_HIGH | _GFP_KSWAPD_RECLAIM, "GFP_ATOMIC"),
+ (_GFP_RECLAIM, "GFP_NOIO"),
+ (_GFP_KSWAPD_RECLAIM | _GFP_NOWARN, "GFP_NOWAIT"),
+ (_GFP_DMA, "GFP_DMA"),
+ (_GFP_DMA32, "GFP_DMA32"),
+ (_GFP_RECLAIM, "__GFP_RECLAIM"),
+ (_GFP_DMA, "__GFP_DMA"),
+ (_GFP_HIGHMEM, "__GFP_HIGHMEM"),
+ (_GFP_DMA32, "__GFP_DMA32"),
+ (_GFP_MOVABLE, "__GFP_MOVABLE"),
+ (_GFP_RECLAIMABLE, "__GFP_RECLAIMABLE"),
+ (_GFP_HIGH, "__GFP_HIGH"),
+ (_GFP_IO, "__GFP_IO"),
+ (_GFP_FS, "__GFP_FS"),
+ (_GFP_ZERO, "__GFP_ZERO"),
+ (_GFP_DIRECT_RECLAIM, "__GFP_DIRECT_RECLAIM"),
+ (_GFP_KSWAPD_RECLAIM, "__GFP_KSWAPD_RECLAIM"),
+ (_GFP_WRITE, "__GFP_WRITE"),
+ (_GFP_NOWARN, "__GFP_NOWARN"),
+ (_GFP_RETRY_MAYFAIL, "__GFP_RETRY_MAYFAIL"),
+ (_GFP_NOFAIL, "__GFP_NOFAIL"),
+ (_GFP_NORETRY, "__GFP_NORETRY"),
+ (_GFP_MEMALLOC, "__GFP_MEMALLOC"),
+ (_GFP_COMP, "__GFP_COMP"),
+ (_GFP_NOMEMALLOC, "__GFP_NOMEMALLOC"),
+ (_GFP_HARDWALL, "__GFP_HARDWALL"),
+ (_GFP_THISNODE, "__GFP_THISNODE"),
+ (_GFP_ACCOUNT, "__GFP_ACCOUNT"),
+ (_GFP_ZEROTAGS, "__GFP_ZEROTAGS"),
+]
+
+
+def trace_begin() -> None:
+ """Called at the start of trace processing."""
+ print("trace_begin")
+
+def trace_end() -> None:
+ """Called at the end of trace processing."""
+ print_unhandled()
+ print("trace_end")
+
+def symbol_str(event_name: str, field_name: str, value: int) -> str:
+ """Resolves symbol values to strings."""
+ # Note: The standalone Python API currently lacks dynamic libtraceevent
+ # formatting (equivalent to _perf_trace_context.symbol_str())
+ if event_name == "irq__softirq_entry" and field_name == "vec":
+ return softirq_vecs.get(value, str(value))
+ return str(value)
+
+def flag_str(event_name: str, field_name: str, value: int) -> str:
+ """Resolves flag values to strings."""
+ # Note: The standalone Python API currently lacks dynamic libtraceevent
+ # formatting (equivalent to _perf_trace_context.flag_str())
+ if event_name == "kmem__kmalloc" and field_name == "gfp_flags":
+ if value == 0:
+ return "none"
+ names = []
+ rem = value
+ for mask, name in GFP_FLAG_NAMES:
+ if (rem & mask) == mask:
+ names.append(name)
+ rem &= ~mask
+ if rem:
+ names.append(f"0x{rem:x}")
+ return "|".join(names)
+ return str(value)
+
+def print_header(event_name: str, sample: perf.sample_event) -> None:
+ """Prints common header for events."""
+ secs = sample.sample_time // 1000000000
+ nsecs = sample.sample_time % 1000000000
+ comm = "[unknown]"
+ try:
+ if session:
+ thread = session.find_thread(sample.sample_tid)
+ if thread:
+ comm = thread.comm() or "[unknown]"
+ except (TypeError, AttributeError):
+ pass
+ print(f"{event_name:<20} {sample.sample_cpu:5} {secs:05}.{nsecs:09} "
+ f"{sample.sample_tid:8} {comm:<20} ", end=' ')
+
+def print_uncommon(sample: perf.sample_event) -> None:
+ """Prints uncommon fields for tracepoints."""
+ # Fallback to 0 if field not found (e.g. on older kernels or if not tracepoint)
+ pc = getattr(sample, 'common_preempt_count', 0)
+ flags = getattr(sample, 'common_flags', 0)
+ lock_depth = getattr(sample, 'common_lock_depth', 0)
+
+ print(f"common_preempt_count={pc}, common_flags={flags}, "
+ f"common_lock_depth={lock_depth}, ", end='')
+
+def irq__softirq_entry(sample: perf.sample_event) -> None:
+ """Handles irq:softirq_entry events."""
+ print_header("irq__softirq_entry", sample)
+ print_uncommon(sample)
+ print(f"vec={symbol_str('irq__softirq_entry', 'vec', getattr(sample, 'vec', 0))}")
+
+def kmem__kmalloc(sample: perf.sample_event) -> None:
+ """Handles kmem:kmalloc events."""
+ print_header("kmem__kmalloc", sample)
+ print_uncommon(sample)
+
+ print(f"call_site={getattr(sample, 'call_site', 0):#x}, "
+ f"ptr={getattr(sample, 'ptr', 0):#x}, "
+ f"bytes_req={getattr(sample, 'bytes_req', 0):d}, "
+ f"bytes_alloc={getattr(sample, 'bytes_alloc', 0):d}, "
+ f"gfp_flags={flag_str('kmem__kmalloc', 'gfp_flags', getattr(sample, 'gfp_flags', 0))}")
+
+def trace_unhandled(event_name: str) -> None:
+ """Tracks unhandled events."""
+ unhandled[event_name] += 1
+
+def print_unhandled() -> None:
+ """Prints summary of unhandled events."""
+ if not unhandled:
+ return
+ print("\nunhandled events:\n")
+ print(f"{'event':<40} {'count':>10}")
+ print("---------------------------------------- -----------")
+ for event_name, count in unhandled.items():
+ print(f"{event_name:<40} {count:10}")
+
+def process_event(sample: perf.sample_event) -> None:
+ """Callback for processing events."""
+ event_name = str(sample.evsel)
+ if event_name.startswith("evsel(irq:softirq_entry)"):
+ irq__softirq_entry(sample)
+ elif "evsel(kmem:kmalloc)" in event_name:
+ kmem__kmalloc(sample)
+ else:
+ trace_unhandled(event_name)
+
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser()
+ ap.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ args = ap.parse_args()
+
+ trace_begin()
+ session = perf.session(perf.data(args.input), sample=process_event)
+ session.process_events()
+ trace_end()
diff --git a/tools/perf/tests/shell/test_check_perf_trace_python.sh b/tools/perf/tests/shell/test_check_perf_trace_python.sh
new file mode 100755
index 000000000000..2c5295134f53
--- /dev/null
+++ b/tools/perf/tests/shell/test_check_perf_trace_python.sh
@@ -0,0 +1,79 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# check-perf-trace python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+# If we don't have the perf python module, we can't test
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+if ! perf check feature -q libtraceevent; then
+ echo "Skipping test, perf built without libtraceevent"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/check-perf-trace.py"
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, check-perf-trace.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_data=""
+
+cleanup() {
+ rm -f "${temp_data}"
+}
+
+trap 'cleanup' EXIT
+trap 'cleanup; exit 1' TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+
+test_file_mode() {
+ echo "Testing check-perf-trace.py..."
+
+ events=""
+ if perf list | grep -q "irq:softirq_entry"; then
+ events="irq:softirq_entry"
+ fi
+ if perf list | grep -q "kmem:kmalloc"; then
+ if [ -n "$events" ]; then
+ events="$events,kmem:kmalloc,kmem:kfree"
+ else
+ events="kmem:kmalloc,kmem:kfree"
+ fi
+ fi
+
+ if [ -z "$events" ]; then
+ echo "Skipping test, no required tracepoints found"
+ exit 2
+ fi
+
+ # Generate events
+ if ! perf record -e "$events" -a -o "${temp_data}" -- sleep 0.5 >/dev/null 2>&1; then
+ echo "Skipping test, perf record failed"
+ exit 2
+ fi
+
+ # Run the script
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" >/dev/null; then
+ echo "File mode test failed."
+ err=1
+ else
+ echo "File mode test passed."
+ fi
+}
+
+test_file_mode
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 38/49] perf python: Port arm-cs-trace-disasm to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (36 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 37/49] perf python: Port check-perf-trace " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 39/49] perf python: Port powerpc-hcalls " Ian Rogers
` (10 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port arm-cs-trace-disasm.py to a standalone script in tools/perf/python/
using the perf module directly.
Improvements compared to the legacy script:
- Encapsulate trace disassembly state in a TraceDisasm class and
migrate CLI parsing from deprecated optparse to argparse.
- Automatically search standard kernel debug paths (find_vmlinux())
when -k/--vmlinux is not specified, and query
perf.config_get("annotate.objdump") for the default objdump binary.
- Bound DISASM_CACHE memory consumption by evicting the cache at 1024
entries and skipping caching of oversized (> 512 lines) objdump
outputs.
- Use sample.srccode() from the perf extension module to annotate
disassembly output with source filenames, line numbers, and source
lines.
Update the ARM CoreSight disassembly shell test
(test_arm_coresight_disasm.sh) to invoke the standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/arm-cs-trace-disasm.py | 350 ++++++++++++++++++
.../coresight/test_arm_coresight_disasm.sh | 27 +-
2 files changed, 371 insertions(+), 6 deletions(-)
create mode 100755 tools/perf/python/arm-cs-trace-disasm.py
diff --git a/tools/perf/python/arm-cs-trace-disasm.py b/tools/perf/python/arm-cs-trace-disasm.py
new file mode 100755
index 000000000000..da5c5a5a3f3c
--- /dev/null
+++ b/tools/perf/python/arm-cs-trace-disasm.py
@@ -0,0 +1,350 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+arm-cs-trace-disasm.py: ARM CoreSight Trace Dump With Disassember using perf python module
+"""
+from __future__ import annotations
+
+import os
+from os import path
+import re
+from subprocess import check_output
+import argparse
+import platform
+import sys
+from typing import Dict, List, Optional
+
+import perf
+
+# Initialize global dicts and regular expression
+DISASM_CACHE: Dict[str, List[str]] = {}
+CPU_DATA: Dict[str, int] = {}
+DISASM_RE = re.compile(r"^\s*([0-9a-fA-F]+):")
+DISASM_FUNC_RE = re.compile(r"^\s*([0-9a-fA-F]+)\s.*:")
+CACHE_SIZE = 1024
+class _State:
+ sample_idx: int = -1
+ source_file_name: Optional[str] = None
+ line_number: Optional[int] = None
+ dso: Optional[str] = None
+
+_STATE = _State()
+
+KVER = platform.release()
+VMLINUX_PATHS = [
+ f"/usr/lib/debug/boot/vmlinux-{KVER}.debug",
+ f"/usr/lib/debug/lib/modules/{KVER}/vmlinux",
+ f"/lib/modules/{KVER}/build/vmlinux",
+ f"/usr/lib/debug/boot/vmlinux-{KVER}",
+ f"/boot/vmlinux-{KVER}",
+ "/boot/vmlinux",
+ "vmlinux"
+]
+
+def default_objdump() -> str:
+ """Return the default objdump path from perf config or 'objdump'."""
+ try:
+ config = perf.config_get("annotate.objdump")
+ return str(config) if config else "objdump"
+ except (AttributeError, TypeError):
+ return "objdump"
+
+def find_vmlinux() -> Optional[str]:
+ """Find the vmlinux file in standard paths."""
+ if hasattr(find_vmlinux, "path"):
+ return getattr(find_vmlinux, "path")
+
+ for v in VMLINUX_PATHS:
+ if os.access(v, os.R_OK):
+ setattr(find_vmlinux, "path", v)
+ return v
+ setattr(find_vmlinux, "path", None)
+ return None
+
+def get_dso_file_path(dso_name: str, dso_build_id: str, vmlinux: Optional[str]) -> str:
+ """Return the path to the DSO file."""
+ buildid_dir = os.environ.get('PERF_BUILDID_DIR')
+ if not buildid_dir:
+ buildid_dir = os.path.join(os.environ.get('HOME', ''), '.debug')
+
+ if dso_name in ("[kernel.kallsyms]", "vmlinux"):
+ if vmlinux:
+ return vmlinux
+ if dso_build_id:
+ for kname in (dso_name, "vmlinux", "[kernel.kallsyms]"):
+ candidate = os.path.join(buildid_dir, kname, dso_build_id, "elf")
+ if os.access(candidate, os.R_OK):
+ return candidate
+ return find_vmlinux() or dso_name
+
+ if dso_name == "[vdso]":
+ append = "/vdso"
+ else:
+ append = "/elf"
+
+ dso_path = buildid_dir + "/" + dso_name + "/" + dso_build_id + append
+ # Replace duplicate slash chars to single slash char
+ dso_path = dso_path.replace('//', '/', 1)
+ return dso_path
+
+def read_disam(dso_fname: str, dso_start: int, start_addr: int,
+ stop_addr: int, objdump: str) -> List[str]:
+ """Read disassembly from a DSO file using objdump."""
+ addr_range = f"{start_addr}:{stop_addr}:{dso_start}:{dso_fname}"
+
+ # Don't let the cache get too big, clear it when it hits max size
+ if len(DISASM_CACHE) > CACHE_SIZE:
+ DISASM_CACHE.clear()
+
+ if addr_range in DISASM_CACHE:
+ disasm_output = DISASM_CACHE[addr_range]
+ else:
+ start_addr = start_addr - dso_start
+ stop_addr = stop_addr - dso_start
+ disasm = [objdump, "-d", "-z",
+ f"--start-address={start_addr:#x}",
+ f"--stop-address={stop_addr:#x}"]
+ disasm += [dso_fname]
+ disasm_output = check_output(disasm).decode('utf-8').split('\n')
+ if len(disasm_output) <= 512:
+ DISASM_CACHE[addr_range] = disasm_output
+
+ return disasm_output
+
+def print_disam(dso_fname: str, dso_start: int, start_addr: int,
+ stop_addr: int, objdump: str) -> None:
+ """Print disassembly for a given address range."""
+ for line in read_disam(dso_fname, dso_start, start_addr, stop_addr, objdump):
+ m = DISASM_FUNC_RE.search(line)
+ if m is None:
+ m = DISASM_RE.search(line)
+ if m is None:
+ continue
+ print(f"\t{line}")
+
+def print_sample(sample: perf.sample_event) -> None:
+ """Print sample details."""
+ print(f"Sample = {{ cpu: {sample.sample_cpu:04d} addr: {sample.sample_addr:016x} "
+ f"phys_addr: {sample.sample_phys_addr:016x} ip: {sample.sample_ip:016x} "
+ f"pid: {sample.sample_pid} tid: {sample.sample_tid} period: {sample.sample_period} "
+ f"time: {sample.sample_time} index: {_STATE.sample_idx}}}")
+
+def common_start_str(comm: str, sample: perf.sample_event) -> str:
+ """Return common start string for sample output."""
+ sec = int(sample.sample_time / 1000000000)
+ ns = sample.sample_time % 1000000000
+ cpu = sample.sample_cpu
+ pid = sample.sample_pid
+ tid = sample.sample_tid
+ return f"{comm:>16s} {pid:5d}/{tid:<5d} [{cpu:04d}] {sec:9d}.{ns:09d} "
+
+def print_srccode(comm: str, sample: perf.sample_event, symbol: str, dso: str) -> None:
+ """Print source code and symbols for a sample."""
+ ip = sample.sample_ip
+ if symbol == "[unknown]":
+ start_str = common_start_str(comm, sample) + f"{ip:x}".rjust(16).ljust(40)
+ else:
+ symoff = 0
+ symoff = getattr(sample, 'sym_offset', 0) or 0
+ offs = f"+{symoff:#x}" if symoff != 0 else ""
+ start_str = common_start_str(comm, sample) + (symbol + offs).ljust(40)
+
+ source_file_name, line_number, source_line = sample.srccode() or (None, 0, None)
+ if source_file_name:
+ if _STATE.line_number == line_number and _STATE.source_file_name == source_file_name:
+ src_str = ""
+ else:
+ if len(source_file_name) > 40:
+ src_file = f"...{source_file_name[-37:]} "
+ else:
+ src_file = source_file_name.ljust(41)
+
+ if source_line is None:
+ src_str = f"{src_file}{line_number:>4d} <source not found>"
+ else:
+ src_str = f"{src_file}{line_number:>4d} {source_line}"
+ _STATE.dso = None
+ elif dso == _STATE.dso:
+ src_str = ""
+ else:
+ src_str = dso
+ _STATE.dso = dso
+
+ _STATE.line_number = line_number
+ _STATE.source_file_name = source_file_name
+
+ print(start_str, src_str)
+
+class TraceDisasm:
+ """Class to handle trace disassembly."""
+ def __init__(self, cli_options: argparse.Namespace):
+ self.options = cli_options
+ self.sample_idx = -1
+ self.session: Optional[perf.session] = None
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Process a single perf event."""
+ self.sample_idx += 1
+ _STATE.sample_idx = self.sample_idx
+
+ if self.options.start_time is not None and sample.sample_time < self.options.start_time:
+ return
+ if self.options.stop_time is not None and sample.sample_time > self.options.stop_time:
+ sys.exit(0)
+ if self.options.start_sample is not None and self.sample_idx < self.options.start_sample:
+ return
+ if self.options.stop_sample is not None and self.sample_idx > self.options.stop_sample:
+ sys.exit(0)
+
+ ev_name = str(sample.evsel)
+ if self.options.verbose:
+ print(f"Event type: {ev_name}")
+ print_sample(sample)
+
+ dso = sample.dso or '[unknown]'
+ symbol = sample.symbol or '[unknown]'
+ dso_bid = (sample.dso_bid.decode('utf-8')
+ if isinstance(sample.dso_bid, bytes)
+ else str(sample.dso_bid or '[unknown]'))
+ dso_start = sample.map_start
+ dso_end = sample.map_end
+ map_pgoff = sample.map_pgoff or 0
+
+ comm = "[unknown]"
+ try:
+ if self.session:
+ thread_info = self.session.find_thread(sample.sample_tid)
+ if thread_info:
+ comm = thread_info.comm() or "[unknown]"
+ except (TypeError, AttributeError):
+ pass
+
+ if dso == '[unknown]':
+ return
+
+ if dso_start is None or dso_end is None:
+ print(f"Failed to find valid dso map for dso {dso}")
+ return
+
+ if "instructions" in ev_name:
+ print_srccode(comm, sample, symbol, dso)
+ return
+
+ if "branches" not in ev_name:
+ return
+
+ self._process_branch(sample, comm, symbol, dso, dso_bid, dso_start, dso_end, map_pgoff)
+
+ def _process_branch(self, sample: perf.sample_event, comm: str, symbol: str, dso: str,
+ dso_bid: str, dso_start: int, dso_end: int, map_pgoff: int) -> None:
+ """Helper to process branch events."""
+ cpu = sample.sample_cpu
+ ip = sample.sample_ip
+ addr = sample.sample_addr
+
+ if CPU_DATA.get(str(cpu) + 'addr') is None:
+ CPU_DATA[str(cpu) + 'addr'] = addr
+ return
+
+ start_addr = CPU_DATA[str(cpu) + 'addr']
+ stop_addr = ip + 4
+
+ # Record for previous sample packet
+ CPU_DATA[str(cpu) + 'addr'] = addr
+
+ # Filter out zero start_address. Optionally identify CS_ETM_TRACE_ON packet
+ if start_addr == 0:
+ if stop_addr == 4 and self.options.verbose:
+ print(f"CPU{cpu}: CS_ETM_TRACE_ON packet is inserted")
+ return
+
+ if start_addr < dso_start or start_addr > dso_end:
+ print(f"Start address {start_addr:#x} is out of range [ {dso_start:#x} .. "
+ f"{dso_end:#x} ] for dso {dso}")
+ return
+
+ if stop_addr < dso_start or stop_addr > dso_end:
+ print(f"Stop address {stop_addr:#x} is out of range [ {dso_start:#x} .. "
+ f"{dso_end:#x} ] for dso {dso}")
+ return
+
+ if self.options.objdump is not None:
+ if dso == "[kernel.kallsyms]":
+ dso_vm_start = 0
+ map_pgoff_local = 0
+ elif dso_start == 0x400000:
+ dso_vm_start = 0
+ map_pgoff_local = 0
+ else:
+ dso_vm_start = dso_start
+ map_pgoff_local = map_pgoff
+
+ dso_fname = get_dso_file_path(dso, dso_bid, self.options.vmlinux)
+ if path.exists(dso_fname):
+ print_disam(dso_fname, dso_vm_start, start_addr + map_pgoff_local,
+ stop_addr + map_pgoff_local, self.options.objdump)
+ else:
+ print(f"Failed to find dso {dso} for address range [ "
+ f"{start_addr + map_pgoff_local:#x} .. {stop_addr + map_pgoff_local:#x} ]")
+
+ print_srccode(comm, sample, symbol, dso)
+
+ def run(self) -> None:
+ """Run the trace disassembly session."""
+ input_file = self.options.input or "perf.data"
+ if input_file != "-" and not os.path.exists(input_file):
+ print(f"Error: {input_file} not found.", file=sys.stderr)
+ sys.exit(1)
+
+ print('ARM CoreSight Trace Data Assembler Dump')
+ try:
+ self.session = perf.session(
+ perf.data(input_file),
+ sample=self.process_event,
+ itrace=self.options.itrace
+ )
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError) as e:
+ print(f"Error opening session: {e}", file=sys.stderr)
+ sys.exit(1)
+
+ self.session.process_events()
+ print('End')
+
+if __name__ == "__main__":
+ def int_arg(v: str) -> int:
+ """Helper for integer command line arguments."""
+ val = int(v)
+ if val < 0:
+ raise argparse.ArgumentTypeError("Argument must be a positive integer")
+ return val
+
+ arg_parser = argparse.ArgumentParser(description="ARM CoreSight Trace Dump With Disassembler")
+ arg_parser.add_argument("-i", "--input", help="input perf.data file")
+ arg_parser.add_argument("-k", "--vmlinux",
+ help="Set path to vmlinux file. Omit to autodetect")
+ arg_parser.add_argument("-d", "--objdump", nargs="?", const=default_objdump(),
+ help="Show disassembly. Can also be used to change the objdump path")
+ arg_parser.add_argument("-v", "--verbose", action="store_true", help="Enable debugging log")
+ arg_parser.add_argument("--start-time", type=int_arg,
+ help="Monotonic clock time of sample to start from.")
+ arg_parser.add_argument("--stop-time", type=int_arg,
+ help="Monotonic clock time of sample to stop at.")
+ arg_parser.add_argument("--itrace", default="b",
+ help="Instruction tracing options.")
+ arg_parser.add_argument("--start-sample", type=int_arg,
+ help="Index of sample to start from.")
+ arg_parser.add_argument("--stop-sample", type=int_arg,
+ help="Index of sample to stop at.")
+
+ parsed_options = arg_parser.parse_args()
+ if (parsed_options.start_time is not None and parsed_options.stop_time is not None and
+ parsed_options.start_time >= parsed_options.stop_time):
+ print("--start-time must less than --stop-time")
+ sys.exit(2)
+ if (parsed_options.start_sample is not None and parsed_options.stop_sample is not None and
+ parsed_options.start_sample >= parsed_options.stop_sample):
+ print("--start-sample must less than --stop-sample")
+ sys.exit(2)
+
+ td = TraceDisasm(parsed_options)
+ td.run()
diff --git a/tools/perf/tests/shell/coresight/test_arm_coresight_disasm.sh b/tools/perf/tests/shell/coresight/test_arm_coresight_disasm.sh
index f3ebad596378..96c6bdc13a17 100755
--- a/tools/perf/tests/shell/coresight/test_arm_coresight_disasm.sh
+++ b/tools/perf/tests/shell/coresight/test_arm_coresight_disasm.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-# Check Arm CoreSight disassembly script completes without errors (exclusive)
# SPDX-License-Identifier: GPL-2.0
+# Check Arm CoreSight disassembly script completes without errors (exclusive)
# The disassembly script reconstructs ranges of instructions and gives these to objdump to
# decode. objdump doesn't like ranges that go backwards, but these are a good indication
@@ -22,9 +22,10 @@ glb_err=1
perfdata_dir=$(mktemp -d /tmp/__perf_test.perf.data.XXXXX)
perfdata=${perfdata_dir}/perf.data
+perfdata2=${perfdata_dir}/perf2.data
file=$(mktemp /tmp/temporary_file.XXXXX)
# Relative path works whether it's installed or running from repo
-script_path=$(dirname "$0")/../../../scripts/python/arm-cs-trace-disasm.py
+script_path=$(dirname "$0")/../../../python/arm-cs-trace-disasm.py
cleanup_files()
{
@@ -44,8 +45,15 @@ branch_search='[[:space:]](bl|b(\.(eq|ne|cs|cc|mi|pl|vs|vc|hi|ls|ge|lt|gt|le|al)
if [ "$(id -u)" == 0 ] && [ -e /proc/kcore ]; then
echo "Testing kernel disassembly"
perf record -o ${perfdata} -e cs_etm//k --kcore -Se -m,64K -- touch $file > /dev/null 2>&1
- perf script -i ${perfdata} --itrace=b -s python:${script_path} -- \
- -d --stop-sample=2 -k ${perfdata}/kcore_dir/kcore 2> /dev/null > ${file}
+ # shellcheck source=lib/setup_python.sh
+ . "$(dirname "$0")"/../lib/setup_python.sh
+ $PYTHON -c "import perf" 2>/dev/null || {
+ echo "Skipping test, perf python module not found"
+ glb_err=2
+ exit 2
+ }
+ $PYTHON ${script_path} -i ${perfdata} --itrace=b -d --stop-sample=2 \
+ -k ${perfdata}/kcore_dir/kcore 2> /dev/null > ${file}
grep -q -E ${branch_search} ${file}
echo "Found kernel branches"
else
@@ -56,8 +64,15 @@ fi
## Test user ##
echo "Testing userspace disassembly"
perf record -o ${perfdata} -e cs_etm//u -Se -m,64K -- touch $file > /dev/null 2>&1
-perf script -i ${perfdata} --itrace=b -s python:${script_path} -- \
- -d --stop-sample=2 2> /dev/null > ${file}
+perf inject --itrace=b -i ${perfdata} -o ${perfdata2}
+# shellcheck source=lib/setup_python.sh
+. "$(dirname "$0")"/../lib/setup_python.sh
+$PYTHON -c "import perf" 2>/dev/null || {
+ echo "Skipping test, perf python module not found"
+ glb_err=2
+ exit 2
+}
+$PYTHON ${script_path} -i ${perfdata2} -d --stop-sample=2 2> /dev/null > ${file}
grep -q -E ${branch_search} ${file}
echo "Found userspace branches"
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 39/49] perf python: Port powerpc-hcalls to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (37 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 38/49] perf python: Port arm-cs-trace-disasm " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 40/49] perf python: Port intel-pt-events and libxed " Ian Rogers
` (9 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port powerpc-hcalls.py from tools/perf/scripts/python/ to a standalone
script in tools/perf/python/:
- Refactor the script into an HCallAnalyzer class with full type
annotations to encapsulate per-CPU hypervisor call entry/exit state.
- Use perf.session for event processing to track hypervisor call entry
and exit timestamps and aggregate min/max/average duration statistics
against HCALL_TABLE.
- Add argparse CLI support (-i/--input) and remove Python 2
compatibility code.
Add a shell test (test_powerpc_hcalls_python.sh) to verify the
standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/powerpc-hcalls.py | 330 ++++++++++++++++++
.../tests/shell/test_powerpc_hcalls_python.sh | 95 +++++
2 files changed, 425 insertions(+)
create mode 100755 tools/perf/python/powerpc-hcalls.py
create mode 100755 tools/perf/tests/shell/test_powerpc_hcalls_python.sh
diff --git a/tools/perf/python/powerpc-hcalls.py b/tools/perf/python/powerpc-hcalls.py
new file mode 100755
index 000000000000..4c354556c0fb
--- /dev/null
+++ b/tools/perf/python/powerpc-hcalls.py
@@ -0,0 +1,330 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0+
+"""
+Hypervisor call statistics
+
+Copyright (C) 2018 Ravi Bangoria, IBM Corporation
+Ported from tools/perf/scripts/python/powerpc-hcalls.py
+"""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+import sys
+import perf
+
+# Hypervisor call table
+HCALL_TABLE = {
+ 4: 'H_REMOVE',
+ 8: 'H_ENTER',
+ 12: 'H_READ',
+ 16: 'H_CLEAR_MOD',
+ 20: 'H_CLEAR_REF',
+ 24: 'H_PROTECT',
+ 28: 'H_GET_TCE',
+ 32: 'H_PUT_TCE',
+ 36: 'H_SET_SPRG0',
+ 40: 'H_SET_DABR',
+ 44: 'H_PAGE_INIT',
+ 48: 'H_SET_ASR',
+ 52: 'H_ASR_ON',
+ 56: 'H_ASR_OFF',
+ 60: 'H_LOGICAL_CI_LOAD',
+ 64: 'H_LOGICAL_CI_STORE',
+ 68: 'H_LOGICAL_CACHE_LOAD',
+ 72: 'H_LOGICAL_CACHE_STORE',
+ 76: 'H_LOGICAL_ICBI',
+ 80: 'H_LOGICAL_DCBF',
+ 84: 'H_GET_TERM_CHAR',
+ 88: 'H_PUT_TERM_CHAR',
+ 92: 'H_REAL_TO_LOGICAL',
+ 96: 'H_HYPERVISOR_DATA',
+ 100: 'H_EOI',
+ 104: 'H_CPPR',
+ 108: 'H_IPI',
+ 112: 'H_IPOLL',
+ 116: 'H_XIRR',
+ 120: 'H_MIGRATE_DMA',
+ 124: 'H_PERFMON',
+ 220: 'H_REGISTER_VPA',
+ 224: 'H_CEDE',
+ 228: 'H_CONFER',
+ 232: 'H_PROD',
+ 236: 'H_GET_PPP',
+ 240: 'H_SET_PPP',
+ 244: 'H_PURR',
+ 248: 'H_PIC',
+ 252: 'H_REG_CRQ',
+ 256: 'H_FREE_CRQ',
+ 260: 'H_VIO_SIGNAL',
+ 264: 'H_SEND_CRQ',
+ 272: 'H_COPY_RDMA',
+ 276: 'H_REGISTER_LOGICAL_LAN',
+ 280: 'H_FREE_LOGICAL_LAN',
+ 284: 'H_ADD_LOGICAL_LAN_BUFFER',
+ 288: 'H_SEND_LOGICAL_LAN',
+ 292: 'H_BULK_REMOVE',
+ 304: 'H_MULTICAST_CTRL',
+ 308: 'H_SET_XDABR',
+ 312: 'H_STUFF_TCE',
+ 316: 'H_PUT_TCE_INDIRECT',
+ 332: 'H_CHANGE_LOGICAL_LAN_MAC',
+ 336: 'H_VTERM_PARTNER_INFO',
+ 340: 'H_REGISTER_VTERM',
+ 344: 'H_FREE_VTERM',
+ 348: 'H_RESET_EVENTS',
+ 352: 'H_ALLOC_RESOURCE',
+ 356: 'H_FREE_RESOURCE',
+ 360: 'H_MODIFY_QP',
+ 364: 'H_QUERY_QP',
+ 368: 'H_REREGISTER_PMR',
+ 372: 'H_REGISTER_SMR',
+ 376: 'H_QUERY_MR',
+ 380: 'H_QUERY_MW',
+ 384: 'H_QUERY_HCA',
+ 388: 'H_QUERY_PORT',
+ 392: 'H_MODIFY_PORT',
+ 396: 'H_DEFINE_AQP1',
+ 400: 'H_GET_TRACE_BUFFER',
+ 404: 'H_DEFINE_AQP0',
+ 408: 'H_RESIZE_MR',
+ 412: 'H_ATTACH_MCQP',
+ 416: 'H_DETACH_MCQP',
+ 420: 'H_CREATE_RPT',
+ 424: 'H_REMOVE_RPT',
+ 428: 'H_REGISTER_RPAGES',
+ 432: 'H_DISABLE_AND_GET',
+ 436: 'H_ERROR_DATA',
+ 440: 'H_GET_HCA_INFO',
+ 444: 'H_GET_PERF_COUNT',
+ 448: 'H_MANAGE_TRACE',
+ 456: 'H_GET_CPU_CHARACTERISTICS',
+ 468: 'H_FREE_LOGICAL_LAN_BUFFER',
+ 472: 'H_POLL_PENDING',
+ 484: 'H_QUERY_INT_STATE',
+ 580: 'H_ILLAN_ATTRIBUTES',
+ 584: 'H_ADD_LOGICAL_LAN_BUFFERS',
+ 592: 'H_MODIFY_HEA_QP',
+ 596: 'H_QUERY_HEA_QP',
+ 600: 'H_QUERY_HEA',
+ 604: 'H_QUERY_HEA_PORT',
+ 608: 'H_MODIFY_HEA_PORT',
+ 612: 'H_REG_BCMC',
+ 616: 'H_DEREG_BCMC',
+ 620: 'H_REGISTER_HEA_RPAGES',
+ 624: 'H_DISABLE_AND_GET_HEA',
+ 628: 'H_GET_HEA_INFO',
+ 632: 'H_ALLOC_HEA_RESOURCE',
+ 644: 'H_ADD_CONN',
+ 648: 'H_DEL_CONN',
+ 664: 'H_JOIN',
+ 672: 'H_VASI_SIGNAL',
+ 676: 'H_VASI_STATE',
+ 680: 'H_VIOCTL',
+ 688: 'H_ENABLE_CRQ',
+ 696: 'H_GET_EM_PARMS',
+ 720: 'H_SET_MPP',
+ 724: 'H_GET_MPP',
+ 732: 'H_REG_SUB_CRQ',
+ 736: 'H_FREE_SUB_CRQ',
+ 740: 'H_SEND_SUB_CRQ',
+ 744: 'H_SEND_SUB_CRQ_INDIRECT',
+ 748: 'H_HOME_NODE_ASSOCIATIVITY',
+ 756: 'H_BEST_ENERGY',
+ 764: 'H_XIRR_X',
+ 768: 'H_RANDOM',
+ 772: 'H_COP',
+ 788: 'H_GET_MPP_X',
+ 796: 'H_SET_MODE',
+ 808: 'H_BLOCK_REMOVE',
+ 856: 'H_CLEAR_HPT',
+ 864: 'H_REQUEST_VMC',
+ 876: 'H_RESIZE_HPT_PREPARE',
+ 880: 'H_RESIZE_HPT_COMMIT',
+ 892: 'H_REGISTER_PROC_TBL',
+ 896: 'H_SIGNAL_SYS_RESET',
+ 904: 'H_ALLOCATE_VAS_WINDOW',
+ 908: 'H_MODIFY_VAS_WINDOW',
+ 912: 'H_DEALLOCATE_VAS_WINDOW',
+ 916: 'H_QUERY_VAS_WINDOW',
+ 920: 'H_QUERY_VAS_CAPABILITIES',
+ 924: 'H_QUERY_NX_CAPABILITIES',
+ 928: 'H_GET_NX_FAULT',
+ 936: 'H_INT_GET_SOURCE_INFO',
+ 940: 'H_INT_SET_SOURCE_CONFIG',
+ 944: 'H_INT_GET_SOURCE_CONFIG',
+ 948: 'H_INT_GET_QUEUE_INFO',
+ 952: 'H_INT_SET_QUEUE_CONFIG',
+ 956: 'H_INT_GET_QUEUE_CONFIG',
+ 960: 'H_INT_SET_OS_REPORTING_LINE',
+ 964: 'H_INT_GET_OS_REPORTING_LINE',
+ 968: 'H_INT_ESB',
+ 972: 'H_INT_SYNC',
+ 976: 'H_INT_RESET',
+ 996: 'H_SCM_READ_METADATA',
+ 1000: 'H_SCM_WRITE_METADATA',
+ 1004: 'H_SCM_BIND_MEM',
+ 1008: 'H_SCM_UNBIND_MEM',
+ 1012: 'H_SCM_QUERY_BLOCK_MEM_BINDING',
+ 1016: 'H_SCM_QUERY_LOGICAL_MEM_BINDING',
+ 1020: 'H_SCM_UNBIND_ALL',
+ 1024: 'H_SCM_HEALTH',
+ 1048: 'H_SCM_PERFORMANCE_STATS',
+ 1052: 'H_PKS_GET_CONFIG',
+ 1056: 'H_PKS_SET_PASSWORD',
+ 1060: 'H_PKS_GEN_PASSWORD',
+ 1068: 'H_PKS_WRITE_OBJECT',
+ 1072: 'H_PKS_GEN_KEY',
+ 1076: 'H_PKS_READ_OBJECT',
+ 1080: 'H_PKS_REMOVE_OBJECT',
+ 1084: 'H_PKS_CONFIRM_OBJECT_FLUSHED',
+ 1096: 'H_RPT_INVALIDATE',
+ 1100: 'H_SCM_FLUSH',
+ 1104: 'H_GET_ENERGY_SCALE_INFO',
+ 1108: 'H_PKS_SIGNED_UPDATE',
+ 1112: 'H_HTM',
+ 1116: 'H_WATCHDOG',
+ # Platform specific hcalls used by KVM on PowerVM
+ 1120: 'H_GUEST_GET_CAPABILITIES',
+ 1124: 'H_GUEST_SET_CAPABILITIES',
+ 1136: 'H_GUEST_CREATE',
+ 1140: 'H_GUEST_CREATE_VCPU',
+ 1144: 'H_GUEST_GET_STATE',
+ 1148: 'H_GUEST_SET_STATE',
+ 1152: 'H_GUEST_RUN_VCPU',
+ 1156: 'H_GUEST_COPY_MEMORY',
+ 1160: 'H_GUEST_DELETE',
+ # Key wrapping hcalls
+ 1168: 'H_PKS_WRAP_OBJECT',
+ 1172: 'H_PKS_UNWRAP_OBJECT',
+ # Platform-specific hcalls used by the Ultravisor
+ 61184: 'H_SVM_PAGE_IN',
+ 61188: 'H_SVM_PAGE_OUT',
+ 61192: 'H_SVM_INIT_START',
+ 61196: 'H_SVM_INIT_DONE',
+ 61204: 'H_SVM_INIT_ABORT',
+ # Platform specific hcalls used by KVM
+ 61440: 'H_RTAS',
+ # Platform specific hcalls used by QEMU/SLOF
+ 61441: 'H_LOGICAL_MEMOP',
+ 61442: 'H_CAS',
+ 61443: 'H_UPDATE_DT',
+ # Platform specific hcalls provided by PHYP
+ 61560: 'H_GET_24X7_CATALOG_PAGE',
+ 61564: 'H_GET_24X7_DATA',
+ 61568: 'H_GET_PERF_COUNTER_INFO',
+ # Platform-specific hcalls used for nested HV KVM
+ 63488: 'H_SET_PARTITION_TABLE',
+ 63492: 'H_ENTER_NESTED',
+ 63496: 'H_TLB_INVALIDATE',
+ 63500: 'H_COPY_TOFROM_GUEST',
+}
+
+
+class HCallAnalyzer:
+ """Analyzes hypervisor calls and aggregates statistics."""
+
+ def __init__(self, cmd_args) -> None:
+ self.args = cmd_args
+ # output: {opcode: {'min': min, 'max': max, 'time': time, 'cnt': cnt}}
+ self.output: dict[int, dict[str, 'int | float']] = \
+ defaultdict(lambda: {'time': 0, 'cnt': 0, 'min': float('inf'), 'max': 0})
+ # d_enter: {cpu: {opcode: nsec}}
+ self.d_enter: dict[int, dict[int, int]] = {}
+ self.print_ptrn = '%-28s%10s%10s%10s%10s'
+
+ def hcall_table_lookup(self, opcode: int) -> str:
+ """Lookup hcall name by opcode."""
+ return HCALL_TABLE.get(opcode, str(opcode))
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Process a single sample event."""
+ name = str(sample.evsel)
+ if name not in ("evsel(powerpc:hcall_entry)", "evsel(powerpc:hcall_exit)"):
+ return
+
+ try:
+ cpu = sample.sample_cpu
+ opcode = sample.opcode
+ except AttributeError:
+ print("ERROR: tracepoint fields missed (is libtraceevent enabled?)",
+ file=sys.stderr)
+ sys.exit(1)
+
+ # sample.sample_time represents the nsecs
+ time = sample.sample_time
+
+ if opcode < 0 or cpu < 0:
+ return
+
+ if name == "evsel(powerpc:hcall_entry)":
+ if cpu not in self.d_enter:
+ self.d_enter[cpu] = {}
+ self.d_enter[cpu][opcode] = time
+ elif name == "evsel(powerpc:hcall_exit)":
+ if cpu in self.d_enter and opcode in self.d_enter[cpu]:
+ time_entry = self.d_enter[cpu][opcode]
+ diff = time - time_entry
+ del self.d_enter[cpu][opcode]
+
+ stats = self.output[opcode]
+ stats['time'] += diff
+ stats['cnt'] += 1
+ if diff < stats['min']:
+ stats['min'] = diff
+ if diff > stats['max']:
+ stats['max'] = diff
+
+ def print_summary(self) -> None:
+ """Print aggregated statistics."""
+ print(self.print_ptrn % ('hcall', 'count', 'min(ns)', 'max(ns)', 'avg(ns)'))
+ print('-' * 68)
+
+ def sort_output(opcode):
+ stats = self.output[opcode]
+ sort_by = getattr(self.args, 'sort', None)
+ if sort_by == 'min':
+ return stats['min']
+ if sort_by == 'max':
+ return stats['max']
+ if sort_by == 'avg':
+ return stats['time'] // stats['cnt']
+ return stats['cnt']
+
+ for opcode in sorted(self.output.keys(), key=sort_output, reverse=True):
+ h_name = self.hcall_table_lookup(opcode)
+ stats = self.output[opcode]
+ time = stats['time']
+ cnt = stats['cnt']
+ min_t = stats['min']
+ max_t = stats['max']
+
+ # Avoid float representation for large integers if possible,
+ # or use formatted strings. Legacy used time//cnt.
+ avg_t = time // cnt if cnt > 0 else 0
+
+ # If min was not updated, it remains inf, but cnt should be > 0 if in output
+ if min_t == float('inf'):
+ min_t = 0
+
+ print(self.print_ptrn % (h_name, cnt, int(min_t), int(max_t), avg_t))
+
+
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser(description="Hypervisor call statistics")
+ ap.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ ap.add_argument("-s", "--sort", default="count",
+ choices=["count", "min", "max", "avg"], help="Sort key")
+ args = ap.parse_args()
+
+ analyzer = HCallAnalyzer(args)
+
+ try:
+ session = perf.session(perf.data(args.input), sample=analyzer.process_event)
+ session.process_events()
+ analyzer.print_summary()
+ except KeyboardInterrupt:
+ analyzer.print_summary()
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError) as e:
+ print(f"Error processing events: {e}")
+ sys.exit(1)
diff --git a/tools/perf/tests/shell/test_powerpc_hcalls_python.sh b/tools/perf/tests/shell/test_powerpc_hcalls_python.sh
new file mode 100755
index 000000000000..5316c28f243f
--- /dev/null
+++ b/tools/perf/tests/shell/test_powerpc_hcalls_python.sh
@@ -0,0 +1,95 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# powerpc-hcalls python test
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/powerpc-hcalls.py"
+
+if ! perf check feature -q libtraceevent > /dev/null 2>&1; then
+ echo "Skipping test, libtraceevent is disabled"
+ exit 2
+fi
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, powerpc-hcalls.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+cleanup() {
+ rm -f "${temp_data}" "${temp_out}"
+}
+trap 'cleanup' EXIT TERM INT
+
+temp_data=$(mktemp /tmp/perf.data.XXXXXX)
+temp_out=$(mktemp /tmp/perf.out.XXXXXX)
+
+echo "Testing powerpc-hcalls.py..."
+
+# Verify HCallAnalyzer aggregation logic on synthetic hcall_entry/hcall_exit samples
+if ! "$PYTHON" - "$script_path" > "${temp_out}" <<'EOF'
+import argparse
+import importlib.util
+import sys
+from types import SimpleNamespace
+
+spec = importlib.util.spec_from_file_location("powerpc_hcalls", sys.argv[1])
+assert spec and spec.loader
+mod = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(mod)
+
+analyzer = mod.HCallAnalyzer(argparse.Namespace(sort='count'))
+analyzer.process_event(SimpleNamespace(
+ evsel="evsel(powerpc:hcall_entry)", sample_cpu=0, sample_time=1000, opcode=4))
+analyzer.process_event(SimpleNamespace(
+ evsel="evsel(powerpc:hcall_exit)", sample_cpu=0, sample_time=2500, opcode=4))
+analyzer.print_summary()
+EOF
+then
+ echo "powerpc-hcalls.py synthetic aggregation test failed"
+ exit 1
+fi
+
+if ! grep -q "H_REMOVE.*1.*1500.*1500.*1500" "${temp_out}"; then
+ echo "Failed to find aggregated H_REMOVE metrics in output"
+ exit 1
+fi
+
+# Create a perf.data file if powerpc hcall tracepoints are available on this host.
+if ! perf record -e powerpc:hcall_entry,powerpc:hcall_exit -a -o "${temp_data}" \
+ -- perf test -w noploop >/dev/null 2>&1; then
+ echo "Skipping live record test, powerpc hcall tracepoints not available"
+ exit 0
+fi
+
+if [ ! -s "${temp_data}" ]; then
+ echo "Skipping live record test, perf record failed to create data"
+ exit 0
+fi
+
+# Check that the script executes on recorded perf.data
+if ! "$PYTHON" "$script_path" -i "${temp_data}" > "${temp_out}"; then
+ echo "powerpc-hcalls.py test failed"
+ err=1
+else
+ if ! grep -q "hcall.*count.*min.*max.*avg" "${temp_out}"; then
+ echo "Failed to find the metrics table header"
+ err=1
+ else
+ echo "powerpc-hcalls test passed."
+ fi
+fi
+rm -f "${temp_out}"
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 40/49] perf python: Port intel-pt-events and libxed to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (38 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 39/49] perf python: Port powerpc-hcalls " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 41/49] perf test: Migrate Intel PT virtual LBR test to Python API Ian Rogers
` (8 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port intel-pt-events.py and libxed.py from tools/perf/scripts/python/
to standalone modules in tools/perf/python/:
- Refactor intel-pt-events.py into an IntelPTAnalyzer class with full
type annotations to encapsulate trace state and stashed output.
- Configure instruction trace decoding directly via perf.session's
itrace option and context_switch callback.
- Dynamically select 32-bit vs 64-bit disassembly mode in libxed.py
using session.is_64_bit and annotate instructions with source lines
via sample.srccode().
- Rename methods in libxed.py to snake_case (instruction, set_mode,
disassemble_one) and remove Python 2 compatibility code.
Add a shell test (test_intel_pt_events_python.sh) to verify the
standalone script.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/intel-pt-events.py | 606 ++++++++++++++++++
tools/perf/python/libxed.py | 123 ++++
.../shell/test_intel_pt_events_python.sh | 69 ++
3 files changed, 798 insertions(+)
create mode 100755 tools/perf/python/intel-pt-events.py
create mode 100755 tools/perf/python/libxed.py
create mode 100755 tools/perf/tests/shell/test_intel_pt_events_python.sh
diff --git a/tools/perf/python/intel-pt-events.py b/tools/perf/python/intel-pt-events.py
new file mode 100755
index 000000000000..f5c7d81b86b9
--- /dev/null
+++ b/tools/perf/python/intel-pt-events.py
@@ -0,0 +1,606 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+Print Intel PT Events including Power Events and PTWRITE.
+Ported from tools/perf/scripts/python/intel-pt-events.py
+"""
+from __future__ import annotations
+
+import argparse
+from typing import Dict, List, Tuple
+import contextlib
+from ctypes import addressof, create_string_buffer
+import io
+import os
+import struct
+import sys
+from typing import Any, Optional
+import typing
+import perf
+
+try:
+ from libxed import LibXED as _LibXED
+ LibXED: Optional[type[_LibXED]] = _LibXED
+except ImportError:
+ LibXED = None
+
+
+def sample_flags_to_name(flags: int) -> str:
+ """Implement perf's sample_flags_to_name."""
+ if not isinstance(flags, int) or flags == 0:
+ return "".ljust(21)
+ sample_flags = [
+ ((1 << 0) | (1 << 1), "call"),
+ ((1 << 0) | (1 << 2), "return"),
+ ((1 << 0) | (1 << 3), "jcc"),
+ ((1 << 0), "jmp"),
+ ((1 << 0) | (1 << 1) | (1 << 6), "int"),
+ ((1 << 0) | (1 << 2) | (1 << 6), "iret"),
+ ((1 << 0) | (1 << 1) | (1 << 4), "syscall"),
+ ((1 << 0) | (1 << 2) | (1 << 4), "sysret"),
+ ((1 << 0) | (1 << 5), "async"),
+ ((1 << 0) | (1 << 1) | (1 << 5) | (1 << 6), "hw int"),
+ ((1 << 0) | (1 << 7), "tx abrt"),
+ ((1 << 0) | (1 << 8), "tr strt"),
+ ((1 << 0) | (1 << 9), "tr end"),
+ ((1 << 0) | (1 << 1) | (1 << 11), "vmentry"),
+ ((1 << 0) | (1 << 1) | (1 << 12), "vmexit"),
+ ]
+ additional_mask = (1 << 10) | (1 << 13) | (1 << 14)
+ branch_event_mask = (1 << 15) | (1 << 16)
+ xf = flags & additional_mask
+ rem_flags = flags & ~additional_mask
+
+ if rem_flags & (1 << 8):
+ prefix = "tr strt "
+ elif rem_flags & (1 << 9):
+ prefix = "tr end "
+ else:
+ prefix = ""
+
+ rem_flags &= ~((1 << 8) | (1 << 9))
+ types = rem_flags & ~branch_event_mask
+ type_name = ""
+ for f_mask, f_name in sample_flags:
+ if f_mask == types:
+ type_name = f_name
+ break
+
+ s = prefix + type_name
+ ev_parts = []
+ if rem_flags & (1 << 15):
+ ev_parts.append("miss")
+ if rem_flags & (1 << 16):
+ ev_parts.append("not_taken")
+ if ev_parts:
+ s += "/" + ",".join(ev_parts) + "/"
+
+ if xf:
+ xs = "("
+ if xf & (1 << 10):
+ xs += "x"
+ if xf & (1 << 13):
+ xs += "D"
+ if xf & (1 << 14):
+ xs += "t"
+ xs += ")"
+ if len(s) + len(xs) < 21:
+ s = s + xs.rjust(21 - len(s))
+ else:
+ s = s + " " + xs
+ return s.ljust(21)
+
+class IntelPTAnalyzer:
+ """Analyzes Intel PT events and prints details."""
+
+ def __init__(self, cfg: argparse.Namespace):
+ self.args = cfg
+ self.session: Optional[perf.session] = None
+ self.insn = False
+ self.src = False
+ self.source_file_name: Optional[str] = None
+ self.line_number: int = 0
+ self.dso: Optional[str] = None
+ self.stash_dict: Dict[int, List[str]] = {}
+ self.output: Any = None
+ self.output_pos: int = 0
+ self.cpu: int = -1
+ self.time: int = 0
+ self.switch_str: Dict[int, str] = {}
+
+ if cfg.insn_trace:
+ print("Intel PT Instruction Trace")
+ self.insn = True
+ elif cfg.src_trace:
+ print("Intel PT Source Trace")
+ self.insn = True
+ self.src = True
+ else:
+ print("Intel PT Branch Trace, Power Events, Event Trace and PTWRITE")
+
+ self.disassembler: Any = None
+ if self.insn and LibXED is not None:
+ try:
+ self.disassembler = LibXED()
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError,
+ ImportError) as e:
+ print(f"Failed to initialize LibXED: {e}")
+ self.disassembler = None
+
+ def print_ptwrite(self, raw_buf: bytes) -> None:
+ """Print PTWRITE data."""
+ if not raw_buf:
+ return
+ try:
+ data = struct.unpack_from("<IQ", raw_buf)
+ flags = data[0]
+ payload = data[1]
+ except struct.error:
+ return
+ exact_ip = flags & 1
+ try:
+ s = payload.to_bytes(8, "little").decode("ascii").rstrip("\x00")
+ if not s.isprintable():
+ s = ""
+ except (UnicodeDecodeError, ValueError):
+ s = ""
+ print(f"IP: {exact_ip} payload: {payload:#x} {s}", end=' ')
+
+ def print_cbr(self, raw_buf: bytes) -> None:
+ """Print CBR data."""
+ if len(raw_buf) < 12:
+ return
+ try:
+ data = struct.unpack_from("<BBBBII", raw_buf)
+ except struct.error:
+ return
+ cbr = data[0]
+ f = (data[4] + 500) // 1000
+ if data[2] == 0:
+ return
+ p = ((cbr * 1000 // data[2]) + 5) // 10
+ print(f"{cbr:3d} freq: {f:4d} MHz ({p:3d}%)", end=' ')
+
+ def print_mwait(self, raw_buf: bytes) -> None:
+ """Print MWAIT data."""
+ try:
+ data = struct.unpack_from("<IQ", raw_buf)
+ except struct.error:
+ return
+ payload = data[1]
+ hints = payload & 0xff
+ extensions = (payload >> 32) & 0x3
+ print(f"hints: {hints:#x} extensions: {extensions:#x}", end=' ')
+
+ def print_pwre(self, raw_buf: bytes) -> None:
+ """Print PWRE data."""
+ try:
+ data = struct.unpack_from("<IQ", raw_buf)
+ except struct.error:
+ return
+ payload = data[1]
+ hw = (payload >> 7) & 1
+ cstate = (payload >> 12) & 0xf
+ subcstate = (payload >> 8) & 0xf
+ print(f"hw: {hw} cstate: {cstate} sub-cstate: {subcstate}", end=' ')
+
+ def print_exstop(self, raw_buf: bytes) -> None:
+ """Print EXSTOP data."""
+ try:
+ data = struct.unpack_from("<I", raw_buf)
+ except struct.error:
+ return
+ flags = data[0]
+ exact_ip = flags & 1
+ print(f"IP: {exact_ip}", end=' ')
+
+ def print_pwrx(self, raw_buf: bytes) -> None:
+ """Print PWRX data."""
+ try:
+ data = struct.unpack_from("<IQ", raw_buf)
+ except struct.error:
+ return
+ payload = data[1]
+ deepest_cstate = payload & 0xf
+ last_cstate = (payload >> 4) & 0xf
+ wake_reason = (payload >> 8) & 0xf
+ print(f"deepest cstate: {deepest_cstate} last cstate: {last_cstate} "
+ f"wake reason: {wake_reason:#x}", end=' ')
+
+ def print_psb(self, raw_buf: bytes) -> None:
+ """Print PSB data."""
+ try:
+ data = struct.unpack_from("<IQ", raw_buf)
+ except struct.error:
+ return
+ offset = data[1]
+ print(f"offset: {offset:#x}", end=' ')
+
+ def print_evt(self, raw_buf: bytes) -> None:
+ """Print EVT data."""
+ glb_cfe = ["", "INTR", "IRET", "SMI", "RSM", "SIPI", "INIT", "VMENTRY", "VMEXIT",
+ "VMEXIT_INTR", "SHUTDOWN", "", "UINT", "UIRET"] + [""] * 18
+ glb_evd = ["", "PFA", "VMXQ", "VMXR"] + [""] * 60
+
+ try:
+ data = struct.unpack_from("<BBH", raw_buf)
+ except struct.error:
+ return
+ typ = data[0] & 0x1f
+ ip_flag = (data[0] & 0x80) >> 7
+ vector = data[1]
+ evd_cnt = data[2]
+ s = glb_cfe[typ]
+ if s:
+ print(f" cfe: {s} IP: {ip_flag} vector: {vector}", end=' ')
+ else:
+ print(f" cfe: {typ} IP: {ip_flag} vector: {vector}", end=' ')
+ pos = 4
+ for _ in range(evd_cnt):
+ if len(raw_buf) < pos + 16:
+ break
+ try:
+ data = struct.unpack_from("<QQ", raw_buf, pos)
+ except struct.error:
+ return
+ et = data[0] & 0x3f
+ s = glb_evd[et]
+ if s:
+ print(f"{s}: {data[1]:#x}", end=' ')
+ else:
+ print(f"EVD_{et}: {data[1]:#x}", end=' ')
+ pos += 16
+
+ def print_iflag(self, raw_buf: bytes) -> None:
+ """Print IFLAG data."""
+ try:
+ data = struct.unpack_from("<IQ", raw_buf)
+ except struct.error:
+ return
+ iflag = data[0] & 1
+ old_iflag = iflag ^ 1
+ via_branch = data[0] & 2
+ s = "via" if via_branch else "non"
+ print(f"IFLAG: {old_iflag}->{iflag} {s} branch", end=' ')
+
+ def common_start_str(self, comm: str, sample: perf.sample_event) -> str:
+ """Return common start string for display."""
+ ts = sample.sample_time
+ cpu = sample.sample_cpu
+ pid = sample.sample_pid
+ tid = sample.sample_tid
+ machine_pid = getattr(sample, "machine_pid", 0)
+ if machine_pid:
+ vcpu = getattr(sample, "vcpu", -1)
+ return (f"VM:{machine_pid:5d} VCPU:{vcpu:03d} {comm:>16s} {pid:5d}/{tid:<5d} "
+ f"[{cpu:03d}] {ts // 1000000000:9d}.{ts % 1000000000:09d} ")
+ return (f"{comm:>16s} {pid:5d}/{tid:<5d} [{cpu:03d}] "
+ f"{ts // 1000000000:9d}.{ts % 1000000000:09d} ")
+
+ def print_common_start(self, comm: str, sample: perf.sample_event, name: str) -> None:
+ """Print common start info."""
+ flags_disp = sample_flags_to_name(getattr(sample, "flags", 0))
+ print(self.common_start_str(comm, sample) + f"{name:>8s} {flags_disp:>21s}", end=' ')
+
+ def print_instructions_start(self, comm: str, sample: perf.sample_event) -> None:
+ """Print instructions start info."""
+ raw_flags = getattr(sample, "flags", 0)
+ if isinstance(raw_flags, int) and (raw_flags & (1 << 10)):
+ print(self.common_start_str(comm, sample) + "x", end=' ')
+ else:
+ print(self.common_start_str(comm, sample), end=' ')
+
+ def disassem(self, insn: bytes, ip: int) -> Tuple[int, str]:
+ """Disassemble instruction using LibXED."""
+ inst = self.disassembler.instruction()
+ is_64_bit = getattr(self.session, "is_64_bit", True) if self.session else True
+ self.disassembler.set_mode(inst, 0 if is_64_bit else 1)
+ buf = create_string_buffer(insn, 64)
+ return self.disassembler.disassemble_one(inst, addressof(buf), len(insn), ip)
+
+ def print_common_ip(self, sample: perf.sample_event, symbol: str, dso: str) -> None:
+ """Print IP and symbol info."""
+ ip = sample.sample_ip
+ offs = f"+{sample.sym_offset:#x}" if getattr(sample, "sym_offset", None) is not None else ""
+ cyc_cnt = getattr(sample, "sample_cyc_count", 0)
+ if cyc_cnt:
+ insn_cnt = getattr(sample, "sample_insn_count", 0)
+ ipc_str = f" IPC: {insn_cnt / cyc_cnt:#.2f} ({insn_cnt}/{cyc_cnt})"
+ else:
+ ipc_str = ""
+
+ if self.insn and self.disassembler is not None:
+ try:
+ insn = sample.insn()
+ except AttributeError:
+ insn = None
+ if insn:
+ cnt, text = self.disassem(insn, ip)
+ byte_str = (f"{ip:x}").rjust(16)
+ for k in range(cnt):
+ byte_str += f" {insn[k]:02x}"
+ print(f"{byte_str:<40s} {text:<30s}", end=' ')
+ else:
+ print(f"{ip:16x}", end=' ')
+ print(f"{symbol}{offs} ({dso})", end=' ')
+ else:
+ print(f"{ip:16x} {symbol}{offs} ({dso})", end=' ')
+
+ addr_correlates_sym = getattr(sample, "addr_symbol", None) is not None
+ if addr_correlates_sym:
+ addr = getattr(sample, "sample_addr", getattr(sample, "addr", 0))
+ addr_dso = (sample.addr_dso or '[unknown]')
+ addr_symbol = (sample.addr_symbol or '[unknown]')
+ addr_offs = (f"+{sample.addr_sym_offset:#x}"
+ if getattr(sample, "addr_sym_offset", None) is not None else "")
+ print(f"=> {addr:x} {addr_symbol}{addr_offs} ({addr_dso}){ipc_str}")
+ else:
+ print(ipc_str)
+
+ def print_srccode(self, comm: str, sample: perf.sample_event,
+ symbol: str, dso: str, with_insn: bool) -> None:
+ """Print source code info."""
+ ip = sample.sample_ip
+ if symbol == "[unknown]":
+ start_str = self.common_start_str(comm, sample) + (f"{ip:x}").rjust(16).ljust(40)
+ else:
+ offs = (f"+{sample.sym_offset:#x}"
+ if getattr(sample, "sym_offset", None) is not None else "")
+ start_str = self.common_start_str(comm, sample) + (symbol + offs).ljust(40)
+
+ if with_insn and self.insn and self.disassembler is not None:
+ try:
+ insn = sample.insn()
+ except AttributeError:
+ insn = None
+ if insn:
+ _, text = self.disassem(insn, ip)
+ start_str += text.ljust(30)
+
+ source_file_name: typing.Any = None
+ line_number: typing.Any = 0
+ source_line: typing.Any = None
+ try:
+ res_srcc = sample.srccode()
+ if res_srcc:
+ source_file_name, line_number, source_line = res_srcc
+ except (AttributeError, ValueError, TypeError):
+ pass
+
+ if source_file_name:
+ if self.line_number == line_number and self.source_file_name == source_file_name:
+ src_str = ""
+ else:
+ if len(source_file_name) > 40:
+ src_file = ("..." + source_file_name[-37:]) + " "
+ else:
+ src_file = source_file_name.ljust(41)
+ if source_line is None:
+ src_str = src_file + str(line_number).rjust(4) + " <source not found>"
+ else:
+ src_str = src_file + str(line_number).rjust(4) + " " + source_line
+ self.dso = None
+ elif dso == self.dso:
+ src_str = ""
+ else:
+ src_str = dso
+ self.dso = dso
+
+ self.line_number = line_number
+ self.source_file_name = source_file_name
+ print(start_str, src_str)
+
+ def do_process_event(self, sample: perf.sample_event) -> None:
+ """Process event and print info."""
+ cpu = getattr(sample, "sample_cpu", getattr(sample, "cpu", 0))
+ if cpu in self.switch_str:
+ print(self.switch_str[cpu])
+ del self.switch_str[cpu]
+
+ comm = "Unknown"
+ if hasattr(self, 'session') and self.session:
+ try:
+ thread = self.session.find_thread(sample.sample_pid, sample.sample_tid)
+ if thread:
+ comm = thread.comm() or "Unknown"
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError):
+ pass
+ # Python < 3.9 compatibility
+ name = str(sample.evsel)
+ if name.startswith("evsel("):
+ name = name[6:-1]
+ dso = (sample.dso or '[unknown]')
+ symbol = (sample.symbol or '[unknown]')
+
+
+ raw_buf = getattr(sample, 'raw_buf', b'') or b''
+
+ if name.startswith("instructions"):
+ if self.src:
+ self.print_srccode(comm, sample, symbol, dso, True)
+ else:
+ self.print_instructions_start(comm, sample)
+ self.print_common_ip(sample, symbol, dso)
+ elif name.startswith("branches"):
+ if self.src:
+ self.print_srccode(comm, sample, symbol, dso, False)
+ else:
+ self.print_common_start(comm, sample, name)
+ self.print_common_ip(sample, symbol, dso)
+ elif name == "ptwrite":
+ self.print_common_start(comm, sample, name)
+ self.print_ptwrite(raw_buf)
+ self.print_common_ip(sample, symbol, dso)
+ elif name == "cbr":
+ self.print_common_start(comm, sample, name)
+ self.print_cbr(raw_buf)
+ self.print_common_ip(sample, symbol, dso)
+ elif name == "mwait":
+ self.print_common_start(comm, sample, name)
+ self.print_mwait(raw_buf)
+ self.print_common_ip(sample, symbol, dso)
+ elif name == "pwre":
+ self.print_common_start(comm, sample, name)
+ self.print_pwre(raw_buf)
+ self.print_common_ip(sample, symbol, dso)
+ elif name == "exstop":
+ self.print_common_start(comm, sample, name)
+ self.print_exstop(raw_buf)
+ self.print_common_ip(sample, symbol, dso)
+ elif name == "pwrx":
+ self.print_common_start(comm, sample, name)
+ self.print_pwrx(raw_buf)
+ self.print_common_ip(sample, symbol, dso)
+ elif name == "psb":
+ self.print_common_start(comm, sample, name)
+ self.print_psb(raw_buf)
+ self.print_common_ip(sample, symbol, dso)
+ elif name == "evt":
+ self.print_common_start(comm, sample, name)
+ self.print_evt(raw_buf)
+ self.print_common_ip(sample, symbol, dso)
+ elif name == "iflag":
+ self.print_common_start(comm, sample, name)
+ self.print_iflag(raw_buf)
+ self.print_common_ip(sample, symbol, dso)
+ else:
+ self.print_common_start(comm, sample, name)
+ self.print_common_ip(sample, symbol, dso)
+
+ def interleave_events(self, sample: perf.sample_event) -> None:
+ """Interleave output to avoid garbled lines from different CPUs."""
+ self.cpu = sample.sample_cpu
+ ts = sample.sample_time
+
+ if self.time != ts:
+ self.time = ts
+ self.flush_stashed_output()
+
+ self.output_pos = 0
+ with contextlib.redirect_stdout(io.StringIO()) as self.output:
+ self.do_process_event(sample)
+
+ self.stash_output()
+
+ def stash_output(self) -> None:
+ """Stash output for later flushing."""
+ output_str = self.output.getvalue()[self.output_pos:]
+ n = len(output_str)
+ if n:
+ self.output_pos += n
+ if self.cpu not in self.stash_dict:
+ self.stash_dict[self.cpu] = []
+ self.stash_dict[self.cpu].append(output_str)
+ if len(self.stash_dict[self.cpu]) > 1000:
+ self.flush_stashed_output()
+
+ def flush_stashed_output(self) -> None:
+ """Flush stashed output."""
+ while self.stash_dict:
+ cpus = list(self.stash_dict.keys())
+ for cpu in cpus:
+ items = self.stash_dict[cpu]
+ countdown = self.args.interleave
+ while len(items) and countdown:
+ sys.stdout.write(items[0])
+ del items[0]
+ countdown -= 1
+ if not items:
+ del self.stash_dict[cpu]
+
+ def process_context_switch(self, event: perf.switch_event) -> None:
+ """Process context switch."""
+ cpu = getattr(event, "sample_cpu", getattr(event, "cpu", 0))
+ pid = getattr(event, "sample_pid", getattr(event, "pid", 0))
+ tid = getattr(event, "sample_tid", getattr(event, "tid", 0))
+ ts = getattr(event, "sample_time", getattr(event, "time", 0))
+ np_pid = getattr(event, "next_prev_pid", None)
+ np_tid = getattr(event, "next_prev_tid", None)
+ machine_pid = getattr(event, "machine_pid", -1)
+ vcpu = getattr(event, "vcpu", -1)
+ misc = getattr(event, "misc", 0)
+ out = bool(misc & (1 << 13))
+ out_preempt = bool(misc & (1 << 14))
+
+ if self.args.interleave:
+ self.flush_stashed_output()
+
+ if out:
+ out_str = "Switch out "
+ else:
+ out_str = "Switch In "
+
+ preempt_str = "preempt" if out_preempt else ""
+
+ if machine_pid == -1:
+ machine_str = ""
+ elif vcpu == -1:
+ machine_str = f"machine PID {machine_pid}"
+ else:
+ machine_str = f"machine PID {machine_pid} VCPU {vcpu}"
+
+ np_str = f"{np_pid:5d}/{np_tid:<5d} " if np_pid is not None and np_tid is not None else ""
+ c_str = (f"{out_str:>16s} {pid:5d}/{tid:<5d} [{cpu:03d}] "
+ f"{ts // 1000000000:9d}.{ts % 1000000000:09d} "
+ f"{np_str}{machine_str} {preempt_str}")
+
+ if self.args.all_switch_events:
+ print(c_str)
+ else:
+ self.switch_str[cpu] = c_str
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Wrapper to handle interleaving and exceptions."""
+ try:
+ if self.args.interleave:
+ self.interleave_events(sample)
+ else:
+ self.do_process_event(sample)
+ except BrokenPipeError:
+ # Stop python printing broken pipe errors and traceback
+ sys.stdout = open(os.devnull, 'w', encoding='utf-8')
+ sys.exit(1)
+
+
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser()
+ ap.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ ap.add_argument("--insn-trace", action='store_true')
+ ap.add_argument("--src-trace", action='store_true')
+ ap.add_argument("--all-switch-events", action='store_true')
+ ap.add_argument("--interleave", type=int, nargs='?', const=4, default=0)
+ ap.add_argument("--itrace", default=None, help="itrace options")
+ args = ap.parse_args()
+
+ if args.itrace is None:
+ if args.insn_trace or args.src_trace:
+ args.itrace = "i0nsepwxI"
+ else:
+ args.itrace = "bepwxI"
+ analyzer = IntelPTAnalyzer(args)
+
+ try:
+ # Note: Python API currently lacks auxtrace_error callbacks affecting
+ # chronological interleaving
+ session = perf.session(
+ perf.data(args.input),
+ sample=analyzer.process_event,
+ context_switch=analyzer.process_context_switch,
+ itrace=args.itrace
+ )
+ analyzer.session = session
+ session.process_events()
+ if args.interleave:
+ analyzer.flush_stashed_output()
+ print("End")
+ except KeyboardInterrupt:
+ if args.interleave:
+ analyzer.flush_stashed_output()
+ print("End")
+ except BrokenPipeError:
+ sys.exit(0)
+ except (OSError, ValueError, KeyError, RuntimeError, TypeError, AttributeError):
+ import traceback
+ traceback.print_exc()
+ sys.exit(1)
+
+
diff --git a/tools/perf/python/libxed.py b/tools/perf/python/libxed.py
new file mode 100755
index 000000000000..900bd3e0184b
--- /dev/null
+++ b/tools/perf/python/libxed.py
@@ -0,0 +1,123 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+Python wrapper for libxed.so
+Ported from tools/perf/scripts/python/libxed.py
+"""
+from __future__ import annotations
+
+from ctypes import CDLL, Structure, create_string_buffer, addressof, sizeof, \
+ c_void_p, c_byte, c_int, c_uint, c_ulonglong
+
+# To use Intel XED, libxed.so must be present. To build and install
+# libxed.so:
+# git clone https://github.com/intelxed/mbuild.git mbuild
+# git clone https://github.com/intelxed/xed
+# cd xed
+# ./mfile.py --share
+# sudo ./mfile.py --prefix=/usr/local install
+# sudo ldconfig
+#
+
+
+class XedStateT(Structure):
+ """xed_state_t structure."""
+ _fields_ = [
+ ("mode", c_int),
+ ("width", c_int)
+ ]
+
+
+class XEDInstruction():
+ """Represents a decoded instruction."""
+
+ def __init__(self, libxed):
+ # Current xed_decoded_inst_t structure is 192 bytes. Use 512 to allow for future expansion
+ xedd_t = c_byte * 512
+ self.xedd = xedd_t()
+ self.xedp = addressof(self.xedd)
+ libxed.xed_decoded_inst_zero(self.xedp)
+ self.state = XedStateT()
+ self.statep = addressof(self.state)
+ # Buffer for disassembled instruction text
+ self.buffer = create_string_buffer(256)
+ self.bufferp = addressof(self.buffer)
+
+
+class LibXED():
+ """Wrapper for libxed.so."""
+
+ def __init__(self):
+ try:
+ self.libxed = CDLL("libxed.so")
+ except OSError:
+ self.libxed = None
+ if not self.libxed:
+ try:
+ self.libxed = CDLL("/usr/local/lib/libxed.so")
+ except OSError:
+ self.libxed = None
+
+ if not self.libxed:
+ raise ImportError("libxed.so not found. Please install Intel XED.")
+
+ self.xed_tables_init = self.libxed.xed_tables_init
+ self.xed_tables_init.restype = None
+ self.xed_tables_init.argtypes = []
+
+ self.xed_decoded_inst_zero = self.libxed.xed_decoded_inst_zero
+ self.xed_decoded_inst_zero.restype = None
+ self.xed_decoded_inst_zero.argtypes = [c_void_p]
+
+ self.xed_operand_values_set_mode = self.libxed.xed_operand_values_set_mode
+ self.xed_operand_values_set_mode.restype = None
+ self.xed_operand_values_set_mode.argtypes = [c_void_p, c_void_p]
+
+ self.xed_decoded_inst_zero_keep_mode = self.libxed.xed_decoded_inst_zero_keep_mode
+ self.xed_decoded_inst_zero_keep_mode.restype = None
+ self.xed_decoded_inst_zero_keep_mode.argtypes = [c_void_p]
+
+ self.xed_decode = self.libxed.xed_decode
+ self.xed_decode.restype = c_int
+ self.xed_decode.argtypes = [c_void_p, c_void_p, c_uint]
+
+ self.xed_format_context = self.libxed.xed_format_context
+ self.xed_format_context.restype = c_uint
+ self.xed_format_context.argtypes = [
+ c_int, c_void_p, c_void_p, c_int, c_ulonglong, c_void_p, c_void_p
+ ]
+
+ self.xed_decoded_inst_get_length = self.libxed.xed_decoded_inst_get_length
+ self.xed_decoded_inst_get_length.restype = c_uint
+ self.xed_decoded_inst_get_length.argtypes = [c_void_p]
+
+ self.xed_tables_init()
+
+ def instruction(self):
+ """Create a new XEDInstruction."""
+ return XEDInstruction(self)
+
+ def set_mode(self, inst, mode):
+ """Set 32-bit or 64-bit mode."""
+ if mode:
+ inst.state.mode = 4 # 32-bit
+ inst.state.width = 4 # 4 bytes
+ else:
+ inst.state.mode = 1 # 64-bit
+ inst.state.width = 8 # 8 bytes
+ self.xed_operand_values_set_mode(inst.xedp, inst.statep)
+
+ def disassemble_one(self, inst, bytes_ptr, bytes_cnt, ip):
+ """Disassemble one instruction."""
+ self.xed_decoded_inst_zero_keep_mode(inst.xedp)
+ err = self.xed_decode(inst.xedp, bytes_ptr, bytes_cnt)
+ if err:
+ return 0, ""
+ # Use AT&T mode (2), alternative is Intel (3)
+ ok = self.xed_format_context(2, inst.xedp, inst.bufferp, sizeof(inst.buffer), ip, 0, 0)
+ if not ok:
+ return 0, ""
+
+ result = inst.buffer.value.decode('utf-8')
+ # Return instruction length and the disassembled instruction text
+ return self.xed_decoded_inst_get_length(inst.xedp), result
diff --git a/tools/perf/tests/shell/test_intel_pt_events_python.sh b/tools/perf/tests/shell/test_intel_pt_events_python.sh
new file mode 100755
index 000000000000..0d397792ced8
--- /dev/null
+++ b/tools/perf/tests/shell/test_intel_pt_events_python.sh
@@ -0,0 +1,69 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# intel-pt-events python test (exclusive)
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/intel-pt-events.py"
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, intel-pt-events.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_dir=""
+temp_data=""
+temp_out=""
+
+cleanup() {
+ [ -n "${temp_dir}" ] && rm -rf "${temp_dir}"
+}
+
+trap 'cleanup' EXIT TERM INT
+
+temp_dir=$(mktemp -d /tmp/perf.ipt.XXXXXX)
+temp_data="${temp_dir}/perf.data"
+temp_out="${temp_dir}/perf.out"
+
+test_intel_pt() {
+ echo "Testing intel-pt-events.py with intel_pt..."
+
+ rm -f "${temp_data}" "${temp_out}"
+ # Generate some intel_pt events; use a subshell that waits for uname so
+ # uname's AUX buffer is flushed before the parent workload exits.
+ if ! perf record -B -N --no-bpf-event -e intel_pt//u -o "${temp_data}" \
+ -- sh -c "uname; true" >/dev/null 2>&1; then
+ echo "Skipping intel_pt test, intel_pt not available."
+ exit 2
+ fi
+
+ # Run the script and check output
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" > "${temp_out}"; then
+ echo "intel-pt-events.py test failed."
+ err=1
+ else
+ if ! grep -q "Intel PT Branch Trace" "${temp_out}" || \
+ ! grep -q "uname" "${temp_out}"; then
+ echo "Failed to find expected output: $(cat "${temp_out}")"
+ err=1
+ else
+ echo "intel-pt-events test passed."
+ fi
+ fi
+ rm -f "${temp_out}"
+}
+
+test_intel_pt
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 41/49] perf test: Migrate Intel PT virtual LBR test to Python API
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (39 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 40/49] perf python: Port intel-pt-events and libxed " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 42/49] perf python: Port export-to-sqlite to perf module Ian Rogers
` (7 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Migrate the Intel PT virtual LBR test from generating an inline legacy
perf script callback to using a standalone Python script
(perf_brstack_max.py) with the brstack iterator API in the perf Python
module.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
.../perf/tests/shell/lib/perf_brstack_max.py | 37 +++++++++++++++++
tools/perf/tests/shell/test_intel_pt.sh | 41 ++++++++-----------
2 files changed, 54 insertions(+), 24 deletions(-)
create mode 100644 tools/perf/tests/shell/lib/perf_brstack_max.py
diff --git a/tools/perf/tests/shell/lib/perf_brstack_max.py b/tools/perf/tests/shell/lib/perf_brstack_max.py
new file mode 100644
index 000000000000..e2f5362de5e6
--- /dev/null
+++ b/tools/perf/tests/shell/lib/perf_brstack_max.py
@@ -0,0 +1,37 @@
+#!/usr/bin/python
+# SPDX-License-Identifier: GPL-2.0
+# Determine the maximum size of branch stacks in a perf.data file.
+
+import argparse
+import sys
+
+import perf
+
+def main():
+ ap = argparse.ArgumentParser()
+ ap.add_argument("-i", "--input", default="perf.data", help="Input file name")
+ args = ap.parse_args()
+
+ bmax = 0
+
+ def process_event(sample):
+ nonlocal bmax
+ try:
+ brstack = sample.brstack
+ if brstack:
+ n = len(list(brstack))
+ if n > bmax:
+ bmax = n
+ except AttributeError:
+ pass
+
+ try:
+ session = perf.session(perf.data(args.input), sample=process_event)
+ session.process_events()
+ print("max brstack", bmax)
+ except Exception as e:
+ print(f"Error processing events: {e}", file=sys.stderr)
+ sys.exit(1)
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/perf/tests/shell/test_intel_pt.sh b/tools/perf/tests/shell/test_intel_pt.sh
index 26243ff760ec..e512d0c88dd0 100755
--- a/tools/perf/tests/shell/test_intel_pt.sh
+++ b/tools/perf/tests/shell/test_intel_pt.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-# Miscellaneous Intel PT testing (exclusive)
# SPDX-License-Identifier: GPL-2.0
+# Miscellaneous Intel PT testing (exclusive)
set -e
@@ -22,7 +22,7 @@ perfdatafile="${temp_dir}/test-perf.data"
outfile="${temp_dir}/test-out.txt"
errfile="${temp_dir}/test-err.txt"
awkscript="${temp_dir}/awkscript"
-maxbrstack="${temp_dir}/maxbrstack.py"
+
cleanup()
{
@@ -376,34 +376,27 @@ test_kernel_trace()
test_virtual_lbr()
{
echo "--- Test virtual LBR ---"
- # Check if python script is supported
- libpython=$(perf version --build-options | grep python | grep -cv OFF)
- if [ "${libpython}" != "1" ] ; then
- echo "SKIP: python scripting is not supported"
+ # Check if python is available
+ if ! command -v python3 >/dev/null 2>&1 && ! command -v python >/dev/null 2>&1; then
+ echo "SKIP: python not found"
return 2
fi
- # Python script to determine the maximum size of branch stacks
- cat << "_end_of_file_" > "${maxbrstack}"
-from __future__ import print_function
-
-bmax = 0
+ # shellcheck source=lib/setup_python.sh
+ . "$(dirname "$0")"/lib/setup_python.sh
-def process_event(param_dict):
- if "brstack" in param_dict:
- brstack = param_dict["brstack"]
- n = len(brstack)
- global bmax
- if n > bmax:
- bmax = n
-
-def trace_end():
- print("max brstack", bmax)
-_end_of_file_
+ if ! $PYTHON -c 'import perf' > /dev/null 2>&1; then
+ echo "SKIP: Python perf module not found"
+ return 2
+ fi
# Check if virtual lbr is working
- perf_record_no_bpf -o "${perfdatafile}" --aux-sample -e '{intel_pt//,cycles}:u' uname
- times_val=$(perf script -i "${perfdatafile}" --itrace=L -s "${maxbrstack}" 2>/dev/null | grep "max brstack " | cut -d " " -f 3)
+ perf_record_no_bpf -o "${tmpfile}" --aux-sample \
+ -e '{intel_pt//,cycles}:u' perf test -w brstack
+ perf inject --itrace=L -i "${tmpfile}" -o "${perfdatafile}"
+ output=$($PYTHON "$(dirname "$0")"/lib/perf_brstack_max.py -i "${perfdatafile}")
+ echo "Debug: perf_brstack_max.py output: $output"
+ times_val=$(echo "$output" | grep "max brstack " | cut -d " " -f 3)
case "${times_val}" in
[0-9]*) ;;
*) times_val=0;;
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 42/49] perf python: Port export-to-sqlite to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (40 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 41/49] perf test: Migrate Intel PT virtual LBR test to Python API Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 43/49] perf python: Port export-to-postgresql " Ian Rogers
` (6 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port export-to-sqlite.py to a standalone script in tools/perf/python/
using the perf module and Python's standard library sqlite3 module.
Improvements compared to the legacy script:
- Remove the dependency on PySide/QtSql by using Python's built-in
sqlite3 module in DatabaseExporter, allowing SQLite export on headless
and minimal systems without Qt installed.
- Support Intel PT and hardware instruction trace export via
perf.session(itrace=...) and perf.call_return callbacks, reconstructing
relational call_paths and calls tables and decoding synthesized PT
payloads (ptwrite, cbr, mwait, pwre, exstop, pwrx).
- Export context_switches via perf.session's context_switch callback
and map thread and comm IDs to their relational database keys.
- Manage temporary staging files inside an isolated tempfile.mkdtemp()
directory with guaranteed cleanup in a finally block.
Update Documentation/db-export.txt and add a shell test
(test_export_to_sqlite_python.sh) to verify the standalone exporter.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/Documentation/db-export.txt | 2 +-
tools/perf/python/export-to-sqlite.py | 893 ++++++++++++++++++
.../shell/test_export_to_sqlite_python.sh | 108 +++
3 files changed, 1002 insertions(+), 1 deletion(-)
create mode 100755 tools/perf/python/export-to-sqlite.py
create mode 100755 tools/perf/tests/shell/test_export_to_sqlite_python.sh
diff --git a/tools/perf/Documentation/db-export.txt b/tools/perf/Documentation/db-export.txt
index 52ffccb02d55..b43f12b96973 100644
--- a/tools/perf/Documentation/db-export.txt
+++ b/tools/perf/Documentation/db-export.txt
@@ -7,7 +7,7 @@ perf tool's python scripting engine:
supports scripts:
- tools/perf/scripts/python/export-to-sqlite.py
+ tools/perf/python/export-to-sqlite.py
tools/perf/scripts/python/export-to-postgresql.py
which export data to a SQLite3 or PostgreSQL database.
diff --git a/tools/perf/python/export-to-sqlite.py b/tools/perf/python/export-to-sqlite.py
new file mode 100755
index 000000000000..34a4840b42df
--- /dev/null
+++ b/tools/perf/python/export-to-sqlite.py
@@ -0,0 +1,893 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+Export perf data to a sqlite3 database.
+
+This script has been ported to use the modern perf Python module and the
+standard library sqlite3 module. It no longer requires PySide2 or QtSql
+for exporting.
+
+Examples of using this script with Intel PT:
+
+ $ perf record -e intel_pt//u ls
+ $ python export-to-sqlite.py -i perf.data -o pt_example
+
+To browse the database, sqlite3 can be used e.g.
+
+ $ sqlite3 pt_example
+ sqlite> .header on
+ sqlite> select * from samples_view where id < 10;
+ sqlite> .mode column
+ sqlite> select * from samples_view where id < 10;
+ sqlite> .tables
+ sqlite> .schema samples_view
+ sqlite> .quit
+
+An example of using the database is provided by the script
+exported-sql-viewer.py. Refer to that script for details.
+
+Ported from tools/perf/scripts/python/export-to-sqlite.py
+"""
+
+from __future__ import annotations
+import typing
+import argparse
+import os
+import shutil
+import sqlite3
+import struct
+import sys
+import tempfile
+from typing import Dict, Optional
+import perf
+
+
+class DatabaseExporter:
+ """Handles database connection and exporting of perf events."""
+
+ def __init__(self, db_path: str):
+ self.con = sqlite3.connect(db_path)
+ self.con.execute("PRAGMA journal_mode = MEMORY")
+ self.session: Optional[perf.session] = None
+ self.sample_count = 0
+
+ # Caches and counters grouped to reduce instance attributes
+ self.caches: Dict[str, dict] = {
+ 'machines': {},
+ 'threads': {},
+ 'comms': {},
+ 'dsos': {},
+ 'symbols': {},
+ 'events': {},
+ 'branch_types': {},
+ 'call_paths': {}
+ }
+
+ self.next_id = {
+ 'machine': 1,
+ 'thread': 1,
+ 'comm': 1,
+ 'dso': 1,
+ 'symbol': 1,
+ 'event': 1,
+ 'branch_type': 1,
+ 'call_path': 1
+ }
+
+ self.create_tables()
+
+ def create_tables(self) -> None:
+ """Create database tables."""
+ self.con.execute("""
+ CREATE TABLE IF NOT EXISTS selected_events (
+ id INTEGER NOT NULL PRIMARY KEY,
+ name VARCHAR(80))
+ """)
+ self.con.execute("""
+ CREATE TABLE IF NOT EXISTS machines (
+ id INTEGER NOT NULL PRIMARY KEY,
+ pid INTEGER,
+ root_dir VARCHAR(4096))
+ """)
+ self.con.execute("""
+ CREATE TABLE IF NOT EXISTS threads (
+ id INTEGER NOT NULL PRIMARY KEY,
+ machine_id BIGINT,
+ process_id BIGINT,
+ pid INTEGER,
+ tid INTEGER)
+ """)
+ self.con.execute("""
+ CREATE TABLE IF NOT EXISTS comms (
+ id INTEGER NOT NULL PRIMARY KEY,
+ comm VARCHAR(16),
+ c_thread_id BIGINT,
+ c_time BIGINT,
+ exec_flag BOOLEAN)
+ """)
+ self.con.execute("""
+ CREATE TABLE IF NOT EXISTS comm_threads (
+ id INTEGER NOT NULL PRIMARY KEY,
+ comm_id BIGINT,
+ thread_id BIGINT)
+ """)
+ self.con.execute("""
+ CREATE TABLE IF NOT EXISTS dsos (
+ id INTEGER NOT NULL PRIMARY KEY,
+ machine_id BIGINT,
+ short_name VARCHAR(256),
+ long_name VARCHAR(4096),
+ build_id VARCHAR(64))
+ """)
+ self.con.execute("""
+ CREATE TABLE IF NOT EXISTS symbols (
+ id INTEGER NOT NULL PRIMARY KEY,
+ dso_id BIGINT,
+ sym_start BIGINT,
+ sym_end BIGINT,
+ binding INTEGER,
+ name VARCHAR(2048))
+ """)
+ self.con.execute("""
+ CREATE TABLE IF NOT EXISTS branch_types (
+ id INTEGER NOT NULL PRIMARY KEY,
+ name VARCHAR(80))
+ """)
+ self.con.execute("""
+ CREATE TABLE IF NOT EXISTS samples (
+ id INTEGER NOT NULL PRIMARY KEY,
+ evsel_id BIGINT,
+ machine_id BIGINT,
+ thread_id BIGINT,
+ comm_id BIGINT,
+ dso_id BIGINT,
+ symbol_id BIGINT,
+ sym_offset BIGINT,
+ ip BIGINT,
+ time BIGINT,
+ cpu INTEGER,
+ to_dso_id BIGINT,
+ to_symbol_id BIGINT,
+ to_sym_offset BIGINT,
+ to_ip BIGINT,
+ period BIGINT,
+ weight BIGINT,
+ transaction_ BIGINT,
+ data_src BIGINT,
+ branch_type INTEGER,
+ in_tx BOOLEAN,
+ call_path_id BIGINT,
+ insn_count BIGINT,
+ cyc_count BIGINT,
+ flags INTEGER)
+ """)
+ self.con.execute('''
+ CREATE TABLE IF NOT EXISTS calls (
+ id INTEGER NOT NULL PRIMARY KEY,
+ thread_id BIGINT,
+ comm_id BIGINT,
+ call_path_id BIGINT,
+ call_time BIGINT,
+ return_time BIGINT,
+ branch_count BIGINT,
+ call_id BIGINT,
+ return_id BIGINT,
+ parent_call_path_id BIGINT,
+ flags INTEGER,
+ parent_id BIGINT,
+ insn_count BIGINT,
+ cyc_count BIGINT
+ )
+ ''')
+ self.con.execute('''
+ CREATE TABLE IF NOT EXISTS call_paths (
+ id INTEGER NOT NULL PRIMARY KEY,
+ parent_id BIGINT,
+ symbol_id BIGINT,
+ ip BIGINT
+ )
+ ''')
+
+ self.con.execute('''
+ CREATE TABLE IF NOT EXISTS cbr (
+ id integer primary key,
+ cbr integer,
+ mhz integer,
+ percent integer
+ )
+ ''')
+ self.con.execute('''
+ CREATE TABLE IF NOT EXISTS mwait (
+ id integer primary key,
+ hints integer,
+ extensions integer
+ )
+ ''')
+
+
+
+ self.con.execute('''
+ CREATE TABLE IF NOT EXISTS context_switches (
+ id integer primary key,
+ machine_id bigint,
+ time bigint,
+ cpu integer,
+ thread_out_id bigint,
+ comm_out_id bigint,
+ thread_in_id bigint,
+ comm_in_id bigint,
+ flags integer
+ )
+ ''')
+ self.con.execute(
+ "CREATE TABLE IF NOT EXISTS ptwrite ("
+ "id integer primary key, payload integer, exact_ip integer"
+ ")"
+ )
+ self.con.execute(
+ "CREATE TABLE IF NOT EXISTS pwre ("
+ "id integer primary key, hw integer, cstate integer, subcstate integer, "
+ "hw_name text, cstate_name text"
+ ")"
+ )
+ self.con.execute(
+ "CREATE TABLE IF NOT EXISTS exstop ("
+ "id integer primary key, exact_ip integer"
+ ")"
+ )
+ self.con.execute(
+ "CREATE TABLE IF NOT EXISTS pwrx ("
+ "id integer primary key, deepest_cstate integer, last_cstate integer, "
+ "wake_reason integer"
+ ")"
+ )
+
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS machines_view AS "
+ "SELECT id, pid, root_dir, "
+ "CASE WHEN id=0 THEN 'unknown' WHEN pid=-1 THEN 'host' ELSE 'guest' END "
+ "AS host_or_guest FROM machines"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS dsos_view AS "
+ "SELECT id, machine_id, "
+ "(SELECT host_or_guest FROM machines_view WHERE id = machine_id) AS host_or_guest, "
+ "short_name, long_name, build_id FROM dsos"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS symbols_view AS "
+ "SELECT id, name, (SELECT short_name FROM dsos WHERE id=dso_id) AS dso, "
+ "dso_id, sym_start, sym_end, "
+ "CASE WHEN binding=0 THEN 'local' WHEN binding=1 THEN 'global' ELSE 'weak' END "
+ "AS binding FROM symbols"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS threads_view AS "
+ "SELECT id, machine_id, "
+ "(SELECT host_or_guest FROM machines_view WHERE id = machine_id) AS host_or_guest, "
+ "process_id, pid, tid FROM threads"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS comm_threads_view AS "
+ "SELECT comm_id, (SELECT comm FROM comms WHERE id = comm_id) AS command, "
+ "thread_id, (SELECT pid FROM threads WHERE id = thread_id) AS pid, "
+ "(SELECT tid FROM threads WHERE id = thread_id) AS tid FROM comm_threads"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS call_paths_view AS "
+ "SELECT c.id, printf('%x', c.ip) AS ip, c.symbol_id, "
+ "(SELECT name FROM symbols WHERE id = c.symbol_id) AS symbol, "
+ "(SELECT dso_id FROM symbols WHERE id = c.symbol_id) AS dso_id, "
+ "(SELECT dso FROM symbols_view WHERE id = c.symbol_id) AS dso_short_name, "
+ "c.parent_id, printf('%x', p.ip) AS parent_ip, p.symbol_id AS parent_symbol_id, "
+ "(SELECT name FROM symbols WHERE id = p.symbol_id) AS parent_symbol, "
+ "(SELECT dso_id FROM symbols WHERE id = p.symbol_id) AS parent_dso_id, "
+ "(SELECT dso FROM symbols_view WHERE id = p.symbol_id) AS parent_dso_short_name "
+ "FROM call_paths c LEFT JOIN call_paths p ON p.id = c.parent_id"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS calls_view AS "
+ "SELECT calls.id, thread_id, "
+ "(SELECT pid FROM threads WHERE id = thread_id) AS pid, "
+ "(SELECT tid FROM threads WHERE id = thread_id) AS tid, "
+ "(SELECT comm FROM comms WHERE id = comm_id) AS command, "
+ "call_path_id, printf('%x', ip) AS ip, symbol_id, "
+ "(SELECT name FROM symbols WHERE id = symbol_id) AS symbol, "
+ "call_time, return_time, return_time - call_time AS elapsed_time, "
+ "branch_count, insn_count, cyc_count, "
+ "CASE WHEN cyc_count=0 THEN CAST(0 AS FLOAT) "
+ "ELSE CAST(insn_count AS FLOAT) / cyc_count END AS IPC, "
+ "call_id, return_id, "
+ "CASE WHEN flags=0 THEN '' WHEN flags=1 THEN 'no call' "
+ "WHEN flags=2 THEN 'no return' WHEN flags=3 THEN 'no call/return' "
+ "WHEN flags=6 THEN 'jmp' ELSE flags END AS flags, "
+ "parent_call_path_id, calls.parent_id "
+ "FROM calls INNER JOIN call_paths ON call_paths.id = call_path_id"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS samples_view AS "
+ "SELECT id, time, cpu, "
+ "(SELECT pid FROM threads WHERE id = thread_id) AS pid, "
+ "(SELECT tid FROM threads WHERE id = thread_id) AS tid, "
+ "(SELECT comm FROM comms WHERE id = comm_id) AS command, "
+ "(SELECT name FROM selected_events WHERE id = evsel_id) AS event, "
+ "printf('%x', ip) AS ip_hex, "
+ "(SELECT name FROM symbols WHERE id = symbol_id) AS symbol, sym_offset, "
+ "(SELECT short_name FROM dsos WHERE id = dso_id) AS dso_short_name, "
+ "printf('%x', to_ip) AS to_ip_hex, "
+ "(SELECT name FROM symbols WHERE id = to_symbol_id) AS to_symbol, to_sym_offset, "
+ "(SELECT short_name FROM dsos WHERE id = to_dso_id) AS to_dso_short_name, "
+ "CASE WHEN branch_type=1 THEN 'jmp' WHEN branch_type=3 THEN 'call' "
+ "WHEN branch_type=5 THEN 'return' WHEN branch_type=9 THEN 'jcc' "
+ "WHEN (branch_type & 64) THEN 'interrupt' "
+ "WHEN (branch_type & 128) THEN 'tx abort' "
+ "WHEN (branch_type & 256) THEN 'trace begin' "
+ "WHEN (branch_type & 512) THEN 'trace end' "
+ "WHEN (branch_type & 2048) THEN 'vmentry' "
+ "WHEN (branch_type & 4096) THEN 'vmexit' "
+ "ELSE branch_type END AS branch_type_name, "
+ "in_tx, call_path_id, insn_count, cyc_count, "
+ "CASE WHEN cyc_count=0 THEN CAST(0 AS FLOAT) "
+ "ELSE CAST(insn_count AS FLOAT) / cyc_count END AS IPC, flags FROM samples"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS ptwrite_view AS "
+ "SELECT ptwrite.id, time, cpu, printf('%x', payload) AS payload_hex, "
+ "CASE WHEN exact_ip=0 THEN 'False' ELSE 'True' END AS exact_ip "
+ "FROM ptwrite INNER JOIN samples ON samples.id = ptwrite.id"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS cbr_view AS "
+ "SELECT cbr.id, time, cpu, cbr, mhz, percent "
+ "FROM cbr INNER JOIN samples ON samples.id = cbr.id"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS mwait_view AS "
+ "SELECT mwait.id, time, cpu, printf('%x', hints) AS hints_hex, "
+ "printf('%x', extensions) AS extensions_hex "
+ "FROM mwait INNER JOIN samples ON samples.id = mwait.id"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS pwre_view AS "
+ "SELECT pwre.id, time, cpu, cstate, subcstate, "
+ "CASE WHEN hw=0 THEN 'False' ELSE 'True' END AS hw "
+ "FROM pwre INNER JOIN samples ON samples.id = pwre.id"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS exstop_view AS "
+ "SELECT exstop.id, time, cpu, "
+ "CASE WHEN exact_ip=0 THEN 'False' ELSE 'True' END AS exact_ip "
+ "FROM exstop INNER JOIN samples ON samples.id = exstop.id"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS pwrx_view AS "
+ "SELECT pwrx.id, time, cpu, deepest_cstate, last_cstate, "
+ "CASE WHEN wake_reason=1 THEN 'Interrupt' "
+ "WHEN wake_reason=2 THEN 'Timer Deadline' "
+ "WHEN wake_reason=4 THEN 'Monitored Address' "
+ "WHEN wake_reason=8 THEN 'HW' WHEN wake_reason=16 THEN 'Other' "
+ "ELSE wake_reason END AS wake_reason "
+ "FROM pwrx INNER JOIN samples ON samples.id = pwrx.id"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS power_events_view AS "
+ "SELECT samples.id, time, cpu, selected_events.name AS event, "
+ "CASE WHEN selected_events.name='cbr' THEN "
+ "(SELECT cbr FROM cbr WHERE cbr.id = samples.id) ELSE \"\" END AS cbr, "
+ "CASE WHEN selected_events.name='cbr' THEN "
+ "(SELECT mhz FROM cbr WHERE cbr.id = samples.id) ELSE \"\" END AS mhz, "
+ "CASE WHEN selected_events.name='cbr' THEN "
+ "(SELECT percent FROM cbr WHERE cbr.id = samples.id) ELSE \"\" END AS percent, "
+ "CASE WHEN selected_events.name='mwait' THEN "
+ "(SELECT printf('%x', hints) FROM mwait WHERE mwait.id = samples.id) "
+ "ELSE \"\" END AS hints_hex, "
+ "CASE WHEN selected_events.name='mwait' THEN "
+ "(SELECT printf('%x', extensions) FROM mwait WHERE mwait.id = samples.id) "
+ "ELSE \"\" END AS extensions_hex, "
+ "CASE WHEN selected_events.name='pwre' THEN "
+ "(SELECT cstate FROM pwre WHERE pwre.id = samples.id) ELSE \"\" END AS cstate, "
+ "CASE WHEN selected_events.name='pwre' THEN "
+ "(SELECT subcstate FROM pwre WHERE pwre.id = samples.id) ELSE \"\" END AS subcstate, "
+ "CASE WHEN selected_events.name='pwre' THEN "
+ "(SELECT hw FROM pwre WHERE pwre.id = samples.id) ELSE \"\" END AS hw, "
+ "CASE WHEN selected_events.name='exstop' THEN "
+ "(SELECT exact_ip FROM exstop WHERE exstop.id = samples.id) "
+ "ELSE \"\" END AS exact_ip, "
+ "CASE WHEN selected_events.name='pwrx' THEN "
+ "(SELECT deepest_cstate FROM pwrx WHERE pwrx.id = samples.id) "
+ "ELSE \"\" END AS deepest_cstate, "
+ "CASE WHEN selected_events.name='pwrx' THEN "
+ "(SELECT last_cstate FROM pwrx WHERE pwrx.id = samples.id) "
+ "ELSE \"\" END AS last_cstate, "
+ "CASE WHEN selected_events.name='pwrx' THEN "
+ "(SELECT wake_reason FROM pwrx WHERE pwrx.id = samples.id) "
+ "ELSE \"\" END AS wake_reason "
+ "FROM samples INNER JOIN selected_events ON selected_events.id = evsel_id "
+ "WHERE selected_events.name IN ('cbr','mwait','pwre','exstop','pwrx')"
+ )
+ self.con.execute(
+ "CREATE VIEW IF NOT EXISTS context_switches_view AS "
+ "SELECT context_switches.id, context_switches.machine_id, "
+ "context_switches.time, context_switches.cpu, "
+ "th_out.pid AS pid_out, th_out.tid AS tid_out, comm_out.comm AS comm_out, "
+ "th_in.pid AS pid_in, th_in.tid AS tid_in, comm_in.comm AS comm_in, "
+ "CASE WHEN flags=0 THEN 'in' WHEN flags=1 THEN 'out' "
+ "WHEN flags=3 THEN 'out preempt' ELSE flags END AS flags "
+ "FROM context_switches "
+ "INNER JOIN threads AS th_out ON th_out.id = context_switches.thread_out_id "
+ "INNER JOIN threads AS th_in ON th_in.id = context_switches.thread_in_id "
+ "INNER JOIN comms AS comm_out ON comm_out.id = context_switches.comm_out_id "
+ "INNER JOIN comms AS comm_in ON comm_in.id = context_switches.comm_in_id"
+ )
+
+
+
+
+
+ # id == 0 means unknown. It is easier to create records for them than
+ # replace the zeroes with NULLs
+ self.con.execute("INSERT OR IGNORE INTO selected_events VALUES (0, 'unknown')")
+ self.con.execute("INSERT OR IGNORE INTO machines VALUES (0, 0, 'unknown')")
+ self.con.execute("INSERT OR IGNORE INTO threads VALUES (0, 0, 0, -1, -1)")
+ self.con.execute("INSERT OR IGNORE INTO comms VALUES (0, 'unknown', 0, 0, 0)")
+ self.con.execute("INSERT OR IGNORE INTO dsos VALUES (0, 0, 'unknown', 'unknown', '')")
+ self.con.execute("INSERT OR IGNORE INTO call_paths VALUES (0, 0, 0, 0)")
+ self.con.execute("INSERT OR IGNORE INTO symbols VALUES (0, 0, 0, 0, 0, 'unknown')")
+
+ self.caches['events']['unknown'] = 0
+ self.caches['threads'][(0, -1, -1)] = 0
+ self.caches['threads'][(0, 0, 0)] = 0
+ self.caches['comms'][('unknown', 0)] = 0
+ self.caches['dsos'][(0, 'unknown', 'unknown', '')] = 0
+ self.caches['symbols'][(0, 'unknown', 0, 0)] = 0
+ # Initialize comm_threads mapping
+ self.comm_threads_cache: set[tuple[int, int]] = set()
+ self.next_comm_thread_id = 1
+
+ def _exec(self, sql: str, params: tuple[typing.Any, ...] = ()) -> sqlite3.Cursor:
+ """Execute SQL statement converting unsigned 64-bit ints to signed 64-bit."""
+ conv_params = tuple(
+ p - 0x10000000000000000 if isinstance(p, int) and p >= 0x8000000000000000 else p
+ for p in params
+ )
+ return self.con.execute(sql, conv_params)
+
+ def get_machine_id(self, machine_pid: Optional[int]) -> int:
+ """Get or create machine ID."""
+ if machine_pid is None or machine_pid <= 0 or machine_pid > 0x7fffffff:
+ machine_pid = -1
+ if machine_pid in self.caches['machines']:
+ return self.caches['machines'][machine_pid]
+ machine_id = self.next_id['machine']
+ self.next_id['machine'] += 1
+ self._exec("INSERT INTO machines VALUES (?, ?, ?)",
+ (machine_id, machine_pid, ""))
+ self.caches['machines'][machine_pid] = machine_id
+ return machine_id
+
+ def get_event_id(self, name: str) -> int:
+ """Get or create event ID."""
+ if name in self.caches['events']:
+ return self.caches['events'][name]
+ event_id = self.next_id['event']
+ self._exec("INSERT INTO selected_events VALUES (?, ?)",
+ (event_id, name))
+ self.caches['events'][name] = event_id
+ self.next_id['event'] += 1
+ return event_id
+
+ def get_thread_id(self, machine_id: int, pid: Optional[int], tid: Optional[int]) -> int:
+ """Get or create thread ID."""
+ if pid is None:
+ pid = -1
+ elif pid > 0x7fffffff:
+ pid -= 0x100000000
+ if tid is None:
+ tid = -1
+ elif tid > 0x7fffffff:
+ tid -= 0x100000000
+ key = (machine_id, pid, tid)
+ if key in self.caches['threads']:
+ return self.caches['threads'][key]
+ thread_id = self.next_id['thread']
+ self.next_id['thread'] += 1
+ self.caches['threads'][key] = thread_id
+ process_id = thread_id if pid == tid else self.get_thread_id(machine_id, pid, pid)
+ self._exec("INSERT INTO threads VALUES (?, ?, ?, ?, ?)",
+ (thread_id, machine_id, process_id, pid, tid))
+ return thread_id
+
+ def get_comm_id(self, comm: str, thread_id: int) -> int:
+ """Get or create comm ID."""
+ key = (comm, thread_id)
+ if key not in self.caches['comms']:
+ comm_id = self.next_id['comm']
+ self._exec("INSERT INTO comms VALUES (?, ?, ?, ?, ?)",
+ (comm_id, comm, thread_id, 0, 0))
+ self.caches['comms'][key] = comm_id
+ self.next_id['comm'] += 1
+ comm_id = self.caches['comms'][key]
+ mapping_key = (comm_id, thread_id)
+ if mapping_key not in self.comm_threads_cache:
+ comm_thread_id = self.next_comm_thread_id
+ self._exec("INSERT INTO comm_threads VALUES (?, ?, ?)",
+ (comm_thread_id, comm_id, thread_id))
+ self.comm_threads_cache.add(mapping_key)
+ self.next_comm_thread_id += 1
+ return comm_id
+
+ def get_dso_id(self, short_name: str, long_name: str,
+ build_id: str, machine_id: int = 0) -> int:
+ """Get or create DSO ID."""
+ key = (machine_id, short_name, long_name, build_id)
+ if key in self.caches['dsos']:
+ return self.caches['dsos'][key]
+ short_key = (machine_id, short_name)
+ if not build_id and short_key in self.caches['dsos']:
+ return self.caches['dsos'][short_key]
+ dso_id = self.next_id['dso']
+ self._exec("INSERT INTO dsos VALUES (?, ?, ?, ?, ?)",
+ (dso_id, machine_id, short_name, long_name, build_id))
+ self.caches['dsos'][key] = dso_id
+ self.caches['dsos'][short_key] = dso_id
+ self.next_id['dso'] += 1
+ return dso_id
+
+ def get_symbol_id(self, dso_id: int, name: str, start: int,
+ end: int) -> int:
+ """Get or create symbol ID."""
+ key = (dso_id, name, start, end)
+ if key in self.caches['symbols']:
+ return self.caches['symbols'][key]
+ short_key = (dso_id, name)
+ if start == 0 and end == 0 and short_key in self.caches['symbols']:
+ return self.caches['symbols'][short_key]
+ symbol_id = self.next_id['symbol']
+ self._exec("INSERT INTO symbols VALUES (?, ?, ?, ?, ?, ?)",
+ (symbol_id, dso_id, start, end, 0, name))
+ self.caches['symbols'][key] = symbol_id
+ self.caches['symbols'][short_key] = symbol_id
+ self.next_id['symbol'] += 1
+ return symbol_id
+
+ def get_call_path_id(self, parent_id: int, symbol_id: int,
+ ip: int) -> int:
+ """Get or create call path ID."""
+ key = (parent_id, symbol_id, ip)
+ if key in self.caches['call_paths']:
+ return self.caches['call_paths'][key]
+ call_path_id = self.next_id['call_path']
+ self._exec("INSERT INTO call_paths VALUES (?, ?, ?, ?)",
+ (call_path_id, parent_id, symbol_id, ip))
+ self.caches['call_paths'][key] = call_path_id
+ self.next_id['call_path'] += 1
+ return call_path_id
+
+ def process_event(self, sample: perf.sample_event) -> None:
+ """Callback for processing events."""
+
+ machine_pid = getattr(sample, 'machine_pid', 0)
+ machine_id = self.get_machine_id(machine_pid)
+ thread_id = self.get_thread_id(machine_id, sample.sample_pid, sample.sample_tid)
+
+ comm = "Unknown_comm"
+ try:
+ if self.session is not None:
+ proc = self.session.find_thread(sample.sample_pid, sample.sample_tid)
+ if proc:
+ comm_name = proc.comm()
+ if comm_name:
+ comm = comm_name
+ except TypeError:
+ pass
+ comm_id = self.get_comm_id(comm, thread_id)
+
+ dso_bid = (sample.dso_bid.decode('utf-8')
+ if isinstance(sample.dso_bid, bytes) else str(sample.dso_bid or ""))
+ dso_id = self.get_dso_id(
+ sample.dso or "Unknown_dso",
+ sample.dso_long_name or "Unknown_dso_long",
+ dso_bid,
+ machine_id
+ )
+
+ symbol_id = self.get_symbol_id(
+ dso_id,
+ sample.symbol or "Unknown_symbol",
+ sample.sym_start or 0,
+ sample.sym_end or 0
+ )
+
+ # Handle callchain
+ call_path_id = 0
+ if hasattr(sample, 'callchain') and sample.callchain:
+ parent_id = 0
+ for node in reversed(sample.callchain):
+ dso_name = node.dso or "Unknown_dso"
+ symbol_name = node.symbol or "Unknown_symbol"
+
+ node_dso_id = self.get_dso_id(dso_name, dso_name, "", machine_id)
+ node_symbol_id = self.get_symbol_id(node_dso_id, symbol_name, 0, 0)
+
+ parent_id = self.get_call_path_id(parent_id, node_symbol_id, node.ip)
+ call_path_id = parent_id
+ else:
+ call_path_id = 0
+
+ # Insert sample
+ event_name_str = str(sample.evsel)
+ if event_name_str.startswith("evsel(") and event_name_str.endswith(")"):
+ event_name_str = event_name_str[6:-1]
+ cursor = self._exec("""
+ INSERT INTO samples VALUES (
+ NULL, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?
+ )
+ """, (
+ self.get_event_id(event_name_str),
+ machine_id, thread_id, comm_id,
+ dso_id, symbol_id,
+ (getattr(sample, 'sym_offset') if getattr(sample, 'sym_offset', None) is not None
+ else (sample.sample_ip - (getattr(sample, 'sym_start', None) or 0))),
+ sample.sample_ip, sample.sample_time, sample.sample_cpu,
+ self.get_dso_id(sample.addr_dso or "Unknown_dso",
+ sample.addr_dso or "Unknown_dso_long", "", machine_id),
+ self.get_symbol_id(
+ self.get_dso_id(sample.addr_dso or "Unknown_dso",
+ sample.addr_dso or "Unknown_dso_long", "", machine_id),
+ sample.addr_symbol or "Unknown_symbol", 0, 0
+ ),
+ sample.addr_sym_offset or 0,
+ sample.sample_addr or 0,
+ sample.sample_period or 0,
+ sample.sample_weight or 0,
+ sample.transaction or 0,
+ sample.sample_data_src,
+ sample.branch_type or 0,
+ sample.in_tx or 0,
+ call_path_id,
+ sample.sample_insn_count,
+ sample.sample_cyc_count,
+ getattr(sample, "flags", 0) # flags
+ ))
+ sample_id = cursor.lastrowid
+
+
+
+ # Handle Intel PT specific raw payloads mathematically equivalent to
+ # legacy synth_data unpacking
+
+ if event_name_str == "ptwrite" and hasattr(sample, "raw_buf"):
+ try:
+ flags, payload = struct.unpack_from("<IQ", sample.raw_buf)
+ self._exec("INSERT INTO ptwrite VALUES (?, ?, ?)",
+ (sample_id, payload, flags & 1))
+ except struct.error:
+ pass
+
+ elif event_name_str == "cbr" and hasattr(sample, "raw_buf"):
+ try:
+ data = struct.unpack_from("<BBBBII", sample.raw_buf)
+ cbr_val, freq, max_freq = data[0], data[2], data[4]
+ MHz = (max_freq + 500) // 1000
+ percent = ((cbr_val * 1000 // freq) + 5) // 10 if freq else 0
+ self._exec("INSERT INTO cbr VALUES (?, ?, ?, ?)",
+ (sample_id, cbr_val, MHz, percent))
+ except struct.error:
+ pass
+
+ elif event_name_str == "mwait" and hasattr(sample, "raw_buf"):
+ try:
+ flags, payload = struct.unpack_from("<IQ", sample.raw_buf)
+ hints = payload & 0xff
+ extensions = (payload >> 32) & 0x3
+ self._exec("INSERT INTO mwait VALUES (?, ?, ?)",
+ (sample_id, hints, extensions))
+ except struct.error:
+ pass
+
+ elif event_name_str == "pwre" and hasattr(sample, "raw_buf"):
+ try:
+ flags, payload = struct.unpack_from("<IQ", sample.raw_buf)
+ hw = (payload >> 7) & 1
+ cstate = (payload >> 12) & 0xf
+ subcstate = (payload >> 8) & 0xf
+ self._exec("INSERT INTO pwre (id, cstate, subcstate, hw) VALUES (?, ?, ?, ?)",
+ (sample_id, cstate, subcstate, hw))
+ except struct.error:
+ pass
+
+ elif event_name_str == "exstop" and hasattr(sample, "raw_buf"):
+ try:
+ flags = struct.unpack_from("<I", sample.raw_buf)[0]
+ self._exec("INSERT INTO exstop VALUES (?, ?)",
+ (sample_id, flags & 1))
+ except struct.error:
+ pass
+
+ elif event_name_str == "pwrx" and hasattr(sample, "raw_buf"):
+ try:
+ flags, payload = struct.unpack_from("<IQ", sample.raw_buf)
+ deepest_cstate = payload & 0xf
+ last_cstate = (payload >> 4) & 0xf
+ wake_reason = (payload >> 8) & 0xf
+ self._exec("INSERT INTO pwrx VALUES (?, ?, ?, ?)",
+ (sample_id, deepest_cstate, last_cstate, wake_reason))
+ except struct.error:
+ pass
+
+ self.sample_count += 1
+ if self.sample_count % 10000 == 0:
+ self.commit()
+
+
+
+ def get_call_path_ids(self, call_path: typing.Any, machine_id: int = 0) -> tuple[int, int]:
+ """Add a perf.callchain as call_paths rows, return its id and parent's."""
+ parent_id = 0
+ call_path_id = 0
+ for node in call_path:
+ dso_name = node.dso or "Unknown_dso"
+ symbol_name = node.symbol or "Unknown_symbol"
+ dso_id = self.get_dso_id(dso_name, dso_name, "", machine_id)
+ symbol_id = self.get_symbol_id(dso_id, symbol_name, 0, 0)
+ parent_id = call_path_id
+ call_path_id = self.get_call_path_id(parent_id, symbol_id, node.ip)
+ return call_path_id, parent_id
+
+ def process_call_return(self, cr: perf.call_return) -> None:
+ """Callback for processing call_return events."""
+ machine_id = self.get_machine_id(getattr(cr, "machine_pid", None))
+ thread_id = self.get_thread_id(machine_id, cr.pid, cr.tid)
+ comm_id = self.get_comm_id(cr.comm or "Unknown_comm", thread_id)
+ call_path_id, parent_call_path_id = self.get_call_path_ids(cr.call_path or [], machine_id)
+ # calls.id and calls.parent_id use the db_id of the perf module, as a
+ # call is given an id before it returns its parent cannot be numbered
+ # here.
+ self._exec(
+ "INSERT INTO calls VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
+ (cr.db_id, thread_id, comm_id, call_path_id,
+ cr.call_time, cr.return_time,
+ cr.branch_count, cr.call_ref,
+ cr.return_ref, parent_call_path_id,
+ cr.flags, cr.parent_id,
+ cr.insn_count, cr.cyc_count)
+ )
+
+
+ def process_context_switch(self, event: typing.Any) -> None:
+ """Callback for processing context switch events."""
+ misc = getattr(event, 'misc', 0)
+ out = bool(misc & (1 << 13)) # PERF_RECORD_MISC_SWITCH_OUT
+ out_preempt = bool(misc & (1 << 14)) # PERF_RECORD_MISC_SWITCH_OUT_PREEMPT
+ flags = (1 if out else 0) | ((1 if out_preempt else 0) << 1)
+ machine_pid = getattr(event, 'machine_pid', None)
+ machine_id = self.get_machine_id(machine_pid)
+
+ sample_pid = getattr(event, 'sample_pid', None)
+ if sample_pid is None:
+ sample_pid = -1
+ elif sample_pid > 0x7fffffff:
+ sample_pid -= 0x100000000
+ sample_tid = getattr(event, 'sample_tid', None)
+ if sample_tid is None:
+ sample_tid = -1
+ elif sample_tid > 0x7fffffff:
+ sample_tid -= 0x100000000
+ next_prev_pid = getattr(event, 'next_prev_pid', None)
+ if next_prev_pid is None:
+ next_prev_pid = -1
+ elif next_prev_pid > 0x7fffffff:
+ next_prev_pid -= 0x100000000
+ next_prev_tid = getattr(event, 'next_prev_tid', None)
+ if next_prev_tid is None:
+ next_prev_tid = -1
+ elif next_prev_tid > 0x7fffffff:
+ next_prev_tid -= 0x100000000
+
+ th_a_id = self.get_thread_id(machine_id, sample_pid, sample_tid)
+ comm_a = "Unknown_comm"
+ if self.session:
+ try:
+ proc = self.session.find_thread(sample_pid, sample_tid)
+ if proc:
+ comm_a_name = proc.comm()
+ if comm_a_name:
+ comm_a = comm_a_name
+ except TypeError:
+ pass
+ comm_a_id = self.get_comm_id(comm_a, th_a_id)
+
+ th_b_id = self.get_thread_id(machine_id, next_prev_pid, next_prev_tid)
+ comm_b = "Unknown_comm"
+ if self.session:
+ try:
+ proc = self.session.find_thread(next_prev_pid, next_prev_tid)
+ if proc:
+ comm_b_name = proc.comm()
+ if comm_b_name:
+ comm_b = comm_b_name
+ except TypeError:
+ pass
+ comm_b_id = self.get_comm_id(comm_b, th_b_id)
+
+ if out:
+ th_out_id = th_a_id
+ comm_out_id = comm_a_id
+ th_in_id = th_b_id
+ comm_in_id = comm_b_id
+ else:
+ th_out_id = th_b_id
+ comm_out_id = comm_b_id
+ th_in_id = th_a_id
+ comm_in_id = comm_a_id
+
+ self._exec(
+ "INSERT INTO context_switches VALUES (NULL, ?, ?, ?, ?, ?, ?, ?, ?)",
+ (machine_id, getattr(event, 'sample_time', 0), getattr(event, 'sample_cpu', 0) or 0,
+ th_out_id, comm_out_id, th_in_id, comm_in_id, flags)
+ )
+
+ def commit(self) -> None:
+ """Commit transaction."""
+ self.con.commit()
+
+ def close(self) -> None:
+ """Close connection."""
+ self.con.close()
+
+
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser(
+ description="Export perf data to a sqlite3 database")
+ ap.add_argument("-i", "--input", default="perf.data",
+ help="Input file name")
+ ap.add_argument("-o", "--output", default="perf.db",
+ help="Output database name")
+ ap.add_argument("--itrace", help="Instruction Tracing options (e.g. crpt)")
+ args = ap.parse_args()
+
+ try:
+ fd = os.open(args.output, os.O_CREAT | os.O_EXCL | os.O_WRONLY, 0o600)
+ except FileExistsError:
+ print(f"Error: {args.output} already exists")
+ sys.exit(1)
+
+ tmp_dir = None
+ exporter = None
+ succeeded = False
+ try:
+ tmp_dir = tempfile.mkdtemp(prefix="perf-export-sqlite-")
+ tmp_db = os.path.join(tmp_dir, "perf.db")
+ exporter = DatabaseExporter(tmp_db)
+ session = perf.session(perf.data(args.input),
+ sample=exporter.process_event,
+ context_switch=exporter.process_context_switch,
+ call_return=exporter.process_call_return,
+ itrace=args.itrace)
+ exporter.session = session
+ session.process_events()
+ exporter.session = None
+ exporter.commit()
+ exporter.close()
+ exporter = None
+ with open(tmp_db, "rb") as src, os.fdopen(fd, "wb", closefd=False) as dst:
+ shutil.copyfileobj(src, dst)
+ succeeded = True
+ print(f"Successfully exported to {args.output}")
+ except (OSError, RuntimeError, ValueError, sqlite3.Error):
+ import traceback
+ traceback.print_exc()
+ finally:
+ if exporter is not None:
+ exporter.session = None
+ exporter.close()
+ os.close(fd)
+ if tmp_dir is not None:
+ shutil.rmtree(tmp_dir, ignore_errors=True)
+ if not succeeded:
+ try:
+ os.remove(args.output)
+ except OSError:
+ pass
+ if not succeeded:
+ sys.exit(1)
diff --git a/tools/perf/tests/shell/test_export_to_sqlite_python.sh b/tools/perf/tests/shell/test_export_to_sqlite_python.sh
new file mode 100755
index 000000000000..342e1734af8f
--- /dev/null
+++ b/tools/perf/tests/shell/test_export_to_sqlite_python.sh
@@ -0,0 +1,108 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# export-to-sqlite python test (exclusive)
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+# If we don't have sqlite3, we can't test
+if ! "$PYTHON" -c 'import sqlite3' > /dev/null 2>&1; then
+ echo "Skipping test, sqlite3 module not found"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/export-to-sqlite.py"
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, export-to-sqlite.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_dir=""
+temp_data=""
+temp_db=""
+
+cleanup() {
+ [ -n "${temp_dir}" ] && rm -rf "${temp_dir}"
+}
+
+trap 'cleanup' EXIT
+trap 'cleanup; exit 1' TERM INT
+
+temp_dir=$(mktemp -d /tmp/perf.sqlite.XXXXXX)
+temp_data="${temp_dir}/perf.data"
+temp_db="${temp_dir}/perf.export.db"
+
+test_file_mode() {
+ echo "Testing export-to-sqlite.py..."
+
+ # Generate events with callchains and context switches if supported
+ if ! perf record -g --switch-events -o "${temp_data}" \
+ -- perf test -w noploop >/dev/null 2>&1 && \
+ ! perf record -g -o "${temp_data}" -- perf test -w noploop >/dev/null 2>&1; then
+ echo "Skipping test, perf record failed"
+ exit 2
+ fi
+
+ # Run the script
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" -o "${temp_db}" >/dev/null; then
+ echo "File mode test failed."
+ err=1
+ else
+ # Check DB tables (samples, call_paths, threads, comms)
+ query="import sqlite3; c = sqlite3.connect('${temp_db}'); "
+ query="${query}s = c.execute('SELECT COUNT(*) FROM samples').fetchone()[0]; "
+ query="${query}cp = c.execute('SELECT COUNT(*) FROM call_paths').fetchone()[0]; "
+ query="${query}exit(0 if s > 0 and cp > 0 else 1)"
+ if ! "$PYTHON" -c "$query" >/dev/null 2>&1; then
+ echo "SQLite validation failed."
+ err=1
+ else
+ echo "File mode test passed."
+ fi
+ fi
+}
+
+test_intel_pt() {
+ echo "Testing export-to-sqlite.py with intel_pt..."
+
+ rm -f "${temp_db}" "${temp_data}"
+ # Generate some intel_pt events; use a subshell that waits for uname
+ if ! perf record -B -N --no-bpf-event -e intel_pt//u -o "${temp_data}" \
+ -- sh -c "uname; true" >/dev/null 2>&1; then
+ echo "Skipping intel_pt test, intel_pt not available."
+ return 0
+ fi
+
+ # Run the script with --itrace cr to synthesize call_returns
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" -o "${temp_db}" --itrace cr; then
+ echo "intel_pt file mode test failed."
+ err=1
+ else
+ # Check DB for calls
+ query="import sqlite3; c = sqlite3.connect('${temp_db}'); "
+ query="${query}r = c.execute('SELECT COUNT(*) FROM calls').fetchone()[0]; "
+ query="${query}exit(1 if r == 0 else 0)"
+ if ! "$PYTHON" -c "$query" >/dev/null 2>&1; then
+ echo "SQLite intel_pt validation failed (no calls found)."
+ err=1
+ else
+ echo "intel_pt test passed (cr validated)."
+ fi
+ fi
+}
+
+test_file_mode
+test_intel_pt
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 43/49] perf python: Port export-to-postgresql to perf module
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (41 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 42/49] perf python: Port export-to-sqlite to perf module Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 44/49] perf python: Move and clean up exported-sql-viewer.py Ian Rogers
` (5 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Port export-to-postgresql.py to a standalone script in
tools/perf/python/ using the perf module and libpq via ctypes.
Improvements compared to the legacy script:
- Remove the dependency on PySide/QtSql for database creation and DDL
execution by driving libpq directly via ctypes (PQconnectdb, PQexec,
PQputCopyData, PQputCopyEnd) and streaming binary PostgreSQL COPY
files in PostgresExporter, enabling export on headless servers without
Qt installed.
- Harden database connection and SQL identifier handling by rejecting
URI ('://') and connection-parameter ('=') injection strings in
connect(), escaping single quotes and backslashes in connection
strings, and quoting SQL identifiers (quote_ident).
- Support Intel PT and instruction trace export via
perf.session(itrace=...) and perf.call_return callbacks, reconstructing
relational call_paths and calls tables (including id=0 placeholder
rows and callfk/returnfk foreign keys) and unpacking synthesized PT
payloads (ptwrite, cbr, mwait, pwre, exstop, pwrx).
- Use a dedicated context_switch ID counter so context-switch exports
do not advance sample database IDs out of sync with call_return
references.
Update Documentation/db-export.txt and add a shell test
(test_export_to_postgresql_python.sh) to verify the standalone exporter.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/Documentation/db-export.txt | 2 +-
tools/perf/python/export-to-postgresql.py | 1320 +++++++++++++++++
.../shell/test_export_to_postgresql_python.sh | 119 ++
3 files changed, 1440 insertions(+), 1 deletion(-)
create mode 100755 tools/perf/python/export-to-postgresql.py
create mode 100755 tools/perf/tests/shell/test_export_to_postgresql_python.sh
diff --git a/tools/perf/Documentation/db-export.txt b/tools/perf/Documentation/db-export.txt
index b43f12b96973..20024e1f9164 100644
--- a/tools/perf/Documentation/db-export.txt
+++ b/tools/perf/Documentation/db-export.txt
@@ -8,7 +8,7 @@ perf tool's python scripting engine:
supports scripts:
tools/perf/python/export-to-sqlite.py
- tools/perf/scripts/python/export-to-postgresql.py
+ tools/perf/python/export-to-postgresql.py
which export data to a SQLite3 or PostgreSQL database.
diff --git a/tools/perf/python/export-to-postgresql.py b/tools/perf/python/export-to-postgresql.py
new file mode 100755
index 000000000000..32b077f38bec
--- /dev/null
+++ b/tools/perf/python/export-to-postgresql.py
@@ -0,0 +1,1320 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+r"""
+Export perf data to a postgresql database.
+
+This script has been ported to use the modern perf Python module and
+libpq via ctypes. It no longer requires PySide2 or QtSql for exporting.
+
+The script assumes postgresql is running on the local machine and that the
+user has postgresql permissions to create databases.
+
+An example of using this script with Intel PT:
+
+ $ perf record -e intel_pt//u ls
+ $ python export-to-postgresql.py -i perf.data -o pt_example
+
+To browse the database, psql can be used e.g.
+
+ $ psql pt_example
+ pt_example=# select * from samples_view where id < 100;
+ pt_example=# \d+
+ pt_example=# \d+ samples_view
+ pt_example=# \q
+
+An example of using the database is provided by the script
+exported-sql-viewer.py. Refer to that script for details.
+
+Tables:
+
+ The tables largely correspond to perf tools' data structures. They are
+ largely self-explanatory.
+
+ samples
+ 'samples' is the main table. It represents what instruction was
+ executing at a point in time when something (a selected event)
+ happened. The memory address is the instruction pointer or 'ip'.
+
+ branch_types
+ 'branch_types' provides descriptions for each type of branch.
+
+ comm_threads
+ 'comm_threads' shows how 'comms' relates to 'threads'.
+
+ comms
+ 'comms' contains a record for each 'comm' - the name given to the
+ executable that is running.
+
+ dsos
+ 'dsos' contains a record for each executable file or library.
+
+ machines
+ 'machines' can be used to distinguish virtual machines if
+ virtualization is supported.
+
+ selected_events
+ 'selected_events' contains a record for each kind of event that
+ has been sampled.
+
+ symbols
+ 'symbols' contains a record for each symbol. Only symbols that
+ have samples are present.
+
+ threads
+ 'threads' contains a record for each thread.
+
+Views:
+
+ Most of the tables have views for more friendly display. The views are:
+
+ comm_threads_view
+ dsos_view
+ machines_view
+ samples_view
+ symbols_view
+ threads_view
+
+Ported from tools/perf/scripts/python/export-to-postgresql.py
+"""
+
+from __future__ import annotations
+import typing
+import argparse
+from ctypes import CDLL, c_char_p, c_int, c_void_p, c_ubyte
+import ctypes.util
+import os
+import shutil
+import struct
+import tempfile
+import sys
+from typing import Any, Dict, Optional
+import perf
+
+# Need to access PostgreSQL C library directly to use COPY FROM STDIN
+libpq_name = ctypes.util.find_library("pq")
+if not libpq_name:
+ libpq_name = "libpq.so.5"
+
+try:
+ libpq = CDLL(libpq_name)
+except OSError as e:
+ print(f"Error loading {libpq_name}: {e}")
+ print("Please ensure PostgreSQL client library is installed.")
+ sys.exit(1)
+
+PQconnectdb = libpq.PQconnectdb
+PQconnectdb.restype = c_void_p
+PQconnectdb.argtypes = [c_char_p]
+PQfinish = libpq.PQfinish
+PQfinish.argtypes = [c_void_p]
+PQstatus = libpq.PQstatus
+PQstatus.restype = c_int
+PQstatus.argtypes = [c_void_p]
+libpq.PQerrorMessage.restype = c_char_p
+libpq.PQerrorMessage.argtypes = [c_void_p]
+PQexec = libpq.PQexec
+PQexec.restype = c_void_p
+PQexec.argtypes = [c_void_p, c_char_p]
+PQresultStatus = libpq.PQresultStatus
+PQresultStatus.restype = c_int
+PQresultStatus.argtypes = [c_void_p]
+PQputCopyData = libpq.PQputCopyData
+PQputCopyData.restype = c_int
+PQputCopyData.argtypes = [c_void_p, c_void_p, c_int]
+PQputCopyEnd = libpq.PQputCopyEnd
+PQputCopyEnd.restype = c_int
+PQputCopyEnd.argtypes = [c_void_p, c_void_p]
+PQgetResult = libpq.PQgetResult
+PQgetResult.restype = c_void_p
+PQgetResult.argtypes = [c_void_p]
+PQclear = libpq.PQclear
+PQclear.argtypes = [c_void_p]
+
+
+def toserverstr(s: str) -> bytes:
+ """Convert string to server encoding (UTF-8)."""
+ return bytes(s, "UTF_8")
+
+
+def toclientstr(s: str) -> bytes:
+ """Convert string to client encoding (UTF-8)."""
+ return bytes(s, "UTF_8")
+
+
+
+
+PERF_IP_FLAG_BRANCH = 1 << 0
+PERF_IP_FLAG_CALL = 1 << 1
+PERF_IP_FLAG_RETURN = 1 << 2
+PERF_IP_FLAG_CONDITIONAL = 1 << 3
+PERF_IP_FLAG_SYSCALLRET = 1 << 4
+PERF_IP_FLAG_ASYNC = 1 << 5
+PERF_IP_FLAG_INTERRUPT = 1 << 6
+PERF_IP_FLAG_TX_ABORT = 1 << 7
+PERF_IP_FLAG_TRACE_BEGIN = 1 << 8
+PERF_IP_FLAG_TRACE_END = 1 << 9
+PERF_IP_FLAG_IN_TX = 1 << 10
+PERF_IP_FLAG_VMENTRY = 1 << 11
+PERF_IP_FLAG_VMEXIT = 1 << 12
+
+BRANCH_TYPES = [
+ (0, "no branch"),
+ (PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL, "call"),
+ (PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN, "return"),
+ (PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CONDITIONAL, "conditional jump"),
+ (PERF_IP_FLAG_BRANCH, "unconditional jump"),
+ (PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_INTERRUPT, "software interrupt"),
+ (PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN | PERF_IP_FLAG_INTERRUPT, "return from interrupt"),
+ (PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_SYSCALLRET, "system call"),
+ (PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN | PERF_IP_FLAG_SYSCALLRET,
+ "return from system call"),
+ (PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_ASYNC, "asynchronous branch"),
+ (PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_ASYNC | PERF_IP_FLAG_INTERRUPT,
+ "hardware interrupt"),
+ (PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TX_ABORT, "transaction abort"),
+ (PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TRACE_BEGIN, "trace begin"),
+ (PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TRACE_END, "trace end"),
+ (PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_VMENTRY, "vm entry"),
+ (PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_VMEXIT, "vm exit"),
+]
+for _b_type, _b_name in list(BRANCH_TYPES):
+ if (_b_type == PERF_IP_FLAG_BRANCH or
+ (_b_type & (PERF_IP_FLAG_TRACE_BEGIN | PERF_IP_FLAG_TRACE_END))):
+ continue
+ BRANCH_TYPES.append((_b_type | PERF_IP_FLAG_TRACE_BEGIN, f"trace begin / {_b_name}"))
+ BRANCH_TYPES.append((_b_type | PERF_IP_FLAG_TRACE_END, f"{_b_name} / trace end"))
+
+class PostgresExporter:
+ """Handles PostgreSQL connection and exporting of perf events."""
+
+ def __init__(self, dbname: str):
+ self.dbname = dbname
+ self.conn = None
+ self.session: Optional[perf.session] = None
+ self.output_dir_name = tempfile.mkdtemp(prefix=dbname + "-perf-data-")
+ self.created_output_dir = True
+
+ self.file_header = struct.pack("!11sii", b"PGCOPY\n\377\r\n\0", 0, 0)
+ self.file_trailer = b"\377\377"
+
+ # Caches and counters grouped to reduce instance attributes
+ self.caches: Dict[str, dict] = {
+ 'machines': {0: 0},
+ 'threads': {},
+ 'comms': {},
+ 'dsos': {},
+ 'symbols': {},
+ 'events': {},
+ 'branch_types': {},
+ 'call_paths': {}
+ }
+
+ self.next_id = {
+ 'machine': 1,
+ 'thread': 1,
+ 'comm': 1,
+ 'dso': 1,
+ 'symbol': 1,
+ 'event': 1,
+ 'branch_type': 1,
+ 'comm_thread': 1,
+ 'call_path': 1,
+ 'sample': 1,
+ 'call': 1,
+ 'context_switch': 1
+ }
+
+ self.files: Dict[str, Any] = {}
+ self.unhandled_count = 0
+
+ def connect(self, db_to_use: str) -> None:
+ """Connect to database."""
+ if "://" in db_to_use or "=" in db_to_use:
+ raise ValueError(f"Invalid database name: {db_to_use}")
+ safe_db = db_to_use.replace('\\', '\\\\').replace("'", "''")
+ conn_str = toclientstr(f"dbname='{safe_db}'")
+ self.conn = PQconnectdb(conn_str)
+ if PQstatus(self.conn) != 0:
+ PQfinish(self.conn)
+ self.conn = None
+ raise RuntimeError(f"PQconnectdb failed for {db_to_use}")
+
+ def disconnect(self) -> None:
+ """Disconnect from database."""
+ if self.conn:
+ PQfinish(self.conn)
+ self.conn = None
+
+ def do_query(self, sql: str) -> None:
+ """Execute a query and check status."""
+ res = PQexec(self.conn, toserverstr(sql))
+ status = PQresultStatus(res)
+ PQclear(res)
+ if status not in (1, 2): # PGRES_COMMAND_OK, PGRES_TUPLES_OK
+ error_msg = libpq.PQerrorMessage(self.conn).decode('utf-8')
+ raise RuntimeError(f"Query failed: {sql}. Error: {error_msg}")
+
+
+ def open_output_file(self, file_name: str):
+ """Open intermediate binary file."""
+ path_name = self.output_dir_name + "/" + file_name
+ f = open(path_name, "wb+")
+ f.write(self.file_header)
+ return f
+
+ def close_output_file(self, f):
+ """Close intermediate binary file."""
+ f.write(self.file_trailer)
+ f.close()
+
+ def copy_output_file(self, path_name: str, table_name: str):
+ """Copy intermediate file to database."""
+ sql = f"COPY {table_name} FROM STDIN (FORMAT 'binary')"
+ res = PQexec(self.conn, toserverstr(sql))
+ if PQresultStatus(res) != 4: # PGRES_COPY_IN
+ PQclear(res)
+ err = libpq.PQerrorMessage(self.conn).decode()
+ raise RuntimeError(f"COPY FROM STDIN PQexec failed for {table_name}: {err}")
+ PQclear(res)
+
+ with open(path_name, "rb") as f:
+ data = f.read(65536)
+ while len(data) > 0:
+ c_data = (c_ubyte * len(data)).from_buffer_copy(data)
+ ret = PQputCopyData(self.conn, c_data, len(data))
+ if ret != 1:
+ raise RuntimeError(f"PQputCopyData failed for {table_name}")
+ data = f.read(65536)
+
+ ret = PQputCopyEnd(self.conn, None)
+ if ret != 1:
+ err = libpq.PQerrorMessage(self.conn).decode()
+ raise RuntimeError(f"PQputCopyEnd failed for {table_name}: {err}")
+
+ res = PQgetResult(self.conn)
+ while res:
+ status = PQresultStatus(res)
+ if status != 1: # PGRES_COMMAND_OK
+ error_msg = libpq.PQerrorMessage(self.conn).decode('utf-8')
+ PQclear(res)
+ raise RuntimeError(
+ f"COPY completion failed for {table_name}. Status {status}: {error_msg}"
+ )
+ PQclear(res)
+ res = PQgetResult(self.conn)
+
+
+
+ def setup_db(self) -> None:
+ """Create database and tables. MUST be called after init."""
+ self.created_output_dir = True
+
+ self.connect('postgres')
+ try:
+ db_name = self.dbname.replace('"', '""')
+ self.do_query(f'CREATE DATABASE "{db_name}"')
+ except Exception as e:
+ shutil.rmtree(self.output_dir_name, ignore_errors=True)
+ raise e
+ self.disconnect()
+
+ self.connect(self.dbname)
+ self.do_query("SET client_min_messages TO WARNING")
+
+ self.do_query("""
+ CREATE TABLE selected_events (
+ id bigint NOT NULL,
+ name varchar(80))
+ """)
+ self.do_query("""
+ CREATE TABLE machines (
+ id bigint NOT NULL,
+ pid integer,
+ root_dir varchar(4096))
+ """)
+ self.do_query("""
+ CREATE TABLE threads (
+ id bigint NOT NULL,
+ machine_id bigint,
+ process_id bigint,
+ pid integer,
+ tid integer)
+ """)
+ self.do_query("""
+ CREATE TABLE comms (
+ id bigint NOT NULL,
+ comm varchar(16),
+ c_thread_id bigint,
+ c_time bigint,
+ exec_flag boolean)
+ """)
+ self.do_query("""
+ CREATE TABLE comm_threads (
+ id bigint NOT NULL,
+ comm_id bigint,
+ thread_id bigint)
+ """)
+ self.do_query("""
+ CREATE TABLE dsos (
+ id bigint NOT NULL,
+ machine_id bigint,
+ short_name varchar(256),
+ long_name varchar(4096),
+ build_id varchar(64))
+ """)
+ self.do_query("""
+ CREATE TABLE symbols (
+ id bigint NOT NULL,
+ dso_id bigint,
+ sym_start bigint,
+ sym_end bigint,
+ binding integer,
+ name varchar(2048))
+ """)
+ self.do_query("""
+ CREATE TABLE branch_types (
+ id integer NOT NULL,
+ name varchar(80))
+ """)
+ self.do_query("""
+ CREATE TABLE samples (
+ id bigint NOT NULL,
+ evsel_id bigint,
+ machine_id bigint,
+ thread_id bigint,
+ comm_id bigint,
+ dso_id bigint,
+ symbol_id bigint,
+ sym_offset bigint,
+ ip bigint,
+ time bigint,
+ cpu integer,
+ to_dso_id bigint,
+ to_symbol_id bigint,
+ to_sym_offset bigint,
+ to_ip bigint,
+ period bigint,
+ weight bigint,
+ transaction_ bigint,
+ data_src bigint,
+ branch_type integer,
+ in_tx boolean,
+ call_path_id bigint,
+ insn_count bigint,
+ cyc_count bigint,
+ flags integer)
+ """)
+ self.do_query('''
+ CREATE TABLE context_switches (
+ id bigint,
+ machine_id bigint,
+ time bigint,
+ cpu integer,
+ thread_out_id bigint,
+ comm_out_id bigint,
+ thread_in_id bigint,
+ comm_in_id bigint,
+ flags integer
+ )
+ ''')
+ self.do_query('''
+ CREATE TABLE call_paths (
+ id bigint NOT NULL,
+ parent_id bigint,
+ symbol_id bigint,
+ ip bigint)
+ ''')
+ self.do_query('''
+ CREATE TABLE calls (
+ id bigint NOT NULL,
+ thread_id bigint,
+ comm_id bigint,
+ call_path_id bigint,
+ call_time bigint,
+ return_time bigint,
+ branch_count bigint,
+ call_id bigint,
+ return_id bigint,
+ parent_call_path_id bigint,
+ flags integer,
+ parent_id bigint,
+ insn_count bigint,
+ cyc_count bigint
+ )
+ ''')
+ self.do_query('''
+ CREATE TABLE ptwrite (
+ id bigint NOT NULL,
+ payload bigint,
+ exact_ip boolean
+ )
+ ''')
+ self.do_query('''
+ CREATE TABLE cbr (
+ id bigint NOT NULL,
+ cbr integer,
+ mhz integer,
+ percent integer
+ )
+ ''')
+ self.do_query('''
+ CREATE TABLE mwait (
+ id bigint NOT NULL,
+ hints integer,
+ extensions integer
+ )
+ ''')
+ self.do_query('''
+ CREATE TABLE pwre (
+ id bigint NOT NULL,
+ cstate integer,
+ subcstate integer,
+ hw boolean
+ )
+ ''')
+ self.do_query('''
+ CREATE TABLE exstop (
+ id bigint NOT NULL,
+ exact_ip boolean
+ )
+ ''')
+ self.do_query('''
+ CREATE TABLE pwrx (
+ id bigint NOT NULL,
+ deepest_cstate integer,
+ last_cstate integer,
+ wake_reason integer
+ )
+ ''')
+
+
+ self.files['evsel'] = self.open_output_file("evsel_table.bin")
+ self.files['machine'] = self.open_output_file("machine_table.bin")
+ self.files['thread'] = self.open_output_file("thread_table.bin")
+ self.files['comm'] = self.open_output_file("comm_table.bin")
+ self.files['comm_thread'] = self.open_output_file("comm_thread_table.bin")
+ self.files['dso'] = self.open_output_file("dso_table.bin")
+ self.files['symbol'] = self.open_output_file("symbol_table.bin")
+ self.files['branch_type'] = self.open_output_file("branch_type_table.bin")
+ self.files['sample'] = self.open_output_file("sample_table.bin")
+ self.files['context_switches'] = self.open_output_file("context_switches_table.bin")
+ self.files['call'] = self.open_output_file("call_table.bin")
+ self.files['call_path'] = self.open_output_file("call_path_table.bin")
+ self.files['ptwrite'] = self.open_output_file("ptwrite_table.bin")
+ self.files['cbr'] = self.open_output_file("cbr_table.bin")
+ self.files['mwait'] = self.open_output_file("mwait_table.bin")
+ self.files['pwre'] = self.open_output_file("pwre_table.bin")
+ self.files['exstop'] = self.open_output_file("exstop_table.bin")
+ self.files['pwrx'] = self.open_output_file("pwrx_table.bin")
+
+ self.write_evsel(0, "unknown")
+ self.write_machine(0, 0, "unknown")
+ self.write_thread(0, 0, 0, -1, -1)
+ self.write_comm(0, "unknown", 0, 0, 0)
+ self.write_dso(0, 0, "unknown", "unknown", "")
+ self.write_symbol(0, 0, 0, 0, 0, "unknown")
+ self.write_call_path(0, 0, 0, 0)
+ self.files['sample'].write(struct.pack(
+ "!hiqiqiqiqiqiqiqiQiQiQiIiqiqiQiQiQiQiQiQiiiBiqiQiQii",
+ 25, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 4, 0,
+ 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 4, 0, 1, 0, 8, 0, 8, 0, 8, 0, 4, 0
+ ))
+ self.files['call'].write(struct.pack(
+ "!hiqiqiqiqiqiqiqiqiqiqiiiqiqiq",
+ 14, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 8, 0, 4, 0, 8, 0, 8, 0, 8, 0
+ ))
+ for b_type, b_name in BRANCH_TYPES:
+ self.write_branch_type(b_type, b_name)
+
+ def write_branch_type(self, branch_type: int, name: str) -> None:
+ name_bytes = toserverstr(name)
+ n = len(name_bytes)
+ fmt = "!hiii" + str(n) + "s"
+ value = struct.pack(fmt, 2, 4, branch_type, n, name_bytes)
+ self.files['branch_type'].write(value)
+
+ def write_ptwrite(self, id_: int, raw_buf: bytes) -> None:
+ data = struct.unpack_from("<IQ", raw_buf)
+ flags, payload = data[0], data[1]
+ value = struct.pack("!hiqiQiB", 3, 8, id_, 8, payload, 1, flags & 1)
+ self.files['ptwrite'].write(value)
+
+ def write_cbr(self, id_: int, raw_buf: bytes) -> None:
+ data = struct.unpack_from("<BBBBII", raw_buf)
+ cbr_val, freq, max_freq = data[0], data[2], data[4]
+ mhz = (max_freq + 500) // 1000
+ percent = ((cbr_val * 1000 // freq) + 5) // 10 if freq else 0
+ value = struct.pack("!hiqiiiiii", 4, 8, id_, 4, cbr_val, 4, int(mhz), 4, int(percent))
+ self.files['cbr'].write(value)
+
+ def write_mwait(self, id_: int, raw_buf: bytes) -> None:
+ data = struct.unpack_from("<IQ", raw_buf)
+ payload = data[1]
+ hints = payload & 0xff
+ extensions = (payload >> 32) & 0x3
+ value = struct.pack("!hiqiiii", 3, 8, id_, 4, hints, 4, extensions)
+ self.files['mwait'].write(value)
+
+ def write_pwre(self, id_: int, raw_buf: bytes) -> None:
+ data = struct.unpack_from("<IQ", raw_buf)
+ payload = data[1]
+ hw = (payload >> 7) & 1
+ cstate = (payload >> 12) & 0xf
+ subcstate = (payload >> 8) & 0xf
+ value = struct.pack("!hiqiiiiiB", 4, 8, id_, 4, cstate, 4, subcstate, 1, hw)
+ self.files['pwre'].write(value)
+
+ def write_exstop(self, id_: int, raw_buf: bytes) -> None:
+ data = struct.unpack_from("<I", raw_buf)
+ flags = data[0]
+ value = struct.pack("!hiqiB", 2, 8, id_, 1, flags & 1)
+ self.files['exstop'].write(value)
+
+ def write_pwrx(self, id_: int, raw_buf: bytes) -> None:
+ data = struct.unpack_from("<IQ", raw_buf)
+ payload = data[1]
+ deepest = payload & 0xf
+ last = (payload >> 4) & 0xf
+ wake = (payload >> 8) & 0xf
+ value = struct.pack("!hiqiiiiii", 4, 8, id_, 4, deepest, 4, last, 4, wake)
+ self.files['pwrx'].write(value)
+
+ def write_evsel(self, evsel_id: int, name: str) -> None:
+ """Write event to binary file."""
+ name_bytes = toserverstr(name)
+ n = len(name_bytes)
+ fmt = "!hiqi" + str(n) + "s"
+ value = struct.pack(fmt, 2, 8, evsel_id, n, name_bytes)
+ self.files['evsel'].write(value)
+
+ def write_machine(self, machine_id: int, pid: int, root_dir: str) -> None:
+ """Write machine to binary file."""
+ rd_bytes = toserverstr(root_dir)
+ n = len(rd_bytes)
+ fmt = "!hiqiii" + str(n) + "s"
+ value = struct.pack(fmt, 3, 8, machine_id, 4, pid, n, rd_bytes)
+ self.files['machine'].write(value)
+
+
+ def write_thread(self, thread_id: int, machine_id: int, process_id: int,
+ pid: int, tid: int) -> None:
+ """Write thread to binary file."""
+ value = struct.pack("!hiqiqiqiIiI", 5, 8, thread_id, 8, machine_id,
+ 8, process_id, 4, pid & 0xffffffff, 4, tid & 0xffffffff)
+ self.files['thread'].write(value)
+
+
+ def write_comm(self, comm_id: int, comm_str: str, thread_id: int,
+ time: int, exec_flag: int) -> None:
+ """Write comm to binary file."""
+ comm_bytes = toserverstr(comm_str)
+ n = len(comm_bytes)
+ fmt = "!hiqi" + str(n) + "s" + "iqiqiB"
+ value = struct.pack(fmt, 5, 8, comm_id, n, comm_bytes, 8,
+ thread_id, 8, time, 1, exec_flag)
+ self.files['comm'].write(value)
+
+ def write_comm_thread(self, comm_thread_id: int, comm_id: int,
+ thread_id: int) -> None:
+ """Write comm_thread to binary file."""
+ fmt = "!hiqiqiq"
+ value = struct.pack(fmt, 3, 8, comm_thread_id, 8, comm_id, 8, thread_id)
+ self.files['comm_thread'].write(value)
+
+
+ def write_dso(self, dso_id: int, machine_id: int, short_name: str,
+ long_name: str, build_id: str) -> None:
+ """Write DSO to binary file."""
+ sn_bytes = toserverstr(short_name)
+ ln_bytes = toserverstr(long_name)
+ bi_bytes = toserverstr(build_id)
+ n1, n2, n3 = len(sn_bytes), len(ln_bytes), len(bi_bytes)
+ fmt = "!hiqiqi" + str(n1) + "si" + str(n2) + "si" + str(n3) + "s"
+ value = struct.pack(fmt, 5, 8, dso_id, 8, machine_id, n1,
+ sn_bytes, n2, ln_bytes, n3, bi_bytes)
+ self.files['dso'].write(value)
+
+
+ def write_symbol(self, symbol_id: int, dso_id: int, sym_start: int,
+ sym_end: int, binding: int, symbol_name: str) -> None:
+ """Write symbol to binary file."""
+ name_bytes = toserverstr(symbol_name)
+ n = len(name_bytes)
+ fmt = "!hiqiqiQiQiii" + str(n) + "s"
+ value = struct.pack(fmt, 6, 8, symbol_id, 8, dso_id, 8,
+ sym_start, 8, sym_end, 4, binding, n, name_bytes)
+ self.files['symbol'].write(value)
+
+ def write_call_path(self, cp_id: int, parent_id: int, symbol_id: int,
+ ip: int) -> None:
+ """Write call path to binary file."""
+ fmt = "!hiqiqiqiQ"
+ value = struct.pack(fmt, 4, 8, cp_id, 8, parent_id, 8, symbol_id, 8, ip)
+ self.files['call_path'].write(value)
+
+
+ def write_sample(self, sample_id: int, evsel_id: int, machine_id: int, thread_id: int,
+ comm_id: int, dso_id: int, symbol_id: int,
+ sample: perf.sample_event, call_path_id: int) -> None:
+ """Write sample to binary file."""
+ addr_dso_id = self.get_dso_id(
+ sample.addr_dso or "Unknown_dso", sample.addr_dso or "Unknown_dso_long", "",
+ machine_id
+ )
+ addr_symbol_id = self.get_symbol_id(
+ addr_dso_id, sample.addr_symbol or "Unknown_symbol", 0, 0
+ )
+ value = struct.pack(
+ "!hiqiqiqiqiqiqiqiQiQiQiIiqiqiQiQiQiQiQiQiiiBiqiQiQii",
+ 25, 8, sample_id, 8, evsel_id, 8, machine_id, 8, thread_id, 8, comm_id,
+ 8, dso_id, 8, symbol_id, 8, getattr(sample, 'sym_offset', 0) or 0,
+ 8, getattr(sample, 'sample_ip', 0) or 0,
+ 8, getattr(sample, 'sample_time', 0) or 0,
+ 4, getattr(sample, 'sample_cpu', 0) & 0xffffffff,
+ 8, addr_dso_id,
+ 8, addr_symbol_id,
+ 8, getattr(sample, 'addr_sym_offset', 0) or 0,
+ 8, getattr(sample, 'sample_addr', 0) or 0,
+ 8, getattr(sample, 'sample_period', 0) or 0,
+ 8, getattr(sample, 'sample_weight', 0) or 0,
+ 8, getattr(sample, 'transaction', 0) or 0,
+ 8, getattr(sample, 'sample_data_src', 0) or 0 or 0,
+ 4, getattr(sample, 'branch_type', 0) or 0,
+ 1, getattr(sample, 'in_tx', 0) or 0,
+ 8, call_path_id,
+ 8, getattr(sample, 'sample_insn_count', 0) or 0 or 0,
+ 8, getattr(sample, 'sample_cyc_count', 0) or 0 or 0,
+ 4, getattr(sample, 'flags', 0) or 0
+ )
+ self.files['sample'].write(value)
+
+
+ def get_machine_id(self, machine_pid: Optional[int]) -> int:
+ """Get or create machine ID."""
+ if machine_pid is None or machine_pid <= 0 or machine_pid > 0x7fffffff:
+ machine_pid = -1
+ if machine_pid in self.caches['machines']:
+ return self.caches['machines'][machine_pid]
+ machine_id = self.next_id['machine']
+ self.next_id['machine'] += 1
+ self.write_machine(machine_id, machine_pid, "")
+ self.caches['machines'][machine_pid] = machine_id
+ return machine_id
+
+ def get_event_id(self, name: str) -> int:
+ """Get or create event ID."""
+ if name in self.caches['events']:
+ return self.caches['events'][name]
+ event_id = self.next_id['event']
+ self.write_evsel(event_id, name)
+ self.caches['events'][name] = event_id
+ self.next_id['event'] += 1
+ return event_id
+
+ def get_thread_id(self, machine_id: int, pid: Optional[int], tid: Optional[int]) -> int:
+ """Get or create thread ID."""
+ if pid is None:
+ pid = -1
+ elif pid > 0x7fffffff:
+ pid -= 0x100000000
+ if tid is None:
+ tid = -1
+ elif tid > 0x7fffffff:
+ tid -= 0x100000000
+ key = (machine_id, pid, tid)
+ if key in self.caches['threads']:
+ return self.caches['threads'][key]
+ process_id = self.get_thread_id(machine_id, pid, pid) if tid != pid else -1
+ thread_id = self.next_id['thread']
+ if process_id == -1: process_id = thread_id
+ self.write_thread(thread_id, machine_id, process_id, pid, tid)
+ self.caches['threads'][key] = thread_id
+ self.next_id['thread'] += 1
+ return thread_id
+
+ def get_comm_id(self, comm: str, thread_id: int) -> int:
+ """Get or create comm ID."""
+ c_key = (comm, thread_id)
+ if c_key in self.caches['comms']:
+ comm_id = self.caches['comms'][c_key]
+ else:
+ comm_id = self.next_id['comm']
+ self.write_comm(comm_id, comm, thread_id, 0, 0)
+ self.caches['comms'][c_key] = comm_id
+ self.next_id['comm'] += 1
+
+ key = (comm_id, thread_id)
+ if 'comm_threads' not in self.caches:
+ self.caches['comm_threads'] = {}
+ if key not in self.caches['comm_threads']:
+ comm_thread_id = self.next_id['comm_thread']
+ self.write_comm_thread(comm_thread_id, comm_id, thread_id)
+ self.caches['comm_threads'][key] = True
+ self.next_id['comm_thread'] += 1
+
+ return comm_id
+
+ def get_dso_id(self, short_name: str, long_name: str,
+ build_id: str, machine_id: int = 0) -> int:
+ """Get or create DSO ID."""
+ key = (machine_id, short_name, long_name, build_id)
+ if key in self.caches['dsos']:
+ return self.caches['dsos'][key]
+ short_key = (machine_id, short_name)
+ if not build_id and short_key in self.caches['dsos']:
+ return self.caches['dsos'][short_key]
+ dso_id = self.next_id['dso']
+ self.write_dso(dso_id, machine_id, short_name, long_name, build_id)
+ self.caches['dsos'][key] = dso_id
+ self.caches['dsos'][short_key] = dso_id
+ self.next_id['dso'] += 1
+ return dso_id
+
+ def get_symbol_id(self, dso_id: int, name: str, start: int,
+ end: int) -> int:
+ """Get or create symbol ID."""
+ key = (dso_id, name)
+ if key in self.caches['symbols']:
+ return self.caches['symbols'][key]
+ symbol_id = self.next_id['symbol']
+ self.write_symbol(symbol_id, dso_id, start, end, 0, name)
+ self.caches['symbols'][key] = symbol_id
+ self.next_id['symbol'] += 1
+ return symbol_id
+
+ def get_call_path_id(self, parent_id: int, symbol_id: int,
+ ip: int) -> int:
+ """Get or create call path ID."""
+ key = (parent_id, symbol_id, ip)
+ if key in self.caches['call_paths']:
+ return self.caches['call_paths'][key]
+ call_path_id = self.next_id['call_path']
+ self.write_call_path(call_path_id, parent_id, symbol_id, ip)
+ self.caches['call_paths'][key] = call_path_id
+ self.next_id['call_path'] += 1
+ return call_path_id
+
+
+ def process_context_switch(self, event: typing.Any) -> None:
+ """Callback for processing context switch events."""
+ misc = getattr(event, 'misc', 0)
+ out = bool(misc & (1 << 13))
+ out_preempt = bool(misc & (1 << 14))
+ flags = (1 if out else 0) | ((1 if out_preempt else 0) << 1)
+ machine_id = self.get_machine_id(getattr(event, 'machine_pid', None))
+
+ sample_pid = getattr(event, 'sample_pid', None)
+ if sample_pid is None:
+ sample_pid = -1
+ elif sample_pid > 0x7fffffff:
+ sample_pid -= 0x100000000
+ sample_tid = getattr(event, 'sample_tid', None)
+ if sample_tid is None:
+ sample_tid = -1
+ elif sample_tid > 0x7fffffff:
+ sample_tid -= 0x100000000
+ next_prev_pid = getattr(event, 'next_prev_pid', None)
+ if next_prev_pid is None:
+ next_prev_pid = -1
+ elif next_prev_pid > 0x7fffffff:
+ next_prev_pid -= 0x100000000
+ next_prev_tid = getattr(event, 'next_prev_tid', None)
+ if next_prev_tid is None:
+ next_prev_tid = -1
+ elif next_prev_tid > 0x7fffffff:
+ next_prev_tid -= 0x100000000
+
+ th_a_id = self.get_thread_id(machine_id, sample_pid, sample_tid)
+ comm_a = "Unknown_comm"
+ if self.session:
+ try:
+ proc = self.session.find_thread(sample_pid, sample_tid)
+ if proc:
+ comm_a_name = proc.comm() or "Unknown"
+ if comm_a_name:
+ comm_a = comm_a_name
+ except TypeError:
+ pass
+ comm_a_id = self.get_comm_id(comm_a, th_a_id)
+
+ th_b_id = self.get_thread_id(machine_id, next_prev_pid, next_prev_tid)
+ comm_b = "Unknown_comm"
+ if self.session:
+ try:
+ proc = self.session.find_thread(next_prev_pid, next_prev_tid)
+ if proc:
+ comm_b_name = proc.comm() or "Unknown"
+ if comm_b_name:
+ comm_b = comm_b_name
+ except TypeError:
+ pass
+ comm_b_id = self.get_comm_id(comm_b, th_b_id)
+
+ if out:
+ th_out_id = th_a_id
+ comm_out_id = comm_a_id
+ th_in_id = th_b_id
+ comm_in_id = comm_b_id
+ else:
+ th_out_id = th_b_id
+ comm_out_id = comm_b_id
+ th_in_id = th_a_id
+ comm_in_id = comm_a_id
+
+ cs_id = self.next_id['context_switch']
+ self.next_id['context_switch'] += 1
+ time = getattr(event, "time", getattr(event, "sample_time", 0))
+ cpu = getattr(event, "sample_cpu", 0)
+ if cpu > 0x7fffffff:
+ cpu -= 0x100000000
+
+ # 9 columns total:
+ fmt = "!hiqiqiqiiiqiqiqiqii"
+ value = struct.pack(
+ fmt, 9, 8, cs_id, 8, machine_id, 8, time, 4, cpu,
+ 8, th_out_id, 8, comm_out_id, 8, th_in_id, 8, comm_in_id, 4, flags
+ )
+ self.files["context_switches"].write(value)
+
+
+ def get_call_path_ids(self, call_path: typing.Any, machine_id: int = 0) -> tuple[int, int]:
+ """Add a perf.callchain as call_path rows, return its id and parent's."""
+ parent_id = 0
+ call_path_id = 0
+ for node in call_path:
+ dso_name = node.dso or "Unknown_dso"
+ symbol_name = node.symbol or "Unknown_symbol"
+ dso_id = self.get_dso_id(dso_name, dso_name, "", machine_id)
+ symbol_id = self.get_symbol_id(dso_id, symbol_name, 0, 0)
+ parent_id = call_path_id
+ call_path_id = self.get_call_path_id(parent_id, symbol_id, node.ip)
+ return call_path_id, parent_id
+
+ def process_call_return(self, cr: perf.call_return) -> None:
+ """Callback for processing call_return events."""
+ machine_id = self.get_machine_id(getattr(cr, "machine_pid", 0) or 0)
+ thread_id = self.get_thread_id(machine_id,
+ cr.pid if cr.pid is not None else -1,
+ cr.tid if cr.tid is not None else -1)
+ comm_id = self.get_comm_id(cr.comm or "Unknown_comm", thread_id)
+ call_path_id, parent_call_path_id = self.get_call_path_ids(cr.call_path or [], machine_id)
+
+ fmt = "!hiqiqiqiqiqiqiqiqiqiqiiiqiqiq"
+ # call.id and call.parent_id use the db_id of the perf module, as a call
+ # is given an id before it returns its parent cannot be numbered here.
+ value = struct.pack(
+ fmt, 14, 8, cr.db_id, 8, thread_id, 8, comm_id, 8, call_path_id,
+ 8, cr.call_time, 8, cr.return_time, 8, cr.branch_count,
+ 8, cr.call_ref, 8, cr.return_ref,
+ 8, parent_call_path_id, 4, cr.flags, 8, cr.parent_id,
+ 8, cr.insn_count, 8, cr.cyc_count
+ )
+ self.files['call'].write(value)
+
+ def process_event(self, sample: typing.Any) -> None:
+ """Callback for processing events."""
+
+ machine_db_id = getattr(sample, 'machine_pid', 0)
+ machine_id = self.get_machine_id(machine_db_id)
+ thread_id = self.get_thread_id(machine_id, sample.sample_pid, sample.sample_tid)
+
+ comm = "Unknown_comm"
+ try:
+ if self.session is not None:
+ proc = self.session.find_thread(sample.sample_pid, sample.sample_tid)
+ if proc:
+ comm = proc.comm() or "Unknown"
+ except TypeError:
+ pass
+ comm_id = self.get_comm_id(comm, thread_id)
+
+ dso_id = self.get_dso_id(
+ sample.dso or "Unknown_dso",
+ sample.dso_long_name or "Unknown_dso_long",
+ sample.dso_bid or "",
+ machine_id
+ )
+
+ symbol_id = self.get_symbol_id(
+ dso_id,
+ sample.symbol or "Unknown_symbol",
+ sample.sym_start or 0,
+ sample.sym_end or 0
+ )
+
+ call_path_id = 0
+ if hasattr(sample, 'callchain') and sample.callchain:
+ parent_id = 0
+ for node in reversed(sample.callchain):
+ node_dso = getattr(node, 'dso', None) or getattr(node, 'map', None)
+ node_symbol = getattr(node, 'symbol', None) or getattr(node, 'sym', None)
+
+ dso_name = "Unknown_dso"
+ if node_dso:
+ dso_name = (node_dso if isinstance(node_dso, str)
+ else getattr(node_dso, 'name', "Unknown_dso") or "Unknown_dso")
+
+ symbol_name = "Unknown_symbol"
+ if node_symbol:
+ symbol_name = (node_symbol if isinstance(node_symbol, str)
+ else getattr(node_symbol, 'name', "Unknown_symbol")
+ or "Unknown_symbol")
+
+ node_dso_id = self.get_dso_id(dso_name, dso_name, "", machine_id)
+ node_symbol_id = self.get_symbol_id(node_dso_id, symbol_name, 0, 0)
+
+ parent_id = self.get_call_path_id(parent_id, node_symbol_id, node.ip)
+ call_path_id = parent_id
+
+ sample_id = self.next_id['sample']
+ evsel_name = str(sample.evsel)
+ if evsel_name.startswith("evsel(") and evsel_name.endswith(")"):
+ evsel_name = evsel_name[6:-1]
+ self.write_sample(sample_id,
+ self.get_event_id(evsel_name),
+ machine_id, thread_id, comm_id, dso_id, symbol_id, sample,
+ call_path_id)
+
+ event_name_str = evsel_name
+ if hasattr(sample, "raw_buf"):
+ if event_name_str == "ptwrite":
+ self.write_ptwrite(sample_id, sample.raw_buf)
+ elif event_name_str == "cbr":
+ self.write_cbr(sample_id, sample.raw_buf)
+ elif event_name_str == "mwait":
+ self.write_mwait(sample_id, sample.raw_buf)
+ elif event_name_str == "pwre":
+ self.write_pwre(sample_id, sample.raw_buf)
+ elif event_name_str == "exstop":
+ self.write_exstop(sample_id, sample.raw_buf)
+ elif event_name_str == "pwrx":
+ self.write_pwrx(sample_id, sample.raw_buf)
+
+ self.next_id['sample'] += 1
+
+ def finalize(self) -> None:
+ """Copy files to database and add keys/views."""
+ print("Copying to database...")
+ for name, f in self.files.items():
+ self.close_output_file(f)
+
+
+ table_mapping = {
+ 'evsel': 'selected_events',
+ 'machine': 'machines',
+ 'thread': 'threads',
+ 'comm': 'comms',
+ 'comm_thread': 'comm_threads',
+ 'dso': 'dsos',
+ 'symbol': 'symbols',
+ 'branch_type': 'branch_types',
+ 'sample': 'samples',
+ 'call': 'calls',
+ 'call_path': 'call_paths',
+ 'context_switches': 'context_switches',
+ 'ptwrite': 'ptwrite',
+ 'cbr': 'cbr',
+ 'mwait': 'mwait',
+ 'pwre': 'pwre',
+ 'exstop': 'exstop',
+ 'pwrx': 'pwrx'
+ }
+ for name, f in self.files.items():
+ table_name = table_mapping.get(name, name + "s")
+ self.copy_output_file(f.name, table_name)
+
+
+ print("Removing intermediate files...")
+ for name, f in self.files.items():
+ os.unlink(f.name)
+ shutil.rmtree(self.output_dir_name, ignore_errors=True)
+
+ print("Adding primary keys")
+ self.do_query("ALTER TABLE selected_events ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE machines ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE threads ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE comms ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE comm_threads ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE dsos ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE symbols ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE branch_types ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE samples ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE call_paths ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE calls ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE ptwrite ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE cbr ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE mwait ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE pwre ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE exstop ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE pwrx ADD PRIMARY KEY (id)")
+ self.do_query("ALTER TABLE context_switches ADD PRIMARY KEY (id)")
+
+ print("Adding foreign keys")
+ self.do_query('ALTER TABLE threads '
+ 'ADD CONSTRAINT machinefk FOREIGN KEY (machine_id) REFERENCES machines (id),'
+ 'ADD CONSTRAINT processfk FOREIGN KEY (process_id) REFERENCES threads (id)')
+ self.do_query('ALTER TABLE comms '
+ 'ADD CONSTRAINT threadfk FOREIGN KEY (c_thread_id) REFERENCES threads (id)')
+ self.do_query('ALTER TABLE comm_threads '
+ 'ADD CONSTRAINT commfk FOREIGN KEY (comm_id) REFERENCES comms (id),'
+ 'ADD CONSTRAINT threadfk FOREIGN KEY (thread_id) REFERENCES threads (id)')
+ self.do_query('ALTER TABLE dsos '
+ 'ADD CONSTRAINT machinefk FOREIGN KEY (machine_id) REFERENCES machines (id)')
+ self.do_query('ALTER TABLE symbols '
+ 'ADD CONSTRAINT dsofk FOREIGN KEY (dso_id) REFERENCES dsos (id)')
+ self.do_query('ALTER TABLE samples '
+ 'ADD CONSTRAINT evselfk FOREIGN KEY (evsel_id) '
+ 'REFERENCES selected_events (id),'
+ 'ADD CONSTRAINT machinefk FOREIGN KEY (machine_id) REFERENCES machines (id),'
+ 'ADD CONSTRAINT threadfk FOREIGN KEY (thread_id) REFERENCES threads (id),'
+ 'ADD CONSTRAINT commfk FOREIGN KEY (comm_id) REFERENCES comms (id),'
+ 'ADD CONSTRAINT dsofk FOREIGN KEY (dso_id) REFERENCES dsos (id),'
+ 'ADD CONSTRAINT symbolfk FOREIGN KEY (symbol_id) REFERENCES symbols (id),'
+ 'ADD CONSTRAINT todsofk FOREIGN KEY (to_dso_id) REFERENCES dsos (id),'
+ 'ADD CONSTRAINT tosymbolfk FOREIGN KEY (to_symbol_id) '
+ 'REFERENCES symbols (id)')
+ self.do_query('ALTER TABLE call_paths '
+ 'ADD CONSTRAINT parentfk FOREIGN KEY (parent_id) REFERENCES call_paths (id),'
+ 'ADD CONSTRAINT symbolfk FOREIGN KEY (symbol_id) REFERENCES symbols (id)')
+ self.do_query('ALTER TABLE calls '
+ 'ADD CONSTRAINT threadfk FOREIGN KEY (thread_id) REFERENCES threads (id),'
+ 'ADD CONSTRAINT commfk FOREIGN KEY (comm_id) REFERENCES comms (id),'
+ 'ADD CONSTRAINT call_pathfk FOREIGN KEY (call_path_id) '
+ 'REFERENCES call_paths (id),'
+ 'ADD CONSTRAINT callfk FOREIGN KEY (call_id) REFERENCES samples (id),'
+ 'ADD CONSTRAINT returnfk FOREIGN KEY (return_id) REFERENCES samples (id),'
+ 'ADD CONSTRAINT parent_call_pathfk FOREIGN KEY (parent_call_path_id) '
+ 'REFERENCES call_paths (id)')
+ self.do_query('CREATE INDEX pcpid_idx ON calls (parent_call_path_id)')
+ self.do_query('CREATE INDEX pid_idx ON calls (parent_id)')
+ self.do_query('ALTER TABLE comms ADD has_calls boolean')
+ self.do_query('UPDATE comms SET has_calls = TRUE WHERE comms.id IN '
+ '(SELECT DISTINCT comm_id FROM calls)')
+ self.do_query('ALTER TABLE context_switches '
+ 'ADD CONSTRAINT machinefk FOREIGN KEY (machine_id) REFERENCES machines (id),'
+ 'ADD CONSTRAINT toutfk FOREIGN KEY (thread_out_id) REFERENCES threads (id),'
+ 'ADD CONSTRAINT tinfk FOREIGN KEY (thread_in_id) REFERENCES threads (id),'
+ 'ADD CONSTRAINT coutfk FOREIGN KEY (comm_out_id) REFERENCES comms (id),'
+ 'ADD CONSTRAINT cinfk FOREIGN KEY (comm_in_id) REFERENCES comms (id)')
+ self.do_query('ALTER TABLE ptwrite '
+ 'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
+ self.do_query('ALTER TABLE cbr '
+ 'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
+ self.do_query('ALTER TABLE mwait '
+ 'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
+ self.do_query('ALTER TABLE pwre '
+ 'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
+ self.do_query('ALTER TABLE exstop '
+ 'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
+ self.do_query('ALTER TABLE pwrx '
+ 'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
+
+ print("Creating views...")
+ self.do_query(
+ "CREATE VIEW machines_view AS "
+ "SELECT id, pid, root_dir, "
+ "CASE WHEN id=0 THEN 'unknown' WHEN pid=-1 THEN 'host' ELSE 'guest' END "
+ "AS host_or_guest FROM machines"
+ )
+ self.do_query(
+ "CREATE VIEW dsos_view AS "
+ "SELECT id, machine_id, "
+ "(SELECT host_or_guest FROM machines_view WHERE id = machine_id) AS host_or_guest, "
+ "short_name, long_name, build_id FROM dsos"
+ )
+ self.do_query(
+ "CREATE VIEW symbols_view AS "
+ "SELECT id, name, (SELECT short_name FROM dsos WHERE id=dso_id) AS dso, "
+ "dso_id, sym_start, sym_end, "
+ "CASE WHEN binding=0 THEN 'local' WHEN binding=1 THEN 'global' ELSE 'weak' END "
+ "AS binding FROM symbols"
+ )
+ self.do_query(
+ "CREATE VIEW threads_view AS "
+ "SELECT id, machine_id, "
+ "(SELECT host_or_guest FROM machines_view WHERE id = machine_id) AS host_or_guest, "
+ "process_id, pid, tid FROM threads"
+ )
+ self.do_query(
+ "CREATE VIEW comm_threads_view AS "
+ "SELECT comm_id, (SELECT comm FROM comms WHERE id = comm_id) AS command, "
+ "thread_id, (SELECT pid FROM threads WHERE id = thread_id) AS pid, "
+ "(SELECT tid FROM threads WHERE id = thread_id) AS tid FROM comm_threads"
+ )
+ self.do_query(
+ "CREATE VIEW call_paths_view AS "
+ "SELECT c.id, to_hex(c.ip) AS ip, c.symbol_id, "
+ "(SELECT name FROM symbols WHERE id = c.symbol_id) AS symbol, "
+ "(SELECT dso_id FROM symbols WHERE id = c.symbol_id) AS dso_id, "
+ "(SELECT dso FROM symbols_view WHERE id = c.symbol_id) AS dso_short_name, "
+ "c.parent_id, to_hex(p.ip) AS parent_ip, p.symbol_id AS parent_symbol_id, "
+ "(SELECT name FROM symbols WHERE id = p.symbol_id) AS parent_symbol, "
+ "(SELECT dso_id FROM symbols WHERE id = p.symbol_id) AS parent_dso_id, "
+ "(SELECT dso FROM symbols_view WHERE id = p.symbol_id) AS parent_dso_short_name "
+ "FROM call_paths c LEFT JOIN call_paths p ON p.id = c.parent_id"
+ )
+ self.do_query(
+ "CREATE VIEW calls_view AS "
+ "SELECT calls.id, thread_id, "
+ "(SELECT pid FROM threads WHERE id = thread_id) AS pid, "
+ "(SELECT tid FROM threads WHERE id = thread_id) AS tid, "
+ "(SELECT comm FROM comms WHERE id = comm_id) AS command, "
+ "call_path_id, to_hex(ip) AS ip, symbol_id, "
+ "(SELECT name FROM symbols WHERE id = symbol_id) AS symbol, "
+ "call_time, return_time, return_time - call_time AS elapsed_time, "
+ "branch_count, insn_count, cyc_count, "
+ "CASE WHEN cyc_count=0 THEN CAST(0 AS FLOAT) "
+ "ELSE CAST(insn_count AS FLOAT) / cyc_count END AS IPC, "
+ "call_id, return_id, "
+ "CASE WHEN flags=0 THEN '' WHEN flags=1 THEN 'no call' "
+ "WHEN flags=2 THEN 'no return' WHEN flags=3 THEN 'no call/return' "
+ "WHEN flags=4 THEN 'jmp' ELSE CAST(flags AS VARCHAR(6)) END AS flags, "
+ "parent_call_path_id, calls.parent_id "
+ "FROM calls INNER JOIN call_paths ON call_paths.id = call_path_id"
+ )
+ self.do_query(
+ "CREATE VIEW samples_view AS "
+ "SELECT id, time, cpu, "
+ "(SELECT pid FROM threads WHERE id = thread_id) AS pid, "
+ "(SELECT tid FROM threads WHERE id = thread_id) AS tid, "
+ "(SELECT comm FROM comms WHERE id = comm_id) AS command, "
+ "(SELECT name FROM selected_events WHERE id = evsel_id) AS event, "
+ "to_hex(ip) AS ip_hex, "
+ "(SELECT name FROM symbols WHERE id = symbol_id) AS symbol, sym_offset, "
+ "(SELECT short_name FROM dsos WHERE id = dso_id) AS dso_short_name, "
+ "to_hex(to_ip) AS to_ip_hex, "
+ "(SELECT name FROM symbols WHERE id = to_symbol_id) AS to_symbol, to_sym_offset, "
+ "(SELECT short_name FROM dsos WHERE id = to_dso_id) AS to_dso_short_name, "
+ "(SELECT name FROM branch_types WHERE id = branch_type) AS branch_type_name, "
+ "in_tx, call_path_id, insn_count, cyc_count, "
+ "CASE WHEN cyc_count=0 THEN CAST(0 AS FLOAT) "
+ "ELSE CAST(insn_count AS FLOAT) / cyc_count END AS IPC, flags FROM samples"
+ )
+ self.do_query(
+ "CREATE VIEW context_switches_view AS "
+ "SELECT context_switches.id, context_switches.machine_id, "
+ "context_switches.time, context_switches.cpu, "
+ "th_out.pid AS pid_out, th_out.tid AS tid_out, comm_out.comm AS comm_out, "
+ "th_in.pid AS pid_in, th_in.tid AS tid_in, comm_in.comm AS comm_in, "
+ "CASE WHEN flags=0 THEN 'in' WHEN flags=1 THEN 'out' "
+ "WHEN flags=3 THEN 'out preempt' ELSE CAST(flags AS VARCHAR(6)) END AS flags "
+ "FROM context_switches "
+ "INNER JOIN threads AS th_out ON th_out.id = context_switches.thread_out_id "
+ "INNER JOIN threads AS th_in ON th_in.id = context_switches.thread_in_id "
+ "INNER JOIN comms AS comm_out ON comm_out.id = context_switches.comm_out_id "
+ "INNER JOIN comms AS comm_in ON comm_in.id = context_switches.comm_in_id"
+ )
+ self.do_query(
+ "CREATE VIEW ptwrite_view AS "
+ "SELECT ptwrite.id, time, cpu, to_hex(payload) AS payload_hex, "
+ "CASE WHEN exact_ip=FALSE THEN 'False' ELSE 'True' END AS exact_ip "
+ "FROM ptwrite INNER JOIN samples ON samples.id = ptwrite.id"
+ )
+ self.do_query(
+ "CREATE VIEW cbr_view AS "
+ "SELECT cbr.id, time, cpu, cbr, mhz, percent "
+ "FROM cbr INNER JOIN samples ON samples.id = cbr.id"
+ )
+ self.do_query(
+ "CREATE VIEW mwait_view AS "
+ "SELECT mwait.id, time, cpu, to_hex(hints) AS hints_hex, "
+ "to_hex(extensions) AS extensions_hex "
+ "FROM mwait INNER JOIN samples ON samples.id = mwait.id"
+ )
+ self.do_query(
+ "CREATE VIEW pwre_view AS "
+ "SELECT pwre.id, time, cpu, cstate, subcstate, "
+ "CASE WHEN hw=FALSE THEN 'False' ELSE 'True' END AS hw "
+ "FROM pwre INNER JOIN samples ON samples.id = pwre.id"
+ )
+ self.do_query(
+ "CREATE VIEW exstop_view AS "
+ "SELECT exstop.id, time, cpu, "
+ "CASE WHEN exact_ip=FALSE THEN 'False' ELSE 'True' END AS exact_ip "
+ "FROM exstop INNER JOIN samples ON samples.id = exstop.id"
+ )
+ self.do_query(
+ "CREATE VIEW pwrx_view AS "
+ "SELECT pwrx.id, time, cpu, deepest_cstate, last_cstate, "
+ "CASE WHEN wake_reason=1 THEN 'Interrupt' "
+ "WHEN wake_reason=2 THEN 'Timer Deadline' "
+ "WHEN wake_reason=4 THEN 'Monitored Address' "
+ "WHEN wake_reason=8 THEN 'HW' "
+ "ELSE CAST ( wake_reason AS VARCHAR(2) ) END AS wake_reason "
+ "FROM pwrx INNER JOIN samples ON samples.id = pwrx.id"
+ )
+ self.do_query(
+ "CREATE VIEW power_events_view AS "
+ "SELECT samples.id, samples.time, samples.cpu, selected_events.name AS event, "
+ "FORMAT('%6s', cbr.cbr) AS cbr, FORMAT('%6s', cbr.mhz) AS MHz, "
+ "FORMAT('%5s', cbr.percent) AS percent, to_hex(mwait.hints) AS hints_hex, "
+ "to_hex(mwait.extensions) AS extensions_hex, FORMAT('%3s', pwre.cstate) AS cstate, "
+ "FORMAT('%3s', pwre.subcstate) AS subcstate, "
+ "CASE WHEN pwre.hw=FALSE THEN 'False' WHEN pwre.hw=TRUE THEN 'True' "
+ "ELSE NULL END AS hw, "
+ "CASE WHEN exstop.exact_ip=FALSE THEN 'False' WHEN exstop.exact_ip=TRUE THEN 'True' "
+ "ELSE NULL END AS exact_ip, FORMAT('%3s', pwrx.deepest_cstate) AS deepest_cstate, "
+ "FORMAT('%3s', pwrx.last_cstate) AS last_cstate, "
+ "CASE WHEN pwrx.wake_reason=1 THEN 'Interrupt' "
+ "WHEN wake_reason=2 THEN 'Timer Deadline' "
+ "WHEN wake_reason=4 THEN 'Monitored Address' "
+ "WHEN wake_reason=8 THEN 'HW' "
+ "ELSE FORMAT('%2s', pwrx.wake_reason) END AS wake_reason "
+ "FROM cbr FULL JOIN mwait ON mwait.id = cbr.id "
+ "FULL JOIN pwre ON pwre.id = cbr.id FULL JOIN exstop ON exstop.id = cbr.id "
+ "FULL JOIN pwrx ON pwrx.id = cbr.id INNER JOIN samples ON samples.id = "
+ "coalesce(cbr.id, mwait.id, pwre.id, exstop.id, pwrx.id) "
+ "INNER JOIN selected_events ON selected_events.id = samples.evsel_id "
+ "ORDER BY samples.id"
+ )
+
+
+if __name__ == "__main__":
+ ap = argparse.ArgumentParser(
+ description="Export perf data to a postgresql database")
+ ap.add_argument("-i", "--input", default="perf.data",
+ help="Input file name")
+ ap.add_argument("-o", "--output", required=True,
+ help="Output database name")
+ ap.add_argument("--itrace", default=None,
+ help="itrace options, e.g. cr")
+ args = ap.parse_args()
+
+ exporter = PostgresExporter(args.output)
+
+ session = None
+ succeeded = False
+ caught_error = False
+ try:
+ exporter.setup_db()
+ session = perf.session(perf.data(args.input),
+ context_switch=exporter.process_context_switch,
+ sample=exporter.process_event,
+ call_return=exporter.process_call_return,
+ itrace=args.itrace)
+ exporter.session = session
+ session.process_events()
+ exporter.session = None
+ exporter.finalize()
+ print(f"Successfully exported to {args.output}")
+ succeeded = True
+ except (OSError, RuntimeError, ValueError, KeyboardInterrupt) as e:
+ if not isinstance(e, (KeyboardInterrupt, SystemExit)):
+ import traceback
+ traceback.print_exc()
+ caught_error = True
+ finally:
+ exporter.session = None
+ for out_file in exporter.files.values():
+ try:
+ if not out_file.closed:
+ out_file.close()
+ except OSError:
+ pass
+ exporter.disconnect()
+ if not succeeded:
+ if (getattr(exporter, 'created_output_dir', False) and
+ os.path.exists(exporter.output_dir_name)):
+ shutil.rmtree(exporter.output_dir_name, ignore_errors=True)
+ if caught_error:
+ sys.exit(1)
diff --git a/tools/perf/tests/shell/test_export_to_postgresql_python.sh b/tools/perf/tests/shell/test_export_to_postgresql_python.sh
new file mode 100755
index 000000000000..757ef867fb93
--- /dev/null
+++ b/tools/perf/tests/shell/test_export_to_postgresql_python.sh
@@ -0,0 +1,119 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# export-to-postgresql python test (exclusive)
+
+set -e
+
+shelldir=$(dirname "$0")
+# shellcheck source=lib/setup_python.sh
+. "${shelldir}"/lib/setup_python.sh
+
+if ! "$PYTHON" -c 'import perf' > /dev/null 2>&1; then
+ echo "Skipping test, perf python module not found"
+ exit 2
+fi
+
+# If we don't have psql, we can't test
+if ! command -v psql >/dev/null 2>&1; then
+ echo "Skipping test, psql not found"
+ exit 2
+fi
+
+# Check if we can connect to postgres and create a database
+if ! psql -c "SELECT 1" postgres >/dev/null 2>&1; then
+ echo "Skipping test, cannot connect to postgres"
+ exit 2
+fi
+
+script_dir="$(dirname "$0")/../../python"
+script_path="${script_dir}/export-to-postgresql.py"
+
+if [ ! -f "$script_path" ]; then
+ echo "Skipping test, export-to-postgresql.py not found at $script_path"
+ exit 2
+fi
+
+err=0
+temp_dir=""
+temp_data=""
+temp_db=""
+
+cleanup() {
+ [ -n "${temp_dir}" ] && rm -rf "${temp_dir}"
+ psql -c "DROP DATABASE IF EXISTS ${temp_db}" postgres >/dev/null 2>&1 || true
+}
+
+trap 'cleanup' EXIT TERM INT
+
+temp_dir=$(mktemp -d /tmp/perf.pg.XXXXXX)
+temp_data="${temp_dir}/perf.data"
+temp_db="perf_test_db_$$"
+
+test_file_mode() {
+ echo "Testing export-to-postgresql.py..."
+
+ # Verify that invalid URI / key-value database names are rejected
+ if "$PYTHON" "$script_path" -i /dev/null -o "dbname=foo host=evil" >/dev/null 2>&1; then
+ echo "Connection parameter injection check failed."
+ err=1
+ fi
+
+ # Generate events with callchains and context switches
+ if ! perf record -g --switch-events -o "${temp_data}" \
+ -- perf test -w noploop >/dev/null 2>&1 && \
+ ! perf record -g -o "${temp_data}" -- perf test -w noploop >/dev/null 2>&1; then
+ echo "Skipping test, perf record failed"
+ exit 2
+ fi
+
+ # Ensure clean db start
+ psql -c "DROP DATABASE IF EXISTS ${temp_db}" postgres >/dev/null 2>&1 || true
+
+ # Run the script
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" -o "${temp_db}" >/dev/null; then
+ echo "File mode test failed."
+ err=1
+ else
+ # Check DB
+ if ! psql -d "${temp_db}" -t -c 'SELECT COUNT(*) FROM samples WHERE id > 0;' | \
+ grep -q '[1-9]'; then
+ echo "PostgreSQL validation failed."
+ err=1
+ else
+ echo "File mode test passed."
+ fi
+ fi
+}
+
+test_intel_pt() {
+ echo "Testing export-to-postgresql.py with intel_pt..."
+
+ psql -c "DROP DATABASE IF EXISTS ${temp_db}" postgres >/dev/null 2>&1 || true
+ rm -f "${temp_data}"
+ # Generate some intel_pt events; use a subshell that waits for uname
+ if ! perf record -B -N --no-bpf-event -e intel_pt//u -o "${temp_data}" \
+ -- sh -c "uname; true" >/dev/null 2>&1; then
+ echo "Skipping intel_pt test, intel_pt not available."
+ return 0
+ fi
+
+ # Run the script with --itrace cr to synthesize call_returns
+ if ! "$PYTHON" "$script_path" -i "${temp_data}" -o "${temp_db}" --itrace cr >/dev/null; then
+ echo "intel_pt file mode test failed."
+ err=1
+ else
+ # Check DB for calls
+ if ! psql -d "${temp_db}" -t -c 'SELECT COUNT(*) FROM calls WHERE id > 0;' | \
+ grep -q '[1-9]'; then
+ echo "PostgreSQL intel_pt validation failed (no calls found)."
+ err=1
+ else
+ echo "intel_pt test passed (cr validated)."
+ fi
+ fi
+}
+
+test_file_mode
+test_intel_pt
+
+exit $err
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 44/49] perf python: Move and clean up exported-sql-viewer.py
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (42 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 43/49] perf python: Port export-to-postgresql " Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 45/49] perf python: Move and clean up parallel-perf.py Ian Rogers
` (4 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Move exported-sql-viewer.py from tools/perf/scripts/python/ to
tools/perf/python/ as it is a standalone PySide/PyQt GUI application
for viewing SQLite and PostgreSQL databases exported by perf script.
Improvements and cleanups compared to the legacy script:
- Add support for PySide6 (falling back to PySide2 and PySide) loaded
dynamically via importlib, and provide stub base classes (_QtBase,
_QtWidget) so static analysis (mypy and pylint) and --help-only
execution succeed when Qt bindings are not installed.
- Fix a Python 3 NameError crash in HBoxLayout where expanded_mark and
not_expanded_mark called Python 2's removed unicode() builtin instead
of chr(0x25BC) and chr(0x25B6).
- Replace instance-level method monkey-patching
(self.mousePressEvent = self.MousePressEvent) on SwitchGraphView with
proper class methods gated by self.rb_enabled so Qt virtual method
dispatch works cleanly across PySide2 and PySide6.
- Remove Python 2 constructs (cPickle, xrange), wildcard imports, bare
'except:' clauses, and variable names shadowing Python builtins.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/exported-sql-viewer.py | 5104 +++++++++++++++++
.../scripts/python/exported-sql-viewer.py | 5030 ----------------
2 files changed, 5104 insertions(+), 5030 deletions(-)
create mode 100755 tools/perf/python/exported-sql-viewer.py
delete mode 100755 tools/perf/scripts/python/exported-sql-viewer.py
diff --git a/tools/perf/python/exported-sql-viewer.py b/tools/perf/python/exported-sql-viewer.py
new file mode 100755
index 000000000000..bdd03c7fd5b6
--- /dev/null
+++ b/tools/perf/python/exported-sql-viewer.py
@@ -0,0 +1,5104 @@
+#!/usr/bin/env python
+# SPDX-License-Identifier: GPL-2.0
+"""exported-sql-viewer.py: view data from sql database."""
+# Copyright (c) 2014-2018, Intel Corporation.
+
+# To use this script you will need to have exported data using either the
+# export-to-sqlite.py or the export-to-postgresql.py script. Refer to those
+# scripts for details.
+#
+# Following on from the example in the export scripts, a
+# call-graph can be displayed for the pt_example database like this:
+#
+# python exported-sql-viewer.py pt_example
+#
+# Note that for PostgreSQL, this script supports connecting to remote databases
+# by setting hostname, port, username, password, and dbname e.g.
+#
+# python exported-sql-viewer.py \
+# "hostname=myhost username=myuser password=mypassword dbname=pt_example"
+#
+# The result is a GUI window with a tree representing a context-sensitive
+# call-graph. Expanding a couple of levels of the tree and adjusting column
+# widths to suit will display something like:
+#
+# Call Graph: pt_example
+# Call Path Object Count Time(ns) Time(%) Branch Count Branch Count(%)
+# v- ls
+# v- 2638:2638
+# v- _start ld-2.19.so 1 10074071 100.0 211135 100.0
+# |- unknown unknown 1 13198 0.1 1 0.0
+# >- _dl_start ld-2.19.so 1 1400980 13.9 19637 9.3
+# >- _d_linit_internal ld-2.19.so 1 448152 4.4 11094 5.3
+# v-__libc_start_main@plt ls 1 8211741 81.5 180397 85.4
+# >- _dl_fixup ld-2.19.so 1 7607 0.1 108 0.1
+# >- __cxa_atexit libc-2.19.so 1 11737 0.1 10 0.0
+# >- __libc_csu_init ls 1 10354 0.1 10 0.0
+# |- _setjmp libc-2.19.so 1 0 0.0 4 0.0
+# v- main ls 1 8182043 99.6 180254 99.9
+#
+# Points to note:
+# The top level is a command name (comm)
+# The next level is a thread (pid:tid)
+# Subsequent levels are functions
+# 'Count' is the number of calls
+# 'Time' is the elapsed time until the function returns
+# Percentages are relative to the level above
+# 'Branch Count' is the total number of branches for that function and all
+# functions that it calls
+
+# There is also a "All branches" report, which displays branches and
+# possibly disassembly. However, presently, the only supported disassembler is
+# Intel XED, and additionally the object code must be present in perf build ID
+# cache. To use Intel XED, libxed.so must be present. To build and install
+# libxed.so:
+# git clone https://github.com/intelxed/mbuild.git mbuild
+# git clone https://github.com/intelxed/xed
+# cd xed
+# ./mfile.py --share
+# sudo ./mfile.py --prefix=/usr/local install
+# sudo ldconfig
+#
+# Example report:
+#
+# Time CPU Command PID TID Branch Type In Tx Branch
+# 8107675239590 2 ls 22011 22011 return from interrupt No ffffffff86a00a67 native_irq_return_iret ([kernel]) -> 7fab593ea260 _start (ld-2.19.so)
+# 7fab593ea260 48 89 e7 mov %rsp, %rdi
+# 8107675239899 2 ls 22011 22011 hardware interrupt No 7fab593ea260 _start (ld-2.19.so) -> ffffffff86a012e0 page_fault ([kernel])
+# 8107675241900 2 ls 22011 22011 return from interrupt No ffffffff86a00a67 native_irq_return_iret ([kernel]) -> 7fab593ea260 _start (ld-2.19.so)
+# 7fab593ea260 48 89 e7 mov %rsp, %rdi
+# 7fab593ea263 e8 c8 06 00 00 callq 0x7fab593ea930
+# 8107675241900 2 ls 22011 22011 call No 7fab593ea263 _start+0x3 (ld-2.19.so) -> 7fab593ea930 _dl_start (ld-2.19.so)
+# 7fab593ea930 55 pushq %rbp
+# 7fab593ea931 48 89 e5 mov %rsp, %rbp
+# 7fab593ea934 41 57 pushq %r15
+# 7fab593ea936 41 56 pushq %r14
+# 7fab593ea938 41 55 pushq %r13
+# 7fab593ea93a 41 54 pushq %r12
+# 7fab593ea93c 53 pushq %rbx
+# 7fab593ea93d 48 89 fb mov %rdi, %rbx
+# 7fab593ea940 48 83 ec 68 sub $0x68, %rsp
+# 7fab593ea944 0f 31 rdtsc
+# 7fab593ea946 48 c1 e2 20 shl $0x20, %rdx
+# 7fab593ea94a 89 c0 mov %eax, %eax
+# 7fab593ea94c 48 09 c2 or %rax, %rdx
+# 7fab593ea94f 48 8b 05 1a 15 22 00 movq 0x22151a(%rip), %rax
+# 8107675242232 2 ls 22011 22011 hardware interrupt No 7fab593ea94f _dl_start+0x1f (ld-2.19.so) -> ffffffff86a012e0 page_fault ([kernel])
+# 8107675242900 2 ls 22011 22011 return from interrupt No ffffffff86a00a67 native_irq_return_iret ([kernel]) -> 7fab593ea94f _dl_start+0x1f (ld-2.19.so)
+# 7fab593ea94f 48 8b 05 1a 15 22 00 movq 0x22151a(%rip), %rax
+# 7fab593ea956 48 89 15 3b 13 22 00 movq %rdx, 0x22133b(%rip)
+# 8107675243232 2 ls 22011 22011 hardware interrupt No 7fab593ea956 _dl_start+0x26 (ld-2.19.so) -> ffffffff86a012e0 page_fault ([kernel])
+
+from __future__ import print_function
+
+import sys
+# Only change warnings if the python -W option was not used
+if not sys.warnoptions:
+ import warnings
+ # PySide2 causes deprecation warnings, ignore them.
+ warnings.filterwarnings("ignore", category=DeprecationWarning)
+import argparse
+import weakref
+import threading
+import pickle
+glb_nsz = 16
+import re
+import os
+import random
+import copy
+import math
+from libxed import LibXED
+
+import importlib
+from typing import Any
+
+class _QtBase:
+ def __init__(self, *args: Any, **kwargs: Any) -> None:
+ pass
+ def __call__(self, *args: Any, **kwargs: Any) -> Any:
+ return self
+ def __getattr__(self, name: str) -> Any:
+ return _QtBase()
+ def mousePressEvent(self, event: Any) -> None:
+ pass
+ def mouseMoveEvent(self, event: Any) -> None:
+ pass
+ def mouseReleaseEvent(self, event: Any) -> None:
+ pass
+ def resizeEvent(self, event: Any) -> None:
+ pass
+ def changeEvent(self, event: Any) -> None:
+ pass
+
+class _QtWidget(_QtBase):
+ pass
+
+class _QtMdiSubWindow(_QtWidget):
+ pass
+
+QAbstractItemModel: Any = _QtBase
+QAbstractItemView: Any = _QtBase
+QAbstractTableModel: Any = _QtBase
+QAction: Any = _QtBase
+QApplication: Any = _QtBase
+QCheckBox: Any = _QtBase
+QColor: Any = _QtBase
+QComboBox: Any = _QtBase
+QDialog: Any = _QtWidget
+QEvent: Any = _QtBase
+QFont: Any = _QtBase
+QFontMetrics: Any = _QtBase
+QGraphicsItem: Any = _QtBase
+QGraphicsLineItem: Any = _QtBase
+QGraphicsScene: Any = _QtBase
+QGraphicsSimpleTextItem: Any = _QtBase
+QGraphicsView: Any = _QtBase
+QGridLayout: Any = _QtBase
+QHBoxLayout: Any = _QtBase
+QKeySequence: Any = _QtBase
+QLabel: Any = _QtBase
+QLineEdit: Any = _QtBase
+QMainWindow: Any = _QtWidget
+QMdiArea: Any = _QtBase
+QMdiSubWindow: Any = _QtMdiSubWindow
+QMenu: Any = _QtBase
+QMessageBox: Any = _QtBase
+QModelIndex: Any = _QtBase
+QObject: Any = _QtBase
+QPalette: Any = _QtBase
+QPoint: Any = _QtBase
+QPointF: Any = _QtBase
+QProgressBar: Any = _QtBase
+QPushButton: Any = _QtBase
+QRect: Any = _QtBase
+QRectF: Any = _QtBase
+QRubberBand: Any = _QtBase
+QSize: Any = _QtBase
+QSizePolicy: Any = _QtBase
+QSortFilterProxyModel: Any = _QtBase
+QSpinBox: Any = _QtBase
+QSplitter: Any = _QtWidget
+QSqlDatabase: Any = _QtBase
+QSqlQuery: Any = _QtBase
+QStyle: Any = _QtBase
+Qt: Any = _QtBase
+QTableView: Any = _QtWidget
+QTextBrowser: Any = _QtWidget
+QThread: Any = _QtBase
+QToolButton: Any = _QtBase
+QTreeView: Any = _QtWidget
+QVBoxLayout: Any = _QtBase
+qVersion: Any = _QtBase
+QWidget: Any = _QtWidget
+Signal: Any = _QtBase
+
+pyside_version_1 = False
+for _pkg in ("PySide6", "PySide2"):
+ try:
+ for _mod_name in ("QtCore", "QtGui", "QtSql", "QtWidgets"):
+ _mod = importlib.import_module(f"{_pkg}.{_mod_name}")
+ for _sym in dir(_mod):
+ if not _sym.startswith("_"):
+ globals()[_sym] = getattr(_mod, _sym)
+ break
+ except ImportError:
+ pass
+
+
+from decimal import Decimal, ROUND_HALF_UP
+from ctypes import create_string_buffer, addressof, sizeof, \
+ c_void_p, c_bool, c_char, c_longlong
+from multiprocessing import Process, Array, Value, Event
+
+
+
+def printerr(*args, **keyword_args):
+ print(*args, file=sys.stderr, **keyword_args)
+
+# Data formatting helpers
+
+def tohex(ip):
+ if ip < 0:
+ ip += 1 << 64
+ return "%x" % ip
+
+def offstr(offset):
+ if offset:
+ return "+0x%x" % offset
+ return ""
+
+def dsoname(name):
+ if name == "[kernel.kallsyms]":
+ return "[kernel]"
+ return name
+
+def findnth(s, sub, n, offs=0):
+ pos = s.find(sub)
+ if pos < 0:
+ return pos
+ if n <= 1:
+ return offs + pos
+ return findnth(s[pos + 1:], sub, n - 1, offs + pos + 1)
+
+# Percent to one decimal place
+
+def PercentToOneDP(n, d):
+ if not d:
+ return "0.0"
+ x = (n * Decimal(100)) / d
+ return str(x.quantize(Decimal(".1"), rounding=ROUND_HALF_UP))
+
+# Helper for queries that must not fail
+
+def QueryExec(query, stmt):
+ ret = query.exec_(stmt)
+ if not ret:
+ raise RuntimeError("Query failed: " + query.lastError().text())
+
+# Background thread
+
+class Thread(QThread):
+
+ done = Signal(object)
+
+ def __init__(self, task, param=None, parent=None):
+ super(Thread, self).__init__(parent)
+ self.task = task
+ self.param = param
+
+ def run(self):
+ while True:
+ if self.param is None:
+ done, result = self.task()
+ else:
+ done, result = self.task(self.param)
+ self.done.emit(result)
+ if done:
+ break
+
+# Tree data model
+
+class TreeModel(QAbstractItemModel):
+
+ def __init__(self, glb, params, parent=None):
+ super(TreeModel, self).__init__(parent)
+ self.glb = glb
+ self.params = params
+ self.root = self.GetRoot()
+ self.last_row_read = 0
+
+ def Item(self, parent):
+ if parent.isValid():
+ return parent.internalPointer()
+ else:
+ return self.root
+
+ def rowCount(self, parent):
+ result = self.Item(parent).childCount()
+ if result < 0:
+ result = 0
+ self.dataChanged.emit(parent, parent)
+ return result
+
+ def hasChildren(self, parent):
+ return self.Item(parent).hasChildren()
+
+ def headerData(self, section, orientation, role):
+ if role == Qt.TextAlignmentRole:
+ return self.columnAlignment(section)
+ if role != Qt.DisplayRole:
+ return None
+ if orientation != Qt.Horizontal:
+ return None
+ return self.columnHeader(section)
+
+ def parent(self, child):
+ child_item = child.internalPointer()
+ if child_item is self.root:
+ return QModelIndex()
+ parent_item = child_item.getParentItem()
+ return self.createIndex(parent_item.getRow(), 0, parent_item)
+
+ def index(self, row, column, parent):
+ child_item = self.Item(parent).getChildItem(row)
+ return self.createIndex(row, column, child_item)
+
+ def DisplayData(self, item, index):
+ return item.getData(index.column())
+
+ def FetchIfNeeded(self, row):
+ if row > self.last_row_read:
+ self.last_row_read = row
+ if row + 10 >= self.root.child_count:
+ self.fetcher.Fetch(glb_chunk_sz)
+
+ def columnAlignment(self, _column):
+ return Qt.AlignLeft
+
+ def columnFont(self, _column):
+ return None
+
+ def data(self, index, role):
+ if role == Qt.TextAlignmentRole:
+ return self.columnAlignment(index.column())
+ if role == Qt.FontRole:
+ return self.columnFont(index.column())
+ if role != Qt.DisplayRole:
+ return None
+ item = index.internalPointer()
+ return self.DisplayData(item, index)
+
+# Table data model
+
+class TableModel(QAbstractTableModel):
+
+ def __init__(self, parent=None):
+ super(TableModel, self).__init__(parent)
+ self.child_count = 0
+ self.child_items = []
+ self.last_row_read = 0
+
+ def Item(self, parent):
+ if parent.isValid():
+ return parent.internalPointer()
+ else:
+ return self
+
+ def rowCount(self, _parent):
+ return self.child_count
+
+ def headerData(self, section, orientation, role):
+ if role == Qt.TextAlignmentRole:
+ return self.columnAlignment(section)
+ if role != Qt.DisplayRole:
+ return None
+ if orientation != Qt.Horizontal:
+ return None
+ return self.columnHeader(section)
+
+ def index(self, row, column, _parent):
+ return self.createIndex(row, column, self.child_items[row])
+
+ def DisplayData(self, item, index):
+ return item.getData(index.column())
+
+ def FetchIfNeeded(self, row):
+ if row > self.last_row_read:
+ self.last_row_read = row
+ if row + 10 >= self.child_count:
+ self.fetcher.Fetch(glb_chunk_sz)
+
+ def columnAlignment(self, _column):
+ return Qt.AlignLeft
+
+ def columnFont(self, _column):
+ return None
+
+ def data(self, index, role):
+ if role == Qt.TextAlignmentRole:
+ return self.columnAlignment(index.column())
+ if role == Qt.FontRole:
+ return self.columnFont(index.column())
+ if role != Qt.DisplayRole:
+ return None
+ item = index.internalPointer()
+ return self.DisplayData(item, index)
+
+# Model cache
+
+model_cache: Any = weakref.WeakValueDictionary()
+model_cache_lock = threading.Lock()
+
+def LookupCreateModel(model_name, create_fn):
+ model_cache_lock.acquire()
+ try:
+ model = model_cache[model_name]
+ except KeyError:
+ model = None
+ if model is None:
+ model = create_fn()
+ model_cache[model_name] = model
+ model_cache_lock.release()
+ return model
+
+def LookupModel(model_name):
+ model_cache_lock.acquire()
+ try:
+ model = model_cache[model_name]
+ except KeyError:
+ model = None
+ model_cache_lock.release()
+ return model
+
+# Find bar
+
+class FindBar():
+
+ def __init__(self, parent, finder, is_reg_expr=False):
+ self.finder = finder
+ self.context = []
+ self.last_value = None
+ self.last_pattern = None
+
+ label = QLabel("Find:")
+ label.setSizePolicy(QSizePolicy.Fixed, QSizePolicy.Fixed)
+
+ self.textbox = QComboBox()
+ self.textbox.setEditable(True)
+ self.textbox.currentIndexChanged.connect(self.ValueChanged)
+
+ self.progress = QProgressBar()
+ self.progress.setRange(0, 0)
+ self.progress.hide()
+
+ if is_reg_expr:
+ self.pattern = QCheckBox("Regular Expression")
+ else:
+ self.pattern = QCheckBox("Pattern")
+ self.pattern.setSizePolicy(QSizePolicy.Fixed, QSizePolicy.Fixed)
+
+ self.next_button = QToolButton()
+ self.next_button.setIcon(parent.style().standardIcon(QStyle.SP_ArrowDown))
+ self.next_button.released.connect(lambda: self.NextPrev(1))
+
+ self.prev_button = QToolButton()
+ self.prev_button.setIcon(parent.style().standardIcon(QStyle.SP_ArrowUp))
+ self.prev_button.released.connect(lambda: self.NextPrev(-1))
+
+ self.close_button = QToolButton()
+ self.close_button.setIcon(parent.style().standardIcon(QStyle.SP_DockWidgetCloseButton))
+ self.close_button.released.connect(self.Deactivate)
+
+ self.hbox = QHBoxLayout()
+ self.hbox.setContentsMargins(0, 0, 0, 0)
+
+ self.hbox.addWidget(label)
+ self.hbox.addWidget(self.textbox)
+ self.hbox.addWidget(self.progress)
+ self.hbox.addWidget(self.pattern)
+ self.hbox.addWidget(self.next_button)
+ self.hbox.addWidget(self.prev_button)
+ self.hbox.addWidget(self.close_button)
+
+ self.bar = QWidget()
+ self.bar.setLayout(self.hbox)
+ self.bar.hide()
+
+ def Widget(self):
+ return self.bar
+
+ def Activate(self):
+ self.bar.show()
+ self.textbox.lineEdit().selectAll()
+ self.textbox.setFocus()
+
+ def Deactivate(self):
+ self.bar.hide()
+
+ def Busy(self):
+ self.textbox.setEnabled(False)
+ self.pattern.hide()
+ self.next_button.hide()
+ self.prev_button.hide()
+ self.progress.show()
+
+ def Idle(self):
+ self.textbox.setEnabled(True)
+ self.progress.hide()
+ self.pattern.show()
+ self.next_button.show()
+ self.prev_button.show()
+
+ def Find(self, direction):
+ value = self.textbox.currentText()
+ pattern = self.pattern.isChecked()
+ self.last_value = value
+ self.last_pattern = pattern
+ self.finder.Find(value, direction, pattern, self.context)
+
+ def ValueChanged(self):
+ _value = self.textbox.currentText()
+ pattern = self.pattern.isChecked()
+ index = self.textbox.currentIndex()
+ data = self.textbox.itemData(index)
+ # Store the pattern in the combo box to keep it with the text value
+ if data == None:
+ self.textbox.setItemData(index, pattern)
+ else:
+ self.pattern.setChecked(data)
+ self.Find(0)
+
+ def NextPrev(self, direction):
+ value = self.textbox.currentText()
+ pattern = self.pattern.isChecked()
+ if value != self.last_value:
+ index = self.textbox.findText(value)
+ # Allow for a button press before the value has been added to the combo box
+ if index < 0:
+ index = self.textbox.count()
+ self.textbox.addItem(value, pattern)
+ self.textbox.setCurrentIndex(index)
+ return
+ else:
+ self.textbox.setItemData(index, pattern)
+ elif pattern != self.last_pattern:
+ # Keep the pattern recorded in the combo box up to date
+ index = self.textbox.currentIndex()
+ self.textbox.setItemData(index, pattern)
+ self.Find(direction)
+
+ def NotFound(self):
+ QMessageBox.information(self.bar, "Find", "'" + self.textbox.currentText() + "' not found")
+
+# Context-sensitive call graph data model item base
+
+class CallGraphLevelItemBase(object):
+ def Select(self) -> None:
+ pass
+ def data(self, _column: int, _role: int) -> Any:
+ return None
+
+ def __init__(self, glb, params, row, parent_item):
+ self.glb = glb
+ self.params = params
+ self.row = row
+ self.parent_item = parent_item
+ self.query_done = False
+ self.child_count = 0
+ self.child_items = []
+ if parent_item:
+ self.level = parent_item.level + 1
+ else:
+ self.level = 0
+
+ def getChildItem(self, row):
+ return self.child_items[row]
+
+ def getParentItem(self):
+ return self.parent_item
+
+ def getRow(self):
+ return self.row
+
+ def childCount(self):
+ if not self.query_done:
+ self.Select()
+ if not self.child_count:
+ return -1
+ return self.child_count
+
+ def hasChildren(self):
+ if not self.query_done:
+ return True
+ return self.child_count > 0
+
+ def getData(self, column):
+ return self.data[column]
+
+# Context-sensitive call graph data model level 2+ item base
+
+class CallGraphLevelTwoPlusItemBase(CallGraphLevelItemBase):
+
+ def __init__(self, glb, params, row, comm_id, thread_id, call_path_id, time, insn_cnt, cyc_cnt, branch_count, parent_item):
+ super(CallGraphLevelTwoPlusItemBase, self).__init__(glb, params, row, parent_item)
+ self.comm_id = comm_id
+ self.thread_id = thread_id
+ self.call_path_id = call_path_id
+ self.insn_cnt = insn_cnt
+ self.cyc_cnt = cyc_cnt
+ self.branch_count = branch_count
+ self.time = time
+
+ def Select(self):
+ self.query_done = True
+ query = QSqlQuery(self.glb.db)
+ if self.params.have_ipc:
+ ipc_str = ", SUM(insn_count), SUM(cyc_count)"
+ else:
+ ipc_str = ""
+ QueryExec(query, "SELECT call_path_id, name, short_name, COUNT(calls.id), SUM(return_time - call_time)" + ipc_str + ", SUM(branch_count)"
+ " FROM calls"
+ " INNER JOIN call_paths ON calls.call_path_id = call_paths.id"
+ " INNER JOIN symbols ON call_paths.symbol_id = symbols.id"
+ " INNER JOIN dsos ON symbols.dso_id = dsos.id"
+ " WHERE parent_call_path_id = " + str(self.call_path_id) +
+ " AND comm_id = " + str(self.comm_id) +
+ " AND thread_id = " + str(self.thread_id) +
+ " GROUP BY call_path_id, name, short_name"
+ " ORDER BY call_path_id")
+ while query.next():
+ if self.params.have_ipc:
+ insn_cnt = int(query.value(5))
+ cyc_cnt = int(query.value(6))
+ branch_count = int(query.value(7))
+ else:
+ insn_cnt = 0
+ cyc_cnt = 0
+ branch_count = int(query.value(5))
+ child_item = CallGraphLevelThreeItem(self.glb, self.params, self.child_count, self.comm_id, self.thread_id, query.value(0), query.value(1), query.value(2), query.value(3), int(query.value(4)), insn_cnt, cyc_cnt, branch_count, self)
+ self.child_items.append(child_item)
+ self.child_count += 1
+
+# Context-sensitive call graph data model level three item
+
+class CallGraphLevelThreeItem(CallGraphLevelTwoPlusItemBase):
+
+ def __init__(self, glb, params, row, comm_id, thread_id, call_path_id, name, dso, count, time, insn_cnt, cyc_cnt, branch_count, parent_item):
+ super(CallGraphLevelThreeItem, self).__init__(glb, params, row, comm_id, thread_id, call_path_id, time, insn_cnt, cyc_cnt, branch_count, parent_item)
+ dso = dsoname(dso)
+ if self.params.have_ipc:
+ insn_pcnt = PercentToOneDP(insn_cnt, parent_item.insn_cnt)
+ cyc_pcnt = PercentToOneDP(cyc_cnt, parent_item.cyc_cnt)
+ br_pcnt = PercentToOneDP(branch_count, parent_item.branch_count)
+ ipc = CalcIPC(cyc_cnt, insn_cnt)
+ self.data = [ name, dso, str(count), str(time), PercentToOneDP(time, parent_item.time), str(insn_cnt), insn_pcnt, str(cyc_cnt), cyc_pcnt, ipc, str(branch_count), br_pcnt ]
+ else:
+ self.data = [ name, dso, str(count), str(time), PercentToOneDP(time, parent_item.time), str(branch_count), PercentToOneDP(branch_count, parent_item.branch_count) ]
+ self.dbid = call_path_id
+
+# Context-sensitive call graph data model level two item
+
+class CallGraphLevelTwoItem(CallGraphLevelTwoPlusItemBase):
+
+ def __init__(self, glb, params, row, comm_id, thread_id, pid, tid, parent_item):
+ super(CallGraphLevelTwoItem, self).__init__(glb, params, row, comm_id, thread_id, 1, 0, 0, 0, 0, parent_item)
+ if self.params.have_ipc:
+ self.data = [str(pid) + ":" + str(tid), "", "", "", "", "", "", "", "", "", "", ""]
+ else:
+ self.data = [str(pid) + ":" + str(tid), "", "", "", "", "", ""]
+ self.dbid = thread_id
+
+ def Select(self):
+ super(CallGraphLevelTwoItem, self).Select()
+ for child_item in self.child_items:
+ self.time += child_item.time
+ self.insn_cnt += child_item.insn_cnt
+ self.cyc_cnt += child_item.cyc_cnt
+ self.branch_count += child_item.branch_count
+ for child_item in self.child_items:
+ child_item.data[4] = PercentToOneDP(child_item.time, self.time)
+ if self.params.have_ipc:
+ child_item.data[6] = PercentToOneDP(child_item.insn_cnt, self.insn_cnt)
+ child_item.data[8] = PercentToOneDP(child_item.cyc_cnt, self.cyc_cnt)
+ child_item.data[11] = PercentToOneDP(child_item.branch_count, self.branch_count)
+ else:
+ child_item.data[6] = PercentToOneDP(child_item.branch_count, self.branch_count)
+
+# Context-sensitive call graph data model level one item
+
+class CallGraphLevelOneItem(CallGraphLevelItemBase):
+
+ def __init__(self, glb, params, row, comm_id, comm, parent_item):
+ super(CallGraphLevelOneItem, self).__init__(glb, params, row, parent_item)
+ if self.params.have_ipc:
+ self.data = [comm, "", "", "", "", "", "", "", "", "", "", ""]
+ else:
+ self.data = [comm, "", "", "", "", "", ""]
+ self.dbid = comm_id
+
+ def Select(self):
+ self.query_done = True
+ query = QSqlQuery(self.glb.db)
+ QueryExec(query, "SELECT thread_id, pid, tid"
+ " FROM comm_threads"
+ " INNER JOIN threads ON thread_id = threads.id"
+ " WHERE comm_id = " + str(self.dbid))
+ while query.next():
+ child_item = CallGraphLevelTwoItem(self.glb, self.params, self.child_count, self.dbid, query.value(0), query.value(1), query.value(2), self)
+ self.child_items.append(child_item)
+ self.child_count += 1
+
+# Context-sensitive call graph data model root item
+
+class CallGraphRootItem(CallGraphLevelItemBase):
+
+ def __init__(self, glb, params):
+ super(CallGraphRootItem, self).__init__(glb, params, 0, None)
+ self.dbid = 0
+ self.query_done = True
+ if_has_calls = ""
+ if IsSelectable(glb.db, "comms", columns = "has_calls"):
+ if_has_calls = " WHERE has_calls = " + glb.dbref.TRUE
+ query = QSqlQuery(glb.db)
+ QueryExec(query, "SELECT id, comm FROM comms" + if_has_calls)
+ while query.next():
+ if not query.value(0):
+ continue
+ child_item = CallGraphLevelOneItem(glb, params, self.child_count, query.value(0), query.value(1), self)
+ self.child_items.append(child_item)
+ self.child_count += 1
+
+# Call graph model parameters
+
+class CallGraphModelParams():
+
+ def __init__(self, glb, _parent=None):
+ self.have_ipc = IsSelectable(glb.db, "calls", columns = "insn_count, cyc_count")
+
+# Context-sensitive call graph data model base
+
+class CallGraphModelBase(TreeModel):
+
+ def __init__(self, glb, parent=None):
+ super(CallGraphModelBase, self).__init__(glb, CallGraphModelParams(glb), parent)
+
+ def FindSelect(self, value, pattern, query):
+ if pattern:
+ # postgresql and sqlite pattern patching differences:
+ # postgresql LIKE is case sensitive but sqlite LIKE is not
+ # postgresql LIKE allows % and _ to be escaped with \ but sqlite LIKE does not
+ # postgresql supports ILIKE which is case insensitive
+ # sqlite supports GLOB (text only) which uses * and ? and is case sensitive
+ if not self.glb.dbref.is_sqlite3:
+ # Escape % and _
+ s = value.replace("%", "\\%")
+ s = s.replace("_", "\\_")
+ # Translate * and ? into SQL LIKE pattern characters % and _
+ if sys.version_info[0] == 3:
+ trans = str.maketrans("*?", "%_")
+ else:
+ trans = bytes.maketrans("*?", "%_")
+ match = " LIKE '" + str(s).translate(trans) + "'"
+ else:
+ match = " GLOB '" + str(value) + "'"
+ else:
+ match = " = '" + str(value) + "'"
+ self.DoFindSelect(query, match)
+
+ def Found(self, query, found):
+ if found:
+ return self.FindPath(query)
+ return []
+
+ def FindValue(self, value, pattern, query, last_value, last_pattern):
+ if last_value == value and pattern == last_pattern:
+ found = query.first()
+ else:
+ self.FindSelect(value, pattern, query)
+ found = query.next()
+ return self.Found(query, found)
+
+ def FindNext(self, query):
+ found = query.next()
+ if not found:
+ found = query.first()
+ return self.Found(query, found)
+
+ def FindPrev(self, query):
+ found = query.previous()
+ if not found:
+ found = query.last()
+ return self.Found(query, found)
+
+ def FindThread(self, c):
+ if c.direction == 0 or c.value != c.last_value or c.pattern != c.last_pattern:
+ ids = self.FindValue(c.value, c.pattern, c.query, c.last_value, c.last_pattern)
+ elif c.direction > 0:
+ ids = self.FindNext(c.query)
+ else:
+ ids = self.FindPrev(c.query)
+ return (True, ids)
+
+ def Find(self, value, direction, pattern, context, callback):
+ class Context():
+ def __init__(self, *x):
+ self.value, self.direction, self.pattern, self.query, self.last_value, self.last_pattern = x
+ def Update(self, *x):
+ self.value, self.direction, self.pattern, self.last_value, self.last_pattern = x + (self.value, self.pattern)
+ if len(context):
+ context[0].Update(value, direction, pattern)
+ else:
+ context.append(Context(value, direction, pattern, QSqlQuery(self.glb.db), None, None))
+ # Use a thread so the UI is not blocked during the SELECT
+ thread = Thread(self.FindThread, context[0])
+ thread.done.connect(lambda ids, t=thread, c=callback: self.FindDone(t, c, ids), Qt.QueuedConnection)
+ thread.start()
+
+ def FindDone(self, _thread, callback, ids):
+ callback(ids)
+
+# Context-sensitive call graph data model
+
+class CallGraphModel(CallGraphModelBase):
+
+ def GetRoot(self):
+ return CallGraphRootItem(self.glb, self.params)
+
+ def columnCount(self, _parent=None):
+ if self.params.have_ipc:
+ return 12
+ else:
+ return 7
+
+ def columnHeader(self, column):
+ if self.params.have_ipc:
+ headers = ["Call Path", "Object", "Count ", "Time (ns) ", "Time (%) ", "Insn Cnt", "Insn Cnt (%)", "Cyc Cnt", "Cyc Cnt (%)", "IPC", "Branch Count ", "Branch Count (%) "]
+ else:
+ headers = ["Call Path", "Object", "Count ", "Time (ns) ", "Time (%) ", "Branch Count ", "Branch Count (%) "]
+ return headers[column]
+
+ def columnAlignment(self, column):
+ if self.params.have_ipc:
+ alignment = [ Qt.AlignLeft, Qt.AlignLeft, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight ]
+ else:
+ alignment = [ Qt.AlignLeft, Qt.AlignLeft, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight ]
+ return alignment[column]
+
+ def DoFindSelect(self, query, match):
+ QueryExec(query, "SELECT call_path_id, comm_id, thread_id"
+ " FROM calls"
+ " INNER JOIN call_paths ON calls.call_path_id = call_paths.id"
+ " INNER JOIN symbols ON call_paths.symbol_id = symbols.id"
+ " WHERE calls.id <> 0"
+ " AND symbols.name" + match +
+ " GROUP BY comm_id, thread_id, call_path_id"
+ " ORDER BY comm_id, thread_id, call_path_id")
+
+ def FindPath(self, query):
+ # Turn the query result into a list of ids that the tree view can walk
+ # to open the tree at the right place.
+ ids = []
+ parent_id = query.value(0)
+ while parent_id:
+ ids.insert(0, parent_id)
+ q2 = QSqlQuery(self.glb.db)
+ QueryExec(q2, "SELECT parent_id"
+ " FROM call_paths"
+ " WHERE id = " + str(parent_id))
+ if not q2.next():
+ break
+ parent_id = q2.value(0)
+ # The call path root is not used
+ if ids[0] == 1:
+ del ids[0]
+ ids.insert(0, query.value(2))
+ ids.insert(0, query.value(1))
+ return ids
+
+# Call tree data model level 2+ item base
+
+class CallTreeLevelTwoPlusItemBase(CallGraphLevelItemBase):
+
+ def __init__(self, glb, params, row, comm_id, thread_id, calls_id, call_time, time, insn_cnt, cyc_cnt, branch_count, parent_item):
+ super(CallTreeLevelTwoPlusItemBase, self).__init__(glb, params, row, parent_item)
+ self.comm_id = comm_id
+ self.thread_id = thread_id
+ self.calls_id = calls_id
+ self.call_time = call_time
+ self.time = time
+ self.insn_cnt = insn_cnt
+ self.cyc_cnt = cyc_cnt
+ self.branch_count = branch_count
+
+ def Select(self):
+ self.query_done = True
+ if self.calls_id == 0:
+ comm_thread = " AND comm_id = " + str(self.comm_id) + " AND thread_id = " + str(self.thread_id)
+ else:
+ comm_thread = ""
+ if self.params.have_ipc:
+ ipc_str = ", insn_count, cyc_count"
+ else:
+ ipc_str = ""
+ query = QSqlQuery(self.glb.db)
+ QueryExec(query, "SELECT calls.id, name, short_name, call_time, return_time - call_time" + ipc_str + ", branch_count"
+ " FROM calls"
+ " INNER JOIN call_paths ON calls.call_path_id = call_paths.id"
+ " INNER JOIN symbols ON call_paths.symbol_id = symbols.id"
+ " INNER JOIN dsos ON symbols.dso_id = dsos.id"
+ " WHERE calls.parent_id = " + str(self.calls_id) + comm_thread +
+ " ORDER BY call_time, calls.id")
+ while query.next():
+ if self.params.have_ipc:
+ insn_cnt = int(query.value(5))
+ cyc_cnt = int(query.value(6))
+ branch_count = int(query.value(7))
+ else:
+ insn_cnt = 0
+ cyc_cnt = 0
+ branch_count = int(query.value(5))
+ child_item = CallTreeLevelThreeItem(self.glb, self.params, self.child_count, self.comm_id, self.thread_id, query.value(0), query.value(1), query.value(2), query.value(3), int(query.value(4)), insn_cnt, cyc_cnt, branch_count, self)
+ self.child_items.append(child_item)
+ self.child_count += 1
+
+# Call tree data model level three item
+
+class CallTreeLevelThreeItem(CallTreeLevelTwoPlusItemBase):
+
+ def __init__(self, glb, params, row, comm_id, thread_id, calls_id, name, dso, call_time, time, insn_cnt, cyc_cnt, branch_count, parent_item):
+ super(CallTreeLevelThreeItem, self).__init__(glb, params, row, comm_id, thread_id, calls_id, call_time, time, insn_cnt, cyc_cnt, branch_count, parent_item)
+ dso = dsoname(dso)
+ if self.params.have_ipc:
+ insn_pcnt = PercentToOneDP(insn_cnt, parent_item.insn_cnt)
+ cyc_pcnt = PercentToOneDP(cyc_cnt, parent_item.cyc_cnt)
+ br_pcnt = PercentToOneDP(branch_count, parent_item.branch_count)
+ ipc = CalcIPC(cyc_cnt, insn_cnt)
+ self.data = [ name, dso, str(call_time), str(time), PercentToOneDP(time, parent_item.time), str(insn_cnt), insn_pcnt, str(cyc_cnt), cyc_pcnt, ipc, str(branch_count), br_pcnt ]
+ else:
+ self.data = [ name, dso, str(call_time), str(time), PercentToOneDP(time, parent_item.time), str(branch_count), PercentToOneDP(branch_count, parent_item.branch_count) ]
+ self.dbid = calls_id
+
+# Call tree data model level two item
+
+class CallTreeLevelTwoItem(CallTreeLevelTwoPlusItemBase):
+
+ def __init__(self, glb, params, row, comm_id, thread_id, pid, tid, parent_item):
+ super(CallTreeLevelTwoItem, self).__init__(glb, params, row, comm_id, thread_id, 0, 0, 0, 0, 0, 0, parent_item)
+ if self.params.have_ipc:
+ self.data = [str(pid) + ":" + str(tid), "", "", "", "", "", "", "", "", "", "", ""]
+ else:
+ self.data = [str(pid) + ":" + str(tid), "", "", "", "", "", ""]
+ self.dbid = thread_id
+
+ def Select(self):
+ super(CallTreeLevelTwoItem, self).Select()
+ for child_item in self.child_items:
+ self.time += child_item.time
+ self.insn_cnt += child_item.insn_cnt
+ self.cyc_cnt += child_item.cyc_cnt
+ self.branch_count += child_item.branch_count
+ for child_item in self.child_items:
+ child_item.data[4] = PercentToOneDP(child_item.time, self.time)
+ if self.params.have_ipc:
+ child_item.data[6] = PercentToOneDP(child_item.insn_cnt, self.insn_cnt)
+ child_item.data[8] = PercentToOneDP(child_item.cyc_cnt, self.cyc_cnt)
+ child_item.data[11] = PercentToOneDP(child_item.branch_count, self.branch_count)
+ else:
+ child_item.data[6] = PercentToOneDP(child_item.branch_count, self.branch_count)
+
+# Call tree data model level one item
+
+class CallTreeLevelOneItem(CallGraphLevelItemBase):
+
+ def __init__(self, glb, params, row, comm_id, comm, parent_item):
+ super(CallTreeLevelOneItem, self).__init__(glb, params, row, parent_item)
+ if self.params.have_ipc:
+ self.data = [comm, "", "", "", "", "", "", "", "", "", "", ""]
+ else:
+ self.data = [comm, "", "", "", "", "", ""]
+ self.dbid = comm_id
+
+ def Select(self):
+ self.query_done = True
+ query = QSqlQuery(self.glb.db)
+ QueryExec(query, "SELECT thread_id, pid, tid"
+ " FROM comm_threads"
+ " INNER JOIN threads ON thread_id = threads.id"
+ " WHERE comm_id = " + str(self.dbid))
+ while query.next():
+ child_item = CallTreeLevelTwoItem(self.glb, self.params, self.child_count, self.dbid, query.value(0), query.value(1), query.value(2), self)
+ self.child_items.append(child_item)
+ self.child_count += 1
+
+# Call tree data model root item
+
+class CallTreeRootItem(CallGraphLevelItemBase):
+
+ def __init__(self, glb, params):
+ super(CallTreeRootItem, self).__init__(glb, params, 0, None)
+ self.dbid = 0
+ self.query_done = True
+ if_has_calls = ""
+ if IsSelectable(glb.db, "comms", columns = "has_calls"):
+ if_has_calls = " WHERE has_calls = " + glb.dbref.TRUE
+ query = QSqlQuery(glb.db)
+ QueryExec(query, "SELECT id, comm FROM comms" + if_has_calls)
+ while query.next():
+ if not query.value(0):
+ continue
+ child_item = CallTreeLevelOneItem(glb, params, self.child_count, query.value(0), query.value(1), self)
+ self.child_items.append(child_item)
+ self.child_count += 1
+
+# Call Tree data model
+
+class CallTreeModel(CallGraphModelBase):
+
+ def GetRoot(self):
+ return CallTreeRootItem(self.glb, self.params)
+
+ def columnCount(self, _parent=None):
+ if self.params.have_ipc:
+ return 12
+ else:
+ return 7
+
+ def columnHeader(self, column):
+ if self.params.have_ipc:
+ headers = ["Call Path", "Object", "Call Time", "Time (ns) ", "Time (%) ", "Insn Cnt", "Insn Cnt (%)", "Cyc Cnt", "Cyc Cnt (%)", "IPC", "Branch Count ", "Branch Count (%) "]
+ else:
+ headers = ["Call Path", "Object", "Call Time", "Time (ns) ", "Time (%) ", "Branch Count ", "Branch Count (%) "]
+ return headers[column]
+
+ def columnAlignment(self, column):
+ if self.params.have_ipc:
+ alignment = [ Qt.AlignLeft, Qt.AlignLeft, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight ]
+ else:
+ alignment = [ Qt.AlignLeft, Qt.AlignLeft, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight ]
+ return alignment[column]
+
+ def DoFindSelect(self, query, match):
+ QueryExec(query, "SELECT calls.id, comm_id, thread_id"
+ " FROM calls"
+ " INNER JOIN call_paths ON calls.call_path_id = call_paths.id"
+ " INNER JOIN symbols ON call_paths.symbol_id = symbols.id"
+ " WHERE calls.id <> 0"
+ " AND symbols.name" + match +
+ " ORDER BY comm_id, thread_id, call_time, calls.id")
+
+ def FindPath(self, query):
+ # Turn the query result into a list of ids that the tree view can walk
+ # to open the tree at the right place.
+ ids = []
+ parent_id = query.value(0)
+ while parent_id:
+ ids.insert(0, parent_id)
+ q2 = QSqlQuery(self.glb.db)
+ QueryExec(q2, "SELECT parent_id"
+ " FROM calls"
+ " WHERE id = " + str(parent_id))
+ if not q2.next():
+ break
+ parent_id = q2.value(0)
+ ids.insert(0, query.value(2))
+ ids.insert(0, query.value(1))
+ return ids
+
+# Vertical layout
+
+class HBoxLayout(QHBoxLayout):
+
+ def __init__(self, *children):
+ super(HBoxLayout, self).__init__()
+
+ self.layout().setContentsMargins(0, 0, 0, 0)
+ for child in children:
+ if child.isWidgetType():
+ self.layout().addWidget(child)
+ else:
+ self.layout().addLayout(child)
+
+# Horizontal layout
+
+class VBoxLayout(QVBoxLayout):
+
+ def __init__(self, *children):
+ super(VBoxLayout, self).__init__()
+
+ self.layout().setContentsMargins(0, 0, 0, 0)
+ for child in children:
+ if child.isWidgetType():
+ self.layout().addWidget(child)
+ else:
+ self.layout().addLayout(child)
+
+# Vertical layout widget
+
+class VBox():
+
+ def __init__(self, *children):
+ self.vbox = QWidget()
+ self.vbox.setLayout(VBoxLayout(*children))
+
+ def Widget(self):
+ return self.vbox
+
+# Tree window base
+
+class TreeWindowBase(QMdiSubWindow):
+
+ def __init__(self, parent=None):
+ super(TreeWindowBase, self).__init__(parent)
+
+ self.model = None
+ self.find_bar = None
+
+ self.view = QTreeView()
+ self.view.setSelectionMode(QAbstractItemView.ContiguousSelection)
+ self.view.CopyCellsToClipboard = CopyTreeCellsToClipboard
+
+ self.context_menu = TreeContextMenu(self.view)
+
+ def DisplayFound(self, ids):
+ if not len(ids):
+ return False
+ parent = QModelIndex()
+ for dbid in ids:
+ found = False
+ n = self.model.rowCount(parent)
+ for row in range(n):
+ child = self.model.index(row, 0, parent)
+ if child.internalPointer().dbid == dbid:
+ found = True
+ self.view.setExpanded(parent, True)
+ self.view.setCurrentIndex(child)
+ parent = child
+ break
+ if not found:
+ break
+ return found
+
+ def Find(self, value, direction, pattern, context):
+ self.view.setFocus()
+ self.find_bar.Busy()
+ self.model.Find(value, direction, pattern, context, self.FindDone)
+
+ def FindDone(self, ids):
+ found = True
+ if not self.DisplayFound(ids):
+ found = False
+ self.find_bar.Idle()
+ if not found:
+ self.find_bar.NotFound()
+
+
+# Context-sensitive call graph window
+
+class CallGraphWindow(TreeWindowBase):
+
+ def __init__(self, glb, parent=None):
+ super(CallGraphWindow, self).__init__(parent)
+
+ self.model = LookupCreateModel("Context-Sensitive Call Graph", lambda x=glb: CallGraphModel(x))
+
+ self.view.setModel(self.model)
+
+ for c, w in ((0, 250), (1, 100), (2, 60), (3, 70), (4, 70), (5, 100)):
+ self.view.setColumnWidth(c, w)
+
+ self.find_bar = FindBar(self, self)
+
+ self.vbox = VBox(self.view, self.find_bar.Widget())
+
+ self.setWidget(self.vbox.Widget())
+
+ AddSubWindow(glb.mainwindow.mdi_area, self, "Context-Sensitive Call Graph")
+
+# Call tree window
+
+class CallTreeWindow(TreeWindowBase):
+
+ def __init__(self, glb, parent=None, thread_at_time=None):
+ super(CallTreeWindow, self).__init__(parent)
+
+ self.model = LookupCreateModel("Call Tree", lambda x=glb: CallTreeModel(x))
+
+ self.view.setModel(self.model)
+
+ for c, w in ((0, 230), (1, 100), (2, 100), (3, 70), (4, 70), (5, 100)):
+ self.view.setColumnWidth(c, w)
+
+ self.find_bar = FindBar(self, self)
+
+ self.vbox = VBox(self.view, self.find_bar.Widget())
+
+ self.setWidget(self.vbox.Widget())
+
+ AddSubWindow(glb.mainwindow.mdi_area, self, "Call Tree")
+
+ if thread_at_time:
+ self.DisplayThreadAtTime(*thread_at_time)
+
+ def DisplayThreadAtTime(self, comm_id, thread_id, time):
+ parent = QModelIndex()
+ for dbid in (comm_id, thread_id):
+ found = False
+ n = self.model.rowCount(parent)
+ for row in range(n):
+ child = self.model.index(row, 0, parent)
+ if child.internalPointer().dbid == dbid:
+ found = True
+ self.view.setExpanded(parent, True)
+ self.view.setCurrentIndex(child)
+ parent = child
+ break
+ if not found:
+ return
+ found = False
+ while True:
+ n = self.model.rowCount(parent)
+ if not n:
+ return
+ last_child = None
+ for row in range(n):
+ self.view.setExpanded(parent, True)
+ child = self.model.index(row, 0, parent)
+ child_call_time = child.internalPointer().call_time
+ if child_call_time < time:
+ last_child = child
+ elif child_call_time == time:
+ self.view.setCurrentIndex(child)
+ return
+ elif child_call_time > time:
+ break
+ if not last_child:
+ if not found:
+ child = self.model.index(0, 0, parent)
+ self.view.setExpanded(parent, True)
+ self.view.setCurrentIndex(child)
+ return
+ found = True
+ self.view.setExpanded(parent, True)
+ self.view.setCurrentIndex(last_child)
+ parent = last_child
+
+# ExecComm() gets the comm_id of the command string that was set when the process exec'd i.e. the program name
+
+def ExecComm(db, thread_id, time):
+ query = QSqlQuery(db)
+ QueryExec(query, "SELECT comm_threads.comm_id, comms.c_time, comms.exec_flag"
+ " FROM comm_threads"
+ " INNER JOIN comms ON comms.id = comm_threads.comm_id"
+ " WHERE comm_threads.thread_id = " + str(thread_id) +
+ " ORDER BY comms.c_time, comms.id")
+ first = None
+ last = None
+ while query.next():
+ if first is None:
+ first = query.value(0)
+ if query.value(2) and Decimal(query.value(1)) <= Decimal(time):
+ last = query.value(0)
+ if not(last is None):
+ return last
+ return first
+
+# Container for (x, y) data
+
+class XY():
+ def __init__(self, x=0, y=0):
+ self.x = x
+ self.y = y
+
+ def __str__(self):
+ return "XY({}, {})".format(str(self.x), str(self.y))
+
+# Container for sub-range data
+
+class Subrange():
+ def __init__(self, lo=0, hi=0):
+ self.lo = lo
+ self.hi = hi
+
+ def __str__(self):
+ return "Subrange({}, {})".format(str(self.lo), str(self.hi))
+
+# Graph data region base class
+
+class GraphDataRegion(object):
+
+ def __init__(self, key, title = "", ordinal = ""):
+ self.key = key
+ self.title = title
+ self.ordinal = ordinal
+
+# Function to sort GraphDataRegion
+
+def GraphDataRegionOrdinal(data_region):
+ return data_region.ordinal
+
+# Attributes for a graph region
+
+class GraphRegionAttribute():
+
+ def __init__(self, colour):
+ self.colour = colour
+
+# Switch graph data region represents a task
+
+class SwitchGraphDataRegion(GraphDataRegion):
+
+ def __init__(self, key, exec_comm_id, pid, tid, comm, thread_id, comm_id):
+ super(SwitchGraphDataRegion, self).__init__(key)
+
+ self.title = str(pid) + " / " + str(tid) + " " + comm
+ # Order graph legend within exec comm by pid / tid / time
+ self.ordinal = str(pid).rjust(16) + str(exec_comm_id).rjust(8) + str(tid).rjust(16)
+ self.exec_comm_id = exec_comm_id
+ self.pid = pid
+ self.tid = tid
+ self.comm = comm
+ self.thread_id = thread_id
+ self.comm_id = comm_id
+
+# Graph data point
+
+class GraphDataPoint():
+
+ def __init__(self, data, index, x, y, altx=None, alty=None, hregion=None, vregion=None):
+ self.data = data
+ self.index = index
+ self.x = x
+ self.y = y
+ self.altx = altx
+ self.alty = alty
+ self.hregion = hregion
+ self.vregion = vregion
+
+# Graph data (single graph) base class
+
+class GraphData(object):
+
+ def __init__(self, collection, xbase=Decimal(0), ybase=Decimal(0)):
+ self.collection = collection
+ self.points = []
+ self.xbase = xbase
+ self.ybase = ybase
+ self.title = ""
+
+ def AddPoint(self, x, y, altx=None, alty=None, hregion=None, vregion=None):
+ index = len(self.points)
+
+ x = float(Decimal(x) - self.xbase)
+ y = float(Decimal(y) - self.ybase)
+
+ self.points.append(GraphDataPoint(self, index, x, y, altx, alty, hregion, vregion))
+
+ def XToData(self, x):
+ return Decimal(x) + self.xbase
+
+ def YToData(self, y):
+ return Decimal(y) + self.ybase
+
+# Switch graph data (for one CPU)
+
+class SwitchGraphData(GraphData):
+
+ def __init__(self, db, collection, cpu, xbase):
+ super(SwitchGraphData, self).__init__(collection, xbase)
+
+ self.cpu = cpu
+ self.title = "CPU " + str(cpu)
+ self.SelectSwitches(db)
+
+ def SelectComms(self, db, thread_id, last_comm_id, start_time, end_time):
+ query = QSqlQuery(db)
+ QueryExec(query, "SELECT id, c_time"
+ " FROM comms"
+ " WHERE c_thread_id = " + str(thread_id) +
+ " AND exec_flag = " + self.collection.glb.dbref.TRUE +
+ " AND c_time >= " + str(start_time) +
+ " AND c_time <= " + str(end_time) +
+ " ORDER BY c_time, id")
+ while query.next():
+ comm_id = query.value(0)
+ if comm_id == last_comm_id:
+ continue
+ time = query.value(1)
+ hregion = self.HRegion(db, thread_id, comm_id, time)
+ self.AddPoint(time, 1000, None, None, hregion)
+
+ def SelectSwitches(self, db):
+ last_time = None
+ last_comm_id = None
+ last_thread_id = None
+ query = QSqlQuery(db)
+ QueryExec(query, "SELECT time, thread_out_id, thread_in_id, comm_out_id, comm_in_id, flags"
+ " FROM context_switches"
+ " WHERE machine_id = " + str(self.collection.machine_id) +
+ " AND cpu = " + str(self.cpu) +
+ " ORDER BY time, id")
+ while query.next():
+ flags = int(query.value(5))
+ if flags & 1:
+ # Schedule-out: detect and add exec's
+ if last_thread_id == query.value(1) and last_comm_id is not None and last_comm_id != query.value(3):
+ self.SelectComms(db, last_thread_id, last_comm_id, last_time, query.value(0))
+ continue
+ # Schedule-in: add data point
+ if len(self.points) == 0:
+ start_time = self.collection.glb.StartTime(self.collection.machine_id)
+ hregion = self.HRegion(db, query.value(1), query.value(3), start_time)
+ self.AddPoint(start_time, 1000, None, None, hregion)
+ time = query.value(0)
+ comm_id = query.value(4)
+ thread_id = query.value(2)
+ hregion = self.HRegion(db, thread_id, comm_id, time)
+ self.AddPoint(time, 1000, None, None, hregion)
+ last_time = time
+ last_comm_id = comm_id
+ last_thread_id = thread_id
+
+ def NewHRegion(self, db, key, thread_id, comm_id, time):
+ exec_comm_id = ExecComm(db, thread_id, time)
+ query = QSqlQuery(db)
+ QueryExec(query, "SELECT pid, tid FROM threads WHERE id = " + str(thread_id))
+ if query.next():
+ pid = query.value(0)
+ tid = query.value(1)
+ else:
+ pid = -1
+ tid = -1
+ query = QSqlQuery(db)
+ QueryExec(query, "SELECT comm FROM comms WHERE id = " + str(comm_id))
+ if query.next():
+ comm = query.value(0)
+ else:
+ comm = ""
+ return SwitchGraphDataRegion(key, exec_comm_id, pid, tid, comm, thread_id, comm_id)
+
+ def HRegion(self, db, thread_id, comm_id, time):
+ key = str(thread_id) + ":" + str(comm_id)
+ hregion = self.collection.LookupHRegion(key)
+ if hregion is None:
+ hregion = self.NewHRegion(db, key, thread_id, comm_id, time)
+ self.collection.AddHRegion(key, hregion)
+ return hregion
+
+# Graph data collection (multiple related graphs) base class
+
+class GraphDataCollection(object):
+
+ def __init__(self, glb):
+ self.glb = glb
+ self.data = []
+ self.hregions = {}
+ self.xrangelo = None
+ self.xrangehi = None
+ self.yrangelo = None
+ self.yrangehi = None
+ self.dp = XY(0, 0)
+
+ def AddGraphData(self, data):
+ self.data.append(data)
+
+ def LookupHRegion(self, key):
+ if key in self.hregions:
+ return self.hregions[key]
+ return None
+
+ def AddHRegion(self, key, hregion):
+ self.hregions[key] = hregion
+
+# Switch graph data collection (SwitchGraphData for each CPU)
+
+class SwitchGraphDataCollection(GraphDataCollection):
+
+ def __init__(self, glb, db, machine_id):
+ super(SwitchGraphDataCollection, self).__init__(glb)
+
+ self.machine_id = machine_id
+ self.cpus = self.SelectCPUs(db)
+
+ self.xrangelo = glb.StartTime(machine_id)
+ self.xrangehi = glb.FinishTime(machine_id)
+
+ self.yrangelo = Decimal(0)
+ self.yrangehi = Decimal(1000)
+
+ for cpu in self.cpus:
+ self.AddGraphData(SwitchGraphData(db, self, cpu, self.xrangelo))
+
+ def SelectCPUs(self, db):
+ cpus = []
+ query = QSqlQuery(db)
+ QueryExec(query, "SELECT DISTINCT cpu"
+ " FROM context_switches"
+ " WHERE machine_id = " + str(self.machine_id))
+ while query.next():
+ cpus.append(int(query.value(0)))
+ return sorted(cpus)
+
+# Switch graph data graphics item displays the graphed data
+
+class SwitchGraphDataGraphicsItem(QGraphicsItem):
+
+ def __init__(self, data, graph_width, graph_height, attrs, event_handler, parent=None):
+ super(SwitchGraphDataGraphicsItem, self).__init__(parent)
+
+ self.data = data
+ self.graph_width = graph_width
+ self.graph_height = graph_height
+ self.attrs = attrs
+ self.event_handler = event_handler
+ self.setAcceptHoverEvents(True)
+
+ def boundingRect(self):
+ return QRectF(0, 0, self.graph_width, self.graph_height)
+
+ def PaintPoint(self, painter, last, x):
+ if not(last is None or last.hregion.pid == 0 or x < self.attrs.subrange.x.lo):
+ if last.x < self.attrs.subrange.x.lo:
+ x0 = self.attrs.subrange.x.lo
+ else:
+ x0 = last.x
+ if x > self.attrs.subrange.x.hi:
+ x1 = self.attrs.subrange.x.hi
+ else:
+ x1 = x - 1
+ x0 = self.attrs.XToPixel(x0)
+ x1 = self.attrs.XToPixel(x1)
+
+ y0 = self.attrs.YToPixel(last.y)
+
+ colour = self.attrs.region_attributes[last.hregion.key].colour
+
+ width = x1 - x0 + 1
+ if width < 2:
+ painter.setPen(colour)
+ painter.drawLine(x0, self.graph_height - y0, x0, self.graph_height)
+ else:
+ painter.fillRect(x0, self.graph_height - y0, width, self.graph_height - 1, colour)
+
+ def paint(self, painter, _option, _widget):
+ last = None
+ for point in self.data.points:
+ self.PaintPoint(painter, last, point.x)
+ if point.x > self.attrs.subrange.x.hi:
+ break
+ last = point
+ self.PaintPoint(painter, last, self.attrs.subrange.x.hi + 1)
+
+ def BinarySearchPoint(self, target):
+ lower_pos = 0
+ higher_pos = len(self.data.points)
+ while True:
+ pos = int((lower_pos + higher_pos) / 2)
+ val = self.data.points[pos].x
+ if target >= val:
+ lower_pos = pos
+ else:
+ higher_pos = pos
+ if higher_pos <= lower_pos + 1:
+ return lower_pos
+
+ def XPixelToData(self, x):
+ x = self.attrs.PixelToX(x)
+ if x < self.data.points[0].x:
+ x = 0
+ pos = 0
+ low = True
+ else:
+ pos = self.BinarySearchPoint(x)
+ low = False
+ return (low, pos, self.data.XToData(x))
+
+ def EventToData(self, event):
+ no_data = (None,) * 4
+ if len(self.data.points) < 1:
+ return no_data
+ x = event.pos().x()
+ if x < 0:
+ return no_data
+ _low0, pos0, time_from = self.XPixelToData(x)
+ low1, pos1, time_to = self.XPixelToData(x + 1)
+ hregions = set()
+ hregion_times = []
+ if not low1:
+ for i in range(pos0, pos1 + 1):
+ hregion = self.data.points[i].hregion
+ hregions.add(hregion)
+ if i == pos0:
+ time = time_from
+ else:
+ time = self.data.XToData(self.data.points[i].x)
+ hregion_times.append((hregion, time))
+ return (time_from, time_to, hregions, hregion_times)
+
+ def hoverMoveEvent(self, event):
+ time_from, time_to, hregions, _hregion_times = self.EventToData(event)
+ if time_from is not None:
+ self.event_handler.PointEvent(self.data.cpu, time_from, time_to, hregions)
+
+ def hoverLeaveEvent(self, _event):
+ self.event_handler.NoPointEvent()
+
+ def mousePressEvent(self, event):
+ if event.button() != Qt.RightButton:
+ super(SwitchGraphDataGraphicsItem, self).mousePressEvent(event)
+ return
+ _time_from, _time_to, _hregions, hregion_times = self.EventToData(event)
+ if hregion_times:
+ self.event_handler.RightClickEvent(self.data.cpu, hregion_times, event.screenPos())
+
+# X-axis graphics item
+
+class XAxisGraphicsItem(QGraphicsItem):
+
+ def __init__(self, width, parent=None):
+ super(XAxisGraphicsItem, self).__init__(parent)
+
+ self.width = width
+ self.max_mark_sz = 4
+ self.height = self.max_mark_sz + 1
+
+ def boundingRect(self):
+ return QRectF(0, 0, self.width, self.height)
+
+ def Step(self):
+ attrs = self.parentItem().attrs
+ subrange = attrs.subrange.x
+ t = subrange.hi - subrange.lo
+ s = (3.0 * t) / self.width
+ n = 1.0
+ while s > n:
+ n = n * 10.0
+ return n
+
+ def PaintMarks(self, painter, at_y, lo, hi, step, i):
+ attrs = self.parentItem().attrs
+ x = lo
+ while x <= hi:
+ xp = attrs.XToPixel(x)
+ if i % 10:
+ if i % 5:
+ sz = 1
+ else:
+ sz = 2
+ else:
+ sz = self.max_mark_sz
+ i = 0
+ painter.drawLine(xp, at_y, xp, at_y + sz)
+ x += step
+ i += 1
+
+ def paint(self, painter, _option, _widget):
+ # Using QPainter::drawLine(int x1, int y1, int x2, int y2) so x2 = width -1
+ painter.drawLine(0, 0, self.width - 1, 0)
+ n = self.Step()
+ attrs = self.parentItem().attrs
+ subrange = attrs.subrange.x
+ if subrange.lo:
+ x_offset = n - (subrange.lo % n)
+ else:
+ x_offset = 0.0
+ x = subrange.lo + x_offset
+ i = (x / n) % 10
+ self.PaintMarks(painter, 0, x, subrange.hi, n, i)
+
+ def ScaleDimensions(self):
+ n = self.Step()
+ attrs = self.parentItem().attrs
+ lo = attrs.subrange.x.lo
+ hi = (n * 10.0) + lo
+ width = attrs.XToPixel(hi)
+ if width > 500:
+ width = 0
+ return (n, lo, hi, width)
+
+ def PaintScale(self, painter, at_x, at_y):
+ n, lo, hi, width = self.ScaleDimensions()
+ if not width:
+ return
+ painter.drawLine(at_x, at_y, at_x + width, at_y)
+ self.PaintMarks(painter, at_y, lo, hi, n, 0)
+
+ def ScaleWidth(self):
+ _n, _lo, _hi, width = self.ScaleDimensions()
+ return width
+
+ def ScaleHeight(self):
+ return self.height
+
+ def ScaleUnit(self):
+ return self.Step() * 10
+
+# Scale graphics item base class
+
+class ScaleGraphicsItem(QGraphicsItem):
+
+ def __init__(self, axis, parent=None):
+ super(ScaleGraphicsItem, self).__init__(parent)
+ self.axis = axis
+
+ def boundingRect(self):
+ scale_width = self.axis.ScaleWidth()
+ if not scale_width:
+ return QRectF()
+ return QRectF(0, 0, self.axis.ScaleWidth() + 100, self.axis.ScaleHeight())
+
+ def paint(self, painter, _option, _widget):
+ scale_width = self.axis.ScaleWidth()
+ if not scale_width:
+ return
+ self.axis.PaintScale(painter, 0, 5)
+ x = scale_width + 4
+ painter.drawText(QPointF(x, 10), self.Text())
+
+ def Unit(self):
+ return self.axis.ScaleUnit()
+
+ def Text(self):
+ return ""
+
+# Switch graph scale graphics item
+
+class SwitchScaleGraphicsItem(ScaleGraphicsItem):
+
+ def Text(self):
+ unit = self.Unit()
+ if unit >= 1000000000:
+ unit = int(unit / 1000000000)
+ us = "s"
+ elif unit >= 1000000:
+ unit = int(unit / 1000000)
+ us = "ms"
+ elif unit >= 1000:
+ unit = int(unit / 1000)
+ us = "us"
+ else:
+ unit = int(unit)
+ us = "ns"
+ return " = " + str(unit) + " " + us
+
+# Switch graph graphics item contains graph title, scale, x/y-axis, and the graphed data
+
+class SwitchGraphGraphicsItem(QGraphicsItem):
+
+ def __init__(self, collection, data, attrs, event_handler, first, parent=None):
+ super(SwitchGraphGraphicsItem, self).__init__(parent)
+ self.collection = collection
+ self.data = data
+ self.attrs = attrs
+ self.event_handler = event_handler
+
+ margin = 20
+ title_width = 50
+
+ self.title_graphics = QGraphicsSimpleTextItem(data.title, self)
+
+ self.title_graphics.setPos(margin, margin)
+ graph_width = attrs.XToPixel(attrs.subrange.x.hi) + 1
+ graph_height = attrs.YToPixel(attrs.subrange.y.hi) + 1
+
+ self.graph_origin_x = margin + title_width + margin
+ self.graph_origin_y = graph_height + margin
+
+ _x_axis_size = 1
+ y_axis_size = 1
+ self.yline = QGraphicsLineItem(0, 0, 0, graph_height, self)
+
+ self.x_axis = XAxisGraphicsItem(graph_width, self)
+ self.x_axis.setPos(self.graph_origin_x, self.graph_origin_y + 1)
+
+ if first:
+ self.scale_item = SwitchScaleGraphicsItem(self.x_axis, self)
+ self.scale_item.setPos(self.graph_origin_x, self.graph_origin_y + 10)
+
+ self.yline.setPos(self.graph_origin_x - y_axis_size, self.graph_origin_y - graph_height)
+
+ self.axis_point = QGraphicsLineItem(0, 0, 0, 0, self)
+ self.axis_point.setPos(self.graph_origin_x - 1, self.graph_origin_y +1)
+
+ self.width = self.graph_origin_x + graph_width + margin
+ self.height = self.graph_origin_y + margin
+
+ self.graph = SwitchGraphDataGraphicsItem(data, graph_width, graph_height, attrs, event_handler, self)
+ self.graph.setPos(self.graph_origin_x, self.graph_origin_y - graph_height)
+
+ if parent and 'EnableRubberBand' in dir(parent):
+ parent.EnableRubberBand(self.graph_origin_x, self.graph_origin_x + graph_width - 1, self)
+
+ def boundingRect(self):
+ return QRectF(0, 0, self.width, self.height)
+
+ def paint(self, painter, option, widget):
+ pass
+
+ def RBXToPixel(self, x):
+ return self.attrs.PixelToX(x - self.graph_origin_x)
+
+ def RBXRangeToPixel(self, x0, x1):
+ return (self.RBXToPixel(x0), self.RBXToPixel(x1 + 1))
+
+ def RBPixelToTime(self, x):
+ if x < self.data.points[0].x:
+ return self.data.XToData(0)
+ return self.data.XToData(x)
+
+ def RBEventTimes(self, x0, x1):
+ x0, x1 = self.RBXRangeToPixel(x0, x1)
+ time_from = self.RBPixelToTime(x0)
+ time_to = self.RBPixelToTime(x1)
+ return (time_from, time_to)
+
+ def RBEvent(self, x0, x1):
+ time_from, time_to = self.RBEventTimes(x0, x1)
+ self.event_handler.RangeEvent(time_from, time_to)
+
+ def RBMoveEvent(self, x0, x1):
+ if x1 < x0:
+ x0, x1 = x1, x0
+ self.RBEvent(x0, x1)
+
+ def RBReleaseEvent(self, x0, x1, selection_state):
+ if x1 < x0:
+ x0, x1 = x1, x0
+ x0, x1 = self.RBXRangeToPixel(x0, x1)
+ self.event_handler.SelectEvent(x0, x1, selection_state)
+
+# Graphics item to draw a vertical bracket (used to highlight "forward" sub-range)
+
+class VerticalBracketGraphicsItem(QGraphicsItem):
+
+ def __init__(self, parent=None):
+ super(VerticalBracketGraphicsItem, self).__init__(parent)
+
+ self.width = 0
+ self.height = 0
+ self.hide()
+
+ def SetSize(self, width, height):
+ self.width = width + 1
+ self.height = height + 1
+
+ def boundingRect(self):
+ return QRectF(0, 0, self.width, self.height)
+
+ def paint(self, painter, _option, _widget):
+ colour = QColor(255, 255, 0, 32)
+ painter.fillRect(0, 0, self.width, self.height, colour)
+ x1 = self.width - 1
+ y1 = self.height - 1
+ painter.drawLine(0, 0, x1, 0)
+ painter.drawLine(0, 0, 0, 3)
+ painter.drawLine(x1, 0, x1, 3)
+ painter.drawLine(0, y1, x1, y1)
+ painter.drawLine(0, y1, 0, y1 - 3)
+ painter.drawLine(x1, y1, x1, y1 - 3)
+
+# Graphics item to contain graphs arranged vertically
+
+class VertcalGraphSetGraphicsItem(QGraphicsItem):
+
+ def __init__(self, collection, attrs, event_handler, child_class, parent=None):
+ super(VertcalGraphSetGraphicsItem, self).__init__(parent)
+
+ self.collection = collection
+
+ self.top = 10
+
+ self.width = 0
+ self.height = self.top
+
+ self.rubber_band = None
+ self.rb_enabled = False
+ self.rb_in_view = False
+ self.rb_xlo = 0
+ self.rb_xhi = 0
+ self.rb_event_handler = None
+ self.rb_origin = None
+
+ first = True
+ for data in collection.data:
+ child = child_class(collection, data, attrs, event_handler, first, self)
+ child.setPos(0, self.height + 1)
+ rect = child.boundingRect()
+ if rect.right() > self.width:
+ self.width = rect.right()
+ self.height = self.height + rect.bottom() + 1
+ first = False
+
+ self.bracket = VerticalBracketGraphicsItem(self)
+
+ def EnableRubberBand(self, xlo, xhi, rb_event_handler):
+ if self.rb_enabled:
+ return
+ self.rb_enabled = True
+ self.rb_in_view = False
+ self.setAcceptedMouseButtons(Qt.LeftButton)
+ self.rb_xlo = xlo
+ self.rb_xhi = xhi
+ self.rb_event_handler = rb_event_handler
+
+ def mousePressEvent(self, event):
+ if self.rb_enabled:
+ self.MousePressEvent(event)
+
+ def mouseMoveEvent(self, event):
+ if self.rb_enabled:
+ self.MouseMoveEvent(event)
+
+ def mouseReleaseEvent(self, event):
+ if self.rb_enabled:
+ self.MouseReleaseEvent(event)
+
+ def boundingRect(self):
+ return QRectF(0, 0, self.width, self.height)
+
+ def paint(self, painter, option, widget):
+ pass
+
+ def RubberBandParent(self):
+ scene = self.scene()
+ view = scene.views()[0]
+ viewport = view.viewport()
+ return viewport
+
+ def RubberBandSetGeometry(self, rect):
+ scene_rectf = self.mapRectToScene(QRectF(rect))
+ scene = self.scene()
+ view = scene.views()[0]
+ poly = view.mapFromScene(scene_rectf)
+ self.rubber_band.setGeometry(poly.boundingRect())
+
+ def SetSelection(self, selection_state):
+ if self.rubber_band:
+ if selection_state:
+ self.RubberBandSetGeometry(selection_state)
+ self.rubber_band.show()
+ else:
+ self.rubber_band.hide()
+
+ def SetBracket(self, rect):
+ if rect:
+ x, y, width, height = rect.x(), rect.y(), rect.width(), rect.height()
+ self.bracket.setPos(x, y)
+ self.bracket.SetSize(width, height)
+ self.bracket.show()
+ else:
+ self.bracket.hide()
+
+ def RubberBandX(self, event):
+ x = event.pos().toPoint().x()
+ if x < self.rb_xlo:
+ x = self.rb_xlo
+ elif x > self.rb_xhi:
+ x = self.rb_xhi
+ else:
+ self.rb_in_view = True
+ return x
+
+ def RubberBandRect(self, x):
+ if self.rb_origin.x() <= x:
+ width = x - self.rb_origin.x()
+ rect = QRect(self.rb_origin, QSize(width, self.height))
+ else:
+ width = self.rb_origin.x() - x
+ top_left = QPoint(self.rb_origin.x() - width, self.rb_origin.y())
+ rect = QRect(top_left, QSize(width, self.height))
+ return rect
+
+ def MousePressEvent(self, event):
+ self.rb_in_view = False
+ x = self.RubberBandX(event)
+ self.rb_origin = QPoint(x, self.top)
+ if self.rubber_band is None:
+ self.rubber_band = QRubberBand(QRubberBand.Rectangle, self.RubberBandParent())
+ self.RubberBandSetGeometry(QRect(self.rb_origin, QSize(0, self.height)))
+ if self.rb_in_view:
+ self.rubber_band.show()
+ self.rb_event_handler.RBMoveEvent(x, x)
+ else:
+ self.rubber_band.hide()
+
+ def MouseMoveEvent(self, event):
+ x = self.RubberBandX(event)
+ rect = self.RubberBandRect(x)
+ self.RubberBandSetGeometry(rect)
+ if self.rb_in_view:
+ self.rubber_band.show()
+ self.rb_event_handler.RBMoveEvent(self.rb_origin.x(), x)
+
+ def MouseReleaseEvent(self, event):
+ x = self.RubberBandX(event)
+ if self.rb_in_view:
+ selection_state = self.RubberBandRect(x)
+ else:
+ selection_state = None
+ self.rb_event_handler.RBReleaseEvent(self.rb_origin.x(), x, selection_state)
+
+# Switch graph legend data model
+
+class SwitchGraphLegendModel(QAbstractTableModel):
+
+ def __init__(self, collection, region_attributes, parent=None):
+ super(SwitchGraphLegendModel, self).__init__(parent)
+
+ self.region_attributes = region_attributes
+
+ self.child_items = sorted(collection.hregions.values(), key=GraphDataRegionOrdinal)
+ self.child_count = len(self.child_items)
+
+ self.highlight_set = set()
+
+ self.column_headers = ("pid", "tid", "comm")
+
+ def rowCount(self, _parent):
+ return self.child_count
+
+ def headerData(self, section, orientation, role):
+ if role != Qt.DisplayRole:
+ return None
+ if orientation != Qt.Horizontal:
+ return None
+ return self.columnHeader(section)
+
+ def index(self, row, column, _parent):
+ return self.createIndex(row, column, self.child_items[row])
+
+ def columnCount(self, _parent=None):
+ return len(self.column_headers)
+
+ def columnHeader(self, column):
+ return self.column_headers[column]
+
+ def data(self, index, role):
+ if role == Qt.BackgroundRole:
+ child = self.child_items[index.row()]
+ if child in self.highlight_set:
+ return self.region_attributes[child.key].colour
+ return None
+ if role == Qt.ForegroundRole:
+ child = self.child_items[index.row()]
+ if child in self.highlight_set:
+ return QColor(255, 255, 255)
+ return self.region_attributes[child.key].colour
+ if role != Qt.DisplayRole:
+ return None
+ hregion = self.child_items[index.row()]
+ col = index.column()
+ if col == 0:
+ return hregion.pid
+ if col == 1:
+ return hregion.tid
+ if col == 2:
+ return hregion.comm
+ return None
+
+ def SetHighlight(self, row, _set_highlight):
+ child = self.child_items[row]
+ top_left = self.createIndex(row, 0, child)
+ bottom_right = self.createIndex(row, len(self.column_headers) - 1, child)
+ self.dataChanged.emit(top_left, bottom_right)
+
+ def Highlight(self, highlight_set):
+ for row in range(self.child_count):
+ child = self.child_items[row]
+ if child in self.highlight_set:
+ if child not in highlight_set:
+ self.SetHighlight(row, False)
+ elif child in highlight_set:
+ self.SetHighlight(row, True)
+ self.highlight_set = highlight_set
+
+# Switch graph legend is a table
+
+class SwitchGraphLegend(QWidget):
+
+ def __init__(self, collection, region_attributes, parent=None):
+ super(SwitchGraphLegend, self).__init__(parent)
+
+ self.data_model = SwitchGraphLegendModel(collection, region_attributes)
+
+ self.model = QSortFilterProxyModel()
+ self.model.setSourceModel(self.data_model)
+
+ self.view = QTableView()
+ self.view.setModel(self.model)
+ self.view.setEditTriggers(QAbstractItemView.NoEditTriggers)
+ self.view.verticalHeader().setVisible(False)
+ self.view.sortByColumn(-1, Qt.AscendingOrder)
+ self.view.setSortingEnabled(True)
+ self.view.resizeColumnsToContents()
+ self.view.resizeRowsToContents()
+
+ self.vbox = VBoxLayout(self.view)
+ self.setLayout(self.vbox)
+
+ sz1 = self.view.columnWidth(0) + self.view.columnWidth(1) + self.view.columnWidth(2) + 2
+ sz1 = sz1 + self.view.verticalScrollBar().sizeHint().width()
+ self.saved_size = sz1
+
+ def resizeEvent(self, event):
+ self.saved_size = self.size().width()
+ super(SwitchGraphLegend, self).resizeEvent(event)
+
+ def Highlight(self, highlight_set):
+ self.data_model.Highlight(highlight_set)
+ self.update()
+
+ def changeEvent(self, event):
+ if event.type() == QEvent.FontChange:
+ self.view.resizeRowsToContents()
+ self.view.resizeColumnsToContents()
+ # Need to resize rows again after column resize
+ self.view.resizeRowsToContents()
+ super(SwitchGraphLegend, self).changeEvent(event)
+
+# Random colour generation
+
+def RGBColourTooLight(r, g, _b):
+ if g > 230:
+ return True
+ if g <= 160:
+ return False
+ if r <= 180 and g <= 180:
+ return False
+ if r < 60:
+ return False
+ return True
+
+def GenerateColours(x):
+ cs = [0]
+ for i in range(1, x):
+ cs.append(int((255.0 / i) + 0.5))
+ colours = []
+ for r in cs:
+ for g in cs:
+ for b in cs:
+ # Exclude black and colours that look too light against a white background
+ if (r, g, b) == (0, 0, 0) or RGBColourTooLight(r, g, b):
+ continue
+ colours.append(QColor(r, g, b))
+ return colours
+
+def GenerateNColours(n):
+ for x in range(2, n + 2):
+ colours = GenerateColours(x)
+ if len(colours) >= n:
+ return colours
+ return []
+
+def GenerateNRandomColours(n, seed):
+ colours = GenerateNColours(n)
+ random.seed(seed)
+ random.shuffle(colours)
+ return colours
+
+# Graph attributes, in particular the scale and subrange that change when zooming
+
+class GraphAttributes():
+
+ def __init__(self, scale, subrange, region_attributes, dp):
+ self.scale = scale
+ self.subrange = subrange
+ self.region_attributes = region_attributes
+ # Rounding avoids errors due to finite floating point precision
+ self.dp = dp # data decimal places
+ self.Update()
+
+ def XToPixel(self, x):
+ return int(round((x - self.subrange.x.lo) * self.scale.x, self.pdp.x))
+
+ def YToPixel(self, y):
+ return int(round((y - self.subrange.y.lo) * self.scale.y, self.pdp.y))
+
+ def PixelToXRounded(self, px):
+ return round((round(px, 0) / self.scale.x), self.dp.x) + self.subrange.x.lo
+
+ def PixelToYRounded(self, py):
+ return round((round(py, 0) / self.scale.y), self.dp.y) + self.subrange.y.lo
+
+ def PixelToX(self, px):
+ x = self.PixelToXRounded(px)
+ if self.pdp.x == 0:
+ rt = self.XToPixel(x)
+ if rt > px:
+ return x - 1
+ return x
+
+ def PixelToY(self, py):
+ y = self.PixelToYRounded(py)
+ if self.pdp.y == 0:
+ rt = self.YToPixel(y)
+ if rt > py:
+ return y - 1
+ return y
+
+ def ToPDP(self, dp, scale):
+ # Calculate pixel decimal places:
+ # (10 ** dp) is the minimum delta in the data
+ # scale it to get the minimum delta in pixels
+ # log10 gives the number of decimals places negatively
+ # subtrace 1 to divide by 10
+ # round to the lower negative number
+ # change the sign to get the number of decimals positively
+ x = math.log10((10 ** dp) * scale)
+ if x < 0:
+ x -= 1
+ x = -int(math.floor(x) - 0.1)
+ else:
+ x = 0
+ return x
+
+ def Update(self):
+ x = self.ToPDP(self.dp.x, self.scale.x)
+ y = self.ToPDP(self.dp.y, self.scale.y)
+ self.pdp = XY(x, y) # pixel decimal places
+
+# Switch graph splitter which divides the CPU graphs from the legend
+
+class SwitchGraphSplitter(QSplitter):
+
+ def __init__(self, parent=None):
+ super(SwitchGraphSplitter, self).__init__(parent)
+
+ self.first_time = False
+
+ def resizeEvent(self, event):
+ if self.first_time:
+ self.first_time = False
+ sz1 = self.widget(1).view.columnWidth(0) + self.widget(1).view.columnWidth(1) + self.widget(1).view.columnWidth(2) + 2
+ sz1 = sz1 + self.widget(1).view.verticalScrollBar().sizeHint().width()
+ sz0 = self.size().width() - self.handleWidth() - sz1
+ self.setSizes([sz0, sz1])
+ elif not(self.widget(1).saved_size is None):
+ sz1 = self.widget(1).saved_size
+ sz0 = self.size().width() - self.handleWidth() - sz1
+ self.setSizes([sz0, sz1])
+ super(SwitchGraphSplitter, self).resizeEvent(event)
+
+# Graph widget base class
+
+class GraphWidget(QWidget):
+
+ graph_title_changed = Signal(object)
+
+ def __init__(self, parent=None):
+ super(GraphWidget, self).__init__(parent)
+
+ def GraphTitleChanged(self, title):
+ self.graph_title_changed.emit(title)
+
+ def Title(self):
+ return ""
+
+# Display time in s, ms, us or ns
+
+def ToTimeStr(val):
+ val = Decimal(val)
+ if val >= 1000000000:
+ return "{} s".format((val / 1000000000).quantize(Decimal("0.000000001")))
+ if val >= 1000000:
+ return "{} ms".format((val / 1000000).quantize(Decimal("0.000001")))
+ if val >= 1000:
+ return "{} us".format((val / 1000).quantize(Decimal("0.001")))
+ return "{} ns".format(val.quantize(Decimal("1")))
+
+# Switch (i.e. context switch i.e. Time Chart by CPU) graph widget which contains the CPU graphs and the legend and control buttons
+
+class SwitchGraphWidget(GraphWidget):
+
+ def __init__(self, glb, collection, parent=None):
+ super(SwitchGraphWidget, self).__init__(parent)
+
+ self.glb = glb
+ self.collection = collection
+
+ self.back_state = []
+ self.forward_state = []
+ self.selection_state = (None, None)
+ self.fwd_rect = None
+ self.start_time = self.glb.StartTime(collection.machine_id)
+
+ i = 0
+ hregions = collection.hregions.values()
+ colours = GenerateNRandomColours(len(hregions), 1013)
+ region_attributes = {}
+ for hregion in hregions:
+ if hregion.pid == 0 and hregion.tid == 0:
+ region_attributes[hregion.key] = GraphRegionAttribute(QColor(0, 0, 0))
+ else:
+ region_attributes[hregion.key] = GraphRegionAttribute(colours[i])
+ i = i + 1
+
+ # Default to entire range
+ xsubrange = Subrange(0.0, float(collection.xrangehi - collection.xrangelo) + 1.0)
+ ysubrange = Subrange(0.0, float(collection.yrangehi - collection.yrangelo) + 1.0)
+ subrange = XY(xsubrange, ysubrange)
+
+ scale = self.GetScaleForRange(subrange)
+
+ self.attrs = GraphAttributes(scale, subrange, region_attributes, collection.dp)
+
+ self.item = VertcalGraphSetGraphicsItem(collection, self.attrs, self, SwitchGraphGraphicsItem)
+
+ self.scene = QGraphicsScene()
+ self.scene.addItem(self.item)
+
+ self.view = QGraphicsView(self.scene)
+ self.view.centerOn(0, 0)
+ self.view.setAlignment(Qt.AlignLeft | Qt.AlignTop)
+
+ self.legend = SwitchGraphLegend(collection, region_attributes)
+
+ self.splitter = SwitchGraphSplitter()
+ self.splitter.addWidget(self.view)
+ self.splitter.addWidget(self.legend)
+
+ self.point_label = QLabel("")
+ self.point_label.setSizePolicy(QSizePolicy.Preferred, QSizePolicy.Fixed)
+
+ self.back_button = QToolButton()
+ self.back_button.setIcon(self.style().standardIcon(QStyle.SP_ArrowLeft))
+ self.back_button.setDisabled(True)
+ self.back_button.released.connect(self.Back)
+
+ self.forward_button = QToolButton()
+ self.forward_button.setIcon(self.style().standardIcon(QStyle.SP_ArrowRight))
+ self.forward_button.setDisabled(True)
+ self.forward_button.released.connect(self.Forward)
+
+ self.zoom_button = QToolButton()
+ self.zoom_button.setText("Zoom")
+ self.zoom_button.setDisabled(True)
+ self.zoom_button.released.connect(self.Zoom)
+
+ self.hbox = HBoxLayout(self.back_button, self.forward_button, self.zoom_button, self.point_label)
+
+ self.vbox = VBoxLayout(self.splitter, self.hbox)
+
+ self.setLayout(self.vbox)
+
+ def GetScaleForRangeX(self, xsubrange):
+ # Default graph 1000 pixels wide
+ dflt = 1000.0
+ r = xsubrange.hi - xsubrange.lo
+ return dflt / r
+
+ def GetScaleForRangeY(self, ysubrange):
+ # Default graph 50 pixels high
+ dflt = 50.0
+ r = ysubrange.hi - ysubrange.lo
+ return dflt / r
+
+ def GetScaleForRange(self, subrange):
+ # Default graph 1000 pixels wide, 50 pixels high
+ xscale = self.GetScaleForRangeX(subrange.x)
+ yscale = self.GetScaleForRangeY(subrange.y)
+ return XY(xscale, yscale)
+
+ def PointEvent(self, cpu, time_from, _time_to, hregions):
+ text = "CPU: " + str(cpu)
+ time_from = time_from.quantize(Decimal(1))
+ rel_time_from = time_from - self.glb.StartTime(self.collection.machine_id)
+ text = text + " Time: " + str(time_from) + " (+" + ToTimeStr(rel_time_from) + ")"
+ self.point_label.setText(text)
+ self.legend.Highlight(hregions)
+
+ def RightClickEvent(self, _cpu, hregion_times, pos):
+ if not IsSelectable(self.glb.db, "calls", "WHERE parent_id >= 0"):
+ return
+ menu = QMenu(self.view)
+ for hregion, time in hregion_times:
+ thread_at_time = (hregion.exec_comm_id, hregion.thread_id, time)
+ menu_text = "Show Call Tree for {} {}:{} at {}".format(hregion.comm, hregion.pid, hregion.tid, time)
+ menu.addAction(CreateAction(menu_text, "Show Call Tree", lambda a=None, args=thread_at_time: self.RightClickSelect(args), self.view))
+ menu.exec_(pos)
+
+ def RightClickSelect(self, args):
+ CallTreeWindow(self.glb, self.glb.mainwindow, thread_at_time=args)
+
+ def NoPointEvent(self):
+ self.point_label.setText("")
+ self.legend.Highlight({})
+
+ def RangeEvent(self, time_from, time_to):
+ time_from = time_from.quantize(Decimal(1))
+ time_to = time_to.quantize(Decimal(1))
+ if time_to <= time_from:
+ self.point_label.setText("")
+ return
+ rel_time_from = time_from - self.start_time
+ rel_time_to = time_to - self.start_time
+ text = " Time: " + str(time_from) + " (+" + ToTimeStr(rel_time_from) + ") to: " + str(time_to) + " (+" + ToTimeStr(rel_time_to) + ")"
+ text = text + " duration: " + ToTimeStr(time_to - time_from)
+ self.point_label.setText(text)
+
+ def BackState(self):
+ return (self.attrs.subrange, self.attrs.scale, self.selection_state, self.fwd_rect)
+
+ def PushBackState(self):
+ state = copy.deepcopy(self.BackState())
+ self.back_state.append(state)
+ self.back_button.setEnabled(True)
+
+ def PopBackState(self):
+ self.attrs.subrange, self.attrs.scale, self.selection_state, self.fwd_rect = self.back_state.pop()
+ self.attrs.Update()
+ if not self.back_state:
+ self.back_button.setDisabled(True)
+
+ def PushForwardState(self):
+ state = copy.deepcopy(self.BackState())
+ self.forward_state.append(state)
+ self.forward_button.setEnabled(True)
+
+ def PopForwardState(self):
+ self.attrs.subrange, self.attrs.scale, self.selection_state, self.fwd_rect = self.forward_state.pop()
+ self.attrs.Update()
+ if not self.forward_state:
+ self.forward_button.setDisabled(True)
+
+ def Title(self):
+ time_from = self.collection.xrangelo + Decimal(self.attrs.subrange.x.lo)
+ time_to = self.collection.xrangelo + Decimal(self.attrs.subrange.x.hi)
+ rel_time_from = time_from - self.start_time
+ rel_time_to = time_to - self.start_time
+ title = "+" + ToTimeStr(rel_time_from) + " to +" + ToTimeStr(rel_time_to)
+ title = title + " (" + ToTimeStr(time_to - time_from) + ")"
+ return title
+
+ def Update(self):
+ selected_subrange, selection_state = self.selection_state
+ self.item.SetSelection(selection_state)
+ self.item.SetBracket(self.fwd_rect)
+ self.zoom_button.setDisabled(selected_subrange is None)
+ self.GraphTitleChanged(self.Title())
+ self.item.update(self.item.boundingRect())
+
+ def Back(self):
+ if not self.back_state:
+ return
+ self.PushForwardState()
+ self.PopBackState()
+ self.Update()
+
+ def Forward(self):
+ if not self.forward_state:
+ return
+ self.PushBackState()
+ self.PopForwardState()
+ self.Update()
+
+ def SelectEvent(self, x0, x1, selection_state):
+ if selection_state is None:
+ selected_subrange = None
+ else:
+ if x1 - x0 < 1.0:
+ x1 += 1.0
+ selected_subrange = Subrange(x0, x1)
+ self.selection_state = (selected_subrange, selection_state)
+ self.zoom_button.setDisabled(selected_subrange is None)
+
+ def Zoom(self):
+ selected_subrange, selection_state = self.selection_state
+ if selected_subrange is None:
+ return
+ self.fwd_rect = selection_state
+ self.item.SetSelection(None)
+ self.PushBackState()
+ self.attrs.subrange.x = selected_subrange
+ self.forward_state = []
+ self.forward_button.setDisabled(True)
+ self.selection_state = (None, None)
+ self.fwd_rect = None
+ self.attrs.scale.x = self.GetScaleForRangeX(self.attrs.subrange.x)
+ self.attrs.Update()
+ self.Update()
+
+# Slow initialization - perform non-GUI initialization in a separate thread and put up a modal message box while waiting
+
+class SlowInitClass():
+
+ def __init__(self, glb, title, init_fn):
+ self.init_fn = init_fn
+ self.done = False
+ self.result = None
+
+ self.msg_box = QMessageBox(glb.mainwindow)
+ self.msg_box.setText("Initializing " + title + ". Please wait.")
+ self.msg_box.setWindowTitle("Initializing " + title)
+ self.msg_box.setWindowIcon(glb.mainwindow.style().standardIcon(QStyle.SP_MessageBoxInformation))
+
+ self.init_thread = Thread(self.ThreadFn, glb)
+ self.init_thread.done.connect(self.Done, Qt.QueuedConnection)
+
+ self.init_thread.start()
+
+ def Done(self):
+ self.msg_box.done(0)
+
+ def ThreadFn(self, glb):
+ conn_name = "SlowInitClass" + str(os.getpid())
+ db, _dbname = glb.dbref.Open(conn_name)
+ self.result = self.init_fn(db)
+ self.done = True
+ return (True, 0)
+
+ def Result(self):
+ while not self.done:
+ self.msg_box.exec_()
+ self.init_thread.wait()
+ return self.result
+
+def SlowInit(glb, title, init_fn):
+ init = SlowInitClass(glb, title, init_fn)
+ return init.Result()
+
+# Time chart by CPU window
+
+class TimeChartByCPUWindow(QMdiSubWindow):
+
+ def __init__(self, glb, parent=None):
+ super(TimeChartByCPUWindow, self).__init__(parent)
+
+ self.glb = glb
+ self.machine_id = glb.HostMachineId()
+ self.collection_name = "SwitchGraphDataCollection " + str(self.machine_id)
+
+ collection = LookupModel(self.collection_name)
+ if collection is None:
+ collection = SlowInit(glb, "Time Chart", self.Init)
+
+ self.widget = SwitchGraphWidget(glb, collection, self)
+ self.view = self.widget
+
+ self.base_title = "Time Chart by CPU"
+ self.setWindowTitle(self.base_title + self.widget.Title())
+ self.widget.graph_title_changed.connect(self.GraphTitleChanged)
+
+ self.setWidget(self.widget)
+
+ AddSubWindow(glb.mainwindow.mdi_area, self, self.windowTitle())
+
+ def Init(self, db):
+ return LookupCreateModel(self.collection_name, lambda : SwitchGraphDataCollection(self.glb, db, self.machine_id))
+
+ def GraphTitleChanged(self, title):
+ self.setWindowTitle(self.base_title + " : " + title)
+
+# Child data item finder
+
+class ChildDataItemFinder():
+
+ def __init__(self, root):
+ self.root = root
+ self.value, self.direction, self.pattern, self.last_value, self.last_pattern = (None,) * 5
+ self.rows = []
+ self.pos = 0
+
+ def FindSelect(self):
+ self.rows = []
+ if self.pattern:
+ pattern = re.compile(self.value)
+ for child in self.root.child_items:
+ for column_data in child.data:
+ if re.search(pattern, str(column_data)) is not None:
+ self.rows.append(child.row)
+ break
+ else:
+ for child in self.root.child_items:
+ for column_data in child.data:
+ if self.value in str(column_data):
+ self.rows.append(child.row)
+ break
+
+ def FindValue(self):
+ self.pos = 0
+ if self.last_value != self.value or self.pattern != self.last_pattern:
+ self.FindSelect()
+ if not len(self.rows):
+ return -1
+ return self.rows[self.pos]
+
+ def FindThread(self):
+ if self.direction == 0 or self.value != self.last_value or self.pattern != self.last_pattern:
+ row = self.FindValue()
+ elif len(self.rows):
+ if self.direction > 0:
+ self.pos += 1
+ if self.pos >= len(self.rows):
+ self.pos = 0
+ else:
+ self.pos -= 1
+ if self.pos < 0:
+ self.pos = len(self.rows) - 1
+ row = self.rows[self.pos]
+ else:
+ row = -1
+ return (True, row)
+
+ def Find(self, value, direction, pattern, _context, callback):
+ self.value, self.direction, self.pattern, self.last_value, self.last_pattern = (value, direction,pattern, self.value, self.pattern)
+ # Use a thread so the UI is not blocked
+ thread = Thread(self.FindThread)
+ thread.done.connect(lambda row, t=thread, c=callback: self.FindDone(t, c, row), Qt.QueuedConnection)
+ thread.start()
+
+ def FindDone(self, _thread, callback, row):
+ callback(row)
+
+# Number of database records to fetch in one go
+
+glb_chunk_sz = 10000
+
+# Background process for SQL data fetcher
+
+class SQLFetcherProcess():
+
+ def __init__(self, dbref, sql, buffer, head, tail, fetch_count, fetching_done, process_target, wait_event, fetched_event, prep):
+ # Need a unique connection name
+ conn_name = "SQLFetcher" + str(os.getpid())
+ self.db, _dbname = dbref.Open(conn_name)
+ self.sql = sql
+ self.buffer = buffer
+ self.head = head
+ self.tail = tail
+ self.fetch_count = fetch_count
+ self.fetching_done = fetching_done
+ self.process_target = process_target
+ self.wait_event = wait_event
+ self.fetched_event = fetched_event
+ self.prep = prep
+ self.query = QSqlQuery(self.db)
+ self.query_limit = 0 if "$$last_id$$" in sql else 2
+ self.last_id = -1
+ self.fetched = 0
+ self.more = True
+ self.local_head = self.head.value
+ self.local_tail = self.tail.value
+
+ def Select(self):
+ if self.query_limit:
+ if self.query_limit == 1:
+ return
+ self.query_limit -= 1
+ stmt = self.sql.replace("$$last_id$$", str(self.last_id))
+ QueryExec(self.query, stmt)
+
+ def Next(self):
+ if not self.query.next():
+ self.Select()
+ if not self.query.next():
+ return None
+ self.last_id = self.query.value(0)
+ return self.prep(self.query)
+
+ def WaitForTarget(self):
+ while True:
+ self.wait_event.clear()
+ target = self.process_target.value
+ if target > self.fetched or target < 0:
+ break
+ self.wait_event.wait()
+ return target
+
+ def HasSpace(self, sz):
+ if self.local_tail <= self.local_head:
+ space = len(self.buffer) - self.local_head
+ if space > sz:
+ return True
+ if space >= glb_nsz:
+ # Use 0 (or space < glb_nsz) to mean there is no more at the top of the buffer
+ nd = pickle.dumps(0, pickle.HIGHEST_PROTOCOL)
+ self.buffer[self.local_head : self.local_head + len(nd)] = nd
+ self.local_head = 0
+ if self.local_tail - self.local_head > sz:
+ return True
+ return False
+
+ def WaitForSpace(self, sz):
+ if self.HasSpace(sz):
+ return
+ while True:
+ self.wait_event.clear()
+ self.local_tail = self.tail.value
+ if self.HasSpace(sz):
+ return
+ self.wait_event.wait()
+
+ def AddToBuffer(self, obj):
+ d = pickle.dumps(obj, pickle.HIGHEST_PROTOCOL)
+ n = len(d)
+ nd = pickle.dumps(n, pickle.HIGHEST_PROTOCOL)
+ sz = n + glb_nsz
+ self.WaitForSpace(sz)
+ pos = self.local_head
+ self.buffer[pos : pos + len(nd)] = nd
+ self.buffer[pos + glb_nsz : pos + sz] = d
+ self.local_head += sz
+
+ def FetchBatch(self, batch_size):
+ fetched = 0
+ while batch_size > fetched:
+ obj = self.Next()
+ if obj is None:
+ self.more = False
+ break
+ self.AddToBuffer(obj)
+ fetched += 1
+ if fetched:
+ self.fetched += fetched
+ with self.fetch_count.get_lock():
+ self.fetch_count.value += fetched
+ self.head.value = self.local_head
+ self.fetched_event.set()
+
+ def Run(self):
+ while self.more:
+ target = self.WaitForTarget()
+ if target < 0:
+ break
+ batch_size = min(glb_chunk_sz, target - self.fetched)
+ self.FetchBatch(batch_size)
+ self.fetching_done.value = True
+ self.fetched_event.set()
+
+def SQLFetcherFn(*x):
+ process = SQLFetcherProcess(*x)
+ process.Run()
+
+# SQL data fetcher
+
+class SQLFetcher(QObject):
+
+ done = Signal(object)
+
+ def __init__(self, glb, sql, prep, process_data, parent=None):
+ super(SQLFetcher, self).__init__(parent)
+ self.process_data = process_data
+ self.more = True
+ self.target = 0
+ self.last_target = 0
+ self.fetched = 0
+ self.buffer_size = 16 * 1024 * 1024
+ self.buffer = Array(c_char, self.buffer_size, lock=False)
+ self.head = Value(c_longlong)
+ self.tail = Value(c_longlong)
+ self.local_tail = 0
+ self.fetch_count = Value(c_longlong)
+ self.fetching_done = Value(c_bool)
+ self.last_count = 0
+ self.process_target = Value(c_longlong)
+ self.wait_event = Event()
+ self.fetched_event = Event()
+ glb.AddInstanceToShutdownOnExit(self)
+ self.process = Process(target=SQLFetcherFn, args=(glb.dbref, sql, self.buffer, self.head, self.tail, self.fetch_count, self.fetching_done, self.process_target, self.wait_event, self.fetched_event, prep))
+ self.process.start()
+ self.thread = Thread(self.Thread)
+ self.thread.done.connect(self.ProcessData, Qt.QueuedConnection)
+ self.thread.start()
+
+ def Shutdown(self):
+ # Tell the thread and process to exit
+ self.process_target.value = -1
+ self.wait_event.set()
+ self.more = False
+ self.fetching_done.value = True
+ self.fetched_event.set()
+
+ def Thread(self):
+ if not self.more:
+ return True, 0
+ while True:
+ self.fetched_event.clear()
+ fetch_count = self.fetch_count.value
+ if fetch_count != self.last_count:
+ break
+ if self.fetching_done.value:
+ self.more = False
+ return True, 0
+ self.fetched_event.wait()
+ count = fetch_count - self.last_count
+ self.last_count = fetch_count
+ self.fetched += count
+ return False, count
+
+ def Fetch(self, nr):
+ if not self.more:
+ # -1 inidcates there are no more
+ return -1
+ result = self.fetched
+ extra = result + nr - self.target
+ if extra > 0:
+ self.target += extra
+ # process_target < 0 indicates shutting down
+ if self.process_target.value >= 0:
+ self.process_target.value = self.target
+ self.wait_event.set()
+ return result
+
+ def RemoveFromBuffer(self):
+ pos = self.local_tail
+ if len(self.buffer) - pos < glb_nsz:
+ pos = 0
+ n = pickle.loads(self.buffer[pos : pos + glb_nsz])
+ if n == 0:
+ pos = 0
+ n = pickle.loads(self.buffer[0 : glb_nsz])
+ pos += glb_nsz
+ obj = pickle.loads(self.buffer[pos : pos + n])
+ self.local_tail = pos + n
+ return obj
+
+ def ProcessData(self, count):
+ for _i in range(count):
+ obj = self.RemoveFromBuffer()
+ self.process_data(obj)
+ self.tail.value = self.local_tail
+ self.wait_event.set()
+ self.done.emit(count)
+
+# Fetch more records bar
+
+class FetchMoreRecordsBar():
+
+ def __init__(self, model, parent):
+ self.model = model
+
+ self.label = QLabel("Number of records (x " + "{:,}".format(glb_chunk_sz) + ") to fetch:")
+ self.label.setSizePolicy(QSizePolicy.Fixed, QSizePolicy.Fixed)
+
+ self.fetch_count = QSpinBox()
+ self.fetch_count.setRange(1, 1000000)
+ self.fetch_count.setValue(10)
+ self.fetch_count.setSizePolicy(QSizePolicy.Fixed, QSizePolicy.Fixed)
+
+ self.fetch = QPushButton("Go!")
+ self.fetch.setSizePolicy(QSizePolicy.Fixed, QSizePolicy.Fixed)
+ self.fetch.released.connect(self.FetchMoreRecords)
+
+ self.progress = QProgressBar()
+ self.progress.setRange(0, 100)
+ self.progress.hide()
+
+ self.done_label = QLabel("All records fetched")
+ self.done_label.hide()
+
+ self.spacer = QLabel("")
+
+ self.close_button = QToolButton()
+ self.close_button.setIcon(parent.style().standardIcon(QStyle.SP_DockWidgetCloseButton))
+ self.close_button.released.connect(self.Deactivate)
+
+ self.hbox = QHBoxLayout()
+ self.hbox.setContentsMargins(0, 0, 0, 0)
+
+ self.hbox.addWidget(self.label)
+ self.hbox.addWidget(self.fetch_count)
+ self.hbox.addWidget(self.fetch)
+ self.hbox.addWidget(self.spacer)
+ self.hbox.addWidget(self.progress)
+ self.hbox.addWidget(self.done_label)
+ self.hbox.addWidget(self.close_button)
+
+ self.bar = QWidget()
+ self.bar.setLayout(self.hbox)
+ self.bar.show()
+
+ self.in_progress = False
+ self.start = 0
+ self.model.progress.connect(self.Progress)
+
+ self.done = False
+
+ if not model.HasMoreRecords():
+ self.Done()
+
+ def Widget(self):
+ return self.bar
+
+ def Activate(self):
+ self.bar.show()
+ self.fetch.setFocus()
+
+ def Deactivate(self):
+ self.bar.hide()
+
+ def Enable(self, enable):
+ self.fetch.setEnabled(enable)
+ self.fetch_count.setEnabled(enable)
+
+ def Busy(self):
+ self.Enable(False)
+ self.fetch.hide()
+ self.spacer.hide()
+ self.progress.show()
+
+ def Idle(self):
+ self.in_progress = False
+ self.Enable(True)
+ self.progress.hide()
+ self.fetch.show()
+ self.spacer.show()
+
+ def Target(self):
+ return self.fetch_count.value() * glb_chunk_sz
+
+ def Done(self):
+ self.done = True
+ self.Idle()
+ self.label.hide()
+ self.fetch_count.hide()
+ self.fetch.hide()
+ self.spacer.hide()
+ self.done_label.show()
+
+ def Progress(self, count):
+ if self.in_progress:
+ if count:
+ percent = ((count - self.start) * 100) / self.Target()
+ if percent >= 100:
+ self.Idle()
+ else:
+ self.progress.setValue(percent)
+ if not count:
+ # Count value of zero means no more records
+ self.Done()
+
+ def FetchMoreRecords(self):
+ if self.done:
+ return
+ self.progress.setValue(0)
+ self.Busy()
+ self.in_progress = True
+ self.start = self.model.FetchMoreRecords(self.Target())
+
+# Brance data model level two item
+
+class BranchLevelTwoItem():
+
+ def __init__(self, row, col, text, parent_item):
+ self.row = row
+ self.parent_item = parent_item
+ self.data = [""] * (col + 1)
+ self.data[col] = text
+ self.level = 2
+
+ def getParentItem(self):
+ return self.parent_item
+
+ def getRow(self):
+ return self.row
+
+ def childCount(self):
+ return 0
+
+ def hasChildren(self):
+ return False
+
+ def getData(self, column):
+ return self.data[column]
+
+# Brance data model level one item
+
+class BranchLevelOneItem():
+
+ def __init__(self, glb, row, data, parent_item):
+ self.glb = glb
+ self.row = row
+ self.parent_item = parent_item
+ self.child_count = 0
+ self.child_items = []
+ self.data = data[1:]
+ self.dbid = data[0]
+ self.level = 1
+ self.query_done = False
+ self.br_col = len(self.data) - 1
+
+ def getChildItem(self, row):
+ return self.child_items[row]
+
+ def getParentItem(self):
+ return self.parent_item
+
+ def getRow(self):
+ return self.row
+
+ def Select(self):
+ self.query_done = True
+
+ if not self.glb.have_disassembler:
+ return
+
+ query = QSqlQuery(self.glb.db)
+
+ QueryExec(query, "SELECT cpu, to_dso_id, to_symbol_id, to_sym_offset, short_name, long_name, build_id, sym_start, to_ip"
+ " FROM samples"
+ " INNER JOIN dsos ON samples.to_dso_id = dsos.id"
+ " INNER JOIN symbols ON samples.to_symbol_id = symbols.id"
+ " WHERE samples.id = " + str(self.dbid))
+ if not query.next():
+ return
+ cpu = query.value(0)
+ dso = query.value(1)
+ sym = query.value(2)
+ if dso == 0 or sym == 0:
+ return
+ off = query.value(3)
+ short_name = query.value(4)
+ long_name = query.value(5)
+ build_id = query.value(6)
+ sym_start = query.value(7)
+ ip = query.value(8)
+
+ QueryExec(query, "SELECT samples.dso_id, symbol_id, sym_offset, sym_start"
+ " FROM samples"
+ " INNER JOIN symbols ON samples.symbol_id = symbols.id"
+ " WHERE samples.id > " + str(self.dbid) + " AND cpu = " + str(cpu) +
+ " ORDER BY samples.id"
+ " LIMIT 1")
+ if not query.next():
+ return
+ if query.value(0) != dso:
+ # Cannot disassemble from one dso to another
+ return
+ bsym = query.value(1)
+ boff = query.value(2)
+ bsym_start = query.value(3)
+ if bsym == 0:
+ return
+ tot = bsym_start + boff + 1 - sym_start - off
+ if tot <= 0 or tot > 16384:
+ return
+
+ inst = self.glb.disassembler.Instruction()
+ f = self.glb.FileFromNamesAndBuildId(short_name, long_name, build_id)
+ if not f:
+ return
+ mode = 0 if Is64Bit(f) else 1
+ self.glb.disassembler.SetMode(inst, mode)
+
+ buf_sz = tot + 16
+ buf = create_string_buffer(tot + 16)
+ f.seek(sym_start + off)
+ buf.value = f.read(buf_sz)
+ buf_ptr = addressof(buf)
+ i = 0
+ while tot > 0:
+ cnt, text = self.glb.disassembler.DisassembleOne(inst, buf_ptr, buf_sz, ip)
+ if cnt:
+ byte_str = tohex(ip).rjust(16)
+ for k in range(cnt):
+ byte_str += " %02x" % ord(buf[i])
+ i += 1
+ while k < 15:
+ byte_str += " "
+ k += 1
+ self.child_items.append(BranchLevelTwoItem(0, self.br_col, byte_str + " " + text, self))
+ self.child_count += 1
+ else:
+ return
+ buf_ptr += cnt
+ tot -= cnt
+ buf_sz -= cnt
+ ip += cnt
+
+ def childCount(self):
+ if not self.query_done:
+ self.Select()
+ if not self.child_count:
+ return -1
+ return self.child_count
+
+ def hasChildren(self):
+ if not self.query_done:
+ return True
+ return self.child_count > 0
+
+ def getData(self, column):
+ return self.data[column]
+
+# Brance data model root item
+
+class BranchRootItem():
+
+ def __init__(self):
+ self.child_count = 0
+ self.child_items = []
+ self.level = 0
+
+ def getChildItem(self, row):
+ return self.child_items[row]
+
+ def getParentItem(self):
+ return None
+
+ def getRow(self):
+ return 0
+
+ def childCount(self):
+ return self.child_count
+
+ def hasChildren(self):
+ return self.child_count > 0
+
+ def getData(self, _column):
+ return ""
+
+# Calculate instructions per cycle
+
+def CalcIPC(cyc_cnt, insn_cnt):
+ if cyc_cnt and insn_cnt:
+ ipc = Decimal(float(insn_cnt) / cyc_cnt)
+ ipc = str(ipc.quantize(Decimal(".01"), rounding=ROUND_HALF_UP))
+ else:
+ ipc = "0"
+ return ipc
+
+# Branch data preparation
+
+def BranchDataPrepBr(query, data):
+ data.append(tohex(query.value(8)).rjust(16) + " " + query.value(9) + offstr(query.value(10)) +
+ " (" + dsoname(query.value(11)) + ")" + " -> " +
+ tohex(query.value(12)) + " " + query.value(13) + offstr(query.value(14)) +
+ " (" + dsoname(query.value(15)) + ")")
+
+def BranchDataPrepIPC(query, data):
+ insn_cnt = query.value(16)
+ cyc_cnt = query.value(17)
+ ipc = CalcIPC(cyc_cnt, insn_cnt)
+ data.append(insn_cnt)
+ data.append(cyc_cnt)
+ data.append(ipc)
+
+def BranchDataPrep(query):
+ data = []
+ for i in range(0, 8):
+ data.append(query.value(i))
+ BranchDataPrepBr(query, data)
+ return data
+
+def BranchDataPrepWA(query):
+ data = []
+ data.append(query.value(0))
+ # Workaround pyside failing to handle large integers (i.e. time) in python3 by converting to a string
+ data.append("{:>19}".format(query.value(1)))
+ for i in range(2, 8):
+ data.append(query.value(i))
+ BranchDataPrepBr(query, data)
+ return data
+
+def BranchDataWithIPCPrep(query):
+ data = []
+ for i in range(0, 8):
+ data.append(query.value(i))
+ BranchDataPrepIPC(query, data)
+ BranchDataPrepBr(query, data)
+ return data
+
+def BranchDataWithIPCPrepWA(query):
+ data = []
+ data.append(query.value(0))
+ # Workaround pyside failing to handle large integers (i.e. time) in python3 by converting to a string
+ data.append("{:>19}".format(query.value(1)))
+ for i in range(2, 8):
+ data.append(query.value(i))
+ BranchDataPrepIPC(query, data)
+ BranchDataPrepBr(query, data)
+ return data
+
+# Branch data model
+
+class BranchModel(TreeModel):
+
+ progress = Signal(object)
+
+ def __init__(self, glb, event_id, where_clause, parent=None):
+ super(BranchModel, self).__init__(glb, None, parent)
+ self.event_id = event_id
+ self.more = True
+ self.populated = 0
+ self.have_ipc = IsSelectable(glb.db, "samples", columns = "insn_count, cyc_count")
+ if self.have_ipc:
+ select_ipc = ", insn_count, cyc_count"
+ prep_fn = BranchDataWithIPCPrep
+ prep_wa_fn = BranchDataWithIPCPrepWA
+ else:
+ select_ipc = ""
+ prep_fn = BranchDataPrep
+ prep_wa_fn = BranchDataPrepWA
+ sql = ("SELECT samples.id, time, cpu, comm, pid, tid, branch_types.name,"
+ " CASE WHEN in_tx = '0' THEN 'No' ELSE 'Yes' END,"
+ " ip, symbols.name, sym_offset, dsos.short_name,"
+ " to_ip, to_symbols.name, to_sym_offset, to_dsos.short_name"
+ + select_ipc +
+ " FROM samples"
+ " INNER JOIN comms ON comm_id = comms.id"
+ " INNER JOIN threads ON thread_id = threads.id"
+ " INNER JOIN branch_types ON branch_type = branch_types.id"
+ " INNER JOIN symbols ON symbol_id = symbols.id"
+ " INNER JOIN symbols to_symbols ON to_symbol_id = to_symbols.id"
+ " INNER JOIN dsos ON samples.dso_id = dsos.id"
+ " INNER JOIN dsos AS to_dsos ON samples.to_dso_id = to_dsos.id"
+ " WHERE samples.id > $$last_id$$" + where_clause +
+ " AND evsel_id = " + str(self.event_id) +
+ " ORDER BY samples.id"
+ " LIMIT " + str(glb_chunk_sz))
+ if pyside_version_1 and sys.version_info[0] == 3:
+ prep = prep_fn
+ else:
+ prep = prep_wa_fn
+ self.fetcher = SQLFetcher(glb, sql, prep, self.AddSample)
+ self.fetcher.done.connect(self.Update)
+ self.fetcher.Fetch(glb_chunk_sz)
+
+ def GetRoot(self):
+ return BranchRootItem()
+
+ def columnCount(self, _parent=None):
+ if self.have_ipc:
+ return 11
+ else:
+ return 8
+
+ def columnHeader(self, column):
+ if self.have_ipc:
+ return ("Time", "CPU", "Command", "PID", "TID", "Branch Type", "In Tx", "Insn Cnt", "Cyc Cnt", "IPC", "Branch")[column]
+ else:
+ return ("Time", "CPU", "Command", "PID", "TID", "Branch Type", "In Tx", "Branch")[column]
+
+ def columnFont(self, column):
+ if self.have_ipc:
+ br_col = 10
+ else:
+ br_col = 7
+ if column != br_col:
+ return None
+ return QFont("Monospace")
+
+ def DisplayData(self, item, index):
+ if item.level == 1:
+ self.FetchIfNeeded(item.row)
+ return item.getData(index.column())
+
+ def AddSample(self, data):
+ child = BranchLevelOneItem(self.glb, self.populated, data, self.root)
+ self.root.child_items.append(child)
+ self.populated += 1
+
+ def Update(self, fetched):
+ if not fetched:
+ self.more = False
+ self.progress.emit(0)
+ child_count = self.root.child_count
+ count = self.populated - child_count
+ if count > 0:
+ parent = QModelIndex()
+ self.beginInsertRows(parent, child_count, child_count + count - 1)
+ self.insertRows(child_count, count, parent)
+ self.root.child_count += count
+ self.endInsertRows()
+ self.progress.emit(self.root.child_count)
+
+ def FetchMoreRecords(self, count):
+ current = self.root.child_count
+ if self.more:
+ self.fetcher.Fetch(count)
+ else:
+ self.progress.emit(0)
+ return current
+
+ def HasMoreRecords(self):
+ return self.more
+
+# Report Variables
+
+class ReportVars():
+
+ def __init__(self, name = "", where_clause = "", limit = ""):
+ self.name = name
+ self.where_clause = where_clause
+ self.limit = limit
+
+ def UniqueId(self):
+ return str(self.where_clause + ";" + self.limit)
+
+# Branch window
+
+class BranchWindow(QMdiSubWindow):
+
+ def __init__(self, glb, event_id, report_vars, parent=None):
+ super(BranchWindow, self).__init__(parent)
+
+ model_name = "Branch Events " + str(event_id) + " " + report_vars.UniqueId()
+
+ self.model = LookupCreateModel(model_name, lambda: BranchModel(glb, event_id, report_vars.where_clause))
+
+ self.view = QTreeView()
+ self.view.setUniformRowHeights(True)
+ self.view.setSelectionMode(QAbstractItemView.ContiguousSelection)
+ self.view.CopyCellsToClipboard = CopyTreeCellsToClipboard
+ self.view.setModel(self.model)
+
+ self.ResizeColumnsToContents()
+
+ self.context_menu = TreeContextMenu(self.view)
+
+ self.find_bar = FindBar(self, self, True)
+
+ self.finder = ChildDataItemFinder(self.model.root)
+
+ self.fetch_bar = FetchMoreRecordsBar(self.model, self)
+
+ self.vbox = VBox(self.view, self.find_bar.Widget(), self.fetch_bar.Widget())
+
+ self.setWidget(self.vbox.Widget())
+
+ AddSubWindow(glb.mainwindow.mdi_area, self, report_vars.name + " Branch Events")
+
+ def ResizeColumnToContents(self, column, n):
+ # Using the view's resizeColumnToContents() here is extrememly slow
+ # so implement a crude alternative
+ mm = "MM" if column else "MMMM"
+ font = self.view.font()
+ metrics = QFontMetrics(font)
+ max_w = 0
+ for row in range(n):
+ val = self.model.root.child_items[row].data[column]
+ width = metrics.width(str(val) + mm)
+ max_w = width if width > max_w else max_w
+ val = self.model.columnHeader(column)
+ width = metrics.width(str(val) + mm)
+ max_w = width if width > max_w else max_w
+ self.view.setColumnWidth(column, max_w)
+
+ def ResizeColumnsToContents(self):
+ n = min(self.model.root.child_count, 100)
+ if n < 1:
+ # No data yet, so connect a signal to notify when there is
+ self.model.rowsInserted.connect(self.UpdateColumnWidths)
+ return
+ columns = self.model.columnCount()
+ for i in range(columns):
+ self.ResizeColumnToContents(i, n)
+
+ def UpdateColumnWidths(self, *_x):
+ # This only needs to be done once, so disconnect the signal now
+ self.model.rowsInserted.disconnect(self.UpdateColumnWidths)
+ self.ResizeColumnsToContents()
+
+ def Find(self, value, direction, pattern, context):
+ self.view.setFocus()
+ self.find_bar.Busy()
+ self.finder.Find(value, direction, pattern, context, self.FindDone)
+
+ def FindDone(self, row):
+ self.find_bar.Idle()
+ if row >= 0:
+ self.view.setCurrentIndex(self.model.index(row, 0, QModelIndex()))
+ else:
+ self.find_bar.NotFound()
+
+# Line edit data item
+
+class LineEditDataItem(object):
+
+ def __init__(self, glb, label, placeholder_text, parent, item_id = "", default = ""):
+ self.glb = glb
+ self.label = label
+ self.placeholder_text = placeholder_text
+ self.parent = parent
+ self.id = item_id
+
+ self.value = default
+
+ self.widget = QLineEdit(default)
+ self.widget.editingFinished.connect(self.Validate)
+ self.widget.textChanged.connect(self.Invalidate)
+ self.red = False
+ self.error = ""
+ self.validated = True
+
+ if placeholder_text:
+ self.widget.setPlaceholderText(placeholder_text)
+
+ def TurnTextRed(self):
+ if not self.red:
+ palette = QPalette()
+ palette.setColor(QPalette.Text,Qt.red)
+ self.widget.setPalette(palette)
+ self.red = True
+
+ def TurnTextNormal(self):
+ if self.red:
+ palette = QPalette()
+ self.widget.setPalette(palette)
+ self.red = False
+
+ def InvalidValue(self, value):
+ self.value = ""
+ self.TurnTextRed()
+ self.error = self.label + " invalid value '" + value + "'"
+ self.parent.ShowMessage(self.error)
+
+ def Invalidate(self):
+ self.validated = False
+
+ def DoValidate(self, input_string):
+ self.value = input_string.strip()
+
+ def Validate(self):
+ self.validated = True
+ self.error = ""
+ self.TurnTextNormal()
+ self.parent.ClearMessage()
+ input_string = self.widget.text()
+ if not len(input_string.strip()):
+ self.value = ""
+ return
+ self.DoValidate(input_string)
+
+ def IsValid(self):
+ if not self.validated:
+ self.Validate()
+ if len(self.error):
+ self.parent.ShowMessage(self.error)
+ return False
+ return True
+
+ def IsNumber(self, value):
+ try:
+ x = int(value)
+ except ValueError:
+ x = 0
+ return str(x) == value
+
+# Non-negative integer ranges dialog data item
+
+class NonNegativeIntegerRangesDataItem(LineEditDataItem):
+
+ def __init__(self, glb, label, placeholder_text, column_name, parent):
+ super(NonNegativeIntegerRangesDataItem, self).__init__(glb, label, placeholder_text, parent)
+
+ self.column_name = column_name
+
+ def DoValidate(self, input_string):
+ singles = []
+ ranges = []
+ for value in [x.strip() for x in input_string.split(",")]:
+ if "-" in value:
+ vrange = value.split("-")
+ if len(vrange) != 2 or not self.IsNumber(vrange[0]) or not self.IsNumber(vrange[1]):
+ return self.InvalidValue(value)
+ ranges.append(vrange)
+ else:
+ if not self.IsNumber(value):
+ return self.InvalidValue(value)
+ singles.append(value)
+ ranges = [("(" + self.column_name + " >= " + r[0] + " AND " + self.column_name + " <= " + r[1] + ")") for r in ranges]
+ if len(singles):
+ ranges.append(self.column_name + " IN (" + ",".join(singles) + ")")
+ self.value = " OR ".join(ranges)
+
+# Positive integer dialog data item
+
+class PositiveIntegerDataItem(LineEditDataItem):
+
+ def DoValidate(self, input_string):
+ if not self.IsNumber(input_string.strip()):
+ return self.InvalidValue(input_string)
+ value = int(input_string.strip())
+ if value <= 0:
+ return self.InvalidValue(input_string)
+ self.value = str(value)
+
+# Dialog data item converted and validated using a SQL table
+
+class SQLTableDataItem(LineEditDataItem):
+
+ def __init__(self, glb, label, placeholder_text, table_name, match_column, column_name1, column_name2, parent):
+ super(SQLTableDataItem, self).__init__(glb, label, placeholder_text, parent)
+
+ self.table_name = table_name
+ self.match_column = match_column
+ self.column_name1 = column_name1
+ self.column_name2 = column_name2
+
+ def ValueToIds(self, value):
+ ids = []
+ query = QSqlQuery(self.glb.db)
+ stmt = "SELECT id FROM " + self.table_name + " WHERE " + self.match_column + " = '" + value + "'"
+ ret = query.exec_(stmt)
+ if ret:
+ while query.next():
+ ids.append(str(query.value(0)))
+ return ids
+
+ def DoValidate(self, input_string):
+ all_ids = []
+ for value in [x.strip() for x in input_string.split(",")]:
+ ids = self.ValueToIds(value)
+ if len(ids):
+ all_ids.extend(ids)
+ else:
+ return self.InvalidValue(value)
+ self.value = self.column_name1 + " IN (" + ",".join(all_ids) + ")"
+ if self.column_name2:
+ self.value = "( " + self.value + " OR " + self.column_name2 + " IN (" + ",".join(all_ids) + ") )"
+
+# Sample time ranges dialog data item converted and validated using 'samples' SQL table
+
+class SampleTimeRangesDataItem(LineEditDataItem):
+
+ def __init__(self, glb, label, placeholder_text, column_name, parent):
+ self.column_name = column_name
+
+ self.last_id = 0
+ self.first_time = 0
+ self.last_time = 2 ** 64
+
+ query = QSqlQuery(glb.db)
+ QueryExec(query, "SELECT id, time FROM samples ORDER BY id DESC LIMIT 1")
+ if query.next():
+ self.last_id = int(query.value(0))
+ self.first_time = int(glb.HostStartTime())
+ self.last_time = int(glb.HostFinishTime())
+ if placeholder_text:
+ placeholder_text += ", between " + str(self.first_time) + " and " + str(self.last_time)
+
+ super(SampleTimeRangesDataItem, self).__init__(glb, label, placeholder_text, parent)
+
+ def IdBetween(self, query, lower_id, higher_id, order):
+ QueryExec(query, "SELECT id FROM samples WHERE id > " + str(lower_id) + " AND id < " + str(higher_id) + " ORDER BY id " + order + " LIMIT 1")
+ if query.next():
+ return True, int(query.value(0))
+ else:
+ return False, 0
+
+ def BinarySearchTime(self, lower_id, higher_id, target_time, get_floor):
+ query = QSqlQuery(self.glb.db)
+ while True:
+ next_id = int((lower_id + higher_id) / 2)
+ QueryExec(query, "SELECT time FROM samples WHERE id = " + str(next_id))
+ if not query.next():
+ ok, dbid = self.IdBetween(query, lower_id, next_id, "DESC")
+ if not ok:
+ ok, dbid = self.IdBetween(query, next_id, higher_id, "")
+ if not ok:
+ return str(higher_id)
+ next_id = dbid
+ QueryExec(query, "SELECT time FROM samples WHERE id = " + str(next_id))
+ next_time = int(query.value(0))
+ if get_floor:
+ if target_time > next_time:
+ lower_id = next_id
+ else:
+ higher_id = next_id
+ if higher_id <= lower_id + 1:
+ return str(higher_id)
+ else:
+ if target_time >= next_time:
+ lower_id = next_id
+ else:
+ higher_id = next_id
+ if higher_id <= lower_id + 1:
+ return str(lower_id)
+
+ def ConvertRelativeTime(self, val):
+ mult = 1
+ suffix = val[-2:]
+ if suffix == "ms":
+ mult = 1000000
+ elif suffix == "us":
+ mult = 1000
+ elif suffix == "ns":
+ mult = 1
+ else:
+ return val
+ val = val[:-2].strip()
+ if not self.IsNumber(val):
+ return val
+ val = int(val) * mult
+ if val >= 0:
+ val += self.first_time
+ else:
+ val += self.last_time
+ return str(val)
+
+ def ConvertTimeRange(self, vrange):
+ if vrange[0] == "":
+ vrange[0] = str(self.first_time)
+ if vrange[1] == "":
+ vrange[1] = str(self.last_time)
+ vrange[0] = self.ConvertRelativeTime(vrange[0])
+ vrange[1] = self.ConvertRelativeTime(vrange[1])
+ if not self.IsNumber(vrange[0]) or not self.IsNumber(vrange[1]):
+ return False
+ beg_range = max(int(vrange[0]), self.first_time)
+ end_range = min(int(vrange[1]), self.last_time)
+ if beg_range > self.last_time or end_range < self.first_time:
+ return False
+ vrange[0] = self.BinarySearchTime(0, self.last_id, beg_range, True)
+ vrange[1] = self.BinarySearchTime(1, self.last_id + 1, end_range, False)
+ return True
+
+ def AddTimeRange(self, value, ranges):
+ n = value.count("-")
+ if n == 1:
+ pass
+ elif n == 2:
+ if value.split("-")[1].strip() == "":
+ n = 1
+ elif n == 3:
+ n = 2
+ else:
+ return False
+ pos = findnth(value, "-", n)
+ vrange = [value[:pos].strip() ,value[pos+1:].strip()]
+ if self.ConvertTimeRange(vrange):
+ ranges.append(vrange)
+ return True
+ return False
+
+ def DoValidate(self, input_string):
+ ranges = []
+ for value in [x.strip() for x in input_string.split(",")]:
+ if not self.AddTimeRange(value, ranges):
+ return self.InvalidValue(value)
+ ranges = [("(" + self.column_name + " >= " + r[0] + " AND " + self.column_name + " <= " + r[1] + ")") for r in ranges]
+ self.value = " OR ".join(ranges)
+
+# Report Dialog Base
+
+class ReportDialogBase(QDialog):
+
+ def __init__(self, glb, title, items, partial, parent=None):
+ super(ReportDialogBase, self).__init__(parent)
+
+ self.glb = glb
+
+ self.report_vars = ReportVars()
+
+ self.setWindowTitle(title)
+ self.setMinimumWidth(600)
+
+ self.data_items = [x(glb, self) for x in items]
+
+ self.partial = partial
+
+ self.grid = QGridLayout()
+
+ for row in range(len(self.data_items)):
+ self.grid.addWidget(QLabel(self.data_items[row].label), row, 0)
+ self.grid.addWidget(self.data_items[row].widget, row, 1)
+
+ self.status = QLabel()
+
+ self.ok_button = QPushButton("Ok", self)
+ self.ok_button.setDefault(True)
+ self.ok_button.released.connect(self.Ok)
+ self.ok_button.setSizePolicy(QSizePolicy.Fixed, QSizePolicy.Fixed)
+
+ self.cancel_button = QPushButton("Cancel", self)
+ self.cancel_button.released.connect(self.reject)
+ self.cancel_button.setSizePolicy(QSizePolicy.Fixed, QSizePolicy.Fixed)
+
+ self.hbox = QHBoxLayout()
+ #self.hbox.addStretch()
+ self.hbox.addWidget(self.status)
+ self.hbox.addWidget(self.ok_button)
+ self.hbox.addWidget(self.cancel_button)
+
+ self.vbox = QVBoxLayout()
+ self.vbox.addLayout(self.grid)
+ self.vbox.addLayout(self.hbox)
+
+ self.setLayout(self.vbox)
+
+ def Ok(self):
+ rep_vars = self.report_vars
+ for d in self.data_items:
+ if d.id == "REPORTNAME":
+ rep_vars.name = d.value
+ if not rep_vars.name:
+ self.ShowMessage("Report name is required")
+ return
+ for d in self.data_items:
+ if not d.IsValid():
+ return
+ for d in self.data_items[1:]:
+ if d.id == "LIMIT":
+ rep_vars.limit = d.value
+ elif len(d.value):
+ if len(rep_vars.where_clause):
+ rep_vars.where_clause += " AND "
+ rep_vars.where_clause += d.value
+ if len(rep_vars.where_clause):
+ if self.partial:
+ rep_vars.where_clause = " AND ( " + rep_vars.where_clause + " ) "
+ else:
+ rep_vars.where_clause = " WHERE " + rep_vars.where_clause + " "
+ self.accept()
+
+ def ShowMessage(self, msg):
+ self.status.setText("<font color=#FF0000>" + msg)
+
+ def ClearMessage(self):
+ self.status.setText("")
+
+# Selected branch report creation dialog
+
+class SelectedBranchDialog(ReportDialogBase):
+
+ def __init__(self, glb, parent=None):
+ title = "Selected Branches"
+ items = (lambda g, p: LineEditDataItem(g, "Report name:", "Enter a name to appear in the window title bar", p, "REPORTNAME"),
+ lambda g, p: SampleTimeRangesDataItem(g, "Time ranges:", "Enter time ranges", "samples.id", p),
+ lambda g, p: NonNegativeIntegerRangesDataItem(g, "CPUs:", "Enter CPUs or ranges e.g. 0,5-6", "cpu", p),
+ lambda g, p: SQLTableDataItem(g, "Commands:", "Only branches with these commands will be included", "comms", "comm", "comm_id", "", p),
+ lambda g, p: SQLTableDataItem(g, "PIDs:", "Only branches with these process IDs will be included", "threads", "pid", "thread_id", "", p),
+ lambda g, p: SQLTableDataItem(g, "TIDs:", "Only branches with these thread IDs will be included", "threads", "tid", "thread_id", "", p),
+ lambda g, p: SQLTableDataItem(g, "DSOs:", "Only branches with these DSOs will be included", "dsos", "short_name", "samples.dso_id", "to_dso_id", p),
+ lambda g, p: SQLTableDataItem(g, "Symbols:", "Only branches with these symbols will be included", "symbols", "name", "symbol_id", "to_symbol_id", p),
+ lambda g, p: LineEditDataItem(g, "Raw SQL clause: ", "Enter a raw SQL WHERE clause", p))
+ super(SelectedBranchDialog, self).__init__(glb, title, items, True, parent)
+
+# Event list
+
+def GetEventList(db):
+ events = []
+ query = QSqlQuery(db)
+ QueryExec(query, "SELECT name FROM selected_events WHERE id > 0 ORDER BY id")
+ while query.next():
+ events.append(query.value(0))
+ return events
+
+# Is a table selectable
+
+def IsSelectable(db, table, sql = "", columns = "*"):
+ query = QSqlQuery(db)
+ try:
+ QueryExec(query, "SELECT " + columns + " FROM " + table + " " + sql + " LIMIT 1")
+ except RuntimeError:
+ return False
+ return True
+
+# SQL table data model item
+
+class SQLTableItem():
+
+ def __init__(self, row, data):
+ self.row = row
+ self.data = data
+
+ def getData(self, column):
+ return self.data[column]
+
+# SQL table data model
+
+class SQLTableModel(TableModel):
+
+ progress = Signal(object)
+
+ def __init__(self, glb, sql, column_headers, parent=None):
+ super(SQLTableModel, self).__init__(parent)
+ self.glb = glb
+ self.more = True
+ self.populated = 0
+ self.column_headers = column_headers
+ self.fetcher = SQLFetcher(glb, sql, lambda x, y=len(column_headers): self.SQLTableDataPrep(x, y), self.AddSample)
+ self.fetcher.done.connect(self.Update)
+ self.fetcher.Fetch(glb_chunk_sz)
+
+ def DisplayData(self, item, index):
+ self.FetchIfNeeded(item.row)
+ return item.getData(index.column())
+
+ def AddSample(self, data):
+ child = SQLTableItem(self.populated, data)
+ self.child_items.append(child)
+ self.populated += 1
+
+ def Update(self, fetched):
+ if not fetched:
+ self.more = False
+ self.progress.emit(0)
+ child_count = self.child_count
+ count = self.populated - child_count
+ if count > 0:
+ parent = QModelIndex()
+ self.beginInsertRows(parent, child_count, child_count + count - 1)
+ self.insertRows(child_count, count, parent)
+ self.child_count += count
+ self.endInsertRows()
+ self.progress.emit(self.child_count)
+
+ def FetchMoreRecords(self, count):
+ current = self.child_count
+ if self.more:
+ self.fetcher.Fetch(count)
+ else:
+ self.progress.emit(0)
+ return current
+
+ def HasMoreRecords(self):
+ return self.more
+
+ def columnCount(self, _parent=None):
+ return len(self.column_headers)
+
+ def columnHeader(self, column):
+ return self.column_headers[column]
+
+ def SQLTableDataPrep(self, query, count):
+ data = []
+ for i in range(count):
+ data.append(query.value(i))
+ return data
+
+# SQL automatic table data model
+
+class SQLAutoTableModel(SQLTableModel):
+
+ def __init__(self, glb, table_name, parent=None):
+ sql = "SELECT * FROM " + table_name + " WHERE id > $$last_id$$ ORDER BY id LIMIT " + str(glb_chunk_sz)
+ if table_name == "comm_threads_view":
+ # For now, comm_threads_view has no id column
+ sql = "SELECT * FROM " + table_name + " WHERE comm_id > $$last_id$$ ORDER BY comm_id LIMIT " + str(glb_chunk_sz)
+ column_headers = []
+ query = QSqlQuery(glb.db)
+ if glb.dbref.is_sqlite3:
+ QueryExec(query, "PRAGMA table_info(" + table_name + ")")
+ while query.next():
+ column_headers.append(query.value(1))
+ if table_name == "sqlite_master":
+ sql = "SELECT * FROM " + table_name
+ else:
+ if table_name[:19] == "information_schema.":
+ sql = "SELECT * FROM " + table_name
+ select_table_name = table_name[19:]
+ schema = "information_schema"
+ else:
+ select_table_name = table_name
+ schema = "public"
+ QueryExec(query, "SELECT column_name FROM information_schema.columns WHERE table_schema = '" + schema + "' and table_name = '" + select_table_name + "'")
+ while query.next():
+ column_headers.append(query.value(0))
+ if pyside_version_1 and sys.version_info[0] == 3:
+ if table_name == "samples_view":
+ self.SQLTableDataPrep = self.samples_view_DataPrep
+ if table_name == "samples":
+ self.SQLTableDataPrep = self.samples_DataPrep
+ super(SQLAutoTableModel, self).__init__(glb, sql, column_headers, parent)
+
+ def samples_view_DataPrep(self, query, count):
+ data = []
+ data.append(query.value(0))
+ # Workaround pyside failing to handle large integers (i.e. time) in python3 by converting to a string
+ data.append("{:>19}".format(query.value(1)))
+ for i in range(2, count):
+ data.append(query.value(i))
+ return data
+
+ def samples_DataPrep(self, query, count):
+ data = []
+ for i in range(9):
+ data.append(query.value(i))
+ # Workaround pyside failing to handle large integers (i.e. time) in python3 by converting to a string
+ data.append("{:>19}".format(query.value(9)))
+ for i in range(10, count):
+ data.append(query.value(i))
+ return data
+
+# Base class for custom ResizeColumnsToContents
+
+class ResizeColumnsToContentsBase(QObject):
+
+ def __init__(self, parent=None):
+ super(ResizeColumnsToContentsBase, self).__init__(parent)
+
+ def ResizeColumnToContents(self, column, n):
+ # Using the view's resizeColumnToContents() here is extrememly slow
+ # so implement a crude alternative
+ font = self.view.font()
+ metrics = QFontMetrics(font)
+ max_w = 0
+ for row in range(n):
+ val = self.data_model.child_items[row].data[column]
+ width = metrics.width(str(val) + "MM")
+ max_w = width if width > max_w else max_w
+ val = self.data_model.columnHeader(column)
+ width = metrics.width(str(val) + "MM")
+ max_w = width if width > max_w else max_w
+ self.view.setColumnWidth(column, max_w)
+
+ def ResizeColumnsToContents(self):
+ n = min(self.data_model.child_count, 100)
+ if n < 1:
+ # No data yet, so connect a signal to notify when there is
+ self.data_model.rowsInserted.connect(self.UpdateColumnWidths)
+ return
+ columns = self.data_model.columnCount()
+ for i in range(columns):
+ self.ResizeColumnToContents(i, n)
+
+ def UpdateColumnWidths(self, *_x):
+ # This only needs to be done once, so disconnect the signal now
+ self.data_model.rowsInserted.disconnect(self.UpdateColumnWidths)
+ self.ResizeColumnsToContents()
+
+# Convert value to CSV
+
+def ToCSValue(val):
+ if '"' in val:
+ val = val.replace('"', '""')
+ if "," in val or '"' in val:
+ val = '"' + val + '"'
+ return val
+
+# Key to sort table model indexes by row / column, assuming fewer than 1000 columns
+
+glb_max_cols = 1000
+
+def RowColumnKey(a):
+ return a.row() * glb_max_cols + a.column()
+
+# Copy selected table cells to clipboard
+
+def CopyTableCellsToClipboard(view, as_csv=False, with_hdr=False):
+ indexes = sorted(view.selectedIndexes(), key=RowColumnKey)
+ idx_cnt = len(indexes)
+ if not idx_cnt:
+ return
+ if idx_cnt == 1:
+ with_hdr=False
+ min_row = indexes[0].row()
+ max_row = indexes[0].row()
+ min_col = indexes[0].column()
+ max_col = indexes[0].column()
+ for i in indexes:
+ min_row = min(min_row, i.row())
+ max_row = max(max_row, i.row())
+ min_col = min(min_col, i.column())
+ max_col = max(max_col, i.column())
+ if max_col > glb_max_cols:
+ raise RuntimeError("glb_max_cols is too low")
+ max_width = [0] * (1 + max_col - min_col)
+ for i in indexes:
+ c = i.column() - min_col
+ max_width[c] = max(max_width[c], len(str(i.data())))
+ text = ""
+ pad = ""
+ sep = ""
+ if with_hdr:
+ model = indexes[0].model()
+ for col in range(min_col, max_col + 1):
+ val = model.headerData(col, Qt.Horizontal, Qt.DisplayRole)
+ if as_csv:
+ text += sep + ToCSValue(val)
+ sep = ","
+ else:
+ c = col - min_col
+ max_width[c] = max(max_width[c], len(val))
+ width = max_width[c]
+ align = model.headerData(col, Qt.Horizontal, Qt.TextAlignmentRole)
+ if align & Qt.AlignRight:
+ val = val.rjust(width)
+ text += pad + sep + val
+ pad = " " * (width - len(val))
+ sep = " "
+ text += "\n"
+ pad = ""
+ sep = ""
+ last_row = min_row
+ for i in indexes:
+ if i.row() > last_row:
+ last_row = i.row()
+ text += "\n"
+ pad = ""
+ sep = ""
+ if as_csv:
+ text += sep + ToCSValue(str(i.data()))
+ sep = ","
+ else:
+ width = max_width[i.column() - min_col]
+ if i.data(Qt.TextAlignmentRole) & Qt.AlignRight:
+ val = str(i.data()).rjust(width)
+ else:
+ val = str(i.data())
+ text += pad + sep + val
+ pad = " " * (width - len(val))
+ sep = " "
+ QApplication.clipboard().setText(text)
+
+def CopyTreeCellsToClipboard(view, as_csv=False, with_hdr=False):
+ indexes = view.selectedIndexes()
+ if not len(indexes):
+ return
+
+ selection = view.selectionModel()
+
+ first = None
+ for i in indexes:
+ above = view.indexAbove(i)
+ if not selection.isSelected(above):
+ first = i
+ break
+
+ if first is None:
+ raise RuntimeError("CopyTreeCellsToClipboard internal error")
+
+ model = first.model()
+ row_cnt = 0
+ col_cnt = model.columnCount(first)
+ max_width = [0] * col_cnt
+
+ indent_sz = 2
+ indent_str = " " * indent_sz
+
+ expanded_mark_sz = 2
+ if sys.version_info[0] == 3:
+ expanded_mark = "\u25BC "
+ not_expanded_mark = "\u25B6 "
+ else:
+ expanded_mark = chr(0x25BC) + " "
+ not_expanded_mark = chr(0x25B6) + " "
+ leaf_mark = " "
+
+ if not as_csv:
+ pos = first
+ while True:
+ row_cnt += 1
+ row = pos.row()
+ for c in range(col_cnt):
+ i = pos.sibling(row, c)
+ if c:
+ n = len(str(i.data()))
+ else:
+ n = len(str(i.data()).strip())
+ n += (i.internalPointer().level - 1) * indent_sz
+ n += expanded_mark_sz
+ max_width[c] = max(max_width[c], n)
+ pos = view.indexBelow(pos)
+ if not selection.isSelected(pos):
+ break
+
+ text = ""
+ pad = ""
+ sep = ""
+ if with_hdr:
+ for c in range(col_cnt):
+ val = model.headerData(c, Qt.Horizontal, Qt.DisplayRole).strip()
+ if as_csv:
+ text += sep + ToCSValue(val)
+ sep = ","
+ else:
+ max_width[c] = max(max_width[c], len(val))
+ width = max_width[c]
+ align = model.headerData(c, Qt.Horizontal, Qt.TextAlignmentRole)
+ if align & Qt.AlignRight:
+ val = val.rjust(width)
+ text += pad + sep + val
+ pad = " " * (width - len(val))
+ sep = " "
+ text += "\n"
+ pad = ""
+ sep = ""
+
+ pos = first
+ while True:
+ row = pos.row()
+ for c in range(col_cnt):
+ i = pos.sibling(row, c)
+ val = str(i.data())
+ if not c:
+ if model.hasChildren(i):
+ if view.isExpanded(i):
+ mark = expanded_mark
+ else:
+ mark = not_expanded_mark
+ else:
+ mark = leaf_mark
+ val = indent_str * (i.internalPointer().level - 1) + mark + val.strip()
+ if as_csv:
+ text += sep + ToCSValue(val)
+ sep = ","
+ else:
+ width = max_width[c]
+ if c and i.data(Qt.TextAlignmentRole) & Qt.AlignRight:
+ val = val.rjust(width)
+ text += pad + sep + val
+ pad = " " * (width - len(val))
+ sep = " "
+ pos = view.indexBelow(pos)
+ if not selection.isSelected(pos):
+ break
+ text = text.rstrip() + "\n"
+ pad = ""
+ sep = ""
+
+ QApplication.clipboard().setText(text)
+
+def CopyCellsToClipboard(view, as_csv=False, with_hdr=False):
+ view.CopyCellsToClipboard(view, as_csv, with_hdr)
+
+def CopyCellsToClipboardHdr(view):
+ CopyCellsToClipboard(view, False, True)
+
+def CopyCellsToClipboardCSV(view):
+ CopyCellsToClipboard(view, True, True)
+
+# Context menu
+
+class ContextMenu(object):
+
+ def __init__(self, view):
+ self.view = view
+ self.view.setContextMenuPolicy(Qt.CustomContextMenu)
+ self.view.customContextMenuRequested.connect(self.ShowContextMenu)
+
+ def ShowContextMenu(self, pos):
+ menu = QMenu(self.view)
+ self.AddActions(menu)
+ menu.exec_(self.view.mapToGlobal(pos))
+
+ def AddCopy(self, menu):
+ menu.addAction(CreateAction("&Copy selection", "Copy to clipboard", lambda: CopyCellsToClipboardHdr(self.view), self.view))
+ menu.addAction(CreateAction("Copy selection as CS&V", "Copy to clipboard as CSV", lambda: CopyCellsToClipboardCSV(self.view), self.view))
+
+ def AddActions(self, menu):
+ self.AddCopy(menu)
+
+class TreeContextMenu(ContextMenu):
+
+ def AddActions(self, menu):
+ i = self.view.currentIndex()
+ text = str(i.data()).strip()
+ if len(text):
+ menu.addAction(CreateAction('Copy "' + text + '"', "Copy to clipboard", lambda: QApplication.clipboard().setText(text), self.view))
+ self.AddCopy(menu)
+
+# Table window
+
+class TableWindow(QMdiSubWindow, ResizeColumnsToContentsBase):
+
+ def __init__(self, glb, table_name, parent=None):
+ super(TableWindow, self).__init__(parent)
+
+ self.data_model = LookupCreateModel(table_name + " Table", lambda: SQLAutoTableModel(glb, table_name))
+
+ self.model = QSortFilterProxyModel()
+ self.model.setSourceModel(self.data_model)
+
+ self.view = QTableView()
+ self.view.setModel(self.model)
+ self.view.setEditTriggers(QAbstractItemView.NoEditTriggers)
+ self.view.verticalHeader().setVisible(False)
+ self.view.sortByColumn(-1, Qt.AscendingOrder)
+ self.view.setSortingEnabled(True)
+ self.view.setSelectionMode(QAbstractItemView.ContiguousSelection)
+ self.view.CopyCellsToClipboard = CopyTableCellsToClipboard
+
+ self.ResizeColumnsToContents()
+
+ self.context_menu = ContextMenu(self.view)
+
+ self.find_bar = FindBar(self, self, True)
+
+ self.finder = ChildDataItemFinder(self.data_model)
+
+ self.fetch_bar = FetchMoreRecordsBar(self.data_model, self)
+
+ self.vbox = VBox(self.view, self.find_bar.Widget(), self.fetch_bar.Widget())
+
+ self.setWidget(self.vbox.Widget())
+
+ AddSubWindow(glb.mainwindow.mdi_area, self, table_name + " Table")
+
+ def Find(self, value, direction, pattern, context):
+ self.view.setFocus()
+ self.find_bar.Busy()
+ self.finder.Find(value, direction, pattern, context, self.FindDone)
+
+ def FindDone(self, row):
+ self.find_bar.Idle()
+ if row >= 0:
+ self.view.setCurrentIndex(self.model.mapFromSource(self.data_model.index(row, 0, QModelIndex())))
+ else:
+ self.find_bar.NotFound()
+
+# Table list
+
+def GetTableList(glb):
+ tables = []
+ query = QSqlQuery(glb.db)
+ if glb.dbref.is_sqlite3:
+ QueryExec(query, "SELECT name FROM sqlite_master WHERE type IN ( 'table' , 'view' ) ORDER BY name")
+ else:
+ QueryExec(query, "SELECT table_name FROM information_schema.tables WHERE table_schema = 'public' AND table_type IN ( 'BASE TABLE' , 'VIEW' ) ORDER BY table_name")
+ while query.next():
+ tables.append(query.value(0))
+ if glb.dbref.is_sqlite3:
+ tables.append("sqlite_master")
+ else:
+ tables.append("information_schema.tables")
+ tables.append("information_schema.views")
+ tables.append("information_schema.columns")
+ return tables
+
+# Top Calls data model
+
+class TopCallsModel(SQLTableModel):
+
+ def __init__(self, glb, report_vars, parent=None):
+ text = ""
+ if not glb.dbref.is_sqlite3:
+ text = "::text"
+ limit = ""
+ if len(report_vars.limit):
+ limit = " LIMIT " + report_vars.limit
+ sql = ("SELECT comm, pid, tid, name,"
+ " CASE"
+ " WHEN (short_name = '[kernel.kallsyms]') THEN '[kernel]'" + text +
+ " ELSE short_name"
+ " END AS dso,"
+ " call_time, return_time, (return_time - call_time) AS elapsed_time, branch_count, "
+ " CASE"
+ " WHEN (calls.flags = 1) THEN 'no call'" + text +
+ " WHEN (calls.flags = 2) THEN 'no return'" + text +
+ " WHEN (calls.flags = 3) THEN 'no call/return'" + text +
+ " ELSE ''" + text +
+ " END AS flags"
+ " FROM calls"
+ " INNER JOIN call_paths ON calls.call_path_id = call_paths.id"
+ " INNER JOIN symbols ON call_paths.symbol_id = symbols.id"
+ " INNER JOIN dsos ON symbols.dso_id = dsos.id"
+ " INNER JOIN comms ON calls.comm_id = comms.id"
+ " INNER JOIN threads ON calls.thread_id = threads.id" +
+ report_vars.where_clause +
+ " ORDER BY elapsed_time DESC" +
+ limit
+ )
+ column_headers = ("Command", "PID", "TID", "Symbol", "Object", "Call Time", "Return Time", "Elapsed Time (ns)", "Branch Count", "Flags")
+ self.alignment = (Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignRight, Qt.AlignRight, Qt.AlignLeft)
+ super(TopCallsModel, self).__init__(glb, sql, column_headers, parent)
+
+ def columnAlignment(self, column):
+ return self.alignment[column]
+
+# Top Calls report creation dialog
+
+class TopCallsDialog(ReportDialogBase):
+
+ def __init__(self, glb, parent=None):
+ title = "Top Calls by Elapsed Time"
+ items = (lambda g, p: LineEditDataItem(g, "Report name:", "Enter a name to appear in the window title bar", p, "REPORTNAME"),
+ lambda g, p: SQLTableDataItem(g, "Commands:", "Only calls with these commands will be included", "comms", "comm", "comm_id", "", p),
+ lambda g, p: SQLTableDataItem(g, "PIDs:", "Only calls with these process IDs will be included", "threads", "pid", "thread_id", "", p),
+ lambda g, p: SQLTableDataItem(g, "TIDs:", "Only calls with these thread IDs will be included", "threads", "tid", "thread_id", "", p),
+ lambda g, p: SQLTableDataItem(g, "DSOs:", "Only calls with these DSOs will be included", "dsos", "short_name", "dso_id", "", p),
+ lambda g, p: SQLTableDataItem(g, "Symbols:", "Only calls with these symbols will be included", "symbols", "name", "symbol_id", "", p),
+ lambda g, p: LineEditDataItem(g, "Raw SQL clause: ", "Enter a raw SQL WHERE clause", p),
+ lambda g, p: PositiveIntegerDataItem(g, "Record limit:", "Limit selection to this number of records", p, "LIMIT", "100"))
+ super(TopCallsDialog, self).__init__(glb, title, items, False, parent)
+
+# Top Calls window
+
+class TopCallsWindow(QMdiSubWindow, ResizeColumnsToContentsBase):
+
+ def __init__(self, glb, report_vars, parent=None):
+ super(TopCallsWindow, self).__init__(parent)
+
+ self.data_model = LookupCreateModel("Top Calls " + report_vars.UniqueId(), lambda: TopCallsModel(glb, report_vars))
+ self.model = self.data_model
+
+ self.view = QTableView()
+ self.view.setModel(self.model)
+ self.view.setEditTriggers(QAbstractItemView.NoEditTriggers)
+ self.view.verticalHeader().setVisible(False)
+ self.view.setSelectionMode(QAbstractItemView.ContiguousSelection)
+ self.view.CopyCellsToClipboard = CopyTableCellsToClipboard
+
+ self.context_menu = ContextMenu(self.view)
+
+ self.ResizeColumnsToContents()
+
+ self.find_bar = FindBar(self, self, True)
+
+ self.finder = ChildDataItemFinder(self.model)
+
+ self.fetch_bar = FetchMoreRecordsBar(self.data_model, self)
+
+ self.vbox = VBox(self.view, self.find_bar.Widget(), self.fetch_bar.Widget())
+
+ self.setWidget(self.vbox.Widget())
+
+ AddSubWindow(glb.mainwindow.mdi_area, self, report_vars.name)
+
+ def Find(self, value, direction, pattern, context):
+ self.view.setFocus()
+ self.find_bar.Busy()
+ self.finder.Find(value, direction, pattern, context, self.FindDone)
+
+ def FindDone(self, row):
+ self.find_bar.Idle()
+ if row >= 0:
+ self.view.setCurrentIndex(self.model.index(row, 0, QModelIndex()))
+ else:
+ self.find_bar.NotFound()
+
+# Action Definition
+
+def CreateAction(label, tip, callback, parent=None, shortcut=None):
+ action = QAction(label, parent)
+ if shortcut != None:
+ action.setShortcuts(shortcut)
+ action.setStatusTip(tip)
+ action.triggered.connect(callback)
+ return action
+
+# Typical application actions
+
+def CreateExitAction(app, parent=None):
+ return CreateAction("&Quit", "Exit the application", app.closeAllWindows, parent, QKeySequence.Quit)
+
+# Typical MDI actions
+
+def CreateCloseActiveWindowAction(mdi_area):
+ return CreateAction("Cl&ose", "Close the active window", mdi_area.closeActiveSubWindow, mdi_area)
+
+def CreateCloseAllWindowsAction(mdi_area):
+ return CreateAction("Close &All", "Close all the windows", mdi_area.closeAllSubWindows, mdi_area)
+
+def CreateTileWindowsAction(mdi_area):
+ return CreateAction("&Tile", "Tile the windows", mdi_area.tileSubWindows, mdi_area)
+
+def CreateCascadeWindowsAction(mdi_area):
+ return CreateAction("&Cascade", "Cascade the windows", mdi_area.cascadeSubWindows, mdi_area)
+
+def CreateNextWindowAction(mdi_area):
+ return CreateAction("Ne&xt", "Move the focus to the next window", mdi_area.activateNextSubWindow, mdi_area, QKeySequence.NextChild)
+
+def CreatePreviousWindowAction(mdi_area):
+ return CreateAction("Pre&vious", "Move the focus to the previous window", mdi_area.activatePreviousSubWindow, mdi_area, QKeySequence.PreviousChild)
+
+# Typical MDI window menu
+
+class WindowMenu():
+
+ def __init__(self, mdi_area, menu):
+ self.mdi_area = mdi_area
+ self.window_menu = menu.addMenu("&Windows")
+ self.close_active_window = CreateCloseActiveWindowAction(mdi_area)
+ self.close_all_windows = CreateCloseAllWindowsAction(mdi_area)
+ self.tile_windows = CreateTileWindowsAction(mdi_area)
+ self.cascade_windows = CreateCascadeWindowsAction(mdi_area)
+ self.next_window = CreateNextWindowAction(mdi_area)
+ self.previous_window = CreatePreviousWindowAction(mdi_area)
+ self.window_menu.aboutToShow.connect(self.Update)
+
+ def Update(self):
+ self.window_menu.clear()
+ sub_window_count = len(self.mdi_area.subWindowList())
+ have_sub_windows = sub_window_count != 0
+ self.close_active_window.setEnabled(have_sub_windows)
+ self.close_all_windows.setEnabled(have_sub_windows)
+ self.tile_windows.setEnabled(have_sub_windows)
+ self.cascade_windows.setEnabled(have_sub_windows)
+ self.next_window.setEnabled(have_sub_windows)
+ self.previous_window.setEnabled(have_sub_windows)
+ self.window_menu.addAction(self.close_active_window)
+ self.window_menu.addAction(self.close_all_windows)
+ self.window_menu.addSeparator()
+ self.window_menu.addAction(self.tile_windows)
+ self.window_menu.addAction(self.cascade_windows)
+ self.window_menu.addSeparator()
+ self.window_menu.addAction(self.next_window)
+ self.window_menu.addAction(self.previous_window)
+ if sub_window_count == 0:
+ return
+ self.window_menu.addSeparator()
+ nr = 1
+ for sub_window in self.mdi_area.subWindowList():
+ label = str(nr) + " " + sub_window.name
+ if nr < 10:
+ label = "&" + label
+ action = self.window_menu.addAction(label)
+ action.setCheckable(True)
+ action.setChecked(sub_window == self.mdi_area.activeSubWindow())
+ action.triggered.connect(lambda a=None,x=nr: self.setActiveSubWindow(x))
+ self.window_menu.addAction(action)
+ nr += 1
+
+ def setActiveSubWindow(self, nr):
+ self.mdi_area.setActiveSubWindow(self.mdi_area.subWindowList()[nr - 1])
+
+# Help text
+
+glb_help_text = """
+<h1>Contents</h1>
+<style>
+p.c1 {
+ text-indent: 40px;
+}
+p.c2 {
+ text-indent: 80px;
+}
+}
+</style>
+<p class=c1><a href=#reports>1. Reports</a></p>
+<p class=c2><a href=#callgraph>1.1 Context-Sensitive Call Graph</a></p>
+<p class=c2><a href=#calltree>1.2 Call Tree</a></p>
+<p class=c2><a href=#allbranches>1.3 All branches</a></p>
+<p class=c2><a href=#selectedbranches>1.4 Selected branches</a></p>
+<p class=c2><a href=#topcallsbyelapsedtime>1.5 Top calls by elapsed time</a></p>
+<p class=c1><a href=#charts>2. Charts</a></p>
+<p class=c2><a href=#timechartbycpu>2.1 Time chart by CPU</a></p>
+<p class=c1><a href=#tables>3. Tables</a></p>
+<h1 id=reports>1. Reports</h1>
+<h2 id=callgraph>1.1 Context-Sensitive Call Graph</h2>
+The result is a GUI window with a tree representing a context-sensitive
+call-graph. Expanding a couple of levels of the tree and adjusting column
+widths to suit will display something like:
+<pre>
+ Call Graph: pt_example
+Call Path Object Count Time(ns) Time(%) Branch Count Branch Count(%)
+v- ls
+ v- 2638:2638
+ v- _start ld-2.19.so 1 10074071 100.0 211135 100.0
+ |- unknown unknown 1 13198 0.1 1 0.0
+ >- _dl_start ld-2.19.so 1 1400980 13.9 19637 9.3
+ >- _d_linit_internal ld-2.19.so 1 448152 4.4 11094 5.3
+ v-__libc_start_main@plt ls 1 8211741 81.5 180397 85.4
+ >- _dl_fixup ld-2.19.so 1 7607 0.1 108 0.1
+ >- __cxa_atexit libc-2.19.so 1 11737 0.1 10 0.0
+ >- __libc_csu_init ls 1 10354 0.1 10 0.0
+ |- _setjmp libc-2.19.so 1 0 0.0 4 0.0
+ v- main ls 1 8182043 99.6 180254 99.9
+</pre>
+<h3>Points to note:</h3>
+<ul>
+<li>The top level is a command name (comm)</li>
+<li>The next level is a thread (pid:tid)</li>
+<li>Subsequent levels are functions</li>
+<li>'Count' is the number of calls</li>
+<li>'Time' is the elapsed time until the function returns</li>
+<li>Percentages are relative to the level above</li>
+<li>'Branch Count' is the total number of branches for that function and all functions that it calls
+</ul>
+<h3>Find</h3>
+Ctrl-F displays a Find bar which finds function names by either an exact match or a pattern match.
+The pattern matching symbols are ? for any character and * for zero or more characters.
+<h2 id=calltree>1.2 Call Tree</h2>
+The Call Tree report is very similar to the Context-Sensitive Call Graph, but the data is not aggregated.
+Also the 'Count' column, which would be always 1, is replaced by the 'Call Time'.
+<h2 id=allbranches>1.3 All branches</h2>
+The All branches report displays all branches in chronological order.
+Not all data is fetched immediately. More records can be fetched using the Fetch bar provided.
+<h3>Disassembly</h3>
+Open a branch to display disassembly. This only works if:
+<ol>
+<li>The disassembler is available. Currently, only Intel XED is supported - see <a href=#xed>Intel XED Setup</a></li>
+<li>The object code is available. Currently, only the perf build ID cache is searched for object code.
+The default directory ~/.debug can be overridden by setting environment variable PERF_BUILDID_DIR.
+One exception is kcore where the DSO long name is used (refer dsos_view on the Tables menu),
+or alternatively, set environment variable PERF_KCORE to the kcore file name.</li>
+</ol>
+<h4 id=xed>Intel XED Setup</h4>
+To use Intel XED, libxed.so must be present. To build and install libxed.so:
+<pre>
+git clone https://github.com/intelxed/mbuild.git mbuild
+git clone https://github.com/intelxed/xed
+cd xed
+./mfile.py --share
+sudo ./mfile.py --prefix=/usr/local install
+sudo ldconfig
+</pre>
+<h3>Instructions per Cycle (IPC)</h3>
+If available, IPC information is displayed in columns 'insn_cnt', 'cyc_cnt' and 'IPC'.
+<p><b>Intel PT note:</b> The information applies to the blocks of code ending with, and including, that branch.
+Due to the granularity of timing information, the number of cycles for some code blocks will not be known.
+In that case, 'insn_cnt', 'cyc_cnt' and 'IPC' are zero, but when 'IPC' is displayed it covers the period
+since the previous displayed 'IPC'.
+<h3>Find</h3>
+Ctrl-F displays a Find bar which finds substrings by either an exact match or a regular expression match.
+Refer to Python documentation for the regular expression syntax.
+All columns are searched, but only currently fetched rows are searched.
+<h2 id=selectedbranches>1.4 Selected branches</h2>
+This is the same as the <a href=#allbranches>All branches</a> report but with the data reduced
+by various selection criteria. A dialog box displays available criteria which are AND'ed together.
+<h3>1.4.1 Time ranges</h3>
+The time ranges hint text shows the total time range. Relative time ranges can also be entered in
+ms, us or ns. Also, negative values are relative to the end of trace. Examples:
+<pre>
+ 81073085947329-81073085958238 From 81073085947329 to 81073085958238
+ 100us-200us From 100us to 200us
+ 10ms- From 10ms to the end
+ -100ns The first 100ns
+ -10ms- The last 10ms
+</pre>
+N.B. Due to the granularity of timestamps, there could be no branches in any given time range.
+<h2 id=topcallsbyelapsedtime>1.5 Top calls by elapsed time</h2>
+The Top calls by elapsed time report displays calls in descending order of time elapsed between when the function was called and when it returned.
+The data is reduced by various selection criteria. A dialog box displays available criteria which are AND'ed together.
+If not all data is fetched, a Fetch bar is provided. Ctrl-F displays a Find bar.
+<h1 id=charts>2. Charts</h1>
+<h2 id=timechartbycpu>2.1 Time chart by CPU</h2>
+This chart displays context switch information when that data is available. Refer to context_switches_view on the Tables menu.
+<h3>Features</h3>
+<ol>
+<li>Mouse over to highight the task and show the time</li>
+<li>Drag the mouse to select a region and zoom by pushing the Zoom button</li>
+<li>Go back and forward by pressing the arrow buttons</li>
+<li>If call information is available, right-click to show a call tree opened to that task and time.
+Note, the call tree may take some time to appear, and there may not be call information for the task or time selected.
+</li>
+</ol>
+<h3>Important</h3>
+The graph can be misleading in the following respects:
+<ol>
+<li>The graph shows the first task on each CPU as running from the beginning of the time range.
+Because tracing might start on different CPUs at different times, that is not necessarily the case.
+Refer to context_switches_view on the Tables menu to understand what data the graph is based upon.</li>
+<li>Similarly, the last task on each CPU can be showing running longer than it really was.
+Again, refer to context_switches_view on the Tables menu to understand what data the graph is based upon.</li>
+<li>When the mouse is over a task, the highlighted task might not be visible on the legend without scrolling if the legend does not fit fully in the window</li>
+</ol>
+<h1 id=tables>3. Tables</h1>
+The Tables menu shows all tables and views in the database. Most tables have an associated view
+which displays the information in a more friendly way. Not all data for large tables is fetched
+immediately. More records can be fetched using the Fetch bar provided. Columns can be sorted,
+but that can be slow for large tables.
+<p>There are also tables of database meta-information.
+For SQLite3 databases, the sqlite_master table is included.
+For PostgreSQL databases, information_schema.tables/views/columns are included.
+<h3>Find</h3>
+Ctrl-F displays a Find bar which finds substrings by either an exact match or a regular expression match.
+Refer to Python documentation for the regular expression syntax.
+All columns are searched, but only currently fetched rows are searched.
+<p>N.B. Results are found in id order, so if the table is re-ordered, find-next and find-previous
+will go to the next/previous result in id order, instead of display order.
+"""
+
+# Help window
+
+class HelpWindow(QMdiSubWindow):
+
+ def __init__(self, glb, parent=None):
+ super(HelpWindow, self).__init__(parent)
+
+ self.text = QTextBrowser()
+ self.text.setHtml(glb_help_text)
+ self.text.setReadOnly(True)
+ self.text.setOpenExternalLinks(True)
+
+ self.setWidget(self.text)
+
+ AddSubWindow(glb.mainwindow.mdi_area, self, "Exported SQL Viewer Help")
+
+# Main window that only displays the help text
+
+class HelpOnlyWindow(QMainWindow):
+
+ def __init__(self, parent=None):
+ super(HelpOnlyWindow, self).__init__(parent)
+
+ self.setMinimumSize(200, 100)
+ self.resize(800, 600)
+ self.setWindowTitle("Exported SQL Viewer Help")
+ self.setWindowIcon(self.style().standardIcon(QStyle.SP_MessageBoxInformation))
+
+ self.text = QTextBrowser()
+ self.text.setHtml(glb_help_text)
+ self.text.setReadOnly(True)
+ self.text.setOpenExternalLinks(True)
+
+ self.setCentralWidget(self.text)
+
+# PostqreSQL server version
+
+def PostqreSQLServerVersion(db):
+ query = QSqlQuery(db)
+ QueryExec(query, "SELECT VERSION()")
+ if query.next():
+ v_str = query.value(0)
+ v_list = v_str.strip().split(" ")
+ if v_list[0] == "PostgreSQL" and v_list[2] == "on":
+ return v_list[1]
+ return v_str
+ return "Unknown"
+
+# SQLite version
+
+def SQLiteVersion(db):
+ query = QSqlQuery(db)
+ QueryExec(query, "SELECT sqlite_version()")
+ if query.next():
+ return query.value(0)
+ return "Unknown"
+
+# About dialog
+
+class AboutDialog(QDialog):
+
+ def __init__(self, glb, parent=None):
+ super(AboutDialog, self).__init__(parent)
+
+ self.setWindowTitle("About Exported SQL Viewer")
+ self.setMinimumWidth(300)
+
+ pyside_version = "1" if pyside_version_1 else "2"
+
+ text = "<pre>"
+ text += "Python version: " + sys.version.split(" ")[0] + "\n"
+ text += "PySide version: " + pyside_version + "\n"
+ text += "Qt version: " + qVersion() + "\n"
+ if glb.dbref.is_sqlite3:
+ text += "SQLite version: " + SQLiteVersion(glb.db) + "\n"
+ else:
+ text += "PostqreSQL version: " + PostqreSQLServerVersion(glb.db) + "\n"
+ text += "</pre>"
+
+ self.text = QTextBrowser()
+ self.text.setHtml(text)
+ self.text.setReadOnly(True)
+ self.text.setOpenExternalLinks(True)
+
+ self.vbox = QVBoxLayout()
+ self.vbox.addWidget(self.text)
+
+ self.setLayout(self.vbox)
+
+# Font resize
+
+def ResizeFont(widget, diff):
+ font = widget.font()
+ sz = font.pointSize()
+ font.setPointSize(sz + diff)
+ widget.setFont(font)
+
+def ShrinkFont(widget):
+ ResizeFont(widget, -1)
+
+def EnlargeFont(widget):
+ ResizeFont(widget, 1)
+
+# Unique name for sub-windows
+
+def NumberedWindowName(name, nr):
+ if nr > 1:
+ name += " <" + str(nr) + ">"
+ return name
+
+def UniqueSubWindowName(mdi_area, name):
+ nr = 1
+ while True:
+ unique_name = NumberedWindowName(name, nr)
+ ok = True
+ for sub_window in mdi_area.subWindowList():
+ if sub_window.name == unique_name:
+ ok = False
+ break
+ if ok:
+ return unique_name
+ nr += 1
+
+# Add a sub-window
+
+def AddSubWindow(mdi_area, sub_window, name):
+ unique_name = UniqueSubWindowName(mdi_area, name)
+ sub_window.setMinimumSize(200, 100)
+ sub_window.resize(800, 600)
+ sub_window.setWindowTitle(unique_name)
+ sub_window.setAttribute(Qt.WA_DeleteOnClose)
+ sub_window.setWindowIcon(sub_window.style().standardIcon(QStyle.SP_FileIcon))
+ sub_window.name = unique_name
+ mdi_area.addSubWindow(sub_window)
+ sub_window.show()
+
+# Main window
+
+class MainWindow(QMainWindow):
+
+ def __init__(self, glb, parent=None):
+ super(MainWindow, self).__init__(parent)
+
+ self.glb = glb
+
+ self.setWindowTitle("Exported SQL Viewer: " + glb.dbname)
+ self.setWindowIcon(self.style().standardIcon(QStyle.SP_ComputerIcon))
+ self.setMinimumSize(200, 100)
+
+ self.mdi_area = QMdiArea()
+ self.mdi_area.setHorizontalScrollBarPolicy(Qt.ScrollBarAsNeeded)
+ self.mdi_area.setVerticalScrollBarPolicy(Qt.ScrollBarAsNeeded)
+
+ self.setCentralWidget(self.mdi_area)
+
+ menu = self.menuBar()
+
+ file_menu = menu.addMenu("&File")
+ file_menu.addAction(CreateExitAction(glb.app, self))
+
+ edit_menu = menu.addMenu("&Edit")
+ edit_menu.addAction(CreateAction("&Copy", "Copy to clipboard", self.CopyToClipboard, self, QKeySequence.Copy))
+ edit_menu.addAction(CreateAction("Copy as CS&V", "Copy to clipboard as CSV", self.CopyToClipboardCSV, self))
+ edit_menu.addAction(CreateAction("&Find...", "Find items", self.Find, self, QKeySequence.Find))
+ edit_menu.addAction(CreateAction("Fetch &more records...", "Fetch more records", self.FetchMoreRecords, self, [QKeySequence(Qt.Key_F8)]))
+ edit_menu.addAction(CreateAction("&Shrink Font", "Make text smaller", self.ShrinkFont, self, [QKeySequence("Ctrl+-")]))
+ edit_menu.addAction(CreateAction("&Enlarge Font", "Make text bigger", self.EnlargeFont, self, [QKeySequence("Ctrl++")]))
+
+ reports_menu = menu.addMenu("&Reports")
+ if IsSelectable(glb.db, "calls"):
+ reports_menu.addAction(CreateAction("Context-Sensitive Call &Graph", "Create a new window containing a context-sensitive call graph", self.NewCallGraph, self))
+
+ if IsSelectable(glb.db, "calls", "WHERE parent_id >= 0"):
+ reports_menu.addAction(CreateAction("Call &Tree", "Create a new window containing a call tree", self.NewCallTree, self))
+
+ self.EventMenu(GetEventList(glb.db), reports_menu)
+
+ if IsSelectable(glb.db, "calls"):
+ reports_menu.addAction(CreateAction("&Top calls by elapsed time", "Create a new window displaying top calls by elapsed time", self.NewTopCalls, self))
+
+ if IsSelectable(glb.db, "context_switches"):
+ charts_menu = menu.addMenu("&Charts")
+ charts_menu.addAction(CreateAction("&Time chart by CPU", "Create a new window displaying time charts by CPU", self.TimeChartByCPU, self))
+
+ self.TableMenu(GetTableList(glb), menu)
+
+ self.window_menu = WindowMenu(self.mdi_area, menu)
+
+ help_menu = menu.addMenu("&Help")
+ help_menu.addAction(CreateAction("&Exported SQL Viewer Help", "Helpful information", self.Help, self, QKeySequence.HelpContents))
+ help_menu.addAction(CreateAction("&About Exported SQL Viewer", "About this application", self.About, self))
+
+ def Try(self, fn):
+ win = self.mdi_area.activeSubWindow()
+ if win:
+ try:
+ fn(win.view)
+ except AttributeError:
+ pass
+
+ def CopyToClipboard(self):
+ self.Try(CopyCellsToClipboardHdr)
+
+ def CopyToClipboardCSV(self):
+ self.Try(CopyCellsToClipboardCSV)
+
+ def Find(self):
+ win = self.mdi_area.activeSubWindow()
+ if win:
+ try:
+ win.find_bar.Activate()
+ except AttributeError:
+ pass
+
+ def FetchMoreRecords(self):
+ win = self.mdi_area.activeSubWindow()
+ if win:
+ try:
+ win.fetch_bar.Activate()
+ except AttributeError:
+ pass
+
+ def ShrinkFont(self):
+ self.Try(ShrinkFont)
+
+ def EnlargeFont(self):
+ self.Try(EnlargeFont)
+
+ def EventMenu(self, events, reports_menu):
+ branches_events = 0
+ for event in events:
+ event = event.split(":")[0]
+ if event == "branches":
+ branches_events += 1
+ dbid = 0
+ for event in events:
+ dbid += 1
+ event = event.split(":")[0]
+ if event == "branches":
+ label = "All branches" if branches_events == 1 else "All branches " + "(id=" + dbid + ")"
+ reports_menu.addAction(CreateAction(label, "Create a new window displaying branch events", lambda a=None,x=dbid: self.NewBranchView(x), self))
+ label = "Selected branches" if branches_events == 1 else "Selected branches " + "(id=" + dbid + ")"
+ reports_menu.addAction(CreateAction(label, "Create a new window displaying branch events", lambda a=None,x=dbid: self.NewSelectedBranchView(x), self))
+
+ def TimeChartByCPU(self):
+ TimeChartByCPUWindow(self.glb, self)
+
+ def TableMenu(self, tables, menu):
+ table_menu = menu.addMenu("&Tables")
+ for table in tables:
+ table_menu.addAction(CreateAction(table, "Create a new window containing a table view", lambda a=None,t=table: self.NewTableView(t), self))
+
+ def NewCallGraph(self):
+ CallGraphWindow(self.glb, self)
+
+ def NewCallTree(self):
+ CallTreeWindow(self.glb, self)
+
+ def NewTopCalls(self):
+ dialog = TopCallsDialog(self.glb, self)
+ ret = dialog.exec_()
+ if ret:
+ TopCallsWindow(self.glb, dialog.report_vars, self)
+
+ def NewBranchView(self, event_id):
+ BranchWindow(self.glb, event_id, ReportVars(), self)
+
+ def NewSelectedBranchView(self, event_id):
+ dialog = SelectedBranchDialog(self.glb, self)
+ ret = dialog.exec_()
+ if ret:
+ BranchWindow(self.glb, event_id, dialog.report_vars, self)
+
+ def NewTableView(self, table_name):
+ TableWindow(self.glb, table_name, self)
+
+ def Help(self):
+ HelpWindow(self.glb, self)
+
+ def About(self):
+ dialog = AboutDialog(self.glb, self)
+ dialog.exec_()
+
+def TryOpen(file_name):
+ try:
+ return open(file_name, "rb")
+ except OSError:
+ return None
+
+def Is64Bit(f):
+ result = sizeof(c_void_p)
+ # ELF support only
+ pos = f.tell()
+ f.seek(0)
+ header = f.read(7)
+ f.seek(pos)
+ magic = header[0:4]
+ if sys.version_info[0] == 2:
+ eclass = ord(header[4])
+ encoding = ord(header[5])
+ version = ord(header[6])
+ else:
+ eclass = header[4]
+ encoding = header[5]
+ version = header[6]
+ if magic == chr(127) + "ELF" and eclass > 0 and eclass < 3 and encoding > 0 and encoding < 3 and version == 1:
+ result = True if eclass == 2 else False
+ return result
+
+# Global data
+
+class Glb():
+
+ def __init__(self, dbref, db, dbname):
+ self.dbref = dbref
+ self.db = db
+ self.dbname = dbname
+ self.home_dir = os.path.expanduser("~")
+ self.buildid_dir = os.getenv("PERF_BUILDID_DIR")
+ if self.buildid_dir:
+ self.buildid_dir += "/.build-id/"
+ else:
+ self.buildid_dir = self.home_dir + "/.debug/.build-id/"
+ self.app = None
+ self.mainwindow = None
+ self.instances_to_shutdown_on_exit = weakref.WeakSet()
+ try:
+ self.disassembler = LibXED()
+ self.have_disassembler = True
+ except (OSError, RuntimeError, AttributeError):
+ self.have_disassembler = False
+ self.host_machine_id = 0
+ self.host_start_time = 0
+ self.host_finish_time = 0
+
+ def FileFromBuildId(self, build_id):
+ file_name = self.buildid_dir + build_id[0:2] + "/" + build_id[2:] + "/elf"
+ return TryOpen(file_name)
+
+ def FileFromNamesAndBuildId(self, short_name, long_name, build_id):
+ # Assume current machine i.e. no support for virtualization
+ if short_name[0:7] == "[kernel" and os.path.basename(long_name) == "kcore":
+ file_name = os.getenv("PERF_KCORE")
+ f = TryOpen(file_name) if file_name else None
+ if f:
+ return f
+ # For now, no special handling if long_name is /proc/kcore
+ f = TryOpen(long_name)
+ if f:
+ return f
+ f = self.FileFromBuildId(build_id)
+ if f:
+ return f
+ return None
+
+ def AddInstanceToShutdownOnExit(self, instance):
+ self.instances_to_shutdown_on_exit.add(instance)
+
+ # Shutdown any background processes or threads
+ def ShutdownInstances(self):
+ for x in self.instances_to_shutdown_on_exit:
+ try:
+ x.Shutdown()
+ except AttributeError:
+ pass
+
+ def GetHostMachineId(self):
+ query = QSqlQuery(self.db)
+ QueryExec(query, "SELECT id FROM machines WHERE pid = -1")
+ if query.next():
+ self.host_machine_id = query.value(0)
+ else:
+ self.host_machine_id = 0
+ return self.host_machine_id
+
+ def HostMachineId(self):
+ if self.host_machine_id:
+ return self.host_machine_id
+ return self.GetHostMachineId()
+
+ def SelectValue(self, sql):
+ query = QSqlQuery(self.db)
+ try:
+ QueryExec(query, sql)
+ except RuntimeError:
+ return None
+ if query.next():
+ return Decimal(query.value(0))
+ return None
+
+ def SwitchesMinTime(self, machine_id):
+ return self.SelectValue("SELECT time"
+ " FROM context_switches"
+ " WHERE time != 0 AND machine_id = " + str(machine_id) +
+ " ORDER BY id LIMIT 1")
+
+ def SwitchesMaxTime(self, machine_id):
+ return self.SelectValue("SELECT time"
+ " FROM context_switches"
+ " WHERE time != 0 AND machine_id = " + str(machine_id) +
+ " ORDER BY id DESC LIMIT 1")
+
+ def SamplesMinTime(self, machine_id):
+ return self.SelectValue("SELECT time"
+ " FROM samples"
+ " WHERE time != 0 AND machine_id = " + str(machine_id) +
+ " ORDER BY id LIMIT 1")
+
+ def SamplesMaxTime(self, machine_id):
+ return self.SelectValue("SELECT time"
+ " FROM samples"
+ " WHERE time != 0 AND machine_id = " + str(machine_id) +
+ " ORDER BY id DESC LIMIT 1")
+
+ def CallsMinTime(self, machine_id):
+ return self.SelectValue("SELECT calls.call_time"
+ " FROM calls"
+ " INNER JOIN threads ON threads.thread_id = calls.thread_id"
+ " WHERE calls.call_time != 0 AND threads.machine_id = " + str(machine_id) +
+ " ORDER BY calls.id LIMIT 1")
+
+ def CallsMaxTime(self, machine_id):
+ return self.SelectValue("SELECT calls.return_time"
+ " FROM calls"
+ " INNER JOIN threads ON threads.thread_id = calls.thread_id"
+ " WHERE calls.return_time != 0 AND threads.machine_id = " + str(machine_id) +
+ " ORDER BY calls.return_time DESC LIMIT 1")
+
+ def GetStartTime(self, machine_id):
+ t0 = self.SwitchesMinTime(machine_id)
+ t1 = self.SamplesMinTime(machine_id)
+ t2 = self.CallsMinTime(machine_id)
+ if t0 is None or (not(t1 is None) and t1 < t0):
+ t0 = t1
+ if t0 is None or (not(t2 is None) and t2 < t0):
+ t0 = t2
+ return t0
+
+ def GetFinishTime(self, machine_id):
+ t0 = self.SwitchesMaxTime(machine_id)
+ t1 = self.SamplesMaxTime(machine_id)
+ t2 = self.CallsMaxTime(machine_id)
+ if t0 is None or (not(t1 is None) and t1 > t0):
+ t0 = t1
+ if t0 is None or (not(t2 is None) and t2 > t0):
+ t0 = t2
+ return t0
+
+ def HostStartTime(self):
+ if self.host_start_time:
+ return self.host_start_time
+ self.host_start_time = self.GetStartTime(self.HostMachineId())
+ return self.host_start_time
+
+ def HostFinishTime(self):
+ if self.host_finish_time:
+ return self.host_finish_time
+ self.host_finish_time = self.GetFinishTime(self.HostMachineId())
+ return self.host_finish_time
+
+ def StartTime(self, machine_id):
+ if machine_id == self.HostMachineId():
+ return self.HostStartTime()
+ return self.GetStartTime(machine_id)
+
+ def FinishTime(self, machine_id):
+ if machine_id == self.HostMachineId():
+ return self.HostFinishTime()
+ return self.GetFinishTime(machine_id)
+
+# Database reference
+
+class DBRef():
+
+ def __init__(self, is_sqlite3, dbname):
+ self.is_sqlite3 = is_sqlite3
+ self.dbname = dbname
+ self.TRUE = "TRUE"
+ self.FALSE = "FALSE"
+ # SQLite prior to version 3.23 does not support TRUE and FALSE
+ if self.is_sqlite3:
+ self.TRUE = "1"
+ self.FALSE = "0"
+
+ def Open(self, connection_name):
+ dbname = self.dbname
+ if self.is_sqlite3:
+ db = QSqlDatabase.addDatabase("QSQLITE", connection_name)
+ else:
+ db = QSqlDatabase.addDatabase("QPSQL", connection_name)
+ opts = dbname.split()
+ for opt in opts:
+ if "=" in opt:
+ opt = opt.split("=")
+ if opt[0] == "hostname":
+ db.setHostName(opt[1])
+ elif opt[0] == "port":
+ db.setPort(int(opt[1]))
+ elif opt[0] == "username":
+ db.setUserName(opt[1])
+ elif opt[0] == "password":
+ db.setPassword(opt[1])
+ elif opt[0] == "dbname":
+ dbname = opt[1]
+ else:
+ dbname = opt
+
+ db.setDatabaseName(dbname)
+ if not db.open():
+ raise RuntimeError("Failed to open database " + dbname + " error: " + db.lastError().text())
+ return db, dbname
+
+# Main
+
+def Main():
+ usage_str = "exported-sql-viewer.py [--pyside-version-1] <database name>\n" \
+ " or: exported-sql-viewer.py --help-only"
+ ap = argparse.ArgumentParser(usage = usage_str, add_help = False)
+ ap.add_argument("--pyside-version-1", action='store_true')
+ ap.add_argument("dbname", nargs="?")
+ ap.add_argument("--help-only", action='store_true')
+ args = ap.parse_args()
+
+ if args.help_only:
+ app = QApplication(sys.argv)
+ mainwindow = HelpOnlyWindow()
+ mainwindow.show()
+ err = app.exec_()
+ sys.exit(err)
+
+ dbname = args.dbname
+ if dbname is None:
+ ap.print_usage()
+ print("Too few arguments")
+ sys.exit(1)
+
+ is_sqlite3 = False
+ try:
+ f = open(dbname, "rb")
+ if f.read(15) == b'SQLite format 3':
+ is_sqlite3 = True
+ f.close()
+ except OSError:
+ pass
+
+ dbref = DBRef(is_sqlite3, dbname)
+ db, dbname = dbref.Open("main")
+ glb = Glb(dbref, db, dbname)
+ app = QApplication(sys.argv)
+ glb.app = app
+ mainwindow = MainWindow(glb)
+ glb.mainwindow = mainwindow
+ mainwindow.show()
+ err = app.exec_()
+ glb.ShutdownInstances()
+ db.close()
+ sys.exit(err)
+
+if __name__ == "__main__":
+ Main()
diff --git a/tools/perf/scripts/python/exported-sql-viewer.py b/tools/perf/scripts/python/exported-sql-viewer.py
deleted file mode 100755
index e0b2e7268ef6..000000000000
--- a/tools/perf/scripts/python/exported-sql-viewer.py
+++ /dev/null
@@ -1,5030 +0,0 @@
-#!/usr/bin/env python
-# SPDX-License-Identifier: GPL-2.0
-# exported-sql-viewer.py: view data from sql database
-# Copyright (c) 2014-2018, Intel Corporation.
-
-# To use this script you will need to have exported data using either the
-# export-to-sqlite.py or the export-to-postgresql.py script. Refer to those
-# scripts for details.
-#
-# Following on from the example in the export scripts, a
-# call-graph can be displayed for the pt_example database like this:
-#
-# python tools/perf/scripts/python/exported-sql-viewer.py pt_example
-#
-# Note that for PostgreSQL, this script supports connecting to remote databases
-# by setting hostname, port, username, password, and dbname e.g.
-#
-# python tools/perf/scripts/python/exported-sql-viewer.py "hostname=myhost username=myuser password=mypassword dbname=pt_example"
-#
-# The result is a GUI window with a tree representing a context-sensitive
-# call-graph. Expanding a couple of levels of the tree and adjusting column
-# widths to suit will display something like:
-#
-# Call Graph: pt_example
-# Call Path Object Count Time(ns) Time(%) Branch Count Branch Count(%)
-# v- ls
-# v- 2638:2638
-# v- _start ld-2.19.so 1 10074071 100.0 211135 100.0
-# |- unknown unknown 1 13198 0.1 1 0.0
-# >- _dl_start ld-2.19.so 1 1400980 13.9 19637 9.3
-# >- _d_linit_internal ld-2.19.so 1 448152 4.4 11094 5.3
-# v-__libc_start_main@plt ls 1 8211741 81.5 180397 85.4
-# >- _dl_fixup ld-2.19.so 1 7607 0.1 108 0.1
-# >- __cxa_atexit libc-2.19.so 1 11737 0.1 10 0.0
-# >- __libc_csu_init ls 1 10354 0.1 10 0.0
-# |- _setjmp libc-2.19.so 1 0 0.0 4 0.0
-# v- main ls 1 8182043 99.6 180254 99.9
-#
-# Points to note:
-# The top level is a command name (comm)
-# The next level is a thread (pid:tid)
-# Subsequent levels are functions
-# 'Count' is the number of calls
-# 'Time' is the elapsed time until the function returns
-# Percentages are relative to the level above
-# 'Branch Count' is the total number of branches for that function and all
-# functions that it calls
-
-# There is also a "All branches" report, which displays branches and
-# possibly disassembly. However, presently, the only supported disassembler is
-# Intel XED, and additionally the object code must be present in perf build ID
-# cache. To use Intel XED, libxed.so must be present. To build and install
-# libxed.so:
-# git clone https://github.com/intelxed/mbuild.git mbuild
-# git clone https://github.com/intelxed/xed
-# cd xed
-# ./mfile.py --share
-# sudo ./mfile.py --prefix=/usr/local install
-# sudo ldconfig
-#
-# Example report:
-#
-# Time CPU Command PID TID Branch Type In Tx Branch
-# 8107675239590 2 ls 22011 22011 return from interrupt No ffffffff86a00a67 native_irq_return_iret ([kernel]) -> 7fab593ea260 _start (ld-2.19.so)
-# 7fab593ea260 48 89 e7 mov %rsp, %rdi
-# 8107675239899 2 ls 22011 22011 hardware interrupt No 7fab593ea260 _start (ld-2.19.so) -> ffffffff86a012e0 page_fault ([kernel])
-# 8107675241900 2 ls 22011 22011 return from interrupt No ffffffff86a00a67 native_irq_return_iret ([kernel]) -> 7fab593ea260 _start (ld-2.19.so)
-# 7fab593ea260 48 89 e7 mov %rsp, %rdi
-# 7fab593ea263 e8 c8 06 00 00 callq 0x7fab593ea930
-# 8107675241900 2 ls 22011 22011 call No 7fab593ea263 _start+0x3 (ld-2.19.so) -> 7fab593ea930 _dl_start (ld-2.19.so)
-# 7fab593ea930 55 pushq %rbp
-# 7fab593ea931 48 89 e5 mov %rsp, %rbp
-# 7fab593ea934 41 57 pushq %r15
-# 7fab593ea936 41 56 pushq %r14
-# 7fab593ea938 41 55 pushq %r13
-# 7fab593ea93a 41 54 pushq %r12
-# 7fab593ea93c 53 pushq %rbx
-# 7fab593ea93d 48 89 fb mov %rdi, %rbx
-# 7fab593ea940 48 83 ec 68 sub $0x68, %rsp
-# 7fab593ea944 0f 31 rdtsc
-# 7fab593ea946 48 c1 e2 20 shl $0x20, %rdx
-# 7fab593ea94a 89 c0 mov %eax, %eax
-# 7fab593ea94c 48 09 c2 or %rax, %rdx
-# 7fab593ea94f 48 8b 05 1a 15 22 00 movq 0x22151a(%rip), %rax
-# 8107675242232 2 ls 22011 22011 hardware interrupt No 7fab593ea94f _dl_start+0x1f (ld-2.19.so) -> ffffffff86a012e0 page_fault ([kernel])
-# 8107675242900 2 ls 22011 22011 return from interrupt No ffffffff86a00a67 native_irq_return_iret ([kernel]) -> 7fab593ea94f _dl_start+0x1f (ld-2.19.so)
-# 7fab593ea94f 48 8b 05 1a 15 22 00 movq 0x22151a(%rip), %rax
-# 7fab593ea956 48 89 15 3b 13 22 00 movq %rdx, 0x22133b(%rip)
-# 8107675243232 2 ls 22011 22011 hardware interrupt No 7fab593ea956 _dl_start+0x26 (ld-2.19.so) -> ffffffff86a012e0 page_fault ([kernel])
-
-from __future__ import print_function
-
-import sys
-# Only change warnings if the python -W option was not used
-if not sys.warnoptions:
- import warnings
- # PySide2 causes deprecation warnings, ignore them.
- warnings.filterwarnings("ignore", category=DeprecationWarning)
-import argparse
-import weakref
-import threading
-import string
-try:
- # Python2
- import cPickle as pickle
- # size of pickled integer big enough for record size
- glb_nsz = 8
-except ImportError:
- import pickle
- glb_nsz = 16
-import re
-import os
-import random
-import copy
-import math
-from libxed import LibXED
-
-pyside_version_1 = True
-if not "--pyside-version-1" in sys.argv:
- try:
- from PySide2.QtCore import *
- from PySide2.QtGui import *
- from PySide2.QtSql import *
- from PySide2.QtWidgets import *
- pyside_version_1 = False
- except:
- pass
-
-if pyside_version_1:
- from PySide.QtCore import *
- from PySide.QtGui import *
- from PySide.QtSql import *
-
-from decimal import Decimal, ROUND_HALF_UP
-from ctypes import CDLL, Structure, create_string_buffer, addressof, sizeof, \
- c_void_p, c_bool, c_byte, c_char, c_int, c_uint, c_longlong, c_ulonglong
-from multiprocessing import Process, Array, Value, Event
-
-# xrange is range in Python3
-try:
- xrange
-except NameError:
- xrange = range
-
-def printerr(*args, **keyword_args):
- print(*args, file=sys.stderr, **keyword_args)
-
-# Data formatting helpers
-
-def tohex(ip):
- if ip < 0:
- ip += 1 << 64
- return "%x" % ip
-
-def offstr(offset):
- if offset:
- return "+0x%x" % offset
- return ""
-
-def dsoname(name):
- if name == "[kernel.kallsyms]":
- return "[kernel]"
- return name
-
-def findnth(s, sub, n, offs=0):
- pos = s.find(sub)
- if pos < 0:
- return pos
- if n <= 1:
- return offs + pos
- return findnth(s[pos + 1:], sub, n - 1, offs + pos + 1)
-
-# Percent to one decimal place
-
-def PercentToOneDP(n, d):
- if not d:
- return "0.0"
- x = (n * Decimal(100)) / d
- return str(x.quantize(Decimal(".1"), rounding=ROUND_HALF_UP))
-
-# Helper for queries that must not fail
-
-def QueryExec(query, stmt):
- ret = query.exec_(stmt)
- if not ret:
- raise Exception("Query failed: " + query.lastError().text())
-
-# Background thread
-
-class Thread(QThread):
-
- done = Signal(object)
-
- def __init__(self, task, param=None, parent=None):
- super(Thread, self).__init__(parent)
- self.task = task
- self.param = param
-
- def run(self):
- while True:
- if self.param is None:
- done, result = self.task()
- else:
- done, result = self.task(self.param)
- self.done.emit(result)
- if done:
- break
-
-# Tree data model
-
-class TreeModel(QAbstractItemModel):
-
- def __init__(self, glb, params, parent=None):
- super(TreeModel, self).__init__(parent)
- self.glb = glb
- self.params = params
- self.root = self.GetRoot()
- self.last_row_read = 0
-
- def Item(self, parent):
- if parent.isValid():
- return parent.internalPointer()
- else:
- return self.root
-
- def rowCount(self, parent):
- result = self.Item(parent).childCount()
- if result < 0:
- result = 0
- self.dataChanged.emit(parent, parent)
- return result
-
- def hasChildren(self, parent):
- return self.Item(parent).hasChildren()
-
- def headerData(self, section, orientation, role):
- if role == Qt.TextAlignmentRole:
- return self.columnAlignment(section)
- if role != Qt.DisplayRole:
- return None
- if orientation != Qt.Horizontal:
- return None
- return self.columnHeader(section)
-
- def parent(self, child):
- child_item = child.internalPointer()
- if child_item is self.root:
- return QModelIndex()
- parent_item = child_item.getParentItem()
- return self.createIndex(parent_item.getRow(), 0, parent_item)
-
- def index(self, row, column, parent):
- child_item = self.Item(parent).getChildItem(row)
- return self.createIndex(row, column, child_item)
-
- def DisplayData(self, item, index):
- return item.getData(index.column())
-
- def FetchIfNeeded(self, row):
- if row > self.last_row_read:
- self.last_row_read = row
- if row + 10 >= self.root.child_count:
- self.fetcher.Fetch(glb_chunk_sz)
-
- def columnAlignment(self, column):
- return Qt.AlignLeft
-
- def columnFont(self, column):
- return None
-
- def data(self, index, role):
- if role == Qt.TextAlignmentRole:
- return self.columnAlignment(index.column())
- if role == Qt.FontRole:
- return self.columnFont(index.column())
- if role != Qt.DisplayRole:
- return None
- item = index.internalPointer()
- return self.DisplayData(item, index)
-
-# Table data model
-
-class TableModel(QAbstractTableModel):
-
- def __init__(self, parent=None):
- super(TableModel, self).__init__(parent)
- self.child_count = 0
- self.child_items = []
- self.last_row_read = 0
-
- def Item(self, parent):
- if parent.isValid():
- return parent.internalPointer()
- else:
- return self
-
- def rowCount(self, parent):
- return self.child_count
-
- def headerData(self, section, orientation, role):
- if role == Qt.TextAlignmentRole:
- return self.columnAlignment(section)
- if role != Qt.DisplayRole:
- return None
- if orientation != Qt.Horizontal:
- return None
- return self.columnHeader(section)
-
- def index(self, row, column, parent):
- return self.createIndex(row, column, self.child_items[row])
-
- def DisplayData(self, item, index):
- return item.getData(index.column())
-
- def FetchIfNeeded(self, row):
- if row > self.last_row_read:
- self.last_row_read = row
- if row + 10 >= self.child_count:
- self.fetcher.Fetch(glb_chunk_sz)
-
- def columnAlignment(self, column):
- return Qt.AlignLeft
-
- def columnFont(self, column):
- return None
-
- def data(self, index, role):
- if role == Qt.TextAlignmentRole:
- return self.columnAlignment(index.column())
- if role == Qt.FontRole:
- return self.columnFont(index.column())
- if role != Qt.DisplayRole:
- return None
- item = index.internalPointer()
- return self.DisplayData(item, index)
-
-# Model cache
-
-model_cache = weakref.WeakValueDictionary()
-model_cache_lock = threading.Lock()
-
-def LookupCreateModel(model_name, create_fn):
- model_cache_lock.acquire()
- try:
- model = model_cache[model_name]
- except:
- model = None
- if model is None:
- model = create_fn()
- model_cache[model_name] = model
- model_cache_lock.release()
- return model
-
-def LookupModel(model_name):
- model_cache_lock.acquire()
- try:
- model = model_cache[model_name]
- except:
- model = None
- model_cache_lock.release()
- return model
-
-# Find bar
-
-class FindBar():
-
- def __init__(self, parent, finder, is_reg_expr=False):
- self.finder = finder
- self.context = []
- self.last_value = None
- self.last_pattern = None
-
- label = QLabel("Find:")
- label.setSizePolicy(QSizePolicy.Fixed, QSizePolicy.Fixed)
-
- self.textbox = QComboBox()
- self.textbox.setEditable(True)
- self.textbox.currentIndexChanged.connect(self.ValueChanged)
-
- self.progress = QProgressBar()
- self.progress.setRange(0, 0)
- self.progress.hide()
-
- if is_reg_expr:
- self.pattern = QCheckBox("Regular Expression")
- else:
- self.pattern = QCheckBox("Pattern")
- self.pattern.setSizePolicy(QSizePolicy.Fixed, QSizePolicy.Fixed)
-
- self.next_button = QToolButton()
- self.next_button.setIcon(parent.style().standardIcon(QStyle.SP_ArrowDown))
- self.next_button.released.connect(lambda: self.NextPrev(1))
-
- self.prev_button = QToolButton()
- self.prev_button.setIcon(parent.style().standardIcon(QStyle.SP_ArrowUp))
- self.prev_button.released.connect(lambda: self.NextPrev(-1))
-
- self.close_button = QToolButton()
- self.close_button.setIcon(parent.style().standardIcon(QStyle.SP_DockWidgetCloseButton))
- self.close_button.released.connect(self.Deactivate)
-
- self.hbox = QHBoxLayout()
- self.hbox.setContentsMargins(0, 0, 0, 0)
-
- self.hbox.addWidget(label)
- self.hbox.addWidget(self.textbox)
- self.hbox.addWidget(self.progress)
- self.hbox.addWidget(self.pattern)
- self.hbox.addWidget(self.next_button)
- self.hbox.addWidget(self.prev_button)
- self.hbox.addWidget(self.close_button)
-
- self.bar = QWidget()
- self.bar.setLayout(self.hbox)
- self.bar.hide()
-
- def Widget(self):
- return self.bar
-
- def Activate(self):
- self.bar.show()
- self.textbox.lineEdit().selectAll()
- self.textbox.setFocus()
-
- def Deactivate(self):
- self.bar.hide()
-
- def Busy(self):
- self.textbox.setEnabled(False)
- self.pattern.hide()
- self.next_button.hide()
- self.prev_button.hide()
- self.progress.show()
-
- def Idle(self):
- self.textbox.setEnabled(True)
- self.progress.hide()
- self.pattern.show()
- self.next_button.show()
- self.prev_button.show()
-
- def Find(self, direction):
- value = self.textbox.currentText()
- pattern = self.pattern.isChecked()
- self.last_value = value
- self.last_pattern = pattern
- self.finder.Find(value, direction, pattern, self.context)
-
- def ValueChanged(self):
- value = self.textbox.currentText()
- pattern = self.pattern.isChecked()
- index = self.textbox.currentIndex()
- data = self.textbox.itemData(index)
- # Store the pattern in the combo box to keep it with the text value
- if data == None:
- self.textbox.setItemData(index, pattern)
- else:
- self.pattern.setChecked(data)
- self.Find(0)
-
- def NextPrev(self, direction):
- value = self.textbox.currentText()
- pattern = self.pattern.isChecked()
- if value != self.last_value:
- index = self.textbox.findText(value)
- # Allow for a button press before the value has been added to the combo box
- if index < 0:
- index = self.textbox.count()
- self.textbox.addItem(value, pattern)
- self.textbox.setCurrentIndex(index)
- return
- else:
- self.textbox.setItemData(index, pattern)
- elif pattern != self.last_pattern:
- # Keep the pattern recorded in the combo box up to date
- index = self.textbox.currentIndex()
- self.textbox.setItemData(index, pattern)
- self.Find(direction)
-
- def NotFound(self):
- QMessageBox.information(self.bar, "Find", "'" + self.textbox.currentText() + "' not found")
-
-# Context-sensitive call graph data model item base
-
-class CallGraphLevelItemBase(object):
-
- def __init__(self, glb, params, row, parent_item):
- self.glb = glb
- self.params = params
- self.row = row
- self.parent_item = parent_item
- self.query_done = False
- self.child_count = 0
- self.child_items = []
- if parent_item:
- self.level = parent_item.level + 1
- else:
- self.level = 0
-
- def getChildItem(self, row):
- return self.child_items[row]
-
- def getParentItem(self):
- return self.parent_item
-
- def getRow(self):
- return self.row
-
- def childCount(self):
- if not self.query_done:
- self.Select()
- if not self.child_count:
- return -1
- return self.child_count
-
- def hasChildren(self):
- if not self.query_done:
- return True
- return self.child_count > 0
-
- def getData(self, column):
- return self.data[column]
-
-# Context-sensitive call graph data model level 2+ item base
-
-class CallGraphLevelTwoPlusItemBase(CallGraphLevelItemBase):
-
- def __init__(self, glb, params, row, comm_id, thread_id, call_path_id, time, insn_cnt, cyc_cnt, branch_count, parent_item):
- super(CallGraphLevelTwoPlusItemBase, self).__init__(glb, params, row, parent_item)
- self.comm_id = comm_id
- self.thread_id = thread_id
- self.call_path_id = call_path_id
- self.insn_cnt = insn_cnt
- self.cyc_cnt = cyc_cnt
- self.branch_count = branch_count
- self.time = time
-
- def Select(self):
- self.query_done = True
- query = QSqlQuery(self.glb.db)
- if self.params.have_ipc:
- ipc_str = ", SUM(insn_count), SUM(cyc_count)"
- else:
- ipc_str = ""
- QueryExec(query, "SELECT call_path_id, name, short_name, COUNT(calls.id), SUM(return_time - call_time)" + ipc_str + ", SUM(branch_count)"
- " FROM calls"
- " INNER JOIN call_paths ON calls.call_path_id = call_paths.id"
- " INNER JOIN symbols ON call_paths.symbol_id = symbols.id"
- " INNER JOIN dsos ON symbols.dso_id = dsos.id"
- " WHERE parent_call_path_id = " + str(self.call_path_id) +
- " AND comm_id = " + str(self.comm_id) +
- " AND thread_id = " + str(self.thread_id) +
- " GROUP BY call_path_id, name, short_name"
- " ORDER BY call_path_id")
- while query.next():
- if self.params.have_ipc:
- insn_cnt = int(query.value(5))
- cyc_cnt = int(query.value(6))
- branch_count = int(query.value(7))
- else:
- insn_cnt = 0
- cyc_cnt = 0
- branch_count = int(query.value(5))
- child_item = CallGraphLevelThreeItem(self.glb, self.params, self.child_count, self.comm_id, self.thread_id, query.value(0), query.value(1), query.value(2), query.value(3), int(query.value(4)), insn_cnt, cyc_cnt, branch_count, self)
- self.child_items.append(child_item)
- self.child_count += 1
-
-# Context-sensitive call graph data model level three item
-
-class CallGraphLevelThreeItem(CallGraphLevelTwoPlusItemBase):
-
- def __init__(self, glb, params, row, comm_id, thread_id, call_path_id, name, dso, count, time, insn_cnt, cyc_cnt, branch_count, parent_item):
- super(CallGraphLevelThreeItem, self).__init__(glb, params, row, comm_id, thread_id, call_path_id, time, insn_cnt, cyc_cnt, branch_count, parent_item)
- dso = dsoname(dso)
- if self.params.have_ipc:
- insn_pcnt = PercentToOneDP(insn_cnt, parent_item.insn_cnt)
- cyc_pcnt = PercentToOneDP(cyc_cnt, parent_item.cyc_cnt)
- br_pcnt = PercentToOneDP(branch_count, parent_item.branch_count)
- ipc = CalcIPC(cyc_cnt, insn_cnt)
- self.data = [ name, dso, str(count), str(time), PercentToOneDP(time, parent_item.time), str(insn_cnt), insn_pcnt, str(cyc_cnt), cyc_pcnt, ipc, str(branch_count), br_pcnt ]
- else:
- self.data = [ name, dso, str(count), str(time), PercentToOneDP(time, parent_item.time), str(branch_count), PercentToOneDP(branch_count, parent_item.branch_count) ]
- self.dbid = call_path_id
-
-# Context-sensitive call graph data model level two item
-
-class CallGraphLevelTwoItem(CallGraphLevelTwoPlusItemBase):
-
- def __init__(self, glb, params, row, comm_id, thread_id, pid, tid, parent_item):
- super(CallGraphLevelTwoItem, self).__init__(glb, params, row, comm_id, thread_id, 1, 0, 0, 0, 0, parent_item)
- if self.params.have_ipc:
- self.data = [str(pid) + ":" + str(tid), "", "", "", "", "", "", "", "", "", "", ""]
- else:
- self.data = [str(pid) + ":" + str(tid), "", "", "", "", "", ""]
- self.dbid = thread_id
-
- def Select(self):
- super(CallGraphLevelTwoItem, self).Select()
- for child_item in self.child_items:
- self.time += child_item.time
- self.insn_cnt += child_item.insn_cnt
- self.cyc_cnt += child_item.cyc_cnt
- self.branch_count += child_item.branch_count
- for child_item in self.child_items:
- child_item.data[4] = PercentToOneDP(child_item.time, self.time)
- if self.params.have_ipc:
- child_item.data[6] = PercentToOneDP(child_item.insn_cnt, self.insn_cnt)
- child_item.data[8] = PercentToOneDP(child_item.cyc_cnt, self.cyc_cnt)
- child_item.data[11] = PercentToOneDP(child_item.branch_count, self.branch_count)
- else:
- child_item.data[6] = PercentToOneDP(child_item.branch_count, self.branch_count)
-
-# Context-sensitive call graph data model level one item
-
-class CallGraphLevelOneItem(CallGraphLevelItemBase):
-
- def __init__(self, glb, params, row, comm_id, comm, parent_item):
- super(CallGraphLevelOneItem, self).__init__(glb, params, row, parent_item)
- if self.params.have_ipc:
- self.data = [comm, "", "", "", "", "", "", "", "", "", "", ""]
- else:
- self.data = [comm, "", "", "", "", "", ""]
- self.dbid = comm_id
-
- def Select(self):
- self.query_done = True
- query = QSqlQuery(self.glb.db)
- QueryExec(query, "SELECT thread_id, pid, tid"
- " FROM comm_threads"
- " INNER JOIN threads ON thread_id = threads.id"
- " WHERE comm_id = " + str(self.dbid))
- while query.next():
- child_item = CallGraphLevelTwoItem(self.glb, self.params, self.child_count, self.dbid, query.value(0), query.value(1), query.value(2), self)
- self.child_items.append(child_item)
- self.child_count += 1
-
-# Context-sensitive call graph data model root item
-
-class CallGraphRootItem(CallGraphLevelItemBase):
-
- def __init__(self, glb, params):
- super(CallGraphRootItem, self).__init__(glb, params, 0, None)
- self.dbid = 0
- self.query_done = True
- if_has_calls = ""
- if IsSelectable(glb.db, "comms", columns = "has_calls"):
- if_has_calls = " WHERE has_calls = " + glb.dbref.TRUE
- query = QSqlQuery(glb.db)
- QueryExec(query, "SELECT id, comm FROM comms" + if_has_calls)
- while query.next():
- if not query.value(0):
- continue
- child_item = CallGraphLevelOneItem(glb, params, self.child_count, query.value(0), query.value(1), self)
- self.child_items.append(child_item)
- self.child_count += 1
-
-# Call graph model parameters
-
-class CallGraphModelParams():
-
- def __init__(self, glb, parent=None):
- self.have_ipc = IsSelectable(glb.db, "calls", columns = "insn_count, cyc_count")
-
-# Context-sensitive call graph data model base
-
-class CallGraphModelBase(TreeModel):
-
- def __init__(self, glb, parent=None):
- super(CallGraphModelBase, self).__init__(glb, CallGraphModelParams(glb), parent)
-
- def FindSelect(self, value, pattern, query):
- if pattern:
- # postgresql and sqlite pattern patching differences:
- # postgresql LIKE is case sensitive but sqlite LIKE is not
- # postgresql LIKE allows % and _ to be escaped with \ but sqlite LIKE does not
- # postgresql supports ILIKE which is case insensitive
- # sqlite supports GLOB (text only) which uses * and ? and is case sensitive
- if not self.glb.dbref.is_sqlite3:
- # Escape % and _
- s = value.replace("%", "\\%")
- s = s.replace("_", "\\_")
- # Translate * and ? into SQL LIKE pattern characters % and _
- if sys.version_info[0] == 3:
- trans = str.maketrans("*?", "%_")
- else:
- trans = string.maketrans("*?", "%_")
- match = " LIKE '" + str(s).translate(trans) + "'"
- else:
- match = " GLOB '" + str(value) + "'"
- else:
- match = " = '" + str(value) + "'"
- self.DoFindSelect(query, match)
-
- def Found(self, query, found):
- if found:
- return self.FindPath(query)
- return []
-
- def FindValue(self, value, pattern, query, last_value, last_pattern):
- if last_value == value and pattern == last_pattern:
- found = query.first()
- else:
- self.FindSelect(value, pattern, query)
- found = query.next()
- return self.Found(query, found)
-
- def FindNext(self, query):
- found = query.next()
- if not found:
- found = query.first()
- return self.Found(query, found)
-
- def FindPrev(self, query):
- found = query.previous()
- if not found:
- found = query.last()
- return self.Found(query, found)
-
- def FindThread(self, c):
- if c.direction == 0 or c.value != c.last_value or c.pattern != c.last_pattern:
- ids = self.FindValue(c.value, c.pattern, c.query, c.last_value, c.last_pattern)
- elif c.direction > 0:
- ids = self.FindNext(c.query)
- else:
- ids = self.FindPrev(c.query)
- return (True, ids)
-
- def Find(self, value, direction, pattern, context, callback):
- class Context():
- def __init__(self, *x):
- self.value, self.direction, self.pattern, self.query, self.last_value, self.last_pattern = x
- def Update(self, *x):
- self.value, self.direction, self.pattern, self.last_value, self.last_pattern = x + (self.value, self.pattern)
- if len(context):
- context[0].Update(value, direction, pattern)
- else:
- context.append(Context(value, direction, pattern, QSqlQuery(self.glb.db), None, None))
- # Use a thread so the UI is not blocked during the SELECT
- thread = Thread(self.FindThread, context[0])
- thread.done.connect(lambda ids, t=thread, c=callback: self.FindDone(t, c, ids), Qt.QueuedConnection)
- thread.start()
-
- def FindDone(self, thread, callback, ids):
- callback(ids)
-
-# Context-sensitive call graph data model
-
-class CallGraphModel(CallGraphModelBase):
-
- def __init__(self, glb, parent=None):
- super(CallGraphModel, self).__init__(glb, parent)
-
- def GetRoot(self):
- return CallGraphRootItem(self.glb, self.params)
-
- def columnCount(self, parent=None):
- if self.params.have_ipc:
- return 12
- else:
- return 7
-
- def columnHeader(self, column):
- if self.params.have_ipc:
- headers = ["Call Path", "Object", "Count ", "Time (ns) ", "Time (%) ", "Insn Cnt", "Insn Cnt (%)", "Cyc Cnt", "Cyc Cnt (%)", "IPC", "Branch Count ", "Branch Count (%) "]
- else:
- headers = ["Call Path", "Object", "Count ", "Time (ns) ", "Time (%) ", "Branch Count ", "Branch Count (%) "]
- return headers[column]
-
- def columnAlignment(self, column):
- if self.params.have_ipc:
- alignment = [ Qt.AlignLeft, Qt.AlignLeft, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight ]
- else:
- alignment = [ Qt.AlignLeft, Qt.AlignLeft, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight ]
- return alignment[column]
-
- def DoFindSelect(self, query, match):
- QueryExec(query, "SELECT call_path_id, comm_id, thread_id"
- " FROM calls"
- " INNER JOIN call_paths ON calls.call_path_id = call_paths.id"
- " INNER JOIN symbols ON call_paths.symbol_id = symbols.id"
- " WHERE calls.id <> 0"
- " AND symbols.name" + match +
- " GROUP BY comm_id, thread_id, call_path_id"
- " ORDER BY comm_id, thread_id, call_path_id")
-
- def FindPath(self, query):
- # Turn the query result into a list of ids that the tree view can walk
- # to open the tree at the right place.
- ids = []
- parent_id = query.value(0)
- while parent_id:
- ids.insert(0, parent_id)
- q2 = QSqlQuery(self.glb.db)
- QueryExec(q2, "SELECT parent_id"
- " FROM call_paths"
- " WHERE id = " + str(parent_id))
- if not q2.next():
- break
- parent_id = q2.value(0)
- # The call path root is not used
- if ids[0] == 1:
- del ids[0]
- ids.insert(0, query.value(2))
- ids.insert(0, query.value(1))
- return ids
-
-# Call tree data model level 2+ item base
-
-class CallTreeLevelTwoPlusItemBase(CallGraphLevelItemBase):
-
- def __init__(self, glb, params, row, comm_id, thread_id, calls_id, call_time, time, insn_cnt, cyc_cnt, branch_count, parent_item):
- super(CallTreeLevelTwoPlusItemBase, self).__init__(glb, params, row, parent_item)
- self.comm_id = comm_id
- self.thread_id = thread_id
- self.calls_id = calls_id
- self.call_time = call_time
- self.time = time
- self.insn_cnt = insn_cnt
- self.cyc_cnt = cyc_cnt
- self.branch_count = branch_count
-
- def Select(self):
- self.query_done = True
- if self.calls_id == 0:
- comm_thread = " AND comm_id = " + str(self.comm_id) + " AND thread_id = " + str(self.thread_id)
- else:
- comm_thread = ""
- if self.params.have_ipc:
- ipc_str = ", insn_count, cyc_count"
- else:
- ipc_str = ""
- query = QSqlQuery(self.glb.db)
- QueryExec(query, "SELECT calls.id, name, short_name, call_time, return_time - call_time" + ipc_str + ", branch_count"
- " FROM calls"
- " INNER JOIN call_paths ON calls.call_path_id = call_paths.id"
- " INNER JOIN symbols ON call_paths.symbol_id = symbols.id"
- " INNER JOIN dsos ON symbols.dso_id = dsos.id"
- " WHERE calls.parent_id = " + str(self.calls_id) + comm_thread +
- " ORDER BY call_time, calls.id")
- while query.next():
- if self.params.have_ipc:
- insn_cnt = int(query.value(5))
- cyc_cnt = int(query.value(6))
- branch_count = int(query.value(7))
- else:
- insn_cnt = 0
- cyc_cnt = 0
- branch_count = int(query.value(5))
- child_item = CallTreeLevelThreeItem(self.glb, self.params, self.child_count, self.comm_id, self.thread_id, query.value(0), query.value(1), query.value(2), query.value(3), int(query.value(4)), insn_cnt, cyc_cnt, branch_count, self)
- self.child_items.append(child_item)
- self.child_count += 1
-
-# Call tree data model level three item
-
-class CallTreeLevelThreeItem(CallTreeLevelTwoPlusItemBase):
-
- def __init__(self, glb, params, row, comm_id, thread_id, calls_id, name, dso, call_time, time, insn_cnt, cyc_cnt, branch_count, parent_item):
- super(CallTreeLevelThreeItem, self).__init__(glb, params, row, comm_id, thread_id, calls_id, call_time, time, insn_cnt, cyc_cnt, branch_count, parent_item)
- dso = dsoname(dso)
- if self.params.have_ipc:
- insn_pcnt = PercentToOneDP(insn_cnt, parent_item.insn_cnt)
- cyc_pcnt = PercentToOneDP(cyc_cnt, parent_item.cyc_cnt)
- br_pcnt = PercentToOneDP(branch_count, parent_item.branch_count)
- ipc = CalcIPC(cyc_cnt, insn_cnt)
- self.data = [ name, dso, str(call_time), str(time), PercentToOneDP(time, parent_item.time), str(insn_cnt), insn_pcnt, str(cyc_cnt), cyc_pcnt, ipc, str(branch_count), br_pcnt ]
- else:
- self.data = [ name, dso, str(call_time), str(time), PercentToOneDP(time, parent_item.time), str(branch_count), PercentToOneDP(branch_count, parent_item.branch_count) ]
- self.dbid = calls_id
-
-# Call tree data model level two item
-
-class CallTreeLevelTwoItem(CallTreeLevelTwoPlusItemBase):
-
- def __init__(self, glb, params, row, comm_id, thread_id, pid, tid, parent_item):
- super(CallTreeLevelTwoItem, self).__init__(glb, params, row, comm_id, thread_id, 0, 0, 0, 0, 0, 0, parent_item)
- if self.params.have_ipc:
- self.data = [str(pid) + ":" + str(tid), "", "", "", "", "", "", "", "", "", "", ""]
- else:
- self.data = [str(pid) + ":" + str(tid), "", "", "", "", "", ""]
- self.dbid = thread_id
-
- def Select(self):
- super(CallTreeLevelTwoItem, self).Select()
- for child_item in self.child_items:
- self.time += child_item.time
- self.insn_cnt += child_item.insn_cnt
- self.cyc_cnt += child_item.cyc_cnt
- self.branch_count += child_item.branch_count
- for child_item in self.child_items:
- child_item.data[4] = PercentToOneDP(child_item.time, self.time)
- if self.params.have_ipc:
- child_item.data[6] = PercentToOneDP(child_item.insn_cnt, self.insn_cnt)
- child_item.data[8] = PercentToOneDP(child_item.cyc_cnt, self.cyc_cnt)
- child_item.data[11] = PercentToOneDP(child_item.branch_count, self.branch_count)
- else:
- child_item.data[6] = PercentToOneDP(child_item.branch_count, self.branch_count)
-
-# Call tree data model level one item
-
-class CallTreeLevelOneItem(CallGraphLevelItemBase):
-
- def __init__(self, glb, params, row, comm_id, comm, parent_item):
- super(CallTreeLevelOneItem, self).__init__(glb, params, row, parent_item)
- if self.params.have_ipc:
- self.data = [comm, "", "", "", "", "", "", "", "", "", "", ""]
- else:
- self.data = [comm, "", "", "", "", "", ""]
- self.dbid = comm_id
-
- def Select(self):
- self.query_done = True
- query = QSqlQuery(self.glb.db)
- QueryExec(query, "SELECT thread_id, pid, tid"
- " FROM comm_threads"
- " INNER JOIN threads ON thread_id = threads.id"
- " WHERE comm_id = " + str(self.dbid))
- while query.next():
- child_item = CallTreeLevelTwoItem(self.glb, self.params, self.child_count, self.dbid, query.value(0), query.value(1), query.value(2), self)
- self.child_items.append(child_item)
- self.child_count += 1
-
-# Call tree data model root item
-
-class CallTreeRootItem(CallGraphLevelItemBase):
-
- def __init__(self, glb, params):
- super(CallTreeRootItem, self).__init__(glb, params, 0, None)
- self.dbid = 0
- self.query_done = True
- if_has_calls = ""
- if IsSelectable(glb.db, "comms", columns = "has_calls"):
- if_has_calls = " WHERE has_calls = " + glb.dbref.TRUE
- query = QSqlQuery(glb.db)
- QueryExec(query, "SELECT id, comm FROM comms" + if_has_calls)
- while query.next():
- if not query.value(0):
- continue
- child_item = CallTreeLevelOneItem(glb, params, self.child_count, query.value(0), query.value(1), self)
- self.child_items.append(child_item)
- self.child_count += 1
-
-# Call Tree data model
-
-class CallTreeModel(CallGraphModelBase):
-
- def __init__(self, glb, parent=None):
- super(CallTreeModel, self).__init__(glb, parent)
-
- def GetRoot(self):
- return CallTreeRootItem(self.glb, self.params)
-
- def columnCount(self, parent=None):
- if self.params.have_ipc:
- return 12
- else:
- return 7
-
- def columnHeader(self, column):
- if self.params.have_ipc:
- headers = ["Call Path", "Object", "Call Time", "Time (ns) ", "Time (%) ", "Insn Cnt", "Insn Cnt (%)", "Cyc Cnt", "Cyc Cnt (%)", "IPC", "Branch Count ", "Branch Count (%) "]
- else:
- headers = ["Call Path", "Object", "Call Time", "Time (ns) ", "Time (%) ", "Branch Count ", "Branch Count (%) "]
- return headers[column]
-
- def columnAlignment(self, column):
- if self.params.have_ipc:
- alignment = [ Qt.AlignLeft, Qt.AlignLeft, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight ]
- else:
- alignment = [ Qt.AlignLeft, Qt.AlignLeft, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight, Qt.AlignRight ]
- return alignment[column]
-
- def DoFindSelect(self, query, match):
- QueryExec(query, "SELECT calls.id, comm_id, thread_id"
- " FROM calls"
- " INNER JOIN call_paths ON calls.call_path_id = call_paths.id"
- " INNER JOIN symbols ON call_paths.symbol_id = symbols.id"
- " WHERE calls.id <> 0"
- " AND symbols.name" + match +
- " ORDER BY comm_id, thread_id, call_time, calls.id")
-
- def FindPath(self, query):
- # Turn the query result into a list of ids that the tree view can walk
- # to open the tree at the right place.
- ids = []
- parent_id = query.value(0)
- while parent_id:
- ids.insert(0, parent_id)
- q2 = QSqlQuery(self.glb.db)
- QueryExec(q2, "SELECT parent_id"
- " FROM calls"
- " WHERE id = " + str(parent_id))
- if not q2.next():
- break
- parent_id = q2.value(0)
- ids.insert(0, query.value(2))
- ids.insert(0, query.value(1))
- return ids
-
-# Vertical layout
-
-class HBoxLayout(QHBoxLayout):
-
- def __init__(self, *children):
- super(HBoxLayout, self).__init__()
-
- self.layout().setContentsMargins(0, 0, 0, 0)
- for child in children:
- if child.isWidgetType():
- self.layout().addWidget(child)
- else:
- self.layout().addLayout(child)
-
-# Horizontal layout
-
-class VBoxLayout(QVBoxLayout):
-
- def __init__(self, *children):
- super(VBoxLayout, self).__init__()
-
- self.layout().setContentsMargins(0, 0, 0, 0)
- for child in children:
- if child.isWidgetType():
- self.layout().addWidget(child)
- else:
- self.layout().addLayout(child)
-
-# Vertical layout widget
-
-class VBox():
-
- def __init__(self, *children):
- self.vbox = QWidget()
- self.vbox.setLayout(VBoxLayout(*children))
-
- def Widget(self):
- return self.vbox
-
-# Tree window base
-
-class TreeWindowBase(QMdiSubWindow):
-
- def __init__(self, parent=None):
- super(TreeWindowBase, self).__init__(parent)
-
- self.model = None
- self.find_bar = None
-
- self.view = QTreeView()
- self.view.setSelectionMode(QAbstractItemView.ContiguousSelection)
- self.view.CopyCellsToClipboard = CopyTreeCellsToClipboard
-
- self.context_menu = TreeContextMenu(self.view)
-
- def DisplayFound(self, ids):
- if not len(ids):
- return False
- parent = QModelIndex()
- for dbid in ids:
- found = False
- n = self.model.rowCount(parent)
- for row in xrange(n):
- child = self.model.index(row, 0, parent)
- if child.internalPointer().dbid == dbid:
- found = True
- self.view.setExpanded(parent, True)
- self.view.setCurrentIndex(child)
- parent = child
- break
- if not found:
- break
- return found
-
- def Find(self, value, direction, pattern, context):
- self.view.setFocus()
- self.find_bar.Busy()
- self.model.Find(value, direction, pattern, context, self.FindDone)
-
- def FindDone(self, ids):
- found = True
- if not self.DisplayFound(ids):
- found = False
- self.find_bar.Idle()
- if not found:
- self.find_bar.NotFound()
-
-
-# Context-sensitive call graph window
-
-class CallGraphWindow(TreeWindowBase):
-
- def __init__(self, glb, parent=None):
- super(CallGraphWindow, self).__init__(parent)
-
- self.model = LookupCreateModel("Context-Sensitive Call Graph", lambda x=glb: CallGraphModel(x))
-
- self.view.setModel(self.model)
-
- for c, w in ((0, 250), (1, 100), (2, 60), (3, 70), (4, 70), (5, 100)):
- self.view.setColumnWidth(c, w)
-
- self.find_bar = FindBar(self, self)
-
- self.vbox = VBox(self.view, self.find_bar.Widget())
-
- self.setWidget(self.vbox.Widget())
-
- AddSubWindow(glb.mainwindow.mdi_area, self, "Context-Sensitive Call Graph")
-
-# Call tree window
-
-class CallTreeWindow(TreeWindowBase):
-
- def __init__(self, glb, parent=None, thread_at_time=None):
- super(CallTreeWindow, self).__init__(parent)
-
- self.model = LookupCreateModel("Call Tree", lambda x=glb: CallTreeModel(x))
-
- self.view.setModel(self.model)
-
- for c, w in ((0, 230), (1, 100), (2, 100), (3, 70), (4, 70), (5, 100)):
- self.view.setColumnWidth(c, w)
-
- self.find_bar = FindBar(self, self)
-
- self.vbox = VBox(self.view, self.find_bar.Widget())
-
- self.setWidget(self.vbox.Widget())
-
- AddSubWindow(glb.mainwindow.mdi_area, self, "Call Tree")
-
- if thread_at_time:
- self.DisplayThreadAtTime(*thread_at_time)
-
- def DisplayThreadAtTime(self, comm_id, thread_id, time):
- parent = QModelIndex()
- for dbid in (comm_id, thread_id):
- found = False
- n = self.model.rowCount(parent)
- for row in xrange(n):
- child = self.model.index(row, 0, parent)
- if child.internalPointer().dbid == dbid:
- found = True
- self.view.setExpanded(parent, True)
- self.view.setCurrentIndex(child)
- parent = child
- break
- if not found:
- return
- found = False
- while True:
- n = self.model.rowCount(parent)
- if not n:
- return
- last_child = None
- for row in xrange(n):
- self.view.setExpanded(parent, True)
- child = self.model.index(row, 0, parent)
- child_call_time = child.internalPointer().call_time
- if child_call_time < time:
- last_child = child
- elif child_call_time == time:
- self.view.setCurrentIndex(child)
- return
- elif child_call_time > time:
- break
- if not last_child:
- if not found:
- child = self.model.index(0, 0, parent)
- self.view.setExpanded(parent, True)
- self.view.setCurrentIndex(child)
- return
- found = True
- self.view.setExpanded(parent, True)
- self.view.setCurrentIndex(last_child)
- parent = last_child
-
-# ExecComm() gets the comm_id of the command string that was set when the process exec'd i.e. the program name
-
-def ExecComm(db, thread_id, time):
- query = QSqlQuery(db)
- QueryExec(query, "SELECT comm_threads.comm_id, comms.c_time, comms.exec_flag"
- " FROM comm_threads"
- " INNER JOIN comms ON comms.id = comm_threads.comm_id"
- " WHERE comm_threads.thread_id = " + str(thread_id) +
- " ORDER BY comms.c_time, comms.id")
- first = None
- last = None
- while query.next():
- if first is None:
- first = query.value(0)
- if query.value(2) and Decimal(query.value(1)) <= Decimal(time):
- last = query.value(0)
- if not(last is None):
- return last
- return first
-
-# Container for (x, y) data
-
-class XY():
- def __init__(self, x=0, y=0):
- self.x = x
- self.y = y
-
- def __str__(self):
- return "XY({}, {})".format(str(self.x), str(self.y))
-
-# Container for sub-range data
-
-class Subrange():
- def __init__(self, lo=0, hi=0):
- self.lo = lo
- self.hi = hi
-
- def __str__(self):
- return "Subrange({}, {})".format(str(self.lo), str(self.hi))
-
-# Graph data region base class
-
-class GraphDataRegion(object):
-
- def __init__(self, key, title = "", ordinal = ""):
- self.key = key
- self.title = title
- self.ordinal = ordinal
-
-# Function to sort GraphDataRegion
-
-def GraphDataRegionOrdinal(data_region):
- return data_region.ordinal
-
-# Attributes for a graph region
-
-class GraphRegionAttribute():
-
- def __init__(self, colour):
- self.colour = colour
-
-# Switch graph data region represents a task
-
-class SwitchGraphDataRegion(GraphDataRegion):
-
- def __init__(self, key, exec_comm_id, pid, tid, comm, thread_id, comm_id):
- super(SwitchGraphDataRegion, self).__init__(key)
-
- self.title = str(pid) + " / " + str(tid) + " " + comm
- # Order graph legend within exec comm by pid / tid / time
- self.ordinal = str(pid).rjust(16) + str(exec_comm_id).rjust(8) + str(tid).rjust(16)
- self.exec_comm_id = exec_comm_id
- self.pid = pid
- self.tid = tid
- self.comm = comm
- self.thread_id = thread_id
- self.comm_id = comm_id
-
-# Graph data point
-
-class GraphDataPoint():
-
- def __init__(self, data, index, x, y, altx=None, alty=None, hregion=None, vregion=None):
- self.data = data
- self.index = index
- self.x = x
- self.y = y
- self.altx = altx
- self.alty = alty
- self.hregion = hregion
- self.vregion = vregion
-
-# Graph data (single graph) base class
-
-class GraphData(object):
-
- def __init__(self, collection, xbase=Decimal(0), ybase=Decimal(0)):
- self.collection = collection
- self.points = []
- self.xbase = xbase
- self.ybase = ybase
- self.title = ""
-
- def AddPoint(self, x, y, altx=None, alty=None, hregion=None, vregion=None):
- index = len(self.points)
-
- x = float(Decimal(x) - self.xbase)
- y = float(Decimal(y) - self.ybase)
-
- self.points.append(GraphDataPoint(self, index, x, y, altx, alty, hregion, vregion))
-
- def XToData(self, x):
- return Decimal(x) + self.xbase
-
- def YToData(self, y):
- return Decimal(y) + self.ybase
-
-# Switch graph data (for one CPU)
-
-class SwitchGraphData(GraphData):
-
- def __init__(self, db, collection, cpu, xbase):
- super(SwitchGraphData, self).__init__(collection, xbase)
-
- self.cpu = cpu
- self.title = "CPU " + str(cpu)
- self.SelectSwitches(db)
-
- def SelectComms(self, db, thread_id, last_comm_id, start_time, end_time):
- query = QSqlQuery(db)
- QueryExec(query, "SELECT id, c_time"
- " FROM comms"
- " WHERE c_thread_id = " + str(thread_id) +
- " AND exec_flag = " + self.collection.glb.dbref.TRUE +
- " AND c_time >= " + str(start_time) +
- " AND c_time <= " + str(end_time) +
- " ORDER BY c_time, id")
- while query.next():
- comm_id = query.value(0)
- if comm_id == last_comm_id:
- continue
- time = query.value(1)
- hregion = self.HRegion(db, thread_id, comm_id, time)
- self.AddPoint(time, 1000, None, None, hregion)
-
- def SelectSwitches(self, db):
- last_time = None
- last_comm_id = None
- last_thread_id = None
- query = QSqlQuery(db)
- QueryExec(query, "SELECT time, thread_out_id, thread_in_id, comm_out_id, comm_in_id, flags"
- " FROM context_switches"
- " WHERE machine_id = " + str(self.collection.machine_id) +
- " AND cpu = " + str(self.cpu) +
- " ORDER BY time, id")
- while query.next():
- flags = int(query.value(5))
- if flags & 1:
- # Schedule-out: detect and add exec's
- if last_thread_id == query.value(1) and last_comm_id is not None and last_comm_id != query.value(3):
- self.SelectComms(db, last_thread_id, last_comm_id, last_time, query.value(0))
- continue
- # Schedule-in: add data point
- if len(self.points) == 0:
- start_time = self.collection.glb.StartTime(self.collection.machine_id)
- hregion = self.HRegion(db, query.value(1), query.value(3), start_time)
- self.AddPoint(start_time, 1000, None, None, hregion)
- time = query.value(0)
- comm_id = query.value(4)
- thread_id = query.value(2)
- hregion = self.HRegion(db, thread_id, comm_id, time)
- self.AddPoint(time, 1000, None, None, hregion)
- last_time = time
- last_comm_id = comm_id
- last_thread_id = thread_id
-
- def NewHRegion(self, db, key, thread_id, comm_id, time):
- exec_comm_id = ExecComm(db, thread_id, time)
- query = QSqlQuery(db)
- QueryExec(query, "SELECT pid, tid FROM threads WHERE id = " + str(thread_id))
- if query.next():
- pid = query.value(0)
- tid = query.value(1)
- else:
- pid = -1
- tid = -1
- query = QSqlQuery(db)
- QueryExec(query, "SELECT comm FROM comms WHERE id = " + str(comm_id))
- if query.next():
- comm = query.value(0)
- else:
- comm = ""
- return SwitchGraphDataRegion(key, exec_comm_id, pid, tid, comm, thread_id, comm_id)
-
- def HRegion(self, db, thread_id, comm_id, time):
- key = str(thread_id) + ":" + str(comm_id)
- hregion = self.collection.LookupHRegion(key)
- if hregion is None:
- hregion = self.NewHRegion(db, key, thread_id, comm_id, time)
- self.collection.AddHRegion(key, hregion)
- return hregion
-
-# Graph data collection (multiple related graphs) base class
-
-class GraphDataCollection(object):
-
- def __init__(self, glb):
- self.glb = glb
- self.data = []
- self.hregions = {}
- self.xrangelo = None
- self.xrangehi = None
- self.yrangelo = None
- self.yrangehi = None
- self.dp = XY(0, 0)
-
- def AddGraphData(self, data):
- self.data.append(data)
-
- def LookupHRegion(self, key):
- if key in self.hregions:
- return self.hregions[key]
- return None
-
- def AddHRegion(self, key, hregion):
- self.hregions[key] = hregion
-
-# Switch graph data collection (SwitchGraphData for each CPU)
-
-class SwitchGraphDataCollection(GraphDataCollection):
-
- def __init__(self, glb, db, machine_id):
- super(SwitchGraphDataCollection, self).__init__(glb)
-
- self.machine_id = machine_id
- self.cpus = self.SelectCPUs(db)
-
- self.xrangelo = glb.StartTime(machine_id)
- self.xrangehi = glb.FinishTime(machine_id)
-
- self.yrangelo = Decimal(0)
- self.yrangehi = Decimal(1000)
-
- for cpu in self.cpus:
- self.AddGraphData(SwitchGraphData(db, self, cpu, self.xrangelo))
-
- def SelectCPUs(self, db):
- cpus = []
- query = QSqlQuery(db)
- QueryExec(query, "SELECT DISTINCT cpu"
- " FROM context_switches"
- " WHERE machine_id = " + str(self.machine_id))
- while query.next():
- cpus.append(int(query.value(0)))
- return sorted(cpus)
-
-# Switch graph data graphics item displays the graphed data
-
-class SwitchGraphDataGraphicsItem(QGraphicsItem):
-
- def __init__(self, data, graph_width, graph_height, attrs, event_handler, parent=None):
- super(SwitchGraphDataGraphicsItem, self).__init__(parent)
-
- self.data = data
- self.graph_width = graph_width
- self.graph_height = graph_height
- self.attrs = attrs
- self.event_handler = event_handler
- self.setAcceptHoverEvents(True)
-
- def boundingRect(self):
- return QRectF(0, 0, self.graph_width, self.graph_height)
-
- def PaintPoint(self, painter, last, x):
- if not(last is None or last.hregion.pid == 0 or x < self.attrs.subrange.x.lo):
- if last.x < self.attrs.subrange.x.lo:
- x0 = self.attrs.subrange.x.lo
- else:
- x0 = last.x
- if x > self.attrs.subrange.x.hi:
- x1 = self.attrs.subrange.x.hi
- else:
- x1 = x - 1
- x0 = self.attrs.XToPixel(x0)
- x1 = self.attrs.XToPixel(x1)
-
- y0 = self.attrs.YToPixel(last.y)
-
- colour = self.attrs.region_attributes[last.hregion.key].colour
-
- width = x1 - x0 + 1
- if width < 2:
- painter.setPen(colour)
- painter.drawLine(x0, self.graph_height - y0, x0, self.graph_height)
- else:
- painter.fillRect(x0, self.graph_height - y0, width, self.graph_height - 1, colour)
-
- def paint(self, painter, option, widget):
- last = None
- for point in self.data.points:
- self.PaintPoint(painter, last, point.x)
- if point.x > self.attrs.subrange.x.hi:
- break;
- last = point
- self.PaintPoint(painter, last, self.attrs.subrange.x.hi + 1)
-
- def BinarySearchPoint(self, target):
- lower_pos = 0
- higher_pos = len(self.data.points)
- while True:
- pos = int((lower_pos + higher_pos) / 2)
- val = self.data.points[pos].x
- if target >= val:
- lower_pos = pos
- else:
- higher_pos = pos
- if higher_pos <= lower_pos + 1:
- return lower_pos
-
- def XPixelToData(self, x):
- x = self.attrs.PixelToX(x)
- if x < self.data.points[0].x:
- x = 0
- pos = 0
- low = True
- else:
- pos = self.BinarySearchPoint(x)
- low = False
- return (low, pos, self.data.XToData(x))
-
- def EventToData(self, event):
- no_data = (None,) * 4
- if len(self.data.points) < 1:
- return no_data
- x = event.pos().x()
- if x < 0:
- return no_data
- low0, pos0, time_from = self.XPixelToData(x)
- low1, pos1, time_to = self.XPixelToData(x + 1)
- hregions = set()
- hregion_times = []
- if not low1:
- for i in xrange(pos0, pos1 + 1):
- hregion = self.data.points[i].hregion
- hregions.add(hregion)
- if i == pos0:
- time = time_from
- else:
- time = self.data.XToData(self.data.points[i].x)
- hregion_times.append((hregion, time))
- return (time_from, time_to, hregions, hregion_times)
-
- def hoverMoveEvent(self, event):
- time_from, time_to, hregions, hregion_times = self.EventToData(event)
- if time_from is not None:
- self.event_handler.PointEvent(self.data.cpu, time_from, time_to, hregions)
-
- def hoverLeaveEvent(self, event):
- self.event_handler.NoPointEvent()
-
- def mousePressEvent(self, event):
- if event.button() != Qt.RightButton:
- super(SwitchGraphDataGraphicsItem, self).mousePressEvent(event)
- return
- time_from, time_to, hregions, hregion_times = self.EventToData(event)
- if hregion_times:
- self.event_handler.RightClickEvent(self.data.cpu, hregion_times, event.screenPos())
-
-# X-axis graphics item
-
-class XAxisGraphicsItem(QGraphicsItem):
-
- def __init__(self, width, parent=None):
- super(XAxisGraphicsItem, self).__init__(parent)
-
- self.width = width
- self.max_mark_sz = 4
- self.height = self.max_mark_sz + 1
-
- def boundingRect(self):
- return QRectF(0, 0, self.width, self.height)
-
- def Step(self):
- attrs = self.parentItem().attrs
- subrange = attrs.subrange.x
- t = subrange.hi - subrange.lo
- s = (3.0 * t) / self.width
- n = 1.0
- while s > n:
- n = n * 10.0
- return n
-
- def PaintMarks(self, painter, at_y, lo, hi, step, i):
- attrs = self.parentItem().attrs
- x = lo
- while x <= hi:
- xp = attrs.XToPixel(x)
- if i % 10:
- if i % 5:
- sz = 1
- else:
- sz = 2
- else:
- sz = self.max_mark_sz
- i = 0
- painter.drawLine(xp, at_y, xp, at_y + sz)
- x += step
- i += 1
-
- def paint(self, painter, option, widget):
- # Using QPainter::drawLine(int x1, int y1, int x2, int y2) so x2 = width -1
- painter.drawLine(0, 0, self.width - 1, 0)
- n = self.Step()
- attrs = self.parentItem().attrs
- subrange = attrs.subrange.x
- if subrange.lo:
- x_offset = n - (subrange.lo % n)
- else:
- x_offset = 0.0
- x = subrange.lo + x_offset
- i = (x / n) % 10
- self.PaintMarks(painter, 0, x, subrange.hi, n, i)
-
- def ScaleDimensions(self):
- n = self.Step()
- attrs = self.parentItem().attrs
- lo = attrs.subrange.x.lo
- hi = (n * 10.0) + lo
- width = attrs.XToPixel(hi)
- if width > 500:
- width = 0
- return (n, lo, hi, width)
-
- def PaintScale(self, painter, at_x, at_y):
- n, lo, hi, width = self.ScaleDimensions()
- if not width:
- return
- painter.drawLine(at_x, at_y, at_x + width, at_y)
- self.PaintMarks(painter, at_y, lo, hi, n, 0)
-
- def ScaleWidth(self):
- n, lo, hi, width = self.ScaleDimensions()
- return width
-
- def ScaleHeight(self):
- return self.height
-
- def ScaleUnit(self):
- return self.Step() * 10
-
-# Scale graphics item base class
-
-class ScaleGraphicsItem(QGraphicsItem):
-
- def __init__(self, axis, parent=None):
- super(ScaleGraphicsItem, self).__init__(parent)
- self.axis = axis
-
- def boundingRect(self):
- scale_width = self.axis.ScaleWidth()
- if not scale_width:
- return QRectF()
- return QRectF(0, 0, self.axis.ScaleWidth() + 100, self.axis.ScaleHeight())
-
- def paint(self, painter, option, widget):
- scale_width = self.axis.ScaleWidth()
- if not scale_width:
- return
- self.axis.PaintScale(painter, 0, 5)
- x = scale_width + 4
- painter.drawText(QPointF(x, 10), self.Text())
-
- def Unit(self):
- return self.axis.ScaleUnit()
-
- def Text(self):
- return ""
-
-# Switch graph scale graphics item
-
-class SwitchScaleGraphicsItem(ScaleGraphicsItem):
-
- def __init__(self, axis, parent=None):
- super(SwitchScaleGraphicsItem, self).__init__(axis, parent)
-
- def Text(self):
- unit = self.Unit()
- if unit >= 1000000000:
- unit = int(unit / 1000000000)
- us = "s"
- elif unit >= 1000000:
- unit = int(unit / 1000000)
- us = "ms"
- elif unit >= 1000:
- unit = int(unit / 1000)
- us = "us"
- else:
- unit = int(unit)
- us = "ns"
- return " = " + str(unit) + " " + us
-
-# Switch graph graphics item contains graph title, scale, x/y-axis, and the graphed data
-
-class SwitchGraphGraphicsItem(QGraphicsItem):
-
- def __init__(self, collection, data, attrs, event_handler, first, parent=None):
- super(SwitchGraphGraphicsItem, self).__init__(parent)
- self.collection = collection
- self.data = data
- self.attrs = attrs
- self.event_handler = event_handler
-
- margin = 20
- title_width = 50
-
- self.title_graphics = QGraphicsSimpleTextItem(data.title, self)
-
- self.title_graphics.setPos(margin, margin)
- graph_width = attrs.XToPixel(attrs.subrange.x.hi) + 1
- graph_height = attrs.YToPixel(attrs.subrange.y.hi) + 1
-
- self.graph_origin_x = margin + title_width + margin
- self.graph_origin_y = graph_height + margin
-
- x_axis_size = 1
- y_axis_size = 1
- self.yline = QGraphicsLineItem(0, 0, 0, graph_height, self)
-
- self.x_axis = XAxisGraphicsItem(graph_width, self)
- self.x_axis.setPos(self.graph_origin_x, self.graph_origin_y + 1)
-
- if first:
- self.scale_item = SwitchScaleGraphicsItem(self.x_axis, self)
- self.scale_item.setPos(self.graph_origin_x, self.graph_origin_y + 10)
-
- self.yline.setPos(self.graph_origin_x - y_axis_size, self.graph_origin_y - graph_height)
-
- self.axis_point = QGraphicsLineItem(0, 0, 0, 0, self)
- self.axis_point.setPos(self.graph_origin_x - 1, self.graph_origin_y +1)
-
- self.width = self.graph_origin_x + graph_width + margin
- self.height = self.graph_origin_y + margin
-
- self.graph = SwitchGraphDataGraphicsItem(data, graph_width, graph_height, attrs, event_handler, self)
- self.graph.setPos(self.graph_origin_x, self.graph_origin_y - graph_height)
-
- if parent and 'EnableRubberBand' in dir(parent):
- parent.EnableRubberBand(self.graph_origin_x, self.graph_origin_x + graph_width - 1, self)
-
- def boundingRect(self):
- return QRectF(0, 0, self.width, self.height)
-
- def paint(self, painter, option, widget):
- pass
-
- def RBXToPixel(self, x):
- return self.attrs.PixelToX(x - self.graph_origin_x)
-
- def RBXRangeToPixel(self, x0, x1):
- return (self.RBXToPixel(x0), self.RBXToPixel(x1 + 1))
-
- def RBPixelToTime(self, x):
- if x < self.data.points[0].x:
- return self.data.XToData(0)
- return self.data.XToData(x)
-
- def RBEventTimes(self, x0, x1):
- x0, x1 = self.RBXRangeToPixel(x0, x1)
- time_from = self.RBPixelToTime(x0)
- time_to = self.RBPixelToTime(x1)
- return (time_from, time_to)
-
- def RBEvent(self, x0, x1):
- time_from, time_to = self.RBEventTimes(x0, x1)
- self.event_handler.RangeEvent(time_from, time_to)
-
- def RBMoveEvent(self, x0, x1):
- if x1 < x0:
- x0, x1 = x1, x0
- self.RBEvent(x0, x1)
-
- def RBReleaseEvent(self, x0, x1, selection_state):
- if x1 < x0:
- x0, x1 = x1, x0
- x0, x1 = self.RBXRangeToPixel(x0, x1)
- self.event_handler.SelectEvent(x0, x1, selection_state)
-
-# Graphics item to draw a vertical bracket (used to highlight "forward" sub-range)
-
-class VerticalBracketGraphicsItem(QGraphicsItem):
-
- def __init__(self, parent=None):
- super(VerticalBracketGraphicsItem, self).__init__(parent)
-
- self.width = 0
- self.height = 0
- self.hide()
-
- def SetSize(self, width, height):
- self.width = width + 1
- self.height = height + 1
-
- def boundingRect(self):
- return QRectF(0, 0, self.width, self.height)
-
- def paint(self, painter, option, widget):
- colour = QColor(255, 255, 0, 32)
- painter.fillRect(0, 0, self.width, self.height, colour)
- x1 = self.width - 1
- y1 = self.height - 1
- painter.drawLine(0, 0, x1, 0)
- painter.drawLine(0, 0, 0, 3)
- painter.drawLine(x1, 0, x1, 3)
- painter.drawLine(0, y1, x1, y1)
- painter.drawLine(0, y1, 0, y1 - 3)
- painter.drawLine(x1, y1, x1, y1 - 3)
-
-# Graphics item to contain graphs arranged vertically
-
-class VertcalGraphSetGraphicsItem(QGraphicsItem):
-
- def __init__(self, collection, attrs, event_handler, child_class, parent=None):
- super(VertcalGraphSetGraphicsItem, self).__init__(parent)
-
- self.collection = collection
-
- self.top = 10
-
- self.width = 0
- self.height = self.top
-
- self.rubber_band = None
- self.rb_enabled = False
-
- first = True
- for data in collection.data:
- child = child_class(collection, data, attrs, event_handler, first, self)
- child.setPos(0, self.height + 1)
- rect = child.boundingRect()
- if rect.right() > self.width:
- self.width = rect.right()
- self.height = self.height + rect.bottom() + 1
- first = False
-
- self.bracket = VerticalBracketGraphicsItem(self)
-
- def EnableRubberBand(self, xlo, xhi, rb_event_handler):
- if self.rb_enabled:
- return
- self.rb_enabled = True
- self.rb_in_view = False
- self.setAcceptedMouseButtons(Qt.LeftButton)
- self.rb_xlo = xlo
- self.rb_xhi = xhi
- self.rb_event_handler = rb_event_handler
- self.mousePressEvent = self.MousePressEvent
- self.mouseMoveEvent = self.MouseMoveEvent
- self.mouseReleaseEvent = self.MouseReleaseEvent
-
- def boundingRect(self):
- return QRectF(0, 0, self.width, self.height)
-
- def paint(self, painter, option, widget):
- pass
-
- def RubberBandParent(self):
- scene = self.scene()
- view = scene.views()[0]
- viewport = view.viewport()
- return viewport
-
- def RubberBandSetGeometry(self, rect):
- scene_rectf = self.mapRectToScene(QRectF(rect))
- scene = self.scene()
- view = scene.views()[0]
- poly = view.mapFromScene(scene_rectf)
- self.rubber_band.setGeometry(poly.boundingRect())
-
- def SetSelection(self, selection_state):
- if self.rubber_band:
- if selection_state:
- self.RubberBandSetGeometry(selection_state)
- self.rubber_band.show()
- else:
- self.rubber_band.hide()
-
- def SetBracket(self, rect):
- if rect:
- x, y, width, height = rect.x(), rect.y(), rect.width(), rect.height()
- self.bracket.setPos(x, y)
- self.bracket.SetSize(width, height)
- self.bracket.show()
- else:
- self.bracket.hide()
-
- def RubberBandX(self, event):
- x = event.pos().toPoint().x()
- if x < self.rb_xlo:
- x = self.rb_xlo
- elif x > self.rb_xhi:
- x = self.rb_xhi
- else:
- self.rb_in_view = True
- return x
-
- def RubberBandRect(self, x):
- if self.rb_origin.x() <= x:
- width = x - self.rb_origin.x()
- rect = QRect(self.rb_origin, QSize(width, self.height))
- else:
- width = self.rb_origin.x() - x
- top_left = QPoint(self.rb_origin.x() - width, self.rb_origin.y())
- rect = QRect(top_left, QSize(width, self.height))
- return rect
-
- def MousePressEvent(self, event):
- self.rb_in_view = False
- x = self.RubberBandX(event)
- self.rb_origin = QPoint(x, self.top)
- if self.rubber_band is None:
- self.rubber_band = QRubberBand(QRubberBand.Rectangle, self.RubberBandParent())
- self.RubberBandSetGeometry(QRect(self.rb_origin, QSize(0, self.height)))
- if self.rb_in_view:
- self.rubber_band.show()
- self.rb_event_handler.RBMoveEvent(x, x)
- else:
- self.rubber_band.hide()
-
- def MouseMoveEvent(self, event):
- x = self.RubberBandX(event)
- rect = self.RubberBandRect(x)
- self.RubberBandSetGeometry(rect)
- if self.rb_in_view:
- self.rubber_band.show()
- self.rb_event_handler.RBMoveEvent(self.rb_origin.x(), x)
-
- def MouseReleaseEvent(self, event):
- x = self.RubberBandX(event)
- if self.rb_in_view:
- selection_state = self.RubberBandRect(x)
- else:
- selection_state = None
- self.rb_event_handler.RBReleaseEvent(self.rb_origin.x(), x, selection_state)
-
-# Switch graph legend data model
-
-class SwitchGraphLegendModel(QAbstractTableModel):
-
- def __init__(self, collection, region_attributes, parent=None):
- super(SwitchGraphLegendModel, self).__init__(parent)
-
- self.region_attributes = region_attributes
-
- self.child_items = sorted(collection.hregions.values(), key=GraphDataRegionOrdinal)
- self.child_count = len(self.child_items)
-
- self.highlight_set = set()
-
- self.column_headers = ("pid", "tid", "comm")
-
- def rowCount(self, parent):
- return self.child_count
-
- def headerData(self, section, orientation, role):
- if role != Qt.DisplayRole:
- return None
- if orientation != Qt.Horizontal:
- return None
- return self.columnHeader(section)
-
- def index(self, row, column, parent):
- return self.createIndex(row, column, self.child_items[row])
-
- def columnCount(self, parent=None):
- return len(self.column_headers)
-
- def columnHeader(self, column):
- return self.column_headers[column]
-
- def data(self, index, role):
- if role == Qt.BackgroundRole:
- child = self.child_items[index.row()]
- if child in self.highlight_set:
- return self.region_attributes[child.key].colour
- return None
- if role == Qt.ForegroundRole:
- child = self.child_items[index.row()]
- if child in self.highlight_set:
- return QColor(255, 255, 255)
- return self.region_attributes[child.key].colour
- if role != Qt.DisplayRole:
- return None
- hregion = self.child_items[index.row()]
- col = index.column()
- if col == 0:
- return hregion.pid
- if col == 1:
- return hregion.tid
- if col == 2:
- return hregion.comm
- return None
-
- def SetHighlight(self, row, set_highlight):
- child = self.child_items[row]
- top_left = self.createIndex(row, 0, child)
- bottom_right = self.createIndex(row, len(self.column_headers) - 1, child)
- self.dataChanged.emit(top_left, bottom_right)
-
- def Highlight(self, highlight_set):
- for row in xrange(self.child_count):
- child = self.child_items[row]
- if child in self.highlight_set:
- if child not in highlight_set:
- self.SetHighlight(row, False)
- elif child in highlight_set:
- self.SetHighlight(row, True)
- self.highlight_set = highlight_set
-
-# Switch graph legend is a table
-
-class SwitchGraphLegend(QWidget):
-
- def __init__(self, collection, region_attributes, parent=None):
- super(SwitchGraphLegend, self).__init__(parent)
-
- self.data_model = SwitchGraphLegendModel(collection, region_attributes)
-
- self.model = QSortFilterProxyModel()
- self.model.setSourceModel(self.data_model)
-
- self.view = QTableView()
- self.view.setModel(self.model)
- self.view.setEditTriggers(QAbstractItemView.NoEditTriggers)
- self.view.verticalHeader().setVisible(False)
- self.view.sortByColumn(-1, Qt.AscendingOrder)
- self.view.setSortingEnabled(True)
- self.view.resizeColumnsToContents()
- self.view.resizeRowsToContents()
-
- self.vbox = VBoxLayout(self.view)
- self.setLayout(self.vbox)
-
- sz1 = self.view.columnWidth(0) + self.view.columnWidth(1) + self.view.columnWidth(2) + 2
- sz1 = sz1 + self.view.verticalScrollBar().sizeHint().width()
- self.saved_size = sz1
-
- def resizeEvent(self, event):
- self.saved_size = self.size().width()
- super(SwitchGraphLegend, self).resizeEvent(event)
-
- def Highlight(self, highlight_set):
- self.data_model.Highlight(highlight_set)
- self.update()
-
- def changeEvent(self, event):
- if event.type() == QEvent.FontChange:
- self.view.resizeRowsToContents()
- self.view.resizeColumnsToContents()
- # Need to resize rows again after column resize
- self.view.resizeRowsToContents()
- super(SwitchGraphLegend, self).changeEvent(event)
-
-# Random colour generation
-
-def RGBColourTooLight(r, g, b):
- if g > 230:
- return True
- if g <= 160:
- return False
- if r <= 180 and g <= 180:
- return False
- if r < 60:
- return False
- return True
-
-def GenerateColours(x):
- cs = [0]
- for i in xrange(1, x):
- cs.append(int((255.0 / i) + 0.5))
- colours = []
- for r in cs:
- for g in cs:
- for b in cs:
- # Exclude black and colours that look too light against a white background
- if (r, g, b) == (0, 0, 0) or RGBColourTooLight(r, g, b):
- continue
- colours.append(QColor(r, g, b))
- return colours
-
-def GenerateNColours(n):
- for x in xrange(2, n + 2):
- colours = GenerateColours(x)
- if len(colours) >= n:
- return colours
- return []
-
-def GenerateNRandomColours(n, seed):
- colours = GenerateNColours(n)
- random.seed(seed)
- random.shuffle(colours)
- return colours
-
-# Graph attributes, in particular the scale and subrange that change when zooming
-
-class GraphAttributes():
-
- def __init__(self, scale, subrange, region_attributes, dp):
- self.scale = scale
- self.subrange = subrange
- self.region_attributes = region_attributes
- # Rounding avoids errors due to finite floating point precision
- self.dp = dp # data decimal places
- self.Update()
-
- def XToPixel(self, x):
- return int(round((x - self.subrange.x.lo) * self.scale.x, self.pdp.x))
-
- def YToPixel(self, y):
- return int(round((y - self.subrange.y.lo) * self.scale.y, self.pdp.y))
-
- def PixelToXRounded(self, px):
- return round((round(px, 0) / self.scale.x), self.dp.x) + self.subrange.x.lo
-
- def PixelToYRounded(self, py):
- return round((round(py, 0) / self.scale.y), self.dp.y) + self.subrange.y.lo
-
- def PixelToX(self, px):
- x = self.PixelToXRounded(px)
- if self.pdp.x == 0:
- rt = self.XToPixel(x)
- if rt > px:
- return x - 1
- return x
-
- def PixelToY(self, py):
- y = self.PixelToYRounded(py)
- if self.pdp.y == 0:
- rt = self.YToPixel(y)
- if rt > py:
- return y - 1
- return y
-
- def ToPDP(self, dp, scale):
- # Calculate pixel decimal places:
- # (10 ** dp) is the minimum delta in the data
- # scale it to get the minimum delta in pixels
- # log10 gives the number of decimals places negatively
- # subtrace 1 to divide by 10
- # round to the lower negative number
- # change the sign to get the number of decimals positively
- x = math.log10((10 ** dp) * scale)
- if x < 0:
- x -= 1
- x = -int(math.floor(x) - 0.1)
- else:
- x = 0
- return x
-
- def Update(self):
- x = self.ToPDP(self.dp.x, self.scale.x)
- y = self.ToPDP(self.dp.y, self.scale.y)
- self.pdp = XY(x, y) # pixel decimal places
-
-# Switch graph splitter which divides the CPU graphs from the legend
-
-class SwitchGraphSplitter(QSplitter):
-
- def __init__(self, parent=None):
- super(SwitchGraphSplitter, self).__init__(parent)
-
- self.first_time = False
-
- def resizeEvent(self, ev):
- if self.first_time:
- self.first_time = False
- sz1 = self.widget(1).view.columnWidth(0) + self.widget(1).view.columnWidth(1) + self.widget(1).view.columnWidth(2) + 2
- sz1 = sz1 + self.widget(1).view.verticalScrollBar().sizeHint().width()
- sz0 = self.size().width() - self.handleWidth() - sz1
- self.setSizes([sz0, sz1])
- elif not(self.widget(1).saved_size is None):
- sz1 = self.widget(1).saved_size
- sz0 = self.size().width() - self.handleWidth() - sz1
- self.setSizes([sz0, sz1])
- super(SwitchGraphSplitter, self).resizeEvent(ev)
-
-# Graph widget base class
-
-class GraphWidget(QWidget):
-
- graph_title_changed = Signal(object)
-
- def __init__(self, parent=None):
- super(GraphWidget, self).__init__(parent)
-
- def GraphTitleChanged(self, title):
- self.graph_title_changed.emit(title)
-
- def Title(self):
- return ""
-
-# Display time in s, ms, us or ns
-
-def ToTimeStr(val):
- val = Decimal(val)
- if val >= 1000000000:
- return "{} s".format((val / 1000000000).quantize(Decimal("0.000000001")))
- if val >= 1000000:
- return "{} ms".format((val / 1000000).quantize(Decimal("0.000001")))
- if val >= 1000:
- return "{} us".format((val / 1000).quantize(Decimal("0.001")))
- return "{} ns".format(val.quantize(Decimal("1")))
-
-# Switch (i.e. context switch i.e. Time Chart by CPU) graph widget which contains the CPU graphs and the legend and control buttons
-
-class SwitchGraphWidget(GraphWidget):
-
- def __init__(self, glb, collection, parent=None):
- super(SwitchGraphWidget, self).__init__(parent)
-
- self.glb = glb
- self.collection = collection
-
- self.back_state = []
- self.forward_state = []
- self.selection_state = (None, None)
- self.fwd_rect = None
- self.start_time = self.glb.StartTime(collection.machine_id)
-
- i = 0
- hregions = collection.hregions.values()
- colours = GenerateNRandomColours(len(hregions), 1013)
- region_attributes = {}
- for hregion in hregions:
- if hregion.pid == 0 and hregion.tid == 0:
- region_attributes[hregion.key] = GraphRegionAttribute(QColor(0, 0, 0))
- else:
- region_attributes[hregion.key] = GraphRegionAttribute(colours[i])
- i = i + 1
-
- # Default to entire range
- xsubrange = Subrange(0.0, float(collection.xrangehi - collection.xrangelo) + 1.0)
- ysubrange = Subrange(0.0, float(collection.yrangehi - collection.yrangelo) + 1.0)
- subrange = XY(xsubrange, ysubrange)
-
- scale = self.GetScaleForRange(subrange)
-
- self.attrs = GraphAttributes(scale, subrange, region_attributes, collection.dp)
-
- self.item = VertcalGraphSetGraphicsItem(collection, self.attrs, self, SwitchGraphGraphicsItem)
-
- self.scene = QGraphicsScene()
- self.scene.addItem(self.item)
-
- self.view = QGraphicsView(self.scene)
- self.view.centerOn(0, 0)
- self.view.setAlignment(Qt.AlignLeft | Qt.AlignTop)
-
- self.legend = SwitchGraphLegend(collection, region_attributes)
-
- self.splitter = SwitchGraphSplitter()
- self.splitter.addWidget(self.view)
- self.splitter.addWidget(self.legend)
-
- self.point_label = QLabel("")
- self.point_label.setSizePolicy(QSizePolicy.Preferred, QSizePolicy.Fixed)
-
- self.back_button = QToolButton()
- self.back_button.setIcon(self.style().standardIcon(QStyle.SP_ArrowLeft))
- self.back_button.setDisabled(True)
- self.back_button.released.connect(lambda: self.Back())
-
- self.forward_button = QToolButton()
- self.forward_button.setIcon(self.style().standardIcon(QStyle.SP_ArrowRight))
- self.forward_button.setDisabled(True)
- self.forward_button.released.connect(lambda: self.Forward())
-
- self.zoom_button = QToolButton()
- self.zoom_button.setText("Zoom")
- self.zoom_button.setDisabled(True)
- self.zoom_button.released.connect(lambda: self.Zoom())
-
- self.hbox = HBoxLayout(self.back_button, self.forward_button, self.zoom_button, self.point_label)
-
- self.vbox = VBoxLayout(self.splitter, self.hbox)
-
- self.setLayout(self.vbox)
-
- def GetScaleForRangeX(self, xsubrange):
- # Default graph 1000 pixels wide
- dflt = 1000.0
- r = xsubrange.hi - xsubrange.lo
- return dflt / r
-
- def GetScaleForRangeY(self, ysubrange):
- # Default graph 50 pixels high
- dflt = 50.0
- r = ysubrange.hi - ysubrange.lo
- return dflt / r
-
- def GetScaleForRange(self, subrange):
- # Default graph 1000 pixels wide, 50 pixels high
- xscale = self.GetScaleForRangeX(subrange.x)
- yscale = self.GetScaleForRangeY(subrange.y)
- return XY(xscale, yscale)
-
- def PointEvent(self, cpu, time_from, time_to, hregions):
- text = "CPU: " + str(cpu)
- time_from = time_from.quantize(Decimal(1))
- rel_time_from = time_from - self.glb.StartTime(self.collection.machine_id)
- text = text + " Time: " + str(time_from) + " (+" + ToTimeStr(rel_time_from) + ")"
- self.point_label.setText(text)
- self.legend.Highlight(hregions)
-
- def RightClickEvent(self, cpu, hregion_times, pos):
- if not IsSelectable(self.glb.db, "calls", "WHERE parent_id >= 0"):
- return
- menu = QMenu(self.view)
- for hregion, time in hregion_times:
- thread_at_time = (hregion.exec_comm_id, hregion.thread_id, time)
- menu_text = "Show Call Tree for {} {}:{} at {}".format(hregion.comm, hregion.pid, hregion.tid, time)
- menu.addAction(CreateAction(menu_text, "Show Call Tree", lambda a=None, args=thread_at_time: self.RightClickSelect(args), self.view))
- menu.exec_(pos)
-
- def RightClickSelect(self, args):
- CallTreeWindow(self.glb, self.glb.mainwindow, thread_at_time=args)
-
- def NoPointEvent(self):
- self.point_label.setText("")
- self.legend.Highlight({})
-
- def RangeEvent(self, time_from, time_to):
- time_from = time_from.quantize(Decimal(1))
- time_to = time_to.quantize(Decimal(1))
- if time_to <= time_from:
- self.point_label.setText("")
- return
- rel_time_from = time_from - self.start_time
- rel_time_to = time_to - self.start_time
- text = " Time: " + str(time_from) + " (+" + ToTimeStr(rel_time_from) + ") to: " + str(time_to) + " (+" + ToTimeStr(rel_time_to) + ")"
- text = text + " duration: " + ToTimeStr(time_to - time_from)
- self.point_label.setText(text)
-
- def BackState(self):
- return (self.attrs.subrange, self.attrs.scale, self.selection_state, self.fwd_rect)
-
- def PushBackState(self):
- state = copy.deepcopy(self.BackState())
- self.back_state.append(state)
- self.back_button.setEnabled(True)
-
- def PopBackState(self):
- self.attrs.subrange, self.attrs.scale, self.selection_state, self.fwd_rect = self.back_state.pop()
- self.attrs.Update()
- if not self.back_state:
- self.back_button.setDisabled(True)
-
- def PushForwardState(self):
- state = copy.deepcopy(self.BackState())
- self.forward_state.append(state)
- self.forward_button.setEnabled(True)
-
- def PopForwardState(self):
- self.attrs.subrange, self.attrs.scale, self.selection_state, self.fwd_rect = self.forward_state.pop()
- self.attrs.Update()
- if not self.forward_state:
- self.forward_button.setDisabled(True)
-
- def Title(self):
- time_from = self.collection.xrangelo + Decimal(self.attrs.subrange.x.lo)
- time_to = self.collection.xrangelo + Decimal(self.attrs.subrange.x.hi)
- rel_time_from = time_from - self.start_time
- rel_time_to = time_to - self.start_time
- title = "+" + ToTimeStr(rel_time_from) + " to +" + ToTimeStr(rel_time_to)
- title = title + " (" + ToTimeStr(time_to - time_from) + ")"
- return title
-
- def Update(self):
- selected_subrange, selection_state = self.selection_state
- self.item.SetSelection(selection_state)
- self.item.SetBracket(self.fwd_rect)
- self.zoom_button.setDisabled(selected_subrange is None)
- self.GraphTitleChanged(self.Title())
- self.item.update(self.item.boundingRect())
-
- def Back(self):
- if not self.back_state:
- return
- self.PushForwardState()
- self.PopBackState()
- self.Update()
-
- def Forward(self):
- if not self.forward_state:
- return
- self.PushBackState()
- self.PopForwardState()
- self.Update()
-
- def SelectEvent(self, x0, x1, selection_state):
- if selection_state is None:
- selected_subrange = None
- else:
- if x1 - x0 < 1.0:
- x1 += 1.0
- selected_subrange = Subrange(x0, x1)
- self.selection_state = (selected_subrange, selection_state)
- self.zoom_button.setDisabled(selected_subrange is None)
-
- def Zoom(self):
- selected_subrange, selection_state = self.selection_state
- if selected_subrange is None:
- return
- self.fwd_rect = selection_state
- self.item.SetSelection(None)
- self.PushBackState()
- self.attrs.subrange.x = selected_subrange
- self.forward_state = []
- self.forward_button.setDisabled(True)
- self.selection_state = (None, None)
- self.fwd_rect = None
- self.attrs.scale.x = self.GetScaleForRangeX(self.attrs.subrange.x)
- self.attrs.Update()
- self.Update()
-
-# Slow initialization - perform non-GUI initialization in a separate thread and put up a modal message box while waiting
-
-class SlowInitClass():
-
- def __init__(self, glb, title, init_fn):
- self.init_fn = init_fn
- self.done = False
- self.result = None
-
- self.msg_box = QMessageBox(glb.mainwindow)
- self.msg_box.setText("Initializing " + title + ". Please wait.")
- self.msg_box.setWindowTitle("Initializing " + title)
- self.msg_box.setWindowIcon(glb.mainwindow.style().standardIcon(QStyle.SP_MessageBoxInformation))
-
- self.init_thread = Thread(self.ThreadFn, glb)
- self.init_thread.done.connect(lambda: self.Done(), Qt.QueuedConnection)
-
- self.init_thread.start()
-
- def Done(self):
- self.msg_box.done(0)
-
- def ThreadFn(self, glb):
- conn_name = "SlowInitClass" + str(os.getpid())
- db, dbname = glb.dbref.Open(conn_name)
- self.result = self.init_fn(db)
- self.done = True
- return (True, 0)
-
- def Result(self):
- while not self.done:
- self.msg_box.exec_()
- self.init_thread.wait()
- return self.result
-
-def SlowInit(glb, title, init_fn):
- init = SlowInitClass(glb, title, init_fn)
- return init.Result()
-
-# Time chart by CPU window
-
-class TimeChartByCPUWindow(QMdiSubWindow):
-
- def __init__(self, glb, parent=None):
- super(TimeChartByCPUWindow, self).__init__(parent)
-
- self.glb = glb
- self.machine_id = glb.HostMachineId()
- self.collection_name = "SwitchGraphDataCollection " + str(self.machine_id)
-
- collection = LookupModel(self.collection_name)
- if collection is None:
- collection = SlowInit(glb, "Time Chart", self.Init)
-
- self.widget = SwitchGraphWidget(glb, collection, self)
- self.view = self.widget
-
- self.base_title = "Time Chart by CPU"
- self.setWindowTitle(self.base_title + self.widget.Title())
- self.widget.graph_title_changed.connect(self.GraphTitleChanged)
-
- self.setWidget(self.widget)
-
- AddSubWindow(glb.mainwindow.mdi_area, self, self.windowTitle())
-
- def Init(self, db):
- return LookupCreateModel(self.collection_name, lambda : SwitchGraphDataCollection(self.glb, db, self.machine_id))
-
- def GraphTitleChanged(self, title):
- self.setWindowTitle(self.base_title + " : " + title)
-
-# Child data item finder
-
-class ChildDataItemFinder():
-
- def __init__(self, root):
- self.root = root
- self.value, self.direction, self.pattern, self.last_value, self.last_pattern = (None,) * 5
- self.rows = []
- self.pos = 0
-
- def FindSelect(self):
- self.rows = []
- if self.pattern:
- pattern = re.compile(self.value)
- for child in self.root.child_items:
- for column_data in child.data:
- if re.search(pattern, str(column_data)) is not None:
- self.rows.append(child.row)
- break
- else:
- for child in self.root.child_items:
- for column_data in child.data:
- if self.value in str(column_data):
- self.rows.append(child.row)
- break
-
- def FindValue(self):
- self.pos = 0
- if self.last_value != self.value or self.pattern != self.last_pattern:
- self.FindSelect()
- if not len(self.rows):
- return -1
- return self.rows[self.pos]
-
- def FindThread(self):
- if self.direction == 0 or self.value != self.last_value or self.pattern != self.last_pattern:
- row = self.FindValue()
- elif len(self.rows):
- if self.direction > 0:
- self.pos += 1
- if self.pos >= len(self.rows):
- self.pos = 0
- else:
- self.pos -= 1
- if self.pos < 0:
- self.pos = len(self.rows) - 1
- row = self.rows[self.pos]
- else:
- row = -1
- return (True, row)
-
- def Find(self, value, direction, pattern, context, callback):
- self.value, self.direction, self.pattern, self.last_value, self.last_pattern = (value, direction,pattern, self.value, self.pattern)
- # Use a thread so the UI is not blocked
- thread = Thread(self.FindThread)
- thread.done.connect(lambda row, t=thread, c=callback: self.FindDone(t, c, row), Qt.QueuedConnection)
- thread.start()
-
- def FindDone(self, thread, callback, row):
- callback(row)
-
-# Number of database records to fetch in one go
-
-glb_chunk_sz = 10000
-
-# Background process for SQL data fetcher
-
-class SQLFetcherProcess():
-
- def __init__(self, dbref, sql, buffer, head, tail, fetch_count, fetching_done, process_target, wait_event, fetched_event, prep):
- # Need a unique connection name
- conn_name = "SQLFetcher" + str(os.getpid())
- self.db, dbname = dbref.Open(conn_name)
- self.sql = sql
- self.buffer = buffer
- self.head = head
- self.tail = tail
- self.fetch_count = fetch_count
- self.fetching_done = fetching_done
- self.process_target = process_target
- self.wait_event = wait_event
- self.fetched_event = fetched_event
- self.prep = prep
- self.query = QSqlQuery(self.db)
- self.query_limit = 0 if "$$last_id$$" in sql else 2
- self.last_id = -1
- self.fetched = 0
- self.more = True
- self.local_head = self.head.value
- self.local_tail = self.tail.value
-
- def Select(self):
- if self.query_limit:
- if self.query_limit == 1:
- return
- self.query_limit -= 1
- stmt = self.sql.replace("$$last_id$$", str(self.last_id))
- QueryExec(self.query, stmt)
-
- def Next(self):
- if not self.query.next():
- self.Select()
- if not self.query.next():
- return None
- self.last_id = self.query.value(0)
- return self.prep(self.query)
-
- def WaitForTarget(self):
- while True:
- self.wait_event.clear()
- target = self.process_target.value
- if target > self.fetched or target < 0:
- break
- self.wait_event.wait()
- return target
-
- def HasSpace(self, sz):
- if self.local_tail <= self.local_head:
- space = len(self.buffer) - self.local_head
- if space > sz:
- return True
- if space >= glb_nsz:
- # Use 0 (or space < glb_nsz) to mean there is no more at the top of the buffer
- nd = pickle.dumps(0, pickle.HIGHEST_PROTOCOL)
- self.buffer[self.local_head : self.local_head + len(nd)] = nd
- self.local_head = 0
- if self.local_tail - self.local_head > sz:
- return True
- return False
-
- def WaitForSpace(self, sz):
- if self.HasSpace(sz):
- return
- while True:
- self.wait_event.clear()
- self.local_tail = self.tail.value
- if self.HasSpace(sz):
- return
- self.wait_event.wait()
-
- def AddToBuffer(self, obj):
- d = pickle.dumps(obj, pickle.HIGHEST_PROTOCOL)
- n = len(d)
- nd = pickle.dumps(n, pickle.HIGHEST_PROTOCOL)
- sz = n + glb_nsz
- self.WaitForSpace(sz)
- pos = self.local_head
- self.buffer[pos : pos + len(nd)] = nd
- self.buffer[pos + glb_nsz : pos + sz] = d
- self.local_head += sz
-
- def FetchBatch(self, batch_size):
- fetched = 0
- while batch_size > fetched:
- obj = self.Next()
- if obj is None:
- self.more = False
- break
- self.AddToBuffer(obj)
- fetched += 1
- if fetched:
- self.fetched += fetched
- with self.fetch_count.get_lock():
- self.fetch_count.value += fetched
- self.head.value = self.local_head
- self.fetched_event.set()
-
- def Run(self):
- while self.more:
- target = self.WaitForTarget()
- if target < 0:
- break
- batch_size = min(glb_chunk_sz, target - self.fetched)
- self.FetchBatch(batch_size)
- self.fetching_done.value = True
- self.fetched_event.set()
-
-def SQLFetcherFn(*x):
- process = SQLFetcherProcess(*x)
- process.Run()
-
-# SQL data fetcher
-
-class SQLFetcher(QObject):
-
- done = Signal(object)
-
- def __init__(self, glb, sql, prep, process_data, parent=None):
- super(SQLFetcher, self).__init__(parent)
- self.process_data = process_data
- self.more = True
- self.target = 0
- self.last_target = 0
- self.fetched = 0
- self.buffer_size = 16 * 1024 * 1024
- self.buffer = Array(c_char, self.buffer_size, lock=False)
- self.head = Value(c_longlong)
- self.tail = Value(c_longlong)
- self.local_tail = 0
- self.fetch_count = Value(c_longlong)
- self.fetching_done = Value(c_bool)
- self.last_count = 0
- self.process_target = Value(c_longlong)
- self.wait_event = Event()
- self.fetched_event = Event()
- glb.AddInstanceToShutdownOnExit(self)
- self.process = Process(target=SQLFetcherFn, args=(glb.dbref, sql, self.buffer, self.head, self.tail, self.fetch_count, self.fetching_done, self.process_target, self.wait_event, self.fetched_event, prep))
- self.process.start()
- self.thread = Thread(self.Thread)
- self.thread.done.connect(self.ProcessData, Qt.QueuedConnection)
- self.thread.start()
-
- def Shutdown(self):
- # Tell the thread and process to exit
- self.process_target.value = -1
- self.wait_event.set()
- self.more = False
- self.fetching_done.value = True
- self.fetched_event.set()
-
- def Thread(self):
- if not self.more:
- return True, 0
- while True:
- self.fetched_event.clear()
- fetch_count = self.fetch_count.value
- if fetch_count != self.last_count:
- break
- if self.fetching_done.value:
- self.more = False
- return True, 0
- self.fetched_event.wait()
- count = fetch_count - self.last_count
- self.last_count = fetch_count
- self.fetched += count
- return False, count
-
- def Fetch(self, nr):
- if not self.more:
- # -1 inidcates there are no more
- return -1
- result = self.fetched
- extra = result + nr - self.target
- if extra > 0:
- self.target += extra
- # process_target < 0 indicates shutting down
- if self.process_target.value >= 0:
- self.process_target.value = self.target
- self.wait_event.set()
- return result
-
- def RemoveFromBuffer(self):
- pos = self.local_tail
- if len(self.buffer) - pos < glb_nsz:
- pos = 0
- n = pickle.loads(self.buffer[pos : pos + glb_nsz])
- if n == 0:
- pos = 0
- n = pickle.loads(self.buffer[0 : glb_nsz])
- pos += glb_nsz
- obj = pickle.loads(self.buffer[pos : pos + n])
- self.local_tail = pos + n
- return obj
-
- def ProcessData(self, count):
- for i in xrange(count):
- obj = self.RemoveFromBuffer()
- self.process_data(obj)
- self.tail.value = self.local_tail
- self.wait_event.set()
- self.done.emit(count)
-
-# Fetch more records bar
-
-class FetchMoreRecordsBar():
-
- def __init__(self, model, parent):
- self.model = model
-
- self.label = QLabel("Number of records (x " + "{:,}".format(glb_chunk_sz) + ") to fetch:")
- self.label.setSizePolicy(QSizePolicy.Fixed, QSizePolicy.Fixed)
-
- self.fetch_count = QSpinBox()
- self.fetch_count.setRange(1, 1000000)
- self.fetch_count.setValue(10)
- self.fetch_count.setSizePolicy(QSizePolicy.Fixed, QSizePolicy.Fixed)
-
- self.fetch = QPushButton("Go!")
- self.fetch.setSizePolicy(QSizePolicy.Fixed, QSizePolicy.Fixed)
- self.fetch.released.connect(self.FetchMoreRecords)
-
- self.progress = QProgressBar()
- self.progress.setRange(0, 100)
- self.progress.hide()
-
- self.done_label = QLabel("All records fetched")
- self.done_label.hide()
-
- self.spacer = QLabel("")
-
- self.close_button = QToolButton()
- self.close_button.setIcon(parent.style().standardIcon(QStyle.SP_DockWidgetCloseButton))
- self.close_button.released.connect(self.Deactivate)
-
- self.hbox = QHBoxLayout()
- self.hbox.setContentsMargins(0, 0, 0, 0)
-
- self.hbox.addWidget(self.label)
- self.hbox.addWidget(self.fetch_count)
- self.hbox.addWidget(self.fetch)
- self.hbox.addWidget(self.spacer)
- self.hbox.addWidget(self.progress)
- self.hbox.addWidget(self.done_label)
- self.hbox.addWidget(self.close_button)
-
- self.bar = QWidget()
- self.bar.setLayout(self.hbox)
- self.bar.show()
-
- self.in_progress = False
- self.model.progress.connect(self.Progress)
-
- self.done = False
-
- if not model.HasMoreRecords():
- self.Done()
-
- def Widget(self):
- return self.bar
-
- def Activate(self):
- self.bar.show()
- self.fetch.setFocus()
-
- def Deactivate(self):
- self.bar.hide()
-
- def Enable(self, enable):
- self.fetch.setEnabled(enable)
- self.fetch_count.setEnabled(enable)
-
- def Busy(self):
- self.Enable(False)
- self.fetch.hide()
- self.spacer.hide()
- self.progress.show()
-
- def Idle(self):
- self.in_progress = False
- self.Enable(True)
- self.progress.hide()
- self.fetch.show()
- self.spacer.show()
-
- def Target(self):
- return self.fetch_count.value() * glb_chunk_sz
-
- def Done(self):
- self.done = True
- self.Idle()
- self.label.hide()
- self.fetch_count.hide()
- self.fetch.hide()
- self.spacer.hide()
- self.done_label.show()
-
- def Progress(self, count):
- if self.in_progress:
- if count:
- percent = ((count - self.start) * 100) / self.Target()
- if percent >= 100:
- self.Idle()
- else:
- self.progress.setValue(percent)
- if not count:
- # Count value of zero means no more records
- self.Done()
-
- def FetchMoreRecords(self):
- if self.done:
- return
- self.progress.setValue(0)
- self.Busy()
- self.in_progress = True
- self.start = self.model.FetchMoreRecords(self.Target())
-
-# Brance data model level two item
-
-class BranchLevelTwoItem():
-
- def __init__(self, row, col, text, parent_item):
- self.row = row
- self.parent_item = parent_item
- self.data = [""] * (col + 1)
- self.data[col] = text
- self.level = 2
-
- def getParentItem(self):
- return self.parent_item
-
- def getRow(self):
- return self.row
-
- def childCount(self):
- return 0
-
- def hasChildren(self):
- return False
-
- def getData(self, column):
- return self.data[column]
-
-# Brance data model level one item
-
-class BranchLevelOneItem():
-
- def __init__(self, glb, row, data, parent_item):
- self.glb = glb
- self.row = row
- self.parent_item = parent_item
- self.child_count = 0
- self.child_items = []
- self.data = data[1:]
- self.dbid = data[0]
- self.level = 1
- self.query_done = False
- self.br_col = len(self.data) - 1
-
- def getChildItem(self, row):
- return self.child_items[row]
-
- def getParentItem(self):
- return self.parent_item
-
- def getRow(self):
- return self.row
-
- def Select(self):
- self.query_done = True
-
- if not self.glb.have_disassembler:
- return
-
- query = QSqlQuery(self.glb.db)
-
- QueryExec(query, "SELECT cpu, to_dso_id, to_symbol_id, to_sym_offset, short_name, long_name, build_id, sym_start, to_ip"
- " FROM samples"
- " INNER JOIN dsos ON samples.to_dso_id = dsos.id"
- " INNER JOIN symbols ON samples.to_symbol_id = symbols.id"
- " WHERE samples.id = " + str(self.dbid))
- if not query.next():
- return
- cpu = query.value(0)
- dso = query.value(1)
- sym = query.value(2)
- if dso == 0 or sym == 0:
- return
- off = query.value(3)
- short_name = query.value(4)
- long_name = query.value(5)
- build_id = query.value(6)
- sym_start = query.value(7)
- ip = query.value(8)
-
- QueryExec(query, "SELECT samples.dso_id, symbol_id, sym_offset, sym_start"
- " FROM samples"
- " INNER JOIN symbols ON samples.symbol_id = symbols.id"
- " WHERE samples.id > " + str(self.dbid) + " AND cpu = " + str(cpu) +
- " ORDER BY samples.id"
- " LIMIT 1")
- if not query.next():
- return
- if query.value(0) != dso:
- # Cannot disassemble from one dso to another
- return
- bsym = query.value(1)
- boff = query.value(2)
- bsym_start = query.value(3)
- if bsym == 0:
- return
- tot = bsym_start + boff + 1 - sym_start - off
- if tot <= 0 or tot > 16384:
- return
-
- inst = self.glb.disassembler.Instruction()
- f = self.glb.FileFromNamesAndBuildId(short_name, long_name, build_id)
- if not f:
- return
- mode = 0 if Is64Bit(f) else 1
- self.glb.disassembler.SetMode(inst, mode)
-
- buf_sz = tot + 16
- buf = create_string_buffer(tot + 16)
- f.seek(sym_start + off)
- buf.value = f.read(buf_sz)
- buf_ptr = addressof(buf)
- i = 0
- while tot > 0:
- cnt, text = self.glb.disassembler.DisassembleOne(inst, buf_ptr, buf_sz, ip)
- if cnt:
- byte_str = tohex(ip).rjust(16)
- for k in xrange(cnt):
- byte_str += " %02x" % ord(buf[i])
- i += 1
- while k < 15:
- byte_str += " "
- k += 1
- self.child_items.append(BranchLevelTwoItem(0, self.br_col, byte_str + " " + text, self))
- self.child_count += 1
- else:
- return
- buf_ptr += cnt
- tot -= cnt
- buf_sz -= cnt
- ip += cnt
-
- def childCount(self):
- if not self.query_done:
- self.Select()
- if not self.child_count:
- return -1
- return self.child_count
-
- def hasChildren(self):
- if not self.query_done:
- return True
- return self.child_count > 0
-
- def getData(self, column):
- return self.data[column]
-
-# Brance data model root item
-
-class BranchRootItem():
-
- def __init__(self):
- self.child_count = 0
- self.child_items = []
- self.level = 0
-
- def getChildItem(self, row):
- return self.child_items[row]
-
- def getParentItem(self):
- return None
-
- def getRow(self):
- return 0
-
- def childCount(self):
- return self.child_count
-
- def hasChildren(self):
- return self.child_count > 0
-
- def getData(self, column):
- return ""
-
-# Calculate instructions per cycle
-
-def CalcIPC(cyc_cnt, insn_cnt):
- if cyc_cnt and insn_cnt:
- ipc = Decimal(float(insn_cnt) / cyc_cnt)
- ipc = str(ipc.quantize(Decimal(".01"), rounding=ROUND_HALF_UP))
- else:
- ipc = "0"
- return ipc
-
-# Branch data preparation
-
-def BranchDataPrepBr(query, data):
- data.append(tohex(query.value(8)).rjust(16) + " " + query.value(9) + offstr(query.value(10)) +
- " (" + dsoname(query.value(11)) + ")" + " -> " +
- tohex(query.value(12)) + " " + query.value(13) + offstr(query.value(14)) +
- " (" + dsoname(query.value(15)) + ")")
-
-def BranchDataPrepIPC(query, data):
- insn_cnt = query.value(16)
- cyc_cnt = query.value(17)
- ipc = CalcIPC(cyc_cnt, insn_cnt)
- data.append(insn_cnt)
- data.append(cyc_cnt)
- data.append(ipc)
-
-def BranchDataPrep(query):
- data = []
- for i in xrange(0, 8):
- data.append(query.value(i))
- BranchDataPrepBr(query, data)
- return data
-
-def BranchDataPrepWA(query):
- data = []
- data.append(query.value(0))
- # Workaround pyside failing to handle large integers (i.e. time) in python3 by converting to a string
- data.append("{:>19}".format(query.value(1)))
- for i in xrange(2, 8):
- data.append(query.value(i))
- BranchDataPrepBr(query, data)
- return data
-
-def BranchDataWithIPCPrep(query):
- data = []
- for i in xrange(0, 8):
- data.append(query.value(i))
- BranchDataPrepIPC(query, data)
- BranchDataPrepBr(query, data)
- return data
-
-def BranchDataWithIPCPrepWA(query):
- data = []
- data.append(query.value(0))
- # Workaround pyside failing to handle large integers (i.e. time) in python3 by converting to a string
- data.append("{:>19}".format(query.value(1)))
- for i in xrange(2, 8):
- data.append(query.value(i))
- BranchDataPrepIPC(query, data)
- BranchDataPrepBr(query, data)
- return data
-
-# Branch data model
-
-class BranchModel(TreeModel):
-
- progress = Signal(object)
-
- def __init__(self, glb, event_id, where_clause, parent=None):
- super(BranchModel, self).__init__(glb, None, parent)
- self.event_id = event_id
- self.more = True
- self.populated = 0
- self.have_ipc = IsSelectable(glb.db, "samples", columns = "insn_count, cyc_count")
- if self.have_ipc:
- select_ipc = ", insn_count, cyc_count"
- prep_fn = BranchDataWithIPCPrep
- prep_wa_fn = BranchDataWithIPCPrepWA
- else:
- select_ipc = ""
- prep_fn = BranchDataPrep
- prep_wa_fn = BranchDataPrepWA
- sql = ("SELECT samples.id, time, cpu, comm, pid, tid, branch_types.name,"
- " CASE WHEN in_tx = '0' THEN 'No' ELSE 'Yes' END,"
- " ip, symbols.name, sym_offset, dsos.short_name,"
- " to_ip, to_symbols.name, to_sym_offset, to_dsos.short_name"
- + select_ipc +
- " FROM samples"
- " INNER JOIN comms ON comm_id = comms.id"
- " INNER JOIN threads ON thread_id = threads.id"
- " INNER JOIN branch_types ON branch_type = branch_types.id"
- " INNER JOIN symbols ON symbol_id = symbols.id"
- " INNER JOIN symbols to_symbols ON to_symbol_id = to_symbols.id"
- " INNER JOIN dsos ON samples.dso_id = dsos.id"
- " INNER JOIN dsos AS to_dsos ON samples.to_dso_id = to_dsos.id"
- " WHERE samples.id > $$last_id$$" + where_clause +
- " AND evsel_id = " + str(self.event_id) +
- " ORDER BY samples.id"
- " LIMIT " + str(glb_chunk_sz))
- if pyside_version_1 and sys.version_info[0] == 3:
- prep = prep_fn
- else:
- prep = prep_wa_fn
- self.fetcher = SQLFetcher(glb, sql, prep, self.AddSample)
- self.fetcher.done.connect(self.Update)
- self.fetcher.Fetch(glb_chunk_sz)
-
- def GetRoot(self):
- return BranchRootItem()
-
- def columnCount(self, parent=None):
- if self.have_ipc:
- return 11
- else:
- return 8
-
- def columnHeader(self, column):
- if self.have_ipc:
- return ("Time", "CPU", "Command", "PID", "TID", "Branch Type", "In Tx", "Insn Cnt", "Cyc Cnt", "IPC", "Branch")[column]
- else:
- return ("Time", "CPU", "Command", "PID", "TID", "Branch Type", "In Tx", "Branch")[column]
-
- def columnFont(self, column):
- if self.have_ipc:
- br_col = 10
- else:
- br_col = 7
- if column != br_col:
- return None
- return QFont("Monospace")
-
- def DisplayData(self, item, index):
- if item.level == 1:
- self.FetchIfNeeded(item.row)
- return item.getData(index.column())
-
- def AddSample(self, data):
- child = BranchLevelOneItem(self.glb, self.populated, data, self.root)
- self.root.child_items.append(child)
- self.populated += 1
-
- def Update(self, fetched):
- if not fetched:
- self.more = False
- self.progress.emit(0)
- child_count = self.root.child_count
- count = self.populated - child_count
- if count > 0:
- parent = QModelIndex()
- self.beginInsertRows(parent, child_count, child_count + count - 1)
- self.insertRows(child_count, count, parent)
- self.root.child_count += count
- self.endInsertRows()
- self.progress.emit(self.root.child_count)
-
- def FetchMoreRecords(self, count):
- current = self.root.child_count
- if self.more:
- self.fetcher.Fetch(count)
- else:
- self.progress.emit(0)
- return current
-
- def HasMoreRecords(self):
- return self.more
-
-# Report Variables
-
-class ReportVars():
-
- def __init__(self, name = "", where_clause = "", limit = ""):
- self.name = name
- self.where_clause = where_clause
- self.limit = limit
-
- def UniqueId(self):
- return str(self.where_clause + ";" + self.limit)
-
-# Branch window
-
-class BranchWindow(QMdiSubWindow):
-
- def __init__(self, glb, event_id, report_vars, parent=None):
- super(BranchWindow, self).__init__(parent)
-
- model_name = "Branch Events " + str(event_id) + " " + report_vars.UniqueId()
-
- self.model = LookupCreateModel(model_name, lambda: BranchModel(glb, event_id, report_vars.where_clause))
-
- self.view = QTreeView()
- self.view.setUniformRowHeights(True)
- self.view.setSelectionMode(QAbstractItemView.ContiguousSelection)
- self.view.CopyCellsToClipboard = CopyTreeCellsToClipboard
- self.view.setModel(self.model)
-
- self.ResizeColumnsToContents()
-
- self.context_menu = TreeContextMenu(self.view)
-
- self.find_bar = FindBar(self, self, True)
-
- self.finder = ChildDataItemFinder(self.model.root)
-
- self.fetch_bar = FetchMoreRecordsBar(self.model, self)
-
- self.vbox = VBox(self.view, self.find_bar.Widget(), self.fetch_bar.Widget())
-
- self.setWidget(self.vbox.Widget())
-
- AddSubWindow(glb.mainwindow.mdi_area, self, report_vars.name + " Branch Events")
-
- def ResizeColumnToContents(self, column, n):
- # Using the view's resizeColumnToContents() here is extrememly slow
- # so implement a crude alternative
- mm = "MM" if column else "MMMM"
- font = self.view.font()
- metrics = QFontMetrics(font)
- max = 0
- for row in xrange(n):
- val = self.model.root.child_items[row].data[column]
- len = metrics.width(str(val) + mm)
- max = len if len > max else max
- val = self.model.columnHeader(column)
- len = metrics.width(str(val) + mm)
- max = len if len > max else max
- self.view.setColumnWidth(column, max)
-
- def ResizeColumnsToContents(self):
- n = min(self.model.root.child_count, 100)
- if n < 1:
- # No data yet, so connect a signal to notify when there is
- self.model.rowsInserted.connect(self.UpdateColumnWidths)
- return
- columns = self.model.columnCount()
- for i in xrange(columns):
- self.ResizeColumnToContents(i, n)
-
- def UpdateColumnWidths(self, *x):
- # This only needs to be done once, so disconnect the signal now
- self.model.rowsInserted.disconnect(self.UpdateColumnWidths)
- self.ResizeColumnsToContents()
-
- def Find(self, value, direction, pattern, context):
- self.view.setFocus()
- self.find_bar.Busy()
- self.finder.Find(value, direction, pattern, context, self.FindDone)
-
- def FindDone(self, row):
- self.find_bar.Idle()
- if row >= 0:
- self.view.setCurrentIndex(self.model.index(row, 0, QModelIndex()))
- else:
- self.find_bar.NotFound()
-
-# Line edit data item
-
-class LineEditDataItem(object):
-
- def __init__(self, glb, label, placeholder_text, parent, id = "", default = ""):
- self.glb = glb
- self.label = label
- self.placeholder_text = placeholder_text
- self.parent = parent
- self.id = id
-
- self.value = default
-
- self.widget = QLineEdit(default)
- self.widget.editingFinished.connect(self.Validate)
- self.widget.textChanged.connect(self.Invalidate)
- self.red = False
- self.error = ""
- self.validated = True
-
- if placeholder_text:
- self.widget.setPlaceholderText(placeholder_text)
-
- def TurnTextRed(self):
- if not self.red:
- palette = QPalette()
- palette.setColor(QPalette.Text,Qt.red)
- self.widget.setPalette(palette)
- self.red = True
-
- def TurnTextNormal(self):
- if self.red:
- palette = QPalette()
- self.widget.setPalette(palette)
- self.red = False
-
- def InvalidValue(self, value):
- self.value = ""
- self.TurnTextRed()
- self.error = self.label + " invalid value '" + value + "'"
- self.parent.ShowMessage(self.error)
-
- def Invalidate(self):
- self.validated = False
-
- def DoValidate(self, input_string):
- self.value = input_string.strip()
-
- def Validate(self):
- self.validated = True
- self.error = ""
- self.TurnTextNormal()
- self.parent.ClearMessage()
- input_string = self.widget.text()
- if not len(input_string.strip()):
- self.value = ""
- return
- self.DoValidate(input_string)
-
- def IsValid(self):
- if not self.validated:
- self.Validate()
- if len(self.error):
- self.parent.ShowMessage(self.error)
- return False
- return True
-
- def IsNumber(self, value):
- try:
- x = int(value)
- except:
- x = 0
- return str(x) == value
-
-# Non-negative integer ranges dialog data item
-
-class NonNegativeIntegerRangesDataItem(LineEditDataItem):
-
- def __init__(self, glb, label, placeholder_text, column_name, parent):
- super(NonNegativeIntegerRangesDataItem, self).__init__(glb, label, placeholder_text, parent)
-
- self.column_name = column_name
-
- def DoValidate(self, input_string):
- singles = []
- ranges = []
- for value in [x.strip() for x in input_string.split(",")]:
- if "-" in value:
- vrange = value.split("-")
- if len(vrange) != 2 or not self.IsNumber(vrange[0]) or not self.IsNumber(vrange[1]):
- return self.InvalidValue(value)
- ranges.append(vrange)
- else:
- if not self.IsNumber(value):
- return self.InvalidValue(value)
- singles.append(value)
- ranges = [("(" + self.column_name + " >= " + r[0] + " AND " + self.column_name + " <= " + r[1] + ")") for r in ranges]
- if len(singles):
- ranges.append(self.column_name + " IN (" + ",".join(singles) + ")")
- self.value = " OR ".join(ranges)
-
-# Positive integer dialog data item
-
-class PositiveIntegerDataItem(LineEditDataItem):
-
- def __init__(self, glb, label, placeholder_text, parent, id = "", default = ""):
- super(PositiveIntegerDataItem, self).__init__(glb, label, placeholder_text, parent, id, default)
-
- def DoValidate(self, input_string):
- if not self.IsNumber(input_string.strip()):
- return self.InvalidValue(input_string)
- value = int(input_string.strip())
- if value <= 0:
- return self.InvalidValue(input_string)
- self.value = str(value)
-
-# Dialog data item converted and validated using a SQL table
-
-class SQLTableDataItem(LineEditDataItem):
-
- def __init__(self, glb, label, placeholder_text, table_name, match_column, column_name1, column_name2, parent):
- super(SQLTableDataItem, self).__init__(glb, label, placeholder_text, parent)
-
- self.table_name = table_name
- self.match_column = match_column
- self.column_name1 = column_name1
- self.column_name2 = column_name2
-
- def ValueToIds(self, value):
- ids = []
- query = QSqlQuery(self.glb.db)
- stmt = "SELECT id FROM " + self.table_name + " WHERE " + self.match_column + " = '" + value + "'"
- ret = query.exec_(stmt)
- if ret:
- while query.next():
- ids.append(str(query.value(0)))
- return ids
-
- def DoValidate(self, input_string):
- all_ids = []
- for value in [x.strip() for x in input_string.split(",")]:
- ids = self.ValueToIds(value)
- if len(ids):
- all_ids.extend(ids)
- else:
- return self.InvalidValue(value)
- self.value = self.column_name1 + " IN (" + ",".join(all_ids) + ")"
- if self.column_name2:
- self.value = "( " + self.value + " OR " + self.column_name2 + " IN (" + ",".join(all_ids) + ") )"
-
-# Sample time ranges dialog data item converted and validated using 'samples' SQL table
-
-class SampleTimeRangesDataItem(LineEditDataItem):
-
- def __init__(self, glb, label, placeholder_text, column_name, parent):
- self.column_name = column_name
-
- self.last_id = 0
- self.first_time = 0
- self.last_time = 2 ** 64
-
- query = QSqlQuery(glb.db)
- QueryExec(query, "SELECT id, time FROM samples ORDER BY id DESC LIMIT 1")
- if query.next():
- self.last_id = int(query.value(0))
- self.first_time = int(glb.HostStartTime())
- self.last_time = int(glb.HostFinishTime())
- if placeholder_text:
- placeholder_text += ", between " + str(self.first_time) + " and " + str(self.last_time)
-
- super(SampleTimeRangesDataItem, self).__init__(glb, label, placeholder_text, parent)
-
- def IdBetween(self, query, lower_id, higher_id, order):
- QueryExec(query, "SELECT id FROM samples WHERE id > " + str(lower_id) + " AND id < " + str(higher_id) + " ORDER BY id " + order + " LIMIT 1")
- if query.next():
- return True, int(query.value(0))
- else:
- return False, 0
-
- def BinarySearchTime(self, lower_id, higher_id, target_time, get_floor):
- query = QSqlQuery(self.glb.db)
- while True:
- next_id = int((lower_id + higher_id) / 2)
- QueryExec(query, "SELECT time FROM samples WHERE id = " + str(next_id))
- if not query.next():
- ok, dbid = self.IdBetween(query, lower_id, next_id, "DESC")
- if not ok:
- ok, dbid = self.IdBetween(query, next_id, higher_id, "")
- if not ok:
- return str(higher_id)
- next_id = dbid
- QueryExec(query, "SELECT time FROM samples WHERE id = " + str(next_id))
- next_time = int(query.value(0))
- if get_floor:
- if target_time > next_time:
- lower_id = next_id
- else:
- higher_id = next_id
- if higher_id <= lower_id + 1:
- return str(higher_id)
- else:
- if target_time >= next_time:
- lower_id = next_id
- else:
- higher_id = next_id
- if higher_id <= lower_id + 1:
- return str(lower_id)
-
- def ConvertRelativeTime(self, val):
- mult = 1
- suffix = val[-2:]
- if suffix == "ms":
- mult = 1000000
- elif suffix == "us":
- mult = 1000
- elif suffix == "ns":
- mult = 1
- else:
- return val
- val = val[:-2].strip()
- if not self.IsNumber(val):
- return val
- val = int(val) * mult
- if val >= 0:
- val += self.first_time
- else:
- val += self.last_time
- return str(val)
-
- def ConvertTimeRange(self, vrange):
- if vrange[0] == "":
- vrange[0] = str(self.first_time)
- if vrange[1] == "":
- vrange[1] = str(self.last_time)
- vrange[0] = self.ConvertRelativeTime(vrange[0])
- vrange[1] = self.ConvertRelativeTime(vrange[1])
- if not self.IsNumber(vrange[0]) or not self.IsNumber(vrange[1]):
- return False
- beg_range = max(int(vrange[0]), self.first_time)
- end_range = min(int(vrange[1]), self.last_time)
- if beg_range > self.last_time or end_range < self.first_time:
- return False
- vrange[0] = self.BinarySearchTime(0, self.last_id, beg_range, True)
- vrange[1] = self.BinarySearchTime(1, self.last_id + 1, end_range, False)
- return True
-
- def AddTimeRange(self, value, ranges):
- n = value.count("-")
- if n == 1:
- pass
- elif n == 2:
- if value.split("-")[1].strip() == "":
- n = 1
- elif n == 3:
- n = 2
- else:
- return False
- pos = findnth(value, "-", n)
- vrange = [value[:pos].strip() ,value[pos+1:].strip()]
- if self.ConvertTimeRange(vrange):
- ranges.append(vrange)
- return True
- return False
-
- def DoValidate(self, input_string):
- ranges = []
- for value in [x.strip() for x in input_string.split(",")]:
- if not self.AddTimeRange(value, ranges):
- return self.InvalidValue(value)
- ranges = [("(" + self.column_name + " >= " + r[0] + " AND " + self.column_name + " <= " + r[1] + ")") for r in ranges]
- self.value = " OR ".join(ranges)
-
-# Report Dialog Base
-
-class ReportDialogBase(QDialog):
-
- def __init__(self, glb, title, items, partial, parent=None):
- super(ReportDialogBase, self).__init__(parent)
-
- self.glb = glb
-
- self.report_vars = ReportVars()
-
- self.setWindowTitle(title)
- self.setMinimumWidth(600)
-
- self.data_items = [x(glb, self) for x in items]
-
- self.partial = partial
-
- self.grid = QGridLayout()
-
- for row in xrange(len(self.data_items)):
- self.grid.addWidget(QLabel(self.data_items[row].label), row, 0)
- self.grid.addWidget(self.data_items[row].widget, row, 1)
-
- self.status = QLabel()
-
- self.ok_button = QPushButton("Ok", self)
- self.ok_button.setDefault(True)
- self.ok_button.released.connect(self.Ok)
- self.ok_button.setSizePolicy(QSizePolicy.Fixed, QSizePolicy.Fixed)
-
- self.cancel_button = QPushButton("Cancel", self)
- self.cancel_button.released.connect(self.reject)
- self.cancel_button.setSizePolicy(QSizePolicy.Fixed, QSizePolicy.Fixed)
-
- self.hbox = QHBoxLayout()
- #self.hbox.addStretch()
- self.hbox.addWidget(self.status)
- self.hbox.addWidget(self.ok_button)
- self.hbox.addWidget(self.cancel_button)
-
- self.vbox = QVBoxLayout()
- self.vbox.addLayout(self.grid)
- self.vbox.addLayout(self.hbox)
-
- self.setLayout(self.vbox)
-
- def Ok(self):
- vars = self.report_vars
- for d in self.data_items:
- if d.id == "REPORTNAME":
- vars.name = d.value
- if not vars.name:
- self.ShowMessage("Report name is required")
- return
- for d in self.data_items:
- if not d.IsValid():
- return
- for d in self.data_items[1:]:
- if d.id == "LIMIT":
- vars.limit = d.value
- elif len(d.value):
- if len(vars.where_clause):
- vars.where_clause += " AND "
- vars.where_clause += d.value
- if len(vars.where_clause):
- if self.partial:
- vars.where_clause = " AND ( " + vars.where_clause + " ) "
- else:
- vars.where_clause = " WHERE " + vars.where_clause + " "
- self.accept()
-
- def ShowMessage(self, msg):
- self.status.setText("<font color=#FF0000>" + msg)
-
- def ClearMessage(self):
- self.status.setText("")
-
-# Selected branch report creation dialog
-
-class SelectedBranchDialog(ReportDialogBase):
-
- def __init__(self, glb, parent=None):
- title = "Selected Branches"
- items = (lambda g, p: LineEditDataItem(g, "Report name:", "Enter a name to appear in the window title bar", p, "REPORTNAME"),
- lambda g, p: SampleTimeRangesDataItem(g, "Time ranges:", "Enter time ranges", "samples.id", p),
- lambda g, p: NonNegativeIntegerRangesDataItem(g, "CPUs:", "Enter CPUs or ranges e.g. 0,5-6", "cpu", p),
- lambda g, p: SQLTableDataItem(g, "Commands:", "Only branches with these commands will be included", "comms", "comm", "comm_id", "", p),
- lambda g, p: SQLTableDataItem(g, "PIDs:", "Only branches with these process IDs will be included", "threads", "pid", "thread_id", "", p),
- lambda g, p: SQLTableDataItem(g, "TIDs:", "Only branches with these thread IDs will be included", "threads", "tid", "thread_id", "", p),
- lambda g, p: SQLTableDataItem(g, "DSOs:", "Only branches with these DSOs will be included", "dsos", "short_name", "samples.dso_id", "to_dso_id", p),
- lambda g, p: SQLTableDataItem(g, "Symbols:", "Only branches with these symbols will be included", "symbols", "name", "symbol_id", "to_symbol_id", p),
- lambda g, p: LineEditDataItem(g, "Raw SQL clause: ", "Enter a raw SQL WHERE clause", p))
- super(SelectedBranchDialog, self).__init__(glb, title, items, True, parent)
-
-# Event list
-
-def GetEventList(db):
- events = []
- query = QSqlQuery(db)
- QueryExec(query, "SELECT name FROM selected_events WHERE id > 0 ORDER BY id")
- while query.next():
- events.append(query.value(0))
- return events
-
-# Is a table selectable
-
-def IsSelectable(db, table, sql = "", columns = "*"):
- query = QSqlQuery(db)
- try:
- QueryExec(query, "SELECT " + columns + " FROM " + table + " " + sql + " LIMIT 1")
- except:
- return False
- return True
-
-# SQL table data model item
-
-class SQLTableItem():
-
- def __init__(self, row, data):
- self.row = row
- self.data = data
-
- def getData(self, column):
- return self.data[column]
-
-# SQL table data model
-
-class SQLTableModel(TableModel):
-
- progress = Signal(object)
-
- def __init__(self, glb, sql, column_headers, parent=None):
- super(SQLTableModel, self).__init__(parent)
- self.glb = glb
- self.more = True
- self.populated = 0
- self.column_headers = column_headers
- self.fetcher = SQLFetcher(glb, sql, lambda x, y=len(column_headers): self.SQLTableDataPrep(x, y), self.AddSample)
- self.fetcher.done.connect(self.Update)
- self.fetcher.Fetch(glb_chunk_sz)
-
- def DisplayData(self, item, index):
- self.FetchIfNeeded(item.row)
- return item.getData(index.column())
-
- def AddSample(self, data):
- child = SQLTableItem(self.populated, data)
- self.child_items.append(child)
- self.populated += 1
-
- def Update(self, fetched):
- if not fetched:
- self.more = False
- self.progress.emit(0)
- child_count = self.child_count
- count = self.populated - child_count
- if count > 0:
- parent = QModelIndex()
- self.beginInsertRows(parent, child_count, child_count + count - 1)
- self.insertRows(child_count, count, parent)
- self.child_count += count
- self.endInsertRows()
- self.progress.emit(self.child_count)
-
- def FetchMoreRecords(self, count):
- current = self.child_count
- if self.more:
- self.fetcher.Fetch(count)
- else:
- self.progress.emit(0)
- return current
-
- def HasMoreRecords(self):
- return self.more
-
- def columnCount(self, parent=None):
- return len(self.column_headers)
-
- def columnHeader(self, column):
- return self.column_headers[column]
-
- def SQLTableDataPrep(self, query, count):
- data = []
- for i in xrange(count):
- data.append(query.value(i))
- return data
-
-# SQL automatic table data model
-
-class SQLAutoTableModel(SQLTableModel):
-
- def __init__(self, glb, table_name, parent=None):
- sql = "SELECT * FROM " + table_name + " WHERE id > $$last_id$$ ORDER BY id LIMIT " + str(glb_chunk_sz)
- if table_name == "comm_threads_view":
- # For now, comm_threads_view has no id column
- sql = "SELECT * FROM " + table_name + " WHERE comm_id > $$last_id$$ ORDER BY comm_id LIMIT " + str(glb_chunk_sz)
- column_headers = []
- query = QSqlQuery(glb.db)
- if glb.dbref.is_sqlite3:
- QueryExec(query, "PRAGMA table_info(" + table_name + ")")
- while query.next():
- column_headers.append(query.value(1))
- if table_name == "sqlite_master":
- sql = "SELECT * FROM " + table_name
- else:
- if table_name[:19] == "information_schema.":
- sql = "SELECT * FROM " + table_name
- select_table_name = table_name[19:]
- schema = "information_schema"
- else:
- select_table_name = table_name
- schema = "public"
- QueryExec(query, "SELECT column_name FROM information_schema.columns WHERE table_schema = '" + schema + "' and table_name = '" + select_table_name + "'")
- while query.next():
- column_headers.append(query.value(0))
- if pyside_version_1 and sys.version_info[0] == 3:
- if table_name == "samples_view":
- self.SQLTableDataPrep = self.samples_view_DataPrep
- if table_name == "samples":
- self.SQLTableDataPrep = self.samples_DataPrep
- super(SQLAutoTableModel, self).__init__(glb, sql, column_headers, parent)
-
- def samples_view_DataPrep(self, query, count):
- data = []
- data.append(query.value(0))
- # Workaround pyside failing to handle large integers (i.e. time) in python3 by converting to a string
- data.append("{:>19}".format(query.value(1)))
- for i in xrange(2, count):
- data.append(query.value(i))
- return data
-
- def samples_DataPrep(self, query, count):
- data = []
- for i in xrange(9):
- data.append(query.value(i))
- # Workaround pyside failing to handle large integers (i.e. time) in python3 by converting to a string
- data.append("{:>19}".format(query.value(9)))
- for i in xrange(10, count):
- data.append(query.value(i))
- return data
-
-# Base class for custom ResizeColumnsToContents
-
-class ResizeColumnsToContentsBase(QObject):
-
- def __init__(self, parent=None):
- super(ResizeColumnsToContentsBase, self).__init__(parent)
-
- def ResizeColumnToContents(self, column, n):
- # Using the view's resizeColumnToContents() here is extrememly slow
- # so implement a crude alternative
- font = self.view.font()
- metrics = QFontMetrics(font)
- max = 0
- for row in xrange(n):
- val = self.data_model.child_items[row].data[column]
- len = metrics.width(str(val) + "MM")
- max = len if len > max else max
- val = self.data_model.columnHeader(column)
- len = metrics.width(str(val) + "MM")
- max = len if len > max else max
- self.view.setColumnWidth(column, max)
-
- def ResizeColumnsToContents(self):
- n = min(self.data_model.child_count, 100)
- if n < 1:
- # No data yet, so connect a signal to notify when there is
- self.data_model.rowsInserted.connect(self.UpdateColumnWidths)
- return
- columns = self.data_model.columnCount()
- for i in xrange(columns):
- self.ResizeColumnToContents(i, n)
-
- def UpdateColumnWidths(self, *x):
- # This only needs to be done once, so disconnect the signal now
- self.data_model.rowsInserted.disconnect(self.UpdateColumnWidths)
- self.ResizeColumnsToContents()
-
-# Convert value to CSV
-
-def ToCSValue(val):
- if '"' in val:
- val = val.replace('"', '""')
- if "," in val or '"' in val:
- val = '"' + val + '"'
- return val
-
-# Key to sort table model indexes by row / column, assuming fewer than 1000 columns
-
-glb_max_cols = 1000
-
-def RowColumnKey(a):
- return a.row() * glb_max_cols + a.column()
-
-# Copy selected table cells to clipboard
-
-def CopyTableCellsToClipboard(view, as_csv=False, with_hdr=False):
- indexes = sorted(view.selectedIndexes(), key=RowColumnKey)
- idx_cnt = len(indexes)
- if not idx_cnt:
- return
- if idx_cnt == 1:
- with_hdr=False
- min_row = indexes[0].row()
- max_row = indexes[0].row()
- min_col = indexes[0].column()
- max_col = indexes[0].column()
- for i in indexes:
- min_row = min(min_row, i.row())
- max_row = max(max_row, i.row())
- min_col = min(min_col, i.column())
- max_col = max(max_col, i.column())
- if max_col > glb_max_cols:
- raise RuntimeError("glb_max_cols is too low")
- max_width = [0] * (1 + max_col - min_col)
- for i in indexes:
- c = i.column() - min_col
- max_width[c] = max(max_width[c], len(str(i.data())))
- text = ""
- pad = ""
- sep = ""
- if with_hdr:
- model = indexes[0].model()
- for col in range(min_col, max_col + 1):
- val = model.headerData(col, Qt.Horizontal, Qt.DisplayRole)
- if as_csv:
- text += sep + ToCSValue(val)
- sep = ","
- else:
- c = col - min_col
- max_width[c] = max(max_width[c], len(val))
- width = max_width[c]
- align = model.headerData(col, Qt.Horizontal, Qt.TextAlignmentRole)
- if align & Qt.AlignRight:
- val = val.rjust(width)
- text += pad + sep + val
- pad = " " * (width - len(val))
- sep = " "
- text += "\n"
- pad = ""
- sep = ""
- last_row = min_row
- for i in indexes:
- if i.row() > last_row:
- last_row = i.row()
- text += "\n"
- pad = ""
- sep = ""
- if as_csv:
- text += sep + ToCSValue(str(i.data()))
- sep = ","
- else:
- width = max_width[i.column() - min_col]
- if i.data(Qt.TextAlignmentRole) & Qt.AlignRight:
- val = str(i.data()).rjust(width)
- else:
- val = str(i.data())
- text += pad + sep + val
- pad = " " * (width - len(val))
- sep = " "
- QApplication.clipboard().setText(text)
-
-def CopyTreeCellsToClipboard(view, as_csv=False, with_hdr=False):
- indexes = view.selectedIndexes()
- if not len(indexes):
- return
-
- selection = view.selectionModel()
-
- first = None
- for i in indexes:
- above = view.indexAbove(i)
- if not selection.isSelected(above):
- first = i
- break
-
- if first is None:
- raise RuntimeError("CopyTreeCellsToClipboard internal error")
-
- model = first.model()
- row_cnt = 0
- col_cnt = model.columnCount(first)
- max_width = [0] * col_cnt
-
- indent_sz = 2
- indent_str = " " * indent_sz
-
- expanded_mark_sz = 2
- if sys.version_info[0] == 3:
- expanded_mark = "\u25BC "
- not_expanded_mark = "\u25B6 "
- else:
- expanded_mark = unicode(chr(0xE2) + chr(0x96) + chr(0xBC) + " ", "utf-8")
- not_expanded_mark = unicode(chr(0xE2) + chr(0x96) + chr(0xB6) + " ", "utf-8")
- leaf_mark = " "
-
- if not as_csv:
- pos = first
- while True:
- row_cnt += 1
- row = pos.row()
- for c in range(col_cnt):
- i = pos.sibling(row, c)
- if c:
- n = len(str(i.data()))
- else:
- n = len(str(i.data()).strip())
- n += (i.internalPointer().level - 1) * indent_sz
- n += expanded_mark_sz
- max_width[c] = max(max_width[c], n)
- pos = view.indexBelow(pos)
- if not selection.isSelected(pos):
- break
-
- text = ""
- pad = ""
- sep = ""
- if with_hdr:
- for c in range(col_cnt):
- val = model.headerData(c, Qt.Horizontal, Qt.DisplayRole).strip()
- if as_csv:
- text += sep + ToCSValue(val)
- sep = ","
- else:
- max_width[c] = max(max_width[c], len(val))
- width = max_width[c]
- align = model.headerData(c, Qt.Horizontal, Qt.TextAlignmentRole)
- if align & Qt.AlignRight:
- val = val.rjust(width)
- text += pad + sep + val
- pad = " " * (width - len(val))
- sep = " "
- text += "\n"
- pad = ""
- sep = ""
-
- pos = first
- while True:
- row = pos.row()
- for c in range(col_cnt):
- i = pos.sibling(row, c)
- val = str(i.data())
- if not c:
- if model.hasChildren(i):
- if view.isExpanded(i):
- mark = expanded_mark
- else:
- mark = not_expanded_mark
- else:
- mark = leaf_mark
- val = indent_str * (i.internalPointer().level - 1) + mark + val.strip()
- if as_csv:
- text += sep + ToCSValue(val)
- sep = ","
- else:
- width = max_width[c]
- if c and i.data(Qt.TextAlignmentRole) & Qt.AlignRight:
- val = val.rjust(width)
- text += pad + sep + val
- pad = " " * (width - len(val))
- sep = " "
- pos = view.indexBelow(pos)
- if not selection.isSelected(pos):
- break
- text = text.rstrip() + "\n"
- pad = ""
- sep = ""
-
- QApplication.clipboard().setText(text)
-
-def CopyCellsToClipboard(view, as_csv=False, with_hdr=False):
- view.CopyCellsToClipboard(view, as_csv, with_hdr)
-
-def CopyCellsToClipboardHdr(view):
- CopyCellsToClipboard(view, False, True)
-
-def CopyCellsToClipboardCSV(view):
- CopyCellsToClipboard(view, True, True)
-
-# Context menu
-
-class ContextMenu(object):
-
- def __init__(self, view):
- self.view = view
- self.view.setContextMenuPolicy(Qt.CustomContextMenu)
- self.view.customContextMenuRequested.connect(self.ShowContextMenu)
-
- def ShowContextMenu(self, pos):
- menu = QMenu(self.view)
- self.AddActions(menu)
- menu.exec_(self.view.mapToGlobal(pos))
-
- def AddCopy(self, menu):
- menu.addAction(CreateAction("&Copy selection", "Copy to clipboard", lambda: CopyCellsToClipboardHdr(self.view), self.view))
- menu.addAction(CreateAction("Copy selection as CS&V", "Copy to clipboard as CSV", lambda: CopyCellsToClipboardCSV(self.view), self.view))
-
- def AddActions(self, menu):
- self.AddCopy(menu)
-
-class TreeContextMenu(ContextMenu):
-
- def __init__(self, view):
- super(TreeContextMenu, self).__init__(view)
-
- def AddActions(self, menu):
- i = self.view.currentIndex()
- text = str(i.data()).strip()
- if len(text):
- menu.addAction(CreateAction('Copy "' + text + '"', "Copy to clipboard", lambda: QApplication.clipboard().setText(text), self.view))
- self.AddCopy(menu)
-
-# Table window
-
-class TableWindow(QMdiSubWindow, ResizeColumnsToContentsBase):
-
- def __init__(self, glb, table_name, parent=None):
- super(TableWindow, self).__init__(parent)
-
- self.data_model = LookupCreateModel(table_name + " Table", lambda: SQLAutoTableModel(glb, table_name))
-
- self.model = QSortFilterProxyModel()
- self.model.setSourceModel(self.data_model)
-
- self.view = QTableView()
- self.view.setModel(self.model)
- self.view.setEditTriggers(QAbstractItemView.NoEditTriggers)
- self.view.verticalHeader().setVisible(False)
- self.view.sortByColumn(-1, Qt.AscendingOrder)
- self.view.setSortingEnabled(True)
- self.view.setSelectionMode(QAbstractItemView.ContiguousSelection)
- self.view.CopyCellsToClipboard = CopyTableCellsToClipboard
-
- self.ResizeColumnsToContents()
-
- self.context_menu = ContextMenu(self.view)
-
- self.find_bar = FindBar(self, self, True)
-
- self.finder = ChildDataItemFinder(self.data_model)
-
- self.fetch_bar = FetchMoreRecordsBar(self.data_model, self)
-
- self.vbox = VBox(self.view, self.find_bar.Widget(), self.fetch_bar.Widget())
-
- self.setWidget(self.vbox.Widget())
-
- AddSubWindow(glb.mainwindow.mdi_area, self, table_name + " Table")
-
- def Find(self, value, direction, pattern, context):
- self.view.setFocus()
- self.find_bar.Busy()
- self.finder.Find(value, direction, pattern, context, self.FindDone)
-
- def FindDone(self, row):
- self.find_bar.Idle()
- if row >= 0:
- self.view.setCurrentIndex(self.model.mapFromSource(self.data_model.index(row, 0, QModelIndex())))
- else:
- self.find_bar.NotFound()
-
-# Table list
-
-def GetTableList(glb):
- tables = []
- query = QSqlQuery(glb.db)
- if glb.dbref.is_sqlite3:
- QueryExec(query, "SELECT name FROM sqlite_master WHERE type IN ( 'table' , 'view' ) ORDER BY name")
- else:
- QueryExec(query, "SELECT table_name FROM information_schema.tables WHERE table_schema = 'public' AND table_type IN ( 'BASE TABLE' , 'VIEW' ) ORDER BY table_name")
- while query.next():
- tables.append(query.value(0))
- if glb.dbref.is_sqlite3:
- tables.append("sqlite_master")
- else:
- tables.append("information_schema.tables")
- tables.append("information_schema.views")
- tables.append("information_schema.columns")
- return tables
-
-# Top Calls data model
-
-class TopCallsModel(SQLTableModel):
-
- def __init__(self, glb, report_vars, parent=None):
- text = ""
- if not glb.dbref.is_sqlite3:
- text = "::text"
- limit = ""
- if len(report_vars.limit):
- limit = " LIMIT " + report_vars.limit
- sql = ("SELECT comm, pid, tid, name,"
- " CASE"
- " WHEN (short_name = '[kernel.kallsyms]') THEN '[kernel]'" + text +
- " ELSE short_name"
- " END AS dso,"
- " call_time, return_time, (return_time - call_time) AS elapsed_time, branch_count, "
- " CASE"
- " WHEN (calls.flags = 1) THEN 'no call'" + text +
- " WHEN (calls.flags = 2) THEN 'no return'" + text +
- " WHEN (calls.flags = 3) THEN 'no call/return'" + text +
- " ELSE ''" + text +
- " END AS flags"
- " FROM calls"
- " INNER JOIN call_paths ON calls.call_path_id = call_paths.id"
- " INNER JOIN symbols ON call_paths.symbol_id = symbols.id"
- " INNER JOIN dsos ON symbols.dso_id = dsos.id"
- " INNER JOIN comms ON calls.comm_id = comms.id"
- " INNER JOIN threads ON calls.thread_id = threads.id" +
- report_vars.where_clause +
- " ORDER BY elapsed_time DESC" +
- limit
- )
- column_headers = ("Command", "PID", "TID", "Symbol", "Object", "Call Time", "Return Time", "Elapsed Time (ns)", "Branch Count", "Flags")
- self.alignment = (Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignLeft, Qt.AlignRight, Qt.AlignRight, Qt.AlignLeft)
- super(TopCallsModel, self).__init__(glb, sql, column_headers, parent)
-
- def columnAlignment(self, column):
- return self.alignment[column]
-
-# Top Calls report creation dialog
-
-class TopCallsDialog(ReportDialogBase):
-
- def __init__(self, glb, parent=None):
- title = "Top Calls by Elapsed Time"
- items = (lambda g, p: LineEditDataItem(g, "Report name:", "Enter a name to appear in the window title bar", p, "REPORTNAME"),
- lambda g, p: SQLTableDataItem(g, "Commands:", "Only calls with these commands will be included", "comms", "comm", "comm_id", "", p),
- lambda g, p: SQLTableDataItem(g, "PIDs:", "Only calls with these process IDs will be included", "threads", "pid", "thread_id", "", p),
- lambda g, p: SQLTableDataItem(g, "TIDs:", "Only calls with these thread IDs will be included", "threads", "tid", "thread_id", "", p),
- lambda g, p: SQLTableDataItem(g, "DSOs:", "Only calls with these DSOs will be included", "dsos", "short_name", "dso_id", "", p),
- lambda g, p: SQLTableDataItem(g, "Symbols:", "Only calls with these symbols will be included", "symbols", "name", "symbol_id", "", p),
- lambda g, p: LineEditDataItem(g, "Raw SQL clause: ", "Enter a raw SQL WHERE clause", p),
- lambda g, p: PositiveIntegerDataItem(g, "Record limit:", "Limit selection to this number of records", p, "LIMIT", "100"))
- super(TopCallsDialog, self).__init__(glb, title, items, False, parent)
-
-# Top Calls window
-
-class TopCallsWindow(QMdiSubWindow, ResizeColumnsToContentsBase):
-
- def __init__(self, glb, report_vars, parent=None):
- super(TopCallsWindow, self).__init__(parent)
-
- self.data_model = LookupCreateModel("Top Calls " + report_vars.UniqueId(), lambda: TopCallsModel(glb, report_vars))
- self.model = self.data_model
-
- self.view = QTableView()
- self.view.setModel(self.model)
- self.view.setEditTriggers(QAbstractItemView.NoEditTriggers)
- self.view.verticalHeader().setVisible(False)
- self.view.setSelectionMode(QAbstractItemView.ContiguousSelection)
- self.view.CopyCellsToClipboard = CopyTableCellsToClipboard
-
- self.context_menu = ContextMenu(self.view)
-
- self.ResizeColumnsToContents()
-
- self.find_bar = FindBar(self, self, True)
-
- self.finder = ChildDataItemFinder(self.model)
-
- self.fetch_bar = FetchMoreRecordsBar(self.data_model, self)
-
- self.vbox = VBox(self.view, self.find_bar.Widget(), self.fetch_bar.Widget())
-
- self.setWidget(self.vbox.Widget())
-
- AddSubWindow(glb.mainwindow.mdi_area, self, report_vars.name)
-
- def Find(self, value, direction, pattern, context):
- self.view.setFocus()
- self.find_bar.Busy()
- self.finder.Find(value, direction, pattern, context, self.FindDone)
-
- def FindDone(self, row):
- self.find_bar.Idle()
- if row >= 0:
- self.view.setCurrentIndex(self.model.index(row, 0, QModelIndex()))
- else:
- self.find_bar.NotFound()
-
-# Action Definition
-
-def CreateAction(label, tip, callback, parent=None, shortcut=None):
- action = QAction(label, parent)
- if shortcut != None:
- action.setShortcuts(shortcut)
- action.setStatusTip(tip)
- action.triggered.connect(callback)
- return action
-
-# Typical application actions
-
-def CreateExitAction(app, parent=None):
- return CreateAction("&Quit", "Exit the application", app.closeAllWindows, parent, QKeySequence.Quit)
-
-# Typical MDI actions
-
-def CreateCloseActiveWindowAction(mdi_area):
- return CreateAction("Cl&ose", "Close the active window", mdi_area.closeActiveSubWindow, mdi_area)
-
-def CreateCloseAllWindowsAction(mdi_area):
- return CreateAction("Close &All", "Close all the windows", mdi_area.closeAllSubWindows, mdi_area)
-
-def CreateTileWindowsAction(mdi_area):
- return CreateAction("&Tile", "Tile the windows", mdi_area.tileSubWindows, mdi_area)
-
-def CreateCascadeWindowsAction(mdi_area):
- return CreateAction("&Cascade", "Cascade the windows", mdi_area.cascadeSubWindows, mdi_area)
-
-def CreateNextWindowAction(mdi_area):
- return CreateAction("Ne&xt", "Move the focus to the next window", mdi_area.activateNextSubWindow, mdi_area, QKeySequence.NextChild)
-
-def CreatePreviousWindowAction(mdi_area):
- return CreateAction("Pre&vious", "Move the focus to the previous window", mdi_area.activatePreviousSubWindow, mdi_area, QKeySequence.PreviousChild)
-
-# Typical MDI window menu
-
-class WindowMenu():
-
- def __init__(self, mdi_area, menu):
- self.mdi_area = mdi_area
- self.window_menu = menu.addMenu("&Windows")
- self.close_active_window = CreateCloseActiveWindowAction(mdi_area)
- self.close_all_windows = CreateCloseAllWindowsAction(mdi_area)
- self.tile_windows = CreateTileWindowsAction(mdi_area)
- self.cascade_windows = CreateCascadeWindowsAction(mdi_area)
- self.next_window = CreateNextWindowAction(mdi_area)
- self.previous_window = CreatePreviousWindowAction(mdi_area)
- self.window_menu.aboutToShow.connect(self.Update)
-
- def Update(self):
- self.window_menu.clear()
- sub_window_count = len(self.mdi_area.subWindowList())
- have_sub_windows = sub_window_count != 0
- self.close_active_window.setEnabled(have_sub_windows)
- self.close_all_windows.setEnabled(have_sub_windows)
- self.tile_windows.setEnabled(have_sub_windows)
- self.cascade_windows.setEnabled(have_sub_windows)
- self.next_window.setEnabled(have_sub_windows)
- self.previous_window.setEnabled(have_sub_windows)
- self.window_menu.addAction(self.close_active_window)
- self.window_menu.addAction(self.close_all_windows)
- self.window_menu.addSeparator()
- self.window_menu.addAction(self.tile_windows)
- self.window_menu.addAction(self.cascade_windows)
- self.window_menu.addSeparator()
- self.window_menu.addAction(self.next_window)
- self.window_menu.addAction(self.previous_window)
- if sub_window_count == 0:
- return
- self.window_menu.addSeparator()
- nr = 1
- for sub_window in self.mdi_area.subWindowList():
- label = str(nr) + " " + sub_window.name
- if nr < 10:
- label = "&" + label
- action = self.window_menu.addAction(label)
- action.setCheckable(True)
- action.setChecked(sub_window == self.mdi_area.activeSubWindow())
- action.triggered.connect(lambda a=None,x=nr: self.setActiveSubWindow(x))
- self.window_menu.addAction(action)
- nr += 1
-
- def setActiveSubWindow(self, nr):
- self.mdi_area.setActiveSubWindow(self.mdi_area.subWindowList()[nr - 1])
-
-# Help text
-
-glb_help_text = """
-<h1>Contents</h1>
-<style>
-p.c1 {
- text-indent: 40px;
-}
-p.c2 {
- text-indent: 80px;
-}
-}
-</style>
-<p class=c1><a href=#reports>1. Reports</a></p>
-<p class=c2><a href=#callgraph>1.1 Context-Sensitive Call Graph</a></p>
-<p class=c2><a href=#calltree>1.2 Call Tree</a></p>
-<p class=c2><a href=#allbranches>1.3 All branches</a></p>
-<p class=c2><a href=#selectedbranches>1.4 Selected branches</a></p>
-<p class=c2><a href=#topcallsbyelapsedtime>1.5 Top calls by elapsed time</a></p>
-<p class=c1><a href=#charts>2. Charts</a></p>
-<p class=c2><a href=#timechartbycpu>2.1 Time chart by CPU</a></p>
-<p class=c1><a href=#tables>3. Tables</a></p>
-<h1 id=reports>1. Reports</h1>
-<h2 id=callgraph>1.1 Context-Sensitive Call Graph</h2>
-The result is a GUI window with a tree representing a context-sensitive
-call-graph. Expanding a couple of levels of the tree and adjusting column
-widths to suit will display something like:
-<pre>
- Call Graph: pt_example
-Call Path Object Count Time(ns) Time(%) Branch Count Branch Count(%)
-v- ls
- v- 2638:2638
- v- _start ld-2.19.so 1 10074071 100.0 211135 100.0
- |- unknown unknown 1 13198 0.1 1 0.0
- >- _dl_start ld-2.19.so 1 1400980 13.9 19637 9.3
- >- _d_linit_internal ld-2.19.so 1 448152 4.4 11094 5.3
- v-__libc_start_main@plt ls 1 8211741 81.5 180397 85.4
- >- _dl_fixup ld-2.19.so 1 7607 0.1 108 0.1
- >- __cxa_atexit libc-2.19.so 1 11737 0.1 10 0.0
- >- __libc_csu_init ls 1 10354 0.1 10 0.0
- |- _setjmp libc-2.19.so 1 0 0.0 4 0.0
- v- main ls 1 8182043 99.6 180254 99.9
-</pre>
-<h3>Points to note:</h3>
-<ul>
-<li>The top level is a command name (comm)</li>
-<li>The next level is a thread (pid:tid)</li>
-<li>Subsequent levels are functions</li>
-<li>'Count' is the number of calls</li>
-<li>'Time' is the elapsed time until the function returns</li>
-<li>Percentages are relative to the level above</li>
-<li>'Branch Count' is the total number of branches for that function and all functions that it calls
-</ul>
-<h3>Find</h3>
-Ctrl-F displays a Find bar which finds function names by either an exact match or a pattern match.
-The pattern matching symbols are ? for any character and * for zero or more characters.
-<h2 id=calltree>1.2 Call Tree</h2>
-The Call Tree report is very similar to the Context-Sensitive Call Graph, but the data is not aggregated.
-Also the 'Count' column, which would be always 1, is replaced by the 'Call Time'.
-<h2 id=allbranches>1.3 All branches</h2>
-The All branches report displays all branches in chronological order.
-Not all data is fetched immediately. More records can be fetched using the Fetch bar provided.
-<h3>Disassembly</h3>
-Open a branch to display disassembly. This only works if:
-<ol>
-<li>The disassembler is available. Currently, only Intel XED is supported - see <a href=#xed>Intel XED Setup</a></li>
-<li>The object code is available. Currently, only the perf build ID cache is searched for object code.
-The default directory ~/.debug can be overridden by setting environment variable PERF_BUILDID_DIR.
-One exception is kcore where the DSO long name is used (refer dsos_view on the Tables menu),
-or alternatively, set environment variable PERF_KCORE to the kcore file name.</li>
-</ol>
-<h4 id=xed>Intel XED Setup</h4>
-To use Intel XED, libxed.so must be present. To build and install libxed.so:
-<pre>
-git clone https://github.com/intelxed/mbuild.git mbuild
-git clone https://github.com/intelxed/xed
-cd xed
-./mfile.py --share
-sudo ./mfile.py --prefix=/usr/local install
-sudo ldconfig
-</pre>
-<h3>Instructions per Cycle (IPC)</h3>
-If available, IPC information is displayed in columns 'insn_cnt', 'cyc_cnt' and 'IPC'.
-<p><b>Intel PT note:</b> The information applies to the blocks of code ending with, and including, that branch.
-Due to the granularity of timing information, the number of cycles for some code blocks will not be known.
-In that case, 'insn_cnt', 'cyc_cnt' and 'IPC' are zero, but when 'IPC' is displayed it covers the period
-since the previous displayed 'IPC'.
-<h3>Find</h3>
-Ctrl-F displays a Find bar which finds substrings by either an exact match or a regular expression match.
-Refer to Python documentation for the regular expression syntax.
-All columns are searched, but only currently fetched rows are searched.
-<h2 id=selectedbranches>1.4 Selected branches</h2>
-This is the same as the <a href=#allbranches>All branches</a> report but with the data reduced
-by various selection criteria. A dialog box displays available criteria which are AND'ed together.
-<h3>1.4.1 Time ranges</h3>
-The time ranges hint text shows the total time range. Relative time ranges can also be entered in
-ms, us or ns. Also, negative values are relative to the end of trace. Examples:
-<pre>
- 81073085947329-81073085958238 From 81073085947329 to 81073085958238
- 100us-200us From 100us to 200us
- 10ms- From 10ms to the end
- -100ns The first 100ns
- -10ms- The last 10ms
-</pre>
-N.B. Due to the granularity of timestamps, there could be no branches in any given time range.
-<h2 id=topcallsbyelapsedtime>1.5 Top calls by elapsed time</h2>
-The Top calls by elapsed time report displays calls in descending order of time elapsed between when the function was called and when it returned.
-The data is reduced by various selection criteria. A dialog box displays available criteria which are AND'ed together.
-If not all data is fetched, a Fetch bar is provided. Ctrl-F displays a Find bar.
-<h1 id=charts>2. Charts</h1>
-<h2 id=timechartbycpu>2.1 Time chart by CPU</h2>
-This chart displays context switch information when that data is available. Refer to context_switches_view on the Tables menu.
-<h3>Features</h3>
-<ol>
-<li>Mouse over to highight the task and show the time</li>
-<li>Drag the mouse to select a region and zoom by pushing the Zoom button</li>
-<li>Go back and forward by pressing the arrow buttons</li>
-<li>If call information is available, right-click to show a call tree opened to that task and time.
-Note, the call tree may take some time to appear, and there may not be call information for the task or time selected.
-</li>
-</ol>
-<h3>Important</h3>
-The graph can be misleading in the following respects:
-<ol>
-<li>The graph shows the first task on each CPU as running from the beginning of the time range.
-Because tracing might start on different CPUs at different times, that is not necessarily the case.
-Refer to context_switches_view on the Tables menu to understand what data the graph is based upon.</li>
-<li>Similarly, the last task on each CPU can be showing running longer than it really was.
-Again, refer to context_switches_view on the Tables menu to understand what data the graph is based upon.</li>
-<li>When the mouse is over a task, the highlighted task might not be visible on the legend without scrolling if the legend does not fit fully in the window</li>
-</ol>
-<h1 id=tables>3. Tables</h1>
-The Tables menu shows all tables and views in the database. Most tables have an associated view
-which displays the information in a more friendly way. Not all data for large tables is fetched
-immediately. More records can be fetched using the Fetch bar provided. Columns can be sorted,
-but that can be slow for large tables.
-<p>There are also tables of database meta-information.
-For SQLite3 databases, the sqlite_master table is included.
-For PostgreSQL databases, information_schema.tables/views/columns are included.
-<h3>Find</h3>
-Ctrl-F displays a Find bar which finds substrings by either an exact match or a regular expression match.
-Refer to Python documentation for the regular expression syntax.
-All columns are searched, but only currently fetched rows are searched.
-<p>N.B. Results are found in id order, so if the table is re-ordered, find-next and find-previous
-will go to the next/previous result in id order, instead of display order.
-"""
-
-# Help window
-
-class HelpWindow(QMdiSubWindow):
-
- def __init__(self, glb, parent=None):
- super(HelpWindow, self).__init__(parent)
-
- self.text = QTextBrowser()
- self.text.setHtml(glb_help_text)
- self.text.setReadOnly(True)
- self.text.setOpenExternalLinks(True)
-
- self.setWidget(self.text)
-
- AddSubWindow(glb.mainwindow.mdi_area, self, "Exported SQL Viewer Help")
-
-# Main window that only displays the help text
-
-class HelpOnlyWindow(QMainWindow):
-
- def __init__(self, parent=None):
- super(HelpOnlyWindow, self).__init__(parent)
-
- self.setMinimumSize(200, 100)
- self.resize(800, 600)
- self.setWindowTitle("Exported SQL Viewer Help")
- self.setWindowIcon(self.style().standardIcon(QStyle.SP_MessageBoxInformation))
-
- self.text = QTextBrowser()
- self.text.setHtml(glb_help_text)
- self.text.setReadOnly(True)
- self.text.setOpenExternalLinks(True)
-
- self.setCentralWidget(self.text)
-
-# PostqreSQL server version
-
-def PostqreSQLServerVersion(db):
- query = QSqlQuery(db)
- QueryExec(query, "SELECT VERSION()")
- if query.next():
- v_str = query.value(0)
- v_list = v_str.strip().split(" ")
- if v_list[0] == "PostgreSQL" and v_list[2] == "on":
- return v_list[1]
- return v_str
- return "Unknown"
-
-# SQLite version
-
-def SQLiteVersion(db):
- query = QSqlQuery(db)
- QueryExec(query, "SELECT sqlite_version()")
- if query.next():
- return query.value(0)
- return "Unknown"
-
-# About dialog
-
-class AboutDialog(QDialog):
-
- def __init__(self, glb, parent=None):
- super(AboutDialog, self).__init__(parent)
-
- self.setWindowTitle("About Exported SQL Viewer")
- self.setMinimumWidth(300)
-
- pyside_version = "1" if pyside_version_1 else "2"
-
- text = "<pre>"
- text += "Python version: " + sys.version.split(" ")[0] + "\n"
- text += "PySide version: " + pyside_version + "\n"
- text += "Qt version: " + qVersion() + "\n"
- if glb.dbref.is_sqlite3:
- text += "SQLite version: " + SQLiteVersion(glb.db) + "\n"
- else:
- text += "PostqreSQL version: " + PostqreSQLServerVersion(glb.db) + "\n"
- text += "</pre>"
-
- self.text = QTextBrowser()
- self.text.setHtml(text)
- self.text.setReadOnly(True)
- self.text.setOpenExternalLinks(True)
-
- self.vbox = QVBoxLayout()
- self.vbox.addWidget(self.text)
-
- self.setLayout(self.vbox)
-
-# Font resize
-
-def ResizeFont(widget, diff):
- font = widget.font()
- sz = font.pointSize()
- font.setPointSize(sz + diff)
- widget.setFont(font)
-
-def ShrinkFont(widget):
- ResizeFont(widget, -1)
-
-def EnlargeFont(widget):
- ResizeFont(widget, 1)
-
-# Unique name for sub-windows
-
-def NumberedWindowName(name, nr):
- if nr > 1:
- name += " <" + str(nr) + ">"
- return name
-
-def UniqueSubWindowName(mdi_area, name):
- nr = 1
- while True:
- unique_name = NumberedWindowName(name, nr)
- ok = True
- for sub_window in mdi_area.subWindowList():
- if sub_window.name == unique_name:
- ok = False
- break
- if ok:
- return unique_name
- nr += 1
-
-# Add a sub-window
-
-def AddSubWindow(mdi_area, sub_window, name):
- unique_name = UniqueSubWindowName(mdi_area, name)
- sub_window.setMinimumSize(200, 100)
- sub_window.resize(800, 600)
- sub_window.setWindowTitle(unique_name)
- sub_window.setAttribute(Qt.WA_DeleteOnClose)
- sub_window.setWindowIcon(sub_window.style().standardIcon(QStyle.SP_FileIcon))
- sub_window.name = unique_name
- mdi_area.addSubWindow(sub_window)
- sub_window.show()
-
-# Main window
-
-class MainWindow(QMainWindow):
-
- def __init__(self, glb, parent=None):
- super(MainWindow, self).__init__(parent)
-
- self.glb = glb
-
- self.setWindowTitle("Exported SQL Viewer: " + glb.dbname)
- self.setWindowIcon(self.style().standardIcon(QStyle.SP_ComputerIcon))
- self.setMinimumSize(200, 100)
-
- self.mdi_area = QMdiArea()
- self.mdi_area.setHorizontalScrollBarPolicy(Qt.ScrollBarAsNeeded)
- self.mdi_area.setVerticalScrollBarPolicy(Qt.ScrollBarAsNeeded)
-
- self.setCentralWidget(self.mdi_area)
-
- menu = self.menuBar()
-
- file_menu = menu.addMenu("&File")
- file_menu.addAction(CreateExitAction(glb.app, self))
-
- edit_menu = menu.addMenu("&Edit")
- edit_menu.addAction(CreateAction("&Copy", "Copy to clipboard", self.CopyToClipboard, self, QKeySequence.Copy))
- edit_menu.addAction(CreateAction("Copy as CS&V", "Copy to clipboard as CSV", self.CopyToClipboardCSV, self))
- edit_menu.addAction(CreateAction("&Find...", "Find items", self.Find, self, QKeySequence.Find))
- edit_menu.addAction(CreateAction("Fetch &more records...", "Fetch more records", self.FetchMoreRecords, self, [QKeySequence(Qt.Key_F8)]))
- edit_menu.addAction(CreateAction("&Shrink Font", "Make text smaller", self.ShrinkFont, self, [QKeySequence("Ctrl+-")]))
- edit_menu.addAction(CreateAction("&Enlarge Font", "Make text bigger", self.EnlargeFont, self, [QKeySequence("Ctrl++")]))
-
- reports_menu = menu.addMenu("&Reports")
- if IsSelectable(glb.db, "calls"):
- reports_menu.addAction(CreateAction("Context-Sensitive Call &Graph", "Create a new window containing a context-sensitive call graph", self.NewCallGraph, self))
-
- if IsSelectable(glb.db, "calls", "WHERE parent_id >= 0"):
- reports_menu.addAction(CreateAction("Call &Tree", "Create a new window containing a call tree", self.NewCallTree, self))
-
- self.EventMenu(GetEventList(glb.db), reports_menu)
-
- if IsSelectable(glb.db, "calls"):
- reports_menu.addAction(CreateAction("&Top calls by elapsed time", "Create a new window displaying top calls by elapsed time", self.NewTopCalls, self))
-
- if IsSelectable(glb.db, "context_switches"):
- charts_menu = menu.addMenu("&Charts")
- charts_menu.addAction(CreateAction("&Time chart by CPU", "Create a new window displaying time charts by CPU", self.TimeChartByCPU, self))
-
- self.TableMenu(GetTableList(glb), menu)
-
- self.window_menu = WindowMenu(self.mdi_area, menu)
-
- help_menu = menu.addMenu("&Help")
- help_menu.addAction(CreateAction("&Exported SQL Viewer Help", "Helpful information", self.Help, self, QKeySequence.HelpContents))
- help_menu.addAction(CreateAction("&About Exported SQL Viewer", "About this application", self.About, self))
-
- def Try(self, fn):
- win = self.mdi_area.activeSubWindow()
- if win:
- try:
- fn(win.view)
- except:
- pass
-
- def CopyToClipboard(self):
- self.Try(CopyCellsToClipboardHdr)
-
- def CopyToClipboardCSV(self):
- self.Try(CopyCellsToClipboardCSV)
-
- def Find(self):
- win = self.mdi_area.activeSubWindow()
- if win:
- try:
- win.find_bar.Activate()
- except:
- pass
-
- def FetchMoreRecords(self):
- win = self.mdi_area.activeSubWindow()
- if win:
- try:
- win.fetch_bar.Activate()
- except:
- pass
-
- def ShrinkFont(self):
- self.Try(ShrinkFont)
-
- def EnlargeFont(self):
- self.Try(EnlargeFont)
-
- def EventMenu(self, events, reports_menu):
- branches_events = 0
- for event in events:
- event = event.split(":")[0]
- if event == "branches":
- branches_events += 1
- dbid = 0
- for event in events:
- dbid += 1
- event = event.split(":")[0]
- if event == "branches":
- label = "All branches" if branches_events == 1 else "All branches " + "(id=" + dbid + ")"
- reports_menu.addAction(CreateAction(label, "Create a new window displaying branch events", lambda a=None,x=dbid: self.NewBranchView(x), self))
- label = "Selected branches" if branches_events == 1 else "Selected branches " + "(id=" + dbid + ")"
- reports_menu.addAction(CreateAction(label, "Create a new window displaying branch events", lambda a=None,x=dbid: self.NewSelectedBranchView(x), self))
-
- def TimeChartByCPU(self):
- TimeChartByCPUWindow(self.glb, self)
-
- def TableMenu(self, tables, menu):
- table_menu = menu.addMenu("&Tables")
- for table in tables:
- table_menu.addAction(CreateAction(table, "Create a new window containing a table view", lambda a=None,t=table: self.NewTableView(t), self))
-
- def NewCallGraph(self):
- CallGraphWindow(self.glb, self)
-
- def NewCallTree(self):
- CallTreeWindow(self.glb, self)
-
- def NewTopCalls(self):
- dialog = TopCallsDialog(self.glb, self)
- ret = dialog.exec_()
- if ret:
- TopCallsWindow(self.glb, dialog.report_vars, self)
-
- def NewBranchView(self, event_id):
- BranchWindow(self.glb, event_id, ReportVars(), self)
-
- def NewSelectedBranchView(self, event_id):
- dialog = SelectedBranchDialog(self.glb, self)
- ret = dialog.exec_()
- if ret:
- BranchWindow(self.glb, event_id, dialog.report_vars, self)
-
- def NewTableView(self, table_name):
- TableWindow(self.glb, table_name, self)
-
- def Help(self):
- HelpWindow(self.glb, self)
-
- def About(self):
- dialog = AboutDialog(self.glb, self)
- dialog.exec_()
-
-def TryOpen(file_name):
- try:
- return open(file_name, "rb")
- except:
- return None
-
-def Is64Bit(f):
- result = sizeof(c_void_p)
- # ELF support only
- pos = f.tell()
- f.seek(0)
- header = f.read(7)
- f.seek(pos)
- magic = header[0:4]
- if sys.version_info[0] == 2:
- eclass = ord(header[4])
- encoding = ord(header[5])
- version = ord(header[6])
- else:
- eclass = header[4]
- encoding = header[5]
- version = header[6]
- if magic == chr(127) + "ELF" and eclass > 0 and eclass < 3 and encoding > 0 and encoding < 3 and version == 1:
- result = True if eclass == 2 else False
- return result
-
-# Global data
-
-class Glb():
-
- def __init__(self, dbref, db, dbname):
- self.dbref = dbref
- self.db = db
- self.dbname = dbname
- self.home_dir = os.path.expanduser("~")
- self.buildid_dir = os.getenv("PERF_BUILDID_DIR")
- if self.buildid_dir:
- self.buildid_dir += "/.build-id/"
- else:
- self.buildid_dir = self.home_dir + "/.debug/.build-id/"
- self.app = None
- self.mainwindow = None
- self.instances_to_shutdown_on_exit = weakref.WeakSet()
- try:
- self.disassembler = LibXED()
- self.have_disassembler = True
- except:
- self.have_disassembler = False
- self.host_machine_id = 0
- self.host_start_time = 0
- self.host_finish_time = 0
-
- def FileFromBuildId(self, build_id):
- file_name = self.buildid_dir + build_id[0:2] + "/" + build_id[2:] + "/elf"
- return TryOpen(file_name)
-
- def FileFromNamesAndBuildId(self, short_name, long_name, build_id):
- # Assume current machine i.e. no support for virtualization
- if short_name[0:7] == "[kernel" and os.path.basename(long_name) == "kcore":
- file_name = os.getenv("PERF_KCORE")
- f = TryOpen(file_name) if file_name else None
- if f:
- return f
- # For now, no special handling if long_name is /proc/kcore
- f = TryOpen(long_name)
- if f:
- return f
- f = self.FileFromBuildId(build_id)
- if f:
- return f
- return None
-
- def AddInstanceToShutdownOnExit(self, instance):
- self.instances_to_shutdown_on_exit.add(instance)
-
- # Shutdown any background processes or threads
- def ShutdownInstances(self):
- for x in self.instances_to_shutdown_on_exit:
- try:
- x.Shutdown()
- except:
- pass
-
- def GetHostMachineId(self):
- query = QSqlQuery(self.db)
- QueryExec(query, "SELECT id FROM machines WHERE pid = -1")
- if query.next():
- self.host_machine_id = query.value(0)
- else:
- self.host_machine_id = 0
- return self.host_machine_id
-
- def HostMachineId(self):
- if self.host_machine_id:
- return self.host_machine_id
- return self.GetHostMachineId()
-
- def SelectValue(self, sql):
- query = QSqlQuery(self.db)
- try:
- QueryExec(query, sql)
- except:
- return None
- if query.next():
- return Decimal(query.value(0))
- return None
-
- def SwitchesMinTime(self, machine_id):
- return self.SelectValue("SELECT time"
- " FROM context_switches"
- " WHERE time != 0 AND machine_id = " + str(machine_id) +
- " ORDER BY id LIMIT 1")
-
- def SwitchesMaxTime(self, machine_id):
- return self.SelectValue("SELECT time"
- " FROM context_switches"
- " WHERE time != 0 AND machine_id = " + str(machine_id) +
- " ORDER BY id DESC LIMIT 1")
-
- def SamplesMinTime(self, machine_id):
- return self.SelectValue("SELECT time"
- " FROM samples"
- " WHERE time != 0 AND machine_id = " + str(machine_id) +
- " ORDER BY id LIMIT 1")
-
- def SamplesMaxTime(self, machine_id):
- return self.SelectValue("SELECT time"
- " FROM samples"
- " WHERE time != 0 AND machine_id = " + str(machine_id) +
- " ORDER BY id DESC LIMIT 1")
-
- def CallsMinTime(self, machine_id):
- return self.SelectValue("SELECT calls.call_time"
- " FROM calls"
- " INNER JOIN threads ON threads.thread_id = calls.thread_id"
- " WHERE calls.call_time != 0 AND threads.machine_id = " + str(machine_id) +
- " ORDER BY calls.id LIMIT 1")
-
- def CallsMaxTime(self, machine_id):
- return self.SelectValue("SELECT calls.return_time"
- " FROM calls"
- " INNER JOIN threads ON threads.thread_id = calls.thread_id"
- " WHERE calls.return_time != 0 AND threads.machine_id = " + str(machine_id) +
- " ORDER BY calls.return_time DESC LIMIT 1")
-
- def GetStartTime(self, machine_id):
- t0 = self.SwitchesMinTime(machine_id)
- t1 = self.SamplesMinTime(machine_id)
- t2 = self.CallsMinTime(machine_id)
- if t0 is None or (not(t1 is None) and t1 < t0):
- t0 = t1
- if t0 is None or (not(t2 is None) and t2 < t0):
- t0 = t2
- return t0
-
- def GetFinishTime(self, machine_id):
- t0 = self.SwitchesMaxTime(machine_id)
- t1 = self.SamplesMaxTime(machine_id)
- t2 = self.CallsMaxTime(machine_id)
- if t0 is None or (not(t1 is None) and t1 > t0):
- t0 = t1
- if t0 is None or (not(t2 is None) and t2 > t0):
- t0 = t2
- return t0
-
- def HostStartTime(self):
- if self.host_start_time:
- return self.host_start_time
- self.host_start_time = self.GetStartTime(self.HostMachineId())
- return self.host_start_time
-
- def HostFinishTime(self):
- if self.host_finish_time:
- return self.host_finish_time
- self.host_finish_time = self.GetFinishTime(self.HostMachineId())
- return self.host_finish_time
-
- def StartTime(self, machine_id):
- if machine_id == self.HostMachineId():
- return self.HostStartTime()
- return self.GetStartTime(machine_id)
-
- def FinishTime(self, machine_id):
- if machine_id == self.HostMachineId():
- return self.HostFinishTime()
- return self.GetFinishTime(machine_id)
-
-# Database reference
-
-class DBRef():
-
- def __init__(self, is_sqlite3, dbname):
- self.is_sqlite3 = is_sqlite3
- self.dbname = dbname
- self.TRUE = "TRUE"
- self.FALSE = "FALSE"
- # SQLite prior to version 3.23 does not support TRUE and FALSE
- if self.is_sqlite3:
- self.TRUE = "1"
- self.FALSE = "0"
-
- def Open(self, connection_name):
- dbname = self.dbname
- if self.is_sqlite3:
- db = QSqlDatabase.addDatabase("QSQLITE", connection_name)
- else:
- db = QSqlDatabase.addDatabase("QPSQL", connection_name)
- opts = dbname.split()
- for opt in opts:
- if "=" in opt:
- opt = opt.split("=")
- if opt[0] == "hostname":
- db.setHostName(opt[1])
- elif opt[0] == "port":
- db.setPort(int(opt[1]))
- elif opt[0] == "username":
- db.setUserName(opt[1])
- elif opt[0] == "password":
- db.setPassword(opt[1])
- elif opt[0] == "dbname":
- dbname = opt[1]
- else:
- dbname = opt
-
- db.setDatabaseName(dbname)
- if not db.open():
- raise Exception("Failed to open database " + dbname + " error: " + db.lastError().text())
- return db, dbname
-
-# Main
-
-def Main():
- usage_str = "exported-sql-viewer.py [--pyside-version-1] <database name>\n" \
- " or: exported-sql-viewer.py --help-only"
- ap = argparse.ArgumentParser(usage = usage_str, add_help = False)
- ap.add_argument("--pyside-version-1", action='store_true')
- ap.add_argument("dbname", nargs="?")
- ap.add_argument("--help-only", action='store_true')
- args = ap.parse_args()
-
- if args.help_only:
- app = QApplication(sys.argv)
- mainwindow = HelpOnlyWindow()
- mainwindow.show()
- err = app.exec_()
- sys.exit(err)
-
- dbname = args.dbname
- if dbname is None:
- ap.print_usage()
- print("Too few arguments")
- sys.exit(1)
-
- is_sqlite3 = False
- try:
- f = open(dbname, "rb")
- if f.read(15) == b'SQLite format 3':
- is_sqlite3 = True
- f.close()
- except:
- pass
-
- dbref = DBRef(is_sqlite3, dbname)
- db, dbname = dbref.Open("main")
- glb = Glb(dbref, db, dbname)
- app = QApplication(sys.argv)
- glb.app = app
- mainwindow = MainWindow(glb)
- glb.mainwindow = mainwindow
- mainwindow.show()
- err = app.exec_()
- glb.ShutdownInstances()
- db.close()
- sys.exit(err)
-
-if __name__ == "__main__":
- Main()
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 45/49] perf python: Move and clean up parallel-perf.py
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (43 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 44/49] perf python: Move and clean up exported-sql-viewer.py Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 46/49] perf: Remove libpython support and legacy Python scripts Ian Rogers
` (3 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Move parallel-perf.py from tools/perf/scripts/python/ to
tools/perf/python/ as it is a standalone Python utility that invokes
'perf script' in parallel across time slices and CPUs. Update
tools/perf/tests/shell/script.sh accordingly.
Also fix bugs and clean up the script to pass mypy and pylint without
suppression comments:
- Fix Work.command() to return the shlex.quote()-escaped command string
(previously sh_cmd was computed with shlex.quote() and discarded in
favor of unquoted self.cmd).
- Close self.popen.stdout in the parent process after spawning the
consumer subprocess in Work.start() when --pipe-to is used, ensuring
the producer receives SIGPIPE if the consumer exits early and avoiding
leaking the pipe file descriptor.
- Convert method and function names to snake_case, specify utf-8 file
encodings, and add type annotations and docstrings.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/python/parallel-perf.py | 1250 ++++++++++++++++++++
tools/perf/scripts/python/parallel-perf.py | 989 ----------------
tools/perf/tests/shell/script.sh | 4 +-
3 files changed, 1252 insertions(+), 991 deletions(-)
create mode 100755 tools/perf/python/parallel-perf.py
delete mode 100755 tools/perf/scripts/python/parallel-perf.py
diff --git a/tools/perf/python/parallel-perf.py b/tools/perf/python/parallel-perf.py
new file mode 100755
index 000000000000..c545933e1cdd
--- /dev/null
+++ b/tools/perf/python/parallel-perf.py
@@ -0,0 +1,1250 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""Parallel perf script."""
+#
+# run a perf script command multiple times in parallel, using perf script
+# options --cpu and --time so that each job processes a different chunk
+# of the data.
+#
+# Copyright (c) 2024, Intel Corporation.
+
+import subprocess
+import argparse
+import pathlib
+import shlex
+import time
+from typing import Any
+import copy
+import sys
+import os
+import re
+
+glb_prog_name = "parallel-perf.py"
+glb_min_interval = 10.0
+glb_min_samples = 64
+
+
+class Verbosity():
+
+ def __init__(self, quiet: bool = False, verbose: bool = False,
+ debug: bool = False) -> None:
+ """__init__."""
+
+ self.normal = True
+ self.verbose = verbose
+ self.debug = debug
+ self.self_test = True
+ if self.debug:
+ self.verbose = True
+ if self.verbose:
+ quiet = False
+ if quiet:
+ self.normal = False
+
+# Manage work (start/wait/kill), as represented by a subprocess.Popen command
+
+
+class Work():
+
+ def __init__(self, cmd: list[str], pipe_to: str,
+ output_dir: str = ".") -> None:
+ """__init__."""
+
+ self.popen: Any = None
+ self.consumer: Any = None
+ self.cmd = cmd
+ self.pipe_to = pipe_to
+ self.output_dir = output_dir
+ self.cmdout_name = f"{output_dir}/cmd.txt"
+ self.stdout_name = f"{output_dir}/out.txt"
+ self.stderr_name = f"{output_dir}/err.txt"
+
+ def command(self):
+ """command."""
+
+ return " ".join(shlex.quote(x) for x in self.cmd)
+
+ def stdout(self):
+ """stdout."""
+
+ return open(self.stdout_name, "w", encoding="utf-8")
+
+ def stderr(self):
+ """stderr."""
+
+ return open(self.stderr_name, "w", encoding="utf-8")
+
+ def create_output_dir(self):
+ """create_output_dir."""
+
+ pathlib.Path(self.output_dir).mkdir(parents=True, exist_ok=True)
+
+ def start(self):
+ """start."""
+
+ if self.popen:
+ return
+ self.create_output_dir()
+ with open(self.cmdout_name, "w", encoding="utf-8") as f:
+ f.write(self.command())
+ f.write("\n")
+ stdout = self.stdout()
+ stderr = self.stderr()
+ if self.pipe_to:
+ self.popen = subprocess.Popen(
+ self.cmd, stdout=subprocess.PIPE, stderr=stderr)
+ args = shlex.split(self.pipe_to)
+ self.consumer = subprocess.Popen(
+ args, stdin=self.popen.stdout, stdout=stdout, stderr=stderr)
+ # The consumer now owns the read end of the pipe. Close this
+ # process's copy so that the producer receives SIGPIPE if the
+ # consumer exits early, and so that the descriptor isn't leaked
+ # for the lifetime of this Work.
+ self.popen.stdout.close()
+ self.popen.stdout = None
+ else:
+ self.popen = subprocess.Popen(
+ self.cmd, stdout=stdout, stderr=stderr)
+
+ def remove_empty_err_file(self):
+ """remove_empty_err_file."""
+
+ if os.path.exists(self.stderr_name):
+ if os.path.getsize(self.stderr_name) == 0:
+ os.unlink(self.stderr_name)
+
+ def errors(self):
+ """errors."""
+
+ if os.path.exists(self.stderr_name):
+ if os.path.getsize(self.stderr_name) != 0:
+ return [f"Non-empty error file {self.stderr_name}"]
+ return []
+
+ def tidy_up(self):
+ """tidy_up."""
+
+ self.remove_empty_err_file()
+
+ def raw_poll_wait(self, p, wait):
+ """raw_poll_wait."""
+
+ if wait:
+ return p.wait()
+ return p.poll()
+
+ def poll(self, wait=False):
+ """poll."""
+
+ if not self.popen:
+ return None
+ result = self.raw_poll_wait(self.popen, wait)
+ if self.consumer:
+ res = result
+ result = self.raw_poll_wait(self.consumer, wait)
+ if result is not None and res is None:
+ self.popen.kill()
+ result = None
+ elif result == 0 and res is not None and res != 0:
+ result = res
+ if result is not None:
+ self.tidy_up()
+ return result
+
+ def wait(self):
+ """wait."""
+
+ return self.poll(wait=True)
+
+ def kill(self):
+ """kill."""
+
+ if not self.popen:
+ return
+ self.popen.kill()
+ if self.consumer:
+ self.consumer.kill()
+
+
+def kill_work(worklist, _verbosity):
+ """kill_work."""
+
+ for w in worklist:
+ w.kill()
+ for w in worklist:
+ w.wait()
+
+
+def number_of_cp_us():
+ """number_of_cp_us."""
+
+ return os.sysconf("SC_NPROCESSORS_ONLN")
+
+
+def nano_secs_to_secs_str(x):
+ """nano_secs_to_secs_str."""
+
+ if x is None:
+ return ""
+ x = str(x)
+ if len(x) < 10:
+ x = "0" * (10 - len(x)) + x
+ return x[:len(x) - 9] + "." + x[-9:]
+
+
+def insert_option_after(cmd, option, after):
+ """insert_option_after."""
+
+ try:
+ pos = cmd.index(after)
+ cmd.insert(pos + 1, option)
+ except (OSError, ValueError, RuntimeError):
+ cmd.append(option)
+
+
+def create_work_list(cmd, pipe_to, output_dir, cpus, time_ranges_by_cpu):
+ """create_work_list."""
+
+ max_len = len(str(cpus[-1]))
+ cpu_dir_fmt = f"cpu-%.{max_len}u"
+ worklist = []
+ pos = 0
+ for cpu in cpus:
+ if cpu >= 0:
+ cpu_dir = os.path.join(output_dir, cpu_dir_fmt % cpu)
+ cpu_option = f"--cpu={cpu}"
+ else:
+ cpu_dir = output_dir
+ cpu_option = None
+
+ tr_dir_fmt = "time-range"
+
+ if len(time_ranges_by_cpu) > 1:
+ time_ranges = time_ranges_by_cpu[pos]
+ tr_dir_fmt += f"-{pos}"
+ pos += 1
+ else:
+ time_ranges = time_ranges_by_cpu[0]
+
+ max_len = len(str(len(time_ranges)))
+ tr_dir_fmt += f"-%.{max_len}u"
+
+ i = 0
+ for r in time_ranges:
+ if r == [None, None]:
+ time_option = None
+ work_output_dir = cpu_dir
+ else:
+ time_option = "--time=" + \
+ nano_secs_to_secs_str(r[0]) + "," + \
+ nano_secs_to_secs_str(r[1])
+ work_output_dir = os.path.join(cpu_dir, tr_dir_fmt % i)
+ i += 1
+ work_cmd = list(cmd)
+ if time_option is not None:
+ insert_option_after(work_cmd, time_option, "script")
+ if cpu_option is not None:
+ insert_option_after(work_cmd, cpu_option, "script")
+ w = Work(work_cmd, pipe_to, work_output_dir)
+ worklist.append(w)
+ return worklist
+
+
+def do_run_work(worklist: list[Work], nr_jobs: int,
+ _verbosity: Verbosity) -> bool:
+ """do_run_work."""
+
+ nr_to_do = len(worklist)
+ not_started = list(worklist)
+ running: list[Work] = []
+ done: list[Work] = []
+ chg = False
+ while True:
+ nr_done = len(done)
+ if chg and _verbosity.normal:
+ nr_run = len(running)
+ print(
+ f"\rThere are {nr_to_do} jobs: {nr_done} completed, {nr_run} running",
+ flush=True, end=" ")
+ if _verbosity.verbose:
+ print()
+ chg = False
+ if nr_done == nr_to_do:
+ break
+ while len(running) < nr_jobs and len(not_started):
+ w = not_started.pop(0)
+ running.append(w)
+ if _verbosity.verbose:
+ print("Starting:", w.command())
+ w.start()
+ chg = True
+ if len(running):
+ time.sleep(0.1)
+ finished: list[Work] = []
+ not_finished: list[Work] = []
+ while len(running):
+ w = running.pop(0)
+ r = w.poll()
+ if r is None:
+ not_finished.append(w)
+ continue
+ if r == 0:
+ if _verbosity.verbose:
+ print("Finished:", w.command())
+ finished.append(w)
+ chg = True
+ continue
+ if _verbosity.normal and not _verbosity.verbose:
+ print()
+ print("Job failed!\n return code:", r,
+ "\n command: ", w.command())
+ if w.pipe_to:
+ print(" piped to: ", w.pipe_to)
+ print("Killing outstanding jobs")
+ kill_work(not_finished, _verbosity)
+ kill_work(running, _verbosity)
+ return False
+ running = not_finished
+ done += finished
+ errorlist: list[str] = []
+ for w in worklist:
+ errorlist += w.errors()
+ if len(errorlist):
+ print("errors:")
+ for e in errorlist:
+ print(e)
+ elif _verbosity.normal:
+ print("\r", " "*50, "\rAll jobs finished successfully", flush=True)
+ return True
+
+
+def run_work(worklist: list[Work], nr_jobs: int = number_of_cp_us(),
+ _verbosity: Verbosity = Verbosity()) -> bool:
+ """run_work."""
+ try:
+ return do_run_work(worklist, nr_jobs, _verbosity)
+ except (OSError, ValueError, RuntimeError, KeyboardInterrupt):
+ for w in worklist:
+ w.kill()
+ raise
+
+
+def read_header(perf, file_name):
+ """read_header."""
+
+ cmd = [perf, "script", "--header-only", "--input", file_name]
+ with subprocess.Popen(cmd, stdout=subprocess.PIPE) as proc:
+ out = proc.stdout.read() if proc.stdout else b""
+ return out.decode("utf-8")
+
+
+def parse_header(hdr):
+ """parse_header."""
+
+ result = {}
+ lines = hdr.split("\n")
+ for line in lines:
+ if ":" in line and line[0] == "#":
+ pos = line.index(":")
+ name = line[1:pos-1].strip()
+ value = line[pos+1:].strip()
+ if name in result:
+ orig_name = name
+ nr = 2
+ while True:
+ name = f"{orig_name} {nr}"
+ if name not in result:
+ break
+ nr += 1
+ result[name] = value
+ return result
+
+
+def header_field(hdr_dict, hdr_fld):
+ """header_field."""
+
+ if hdr_fld not in hdr_dict:
+ raise RuntimeError(f"'{hdr_fld}' missing from header information")
+ return hdr_dict[hdr_fld]
+
+# Represent the position of an option within a command string
+# and provide the option value and/or remove the option
+
+
+class OptPos():
+
+ def init(self, opt_element=-1, value_element=-1, opt_pos=-1, value_pos=-1, error=None):
+ """init."""
+
+ self.opt_element = opt_element # list element that contains option
+ self.value_element = value_element # list element that contains option value
+ self.opt_pos = opt_pos # string position of option
+ self.value_pos = value_pos # string position of value
+ self.error = error # error message string
+
+ def __init__(self, args, short_name, long_name, default=None):
+ """__init__."""
+
+ self.args = list(args)
+ self.default = default
+ n = 2 + len(long_name)
+ m = len(short_name)
+ pos = -1
+ for opt in args:
+ pos += 1
+ if m and opt[:2] == f"-{short_name}":
+ if len(opt) == 2:
+ if pos + 1 < len(args):
+ self.init(pos, pos + 1, 0, 0)
+ else:
+ self.init(error=f"-{short_name} option missing value")
+ else:
+ self.init(pos, pos, 0, 2)
+ return
+ if opt[:n] == f"--{long_name}":
+ if len(opt) == n:
+ if pos + 1 < len(args):
+ self.init(pos, pos + 1, 0, 0)
+ else:
+ self.init(error=f"--{long_name} option missing value")
+ elif opt[n] == "=":
+ self.init(pos, pos, 0, n + 1)
+ else:
+ self.init(error=f"--{long_name} option expected '='")
+ return
+ if m and opt[:1] == "-" and opt[:2] != "--" and short_name in opt:
+ ipos = opt.index(short_name)
+ if "-" in opt[1:]:
+ hpos = opt[1:].index("-")
+ if hpos < ipos:
+ continue
+ if ipos + 1 == len(opt):
+ if pos + 1 < len(args):
+ self.init(pos, pos + 1, ipos, 0)
+ else:
+ self.init(error=f"-{short_name} option missing value")
+ else:
+ self.init(pos, pos, ipos, ipos + 1)
+ return
+ self.init()
+
+ def value(self):
+ """value."""
+
+ if self.opt_element >= 0:
+ if self.opt_element != self.value_element:
+ return self.args[self.value_element]
+ else:
+ return self.args[self.value_element][self.value_pos:]
+ return self.default
+
+ def remove(self, args):
+ """remove."""
+
+ if self.opt_element == -1:
+ return
+ if self.opt_element != self.value_element:
+ del args[self.value_element]
+ if self.opt_pos:
+ args[self.opt_element] = args[self.opt_element][:self.opt_pos]
+ else:
+ del args[self.opt_element]
+
+
+def determine_input_file_name(cmd):
+ """determine_input_file_name."""
+
+ p = OptPos(cmd, "i", "input", "perf.data")
+ if p.error:
+ raise RuntimeError(f"perf command {p.error}")
+ file_name = p.value()
+ if not os.path.exists(file_name):
+ raise RuntimeError(f"perf command input file '{file_name}' not found")
+ return file_name
+
+
+def read_option(args, short_name, long_name, err_prefix, remove=False):
+ """read_option."""
+
+ p = OptPos(args, short_name, long_name)
+ if p.error:
+ raise RuntimeError(f"{err_prefix}{p.error}")
+ value = p.value()
+ if remove:
+ p.remove(args)
+ return value
+
+
+def extract_option(args, short_name, long_name, err_prefix):
+ """extract_option."""
+
+ return read_option(args, short_name, long_name, err_prefix, True)
+
+
+def read_perf_option(args, short_name, long_name):
+ """read_perf_option."""
+
+ return read_option(args, short_name, long_name, "perf command ")
+
+
+def extract_perf_option(args, short_name, long_name):
+ """extract_perf_option."""
+
+ return extract_option(args, short_name, long_name, "perf command ")
+
+
+def perf_double_quick_commands(cmd, file_name):
+ """perf_double_quick_commands."""
+
+ cpu_str = read_perf_option(cmd, "C", "cpu")
+ time_str = read_perf_option(cmd, "", "time")
+ # Use double-quick sampling to determine trace data density
+ times_cmd = ["perf", "script", "--ns",
+ "--input", file_name, "--itrace=qqi"]
+ if cpu_str is not None and cpu_str != "":
+ times_cmd.append(f"--cpu={cpu_str}")
+ if time_str is not None and time_str != "":
+ times_cmd.append(f"--time={time_str}")
+ cnts_cmd = list(times_cmd)
+ cnts_cmd.append("-Fcpu")
+ times_cmd.append("-Fcpu,time")
+ return cnts_cmd, times_cmd
+
+
+class CPUTimeRange():
+ def __init__(self, cpu):
+ """__init__."""
+
+ self.cpu = cpu
+ self.sample_cnt = 0
+ self.time_ranges = None
+ self.interval = 0
+ self.interval_remaining = 0
+ self.remaining = 0
+ self.tr_pos = 0
+
+
+def calc_time_ranges_by_cpu(line, cpu, cpu_time_ranges, max_time):
+ """calc_time_ranges_by_cpu."""
+
+ cpu_time_range = cpu_time_ranges[cpu]
+ cpu_time_range.remaining -= 1
+ cpu_time_range.interval_remaining -= 1
+ if cpu_time_range.remaining == 0:
+ cpu_time_range.time_ranges[cpu_time_range.tr_pos][1] = max_time
+ return
+ if cpu_time_range.interval_remaining == 0:
+ ts = time_val(line[1][:-1], 0)
+ time_ranges = cpu_time_range.time_ranges
+ time_ranges[cpu_time_range.tr_pos][1] = ts - 1
+ time_ranges.append([ts, max_time])
+ cpu_time_range.tr_pos += 1
+ cpu_time_range.interval_remaining = cpu_time_range.interval
+
+
+def count_samples_by_cpu(_line: list[str], cpu: int,
+ cpu_time_ranges: list[Any]) -> None:
+ """count_samples_by_cpu."""
+
+ try:
+ cpu_time_ranges[cpu].sample_cnt += 1
+ except (OSError, ValueError, RuntimeError, IndexError):
+ print("exception")
+ print("cpu", cpu)
+ print("len(cpu_time_ranges)", len(cpu_time_ranges))
+ raise
+
+
+def process_command_output_lines(cmd, per_cpu, fn, *x):
+ """process_command_output_lines."""
+
+ # Assume CPU number is at beginning of line and enclosed by []
+ pat = re.compile(r"\s*\[[0-9]+\]")
+ p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
+ while True:
+ line = p.stdout.readline()
+ if line:
+ line = line.decode("utf-8")
+ if pat.match(line):
+ line = line.split()
+ if per_cpu:
+ # Assumes CPU number is enclosed by []
+ cpu = int(line[0][1:-1])
+ else:
+ cpu = 0
+ fn(line, cpu, *x)
+ else:
+ break
+ p.wait()
+
+
+def intersect_time_ranges(new_time_ranges, time_ranges):
+ """intersect_time_ranges."""
+
+ pos = 0
+ new_pos = 0
+ # Can assume len(time_ranges) != 0 and len(new_time_ranges) != 0
+ # Note also, there *must* be at least one intersection.
+ while pos < len(time_ranges) and new_pos < len(new_time_ranges):
+ # new end < old start => no intersection, remove new
+ if new_time_ranges[new_pos][1] < time_ranges[pos][0]:
+ del new_time_ranges[new_pos]
+ continue
+ # new start > old end => no intersection, check next
+ if new_time_ranges[new_pos][0] > time_ranges[pos][1]:
+ pos += 1
+ if pos < len(time_ranges):
+ continue
+ # no next, so remove remaining
+ while new_pos < len(new_time_ranges):
+ del new_time_ranges[new_pos]
+ return
+ # Found an intersection
+ # new start < old start => adjust new start = old start
+ if new_time_ranges[new_pos][0] < time_ranges[pos][0]:
+ new_time_ranges[new_pos][0] = time_ranges[pos][0]
+ # new end > old end => keep the overlap, insert the remainder
+ if new_time_ranges[new_pos][1] > time_ranges[pos][1]:
+ r = [time_ranges[pos][1] + 1, new_time_ranges[new_pos][1]]
+ new_time_ranges[new_pos][1] = time_ranges[pos][1]
+ new_pos += 1
+ new_time_ranges.insert(new_pos, r)
+ continue
+ # new [start, end] is within old [start, end]
+ new_pos += 1
+
+
+def split_time_ranges_by_trace_data_density(
+ time_ranges, cpus, nr, cmd, file_name, per_cpu, min_size, min_interval, _verbosity
+):
+ """split_time_ranges_by_trace_data_density."""
+
+ if _verbosity.normal:
+ print("\rAnalyzing...", flush=True, end=" ")
+ if _verbosity.verbose:
+ print()
+ cnts_cmd, times_cmd = perf_double_quick_commands(cmd, file_name)
+
+ nr_cpus = cpus[-1] + 1 if per_cpu else 1
+ if per_cpu:
+ nr_cpus = cpus[-1] + 1
+ cpu_time_ranges = [CPUTimeRange(cpu) for cpu in range(nr_cpus)]
+ else:
+ nr_cpus = 1
+ cpu_time_ranges = [CPUTimeRange(-1)]
+
+ if _verbosity.debug:
+ print("nr_cpus", nr_cpus)
+ print("cnts_cmd", cnts_cmd)
+ print("times_cmd", times_cmd)
+
+ # Count the number of "double quick" samples per CPU
+ process_command_output_lines(
+ cnts_cmd, per_cpu, count_samples_by_cpu, cpu_time_ranges)
+
+ tot = 0
+ mx = 0
+ for cpu_time_range in cpu_time_ranges:
+ cnt = cpu_time_range.sample_cnt
+ tot += cnt
+ if cnt > mx:
+ mx = cnt
+ if _verbosity.debug:
+ print("cpu:", cpu_time_range.cpu, "sample_cnt", cnt)
+
+ if min_size < 1:
+ min_size = 1
+
+ if mx < min_size:
+ # Too little data to be worth splitting
+ if _verbosity.debug:
+ print("Too little data to split by time")
+ if nr == 0:
+ nr = 1
+ return [split_time_ranges_into_n(time_ranges, nr, min_interval)]
+
+ if nr:
+ divisor = nr
+ min_size = 1
+ else:
+ divisor = number_of_cp_us()
+
+ interval = int(round(tot / divisor, 0))
+ if interval < min_size:
+ interval = min_size
+
+ if _verbosity.debug:
+ print("divisor", divisor)
+ print("min_size", min_size)
+ print("interval", interval)
+
+ min_time = time_ranges[0][0]
+ max_time = time_ranges[-1][1]
+
+ for cpu_time_range in cpu_time_ranges:
+ cnt = cpu_time_range.sample_cnt
+ if cnt == 0:
+ cpu_time_range.time_ranges = copy.deepcopy(time_ranges)
+ continue
+ # Adjust target interval for CPU to give approximately equal interval sizes
+ # Determine number of intervals, rounding to nearest integer
+ n = int(round(cnt / interval, 0))
+ if n < 1:
+ n = 1
+ # Determine interval size, rounding up
+ d, m = divmod(cnt, n)
+ if m:
+ d += 1
+ cpu_time_range.interval = d
+ cpu_time_range.interval_remaining = d
+ cpu_time_range.remaining = cnt
+ # init. time ranges for each CPU with the start time
+ cpu_time_range.time_ranges = [[min_time, max_time]]
+
+ # Set time ranges so that the same number of "double quick" samples
+ # will fall into each time range.
+ process_command_output_lines(
+ times_cmd, per_cpu, calc_time_ranges_by_cpu, cpu_time_ranges, max_time)
+
+ for cpu_time_range in cpu_time_ranges:
+ if cpu_time_range.sample_cnt:
+ intersect_time_ranges(cpu_time_range.time_ranges, time_ranges)
+
+ return [cpu_time_ranges[cpu].time_ranges for cpu in cpus]
+
+
+def split_single_time_range_into_n(time_range, n):
+ """split_single_time_range_into_n."""
+
+ if n <= 1:
+ return [time_range]
+ start = time_range[0]
+ end = time_range[1]
+ duration = int((end - start + 1) / n)
+ if duration < 1:
+ return [time_range]
+ time_ranges = []
+ for _i in range(n):
+ time_ranges.append([start, start + duration - 1])
+ start += duration
+ time_ranges[-1][1] = end
+ return time_ranges
+
+
+def time_range_duration(r):
+ """time_range_duration."""
+
+ return r[1] - r[0] + 1
+
+
+def total_duration(time_ranges):
+ """total_duration."""
+
+ duration = 0
+ for r in time_ranges:
+ duration += time_range_duration(r)
+ return duration
+
+
+def split_time_ranges_by_interval(time_ranges, interval):
+ """split_time_ranges_by_interval."""
+
+ new_ranges = []
+ for r in time_ranges:
+ duration = time_range_duration(r)
+ n = duration / interval
+ n = int(round(n, 0))
+ new_ranges += split_single_time_range_into_n(r, n)
+ return new_ranges
+
+
+def split_time_ranges_into_n(time_ranges, n, min_interval):
+ """split_time_ranges_into_n."""
+
+ if n <= len(time_ranges):
+ return time_ranges
+ duration = total_duration(time_ranges)
+ interval = duration / n
+ if interval < min_interval:
+ interval = min_interval
+ return split_time_ranges_by_interval(time_ranges, interval)
+
+
+def recombine_time_ranges(tr):
+ """recombine_time_ranges."""
+
+ new_tr = copy.deepcopy(tr)
+ i = 1
+ while i < len(new_tr):
+ # if prev end + 1 == cur start, combine them
+ if new_tr[i - 1][1] + 1 == new_tr[i][0]:
+ new_tr[i][0] = new_tr[i - 1][0]
+ del new_tr[i - 1]
+ else:
+ i += 1
+ return new_tr
+
+
+def open_time_range_ends(time_ranges, min_time, max_time):
+ """open_time_range_ends."""
+
+ if time_ranges[0][0] <= min_time:
+ time_ranges[0][0] = None
+ if time_ranges[-1][1] >= max_time:
+ time_ranges[-1][1] = None
+
+
+def bad_time_str(time_str):
+ """bad_time_str."""
+
+ raise RuntimeError(
+ f"perf command bad time option: '{time_str}'\n"
+ "Check also 'time of first sample' and 'time of last sample' "
+ "in perf script --header-only"
+ )
+
+
+def validate_time_ranges(time_ranges, time_str):
+ """validate_time_ranges."""
+
+ n = len(time_ranges)
+ for i in range(n):
+ start = time_ranges[i][0]
+ end = time_ranges[i][1]
+ if i != 0 and start <= time_ranges[i - 1][1]:
+ bad_time_str(time_str)
+ if start > end:
+ bad_time_str(time_str)
+
+
+def time_val(s, dflt):
+ """time_val."""
+
+ s = s.strip()
+ if s == "":
+ return dflt
+ a = s.split(".")
+ if len(a) > 2:
+ raise RuntimeError(f"Bad time value'{s}'")
+ x = int(a[0])
+ if x < 0:
+ raise RuntimeError("Negative time not allowed")
+ x *= 1000000000
+ if len(a) > 1:
+ x += int((a[1] + "000000000")[:9])
+ return x
+
+
+def bad_cpu_str(cpu_str):
+ """bad_cpu_str."""
+
+ raise RuntimeError(
+ f"perf command bad cpu option: '{cpu_str}'\n"
+ "Check also 'nrcpus avail' in perf script --header-only"
+ )
+
+
+def parse_time_str(time_str, min_time, max_time):
+ """parse_time_str."""
+
+ if time_str is None or time_str == "":
+ return [[min_time, max_time]]
+ time_ranges = []
+ for r in time_str.split():
+ a = r.split(",")
+ if len(a) != 2:
+ bad_time_str(time_str)
+ try:
+ start = time_val(a[0], min_time)
+ end = time_val(a[1], max_time)
+ except (OSError, ValueError, RuntimeError):
+ bad_time_str(time_str)
+ time_ranges.append([start, end])
+ validate_time_ranges(time_ranges, time_str)
+ return time_ranges
+
+
+def parse_cpu_str(cpu_str, nr_cpus):
+ """parse_cpu_str."""
+
+ if cpu_str is None or cpu_str == "":
+ return [-1]
+ cpus = []
+ for r in cpu_str.split(","):
+ a = r.split("-")
+ if len(a) < 1 or len(a) > 2:
+ bad_cpu_str(cpu_str)
+ try:
+ start = int(a[0].strip())
+ if len(a) > 1:
+ end = int(a[1].strip())
+ else:
+ end = start
+ except (OSError, ValueError, RuntimeError):
+ bad_cpu_str(cpu_str)
+ if start < 0 or end < 0 or end < start or end >= nr_cpus:
+ bad_cpu_str(cpu_str)
+ cpus.extend(range(start, end + 1))
+ cpus = list(set(cpus)) # remove duplicates
+ cpus.sort()
+ return cpus
+
+
+class ParallelPerf():
+
+ def __init__(self, a):
+ """init."""
+ self.nr = 0
+ self.jobs = 0
+ self.file_name = None
+ self.hdr = None
+ self.hdr_dict = None
+ self.cmd_line = None
+ self.min_time = None
+ self.max_time = None
+ self.time_str = None
+ self.time_ranges = None
+ self.cpu_str = None
+ self.cpus = None
+ self.split_time_ranges_for_each_cpu = None
+ self.worklist = None
+ self.per_cpu = None
+ self.interval = None
+ self.min_size = None
+ self.min_interval = None
+ self.pipe_to = None
+ self.output_dir = None
+ self.cmd = getattr(a, 'cmd', None)
+ self.no_per_cpu = getattr(a, 'no_per_cpu', False)
+ self.dry_run = getattr(a, 'dry_run', False)
+ self._verbosity = getattr(a, "_verbosity", Verbosity())
+
+ for arg_name in vars(a):
+ setattr(self, arg_name, getattr(a, arg_name))
+ self.orig_nr = self.nr
+ self.orig_cmd = list(self.cmd)
+ self.perf = self.cmd[0]
+ if os.path.exists(self.output_dir):
+ raise RuntimeError(f"Output '{self.output_dir}' already exists")
+ if self.jobs < 0 or self.nr < 0 or self.interval < 0:
+ raise RuntimeError(
+ "Bad options (negative values): try -h option for help")
+ if self.nr != 0 and self.interval != 0:
+ raise RuntimeError(
+ "Cannot specify number of time subdivisions and time interval")
+ if self.jobs == 0:
+ self.jobs = number_of_cp_us()
+ if self.nr == 0 and self.interval == 0:
+ if self.per_cpu:
+ self.nr = 1
+ else:
+ self.nr = self.jobs
+
+ def init(self):
+ """init."""
+
+ if self._verbosity.debug:
+ print("cmd", self.cmd)
+ self.file_name = determine_input_file_name(self.cmd)
+ self.hdr = read_header(self.perf, self.file_name)
+ self.hdr_dict = parse_header(self.hdr)
+ self.cmd_line = header_field(self.hdr_dict, "cmdline")
+
+ def extract_time_info(self):
+ """extract_time_info."""
+
+ self.min_time = time_val(header_field(
+ self.hdr_dict, "time of first sample"), 0)
+ self.max_time = time_val(header_field(
+ self.hdr_dict, "time of last sample"), 0)
+ self.time_str = extract_perf_option(self.cmd, "", "time")
+ self.time_ranges = parse_time_str(
+ self.time_str, self.min_time, self.max_time)
+ if self._verbosity.debug:
+ print("time_ranges", self.time_ranges)
+
+ def extract_cpu_info(self):
+ """extract_cpu_info."""
+
+ if self.per_cpu:
+ nr_cpus = int(header_field(self.hdr_dict, "nrcpus avail"))
+ self.cpu_str = extract_perf_option(self.cmd, "C", "cpu")
+ if self.cpu_str is None or self.cpu_str == "":
+ self.cpus = [x for x in range(nr_cpus)]
+ else:
+ self.cpus = parse_cpu_str(self.cpu_str, nr_cpus)
+ else:
+ self.cpu_str = None
+ self.cpus = [-1]
+ if self._verbosity.debug:
+ print("cpus", self.cpus)
+
+ def is_intel_pt(self):
+ """is_intel_pt."""
+
+ return self.cmd_line.find("intel_pt") >= 0
+
+ def split_time_ranges(self):
+ """split_time_ranges."""
+
+ if self.is_intel_pt() and self.interval == 0:
+ self.split_time_ranges_for_each_cpu = (
+ split_time_ranges_by_trace_data_density(
+ self.time_ranges, self.cpus, self.orig_nr,
+ self.orig_cmd, self.file_name, self.per_cpu,
+ self.min_size, self.min_interval, self._verbosity
+ )
+ )
+ elif self.nr:
+ self.split_time_ranges_for_each_cpu = [split_time_ranges_into_n(
+ self.time_ranges, self.nr, self.min_interval)]
+ else:
+ self.split_time_ranges_for_each_cpu = [
+ split_time_ranges_by_interval(self.time_ranges, self.interval)]
+
+ def check_time_ranges(self):
+ """check_time_ranges."""
+
+ for tr in self.split_time_ranges_for_each_cpu:
+ # Re-combined time ranges should be the same
+ new_tr = recombine_time_ranges(tr)
+ if new_tr != self.time_ranges:
+ if self._verbosity.debug:
+ print("tr", tr)
+ print("new_tr", new_tr)
+ raise RuntimeError("Self test failed!")
+
+ def open_time_range_ends(self):
+ """open_time_range_ends."""
+
+ for time_ranges in self.split_time_ranges_for_each_cpu:
+ open_time_range_ends(time_ranges, self.min_time, self.max_time)
+
+ def create_work_list(self):
+ """create_work_list."""
+
+ self.worklist = create_work_list(
+ self.cmd, self.pipe_to, self.output_dir, self.cpus, self.split_time_ranges_for_each_cpu)
+
+ def perf_data_recorded_per_cpu(self):
+ """perf_data_recorded_per_cpu."""
+
+ if "--per-thread" in self.cmd_line.split():
+ return False
+ return True
+
+ def default_to_per_cpu(self):
+ """default_to_per_cpu."""
+
+ # --no-per-cpu option takes precedence
+ if self.no_per_cpu:
+ return False
+ if not self.perf_data_recorded_per_cpu():
+ return False
+ # Default to per-cpu for Intel PT data that was recorded per-cpu,
+ # because decoding can be done for each CPU separately.
+ if self.is_intel_pt():
+ return True
+ return False
+
+ def config(self):
+ """config."""
+
+ self.init()
+ self.extract_time_info()
+ if not self.per_cpu:
+ self.per_cpu = self.default_to_per_cpu()
+ if self._verbosity.debug:
+ print("per_cpu", self.per_cpu)
+ self.extract_cpu_info()
+ self.split_time_ranges()
+ if self._verbosity.self_test:
+ self.check_time_ranges()
+ # Prefer open-ended time range to starting / ending with min_time / max_time resp.
+ self.open_time_range_ends()
+ self.create_work_list()
+
+ def run(self):
+ """run."""
+
+ if self.dry_run:
+ print(len(self.worklist), "jobs:")
+ for w in self.worklist:
+ print(w.command())
+ return True
+ result = run_work(self.worklist, self.jobs, _verbosity=self._verbosity)
+ if self._verbosity.verbose:
+ print(glb_prog_name, "done")
+ return result
+
+
+def run_parallel_perf(a):
+ """run_parallel_perf."""
+
+ pp = ParallelPerf(a)
+ pp.config()
+ return pp.run()
+
+
+def main(args):
+ """main."""
+
+ ap = argparse.ArgumentParser(
+ prog=glb_prog_name, formatter_class=argparse.RawDescriptionHelpFormatter,
+ description="""
+run a perf script command multiple times in parallel, using perf script options
+--cpu and --time so that each job processes a different chunk of the data.
+""",
+ epilog="""
+Follow the options by '--' and then the perf script command e.g.
+
+ $ perf record -a -- sleep 10
+ $ parallel-perf.py --nr=4 -- perf script --ns
+ All jobs finished successfully
+ $ tree parallel-perf-output/
+ parallel-perf-output/
+ ├── time-range-0
+ │ ├── cmd.txt
+ │ └── out.txt
+ ├── time-range-1
+ │ ├── cmd.txt
+ │ └── out.txt
+ ├── time-range-2
+ │ ├── cmd.txt
+ │ └── out.txt
+ └── time-range-3
+ ├── cmd.txt
+ └── out.txt
+ $ find parallel-perf-output -name cmd.txt | sort | xargs grep -H .
+ parallel-perf-output/time-range-0/cmd.txt:perf script --time=,9466.504461499 --ns
+ parallel-perf-output/time-range-1/cmd.txt:perf script --time=9466.504461500,9469.005396999 --ns
+ parallel-perf-output/time-range-2/cmd.txt:perf script --time=9469.005397000,9471.506332499 --ns
+ parallel-perf-output/time-range-3/cmd.txt:perf script --time=9471.506332500, --ns
+
+Any perf script command can be used, including the use of perf script options
+--dlfilter and --script, so that the benefit of running parallel jobs
+naturally extends to them also.
+
+If option --pipe-to is used, standard output is first piped through that
+command. Beware, if the command fails (e.g. grep with no matches), it will be
+considered a fatal error.
+
+Final standard output is redirected to files named out.txt in separate
+subdirectories under the output directory. Similarly, standard error is
+written to files named err.txt. In addition, files named cmd.txt contain the
+corresponding perf script command. After processing, err.txt files are removed
+if they are empty.
+
+If any job exits with a non-zero exit code, then all jobs are killed and no
+more are started. A message is printed if any job results in a non-empty
+err.txt file.
+
+There is a separate output subdirectory for each time range. If the --per-cpu
+option is used, these are further grouped under cpu-n subdirectories, e.g.
+
+ $ parallel-perf.py --per-cpu --nr=2 -- perf script --ns --cpu=0,1
+ All jobs finished successfully
+ $ tree parallel-perf-output
+ parallel-perf-output/
+ ├── cpu-0
+ │ ├── time-range-0
+ │ │ ├── cmd.txt
+ │ │ └── out.txt
+ │ └── time-range-1
+ │ ├── cmd.txt
+ │ └── out.txt
+ └── cpu-1
+ ├── time-range-0
+ │ ├── cmd.txt
+ │ └── out.txt
+ └── time-range-1
+ ├── cmd.txt
+ └── out.txt
+ $ find parallel-perf-output -name cmd.txt | sort | xargs grep -H .
+ parallel-perf-output/cpu-0/time-range-0/cmd.txt:perf script --cpu=0 --time=,9469.005396999 --ns
+ parallel-perf-output/cpu-0/time-range-1/cmd.txt:perf script --cpu=0 --time=9469.005397000, --ns
+ parallel-perf-output/cpu-1/time-range-0/cmd.txt:perf script --cpu=1 --time=,9469.005396999 --ns
+ parallel-perf-output/cpu-1/time-range-1/cmd.txt:perf script --cpu=1 --time=9469.005397000, --ns
+
+Subdivisions of time range, and cpus if the --per-cpu option is used, are
+expressed by the --time and --cpu perf script options respectively. If the
+supplied perf script command has a --time option, then that time range is
+subdivided, otherwise the time range given by 'time of first sample' to
+'time of last sample' is used (refer perf script --header-only). Similarly, the
+supplied perf script command may provide a --cpu option, and only those CPUs
+will be processed.
+
+To prevent time intervals becoming too small, the --min-interval option can
+be used.
+
+Note there is special handling for processing Intel PT traces. If an interval is
+not specified and the perf record command contained the intel_pt event, then the
+time range will be subdivided in order to produce subdivisions that contain
+approximately the same amount of trace data. That is accomplished by counting
+double-quick (--itrace=qqi) samples, and choosing time ranges that encompass
+approximately the same number of samples. In that case, time ranges may not be
+the same for each CPU processed. For Intel PT, --per-cpu is the default, but
+that can be overridden by --no-per-cpu. Note, for Intel PT, double-quick
+decoding produces 1 sample for each PSB synchronization packet, which in turn
+come after a certain number of bytes output, determined by psb_period (refer
+perf Intel PT documentation). The minimum number of double-quick samples that
+will define a time range can be set by the --min_size option, which defaults to
+64.
+""")
+ ap.add_argument("-o", "--output-dir", default="parallel-perf-output",
+ help="output directory (default 'parallel-perf-output')")
+ ap.add_argument("-j", "--jobs", type=int, default=0,
+ help="maximum number of jobs to run in parallel at one time "
+ "(default is the number of CPUs)")
+ ap.add_argument("-n", "--nr", type=int, default=0,
+ help="number of time subdivisions (default is the number of jobs)")
+ ap.add_argument("-i", "--interval", type=float, default=0,
+ help="subdivide the time range using this time interval "
+ "(in seconds e.g. 0.1 for a tenth of a second)")
+ ap.add_argument("-c", "--per-cpu", action="store_true",
+ help="process data for each CPU in parallel")
+ ap.add_argument("-m", "--min-interval", type=float, default=glb_min_interval,
+ help=f"minimum interval (default {glb_min_interval} seconds)")
+ ap.add_argument("-p", "--pipe-to",
+ help="command to pipe output to (optional)")
+ ap.add_argument("-N", "--no-per-cpu", action="store_true",
+ help="do not process data for each CPU in parallel")
+ ap.add_argument("-b", "--min_size", type=int, default=glb_min_samples,
+ help="minimum data size (for Intel PT in PSBs)")
+ ap.add_argument("-D", "--dry-run", action="store_true",
+ help="do not run any jobs, just show the perf script commands")
+ ap.add_argument("-q", "--quiet", action="store_true",
+ help="do not print any messages except errors")
+ ap.add_argument("-v", "--verbose", action="store_true",
+ help="print more messages")
+ ap.add_argument("-d", "--debug", action="store_true",
+ help="print debugging messages")
+ cmd_line = list(args)
+ try:
+ split_pos = cmd_line.index("--")
+ cmd = cmd_line[split_pos + 1:]
+ args = cmd_line[:split_pos]
+ except (OSError, ValueError, RuntimeError):
+ cmd = None
+ args = cmd_line
+ a = ap.parse_args(args=args[1:])
+ a.cmd = cmd
+ setattr(a, "_verbosity", Verbosity(a.quiet, a.verbose, a.debug))
+ try:
+ if not a.cmd:
+ if a.cmd is None and len(args) <= 1:
+ ap.print_help()
+ return True
+ raise RuntimeError(
+ "command line must contain '--' before perf command")
+ return run_parallel_perf(a)
+ except (OSError, ValueError, RuntimeError, KeyboardInterrupt) as e:
+ print("Fatal error: ", str(e))
+ if a.debug:
+ raise
+ return False
+
+
+if __name__ == "__main__":
+ if not main(sys.argv):
+ sys.exit(1)
diff --git a/tools/perf/scripts/python/parallel-perf.py b/tools/perf/scripts/python/parallel-perf.py
deleted file mode 100755
index be85fd7f6632..000000000000
--- a/tools/perf/scripts/python/parallel-perf.py
+++ /dev/null
@@ -1,989 +0,0 @@
-#!/usr/bin/env python3
-# SPDX-License-Identifier: GPL-2.0
-#
-# Run a perf script command multiple times in parallel, using perf script
-# options --cpu and --time so that each job processes a different chunk
-# of the data.
-#
-# Copyright (c) 2024, Intel Corporation.
-
-import subprocess
-import argparse
-import pathlib
-import shlex
-import time
-import copy
-import sys
-import os
-import re
-
-glb_prog_name = "parallel-perf.py"
-glb_min_interval = 10.0
-glb_min_samples = 64
-
-class Verbosity():
-
- def __init__(self, quiet=False, verbose=False, debug=False):
- self.normal = True
- self.verbose = verbose
- self.debug = debug
- self.self_test = True
- if self.debug:
- self.verbose = True
- if self.verbose:
- quiet = False
- if quiet:
- self.normal = False
-
-# Manage work (Start/Wait/Kill), as represented by a subprocess.Popen command
-class Work():
-
- def __init__(self, cmd, pipe_to, output_dir="."):
- self.popen = None
- self.consumer = None
- self.cmd = cmd
- self.pipe_to = pipe_to
- self.output_dir = output_dir
- self.cmdout_name = f"{output_dir}/cmd.txt"
- self.stdout_name = f"{output_dir}/out.txt"
- self.stderr_name = f"{output_dir}/err.txt"
-
- def Command(self):
- sh_cmd = [ shlex.quote(x) for x in self.cmd ]
- return " ".join(self.cmd)
-
- def Stdout(self):
- return open(self.stdout_name, "w")
-
- def Stderr(self):
- return open(self.stderr_name, "w")
-
- def CreateOutputDir(self):
- pathlib.Path(self.output_dir).mkdir(parents=True, exist_ok=True)
-
- def Start(self):
- if self.popen:
- return
- self.CreateOutputDir()
- with open(self.cmdout_name, "w") as f:
- f.write(self.Command())
- f.write("\n")
- stdout = self.Stdout()
- stderr = self.Stderr()
- if self.pipe_to:
- self.popen = subprocess.Popen(self.cmd, stdout=subprocess.PIPE, stderr=stderr)
- args = shlex.split(self.pipe_to)
- self.consumer = subprocess.Popen(args, stdin=self.popen.stdout, stdout=stdout, stderr=stderr)
- else:
- self.popen = subprocess.Popen(self.cmd, stdout=stdout, stderr=stderr)
-
- def RemoveEmptyErrFile(self):
- if os.path.exists(self.stderr_name):
- if os.path.getsize(self.stderr_name) == 0:
- os.unlink(self.stderr_name)
-
- def Errors(self):
- if os.path.exists(self.stderr_name):
- if os.path.getsize(self.stderr_name) != 0:
- return [ f"Non-empty error file {self.stderr_name}" ]
- return []
-
- def TidyUp(self):
- self.RemoveEmptyErrFile()
-
- def RawPollWait(self, p, wait):
- if wait:
- return p.wait()
- return p.poll()
-
- def Poll(self, wait=False):
- if not self.popen:
- return None
- result = self.RawPollWait(self.popen, wait)
- if self.consumer:
- res = result
- result = self.RawPollWait(self.consumer, wait)
- if result != None and res == None:
- self.popen.kill()
- result = None
- elif result == 0 and res != None and res != 0:
- result = res
- if result != None:
- self.TidyUp()
- return result
-
- def Wait(self):
- return self.Poll(wait=True)
-
- def Kill(self):
- if not self.popen:
- return
- self.popen.kill()
- if self.consumer:
- self.consumer.kill()
-
-def KillWork(worklist, verbosity):
- for w in worklist:
- w.Kill()
- for w in worklist:
- w.Wait()
-
-def NumberOfCPUs():
- return os.sysconf("SC_NPROCESSORS_ONLN")
-
-def NanoSecsToSecsStr(x):
- if x == None:
- return ""
- x = str(x)
- if len(x) < 10:
- x = "0" * (10 - len(x)) + x
- return x[:len(x) - 9] + "." + x[-9:]
-
-def InsertOptionAfter(cmd, option, after):
- try:
- pos = cmd.index(after)
- cmd.insert(pos + 1, option)
- except:
- cmd.append(option)
-
-def CreateWorkList(cmd, pipe_to, output_dir, cpus, time_ranges_by_cpu):
- max_len = len(str(cpus[-1]))
- cpu_dir_fmt = f"cpu-%.{max_len}u"
- worklist = []
- pos = 0
- for cpu in cpus:
- if cpu >= 0:
- cpu_dir = os.path.join(output_dir, cpu_dir_fmt % cpu)
- cpu_option = f"--cpu={cpu}"
- else:
- cpu_dir = output_dir
- cpu_option = None
-
- tr_dir_fmt = "time-range"
-
- if len(time_ranges_by_cpu) > 1:
- time_ranges = time_ranges_by_cpu[pos]
- tr_dir_fmt += f"-{pos}"
- pos += 1
- else:
- time_ranges = time_ranges_by_cpu[0]
-
- max_len = len(str(len(time_ranges)))
- tr_dir_fmt += f"-%.{max_len}u"
-
- i = 0
- for r in time_ranges:
- if r == [None, None]:
- time_option = None
- work_output_dir = cpu_dir
- else:
- time_option = "--time=" + NanoSecsToSecsStr(r[0]) + "," + NanoSecsToSecsStr(r[1])
- work_output_dir = os.path.join(cpu_dir, tr_dir_fmt % i)
- i += 1
- work_cmd = list(cmd)
- if time_option != None:
- InsertOptionAfter(work_cmd, time_option, "script")
- if cpu_option != None:
- InsertOptionAfter(work_cmd, cpu_option, "script")
- w = Work(work_cmd, pipe_to, work_output_dir)
- worklist.append(w)
- return worklist
-
-def DoRunWork(worklist, nr_jobs, verbosity):
- nr_to_do = len(worklist)
- not_started = list(worklist)
- running = []
- done = []
- chg = False
- while True:
- nr_done = len(done)
- if chg and verbosity.normal:
- nr_run = len(running)
- print(f"\rThere are {nr_to_do} jobs: {nr_done} completed, {nr_run} running", flush=True, end=" ")
- if verbosity.verbose:
- print()
- chg = False
- if nr_done == nr_to_do:
- break
- while len(running) < nr_jobs and len(not_started):
- w = not_started.pop(0)
- running.append(w)
- if verbosity.verbose:
- print("Starting:", w.Command())
- w.Start()
- chg = True
- if len(running):
- time.sleep(0.1)
- finished = []
- not_finished = []
- while len(running):
- w = running.pop(0)
- r = w.Poll()
- if r == None:
- not_finished.append(w)
- continue
- if r == 0:
- if verbosity.verbose:
- print("Finished:", w.Command())
- finished.append(w)
- chg = True
- continue
- if verbosity.normal and not verbosity.verbose:
- print()
- print("Job failed!\n return code:", r, "\n command: ", w.Command())
- if w.pipe_to:
- print(" piped to: ", w.pipe_to)
- print("Killing outstanding jobs")
- KillWork(not_finished, verbosity)
- KillWork(running, verbosity)
- return False
- running = not_finished
- done += finished
- errorlist = []
- for w in worklist:
- errorlist += w.Errors()
- if len(errorlist):
- print("Errors:")
- for e in errorlist:
- print(e)
- elif verbosity.normal:
- print("\r"," "*50, "\rAll jobs finished successfully", flush=True)
- return True
-
-def RunWork(worklist, nr_jobs=NumberOfCPUs(), verbosity=Verbosity()):
- try:
- return DoRunWork(worklist, nr_jobs, verbosity)
- except:
- for w in worklist:
- w.Kill()
- raise
- return True
-
-def ReadHeader(perf, file_name):
- return subprocess.Popen([perf, "script", "--header-only", "--input", file_name], stdout=subprocess.PIPE).stdout.read().decode("utf-8")
-
-def ParseHeader(hdr):
- result = {}
- lines = hdr.split("\n")
- for line in lines:
- if ":" in line and line[0] == "#":
- pos = line.index(":")
- name = line[1:pos-1].strip()
- value = line[pos+1:].strip()
- if name in result:
- orig_name = name
- nr = 2
- while True:
- name = f"{orig_name} {nr}"
- if name not in result:
- break
- nr += 1
- result[name] = value
- return result
-
-def HeaderField(hdr_dict, hdr_fld):
- if hdr_fld not in hdr_dict:
- raise Exception(f"'{hdr_fld}' missing from header information")
- return hdr_dict[hdr_fld]
-
-# Represent the position of an option within a command string
-# and provide the option value and/or remove the option
-class OptPos():
-
- def Init(self, opt_element=-1, value_element=-1, opt_pos=-1, value_pos=-1, error=None):
- self.opt_element = opt_element # list element that contains option
- self.value_element = value_element # list element that contains option value
- self.opt_pos = opt_pos # string position of option
- self.value_pos = value_pos # string position of value
- self.error = error # error message string
-
- def __init__(self, args, short_name, long_name, default=None):
- self.args = list(args)
- self.default = default
- n = 2 + len(long_name)
- m = len(short_name)
- pos = -1
- for opt in args:
- pos += 1
- if m and opt[:2] == f"-{short_name}":
- if len(opt) == 2:
- if pos + 1 < len(args):
- self.Init(pos, pos + 1, 0, 0)
- else:
- self.Init(error = f"-{short_name} option missing value")
- else:
- self.Init(pos, pos, 0, 2)
- return
- if opt[:n] == f"--{long_name}":
- if len(opt) == n:
- if pos + 1 < len(args):
- self.Init(pos, pos + 1, 0, 0)
- else:
- self.Init(error = f"--{long_name} option missing value")
- elif opt[n] == "=":
- self.Init(pos, pos, 0, n + 1)
- else:
- self.Init(error = f"--{long_name} option expected '='")
- return
- if m and opt[:1] == "-" and opt[:2] != "--" and short_name in opt:
- ipos = opt.index(short_name)
- if "-" in opt[1:]:
- hpos = opt[1:].index("-")
- if hpos < ipos:
- continue
- if ipos + 1 == len(opt):
- if pos + 1 < len(args):
- self.Init(pos, pos + 1, ipos, 0)
- else:
- self.Init(error = f"-{short_name} option missing value")
- else:
- self.Init(pos, pos, ipos, ipos + 1)
- return
- self.Init()
-
- def Value(self):
- if self.opt_element >= 0:
- if self.opt_element != self.value_element:
- return self.args[self.value_element]
- else:
- return self.args[self.value_element][self.value_pos:]
- return self.default
-
- def Remove(self, args):
- if self.opt_element == -1:
- return
- if self.opt_element != self.value_element:
- del args[self.value_element]
- if self.opt_pos:
- args[self.opt_element] = args[self.opt_element][:self.opt_pos]
- else:
- del args[self.opt_element]
-
-def DetermineInputFileName(cmd):
- p = OptPos(cmd, "i", "input", "perf.data")
- if p.error:
- raise Exception(f"perf command {p.error}")
- file_name = p.Value()
- if not os.path.exists(file_name):
- raise Exception(f"perf command input file '{file_name}' not found")
- return file_name
-
-def ReadOption(args, short_name, long_name, err_prefix, remove=False):
- p = OptPos(args, short_name, long_name)
- if p.error:
- raise Exception(f"{err_prefix}{p.error}")
- value = p.Value()
- if remove:
- p.Remove(args)
- return value
-
-def ExtractOption(args, short_name, long_name, err_prefix):
- return ReadOption(args, short_name, long_name, err_prefix, True)
-
-def ReadPerfOption(args, short_name, long_name):
- return ReadOption(args, short_name, long_name, "perf command ")
-
-def ExtractPerfOption(args, short_name, long_name):
- return ExtractOption(args, short_name, long_name, "perf command ")
-
-def PerfDoubleQuickCommands(cmd, file_name):
- cpu_str = ReadPerfOption(cmd, "C", "cpu")
- time_str = ReadPerfOption(cmd, "", "time")
- # Use double-quick sampling to determine trace data density
- times_cmd = ["perf", "script", "--ns", "--input", file_name, "--itrace=qqi"]
- if cpu_str != None and cpu_str != "":
- times_cmd.append(f"--cpu={cpu_str}")
- if time_str != None and time_str != "":
- times_cmd.append(f"--time={time_str}")
- cnts_cmd = list(times_cmd)
- cnts_cmd.append("-Fcpu")
- times_cmd.append("-Fcpu,time")
- return cnts_cmd, times_cmd
-
-class CPUTimeRange():
- def __init__(self, cpu):
- self.cpu = cpu
- self.sample_cnt = 0
- self.time_ranges = None
- self.interval = 0
- self.interval_remaining = 0
- self.remaining = 0
- self.tr_pos = 0
-
-def CalcTimeRangesByCPU(line, cpu, cpu_time_ranges, max_time):
- cpu_time_range = cpu_time_ranges[cpu]
- cpu_time_range.remaining -= 1
- cpu_time_range.interval_remaining -= 1
- if cpu_time_range.remaining == 0:
- cpu_time_range.time_ranges[cpu_time_range.tr_pos][1] = max_time
- return
- if cpu_time_range.interval_remaining == 0:
- time = TimeVal(line[1][:-1], 0)
- time_ranges = cpu_time_range.time_ranges
- time_ranges[cpu_time_range.tr_pos][1] = time - 1
- time_ranges.append([time, max_time])
- cpu_time_range.tr_pos += 1
- cpu_time_range.interval_remaining = cpu_time_range.interval
-
-def CountSamplesByCPU(line, cpu, cpu_time_ranges):
- try:
- cpu_time_ranges[cpu].sample_cnt += 1
- except:
- print("exception")
- print("cpu", cpu)
- print("len(cpu_time_ranges)", len(cpu_time_ranges))
- raise
-
-def ProcessCommandOutputLines(cmd, per_cpu, fn, *x):
- # Assume CPU number is at beginning of line and enclosed by []
- pat = re.compile(r"\s*\[[0-9]+\]")
- p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
- while True:
- line = p.stdout.readline()
- if line:
- line = line.decode("utf-8")
- if pat.match(line):
- line = line.split()
- if per_cpu:
- # Assumes CPU number is enclosed by []
- cpu = int(line[0][1:-1])
- else:
- cpu = 0
- fn(line, cpu, *x)
- else:
- break
- p.wait()
-
-def IntersectTimeRanges(new_time_ranges, time_ranges):
- pos = 0
- new_pos = 0
- # Can assume len(time_ranges) != 0 and len(new_time_ranges) != 0
- # Note also, there *must* be at least one intersection.
- while pos < len(time_ranges) and new_pos < len(new_time_ranges):
- # new end < old start => no intersection, remove new
- if new_time_ranges[new_pos][1] < time_ranges[pos][0]:
- del new_time_ranges[new_pos]
- continue
- # new start > old end => no intersection, check next
- if new_time_ranges[new_pos][0] > time_ranges[pos][1]:
- pos += 1
- if pos < len(time_ranges):
- continue
- # no next, so remove remaining
- while new_pos < len(new_time_ranges):
- del new_time_ranges[new_pos]
- return
- # Found an intersection
- # new start < old start => adjust new start = old start
- if new_time_ranges[new_pos][0] < time_ranges[pos][0]:
- new_time_ranges[new_pos][0] = time_ranges[pos][0]
- # new end > old end => keep the overlap, insert the remainder
- if new_time_ranges[new_pos][1] > time_ranges[pos][1]:
- r = [ time_ranges[pos][1] + 1, new_time_ranges[new_pos][1] ]
- new_time_ranges[new_pos][1] = time_ranges[pos][1]
- new_pos += 1
- new_time_ranges.insert(new_pos, r)
- continue
- # new [start, end] is within old [start, end]
- new_pos += 1
-
-def SplitTimeRangesByTraceDataDensity(time_ranges, cpus, nr, cmd, file_name, per_cpu, min_size, min_interval, verbosity):
- if verbosity.normal:
- print("\rAnalyzing...", flush=True, end=" ")
- if verbosity.verbose:
- print()
- cnts_cmd, times_cmd = PerfDoubleQuickCommands(cmd, file_name)
-
- nr_cpus = cpus[-1] + 1 if per_cpu else 1
- if per_cpu:
- nr_cpus = cpus[-1] + 1
- cpu_time_ranges = [ CPUTimeRange(cpu) for cpu in range(nr_cpus) ]
- else:
- nr_cpus = 1
- cpu_time_ranges = [ CPUTimeRange(-1) ]
-
- if verbosity.debug:
- print("nr_cpus", nr_cpus)
- print("cnts_cmd", cnts_cmd)
- print("times_cmd", times_cmd)
-
- # Count the number of "double quick" samples per CPU
- ProcessCommandOutputLines(cnts_cmd, per_cpu, CountSamplesByCPU, cpu_time_ranges)
-
- tot = 0
- mx = 0
- for cpu_time_range in cpu_time_ranges:
- cnt = cpu_time_range.sample_cnt
- tot += cnt
- if cnt > mx:
- mx = cnt
- if verbosity.debug:
- print("cpu:", cpu_time_range.cpu, "sample_cnt", cnt)
-
- if min_size < 1:
- min_size = 1
-
- if mx < min_size:
- # Too little data to be worth splitting
- if verbosity.debug:
- print("Too little data to split by time")
- if nr == 0:
- nr = 1
- return [ SplitTimeRangesIntoN(time_ranges, nr, min_interval) ]
-
- if nr:
- divisor = nr
- min_size = 1
- else:
- divisor = NumberOfCPUs()
-
- interval = int(round(tot / divisor, 0))
- if interval < min_size:
- interval = min_size
-
- if verbosity.debug:
- print("divisor", divisor)
- print("min_size", min_size)
- print("interval", interval)
-
- min_time = time_ranges[0][0]
- max_time = time_ranges[-1][1]
-
- for cpu_time_range in cpu_time_ranges:
- cnt = cpu_time_range.sample_cnt
- if cnt == 0:
- cpu_time_range.time_ranges = copy.deepcopy(time_ranges)
- continue
- # Adjust target interval for CPU to give approximately equal interval sizes
- # Determine number of intervals, rounding to nearest integer
- n = int(round(cnt / interval, 0))
- if n < 1:
- n = 1
- # Determine interval size, rounding up
- d, m = divmod(cnt, n)
- if m:
- d += 1
- cpu_time_range.interval = d
- cpu_time_range.interval_remaining = d
- cpu_time_range.remaining = cnt
- # Init. time ranges for each CPU with the start time
- cpu_time_range.time_ranges = [ [min_time, max_time] ]
-
- # Set time ranges so that the same number of "double quick" samples
- # will fall into each time range.
- ProcessCommandOutputLines(times_cmd, per_cpu, CalcTimeRangesByCPU, cpu_time_ranges, max_time)
-
- for cpu_time_range in cpu_time_ranges:
- if cpu_time_range.sample_cnt:
- IntersectTimeRanges(cpu_time_range.time_ranges, time_ranges)
-
- return [cpu_time_ranges[cpu].time_ranges for cpu in cpus]
-
-def SplitSingleTimeRangeIntoN(time_range, n):
- if n <= 1:
- return [time_range]
- start = time_range[0]
- end = time_range[1]
- duration = int((end - start + 1) / n)
- if duration < 1:
- return [time_range]
- time_ranges = []
- for i in range(n):
- time_ranges.append([start, start + duration - 1])
- start += duration
- time_ranges[-1][1] = end
- return time_ranges
-
-def TimeRangeDuration(r):
- return r[1] - r[0] + 1
-
-def TotalDuration(time_ranges):
- duration = 0
- for r in time_ranges:
- duration += TimeRangeDuration(r)
- return duration
-
-def SplitTimeRangesByInterval(time_ranges, interval):
- new_ranges = []
- for r in time_ranges:
- duration = TimeRangeDuration(r)
- n = duration / interval
- n = int(round(n, 0))
- new_ranges += SplitSingleTimeRangeIntoN(r, n)
- return new_ranges
-
-def SplitTimeRangesIntoN(time_ranges, n, min_interval):
- if n <= len(time_ranges):
- return time_ranges
- duration = TotalDuration(time_ranges)
- interval = duration / n
- if interval < min_interval:
- interval = min_interval
- return SplitTimeRangesByInterval(time_ranges, interval)
-
-def RecombineTimeRanges(tr):
- new_tr = copy.deepcopy(tr)
- n = len(new_tr)
- i = 1
- while i < len(new_tr):
- # if prev end + 1 == cur start, combine them
- if new_tr[i - 1][1] + 1 == new_tr[i][0]:
- new_tr[i][0] = new_tr[i - 1][0]
- del new_tr[i - 1]
- else:
- i += 1
- return new_tr
-
-def OpenTimeRangeEnds(time_ranges, min_time, max_time):
- if time_ranges[0][0] <= min_time:
- time_ranges[0][0] = None
- if time_ranges[-1][1] >= max_time:
- time_ranges[-1][1] = None
-
-def BadTimeStr(time_str):
- raise Exception(f"perf command bad time option: '{time_str}'\nCheck also 'time of first sample' and 'time of last sample' in perf script --header-only")
-
-def ValidateTimeRanges(time_ranges, time_str):
- n = len(time_ranges)
- for i in range(n):
- start = time_ranges[i][0]
- end = time_ranges[i][1]
- if i != 0 and start <= time_ranges[i - 1][1]:
- BadTimeStr(time_str)
- if start > end:
- BadTimeStr(time_str)
-
-def TimeVal(s, dflt):
- s = s.strip()
- if s == "":
- return dflt
- a = s.split(".")
- if len(a) > 2:
- raise Exception(f"Bad time value'{s}'")
- x = int(a[0])
- if x < 0:
- raise Exception("Negative time not allowed")
- x *= 1000000000
- if len(a) > 1:
- x += int((a[1] + "000000000")[:9])
- return x
-
-def BadCPUStr(cpu_str):
- raise Exception(f"perf command bad cpu option: '{cpu_str}'\nCheck also 'nrcpus avail' in perf script --header-only")
-
-def ParseTimeStr(time_str, min_time, max_time):
- if time_str == None or time_str == "":
- return [[min_time, max_time]]
- time_ranges = []
- for r in time_str.split():
- a = r.split(",")
- if len(a) != 2:
- BadTimeStr(time_str)
- try:
- start = TimeVal(a[0], min_time)
- end = TimeVal(a[1], max_time)
- except:
- BadTimeStr(time_str)
- time_ranges.append([start, end])
- ValidateTimeRanges(time_ranges, time_str)
- return time_ranges
-
-def ParseCPUStr(cpu_str, nr_cpus):
- if cpu_str == None or cpu_str == "":
- return [-1]
- cpus = []
- for r in cpu_str.split(","):
- a = r.split("-")
- if len(a) < 1 or len(a) > 2:
- BadCPUStr(cpu_str)
- try:
- start = int(a[0].strip())
- if len(a) > 1:
- end = int(a[1].strip())
- else:
- end = start
- except:
- BadCPUStr(cpu_str)
- if start < 0 or end < 0 or end < start or end >= nr_cpus:
- BadCPUStr(cpu_str)
- cpus.extend(range(start, end + 1))
- cpus = list(set(cpus)) # Remove duplicates
- cpus.sort()
- return cpus
-
-class ParallelPerf():
-
- def __init__(self, a):
- for arg_name in vars(a):
- setattr(self, arg_name, getattr(a, arg_name))
- self.orig_nr = self.nr
- self.orig_cmd = list(self.cmd)
- self.perf = self.cmd[0]
- if os.path.exists(self.output_dir):
- raise Exception(f"Output '{self.output_dir}' already exists")
- if self.jobs < 0 or self.nr < 0 or self.interval < 0:
- raise Exception("Bad options (negative values): try -h option for help")
- if self.nr != 0 and self.interval != 0:
- raise Exception("Cannot specify number of time subdivisions and time interval")
- if self.jobs == 0:
- self.jobs = NumberOfCPUs()
- if self.nr == 0 and self.interval == 0:
- if self.per_cpu:
- self.nr = 1
- else:
- self.nr = self.jobs
-
- def Init(self):
- if self.verbosity.debug:
- print("cmd", self.cmd)
- self.file_name = DetermineInputFileName(self.cmd)
- self.hdr = ReadHeader(self.perf, self.file_name)
- self.hdr_dict = ParseHeader(self.hdr)
- self.cmd_line = HeaderField(self.hdr_dict, "cmdline")
-
- def ExtractTimeInfo(self):
- self.min_time = TimeVal(HeaderField(self.hdr_dict, "time of first sample"), 0)
- self.max_time = TimeVal(HeaderField(self.hdr_dict, "time of last sample"), 0)
- self.time_str = ExtractPerfOption(self.cmd, "", "time")
- self.time_ranges = ParseTimeStr(self.time_str, self.min_time, self.max_time)
- if self.verbosity.debug:
- print("time_ranges", self.time_ranges)
-
- def ExtractCPUInfo(self):
- if self.per_cpu:
- nr_cpus = int(HeaderField(self.hdr_dict, "nrcpus avail"))
- self.cpu_str = ExtractPerfOption(self.cmd, "C", "cpu")
- if self.cpu_str == None or self.cpu_str == "":
- self.cpus = [ x for x in range(nr_cpus) ]
- else:
- self.cpus = ParseCPUStr(self.cpu_str, nr_cpus)
- else:
- self.cpu_str = None
- self.cpus = [-1]
- if self.verbosity.debug:
- print("cpus", self.cpus)
-
- def IsIntelPT(self):
- return self.cmd_line.find("intel_pt") >= 0
-
- def SplitTimeRanges(self):
- if self.IsIntelPT() and self.interval == 0:
- self.split_time_ranges_for_each_cpu = \
- SplitTimeRangesByTraceDataDensity(self.time_ranges, self.cpus, self.orig_nr,
- self.orig_cmd, self.file_name, self.per_cpu,
- self.min_size, self.min_interval, self.verbosity)
- elif self.nr:
- self.split_time_ranges_for_each_cpu = [ SplitTimeRangesIntoN(self.time_ranges, self.nr, self.min_interval) ]
- else:
- self.split_time_ranges_for_each_cpu = [ SplitTimeRangesByInterval(self.time_ranges, self.interval) ]
-
- def CheckTimeRanges(self):
- for tr in self.split_time_ranges_for_each_cpu:
- # Re-combined time ranges should be the same
- new_tr = RecombineTimeRanges(tr)
- if new_tr != self.time_ranges:
- if self.verbosity.debug:
- print("tr", tr)
- print("new_tr", new_tr)
- raise Exception("Self test failed!")
-
- def OpenTimeRangeEnds(self):
- for time_ranges in self.split_time_ranges_for_each_cpu:
- OpenTimeRangeEnds(time_ranges, self.min_time, self.max_time)
-
- def CreateWorkList(self):
- self.worklist = CreateWorkList(self.cmd, self.pipe_to, self.output_dir, self.cpus, self.split_time_ranges_for_each_cpu)
-
- def PerfDataRecordedPerCPU(self):
- if "--per-thread" in self.cmd_line.split():
- return False
- return True
-
- def DefaultToPerCPU(self):
- # --no-per-cpu option takes precedence
- if self.no_per_cpu:
- return False
- if not self.PerfDataRecordedPerCPU():
- return False
- # Default to per-cpu for Intel PT data that was recorded per-cpu,
- # because decoding can be done for each CPU separately.
- if self.IsIntelPT():
- return True
- return False
-
- def Config(self):
- self.Init()
- self.ExtractTimeInfo()
- if not self.per_cpu:
- self.per_cpu = self.DefaultToPerCPU()
- if self.verbosity.debug:
- print("per_cpu", self.per_cpu)
- self.ExtractCPUInfo()
- self.SplitTimeRanges()
- if self.verbosity.self_test:
- self.CheckTimeRanges()
- # Prefer open-ended time range to starting / ending with min_time / max_time resp.
- self.OpenTimeRangeEnds()
- self.CreateWorkList()
-
- def Run(self):
- if self.dry_run:
- print(len(self.worklist),"jobs:")
- for w in self.worklist:
- print(w.Command())
- return True
- result = RunWork(self.worklist, self.jobs, verbosity=self.verbosity)
- if self.verbosity.verbose:
- print(glb_prog_name, "done")
- return result
-
-def RunParallelPerf(a):
- pp = ParallelPerf(a)
- pp.Config()
- return pp.Run()
-
-def Main(args):
- ap = argparse.ArgumentParser(
- prog=glb_prog_name, formatter_class = argparse.RawDescriptionHelpFormatter,
- description =
-"""
-Run a perf script command multiple times in parallel, using perf script options
---cpu and --time so that each job processes a different chunk of the data.
-""",
- epilog =
-"""
-Follow the options by '--' and then the perf script command e.g.
-
- $ perf record -a -- sleep 10
- $ parallel-perf.py --nr=4 -- perf script --ns
- All jobs finished successfully
- $ tree parallel-perf-output/
- parallel-perf-output/
- ├── time-range-0
- │ ├── cmd.txt
- │ └── out.txt
- ├── time-range-1
- │ ├── cmd.txt
- │ └── out.txt
- ├── time-range-2
- │ ├── cmd.txt
- │ └── out.txt
- └── time-range-3
- ├── cmd.txt
- └── out.txt
- $ find parallel-perf-output -name cmd.txt | sort | xargs grep -H .
- parallel-perf-output/time-range-0/cmd.txt:perf script --time=,9466.504461499 --ns
- parallel-perf-output/time-range-1/cmd.txt:perf script --time=9466.504461500,9469.005396999 --ns
- parallel-perf-output/time-range-2/cmd.txt:perf script --time=9469.005397000,9471.506332499 --ns
- parallel-perf-output/time-range-3/cmd.txt:perf script --time=9471.506332500, --ns
-
-Any perf script command can be used, including the use of perf script options
---dlfilter and --script, so that the benefit of running parallel jobs
-naturally extends to them also.
-
-If option --pipe-to is used, standard output is first piped through that
-command. Beware, if the command fails (e.g. grep with no matches), it will be
-considered a fatal error.
-
-Final standard output is redirected to files named out.txt in separate
-subdirectories under the output directory. Similarly, standard error is
-written to files named err.txt. In addition, files named cmd.txt contain the
-corresponding perf script command. After processing, err.txt files are removed
-if they are empty.
-
-If any job exits with a non-zero exit code, then all jobs are killed and no
-more are started. A message is printed if any job results in a non-empty
-err.txt file.
-
-There is a separate output subdirectory for each time range. If the --per-cpu
-option is used, these are further grouped under cpu-n subdirectories, e.g.
-
- $ parallel-perf.py --per-cpu --nr=2 -- perf script --ns --cpu=0,1
- All jobs finished successfully
- $ tree parallel-perf-output
- parallel-perf-output/
- ├── cpu-0
- │ ├── time-range-0
- │ │ ├── cmd.txt
- │ │ └── out.txt
- │ └── time-range-1
- │ ├── cmd.txt
- │ └── out.txt
- └── cpu-1
- ├── time-range-0
- │ ├── cmd.txt
- │ └── out.txt
- └── time-range-1
- ├── cmd.txt
- └── out.txt
- $ find parallel-perf-output -name cmd.txt | sort | xargs grep -H .
- parallel-perf-output/cpu-0/time-range-0/cmd.txt:perf script --cpu=0 --time=,9469.005396999 --ns
- parallel-perf-output/cpu-0/time-range-1/cmd.txt:perf script --cpu=0 --time=9469.005397000, --ns
- parallel-perf-output/cpu-1/time-range-0/cmd.txt:perf script --cpu=1 --time=,9469.005396999 --ns
- parallel-perf-output/cpu-1/time-range-1/cmd.txt:perf script --cpu=1 --time=9469.005397000, --ns
-
-Subdivisions of time range, and cpus if the --per-cpu option is used, are
-expressed by the --time and --cpu perf script options respectively. If the
-supplied perf script command has a --time option, then that time range is
-subdivided, otherwise the time range given by 'time of first sample' to
-'time of last sample' is used (refer perf script --header-only). Similarly, the
-supplied perf script command may provide a --cpu option, and only those CPUs
-will be processed.
-
-To prevent time intervals becoming too small, the --min-interval option can
-be used.
-
-Note there is special handling for processing Intel PT traces. If an interval is
-not specified and the perf record command contained the intel_pt event, then the
-time range will be subdivided in order to produce subdivisions that contain
-approximately the same amount of trace data. That is accomplished by counting
-double-quick (--itrace=qqi) samples, and choosing time ranges that encompass
-approximately the same number of samples. In that case, time ranges may not be
-the same for each CPU processed. For Intel PT, --per-cpu is the default, but
-that can be overridden by --no-per-cpu. Note, for Intel PT, double-quick
-decoding produces 1 sample for each PSB synchronization packet, which in turn
-come after a certain number of bytes output, determined by psb_period (refer
-perf Intel PT documentation). The minimum number of double-quick samples that
-will define a time range can be set by the --min_size option, which defaults to
-64.
-""")
- ap.add_argument("-o", "--output-dir", default="parallel-perf-output", help="output directory (default 'parallel-perf-output')")
- ap.add_argument("-j", "--jobs", type=int, default=0, help="maximum number of jobs to run in parallel at one time (default is the number of CPUs)")
- ap.add_argument("-n", "--nr", type=int, default=0, help="number of time subdivisions (default is the number of jobs)")
- ap.add_argument("-i", "--interval", type=float, default=0, help="subdivide the time range using this time interval (in seconds e.g. 0.1 for a tenth of a second)")
- ap.add_argument("-c", "--per-cpu", action="store_true", help="process data for each CPU in parallel")
- ap.add_argument("-m", "--min-interval", type=float, default=glb_min_interval, help=f"minimum interval (default {glb_min_interval} seconds)")
- ap.add_argument("-p", "--pipe-to", help="command to pipe output to (optional)")
- ap.add_argument("-N", "--no-per-cpu", action="store_true", help="do not process data for each CPU in parallel")
- ap.add_argument("-b", "--min_size", type=int, default=glb_min_samples, help="minimum data size (for Intel PT in PSBs)")
- ap.add_argument("-D", "--dry-run", action="store_true", help="do not run any jobs, just show the perf script commands")
- ap.add_argument("-q", "--quiet", action="store_true", help="do not print any messages except errors")
- ap.add_argument("-v", "--verbose", action="store_true", help="print more messages")
- ap.add_argument("-d", "--debug", action="store_true", help="print debugging messages")
- cmd_line = list(args)
- try:
- split_pos = cmd_line.index("--")
- cmd = cmd_line[split_pos + 1:]
- args = cmd_line[:split_pos]
- except:
- cmd = None
- args = cmd_line
- a = ap.parse_args(args=args[1:])
- a.cmd = cmd
- a.verbosity = Verbosity(a.quiet, a.verbose, a.debug)
- try:
- if a.cmd == None:
- if len(args) <= 1:
- ap.print_help()
- return True
- raise Exception("Command line must contain '--' before perf command")
- return RunParallelPerf(a)
- except Exception as e:
- print("Fatal error: ", str(e))
- if a.debug:
- raise
- return False
-
-if __name__ == "__main__":
- if not Main(sys.argv):
- sys.exit(1)
diff --git a/tools/perf/tests/shell/script.sh b/tools/perf/tests/shell/script.sh
index 7007f1cdf761..254fc3ae94e7 100755
--- a/tools/perf/tests/shell/script.sh
+++ b/tools/perf/tests/shell/script.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-# perf script tests
# SPDX-License-Identifier: GPL-2.0
+# perf script tests
set -e
@@ -76,7 +76,7 @@ test_parallel_perf()
err=2
return
fi
- pp=$(dirname "$0")/../../scripts/python/parallel-perf.py
+ pp=$(dirname "$0")/../../python/parallel-perf.py
if [ ! -f "${pp}" ] ; then
echo "SKIP: parallel-perf.py script not found "
err=2
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 46/49] perf: Remove libpython support and legacy Python scripts
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (44 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 45/49] perf python: Move and clean up parallel-perf.py Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 47/49] perf Makefile: Update Python script installation path Ian Rogers
` (2 subsequent siblings)
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Remove embedded Python interpreter support (libpython) from perf, as
all Python scripts have been migrated to standalone scripts using the
perf Python extension module.
Changes include:
- Remove libpython detection and build flags from Makefile.config.
- Remove legacy Python script installation rules from Makefile.perf.
- Delete tools/perf/util/scripting-engines/trace-event-python.c and
tools/perf/scripts/python/Perf-Trace-Util/Context.c.
- Remove Python scripting engine registration from
trace-event-scripting.c.
- Remove libpython from the supported features list in builtin-check.c
and Documentation/perf-check.txt.
- Delete the legacy Python scripts and bin wrappers in
tools/perf/scripts/python/ and update shell tests.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/Documentation/perf-check.txt | 1 -
tools/perf/Makefile.config | 26 +-
tools/perf/Makefile.perf | 7 +-
tools/perf/builtin-check.c | 2 +-
tools/perf/scripts/Build | 27 -
.../perf/scripts/python/Perf-Trace-Util/Build | 4 -
.../scripts/python/Perf-Trace-Util/Context.c | 225 --
.../Perf-Trace-Util/lib/Perf/Trace/Core.py | 116 -
.../lib/Perf/Trace/EventClass.py | 97 -
.../lib/Perf/Trace/SchedGui.py | 184 --
.../Perf-Trace-Util/lib/Perf/Trace/Util.py | 92 -
.../scripts/python/arm-cs-trace-disasm.py | 356 ---
.../python/bin/compaction-times-record | 2 -
.../python/bin/compaction-times-report | 4 -
.../python/bin/event_analyzing_sample-record | 8 -
.../python/bin/event_analyzing_sample-report | 3 -
.../python/bin/export-to-postgresql-record | 8 -
.../python/bin/export-to-postgresql-report | 29 -
.../python/bin/export-to-sqlite-record | 8 -
.../python/bin/export-to-sqlite-report | 29 -
.../python/bin/failed-syscalls-by-pid-record | 3 -
.../python/bin/failed-syscalls-by-pid-report | 10 -
.../perf/scripts/python/bin/flamegraph-record | 2 -
.../perf/scripts/python/bin/flamegraph-report | 3 -
.../python/bin/futex-contention-record | 2 -
.../python/bin/futex-contention-report | 4 -
tools/perf/scripts/python/bin/gecko-record | 2 -
tools/perf/scripts/python/bin/gecko-report | 7 -
.../scripts/python/bin/intel-pt-events-record | 13 -
.../scripts/python/bin/intel-pt-events-report | 3 -
.../scripts/python/bin/mem-phys-addr-record | 19 -
.../scripts/python/bin/mem-phys-addr-report | 3 -
.../scripts/python/bin/net_dropmonitor-record | 2 -
.../scripts/python/bin/net_dropmonitor-report | 4 -
.../scripts/python/bin/netdev-times-record | 8 -
.../scripts/python/bin/netdev-times-report | 5 -
.../scripts/python/bin/powerpc-hcalls-record | 2 -
.../scripts/python/bin/powerpc-hcalls-report | 2 -
.../scripts/python/bin/sched-migration-record | 2 -
.../scripts/python/bin/sched-migration-report | 3 -
tools/perf/scripts/python/bin/sctop-record | 3 -
tools/perf/scripts/python/bin/sctop-report | 24 -
.../scripts/python/bin/stackcollapse-record | 8 -
.../scripts/python/bin/stackcollapse-report | 3 -
.../python/bin/syscall-counts-by-pid-record | 3 -
.../python/bin/syscall-counts-by-pid-report | 10 -
.../scripts/python/bin/syscall-counts-record | 3 -
.../scripts/python/bin/syscall-counts-report | 10 -
.../scripts/python/bin/task-analyzer-record | 2 -
.../scripts/python/bin/task-analyzer-report | 3 -
tools/perf/scripts/python/check-perf-trace.py | 84 -
tools/perf/scripts/python/compaction-times.py | 311 ---
.../scripts/python/event_analyzing_sample.py | 192 --
.../scripts/python/export-to-postgresql.py | 1114 --------
tools/perf/scripts/python/export-to-sqlite.py | 799 ------
.../scripts/python/failed-syscalls-by-pid.py | 79 -
tools/perf/scripts/python/flamegraph.py | 267 --
tools/perf/scripts/python/futex-contention.py | 57 -
tools/perf/scripts/python/gecko.py | 395 ---
tools/perf/scripts/python/intel-pt-events.py | 494 ----
tools/perf/scripts/python/libxed.py | 107 -
tools/perf/scripts/python/mem-phys-addr.py | 127 -
tools/perf/scripts/python/net_dropmonitor.py | 78 -
tools/perf/scripts/python/netdev-times.py | 473 ----
tools/perf/scripts/python/powerpc-hcalls.py | 335 ---
tools/perf/scripts/python/sched-migration.py | 462 ----
tools/perf/scripts/python/sctop.py | 89 -
tools/perf/scripts/python/stackcollapse.py | 127 -
tools/perf/scripts/python/stat-cpi.py | 79 -
.../scripts/python/syscall-counts-by-pid.py | 75 -
tools/perf/scripts/python/syscall-counts.py | 65 -
tools/perf/scripts/python/task-analyzer.py | 934 -------
tools/perf/tests/shell/script.sh | 39 -
tools/perf/tests/shell/script_python.sh | 113 -
tools/perf/util/scripting-engines/Build | 4 -
.../scripting-engines/trace-event-python.c | 2333 -----------------
tools/perf/util/trace-event-scripting.c | 9 -
77 files changed, 5 insertions(+), 10632 deletions(-)
delete mode 100644 tools/perf/scripts/python/Perf-Trace-Util/Build
delete mode 100644 tools/perf/scripts/python/Perf-Trace-Util/Context.c
delete mode 100644 tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/Core.py
delete mode 100755 tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/EventClass.py
delete mode 100644 tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/SchedGui.py
delete mode 100644 tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/Util.py
delete mode 100755 tools/perf/scripts/python/arm-cs-trace-disasm.py
delete mode 100644 tools/perf/scripts/python/bin/compaction-times-record
delete mode 100644 tools/perf/scripts/python/bin/compaction-times-report
delete mode 100644 tools/perf/scripts/python/bin/event_analyzing_sample-record
delete mode 100644 tools/perf/scripts/python/bin/event_analyzing_sample-report
delete mode 100644 tools/perf/scripts/python/bin/export-to-postgresql-record
delete mode 100644 tools/perf/scripts/python/bin/export-to-postgresql-report
delete mode 100644 tools/perf/scripts/python/bin/export-to-sqlite-record
delete mode 100644 tools/perf/scripts/python/bin/export-to-sqlite-report
delete mode 100644 tools/perf/scripts/python/bin/failed-syscalls-by-pid-record
delete mode 100644 tools/perf/scripts/python/bin/failed-syscalls-by-pid-report
delete mode 100755 tools/perf/scripts/python/bin/flamegraph-record
delete mode 100755 tools/perf/scripts/python/bin/flamegraph-report
delete mode 100644 tools/perf/scripts/python/bin/futex-contention-record
delete mode 100644 tools/perf/scripts/python/bin/futex-contention-report
delete mode 100644 tools/perf/scripts/python/bin/gecko-record
delete mode 100755 tools/perf/scripts/python/bin/gecko-report
delete mode 100644 tools/perf/scripts/python/bin/intel-pt-events-record
delete mode 100644 tools/perf/scripts/python/bin/intel-pt-events-report
delete mode 100644 tools/perf/scripts/python/bin/mem-phys-addr-record
delete mode 100644 tools/perf/scripts/python/bin/mem-phys-addr-report
delete mode 100755 tools/perf/scripts/python/bin/net_dropmonitor-record
delete mode 100755 tools/perf/scripts/python/bin/net_dropmonitor-report
delete mode 100644 tools/perf/scripts/python/bin/netdev-times-record
delete mode 100644 tools/perf/scripts/python/bin/netdev-times-report
delete mode 100644 tools/perf/scripts/python/bin/powerpc-hcalls-record
delete mode 100644 tools/perf/scripts/python/bin/powerpc-hcalls-report
delete mode 100644 tools/perf/scripts/python/bin/sched-migration-record
delete mode 100644 tools/perf/scripts/python/bin/sched-migration-report
delete mode 100644 tools/perf/scripts/python/bin/sctop-record
delete mode 100644 tools/perf/scripts/python/bin/sctop-report
delete mode 100755 tools/perf/scripts/python/bin/stackcollapse-record
delete mode 100755 tools/perf/scripts/python/bin/stackcollapse-report
delete mode 100644 tools/perf/scripts/python/bin/syscall-counts-by-pid-record
delete mode 100644 tools/perf/scripts/python/bin/syscall-counts-by-pid-report
delete mode 100644 tools/perf/scripts/python/bin/syscall-counts-record
delete mode 100644 tools/perf/scripts/python/bin/syscall-counts-report
delete mode 100755 tools/perf/scripts/python/bin/task-analyzer-record
delete mode 100755 tools/perf/scripts/python/bin/task-analyzer-report
delete mode 100644 tools/perf/scripts/python/check-perf-trace.py
delete mode 100644 tools/perf/scripts/python/compaction-times.py
delete mode 100644 tools/perf/scripts/python/event_analyzing_sample.py
delete mode 100644 tools/perf/scripts/python/export-to-postgresql.py
delete mode 100644 tools/perf/scripts/python/export-to-sqlite.py
delete mode 100644 tools/perf/scripts/python/failed-syscalls-by-pid.py
delete mode 100755 tools/perf/scripts/python/flamegraph.py
delete mode 100644 tools/perf/scripts/python/futex-contention.py
delete mode 100644 tools/perf/scripts/python/gecko.py
delete mode 100644 tools/perf/scripts/python/intel-pt-events.py
delete mode 100644 tools/perf/scripts/python/libxed.py
delete mode 100644 tools/perf/scripts/python/mem-phys-addr.py
delete mode 100755 tools/perf/scripts/python/net_dropmonitor.py
delete mode 100644 tools/perf/scripts/python/netdev-times.py
delete mode 100644 tools/perf/scripts/python/powerpc-hcalls.py
delete mode 100644 tools/perf/scripts/python/sched-migration.py
delete mode 100644 tools/perf/scripts/python/sctop.py
delete mode 100755 tools/perf/scripts/python/stackcollapse.py
delete mode 100644 tools/perf/scripts/python/stat-cpi.py
delete mode 100644 tools/perf/scripts/python/syscall-counts-by-pid.py
delete mode 100644 tools/perf/scripts/python/syscall-counts.py
delete mode 100755 tools/perf/scripts/python/task-analyzer.py
delete mode 100755 tools/perf/tests/shell/script_python.sh
delete mode 100644 tools/perf/util/scripting-engines/trace-event-python.c
diff --git a/tools/perf/Documentation/perf-check.txt b/tools/perf/Documentation/perf-check.txt
index 09e1d35677f5..3d169e5bb372 100644
--- a/tools/perf/Documentation/perf-check.txt
+++ b/tools/perf/Documentation/perf-check.txt
@@ -60,7 +60,6 @@ feature::
libopencsd / HAVE_CSTRACE_SUPPORT
libperl / HAVE_LIBPERL_SUPPORT
libpfm4 / HAVE_LIBPFM
- libpython / HAVE_LIBPYTHON_SUPPORT
libslang / HAVE_SLANG_SUPPORT
libtraceevent / HAVE_LIBTRACEEVENT
libunwind / HAVE_LIBUNWIND_SUPPORT
diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index 4ee7393a39f9..6e03d8f808b5 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -793,26 +793,7 @@ ifdef GTK4
endif
endif
-ifdef LIBPERL
- PERL_EMBED_LDOPTS = $(shell perl -MExtUtils::Embed -e ldopts 2>/dev/null)
- PERL_EMBED_LDFLAGS = $(call strip-libs,$(PERL_EMBED_LDOPTS))
- PERL_EMBED_LIBADD = $(call grep-libs,$(PERL_EMBED_LDOPTS))
- PERL_EMBED_CCOPTS = $(shell perl -MExtUtils::Embed -e ccopts 2>/dev/null)
- PERL_EMBED_CCOPTS := $(filter-out -specs=%,$(PERL_EMBED_CCOPTS))
- PERL_EMBED_CCOPTS := $(filter-out -flto% -ffat-lto-objects, $(PERL_EMBED_CCOPTS))
- PERL_EMBED_LDOPTS := $(filter-out -specs=%,$(PERL_EMBED_LDOPTS))
- FLAGS_PERL_EMBED=$(PERL_EMBED_CCOPTS) $(PERL_EMBED_LDOPTS)
-
- $(call feature_check,libperl)
- ifneq ($(feature-libperl), 1)
- $(error Missing perl devel files. Please install perl-ExtUtils-Embed/libperl-dev)
- else
- LDFLAGS += $(PERL_EMBED_LDFLAGS)
- EXTLIBS += $(PERL_EMBED_LIBADD)
- CFLAGS += -DHAVE_LIBPERL_SUPPORT
- $(call detected,CONFIG_LIBPERL)
- endif
-endif
+
ifeq ($(feature-timerfd), 1)
CFLAGS += -DHAVE_TIMERFD_SUPPORT
@@ -844,8 +825,7 @@ else
ifneq ($(feature-libpython), 1)
$(call disable-python,No 'Python.h' was found: disables Python support - please install python-devel/python-dev)
else
- LDFLAGS += $(PYTHON_EMBED_LDFLAGS)
- EXTLIBS += $(PYTHON_EMBED_LIBADD)
+ CFLAGS += -DHAVE_LIBPYTHON_SUPPORT
PYTHON_SETUPTOOLS_INSTALLED := $(shell $(PYTHON) -c 'import setuptools;' 2> /dev/null && echo "yes" || echo "no")
ifeq ($(PYTHON_SETUPTOOLS_INSTALLED), yes)
PYTHON_EXTENSION_SUFFIX := $(shell $(PYTHON) -c 'from importlib import machinery; print(machinery.EXTENSION_SUFFIXES[0])')
@@ -856,8 +836,6 @@ else
else
$(warning Missing python setuptools, the python binding won't be built, please install python3-setuptools or equivalent)
endif
- CFLAGS += -DHAVE_LIBPYTHON_SUPPORT
- $(call detected,CONFIG_LIBPYTHON)
ifeq ($(filter -fPIC,$(CFLAGS)),)
# Building a shared library requires position independent code.
CFLAGS += -fPIC
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index abd377c16435..45e5a860d537 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -900,11 +900,8 @@ ifdef LIBPERL
endif
ifndef NO_LIBPYTHON
$(call QUIET_INSTALL, python-scripts) \
- $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/Perf-Trace-Util/lib/Perf/Trace'; \
- $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/bin'; \
- $(INSTALL) scripts/python/Perf-Trace-Util/lib/Perf/Trace/* -m 644 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/Perf-Trace-Util/lib/Perf/Trace'; \
- $(INSTALL) scripts/python/*.py -m 644 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'; \
- $(INSTALL) scripts/python/bin/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/bin'
+ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'; \
+ $(INSTALL) python/*.py -m 644 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'
$(call QUIET_INSTALL, python-scripts-standalone) \
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/python'; \
$(INSTALL) python/*.py -m 755 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/python'
diff --git a/tools/perf/builtin-check.c b/tools/perf/builtin-check.c
index 60437650c50f..35272aaeb613 100644
--- a/tools/perf/builtin-check.c
+++ b/tools/perf/builtin-check.c
@@ -52,8 +52,8 @@ struct feature_status supported_features[] = {
FEATURE_STATUS("libnuma", HAVE_LIBNUMA_SUPPORT),
FEATURE_STATUS("libopencsd", HAVE_CSTRACE_SUPPORT),
FEATURE_STATUS_TIP("libperl", HAVE_LIBPERL_SUPPORT, "Deprecated, use LIBPERL=1 and install perl-ExtUtils-Embed/libperl-dev to build with it"),
+ FEATURE_STATUS("python-module", HAVE_LIBPYTHON_SUPPORT),
FEATURE_STATUS("libpfm4", HAVE_LIBPFM),
- FEATURE_STATUS("libpython", HAVE_LIBPYTHON_SUPPORT),
FEATURE_STATUS("libslang", HAVE_SLANG_SUPPORT),
FEATURE_STATUS("libtraceevent", HAVE_LIBTRACEEVENT),
FEATURE_STATUS_TIP("libunwind", HAVE_LIBUNWIND_SUPPORT, "Deprecated, use LIBUNWIND=1 and install libunwind-dev[el] to build with it"),
diff --git a/tools/perf/scripts/Build b/tools/perf/scripts/Build
index 91229a1fe3ff..fbeab8fff88b 100644
--- a/tools/perf/scripts/Build
+++ b/tools/perf/scripts/Build
@@ -1,30 +1,3 @@
ifeq ($(CONFIG_LIBTRACEEVENT),y)
perf-util-$(CONFIG_LIBPERL) += perl/Perf-Trace-Util/
endif
-perf-util-$(CONFIG_LIBPYTHON) += python/Perf-Trace-Util/
-
-ifdef MYPY
- PY_TESTS := $(shell find python -type f -name '*.py')
- MYPY_TEST_LOGS := $(PY_TESTS:python/%=python/%.mypy_log)
-else
- MYPY_TEST_LOGS :=
-endif
-
-$(OUTPUT)%.mypy_log: %
- $(call rule_mkdir)
- $(Q)$(call echo-cmd,test)mypy "$<" > $@ || (cat $@ && rm $@ && false)
-
-perf-y += $(MYPY_TEST_LOGS)
-
-ifdef PYLINT
- PY_TESTS := $(shell find python -type f -name '*.py')
- PYLINT_TEST_LOGS := $(PY_TESTS:python/%=python/%.pylint_log)
-else
- PYLINT_TEST_LOGS :=
-endif
-
-$(OUTPUT)%.pylint_log: %
- $(call rule_mkdir)
- $(Q)$(call echo-cmd,test)pylint "$<" > $@ || (cat $@ && rm $@ && false)
-
-perf-y += $(PYLINT_TEST_LOGS)
diff --git a/tools/perf/scripts/python/Perf-Trace-Util/Build b/tools/perf/scripts/python/Perf-Trace-Util/Build
deleted file mode 100644
index be3710c61320..000000000000
--- a/tools/perf/scripts/python/Perf-Trace-Util/Build
+++ /dev/null
@@ -1,4 +0,0 @@
-perf-util-y += Context.o
-
-# -Wno-declaration-after-statement: The python headers have mixed code with declarations (decls after asserts, for instance)
-CFLAGS_Context.o += $(PYTHON_EMBED_CCOPTS) -Wno-redundant-decls -Wno-strict-prototypes -Wno-unused-parameter -Wno-nested-externs -Wno-declaration-after-statement
diff --git a/tools/perf/scripts/python/Perf-Trace-Util/Context.c b/tools/perf/scripts/python/Perf-Trace-Util/Context.c
deleted file mode 100644
index c19f44610983..000000000000
--- a/tools/perf/scripts/python/Perf-Trace-Util/Context.c
+++ /dev/null
@@ -1,225 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * Context.c. Python interfaces for perf script.
- *
- * Copyright (C) 2010 Tom Zanussi <tzanussi@gmail.com>
- */
-
-/*
- * Use Py_ssize_t for '#' formats to avoid DeprecationWarning: PY_SSIZE_T_CLEAN
- * will be required for '#' formats.
- */
-#define PY_SSIZE_T_CLEAN
-
-#include <Python.h>
-#include "../../../util/config.h"
-#include "../../../util/trace-event.h"
-#include "../../../util/event.h"
-#include "../../../util/symbol.h"
-#include "../../../util/thread.h"
-#include "../../../util/map.h"
-#include "../../../util/maps.h"
-#include "../../../util/auxtrace.h"
-#include "../../../util/session.h"
-#include "../../../util/srcline.h"
-#include "../../../util/srccode.h"
-
-#define _PyCapsule_GetPointer(arg1, arg2) \
- PyCapsule_GetPointer((arg1), (arg2))
-#define _PyBytes_FromStringAndSize(arg1, arg2) \
- PyBytes_FromStringAndSize((arg1), (arg2))
-#define _PyUnicode_AsUTF8(arg) \
- PyUnicode_AsUTF8(arg)
-
-PyMODINIT_FUNC PyInit_perf_trace_context(void);
-
-static struct scripting_context *get_args(PyObject *args, const char *name, PyObject **arg2)
-{
- int cnt = 1 + !!arg2;
- PyObject *context;
-
- if (!PyArg_UnpackTuple(args, name, 1, cnt, &context, arg2))
- return NULL;
-
- return _PyCapsule_GetPointer(context, NULL);
-}
-
-static struct scripting_context *get_scripting_context(PyObject *args)
-{
- return get_args(args, "context", NULL);
-}
-
-#ifdef HAVE_LIBTRACEEVENT
-static PyObject *perf_trace_context_common_pc(PyObject *obj, PyObject *args)
-{
- struct scripting_context *c = get_scripting_context(args);
-
- if (!c)
- return NULL;
-
- return Py_BuildValue("i", common_pc(c));
-}
-
-static PyObject *perf_trace_context_common_flags(PyObject *obj,
- PyObject *args)
-{
- struct scripting_context *c = get_scripting_context(args);
-
- if (!c)
- return NULL;
-
- return Py_BuildValue("i", common_flags(c));
-}
-
-static PyObject *perf_trace_context_common_lock_depth(PyObject *obj,
- PyObject *args)
-{
- struct scripting_context *c = get_scripting_context(args);
-
- if (!c)
- return NULL;
-
- return Py_BuildValue("i", common_lock_depth(c));
-}
-#endif
-
-static PyObject *perf_sample_insn(PyObject *obj, PyObject *args)
-{
- struct scripting_context *c = get_scripting_context(args);
-
- if (!c)
- return NULL;
-
- if (c->sample->ip && !c->sample->insn_len && thread__maps(c->al->thread)) {
- struct machine *machine = maps__machine(thread__maps(c->al->thread));
-
- perf_sample__fetch_insn(c->sample, c->al->thread, machine);
- }
- if (!c->sample->insn_len)
- Py_RETURN_NONE; /* N.B. This is a return statement */
-
- return _PyBytes_FromStringAndSize(c->sample->insn, c->sample->insn_len);
-}
-
-static PyObject *perf_set_itrace_options(PyObject *obj, PyObject *args)
-{
- struct scripting_context *c;
- const char *itrace_options;
- int retval = -1;
- PyObject *str;
-
- c = get_args(args, "itrace_options", &str);
- if (!c)
- return NULL;
-
- if (!c->session || !c->session->itrace_synth_opts)
- goto out;
-
- if (c->session->itrace_synth_opts->set) {
- retval = 1;
- goto out;
- }
-
- itrace_options = _PyUnicode_AsUTF8(str);
-
- retval = itrace_do_parse_synth_opts(c->session->itrace_synth_opts, itrace_options, 0);
-out:
- return Py_BuildValue("i", retval);
-}
-
-static PyObject *perf_sample_src(PyObject *obj, PyObject *args, bool get_srccode)
-{
- struct scripting_context *c = get_scripting_context(args);
- unsigned int line = 0;
- char *srcfile = NULL;
- char *srccode = NULL;
- PyObject *result;
- struct map *map;
- struct dso *dso;
- int len = 0;
- u64 addr;
-
- if (!c)
- return NULL;
-
- map = c->al->map;
- addr = c->al->addr;
- dso = map ? map__dso(map) : NULL;
-
- if (dso)
- srcfile = get_srcline_split(dso, map__rip_2objdump(map, addr), &line);
-
- if (get_srccode) {
- if (srcfile)
- srccode = find_sourceline(srcfile, line, &len);
- result = Py_BuildValue("(sIs#)", srcfile, line, srccode, (Py_ssize_t)len);
- } else {
- result = Py_BuildValue("(sI)", srcfile, line);
- }
-
- free(srcfile);
-
- return result;
-}
-
-static PyObject *perf_sample_srcline(PyObject *obj, PyObject *args)
-{
- return perf_sample_src(obj, args, false);
-}
-
-static PyObject *perf_sample_srccode(PyObject *obj, PyObject *args)
-{
- return perf_sample_src(obj, args, true);
-}
-
-static PyObject *__perf_config_get(PyObject *obj, PyObject *args)
-{
- const char *config_name;
-
- if (!PyArg_ParseTuple(args, "s", &config_name))
- return NULL;
- return Py_BuildValue("s", perf_config_get(config_name));
-}
-
-static PyMethodDef ContextMethods[] = {
-#ifdef HAVE_LIBTRACEEVENT
- { "common_pc", perf_trace_context_common_pc, METH_VARARGS,
- "Get the common preempt count event field value."},
- { "common_flags", perf_trace_context_common_flags, METH_VARARGS,
- "Get the common flags event field value."},
- { "common_lock_depth", perf_trace_context_common_lock_depth,
- METH_VARARGS, "Get the common lock depth event field value."},
-#endif
- { "perf_sample_insn", perf_sample_insn,
- METH_VARARGS, "Get the machine code instruction."},
- { "perf_set_itrace_options", perf_set_itrace_options,
- METH_VARARGS, "Set --itrace options."},
- { "perf_sample_srcline", perf_sample_srcline,
- METH_VARARGS, "Get source file name and line number."},
- { "perf_sample_srccode", perf_sample_srccode,
- METH_VARARGS, "Get source file name, line number and line."},
- { "perf_config_get", __perf_config_get, METH_VARARGS, "Get perf config entry"},
- { NULL, NULL, 0, NULL}
-};
-
-PyMODINIT_FUNC PyInit_perf_trace_context(void)
-{
- static struct PyModuleDef moduledef = {
- PyModuleDef_HEAD_INIT,
- "perf_trace_context", /* m_name */
- "", /* m_doc */
- -1, /* m_size */
- ContextMethods, /* m_methods */
- NULL, /* m_reload */
- NULL, /* m_traverse */
- NULL, /* m_clear */
- NULL, /* m_free */
- };
- PyObject *mod;
-
- mod = PyModule_Create(&moduledef);
- /* Add perf_script_context to the module so it can be imported */
- PyObject_SetAttrString(mod, "perf_script_context", Py_None);
-
- return mod;
-}
diff --git a/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/Core.py b/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/Core.py
deleted file mode 100644
index 54ace2f6bc36..000000000000
--- a/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/Core.py
+++ /dev/null
@@ -1,116 +0,0 @@
-# Core.py - Python extension for perf script, core functions
-#
-# Copyright (C) 2010 by Tom Zanussi <tzanussi@gmail.com>
-#
-# This software may be distributed under the terms of the GNU General
-# Public License ("GPL") version 2 as published by the Free Software
-# Foundation.
-
-from collections import defaultdict
-
-def autodict():
- return defaultdict(autodict)
-
-flag_fields = autodict()
-symbolic_fields = autodict()
-
-def define_flag_field(event_name, field_name, delim):
- flag_fields[event_name][field_name]['delim'] = delim
-
-def define_flag_value(event_name, field_name, value, field_str):
- flag_fields[event_name][field_name]['values'][value] = field_str
-
-def define_symbolic_field(event_name, field_name):
- # nothing to do, really
- pass
-
-def define_symbolic_value(event_name, field_name, value, field_str):
- symbolic_fields[event_name][field_name]['values'][value] = field_str
-
-def flag_str(event_name, field_name, value):
- string = ""
-
- if flag_fields[event_name][field_name]:
- print_delim = 0
- for idx in sorted(flag_fields[event_name][field_name]['values']):
- if not value and not idx:
- string += flag_fields[event_name][field_name]['values'][idx]
- break
- if idx and (value & idx) == idx:
- if print_delim and flag_fields[event_name][field_name]['delim']:
- string += " " + flag_fields[event_name][field_name]['delim'] + " "
- string += flag_fields[event_name][field_name]['values'][idx]
- print_delim = 1
- value &= ~idx
-
- return string
-
-def symbol_str(event_name, field_name, value):
- string = ""
-
- if symbolic_fields[event_name][field_name]:
- for idx in sorted(symbolic_fields[event_name][field_name]['values']):
- if not value and not idx:
- string = symbolic_fields[event_name][field_name]['values'][idx]
- break
- if (value == idx):
- string = symbolic_fields[event_name][field_name]['values'][idx]
- break
-
- return string
-
-trace_flags = { 0x00: "NONE", \
- 0x01: "IRQS_OFF", \
- 0x02: "IRQS_NOSUPPORT", \
- 0x04: "NEED_RESCHED", \
- 0x08: "HARDIRQ", \
- 0x10: "SOFTIRQ" }
-
-def trace_flag_str(value):
- string = ""
- print_delim = 0
-
- for idx in trace_flags:
- if not value and not idx:
- string += "NONE"
- break
-
- if idx and (value & idx) == idx:
- if print_delim:
- string += " | ";
- string += trace_flags[idx]
- print_delim = 1
- value &= ~idx
-
- return string
-
-
-def taskState(state):
- states = {
- 0 : "R",
- 1 : "S",
- 2 : "D",
- 64: "DEAD"
- }
-
- if state not in states:
- return "Unknown"
-
- return states[state]
-
-
-class EventHeaders:
- def __init__(self, common_cpu, common_secs, common_nsecs,
- common_pid, common_comm, common_callchain):
- self.cpu = common_cpu
- self.secs = common_secs
- self.nsecs = common_nsecs
- self.pid = common_pid
- self.comm = common_comm
- self.callchain = common_callchain
-
- def ts(self):
- return (self.secs * (10 ** 9)) + self.nsecs
-
- def ts_format(self):
- return "%d.%d" % (self.secs, int(self.nsecs / 1000))
diff --git a/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/EventClass.py b/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/EventClass.py
deleted file mode 100755
index 21a7a1298094..000000000000
--- a/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/EventClass.py
+++ /dev/null
@@ -1,97 +0,0 @@
-# EventClass.py
-# SPDX-License-Identifier: GPL-2.0
-#
-# This is a library defining some events types classes, which could
-# be used by other scripts to analyzing the perf samples.
-#
-# Currently there are just a few classes defined for examples,
-# PerfEvent is the base class for all perf event sample, PebsEvent
-# is a HW base Intel x86 PEBS event, and user could add more SW/HW
-# event classes based on requirements.
-from __future__ import print_function
-
-import struct
-
-# Event types, user could add more here
-EVTYPE_GENERIC = 0
-EVTYPE_PEBS = 1 # Basic PEBS event
-EVTYPE_PEBS_LL = 2 # PEBS event with load latency info
-EVTYPE_IBS = 3
-
-#
-# Currently we don't have good way to tell the event type, but by
-# the size of raw buffer, raw PEBS event with load latency data's
-# size is 176 bytes, while the pure PEBS event's size is 144 bytes.
-#
-def create_event(name, comm, dso, symbol, raw_buf):
- if (len(raw_buf) == 144):
- event = PebsEvent(name, comm, dso, symbol, raw_buf)
- elif (len(raw_buf) == 176):
- event = PebsNHM(name, comm, dso, symbol, raw_buf)
- else:
- event = PerfEvent(name, comm, dso, symbol, raw_buf)
-
- return event
-
-class PerfEvent(object):
- event_num = 0
- def __init__(self, name, comm, dso, symbol, raw_buf, ev_type=EVTYPE_GENERIC):
- self.name = name
- self.comm = comm
- self.dso = dso
- self.symbol = symbol
- self.raw_buf = raw_buf
- self.ev_type = ev_type
- PerfEvent.event_num += 1
-
- def show(self):
- print("PMU event: name=%12s, symbol=%24s, comm=%8s, dso=%12s" %
- (self.name, self.symbol, self.comm, self.dso))
-
-#
-# Basic Intel PEBS (Precise Event-based Sampling) event, whose raw buffer
-# contains the context info when that event happened: the EFLAGS and
-# linear IP info, as well as all the registers.
-#
-class PebsEvent(PerfEvent):
- pebs_num = 0
- def __init__(self, name, comm, dso, symbol, raw_buf, ev_type=EVTYPE_PEBS):
- tmp_buf=raw_buf[0:80]
- flags, ip, ax, bx, cx, dx, si, di, bp, sp = struct.unpack('QQQQQQQQQQ', tmp_buf)
- self.flags = flags
- self.ip = ip
- self.ax = ax
- self.bx = bx
- self.cx = cx
- self.dx = dx
- self.si = si
- self.di = di
- self.bp = bp
- self.sp = sp
-
- PerfEvent.__init__(self, name, comm, dso, symbol, raw_buf, ev_type)
- PebsEvent.pebs_num += 1
- del tmp_buf
-
-#
-# Intel Nehalem and Westmere support PEBS plus Load Latency info which lie
-# in the four 64 bit words write after the PEBS data:
-# Status: records the IA32_PERF_GLOBAL_STATUS register value
-# DLA: Data Linear Address (EIP)
-# DSE: Data Source Encoding, where the latency happens, hit or miss
-# in L1/L2/L3 or IO operations
-# LAT: the actual latency in cycles
-#
-class PebsNHM(PebsEvent):
- pebs_nhm_num = 0
- def __init__(self, name, comm, dso, symbol, raw_buf, ev_type=EVTYPE_PEBS_LL):
- tmp_buf=raw_buf[144:176]
- status, dla, dse, lat = struct.unpack('QQQQ', tmp_buf)
- self.status = status
- self.dla = dla
- self.dse = dse
- self.lat = lat
-
- PebsEvent.__init__(self, name, comm, dso, symbol, raw_buf, ev_type)
- PebsNHM.pebs_nhm_num += 1
- del tmp_buf
diff --git a/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/SchedGui.py b/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/SchedGui.py
deleted file mode 100644
index cac7b2542ee8..000000000000
--- a/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/SchedGui.py
+++ /dev/null
@@ -1,184 +0,0 @@
-# SchedGui.py - Python extension for perf script, basic GUI code for
-# traces drawing and overview.
-#
-# Copyright (C) 2010 by Frederic Weisbecker <fweisbec@gmail.com>
-#
-# This software is distributed under the terms of the GNU General
-# Public License ("GPL") version 2 as published by the Free Software
-# Foundation.
-
-
-try:
- import wx
-except ImportError:
- raise ImportError("You need to install the wxpython lib for this script")
-
-
-class RootFrame(wx.Frame):
- Y_OFFSET = 100
- RECT_HEIGHT = 100
- RECT_SPACE = 50
- EVENT_MARKING_WIDTH = 5
-
- def __init__(self, sched_tracer, title, parent = None, id = -1):
- wx.Frame.__init__(self, parent, id, title)
-
- (self.screen_width, self.screen_height) = wx.GetDisplaySize()
- self.screen_width -= 10
- self.screen_height -= 10
- self.zoom = 0.5
- self.scroll_scale = 20
- self.sched_tracer = sched_tracer
- self.sched_tracer.set_root_win(self)
- (self.ts_start, self.ts_end) = sched_tracer.interval()
- self.update_width_virtual()
- self.nr_rects = sched_tracer.nr_rectangles() + 1
- self.height_virtual = RootFrame.Y_OFFSET + (self.nr_rects * (RootFrame.RECT_HEIGHT + RootFrame.RECT_SPACE))
-
- # whole window panel
- self.panel = wx.Panel(self, size=(self.screen_width, self.screen_height))
-
- # scrollable container
- self.scroll = wx.ScrolledWindow(self.panel)
- self.scroll.SetScrollbars(self.scroll_scale, self.scroll_scale, self.width_virtual / self.scroll_scale, self.height_virtual / self.scroll_scale)
- self.scroll.EnableScrolling(True, True)
- self.scroll.SetFocus()
-
- # scrollable drawing area
- self.scroll_panel = wx.Panel(self.scroll, size=(self.screen_width - 15, self.screen_height / 2))
- self.scroll_panel.Bind(wx.EVT_PAINT, self.on_paint)
- self.scroll_panel.Bind(wx.EVT_KEY_DOWN, self.on_key_press)
- self.scroll_panel.Bind(wx.EVT_LEFT_DOWN, self.on_mouse_down)
- self.scroll.Bind(wx.EVT_PAINT, self.on_paint)
- self.scroll.Bind(wx.EVT_KEY_DOWN, self.on_key_press)
- self.scroll.Bind(wx.EVT_LEFT_DOWN, self.on_mouse_down)
-
- self.scroll.Fit()
- self.Fit()
-
- self.scroll_panel.SetDimensions(-1, -1, self.width_virtual, self.height_virtual, wx.SIZE_USE_EXISTING)
-
- self.txt = None
-
- self.Show(True)
-
- def us_to_px(self, val):
- return val / (10 ** 3) * self.zoom
-
- def px_to_us(self, val):
- return (val / self.zoom) * (10 ** 3)
-
- def scroll_start(self):
- (x, y) = self.scroll.GetViewStart()
- return (x * self.scroll_scale, y * self.scroll_scale)
-
- def scroll_start_us(self):
- (x, y) = self.scroll_start()
- return self.px_to_us(x)
-
- def paint_rectangle_zone(self, nr, color, top_color, start, end):
- offset_px = self.us_to_px(start - self.ts_start)
- width_px = self.us_to_px(end - self.ts_start)
-
- offset_py = RootFrame.Y_OFFSET + (nr * (RootFrame.RECT_HEIGHT + RootFrame.RECT_SPACE))
- width_py = RootFrame.RECT_HEIGHT
-
- dc = self.dc
-
- if top_color is not None:
- (r, g, b) = top_color
- top_color = wx.Colour(r, g, b)
- brush = wx.Brush(top_color, wx.SOLID)
- dc.SetBrush(brush)
- dc.DrawRectangle(offset_px, offset_py, width_px, RootFrame.EVENT_MARKING_WIDTH)
- width_py -= RootFrame.EVENT_MARKING_WIDTH
- offset_py += RootFrame.EVENT_MARKING_WIDTH
-
- (r ,g, b) = color
- color = wx.Colour(r, g, b)
- brush = wx.Brush(color, wx.SOLID)
- dc.SetBrush(brush)
- dc.DrawRectangle(offset_px, offset_py, width_px, width_py)
-
- def update_rectangles(self, dc, start, end):
- start += self.ts_start
- end += self.ts_start
- self.sched_tracer.fill_zone(start, end)
-
- def on_paint(self, event):
- dc = wx.PaintDC(self.scroll_panel)
- self.dc = dc
-
- width = min(self.width_virtual, self.screen_width)
- (x, y) = self.scroll_start()
- start = self.px_to_us(x)
- end = self.px_to_us(x + width)
- self.update_rectangles(dc, start, end)
-
- def rect_from_ypixel(self, y):
- y -= RootFrame.Y_OFFSET
- rect = y / (RootFrame.RECT_HEIGHT + RootFrame.RECT_SPACE)
- height = y % (RootFrame.RECT_HEIGHT + RootFrame.RECT_SPACE)
-
- if rect < 0 or rect > self.nr_rects - 1 or height > RootFrame.RECT_HEIGHT:
- return -1
-
- return rect
-
- def update_summary(self, txt):
- if self.txt:
- self.txt.Destroy()
- self.txt = wx.StaticText(self.panel, -1, txt, (0, (self.screen_height / 2) + 50))
-
-
- def on_mouse_down(self, event):
- (x, y) = event.GetPositionTuple()
- rect = self.rect_from_ypixel(y)
- if rect == -1:
- return
-
- t = self.px_to_us(x) + self.ts_start
-
- self.sched_tracer.mouse_down(rect, t)
-
-
- def update_width_virtual(self):
- self.width_virtual = self.us_to_px(self.ts_end - self.ts_start)
-
- def __zoom(self, x):
- self.update_width_virtual()
- (xpos, ypos) = self.scroll.GetViewStart()
- xpos = self.us_to_px(x) / self.scroll_scale
- self.scroll.SetScrollbars(self.scroll_scale, self.scroll_scale, self.width_virtual / self.scroll_scale, self.height_virtual / self.scroll_scale, xpos, ypos)
- self.Refresh()
-
- def zoom_in(self):
- x = self.scroll_start_us()
- self.zoom *= 2
- self.__zoom(x)
-
- def zoom_out(self):
- x = self.scroll_start_us()
- self.zoom /= 2
- self.__zoom(x)
-
-
- def on_key_press(self, event):
- key = event.GetRawKeyCode()
- if key == ord("+"):
- self.zoom_in()
- return
- if key == ord("-"):
- self.zoom_out()
- return
-
- key = event.GetKeyCode()
- (x, y) = self.scroll.GetViewStart()
- if key == wx.WXK_RIGHT:
- self.scroll.Scroll(x + 1, y)
- elif key == wx.WXK_LEFT:
- self.scroll.Scroll(x - 1, y)
- elif key == wx.WXK_DOWN:
- self.scroll.Scroll(x, y + 1)
- elif key == wx.WXK_UP:
- self.scroll.Scroll(x, y - 1)
diff --git a/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/Util.py b/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/Util.py
deleted file mode 100644
index b75d31858e54..000000000000
--- a/tools/perf/scripts/python/Perf-Trace-Util/lib/Perf/Trace/Util.py
+++ /dev/null
@@ -1,92 +0,0 @@
-# Util.py - Python extension for perf script, miscellaneous utility code
-#
-# Copyright (C) 2010 by Tom Zanussi <tzanussi@gmail.com>
-#
-# This software may be distributed under the terms of the GNU General
-# Public License ("GPL") version 2 as published by the Free Software
-# Foundation.
-from __future__ import print_function
-
-import errno, os
-
-FUTEX_WAIT = 0
-FUTEX_WAKE = 1
-FUTEX_PRIVATE_FLAG = 128
-FUTEX_CLOCK_REALTIME = 256
-FUTEX_CMD_MASK = ~(FUTEX_PRIVATE_FLAG | FUTEX_CLOCK_REALTIME)
-
-NSECS_PER_SEC = 1000000000
-
-def avg(total, n):
- return total / n
-
-def nsecs(secs, nsecs):
- return secs * NSECS_PER_SEC + nsecs
-
-def nsecs_secs(nsecs):
- return nsecs / NSECS_PER_SEC
-
-def nsecs_nsecs(nsecs):
- return nsecs % NSECS_PER_SEC
-
-def nsecs_str(nsecs):
- str = "%5u.%09u" % (nsecs_secs(nsecs), nsecs_nsecs(nsecs)),
- return str
-
-def add_stats(dict, key, value):
- if key not in dict:
- dict[key] = (value, value, value, 1)
- else:
- min, max, avg, count = dict[key]
- if value < min:
- min = value
- if value > max:
- max = value
- avg = (avg + value) / 2
- dict[key] = (min, max, avg, count + 1)
-
-def clear_term():
- print("\x1b[H\x1b[2J")
-
-audit_package_warned = False
-
-try:
- import audit
- machine_to_id = {
- 'x86_64': audit.MACH_86_64,
- 'aarch64': audit.MACH_AARCH64,
- 'alpha' : audit.MACH_ALPHA,
- 'ia64' : audit.MACH_IA64,
- 'ppc' : audit.MACH_PPC,
- 'ppc64' : audit.MACH_PPC64,
- 'ppc64le' : audit.MACH_PPC64LE,
- 's390' : audit.MACH_S390,
- 's390x' : audit.MACH_S390X,
- 'i386' : audit.MACH_X86,
- 'i586' : audit.MACH_X86,
- 'i686' : audit.MACH_X86,
- }
- try:
- machine_to_id['armeb'] = audit.MACH_ARMEB
- except:
- pass
- machine_id = machine_to_id[os.uname()[4]]
-except:
- if not audit_package_warned:
- audit_package_warned = True
- print("Install the python-audit package to get syscall names.\n"
- "For example:\n # apt-get install python3-audit (Ubuntu)"
- "\n # yum install python3-audit (Fedora)"
- "\n etc.\n")
-
-def syscall_name(id):
- try:
- return audit.audit_syscall_to_name(id, machine_id)
- except:
- return str(id)
-
-def strerror(nr):
- try:
- return errno.errorcode[abs(nr)]
- except:
- return "Unknown %d errno" % nr
diff --git a/tools/perf/scripts/python/arm-cs-trace-disasm.py b/tools/perf/scripts/python/arm-cs-trace-disasm.py
deleted file mode 100755
index 42579f858684..000000000000
--- a/tools/perf/scripts/python/arm-cs-trace-disasm.py
+++ /dev/null
@@ -1,356 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# arm-cs-trace-disasm.py: ARM CoreSight Trace Dump With Disassember
-#
-# Author: Tor Jeremiassen <tor@ti.com>
-# Mathieu Poirier <mathieu.poirier@linaro.org>
-# Leo Yan <leo.yan@linaro.org>
-# Al Grant <Al.Grant@arm.com>
-
-from __future__ import print_function
-import os
-from os import path
-import re
-from subprocess import *
-import argparse
-import platform
-
-from perf_trace_context import perf_sample_srccode, perf_config_get
-
-# Below are some example commands for using this script.
-# Note a --kcore recording is required for accurate decode
-# due to the alternatives patching mechanism. In addition to this,
-# source line info comes from Perf, and when using kcore there is
-# no debug info. The following lists the supported features in each mode:
-#
-# +-----------+-----------------+------------------+------------------+
-# | Recording | Accurate decode | Source line dump | Disassembly dump |
-# +-----------+-----------------+------------------+------------------+
-# | --kcore | yes | no | yes |
-# | normal | no | yes (inaccurate) | yes (inaccurate) |
-# +-----------+-----------------+------------------+------------------+
-#
-# Output disassembly with objdump and auto detect vmlinux
-# (when running on same machine.):
-# perf script --itrace=b -s scripts/python/arm-cs-trace-disasm.py \
-# -- -d
-#
-# Output disassembly with llvm-objdump:
-# perf script --itrace=b -s scripts/python/arm-cs-trace-disasm.py \
-# -- -d llvm-objdump-11 -k path/to/vmlinux
-#
-# Output accurate disassembly by passing kcore to script:
-# perf script --itrace=b -s scripts/python/arm-cs-trace-disasm.py \
-# -- -d -k perf.data/kcore_dir/kcore
-#
-# Output only source line and symbols:
-# perf script --itrace=b -s scripts/python/arm-cs-trace-disasm.py
-
-def default_objdump():
- config = perf_config_get("annotate.objdump")
- return config if config else "objdump"
-
-# Command line parsing.
-def int_arg(v):
- v = int(v)
- if v < 0:
- raise argparse.ArgumentTypeError("Argument must be a positive integer")
- return v
-
-args = argparse.ArgumentParser()
-args.add_argument("-k", "--vmlinux",
- help="Set path to vmlinux or kcore file. Omit to autodetect if running on same machine")
-args.add_argument("-d", "--objdump", nargs="?", const=default_objdump(),
- help="Show disassembly. Can also be used to change the objdump path"),
-args.add_argument("-v", "--verbose", action="store_true", help="Enable debugging log")
-args.add_argument("--start-time", type=int_arg, help="Monotonic clock time of sample to start from. "
- "See 'time' field on samples in -v mode.")
-args.add_argument("--stop-time", type=int_arg, help="Monotonic clock time of sample to stop at. "
- "See 'time' field on samples in -v mode.")
-args.add_argument("--start-sample", type=int_arg, help="Index of sample to start from. "
- "See 'index' field on samples in -v mode.")
-args.add_argument("--stop-sample", type=int_arg, help="Index of sample to stop at. "
- "See 'index' field on samples in -v mode.")
-
-options = args.parse_args()
-if (options.start_time and options.stop_time and
- options.start_time >= options.stop_time):
- print("--start-time must less than --stop-time")
- exit(2)
-if (options.start_sample and options.stop_sample and
- options.start_sample >= options.stop_sample):
- print("--start-sample must less than --stop-sample")
- exit(2)
-
-# Initialize global dicts and regular expression
-disasm_cache = dict()
-cpu_data = dict()
-disasm_re = re.compile(r"^\s*([0-9a-fA-F]+):")
-disasm_func_re = re.compile(r"^\s*([0-9a-fA-F]+)\s.*:")
-cache_size = 64*1024
-sample_idx = -1
-
-glb_source_file_name = None
-glb_line_number = None
-glb_dso = None
-
-kver = platform.release()
-vmlinux_paths = [
- f"/usr/lib/debug/boot/vmlinux-{kver}.debug",
- f"/usr/lib/debug/lib/modules/{kver}/vmlinux",
- f"/lib/modules/{kver}/build/vmlinux",
- f"/usr/lib/debug/boot/vmlinux-{kver}",
- f"/boot/vmlinux-{kver}",
- f"/boot/vmlinux",
- f"vmlinux"
-]
-
-def get_optional(perf_dict, field):
- if field in perf_dict:
- return perf_dict[field]
- return "[unknown]"
-
-def get_offset(perf_dict, field):
- if field in perf_dict:
- return "+%#x" % perf_dict[field]
- return ""
-
-def find_vmlinux():
- if hasattr(find_vmlinux, "path"):
- return find_vmlinux.path
-
- for v in vmlinux_paths:
- if os.access(v, os.R_OK):
- find_vmlinux.path = v
- break
- else:
- find_vmlinux.path = None
-
- return find_vmlinux.path
-
-def get_dso_file_path(dso_name, dso_build_id):
- if (dso_name == "[kernel.kallsyms]" or dso_name == "vmlinux"):
- if (options.vmlinux):
- return options.vmlinux;
- else:
- return find_vmlinux() if find_vmlinux() else dso_name
-
- if (dso_name == "[vdso]") :
- append = "/vdso"
- else:
- append = "/elf"
-
- dso_path = os.environ['PERF_BUILDID_DIR'] + "/" + dso_name + "/" + dso_build_id + append;
- # Replace duplicate slash chars to single slash char
- dso_path = dso_path.replace('//', '/', 1)
- return dso_path
-
-def read_disam(dso_fname, dso_start, start_addr, stop_addr):
- addr_range = str(start_addr) + ":" + str(stop_addr) + ":" + dso_fname
-
- # Don't let the cache get too big, clear it when it hits max size
- if (len(disasm_cache) > cache_size):
- disasm_cache.clear();
-
- if addr_range in disasm_cache:
- disasm_output = disasm_cache[addr_range];
- else:
- start_addr = start_addr - dso_start;
- stop_addr = stop_addr - dso_start;
- disasm = [ options.objdump, "-d", "-z",
- "--start-address="+format(start_addr,"#x"),
- "--stop-address="+format(stop_addr,"#x") ]
- disasm += [ dso_fname ]
- disasm_output = check_output(disasm).decode('utf-8').split('\n')
- disasm_cache[addr_range] = disasm_output
-
- return disasm_output
-
-def print_disam(dso_fname, dso_start, start_addr, stop_addr):
- for line in read_disam(dso_fname, dso_start, start_addr, stop_addr):
- m = disasm_func_re.search(line)
- if m is None:
- m = disasm_re.search(line)
- if m is None:
- continue
- print("\t" + line)
-
-def print_sample(sample):
- print("Sample = { cpu: %04d addr: 0x%016x phys_addr: 0x%016x ip: 0x%016x " \
- "pid: %d tid: %d period: %d time: %d index: %d}" % \
- (sample['cpu'], sample['addr'], sample['phys_addr'], \
- sample['ip'], sample['pid'], sample['tid'], \
- sample['period'], sample['time'], sample_idx))
-
-def trace_begin():
- print('ARM CoreSight Trace Data Assembler Dump')
-
-def trace_end():
- print('End')
-
-def trace_unhandled(event_name, context, event_fields_dict):
- print(' '.join(['%s=%s'%(k,str(v))for k,v in sorted(event_fields_dict.items())]))
-
-def common_start_str(comm, sample):
- sec = int(sample["time"] / 1000000000)
- ns = sample["time"] % 1000000000
- cpu = sample["cpu"]
- pid = sample["pid"]
- tid = sample["tid"]
- return "%16s %5u/%-5u [%04u] %9u.%09u " % (comm, pid, tid, cpu, sec, ns)
-
-# This code is copied from intel-pt-events.py for printing source code
-# line and symbols.
-def print_srccode(comm, param_dict, sample, symbol, dso):
- ip = sample["ip"]
- if symbol == "[unknown]":
- start_str = common_start_str(comm, sample) + ("%x" % ip).rjust(16).ljust(40)
- else:
- offs = get_offset(param_dict, "symoff")
- start_str = common_start_str(comm, sample) + (symbol + offs).ljust(40)
-
- global glb_source_file_name
- global glb_line_number
- global glb_dso
-
- source_file_name, line_number, source_line = perf_sample_srccode(perf_script_context)
- if source_file_name:
- if glb_line_number == line_number and glb_source_file_name == source_file_name:
- src_str = ""
- else:
- if len(source_file_name) > 40:
- src_file = ("..." + source_file_name[-37:]) + " "
- else:
- src_file = source_file_name.ljust(41)
-
- if source_line is None:
- src_str = src_file + str(line_number).rjust(4) + " <source not found>"
- else:
- src_str = src_file + str(line_number).rjust(4) + " " + source_line
- glb_dso = None
- elif dso == glb_dso:
- src_str = ""
- else:
- src_str = dso
- glb_dso = dso
-
- glb_line_number = line_number
- glb_source_file_name = source_file_name
-
- print(start_str, src_str)
-
-def process_event(param_dict):
- global cache_size
- global options
- global sample_idx
-
- sample = param_dict["sample"]
- comm = param_dict["comm"]
-
- name = param_dict["ev_name"]
- dso = get_optional(param_dict, "dso")
- dso_bid = get_optional(param_dict, "dso_bid")
- dso_start = get_optional(param_dict, "dso_map_start")
- dso_end = get_optional(param_dict, "dso_map_end")
- symbol = get_optional(param_dict, "symbol")
- map_pgoff = get_optional(param_dict, "map_pgoff")
- # check for valid map offset
- if (str(map_pgoff) == '[unknown]'):
- map_pgoff = 0
-
- cpu = sample["cpu"]
- ip = sample["ip"]
- addr = sample["addr"]
-
- sample_idx += 1
-
- if (options.start_time and sample["time"] < options.start_time):
- return
- if (options.stop_time and sample["time"] > options.stop_time):
- exit(0)
- if (options.start_sample and sample_idx < options.start_sample):
- return
- if (options.stop_sample and sample_idx > options.stop_sample):
- exit(0)
-
- if (options.verbose == True):
- print("Event type: %s" % name)
- print_sample(sample)
-
- # Initialize CPU data if it's empty, and directly return back
- # if this is the first tracing event for this CPU.
- if (cpu_data.get(str(cpu) + 'addr') == None):
- cpu_data[str(cpu) + 'addr'] = addr
- return
-
- # If cannot find dso so cannot dump assembler, bail out
- if (dso == '[unknown]'):
- return
-
- # Validate dso start and end addresses
- if ((dso_start == '[unknown]') or (dso_end == '[unknown]')):
- print("Failed to find valid dso map for dso %s" % dso)
- return
-
- if (name[0:12] == "instructions"):
- print_srccode(comm, param_dict, sample, symbol, dso)
- return
-
- # Don't proceed if this event is not a branch sample, .
- if (name[0:8] != "branches"):
- return
-
- # The format for packet is:
- #
- # +------------+------------+------------+
- # sample_prev: | addr | ip | cpu |
- # +------------+------------+------------+
- # sample_next: | addr | ip | cpu |
- # +------------+------------+------------+
- #
- # We need to combine the two continuous packets to get the instruction
- # range for sample_prev::cpu:
- #
- # [ sample_prev::addr .. sample_next::ip ]
- #
- # For this purose, sample_prev::addr is stored into cpu_data structure
- # and read back for 'start_addr' when the new packet comes, and we need
- # to use sample_next::ip to calculate 'stop_addr', plusing extra 4 for
- # 'stop_addr' is for the sake of objdump so the final assembler dump can
- # include last instruction for sample_next::ip.
- start_addr = cpu_data[str(cpu) + 'addr']
- stop_addr = ip + 4
-
- # Record for previous sample packet
- cpu_data[str(cpu) + 'addr'] = addr
-
- # Filter out zero start_address. Optionally identify CS_ETM_TRACE_ON packet
- if (start_addr == 0):
- if ((stop_addr == 4) and (options.verbose == True)):
- print("CPU%d: CS_ETM_TRACE_ON packet is inserted" % cpu)
- return
-
- if (start_addr < int(dso_start) or start_addr > int(dso_end)):
- print("Start address 0x%x is out of range [ 0x%x .. 0x%x ] for dso %s" % (start_addr, int(dso_start), int(dso_end), dso))
- return
-
- if (stop_addr < int(dso_start) or stop_addr > int(dso_end)):
- print("Stop address 0x%x is out of range [ 0x%x .. 0x%x ] for dso %s" % (stop_addr, int(dso_start), int(dso_end), dso))
- return
-
- if (options.objdump != None):
- # It doesn't need to decrease virtual memory offset for disassembly
- # for kernel dso and executable file dso, so in this case we set
- # vm_start to zero.
- if (dso == "[kernel.kallsyms]" or dso_start == 0x400000):
- dso_vm_start = 0
- map_pgoff = 0
- else:
- dso_vm_start = int(dso_start)
-
- dso_fname = get_dso_file_path(dso, dso_bid)
- if path.exists(dso_fname):
- print_disam(dso_fname, dso_vm_start, start_addr + map_pgoff, stop_addr + map_pgoff)
- else:
- print("Failed to find dso %s for address range [ 0x%x .. 0x%x ]" % (dso, start_addr + map_pgoff, stop_addr + map_pgoff))
-
- print_srccode(comm, param_dict, sample, symbol, dso)
diff --git a/tools/perf/scripts/python/bin/compaction-times-record b/tools/perf/scripts/python/bin/compaction-times-record
deleted file mode 100644
index 6edcd40e14e8..000000000000
--- a/tools/perf/scripts/python/bin/compaction-times-record
+++ /dev/null
@@ -1,2 +0,0 @@
-#!/bin/bash
-perf record -e compaction:mm_compaction_begin -e compaction:mm_compaction_end -e compaction:mm_compaction_migratepages -e compaction:mm_compaction_isolate_migratepages -e compaction:mm_compaction_isolate_freepages $@
diff --git a/tools/perf/scripts/python/bin/compaction-times-report b/tools/perf/scripts/python/bin/compaction-times-report
deleted file mode 100644
index 3dc13897cfde..000000000000
--- a/tools/perf/scripts/python/bin/compaction-times-report
+++ /dev/null
@@ -1,4 +0,0 @@
-#!/bin/bash
-#description: display time taken by mm compaction
-#args: [-h] [-u] [-p|-pv] [-t | [-m] [-fs] [-ms]] [pid|pid-range|comm-regex]
-perf script -s "$PERF_EXEC_PATH"/scripts/python/compaction-times.py $@
diff --git a/tools/perf/scripts/python/bin/event_analyzing_sample-record b/tools/perf/scripts/python/bin/event_analyzing_sample-record
deleted file mode 100644
index 5ce652dabd02..000000000000
--- a/tools/perf/scripts/python/bin/event_analyzing_sample-record
+++ /dev/null
@@ -1,8 +0,0 @@
-#!/bin/bash
-
-#
-# event_analyzing_sample.py can cover all type of perf samples including
-# the tracepoints, so no special record requirements, just record what
-# you want to analyze.
-#
-perf record $@
diff --git a/tools/perf/scripts/python/bin/event_analyzing_sample-report b/tools/perf/scripts/python/bin/event_analyzing_sample-report
deleted file mode 100644
index 0941fc94e158..000000000000
--- a/tools/perf/scripts/python/bin/event_analyzing_sample-report
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-# description: analyze all perf samples
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/event_analyzing_sample.py
diff --git a/tools/perf/scripts/python/bin/export-to-postgresql-record b/tools/perf/scripts/python/bin/export-to-postgresql-record
deleted file mode 100644
index 221d66e05713..000000000000
--- a/tools/perf/scripts/python/bin/export-to-postgresql-record
+++ /dev/null
@@ -1,8 +0,0 @@
-#!/bin/bash
-
-#
-# export perf data to a postgresql database. Can cover
-# perf ip samples (excluding the tracepoints). No special
-# record requirements, just record what you want to export.
-#
-perf record $@
diff --git a/tools/perf/scripts/python/bin/export-to-postgresql-report b/tools/perf/scripts/python/bin/export-to-postgresql-report
deleted file mode 100644
index cd335b6e2a01..000000000000
--- a/tools/perf/scripts/python/bin/export-to-postgresql-report
+++ /dev/null
@@ -1,29 +0,0 @@
-#!/bin/bash
-# description: export perf data to a postgresql database
-# args: [database name] [columns] [calls]
-n_args=0
-for i in "$@"
-do
- if expr match "$i" "-" > /dev/null ; then
- break
- fi
- n_args=$(( $n_args + 1 ))
-done
-if [ "$n_args" -gt 3 ] ; then
- echo "usage: export-to-postgresql-report [database name] [columns] [calls]"
- exit
-fi
-if [ "$n_args" -gt 2 ] ; then
- dbname=$1
- columns=$2
- calls=$3
- shift 3
-elif [ "$n_args" -gt 1 ] ; then
- dbname=$1
- columns=$2
- shift 2
-elif [ "$n_args" -gt 0 ] ; then
- dbname=$1
- shift
-fi
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/export-to-postgresql.py $dbname $columns $calls
diff --git a/tools/perf/scripts/python/bin/export-to-sqlite-record b/tools/perf/scripts/python/bin/export-to-sqlite-record
deleted file mode 100644
index 070204fd6d00..000000000000
--- a/tools/perf/scripts/python/bin/export-to-sqlite-record
+++ /dev/null
@@ -1,8 +0,0 @@
-#!/bin/bash
-
-#
-# export perf data to a sqlite3 database. Can cover
-# perf ip samples (excluding the tracepoints). No special
-# record requirements, just record what you want to export.
-#
-perf record $@
diff --git a/tools/perf/scripts/python/bin/export-to-sqlite-report b/tools/perf/scripts/python/bin/export-to-sqlite-report
deleted file mode 100644
index 5ff6033e70ba..000000000000
--- a/tools/perf/scripts/python/bin/export-to-sqlite-report
+++ /dev/null
@@ -1,29 +0,0 @@
-#!/bin/bash
-# description: export perf data to a sqlite3 database
-# args: [database name] [columns] [calls]
-n_args=0
-for i in "$@"
-do
- if expr match "$i" "-" > /dev/null ; then
- break
- fi
- n_args=$(( $n_args + 1 ))
-done
-if [ "$n_args" -gt 3 ] ; then
- echo "usage: export-to-sqlite-report [database name] [columns] [calls]"
- exit
-fi
-if [ "$n_args" -gt 2 ] ; then
- dbname=$1
- columns=$2
- calls=$3
- shift 3
-elif [ "$n_args" -gt 1 ] ; then
- dbname=$1
- columns=$2
- shift 2
-elif [ "$n_args" -gt 0 ] ; then
- dbname=$1
- shift
-fi
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/export-to-sqlite.py $dbname $columns $calls
diff --git a/tools/perf/scripts/python/bin/failed-syscalls-by-pid-record b/tools/perf/scripts/python/bin/failed-syscalls-by-pid-record
deleted file mode 100644
index 74685f318379..000000000000
--- a/tools/perf/scripts/python/bin/failed-syscalls-by-pid-record
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-(perf record -e raw_syscalls:sys_exit $@ || \
- perf record -e syscalls:sys_exit $@) 2> /dev/null
diff --git a/tools/perf/scripts/python/bin/failed-syscalls-by-pid-report b/tools/perf/scripts/python/bin/failed-syscalls-by-pid-report
deleted file mode 100644
index fda5096d0cbf..000000000000
--- a/tools/perf/scripts/python/bin/failed-syscalls-by-pid-report
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/bash
-# description: system-wide failed syscalls, by pid
-# args: [comm]
-if [ $# -gt 0 ] ; then
- if ! expr match "$1" "-" > /dev/null ; then
- comm=$1
- shift
- fi
-fi
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/failed-syscalls-by-pid.py $comm
diff --git a/tools/perf/scripts/python/bin/flamegraph-record b/tools/perf/scripts/python/bin/flamegraph-record
deleted file mode 100755
index 7df5a19c0163..000000000000
--- a/tools/perf/scripts/python/bin/flamegraph-record
+++ /dev/null
@@ -1,2 +0,0 @@
-#!/bin/bash
-perf record -g "$@"
diff --git a/tools/perf/scripts/python/bin/flamegraph-report b/tools/perf/scripts/python/bin/flamegraph-report
deleted file mode 100755
index 453a6918afbe..000000000000
--- a/tools/perf/scripts/python/bin/flamegraph-report
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-# description: create flame graphs
-perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py "$@"
diff --git a/tools/perf/scripts/python/bin/futex-contention-record b/tools/perf/scripts/python/bin/futex-contention-record
deleted file mode 100644
index b1495c9a9b20..000000000000
--- a/tools/perf/scripts/python/bin/futex-contention-record
+++ /dev/null
@@ -1,2 +0,0 @@
-#!/bin/bash
-perf record -e syscalls:sys_enter_futex -e syscalls:sys_exit_futex $@
diff --git a/tools/perf/scripts/python/bin/futex-contention-report b/tools/perf/scripts/python/bin/futex-contention-report
deleted file mode 100644
index 6c44271091ab..000000000000
--- a/tools/perf/scripts/python/bin/futex-contention-report
+++ /dev/null
@@ -1,4 +0,0 @@
-#!/bin/bash
-# description: futext contention measurement
-
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/futex-contention.py
diff --git a/tools/perf/scripts/python/bin/gecko-record b/tools/perf/scripts/python/bin/gecko-record
deleted file mode 100644
index f0d1aa55f171..000000000000
--- a/tools/perf/scripts/python/bin/gecko-record
+++ /dev/null
@@ -1,2 +0,0 @@
-#!/bin/bash
-perf record -F 99 -g "$@"
diff --git a/tools/perf/scripts/python/bin/gecko-report b/tools/perf/scripts/python/bin/gecko-report
deleted file mode 100755
index 1867ec8d9757..000000000000
--- a/tools/perf/scripts/python/bin/gecko-report
+++ /dev/null
@@ -1,7 +0,0 @@
-#!/bin/bash
-# description: create firefox gecko profile json format from perf.data
-if [ "$*" = "-i -" ]; then
-perf script -s "$PERF_EXEC_PATH"/scripts/python/gecko.py
-else
-perf script -s "$PERF_EXEC_PATH"/scripts/python/gecko.py -- "$@"
-fi
diff --git a/tools/perf/scripts/python/bin/intel-pt-events-record b/tools/perf/scripts/python/bin/intel-pt-events-record
deleted file mode 100644
index 6b9877cfe23e..000000000000
--- a/tools/perf/scripts/python/bin/intel-pt-events-record
+++ /dev/null
@@ -1,13 +0,0 @@
-#!/bin/bash
-
-#
-# print Intel PT Events including Power Events and PTWRITE. The intel_pt PMU
-# event needs to be specified with appropriate config terms.
-#
-if ! echo "$@" | grep -q intel_pt ; then
- echo "Options must include the Intel PT event e.g. -e intel_pt/pwr_evt,ptw/"
- echo "and for power events it probably needs to be system wide i.e. -a option"
- echo "For example: -a -e intel_pt/pwr_evt,branch=0/ sleep 1"
- exit 1
-fi
-perf record $@
diff --git a/tools/perf/scripts/python/bin/intel-pt-events-report b/tools/perf/scripts/python/bin/intel-pt-events-report
deleted file mode 100644
index beeac3fde9db..000000000000
--- a/tools/perf/scripts/python/bin/intel-pt-events-report
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-# description: print Intel PT Events including Power Events and PTWRITE
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/intel-pt-events.py
diff --git a/tools/perf/scripts/python/bin/mem-phys-addr-record b/tools/perf/scripts/python/bin/mem-phys-addr-record
deleted file mode 100644
index 5a875122a904..000000000000
--- a/tools/perf/scripts/python/bin/mem-phys-addr-record
+++ /dev/null
@@ -1,19 +0,0 @@
-#!/bin/bash
-
-#
-# Profiling physical memory by all retired load instructions/uops event
-# MEM_INST_RETIRED.ALL_LOADS or MEM_UOPS_RETIRED.ALL_LOADS
-#
-
-load=`perf list | grep mem_inst_retired.all_loads`
-if [ -z "$load" ]; then
- load=`perf list | grep mem_uops_retired.all_loads`
-fi
-if [ -z "$load" ]; then
- echo "There is no event to count all retired load instructions/uops."
- exit 1
-fi
-
-arg=$(echo $load | tr -d ' ')
-arg="$arg:P"
-perf record --phys-data -e $arg $@
diff --git a/tools/perf/scripts/python/bin/mem-phys-addr-report b/tools/perf/scripts/python/bin/mem-phys-addr-report
deleted file mode 100644
index 3f2b847e2eab..000000000000
--- a/tools/perf/scripts/python/bin/mem-phys-addr-report
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-# description: resolve physical address samples
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/mem-phys-addr.py
diff --git a/tools/perf/scripts/python/bin/net_dropmonitor-record b/tools/perf/scripts/python/bin/net_dropmonitor-record
deleted file mode 100755
index 423fb81dadae..000000000000
--- a/tools/perf/scripts/python/bin/net_dropmonitor-record
+++ /dev/null
@@ -1,2 +0,0 @@
-#!/bin/bash
-perf record -e skb:kfree_skb $@
diff --git a/tools/perf/scripts/python/bin/net_dropmonitor-report b/tools/perf/scripts/python/bin/net_dropmonitor-report
deleted file mode 100755
index 8d698f5a06aa..000000000000
--- a/tools/perf/scripts/python/bin/net_dropmonitor-report
+++ /dev/null
@@ -1,4 +0,0 @@
-#!/bin/bash
-# description: display a table of dropped frames
-
-perf script -s "$PERF_EXEC_PATH"/scripts/python/net_dropmonitor.py $@
diff --git a/tools/perf/scripts/python/bin/netdev-times-record b/tools/perf/scripts/python/bin/netdev-times-record
deleted file mode 100644
index 558754b840a9..000000000000
--- a/tools/perf/scripts/python/bin/netdev-times-record
+++ /dev/null
@@ -1,8 +0,0 @@
-#!/bin/bash
-perf record -e net:net_dev_xmit -e net:net_dev_queue \
- -e net:netif_receive_skb -e net:netif_rx \
- -e skb:consume_skb -e skb:kfree_skb \
- -e skb:skb_copy_datagram_iovec -e napi:napi_poll \
- -e irq:irq_handler_entry -e irq:irq_handler_exit \
- -e irq:softirq_entry -e irq:softirq_exit \
- -e irq:softirq_raise $@
diff --git a/tools/perf/scripts/python/bin/netdev-times-report b/tools/perf/scripts/python/bin/netdev-times-report
deleted file mode 100644
index 8f759291da86..000000000000
--- a/tools/perf/scripts/python/bin/netdev-times-report
+++ /dev/null
@@ -1,5 +0,0 @@
-#!/bin/bash
-# description: display a process of packet and processing time
-# args: [tx] [rx] [dev=] [debug]
-
-perf script -s "$PERF_EXEC_PATH"/scripts/python/netdev-times.py $@
diff --git a/tools/perf/scripts/python/bin/powerpc-hcalls-record b/tools/perf/scripts/python/bin/powerpc-hcalls-record
deleted file mode 100644
index b7402aa9147d..000000000000
--- a/tools/perf/scripts/python/bin/powerpc-hcalls-record
+++ /dev/null
@@ -1,2 +0,0 @@
-#!/bin/bash
-perf record -e "{powerpc:hcall_entry,powerpc:hcall_exit}" $@
diff --git a/tools/perf/scripts/python/bin/powerpc-hcalls-report b/tools/perf/scripts/python/bin/powerpc-hcalls-report
deleted file mode 100644
index dd32ad7465f6..000000000000
--- a/tools/perf/scripts/python/bin/powerpc-hcalls-report
+++ /dev/null
@@ -1,2 +0,0 @@
-#!/bin/bash
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/powerpc-hcalls.py
diff --git a/tools/perf/scripts/python/bin/sched-migration-record b/tools/perf/scripts/python/bin/sched-migration-record
deleted file mode 100644
index 7493fddbe995..000000000000
--- a/tools/perf/scripts/python/bin/sched-migration-record
+++ /dev/null
@@ -1,2 +0,0 @@
-#!/bin/bash
-perf record -m 16384 -e sched:sched_wakeup -e sched:sched_wakeup_new -e sched:sched_switch -e sched:sched_migrate_task $@
diff --git a/tools/perf/scripts/python/bin/sched-migration-report b/tools/perf/scripts/python/bin/sched-migration-report
deleted file mode 100644
index 68b037a1849b..000000000000
--- a/tools/perf/scripts/python/bin/sched-migration-report
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-# description: sched migration overview
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/sched-migration.py
diff --git a/tools/perf/scripts/python/bin/sctop-record b/tools/perf/scripts/python/bin/sctop-record
deleted file mode 100644
index d6940841e54f..000000000000
--- a/tools/perf/scripts/python/bin/sctop-record
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-(perf record -e raw_syscalls:sys_enter $@ || \
- perf record -e syscalls:sys_enter $@) 2> /dev/null
diff --git a/tools/perf/scripts/python/bin/sctop-report b/tools/perf/scripts/python/bin/sctop-report
deleted file mode 100644
index c32db294124d..000000000000
--- a/tools/perf/scripts/python/bin/sctop-report
+++ /dev/null
@@ -1,24 +0,0 @@
-#!/bin/bash
-# description: syscall top
-# args: [comm] [interval]
-n_args=0
-for i in "$@"
-do
- if expr match "$i" "-" > /dev/null ; then
- break
- fi
- n_args=$(( $n_args + 1 ))
-done
-if [ "$n_args" -gt 2 ] ; then
- echo "usage: sctop-report [comm] [interval]"
- exit
-fi
-if [ "$n_args" -gt 1 ] ; then
- comm=$1
- interval=$2
- shift 2
-elif [ "$n_args" -gt 0 ] ; then
- interval=$1
- shift
-fi
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/sctop.py $comm $interval
diff --git a/tools/perf/scripts/python/bin/stackcollapse-record b/tools/perf/scripts/python/bin/stackcollapse-record
deleted file mode 100755
index 9d8f9f0f3a17..000000000000
--- a/tools/perf/scripts/python/bin/stackcollapse-record
+++ /dev/null
@@ -1,8 +0,0 @@
-#!/bin/sh
-
-#
-# stackcollapse.py can cover all type of perf samples including
-# the tracepoints, so no special record requirements, just record what
-# you want to analyze.
-#
-perf record "$@"
diff --git a/tools/perf/scripts/python/bin/stackcollapse-report b/tools/perf/scripts/python/bin/stackcollapse-report
deleted file mode 100755
index 21a356bd27f6..000000000000
--- a/tools/perf/scripts/python/bin/stackcollapse-report
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/sh
-# description: produce callgraphs in short form for scripting use
-perf script -s "$PERF_EXEC_PATH"/scripts/python/stackcollapse.py "$@"
diff --git a/tools/perf/scripts/python/bin/syscall-counts-by-pid-record b/tools/perf/scripts/python/bin/syscall-counts-by-pid-record
deleted file mode 100644
index d6940841e54f..000000000000
--- a/tools/perf/scripts/python/bin/syscall-counts-by-pid-record
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-(perf record -e raw_syscalls:sys_enter $@ || \
- perf record -e syscalls:sys_enter $@) 2> /dev/null
diff --git a/tools/perf/scripts/python/bin/syscall-counts-by-pid-report b/tools/perf/scripts/python/bin/syscall-counts-by-pid-report
deleted file mode 100644
index 16eb8d65c543..000000000000
--- a/tools/perf/scripts/python/bin/syscall-counts-by-pid-report
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/bash
-# description: system-wide syscall counts, by pid
-# args: [comm]
-if [ $# -gt 0 ] ; then
- if ! expr match "$1" "-" > /dev/null ; then
- comm=$1
- shift
- fi
-fi
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/syscall-counts-by-pid.py $comm
diff --git a/tools/perf/scripts/python/bin/syscall-counts-record b/tools/perf/scripts/python/bin/syscall-counts-record
deleted file mode 100644
index d6940841e54f..000000000000
--- a/tools/perf/scripts/python/bin/syscall-counts-record
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-(perf record -e raw_syscalls:sys_enter $@ || \
- perf record -e syscalls:sys_enter $@) 2> /dev/null
diff --git a/tools/perf/scripts/python/bin/syscall-counts-report b/tools/perf/scripts/python/bin/syscall-counts-report
deleted file mode 100644
index 0f0e9d453bb4..000000000000
--- a/tools/perf/scripts/python/bin/syscall-counts-report
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/bash
-# description: system-wide syscall counts
-# args: [comm]
-if [ $# -gt 0 ] ; then
- if ! expr match "$1" "-" > /dev/null ; then
- comm=$1
- shift
- fi
-fi
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/python/syscall-counts.py $comm
diff --git a/tools/perf/scripts/python/bin/task-analyzer-record b/tools/perf/scripts/python/bin/task-analyzer-record
deleted file mode 100755
index 0f6b51bb2767..000000000000
--- a/tools/perf/scripts/python/bin/task-analyzer-record
+++ /dev/null
@@ -1,2 +0,0 @@
-#!/bin/bash
-perf record -e sched:sched_switch -e sched:sched_migrate_task "$@"
diff --git a/tools/perf/scripts/python/bin/task-analyzer-report b/tools/perf/scripts/python/bin/task-analyzer-report
deleted file mode 100755
index 4b16a8cc40a0..000000000000
--- a/tools/perf/scripts/python/bin/task-analyzer-report
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-# description: analyze timings of tasks
-perf script -s "$PERF_EXEC_PATH"/scripts/python/task-analyzer.py -- "$@"
diff --git a/tools/perf/scripts/python/check-perf-trace.py b/tools/perf/scripts/python/check-perf-trace.py
deleted file mode 100644
index d2c22954800d..000000000000
--- a/tools/perf/scripts/python/check-perf-trace.py
+++ /dev/null
@@ -1,84 +0,0 @@
-# perf script event handlers, generated by perf script -g python
-# (c) 2010, Tom Zanussi <tzanussi@gmail.com>
-# Licensed under the terms of the GNU GPL License version 2
-#
-# This script tests basic functionality such as flag and symbol
-# strings, common_xxx() calls back into perf, begin, end, unhandled
-# events, etc. Basically, if this script runs successfully and
-# displays expected results, Python scripting support should be ok.
-
-from __future__ import print_function
-
-import os
-import sys
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from Core import *
-from perf_trace_context import *
-
-unhandled = autodict()
-
-def trace_begin():
- print("trace_begin")
- pass
-
-def trace_end():
- print_unhandled()
-
-def irq__softirq_entry(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, vec):
- print_header(event_name, common_cpu, common_secs, common_nsecs,
- common_pid, common_comm)
-
- print_uncommon(context)
-
- print("vec=%s" % (symbol_str("irq__softirq_entry", "vec", vec)))
-
-def kmem__kmalloc(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, call_site, ptr, bytes_req, bytes_alloc,
- gfp_flags):
- print_header(event_name, common_cpu, common_secs, common_nsecs,
- common_pid, common_comm)
-
- print_uncommon(context)
-
- print("call_site=%u, ptr=%u, bytes_req=%u, "
- "bytes_alloc=%u, gfp_flags=%s" %
- (call_site, ptr, bytes_req, bytes_alloc,
- flag_str("kmem__kmalloc", "gfp_flags", gfp_flags)))
-
-def trace_unhandled(event_name, context, event_fields_dict):
- try:
- unhandled[event_name] += 1
- except TypeError:
- unhandled[event_name] = 1
-
-def print_header(event_name, cpu, secs, nsecs, pid, comm):
- print("%-20s %5u %05u.%09u %8u %-20s " %
- (event_name, cpu, secs, nsecs, pid, comm),
- end=' ')
-
-# print trace fields not included in handler args
-def print_uncommon(context):
- print("common_preempt_count=%d, common_flags=%s, "
- "common_lock_depth=%d, " %
- (common_pc(context), trace_flag_str(common_flags(context)),
- common_lock_depth(context)))
-
-def print_unhandled():
- keys = unhandled.keys()
- if not keys:
- return
-
- print("\nunhandled events:\n")
-
- print("%-40s %10s" % ("event", "count"))
- print("%-40s %10s" % ("----------------------------------------",
- "-----------"))
-
- for event_name in keys:
- print("%-40s %10d\n" % (event_name, unhandled[event_name]))
diff --git a/tools/perf/scripts/python/compaction-times.py b/tools/perf/scripts/python/compaction-times.py
deleted file mode 100644
index 9401f7c14747..000000000000
--- a/tools/perf/scripts/python/compaction-times.py
+++ /dev/null
@@ -1,311 +0,0 @@
-# report time spent in compaction
-# Licensed under the terms of the GNU GPL License version 2
-
-# testing:
-# 'echo 1 > /proc/sys/vm/compact_memory' to force compaction of all zones
-
-import os
-import sys
-import re
-
-import signal
-signal.signal(signal.SIGPIPE, signal.SIG_DFL)
-
-usage = "usage: perf script report compaction-times.py -- [-h] [-u] [-p|-pv] [-t | [-m] [-fs] [-ms]] [pid|pid-range|comm-regex]\n"
-
-class popt:
- DISP_DFL = 0
- DISP_PROC = 1
- DISP_PROC_VERBOSE=2
-
-class topt:
- DISP_TIME = 0
- DISP_MIG = 1
- DISP_ISOLFREE = 2
- DISP_ISOLMIG = 4
- DISP_ALL = 7
-
-class comm_filter:
- def __init__(self, re):
- self.re = re
-
- def filter(self, pid, comm):
- m = self.re.search(comm)
- return m == None or m.group() == ""
-
-class pid_filter:
- def __init__(self, low, high):
- self.low = (0 if low == "" else int(low))
- self.high = (0 if high == "" else int(high))
-
- def filter(self, pid, comm):
- return not (pid >= self.low and (self.high == 0 or pid <= self.high))
-
-def set_type(t):
- global opt_disp
- opt_disp = (t if opt_disp == topt.DISP_ALL else opt_disp|t)
-
-def ns(sec, nsec):
- return (sec * 1000000000) + nsec
-
-def time(ns):
- return "%dns" % ns if opt_ns else "%dus" % (round(ns, -3) / 1000)
-
-class pair:
- def __init__(self, aval, bval, alabel = None, blabel = None):
- self.alabel = alabel
- self.blabel = blabel
- self.aval = aval
- self.bval = bval
-
- def __add__(self, rhs):
- self.aval += rhs.aval
- self.bval += rhs.bval
- return self
-
- def __str__(self):
- return "%s=%d %s=%d" % (self.alabel, self.aval, self.blabel, self.bval)
-
-class cnode:
- def __init__(self, ns):
- self.ns = ns
- self.migrated = pair(0, 0, "moved", "failed")
- self.fscan = pair(0,0, "scanned", "isolated")
- self.mscan = pair(0,0, "scanned", "isolated")
-
- def __add__(self, rhs):
- self.ns += rhs.ns
- self.migrated += rhs.migrated
- self.fscan += rhs.fscan
- self.mscan += rhs.mscan
- return self
-
- def __str__(self):
- prev = 0
- s = "%s " % time(self.ns)
- if (opt_disp & topt.DISP_MIG):
- s += "migration: %s" % self.migrated
- prev = 1
- if (opt_disp & topt.DISP_ISOLFREE):
- s += "%sfree_scanner: %s" % (" " if prev else "", self.fscan)
- prev = 1
- if (opt_disp & topt.DISP_ISOLMIG):
- s += "%smigration_scanner: %s" % (" " if prev else "", self.mscan)
- return s
-
- def complete(self, secs, nsecs):
- self.ns = ns(secs, nsecs) - self.ns
-
- def increment(self, migrated, fscan, mscan):
- if (migrated != None):
- self.migrated += migrated
- if (fscan != None):
- self.fscan += fscan
- if (mscan != None):
- self.mscan += mscan
-
-
-class chead:
- heads = {}
- val = cnode(0);
- fobj = None
-
- @classmethod
- def add_filter(cls, filter):
- cls.fobj = filter
-
- @classmethod
- def create_pending(cls, pid, comm, start_secs, start_nsecs):
- filtered = 0
- try:
- head = cls.heads[pid]
- filtered = head.is_filtered()
- except KeyError:
- if cls.fobj != None:
- filtered = cls.fobj.filter(pid, comm)
- head = cls.heads[pid] = chead(comm, pid, filtered)
-
- if not filtered:
- head.mark_pending(start_secs, start_nsecs)
-
- @classmethod
- def increment_pending(cls, pid, migrated, fscan, mscan):
- head = cls.heads[pid]
- if not head.is_filtered():
- if head.is_pending():
- head.do_increment(migrated, fscan, mscan)
- else:
- sys.stderr.write("missing start compaction event for pid %d\n" % pid)
-
- @classmethod
- def complete_pending(cls, pid, secs, nsecs):
- head = cls.heads[pid]
- if not head.is_filtered():
- if head.is_pending():
- head.make_complete(secs, nsecs)
- else:
- sys.stderr.write("missing start compaction event for pid %d\n" % pid)
-
- @classmethod
- def gen(cls):
- if opt_proc != popt.DISP_DFL:
- for i in cls.heads:
- yield cls.heads[i]
-
- @classmethod
- def str(cls):
- return cls.val
-
- def __init__(self, comm, pid, filtered):
- self.comm = comm
- self.pid = pid
- self.val = cnode(0)
- self.pending = None
- self.filtered = filtered
- self.list = []
-
- def __add__(self, rhs):
- self.ns += rhs.ns
- self.val += rhs.val
- return self
-
- def mark_pending(self, secs, nsecs):
- self.pending = cnode(ns(secs, nsecs))
-
- def do_increment(self, migrated, fscan, mscan):
- self.pending.increment(migrated, fscan, mscan)
-
- def make_complete(self, secs, nsecs):
- self.pending.complete(secs, nsecs)
- chead.val += self.pending
-
- if opt_proc != popt.DISP_DFL:
- self.val += self.pending
-
- if opt_proc == popt.DISP_PROC_VERBOSE:
- self.list.append(self.pending)
- self.pending = None
-
- def enumerate(self):
- if opt_proc == popt.DISP_PROC_VERBOSE and not self.is_filtered():
- for i, pelem in enumerate(self.list):
- sys.stdout.write("%d[%s].%d: %s\n" % (self.pid, self.comm, i+1, pelem))
-
- def is_pending(self):
- return self.pending != None
-
- def is_filtered(self):
- return self.filtered
-
- def display(self):
- if not self.is_filtered():
- sys.stdout.write("%d[%s]: %s\n" % (self.pid, self.comm, self.val))
-
-
-def trace_end():
- sys.stdout.write("total: %s\n" % chead.str())
- for i in chead.gen():
- i.display(),
- i.enumerate()
-
-def compaction__mm_compaction_migratepages(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, nr_migrated, nr_failed):
-
- chead.increment_pending(common_pid,
- pair(nr_migrated, nr_failed), None, None)
-
-def compaction__mm_compaction_isolate_freepages(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, start_pfn, end_pfn, nr_scanned, nr_taken):
-
- chead.increment_pending(common_pid,
- None, pair(nr_scanned, nr_taken), None)
-
-def compaction__mm_compaction_isolate_migratepages(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, start_pfn, end_pfn, nr_scanned, nr_taken):
-
- chead.increment_pending(common_pid,
- None, None, pair(nr_scanned, nr_taken))
-
-def compaction__mm_compaction_end(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, zone_start, migrate_start, free_start, zone_end,
- sync, status):
-
- chead.complete_pending(common_pid, common_secs, common_nsecs)
-
-def compaction__mm_compaction_begin(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, zone_start, migrate_start, free_start, zone_end,
- sync):
-
- chead.create_pending(common_pid, common_comm, common_secs, common_nsecs)
-
-def pr_help():
- global usage
-
- sys.stdout.write(usage)
- sys.stdout.write("\n")
- sys.stdout.write("-h display this help\n")
- sys.stdout.write("-p display by process\n")
- sys.stdout.write("-pv display by process (verbose)\n")
- sys.stdout.write("-t display stall times only\n")
- sys.stdout.write("-m display stats for migration\n")
- sys.stdout.write("-fs display stats for free scanner\n")
- sys.stdout.write("-ms display stats for migration scanner\n")
- sys.stdout.write("-u display results in microseconds (default nanoseconds)\n")
-
-
-comm_re = None
-pid_re = None
-pid_regex = r"^(\d*)-(\d*)$|^(\d*)$"
-
-opt_proc = popt.DISP_DFL
-opt_disp = topt.DISP_ALL
-
-opt_ns = True
-
-argc = len(sys.argv) - 1
-if argc >= 1:
- pid_re = re.compile(pid_regex)
-
- for i, opt in enumerate(sys.argv[1:]):
- if opt[0] == "-":
- if opt == "-h":
- pr_help()
- exit(0);
- elif opt == "-p":
- opt_proc = popt.DISP_PROC
- elif opt == "-pv":
- opt_proc = popt.DISP_PROC_VERBOSE
- elif opt == '-u':
- opt_ns = False
- elif opt == "-t":
- set_type(topt.DISP_TIME)
- elif opt == "-m":
- set_type(topt.DISP_MIG)
- elif opt == "-fs":
- set_type(topt.DISP_ISOLFREE)
- elif opt == "-ms":
- set_type(topt.DISP_ISOLMIG)
- else:
- sys.exit(usage)
-
- elif i == argc - 1:
- m = pid_re.search(opt)
- if m != None and m.group() != "":
- if m.group(3) != None:
- f = pid_filter(m.group(3), m.group(3))
- else:
- f = pid_filter(m.group(1), m.group(2))
- else:
- try:
- comm_re=re.compile(opt)
- except:
- sys.stderr.write("invalid regex '%s'" % opt)
- sys.exit(usage)
- f = comm_filter(comm_re)
-
- chead.add_filter(f)
diff --git a/tools/perf/scripts/python/event_analyzing_sample.py b/tools/perf/scripts/python/event_analyzing_sample.py
deleted file mode 100644
index aa1e2cfa26a6..000000000000
--- a/tools/perf/scripts/python/event_analyzing_sample.py
+++ /dev/null
@@ -1,192 +0,0 @@
-# event_analyzing_sample.py: general event handler in python
-# SPDX-License-Identifier: GPL-2.0
-#
-# Current perf report is already very powerful with the annotation integrated,
-# and this script is not trying to be as powerful as perf report, but
-# providing end user/developer a flexible way to analyze the events other
-# than trace points.
-#
-# The 2 database related functions in this script just show how to gather
-# the basic information, and users can modify and write their own functions
-# according to their specific requirement.
-#
-# The first function "show_general_events" just does a basic grouping for all
-# generic events with the help of sqlite, and the 2nd one "show_pebs_ll" is
-# for a x86 HW PMU event: PEBS with load latency data.
-#
-
-from __future__ import print_function
-
-import os
-import sys
-import math
-import struct
-import sqlite3
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import *
-from EventClass import *
-
-#
-# If the perf.data has a big number of samples, then the insert operation
-# will be very time consuming (about 10+ minutes for 10000 samples) if the
-# .db database is on disk. Move the .db file to RAM based FS to speedup
-# the handling, which will cut the time down to several seconds.
-#
-con = sqlite3.connect("/dev/shm/perf.db")
-con.isolation_level = None
-
-def trace_begin():
- print("In trace_begin:\n")
-
- #
- # Will create several tables at the start, pebs_ll is for PEBS data with
- # load latency info, while gen_events is for general event.
- #
- con.execute("""
- create table if not exists gen_events (
- name text,
- symbol text,
- comm text,
- dso text
- );""")
- con.execute("""
- create table if not exists pebs_ll (
- name text,
- symbol text,
- comm text,
- dso text,
- flags integer,
- ip integer,
- status integer,
- dse integer,
- dla integer,
- lat integer
- );""")
-
-#
-# Create and insert event object to a database so that user could
-# do more analysis with simple database commands.
-#
-def process_event(param_dict):
- event_attr = param_dict["attr"]
- sample = param_dict["sample"]
- raw_buf = param_dict["raw_buf"]
- comm = param_dict["comm"]
- name = param_dict["ev_name"]
-
- # Symbol and dso info are not always resolved
- if ("dso" in param_dict):
- dso = param_dict["dso"]
- else:
- dso = "Unknown_dso"
-
- if ("symbol" in param_dict):
- symbol = param_dict["symbol"]
- else:
- symbol = "Unknown_symbol"
-
- # Create the event object and insert it to the right table in database
- event = create_event(name, comm, dso, symbol, raw_buf)
- insert_db(event)
-
-def insert_db(event):
- if event.ev_type == EVTYPE_GENERIC:
- con.execute("insert into gen_events values(?, ?, ?, ?)",
- (event.name, event.symbol, event.comm, event.dso))
- elif event.ev_type == EVTYPE_PEBS_LL:
- event.ip &= 0x7fffffffffffffff
- event.dla &= 0x7fffffffffffffff
- con.execute("insert into pebs_ll values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
- (event.name, event.symbol, event.comm, event.dso, event.flags,
- event.ip, event.status, event.dse, event.dla, event.lat))
-
-def trace_end():
- print("In trace_end:\n")
- # We show the basic info for the 2 type of event classes
- show_general_events()
- show_pebs_ll()
- con.close()
-
-#
-# As the event number may be very big, so we can't use linear way
-# to show the histogram in real number, but use a log2 algorithm.
-#
-
-def num2sym(num):
- # Each number will have at least one '#'
- snum = '#' * (int)(math.log(num, 2) + 1)
- return snum
-
-def show_general_events():
-
- # Check the total record number in the table
- count = con.execute("select count(*) from gen_events")
- for t in count:
- print("There is %d records in gen_events table" % t[0])
- if t[0] == 0:
- return
-
- print("Statistics about the general events grouped by thread/symbol/dso: \n")
-
- # Group by thread
- commq = con.execute("select comm, count(comm) from gen_events group by comm order by -count(comm)")
- print("\n%16s %8s %16s\n%s" % ("comm", "number", "histogram", "="*42))
- for row in commq:
- print("%16s %8d %s" % (row[0], row[1], num2sym(row[1])))
-
- # Group by symbol
- print("\n%32s %8s %16s\n%s" % ("symbol", "number", "histogram", "="*58))
- symbolq = con.execute("select symbol, count(symbol) from gen_events group by symbol order by -count(symbol)")
- for row in symbolq:
- print("%32s %8d %s" % (row[0], row[1], num2sym(row[1])))
-
- # Group by dso
- print("\n%40s %8s %16s\n%s" % ("dso", "number", "histogram", "="*74))
- dsoq = con.execute("select dso, count(dso) from gen_events group by dso order by -count(dso)")
- for row in dsoq:
- print("%40s %8d %s" % (row[0], row[1], num2sym(row[1])))
-
-#
-# This function just shows the basic info, and we could do more with the
-# data in the tables, like checking the function parameters when some
-# big latency events happen.
-#
-def show_pebs_ll():
-
- count = con.execute("select count(*) from pebs_ll")
- for t in count:
- print("There is %d records in pebs_ll table" % t[0])
- if t[0] == 0:
- return
-
- print("Statistics about the PEBS Load Latency events grouped by thread/symbol/dse/latency: \n")
-
- # Group by thread
- commq = con.execute("select comm, count(comm) from pebs_ll group by comm order by -count(comm)")
- print("\n%16s %8s %16s\n%s" % ("comm", "number", "histogram", "="*42))
- for row in commq:
- print("%16s %8d %s" % (row[0], row[1], num2sym(row[1])))
-
- # Group by symbol
- print("\n%32s %8s %16s\n%s" % ("symbol", "number", "histogram", "="*58))
- symbolq = con.execute("select symbol, count(symbol) from pebs_ll group by symbol order by -count(symbol)")
- for row in symbolq:
- print("%32s %8d %s" % (row[0], row[1], num2sym(row[1])))
-
- # Group by dse
- dseq = con.execute("select dse, count(dse) from pebs_ll group by dse order by -count(dse)")
- print("\n%32s %8s %16s\n%s" % ("dse", "number", "histogram", "="*58))
- for row in dseq:
- print("%32s %8d %s" % (row[0], row[1], num2sym(row[1])))
-
- # Group by latency
- latq = con.execute("select lat, count(lat) from pebs_ll group by lat order by lat")
- print("\n%32s %8s %16s\n%s" % ("latency", "number", "histogram", "="*58))
- for row in latq:
- print("%32s %8d %s" % (row[0], row[1], num2sym(row[1])))
-
-def trace_unhandled(event_name, context, event_fields_dict):
- print (' '.join(['%s=%s'%(k,str(v))for k,v in sorted(event_fields_dict.items())]))
diff --git a/tools/perf/scripts/python/export-to-postgresql.py b/tools/perf/scripts/python/export-to-postgresql.py
deleted file mode 100644
index 3a6bdcd74e60..000000000000
--- a/tools/perf/scripts/python/export-to-postgresql.py
+++ /dev/null
@@ -1,1114 +0,0 @@
-# export-to-postgresql.py: export perf data to a postgresql database
-# Copyright (c) 2014, Intel Corporation.
-#
-# This program is free software; you can redistribute it and/or modify it
-# under the terms and conditions of the GNU General Public License,
-# version 2, as published by the Free Software Foundation.
-#
-# This program is distributed in the hope it will be useful, but WITHOUT
-# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
-# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
-# more details.
-
-from __future__ import print_function
-
-import os
-import sys
-import struct
-import datetime
-
-# To use this script you will need to have installed package python-pyside which
-# provides LGPL-licensed Python bindings for Qt. You will also need the package
-# libqt4-sql-psql for Qt postgresql support.
-#
-# The script assumes postgresql is running on the local machine and that the
-# user has postgresql permissions to create databases. Examples of installing
-# postgresql and adding such a user are:
-#
-# fedora:
-#
-# $ sudo yum install postgresql postgresql-server qt-postgresql
-# $ sudo su - postgres -c initdb
-# $ sudo service postgresql start
-# $ sudo su - postgres
-# $ createuser -s <your user id here> # Older versions may not support -s, in which case answer the prompt below:
-# Shall the new role be a superuser? (y/n) y
-# $ sudo yum install python-pyside
-#
-# Alternately, to use Python3 and/or pyside 2, one of the following:
-# $ sudo yum install python3-pyside
-# $ pip install --user PySide2
-# $ pip3 install --user PySide2
-#
-# ubuntu:
-#
-# $ sudo apt-get install postgresql
-# $ sudo su - postgres
-# $ createuser -s <your user id here>
-# $ sudo apt-get install python-pyside.qtsql libqt4-sql-psql
-#
-# Alternately, to use Python3 and/or pyside 2, one of the following:
-#
-# $ sudo apt-get install python3-pyside.qtsql libqt4-sql-psql
-# $ sudo apt-get install python-pyside2.qtsql libqt5sql5-psql
-# $ sudo apt-get install python3-pyside2.qtsql libqt5sql5-psql
-#
-# An example of using this script with Intel PT:
-#
-# $ perf record -e intel_pt//u ls
-# $ perf script -s ~/libexec/perf-core/scripts/python/export-to-postgresql.py pt_example branches calls
-# 2015-05-29 12:49:23.464364 Creating database...
-# 2015-05-29 12:49:26.281717 Writing to intermediate files...
-# 2015-05-29 12:49:27.190383 Copying to database...
-# 2015-05-29 12:49:28.140451 Removing intermediate files...
-# 2015-05-29 12:49:28.147451 Adding primary keys
-# 2015-05-29 12:49:28.655683 Adding foreign keys
-# 2015-05-29 12:49:29.365350 Done
-#
-# To browse the database, psql can be used e.g.
-#
-# $ psql pt_example
-# pt_example=# select * from samples_view where id < 100;
-# pt_example=# \d+
-# pt_example=# \d+ samples_view
-# pt_example=# \q
-#
-# An example of using the database is provided by the script
-# exported-sql-viewer.py. Refer to that script for details.
-#
-# Tables:
-#
-# The tables largely correspond to perf tools' data structures. They are largely self-explanatory.
-#
-# samples
-#
-# 'samples' is the main table. It represents what instruction was executing at a point in time
-# when something (a selected event) happened. The memory address is the instruction pointer or 'ip'.
-#
-# calls
-#
-# 'calls' represents function calls and is related to 'samples' by 'call_id' and 'return_id'.
-# 'calls' is only created when the 'calls' option to this script is specified.
-#
-# call_paths
-#
-# 'call_paths' represents all the call stacks. Each 'call' has an associated record in 'call_paths'.
-# 'calls_paths' is only created when the 'calls' option to this script is specified.
-#
-# branch_types
-#
-# 'branch_types' provides descriptions for each type of branch.
-#
-# comm_threads
-#
-# 'comm_threads' shows how 'comms' relates to 'threads'.
-#
-# comms
-#
-# 'comms' contains a record for each 'comm' - the name given to the executable that is running.
-#
-# dsos
-#
-# 'dsos' contains a record for each executable file or library.
-#
-# machines
-#
-# 'machines' can be used to distinguish virtual machines if virtualization is supported.
-#
-# selected_events
-#
-# 'selected_events' contains a record for each kind of event that has been sampled.
-#
-# symbols
-#
-# 'symbols' contains a record for each symbol. Only symbols that have samples are present.
-#
-# threads
-#
-# 'threads' contains a record for each thread.
-#
-# Views:
-#
-# Most of the tables have views for more friendly display. The views are:
-#
-# calls_view
-# call_paths_view
-# comm_threads_view
-# dsos_view
-# machines_view
-# samples_view
-# symbols_view
-# threads_view
-#
-# More examples of browsing the database with psql:
-# Note that some of the examples are not the most optimal SQL query.
-# Note that call information is only available if the script's 'calls' option has been used.
-#
-# Top 10 function calls (not aggregated by symbol):
-#
-# SELECT * FROM calls_view ORDER BY elapsed_time DESC LIMIT 10;
-#
-# Top 10 function calls (aggregated by symbol):
-#
-# SELECT symbol_id,(SELECT name FROM symbols WHERE id = symbol_id) AS symbol,
-# SUM(elapsed_time) AS tot_elapsed_time,SUM(branch_count) AS tot_branch_count
-# FROM calls_view GROUP BY symbol_id ORDER BY tot_elapsed_time DESC LIMIT 10;
-#
-# Note that the branch count gives a rough estimation of cpu usage, so functions
-# that took a long time but have a relatively low branch count must have spent time
-# waiting.
-#
-# Find symbols by pattern matching on part of the name (e.g. names containing 'alloc'):
-#
-# SELECT * FROM symbols_view WHERE name LIKE '%alloc%';
-#
-# Top 10 function calls for a specific symbol (e.g. whose symbol_id is 187):
-#
-# SELECT * FROM calls_view WHERE symbol_id = 187 ORDER BY elapsed_time DESC LIMIT 10;
-#
-# Show function calls made by function in the same context (i.e. same call path) (e.g. one with call_path_id 254):
-#
-# SELECT * FROM calls_view WHERE parent_call_path_id = 254;
-#
-# Show branches made during a function call (e.g. where call_id is 29357 and return_id is 29370 and tid is 29670)
-#
-# SELECT * FROM samples_view WHERE id >= 29357 AND id <= 29370 AND tid = 29670 AND event LIKE 'branches%';
-#
-# Show transactions:
-#
-# SELECT * FROM samples_view WHERE event = 'transactions';
-#
-# Note transaction start has 'in_tx' true whereas, transaction end has 'in_tx' false.
-# Transaction aborts have branch_type_name 'transaction abort'
-#
-# Show transaction aborts:
-#
-# SELECT * FROM samples_view WHERE event = 'transactions' AND branch_type_name = 'transaction abort';
-#
-# To print a call stack requires walking the call_paths table. For example this python script:
-# #!/usr/bin/python2
-#
-# import sys
-# from PySide.QtSql import *
-#
-# if __name__ == '__main__':
-# if (len(sys.argv) < 3):
-# print >> sys.stderr, "Usage is: printcallstack.py <database name> <call_path_id>"
-# raise Exception("Too few arguments")
-# dbname = sys.argv[1]
-# call_path_id = sys.argv[2]
-# db = QSqlDatabase.addDatabase('QPSQL')
-# db.setDatabaseName(dbname)
-# if not db.open():
-# raise Exception("Failed to open database " + dbname + " error: " + db.lastError().text())
-# query = QSqlQuery(db)
-# print " id ip symbol_id symbol dso_id dso_short_name"
-# while call_path_id != 0 and call_path_id != 1:
-# ret = query.exec_('SELECT * FROM call_paths_view WHERE id = ' + str(call_path_id))
-# if not ret:
-# raise Exception("Query failed: " + query.lastError().text())
-# if not query.next():
-# raise Exception("Query failed")
-# print "{0:>6} {1:>10} {2:>9} {3:<30} {4:>6} {5:<30}".format(query.value(0), query.value(1), query.value(2), query.value(3), query.value(4), query.value(5))
-# call_path_id = query.value(6)
-
-pyside_version_1 = True
-if not "pyside-version-1" in sys.argv:
- try:
- from PySide2.QtSql import *
- pyside_version_1 = False
- except:
- pass
-
-if pyside_version_1:
- from PySide.QtSql import *
-
-if sys.version_info < (3, 0):
- def toserverstr(str):
- return str
- def toclientstr(str):
- return str
-else:
- # Assume UTF-8 server_encoding and client_encoding
- def toserverstr(str):
- return bytes(str, "UTF_8")
- def toclientstr(str):
- return bytes(str, "UTF_8")
-
-# Need to access PostgreSQL C library directly to use COPY FROM STDIN
-from ctypes import *
-libpq = CDLL("libpq.so.5")
-PQconnectdb = libpq.PQconnectdb
-PQconnectdb.restype = c_void_p
-PQconnectdb.argtypes = [ c_char_p ]
-PQfinish = libpq.PQfinish
-PQfinish.argtypes = [ c_void_p ]
-PQstatus = libpq.PQstatus
-PQstatus.restype = c_int
-PQstatus.argtypes = [ c_void_p ]
-PQexec = libpq.PQexec
-PQexec.restype = c_void_p
-PQexec.argtypes = [ c_void_p, c_char_p ]
-PQresultStatus = libpq.PQresultStatus
-PQresultStatus.restype = c_int
-PQresultStatus.argtypes = [ c_void_p ]
-PQputCopyData = libpq.PQputCopyData
-PQputCopyData.restype = c_int
-PQputCopyData.argtypes = [ c_void_p, c_void_p, c_int ]
-PQputCopyEnd = libpq.PQputCopyEnd
-PQputCopyEnd.restype = c_int
-PQputCopyEnd.argtypes = [ c_void_p, c_void_p ]
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-# These perf imports are not used at present
-#from perf_trace_context import *
-#from Core import *
-
-perf_db_export_mode = True
-perf_db_export_calls = False
-perf_db_export_callchains = False
-
-def printerr(*args, **kw_args):
- print(*args, file=sys.stderr, **kw_args)
-
-def printdate(*args, **kw_args):
- print(datetime.datetime.today(), *args, sep=' ', **kw_args)
-
-def usage():
- printerr("Usage is: export-to-postgresql.py <database name> [<columns>] [<calls>] [<callchains>] [<pyside-version-1>]");
- printerr("where: columns 'all' or 'branches'");
- printerr(" calls 'calls' => create calls and call_paths table");
- printerr(" callchains 'callchains' => create call_paths table");
- printerr(" pyside-version-1 'pyside-version-1' => use pyside version 1");
- raise Exception("Too few or bad arguments")
-
-if (len(sys.argv) < 2):
- usage()
-
-dbname = sys.argv[1]
-
-if (len(sys.argv) >= 3):
- columns = sys.argv[2]
-else:
- columns = "all"
-
-if columns not in ("all", "branches"):
- usage()
-
-branches = (columns == "branches")
-
-for i in range(3,len(sys.argv)):
- if (sys.argv[i] == "calls"):
- perf_db_export_calls = True
- elif (sys.argv[i] == "callchains"):
- perf_db_export_callchains = True
- elif (sys.argv[i] == "pyside-version-1"):
- pass
- else:
- usage()
-
-output_dir_name = os.getcwd() + "/" + dbname + "-perf-data"
-os.mkdir(output_dir_name)
-
-def do_query(q, s):
- if (q.exec_(s)):
- return
- raise Exception("Query failed: " + q.lastError().text())
-
-printdate("Creating database...")
-
-db = QSqlDatabase.addDatabase('QPSQL')
-query = QSqlQuery(db)
-db.setDatabaseName('postgres')
-db.open()
-try:
- do_query(query, 'CREATE DATABASE ' + dbname)
-except:
- os.rmdir(output_dir_name)
- raise
-query.finish()
-query.clear()
-db.close()
-
-db.setDatabaseName(dbname)
-db.open()
-
-query = QSqlQuery(db)
-do_query(query, 'SET client_min_messages TO WARNING')
-
-do_query(query, 'CREATE TABLE selected_events ('
- 'id bigint NOT NULL,'
- 'name varchar(80))')
-do_query(query, 'CREATE TABLE machines ('
- 'id bigint NOT NULL,'
- 'pid integer,'
- 'root_dir varchar(4096))')
-do_query(query, 'CREATE TABLE threads ('
- 'id bigint NOT NULL,'
- 'machine_id bigint,'
- 'process_id bigint,'
- 'pid integer,'
- 'tid integer)')
-do_query(query, 'CREATE TABLE comms ('
- 'id bigint NOT NULL,'
- 'comm varchar(16),'
- 'c_thread_id bigint,'
- 'c_time bigint,'
- 'exec_flag boolean)')
-do_query(query, 'CREATE TABLE comm_threads ('
- 'id bigint NOT NULL,'
- 'comm_id bigint,'
- 'thread_id bigint)')
-do_query(query, 'CREATE TABLE dsos ('
- 'id bigint NOT NULL,'
- 'machine_id bigint,'
- 'short_name varchar(256),'
- 'long_name varchar(4096),'
- 'build_id varchar(64))')
-do_query(query, 'CREATE TABLE symbols ('
- 'id bigint NOT NULL,'
- 'dso_id bigint,'
- 'sym_start bigint,'
- 'sym_end bigint,'
- 'binding integer,'
- 'name varchar(2048))')
-do_query(query, 'CREATE TABLE branch_types ('
- 'id integer NOT NULL,'
- 'name varchar(80))')
-
-if branches:
- do_query(query, 'CREATE TABLE samples ('
- 'id bigint NOT NULL,'
- 'evsel_id bigint,'
- 'machine_id bigint,'
- 'thread_id bigint,'
- 'comm_id bigint,'
- 'dso_id bigint,'
- 'symbol_id bigint,'
- 'sym_offset bigint,'
- 'ip bigint,'
- 'time bigint,'
- 'cpu integer,'
- 'to_dso_id bigint,'
- 'to_symbol_id bigint,'
- 'to_sym_offset bigint,'
- 'to_ip bigint,'
- 'branch_type integer,'
- 'in_tx boolean,'
- 'call_path_id bigint,'
- 'insn_count bigint,'
- 'cyc_count bigint,'
- 'flags integer)')
-else:
- do_query(query, 'CREATE TABLE samples ('
- 'id bigint NOT NULL,'
- 'evsel_id bigint,'
- 'machine_id bigint,'
- 'thread_id bigint,'
- 'comm_id bigint,'
- 'dso_id bigint,'
- 'symbol_id bigint,'
- 'sym_offset bigint,'
- 'ip bigint,'
- 'time bigint,'
- 'cpu integer,'
- 'to_dso_id bigint,'
- 'to_symbol_id bigint,'
- 'to_sym_offset bigint,'
- 'to_ip bigint,'
- 'period bigint,'
- 'weight bigint,'
- 'transaction bigint,'
- 'data_src bigint,'
- 'branch_type integer,'
- 'in_tx boolean,'
- 'call_path_id bigint,'
- 'insn_count bigint,'
- 'cyc_count bigint,'
- 'flags integer)')
-
-if perf_db_export_calls or perf_db_export_callchains:
- do_query(query, 'CREATE TABLE call_paths ('
- 'id bigint NOT NULL,'
- 'parent_id bigint,'
- 'symbol_id bigint,'
- 'ip bigint)')
-if perf_db_export_calls:
- do_query(query, 'CREATE TABLE calls ('
- 'id bigint NOT NULL,'
- 'thread_id bigint,'
- 'comm_id bigint,'
- 'call_path_id bigint,'
- 'call_time bigint,'
- 'return_time bigint,'
- 'branch_count bigint,'
- 'call_id bigint,'
- 'return_id bigint,'
- 'parent_call_path_id bigint,'
- 'flags integer,'
- 'parent_id bigint,'
- 'insn_count bigint,'
- 'cyc_count bigint)')
-
-do_query(query, 'CREATE TABLE ptwrite ('
- 'id bigint NOT NULL,'
- 'payload bigint,'
- 'exact_ip boolean)')
-
-do_query(query, 'CREATE TABLE cbr ('
- 'id bigint NOT NULL,'
- 'cbr integer,'
- 'mhz integer,'
- 'percent integer)')
-
-do_query(query, 'CREATE TABLE mwait ('
- 'id bigint NOT NULL,'
- 'hints integer,'
- 'extensions integer)')
-
-do_query(query, 'CREATE TABLE pwre ('
- 'id bigint NOT NULL,'
- 'cstate integer,'
- 'subcstate integer,'
- 'hw boolean)')
-
-do_query(query, 'CREATE TABLE exstop ('
- 'id bigint NOT NULL,'
- 'exact_ip boolean)')
-
-do_query(query, 'CREATE TABLE pwrx ('
- 'id bigint NOT NULL,'
- 'deepest_cstate integer,'
- 'last_cstate integer,'
- 'wake_reason integer)')
-
-do_query(query, 'CREATE TABLE context_switches ('
- 'id bigint NOT NULL,'
- 'machine_id bigint,'
- 'time bigint,'
- 'cpu integer,'
- 'thread_out_id bigint,'
- 'comm_out_id bigint,'
- 'thread_in_id bigint,'
- 'comm_in_id bigint,'
- 'flags integer)')
-
-do_query(query, 'CREATE VIEW machines_view AS '
- 'SELECT '
- 'id,'
- 'pid,'
- 'root_dir,'
- 'CASE WHEN id=0 THEN \'unknown\' WHEN pid=-1 THEN \'host\' ELSE \'guest\' END AS host_or_guest'
- ' FROM machines')
-
-do_query(query, 'CREATE VIEW dsos_view AS '
- 'SELECT '
- 'id,'
- 'machine_id,'
- '(SELECT host_or_guest FROM machines_view WHERE id = machine_id) AS host_or_guest,'
- 'short_name,'
- 'long_name,'
- 'build_id'
- ' FROM dsos')
-
-do_query(query, 'CREATE VIEW symbols_view AS '
- 'SELECT '
- 'id,'
- 'name,'
- '(SELECT short_name FROM dsos WHERE id=dso_id) AS dso,'
- 'dso_id,'
- 'sym_start,'
- 'sym_end,'
- 'CASE WHEN binding=0 THEN \'local\' WHEN binding=1 THEN \'global\' ELSE \'weak\' END AS binding'
- ' FROM symbols')
-
-do_query(query, 'CREATE VIEW threads_view AS '
- 'SELECT '
- 'id,'
- 'machine_id,'
- '(SELECT host_or_guest FROM machines_view WHERE id = machine_id) AS host_or_guest,'
- 'process_id,'
- 'pid,'
- 'tid'
- ' FROM threads')
-
-do_query(query, 'CREATE VIEW comm_threads_view AS '
- 'SELECT '
- 'comm_id,'
- '(SELECT comm FROM comms WHERE id = comm_id) AS command,'
- 'thread_id,'
- '(SELECT pid FROM threads WHERE id = thread_id) AS pid,'
- '(SELECT tid FROM threads WHERE id = thread_id) AS tid'
- ' FROM comm_threads')
-
-if perf_db_export_calls or perf_db_export_callchains:
- do_query(query, 'CREATE VIEW call_paths_view AS '
- 'SELECT '
- 'c.id,'
- 'to_hex(c.ip) AS ip,'
- 'c.symbol_id,'
- '(SELECT name FROM symbols WHERE id = c.symbol_id) AS symbol,'
- '(SELECT dso_id FROM symbols WHERE id = c.symbol_id) AS dso_id,'
- '(SELECT dso FROM symbols_view WHERE id = c.symbol_id) AS dso_short_name,'
- 'c.parent_id,'
- 'to_hex(p.ip) AS parent_ip,'
- 'p.symbol_id AS parent_symbol_id,'
- '(SELECT name FROM symbols WHERE id = p.symbol_id) AS parent_symbol,'
- '(SELECT dso_id FROM symbols WHERE id = p.symbol_id) AS parent_dso_id,'
- '(SELECT dso FROM symbols_view WHERE id = p.symbol_id) AS parent_dso_short_name'
- ' FROM call_paths c INNER JOIN call_paths p ON p.id = c.parent_id')
-if perf_db_export_calls:
- do_query(query, 'CREATE VIEW calls_view AS '
- 'SELECT '
- 'calls.id,'
- 'thread_id,'
- '(SELECT pid FROM threads WHERE id = thread_id) AS pid,'
- '(SELECT tid FROM threads WHERE id = thread_id) AS tid,'
- '(SELECT comm FROM comms WHERE id = comm_id) AS command,'
- 'call_path_id,'
- 'to_hex(ip) AS ip,'
- 'symbol_id,'
- '(SELECT name FROM symbols WHERE id = symbol_id) AS symbol,'
- 'call_time,'
- 'return_time,'
- 'return_time - call_time AS elapsed_time,'
- 'branch_count,'
- 'insn_count,'
- 'cyc_count,'
- 'CASE WHEN cyc_count=0 THEN CAST(0 AS NUMERIC(20, 2)) ELSE CAST((CAST(insn_count AS FLOAT) / cyc_count) AS NUMERIC(20, 2)) END AS IPC,'
- 'call_id,'
- 'return_id,'
- 'CASE WHEN flags=0 THEN \'\' WHEN flags=1 THEN \'no call\' WHEN flags=2 THEN \'no return\' WHEN flags=3 THEN \'no call/return\' WHEN flags=6 THEN \'jump\' ELSE CAST ( flags AS VARCHAR(6) ) END AS flags,'
- 'parent_call_path_id,'
- 'calls.parent_id'
- ' FROM calls INNER JOIN call_paths ON call_paths.id = call_path_id')
-
-do_query(query, 'CREATE VIEW samples_view AS '
- 'SELECT '
- 'id,'
- 'time,'
- 'cpu,'
- '(SELECT pid FROM threads WHERE id = thread_id) AS pid,'
- '(SELECT tid FROM threads WHERE id = thread_id) AS tid,'
- '(SELECT comm FROM comms WHERE id = comm_id) AS command,'
- '(SELECT name FROM selected_events WHERE id = evsel_id) AS event,'
- 'to_hex(ip) AS ip_hex,'
- '(SELECT name FROM symbols WHERE id = symbol_id) AS symbol,'
- 'sym_offset,'
- '(SELECT short_name FROM dsos WHERE id = dso_id) AS dso_short_name,'
- 'to_hex(to_ip) AS to_ip_hex,'
- '(SELECT name FROM symbols WHERE id = to_symbol_id) AS to_symbol,'
- 'to_sym_offset,'
- '(SELECT short_name FROM dsos WHERE id = to_dso_id) AS to_dso_short_name,'
- '(SELECT name FROM branch_types WHERE id = branch_type) AS branch_type_name,'
- 'in_tx,'
- 'insn_count,'
- 'cyc_count,'
- 'CASE WHEN cyc_count=0 THEN CAST(0 AS NUMERIC(20, 2)) ELSE CAST((CAST(insn_count AS FLOAT) / cyc_count) AS NUMERIC(20, 2)) END AS IPC,'
- 'flags'
- ' FROM samples')
-
-do_query(query, 'CREATE VIEW ptwrite_view AS '
- 'SELECT '
- 'ptwrite.id,'
- 'time,'
- 'cpu,'
- 'to_hex(payload) AS payload_hex,'
- 'CASE WHEN exact_ip=FALSE THEN \'False\' ELSE \'True\' END AS exact_ip'
- ' FROM ptwrite'
- ' INNER JOIN samples ON samples.id = ptwrite.id')
-
-do_query(query, 'CREATE VIEW cbr_view AS '
- 'SELECT '
- 'cbr.id,'
- 'time,'
- 'cpu,'
- 'cbr,'
- 'mhz,'
- 'percent'
- ' FROM cbr'
- ' INNER JOIN samples ON samples.id = cbr.id')
-
-do_query(query, 'CREATE VIEW mwait_view AS '
- 'SELECT '
- 'mwait.id,'
- 'time,'
- 'cpu,'
- 'to_hex(hints) AS hints_hex,'
- 'to_hex(extensions) AS extensions_hex'
- ' FROM mwait'
- ' INNER JOIN samples ON samples.id = mwait.id')
-
-do_query(query, 'CREATE VIEW pwre_view AS '
- 'SELECT '
- 'pwre.id,'
- 'time,'
- 'cpu,'
- 'cstate,'
- 'subcstate,'
- 'CASE WHEN hw=FALSE THEN \'False\' ELSE \'True\' END AS hw'
- ' FROM pwre'
- ' INNER JOIN samples ON samples.id = pwre.id')
-
-do_query(query, 'CREATE VIEW exstop_view AS '
- 'SELECT '
- 'exstop.id,'
- 'time,'
- 'cpu,'
- 'CASE WHEN exact_ip=FALSE THEN \'False\' ELSE \'True\' END AS exact_ip'
- ' FROM exstop'
- ' INNER JOIN samples ON samples.id = exstop.id')
-
-do_query(query, 'CREATE VIEW pwrx_view AS '
- 'SELECT '
- 'pwrx.id,'
- 'time,'
- 'cpu,'
- 'deepest_cstate,'
- 'last_cstate,'
- 'CASE WHEN wake_reason=1 THEN \'Interrupt\''
- ' WHEN wake_reason=2 THEN \'Timer Deadline\''
- ' WHEN wake_reason=4 THEN \'Monitored Address\''
- ' WHEN wake_reason=8 THEN \'HW\''
- ' ELSE CAST ( wake_reason AS VARCHAR(2) )'
- 'END AS wake_reason'
- ' FROM pwrx'
- ' INNER JOIN samples ON samples.id = pwrx.id')
-
-do_query(query, 'CREATE VIEW power_events_view AS '
- 'SELECT '
- 'samples.id,'
- 'samples.time,'
- 'samples.cpu,'
- 'selected_events.name AS event,'
- 'FORMAT(\'%6s\', cbr.cbr) AS cbr,'
- 'FORMAT(\'%6s\', cbr.mhz) AS MHz,'
- 'FORMAT(\'%5s\', cbr.percent) AS percent,'
- 'to_hex(mwait.hints) AS hints_hex,'
- 'to_hex(mwait.extensions) AS extensions_hex,'
- 'FORMAT(\'%3s\', pwre.cstate) AS cstate,'
- 'FORMAT(\'%3s\', pwre.subcstate) AS subcstate,'
- 'CASE WHEN pwre.hw=FALSE THEN \'False\' WHEN pwre.hw=TRUE THEN \'True\' ELSE NULL END AS hw,'
- 'CASE WHEN exstop.exact_ip=FALSE THEN \'False\' WHEN exstop.exact_ip=TRUE THEN \'True\' ELSE NULL END AS exact_ip,'
- 'FORMAT(\'%3s\', pwrx.deepest_cstate) AS deepest_cstate,'
- 'FORMAT(\'%3s\', pwrx.last_cstate) AS last_cstate,'
- 'CASE WHEN pwrx.wake_reason=1 THEN \'Interrupt\''
- ' WHEN pwrx.wake_reason=2 THEN \'Timer Deadline\''
- ' WHEN pwrx.wake_reason=4 THEN \'Monitored Address\''
- ' WHEN pwrx.wake_reason=8 THEN \'HW\''
- ' ELSE FORMAT(\'%2s\', pwrx.wake_reason)'
- 'END AS wake_reason'
- ' FROM cbr'
- ' FULL JOIN mwait ON mwait.id = cbr.id'
- ' FULL JOIN pwre ON pwre.id = cbr.id'
- ' FULL JOIN exstop ON exstop.id = cbr.id'
- ' FULL JOIN pwrx ON pwrx.id = cbr.id'
- ' INNER JOIN samples ON samples.id = coalesce(cbr.id, mwait.id, pwre.id, exstop.id, pwrx.id)'
- ' INNER JOIN selected_events ON selected_events.id = samples.evsel_id'
- ' ORDER BY samples.id')
-
-do_query(query, 'CREATE VIEW context_switches_view AS '
- 'SELECT '
- 'context_switches.id,'
- 'context_switches.machine_id,'
- 'context_switches.time,'
- 'context_switches.cpu,'
- 'th_out.pid AS pid_out,'
- 'th_out.tid AS tid_out,'
- 'comm_out.comm AS comm_out,'
- 'th_in.pid AS pid_in,'
- 'th_in.tid AS tid_in,'
- 'comm_in.comm AS comm_in,'
- 'CASE WHEN context_switches.flags = 0 THEN \'in\''
- ' WHEN context_switches.flags = 1 THEN \'out\''
- ' WHEN context_switches.flags = 3 THEN \'out preempt\''
- ' ELSE CAST ( context_switches.flags AS VARCHAR(11) )'
- 'END AS flags'
- ' FROM context_switches'
- ' INNER JOIN threads AS th_out ON th_out.id = context_switches.thread_out_id'
- ' INNER JOIN threads AS th_in ON th_in.id = context_switches.thread_in_id'
- ' INNER JOIN comms AS comm_out ON comm_out.id = context_switches.comm_out_id'
- ' INNER JOIN comms AS comm_in ON comm_in.id = context_switches.comm_in_id')
-
-file_header = struct.pack("!11sii", b"PGCOPY\n\377\r\n\0", 0, 0)
-file_trailer = b"\377\377"
-
-def open_output_file(file_name):
- path_name = output_dir_name + "/" + file_name
- file = open(path_name, "wb+")
- file.write(file_header)
- return file
-
-def close_output_file(file):
- file.write(file_trailer)
- file.close()
-
-def copy_output_file_direct(file, table_name):
- close_output_file(file)
- sql = "COPY " + table_name + " FROM '" + file.name + "' (FORMAT 'binary')"
- do_query(query, sql)
-
-# Use COPY FROM STDIN because security may prevent postgres from accessing the files directly
-def copy_output_file(file, table_name):
- conn = PQconnectdb(toclientstr("dbname = " + dbname))
- if (PQstatus(conn)):
- raise Exception("COPY FROM STDIN PQconnectdb failed")
- file.write(file_trailer)
- file.seek(0)
- sql = "COPY " + table_name + " FROM STDIN (FORMAT 'binary')"
- res = PQexec(conn, toclientstr(sql))
- if (PQresultStatus(res) != 4):
- raise Exception("COPY FROM STDIN PQexec failed")
- data = file.read(65536)
- while (len(data)):
- ret = PQputCopyData(conn, data, len(data))
- if (ret != 1):
- raise Exception("COPY FROM STDIN PQputCopyData failed, error " + str(ret))
- data = file.read(65536)
- ret = PQputCopyEnd(conn, None)
- if (ret != 1):
- raise Exception("COPY FROM STDIN PQputCopyEnd failed, error " + str(ret))
- PQfinish(conn)
-
-def remove_output_file(file):
- name = file.name
- file.close()
- os.unlink(name)
-
-evsel_file = open_output_file("evsel_table.bin")
-machine_file = open_output_file("machine_table.bin")
-thread_file = open_output_file("thread_table.bin")
-comm_file = open_output_file("comm_table.bin")
-comm_thread_file = open_output_file("comm_thread_table.bin")
-dso_file = open_output_file("dso_table.bin")
-symbol_file = open_output_file("symbol_table.bin")
-branch_type_file = open_output_file("branch_type_table.bin")
-sample_file = open_output_file("sample_table.bin")
-if perf_db_export_calls or perf_db_export_callchains:
- call_path_file = open_output_file("call_path_table.bin")
-if perf_db_export_calls:
- call_file = open_output_file("call_table.bin")
-ptwrite_file = open_output_file("ptwrite_table.bin")
-cbr_file = open_output_file("cbr_table.bin")
-mwait_file = open_output_file("mwait_table.bin")
-pwre_file = open_output_file("pwre_table.bin")
-exstop_file = open_output_file("exstop_table.bin")
-pwrx_file = open_output_file("pwrx_table.bin")
-context_switches_file = open_output_file("context_switches_table.bin")
-
-def trace_begin():
- printdate("Writing to intermediate files...")
- # id == 0 means unknown. It is easier to create records for them than replace the zeroes with NULLs
- evsel_table(0, "unknown")
- machine_table(0, 0, "unknown")
- thread_table(0, 0, 0, -1, -1)
- comm_table(0, "unknown", 0, 0, 0)
- dso_table(0, 0, "unknown", "unknown", "")
- symbol_table(0, 0, 0, 0, 0, "unknown")
- sample_table(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
- if perf_db_export_calls or perf_db_export_callchains:
- call_path_table(0, 0, 0, 0)
- call_return_table(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
-
-unhandled_count = 0
-
-def is_table_empty(table_name):
- do_query(query, 'SELECT * FROM ' + table_name + ' LIMIT 1');
- if query.next():
- return False
- return True
-
-def drop(table_name):
- do_query(query, 'DROP VIEW ' + table_name + '_view');
- do_query(query, 'DROP TABLE ' + table_name);
-
-def trace_end():
- printdate("Copying to database...")
- copy_output_file(evsel_file, "selected_events")
- copy_output_file(machine_file, "machines")
- copy_output_file(thread_file, "threads")
- copy_output_file(comm_file, "comms")
- copy_output_file(comm_thread_file, "comm_threads")
- copy_output_file(dso_file, "dsos")
- copy_output_file(symbol_file, "symbols")
- copy_output_file(branch_type_file, "branch_types")
- copy_output_file(sample_file, "samples")
- if perf_db_export_calls or perf_db_export_callchains:
- copy_output_file(call_path_file, "call_paths")
- if perf_db_export_calls:
- copy_output_file(call_file, "calls")
- copy_output_file(ptwrite_file, "ptwrite")
- copy_output_file(cbr_file, "cbr")
- copy_output_file(mwait_file, "mwait")
- copy_output_file(pwre_file, "pwre")
- copy_output_file(exstop_file, "exstop")
- copy_output_file(pwrx_file, "pwrx")
- copy_output_file(context_switches_file, "context_switches")
-
- printdate("Removing intermediate files...")
- remove_output_file(evsel_file)
- remove_output_file(machine_file)
- remove_output_file(thread_file)
- remove_output_file(comm_file)
- remove_output_file(comm_thread_file)
- remove_output_file(dso_file)
- remove_output_file(symbol_file)
- remove_output_file(branch_type_file)
- remove_output_file(sample_file)
- if perf_db_export_calls or perf_db_export_callchains:
- remove_output_file(call_path_file)
- if perf_db_export_calls:
- remove_output_file(call_file)
- remove_output_file(ptwrite_file)
- remove_output_file(cbr_file)
- remove_output_file(mwait_file)
- remove_output_file(pwre_file)
- remove_output_file(exstop_file)
- remove_output_file(pwrx_file)
- remove_output_file(context_switches_file)
- os.rmdir(output_dir_name)
- printdate("Adding primary keys")
- do_query(query, 'ALTER TABLE selected_events ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE machines ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE threads ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE comms ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE comm_threads ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE dsos ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE symbols ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE branch_types ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE samples ADD PRIMARY KEY (id)')
- if perf_db_export_calls or perf_db_export_callchains:
- do_query(query, 'ALTER TABLE call_paths ADD PRIMARY KEY (id)')
- if perf_db_export_calls:
- do_query(query, 'ALTER TABLE calls ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE ptwrite ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE cbr ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE mwait ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE pwre ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE exstop ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE pwrx ADD PRIMARY KEY (id)')
- do_query(query, 'ALTER TABLE context_switches ADD PRIMARY KEY (id)')
-
- printdate("Adding foreign keys")
- do_query(query, 'ALTER TABLE threads '
- 'ADD CONSTRAINT machinefk FOREIGN KEY (machine_id) REFERENCES machines (id),'
- 'ADD CONSTRAINT processfk FOREIGN KEY (process_id) REFERENCES threads (id)')
- do_query(query, 'ALTER TABLE comms '
- 'ADD CONSTRAINT threadfk FOREIGN KEY (c_thread_id) REFERENCES threads (id)')
- do_query(query, 'ALTER TABLE comm_threads '
- 'ADD CONSTRAINT commfk FOREIGN KEY (comm_id) REFERENCES comms (id),'
- 'ADD CONSTRAINT threadfk FOREIGN KEY (thread_id) REFERENCES threads (id)')
- do_query(query, 'ALTER TABLE dsos '
- 'ADD CONSTRAINT machinefk FOREIGN KEY (machine_id) REFERENCES machines (id)')
- do_query(query, 'ALTER TABLE symbols '
- 'ADD CONSTRAINT dsofk FOREIGN KEY (dso_id) REFERENCES dsos (id)')
- do_query(query, 'ALTER TABLE samples '
- 'ADD CONSTRAINT evselfk FOREIGN KEY (evsel_id) REFERENCES selected_events (id),'
- 'ADD CONSTRAINT machinefk FOREIGN KEY (machine_id) REFERENCES machines (id),'
- 'ADD CONSTRAINT threadfk FOREIGN KEY (thread_id) REFERENCES threads (id),'
- 'ADD CONSTRAINT commfk FOREIGN KEY (comm_id) REFERENCES comms (id),'
- 'ADD CONSTRAINT dsofk FOREIGN KEY (dso_id) REFERENCES dsos (id),'
- 'ADD CONSTRAINT symbolfk FOREIGN KEY (symbol_id) REFERENCES symbols (id),'
- 'ADD CONSTRAINT todsofk FOREIGN KEY (to_dso_id) REFERENCES dsos (id),'
- 'ADD CONSTRAINT tosymbolfk FOREIGN KEY (to_symbol_id) REFERENCES symbols (id)')
- if perf_db_export_calls or perf_db_export_callchains:
- do_query(query, 'ALTER TABLE call_paths '
- 'ADD CONSTRAINT parentfk FOREIGN KEY (parent_id) REFERENCES call_paths (id),'
- 'ADD CONSTRAINT symbolfk FOREIGN KEY (symbol_id) REFERENCES symbols (id)')
- if perf_db_export_calls:
- do_query(query, 'ALTER TABLE calls '
- 'ADD CONSTRAINT threadfk FOREIGN KEY (thread_id) REFERENCES threads (id),'
- 'ADD CONSTRAINT commfk FOREIGN KEY (comm_id) REFERENCES comms (id),'
- 'ADD CONSTRAINT call_pathfk FOREIGN KEY (call_path_id) REFERENCES call_paths (id),'
- 'ADD CONSTRAINT callfk FOREIGN KEY (call_id) REFERENCES samples (id),'
- 'ADD CONSTRAINT returnfk FOREIGN KEY (return_id) REFERENCES samples (id),'
- 'ADD CONSTRAINT parent_call_pathfk FOREIGN KEY (parent_call_path_id) REFERENCES call_paths (id)')
- do_query(query, 'CREATE INDEX pcpid_idx ON calls (parent_call_path_id)')
- do_query(query, 'CREATE INDEX pid_idx ON calls (parent_id)')
- do_query(query, 'ALTER TABLE comms ADD has_calls boolean')
- do_query(query, 'UPDATE comms SET has_calls = TRUE WHERE comms.id IN (SELECT DISTINCT comm_id FROM calls)')
- do_query(query, 'ALTER TABLE ptwrite '
- 'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
- do_query(query, 'ALTER TABLE cbr '
- 'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
- do_query(query, 'ALTER TABLE mwait '
- 'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
- do_query(query, 'ALTER TABLE pwre '
- 'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
- do_query(query, 'ALTER TABLE exstop '
- 'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
- do_query(query, 'ALTER TABLE pwrx '
- 'ADD CONSTRAINT idfk FOREIGN KEY (id) REFERENCES samples (id)')
- do_query(query, 'ALTER TABLE context_switches '
- 'ADD CONSTRAINT machinefk FOREIGN KEY (machine_id) REFERENCES machines (id),'
- 'ADD CONSTRAINT toutfk FOREIGN KEY (thread_out_id) REFERENCES threads (id),'
- 'ADD CONSTRAINT tinfk FOREIGN KEY (thread_in_id) REFERENCES threads (id),'
- 'ADD CONSTRAINT coutfk FOREIGN KEY (comm_out_id) REFERENCES comms (id),'
- 'ADD CONSTRAINT cinfk FOREIGN KEY (comm_in_id) REFERENCES comms (id)')
-
- printdate("Dropping unused tables")
- if is_table_empty("ptwrite"):
- drop("ptwrite")
- if is_table_empty("mwait") and is_table_empty("pwre") and is_table_empty("exstop") and is_table_empty("pwrx"):
- do_query(query, 'DROP VIEW power_events_view');
- drop("mwait")
- drop("pwre")
- drop("exstop")
- drop("pwrx")
- if is_table_empty("cbr"):
- drop("cbr")
- if is_table_empty("context_switches"):
- drop("context_switches")
-
- if (unhandled_count):
- printdate("Warning: ", unhandled_count, " unhandled events")
- printdate("Done")
-
-def trace_unhandled(event_name, context, event_fields_dict):
- global unhandled_count
- unhandled_count += 1
-
-def sched__sched_switch(*x):
- pass
-
-def evsel_table(evsel_id, evsel_name, *x):
- evsel_name = toserverstr(evsel_name)
- n = len(evsel_name)
- fmt = "!hiqi" + str(n) + "s"
- value = struct.pack(fmt, 2, 8, evsel_id, n, evsel_name)
- evsel_file.write(value)
-
-def machine_table(machine_id, pid, root_dir, *x):
- root_dir = toserverstr(root_dir)
- n = len(root_dir)
- fmt = "!hiqiii" + str(n) + "s"
- value = struct.pack(fmt, 3, 8, machine_id, 4, pid, n, root_dir)
- machine_file.write(value)
-
-def thread_table(thread_id, machine_id, process_id, pid, tid, *x):
- value = struct.pack("!hiqiqiqiiii", 5, 8, thread_id, 8, machine_id, 8, process_id, 4, pid, 4, tid)
- thread_file.write(value)
-
-def comm_table(comm_id, comm_str, thread_id, time, exec_flag, *x):
- comm_str = toserverstr(comm_str)
- n = len(comm_str)
- fmt = "!hiqi" + str(n) + "s" + "iqiqiB"
- value = struct.pack(fmt, 5, 8, comm_id, n, comm_str, 8, thread_id, 8, time, 1, exec_flag)
- comm_file.write(value)
-
-def comm_thread_table(comm_thread_id, comm_id, thread_id, *x):
- fmt = "!hiqiqiq"
- value = struct.pack(fmt, 3, 8, comm_thread_id, 8, comm_id, 8, thread_id)
- comm_thread_file.write(value)
-
-def dso_table(dso_id, machine_id, short_name, long_name, build_id, *x):
- short_name = toserverstr(short_name)
- long_name = toserverstr(long_name)
- build_id = toserverstr(build_id)
- n1 = len(short_name)
- n2 = len(long_name)
- n3 = len(build_id)
- fmt = "!hiqiqi" + str(n1) + "si" + str(n2) + "si" + str(n3) + "s"
- value = struct.pack(fmt, 5, 8, dso_id, 8, machine_id, n1, short_name, n2, long_name, n3, build_id)
- dso_file.write(value)
-
-def symbol_table(symbol_id, dso_id, sym_start, sym_end, binding, symbol_name, *x):
- symbol_name = toserverstr(symbol_name)
- n = len(symbol_name)
- fmt = "!hiqiqiqiqiii" + str(n) + "s"
- value = struct.pack(fmt, 6, 8, symbol_id, 8, dso_id, 8, sym_start, 8, sym_end, 4, binding, n, symbol_name)
- symbol_file.write(value)
-
-def branch_type_table(branch_type, name, *x):
- name = toserverstr(name)
- n = len(name)
- fmt = "!hiii" + str(n) + "s"
- value = struct.pack(fmt, 2, 4, branch_type, n, name)
- branch_type_file.write(value)
-
-def sample_table(sample_id, evsel_id, machine_id, thread_id, comm_id, dso_id, symbol_id, sym_offset, ip, time, cpu, to_dso_id, to_symbol_id, to_sym_offset, to_ip, period, weight, transaction, data_src, branch_type, in_tx, call_path_id, insn_cnt, cyc_cnt, flags, *x):
- if branches:
- value = struct.pack("!hiqiqiqiqiqiqiqiqiqiqiiiqiqiqiqiiiBiqiqiqii", 21, 8, sample_id, 8, evsel_id, 8, machine_id, 8, thread_id, 8, comm_id, 8, dso_id, 8, symbol_id, 8, sym_offset, 8, ip, 8, time, 4, cpu, 8, to_dso_id, 8, to_symbol_id, 8, to_sym_offset, 8, to_ip, 4, branch_type, 1, in_tx, 8, call_path_id, 8, insn_cnt, 8, cyc_cnt, 4, flags)
- else:
- value = struct.pack("!hiqiqiqiqiqiqiqiqiqiqiiiqiqiqiqiqiqiqiqiiiBiqiqiqii", 25, 8, sample_id, 8, evsel_id, 8, machine_id, 8, thread_id, 8, comm_id, 8, dso_id, 8, symbol_id, 8, sym_offset, 8, ip, 8, time, 4, cpu, 8, to_dso_id, 8, to_symbol_id, 8, to_sym_offset, 8, to_ip, 8, period, 8, weight, 8, transaction, 8, data_src, 4, branch_type, 1, in_tx, 8, call_path_id, 8, insn_cnt, 8, cyc_cnt, 4, flags)
- sample_file.write(value)
-
-def call_path_table(cp_id, parent_id, symbol_id, ip, *x):
- fmt = "!hiqiqiqiq"
- value = struct.pack(fmt, 4, 8, cp_id, 8, parent_id, 8, symbol_id, 8, ip)
- call_path_file.write(value)
-
-def call_return_table(cr_id, thread_id, comm_id, call_path_id, call_time, return_time, branch_count, call_id, return_id, parent_call_path_id, flags, parent_id, insn_cnt, cyc_cnt, *x):
- fmt = "!hiqiqiqiqiqiqiqiqiqiqiiiqiqiq"
- value = struct.pack(fmt, 14, 8, cr_id, 8, thread_id, 8, comm_id, 8, call_path_id, 8, call_time, 8, return_time, 8, branch_count, 8, call_id, 8, return_id, 8, parent_call_path_id, 4, flags, 8, parent_id, 8, insn_cnt, 8, cyc_cnt)
- call_file.write(value)
-
-def ptwrite(id, raw_buf):
- data = struct.unpack_from("<IQ", raw_buf)
- flags = data[0]
- payload = data[1]
- exact_ip = flags & 1
- value = struct.pack("!hiqiqiB", 3, 8, id, 8, payload, 1, exact_ip)
- ptwrite_file.write(value)
-
-def cbr(id, raw_buf):
- data = struct.unpack_from("<BBBBII", raw_buf)
- cbr = data[0]
- MHz = (data[4] + 500) / 1000
- percent = ((cbr * 1000 / data[2]) + 5) / 10
- value = struct.pack("!hiqiiiiii", 4, 8, id, 4, cbr, 4, int(MHz), 4, int(percent))
- cbr_file.write(value)
-
-def mwait(id, raw_buf):
- data = struct.unpack_from("<IQ", raw_buf)
- payload = data[1]
- hints = payload & 0xff
- extensions = (payload >> 32) & 0x3
- value = struct.pack("!hiqiiii", 3, 8, id, 4, hints, 4, extensions)
- mwait_file.write(value)
-
-def pwre(id, raw_buf):
- data = struct.unpack_from("<IQ", raw_buf)
- payload = data[1]
- hw = (payload >> 7) & 1
- cstate = (payload >> 12) & 0xf
- subcstate = (payload >> 8) & 0xf
- value = struct.pack("!hiqiiiiiB", 4, 8, id, 4, cstate, 4, subcstate, 1, hw)
- pwre_file.write(value)
-
-def exstop(id, raw_buf):
- data = struct.unpack_from("<I", raw_buf)
- flags = data[0]
- exact_ip = flags & 1
- value = struct.pack("!hiqiB", 2, 8, id, 1, exact_ip)
- exstop_file.write(value)
-
-def pwrx(id, raw_buf):
- data = struct.unpack_from("<IQ", raw_buf)
- payload = data[1]
- deepest_cstate = payload & 0xf
- last_cstate = (payload >> 4) & 0xf
- wake_reason = (payload >> 8) & 0xf
- value = struct.pack("!hiqiiiiii", 4, 8, id, 4, deepest_cstate, 4, last_cstate, 4, wake_reason)
- pwrx_file.write(value)
-
-def synth_data(id, config, raw_buf, *x):
- if config == 0:
- ptwrite(id, raw_buf)
- elif config == 1:
- mwait(id, raw_buf)
- elif config == 2:
- pwre(id, raw_buf)
- elif config == 3:
- exstop(id, raw_buf)
- elif config == 4:
- pwrx(id, raw_buf)
- elif config == 5:
- cbr(id, raw_buf)
-
-def context_switch_table(id, machine_id, time, cpu, thread_out_id, comm_out_id, thread_in_id, comm_in_id, flags, *x):
- fmt = "!hiqiqiqiiiqiqiqiqii"
- value = struct.pack(fmt, 9, 8, id, 8, machine_id, 8, time, 4, cpu, 8, thread_out_id, 8, comm_out_id, 8, thread_in_id, 8, comm_in_id, 4, flags)
- context_switches_file.write(value)
diff --git a/tools/perf/scripts/python/export-to-sqlite.py b/tools/perf/scripts/python/export-to-sqlite.py
deleted file mode 100644
index 73c992feb1b9..000000000000
--- a/tools/perf/scripts/python/export-to-sqlite.py
+++ /dev/null
@@ -1,799 +0,0 @@
-# export-to-sqlite.py: export perf data to a sqlite3 database
-# Copyright (c) 2017, Intel Corporation.
-#
-# This program is free software; you can redistribute it and/or modify it
-# under the terms and conditions of the GNU General Public License,
-# version 2, as published by the Free Software Foundation.
-#
-# This program is distributed in the hope it will be useful, but WITHOUT
-# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
-# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
-# more details.
-
-from __future__ import print_function
-
-import os
-import sys
-import struct
-import datetime
-
-# To use this script you will need to have installed package python-pyside which
-# provides LGPL-licensed Python bindings for Qt. You will also need the package
-# libqt4-sql-sqlite for Qt sqlite3 support.
-#
-# Examples of installing pyside:
-#
-# ubuntu:
-#
-# $ sudo apt-get install python-pyside.qtsql libqt4-sql-psql
-#
-# Alternately, to use Python3 and/or pyside 2, one of the following:
-#
-# $ sudo apt-get install python3-pyside.qtsql libqt4-sql-psql
-# $ sudo apt-get install python-pyside2.qtsql libqt5sql5-psql
-# $ sudo apt-get install python3-pyside2.qtsql libqt5sql5-psql
-# fedora:
-#
-# $ sudo yum install python-pyside
-#
-# Alternately, to use Python3 and/or pyside 2, one of the following:
-# $ sudo yum install python3-pyside
-# $ pip install --user PySide2
-# $ pip3 install --user PySide2
-#
-# An example of using this script with Intel PT:
-#
-# $ perf record -e intel_pt//u ls
-# $ perf script -s ~/libexec/perf-core/scripts/python/export-to-sqlite.py pt_example branches calls
-# 2017-07-31 14:26:07.326913 Creating database...
-# 2017-07-31 14:26:07.538097 Writing records...
-# 2017-07-31 14:26:09.889292 Adding indexes
-# 2017-07-31 14:26:09.958746 Done
-#
-# To browse the database, sqlite3 can be used e.g.
-#
-# $ sqlite3 pt_example
-# sqlite> .header on
-# sqlite> select * from samples_view where id < 10;
-# sqlite> .mode column
-# sqlite> select * from samples_view where id < 10;
-# sqlite> .tables
-# sqlite> .schema samples_view
-# sqlite> .quit
-#
-# An example of using the database is provided by the script
-# exported-sql-viewer.py. Refer to that script for details.
-#
-# The database structure is practically the same as created by the script
-# export-to-postgresql.py. Refer to that script for details. A notable
-# difference is the 'transaction' column of the 'samples' table which is
-# renamed 'transaction_' in sqlite because 'transaction' is a reserved word.
-
-pyside_version_1 = True
-if not "pyside-version-1" in sys.argv:
- try:
- from PySide2.QtSql import *
- pyside_version_1 = False
- except:
- pass
-
-if pyside_version_1:
- from PySide.QtSql import *
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-# These perf imports are not used at present
-#from perf_trace_context import *
-#from Core import *
-
-perf_db_export_mode = True
-perf_db_export_calls = False
-perf_db_export_callchains = False
-
-def printerr(*args, **keyword_args):
- print(*args, file=sys.stderr, **keyword_args)
-
-def printdate(*args, **kw_args):
- print(datetime.datetime.today(), *args, sep=' ', **kw_args)
-
-def usage():
- printerr("Usage is: export-to-sqlite.py <database name> [<columns>] [<calls>] [<callchains>] [<pyside-version-1>]");
- printerr("where: columns 'all' or 'branches'");
- printerr(" calls 'calls' => create calls and call_paths table");
- printerr(" callchains 'callchains' => create call_paths table");
- printerr(" pyside-version-1 'pyside-version-1' => use pyside version 1");
- raise Exception("Too few or bad arguments")
-
-if (len(sys.argv) < 2):
- usage()
-
-dbname = sys.argv[1]
-
-if (len(sys.argv) >= 3):
- columns = sys.argv[2]
-else:
- columns = "all"
-
-if columns not in ("all", "branches"):
- usage()
-
-branches = (columns == "branches")
-
-for i in range(3,len(sys.argv)):
- if (sys.argv[i] == "calls"):
- perf_db_export_calls = True
- elif (sys.argv[i] == "callchains"):
- perf_db_export_callchains = True
- elif (sys.argv[i] == "pyside-version-1"):
- pass
- else:
- usage()
-
-def do_query(q, s):
- if (q.exec_(s)):
- return
- raise Exception("Query failed: " + q.lastError().text())
-
-def do_query_(q):
- if (q.exec_()):
- return
- raise Exception("Query failed: " + q.lastError().text())
-
-printdate("Creating database ...")
-
-db_exists = False
-try:
- f = open(dbname)
- f.close()
- db_exists = True
-except:
- pass
-
-if db_exists:
- raise Exception(dbname + " already exists")
-
-db = QSqlDatabase.addDatabase('QSQLITE')
-db.setDatabaseName(dbname)
-db.open()
-
-query = QSqlQuery(db)
-
-do_query(query, 'PRAGMA journal_mode = OFF')
-do_query(query, 'BEGIN TRANSACTION')
-
-do_query(query, 'CREATE TABLE selected_events ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'name varchar(80))')
-do_query(query, 'CREATE TABLE machines ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'pid integer,'
- 'root_dir varchar(4096))')
-do_query(query, 'CREATE TABLE threads ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'machine_id bigint,'
- 'process_id bigint,'
- 'pid integer,'
- 'tid integer)')
-do_query(query, 'CREATE TABLE comms ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'comm varchar(16),'
- 'c_thread_id bigint,'
- 'c_time bigint,'
- 'exec_flag boolean)')
-do_query(query, 'CREATE TABLE comm_threads ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'comm_id bigint,'
- 'thread_id bigint)')
-do_query(query, 'CREATE TABLE dsos ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'machine_id bigint,'
- 'short_name varchar(256),'
- 'long_name varchar(4096),'
- 'build_id varchar(64))')
-do_query(query, 'CREATE TABLE symbols ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'dso_id bigint,'
- 'sym_start bigint,'
- 'sym_end bigint,'
- 'binding integer,'
- 'name varchar(2048))')
-do_query(query, 'CREATE TABLE branch_types ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'name varchar(80))')
-
-if branches:
- do_query(query, 'CREATE TABLE samples ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'evsel_id bigint,'
- 'machine_id bigint,'
- 'thread_id bigint,'
- 'comm_id bigint,'
- 'dso_id bigint,'
- 'symbol_id bigint,'
- 'sym_offset bigint,'
- 'ip bigint,'
- 'time bigint,'
- 'cpu integer,'
- 'to_dso_id bigint,'
- 'to_symbol_id bigint,'
- 'to_sym_offset bigint,'
- 'to_ip bigint,'
- 'branch_type integer,'
- 'in_tx boolean,'
- 'call_path_id bigint,'
- 'insn_count bigint,'
- 'cyc_count bigint,'
- 'flags integer)')
-else:
- do_query(query, 'CREATE TABLE samples ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'evsel_id bigint,'
- 'machine_id bigint,'
- 'thread_id bigint,'
- 'comm_id bigint,'
- 'dso_id bigint,'
- 'symbol_id bigint,'
- 'sym_offset bigint,'
- 'ip bigint,'
- 'time bigint,'
- 'cpu integer,'
- 'to_dso_id bigint,'
- 'to_symbol_id bigint,'
- 'to_sym_offset bigint,'
- 'to_ip bigint,'
- 'period bigint,'
- 'weight bigint,'
- 'transaction_ bigint,'
- 'data_src bigint,'
- 'branch_type integer,'
- 'in_tx boolean,'
- 'call_path_id bigint,'
- 'insn_count bigint,'
- 'cyc_count bigint,'
- 'flags integer)')
-
-if perf_db_export_calls or perf_db_export_callchains:
- do_query(query, 'CREATE TABLE call_paths ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'parent_id bigint,'
- 'symbol_id bigint,'
- 'ip bigint)')
-if perf_db_export_calls:
- do_query(query, 'CREATE TABLE calls ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'thread_id bigint,'
- 'comm_id bigint,'
- 'call_path_id bigint,'
- 'call_time bigint,'
- 'return_time bigint,'
- 'branch_count bigint,'
- 'call_id bigint,'
- 'return_id bigint,'
- 'parent_call_path_id bigint,'
- 'flags integer,'
- 'parent_id bigint,'
- 'insn_count bigint,'
- 'cyc_count bigint)')
-
-do_query(query, 'CREATE TABLE ptwrite ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'payload bigint,'
- 'exact_ip integer)')
-
-do_query(query, 'CREATE TABLE cbr ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'cbr integer,'
- 'mhz integer,'
- 'percent integer)')
-
-do_query(query, 'CREATE TABLE mwait ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'hints integer,'
- 'extensions integer)')
-
-do_query(query, 'CREATE TABLE pwre ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'cstate integer,'
- 'subcstate integer,'
- 'hw integer)')
-
-do_query(query, 'CREATE TABLE exstop ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'exact_ip integer)')
-
-do_query(query, 'CREATE TABLE pwrx ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'deepest_cstate integer,'
- 'last_cstate integer,'
- 'wake_reason integer)')
-
-do_query(query, 'CREATE TABLE context_switches ('
- 'id integer NOT NULL PRIMARY KEY,'
- 'machine_id bigint,'
- 'time bigint,'
- 'cpu integer,'
- 'thread_out_id bigint,'
- 'comm_out_id bigint,'
- 'thread_in_id bigint,'
- 'comm_in_id bigint,'
- 'flags integer)')
-
-# printf was added to sqlite in version 3.8.3
-sqlite_has_printf = False
-try:
- do_query(query, 'SELECT printf("") FROM machines')
- sqlite_has_printf = True
-except:
- pass
-
-def emit_to_hex(x):
- if sqlite_has_printf:
- return 'printf("%x", ' + x + ')'
- else:
- return x
-
-do_query(query, 'CREATE VIEW machines_view AS '
- 'SELECT '
- 'id,'
- 'pid,'
- 'root_dir,'
- 'CASE WHEN id=0 THEN \'unknown\' WHEN pid=-1 THEN \'host\' ELSE \'guest\' END AS host_or_guest'
- ' FROM machines')
-
-do_query(query, 'CREATE VIEW dsos_view AS '
- 'SELECT '
- 'id,'
- 'machine_id,'
- '(SELECT host_or_guest FROM machines_view WHERE id = machine_id) AS host_or_guest,'
- 'short_name,'
- 'long_name,'
- 'build_id'
- ' FROM dsos')
-
-do_query(query, 'CREATE VIEW symbols_view AS '
- 'SELECT '
- 'id,'
- 'name,'
- '(SELECT short_name FROM dsos WHERE id=dso_id) AS dso,'
- 'dso_id,'
- 'sym_start,'
- 'sym_end,'
- 'CASE WHEN binding=0 THEN \'local\' WHEN binding=1 THEN \'global\' ELSE \'weak\' END AS binding'
- ' FROM symbols')
-
-do_query(query, 'CREATE VIEW threads_view AS '
- 'SELECT '
- 'id,'
- 'machine_id,'
- '(SELECT host_or_guest FROM machines_view WHERE id = machine_id) AS host_or_guest,'
- 'process_id,'
- 'pid,'
- 'tid'
- ' FROM threads')
-
-do_query(query, 'CREATE VIEW comm_threads_view AS '
- 'SELECT '
- 'comm_id,'
- '(SELECT comm FROM comms WHERE id = comm_id) AS command,'
- 'thread_id,'
- '(SELECT pid FROM threads WHERE id = thread_id) AS pid,'
- '(SELECT tid FROM threads WHERE id = thread_id) AS tid'
- ' FROM comm_threads')
-
-if perf_db_export_calls or perf_db_export_callchains:
- do_query(query, 'CREATE VIEW call_paths_view AS '
- 'SELECT '
- 'c.id,'
- + emit_to_hex('c.ip') + ' AS ip,'
- 'c.symbol_id,'
- '(SELECT name FROM symbols WHERE id = c.symbol_id) AS symbol,'
- '(SELECT dso_id FROM symbols WHERE id = c.symbol_id) AS dso_id,'
- '(SELECT dso FROM symbols_view WHERE id = c.symbol_id) AS dso_short_name,'
- 'c.parent_id,'
- + emit_to_hex('p.ip') + ' AS parent_ip,'
- 'p.symbol_id AS parent_symbol_id,'
- '(SELECT name FROM symbols WHERE id = p.symbol_id) AS parent_symbol,'
- '(SELECT dso_id FROM symbols WHERE id = p.symbol_id) AS parent_dso_id,'
- '(SELECT dso FROM symbols_view WHERE id = p.symbol_id) AS parent_dso_short_name'
- ' FROM call_paths c INNER JOIN call_paths p ON p.id = c.parent_id')
-if perf_db_export_calls:
- do_query(query, 'CREATE VIEW calls_view AS '
- 'SELECT '
- 'calls.id,'
- 'thread_id,'
- '(SELECT pid FROM threads WHERE id = thread_id) AS pid,'
- '(SELECT tid FROM threads WHERE id = thread_id) AS tid,'
- '(SELECT comm FROM comms WHERE id = comm_id) AS command,'
- 'call_path_id,'
- + emit_to_hex('ip') + ' AS ip,'
- 'symbol_id,'
- '(SELECT name FROM symbols WHERE id = symbol_id) AS symbol,'
- 'call_time,'
- 'return_time,'
- 'return_time - call_time AS elapsed_time,'
- 'branch_count,'
- 'insn_count,'
- 'cyc_count,'
- 'CASE WHEN cyc_count=0 THEN CAST(0 AS FLOAT) ELSE ROUND(CAST(insn_count AS FLOAT) / cyc_count, 2) END AS IPC,'
- 'call_id,'
- 'return_id,'
- 'CASE WHEN flags=0 THEN \'\' WHEN flags=1 THEN \'no call\' WHEN flags=2 THEN \'no return\' WHEN flags=3 THEN \'no call/return\' WHEN flags=6 THEN \'jump\' ELSE flags END AS flags,'
- 'parent_call_path_id,'
- 'calls.parent_id'
- ' FROM calls INNER JOIN call_paths ON call_paths.id = call_path_id')
-
-do_query(query, 'CREATE VIEW samples_view AS '
- 'SELECT '
- 'id,'
- 'time,'
- 'cpu,'
- '(SELECT pid FROM threads WHERE id = thread_id) AS pid,'
- '(SELECT tid FROM threads WHERE id = thread_id) AS tid,'
- '(SELECT comm FROM comms WHERE id = comm_id) AS command,'
- '(SELECT name FROM selected_events WHERE id = evsel_id) AS event,'
- + emit_to_hex('ip') + ' AS ip_hex,'
- '(SELECT name FROM symbols WHERE id = symbol_id) AS symbol,'
- 'sym_offset,'
- '(SELECT short_name FROM dsos WHERE id = dso_id) AS dso_short_name,'
- + emit_to_hex('to_ip') + ' AS to_ip_hex,'
- '(SELECT name FROM symbols WHERE id = to_symbol_id) AS to_symbol,'
- 'to_sym_offset,'
- '(SELECT short_name FROM dsos WHERE id = to_dso_id) AS to_dso_short_name,'
- '(SELECT name FROM branch_types WHERE id = branch_type) AS branch_type_name,'
- 'in_tx,'
- 'insn_count,'
- 'cyc_count,'
- 'CASE WHEN cyc_count=0 THEN CAST(0 AS FLOAT) ELSE ROUND(CAST(insn_count AS FLOAT) / cyc_count, 2) END AS IPC,'
- 'flags'
- ' FROM samples')
-
-do_query(query, 'CREATE VIEW ptwrite_view AS '
- 'SELECT '
- 'ptwrite.id,'
- 'time,'
- 'cpu,'
- + emit_to_hex('payload') + ' AS payload_hex,'
- 'CASE WHEN exact_ip=0 THEN \'False\' ELSE \'True\' END AS exact_ip'
- ' FROM ptwrite'
- ' INNER JOIN samples ON samples.id = ptwrite.id')
-
-do_query(query, 'CREATE VIEW cbr_view AS '
- 'SELECT '
- 'cbr.id,'
- 'time,'
- 'cpu,'
- 'cbr,'
- 'mhz,'
- 'percent'
- ' FROM cbr'
- ' INNER JOIN samples ON samples.id = cbr.id')
-
-do_query(query, 'CREATE VIEW mwait_view AS '
- 'SELECT '
- 'mwait.id,'
- 'time,'
- 'cpu,'
- + emit_to_hex('hints') + ' AS hints_hex,'
- + emit_to_hex('extensions') + ' AS extensions_hex'
- ' FROM mwait'
- ' INNER JOIN samples ON samples.id = mwait.id')
-
-do_query(query, 'CREATE VIEW pwre_view AS '
- 'SELECT '
- 'pwre.id,'
- 'time,'
- 'cpu,'
- 'cstate,'
- 'subcstate,'
- 'CASE WHEN hw=0 THEN \'False\' ELSE \'True\' END AS hw'
- ' FROM pwre'
- ' INNER JOIN samples ON samples.id = pwre.id')
-
-do_query(query, 'CREATE VIEW exstop_view AS '
- 'SELECT '
- 'exstop.id,'
- 'time,'
- 'cpu,'
- 'CASE WHEN exact_ip=0 THEN \'False\' ELSE \'True\' END AS exact_ip'
- ' FROM exstop'
- ' INNER JOIN samples ON samples.id = exstop.id')
-
-do_query(query, 'CREATE VIEW pwrx_view AS '
- 'SELECT '
- 'pwrx.id,'
- 'time,'
- 'cpu,'
- 'deepest_cstate,'
- 'last_cstate,'
- 'CASE WHEN wake_reason=1 THEN \'Interrupt\''
- ' WHEN wake_reason=2 THEN \'Timer Deadline\''
- ' WHEN wake_reason=4 THEN \'Monitored Address\''
- ' WHEN wake_reason=8 THEN \'HW\''
- ' ELSE wake_reason '
- 'END AS wake_reason'
- ' FROM pwrx'
- ' INNER JOIN samples ON samples.id = pwrx.id')
-
-do_query(query, 'CREATE VIEW power_events_view AS '
- 'SELECT '
- 'samples.id,'
- 'time,'
- 'cpu,'
- 'selected_events.name AS event,'
- 'CASE WHEN selected_events.name=\'cbr\' THEN (SELECT cbr FROM cbr WHERE cbr.id = samples.id) ELSE "" END AS cbr,'
- 'CASE WHEN selected_events.name=\'cbr\' THEN (SELECT mhz FROM cbr WHERE cbr.id = samples.id) ELSE "" END AS mhz,'
- 'CASE WHEN selected_events.name=\'cbr\' THEN (SELECT percent FROM cbr WHERE cbr.id = samples.id) ELSE "" END AS percent,'
- 'CASE WHEN selected_events.name=\'mwait\' THEN (SELECT ' + emit_to_hex('hints') + ' FROM mwait WHERE mwait.id = samples.id) ELSE "" END AS hints_hex,'
- 'CASE WHEN selected_events.name=\'mwait\' THEN (SELECT ' + emit_to_hex('extensions') + ' FROM mwait WHERE mwait.id = samples.id) ELSE "" END AS extensions_hex,'
- 'CASE WHEN selected_events.name=\'pwre\' THEN (SELECT cstate FROM pwre WHERE pwre.id = samples.id) ELSE "" END AS cstate,'
- 'CASE WHEN selected_events.name=\'pwre\' THEN (SELECT subcstate FROM pwre WHERE pwre.id = samples.id) ELSE "" END AS subcstate,'
- 'CASE WHEN selected_events.name=\'pwre\' THEN (SELECT hw FROM pwre WHERE pwre.id = samples.id) ELSE "" END AS hw,'
- 'CASE WHEN selected_events.name=\'exstop\' THEN (SELECT exact_ip FROM exstop WHERE exstop.id = samples.id) ELSE "" END AS exact_ip,'
- 'CASE WHEN selected_events.name=\'pwrx\' THEN (SELECT deepest_cstate FROM pwrx WHERE pwrx.id = samples.id) ELSE "" END AS deepest_cstate,'
- 'CASE WHEN selected_events.name=\'pwrx\' THEN (SELECT last_cstate FROM pwrx WHERE pwrx.id = samples.id) ELSE "" END AS last_cstate,'
- 'CASE WHEN selected_events.name=\'pwrx\' THEN (SELECT '
- 'CASE WHEN wake_reason=1 THEN \'Interrupt\''
- ' WHEN wake_reason=2 THEN \'Timer Deadline\''
- ' WHEN wake_reason=4 THEN \'Monitored Address\''
- ' WHEN wake_reason=8 THEN \'HW\''
- ' ELSE wake_reason '
- 'END'
- ' FROM pwrx WHERE pwrx.id = samples.id) ELSE "" END AS wake_reason'
- ' FROM samples'
- ' INNER JOIN selected_events ON selected_events.id = evsel_id'
- ' WHERE selected_events.name IN (\'cbr\',\'mwait\',\'exstop\',\'pwre\',\'pwrx\')')
-
-do_query(query, 'CREATE VIEW context_switches_view AS '
- 'SELECT '
- 'context_switches.id,'
- 'context_switches.machine_id,'
- 'context_switches.time,'
- 'context_switches.cpu,'
- 'th_out.pid AS pid_out,'
- 'th_out.tid AS tid_out,'
- 'comm_out.comm AS comm_out,'
- 'th_in.pid AS pid_in,'
- 'th_in.tid AS tid_in,'
- 'comm_in.comm AS comm_in,'
- 'CASE WHEN context_switches.flags = 0 THEN \'in\''
- ' WHEN context_switches.flags = 1 THEN \'out\''
- ' WHEN context_switches.flags = 3 THEN \'out preempt\''
- ' ELSE context_switches.flags '
- 'END AS flags'
- ' FROM context_switches'
- ' INNER JOIN threads AS th_out ON th_out.id = context_switches.thread_out_id'
- ' INNER JOIN threads AS th_in ON th_in.id = context_switches.thread_in_id'
- ' INNER JOIN comms AS comm_out ON comm_out.id = context_switches.comm_out_id'
- ' INNER JOIN comms AS comm_in ON comm_in.id = context_switches.comm_in_id')
-
-do_query(query, 'END TRANSACTION')
-
-evsel_query = QSqlQuery(db)
-evsel_query.prepare("INSERT INTO selected_events VALUES (?, ?)")
-machine_query = QSqlQuery(db)
-machine_query.prepare("INSERT INTO machines VALUES (?, ?, ?)")
-thread_query = QSqlQuery(db)
-thread_query.prepare("INSERT INTO threads VALUES (?, ?, ?, ?, ?)")
-comm_query = QSqlQuery(db)
-comm_query.prepare("INSERT INTO comms VALUES (?, ?, ?, ?, ?)")
-comm_thread_query = QSqlQuery(db)
-comm_thread_query.prepare("INSERT INTO comm_threads VALUES (?, ?, ?)")
-dso_query = QSqlQuery(db)
-dso_query.prepare("INSERT INTO dsos VALUES (?, ?, ?, ?, ?)")
-symbol_query = QSqlQuery(db)
-symbol_query.prepare("INSERT INTO symbols VALUES (?, ?, ?, ?, ?, ?)")
-branch_type_query = QSqlQuery(db)
-branch_type_query.prepare("INSERT INTO branch_types VALUES (?, ?)")
-sample_query = QSqlQuery(db)
-if branches:
- sample_query.prepare("INSERT INTO samples VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)")
-else:
- sample_query.prepare("INSERT INTO samples VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)")
-if perf_db_export_calls or perf_db_export_callchains:
- call_path_query = QSqlQuery(db)
- call_path_query.prepare("INSERT INTO call_paths VALUES (?, ?, ?, ?)")
-if perf_db_export_calls:
- call_query = QSqlQuery(db)
- call_query.prepare("INSERT INTO calls VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)")
-ptwrite_query = QSqlQuery(db)
-ptwrite_query.prepare("INSERT INTO ptwrite VALUES (?, ?, ?)")
-cbr_query = QSqlQuery(db)
-cbr_query.prepare("INSERT INTO cbr VALUES (?, ?, ?, ?)")
-mwait_query = QSqlQuery(db)
-mwait_query.prepare("INSERT INTO mwait VALUES (?, ?, ?)")
-pwre_query = QSqlQuery(db)
-pwre_query.prepare("INSERT INTO pwre VALUES (?, ?, ?, ?)")
-exstop_query = QSqlQuery(db)
-exstop_query.prepare("INSERT INTO exstop VALUES (?, ?)")
-pwrx_query = QSqlQuery(db)
-pwrx_query.prepare("INSERT INTO pwrx VALUES (?, ?, ?, ?)")
-context_switch_query = QSqlQuery(db)
-context_switch_query.prepare("INSERT INTO context_switches VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)")
-
-def trace_begin():
- printdate("Writing records...")
- do_query(query, 'BEGIN TRANSACTION')
- # id == 0 means unknown. It is easier to create records for them than replace the zeroes with NULLs
- evsel_table(0, "unknown")
- machine_table(0, 0, "unknown")
- thread_table(0, 0, 0, -1, -1)
- comm_table(0, "unknown", 0, 0, 0)
- dso_table(0, 0, "unknown", "unknown", "")
- symbol_table(0, 0, 0, 0, 0, "unknown")
- sample_table(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
- if perf_db_export_calls or perf_db_export_callchains:
- call_path_table(0, 0, 0, 0)
- call_return_table(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
-
-unhandled_count = 0
-
-def is_table_empty(table_name):
- do_query(query, 'SELECT * FROM ' + table_name + ' LIMIT 1');
- if query.next():
- return False
- return True
-
-def drop(table_name):
- do_query(query, 'DROP VIEW ' + table_name + '_view');
- do_query(query, 'DROP TABLE ' + table_name);
-
-def trace_end():
- do_query(query, 'END TRANSACTION')
-
- printdate("Adding indexes")
- if perf_db_export_calls:
- do_query(query, 'CREATE INDEX pcpid_idx ON calls (parent_call_path_id)')
- do_query(query, 'CREATE INDEX pid_idx ON calls (parent_id)')
- do_query(query, 'ALTER TABLE comms ADD has_calls boolean')
- do_query(query, 'UPDATE comms SET has_calls = 1 WHERE comms.id IN (SELECT DISTINCT comm_id FROM calls)')
-
- printdate("Dropping unused tables")
- if is_table_empty("ptwrite"):
- drop("ptwrite")
- if is_table_empty("mwait") and is_table_empty("pwre") and is_table_empty("exstop") and is_table_empty("pwrx"):
- do_query(query, 'DROP VIEW power_events_view');
- drop("mwait")
- drop("pwre")
- drop("exstop")
- drop("pwrx")
- if is_table_empty("cbr"):
- drop("cbr")
- if is_table_empty("context_switches"):
- drop("context_switches")
-
- if (unhandled_count):
- printdate("Warning: ", unhandled_count, " unhandled events")
- printdate("Done")
-
-def trace_unhandled(event_name, context, event_fields_dict):
- global unhandled_count
- unhandled_count += 1
-
-def sched__sched_switch(*x):
- pass
-
-def bind_exec(q, n, x):
- for xx in x[0:n]:
- q.addBindValue(str(xx))
- do_query_(q)
-
-def evsel_table(*x):
- bind_exec(evsel_query, 2, x)
-
-def machine_table(*x):
- bind_exec(machine_query, 3, x)
-
-def thread_table(*x):
- bind_exec(thread_query, 5, x)
-
-def comm_table(*x):
- bind_exec(comm_query, 5, x)
-
-def comm_thread_table(*x):
- bind_exec(comm_thread_query, 3, x)
-
-def dso_table(*x):
- bind_exec(dso_query, 5, x)
-
-def symbol_table(*x):
- bind_exec(symbol_query, 6, x)
-
-def branch_type_table(*x):
- bind_exec(branch_type_query, 2, x)
-
-def sample_table(*x):
- if branches:
- for xx in x[0:15]:
- sample_query.addBindValue(str(xx))
- for xx in x[19:25]:
- sample_query.addBindValue(str(xx))
- do_query_(sample_query)
- else:
- bind_exec(sample_query, 25, x)
-
-def call_path_table(*x):
- bind_exec(call_path_query, 4, x)
-
-def call_return_table(*x):
- bind_exec(call_query, 14, x)
-
-def ptwrite(id, raw_buf):
- data = struct.unpack_from("<IQ", raw_buf)
- flags = data[0]
- payload = data[1]
- exact_ip = flags & 1
- ptwrite_query.addBindValue(str(id))
- ptwrite_query.addBindValue(str(payload))
- ptwrite_query.addBindValue(str(exact_ip))
- do_query_(ptwrite_query)
-
-def cbr(id, raw_buf):
- data = struct.unpack_from("<BBBBII", raw_buf)
- cbr = data[0]
- MHz = (data[4] + 500) / 1000
- percent = ((cbr * 1000 / data[2]) + 5) / 10
- cbr_query.addBindValue(str(id))
- cbr_query.addBindValue(str(cbr))
- cbr_query.addBindValue(str(MHz))
- cbr_query.addBindValue(str(percent))
- do_query_(cbr_query)
-
-def mwait(id, raw_buf):
- data = struct.unpack_from("<IQ", raw_buf)
- payload = data[1]
- hints = payload & 0xff
- extensions = (payload >> 32) & 0x3
- mwait_query.addBindValue(str(id))
- mwait_query.addBindValue(str(hints))
- mwait_query.addBindValue(str(extensions))
- do_query_(mwait_query)
-
-def pwre(id, raw_buf):
- data = struct.unpack_from("<IQ", raw_buf)
- payload = data[1]
- hw = (payload >> 7) & 1
- cstate = (payload >> 12) & 0xf
- subcstate = (payload >> 8) & 0xf
- pwre_query.addBindValue(str(id))
- pwre_query.addBindValue(str(cstate))
- pwre_query.addBindValue(str(subcstate))
- pwre_query.addBindValue(str(hw))
- do_query_(pwre_query)
-
-def exstop(id, raw_buf):
- data = struct.unpack_from("<I", raw_buf)
- flags = data[0]
- exact_ip = flags & 1
- exstop_query.addBindValue(str(id))
- exstop_query.addBindValue(str(exact_ip))
- do_query_(exstop_query)
-
-def pwrx(id, raw_buf):
- data = struct.unpack_from("<IQ", raw_buf)
- payload = data[1]
- deepest_cstate = payload & 0xf
- last_cstate = (payload >> 4) & 0xf
- wake_reason = (payload >> 8) & 0xf
- pwrx_query.addBindValue(str(id))
- pwrx_query.addBindValue(str(deepest_cstate))
- pwrx_query.addBindValue(str(last_cstate))
- pwrx_query.addBindValue(str(wake_reason))
- do_query_(pwrx_query)
-
-def synth_data(id, config, raw_buf, *x):
- if config == 0:
- ptwrite(id, raw_buf)
- elif config == 1:
- mwait(id, raw_buf)
- elif config == 2:
- pwre(id, raw_buf)
- elif config == 3:
- exstop(id, raw_buf)
- elif config == 4:
- pwrx(id, raw_buf)
- elif config == 5:
- cbr(id, raw_buf)
-
-def context_switch_table(*x):
- bind_exec(context_switch_query, 9, x)
diff --git a/tools/perf/scripts/python/failed-syscalls-by-pid.py b/tools/perf/scripts/python/failed-syscalls-by-pid.py
deleted file mode 100644
index 310efe5e7e23..000000000000
--- a/tools/perf/scripts/python/failed-syscalls-by-pid.py
+++ /dev/null
@@ -1,79 +0,0 @@
-# failed system call counts, by pid
-# (c) 2010, Tom Zanussi <tzanussi@gmail.com>
-# Licensed under the terms of the GNU GPL License version 2
-#
-# Displays system-wide failed system call totals, broken down by pid.
-# If a [comm] arg is specified, only syscalls called by [comm] are displayed.
-
-from __future__ import print_function
-
-import os
-import sys
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import *
-from Core import *
-from Util import *
-
-usage = "perf script -s syscall-counts-by-pid.py [comm|pid]\n";
-
-for_comm = None
-for_pid = None
-
-if len(sys.argv) > 2:
- sys.exit(usage)
-
-if len(sys.argv) > 1:
- try:
- for_pid = int(sys.argv[1])
- except:
- for_comm = sys.argv[1]
-
-syscalls = autodict()
-
-def trace_begin():
- print("Press control+C to stop and show the summary")
-
-def trace_end():
- print_error_totals()
-
-def raw_syscalls__sys_exit(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, id, ret):
- if (for_comm and common_comm != for_comm) or \
- (for_pid and common_pid != for_pid ):
- return
-
- if ret < 0:
- try:
- syscalls[common_comm][common_pid][id][ret] += 1
- except TypeError:
- syscalls[common_comm][common_pid][id][ret] = 1
-
-def syscalls__sys_exit(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- id, ret):
- raw_syscalls__sys_exit(**locals())
-
-def print_error_totals():
- if for_comm is not None:
- print("\nsyscall errors for %s:\n" % (for_comm))
- else:
- print("\nsyscall errors:\n")
-
- print("%-30s %10s" % ("comm [pid]", "count"))
- print("%-30s %10s" % ("------------------------------", "----------"))
-
- comm_keys = syscalls.keys()
- for comm in comm_keys:
- pid_keys = syscalls[comm].keys()
- for pid in pid_keys:
- print("\n%s [%d]" % (comm, pid))
- id_keys = syscalls[comm][pid].keys()
- for id in id_keys:
- print(" syscall: %-16s" % syscall_name(id))
- ret_keys = syscalls[comm][pid][id].keys()
- for ret, val in sorted(syscalls[comm][pid][id].items(), key = lambda kv: (kv[1], kv[0]), reverse = True):
- print(" err = %-20s %10d" % (strerror(ret), val))
diff --git a/tools/perf/scripts/python/flamegraph.py b/tools/perf/scripts/python/flamegraph.py
deleted file mode 100755
index ad735990c5be..000000000000
--- a/tools/perf/scripts/python/flamegraph.py
+++ /dev/null
@@ -1,267 +0,0 @@
-# flamegraph.py - create flame graphs from perf samples
-# SPDX-License-Identifier: GPL-2.0
-#
-# Usage:
-#
-# perf record -a -g -F 99 sleep 60
-# perf script report flamegraph
-#
-# Combined:
-#
-# perf script flamegraph -a -F 99 sleep 60
-#
-# Written by Andreas Gerstmayr <agerstmayr@redhat.com>
-# Flame Graphs invented by Brendan Gregg <bgregg@netflix.com>
-# Works in tandem with d3-flame-graph by Martin Spier <mspier@netflix.com>
-#
-# pylint: disable=missing-module-docstring
-# pylint: disable=missing-class-docstring
-# pylint: disable=missing-function-docstring
-
-import argparse
-import hashlib
-import io
-import json
-import os
-import subprocess
-import sys
-from typing import Dict, Optional, Union
-import urllib.request
-
-MINIMAL_HTML = """<head>
- <link rel="stylesheet" type="text/css" href="https://cdn.jsdelivr.net/npm/d3-flame-graph@4.1.3/dist/d3-flamegraph.css">
-</head>
-<body>
- <div id="chart"></div>
- <script type="text/javascript" src="https://d3js.org/d3.v7.js"></script>
- <script type="text/javascript" src="https://cdn.jsdelivr.net/npm/d3-flame-graph@4.1.3/dist/d3-flamegraph.min.js"></script>
- <script type="text/javascript">
- const stacks = [/** @flamegraph_json **/];
- // Note, options is unused.
- const options = [/** @options_json **/];
-
- var chart = flamegraph();
- d3.select("#chart")
- .datum(stacks[0])
- .call(chart);
- </script>
-</body>
-"""
-
-# pylint: disable=too-few-public-methods
-class Node:
- def __init__(self, name: str, libtype: str):
- self.name = name
- # "root" | "kernel" | ""
- # "" indicates user space
- self.libtype = libtype
- self.value: int = 0
- self.children: list[Node] = []
-
- def to_json(self) -> Dict[str, Union[str, int, list[Dict]]]:
- return {
- "n": self.name,
- "l": self.libtype,
- "v": self.value,
- "c": [x.to_json() for x in self.children]
- }
-
-
-class FlameGraphCLI:
- def __init__(self, args):
- self.args = args
- self.stack = Node("all", "root")
-
- @staticmethod
- def get_libtype_from_dso(dso: Optional[str]) -> str:
- """
- when kernel-debuginfo is installed,
- dso points to /usr/lib/debug/lib/modules/*/vmlinux
- """
- if dso and (dso == "[kernel.kallsyms]" or dso.endswith("/vmlinux")):
- return "kernel"
-
- return ""
-
- @staticmethod
- def find_or_create_node(node: Node, name: str, libtype: str) -> Node:
- for child in node.children:
- if child.name == name:
- return child
-
- child = Node(name, libtype)
- node.children.append(child)
- return child
-
- def process_event(self, event) -> None:
- # ignore events where the event name does not match
- # the one specified by the user
- if self.args.event_name and event.get("ev_name") != self.args.event_name:
- return
-
- pid = event.get("sample", {}).get("pid", 0)
- # event["dso"] sometimes contains /usr/lib/debug/lib/modules/*/vmlinux
- # for user-space processes; let's use pid for kernel or user-space distinction
- if pid == 0:
- comm = event["comm"]
- libtype = "kernel"
- else:
- comm = f"{event['comm']} ({pid})"
- libtype = ""
- node = self.find_or_create_node(self.stack, comm, libtype)
-
- if "callchain" in event:
- for entry in reversed(event["callchain"]):
- name = entry.get("sym", {}).get("name", "[unknown]")
- libtype = self.get_libtype_from_dso(entry.get("dso"))
- node = self.find_or_create_node(node, name, libtype)
- else:
- name = event.get("symbol", "[unknown]")
- libtype = self.get_libtype_from_dso(event.get("dso"))
- node = self.find_or_create_node(node, name, libtype)
- node.value += 1
-
- def get_report_header(self) -> str:
- if self.args.input == "-":
- # when this script is invoked with "perf script flamegraph",
- # no perf.data is created and we cannot read the header of it
- return ""
-
- try:
- # if the file name other than perf.data is given,
- # we read the header of that file
- if self.args.input:
- output = subprocess.check_output(["perf", "report", "--header-only",
- "-i", self.args.input])
- else:
- output = subprocess.check_output(["perf", "report", "--header-only"])
-
- result = output.decode("utf-8")
- if self.args.event_name:
- result += "\nFocused event: " + self.args.event_name
- return result
- except Exception as err: # pylint: disable=broad-except
- print(f"Error reading report header: {err}", file=sys.stderr)
- return ""
-
- def trace_end(self) -> None:
- stacks_json = json.dumps(self.stack, default=lambda x: x.to_json())
-
- if self.args.format == "html":
- report_header = self.get_report_header()
- options = {
- "colorscheme": self.args.colorscheme,
- "context": report_header
- }
- options_json = json.dumps(options)
-
- template_md5sum = None
- if self.args.format == "html":
- if os.path.isfile(self.args.template):
- template = f"file://{self.args.template}"
- else:
- if not self.args.allow_download:
- print(f"""Warning: Flame Graph template '{self.args.template}'
-does not exist. To avoid this please install a package such as the
-js-d3-flame-graph or libjs-d3-flame-graph, specify an existing flame
-graph template (--template PATH) or use another output format (--format
-FORMAT).""",
- file=sys.stderr)
- if self.args.input == "-":
- print(
-"""Not attempting to download Flame Graph template as script command line
-input is disabled due to using live mode. If you want to download the
-template retry without live mode. For example, use 'perf record -a -g
--F 99 sleep 60' and 'perf script report flamegraph'. Alternatively,
-download the template from:
-https://cdn.jsdelivr.net/npm/d3-flame-graph@4.1.3/dist/templates/d3-flamegraph-base.html
-and place it at:
-/usr/share/d3-flame-graph/d3-flamegraph-base.html""",
- file=sys.stderr)
- sys.exit(1)
- s = None
- while s not in ["y", "n"]:
- s = input("Do you wish to download a template from cdn.jsdelivr.net?" +
- "(this warning can be suppressed with --allow-download) [yn] "
- ).lower()
- if s == "n":
- sys.exit(1)
- template = ("https://cdn.jsdelivr.net/npm/d3-flame-graph@4.1.3/dist/templates/"
- "d3-flamegraph-base.html")
- template_md5sum = "143e0d06ba69b8370b9848dcd6ae3f36"
-
- try:
- with urllib.request.urlopen(template) as url_template:
- output_str = "".join([
- l.decode("utf-8") for l in url_template.readlines()
- ])
- except Exception as err:
- print(f"Error reading template {template}: {err}\n"
- "a minimal flame graph will be generated", file=sys.stderr)
- output_str = MINIMAL_HTML
- template_md5sum = None
-
- if template_md5sum:
- download_md5sum = hashlib.md5(output_str.encode("utf-8")).hexdigest()
- if download_md5sum != template_md5sum:
- s = None
- while s not in ["y", "n"]:
- s = input(f"""Unexpected template md5sum.
-{download_md5sum} != {template_md5sum}, for:
-{output_str}
-continue?[yn] """).lower()
- if s == "n":
- sys.exit(1)
-
- output_str = output_str.replace("/** @options_json **/", options_json)
- output_str = output_str.replace("/** @flamegraph_json **/", stacks_json)
-
- output_fn = self.args.output or "flamegraph.html"
- else:
- output_str = stacks_json
- output_fn = self.args.output or "stacks.json"
-
- if output_fn == "-":
- with io.open(sys.stdout.fileno(), "w", encoding="utf-8", closefd=False) as out:
- out.write(output_str)
- else:
- print(f"dumping data to {output_fn}")
- try:
- with io.open(output_fn, "w", encoding="utf-8") as out:
- out.write(output_str)
- except IOError as err:
- print(f"Error writing output file: {err}", file=sys.stderr)
- sys.exit(1)
-
-
-if __name__ == "__main__":
- parser = argparse.ArgumentParser(description="Create flame graphs.")
- parser.add_argument("-f", "--format",
- default="html", choices=["json", "html"],
- help="output file format")
- parser.add_argument("-o", "--output",
- help="output file name")
- parser.add_argument("--template",
- default="/usr/share/d3-flame-graph/d3-flamegraph-base.html",
- help="path to flame graph HTML template")
- parser.add_argument("--colorscheme",
- default="blue-green",
- help="flame graph color scheme",
- choices=["blue-green", "orange"])
- parser.add_argument("-i", "--input",
- help=argparse.SUPPRESS)
- parser.add_argument("--allow-download",
- default=False,
- action="store_true",
- help="allow unprompted downloading of HTML template")
- parser.add_argument("-e", "--event",
- default="",
- dest="event_name",
- type=str,
- help="specify the event to generate flamegraph for")
-
- cli_args = parser.parse_args()
- cli = FlameGraphCLI(cli_args)
-
- process_event = cli.process_event
- trace_end = cli.trace_end
diff --git a/tools/perf/scripts/python/futex-contention.py b/tools/perf/scripts/python/futex-contention.py
deleted file mode 100644
index 7e884d46f920..000000000000
--- a/tools/perf/scripts/python/futex-contention.py
+++ /dev/null
@@ -1,57 +0,0 @@
-# futex contention
-# (c) 2010, Arnaldo Carvalho de Melo <acme@redhat.com>
-# Licensed under the terms of the GNU GPL License version 2
-#
-# Translation of:
-#
-# http://sourceware.org/systemtap/wiki/WSFutexContention
-#
-# to perf python scripting.
-#
-# Measures futex contention
-
-from __future__ import print_function
-
-import os
-import sys
-sys.path.append(os.environ['PERF_EXEC_PATH'] +
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-from Util import *
-
-process_names = {}
-thread_thislock = {}
-thread_blocktime = {}
-
-lock_waits = {} # long-lived stats on (tid,lock) blockage elapsed time
-process_names = {} # long-lived pid-to-execname mapping
-
-
-def syscalls__sys_enter_futex(event, ctxt, cpu, s, ns, tid, comm, callchain,
- nr, uaddr, op, val, utime, uaddr2, val3):
- cmd = op & FUTEX_CMD_MASK
- if cmd != FUTEX_WAIT:
- return # we don't care about originators of WAKE events
-
- process_names[tid] = comm
- thread_thislock[tid] = uaddr
- thread_blocktime[tid] = nsecs(s, ns)
-
-
-def syscalls__sys_exit_futex(event, ctxt, cpu, s, ns, tid, comm, callchain,
- nr, ret):
- if tid in thread_blocktime:
- elapsed = nsecs(s, ns) - thread_blocktime[tid]
- add_stats(lock_waits, (tid, thread_thislock[tid]), elapsed)
- del thread_blocktime[tid]
- del thread_thislock[tid]
-
-
-def trace_begin():
- print("Press control+C to stop and show the summary")
-
-
-def trace_end():
- for (tid, lock) in lock_waits:
- min, max, avg, count = lock_waits[tid, lock]
- print("%s[%d] lock %x contended %d times, %d avg ns [max: %d ns, min %d ns]" %
- (process_names[tid], tid, lock, count, avg, max, min))
diff --git a/tools/perf/scripts/python/gecko.py b/tools/perf/scripts/python/gecko.py
deleted file mode 100644
index bc5a72f94bfa..000000000000
--- a/tools/perf/scripts/python/gecko.py
+++ /dev/null
@@ -1,395 +0,0 @@
-# gecko.py - Convert perf record output to Firefox's gecko profile format
-# SPDX-License-Identifier: GPL-2.0
-#
-# The script converts perf.data to Gecko Profile Format,
-# which can be read by https://profiler.firefox.com/.
-#
-# Usage:
-#
-# perf record -a -g -F 99 sleep 60
-# perf script report gecko
-#
-# Combined:
-#
-# perf script gecko -F 99 -a sleep 60
-
-import os
-import sys
-import time
-import json
-import string
-import random
-import argparse
-import threading
-import webbrowser
-import urllib.parse
-from os import system
-from functools import reduce
-from dataclasses import dataclass, field
-from http.server import HTTPServer, SimpleHTTPRequestHandler, test
-from typing import List, Dict, Optional, NamedTuple, Set, Tuple, Any
-
-# Add the Perf-Trace-Util library to the Python path
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import *
-from Core import *
-
-StringID = int
-StackID = int
-FrameID = int
-CategoryID = int
-Milliseconds = float
-
-# start_time is intialiazed only once for the all event traces.
-start_time = None
-
-# https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/profile.js#L425
-# Follow Brendan Gregg's Flamegraph convention: orange for kernel and yellow for user space by default.
-CATEGORIES = None
-
-# The product name is used by the profiler UI to show the Operating system and Processor.
-PRODUCT = os.popen('uname -op').read().strip()
-
-# store the output file
-output_file = None
-
-# Here key = tid, value = Thread
-tid_to_thread = dict()
-
-# The HTTP server is used to serve the profile to the profiler UI.
-http_server_thread = None
-
-# The category index is used by the profiler UI to show the color of the flame graph.
-USER_CATEGORY_INDEX = 0
-KERNEL_CATEGORY_INDEX = 1
-
-# https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L156
-class Frame(NamedTuple):
- string_id: StringID
- relevantForJS: bool
- innerWindowID: int
- implementation: None
- optimizations: None
- line: None
- column: None
- category: CategoryID
- subcategory: int
-
-# https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L216
-class Stack(NamedTuple):
- prefix_id: Optional[StackID]
- frame_id: FrameID
-
-# https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L90
-class Sample(NamedTuple):
- stack_id: Optional[StackID]
- time_ms: Milliseconds
- responsiveness: int
-
-@dataclass
-class Thread:
- """A builder for a profile of the thread.
-
- Attributes:
- comm: Thread command-line (name).
- pid: process ID of containing process.
- tid: thread ID.
- samples: Timeline of profile samples.
- frameTable: interned stack frame ID -> stack frame.
- stringTable: interned string ID -> string.
- stringMap: interned string -> string ID.
- stackTable: interned stack ID -> stack.
- stackMap: (stack prefix ID, leaf stack frame ID) -> interned Stack ID.
- frameMap: Stack Frame string -> interned Frame ID.
- comm: str
- pid: int
- tid: int
- samples: List[Sample] = field(default_factory=list)
- frameTable: List[Frame] = field(default_factory=list)
- stringTable: List[str] = field(default_factory=list)
- stringMap: Dict[str, int] = field(default_factory=dict)
- stackTable: List[Stack] = field(default_factory=list)
- stackMap: Dict[Tuple[Optional[int], int], int] = field(default_factory=dict)
- frameMap: Dict[str, int] = field(default_factory=dict)
- """
- comm: str
- pid: int
- tid: int
- samples: List[Sample] = field(default_factory=list)
- frameTable: List[Frame] = field(default_factory=list)
- stringTable: List[str] = field(default_factory=list)
- stringMap: Dict[str, int] = field(default_factory=dict)
- stackTable: List[Stack] = field(default_factory=list)
- stackMap: Dict[Tuple[Optional[int], int], int] = field(default_factory=dict)
- frameMap: Dict[str, int] = field(default_factory=dict)
-
- def _intern_stack(self, frame_id: int, prefix_id: Optional[int]) -> int:
- """Gets a matching stack, or saves the new stack. Returns a Stack ID."""
- key = f"{frame_id}" if prefix_id is None else f"{frame_id},{prefix_id}"
- # key = (prefix_id, frame_id)
- stack_id = self.stackMap.get(key)
- if stack_id is None:
- # return stack_id
- stack_id = len(self.stackTable)
- self.stackTable.append(Stack(prefix_id=prefix_id, frame_id=frame_id))
- self.stackMap[key] = stack_id
- return stack_id
-
- def _intern_string(self, string: str) -> int:
- """Gets a matching string, or saves the new string. Returns a String ID."""
- string_id = self.stringMap.get(string)
- if string_id is not None:
- return string_id
- string_id = len(self.stringTable)
- self.stringTable.append(string)
- self.stringMap[string] = string_id
- return string_id
-
- def _intern_frame(self, frame_str: str) -> int:
- """Gets a matching stack frame, or saves the new frame. Returns a Frame ID."""
- frame_id = self.frameMap.get(frame_str)
- if frame_id is not None:
- return frame_id
- frame_id = len(self.frameTable)
- self.frameMap[frame_str] = frame_id
- string_id = self._intern_string(frame_str)
-
- symbol_name_to_category = KERNEL_CATEGORY_INDEX if frame_str.find('kallsyms') != -1 \
- or frame_str.find('/vmlinux') != -1 \
- or frame_str.endswith('.ko)') \
- else USER_CATEGORY_INDEX
-
- self.frameTable.append(Frame(
- string_id=string_id,
- relevantForJS=False,
- innerWindowID=0,
- implementation=None,
- optimizations=None,
- line=None,
- column=None,
- category=symbol_name_to_category,
- subcategory=None,
- ))
- return frame_id
-
- def _add_sample(self, comm: str, stack: List[str], time_ms: Milliseconds) -> None:
- """Add a timestamped stack trace sample to the thread builder.
- Args:
- comm: command-line (name) of the thread at this sample
- stack: sampled stack frames. Root first, leaf last.
- time_ms: timestamp of sample in milliseconds.
- """
- # Ihreads may not set their names right after they are created.
- # Instead, they might do it later. In such situations, to use the latest name they have set.
- if self.comm != comm:
- self.comm = comm
-
- prefix_stack_id = reduce(lambda prefix_id, frame: self._intern_stack
- (self._intern_frame(frame), prefix_id), stack, None)
- if prefix_stack_id is not None:
- self.samples.append(Sample(stack_id=prefix_stack_id,
- time_ms=time_ms,
- responsiveness=0))
-
- def _to_json_dict(self) -> Dict:
- """Converts current Thread to GeckoThread JSON format."""
- # Gecko profile format is row-oriented data as List[List],
- # And a schema for interpreting each index.
- # Schema:
- # https://github.com/firefox-devtools/profiler/blob/main/docs-developer/gecko-profile-format.md
- # https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L230
- return {
- "tid": self.tid,
- "pid": self.pid,
- "name": self.comm,
- # https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L51
- "markers": {
- "schema": {
- "name": 0,
- "startTime": 1,
- "endTime": 2,
- "phase": 3,
- "category": 4,
- "data": 5,
- },
- "data": [],
- },
-
- # https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L90
- "samples": {
- "schema": {
- "stack": 0,
- "time": 1,
- "responsiveness": 2,
- },
- "data": self.samples
- },
-
- # https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L156
- "frameTable": {
- "schema": {
- "location": 0,
- "relevantForJS": 1,
- "innerWindowID": 2,
- "implementation": 3,
- "optimizations": 4,
- "line": 5,
- "column": 6,
- "category": 7,
- "subcategory": 8,
- },
- "data": self.frameTable,
- },
-
- # https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L216
- "stackTable": {
- "schema": {
- "prefix": 0,
- "frame": 1,
- },
- "data": self.stackTable,
- },
- "stringTable": self.stringTable,
- "registerTime": 0,
- "unregisterTime": None,
- "processType": "default",
- }
-
-# Uses perf script python interface to parse each
-# event and store the data in the thread builder.
-def process_event(param_dict: Dict) -> None:
- global start_time
- global tid_to_thread
- time_stamp = (param_dict['sample']['time'] // 1000) / 1000
- pid = param_dict['sample']['pid']
- tid = param_dict['sample']['tid']
- comm = param_dict['comm']
-
- # Start time is the time of the first sample
- if not start_time:
- start_time = time_stamp
-
- # Parse and append the callchain of the current sample into a stack.
- stack = []
- if param_dict['callchain']:
- for call in param_dict['callchain']:
- if 'sym' not in call:
- continue
- stack.append(f'{call["sym"]["name"]} (in {call["dso"]})')
- if len(stack) != 0:
- # Reverse the stack, as root come first and the leaf at the end.
- stack = stack[::-1]
-
- # During perf record if -g is not used, the callchain is not available.
- # In that case, the symbol and dso are available in the event parameters.
- else:
- func = param_dict['symbol'] if 'symbol' in param_dict else '[unknown]'
- dso = param_dict['dso'] if 'dso' in param_dict else '[unknown]'
- stack.append(f'{func} (in {dso})')
-
- # Add sample to the specific thread.
- thread = tid_to_thread.get(tid)
- if thread is None:
- thread = Thread(comm=comm, pid=pid, tid=tid)
- tid_to_thread[tid] = thread
- thread._add_sample(comm=comm, stack=stack, time_ms=time_stamp)
-
-def trace_begin() -> None:
- global output_file
- if (output_file is None):
- print("Staring Firefox Profiler on your default browser...")
- global http_server_thread
- http_server_thread = threading.Thread(target=test, args=(CORSRequestHandler, HTTPServer,))
- http_server_thread.daemon = True
- http_server_thread.start()
-
-# Trace_end runs at the end and will be used to aggregate
-# the data into the final json object and print it out to stdout.
-def trace_end() -> None:
- global output_file
- threads = [thread._to_json_dict() for thread in tid_to_thread.values()]
-
- # Schema: https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L305
- gecko_profile_with_meta = {
- "meta": {
- "interval": 1,
- "processType": 0,
- "product": PRODUCT,
- "stackwalk": 1,
- "debug": 0,
- "gcpoison": 0,
- "asyncstack": 1,
- "startTime": start_time,
- "shutdownTime": None,
- "version": 24,
- "presymbolicated": True,
- "categories": CATEGORIES,
- "markerSchema": [],
- },
- "libs": [],
- "threads": threads,
- "processes": [],
- "pausedRanges": [],
- }
- # launch the profiler on local host if not specified --save-only args, otherwise print to file
- if (output_file is None):
- output_file = 'gecko_profile.json'
- with open(output_file, 'w') as f:
- json.dump(gecko_profile_with_meta, f, indent=2)
- launchFirefox(output_file)
- time.sleep(1)
- print(f'[ perf gecko: Captured and wrote into {output_file} ]')
- else:
- print(f'[ perf gecko: Captured and wrote into {output_file} ]')
- with open(output_file, 'w') as f:
- json.dump(gecko_profile_with_meta, f, indent=2)
-
-# Used to enable Cross-Origin Resource Sharing (CORS) for requests coming from 'https://profiler.firefox.com', allowing it to access resources from this server.
-class CORSRequestHandler(SimpleHTTPRequestHandler):
- def end_headers (self):
- self.send_header('Access-Control-Allow-Origin', 'https://profiler.firefox.com')
- SimpleHTTPRequestHandler.end_headers(self)
-
-# start a local server to serve the gecko_profile.json file to the profiler.firefox.com
-def launchFirefox(file):
- safe_string = urllib.parse.quote_plus(f'http://localhost:8000/{file}')
- url = 'https://profiler.firefox.com/from-url/' + safe_string
- webbrowser.open(f'{url}')
-
-def main() -> None:
- global output_file
- global CATEGORIES
- parser = argparse.ArgumentParser(description="Convert perf.data to Firefox\'s Gecko Profile format which can be uploaded to profiler.firefox.com for visualization")
-
- # Add the command-line options
- # Colors must be defined according to this:
- # https://github.com/firefox-devtools/profiler/blob/50124adbfa488adba6e2674a8f2618cf34b59cd2/res/css/categories.css
- parser.add_argument('--user-color', default='yellow', help='Color for the User category', choices=['yellow', 'blue', 'purple', 'green', 'orange', 'red', 'grey', 'magenta'])
- parser.add_argument('--kernel-color', default='orange', help='Color for the Kernel category', choices=['yellow', 'blue', 'purple', 'green', 'orange', 'red', 'grey', 'magenta'])
- # If --save-only is specified, the output will be saved to a file instead of opening Firefox's profiler directly.
- parser.add_argument('--save-only', help='Save the output to a file instead of opening Firefox\'s profiler')
-
- # Parse the command-line arguments
- args = parser.parse_args()
- # Access the values provided by the user
- user_color = args.user_color
- kernel_color = args.kernel_color
- output_file = args.save_only
-
- CATEGORIES = [
- {
- "name": 'User',
- "color": user_color,
- "subcategories": ['Other']
- },
- {
- "name": 'Kernel',
- "color": kernel_color,
- "subcategories": ['Other']
- },
- ]
-
-if __name__ == '__main__':
- main()
diff --git a/tools/perf/scripts/python/intel-pt-events.py b/tools/perf/scripts/python/intel-pt-events.py
deleted file mode 100644
index 346c89bd16d6..000000000000
--- a/tools/perf/scripts/python/intel-pt-events.py
+++ /dev/null
@@ -1,494 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-# intel-pt-events.py: Print Intel PT Events including Power Events and PTWRITE
-# Copyright (c) 2017-2021, Intel Corporation.
-#
-# This program is free software; you can redistribute it and/or modify it
-# under the terms and conditions of the GNU General Public License,
-# version 2, as published by the Free Software Foundation.
-#
-# This program is distributed in the hope it will be useful, but WITHOUT
-# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
-# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
-# more details.
-
-from __future__ import division, print_function
-
-import io
-import os
-import sys
-import struct
-import argparse
-import contextlib
-
-from libxed import LibXED
-from ctypes import create_string_buffer, addressof
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import perf_set_itrace_options, \
- perf_sample_insn, perf_sample_srccode
-
-try:
- broken_pipe_exception = BrokenPipeError
-except:
- broken_pipe_exception = IOError
-
-glb_switch_str = {}
-glb_insn = False
-glb_disassembler = None
-glb_src = False
-glb_source_file_name = None
-glb_line_number = None
-glb_dso = None
-glb_stash_dict = {}
-glb_output = None
-glb_output_pos = 0
-glb_cpu = -1
-glb_time = 0
-
-def get_optional_null(perf_dict, field):
- if field in perf_dict:
- return perf_dict[field]
- return ""
-
-def get_optional_zero(perf_dict, field):
- if field in perf_dict:
- return perf_dict[field]
- return 0
-
-def get_optional_bytes(perf_dict, field):
- if field in perf_dict:
- return perf_dict[field]
- return bytes()
-
-def get_optional(perf_dict, field):
- if field in perf_dict:
- return perf_dict[field]
- return "[unknown]"
-
-def get_offset(perf_dict, field):
- if field in perf_dict:
- return "+%#x" % perf_dict[field]
- return ""
-
-def trace_begin():
- ap = argparse.ArgumentParser(usage = "", add_help = False)
- ap.add_argument("--insn-trace", action='store_true')
- ap.add_argument("--src-trace", action='store_true')
- ap.add_argument("--all-switch-events", action='store_true')
- ap.add_argument("--interleave", type=int, nargs='?', const=4, default=0)
- global glb_args
- global glb_insn
- global glb_src
- glb_args = ap.parse_args()
- if glb_args.insn_trace:
- print("Intel PT Instruction Trace")
- itrace = "i0nsepwxI"
- glb_insn = True
- elif glb_args.src_trace:
- print("Intel PT Source Trace")
- itrace = "i0nsepwxI"
- glb_insn = True
- glb_src = True
- else:
- print("Intel PT Branch Trace, Power Events, Event Trace and PTWRITE")
- itrace = "bepwxI"
- global glb_disassembler
- try:
- glb_disassembler = LibXED()
- except:
- glb_disassembler = None
- perf_set_itrace_options(perf_script_context, itrace)
-
-def trace_end():
- if glb_args.interleave:
- flush_stashed_output()
- print("End")
-
-def trace_unhandled(event_name, context, event_fields_dict):
- print(' '.join(['%s=%s'%(k,str(v))for k,v in sorted(event_fields_dict.items())]))
-
-def stash_output():
- global glb_stash_dict
- global glb_output_pos
- output_str = glb_output.getvalue()[glb_output_pos:]
- n = len(output_str)
- if n:
- glb_output_pos += n
- if glb_cpu not in glb_stash_dict:
- glb_stash_dict[glb_cpu] = []
- glb_stash_dict[glb_cpu].append(output_str)
-
-def flush_stashed_output():
- global glb_stash_dict
- while glb_stash_dict:
- cpus = list(glb_stash_dict.keys())
- # Output at most glb_args.interleave output strings per cpu
- for cpu in cpus:
- items = glb_stash_dict[cpu]
- countdown = glb_args.interleave
- while len(items) and countdown:
- sys.stdout.write(items[0])
- del items[0]
- countdown -= 1
- if not items:
- del glb_stash_dict[cpu]
-
-def print_ptwrite(raw_buf):
- data = struct.unpack_from("<IQ", raw_buf)
- flags = data[0]
- payload = data[1]
- exact_ip = flags & 1
- try:
- s = payload.to_bytes(8, "little").decode("ascii").rstrip("\x00")
- if not s.isprintable():
- s = ""
- except:
- s = ""
- print("IP: %u payload: %#x" % (exact_ip, payload), s, end=' ')
-
-def print_cbr(raw_buf):
- data = struct.unpack_from("<BBBBII", raw_buf)
- cbr = data[0]
- f = (data[4] + 500) / 1000
- p = ((cbr * 1000 / data[2]) + 5) / 10
- print("%3u freq: %4u MHz (%3u%%)" % (cbr, f, p), end=' ')
-
-def print_mwait(raw_buf):
- data = struct.unpack_from("<IQ", raw_buf)
- payload = data[1]
- hints = payload & 0xff
- extensions = (payload >> 32) & 0x3
- print("hints: %#x extensions: %#x" % (hints, extensions), end=' ')
-
-def print_pwre(raw_buf):
- data = struct.unpack_from("<IQ", raw_buf)
- payload = data[1]
- hw = (payload >> 7) & 1
- cstate = (payload >> 12) & 0xf
- subcstate = (payload >> 8) & 0xf
- print("hw: %u cstate: %u sub-cstate: %u" % (hw, cstate, subcstate),
- end=' ')
-
-def print_exstop(raw_buf):
- data = struct.unpack_from("<I", raw_buf)
- flags = data[0]
- exact_ip = flags & 1
- print("IP: %u" % (exact_ip), end=' ')
-
-def print_pwrx(raw_buf):
- data = struct.unpack_from("<IQ", raw_buf)
- payload = data[1]
- deepest_cstate = payload & 0xf
- last_cstate = (payload >> 4) & 0xf
- wake_reason = (payload >> 8) & 0xf
- print("deepest cstate: %u last cstate: %u wake reason: %#x" %
- (deepest_cstate, last_cstate, wake_reason), end=' ')
-
-def print_psb(raw_buf):
- data = struct.unpack_from("<IQ", raw_buf)
- offset = data[1]
- print("offset: %#x" % (offset), end=' ')
-
-glb_cfe = ["", "INTR", "IRET", "SMI", "RSM", "SIPI", "INIT", "VMENTRY", "VMEXIT",
- "VMEXIT_INTR", "SHUTDOWN", "", "UINT", "UIRET"] + [""] * 18
-glb_evd = ["", "PFA", "VMXQ", "VMXR"] + [""] * 60
-
-def print_evt(raw_buf):
- data = struct.unpack_from("<BBH", raw_buf)
- typ = data[0] & 0x1f
- ip_flag = (data[0] & 0x80) >> 7
- vector = data[1]
- evd_cnt = data[2]
- s = glb_cfe[typ]
- if s:
- print(" cfe: %s IP: %u vector: %u" % (s, ip_flag, vector), end=' ')
- else:
- print(" cfe: %u IP: %u vector: %u" % (typ, ip_flag, vector), end=' ')
- pos = 4
- for i in range(evd_cnt):
- data = struct.unpack_from("<QQ", raw_buf)
- et = data[0] & 0x3f
- s = glb_evd[et]
- if s:
- print("%s: %#x" % (s, data[1]), end=' ')
- else:
- print("EVD_%u: %#x" % (et, data[1]), end=' ')
-
-def print_iflag(raw_buf):
- data = struct.unpack_from("<IQ", raw_buf)
- iflag = data[0] & 1
- old_iflag = iflag ^ 1
- via_branch = data[0] & 2
- branch_ip = data[1]
- if via_branch:
- s = "via"
- else:
- s = "non"
- print("IFLAG: %u->%u %s branch" % (old_iflag, iflag, s), end=' ')
-
-def common_start_str(comm, sample):
- ts = sample["time"]
- cpu = sample["cpu"]
- pid = sample["pid"]
- tid = sample["tid"]
- if "machine_pid" in sample:
- machine_pid = sample["machine_pid"]
- vcpu = sample["vcpu"]
- return "VM:%5d VCPU:%03d %16s %5u/%-5u [%03u] %9u.%09u " % (machine_pid, vcpu, comm, pid, tid, cpu, ts / 1000000000, ts %1000000000)
- else:
- return "%16s %5u/%-5u [%03u] %9u.%09u " % (comm, pid, tid, cpu, ts / 1000000000, ts %1000000000)
-
-def print_common_start(comm, sample, name):
- flags_disp = get_optional_null(sample, "flags_disp")
- # Unused fields:
- # period = sample["period"]
- # phys_addr = sample["phys_addr"]
- # weight = sample["weight"]
- # transaction = sample["transaction"]
- # cpumode = get_optional_zero(sample, "cpumode")
- print(common_start_str(comm, sample) + "%8s %21s" % (name, flags_disp), end=' ')
-
-def print_instructions_start(comm, sample):
- if "x" in get_optional_null(sample, "flags"):
- print(common_start_str(comm, sample) + "x", end=' ')
- else:
- print(common_start_str(comm, sample), end=' ')
-
-def disassem(insn, ip):
- inst = glb_disassembler.Instruction()
- glb_disassembler.SetMode(inst, 0) # Assume 64-bit
- buf = create_string_buffer(64)
- buf.value = insn
- return glb_disassembler.DisassembleOne(inst, addressof(buf), len(insn), ip)
-
-def print_common_ip(param_dict, sample, symbol, dso):
- ip = sample["ip"]
- offs = get_offset(param_dict, "symoff")
- if "cyc_cnt" in sample:
- cyc_cnt = sample["cyc_cnt"]
- insn_cnt = get_optional_zero(sample, "insn_cnt")
- ipc_str = " IPC: %#.2f (%u/%u)" % (insn_cnt / cyc_cnt, insn_cnt, cyc_cnt)
- else:
- ipc_str = ""
- if glb_insn and glb_disassembler is not None:
- insn = perf_sample_insn(perf_script_context)
- if insn and len(insn):
- cnt, text = disassem(insn, ip)
- byte_str = ("%x" % ip).rjust(16)
- if sys.version_info.major >= 3:
- for k in range(cnt):
- byte_str += " %02x" % insn[k]
- else:
- for k in xrange(cnt):
- byte_str += " %02x" % ord(insn[k])
- print("%-40s %-30s" % (byte_str, text), end=' ')
- print("%s%s (%s)" % (symbol, offs, dso), end=' ')
- else:
- print("%16x %s%s (%s)" % (ip, symbol, offs, dso), end=' ')
- if "addr_correlates_sym" in sample:
- addr = sample["addr"]
- dso = get_optional(sample, "addr_dso")
- symbol = get_optional(sample, "addr_symbol")
- offs = get_offset(sample, "addr_symoff")
- print("=> %x %s%s (%s)%s" % (addr, symbol, offs, dso, ipc_str))
- else:
- print(ipc_str)
-
-def print_srccode(comm, param_dict, sample, symbol, dso, with_insn):
- ip = sample["ip"]
- if symbol == "[unknown]":
- start_str = common_start_str(comm, sample) + ("%x" % ip).rjust(16).ljust(40)
- else:
- offs = get_offset(param_dict, "symoff")
- start_str = common_start_str(comm, sample) + (symbol + offs).ljust(40)
-
- if with_insn and glb_insn and glb_disassembler is not None:
- insn = perf_sample_insn(perf_script_context)
- if insn and len(insn):
- cnt, text = disassem(insn, ip)
- start_str += text.ljust(30)
-
- global glb_source_file_name
- global glb_line_number
- global glb_dso
-
- source_file_name, line_number, source_line = perf_sample_srccode(perf_script_context)
- if source_file_name:
- if glb_line_number == line_number and glb_source_file_name == source_file_name:
- src_str = ""
- else:
- if len(source_file_name) > 40:
- src_file = ("..." + source_file_name[-37:]) + " "
- else:
- src_file = source_file_name.ljust(41)
- if source_line is None:
- src_str = src_file + str(line_number).rjust(4) + " <source not found>"
- else:
- src_str = src_file + str(line_number).rjust(4) + " " + source_line
- glb_dso = None
- elif dso == glb_dso:
- src_str = ""
- else:
- src_str = dso
- glb_dso = dso
-
- glb_line_number = line_number
- glb_source_file_name = source_file_name
-
- print(start_str, src_str)
-
-def do_process_event(param_dict):
- sample = param_dict["sample"]
- raw_buf = param_dict["raw_buf"]
- comm = param_dict["comm"]
- name = param_dict["ev_name"]
- # Unused fields:
- # callchain = param_dict["callchain"]
- # brstack = param_dict["brstack"]
- # brstacksym = param_dict["brstacksym"]
- # event_attr = param_dict["attr"]
-
- # Symbol and dso info are not always resolved
- dso = get_optional(param_dict, "dso")
- symbol = get_optional(param_dict, "symbol")
-
- cpu = sample["cpu"]
- if cpu in glb_switch_str:
- print(glb_switch_str[cpu])
- del glb_switch_str[cpu]
-
- if name.startswith("instructions"):
- if glb_src:
- print_srccode(comm, param_dict, sample, symbol, dso, True)
- else:
- print_instructions_start(comm, sample)
- print_common_ip(param_dict, sample, symbol, dso)
- elif name.startswith("branches"):
- if glb_src:
- print_srccode(comm, param_dict, sample, symbol, dso, False)
- else:
- print_common_start(comm, sample, name)
- print_common_ip(param_dict, sample, symbol, dso)
- elif name == "ptwrite":
- print_common_start(comm, sample, name)
- print_ptwrite(raw_buf)
- print_common_ip(param_dict, sample, symbol, dso)
- elif name == "cbr":
- print_common_start(comm, sample, name)
- print_cbr(raw_buf)
- print_common_ip(param_dict, sample, symbol, dso)
- elif name == "mwait":
- print_common_start(comm, sample, name)
- print_mwait(raw_buf)
- print_common_ip(param_dict, sample, symbol, dso)
- elif name == "pwre":
- print_common_start(comm, sample, name)
- print_pwre(raw_buf)
- print_common_ip(param_dict, sample, symbol, dso)
- elif name == "exstop":
- print_common_start(comm, sample, name)
- print_exstop(raw_buf)
- print_common_ip(param_dict, sample, symbol, dso)
- elif name == "pwrx":
- print_common_start(comm, sample, name)
- print_pwrx(raw_buf)
- print_common_ip(param_dict, sample, symbol, dso)
- elif name == "psb":
- print_common_start(comm, sample, name)
- print_psb(raw_buf)
- print_common_ip(param_dict, sample, symbol, dso)
- elif name == "evt":
- print_common_start(comm, sample, name)
- print_evt(raw_buf)
- print_common_ip(param_dict, sample, symbol, dso)
- elif name == "iflag":
- print_common_start(comm, sample, name)
- print_iflag(raw_buf)
- print_common_ip(param_dict, sample, symbol, dso)
- else:
- print_common_start(comm, sample, name)
- print_common_ip(param_dict, sample, symbol, dso)
-
-def interleave_events(param_dict):
- global glb_cpu
- global glb_time
- global glb_output
- global glb_output_pos
-
- sample = param_dict["sample"]
- glb_cpu = sample["cpu"]
- ts = sample["time"]
-
- if glb_time != ts:
- glb_time = ts
- flush_stashed_output()
-
- glb_output_pos = 0
- with contextlib.redirect_stdout(io.StringIO()) as glb_output:
- do_process_event(param_dict)
-
- stash_output()
-
-def process_event(param_dict):
- try:
- if glb_args.interleave:
- interleave_events(param_dict)
- else:
- do_process_event(param_dict)
- except broken_pipe_exception:
- # Stop python printing broken pipe errors and traceback
- sys.stdout = open(os.devnull, 'w')
- sys.exit(1)
-
-def auxtrace_error(typ, code, cpu, pid, tid, ip, ts, msg, cpumode, *x):
- if glb_args.interleave:
- flush_stashed_output()
- if len(x) >= 2 and x[0]:
- machine_pid = x[0]
- vcpu = x[1]
- else:
- machine_pid = 0
- vcpu = -1
- try:
- if machine_pid:
- print("VM:%5d VCPU:%03d %16s %5u/%-5u [%03u] %9u.%09u error type %u code %u: %s ip 0x%16x" %
- (machine_pid, vcpu, "Trace error", pid, tid, cpu, ts / 1000000000, ts %1000000000, typ, code, msg, ip))
- else:
- print("%16s %5u/%-5u [%03u] %9u.%09u error type %u code %u: %s ip 0x%16x" %
- ("Trace error", pid, tid, cpu, ts / 1000000000, ts %1000000000, typ, code, msg, ip))
- except broken_pipe_exception:
- # Stop python printing broken pipe errors and traceback
- sys.stdout = open(os.devnull, 'w')
- sys.exit(1)
-
-def context_switch(ts, cpu, pid, tid, np_pid, np_tid, machine_pid, out, out_preempt, *x):
- if glb_args.interleave:
- flush_stashed_output()
- if out:
- out_str = "Switch out "
- else:
- out_str = "Switch In "
- if out_preempt:
- preempt_str = "preempt"
- else:
- preempt_str = ""
- if len(x) >= 2 and x[0]:
- machine_pid = x[0]
- vcpu = x[1]
- else:
- vcpu = None;
- if machine_pid == -1:
- machine_str = ""
- elif vcpu is None:
- machine_str = "machine PID %d" % machine_pid
- else:
- machine_str = "machine PID %d VCPU %d" % (machine_pid, vcpu)
- switch_str = "%16s %5d/%-5d [%03u] %9u.%09u %5d/%-5d %s %s" % \
- (out_str, pid, tid, cpu, ts / 1000000000, ts %1000000000, np_pid, np_tid, machine_str, preempt_str)
- if glb_args.all_switch_events:
- print(switch_str)
- else:
- global glb_switch_str
- glb_switch_str[cpu] = switch_str
diff --git a/tools/perf/scripts/python/libxed.py b/tools/perf/scripts/python/libxed.py
deleted file mode 100644
index 2c70a5a7eb9c..000000000000
--- a/tools/perf/scripts/python/libxed.py
+++ /dev/null
@@ -1,107 +0,0 @@
-#!/usr/bin/env python
-# SPDX-License-Identifier: GPL-2.0
-# libxed.py: Python wrapper for libxed.so
-# Copyright (c) 2014-2021, Intel Corporation.
-
-# To use Intel XED, libxed.so must be present. To build and install
-# libxed.so:
-# git clone https://github.com/intelxed/mbuild.git mbuild
-# git clone https://github.com/intelxed/xed
-# cd xed
-# ./mfile.py --share
-# sudo ./mfile.py --prefix=/usr/local install
-# sudo ldconfig
-#
-
-import sys
-
-from ctypes import CDLL, Structure, create_string_buffer, addressof, sizeof, \
- c_void_p, c_bool, c_byte, c_char, c_int, c_uint, c_longlong, c_ulonglong
-
-# XED Disassembler
-
-class xed_state_t(Structure):
-
- _fields_ = [
- ("mode", c_int),
- ("width", c_int)
- ]
-
-class XEDInstruction():
-
- def __init__(self, libxed):
- # Current xed_decoded_inst_t structure is 192 bytes. Use 512 to allow for future expansion
- xedd_t = c_byte * 512
- self.xedd = xedd_t()
- self.xedp = addressof(self.xedd)
- libxed.xed_decoded_inst_zero(self.xedp)
- self.state = xed_state_t()
- self.statep = addressof(self.state)
- # Buffer for disassembled instruction text
- self.buffer = create_string_buffer(256)
- self.bufferp = addressof(self.buffer)
-
-class LibXED():
-
- def __init__(self):
- try:
- self.libxed = CDLL("libxed.so")
- except:
- self.libxed = None
- if not self.libxed:
- self.libxed = CDLL("/usr/local/lib/libxed.so")
-
- self.xed_tables_init = self.libxed.xed_tables_init
- self.xed_tables_init.restype = None
- self.xed_tables_init.argtypes = []
-
- self.xed_decoded_inst_zero = self.libxed.xed_decoded_inst_zero
- self.xed_decoded_inst_zero.restype = None
- self.xed_decoded_inst_zero.argtypes = [ c_void_p ]
-
- self.xed_operand_values_set_mode = self.libxed.xed_operand_values_set_mode
- self.xed_operand_values_set_mode.restype = None
- self.xed_operand_values_set_mode.argtypes = [ c_void_p, c_void_p ]
-
- self.xed_decoded_inst_zero_keep_mode = self.libxed.xed_decoded_inst_zero_keep_mode
- self.xed_decoded_inst_zero_keep_mode.restype = None
- self.xed_decoded_inst_zero_keep_mode.argtypes = [ c_void_p ]
-
- self.xed_decode = self.libxed.xed_decode
- self.xed_decode.restype = c_int
- self.xed_decode.argtypes = [ c_void_p, c_void_p, c_uint ]
-
- self.xed_format_context = self.libxed.xed_format_context
- self.xed_format_context.restype = c_uint
- self.xed_format_context.argtypes = [ c_int, c_void_p, c_void_p, c_int, c_ulonglong, c_void_p, c_void_p ]
-
- self.xed_tables_init()
-
- def Instruction(self):
- return XEDInstruction(self)
-
- def SetMode(self, inst, mode):
- if mode:
- inst.state.mode = 4 # 32-bit
- inst.state.width = 4 # 4 bytes
- else:
- inst.state.mode = 1 # 64-bit
- inst.state.width = 8 # 8 bytes
- self.xed_operand_values_set_mode(inst.xedp, inst.statep)
-
- def DisassembleOne(self, inst, bytes_ptr, bytes_cnt, ip):
- self.xed_decoded_inst_zero_keep_mode(inst.xedp)
- err = self.xed_decode(inst.xedp, bytes_ptr, bytes_cnt)
- if err:
- return 0, ""
- # Use AT&T mode (2), alternative is Intel (3)
- ok = self.xed_format_context(2, inst.xedp, inst.bufferp, sizeof(inst.buffer), ip, 0, 0)
- if not ok:
- return 0, ""
- if sys.version_info[0] == 2:
- result = inst.buffer.value
- else:
- result = inst.buffer.value.decode()
- # Return instruction length and the disassembled instruction text
- # For now, assume the length is in byte 166
- return inst.xedd[166], result
diff --git a/tools/perf/scripts/python/mem-phys-addr.py b/tools/perf/scripts/python/mem-phys-addr.py
deleted file mode 100644
index 5e237a5a5f1b..000000000000
--- a/tools/perf/scripts/python/mem-phys-addr.py
+++ /dev/null
@@ -1,127 +0,0 @@
-# mem-phys-addr.py: Resolve physical address samples
-# SPDX-License-Identifier: GPL-2.0
-#
-# Copyright (c) 2018, Intel Corporation.
-
-import os
-import sys
-import re
-import bisect
-import collections
-from dataclasses import dataclass
-from typing import (Dict, Optional)
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-@dataclass(frozen=True)
-class IomemEntry:
- """Read from a line in /proc/iomem"""
- begin: int
- end: int
- indent: int
- label: str
-
-# Physical memory layout from /proc/iomem. Key is the indent and then
-# a list of ranges.
-iomem: Dict[int, list[IomemEntry]] = collections.defaultdict(list)
-# Child nodes from the iomem parent.
-children: Dict[IomemEntry, set[IomemEntry]] = collections.defaultdict(set)
-# Maximum indent seen before an entry in the iomem file.
-max_indent: int = 0
-# Count for each range of memory.
-load_mem_type_cnt: Dict[IomemEntry, int] = collections.Counter()
-# Perf event name set from the first sample in the data.
-event_name: Optional[str] = None
-
-def parse_iomem():
- """Populate iomem from /proc/iomem file"""
- global iomem
- global max_indent
- global children
- with open('/proc/iomem', 'r', encoding='ascii') as f:
- for line in f:
- indent = 0
- while line[indent] == ' ':
- indent += 1
- if indent > max_indent:
- max_indent = indent
- m = re.split('-|:', line, 2)
- begin = int(m[0], 16)
- end = int(m[1], 16)
- label = m[2].strip()
- entry = IomemEntry(begin, end, indent, label)
- # Before adding entry, search for a parent node using its begin.
- if indent > 0:
- parent = find_memory_type(begin)
- assert parent, f"Given indent expected a parent for {label}"
- children[parent].add(entry)
- iomem[indent].append(entry)
-
-def find_memory_type(phys_addr) -> Optional[IomemEntry]:
- """Search iomem for the range containing phys_addr with the maximum indent"""
- for i in range(max_indent, -1, -1):
- if i not in iomem:
- continue
- position = bisect.bisect_right(iomem[i], phys_addr,
- key=lambda entry: entry.begin)
- if position is None:
- continue
- iomem_entry = iomem[i][position-1]
- if iomem_entry.begin <= phys_addr <= iomem_entry.end:
- return iomem_entry
- print(f"Didn't find {phys_addr}")
- return None
-
-def print_memory_type():
- print(f"Event: {event_name}")
- print(f"{'Memory type':<40} {'count':>10} {'percentage':>10}")
- print(f"{'-' * 40:<40} {'-' * 10:>10} {'-' * 10:>10}")
- total = sum(load_mem_type_cnt.values())
- # Add count from children into the parent.
- for i in range(max_indent, -1, -1):
- if i not in iomem:
- continue
- for entry in iomem[i]:
- global children
- for child in children[entry]:
- if load_mem_type_cnt[child] > 0:
- load_mem_type_cnt[entry] += load_mem_type_cnt[child]
-
- def print_entries(entries):
- """Print counts from parents down to their children"""
- global children
- for entry in sorted(entries,
- key = lambda entry: load_mem_type_cnt[entry],
- reverse = True):
- count = load_mem_type_cnt[entry]
- if count > 0:
- mem_type = ' ' * entry.indent + f"{entry.begin:x}-{entry.end:x} : {entry.label}"
- percent = 100 * count / total
- print(f"{mem_type:<40} {count:>10} {percent:>10.1f}")
- print_entries(children[entry])
-
- print_entries(iomem[0])
-
-def trace_begin():
- parse_iomem()
-
-def trace_end():
- print_memory_type()
-
-def process_event(param_dict):
- if "sample" not in param_dict:
- return
-
- sample = param_dict["sample"]
- if "phys_addr" not in sample:
- return
-
- phys_addr = sample["phys_addr"]
- entry = find_memory_type(phys_addr)
- if entry:
- load_mem_type_cnt[entry] += 1
-
- global event_name
- if event_name is None:
- event_name = param_dict["ev_name"]
diff --git a/tools/perf/scripts/python/net_dropmonitor.py b/tools/perf/scripts/python/net_dropmonitor.py
deleted file mode 100755
index a97e7a6e0940..000000000000
--- a/tools/perf/scripts/python/net_dropmonitor.py
+++ /dev/null
@@ -1,78 +0,0 @@
-# Monitor the system for dropped packets and proudce a report of drop locations and counts
-# SPDX-License-Identifier: GPL-2.0
-
-from __future__ import print_function
-
-import os
-import sys
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import *
-from Core import *
-from Util import *
-
-drop_log = {}
-kallsyms = []
-
-def get_kallsyms_table():
- global kallsyms
-
- try:
- f = open("/proc/kallsyms", "r")
- except:
- return
-
- for line in f:
- loc = int(line.split()[0], 16)
- name = line.split()[2]
- kallsyms.append((loc, name))
- kallsyms.sort()
-
-def get_sym(sloc):
- loc = int(sloc)
-
- # Invariant: kallsyms[i][0] <= loc for all 0 <= i <= start
- # kallsyms[i][0] > loc for all end <= i < len(kallsyms)
- start, end = -1, len(kallsyms)
- while end != start + 1:
- pivot = (start + end) // 2
- if loc < kallsyms[pivot][0]:
- end = pivot
- else:
- start = pivot
-
- # Now (start == -1 or kallsyms[start][0] <= loc)
- # and (start == len(kallsyms) - 1 or loc < kallsyms[start + 1][0])
- if start >= 0:
- symloc, name = kallsyms[start]
- return (name, loc - symloc)
- else:
- return (None, 0)
-
-def print_drop_table():
- print("%25s %25s %25s" % ("LOCATION", "OFFSET", "COUNT"))
- for i in drop_log.keys():
- (sym, off) = get_sym(i)
- if sym == None:
- sym = i
- print("%25s %25s %25s" % (sym, off, drop_log[i]))
-
-
-def trace_begin():
- print("Starting trace (Ctrl-C to dump results)")
-
-def trace_end():
- print("Gathering kallsyms data")
- get_kallsyms_table()
- print_drop_table()
-
-# called from perf, when it finds a corresponding event
-def skb__kfree_skb(name, context, cpu, sec, nsec, pid, comm, callchain,
- skbaddr, location, protocol, reason):
- slocation = str(location)
- try:
- drop_log[slocation] = drop_log[slocation] + 1
- except:
- drop_log[slocation] = 1
diff --git a/tools/perf/scripts/python/netdev-times.py b/tools/perf/scripts/python/netdev-times.py
deleted file mode 100644
index 30c4bccee5b2..000000000000
--- a/tools/perf/scripts/python/netdev-times.py
+++ /dev/null
@@ -1,473 +0,0 @@
-# Display a process of packets and processed time.
-# SPDX-License-Identifier: GPL-2.0
-# It helps us to investigate networking or network device.
-#
-# options
-# tx: show only tx chart
-# rx: show only rx chart
-# dev=: show only thing related to specified device
-# debug: work with debug mode. It shows buffer status.
-
-from __future__ import print_function
-
-import os
-import sys
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import *
-from Core import *
-from Util import *
-from functools import cmp_to_key
-
-all_event_list = []; # insert all tracepoint event related with this script
-irq_dic = {}; # key is cpu and value is a list which stacks irqs
- # which raise NET_RX softirq
-net_rx_dic = {}; # key is cpu and value include time of NET_RX softirq-entry
- # and a list which stacks receive
-receive_hunk_list = []; # a list which include a sequence of receive events
-rx_skb_list = []; # received packet list for matching
- # skb_copy_datagram_iovec
-
-buffer_budget = 65536; # the budget of rx_skb_list, tx_queue_list and
- # tx_xmit_list
-of_count_rx_skb_list = 0; # overflow count
-
-tx_queue_list = []; # list of packets which pass through dev_queue_xmit
-of_count_tx_queue_list = 0; # overflow count
-
-tx_xmit_list = []; # list of packets which pass through dev_hard_start_xmit
-of_count_tx_xmit_list = 0; # overflow count
-
-tx_free_list = []; # list of packets which is freed
-
-# options
-show_tx = 0;
-show_rx = 0;
-dev = 0; # store a name of device specified by option "dev="
-debug = 0;
-
-# indices of event_info tuple
-EINFO_IDX_NAME= 0
-EINFO_IDX_CONTEXT=1
-EINFO_IDX_CPU= 2
-EINFO_IDX_TIME= 3
-EINFO_IDX_PID= 4
-EINFO_IDX_COMM= 5
-
-# Calculate a time interval(msec) from src(nsec) to dst(nsec)
-def diff_msec(src, dst):
- return (dst - src) / 1000000.0
-
-# Display a process of transmitting a packet
-def print_transmit(hunk):
- if dev != 0 and hunk['dev'].find(dev) < 0:
- return
- print("%7s %5d %6d.%06dsec %12.3fmsec %12.3fmsec" %
- (hunk['dev'], hunk['len'],
- nsecs_secs(hunk['queue_t']),
- nsecs_nsecs(hunk['queue_t'])/1000,
- diff_msec(hunk['queue_t'], hunk['xmit_t']),
- diff_msec(hunk['xmit_t'], hunk['free_t'])))
-
-# Format for displaying rx packet processing
-PF_IRQ_ENTRY= " irq_entry(+%.3fmsec irq=%d:%s)"
-PF_SOFT_ENTRY=" softirq_entry(+%.3fmsec)"
-PF_NAPI_POLL= " napi_poll_exit(+%.3fmsec %s)"
-PF_JOINT= " |"
-PF_WJOINT= " | |"
-PF_NET_RECV= " |---netif_receive_skb(+%.3fmsec skb=%x len=%d)"
-PF_NET_RX= " |---netif_rx(+%.3fmsec skb=%x)"
-PF_CPY_DGRAM= " | skb_copy_datagram_iovec(+%.3fmsec %d:%s)"
-PF_KFREE_SKB= " | kfree_skb(+%.3fmsec location=%x)"
-PF_CONS_SKB= " | consume_skb(+%.3fmsec)"
-
-# Display a process of received packets and interrputs associated with
-# a NET_RX softirq
-def print_receive(hunk):
- show_hunk = 0
- irq_list = hunk['irq_list']
- cpu = irq_list[0]['cpu']
- base_t = irq_list[0]['irq_ent_t']
- # check if this hunk should be showed
- if dev != 0:
- for i in range(len(irq_list)):
- if irq_list[i]['name'].find(dev) >= 0:
- show_hunk = 1
- break
- else:
- show_hunk = 1
- if show_hunk == 0:
- return
-
- print("%d.%06dsec cpu=%d" %
- (nsecs_secs(base_t), nsecs_nsecs(base_t)/1000, cpu))
- for i in range(len(irq_list)):
- print(PF_IRQ_ENTRY %
- (diff_msec(base_t, irq_list[i]['irq_ent_t']),
- irq_list[i]['irq'], irq_list[i]['name']))
- print(PF_JOINT)
- irq_event_list = irq_list[i]['event_list']
- for j in range(len(irq_event_list)):
- irq_event = irq_event_list[j]
- if irq_event['event'] == 'netif_rx':
- print(PF_NET_RX %
- (diff_msec(base_t, irq_event['time']),
- irq_event['skbaddr']))
- print(PF_JOINT)
- print(PF_SOFT_ENTRY %
- diff_msec(base_t, hunk['sirq_ent_t']))
- print(PF_JOINT)
- event_list = hunk['event_list']
- for i in range(len(event_list)):
- event = event_list[i]
- if event['event_name'] == 'napi_poll':
- print(PF_NAPI_POLL %
- (diff_msec(base_t, event['event_t']),
- event['dev']))
- if i == len(event_list) - 1:
- print("")
- else:
- print(PF_JOINT)
- else:
- print(PF_NET_RECV %
- (diff_msec(base_t, event['event_t']),
- event['skbaddr'],
- event['len']))
- if 'comm' in event.keys():
- print(PF_WJOINT)
- print(PF_CPY_DGRAM %
- (diff_msec(base_t, event['comm_t']),
- event['pid'], event['comm']))
- elif 'handle' in event.keys():
- print(PF_WJOINT)
- if event['handle'] == "kfree_skb":
- print(PF_KFREE_SKB %
- (diff_msec(base_t,
- event['comm_t']),
- event['location']))
- elif event['handle'] == "consume_skb":
- print(PF_CONS_SKB %
- diff_msec(base_t,
- event['comm_t']))
- print(PF_JOINT)
-
-def trace_begin():
- global show_tx
- global show_rx
- global dev
- global debug
-
- for i in range(len(sys.argv)):
- if i == 0:
- continue
- arg = sys.argv[i]
- if arg == 'tx':
- show_tx = 1
- elif arg =='rx':
- show_rx = 1
- elif arg.find('dev=',0, 4) >= 0:
- dev = arg[4:]
- elif arg == 'debug':
- debug = 1
- if show_tx == 0 and show_rx == 0:
- show_tx = 1
- show_rx = 1
-
-def trace_end():
- # order all events in time
- all_event_list.sort(key=cmp_to_key(lambda a,b :a[EINFO_IDX_TIME] < b[EINFO_IDX_TIME]))
- # process all events
- for i in range(len(all_event_list)):
- event_info = all_event_list[i]
- name = event_info[EINFO_IDX_NAME]
- if name == 'irq__softirq_exit':
- handle_irq_softirq_exit(event_info)
- elif name == 'irq__softirq_entry':
- handle_irq_softirq_entry(event_info)
- elif name == 'irq__softirq_raise':
- handle_irq_softirq_raise(event_info)
- elif name == 'irq__irq_handler_entry':
- handle_irq_handler_entry(event_info)
- elif name == 'irq__irq_handler_exit':
- handle_irq_handler_exit(event_info)
- elif name == 'napi__napi_poll':
- handle_napi_poll(event_info)
- elif name == 'net__netif_receive_skb':
- handle_netif_receive_skb(event_info)
- elif name == 'net__netif_rx':
- handle_netif_rx(event_info)
- elif name == 'skb__skb_copy_datagram_iovec':
- handle_skb_copy_datagram_iovec(event_info)
- elif name == 'net__net_dev_queue':
- handle_net_dev_queue(event_info)
- elif name == 'net__net_dev_xmit':
- handle_net_dev_xmit(event_info)
- elif name == 'skb__kfree_skb':
- handle_kfree_skb(event_info)
- elif name == 'skb__consume_skb':
- handle_consume_skb(event_info)
- # display receive hunks
- if show_rx:
- for i in range(len(receive_hunk_list)):
- print_receive(receive_hunk_list[i])
- # display transmit hunks
- if show_tx:
- print(" dev len Qdisc "
- " netdevice free")
- for i in range(len(tx_free_list)):
- print_transmit(tx_free_list[i])
- if debug:
- print("debug buffer status")
- print("----------------------------")
- print("xmit Qdisc:remain:%d overflow:%d" %
- (len(tx_queue_list), of_count_tx_queue_list))
- print("xmit netdevice:remain:%d overflow:%d" %
- (len(tx_xmit_list), of_count_tx_xmit_list))
- print("receive:remain:%d overflow:%d" %
- (len(rx_skb_list), of_count_rx_skb_list))
-
-# called from perf, when it finds a correspoinding event
-def irq__softirq_entry(name, context, cpu, sec, nsec, pid, comm, callchain, vec):
- if symbol_str("irq__softirq_entry", "vec", vec) != "NET_RX":
- return
- event_info = (name, context, cpu, nsecs(sec, nsec), pid, comm, vec)
- all_event_list.append(event_info)
-
-def irq__softirq_exit(name, context, cpu, sec, nsec, pid, comm, callchain, vec):
- if symbol_str("irq__softirq_entry", "vec", vec) != "NET_RX":
- return
- event_info = (name, context, cpu, nsecs(sec, nsec), pid, comm, vec)
- all_event_list.append(event_info)
-
-def irq__softirq_raise(name, context, cpu, sec, nsec, pid, comm, callchain, vec):
- if symbol_str("irq__softirq_entry", "vec", vec) != "NET_RX":
- return
- event_info = (name, context, cpu, nsecs(sec, nsec), pid, comm, vec)
- all_event_list.append(event_info)
-
-def irq__irq_handler_entry(name, context, cpu, sec, nsec, pid, comm,
- callchain, irq, irq_name):
- event_info = (name, context, cpu, nsecs(sec, nsec), pid, comm,
- irq, irq_name)
- all_event_list.append(event_info)
-
-def irq__irq_handler_exit(name, context, cpu, sec, nsec, pid, comm, callchain, irq, ret):
- event_info = (name, context, cpu, nsecs(sec, nsec), pid, comm, irq, ret)
- all_event_list.append(event_info)
-
-def napi__napi_poll(name, context, cpu, sec, nsec, pid, comm, callchain, napi,
- dev_name, work=None, budget=None):
- event_info = (name, context, cpu, nsecs(sec, nsec), pid, comm,
- napi, dev_name, work, budget)
- all_event_list.append(event_info)
-
-def net__netif_receive_skb(name, context, cpu, sec, nsec, pid, comm, callchain, skbaddr,
- skblen, dev_name):
- event_info = (name, context, cpu, nsecs(sec, nsec), pid, comm,
- skbaddr, skblen, dev_name)
- all_event_list.append(event_info)
-
-def net__netif_rx(name, context, cpu, sec, nsec, pid, comm, callchain, skbaddr,
- skblen, dev_name):
- event_info = (name, context, cpu, nsecs(sec, nsec), pid, comm,
- skbaddr, skblen, dev_name)
- all_event_list.append(event_info)
-
-def net__net_dev_queue(name, context, cpu, sec, nsec, pid, comm, callchain,
- skbaddr, skblen, dev_name):
- event_info = (name, context, cpu, nsecs(sec, nsec), pid, comm,
- skbaddr, skblen, dev_name)
- all_event_list.append(event_info)
-
-def net__net_dev_xmit(name, context, cpu, sec, nsec, pid, comm, callchain,
- skbaddr, skblen, rc, dev_name):
- event_info = (name, context, cpu, nsecs(sec, nsec), pid, comm,
- skbaddr, skblen, rc ,dev_name)
- all_event_list.append(event_info)
-
-def skb__kfree_skb(name, context, cpu, sec, nsec, pid, comm, callchain,
- skbaddr, location, protocol, reason):
- event_info = (name, context, cpu, nsecs(sec, nsec), pid, comm,
- skbaddr, location, protocol, reason)
- all_event_list.append(event_info)
-
-def skb__consume_skb(name, context, cpu, sec, nsec, pid, comm, callchain,
- skbaddr, location):
- event_info = (name, context, cpu, nsecs(sec, nsec), pid, comm,
- skbaddr)
- all_event_list.append(event_info)
-
-def skb__skb_copy_datagram_iovec(name, context, cpu, sec, nsec, pid, comm, callchain,
- skbaddr, skblen):
- event_info = (name, context, cpu, nsecs(sec, nsec), pid, comm,
- skbaddr, skblen)
- all_event_list.append(event_info)
-
-def handle_irq_handler_entry(event_info):
- (name, context, cpu, time, pid, comm, irq, irq_name) = event_info
- if cpu not in irq_dic.keys():
- irq_dic[cpu] = []
- irq_record = {'irq':irq, 'name':irq_name, 'cpu':cpu, 'irq_ent_t':time}
- irq_dic[cpu].append(irq_record)
-
-def handle_irq_handler_exit(event_info):
- (name, context, cpu, time, pid, comm, irq, ret) = event_info
- if cpu not in irq_dic.keys():
- return
- irq_record = irq_dic[cpu].pop()
- if irq != irq_record['irq']:
- return
- irq_record.update({'irq_ext_t':time})
- # if an irq doesn't include NET_RX softirq, drop.
- if 'event_list' in irq_record.keys():
- irq_dic[cpu].append(irq_record)
-
-def handle_irq_softirq_raise(event_info):
- (name, context, cpu, time, pid, comm, vec) = event_info
- if cpu not in irq_dic.keys() \
- or len(irq_dic[cpu]) == 0:
- return
- irq_record = irq_dic[cpu].pop()
- if 'event_list' in irq_record.keys():
- irq_event_list = irq_record['event_list']
- else:
- irq_event_list = []
- irq_event_list.append({'time':time, 'event':'sirq_raise'})
- irq_record.update({'event_list':irq_event_list})
- irq_dic[cpu].append(irq_record)
-
-def handle_irq_softirq_entry(event_info):
- (name, context, cpu, time, pid, comm, vec) = event_info
- net_rx_dic[cpu] = {'sirq_ent_t':time, 'event_list':[]}
-
-def handle_irq_softirq_exit(event_info):
- (name, context, cpu, time, pid, comm, vec) = event_info
- irq_list = []
- event_list = 0
- if cpu in irq_dic.keys():
- irq_list = irq_dic[cpu]
- del irq_dic[cpu]
- if cpu in net_rx_dic.keys():
- sirq_ent_t = net_rx_dic[cpu]['sirq_ent_t']
- event_list = net_rx_dic[cpu]['event_list']
- del net_rx_dic[cpu]
- if irq_list == [] or event_list == 0:
- return
- rec_data = {'sirq_ent_t':sirq_ent_t, 'sirq_ext_t':time,
- 'irq_list':irq_list, 'event_list':event_list}
- # merge information related to a NET_RX softirq
- receive_hunk_list.append(rec_data)
-
-def handle_napi_poll(event_info):
- (name, context, cpu, time, pid, comm, napi, dev_name,
- work, budget) = event_info
- if cpu in net_rx_dic.keys():
- event_list = net_rx_dic[cpu]['event_list']
- rec_data = {'event_name':'napi_poll',
- 'dev':dev_name, 'event_t':time,
- 'work':work, 'budget':budget}
- event_list.append(rec_data)
-
-def handle_netif_rx(event_info):
- (name, context, cpu, time, pid, comm,
- skbaddr, skblen, dev_name) = event_info
- if cpu not in irq_dic.keys() \
- or len(irq_dic[cpu]) == 0:
- return
- irq_record = irq_dic[cpu].pop()
- if 'event_list' in irq_record.keys():
- irq_event_list = irq_record['event_list']
- else:
- irq_event_list = []
- irq_event_list.append({'time':time, 'event':'netif_rx',
- 'skbaddr':skbaddr, 'skblen':skblen, 'dev_name':dev_name})
- irq_record.update({'event_list':irq_event_list})
- irq_dic[cpu].append(irq_record)
-
-def handle_netif_receive_skb(event_info):
- global of_count_rx_skb_list
-
- (name, context, cpu, time, pid, comm,
- skbaddr, skblen, dev_name) = event_info
- if cpu in net_rx_dic.keys():
- rec_data = {'event_name':'netif_receive_skb',
- 'event_t':time, 'skbaddr':skbaddr, 'len':skblen}
- event_list = net_rx_dic[cpu]['event_list']
- event_list.append(rec_data)
- rx_skb_list.insert(0, rec_data)
- if len(rx_skb_list) > buffer_budget:
- rx_skb_list.pop()
- of_count_rx_skb_list += 1
-
-def handle_net_dev_queue(event_info):
- global of_count_tx_queue_list
-
- (name, context, cpu, time, pid, comm,
- skbaddr, skblen, dev_name) = event_info
- skb = {'dev':dev_name, 'skbaddr':skbaddr, 'len':skblen, 'queue_t':time}
- tx_queue_list.insert(0, skb)
- if len(tx_queue_list) > buffer_budget:
- tx_queue_list.pop()
- of_count_tx_queue_list += 1
-
-def handle_net_dev_xmit(event_info):
- global of_count_tx_xmit_list
-
- (name, context, cpu, time, pid, comm,
- skbaddr, skblen, rc, dev_name) = event_info
- if rc == 0: # NETDEV_TX_OK
- for i in range(len(tx_queue_list)):
- skb = tx_queue_list[i]
- if skb['skbaddr'] == skbaddr:
- skb['xmit_t'] = time
- tx_xmit_list.insert(0, skb)
- del tx_queue_list[i]
- if len(tx_xmit_list) > buffer_budget:
- tx_xmit_list.pop()
- of_count_tx_xmit_list += 1
- return
-
-def handle_kfree_skb(event_info):
- (name, context, cpu, time, pid, comm,
- skbaddr, location, protocol, reason) = event_info
- for i in range(len(tx_queue_list)):
- skb = tx_queue_list[i]
- if skb['skbaddr'] == skbaddr:
- del tx_queue_list[i]
- return
- for i in range(len(tx_xmit_list)):
- skb = tx_xmit_list[i]
- if skb['skbaddr'] == skbaddr:
- skb['free_t'] = time
- tx_free_list.append(skb)
- del tx_xmit_list[i]
- return
- for i in range(len(rx_skb_list)):
- rec_data = rx_skb_list[i]
- if rec_data['skbaddr'] == skbaddr:
- rec_data.update({'handle':"kfree_skb",
- 'comm':comm, 'pid':pid, 'comm_t':time})
- del rx_skb_list[i]
- return
-
-def handle_consume_skb(event_info):
- (name, context, cpu, time, pid, comm, skbaddr) = event_info
- for i in range(len(tx_xmit_list)):
- skb = tx_xmit_list[i]
- if skb['skbaddr'] == skbaddr:
- skb['free_t'] = time
- tx_free_list.append(skb)
- del tx_xmit_list[i]
- return
-
-def handle_skb_copy_datagram_iovec(event_info):
- (name, context, cpu, time, pid, comm, skbaddr, skblen) = event_info
- for i in range(len(rx_skb_list)):
- rec_data = rx_skb_list[i]
- if skbaddr == rec_data['skbaddr']:
- rec_data.update({'handle':"skb_copy_datagram_iovec",
- 'comm':comm, 'pid':pid, 'comm_t':time})
- del rx_skb_list[i]
- return
diff --git a/tools/perf/scripts/python/powerpc-hcalls.py b/tools/perf/scripts/python/powerpc-hcalls.py
deleted file mode 100644
index fedce1b68cad..000000000000
--- a/tools/perf/scripts/python/powerpc-hcalls.py
+++ /dev/null
@@ -1,335 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0+
-#
-# Copyright (C) 2018 Ravi Bangoria, IBM Corporation
-#
-# Hypervisor call statisics
-
-from __future__ import print_function
-
-import os
-import sys
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import *
-from Core import *
-from Util import *
-
-# output: {
-# opcode: {
-# 'min': minimum time nsec
-# 'max': maximum time nsec
-# 'time': average time nsec
-# 'cnt': counter
-# } ...
-# }
-output = {}
-sort_key = 'count'
-
-# d_enter: {
-# cpu: {
-# opcode: nsec
-# } ...
-# }
-d_enter = {}
-
-hcall_table = {
- 4: 'H_REMOVE',
- 8: 'H_ENTER',
- 12: 'H_READ',
- 16: 'H_CLEAR_MOD',
- 20: 'H_CLEAR_REF',
- 24: 'H_PROTECT',
- 28: 'H_GET_TCE',
- 32: 'H_PUT_TCE',
- 36: 'H_SET_SPRG0',
- 40: 'H_SET_DABR',
- 44: 'H_PAGE_INIT',
- 48: 'H_SET_ASR',
- 52: 'H_ASR_ON',
- 56: 'H_ASR_OFF',
- 60: 'H_LOGICAL_CI_LOAD',
- 64: 'H_LOGICAL_CI_STORE',
- 68: 'H_LOGICAL_CACHE_LOAD',
- 72: 'H_LOGICAL_CACHE_STORE',
- 76: 'H_LOGICAL_ICBI',
- 80: 'H_LOGICAL_DCBF',
- 84: 'H_GET_TERM_CHAR',
- 88: 'H_PUT_TERM_CHAR',
- 92: 'H_REAL_TO_LOGICAL',
- 96: 'H_HYPERVISOR_DATA',
- 100: 'H_EOI',
- 104: 'H_CPPR',
- 108: 'H_IPI',
- 112: 'H_IPOLL',
- 116: 'H_XIRR',
- 120: 'H_MIGRATE_DMA',
- 124: 'H_PERFMON',
- 220: 'H_REGISTER_VPA',
- 224: 'H_CEDE',
- 228: 'H_CONFER',
- 232: 'H_PROD',
- 236: 'H_GET_PPP',
- 240: 'H_SET_PPP',
- 244: 'H_PURR',
- 248: 'H_PIC',
- 252: 'H_REG_CRQ',
- 256: 'H_FREE_CRQ',
- 260: 'H_VIO_SIGNAL',
- 264: 'H_SEND_CRQ',
- 272: 'H_COPY_RDMA',
- 276: 'H_REGISTER_LOGICAL_LAN',
- 280: 'H_FREE_LOGICAL_LAN',
- 284: 'H_ADD_LOGICAL_LAN_BUFFER',
- 288: 'H_SEND_LOGICAL_LAN',
- 292: 'H_BULK_REMOVE',
- 304: 'H_MULTICAST_CTRL',
- 308: 'H_SET_XDABR',
- 312: 'H_STUFF_TCE',
- 316: 'H_PUT_TCE_INDIRECT',
- 332: 'H_CHANGE_LOGICAL_LAN_MAC',
- 336: 'H_VTERM_PARTNER_INFO',
- 340: 'H_REGISTER_VTERM',
- 344: 'H_FREE_VTERM',
- 348: 'H_RESET_EVENTS',
- 352: 'H_ALLOC_RESOURCE',
- 356: 'H_FREE_RESOURCE',
- 360: 'H_MODIFY_QP',
- 364: 'H_QUERY_QP',
- 368: 'H_REREGISTER_PMR',
- 372: 'H_REGISTER_SMR',
- 376: 'H_QUERY_MR',
- 380: 'H_QUERY_MW',
- 384: 'H_QUERY_HCA',
- 388: 'H_QUERY_PORT',
- 392: 'H_MODIFY_PORT',
- 396: 'H_DEFINE_AQP1',
- 400: 'H_GET_TRACE_BUFFER',
- 404: 'H_DEFINE_AQP0',
- 408: 'H_RESIZE_MR',
- 412: 'H_ATTACH_MCQP',
- 416: 'H_DETACH_MCQP',
- 420: 'H_CREATE_RPT',
- 424: 'H_REMOVE_RPT',
- 428: 'H_REGISTER_RPAGES',
- 432: 'H_DISABLE_AND_GET',
- 436: 'H_ERROR_DATA',
- 440: 'H_GET_HCA_INFO',
- 444: 'H_GET_PERF_COUNT',
- 448: 'H_MANAGE_TRACE',
- 456: 'H_GET_CPU_CHARACTERISTICS',
- 468: 'H_FREE_LOGICAL_LAN_BUFFER',
- 472: 'H_POLL_PENDING',
- 484: 'H_QUERY_INT_STATE',
- 580: 'H_ILLAN_ATTRIBUTES',
- 584: 'H_ADD_LOGICAL_LAN_BUFFERS',
- 592: 'H_MODIFY_HEA_QP',
- 596: 'H_QUERY_HEA_QP',
- 600: 'H_QUERY_HEA',
- 604: 'H_QUERY_HEA_PORT',
- 608: 'H_MODIFY_HEA_PORT',
- 612: 'H_REG_BCMC',
- 616: 'H_DEREG_BCMC',
- 620: 'H_REGISTER_HEA_RPAGES',
- 624: 'H_DISABLE_AND_GET_HEA',
- 628: 'H_GET_HEA_INFO',
- 632: 'H_ALLOC_HEA_RESOURCE',
- 644: 'H_ADD_CONN',
- 648: 'H_DEL_CONN',
- 664: 'H_JOIN',
- 672: 'H_VASI_SIGNAL',
- 676: 'H_VASI_STATE',
- 680: 'H_VIOCTL',
- 688: 'H_ENABLE_CRQ',
- 696: 'H_GET_EM_PARMS',
- 720: 'H_SET_MPP',
- 724: 'H_GET_MPP',
- 732: 'H_REG_SUB_CRQ',
- 736: 'H_FREE_SUB_CRQ',
- 740: 'H_SEND_SUB_CRQ',
- 744: 'H_SEND_SUB_CRQ_INDIRECT',
- 748: 'H_HOME_NODE_ASSOCIATIVITY',
- 756: 'H_BEST_ENERGY',
- 764: 'H_XIRR_X',
- 768: 'H_RANDOM',
- 772: 'H_COP',
- 788: 'H_GET_MPP_X',
- 796: 'H_SET_MODE',
- 808: 'H_BLOCK_REMOVE',
- 856: 'H_CLEAR_HPT',
- 864: 'H_REQUEST_VMC',
- 876: 'H_RESIZE_HPT_PREPARE',
- 880: 'H_RESIZE_HPT_COMMIT',
- 892: 'H_REGISTER_PROC_TBL',
- 896: 'H_SIGNAL_SYS_RESET',
- 904: 'H_ALLOCATE_VAS_WINDOW',
- 908: 'H_MODIFY_VAS_WINDOW',
- 912: 'H_DEALLOCATE_VAS_WINDOW',
- 916: 'H_QUERY_VAS_WINDOW',
- 920: 'H_QUERY_VAS_CAPABILITIES',
- 924: 'H_QUERY_NX_CAPABILITIES',
- 928: 'H_GET_NX_FAULT',
- 936: 'H_INT_GET_SOURCE_INFO',
- 940: 'H_INT_SET_SOURCE_CONFIG',
- 944: 'H_INT_GET_SOURCE_CONFIG',
- 948: 'H_INT_GET_QUEUE_INFO',
- 952: 'H_INT_SET_QUEUE_CONFIG',
- 956: 'H_INT_GET_QUEUE_CONFIG',
- 960: 'H_INT_SET_OS_REPORTING_LINE',
- 964: 'H_INT_GET_OS_REPORTING_LINE',
- 968: 'H_INT_ESB',
- 972: 'H_INT_SYNC',
- 976: 'H_INT_RESET',
- 996: 'H_SCM_READ_METADATA',
- 1000: 'H_SCM_WRITE_METADATA',
- 1004: 'H_SCM_BIND_MEM',
- 1008: 'H_SCM_UNBIND_MEM',
- 1012: 'H_SCM_QUERY_BLOCK_MEM_BINDING',
- 1016: 'H_SCM_QUERY_LOGICAL_MEM_BINDING',
- 1020: 'H_SCM_UNBIND_ALL',
- 1024: 'H_SCM_HEALTH',
- 1048: 'H_SCM_PERFORMANCE_STATS',
- 1052: 'H_PKS_GET_CONFIG',
- 1056: 'H_PKS_SET_PASSWORD',
- 1060: 'H_PKS_GEN_PASSWORD',
- 1068: 'H_PKS_WRITE_OBJECT',
- 1072: 'H_PKS_GEN_KEY',
- 1076: 'H_PKS_READ_OBJECT',
- 1080: 'H_PKS_REMOVE_OBJECT',
- 1084: 'H_PKS_CONFIRM_OBJECT_FLUSHED',
- 1096: 'H_RPT_INVALIDATE',
- 1100: 'H_SCM_FLUSH',
- 1104: 'H_GET_ENERGY_SCALE_INFO',
- 1108: 'H_PKS_SIGNED_UPDATE',
- 1112: 'H_HTM',
- 1116: 'H_WATCHDOG',
- # Platform specific hcalls used by KVM on PowerVM
- 1120: 'H_GUEST_GET_CAPABILITIES',
- 1124: 'H_GUEST_SET_CAPABILITIES',
- 1136: 'H_GUEST_CREATE',
- 1140: 'H_GUEST_CREATE_VCPU',
- 1144: 'H_GUEST_GET_STATE',
- 1148: 'H_GUEST_SET_STATE',
- 1152: 'H_GUEST_RUN_VCPU',
- 1156: 'H_GUEST_COPY_MEMORY',
- 1160: 'H_GUEST_DELETE',
- # Key wrapping hcalls
- 1168: 'H_PKS_WRAP_OBJECT',
- 1172: 'H_PKS_UNWRAP_OBJECT',
- # Platform-specific hcalls used by the Ultravisor
- 61184: 'H_SVM_PAGE_IN',
- 61188: 'H_SVM_PAGE_OUT',
- 61192: 'H_SVM_INIT_START',
- 61196: 'H_SVM_INIT_DONE',
- 61204: 'H_SVM_INIT_ABORT',
- # Platform specific hcalls used by KVM
- 61440: 'H_RTAS',
- # Platform specific hcalls used by QEMU/SLOF
- 61441: 'H_LOGICAL_MEMOP',
- 61442: 'H_CAS',
- 61443: 'H_UPDATE_DT',
- # Platform specific hcalls provided by PHYP
- 61560: 'H_GET_24X7_CATALOG_PAGE',
- 61564: 'H_GET_24X7_DATA',
- 61568: 'H_GET_PERF_COUNTER_INFO',
- # Platform-specific hcalls used for nested HV KVM
- 63488: 'H_SET_PARTITION_TABLE',
- 63492: 'H_ENTER_NESTED',
- 63496: 'H_TLB_INVALIDATE',
- 63500: 'H_COPY_TOFROM_GUEST',
-}
-
-def hcall_table_lookup(opcode):
- if (opcode in hcall_table):
- return hcall_table[opcode]
- else:
- return opcode
-
-print_ptrn = '%-28s%10s%10s%10s%10s'
-
-def sort_output(opcode):
- stats = output[opcode]
-
- if sort_key == 'min':
- return stats['min']
- if sort_key == 'max':
- return stats['max']
- if sort_key == 'avg':
- return stats['time'] // stats['cnt']
-
- return stats['cnt']
-
-def trace_begin():
- global sort_key
-
- valid_sort_keys = ['count', 'min', 'max', 'avg']
-
- i = 1
- while i < len(sys.argv):
- arg = sys.argv[i]
-
- if arg == '-s' or arg == '--sort':
- if i + 1 >= len(sys.argv):
- print("Error: -s/--sort requires a sort key argument")
- sys.exit(1)
- sort_key = sys.argv[i + 1]
- i += 2
- continue
-
- if arg.startswith('--sort='):
- sort_key = arg.split('=', 1)[1]
- i += 1
- continue
-
- i += 1
-
- if sort_key not in valid_sort_keys:
- print(f"Error: Invalid sort key '{sort_key}'. Valid options are: {', '.join(valid_sort_keys)}")
- sys.exit(1)
-
- print("SORT KEY =", sort_key)
-
-def trace_end():
- print(print_ptrn % ('hcall', 'count', 'min(ns)', 'max(ns)', 'avg(ns)'))
- print('-' * 68)
- for opcode in sorted(output, key = sort_output,
- reverse=True):
- h_name = hcall_table_lookup(opcode)
- time = output[opcode]['time']
- cnt = output[opcode]['cnt']
- min_t = output[opcode]['min']
- max_t = output[opcode]['max']
-
- print(print_ptrn % (h_name, cnt, min_t, max_t, time//cnt))
-
-def powerpc__hcall_exit(name, context, cpu, sec, nsec, pid, comm, callchain,
- opcode, retval):
- if (cpu in d_enter and opcode in d_enter[cpu]):
- diff = nsecs(sec, nsec) - d_enter[cpu][opcode]
-
- if (opcode in output):
- output[opcode]['time'] += diff
- output[opcode]['cnt'] += 1
- if (output[opcode]['min'] > diff):
- output[opcode]['min'] = diff
- if (output[opcode]['max'] < diff):
- output[opcode]['max'] = diff
- else:
- output[opcode] = {
- 'time': diff,
- 'cnt': 1,
- 'min': diff,
- 'max': diff,
- }
-
- del d_enter[cpu][opcode]
-# else:
-# print("Can't find matching hcall_enter event. Ignoring sample")
-
-def powerpc__hcall_entry(event_name, context, cpu, sec, nsec, pid, comm,
- callchain, opcode):
- if (cpu in d_enter):
- d_enter[cpu][opcode] = nsecs(sec, nsec)
- else:
- d_enter[cpu] = {opcode: nsecs(sec, nsec)}
diff --git a/tools/perf/scripts/python/sched-migration.py b/tools/perf/scripts/python/sched-migration.py
deleted file mode 100644
index 8196e3087c9e..000000000000
--- a/tools/perf/scripts/python/sched-migration.py
+++ /dev/null
@@ -1,462 +0,0 @@
-# Cpu task migration overview toy
-#
-# Copyright (C) 2010 Frederic Weisbecker <fweisbec@gmail.com>
-#
-# perf script event handlers have been generated by perf script -g python
-#
-# This software is distributed under the terms of the GNU General
-# Public License ("GPL") version 2 as published by the Free Software
-# Foundation.
-from __future__ import print_function
-
-import os
-import sys
-
-from collections import defaultdict
-try:
- from UserList import UserList
-except ImportError:
- # Python 3: UserList moved to the collections package
- from collections import UserList
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-sys.path.append('scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import *
-from Core import *
-from SchedGui import *
-
-
-threads = { 0 : "idle"}
-
-def thread_name(pid):
- return "%s:%d" % (threads[pid], pid)
-
-class RunqueueEventUnknown:
- @staticmethod
- def color():
- return None
-
- def __repr__(self):
- return "unknown"
-
-class RunqueueEventSleep:
- @staticmethod
- def color():
- return (0, 0, 0xff)
-
- def __init__(self, sleeper):
- self.sleeper = sleeper
-
- def __repr__(self):
- return "%s gone to sleep" % thread_name(self.sleeper)
-
-class RunqueueEventWakeup:
- @staticmethod
- def color():
- return (0xff, 0xff, 0)
-
- def __init__(self, wakee):
- self.wakee = wakee
-
- def __repr__(self):
- return "%s woke up" % thread_name(self.wakee)
-
-class RunqueueEventFork:
- @staticmethod
- def color():
- return (0, 0xff, 0)
-
- def __init__(self, child):
- self.child = child
-
- def __repr__(self):
- return "new forked task %s" % thread_name(self.child)
-
-class RunqueueMigrateIn:
- @staticmethod
- def color():
- return (0, 0xf0, 0xff)
-
- def __init__(self, new):
- self.new = new
-
- def __repr__(self):
- return "task migrated in %s" % thread_name(self.new)
-
-class RunqueueMigrateOut:
- @staticmethod
- def color():
- return (0xff, 0, 0xff)
-
- def __init__(self, old):
- self.old = old
-
- def __repr__(self):
- return "task migrated out %s" % thread_name(self.old)
-
-class RunqueueSnapshot:
- def __init__(self, tasks = [0], event = RunqueueEventUnknown()):
- self.tasks = tuple(tasks)
- self.event = event
-
- def sched_switch(self, prev, prev_state, next):
- event = RunqueueEventUnknown()
-
- if taskState(prev_state) == "R" and next in self.tasks \
- and prev in self.tasks:
- return self
-
- if taskState(prev_state) != "R":
- event = RunqueueEventSleep(prev)
-
- next_tasks = list(self.tasks[:])
- if prev in self.tasks:
- if taskState(prev_state) != "R":
- next_tasks.remove(prev)
- elif taskState(prev_state) == "R":
- next_tasks.append(prev)
-
- if next not in next_tasks:
- next_tasks.append(next)
-
- return RunqueueSnapshot(next_tasks, event)
-
- def migrate_out(self, old):
- if old not in self.tasks:
- return self
- next_tasks = [task for task in self.tasks if task != old]
-
- return RunqueueSnapshot(next_tasks, RunqueueMigrateOut(old))
-
- def __migrate_in(self, new, event):
- if new in self.tasks:
- self.event = event
- return self
- next_tasks = self.tasks[:] + tuple([new])
-
- return RunqueueSnapshot(next_tasks, event)
-
- def migrate_in(self, new):
- return self.__migrate_in(new, RunqueueMigrateIn(new))
-
- def wake_up(self, new):
- return self.__migrate_in(new, RunqueueEventWakeup(new))
-
- def wake_up_new(self, new):
- return self.__migrate_in(new, RunqueueEventFork(new))
-
- def load(self):
- """ Provide the number of tasks on the runqueue.
- Don't count idle"""
- return len(self.tasks) - 1
-
- def __repr__(self):
- ret = self.tasks.__repr__()
- ret += self.origin_tostring()
-
- return ret
-
-class TimeSlice:
- def __init__(self, start, prev):
- self.start = start
- self.prev = prev
- self.end = start
- # cpus that triggered the event
- self.event_cpus = []
- if prev is not None:
- self.total_load = prev.total_load
- self.rqs = prev.rqs.copy()
- else:
- self.rqs = defaultdict(RunqueueSnapshot)
- self.total_load = 0
-
- def __update_total_load(self, old_rq, new_rq):
- diff = new_rq.load() - old_rq.load()
- self.total_load += diff
-
- def sched_switch(self, ts_list, prev, prev_state, next, cpu):
- old_rq = self.prev.rqs[cpu]
- new_rq = old_rq.sched_switch(prev, prev_state, next)
-
- if old_rq is new_rq:
- return
-
- self.rqs[cpu] = new_rq
- self.__update_total_load(old_rq, new_rq)
- ts_list.append(self)
- self.event_cpus = [cpu]
-
- def migrate(self, ts_list, new, old_cpu, new_cpu):
- if old_cpu == new_cpu:
- return
- old_rq = self.prev.rqs[old_cpu]
- out_rq = old_rq.migrate_out(new)
- self.rqs[old_cpu] = out_rq
- self.__update_total_load(old_rq, out_rq)
-
- new_rq = self.prev.rqs[new_cpu]
- in_rq = new_rq.migrate_in(new)
- self.rqs[new_cpu] = in_rq
- self.__update_total_load(new_rq, in_rq)
-
- ts_list.append(self)
-
- if old_rq is not out_rq:
- self.event_cpus.append(old_cpu)
- self.event_cpus.append(new_cpu)
-
- def wake_up(self, ts_list, pid, cpu, fork):
- old_rq = self.prev.rqs[cpu]
- if fork:
- new_rq = old_rq.wake_up_new(pid)
- else:
- new_rq = old_rq.wake_up(pid)
-
- if new_rq is old_rq:
- return
- self.rqs[cpu] = new_rq
- self.__update_total_load(old_rq, new_rq)
- ts_list.append(self)
- self.event_cpus = [cpu]
-
- def next(self, t):
- self.end = t
- return TimeSlice(t, self)
-
-class TimeSliceList(UserList):
- def __init__(self, arg = []):
- self.data = arg
-
- def get_time_slice(self, ts):
- if len(self.data) == 0:
- slice = TimeSlice(ts, TimeSlice(-1, None))
- else:
- slice = self.data[-1].next(ts)
- return slice
-
- def find_time_slice(self, ts):
- start = 0
- end = len(self.data)
- found = -1
- searching = True
- while searching:
- if start == end or start == end - 1:
- searching = False
-
- i = (end + start) / 2
- if self.data[i].start <= ts and self.data[i].end >= ts:
- found = i
- end = i
- continue
-
- if self.data[i].end < ts:
- start = i
-
- elif self.data[i].start > ts:
- end = i
-
- return found
-
- def set_root_win(self, win):
- self.root_win = win
-
- def mouse_down(self, cpu, t):
- idx = self.find_time_slice(t)
- if idx == -1:
- return
-
- ts = self[idx]
- rq = ts.rqs[cpu]
- raw = "CPU: %d\n" % cpu
- raw += "Last event : %s\n" % rq.event.__repr__()
- raw += "Timestamp : %d.%06d\n" % (ts.start / (10 ** 9), (ts.start % (10 ** 9)) / 1000)
- raw += "Duration : %6d us\n" % ((ts.end - ts.start) / (10 ** 6))
- raw += "Load = %d\n" % rq.load()
- for t in rq.tasks:
- raw += "%s \n" % thread_name(t)
-
- self.root_win.update_summary(raw)
-
- def update_rectangle_cpu(self, slice, cpu):
- rq = slice.rqs[cpu]
-
- if slice.total_load != 0:
- load_rate = rq.load() / float(slice.total_load)
- else:
- load_rate = 0
-
- red_power = int(0xff - (0xff * load_rate))
- color = (0xff, red_power, red_power)
-
- top_color = None
-
- if cpu in slice.event_cpus:
- top_color = rq.event.color()
-
- self.root_win.paint_rectangle_zone(cpu, color, top_color, slice.start, slice.end)
-
- def fill_zone(self, start, end):
- i = self.find_time_slice(start)
- if i == -1:
- return
-
- for i in range(i, len(self.data)):
- timeslice = self.data[i]
- if timeslice.start > end:
- return
-
- for cpu in timeslice.rqs:
- self.update_rectangle_cpu(timeslice, cpu)
-
- def interval(self):
- if len(self.data) == 0:
- return (0, 0)
-
- return (self.data[0].start, self.data[-1].end)
-
- def nr_rectangles(self):
- last_ts = self.data[-1]
- max_cpu = 0
- for cpu in last_ts.rqs:
- if cpu > max_cpu:
- max_cpu = cpu
- return max_cpu
-
-
-class SchedEventProxy:
- def __init__(self):
- self.current_tsk = defaultdict(lambda : -1)
- self.timeslices = TimeSliceList()
-
- def sched_switch(self, headers, prev_comm, prev_pid, prev_prio, prev_state,
- next_comm, next_pid, next_prio):
- """ Ensure the task we sched out this cpu is really the one
- we logged. Otherwise we may have missed traces """
-
- on_cpu_task = self.current_tsk[headers.cpu]
-
- if on_cpu_task != -1 and on_cpu_task != prev_pid:
- print("Sched switch event rejected ts: %s cpu: %d prev: %s(%d) next: %s(%d)" % \
- headers.ts_format(), headers.cpu, prev_comm, prev_pid, next_comm, next_pid)
-
- threads[prev_pid] = prev_comm
- threads[next_pid] = next_comm
- self.current_tsk[headers.cpu] = next_pid
-
- ts = self.timeslices.get_time_slice(headers.ts())
- ts.sched_switch(self.timeslices, prev_pid, prev_state, next_pid, headers.cpu)
-
- def migrate(self, headers, pid, prio, orig_cpu, dest_cpu):
- ts = self.timeslices.get_time_slice(headers.ts())
- ts.migrate(self.timeslices, pid, orig_cpu, dest_cpu)
-
- def wake_up(self, headers, comm, pid, success, target_cpu, fork):
- if success == 0:
- return
- ts = self.timeslices.get_time_slice(headers.ts())
- ts.wake_up(self.timeslices, pid, target_cpu, fork)
-
-
-def trace_begin():
- global parser
- parser = SchedEventProxy()
-
-def trace_end():
- app = wx.App(False)
- timeslices = parser.timeslices
- frame = RootFrame(timeslices, "Migration")
- app.MainLoop()
-
-def sched__sched_stat_runtime(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, comm, pid, runtime, vruntime):
- pass
-
-def sched__sched_stat_iowait(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, comm, pid, delay):
- pass
-
-def sched__sched_stat_sleep(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, comm, pid, delay):
- pass
-
-def sched__sched_stat_wait(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, comm, pid, delay):
- pass
-
-def sched__sched_process_fork(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, parent_comm, parent_pid, child_comm, child_pid):
- pass
-
-def sched__sched_process_wait(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, comm, pid, prio):
- pass
-
-def sched__sched_process_exit(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, comm, pid, prio):
- pass
-
-def sched__sched_process_free(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, comm, pid, prio):
- pass
-
-def sched__sched_migrate_task(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, comm, pid, prio, orig_cpu,
- dest_cpu):
- headers = EventHeaders(common_cpu, common_secs, common_nsecs,
- common_pid, common_comm, common_callchain)
- parser.migrate(headers, pid, prio, orig_cpu, dest_cpu)
-
-def sched__sched_switch(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm, common_callchain,
- prev_comm, prev_pid, prev_prio, prev_state,
- next_comm, next_pid, next_prio):
-
- headers = EventHeaders(common_cpu, common_secs, common_nsecs,
- common_pid, common_comm, common_callchain)
- parser.sched_switch(headers, prev_comm, prev_pid, prev_prio, prev_state,
- next_comm, next_pid, next_prio)
-
-def sched__sched_wakeup_new(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, comm, pid, prio, success,
- target_cpu):
- headers = EventHeaders(common_cpu, common_secs, common_nsecs,
- common_pid, common_comm, common_callchain)
- parser.wake_up(headers, comm, pid, success, target_cpu, 1)
-
-def sched__sched_wakeup(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, comm, pid, prio, success,
- target_cpu):
- headers = EventHeaders(common_cpu, common_secs, common_nsecs,
- common_pid, common_comm, common_callchain)
- parser.wake_up(headers, comm, pid, success, target_cpu, 0)
-
-def sched__sched_wait_task(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, comm, pid, prio):
- pass
-
-def sched__sched_kthread_stop_ret(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, ret):
- pass
-
-def sched__sched_kthread_stop(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, comm, pid):
- pass
-
-def trace_unhandled(event_name, context, event_fields_dict):
- pass
diff --git a/tools/perf/scripts/python/sctop.py b/tools/perf/scripts/python/sctop.py
deleted file mode 100644
index 6e0278dcb092..000000000000
--- a/tools/perf/scripts/python/sctop.py
+++ /dev/null
@@ -1,89 +0,0 @@
-# system call top
-# (c) 2010, Tom Zanussi <tzanussi@gmail.com>
-# Licensed under the terms of the GNU GPL License version 2
-#
-# Periodically displays system-wide system call totals, broken down by
-# syscall. If a [comm] arg is specified, only syscalls called by
-# [comm] are displayed. If an [interval] arg is specified, the display
-# will be refreshed every [interval] seconds. The default interval is
-# 3 seconds.
-
-from __future__ import print_function
-
-import os, sys, time
-
-try:
- import thread
-except ImportError:
- import _thread as thread
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import *
-from Core import *
-from Util import *
-
-usage = "perf script -s sctop.py [comm] [interval]\n";
-
-for_comm = None
-default_interval = 3
-interval = default_interval
-
-if len(sys.argv) > 3:
- sys.exit(usage)
-
-if len(sys.argv) > 2:
- for_comm = sys.argv[1]
- interval = int(sys.argv[2])
-elif len(sys.argv) > 1:
- try:
- interval = int(sys.argv[1])
- except ValueError:
- for_comm = sys.argv[1]
- interval = default_interval
-
-syscalls = autodict()
-
-def trace_begin():
- thread.start_new_thread(print_syscall_totals, (interval,))
- pass
-
-def raw_syscalls__sys_enter(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, id, args):
- if for_comm is not None:
- if common_comm != for_comm:
- return
- try:
- syscalls[id] += 1
- except TypeError:
- syscalls[id] = 1
-
-def syscalls__sys_enter(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- id, args):
- raw_syscalls__sys_enter(**locals())
-
-def print_syscall_totals(interval):
- while 1:
- clear_term()
- if for_comm is not None:
- print("\nsyscall events for %s:\n" % (for_comm))
- else:
- print("\nsyscall events:\n")
-
- print("%-40s %10s" % ("event", "count"))
- print("%-40s %10s" %
- ("----------------------------------------",
- "----------"))
-
- for id, val in sorted(syscalls.items(),
- key = lambda kv: (kv[1], kv[0]),
- reverse = True):
- try:
- print("%-40s %10d" % (syscall_name(id), val))
- except TypeError:
- pass
- syscalls.clear()
- time.sleep(interval)
diff --git a/tools/perf/scripts/python/stackcollapse.py b/tools/perf/scripts/python/stackcollapse.py
deleted file mode 100755
index b1c4def1410a..000000000000
--- a/tools/perf/scripts/python/stackcollapse.py
+++ /dev/null
@@ -1,127 +0,0 @@
-# stackcollapse.py - format perf samples with one line per distinct call stack
-# SPDX-License-Identifier: GPL-2.0
-#
-# This script's output has two space-separated fields. The first is a semicolon
-# separated stack including the program name (from the "comm" field) and the
-# function names from the call stack. The second is a count:
-#
-# swapper;start_kernel;rest_init;cpu_idle;default_idle;native_safe_halt 2
-#
-# The file is sorted according to the first field.
-#
-# Input may be created and processed using:
-#
-# perf record -a -g -F 99 sleep 60
-# perf script report stackcollapse > out.stacks-folded
-#
-# (perf script record stackcollapse works too).
-#
-# Written by Paolo Bonzini <pbonzini@redhat.com>
-# Based on Brendan Gregg's stackcollapse-perf.pl script.
-
-from __future__ import print_function
-
-import os
-import sys
-from collections import defaultdict
-from optparse import OptionParser, make_option
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import *
-from Core import *
-from EventClass import *
-
-# command line parsing
-
-option_list = [
- # formatting options for the bottom entry of the stack
- make_option("--include-tid", dest="include_tid",
- action="store_true", default=False,
- help="include thread id in stack"),
- make_option("--include-pid", dest="include_pid",
- action="store_true", default=False,
- help="include process id in stack"),
- make_option("--no-comm", dest="include_comm",
- action="store_false", default=True,
- help="do not separate stacks according to comm"),
- make_option("--tidy-java", dest="tidy_java",
- action="store_true", default=False,
- help="beautify Java signatures"),
- make_option("--kernel", dest="annotate_kernel",
- action="store_true", default=False,
- help="annotate kernel functions with _[k]")
-]
-
-parser = OptionParser(option_list=option_list)
-(opts, args) = parser.parse_args()
-
-if len(args) != 0:
- parser.error("unexpected command line argument")
-if opts.include_tid and not opts.include_comm:
- parser.error("requesting tid but not comm is invalid")
-if opts.include_pid and not opts.include_comm:
- parser.error("requesting pid but not comm is invalid")
-
-# event handlers
-
-lines = defaultdict(lambda: 0)
-
-def process_event(param_dict):
- def tidy_function_name(sym, dso):
- if sym is None:
- sym = '[unknown]'
-
- sym = sym.replace(';', ':')
- if opts.tidy_java:
- # the original stackcollapse-perf.pl script gives the
- # example of converting this:
- # Lorg/mozilla/javascript/MemberBox;.<init>(Ljava/lang/reflect/Method;)V
- # to this:
- # org/mozilla/javascript/MemberBox:.init
- sym = sym.replace('<', '')
- sym = sym.replace('>', '')
- if sym[0] == 'L' and sym.find('/'):
- sym = sym[1:]
- try:
- sym = sym[:sym.index('(')]
- except ValueError:
- pass
-
- if opts.annotate_kernel and dso == '[kernel.kallsyms]':
- return sym + '_[k]'
- else:
- return sym
-
- stack = list()
- if 'callchain' in param_dict:
- for entry in param_dict['callchain']:
- entry.setdefault('sym', dict())
- entry['sym'].setdefault('name', None)
- entry.setdefault('dso', None)
- stack.append(tidy_function_name(entry['sym']['name'],
- entry['dso']))
- else:
- param_dict.setdefault('symbol', None)
- param_dict.setdefault('dso', None)
- stack.append(tidy_function_name(param_dict['symbol'],
- param_dict['dso']))
-
- if opts.include_comm:
- comm = param_dict["comm"].replace(' ', '_')
- sep = "-"
- if opts.include_pid:
- comm = comm + sep + str(param_dict['sample']['pid'])
- sep = "/"
- if opts.include_tid:
- comm = comm + sep + str(param_dict['sample']['tid'])
- stack.append(comm)
-
- stack_string = ';'.join(reversed(stack))
- lines[stack_string] = lines[stack_string] + 1
-
-def trace_end():
- list = sorted(lines)
- for stack in list:
- print("%s %d" % (stack, lines[stack]))
diff --git a/tools/perf/scripts/python/stat-cpi.py b/tools/perf/scripts/python/stat-cpi.py
deleted file mode 100644
index 01fa933ff3cf..000000000000
--- a/tools/perf/scripts/python/stat-cpi.py
+++ /dev/null
@@ -1,79 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-
-from __future__ import print_function
-
-data = {}
-times = []
-threads = []
-cpus = []
-
-def get_key(time, event, cpu, thread):
- return "%d-%s-%d-%d" % (time, event, cpu, thread)
-
-def store_key(time, cpu, thread):
- if (time not in times):
- times.append(time)
-
- if (cpu not in cpus):
- cpus.append(cpu)
-
- if (thread not in threads):
- threads.append(thread)
-
-def store(time, event, cpu, thread, val, ena, run):
- #print("event %s cpu %d, thread %d, time %d, val %d, ena %d, run %d" %
- # (event, cpu, thread, time, val, ena, run))
-
- store_key(time, cpu, thread)
- key = get_key(time, event, cpu, thread)
- data[key] = [ val, ena, run]
-
-def get(time, event, cpu, thread):
- key = get_key(time, event, cpu, thread)
- return data[key][0]
-
-def stat__cycles_k(cpu, thread, time, val, ena, run):
- store(time, "cycles", cpu, thread, val, ena, run);
-
-def stat__instructions_k(cpu, thread, time, val, ena, run):
- store(time, "instructions", cpu, thread, val, ena, run);
-
-def stat__cycles_u(cpu, thread, time, val, ena, run):
- store(time, "cycles", cpu, thread, val, ena, run);
-
-def stat__instructions_u(cpu, thread, time, val, ena, run):
- store(time, "instructions", cpu, thread, val, ena, run);
-
-def stat__cycles(cpu, thread, time, val, ena, run):
- store(time, "cycles", cpu, thread, val, ena, run);
-
-def stat__instructions(cpu, thread, time, val, ena, run):
- store(time, "instructions", cpu, thread, val, ena, run);
-
-def stat__interval(time):
- for cpu in cpus:
- for thread in threads:
- cyc = get(time, "cycles", cpu, thread)
- ins = get(time, "instructions", cpu, thread)
- cpi = 0
-
- if ins != 0:
- cpi = cyc/float(ins)
-
- print("%15f: cpu %d, thread %d -> cpi %f (%d/%d)" % (time/(float(1000000000)), cpu, thread, cpi, cyc, ins))
-
-def trace_end():
- pass
-# XXX trace_end callback could be used as an alternative place
-# to compute same values as in the script above:
-#
-# for time in times:
-# for cpu in cpus:
-# for thread in threads:
-# cyc = get(time, "cycles", cpu, thread)
-# ins = get(time, "instructions", cpu, thread)
-#
-# if ins != 0:
-# cpi = cyc/float(ins)
-#
-# print("time %.9f, cpu %d, thread %d -> cpi %f" % (time/(float(1000000000)), cpu, thread, cpi))
diff --git a/tools/perf/scripts/python/syscall-counts-by-pid.py b/tools/perf/scripts/python/syscall-counts-by-pid.py
deleted file mode 100644
index f254e40c6f0f..000000000000
--- a/tools/perf/scripts/python/syscall-counts-by-pid.py
+++ /dev/null
@@ -1,75 +0,0 @@
-# system call counts, by pid
-# (c) 2010, Tom Zanussi <tzanussi@gmail.com>
-# Licensed under the terms of the GNU GPL License version 2
-#
-# Displays system-wide system call totals, broken down by syscall.
-# If a [comm] arg is specified, only syscalls called by [comm] are displayed.
-
-from __future__ import print_function
-
-import os, sys
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import *
-from Core import *
-from Util import syscall_name
-
-usage = "perf script -s syscall-counts-by-pid.py [comm]\n";
-
-for_comm = None
-for_pid = None
-
-if len(sys.argv) > 2:
- sys.exit(usage)
-
-if len(sys.argv) > 1:
- try:
- for_pid = int(sys.argv[1])
- except:
- for_comm = sys.argv[1]
-
-syscalls = autodict()
-
-def trace_begin():
- print("Press control+C to stop and show the summary")
-
-def trace_end():
- print_syscall_totals()
-
-def raw_syscalls__sys_enter(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, id, args):
- if (for_comm and common_comm != for_comm) or \
- (for_pid and common_pid != for_pid ):
- return
- try:
- syscalls[common_comm][common_pid][id] += 1
- except TypeError:
- syscalls[common_comm][common_pid][id] = 1
-
-def syscalls__sys_enter(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- id, args):
- raw_syscalls__sys_enter(**locals())
-
-def print_syscall_totals():
- if for_comm is not None:
- print("\nsyscall events for %s:\n" % (for_comm))
- else:
- print("\nsyscall events by comm/pid:\n")
-
- print("%-40s %10s" % ("comm [pid]/syscalls", "count"))
- print("%-40s %10s" % ("----------------------------------------",
- "----------"))
-
- comm_keys = syscalls.keys()
- for comm in comm_keys:
- pid_keys = syscalls[comm].keys()
- for pid in pid_keys:
- print("\n%s [%d]" % (comm, pid))
- id_keys = syscalls[comm][pid].keys()
- for id, val in sorted(syscalls[comm][pid].items(),
- key = lambda kv: (kv[1], kv[0]), reverse = True):
- print(" %-38s %10d" % (syscall_name(id), val))
diff --git a/tools/perf/scripts/python/syscall-counts.py b/tools/perf/scripts/python/syscall-counts.py
deleted file mode 100644
index 8adb95ff1664..000000000000
--- a/tools/perf/scripts/python/syscall-counts.py
+++ /dev/null
@@ -1,65 +0,0 @@
-# system call counts
-# (c) 2010, Tom Zanussi <tzanussi@gmail.com>
-# Licensed under the terms of the GNU GPL License version 2
-#
-# Displays system-wide system call totals, broken down by syscall.
-# If a [comm] arg is specified, only syscalls called by [comm] are displayed.
-
-from __future__ import print_function
-
-import os
-import sys
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import *
-from Core import *
-from Util import syscall_name
-
-usage = "perf script -s syscall-counts.py [comm]\n";
-
-for_comm = None
-
-if len(sys.argv) > 2:
- sys.exit(usage)
-
-if len(sys.argv) > 1:
- for_comm = sys.argv[1]
-
-syscalls = autodict()
-
-def trace_begin():
- print("Press control+C to stop and show the summary")
-
-def trace_end():
- print_syscall_totals()
-
-def raw_syscalls__sys_enter(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- common_callchain, id, args):
- if for_comm is not None:
- if common_comm != for_comm:
- return
- try:
- syscalls[id] += 1
- except TypeError:
- syscalls[id] = 1
-
-def syscalls__sys_enter(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm, id, args):
- raw_syscalls__sys_enter(**locals())
-
-def print_syscall_totals():
- if for_comm is not None:
- print("\nsyscall events for %s:\n" % (for_comm))
- else:
- print("\nsyscall events:\n")
-
- print("%-40s %10s" % ("event", "count"))
- print("%-40s %10s" % ("----------------------------------------",
- "-----------"))
-
- for id, val in sorted(syscalls.items(),
- key = lambda kv: (kv[1], kv[0]), reverse = True):
- print("%-40s %10d" % (syscall_name(id), val))
diff --git a/tools/perf/scripts/python/task-analyzer.py b/tools/perf/scripts/python/task-analyzer.py
deleted file mode 100755
index 3f1df9894246..000000000000
--- a/tools/perf/scripts/python/task-analyzer.py
+++ /dev/null
@@ -1,934 +0,0 @@
-# task-analyzer.py - comprehensive perf tasks analysis
-# SPDX-License-Identifier: GPL-2.0
-# Copyright (c) 2022, Hagen Paul Pfeifer <hagen@jauu.net>
-# Licensed under the terms of the GNU GPL License version 2
-#
-# Usage:
-#
-# perf record -e sched:sched_switch -a -- sleep 10
-# perf script report task-analyzer
-#
-
-from __future__ import print_function
-import sys
-import os
-import string
-import argparse
-import decimal
-
-
-sys.path.append(
- os.environ["PERF_EXEC_PATH"] + "/scripts/python/Perf-Trace-Util/lib/Perf/Trace"
-)
-from perf_trace_context import *
-from Core import *
-
-# Definition of possible ASCII color codes
-_COLORS = {
- "grey": "\033[90m",
- "red": "\033[91m",
- "green": "\033[92m",
- "yellow": "\033[93m",
- "blue": "\033[94m",
- "violet": "\033[95m",
- "reset": "\033[0m",
-}
-
-# Columns will have a static size to align everything properly
-# Support of 116 days of active update with nano precision
-LEN_SWITCHED_IN = len("9999999.999999999") # 17
-LEN_SWITCHED_OUT = len("9999999.999999999") # 17
-LEN_CPU = len("000")
-LEN_PID = len("maxvalue") # 8
-LEN_TID = len("maxvalue") # 8
-LEN_COMM = len("max-comms-length") # 16
-LEN_RUNTIME = len("999999.999") # 10
-# Support of 3.45 hours of timespans
-LEN_OUT_IN = len("99999999999.999") # 15
-LEN_OUT_OUT = len("99999999999.999") # 15
-LEN_IN_IN = len("99999999999.999") # 15
-LEN_IN_OUT = len("99999999999.999") # 15
-
-
-# py2/py3 compatibility layer, see PEP469
-try:
- dict.iteritems
-except AttributeError:
- # py3
- def itervalues(d):
- return iter(d.values())
-
- def iteritems(d):
- return iter(d.items())
-
-else:
- # py2
- def itervalues(d):
- return d.itervalues()
-
- def iteritems(d):
- return d.iteritems()
-
-
-def _check_color():
- global _COLORS
- """user enforced no-color or if stdout is no tty we disable colors"""
- if sys.stdout.isatty() and args.stdio_color != "never":
- return
- _COLORS = {
- "grey": "",
- "red": "",
- "green": "",
- "yellow": "",
- "blue": "",
- "violet": "",
- "reset": "",
- }
-
-
-def _parse_args():
- global args
- parser = argparse.ArgumentParser(description="Analyze tasks behavior")
- parser.add_argument(
- "--time-limit",
- default=[],
- help=
- "print tasks only in time[s] window e.g"
- " --time-limit 123.111:789.222(print all between 123.111 and 789.222)"
- " --time-limit 123: (print all from 123)"
- " --time-limit :456 (print all until incl. 456)",
- )
- parser.add_argument(
- "--summary", action="store_true", help="print addtional runtime information"
- )
- parser.add_argument(
- "--summary-only", action="store_true", help="print only summary without traces"
- )
- parser.add_argument(
- "--summary-extended",
- action="store_true",
- help="print the summary with additional information of max inter task times"
- " relative to the prev task",
- )
- parser.add_argument(
- "--ns", action="store_true", help="show timestamps in nanoseconds"
- )
- parser.add_argument(
- "--ms", action="store_true", help="show timestamps in milliseconds"
- )
- parser.add_argument(
- "--extended-times",
- action="store_true",
- help="Show the elapsed times between schedule in/schedule out"
- " of this task and the schedule in/schedule out of previous occurrence"
- " of the same task",
- )
- parser.add_argument(
- "--filter-tasks",
- default=[],
- help="filter out unneeded tasks by tid, pid or processname."
- " E.g --filter-task 1337,/sbin/init ",
- )
- parser.add_argument(
- "--limit-to-tasks",
- default=[],
- help="limit output to selected task by tid, pid, processname."
- " E.g --limit-to-tasks 1337,/sbin/init",
- )
- parser.add_argument(
- "--highlight-tasks",
- default="",
- help="colorize special tasks by their pid/tid/comm."
- " E.g. --highlight-tasks 1:red,mutt:yellow"
- " Colors available: red,grey,yellow,blue,violet,green",
- )
- parser.add_argument(
- "--rename-comms-by-tids",
- default="",
- help="rename task names by using tid (<tid>:<newname>,<tid>:<newname>)"
- " This option is handy for inexpressive processnames like python interpreted"
- " process. E.g --rename 1337:my-python-app",
- )
- parser.add_argument(
- "--stdio-color",
- default="auto",
- choices=["always", "never", "auto"],
- help="always, never or auto, allowing configuring color output"
- " via the command line",
- )
- parser.add_argument(
- "--csv",
- default="",
- help="Write trace to file selected by user. Options, like --ns or --extended"
- "-times are used.",
- )
- parser.add_argument(
- "--csv-summary",
- default="",
- help="Write summary to file selected by user. Options, like --ns or"
- " --summary-extended are used.",
- )
- args = parser.parse_args()
- args.tid_renames = dict()
-
- _argument_filter_sanity_check()
- _argument_prepare_check()
-
-
-def time_uniter(unit):
- picker = {
- "s": 1,
- "ms": 1e3,
- "us": 1e6,
- "ns": 1e9,
- }
- return picker[unit]
-
-
-def _init_db():
- global db
- db = dict()
- db["running"] = dict()
- db["cpu"] = dict()
- db["tid"] = dict()
- db["global"] = []
- if args.summary or args.summary_extended or args.summary_only:
- db["task_info"] = dict()
- db["runtime_info"] = dict()
- # min values for summary depending on the header
- db["task_info"]["pid"] = len("PID")
- db["task_info"]["tid"] = len("TID")
- db["task_info"]["comm"] = len("Comm")
- db["runtime_info"]["runs"] = len("Runs")
- db["runtime_info"]["acc"] = len("Accumulated")
- db["runtime_info"]["max"] = len("Max")
- db["runtime_info"]["max_at"] = len("Max At")
- db["runtime_info"]["min"] = len("Min")
- db["runtime_info"]["mean"] = len("Mean")
- db["runtime_info"]["median"] = len("Median")
- if args.summary_extended:
- db["inter_times"] = dict()
- db["inter_times"]["out_in"] = len("Out-In")
- db["inter_times"]["inter_at"] = len("At")
- db["inter_times"]["out_out"] = len("Out-Out")
- db["inter_times"]["in_in"] = len("In-In")
- db["inter_times"]["in_out"] = len("In-Out")
-
-
-def _median(numbers):
- """phython3 hat statistics module - we have nothing"""
- n = len(numbers)
- index = n // 2
- if n % 2:
- return sorted(numbers)[index]
- return sum(sorted(numbers)[index - 1 : index + 1]) / 2
-
-
-def _mean(numbers):
- return sum(numbers) / len(numbers)
-
-
-class Timespans(object):
- """
- The elapsed time between two occurrences of the same task is being tracked with the
- help of this class. There are 4 of those Timespans Out-Out, In-Out, Out-In and
- In-In.
- The first half of the name signals the first time point of the
- first task. The second half of the name represents the second
- timepoint of the second task.
- """
-
- def __init__(self):
- self._last_start = None
- self._last_finish = None
- self.out_out = -1
- self.in_out = -1
- self.out_in = -1
- self.in_in = -1
- if args.summary_extended:
- self._time_in = -1
- self.max_out_in = -1
- self.max_at = -1
- self.max_in_out = -1
- self.max_in_in = -1
- self.max_out_out = -1
-
- def feed(self, task):
- """
- Called for every recorded trace event to find process pair and calculate the
- task timespans. Chronological ordering, feed does not do reordering
- """
- if not self._last_finish:
- self._last_start = task.time_in(time_unit)
- self._last_finish = task.time_out(time_unit)
- return
- self._time_in = task.time_in()
- time_in = task.time_in(time_unit)
- time_out = task.time_out(time_unit)
- self.in_in = time_in - self._last_start
- self.out_in = time_in - self._last_finish
- self.in_out = time_out - self._last_start
- self.out_out = time_out - self._last_finish
- if args.summary_extended:
- self._update_max_entries()
- self._last_finish = task.time_out(time_unit)
- self._last_start = task.time_in(time_unit)
-
- def _update_max_entries(self):
- if self.in_in > self.max_in_in:
- self.max_in_in = self.in_in
- if self.out_out > self.max_out_out:
- self.max_out_out = self.out_out
- if self.in_out > self.max_in_out:
- self.max_in_out = self.in_out
- if self.out_in > self.max_out_in:
- self.max_out_in = self.out_in
- self.max_at = self._time_in
-
-
-
-class Summary(object):
- """
- Primary instance for calculating the summary output. Processes the whole trace to
- find and memorize relevant data such as mean, max et cetera. This instance handles
- dynamic alignment aspects for summary output.
- """
-
- def __init__(self):
- self._body = []
-
- class AlignmentHelper:
- """
- Used to calculated the alignment for the output of the summary.
- """
- def __init__(self, pid, tid, comm, runs, acc, mean,
- median, min, max, max_at):
- self.pid = pid
- self.tid = tid
- self.comm = comm
- self.runs = runs
- self.acc = acc
- self.mean = mean
- self.median = median
- self.min = min
- self.max = max
- self.max_at = max_at
- if args.summary_extended:
- self.out_in = None
- self.inter_at = None
- self.out_out = None
- self.in_in = None
- self.in_out = None
-
- def _print_header(self):
- '''
- Output is trimmed in _format_stats thus additional adjustment in the header
- is needed, depending on the choice of timeunit. The adjustment corresponds
- to the amount of column titles being adjusted in _column_titles.
- '''
- decimal_precision = 6 if not args.ns else 9
- fmt = " {{:^{}}}".format(sum(db["task_info"].values()))
- fmt += " {{:^{}}}".format(
- sum(db["runtime_info"].values()) - 2 * decimal_precision
- )
- _header = ("Task Information", "Runtime Information")
-
- if args.summary_extended:
- fmt += " {{:^{}}}".format(
- sum(db["inter_times"].values()) - 4 * decimal_precision
- )
- _header += ("Max Inter Task Times",)
- fd_sum.write(fmt.format(*_header) + "\n")
-
- def _column_titles(self):
- """
- Cells are being processed and displayed in different way so an alignment adjust
- is implemented depeding on the choice of the timeunit. The positions of the max
- values are being displayed in grey. Thus in their format two additional {},
- are placed for color set and reset.
- """
- separator, fix_csv_align = _prepare_fmt_sep()
- decimal_precision, time_precision = _prepare_fmt_precision()
- fmt = "{{:>{}}}".format(db["task_info"]["pid"] * fix_csv_align)
- fmt += "{}{{:>{}}}".format(separator, db["task_info"]["tid"] * fix_csv_align)
- fmt += "{}{{:>{}}}".format(separator, db["task_info"]["comm"] * fix_csv_align)
- fmt += "{}{{:>{}}}".format(separator, db["runtime_info"]["runs"] * fix_csv_align)
- fmt += "{}{{:>{}}}".format(separator, db["runtime_info"]["acc"] * fix_csv_align)
- fmt += "{}{{:>{}}}".format(separator, db["runtime_info"]["mean"] * fix_csv_align)
- fmt += "{}{{:>{}}}".format(
- separator, db["runtime_info"]["median"] * fix_csv_align
- )
- fmt += "{}{{:>{}}}".format(
- separator, (db["runtime_info"]["min"] - decimal_precision) * fix_csv_align
- )
- fmt += "{}{{:>{}}}".format(
- separator, (db["runtime_info"]["max"] - decimal_precision) * fix_csv_align
- )
- fmt += "{}{{}}{{:>{}}}{{}}".format(
- separator, (db["runtime_info"]["max_at"] - time_precision) * fix_csv_align
- )
-
- column_titles = ("PID", "TID", "Comm")
- column_titles += ("Runs", "Accumulated", "Mean", "Median", "Min", "Max")
- column_titles += (_COLORS["grey"], "Max At", _COLORS["reset"])
-
- if args.summary_extended:
- fmt += "{}{{:>{}}}".format(
- separator,
- (db["inter_times"]["out_in"] - decimal_precision) * fix_csv_align
- )
- fmt += "{}{{}}{{:>{}}}{{}}".format(
- separator,
- (db["inter_times"]["inter_at"] - time_precision) * fix_csv_align
- )
- fmt += "{}{{:>{}}}".format(
- separator,
- (db["inter_times"]["out_out"] - decimal_precision) * fix_csv_align
- )
- fmt += "{}{{:>{}}}".format(
- separator,
- (db["inter_times"]["in_in"] - decimal_precision) * fix_csv_align
- )
- fmt += "{}{{:>{}}}".format(
- separator,
- (db["inter_times"]["in_out"] - decimal_precision) * fix_csv_align
- )
-
- column_titles += ("Out-In", _COLORS["grey"], "Max At", _COLORS["reset"],
- "Out-Out", "In-In", "In-Out")
-
- fd_sum.write(fmt.format(*column_titles) + "\n")
-
-
- def _task_stats(self):
- """calculates the stats of every task and constructs the printable summary"""
- for tid in sorted(db["tid"]):
- color_one_sample = _COLORS["grey"]
- color_reset = _COLORS["reset"]
- no_executed = 0
- runtimes = []
- time_in = []
- timespans = Timespans()
- for task in db["tid"][tid]:
- pid = task.pid
- comm = task.comm
- no_executed += 1
- runtimes.append(task.runtime(time_unit))
- time_in.append(task.time_in())
- timespans.feed(task)
- if len(runtimes) > 1:
- color_one_sample = ""
- color_reset = ""
- time_max = max(runtimes)
- time_min = min(runtimes)
- max_at = time_in[runtimes.index(max(runtimes))]
-
- # The size of the decimal after sum,mean and median varies, thus we cut
- # the decimal number, by rounding it. It has no impact on the output,
- # because we have a precision of the decimal points at the output.
- time_sum = round(sum(runtimes), 3)
- time_mean = round(_mean(runtimes), 3)
- time_median = round(_median(runtimes), 3)
-
- align_helper = self.AlignmentHelper(pid, tid, comm, no_executed, time_sum,
- time_mean, time_median, time_min, time_max, max_at)
- self._body.append([pid, tid, comm, no_executed, time_sum, color_one_sample,
- time_mean, time_median, time_min, time_max,
- _COLORS["grey"], max_at, _COLORS["reset"], color_reset])
- if args.summary_extended:
- self._body[-1].extend([timespans.max_out_in,
- _COLORS["grey"], timespans.max_at,
- _COLORS["reset"], timespans.max_out_out,
- timespans.max_in_in,
- timespans.max_in_out])
- align_helper.out_in = timespans.max_out_in
- align_helper.inter_at = timespans.max_at
- align_helper.out_out = timespans.max_out_out
- align_helper.in_in = timespans.max_in_in
- align_helper.in_out = timespans.max_in_out
- self._calc_alignments_summary(align_helper)
-
- def _format_stats(self):
- separator, fix_csv_align = _prepare_fmt_sep()
- decimal_precision, time_precision = _prepare_fmt_precision()
- len_pid = db["task_info"]["pid"] * fix_csv_align
- len_tid = db["task_info"]["tid"] * fix_csv_align
- len_comm = db["task_info"]["comm"] * fix_csv_align
- len_runs = db["runtime_info"]["runs"] * fix_csv_align
- len_acc = db["runtime_info"]["acc"] * fix_csv_align
- len_mean = db["runtime_info"]["mean"] * fix_csv_align
- len_median = db["runtime_info"]["median"] * fix_csv_align
- len_min = (db["runtime_info"]["min"] - decimal_precision) * fix_csv_align
- len_max = (db["runtime_info"]["max"] - decimal_precision) * fix_csv_align
- len_max_at = (db["runtime_info"]["max_at"] - time_precision) * fix_csv_align
- if args.summary_extended:
- len_out_in = (
- db["inter_times"]["out_in"] - decimal_precision
- ) * fix_csv_align
- len_inter_at = (
- db["inter_times"]["inter_at"] - time_precision
- ) * fix_csv_align
- len_out_out = (
- db["inter_times"]["out_out"] - decimal_precision
- ) * fix_csv_align
- len_in_in = (db["inter_times"]["in_in"] - decimal_precision) * fix_csv_align
- len_in_out = (
- db["inter_times"]["in_out"] - decimal_precision
- ) * fix_csv_align
-
- fmt = "{{:{}d}}".format(len_pid)
- fmt += "{}{{:{}d}}".format(separator, len_tid)
- fmt += "{}{{:>{}}}".format(separator, len_comm)
- fmt += "{}{{:{}d}}".format(separator, len_runs)
- fmt += "{}{{:{}.{}f}}".format(separator, len_acc, time_precision)
- fmt += "{}{{}}{{:{}.{}f}}".format(separator, len_mean, time_precision)
- fmt += "{}{{:{}.{}f}}".format(separator, len_median, time_precision)
- fmt += "{}{{:{}.{}f}}".format(separator, len_min, time_precision)
- fmt += "{}{{:{}.{}f}}".format(separator, len_max, time_precision)
- fmt += "{}{{}}{{:{}.{}f}}{{}}{{}}".format(
- separator, len_max_at, decimal_precision
- )
- if args.summary_extended:
- fmt += "{}{{:{}.{}f}}".format(separator, len_out_in, time_precision)
- fmt += "{}{{}}{{:{}.{}f}}{{}}".format(
- separator, len_inter_at, decimal_precision
- )
- fmt += "{}{{:{}.{}f}}".format(separator, len_out_out, time_precision)
- fmt += "{}{{:{}.{}f}}".format(separator, len_in_in, time_precision)
- fmt += "{}{{:{}.{}f}}".format(separator, len_in_out, time_precision)
- return fmt
-
-
- def _calc_alignments_summary(self, align_helper):
- # Length is being cut in 3 groups so that further addition is easier to handle.
- # The length of every argument from the alignment helper is being checked if it
- # is longer than the longest until now. In that case the length is being saved.
- for key in db["task_info"]:
- if len(str(getattr(align_helper, key))) > db["task_info"][key]:
- db["task_info"][key] = len(str(getattr(align_helper, key)))
- for key in db["runtime_info"]:
- if len(str(getattr(align_helper, key))) > db["runtime_info"][key]:
- db["runtime_info"][key] = len(str(getattr(align_helper, key)))
- if args.summary_extended:
- for key in db["inter_times"]:
- if len(str(getattr(align_helper, key))) > db["inter_times"][key]:
- db["inter_times"][key] = len(str(getattr(align_helper, key)))
-
-
- def print(self):
- self._task_stats()
- fmt = self._format_stats()
-
- if not args.csv_summary:
- print("\nSummary")
- self._print_header()
- self._column_titles()
- for i in range(len(self._body)):
- fd_sum.write(fmt.format(*tuple(self._body[i])) + "\n")
-
-
-
-class Task(object):
- """ The class is used to handle the information of a given task."""
-
- def __init__(self, id, tid, cpu, comm):
- self.id = id
- self.tid = tid
- self.cpu = cpu
- self.comm = comm
- self.pid = None
- self._time_in = None
- self._time_out = None
-
- def schedule_in_at(self, time):
- """set the time where the task was scheduled in"""
- self._time_in = time
-
- def schedule_out_at(self, time):
- """set the time where the task was scheduled out"""
- self._time_out = time
-
- def time_out(self, unit="s"):
- """return time where a given task was scheduled out"""
- factor = time_uniter(unit)
- return self._time_out * decimal.Decimal(factor)
-
- def time_in(self, unit="s"):
- """return time where a given task was scheduled in"""
- factor = time_uniter(unit)
- return self._time_in * decimal.Decimal(factor)
-
- def runtime(self, unit="us"):
- factor = time_uniter(unit)
- return (self._time_out - self._time_in) * decimal.Decimal(factor)
-
- def update_pid(self, pid):
- self.pid = pid
-
-
-def _task_id(pid, cpu):
- """returns a "unique-enough" identifier, please do not change"""
- return "{}-{}".format(pid, cpu)
-
-
-def _filter_non_printable(unfiltered):
- """comm names may contain loony chars like '\x00000'"""
- filtered = ""
- for char in unfiltered:
- if char not in string.printable:
- continue
- filtered += char
- return filtered
-
-
-def _fmt_header():
- separator, fix_csv_align = _prepare_fmt_sep()
- fmt = "{{:>{}}}".format(LEN_SWITCHED_IN*fix_csv_align)
- fmt += "{}{{:>{}}}".format(separator, LEN_SWITCHED_OUT*fix_csv_align)
- fmt += "{}{{:>{}}}".format(separator, LEN_CPU*fix_csv_align)
- fmt += "{}{{:>{}}}".format(separator, LEN_PID*fix_csv_align)
- fmt += "{}{{:>{}}}".format(separator, LEN_TID*fix_csv_align)
- fmt += "{}{{:>{}}}".format(separator, LEN_COMM*fix_csv_align)
- fmt += "{}{{:>{}}}".format(separator, LEN_RUNTIME*fix_csv_align)
- fmt += "{}{{:>{}}}".format(separator, LEN_OUT_IN*fix_csv_align)
- if args.extended_times:
- fmt += "{}{{:>{}}}".format(separator, LEN_OUT_OUT*fix_csv_align)
- fmt += "{}{{:>{}}}".format(separator, LEN_IN_IN*fix_csv_align)
- fmt += "{}{{:>{}}}".format(separator, LEN_IN_OUT*fix_csv_align)
- return fmt
-
-
-def _fmt_body():
- separator, fix_csv_align = _prepare_fmt_sep()
- decimal_precision, time_precision = _prepare_fmt_precision()
- fmt = "{{}}{{:{}.{}f}}".format(LEN_SWITCHED_IN*fix_csv_align, decimal_precision)
- fmt += "{}{{:{}.{}f}}".format(
- separator, LEN_SWITCHED_OUT*fix_csv_align, decimal_precision
- )
- fmt += "{}{{:{}d}}".format(separator, LEN_CPU*fix_csv_align)
- fmt += "{}{{:{}d}}".format(separator, LEN_PID*fix_csv_align)
- fmt += "{}{{}}{{:{}d}}{{}}".format(separator, LEN_TID*fix_csv_align)
- fmt += "{}{{}}{{:>{}}}".format(separator, LEN_COMM*fix_csv_align)
- fmt += "{}{{:{}.{}f}}".format(separator, LEN_RUNTIME*fix_csv_align, time_precision)
- if args.extended_times:
- fmt += "{}{{:{}.{}f}}".format(separator, LEN_OUT_IN*fix_csv_align, time_precision)
- fmt += "{}{{:{}.{}f}}".format(separator, LEN_OUT_OUT*fix_csv_align, time_precision)
- fmt += "{}{{:{}.{}f}}".format(separator, LEN_IN_IN*fix_csv_align, time_precision)
- fmt += "{}{{:{}.{}f}}{{}}".format(
- separator, LEN_IN_OUT*fix_csv_align, time_precision
- )
- else:
- fmt += "{}{{:{}.{}f}}{{}}".format(
- separator, LEN_OUT_IN*fix_csv_align, time_precision
- )
- return fmt
-
-
-def _print_header():
- fmt = _fmt_header()
- header = ("Switched-In", "Switched-Out", "CPU", "PID", "TID", "Comm", "Runtime",
- "Time Out-In")
- if args.extended_times:
- header += ("Time Out-Out", "Time In-In", "Time In-Out")
- fd_task.write(fmt.format(*header) + "\n")
-
-
-
-def _print_task_finish(task):
- """calculating every entry of a row and printing it immediately"""
- c_row_set = ""
- c_row_reset = ""
- out_in = -1
- out_out = -1
- in_in = -1
- in_out = -1
- fmt = _fmt_body()
- # depending on user provided highlight option we change the color
- # for particular tasks
- if str(task.tid) in args.highlight_tasks_map:
- c_row_set = _COLORS[args.highlight_tasks_map[str(task.tid)]]
- c_row_reset = _COLORS["reset"]
- if task.comm in args.highlight_tasks_map:
- c_row_set = _COLORS[args.highlight_tasks_map[task.comm]]
- c_row_reset = _COLORS["reset"]
- # grey-out entries if PID == TID, they
- # are identical, no threaded model so the
- # thread id (tid) do not matter
- c_tid_set = ""
- c_tid_reset = ""
- if task.pid == task.tid:
- c_tid_set = _COLORS["grey"]
- c_tid_reset = _COLORS["reset"]
- if task.tid in db["tid"]:
- # get last task of tid
- last_tid_task = db["tid"][task.tid][-1]
- # feed the timespan calculate, last in tid db
- # and second the current one
- timespan_gap_tid = Timespans()
- timespan_gap_tid.feed(last_tid_task)
- timespan_gap_tid.feed(task)
- out_in = timespan_gap_tid.out_in
- out_out = timespan_gap_tid.out_out
- in_in = timespan_gap_tid.in_in
- in_out = timespan_gap_tid.in_out
-
-
- if args.extended_times:
- line_out = fmt.format(c_row_set, task.time_in(), task.time_out(), task.cpu,
- task.pid, c_tid_set, task.tid, c_tid_reset, c_row_set, task.comm,
- task.runtime(time_unit), out_in, out_out, in_in, in_out,
- c_row_reset) + "\n"
- else:
- line_out = fmt.format(c_row_set, task.time_in(), task.time_out(), task.cpu,
- task.pid, c_tid_set, task.tid, c_tid_reset, c_row_set, task.comm,
- task.runtime(time_unit), out_in, c_row_reset) + "\n"
- try:
- fd_task.write(line_out)
- except(IOError):
- # don't mangle the output if user SIGINT this script
- sys.exit()
-
-def _record_cleanup(_list):
- """
- no need to store more then one element if --summarize
- is not enabled
- """
- if not args.summary and len(_list) > 1:
- _list = _list[len(_list) - 1 :]
-
-
-def _record_by_tid(task):
- tid = task.tid
- if tid not in db["tid"]:
- db["tid"][tid] = []
- db["tid"][tid].append(task)
- _record_cleanup(db["tid"][tid])
-
-
-def _record_by_cpu(task):
- cpu = task.cpu
- if cpu not in db["cpu"]:
- db["cpu"][cpu] = []
- db["cpu"][cpu].append(task)
- _record_cleanup(db["cpu"][cpu])
-
-
-def _record_global(task):
- """record all executed task, ordered by finish chronological"""
- db["global"].append(task)
- _record_cleanup(db["global"])
-
-
-def _handle_task_finish(tid, cpu, time, perf_sample_dict):
- if tid == 0:
- return
- _id = _task_id(tid, cpu)
- if _id not in db["running"]:
- # may happen, if we missed the switch to
- # event. Seen in combination with --exclude-perf
- # where the start is filtered out, but not the
- # switched in. Probably a bug in exclude-perf
- # option.
- return
- task = db["running"][_id]
- task.schedule_out_at(time)
-
- # record tid, during schedule in the tid
- # is not available, update now
- pid = int(perf_sample_dict["sample"]["pid"])
-
- task.update_pid(pid)
- del db["running"][_id]
-
- # print only tasks which are not being filtered and no print of trace
- # for summary only, but record every task.
- if not _limit_filtered(tid, pid, task.comm) and not args.summary_only:
- _print_task_finish(task)
- _record_by_tid(task)
- _record_by_cpu(task)
- _record_global(task)
-
-
-def _handle_task_start(tid, cpu, comm, time):
- if tid == 0:
- return
- if tid in args.tid_renames:
- comm = args.tid_renames[tid]
- _id = _task_id(tid, cpu)
- if _id in db["running"]:
- # handle corner cases where already running tasks
- # are switched-to again - saw this via --exclude-perf
- # recorded traces. We simple ignore this "second start"
- # event.
- return
- assert _id not in db["running"]
- task = Task(_id, tid, cpu, comm)
- task.schedule_in_at(time)
- db["running"][_id] = task
-
-
-def _time_to_internal(time_ns):
- """
- To prevent float rounding errors we use Decimal internally
- """
- return decimal.Decimal(time_ns) / decimal.Decimal(1e9)
-
-
-def _limit_filtered(tid, pid, comm):
- if args.filter_tasks:
- if str(tid) in args.filter_tasks or comm in args.filter_tasks:
- return True
- else:
- return False
- if args.limit_to_tasks:
- if str(tid) in args.limit_to_tasks or comm in args.limit_to_tasks:
- return False
- else:
- return True
-
-
-def _argument_filter_sanity_check():
- if args.limit_to_tasks and args.filter_tasks:
- sys.exit("Error: Filter and Limit at the same time active.")
- if args.extended_times and args.summary_only:
- sys.exit("Error: Summary only and extended times active.")
- if args.time_limit and ":" not in args.time_limit:
- sys.exit(
- "Error: No bound set for time limit. Please set bound by ':' e.g :123."
- )
- if args.time_limit and (args.summary or args.summary_only or args.summary_extended):
- sys.exit("Error: Cannot set time limit and print summary")
- if args.csv_summary:
- args.summary = True
- if args.csv == args.csv_summary:
- sys.exit("Error: Chosen files for csv and csv summary are the same")
- if args.csv and (args.summary_extended or args.summary) and not args.csv_summary:
- sys.exit("Error: No file chosen to write summary to. Choose with --csv-summary "
- "<file>")
- if args.csv and args.summary_only:
- sys.exit("Error: --csv chosen and --summary-only. Standard task would not be"
- "written to csv file.")
-
-def _argument_prepare_check():
- global time_unit, fd_task, fd_sum
- if args.filter_tasks:
- args.filter_tasks = args.filter_tasks.split(",")
- if args.limit_to_tasks:
- args.limit_to_tasks = args.limit_to_tasks.split(",")
- if args.time_limit:
- args.time_limit = args.time_limit.split(":")
- for rename_tuple in args.rename_comms_by_tids.split(","):
- tid_name = rename_tuple.split(":")
- if len(tid_name) != 2:
- continue
- args.tid_renames[int(tid_name[0])] = tid_name[1]
- args.highlight_tasks_map = dict()
- for highlight_tasks_tuple in args.highlight_tasks.split(","):
- tasks_color_map = highlight_tasks_tuple.split(":")
- # default highlight color to red if no color set by user
- if len(tasks_color_map) == 1:
- tasks_color_map.append("red")
- if args.highlight_tasks and tasks_color_map[1].lower() not in _COLORS:
- sys.exit(
- "Error: Color not defined, please choose from grey,red,green,yellow,blue,"
- "violet"
- )
- if len(tasks_color_map) != 2:
- continue
- args.highlight_tasks_map[tasks_color_map[0]] = tasks_color_map[1]
- time_unit = "us"
- if args.ns:
- time_unit = "ns"
- elif args.ms:
- time_unit = "ms"
-
-
- fd_task = sys.stdout
- if args.csv:
- args.stdio_color = "never"
- fd_task = open(args.csv, "w")
- print("generating csv at",args.csv,)
-
- fd_sum = sys.stdout
- if args.csv_summary:
- args.stdio_color = "never"
- fd_sum = open(args.csv_summary, "w")
- print("generating csv summary at",args.csv_summary)
- if not args.csv:
- args.summary_only = True
-
-
-def _is_within_timelimit(time):
- """
- Check if a time limit was given by parameter, if so ignore the rest. If not,
- process the recorded trace in its entirety.
- """
- if not args.time_limit:
- return True
- lower_time_limit = args.time_limit[0]
- upper_time_limit = args.time_limit[1]
- # check for upper limit
- if upper_time_limit == "":
- if time >= decimal.Decimal(lower_time_limit):
- return True
- # check for lower limit
- if lower_time_limit == "":
- if time <= decimal.Decimal(upper_time_limit):
- return True
- # quit if time exceeds upper limit. Good for big datasets
- else:
- quit()
- if lower_time_limit != "" and upper_time_limit != "":
- if (time >= decimal.Decimal(lower_time_limit) and
- time <= decimal.Decimal(upper_time_limit)):
- return True
- # quit if time exceeds upper limit. Good for big datasets
- elif time > decimal.Decimal(upper_time_limit):
- quit()
-
-def _prepare_fmt_precision():
- decimal_precision = 6
- time_precision = 3
- if args.ns:
- decimal_precision = 9
- time_precision = 0
- return decimal_precision, time_precision
-
-def _prepare_fmt_sep():
- separator = " "
- fix_csv_align = 1
- if args.csv or args.csv_summary:
- separator = ";"
- fix_csv_align = 0
- return separator, fix_csv_align
-
-def trace_unhandled(event_name, context, event_fields_dict, perf_sample_dict):
- pass
-
-
-def trace_begin():
- _parse_args()
- _check_color()
- _init_db()
- if not args.summary_only:
- _print_header()
-
-def trace_end():
- if args.summary or args.summary_extended or args.summary_only:
- Summary().print()
-
-def sched__sched_switch(event_name, context, common_cpu, common_secs, common_nsecs,
- common_pid, common_comm, common_callchain, prev_comm,
- prev_pid, prev_prio, prev_state, next_comm, next_pid,
- next_prio, perf_sample_dict):
- # ignore common_secs & common_nsecs cause we need
- # high res timestamp anyway, using the raw value is
- # faster
- time = _time_to_internal(perf_sample_dict["sample"]["time"])
- if not _is_within_timelimit(time):
- # user specific --time-limit a:b set
- return
-
- next_comm = _filter_non_printable(next_comm)
- _handle_task_finish(prev_pid, common_cpu, time, perf_sample_dict)
- _handle_task_start(next_pid, common_cpu, next_comm, time)
diff --git a/tools/perf/tests/shell/script.sh b/tools/perf/tests/shell/script.sh
index 254fc3ae94e7..902343f7a83c 100755
--- a/tools/perf/tests/shell/script.sh
+++ b/tools/perf/tests/shell/script.sh
@@ -6,8 +6,6 @@ set -e
temp_dir=$(mktemp -d /tmp/perf-test-script.XXXXXXXXXX)
-perfdatafile="${temp_dir}/perf.data"
-db_test="${temp_dir}/db_test.py"
err=0
@@ -31,42 +29,6 @@ trap_cleanup()
trap trap_cleanup EXIT TERM INT
-test_db()
-{
- echo "DB test"
-
- # Check if python script is supported
- if perf version --build-options | grep python | grep -q OFF ; then
- echo "SKIP: python scripting is not supported"
- err=2
- return
- fi
-
- cat << "_end_of_file_" > "${db_test}"
-perf_db_export_mode = True
-perf_db_export_calls = False
-perf_db_export_callchains = True
-
-def sample_table(*args):
- print(f'sample_table({args})')
-
-def call_path_table(*args):
- print(f'call_path_table({args}')
-_end_of_file_
- case $(uname -m)
- in s390x)
- cmd_flags="--call-graph dwarf -e cpu-clock";;
- *)
- cmd_flags="-g";;
- esac
-
- perf record $cmd_flags -o "${perfdatafile}" true
- # Disable lsan to avoid warnings about python memory leaks.
- export ASAN_OPTIONS=detect_leaks=0
- perf script -i "${perfdatafile}" -s "${db_test}"
- export ASAN_OPTIONS=
- echo "DB test [Success]"
-}
test_parallel_perf()
{
@@ -91,7 +53,6 @@ test_parallel_perf()
echo "parallel-perf test [Success]"
}
-test_db
test_parallel_perf
cleanup
diff --git a/tools/perf/tests/shell/script_python.sh b/tools/perf/tests/shell/script_python.sh
deleted file mode 100755
index 6bc66074a31f..000000000000
--- a/tools/perf/tests/shell/script_python.sh
+++ /dev/null
@@ -1,113 +0,0 @@
-#!/bin/bash
-# perf script python tests
-# SPDX-License-Identifier: GPL-2.0
-
-set -e
-
-# set PERF_EXEC_PATH to find scripts in the source directory
-perfdir=$(dirname "$0")/../..
-if [ -e "$perfdir/scripts/python/Perf-Trace-Util" ]; then
- export PERF_EXEC_PATH=$perfdir
-fi
-
-
-perfdata=$(mktemp /tmp/__perf_test_script_python.perf.data.XXXXX)
-generated_script=$(mktemp /tmp/__perf_test_script.XXXXX.py)
-
-cleanup() {
- rm -f "${perfdata}"
- rm -f "${generated_script}"
- trap - EXIT TERM INT
-}
-
-trap_cleanup() {
- echo "Unexpected signal in ${FUNCNAME[1]}"
- cleanup
- exit 1
-}
-trap trap_cleanup TERM INT
-trap cleanup EXIT
-
-check_python_support() {
- if perf check feature -q libpython; then
- return 0
- fi
- echo "perf script python test [Skipped: no libpython support]"
- return 2
-}
-
-test_script() {
- local event_name=$1
- local expected_output=$2
- local record_opts=$3
-
- echo "Testing event: $event_name"
-
- # Try to record. If this fails, it might be permissions or lack of
- # support. Return 2 to indicate "skip this event" rather than "fail
- # test".
- if ! perf record -o "${perfdata}" -e "$event_name" $record_opts -- perf test -w thloop > /dev/null 2>&1; then
- echo "perf script python test [Skipped: failed to record $event_name]"
- return 2
- fi
-
- echo "Generating python script..."
- if ! perf script -i "${perfdata}" -g "${generated_script}"; then
- echo "perf script python test [Failed: script generation for $event_name]"
- return 1
- fi
-
- if [ ! -f "${generated_script}" ]; then
- echo "perf script python test [Failed: script not generated for $event_name]"
- return 1
- fi
-
- # Perf script -g python doesn't generate process_event for generic
- # events so append it manually to test that the callback works.
- if ! grep -q "def process_event" "${generated_script}"; then
- cat <<EOF >> "${generated_script}"
-
-def process_event(param_dict):
- print("param_dict: %s" % param_dict)
-EOF
- fi
-
- echo "Executing python script..."
- output=$(perf script -i "${perfdata}" -s "${generated_script}" 2>&1)
-
- if echo "$output" | grep -q "$expected_output"; then
- echo "perf script python test [Success: $event_name triggered $expected_output]"
- return 0
- else
- echo "perf script python test [Failed: $event_name did not trigger $expected_output]"
- echo "Output was:"
- echo "$output" | head -n 20
- return 1
- fi
-}
-
-check_python_support || exit 2
-
-# Try tracepoint first
-test_script "sched:sched_switch" "sched__sched_switch" "-c 1" && res=0 || res=$?
-
-if [ $res -eq 0 ]; then
- exit 0
-elif [ $res -eq 1 ]; then
- exit 1
-fi
-
-# If tracepoint skipped (res=2), try task-clock
-# For generic events like task-clock, the generated script uses process_event()
-# which prints the param_dict.
-test_script "task-clock" "param_dict" "-c 100" && res=0 || res=$?
-
-if [ $res -eq 0 ]; then
- exit 0
-elif [ $res -eq 1 ]; then
- exit 1
-fi
-
-# If both skipped
-echo "perf script python test [Skipped: Could not record tracepoint or task-clock]"
-exit 2
diff --git a/tools/perf/util/scripting-engines/Build b/tools/perf/util/scripting-engines/Build
index 24f087b0cd11..3f1dc10526f8 100644
--- a/tools/perf/util/scripting-engines/Build
+++ b/tools/perf/util/scripting-engines/Build
@@ -1,9 +1,5 @@
ifeq ($(CONFIG_LIBTRACEEVENT),y)
perf-util-$(CONFIG_LIBPERL) += trace-event-perl.o
endif
-perf-util-$(CONFIG_LIBPYTHON) += trace-event-python.o
CFLAGS_trace-event-perl.o += $(PERL_EMBED_CCOPTS) -Wno-redundant-decls -Wno-strict-prototypes -Wno-unused-parameter -Wno-shadow -Wno-nested-externs -Wno-undef -Wno-switch-default -Wno-bad-function-cast -Wno-declaration-after-statement -Wno-switch-enum -Wno-thread-safety-analysis
-
-# -Wno-declaration-after-statement: The python headers have mixed code with declarations (decls after asserts, for instance)
-CFLAGS_trace-event-python.o += $(PYTHON_EMBED_CCOPTS) -Wno-redundant-decls -Wno-strict-prototypes -Wno-unused-parameter -Wno-shadow -Wno-deprecated-declarations -Wno-switch-enum -Wno-declaration-after-statement
diff --git a/tools/perf/util/scripting-engines/trace-event-python.c b/tools/perf/util/scripting-engines/trace-event-python.c
deleted file mode 100644
index 239104b0337d..000000000000
--- a/tools/perf/util/scripting-engines/trace-event-python.c
+++ /dev/null
@@ -1,2333 +0,0 @@
-/*
- * trace-event-python. Feed trace events to an embedded Python interpreter.
- *
- * Copyright (C) 2010 Tom Zanussi <tzanussi@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#include <Python.h>
-
-#include <inttypes.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <stdbool.h>
-#include <errno.h>
-#include <linux/bitmap.h>
-#include <linux/compiler.h>
-#include <linux/time64.h>
-#ifdef HAVE_LIBTRACEEVENT
-#include <event-parse.h>
-#endif
-
-#include "../build-id.h"
-#include "../counts.h"
-#include "../debug.h"
-#include "../dso.h"
-#include "../callchain.h"
-#include "../env.h"
-#include "../evsel.h"
-#include "../event.h"
-#include "../thread.h"
-#include "../comm.h"
-#include "../machine.h"
-#include "../mem-info.h"
-#include "../db-export.h"
-#include "../thread-stack.h"
-#include "../trace-event.h"
-#include "../call-path.h"
-#include "dwarf-regs.h"
-#include "map.h"
-#include "symbol.h"
-#include "thread_map.h"
-#include "print_binary.h"
-#include "stat.h"
-#include "mem-events.h"
-#include "util/perf_regs.h"
-
-#define _PyUnicode_FromString(arg) \
- PyUnicode_FromString(arg)
-#define _PyUnicode_FromStringAndSize(arg1, arg2) \
- PyUnicode_FromStringAndSize((arg1), (arg2))
-#define _PyBytes_FromStringAndSize(arg1, arg2) \
- PyBytes_FromStringAndSize((arg1), (arg2))
-#define _PyLong_FromLong(arg) \
- PyLong_FromLong(arg)
-#define _PyLong_AsLong(arg) \
- PyLong_AsLong(arg)
-#define _PyCapsule_New(arg1, arg2, arg3) \
- PyCapsule_New((arg1), (arg2), (arg3))
-
-PyMODINIT_FUNC PyInit_perf_trace_context(void);
-
-#ifdef HAVE_LIBTRACEEVENT
-#define TRACE_EVENT_TYPE_MAX \
- ((1 << (sizeof(unsigned short) * 8)) - 1)
-
-#define N_COMMON_FIELDS 7
-
-static char *cur_field_name;
-static int zero_flag_atom;
-#endif
-
-#define MAX_FIELDS 64
-
-extern struct scripting_context *scripting_context;
-
-static PyObject *main_module, *main_dict;
-
-struct tables {
- struct db_export dbe;
- PyObject *evsel_handler;
- PyObject *machine_handler;
- PyObject *thread_handler;
- PyObject *comm_handler;
- PyObject *comm_thread_handler;
- PyObject *dso_handler;
- PyObject *symbol_handler;
- PyObject *branch_type_handler;
- PyObject *sample_handler;
- PyObject *call_path_handler;
- PyObject *call_return_handler;
- PyObject *synth_handler;
- PyObject *context_switch_handler;
- bool db_export_mode;
-};
-
-static struct tables tables_global;
-
-static void handler_call_die(const char *handler_name) __noreturn;
-static void handler_call_die(const char *handler_name)
-{
- PyErr_Print();
- Py_FatalError("problem in Python trace event handler");
- // Py_FatalError does not return
- // but we have to make the compiler happy
- abort();
-}
-
-/*
- * Insert val into the dictionary and decrement the reference counter.
- * This is necessary for dictionaries since PyDict_SetItemString() does not
- * steal a reference, as opposed to PyTuple_SetItem().
- */
-static void pydict_set_item_string_decref(PyObject *dict, const char *key, PyObject *val)
-{
- PyDict_SetItemString(dict, key, val);
- Py_DECREF(val);
-}
-
-static PyObject *get_handler(const char *handler_name)
-{
- PyObject *handler;
-
- handler = PyDict_GetItemString(main_dict, handler_name);
- if (handler && !PyCallable_Check(handler))
- return NULL;
- return handler;
-}
-
-static void call_object(PyObject *handler, PyObject *args, const char *die_msg)
-{
- PyObject *retval;
-
- retval = PyObject_CallObject(handler, args);
- if (retval == NULL)
- handler_call_die(die_msg);
- Py_DECREF(retval);
-}
-
-static void try_call_object(const char *handler_name, PyObject *args)
-{
- PyObject *handler;
-
- handler = get_handler(handler_name);
- if (handler)
- call_object(handler, args, handler_name);
-}
-
-#ifdef HAVE_LIBTRACEEVENT
-static int get_argument_count(PyObject *handler)
-{
- int arg_count = 0;
-
- PyObject *code_obj = PyObject_GetAttrString(handler, "__code__");
- PyErr_Clear();
- if (code_obj) {
- PyObject *arg_count_obj = PyObject_GetAttrString(code_obj,
- "co_argcount");
- if (arg_count_obj) {
- arg_count = (int) _PyLong_AsLong(arg_count_obj);
- Py_DECREF(arg_count_obj);
- }
- Py_DECREF(code_obj);
- }
- return arg_count;
-}
-
-static void define_value(enum tep_print_arg_type field_type,
- const char *ev_name,
- const char *field_name,
- const char *field_value,
- const char *field_str)
-{
- const char *handler_name = "define_flag_value";
- PyObject *t;
- unsigned long long value;
- unsigned n = 0;
-
- if (field_type == TEP_PRINT_SYMBOL)
- handler_name = "define_symbolic_value";
-
- t = PyTuple_New(4);
- if (!t)
- Py_FatalError("couldn't create Python tuple");
-
- value = eval_flag(field_value);
-
- PyTuple_SetItem(t, n++, _PyUnicode_FromString(ev_name));
- PyTuple_SetItem(t, n++, _PyUnicode_FromString(field_name));
- PyTuple_SetItem(t, n++, _PyLong_FromLong(value));
- PyTuple_SetItem(t, n++, _PyUnicode_FromString(field_str));
-
- try_call_object(handler_name, t);
-
- Py_DECREF(t);
-}
-
-static void define_values(enum tep_print_arg_type field_type,
- struct tep_print_flag_sym *field,
- const char *ev_name,
- const char *field_name)
-{
- define_value(field_type, ev_name, field_name, field->value,
- field->str);
-
- if (field->next)
- define_values(field_type, field->next, ev_name, field_name);
-}
-
-static void define_field(enum tep_print_arg_type field_type,
- const char *ev_name,
- const char *field_name,
- const char *delim)
-{
- const char *handler_name = "define_flag_field";
- PyObject *t;
- unsigned n = 0;
-
- if (field_type == TEP_PRINT_SYMBOL)
- handler_name = "define_symbolic_field";
-
- if (field_type == TEP_PRINT_FLAGS)
- t = PyTuple_New(3);
- else
- t = PyTuple_New(2);
- if (!t)
- Py_FatalError("couldn't create Python tuple");
-
- PyTuple_SetItem(t, n++, _PyUnicode_FromString(ev_name));
- PyTuple_SetItem(t, n++, _PyUnicode_FromString(field_name));
- if (field_type == TEP_PRINT_FLAGS)
- PyTuple_SetItem(t, n++, _PyUnicode_FromString(delim));
-
- try_call_object(handler_name, t);
-
- Py_DECREF(t);
-}
-
-static void define_event_symbols(struct tep_event *event,
- const char *ev_name,
- struct tep_print_arg *args)
-{
- if (args == NULL)
- return;
-
- switch (args->type) {
- case TEP_PRINT_NULL:
- break;
- case TEP_PRINT_ATOM:
- define_value(TEP_PRINT_FLAGS, ev_name, cur_field_name, "0",
- args->atom.atom);
- zero_flag_atom = 0;
- break;
- case TEP_PRINT_FIELD:
- free(cur_field_name);
- cur_field_name = strdup(args->field.name);
- break;
- case TEP_PRINT_FLAGS:
- define_event_symbols(event, ev_name, args->flags.field);
- define_field(TEP_PRINT_FLAGS, ev_name, cur_field_name,
- args->flags.delim);
- define_values(TEP_PRINT_FLAGS, args->flags.flags, ev_name,
- cur_field_name);
- break;
- case TEP_PRINT_SYMBOL:
- define_event_symbols(event, ev_name, args->symbol.field);
- define_field(TEP_PRINT_SYMBOL, ev_name, cur_field_name, NULL);
- define_values(TEP_PRINT_SYMBOL, args->symbol.symbols, ev_name,
- cur_field_name);
- break;
- case TEP_PRINT_HEX:
- case TEP_PRINT_HEX_STR:
- define_event_symbols(event, ev_name, args->hex.field);
- define_event_symbols(event, ev_name, args->hex.size);
- break;
- case TEP_PRINT_INT_ARRAY:
- define_event_symbols(event, ev_name, args->int_array.field);
- define_event_symbols(event, ev_name, args->int_array.count);
- define_event_symbols(event, ev_name, args->int_array.el_size);
- break;
- case TEP_PRINT_STRING:
- break;
- case TEP_PRINT_TYPE:
- define_event_symbols(event, ev_name, args->typecast.item);
- break;
- case TEP_PRINT_OP:
- if (strcmp(args->op.op, ":") == 0)
- zero_flag_atom = 1;
- define_event_symbols(event, ev_name, args->op.left);
- define_event_symbols(event, ev_name, args->op.right);
- break;
- default:
- /* gcc warns for these? */
- case TEP_PRINT_BSTRING:
- case TEP_PRINT_DYNAMIC_ARRAY:
- case TEP_PRINT_DYNAMIC_ARRAY_LEN:
- case TEP_PRINT_FUNC:
- case TEP_PRINT_BITMASK:
- /* we should warn... */
- return;
- }
-
- if (args->next)
- define_event_symbols(event, ev_name, args->next);
-}
-
-static PyObject *get_field_numeric_entry(struct tep_event *event,
- struct tep_format_field *field, void *data)
-{
- bool is_array = field->flags & TEP_FIELD_IS_ARRAY;
- PyObject *obj = NULL, *list = NULL;
- unsigned long long val;
- unsigned int item_size, n_items, i;
-
- if (is_array) {
- list = PyList_New(field->arraylen);
- if (!list)
- Py_FatalError("couldn't create Python list");
- item_size = field->size / field->arraylen;
- n_items = field->arraylen;
- } else {
- item_size = field->size;
- n_items = 1;
- }
-
- for (i = 0; i < n_items; i++) {
-
- val = read_size(event, data + field->offset + i * item_size,
- item_size);
- if (field->flags & TEP_FIELD_IS_SIGNED) {
- if ((long long)val >= LONG_MIN &&
- (long long)val <= LONG_MAX)
- obj = _PyLong_FromLong(val);
- else
- obj = PyLong_FromLongLong(val);
- } else {
- if (val <= LONG_MAX)
- obj = _PyLong_FromLong(val);
- else
- obj = PyLong_FromUnsignedLongLong(val);
- }
- if (is_array)
- PyList_SET_ITEM(list, i, obj);
- }
- if (is_array)
- obj = list;
- return obj;
-}
-#endif
-
-static const char *get_dsoname(struct map *map)
-{
- const char *dsoname = "[unknown]";
- struct dso *dso = map ? map__dso(map) : NULL;
-
- if (dso) {
- if (symbol_conf.show_kernel_path && dso__long_name(dso))
- dsoname = dso__long_name(dso);
- else
- dsoname = dso__name(dso);
- }
-
- return dsoname;
-}
-
-static unsigned long get_offset(struct symbol *sym, struct addr_location *al)
-{
- unsigned long offset;
-
- if (al->addr < sym->end)
- offset = al->addr - sym->start;
- else
- offset = al->addr - map__start(al->map) - sym->start;
-
- return offset;
-}
-
-static PyObject *python_process_callchain(struct perf_sample *sample,
- struct addr_location *al)
-{
- PyObject *pylist;
- struct callchain_cursor *cursor;
-
- pylist = PyList_New(0);
- if (!pylist)
- Py_FatalError("couldn't create Python list");
-
- if (!symbol_conf.use_callchain || !sample->callchain)
- goto exit;
-
- cursor = get_tls_callchain_cursor();
- if (thread__resolve_callchain(al->thread, cursor,
- sample, NULL, NULL,
- scripting_max_stack) != 0) {
- pr_err("Failed to resolve callchain. Skipping\n");
- goto exit;
- }
- callchain_cursor_commit(cursor);
-
-
- while (1) {
- PyObject *pyelem;
- struct callchain_cursor_node *node;
- node = callchain_cursor_current(cursor);
- if (!node)
- break;
-
- pyelem = PyDict_New();
- if (!pyelem)
- Py_FatalError("couldn't create Python dictionary");
-
-
- pydict_set_item_string_decref(pyelem, "ip",
- PyLong_FromUnsignedLongLong(node->ip));
-
- if (node->ms.sym) {
- PyObject *pysym = PyDict_New();
- if (!pysym)
- Py_FatalError("couldn't create Python dictionary");
- pydict_set_item_string_decref(pysym, "start",
- PyLong_FromUnsignedLongLong(node->ms.sym->start));
- pydict_set_item_string_decref(pysym, "end",
- PyLong_FromUnsignedLongLong(node->ms.sym->end));
- pydict_set_item_string_decref(pysym, "binding",
- _PyLong_FromLong(symbol__binding(node->ms.sym)));
- pydict_set_item_string_decref(pysym, "name",
- _PyUnicode_FromStringAndSize(node->ms.sym->name,
- node->ms.sym->namelen));
- pydict_set_item_string_decref(pyelem, "sym", pysym);
-
- if (node->ms.map) {
- struct map *map = node->ms.map;
- struct addr_location node_al;
- unsigned long offset;
-
- addr_location__init(&node_al);
- node_al.addr = map__map_ip(map, node->ip);
- node_al.map = map__get(map);
- offset = get_offset(node->ms.sym, &node_al);
- addr_location__exit(&node_al);
-
- pydict_set_item_string_decref(
- pyelem, "sym_off",
- PyLong_FromUnsignedLongLong(offset));
- }
- if (node->srcline && strcmp(":0", node->srcline)) {
- pydict_set_item_string_decref(
- pyelem, "sym_srcline",
- _PyUnicode_FromString(node->srcline));
- }
- }
-
- if (node->ms.map) {
- const char *dsoname = get_dsoname(node->ms.map);
-
- pydict_set_item_string_decref(pyelem, "dso",
- _PyUnicode_FromString(dsoname));
- }
-
- callchain_cursor_advance(cursor);
- PyList_Append(pylist, pyelem);
- Py_DECREF(pyelem);
- }
-
-exit:
- return pylist;
-}
-
-static PyObject *python_process_brstack(struct perf_sample *sample,
- struct thread *thread)
-{
- struct branch_stack *br = sample->branch_stack;
- struct branch_entry *entries = perf_sample__branch_entries(sample);
- PyObject *pylist;
- u64 i;
-
- pylist = PyList_New(0);
- if (!pylist)
- Py_FatalError("couldn't create Python list");
-
- if (!(br && br->nr))
- goto exit;
-
- for (i = 0; i < br->nr; i++) {
- PyObject *pyelem;
- struct addr_location al;
- const char *dsoname;
-
- pyelem = PyDict_New();
- if (!pyelem)
- Py_FatalError("couldn't create Python dictionary");
-
- pydict_set_item_string_decref(pyelem, "from",
- PyLong_FromUnsignedLongLong(entries[i].from));
- pydict_set_item_string_decref(pyelem, "to",
- PyLong_FromUnsignedLongLong(entries[i].to));
- pydict_set_item_string_decref(pyelem, "mispred",
- PyBool_FromLong(entries[i].flags.mispred));
- pydict_set_item_string_decref(pyelem, "predicted",
- PyBool_FromLong(entries[i].flags.predicted));
- pydict_set_item_string_decref(pyelem, "in_tx",
- PyBool_FromLong(entries[i].flags.in_tx));
- pydict_set_item_string_decref(pyelem, "abort",
- PyBool_FromLong(entries[i].flags.abort));
- pydict_set_item_string_decref(pyelem, "cycles",
- PyLong_FromUnsignedLongLong(entries[i].flags.cycles));
-
- addr_location__init(&al);
- thread__find_map_fb(thread, sample->cpumode,
- entries[i].from, &al);
- dsoname = get_dsoname(al.map);
- pydict_set_item_string_decref(pyelem, "from_dsoname",
- _PyUnicode_FromString(dsoname));
-
- thread__find_map_fb(thread, sample->cpumode,
- entries[i].to, &al);
- dsoname = get_dsoname(al.map);
- pydict_set_item_string_decref(pyelem, "to_dsoname",
- _PyUnicode_FromString(dsoname));
-
- addr_location__exit(&al);
- PyList_Append(pylist, pyelem);
- Py_DECREF(pyelem);
- }
-
-exit:
- return pylist;
-}
-
-static int get_symoff(struct symbol *sym, struct addr_location *al,
- bool print_off, char *bf, int size)
-{
- unsigned long offset;
-
- if (!sym || !sym->name[0])
- return scnprintf(bf, size, "%s", "[unknown]");
-
- if (!print_off)
- return scnprintf(bf, size, "%s", sym->name);
-
- offset = get_offset(sym, al);
-
- return scnprintf(bf, size, "%s+0x%x", sym->name, offset);
-}
-
-static int get_br_mspred(struct branch_flags *flags, char *bf, int size)
-{
- if (!flags->mispred && !flags->predicted)
- return scnprintf(bf, size, "%s", "-");
-
- if (flags->mispred)
- return scnprintf(bf, size, "%s", "M");
-
- return scnprintf(bf, size, "%s", "P");
-}
-
-static PyObject *python_process_brstacksym(struct perf_sample *sample,
- struct thread *thread)
-{
- struct branch_stack *br = sample->branch_stack;
- struct branch_entry *entries = perf_sample__branch_entries(sample);
- PyObject *pylist;
- u64 i;
- char bf[512];
-
- pylist = PyList_New(0);
- if (!pylist)
- Py_FatalError("couldn't create Python list");
-
- if (!(br && br->nr))
- goto exit;
-
- for (i = 0; i < br->nr; i++) {
- PyObject *pyelem;
- struct addr_location al;
-
- addr_location__init(&al);
- pyelem = PyDict_New();
- if (!pyelem)
- Py_FatalError("couldn't create Python dictionary");
-
- thread__find_symbol_fb(thread, sample->cpumode,
- entries[i].from, &al);
- get_symoff(al.sym, &al, true, bf, sizeof(bf));
- pydict_set_item_string_decref(pyelem, "from",
- _PyUnicode_FromString(bf));
-
- thread__find_symbol_fb(thread, sample->cpumode,
- entries[i].to, &al);
- get_symoff(al.sym, &al, true, bf, sizeof(bf));
- pydict_set_item_string_decref(pyelem, "to",
- _PyUnicode_FromString(bf));
-
- get_br_mspred(&entries[i].flags, bf, sizeof(bf));
- pydict_set_item_string_decref(pyelem, "pred",
- _PyUnicode_FromString(bf));
-
- if (entries[i].flags.in_tx) {
- pydict_set_item_string_decref(pyelem, "in_tx",
- _PyUnicode_FromString("X"));
- } else {
- pydict_set_item_string_decref(pyelem, "in_tx",
- _PyUnicode_FromString("-"));
- }
-
- if (entries[i].flags.abort) {
- pydict_set_item_string_decref(pyelem, "abort",
- _PyUnicode_FromString("A"));
- } else {
- pydict_set_item_string_decref(pyelem, "abort",
- _PyUnicode_FromString("-"));
- }
-
- PyList_Append(pylist, pyelem);
- Py_DECREF(pyelem);
- addr_location__exit(&al);
- }
-
-exit:
- return pylist;
-}
-
-static PyObject *get_sample_value_as_tuple(struct sample_read_value *value,
- u64 read_format)
-{
- PyObject *t;
-
- t = PyTuple_New(3);
- if (!t)
- Py_FatalError("couldn't create Python tuple");
- PyTuple_SetItem(t, 0, PyLong_FromUnsignedLongLong(value->id));
- PyTuple_SetItem(t, 1, PyLong_FromUnsignedLongLong(value->value));
- if (read_format & PERF_FORMAT_LOST)
- PyTuple_SetItem(t, 2, PyLong_FromUnsignedLongLong(value->lost));
-
- return t;
-}
-
-static void set_sample_read_in_dict(PyObject *dict_sample, struct perf_sample *sample)
-{
- u64 read_format = sample->evsel->core.attr.read_format;
- PyObject *values;
- unsigned int i;
-
- if (read_format & PERF_FORMAT_TOTAL_TIME_ENABLED) {
- pydict_set_item_string_decref(dict_sample, "time_enabled",
- PyLong_FromUnsignedLongLong(sample->read.time_enabled));
- }
-
- if (read_format & PERF_FORMAT_TOTAL_TIME_RUNNING) {
- pydict_set_item_string_decref(dict_sample, "time_running",
- PyLong_FromUnsignedLongLong(sample->read.time_running));
- }
-
- if (read_format & PERF_FORMAT_GROUP)
- values = PyList_New(sample->read.group.nr);
- else
- values = PyList_New(1);
-
- if (!values)
- Py_FatalError("couldn't create Python list");
-
- if (read_format & PERF_FORMAT_GROUP) {
- struct sample_read_value *v = sample->read.group.values;
-
- i = 0;
- sample_read_group__for_each(v, sample->read.group.nr, read_format) {
- PyObject *t = get_sample_value_as_tuple(v, read_format);
- PyList_SET_ITEM(values, i, t);
- i++;
- }
- } else {
- PyObject *t = get_sample_value_as_tuple(&sample->read.one,
- read_format);
- PyList_SET_ITEM(values, 0, t);
- }
- pydict_set_item_string_decref(dict_sample, "values", values);
-}
-
-static void set_sample_datasrc_in_dict(PyObject *dict,
- struct perf_sample *sample)
-{
- struct mem_info *mi = mem_info__new();
- char decode[100];
-
- if (!mi)
- Py_FatalError("couldn't create mem-info");
-
- pydict_set_item_string_decref(dict, "datasrc",
- PyLong_FromUnsignedLongLong(sample->data_src));
-
- mem_info__data_src(mi)->val = sample->data_src;
- perf_script__meminfo_scnprintf(decode, 100, mi);
- mem_info__put(mi);
-
- pydict_set_item_string_decref(dict, "datasrc_decode",
- _PyUnicode_FromString(decode));
-}
-
-static int regs_map(struct regs_dump *regs, uint64_t mask, uint16_t e_machine,
- uint32_t e_flags, char *bf, int size)
-{
- unsigned int i = 0, r;
- int printed = 0;
-
- bf[0] = 0;
-
- if (size <= 0)
- return 0;
-
- if (!regs || !regs->regs)
- return 0;
-
- for_each_set_bit(r, (unsigned long *) &mask, sizeof(mask) * 8) {
- u64 val = regs->regs[i++];
-
- printed += scnprintf(bf + printed, size - printed,
- "%5s:0x%" PRIx64 " ",
- perf_reg_name(r, e_machine, e_flags, regs->abi), val);
- }
-
- return printed;
-}
-
-static void simd_regs_map(struct regs_dump *regs, struct perf_event_attr *attr,
- uint16_t e_machine, char *bf, int size, int printed, bool intr)
-{
- const char *name = "unknown";
- int reg_c, idx, pred_base;
- unsigned int i = 0, j;
- uint16_t qwords;
-
- if (size <= 0 || printed >= size)
- return;
-
- if (!regs || !regs->simd_data)
- return;
-
- if (!regs->nr_vectors && !regs->nr_pred)
- return;
-
- for (reg_c = 0; reg_c < 64; reg_c++) {
- if (!regs->nr_vectors)
- break;
- if (intr) {
- perf_intr_simd_reg_class_bitmap_qwords(e_machine, reg_c,
- &qwords, /*pred=*/false);
- } else {
- perf_user_simd_reg_class_bitmap_qwords(e_machine, reg_c,
- &qwords, /*pred=*/false);
- }
- if (regs->vector_qwords == qwords) {
- name = perf_simd_reg_class_name(e_machine, reg_c, /*pred=*/false);
- break;
- }
- }
-
- for (i = 0; i < regs->nr_vectors; i++) {
- for (j = 0; j < regs->vector_qwords; j++) {
- idx = i * regs->vector_qwords + j;
- if (regs->vector_qwords > 1) {
- printed += scnprintf(bf + printed, size - printed,
- "%5s[%d][%d]:0x%" PRIx64 " ",
- name, i, j, regs->simd_data[idx]);
- } else {
- printed += scnprintf(bf + printed, size - printed,
- "%5s[%d]:0x%" PRIx64 " ",
- name, i, regs->simd_data[idx]);
- }
- }
- }
-
- name = "unknown";
- for (reg_c = 0; reg_c < 64; reg_c++) {
- if (!regs->nr_pred)
- break;
- if (intr) {
- perf_intr_simd_reg_class_bitmap_qwords(e_machine, reg_c,
- &qwords, /*pred=*/true);
- } else {
- perf_user_simd_reg_class_bitmap_qwords(e_machine, reg_c,
- &qwords, /*pred=*/true);
- }
- if (regs->pred_qwords == qwords) {
- name = perf_simd_reg_class_name(e_machine, reg_c, /*pred=*/true);
- break;
- }
- }
-
- pred_base = regs->nr_vectors * regs->vector_qwords;
- for (i = 0; i < regs->nr_pred; i++) {
- for (j = 0; j < regs->pred_qwords; j++) {
- idx = pred_base + i * regs->pred_qwords + j;
- if (regs->pred_qwords > 1) {
- printed += scnprintf(bf + printed, size - printed,
- "%5s[%d][%d]:0x%" PRIx64 " ",
- name, i, j, regs->simd_data[idx]);
- } else {
- printed += scnprintf(bf + printed, size - printed,
- "%5s[%d]:0x%" PRIx64 " ",
- name, i, regs->simd_data[idx]);
- }
- }
- }
-}
-
-#define MAX_REG_SIZE 128
-
-static int set_regs_in_dict(PyObject *dict,
- struct perf_sample *sample,
- uint16_t e_machine,
- uint32_t e_flags)
-{
- struct perf_event_attr *attr = &sample->evsel->core.attr;
- int intr_size, user_size, size;
- struct regs_dump *regs;
- char *bf = NULL;
- int printed;
-
- intr_size = (__sw_hweight64(attr->sample_regs_intr) * MAX_REG_SIZE) + 1;
- user_size = (__sw_hweight64(attr->sample_regs_user) * MAX_REG_SIZE) + 1;
- if (sample->intr_regs && attr->sample_simd_regs_enabled) {
- regs = sample->intr_regs;
- intr_size += (regs->nr_vectors * regs->vector_qwords +
- regs->nr_pred * regs->pred_qwords) * MAX_REG_SIZE;
- }
- if (sample->user_regs && attr->sample_simd_regs_enabled) {
- regs = sample->user_regs;
- user_size += (regs->nr_vectors * regs->vector_qwords +
- regs->nr_pred * regs->pred_qwords) * MAX_REG_SIZE;
- }
- size = intr_size > user_size ? intr_size : user_size;
-
- if (sample->intr_regs) {
- bf = malloc(size);
- if (!bf)
- return -1;
-
- printed = regs_map(sample->intr_regs, attr->sample_regs_intr,
- e_machine, e_flags, bf, size);
- if (attr->sample_simd_regs_enabled) {
- simd_regs_map(sample->intr_regs, attr, e_machine, bf,
- size, printed, true);
- }
-
- pydict_set_item_string_decref(dict, "iregs",
- _PyUnicode_FromString(bf));
- }
-
- if (sample->user_regs) {
- if (!bf) {
- bf = malloc(size);
- if (!bf)
- return -1;
- }
- printed = regs_map(sample->user_regs, attr->sample_regs_user,
- e_machine, e_flags, bf, size);
- if (attr->sample_simd_regs_enabled) {
- simd_regs_map(sample->user_regs, attr, e_machine, bf,
- size, printed, false);
- }
-
- pydict_set_item_string_decref(dict, "uregs",
- _PyUnicode_FromString(bf));
- }
- free(bf);
-
- return 0;
-}
-
-static void set_sym_in_dict(PyObject *dict, struct addr_location *al,
- const char *dso_field, const char *dso_bid_field,
- const char *dso_map_start, const char *dso_map_end,
- const char *sym_field, const char *symoff_field,
- const char *map_pgoff)
-{
- if (al->map) {
- char sbuild_id[SBUILD_ID_SIZE];
- struct dso *dso = map__dso(al->map);
-
- pydict_set_item_string_decref(dict, dso_field,
- _PyUnicode_FromString(dso__name(dso)));
- build_id__snprintf(dso__bid(dso), sbuild_id, sizeof(sbuild_id));
- pydict_set_item_string_decref(dict, dso_bid_field,
- _PyUnicode_FromString(sbuild_id));
- pydict_set_item_string_decref(dict, dso_map_start,
- PyLong_FromUnsignedLong(map__start(al->map)));
- pydict_set_item_string_decref(dict, dso_map_end,
- PyLong_FromUnsignedLong(map__end(al->map)));
- pydict_set_item_string_decref(dict, map_pgoff,
- PyLong_FromUnsignedLongLong(map__pgoff(al->map)));
- }
- if (al->sym) {
- pydict_set_item_string_decref(dict, sym_field,
- _PyUnicode_FromString(al->sym->name));
- pydict_set_item_string_decref(dict, symoff_field,
- PyLong_FromUnsignedLong(get_offset(al->sym, al)));
- }
-}
-
-static void set_sample_flags(PyObject *dict, u32 flags)
-{
- const char *ch = PERF_IP_FLAG_CHARS;
- char *p, str[33];
-
- for (p = str; *ch; ch++, flags >>= 1) {
- if (flags & 1)
- *p++ = *ch;
- }
- *p = 0;
- pydict_set_item_string_decref(dict, "flags", _PyUnicode_FromString(str));
-}
-
-static void python_process_sample_flags(struct perf_sample *sample, PyObject *dict_sample)
-{
- char flags_disp[SAMPLE_FLAGS_BUF_SIZE];
-
- set_sample_flags(dict_sample, sample->flags);
- perf_sample__sprintf_flags(sample->flags, flags_disp, sizeof(flags_disp));
- pydict_set_item_string_decref(dict_sample, "flags_disp",
- _PyUnicode_FromString(flags_disp));
-}
-
-static PyObject *get_perf_sample_dict(struct perf_sample *sample,
- struct addr_location *al,
- struct addr_location *addr_al,
- PyObject *callchain)
-{
- PyObject *dict, *dict_sample, *brstack, *brstacksym;
- uint16_t e_machine = EM_HOST;
- uint32_t e_flags = EF_HOST;
- struct evsel *evsel = sample->evsel;
-
- dict = PyDict_New();
- if (!dict)
- Py_FatalError("couldn't create Python dictionary");
-
- dict_sample = PyDict_New();
- if (!dict_sample)
- Py_FatalError("couldn't create Python dictionary");
-
- pydict_set_item_string_decref(dict, "ev_name", _PyUnicode_FromString(evsel__name(evsel)));
- pydict_set_item_string_decref(dict, "attr", _PyBytes_FromStringAndSize((const char *)&evsel->core.attr, sizeof(evsel->core.attr)));
-
- pydict_set_item_string_decref(dict_sample, "id",
- PyLong_FromUnsignedLongLong(sample->id));
- pydict_set_item_string_decref(dict_sample, "stream_id",
- PyLong_FromUnsignedLongLong(sample->stream_id));
- pydict_set_item_string_decref(dict_sample, "pid",
- _PyLong_FromLong(sample->pid));
- pydict_set_item_string_decref(dict_sample, "tid",
- _PyLong_FromLong(sample->tid));
- pydict_set_item_string_decref(dict_sample, "cpu",
- _PyLong_FromLong(sample->cpu));
- pydict_set_item_string_decref(dict_sample, "ip",
- PyLong_FromUnsignedLongLong(sample->ip));
- pydict_set_item_string_decref(dict_sample, "time",
- PyLong_FromUnsignedLongLong(sample->time));
- pydict_set_item_string_decref(dict_sample, "period",
- PyLong_FromUnsignedLongLong(sample->period));
- pydict_set_item_string_decref(dict_sample, "phys_addr",
- PyLong_FromUnsignedLongLong(sample->phys_addr));
- pydict_set_item_string_decref(dict_sample, "addr",
- PyLong_FromUnsignedLongLong(sample->addr));
- set_sample_read_in_dict(dict_sample, sample);
- pydict_set_item_string_decref(dict_sample, "weight",
- PyLong_FromUnsignedLongLong(sample->weight));
- pydict_set_item_string_decref(dict_sample, "ins_lat",
- PyLong_FromUnsignedLong(sample->ins_lat));
- pydict_set_item_string_decref(dict_sample, "transaction",
- PyLong_FromUnsignedLongLong(sample->transaction));
- set_sample_datasrc_in_dict(dict_sample, sample);
- pydict_set_item_string_decref(dict, "sample", dict_sample);
-
- pydict_set_item_string_decref(dict, "raw_buf", _PyBytes_FromStringAndSize(
- (const char *)sample->raw_data, sample->raw_size));
- pydict_set_item_string_decref(dict, "comm",
- _PyUnicode_FromString(thread__comm_str(al->thread)));
- set_sym_in_dict(dict, al, "dso", "dso_bid", "dso_map_start", "dso_map_end",
- "symbol", "symoff", "map_pgoff");
-
- pydict_set_item_string_decref(dict, "callchain", callchain);
-
- brstack = python_process_brstack(sample, al->thread);
- pydict_set_item_string_decref(dict, "brstack", brstack);
-
- brstacksym = python_process_brstacksym(sample, al->thread);
- pydict_set_item_string_decref(dict, "brstacksym", brstacksym);
-
- if (sample->machine_pid) {
- pydict_set_item_string_decref(dict_sample, "machine_pid",
- _PyLong_FromLong(sample->machine_pid));
- pydict_set_item_string_decref(dict_sample, "vcpu",
- _PyLong_FromLong(sample->vcpu));
- }
-
- pydict_set_item_string_decref(dict_sample, "cpumode",
- _PyLong_FromLong((unsigned long)sample->cpumode));
-
- if (addr_al) {
- pydict_set_item_string_decref(dict_sample, "addr_correlates_sym",
- PyBool_FromLong(1));
- set_sym_in_dict(dict_sample, addr_al, "addr_dso", "addr_dso_bid",
- "addr_dso_map_start", "addr_dso_map_end",
- "addr_symbol", "addr_symoff", "addr_map_pgoff");
- }
-
- if (sample->flags)
- python_process_sample_flags(sample, dict_sample);
-
- /* Instructions per cycle (IPC) */
- if (sample->insn_cnt && sample->cyc_cnt) {
- pydict_set_item_string_decref(dict_sample, "insn_cnt",
- PyLong_FromUnsignedLongLong(sample->insn_cnt));
- pydict_set_item_string_decref(dict_sample, "cyc_cnt",
- PyLong_FromUnsignedLongLong(sample->cyc_cnt));
- }
-
- if (al->thread)
- e_machine = thread__e_machine(al->thread, /*machine=*/NULL, &e_flags);
-
- if (set_regs_in_dict(dict, sample, e_machine, e_flags))
- Py_FatalError("Failed to setting regs in dict");
-
- return dict;
-}
-
-#ifdef HAVE_LIBTRACEEVENT
-static void python_process_tracepoint(struct perf_sample *sample,
- struct addr_location *al,
- struct addr_location *addr_al)
-{
- struct tep_event *event;
- PyObject *handler, *context, *t, *obj = NULL, *callchain;
- PyObject *dict = NULL, *all_entries_dict = NULL;
- static char handler_name[256];
- struct tep_format_field *field;
- unsigned long s, ns;
- unsigned n = 0;
- int pid;
- int cpu = sample->cpu;
- void *data = sample->raw_data;
- unsigned long long nsecs = sample->time;
- const char *comm = thread__comm_str(al->thread);
- const char *default_handler_name = "trace_unhandled";
- DECLARE_BITMAP(events_defined, TRACE_EVENT_TYPE_MAX);
- struct evsel *evsel = sample->evsel;
-
- bitmap_zero(events_defined, TRACE_EVENT_TYPE_MAX);
-
- event = evsel__tp_format(evsel);
- if (!event) {
- snprintf(handler_name, sizeof(handler_name),
- "ug! no event found for type %" PRIu64, (u64)evsel->core.attr.config);
- Py_FatalError(handler_name);
- }
-
- pid = raw_field_value(event, "common_pid", data);
-
- sprintf(handler_name, "%s__%s", event->system, event->name);
-
- if (!__test_and_set_bit(event->id, events_defined))
- define_event_symbols(event, handler_name, event->print_fmt.args);
-
- handler = get_handler(handler_name);
- if (!handler) {
- handler = get_handler(default_handler_name);
- if (!handler)
- return;
- dict = PyDict_New();
- if (!dict)
- Py_FatalError("couldn't create Python dict");
- }
-
- t = PyTuple_New(MAX_FIELDS);
- if (!t)
- Py_FatalError("couldn't create Python tuple");
-
-
- s = nsecs / NSEC_PER_SEC;
- ns = nsecs - s * NSEC_PER_SEC;
-
- context = _PyCapsule_New(scripting_context, NULL, NULL);
-
- PyTuple_SetItem(t, n++, _PyUnicode_FromString(handler_name));
- PyTuple_SetItem(t, n++, context);
-
- /* ip unwinding */
- callchain = python_process_callchain(sample, al);
- /* Need an additional reference for the perf_sample dict */
- Py_INCREF(callchain);
-
- if (!dict) {
- PyTuple_SetItem(t, n++, _PyLong_FromLong(cpu));
- PyTuple_SetItem(t, n++, _PyLong_FromLong(s));
- PyTuple_SetItem(t, n++, _PyLong_FromLong(ns));
- PyTuple_SetItem(t, n++, _PyLong_FromLong(pid));
- PyTuple_SetItem(t, n++, _PyUnicode_FromString(comm));
- PyTuple_SetItem(t, n++, callchain);
- } else {
- pydict_set_item_string_decref(dict, "common_cpu", _PyLong_FromLong(cpu));
- pydict_set_item_string_decref(dict, "common_s", _PyLong_FromLong(s));
- pydict_set_item_string_decref(dict, "common_ns", _PyLong_FromLong(ns));
- pydict_set_item_string_decref(dict, "common_pid", _PyLong_FromLong(pid));
- pydict_set_item_string_decref(dict, "common_comm", _PyUnicode_FromString(comm));
- pydict_set_item_string_decref(dict, "common_callchain", callchain);
- }
- for (field = event->format.fields; field; field = field->next) {
- unsigned int offset, len;
- unsigned long long val;
-
- if (field->flags & TEP_FIELD_IS_ARRAY) {
- offset = field->offset;
- len = field->size;
- if (field->flags & TEP_FIELD_IS_DYNAMIC) {
- val = tep_read_number(scripting_context->pevent,
- data + offset, len);
- offset = val;
- len = offset >> 16;
- offset &= 0xffff;
- if (tep_field_is_relative(field->flags))
- offset += field->offset + field->size;
- }
- if (field->flags & TEP_FIELD_IS_STRING &&
- is_printable_array(data + offset, len)) {
- obj = _PyUnicode_FromString((char *) data + offset);
- } else {
- obj = PyByteArray_FromStringAndSize((const char *) data + offset, len);
- field->flags &= ~TEP_FIELD_IS_STRING;
- }
- } else { /* FIELD_IS_NUMERIC */
- obj = get_field_numeric_entry(event, field, data);
- }
- if (!dict)
- PyTuple_SetItem(t, n++, obj);
- else
- pydict_set_item_string_decref(dict, field->name, obj);
-
- }
-
- if (dict)
- PyTuple_SetItem(t, n++, dict);
-
- if (get_argument_count(handler) == (int) n + 1) {
- all_entries_dict = get_perf_sample_dict(sample, al, addr_al,
- callchain);
- PyTuple_SetItem(t, n++, all_entries_dict);
- } else {
- Py_DECREF(callchain);
- }
-
- if (_PyTuple_Resize(&t, n) == -1)
- Py_FatalError("error resizing Python tuple");
-
- if (!dict)
- call_object(handler, t, handler_name);
- else
- call_object(handler, t, default_handler_name);
-
- Py_DECREF(t);
-}
-#else
-static void python_process_tracepoint(struct perf_sample *sample __maybe_unused,
- struct addr_location *al __maybe_unused,
- struct addr_location *addr_al __maybe_unused)
-{
- fprintf(stderr, "Tracepoint events are not supported because "
- "perf is not linked with libtraceevent.\n");
-}
-#endif
-
-static PyObject *tuple_new(unsigned int sz)
-{
- PyObject *t;
-
- t = PyTuple_New(sz);
- if (!t)
- Py_FatalError("couldn't create Python tuple");
- return t;
-}
-
-static int tuple_set_s64(PyObject *t, unsigned int pos, s64 val)
-{
-#if BITS_PER_LONG == 64
- return PyTuple_SetItem(t, pos, _PyLong_FromLong(val));
-#endif
-#if BITS_PER_LONG == 32
- return PyTuple_SetItem(t, pos, PyLong_FromLongLong(val));
-#endif
-}
-
-/*
- * Databases support only signed 64-bit numbers, so even though we are
- * exporting a u64, it must be as s64.
- */
-#define tuple_set_d64 tuple_set_s64
-
-static int tuple_set_u64(PyObject *t, unsigned int pos, u64 val)
-{
-#if BITS_PER_LONG == 64
- return PyTuple_SetItem(t, pos, PyLong_FromUnsignedLong(val));
-#endif
-#if BITS_PER_LONG == 32
- return PyTuple_SetItem(t, pos, PyLong_FromUnsignedLongLong(val));
-#endif
-}
-
-static int tuple_set_u32(PyObject *t, unsigned int pos, u32 val)
-{
- return PyTuple_SetItem(t, pos, PyLong_FromUnsignedLong(val));
-}
-
-static int tuple_set_s32(PyObject *t, unsigned int pos, s32 val)
-{
- return PyTuple_SetItem(t, pos, _PyLong_FromLong(val));
-}
-
-static int tuple_set_bool(PyObject *t, unsigned int pos, bool val)
-{
- return PyTuple_SetItem(t, pos, PyBool_FromLong(val));
-}
-
-static int tuple_set_string(PyObject *t, unsigned int pos, const char *s)
-{
- return PyTuple_SetItem(t, pos, _PyUnicode_FromString(s));
-}
-
-static int tuple_set_bytes(PyObject *t, unsigned int pos, void *bytes,
- unsigned int sz)
-{
- return PyTuple_SetItem(t, pos, _PyBytes_FromStringAndSize(bytes, sz));
-}
-
-static int python_export_evsel(struct db_export *dbe, struct evsel *evsel)
-{
- struct tables *tables = container_of(dbe, struct tables, dbe);
- PyObject *t;
-
- t = tuple_new(2);
-
- tuple_set_d64(t, 0, evsel->db_id);
- tuple_set_string(t, 1, evsel__name(evsel));
-
- call_object(tables->evsel_handler, t, "evsel_table");
-
- Py_DECREF(t);
-
- return 0;
-}
-
-static int python_export_machine(struct db_export *dbe,
- struct machine *machine)
-{
- struct tables *tables = container_of(dbe, struct tables, dbe);
- PyObject *t;
-
- t = tuple_new(3);
-
- tuple_set_d64(t, 0, machine->db_id);
- tuple_set_s32(t, 1, machine->pid);
- tuple_set_string(t, 2, machine->root_dir ? machine->root_dir : "");
-
- call_object(tables->machine_handler, t, "machine_table");
-
- Py_DECREF(t);
-
- return 0;
-}
-
-static int python_export_thread(struct db_export *dbe, struct thread *thread,
- u64 main_thread_db_id, struct machine *machine)
-{
- struct tables *tables = container_of(dbe, struct tables, dbe);
- PyObject *t;
-
- t = tuple_new(5);
-
- tuple_set_d64(t, 0, thread__db_id(thread));
- tuple_set_d64(t, 1, machine->db_id);
- tuple_set_d64(t, 2, main_thread_db_id);
- tuple_set_s32(t, 3, thread__pid(thread));
- tuple_set_s32(t, 4, thread__tid(thread));
-
- call_object(tables->thread_handler, t, "thread_table");
-
- Py_DECREF(t);
-
- return 0;
-}
-
-static int python_export_comm(struct db_export *dbe, struct comm *comm,
- struct thread *thread)
-{
- struct tables *tables = container_of(dbe, struct tables, dbe);
- PyObject *t;
-
- t = tuple_new(5);
-
- tuple_set_d64(t, 0, comm->db_id);
- tuple_set_string(t, 1, comm__str(comm));
- tuple_set_d64(t, 2, thread__db_id(thread));
- tuple_set_d64(t, 3, comm->start);
- tuple_set_s32(t, 4, comm->exec);
-
- call_object(tables->comm_handler, t, "comm_table");
-
- Py_DECREF(t);
-
- return 0;
-}
-
-static int python_export_comm_thread(struct db_export *dbe, u64 db_id,
- struct comm *comm, struct thread *thread)
-{
- struct tables *tables = container_of(dbe, struct tables, dbe);
- PyObject *t;
-
- t = tuple_new(3);
-
- tuple_set_d64(t, 0, db_id);
- tuple_set_d64(t, 1, comm->db_id);
- tuple_set_d64(t, 2, thread__db_id(thread));
-
- call_object(tables->comm_thread_handler, t, "comm_thread_table");
-
- Py_DECREF(t);
-
- return 0;
-}
-
-static int python_export_dso(struct db_export *dbe, struct dso *dso,
- struct machine *machine)
-{
- struct tables *tables = container_of(dbe, struct tables, dbe);
- char sbuild_id[SBUILD_ID_SIZE];
- PyObject *t;
-
- build_id__snprintf(dso__bid(dso), sbuild_id, sizeof(sbuild_id));
-
- t = tuple_new(5);
-
- tuple_set_d64(t, 0, dso__db_id(dso));
- tuple_set_d64(t, 1, machine->db_id);
- tuple_set_string(t, 2, dso__short_name(dso));
- tuple_set_string(t, 3, dso__long_name(dso));
- tuple_set_string(t, 4, sbuild_id);
-
- call_object(tables->dso_handler, t, "dso_table");
-
- Py_DECREF(t);
-
- return 0;
-}
-
-static int python_export_symbol(struct db_export *dbe, struct symbol *sym,
- struct dso *dso)
-{
- struct tables *tables = container_of(dbe, struct tables, dbe);
- u64 *sym_db_id = symbol__priv(sym);
- PyObject *t;
-
- t = tuple_new(6);
-
- tuple_set_d64(t, 0, *sym_db_id);
- tuple_set_d64(t, 1, dso__db_id(dso));
- tuple_set_d64(t, 2, sym->start);
- tuple_set_d64(t, 3, sym->end);
- tuple_set_s32(t, 4, symbol__binding(sym));
- tuple_set_string(t, 5, sym->name);
-
- call_object(tables->symbol_handler, t, "symbol_table");
-
- Py_DECREF(t);
-
- return 0;
-}
-
-static int python_export_branch_type(struct db_export *dbe, u32 branch_type,
- const char *name)
-{
- struct tables *tables = container_of(dbe, struct tables, dbe);
- PyObject *t;
-
- t = tuple_new(2);
-
- tuple_set_s32(t, 0, branch_type);
- tuple_set_string(t, 1, name);
-
- call_object(tables->branch_type_handler, t, "branch_type_table");
-
- Py_DECREF(t);
-
- return 0;
-}
-
-static void python_export_sample_table(struct db_export *dbe,
- struct export_sample *es)
-{
- struct tables *tables = container_of(dbe, struct tables, dbe);
- PyObject *t;
-
- t = tuple_new(28);
-
- tuple_set_d64(t, 0, es->db_id);
- tuple_set_d64(t, 1, es->sample->evsel->db_id);
- tuple_set_d64(t, 2, maps__machine(thread__maps(es->al->thread))->db_id);
- tuple_set_d64(t, 3, thread__db_id(es->al->thread));
- tuple_set_d64(t, 4, es->comm_db_id);
- tuple_set_d64(t, 5, es->dso_db_id);
- tuple_set_d64(t, 6, es->sym_db_id);
- tuple_set_d64(t, 7, es->offset);
- tuple_set_d64(t, 8, es->sample->ip);
- tuple_set_d64(t, 9, es->sample->time);
- tuple_set_s32(t, 10, es->sample->cpu);
- tuple_set_d64(t, 11, es->addr_dso_db_id);
- tuple_set_d64(t, 12, es->addr_sym_db_id);
- tuple_set_d64(t, 13, es->addr_offset);
- tuple_set_d64(t, 14, es->sample->addr);
- tuple_set_d64(t, 15, es->sample->period);
- tuple_set_d64(t, 16, es->sample->weight);
- tuple_set_d64(t, 17, es->sample->transaction);
- tuple_set_d64(t, 18, es->sample->data_src);
- tuple_set_s32(t, 19, es->sample->flags & PERF_BRANCH_MASK);
- tuple_set_s32(t, 20, !!(es->sample->flags & PERF_IP_FLAG_IN_TX));
- tuple_set_d64(t, 21, es->call_path_id);
- tuple_set_d64(t, 22, es->sample->insn_cnt);
- tuple_set_d64(t, 23, es->sample->cyc_cnt);
- tuple_set_s32(t, 24, es->sample->flags);
- tuple_set_d64(t, 25, es->sample->id);
- tuple_set_d64(t, 26, es->sample->stream_id);
- tuple_set_u32(t, 27, es->sample->ins_lat);
-
- call_object(tables->sample_handler, t, "sample_table");
-
- Py_DECREF(t);
-}
-
-static void python_export_synth(struct db_export *dbe, struct export_sample *es)
-{
- struct tables *tables = container_of(dbe, struct tables, dbe);
- PyObject *t;
-
- t = tuple_new(3);
-
- tuple_set_d64(t, 0, es->db_id);
- tuple_set_d64(t, 1, es->sample->evsel->core.attr.config);
- tuple_set_bytes(t, 2, es->sample->raw_data, es->sample->raw_size);
-
- call_object(tables->synth_handler, t, "synth_data");
-
- Py_DECREF(t);
-}
-
-static int python_export_sample(struct db_export *dbe,
- struct export_sample *es)
-{
- struct tables *tables = container_of(dbe, struct tables, dbe);
-
- python_export_sample_table(dbe, es);
-
- if (es->sample->evsel->core.attr.type == PERF_TYPE_SYNTH && tables->synth_handler)
- python_export_synth(dbe, es);
-
- return 0;
-}
-
-static int python_export_call_path(struct db_export *dbe, struct call_path *cp)
-{
- struct tables *tables = container_of(dbe, struct tables, dbe);
- PyObject *t;
- u64 parent_db_id, sym_db_id;
-
- parent_db_id = cp->parent ? cp->parent->db_id : 0;
- sym_db_id = cp->sym ? *(u64 *)symbol__priv(cp->sym) : 0;
-
- t = tuple_new(4);
-
- tuple_set_d64(t, 0, cp->db_id);
- tuple_set_d64(t, 1, parent_db_id);
- tuple_set_d64(t, 2, sym_db_id);
- tuple_set_d64(t, 3, cp->ip);
-
- call_object(tables->call_path_handler, t, "call_path_table");
-
- Py_DECREF(t);
-
- return 0;
-}
-
-static int python_export_call_return(struct db_export *dbe,
- struct call_return *cr)
-{
- struct tables *tables = container_of(dbe, struct tables, dbe);
- u64 comm_db_id = cr->comm ? cr->comm->db_id : 0;
- PyObject *t;
-
- t = tuple_new(14);
-
- tuple_set_d64(t, 0, cr->db_id);
- tuple_set_d64(t, 1, thread__db_id(cr->thread));
- tuple_set_d64(t, 2, comm_db_id);
- tuple_set_d64(t, 3, cr->cp->db_id);
- tuple_set_d64(t, 4, cr->call_time);
- tuple_set_d64(t, 5, cr->return_time);
- tuple_set_d64(t, 6, cr->branch_count);
- tuple_set_d64(t, 7, cr->call_ref);
- tuple_set_d64(t, 8, cr->return_ref);
- tuple_set_d64(t, 9, cr->cp->parent->db_id);
- tuple_set_s32(t, 10, cr->flags);
- tuple_set_d64(t, 11, cr->parent_db_id);
- tuple_set_d64(t, 12, cr->insn_count);
- tuple_set_d64(t, 13, cr->cyc_count);
-
- call_object(tables->call_return_handler, t, "call_return_table");
-
- Py_DECREF(t);
-
- return 0;
-}
-
-static int python_export_context_switch(struct db_export *dbe, u64 db_id,
- struct machine *machine,
- struct perf_sample *sample,
- u64 th_out_id, u64 comm_out_id,
- u64 th_in_id, u64 comm_in_id, int flags)
-{
- struct tables *tables = container_of(dbe, struct tables, dbe);
- PyObject *t;
-
- t = tuple_new(9);
-
- tuple_set_d64(t, 0, db_id);
- tuple_set_d64(t, 1, machine->db_id);
- tuple_set_d64(t, 2, sample->time);
- tuple_set_s32(t, 3, sample->cpu);
- tuple_set_d64(t, 4, th_out_id);
- tuple_set_d64(t, 5, comm_out_id);
- tuple_set_d64(t, 6, th_in_id);
- tuple_set_d64(t, 7, comm_in_id);
- tuple_set_s32(t, 8, flags);
-
- call_object(tables->context_switch_handler, t, "context_switch");
-
- Py_DECREF(t);
-
- return 0;
-}
-
-static int python_process_call_return(struct call_return *cr, u64 *parent_db_id,
- void *data)
-{
- struct db_export *dbe = data;
-
- return db_export__call_return(dbe, cr, parent_db_id);
-}
-
-static void python_process_general_event(struct perf_sample *sample,
- struct addr_location *al,
- struct addr_location *addr_al)
-{
- PyObject *handler, *t, *dict, *callchain;
- static char handler_name[64];
- unsigned n = 0;
-
- snprintf(handler_name, sizeof(handler_name), "%s", "process_event");
-
- handler = get_handler(handler_name);
- if (!handler)
- return;
-
- /*
- * Use the MAX_FIELDS to make the function expandable, though
- * currently there is only one item for the tuple.
- */
- t = PyTuple_New(MAX_FIELDS);
- if (!t)
- Py_FatalError("couldn't create Python tuple");
-
- /* ip unwinding */
- callchain = python_process_callchain(sample, al);
- dict = get_perf_sample_dict(sample, al, addr_al, callchain);
-
- PyTuple_SetItem(t, n++, dict);
- if (_PyTuple_Resize(&t, n) == -1)
- Py_FatalError("error resizing Python tuple");
-
- call_object(handler, t, handler_name);
-
- Py_DECREF(t);
-}
-
-static void python_process_event(union perf_event *event,
- struct perf_sample *sample,
- struct addr_location *al,
- struct addr_location *addr_al)
-{
- struct tables *tables = &tables_global;
-
- scripting_context__update(scripting_context, event, sample, al, addr_al);
-
- switch (sample->evsel->core.attr.type) {
- case PERF_TYPE_TRACEPOINT:
- python_process_tracepoint(sample, al, addr_al);
- break;
- /* Reserve for future process_hw/sw/raw APIs */
- default:
- if (tables->db_export_mode)
- db_export__sample(&tables->dbe, event, sample, al, addr_al);
- else
- python_process_general_event(sample, al, addr_al);
- }
-}
-
-static void python_process_throttle(union perf_event *event,
- struct perf_sample *sample,
- struct machine *machine)
-{
- const char *handler_name;
- PyObject *handler, *t;
-
- if (event->header.type == PERF_RECORD_THROTTLE)
- handler_name = "throttle";
- else
- handler_name = "unthrottle";
- handler = get_handler(handler_name);
- if (!handler)
- return;
-
- t = tuple_new(6);
- if (!t)
- return;
-
- tuple_set_u64(t, 0, event->throttle.time);
- tuple_set_u64(t, 1, event->throttle.id);
- tuple_set_u64(t, 2, event->throttle.stream_id);
- tuple_set_s32(t, 3, sample->cpu);
- tuple_set_s32(t, 4, sample->pid);
- tuple_set_s32(t, 5, sample->tid);
-
- call_object(handler, t, handler_name);
-
- Py_DECREF(t);
-}
-
-static void python_do_process_switch(union perf_event *event,
- struct perf_sample *sample,
- struct machine *machine)
-{
- const char *handler_name = "context_switch";
- bool out = event->header.misc & PERF_RECORD_MISC_SWITCH_OUT;
- bool out_preempt = out && (event->header.misc & PERF_RECORD_MISC_SWITCH_OUT_PREEMPT);
- pid_t np_pid = -1, np_tid = -1;
- PyObject *handler, *t;
-
- handler = get_handler(handler_name);
- if (!handler)
- return;
-
- if (event->header.type == PERF_RECORD_SWITCH_CPU_WIDE) {
- np_pid = event->context_switch.next_prev_pid;
- np_tid = event->context_switch.next_prev_tid;
- }
-
- t = tuple_new(11);
- if (!t)
- return;
-
- tuple_set_u64(t, 0, sample->time);
- tuple_set_s32(t, 1, sample->cpu);
- tuple_set_s32(t, 2, sample->pid);
- tuple_set_s32(t, 3, sample->tid);
- tuple_set_s32(t, 4, np_pid);
- tuple_set_s32(t, 5, np_tid);
- tuple_set_s32(t, 6, machine->pid);
- tuple_set_bool(t, 7, out);
- tuple_set_bool(t, 8, out_preempt);
- tuple_set_s32(t, 9, sample->machine_pid);
- tuple_set_s32(t, 10, sample->vcpu);
-
- call_object(handler, t, handler_name);
-
- Py_DECREF(t);
-}
-
-static void python_process_switch(union perf_event *event,
- struct perf_sample *sample,
- struct machine *machine)
-{
- struct tables *tables = &tables_global;
-
- if (tables->db_export_mode)
- db_export__switch(&tables->dbe, event, sample, machine);
- else
- python_do_process_switch(event, sample, machine);
-}
-
-static void python_process_auxtrace_error(struct perf_session *session __maybe_unused,
- union perf_event *event)
-{
- struct perf_record_auxtrace_error *e = &event->auxtrace_error;
- u8 cpumode = e->header.misc & PERF_RECORD_MISC_CPUMODE_MASK;
- const char *handler_name = "auxtrace_error";
- unsigned long long tm = e->time;
- const char *msg = e->msg;
- s32 machine_pid = 0, vcpu = 0;
- char msg_buf[MAX_AUXTRACE_ERROR_MSG + 1];
- int msg_max;
- PyObject *handler, *t;
-
- handler = get_handler(handler_name);
- if (!handler)
- return;
-
- if (!e->fmt) {
- tm = 0;
- msg = (const char *)&e->time;
- }
-
- /* Bound msg to the bytes within the event, ensure NUL-termination */
- msg_max = (int)((void *)event + event->header.size - (void *)msg);
- if (msg_max <= 0) {
- msg_buf[0] = '\0';
- } else {
- if (msg_max > (int)sizeof(msg_buf) - 1)
- msg_max = sizeof(msg_buf) - 1;
- memcpy(msg_buf, msg, msg_max);
- msg_buf[msg_max] = '\0';
- }
-
- /* Only access fmt >= 2 fields if the event is large enough */
- if (e->fmt >= 2 &&
- event->header.size >= offsetof(typeof(event->auxtrace_error), vcpu) +
- sizeof(event->auxtrace_error.vcpu)) {
- machine_pid = e->machine_pid;
- vcpu = e->vcpu;
- }
-
- t = tuple_new(11);
-
- tuple_set_u32(t, 0, e->type);
- tuple_set_u32(t, 1, e->code);
- tuple_set_s32(t, 2, e->cpu);
- tuple_set_s32(t, 3, e->pid);
- tuple_set_s32(t, 4, e->tid);
- tuple_set_u64(t, 5, e->ip);
- tuple_set_u64(t, 6, tm);
- tuple_set_string(t, 7, msg_buf);
- tuple_set_u32(t, 8, cpumode);
- tuple_set_s32(t, 9, machine_pid);
- tuple_set_s32(t, 10, vcpu);
-
- call_object(handler, t, handler_name);
-
- Py_DECREF(t);
-}
-
-static void get_handler_name(char *str, size_t size,
- struct evsel *evsel)
-{
- char *p = str;
-
- scnprintf(str, size, "stat__%s", evsel__name(evsel));
-
- while ((p = strchr(p, ':'))) {
- *p = '_';
- p++;
- }
-}
-
-static void
-process_stat(struct evsel *counter, struct perf_cpu cpu, int thread, u64 tstamp,
- struct perf_counts_values *count)
-{
- PyObject *handler, *t;
- static char handler_name[256];
- int n = 0;
-
- t = PyTuple_New(MAX_FIELDS);
- if (!t)
- Py_FatalError("couldn't create Python tuple");
-
- get_handler_name(handler_name, sizeof(handler_name),
- counter);
-
- handler = get_handler(handler_name);
- if (!handler) {
- pr_debug("can't find python handler %s\n", handler_name);
- return;
- }
-
- PyTuple_SetItem(t, n++, _PyLong_FromLong(cpu.cpu));
- PyTuple_SetItem(t, n++, _PyLong_FromLong(thread));
-
- tuple_set_u64(t, n++, tstamp);
- tuple_set_u64(t, n++, count->val);
- tuple_set_u64(t, n++, count->ena);
- tuple_set_u64(t, n++, count->run);
-
- if (_PyTuple_Resize(&t, n) == -1)
- Py_FatalError("error resizing Python tuple");
-
- call_object(handler, t, handler_name);
-
- Py_DECREF(t);
-}
-
-static void python_process_stat(struct perf_stat_config *config,
- struct evsel *counter, u64 tstamp)
-{
- struct perf_thread_map *threads = counter->core.threads;
- struct perf_cpu_map *cpus = counter->core.cpus;
-
- for (int thread = 0; thread < perf_thread_map__nr(threads); thread++) {
- unsigned int idx;
- struct perf_cpu cpu;
-
- perf_cpu_map__for_each_cpu(cpu, idx, cpus) {
- process_stat(counter, cpu,
- perf_thread_map__pid(threads, thread), tstamp,
- perf_counts(counter->counts, idx, thread));
- }
- }
-}
-
-static void python_process_stat_interval(u64 tstamp)
-{
- PyObject *handler, *t;
- static const char handler_name[] = "stat__interval";
- int n = 0;
-
- t = PyTuple_New(MAX_FIELDS);
- if (!t)
- Py_FatalError("couldn't create Python tuple");
-
- handler = get_handler(handler_name);
- if (!handler) {
- pr_debug("can't find python handler %s\n", handler_name);
- return;
- }
-
- tuple_set_u64(t, n++, tstamp);
-
- if (_PyTuple_Resize(&t, n) == -1)
- Py_FatalError("error resizing Python tuple");
-
- call_object(handler, t, handler_name);
-
- Py_DECREF(t);
-}
-
-static int perf_script_context_init(void)
-{
- PyObject *perf_script_context;
- PyObject *perf_trace_context;
- PyObject *dict;
- int ret;
-
- perf_trace_context = PyImport_AddModule("perf_trace_context");
- if (!perf_trace_context)
- return -1;
- dict = PyModule_GetDict(perf_trace_context);
- if (!dict)
- return -1;
-
- perf_script_context = _PyCapsule_New(scripting_context, NULL, NULL);
- if (!perf_script_context)
- return -1;
-
- ret = PyDict_SetItemString(dict, "perf_script_context", perf_script_context);
- if (!ret)
- ret = PyDict_SetItemString(main_dict, "perf_script_context", perf_script_context);
- Py_DECREF(perf_script_context);
- return ret;
-}
-
-static int run_start_sub(void)
-{
- main_module = PyImport_AddModule("__main__");
- if (main_module == NULL)
- return -1;
- Py_INCREF(main_module);
-
- main_dict = PyModule_GetDict(main_module);
- if (main_dict == NULL)
- goto error;
- Py_INCREF(main_dict);
-
- if (perf_script_context_init())
- goto error;
-
- try_call_object("trace_begin", NULL);
-
- return 0;
-
-error:
- Py_XDECREF(main_dict);
- Py_XDECREF(main_module);
- return -1;
-}
-
-#define SET_TABLE_HANDLER_(name, handler_name, table_name) do { \
- tables->handler_name = get_handler(#table_name); \
- if (tables->handler_name) \
- tables->dbe.export_ ## name = python_export_ ## name; \
-} while (0)
-
-#define SET_TABLE_HANDLER(name) \
- SET_TABLE_HANDLER_(name, name ## _handler, name ## _table)
-
-static void set_table_handlers(struct tables *tables)
-{
- const char *perf_db_export_mode = "perf_db_export_mode";
- const char *perf_db_export_calls = "perf_db_export_calls";
- const char *perf_db_export_callchains = "perf_db_export_callchains";
- PyObject *db_export_mode, *db_export_calls, *db_export_callchains;
- bool export_calls = false;
- bool export_callchains = false;
- int ret;
-
- memset(tables, 0, sizeof(struct tables));
- if (db_export__init(&tables->dbe))
- Py_FatalError("failed to initialize export");
-
- db_export_mode = PyDict_GetItemString(main_dict, perf_db_export_mode);
- if (!db_export_mode)
- return;
-
- ret = PyObject_IsTrue(db_export_mode);
- if (ret == -1)
- handler_call_die(perf_db_export_mode);
- if (!ret)
- return;
-
- /* handle export calls */
- tables->dbe.crp = NULL;
- db_export_calls = PyDict_GetItemString(main_dict, perf_db_export_calls);
- if (db_export_calls) {
- ret = PyObject_IsTrue(db_export_calls);
- if (ret == -1)
- handler_call_die(perf_db_export_calls);
- export_calls = !!ret;
- }
-
- if (export_calls) {
- tables->dbe.crp =
- call_return_processor__new(python_process_call_return,
- &tables->dbe);
- if (!tables->dbe.crp)
- Py_FatalError("failed to create calls processor");
- }
-
- /* handle export callchains */
- tables->dbe.cpr = NULL;
- db_export_callchains = PyDict_GetItemString(main_dict,
- perf_db_export_callchains);
- if (db_export_callchains) {
- ret = PyObject_IsTrue(db_export_callchains);
- if (ret == -1)
- handler_call_die(perf_db_export_callchains);
- export_callchains = !!ret;
- }
-
- if (export_callchains) {
- /*
- * Attempt to use the call path root from the call return
- * processor, if the call return processor is in use. Otherwise,
- * we allocate a new call path root. This prevents exporting
- * duplicate call path ids when both are in use simultaneously.
- */
- if (tables->dbe.crp)
- tables->dbe.cpr = tables->dbe.crp->cpr;
- else
- tables->dbe.cpr = call_path_root__new();
-
- if (!tables->dbe.cpr)
- Py_FatalError("failed to create call path root");
- }
-
- tables->db_export_mode = true;
- /*
- * Reserve per symbol space for symbol->db_id via symbol__priv()
- */
- symbol_conf.priv_size = sizeof(u64);
-
- SET_TABLE_HANDLER(evsel);
- SET_TABLE_HANDLER(machine);
- SET_TABLE_HANDLER(thread);
- SET_TABLE_HANDLER(comm);
- SET_TABLE_HANDLER(comm_thread);
- SET_TABLE_HANDLER(dso);
- SET_TABLE_HANDLER(symbol);
- SET_TABLE_HANDLER(branch_type);
- SET_TABLE_HANDLER(sample);
- SET_TABLE_HANDLER(call_path);
- SET_TABLE_HANDLER(call_return);
- SET_TABLE_HANDLER(context_switch);
-
- /*
- * Synthesized events are samples but with architecture-specific data
- * stored in sample->raw_data. They are exported via
- * python_export_sample() and consequently do not need a separate export
- * callback.
- */
- tables->synth_handler = get_handler("synth_data");
-}
-
-static void _free_command_line(wchar_t **command_line, int num)
-{
- int i;
- for (i = 0; i < num; i++)
- PyMem_RawFree(command_line[i]);
- free(command_line);
-}
-
-
-/*
- * Start trace script
- */
-static int python_start_script(const char *script, int argc, const char **argv,
- struct perf_session *session)
-{
- struct tables *tables = &tables_global;
- wchar_t **command_line;
- char buf[PATH_MAX];
- int i, err = 0;
- FILE *fp;
-
- scripting_context->session = session;
- command_line = malloc((argc + 1) * sizeof(wchar_t *));
- if (!command_line)
- return -1;
-
- command_line[0] = Py_DecodeLocale(script, NULL);
- for (i = 1; i < argc + 1; i++)
- command_line[i] = Py_DecodeLocale(argv[i - 1], NULL);
- PyImport_AppendInittab("perf_trace_context", PyInit_perf_trace_context);
- Py_Initialize();
-
- PySys_SetArgv(argc + 1, command_line);
-
- fp = fopen(script, "r");
- if (!fp) {
- sprintf(buf, "Can't open python script \"%s\"", script);
- perror(buf);
- err = -1;
- goto error;
- }
-
- err = PyRun_SimpleFile(fp, script);
- if (err) {
- fprintf(stderr, "Error running python script %s\n", script);
- goto error;
- }
-
- err = run_start_sub();
- if (err) {
- fprintf(stderr, "Error starting python script %s\n", script);
- goto error;
- }
-
- set_table_handlers(tables);
-
- if (tables->db_export_mode) {
- err = db_export__branch_types(&tables->dbe);
- if (err)
- goto error;
- }
-
- _free_command_line(command_line, argc + 1);
-
- return err;
-error:
- Py_Finalize();
- _free_command_line(command_line, argc + 1);
-
- return err;
-}
-
-static int python_flush_script(void)
-{
- return 0;
-}
-
-/*
- * Stop trace script
- */
-static int python_stop_script(void)
-{
- struct tables *tables = &tables_global;
-
- try_call_object("trace_end", NULL);
-
- db_export__exit(&tables->dbe);
-
- Py_XDECREF(main_dict);
- Py_XDECREF(main_module);
- Py_Finalize();
-
- return 0;
-}
-
-#ifdef HAVE_LIBTRACEEVENT
-static int python_generate_script(struct tep_handle *pevent, const char *outfile)
-{
- int i, not_first, count, nr_events;
- struct tep_event **all_events;
- struct tep_event *event = NULL;
- struct tep_format_field *f;
- char fname[PATH_MAX];
- FILE *ofp;
-
- sprintf(fname, "%s.py", outfile);
- ofp = fopen(fname, "w");
- if (ofp == NULL) {
- fprintf(stderr, "couldn't open %s\n", fname);
- return -1;
- }
- fprintf(ofp, "# perf script event handlers, "
- "generated by perf script -g python\n");
-
- fprintf(ofp, "# Licensed under the terms of the GNU GPL"
- " License version 2\n\n");
-
- fprintf(ofp, "# The common_* event handler fields are the most useful "
- "fields common to\n");
-
- fprintf(ofp, "# all events. They don't necessarily correspond to "
- "the 'common_*' fields\n");
-
- fprintf(ofp, "# in the format files. Those fields not available as "
- "handler params can\n");
-
- fprintf(ofp, "# be retrieved using Python functions of the form "
- "common_*(context).\n");
-
- fprintf(ofp, "# See the perf-script-python Documentation for the list "
- "of available functions.\n\n");
-
- fprintf(ofp, "from __future__ import print_function\n\n");
- fprintf(ofp, "import os\n");
- fprintf(ofp, "import sys\n\n");
-
- fprintf(ofp, "sys.path.append(os.environ['PERF_EXEC_PATH'] + \\\n");
- fprintf(ofp, "\t'/scripts/python/Perf-Trace-Util/lib/Perf/Trace')\n");
- fprintf(ofp, "\nfrom perf_trace_context import *\n");
- fprintf(ofp, "from Core import *\n\n\n");
-
- fprintf(ofp, "def trace_begin():\n");
- fprintf(ofp, "\tprint(\"in trace_begin\")\n\n");
-
- fprintf(ofp, "def trace_end():\n");
- fprintf(ofp, "\tprint(\"in trace_end\")\n\n");
-
- nr_events = tep_get_events_count(pevent);
- all_events = tep_list_events(pevent, TEP_EVENT_SORT_ID);
-
- for (i = 0; all_events && i < nr_events; i++) {
- event = all_events[i];
- fprintf(ofp, "def %s__%s(", event->system, event->name);
- fprintf(ofp, "event_name, ");
- fprintf(ofp, "context, ");
- fprintf(ofp, "common_cpu,\n");
- fprintf(ofp, "\tcommon_secs, ");
- fprintf(ofp, "common_nsecs, ");
- fprintf(ofp, "common_pid, ");
- fprintf(ofp, "common_comm,\n\t");
- fprintf(ofp, "common_callchain, ");
-
- not_first = 0;
- count = 0;
-
- for (f = event->format.fields; f; f = f->next) {
- if (not_first++)
- fprintf(ofp, ", ");
- if (++count % 5 == 0)
- fprintf(ofp, "\n\t");
-
- fprintf(ofp, "%s", f->name);
- }
- if (not_first++)
- fprintf(ofp, ", ");
- if (++count % 5 == 0)
- fprintf(ofp, "\n\t\t");
- fprintf(ofp, "perf_sample_dict");
-
- fprintf(ofp, "):\n");
-
- fprintf(ofp, "\t\tprint_header(event_name, common_cpu, "
- "common_secs, common_nsecs,\n\t\t\t"
- "common_pid, common_comm)\n\n");
-
- fprintf(ofp, "\t\tprint(\"");
-
- not_first = 0;
- count = 0;
-
- for (f = event->format.fields; f; f = f->next) {
- if (not_first++)
- fprintf(ofp, ", ");
- if (count && count % 3 == 0) {
- fprintf(ofp, "\" \\\n\t\t\"");
- }
- count++;
-
- fprintf(ofp, "%s=", f->name);
- if (f->flags & TEP_FIELD_IS_STRING ||
- f->flags & TEP_FIELD_IS_FLAG ||
- f->flags & TEP_FIELD_IS_ARRAY ||
- f->flags & TEP_FIELD_IS_SYMBOLIC)
- fprintf(ofp, "%%s");
- else if (f->flags & TEP_FIELD_IS_SIGNED)
- fprintf(ofp, "%%d");
- else
- fprintf(ofp, "%%u");
- }
-
- fprintf(ofp, "\" %% \\\n\t\t(");
-
- not_first = 0;
- count = 0;
-
- for (f = event->format.fields; f; f = f->next) {
- if (not_first++)
- fprintf(ofp, ", ");
-
- if (++count % 5 == 0)
- fprintf(ofp, "\n\t\t");
-
- if (f->flags & TEP_FIELD_IS_FLAG) {
- if ((count - 1) % 5 != 0) {
- fprintf(ofp, "\n\t\t");
- count = 4;
- }
- fprintf(ofp, "flag_str(\"");
- fprintf(ofp, "%s__%s\", ", event->system,
- event->name);
- fprintf(ofp, "\"%s\", %s)", f->name,
- f->name);
- } else if (f->flags & TEP_FIELD_IS_SYMBOLIC) {
- if ((count - 1) % 5 != 0) {
- fprintf(ofp, "\n\t\t");
- count = 4;
- }
- fprintf(ofp, "symbol_str(\"");
- fprintf(ofp, "%s__%s\", ", event->system,
- event->name);
- fprintf(ofp, "\"%s\", %s)", f->name,
- f->name);
- } else
- fprintf(ofp, "%s", f->name);
- }
-
- fprintf(ofp, "))\n\n");
-
- fprintf(ofp, "\t\tprint('Sample: {'+"
- "get_dict_as_string(perf_sample_dict['sample'], ', ')+'}')\n\n");
-
- fprintf(ofp, "\t\tfor node in common_callchain:");
- fprintf(ofp, "\n\t\t\tif 'sym' in node:");
- fprintf(ofp, "\n\t\t\t\tprint(\"\t[%%x] %%s%%s%%s%%s\" %% (");
- fprintf(ofp, "\n\t\t\t\t\tnode['ip'], node['sym']['name'],");
- fprintf(ofp, "\n\t\t\t\t\t\"+0x{:x}\".format(node['sym_off']) if 'sym_off' in node else \"\",");
- fprintf(ofp, "\n\t\t\t\t\t\" ({})\".format(node['dso']) if 'dso' in node else \"\",");
- fprintf(ofp, "\n\t\t\t\t\t\" \" + node['sym_srcline'] if 'sym_srcline' in node else \"\"))");
- fprintf(ofp, "\n\t\t\telse:");
- fprintf(ofp, "\n\t\t\t\tprint(\"\t[%%x]\" %% (node['ip']))\n\n");
- fprintf(ofp, "\t\tprint()\n\n");
-
- }
-
- fprintf(ofp, "def trace_unhandled(event_name, context, "
- "event_fields_dict, perf_sample_dict):\n");
-
- fprintf(ofp, "\t\tprint(get_dict_as_string(event_fields_dict))\n");
- fprintf(ofp, "\t\tprint('Sample: {'+"
- "get_dict_as_string(perf_sample_dict['sample'], ', ')+'}')\n\n");
-
- fprintf(ofp, "def print_header("
- "event_name, cpu, secs, nsecs, pid, comm):\n"
- "\tprint(\"%%-20s %%5u %%05u.%%09u %%8u %%-20s \" %% \\\n\t"
- "(event_name, cpu, secs, nsecs, pid, comm), end=\"\")\n\n");
-
- fprintf(ofp, "def get_dict_as_string(a_dict, delimiter=' '):\n"
- "\treturn delimiter.join"
- "(['%%s=%%s'%%(k,str(v))for k,v in sorted(a_dict.items())])\n");
-
- fclose(ofp);
-
- fprintf(stderr, "generated Python script: %s\n", fname);
-
- return 0;
-}
-#else
-static int python_generate_script(struct tep_handle *pevent __maybe_unused,
- const char *outfile __maybe_unused)
-{
- fprintf(stderr, "Generating Python perf-script is not supported."
- " Install libtraceevent and rebuild perf to enable it.\n"
- "For example:\n # apt install libtraceevent-dev (ubuntu)"
- "\n # yum install libtraceevent-devel (Fedora)"
- "\n etc.\n");
- return -1;
-}
-#endif
-
-struct scripting_ops python_scripting_ops = {
- .name = "Python",
- .dirname = "python",
- .start_script = python_start_script,
- .flush_script = python_flush_script,
- .stop_script = python_stop_script,
- .process_event = python_process_event,
- .process_switch = python_process_switch,
- .process_auxtrace_error = python_process_auxtrace_error,
- .process_stat = python_process_stat,
- .process_stat_interval = python_process_stat_interval,
- .process_throttle = python_process_throttle,
- .generate_script = python_generate_script,
-};
diff --git a/tools/perf/util/trace-event-scripting.c b/tools/perf/util/trace-event-scripting.c
index dc584ac316a3..c78b317978dd 100644
--- a/tools/perf/util/trace-event-scripting.c
+++ b/tools/perf/util/trace-event-scripting.c
@@ -189,19 +189,10 @@ static void register_python_scripting(struct scripting_ops *scripting_ops)
}
}
-#ifndef HAVE_LIBPYTHON_SUPPORT
void setup_python_scripting(void)
{
register_python_scripting(&python_scripting_unsupported_ops);
}
-#else
-extern struct scripting_ops python_scripting_ops;
-
-void setup_python_scripting(void)
-{
- register_python_scripting(&python_scripting_ops);
-}
-#endif
#ifdef HAVE_LIBTRACEEVENT
static void print_perl_unsupported_msg(void)
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 47/49] perf Makefile: Update Python script installation path
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (45 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 46/49] perf: Remove libpython support and legacy Python scripts Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 48/49] perf script: Support standalone scripts and remove embedded scripting Ian Rogers
2026-09-20 5:21 ` [PATCH v1 49/49] perf Documentation: Update for standalone Python scripts Ian Rogers
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Replace the libpython feature test with a python-module feature test
checking for Python C extension build capability, and update feature
test references accordingly.
Remove references to the legacy scripts/python directory and install
standalone Python scripts directly under the python directory in
libexec. Update the TUI script browser (ui/browsers/scripts.c) to
discover standalone scripts from the updated installation path.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/build/Makefile.feature | 4 +-
tools/build/feature/Makefile | 4 +-
tools/build/feature/test-all.c | 6 +-
tools/build/feature/test-libpython.c | 10 --
tools/build/feature/test-python-module.c | 13 ++
tools/perf/Documentation/perf-check.txt | 1 +
tools/perf/Makefile.config | 15 +-
tools/perf/Makefile.perf | 8 +-
tools/perf/builtin-check.c | 2 +-
tools/perf/scripts/install-build-deps.sh | 4 +-
tools/perf/tests/make | 8 +-
tools/perf/ui/browsers/scripts.c | 182 +++++++++++++++--------
12 files changed, 155 insertions(+), 102 deletions(-)
delete mode 100644 tools/build/feature/test-libpython.c
create mode 100644 tools/build/feature/test-python-module.c
diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
index 331f5cdfc34b..3ca7a4f5c7fd 100644
--- a/tools/build/Makefile.feature
+++ b/tools/build/Makefile.feature
@@ -79,7 +79,7 @@ FEATURE_TESTS_BASIC := \
libelf-zstd \
libnuma \
numa_num_possible_cpus \
- libpython \
+ python-module \
libslang \
libtraceevent \
libcpupower \
@@ -145,7 +145,7 @@ FEATURE_DISPLAY ?= \
libelf \
libnuma \
numa_num_possible_cpus \
- libpython \
+ python-module \
libcapstone \
llvm-perf \
zlib \
diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
index cdf89f132074..269af8e8f5cf 100644
--- a/tools/build/feature/Makefile
+++ b/tools/build/feature/Makefile
@@ -32,7 +32,7 @@ FILES= \
test-libnuma.bin \
test-numa_num_possible_cpus.bin \
test-libperl.bin \
- test-libpython.bin \
+ test-python-module.bin \
test-libslang.bin \
test-libtraceevent.bin \
test-libcpupower.bin \
@@ -261,7 +261,7 @@ endif
$(OUTPUT)test-libperl.bin:
$(BUILD) $(FLAGS_PERL_EMBED)
-$(OUTPUT)test-libpython.bin:
+$(OUTPUT)test-python-module.bin:
$(BUILD) $(FLAGS_PYTHON_EMBED)
$(OUTPUT)test-libbfd.bin:
diff --git a/tools/build/feature/test-all.c b/tools/build/feature/test-all.c
index 544563d62950..0ee16eccd9e0 100644
--- a/tools/build/feature/test-all.c
+++ b/tools/build/feature/test-all.c
@@ -10,8 +10,8 @@
* Quirk: Python headers cannot be in arbitrary places, so keep this testcase at
* the top:
*/
-#define main main_test_libpython
-# include "test-libpython.c"
+#define main main_test_python_module
+# include "test-python-module.c"
#undef main
#define main main_test_hello
@@ -148,7 +148,7 @@
int main(int argc, char *argv[])
{
- main_test_libpython();
+ main_test_python_module();
main_test_hello();
main_test_libelf();
main_test_gettid();
diff --git a/tools/build/feature/test-libpython.c b/tools/build/feature/test-libpython.c
deleted file mode 100644
index 371c9113e49d..000000000000
--- a/tools/build/feature/test-libpython.c
+++ /dev/null
@@ -1,10 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-#include <Python.h>
-
-int main(void)
-{
- Py_Initialize();
-
- return 0;
-}
-#undef _GNU_SOURCE
diff --git a/tools/build/feature/test-python-module.c b/tools/build/feature/test-python-module.c
new file mode 100644
index 000000000000..50e9e5062feb
--- /dev/null
+++ b/tools/build/feature/test-python-module.c
@@ -0,0 +1,13 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <Python.h>
+
+int main(void)
+{
+ static struct PyModuleDef moduledef = {
+ PyModuleDef_HEAD_INIT,
+ .m_name = "test",
+ };
+ PyObject *module = PyModule_Create(&moduledef);
+
+ return module ? 0 : -1;
+}
diff --git a/tools/perf/Documentation/perf-check.txt b/tools/perf/Documentation/perf-check.txt
index 3d169e5bb372..a80913794eb6 100644
--- a/tools/perf/Documentation/perf-check.txt
+++ b/tools/perf/Documentation/perf-check.txt
@@ -65,6 +65,7 @@ feature::
libunwind / HAVE_LIBUNWIND_SUPPORT
lzma / HAVE_LZMA_SUPPORT
numa_num_possible_cpus / HAVE_LIBNUMA_SUPPORT
+ python-module / HAVE_PYTHON_MODULE_SUPPORT
zlib / HAVE_ZLIB_SUPPORT
zstd / HAVE_ZSTD_SUPPORT
diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index 6e03d8f808b5..7e05b7943b3b 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -305,7 +305,7 @@ PYTHON_CONFIG_SQ := $(call shell-sq,$(PYTHON_CONFIG))
# Python 3.8 changed the output of `python-config --ldflags` to not include the
# '-lpythonX.Y' flag unless '--embed' is also passed. The feature check for
-# libpython fails if that flag is not included in LDFLAGS
+# python-module fails if that flag is not included in LDFLAGS
ifeq ($(shell $(PYTHON_CONFIG_SQ) --ldflags --embed 2>&1 1>/dev/null; echo $$?), 0)
PYTHON_CONFIG_LDFLAGS := --ldflags --embed
else
@@ -328,8 +328,8 @@ ifdef PYTHON_CONFIG
endif
endif
-FEATURE_CHECK_CFLAGS-libpython := $(PYTHON_EMBED_CCOPTS)
-FEATURE_CHECK_LDFLAGS-libpython := $(PYTHON_EMBED_LDOPTS)
+FEATURE_CHECK_CFLAGS-python-module := $(PYTHON_EMBED_CCOPTS)
+FEATURE_CHECK_LDFLAGS-python-module := $(PYTHON_EMBED_LDOPTS)
FEATURE_CHECK_LDFLAGS-libaio = -lrt
@@ -803,13 +803,12 @@ endif
disable-python = $(eval $(disable-python_code))
define disable-python_code
- CFLAGS += -DNO_LIBPYTHON
$(warning $1)
- NO_LIBPYTHON := 1
+ NO_PYTHON_MODULE := 1
endef
PYTHON_EXTENSION_SUFFIX := '.so'
-ifdef NO_LIBPYTHON
+ifdef NO_PYTHON_MODULE
$(call disable-python,Python support disabled by user)
else
@@ -822,10 +821,10 @@ else
$(call disable-python,No 'python-config' tool was found: disables Python support - please install python-devel/python-dev)
else
- ifneq ($(feature-libpython), 1)
+ ifneq ($(feature-python-module), 1)
$(call disable-python,No 'Python.h' was found: disables Python support - please install python-devel/python-dev)
else
- CFLAGS += -DHAVE_LIBPYTHON_SUPPORT
+ CFLAGS += -DHAVE_PYTHON_MODULE_SUPPORT
PYTHON_SETUPTOOLS_INSTALLED := $(shell $(PYTHON) -c 'import setuptools;' 2> /dev/null && echo "yes" || echo "no")
ifeq ($(PYTHON_SETUPTOOLS_INSTALLED), yes)
PYTHON_EXTENSION_SUFFIX := $(shell $(PYTHON) -c 'from importlib import machinery; print(machinery.EXTENSION_SUFFIXES[0])')
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 45e5a860d537..6bcea56f1f90 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -19,7 +19,7 @@ include ../scripts/utilities.mak
#
# Define LIBPERL to enable perl script extension.
#
-# Define NO_LIBPYTHON to disable python script extension.
+# Define NO_PYTHON_MODULE to disable python script extension.
#
# Define PYTHON to point to the python binary if the default
# `python' is not correct; for example: PYTHON=python2
@@ -898,11 +898,9 @@ ifdef LIBPERL
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/perl/bin'; \
$(INSTALL) scripts/perl/bin/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/perl/bin'
endif
-ifndef NO_LIBPYTHON
+
+ifndef NO_PYTHON_MODULE
$(call QUIET_INSTALL, python-scripts) \
- $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'; \
- $(INSTALL) python/*.py -m 644 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'
- $(call QUIET_INSTALL, python-scripts-standalone) \
$(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/python'; \
$(INSTALL) python/*.py -m 755 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/python'
endif
diff --git a/tools/perf/builtin-check.c b/tools/perf/builtin-check.c
index 35272aaeb613..06711362d37f 100644
--- a/tools/perf/builtin-check.c
+++ b/tools/perf/builtin-check.c
@@ -52,7 +52,7 @@ struct feature_status supported_features[] = {
FEATURE_STATUS("libnuma", HAVE_LIBNUMA_SUPPORT),
FEATURE_STATUS("libopencsd", HAVE_CSTRACE_SUPPORT),
FEATURE_STATUS_TIP("libperl", HAVE_LIBPERL_SUPPORT, "Deprecated, use LIBPERL=1 and install perl-ExtUtils-Embed/libperl-dev to build with it"),
- FEATURE_STATUS("python-module", HAVE_LIBPYTHON_SUPPORT),
+ FEATURE_STATUS("python-module", HAVE_PYTHON_MODULE_SUPPORT),
FEATURE_STATUS("libpfm4", HAVE_LIBPFM),
FEATURE_STATUS("libslang", HAVE_SLANG_SUPPORT),
FEATURE_STATUS("libtraceevent", HAVE_LIBTRACEEVENT),
diff --git a/tools/perf/scripts/install-build-deps.sh b/tools/perf/scripts/install-build-deps.sh
index a601a5260c17..3e53c7bf0f85 100755
--- a/tools/perf/scripts/install-build-deps.sh
+++ b/tools/perf/scripts/install-build-deps.sh
@@ -155,7 +155,7 @@ fedora_pkg_for() {
libcapstone)
echo "capstone-devel"
;;
- libpython)
+ python-module)
echo "python3-devel"
;;
libtraceevent)
@@ -314,7 +314,7 @@ debian_pkg_for() {
libcapstone)
echo "libcapstone-dev"
;;
- libpython)
+ python-module)
echo "python3-dev"
;;
libtraceevent)
diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index f879f8109072..b5eaf326573c 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -77,8 +77,8 @@ make_jevents_all := JEVENTS_ARCH=all
make_no_bpf_skel := BUILD_BPF_SKEL=0
make_gen_vmlinux_h := GEN_VMLINUX_H=1
make_libperl := LIBPERL=1
-make_no_libpython := NO_LIBPYTHON=1
-make_no_scripts := NO_LIBPYTHON=1
+make_no_python_module := NO_PYTHON_MODULE=1
+make_no_scripts := NO_PYTHON_MODULE=1
make_no_slang := NO_SLANG=1
make_no_demangle := NO_DEMANGLE=1
make_no_libelf := NO_LIBELF=1
@@ -118,7 +118,7 @@ make_install_prefix_slash := install prefix=/tmp/krava/
make_static := LDFLAGS=-static NO_PERF_READ_VDSO32=1 NO_PERF_READ_VDSOX32=1 NO_JVMTI=1 NO_LIBTRACEEVENT=1 NO_LIBELF=1
# all the NO_* variable combined
-make_minimal := NO_LIBPYTHON=1
+make_minimal := NO_PYTHON_MODULE=1
make_minimal += NO_DEMANGLE=1 NO_LIBELF=1 NO_BACKTRACE=1
make_minimal += NO_LIBNUMA=1 NO_LIBBIONIC=1 NO_LIBDW=1
make_minimal += NO_LIBBPF=1
@@ -150,7 +150,7 @@ run += make_jevents_all
run += make_no_bpf_skel
run += make_gen_vmlinux_h
run += make_libperl
-run += make_no_libpython
+run += make_no_python_module
run += make_no_scripts
run += make_no_slang
run += make_no_demangle
diff --git a/tools/perf/ui/browsers/scripts.c b/tools/perf/ui/browsers/scripts.c
index 94cc1f427c96..7bf967e6cd19 100644
--- a/tools/perf/ui/browsers/scripts.c
+++ b/tools/perf/ui/browsers/scripts.c
@@ -1,4 +1,11 @@
// SPDX-License-Identifier: GPL-2.0
+#include <dirent.h>
+#include <fcntl.h>
+#include <limits.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
#include "../../util/util.h" // perf_exe()
#include "../util.h"
#include "../../util/evlist.h"
@@ -14,7 +21,6 @@
#include <linux/string.h>
#include <linux/zalloc.h>
#include <subcmd/exec-cmd.h>
-#include <stdlib.h>
#define SCRIPT_NAMELEN 128
#define SCRIPT_MAX_NO 64
@@ -128,7 +134,7 @@ static int check_ev_match(int dir_fd, const char *scriptname, struct perf_sessio
if (!len)
break;
- snprintf(evname, len + 1, "%s", p);
+ snprintf(evname, sizeof(evname), "%.*s", (int)len, p);
match = 0;
evlist__for_each_entry(session->evlist, pos) {
@@ -159,6 +165,7 @@ static int check_ev_match(int dir_fd, const char *scriptname, struct perf_sessio
static int find_scripts(char **scripts_array, char **scripts_path_array, int num,
int pathlen)
{
+ int namelen;
struct dirent *script_dirent, *lang_dirent;
int scripts_dir_fd, lang_dir_fd;
DIR *scripts_dir, *lang_dir;
@@ -180,73 +187,118 @@ static int find_scripts(char **scripts_array, char **scripts_path_array, int num
snprintf(scripts_path, sizeof(scripts_path), "%s/scripts", exec_path);
scripts_dir_fd = open(scripts_path, O_DIRECTORY);
- pr_err("Failed to open directory '%s'", scripts_path);
- if (scripts_dir_fd == -1) {
- perf_session__delete(session);
- return -1;
- }
}
- scripts_dir = fdopendir(scripts_dir_fd);
- if (!scripts_dir) {
- close(scripts_dir_fd);
- perf_session__delete(session);
- return -1;
+ if (scripts_dir_fd != -1) {
+ scripts_dir = fdopendir(scripts_dir_fd);
+ if (scripts_dir) {
+ while ((lang_dirent = readdir(scripts_dir)) != NULL) {
+ if (lang_dirent->d_type != DT_DIR &&
+ (lang_dirent->d_type == DT_UNKNOWN &&
+ !is_directory_at(scripts_dir_fd, lang_dirent->d_name)))
+ continue;
+ if (!strcmp(lang_dirent->d_name, ".") ||
+ !strcmp(lang_dirent->d_name, ".."))
+ continue;
+
+ if (strstr(lang_dirent->d_name, "python"))
+ continue;
+
+ lang_dir_fd = openat(scripts_dir_fd, lang_dirent->d_name,
+ O_DIRECTORY);
+ if (lang_dir_fd == -1)
+ continue;
+ lang_dir = fdopendir(lang_dir_fd);
+ if (!lang_dir) {
+ close(lang_dir_fd);
+ continue;
+ }
+ while ((script_dirent = readdir(lang_dir)) != NULL) {
+ if (script_dirent->d_type == DT_DIR)
+ continue;
+ if (script_dirent->d_type == DT_UNKNOWN &&
+ is_directory_at(lang_dir_fd, script_dirent->d_name))
+ continue;
+ /* Skip those real time scripts: xxxtop.p[yl] */
+ if (strstr(script_dirent->d_name, "top."))
+ continue;
+ if (i >= num)
+ break;
+ scnprintf(scripts_path_array[i], pathlen,
+ "%s/scripts/%s/%s", exec_path,
+ lang_dirent->d_name,
+ script_dirent->d_name);
+ temp = strrchr(script_dirent->d_name, '.');
+ namelen = temp ? (int)(temp - script_dirent->d_name)
+ : (int)strlen(script_dirent->d_name);
+
+ if (namelen >= SCRIPT_NAMELEN)
+ namelen = SCRIPT_NAMELEN - 1;
+ snprintf(scripts_array[i], namelen + 1, "%s",
+ script_dirent->d_name);
+
+ if (check_ev_match(lang_dir_fd, scripts_array[i], session))
+ continue;
+
+ i++;
+ }
+ closedir(lang_dir);
+ }
+ closedir(scripts_dir);
+ } else {
+ close(scripts_dir_fd);
+ }
}
- while ((lang_dirent = readdir(scripts_dir)) != NULL) {
- if (lang_dirent->d_type != DT_DIR &&
- (lang_dirent->d_type == DT_UNKNOWN &&
- !is_directory_at(scripts_dir_fd, lang_dirent->d_name)))
- continue;
- if (!strcmp(lang_dirent->d_name, ".") || !strcmp(lang_dirent->d_name, ".."))
- continue;
-
-#ifndef HAVE_LIBPERL_SUPPORT
- if (strstr(lang_dirent->d_name, "perl"))
- continue;
-#endif
-#ifndef HAVE_LIBPYTHON_SUPPORT
- if (strstr(lang_dirent->d_name, "python"))
- continue;
-#endif
-
- lang_dir_fd = openat(scripts_dir_fd, lang_dirent->d_name, O_DIRECTORY);
- if (lang_dir_fd == -1)
- continue;
- lang_dir = fdopendir(lang_dir_fd);
- if (!lang_dir) {
- close(lang_dir_fd);
- continue;
- }
- while ((script_dirent = readdir(lang_dir)) != NULL) {
- if (script_dirent->d_type == DT_DIR)
- continue;
- if (script_dirent->d_type == DT_UNKNOWN &&
- is_directory_at(lang_dir_fd, script_dirent->d_name))
- continue;
- /* Skip those real time scripts: xxxtop.p[yl] */
- if (strstr(script_dirent->d_name, "top."))
- continue;
- if (i >= num)
- break;
- scnprintf(scripts_path_array[i], pathlen, "%s/scripts/%s/%s",
- exec_path,
- lang_dirent->d_name,
- script_dirent->d_name);
- temp = strchr(script_dirent->d_name, '.');
- snprintf(scripts_array[i],
- (temp - script_dirent->d_name) + 1,
- "%s", script_dirent->d_name);
-
- if (check_ev_match(lang_dir_fd, scripts_array[i], session))
- continue;
-
- i++;
+#ifdef HAVE_PYTHON_MODULE_SUPPORT
+ {
+ char py_scripts_path[PATH_MAX];
+ int py_scripts_dir_fd;
+ DIR *py_scripts_dir;
+ int len;
+
+ snprintf(py_scripts_path, sizeof(py_scripts_path), "%s/python", exec_path);
+ py_scripts_dir_fd = open(py_scripts_path, O_DIRECTORY);
+ if (py_scripts_dir_fd != -1) {
+ py_scripts_dir = fdopendir(py_scripts_dir_fd);
+ if (py_scripts_dir) {
+ while ((script_dirent = readdir(py_scripts_dir)) != NULL) {
+ if (script_dirent->d_type == DT_DIR)
+ continue;
+ if (script_dirent->d_type == DT_UNKNOWN &&
+ is_directory_at(py_scripts_dir_fd,
+ script_dirent->d_name))
+ continue;
+ /* Skip those real time scripts: xxxtop.p[yl] */
+ if (strstr(script_dirent->d_name, "top."))
+ continue;
+ if (i >= num)
+ break;
+ len = strlen(script_dirent->d_name);
+ if (len <= 3 ||
+ strcmp(script_dirent->d_name + len - 3, ".py"))
+ continue;
+
+ scnprintf(scripts_path_array[i], pathlen, "%s/python/%s",
+ exec_path,
+ script_dirent->d_name);
+ temp = strrchr(script_dirent->d_name, '.');
+ namelen = temp ? (int)(temp - script_dirent->d_name)
+ : (int)strlen(script_dirent->d_name);
+
+ if (namelen >= SCRIPT_NAMELEN)
+ namelen = SCRIPT_NAMELEN - 1;
+ snprintf(scripts_array[i], namelen + 1, "%s",
+ script_dirent->d_name);
+
+ i++;
+ }
+ closedir(py_scripts_dir);
+ } else {
+ close(py_scripts_dir_fd);
+ }
}
- closedir(lang_dir);
}
-
- closedir(scripts_dir);
+#endif
perf_session__delete(session);
return i;
}
@@ -264,7 +316,7 @@ static int list_scripts(char *script_name, bool *custom,
int ret = 0;
int max_std, custom_perf;
char pbuf[256];
- const char *perf = perf_exe(pbuf, sizeof pbuf);
+ const char *perf = perf_exe(pbuf, sizeof(pbuf));
struct script_config scriptc = {
.names = (const char **)names,
.paths = paths,
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 48/49] perf script: Support standalone scripts and remove embedded scripting
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (46 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 47/49] perf Makefile: Update Python script installation path Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
2026-09-20 5:21 ` [PATCH v1 49/49] perf Documentation: Update for standalone Python scripts Ian Rogers
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Refactor 'perf script' to launch standalone scripts directly via fork()
and execvp() and remove the legacy embedded Perl and Python scripting
engines:
- Remove the embedded Perl scripting engine
(util/scripting-engines/trace-event-perl.c), Perl scripts, bin
wrappers, and Trace-Util library
(scripts/perl/Perf-Trace-Util/), and script_perl.sh test.
- Remove libperl feature checks from Makefile.config, Makefile.perf,
builtin-check.c, and Documentation/perf-check.txt.
- Remove -g / --gen-script option and scripting_ops dispatch table from
builtin-script.c and trace-event-scripting.c.
- Hide the legacy -s / --script option and update script discovery in
find_script() and list_available_scripts() to prioritize the system
'python' directory over bare filenames in the current directory.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/build/Makefile.feature | 1 -
tools/build/feature/Makefile | 20 -
tools/build/feature/test-libperl.c | 10 -
tools/perf/Build | 1 -
tools/perf/Documentation/perf-check.txt | 1 -
tools/perf/Makefile.config | 1 -
tools/perf/Makefile.perf | 14 -
tools/perf/builtin-check.c | 1 -
tools/perf/builtin-script.c | 824 ++++++++----------
tools/perf/scripts/Build | 3 -
tools/perf/scripts/perl/Perf-Trace-Util/Build | 9 -
.../scripts/perl/Perf-Trace-Util/Context.c | 122 ---
.../scripts/perl/Perf-Trace-Util/Context.xs | 42 -
.../scripts/perl/Perf-Trace-Util/Makefile.PL | 18 -
.../perf/scripts/perl/Perf-Trace-Util/README | 59 --
.../Perf-Trace-Util/lib/Perf/Trace/Context.pm | 55 --
.../Perf-Trace-Util/lib/Perf/Trace/Core.pm | 192 ----
.../Perf-Trace-Util/lib/Perf/Trace/Util.pm | 94 --
.../perf/scripts/perl/Perf-Trace-Util/typemap | 1 -
.../scripts/perl/bin/check-perf-trace-record | 2 -
.../scripts/perl/bin/failed-syscalls-record | 3 -
.../scripts/perl/bin/failed-syscalls-report | 10 -
tools/perf/scripts/perl/bin/rw-by-file-record | 3 -
tools/perf/scripts/perl/bin/rw-by-file-report | 10 -
tools/perf/scripts/perl/bin/rw-by-pid-record | 2 -
tools/perf/scripts/perl/bin/rw-by-pid-report | 3 -
tools/perf/scripts/perl/bin/rwtop-record | 2 -
tools/perf/scripts/perl/bin/rwtop-report | 20 -
.../scripts/perl/bin/wakeup-latency-record | 6 -
.../scripts/perl/bin/wakeup-latency-report | 3 -
tools/perf/scripts/perl/check-perf-trace.pl | 106 ---
tools/perf/scripts/perl/failed-syscalls.pl | 47 -
tools/perf/scripts/perl/rw-by-file.pl | 106 ---
tools/perf/scripts/perl/rw-by-pid.pl | 184 ----
tools/perf/scripts/perl/rwtop.pl | 203 -----
tools/perf/scripts/perl/wakeup-latency.pl | 107 ---
tools/perf/tests/make | 2 -
tools/perf/tests/shell/script_perl.sh | 102 ---
tools/perf/ui/browsers/scripts.c | 161 +---
tools/perf/util/Build | 2 -
tools/perf/util/scripting-engines/Build | 5 -
.../util/scripting-engines/trace-event-perl.c | 770 ----------------
tools/perf/util/trace-event-parse.c | 65 --
tools/perf/util/trace-event-scripting.c | 398 ---------
tools/perf/util/trace-event.h | 72 +-
45 files changed, 355 insertions(+), 3507 deletions(-)
delete mode 100644 tools/build/feature/test-libperl.c
delete mode 100644 tools/perf/scripts/Build
delete mode 100644 tools/perf/scripts/perl/Perf-Trace-Util/Build
delete mode 100644 tools/perf/scripts/perl/Perf-Trace-Util/Context.c
delete mode 100644 tools/perf/scripts/perl/Perf-Trace-Util/Context.xs
delete mode 100644 tools/perf/scripts/perl/Perf-Trace-Util/Makefile.PL
delete mode 100644 tools/perf/scripts/perl/Perf-Trace-Util/README
delete mode 100644 tools/perf/scripts/perl/Perf-Trace-Util/lib/Perf/Trace/Context.pm
delete mode 100644 tools/perf/scripts/perl/Perf-Trace-Util/lib/Perf/Trace/Core.pm
delete mode 100644 tools/perf/scripts/perl/Perf-Trace-Util/lib/Perf/Trace/Util.pm
delete mode 100644 tools/perf/scripts/perl/Perf-Trace-Util/typemap
delete mode 100644 tools/perf/scripts/perl/bin/check-perf-trace-record
delete mode 100644 tools/perf/scripts/perl/bin/failed-syscalls-record
delete mode 100644 tools/perf/scripts/perl/bin/failed-syscalls-report
delete mode 100644 tools/perf/scripts/perl/bin/rw-by-file-record
delete mode 100644 tools/perf/scripts/perl/bin/rw-by-file-report
delete mode 100644 tools/perf/scripts/perl/bin/rw-by-pid-record
delete mode 100644 tools/perf/scripts/perl/bin/rw-by-pid-report
delete mode 100644 tools/perf/scripts/perl/bin/rwtop-record
delete mode 100644 tools/perf/scripts/perl/bin/rwtop-report
delete mode 100644 tools/perf/scripts/perl/bin/wakeup-latency-record
delete mode 100644 tools/perf/scripts/perl/bin/wakeup-latency-report
delete mode 100644 tools/perf/scripts/perl/check-perf-trace.pl
delete mode 100644 tools/perf/scripts/perl/failed-syscalls.pl
delete mode 100644 tools/perf/scripts/perl/rw-by-file.pl
delete mode 100644 tools/perf/scripts/perl/rw-by-pid.pl
delete mode 100644 tools/perf/scripts/perl/rwtop.pl
delete mode 100644 tools/perf/scripts/perl/wakeup-latency.pl
delete mode 100755 tools/perf/tests/shell/script_perl.sh
delete mode 100644 tools/perf/util/scripting-engines/Build
delete mode 100644 tools/perf/util/scripting-engines/trace-event-perl.c
delete mode 100644 tools/perf/util/trace-event-scripting.c
diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
index 3ca7a4f5c7fd..dbef5b6c4e43 100644
--- a/tools/build/Makefile.feature
+++ b/tools/build/Makefile.feature
@@ -121,7 +121,6 @@ FEATURE_TESTS_EXTRA := \
libbfd-liberty \
libbfd-liberty-z \
libopencsd \
- libperl \
llvm \
libbpf \
libpfm4 \
diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
index 269af8e8f5cf..62f9cbf3dee9 100644
--- a/tools/build/feature/Makefile
+++ b/tools/build/feature/Makefile
@@ -31,7 +31,6 @@ FILES= \
test-libdebuginfod.bin \
test-libnuma.bin \
test-numa_num_possible_cpus.bin \
- test-libperl.bin \
test-python-module.bin \
test-libslang.bin \
test-libtraceevent.bin \
@@ -242,25 +241,6 @@ $(OUTPUT)test-libtracefs.bin:
$(OUTPUT)test-gtk4.bin:
$(BUILD) $(shell $(PKG_CONFIG) --libs --cflags gtk4 2>/dev/null)
-grep-libs = $(filter -l%,$(1))
-strip-libs = $(filter-out -l%,$(1))
-
-PERL_EMBED_LDOPTS = $(shell perl -MExtUtils::Embed -e ldopts 2>/dev/null)
-PERL_EMBED_LDFLAGS = $(call strip-libs,$(PERL_EMBED_LDOPTS))
-PERL_EMBED_LIBADD = $(call grep-libs,$(PERL_EMBED_LDOPTS))
-PERL_EMBED_CCOPTS = $(shell perl -MExtUtils::Embed -e ccopts 2>/dev/null)
-FLAGS_PERL_EMBED=$(PERL_EMBED_CCOPTS) $(PERL_EMBED_LDOPTS)
-
-ifeq ($(CC_NO_CLANG), 0)
- PERL_EMBED_LDOPTS := $(filter-out -specs=%,$(PERL_EMBED_LDOPTS))
- PERL_EMBED_CCOPTS := $(filter-out -flto=auto -ffat-lto-objects, $(PERL_EMBED_CCOPTS))
- PERL_EMBED_CCOPTS := $(filter-out -specs=%,$(PERL_EMBED_CCOPTS))
- FLAGS_PERL_EMBED += -Wno-compound-token-split-by-macro
-endif
-
-$(OUTPUT)test-libperl.bin:
- $(BUILD) $(FLAGS_PERL_EMBED)
-
$(OUTPUT)test-python-module.bin:
$(BUILD) $(FLAGS_PYTHON_EMBED)
diff --git a/tools/build/feature/test-libperl.c b/tools/build/feature/test-libperl.c
deleted file mode 100644
index 0415f437eb31..000000000000
--- a/tools/build/feature/test-libperl.c
+++ /dev/null
@@ -1,10 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0
-#include <EXTERN.h>
-#include <perl.h>
-
-int main(void)
-{
- perl_alloc();
-
- return 0;
-}
diff --git a/tools/perf/Build b/tools/perf/Build
index 5bd0e58d7cb0..8a0b5592a973 100644
--- a/tools/perf/Build
+++ b/tools/perf/Build
@@ -61,7 +61,6 @@ perf-util-y += arch/
perf-y += arch/
perf-test-y += arch/
perf-ui-y += ui/
-perf-util-y += scripts/
gtk-y += ui/gtk/
diff --git a/tools/perf/Documentation/perf-check.txt b/tools/perf/Documentation/perf-check.txt
index a80913794eb6..1406a41c5d09 100644
--- a/tools/perf/Documentation/perf-check.txt
+++ b/tools/perf/Documentation/perf-check.txt
@@ -58,7 +58,6 @@ feature::
libLLVM / HAVE_LIBLLVM_SUPPORT
libnuma / HAVE_LIBNUMA_SUPPORT
libopencsd / HAVE_CSTRACE_SUPPORT
- libperl / HAVE_LIBPERL_SUPPORT
libpfm4 / HAVE_LIBPFM
libslang / HAVE_SLANG_SUPPORT
libtraceevent / HAVE_LIBTRACEEVENT
diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index 7e05b7943b3b..5c404289780b 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -1275,7 +1275,6 @@ $(call detected_var,tipdir_SQ)
$(call detected_var,srcdir_SQ)
$(call detected_var,LIBDIR)
$(call detected_var,GTK_CFLAGS)
-$(call detected_var,PERL_EMBED_CCOPTS)
$(call detected_var,PYTHON_EMBED_CCOPTS)
ifneq ($(BISON_FILE_PREFIX_MAP),)
$(call detected_var,BISON_FILE_PREFIX_MAP)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 6bcea56f1f90..57e1a1a04e8a 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -17,7 +17,6 @@ include ../scripts/utilities.mak
#
# Define CROSS_COMPILE as prefix name of compiler if you want cross-builds.
#
-# Define LIBPERL to enable perl script extension.
#
# Define NO_PYTHON_MODULE to disable python script extension.
#
@@ -481,11 +480,6 @@ OTHER_PROGRAMS = $(OUTPUT)perf
ifndef SHELL_PATH
SHELL_PATH = /bin/sh
endif
-ifndef PERL_PATH
- PERL_PATH = /usr/bin/perl
-endif
-
-export PERL_PATH
LIBPERF_BENCH_IN := $(OUTPUT)perf-bench-in.o
LIBPERF_BENCH := $(OUTPUT)libperf-bench.a
@@ -890,14 +884,6 @@ endif
$(INSTALL) $(OUTPUT)perf-archive -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)'
$(call QUIET_INSTALL, perf-iostat) \
$(INSTALL) $(OUTPUT)perf-iostat -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)'
-ifdef LIBPERL
- $(call QUIET_INSTALL, perl-scripts) \
- $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/perl/Perf-Trace-Util/lib/Perf/Trace'; \
- $(INSTALL) scripts/perl/Perf-Trace-Util/lib/Perf/Trace/* -m 644 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/perl/Perf-Trace-Util/lib/Perf/Trace'; \
- $(INSTALL) scripts/perl/*.pl -m 644 -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/perl'; \
- $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/perl/bin'; \
- $(INSTALL) scripts/perl/bin/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/perl/bin'
-endif
ifndef NO_PYTHON_MODULE
$(call QUIET_INSTALL, python-scripts) \
diff --git a/tools/perf/builtin-check.c b/tools/perf/builtin-check.c
index 06711362d37f..1e43e097e7a7 100644
--- a/tools/perf/builtin-check.c
+++ b/tools/perf/builtin-check.c
@@ -51,7 +51,6 @@ struct feature_status supported_features[] = {
FEATURE_STATUS("libLLVM", HAVE_LIBLLVM_SUPPORT),
FEATURE_STATUS("libnuma", HAVE_LIBNUMA_SUPPORT),
FEATURE_STATUS("libopencsd", HAVE_CSTRACE_SUPPORT),
- FEATURE_STATUS_TIP("libperl", HAVE_LIBPERL_SUPPORT, "Deprecated, use LIBPERL=1 and install perl-ExtUtils-Embed/libperl-dev to build with it"),
FEATURE_STATUS("python-module", HAVE_PYTHON_MODULE_SUPPORT),
FEATURE_STATUS("libpfm4", HAVE_LIBPFM),
FEATURE_STATUS("libslang", HAVE_SLANG_SUPPORT),
diff --git a/tools/perf/builtin-script.c b/tools/perf/builtin-script.c
index 0174489d1c0f..c9e1b5ac5c8f 100644
--- a/tools/perf/builtin-script.c
+++ b/tools/perf/builtin-script.c
@@ -3,6 +3,9 @@
#include <inttypes.h>
#include <signal.h>
#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <strings.h>
#include <dirent.h>
#include <fcntl.h>
@@ -18,6 +21,7 @@
#include <sys/param.h>
#include <sys/stat.h>
#include <sys/types.h>
+#include <sys/wait.h>
#include <sys/utsname.h>
#include <unistd.h>
@@ -78,7 +82,6 @@
#endif
static char const *script_name;
-static char const *generate_script_lang;
static bool reltime;
static bool deltatime;
static u64 initial_time;
@@ -88,7 +91,6 @@ static u64 last_timestamp;
static u64 nr_unordered;
static bool no_callchain;
static bool latency_format;
-static bool system_wide;
static bool print_flags;
static const char *cpu_list;
static DECLARE_BITMAP(cpu_bitmap, MAX_NR_CPUS);
@@ -96,6 +98,7 @@ static int max_blocks;
static struct dlfilter *dlfilter;
static int dlargc;
static char **dlargv;
+static unsigned int scripting_max_stack = PERF_MAX_STACK_DEPTH;
enum perf_output_field {
PERF_OUTPUT_COMM = 1ULL << 0,
@@ -1810,6 +1813,144 @@ static int perf_sample__fprintf_bts(struct perf_sample *sample,
return printed;
}
+#define SAMPLE_FLAGS_BUF_SIZE 64
+#define SAMPLE_FLAGS_STR_ALIGNED_SIZE 21
+
+static int sample_flags_to_name(u32 flags, char *str, size_t size)
+{
+ static const struct {
+ u32 flags;
+ const char *name;
+ } sample_flags[] = {
+ {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL, "call"},
+ {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN, "return"},
+ {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CONDITIONAL, "jcc"},
+ {PERF_IP_FLAG_BRANCH, "jmp"},
+ {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_INTERRUPT, "int"},
+ {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN | PERF_IP_FLAG_INTERRUPT, "iret"},
+ {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_SYSCALLRET, "syscall"},
+ {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN | PERF_IP_FLAG_SYSCALLRET, "sysret"},
+ {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_ASYNC, "async"},
+ {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL |
+ PERF_IP_FLAG_ASYNC | PERF_IP_FLAG_INTERRUPT,
+ "hw int"},
+ {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TX_ABORT, "tx abrt"},
+ {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TRACE_BEGIN, "tr strt"},
+ {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TRACE_END, "tr end"},
+ {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_VMENTRY, "vmentry"},
+ {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_VMEXIT, "vmexit"},
+ {0, NULL}
+ };
+ static const struct {
+ u32 flags;
+ const char *name;
+ } branch_events[] = {
+ {PERF_IP_FLAG_BRANCH_MISS, "miss"},
+ {PERF_IP_FLAG_NOT_TAKEN, "not_taken"},
+ {0, NULL}
+ };
+ int i;
+ const char *prefix;
+ int pos = 0, ret, ev_idx = 0;
+ u32 xf = flags & PERF_ADDITIONAL_STATE_MASK;
+ u32 types, events;
+ char xs[16] = { 0 };
+
+ /* Clear additional state bits */
+ flags &= ~PERF_ADDITIONAL_STATE_MASK;
+
+ if (flags & PERF_IP_FLAG_TRACE_BEGIN)
+ prefix = "tr strt ";
+ else if (flags & PERF_IP_FLAG_TRACE_END)
+ prefix = "tr end ";
+ else
+ prefix = "";
+
+ ret = snprintf(str + pos, size - pos, "%s", prefix);
+ if (ret < 0)
+ return ret;
+ pos += ret;
+
+ flags &= ~(PERF_IP_FLAG_TRACE_BEGIN | PERF_IP_FLAG_TRACE_END);
+
+ types = flags & ~PERF_IP_FLAG_BRANCH_EVENT_MASK;
+ for (i = 0; sample_flags[i].name; i++) {
+ if (sample_flags[i].flags != types)
+ continue;
+
+ ret = snprintf(str + pos, size - pos, "%s", sample_flags[i].name);
+ if (ret < 0)
+ return ret;
+ pos += ret;
+ break;
+ }
+
+ events = flags & PERF_IP_FLAG_BRANCH_EVENT_MASK;
+ for (i = 0; branch_events[i].name; i++) {
+ if (!(branch_events[i].flags & events))
+ continue;
+
+ ret = snprintf(str + pos, size - pos, !ev_idx ? "/%s" : ",%s",
+ branch_events[i].name);
+ if (ret < 0)
+ return ret;
+ pos += ret;
+ ev_idx++;
+ }
+
+ /* Add an end character '/' for events */
+ if (ev_idx) {
+ ret = snprintf(str + pos, size - pos, "/");
+ if (ret < 0)
+ return ret;
+ pos += ret;
+ }
+
+ if (!xf)
+ return pos;
+
+ snprintf(xs, sizeof(xs), "(%s%s%s)",
+ flags & PERF_IP_FLAG_IN_TX ? "x" : "",
+ flags & PERF_IP_FLAG_INTR_DISABLE ? "D" : "",
+ flags & PERF_IP_FLAG_INTR_TOGGLE ? "t" : "");
+
+ /* Right align the string if its length is less than the limit */
+ if ((pos + strlen(xs)) < SAMPLE_FLAGS_STR_ALIGNED_SIZE)
+ ret = snprintf(str + pos, size - pos, "%*s",
+ (int)(SAMPLE_FLAGS_STR_ALIGNED_SIZE - ret), xs);
+ else
+ ret = snprintf(str + pos, size - pos, " %s", xs);
+ if (ret < 0)
+ return ret;
+
+ return pos + ret;
+}
+
+static int perf_sample__sprintf_flags(u32 flags, char *str, size_t sz)
+{
+ const char *chars = PERF_IP_FLAG_CHARS;
+ const size_t n = strlen(PERF_IP_FLAG_CHARS);
+ size_t i, pos = 0;
+ int ret;
+
+ ret = sample_flags_to_name(flags, str, sz);
+ if (ret > 0)
+ return ret;
+
+ for (i = 0; i < n; i++, flags >>= 1) {
+ if ((flags & 1) && pos < sz)
+ str[pos++] = chars[i];
+ }
+ for (; i < 32; i++, flags >>= 1) {
+ if ((flags & 1) && pos < sz)
+ str[pos++] = '?';
+ }
+ if (pos < sz)
+ str[pos] = 0;
+
+ return pos;
+}
+
static int perf_sample__fprintf_flags(u32 flags, FILE *fp)
{
char str[SAMPLE_FLAGS_BUF_SIZE];
@@ -2653,8 +2794,6 @@ static void process_event(struct perf_script *script,
fflush(fp);
}
-static struct scripting_ops *scripting_ops;
-
static void __process_stat(struct evsel *counter, u64 tstamp)
{
int nthreads = perf_thread_map__nr(counter->core.threads);
@@ -2689,37 +2828,14 @@ static void __process_stat(struct evsel *counter, u64 tstamp)
static void process_stat(struct evsel *counter, u64 tstamp)
{
- if (scripting_ops && scripting_ops->process_stat)
- scripting_ops->process_stat(&stat_config, counter, tstamp);
- else
- __process_stat(counter, tstamp);
-}
-
-static void process_stat_interval(u64 tstamp)
-{
- if (scripting_ops && scripting_ops->process_stat_interval)
- scripting_ops->process_stat_interval(tstamp);
-}
-
-static void setup_scripting(void)
-{
-#ifdef HAVE_LIBTRACEEVENT
- setup_perl_scripting();
-#endif
- setup_python_scripting();
+ __process_stat(counter, tstamp);
}
-static int flush_scripting(void)
+static void process_stat_interval(u64 tstamp __maybe_unused)
{
- return scripting_ops ? scripting_ops->flush_script() : 0;
}
-static int cleanup_scripting(void)
-{
- pr_debug("\nperf script stopped\n");
- return scripting_ops ? scripting_ops->stop_script() : 0;
-}
static bool filter_cpu(struct perf_sample *sample)
{
@@ -2793,20 +2909,7 @@ static int process_sample_event(const struct perf_tool *tool,
goto out_put;
}
- if (scripting_ops) {
- struct addr_location *addr_al_ptr = NULL;
-
- if ((evsel->core.attr.sample_type & PERF_SAMPLE_ADDR) &&
- sample_addr_correlates_sym(&evsel->core.attr)) {
- if (!addr_al.thread)
- thread__resolve(al.thread, &addr_al, sample);
- addr_al_ptr = &addr_al;
- }
- scripting_ops->process_event(event, sample, &al, addr_al_ptr);
- } else {
- process_event(scr, sample, &al, &addr_al, machine);
- }
-
+ process_event(scr, sample, &al, &addr_al, machine);
out_put:
addr_location__exit(&addr_al);
addr_location__exit(&al);
@@ -3118,8 +3221,7 @@ static int process_switch_event(const struct perf_tool *tool,
if (perf_event__process_switch(tool, event, sample, machine) < 0)
return -1;
- if (scripting_ops && scripting_ops->process_switch && !filter_cpu(sample))
- scripting_ops->process_switch(event, sample, machine);
+
if (!script->show_switch_events)
return 0;
@@ -3128,17 +3230,7 @@ static int process_switch_event(const struct perf_tool *tool,
sample->tid);
}
-static int process_auxtrace_error(const struct perf_tool *tool,
- struct perf_session *session,
- union perf_event *event)
-{
- if (scripting_ops && scripting_ops->process_auxtrace_error) {
- scripting_ops->process_auxtrace_error(session, event);
- return 0;
- }
- return perf_event__process_auxtrace_error(tool, session, event);
-}
static int
process_lost_event(const struct perf_tool *tool,
@@ -3152,12 +3244,11 @@ process_lost_event(const struct perf_tool *tool,
static int
process_throttle_event(const struct perf_tool *tool __maybe_unused,
- union perf_event *event,
- struct perf_sample *sample,
- struct machine *machine)
+ union perf_event *event __maybe_unused,
+ struct perf_sample *sample __maybe_unused,
+ struct machine *machine __maybe_unused)
{
- if (scripting_ops && scripting_ops->process_throttle)
- scripting_ops->process_throttle(event, sample, machine);
+
return 0;
}
@@ -3300,10 +3391,9 @@ static int __cmd_script(struct perf_script *script)
script->tool.mmap = process_mmap_event;
script->tool.mmap2 = process_mmap2_event;
}
- if (script->show_switch_events || (scripting_ops && scripting_ops->process_switch))
+ if (script->show_switch_events)
script->tool.context_switch = process_switch_event;
- if (scripting_ops && scripting_ops->process_auxtrace_error)
- script->tool.auxtrace_error = process_auxtrace_error;
+ script->tool.auxtrace_error = perf_event__process_auxtrace_error;
if (script->show_namespace_events)
script->tool.namespaces = process_namespaces_event;
if (script->show_cgroup_events)
@@ -3340,96 +3430,49 @@ static int __cmd_script(struct perf_script *script)
return ret;
}
-static int list_available_languages_cb(struct scripting_ops *ops, const char *spec)
-{
- fprintf(stderr, " %-42s [%s]\n", spec, ops->name);
- return 0;
-}
-static void list_available_languages(void)
-{
- fprintf(stderr, "\n");
- fprintf(stderr, "Scripting language extensions (used in "
- "perf script -s [spec:]script.[spec]):\n\n");
- script_spec__for_each(&list_available_languages_cb);
- fprintf(stderr, "\n");
-}
/* Find script file relative to current directory or exec path */
static char *find_script(const char *script)
{
char path[PATH_MAX];
+ char *exec_path;
- if (!scripting_ops) {
- const char *ext = strrchr(script, '.');
+ if (strchr(script, '/') && access(script, R_OK) == 0)
+ return strdup(script);
- if (!ext)
- return NULL;
+ exec_path = get_argv_exec_path();
+ if (exec_path) {
+ snprintf(path, sizeof(path), "%s/python/%s", exec_path, script);
+ if (access(path, R_OK) == 0) {
+ free(exec_path);
+ return strdup(path);
+ }
- scripting_ops = script_spec__lookup(++ext);
- if (!scripting_ops)
- return NULL;
+ snprintf(path, sizeof(path), "%s/python/%s.py", exec_path, script);
+ free(exec_path);
+ if (access(path, R_OK) == 0)
+ return strdup(path);
}
- if (access(script, R_OK)) {
- char *exec_path = get_argv_exec_path();
-
- if (!exec_path)
- return NULL;
- snprintf(path, sizeof(path), "%s/scripts/%s/%s",
- exec_path, scripting_ops->dirname, script);
- free(exec_path);
- script = path;
- if (access(script, R_OK))
- return NULL;
+ if (access(script, R_OK) == 0) {
+ if (!strchr(script, '/')) {
+ snprintf(path, sizeof(path), "./%s", script);
+ return strdup(path);
+ }
+ return strdup(script);
}
- return strdup(script);
+
+ /* Failure to find script. */
+ return NULL;
}
static int parse_scriptname(const struct option *opt __maybe_unused,
const char *str, int unset __maybe_unused)
{
- char spec[PATH_MAX];
- const char *script, *ext;
- int len;
-
- if (strcmp(str, "lang") == 0) {
- list_available_languages();
- exit(0);
- }
-
- script = strchr(str, ':');
- if (script) {
- len = script - str;
- if (len >= PATH_MAX) {
- fprintf(stderr, "invalid language specifier");
- return -1;
- }
- strncpy(spec, str, len);
- spec[len] = '\0';
- scripting_ops = script_spec__lookup(spec);
- if (!scripting_ops) {
- fprintf(stderr, "invalid language specifier");
- return -1;
- }
- script++;
- } else {
- script = str;
- ext = strrchr(script, '.');
- if (!ext) {
- fprintf(stderr, "invalid script extension");
- return -1;
- }
- scripting_ops = script_spec__lookup(++ext);
- if (!scripting_ops) {
- fprintf(stderr, "invalid script extension");
- return -1;
- }
- }
-
- script_name = find_script(script);
+ script_name = find_script(str);
if (!script_name)
- script_name = strdup(script);
+ script_name = strdup(str);
return 0;
}
@@ -3625,7 +3668,6 @@ struct script_desc {
struct list_head node;
char *name;
char *half_liner;
- char *args;
};
static LIST_HEAD(script_descs);
@@ -3640,16 +3682,18 @@ static struct script_desc *script_desc__new(const char *name)
return s;
}
-static void script_desc__delete(struct script_desc *s)
-{
- zfree(&s->name);
- zfree(&s->half_liner);
- zfree(&s->args);
- free(s);
-}
+
static void script_desc__add(struct script_desc *s)
{
+ struct script_desc *pos;
+
+ list_for_each_entry(pos, &script_descs, node) {
+ if (s->name && pos->name && strcasecmp(s->name, pos->name) < 0) {
+ list_add_tail(&s->node, &pos->node);
+ return;
+ }
+ }
list_add_tail(&s->node, &script_descs);
}
@@ -3697,15 +3741,66 @@ static int read_script_info(struct script_desc *desc, const char *filename)
{
char line[BUFSIZ], *p;
FILE *fp;
+ bool in_docstring = false;
+ bool found_description = false;
fp = fopen(filename, "r");
if (!fp)
return -1;
while (fgets(line, sizeof(line), fp)) {
+ static const char * const triple_quote_str[] = {
+ "\"\"\"",
+ "'''",
+ "r\"\"\"",
+ };
p = skip_spaces(line);
if (strlen(p) == 0)
continue;
+
+ if (in_docstring) {
+ if (strlen(p) && p[strlen(p) - 1] == '\n')
+ p[strlen(p) - 1] = '\0';
+ zfree(&desc->half_liner);
+ desc->half_liner = strdup(skip_spaces(p));
+ in_docstring = false;
+ found_description = true;
+ break;
+ }
+
+
+ for (size_t i = 0; i < ARRAY_SIZE(triple_quote_str); i++) {
+ const char *quote = triple_quote_str[i];
+ const char *close_quote = quote;
+
+ if (quote[0] == 'r' || quote[0] == 'R')
+ close_quote++;
+
+ if (!strstarts(p, quote))
+ continue;
+
+ p += strlen(quote);
+ p = skip_spaces(p);
+ if (strlen(p) > 0) {
+ if (p[strlen(p) - 1] == '\n')
+ p[strlen(p) - 1] = '\0';
+ p = skip_spaces(p);
+ if (str_ends_with(p, close_quote))
+ p[strlen(p) - strlen(close_quote)] = '\0';
+ zfree(&desc->half_liner);
+ desc->half_liner = strdup(skip_spaces(p));
+ found_description = true;
+ break;
+ }
+ in_docstring = true;
+ break;
+ }
+ if (found_description)
+ break;
+
+ if (in_docstring)
+ continue;
+
if (*p != '#')
continue;
p++;
@@ -3718,14 +3813,17 @@ static int read_script_info(struct script_desc *desc, const char *filename)
if (!strncmp(p, "description:", strlen("description:"))) {
p += strlen("description:");
+ zfree(&desc->half_liner);
desc->half_liner = strdup(skip_spaces(p));
- continue;
+ found_description = true;
+ break;
}
- if (!strncmp(p, "args:", strlen("args:"))) {
- p += strlen("args:");
- desc->args = strdup(skip_spaces(p));
- continue;
+ if (!found_description && strlen(p) > 0 &&
+ strncmp(p, "SPDX-License-Identifier", 23)) {
+ if (!desc->half_liner)
+ desc->half_liner = strdup(p);
+ // Don't set found_description, maybe we find a better "description:" later!
}
}
@@ -3756,23 +3854,21 @@ static int list_available_scripts(const struct option *opt __maybe_unused,
const char *s __maybe_unused,
int unset __maybe_unused)
{
- struct dirent *script_dirent, *lang_dirent;
- char *buf, *scripts_path, *script_path, *lang_path, *first_half;
- DIR *scripts_dir, *lang_dir;
+ struct dirent *script_dirent;
+ char *buf, *scripts_path, *script_path;
+ DIR *scripts_dir;
struct script_desc *desc;
char *script_root;
- buf = malloc(3 * MAXPATHLEN + BUFSIZ);
+ buf = malloc(2 * MAXPATHLEN + BUFSIZ);
if (!buf) {
pr_err("malloc failed\n");
exit(-1);
}
scripts_path = buf;
script_path = buf + MAXPATHLEN;
- lang_path = buf + 2 * MAXPATHLEN;
- first_half = buf + 3 * MAXPATHLEN;
- snprintf(scripts_path, MAXPATHLEN, "%s/scripts", get_argv_exec_path());
+ snprintf(scripts_path, MAXPATHLEN, "%s/python", get_argv_exec_path());
scripts_dir = opendir(scripts_path);
if (!scripts_dir) {
@@ -3784,30 +3880,26 @@ static int list_available_scripts(const struct option *opt __maybe_unused,
exit(-1);
}
- for_each_lang(scripts_path, scripts_dir, lang_dirent) {
- scnprintf(lang_path, MAXPATHLEN, "%s/%s/bin", scripts_path,
- lang_dirent->d_name);
- lang_dir = opendir(lang_path);
- if (!lang_dir)
- continue;
+ while ((script_dirent = readdir(scripts_dir)) != NULL) {
+ if (script_dirent->d_type != DT_DIR &&
+ (script_dirent->d_type != DT_UNKNOWN ||
+ !is_directory(scripts_path, script_dirent))) {
- for_each_script(lang_path, lang_dir, script_dirent) {
- script_root = get_script_root(script_dirent, REPORT_SUFFIX);
+ script_root = get_script_root(script_dirent, ".py");
if (script_root) {
desc = script_desc__findnew(script_root);
scnprintf(script_path, MAXPATHLEN, "%s/%s",
- lang_path, script_dirent->d_name);
+ scripts_path, script_dirent->d_name);
read_script_info(desc, script_path);
free(script_root);
}
}
}
+ closedir(scripts_dir);
- fprintf(stdout, "List of available trace scripts:\n");
+ fprintf(stdout, "List of available scripts:\n");
list_for_each_entry(desc, &script_descs, node) {
- sprintf(first_half, "%s %s", desc->name,
- desc->args ? desc->args : "");
- fprintf(stdout, " %-36s %s\n", first_half,
+ fprintf(stdout, " %-36s %s\n", desc->name,
desc->half_liner ? desc->half_liner : "");
}
@@ -3843,93 +3935,7 @@ static void free_dlarg(void)
free(dlargv);
}
-static char *get_script_path(const char *script_root, const char *suffix)
-{
- struct dirent *script_dirent, *lang_dirent;
- char scripts_path[MAXPATHLEN];
- char script_path[MAXPATHLEN];
- DIR *scripts_dir, *lang_dir;
- char lang_path[MAXPATHLEN];
- char *__script_root;
-
- snprintf(scripts_path, MAXPATHLEN, "%s/scripts", get_argv_exec_path());
-
- scripts_dir = opendir(scripts_path);
- if (!scripts_dir)
- return NULL;
-
- for_each_lang(scripts_path, scripts_dir, lang_dirent) {
- scnprintf(lang_path, MAXPATHLEN, "%s/%s/bin", scripts_path,
- lang_dirent->d_name);
- lang_dir = opendir(lang_path);
- if (!lang_dir)
- continue;
-
- for_each_script(lang_path, lang_dir, script_dirent) {
- __script_root = get_script_root(script_dirent, suffix);
- if (__script_root && !strcmp(script_root, __script_root)) {
- free(__script_root);
- closedir(scripts_dir);
- scnprintf(script_path, MAXPATHLEN, "%s/%s",
- lang_path, script_dirent->d_name);
- closedir(lang_dir);
- return strdup(script_path);
- }
- free(__script_root);
- }
- closedir(lang_dir);
- }
- closedir(scripts_dir);
-
- return NULL;
-}
-
-static bool is_top_script(const char *script_path)
-{
- return ends_with(script_path, "top") != NULL;
-}
-static int has_required_arg(char *script_path)
-{
- struct script_desc *desc;
- int n_args = 0;
- char *p;
-
- desc = script_desc__new(NULL);
-
- if (read_script_info(desc, script_path))
- goto out;
-
- if (!desc->args)
- goto out;
-
- for (p = desc->args; *p; p++)
- if (*p == '<')
- n_args++;
-out:
- script_desc__delete(desc);
-
- return n_args;
-}
-
-static int have_cmd(int argc, const char **argv)
-{
- char **__argv = calloc(argc, sizeof(const char *));
-
- if (!__argv) {
- pr_err("malloc failed\n");
- return -1;
- }
-
- memcpy(__argv, argv, sizeof(const char *) * argc);
- argc = parse_options(argc, (const char **)__argv, record_options,
- NULL, PARSE_OPT_STOP_AT_NON_OPTION);
- free(__argv);
-
- system_wide = (argc == 0);
-
- return 0;
-}
static void script__setup_sample_type(struct perf_script *script)
{
@@ -4115,17 +4121,13 @@ int cmd_script(int argc, const char **argv)
bool show_full_info = false;
bool header = false;
bool header_only = false;
- bool script_started = false;
bool unsorted_dump = false;
bool merge_deferred_callchains = true;
- char *rec_script_path = NULL;
- char *rep_script_path = NULL;
struct perf_session *session;
struct itrace_synth_opts itrace_synth_opts = {
.set = false,
.default_no_sample = true,
};
- char *script_path = NULL;
const char *dlfilter_file = NULL;
const char **__argv;
int i, j, err = 0;
@@ -4146,11 +4148,10 @@ int cmd_script(int argc, const char **argv)
list_available_scripts),
OPT_CALLBACK_NOOPT(0, "list-dlfilters", NULL, NULL, "list available dlfilters",
list_available_dlfilters),
- OPT_CALLBACK('s', "script", NULL, "name",
- "script file name (lang:script name, script name, or *)",
- parse_scriptname),
- OPT_STRING('g', "gen-script", &generate_script_lang, "lang",
- "generate perf-script.xx script in specified language"),
+ { .type = OPTION_CALLBACK, .short_name = 's', .long_name = "script",
+ .value = NULL, .argh = "name",
+ .help = "script file name (lang:script name, script name, or *)",
+ .callback = parse_scriptname, .flags = PARSE_OPT_HIDDEN },
OPT_STRING(0, "dlfilter", &dlfilter_file, "file", "filter .so file name"),
OPT_CALLBACK(0, "dlarg", NULL, "argument", "filter argument",
add_dlarg),
@@ -4179,8 +4180,6 @@ int cmd_script(int argc, const char **argv)
"code_page_size,ins_lat,machine_pid,vcpu,cgroup,retire_lat,"
"brcntr",
parse_output_fields),
- OPT_BOOLEAN('a', "all-cpus", &system_wide,
- "system-wide collection from all CPUs"),
OPT_STRING(0, "dsos", &symbol_conf.dso_list_str, "dso[,dso...]",
"only consider symbols in these DSOs"),
OPT_STRING('S', "symbols", &symbol_conf.sym_list_str, "symbol[,symbol...]",
@@ -4277,22 +4276,17 @@ int cmd_script(int argc, const char **argv)
OPTS_EVSWITCH(&script.evswitch),
OPT_END()
};
- const char * const script_subcommands[] = { "record", "report", NULL };
const char *script_usage[] = {
"perf script [<options>]",
- "perf script [<options>] record <script> [<record-options>] <command>",
- "perf script [<options>] report <script> [script-args]",
- "perf script [<options>] <script> [<record-options>] <command>",
- "perf script [<options>] <top-script> [script-args]",
+ "perf script [<options>] <script> [script-args]",
NULL
};
struct perf_env *env;
perf_set_singlethreaded();
- setup_scripting();
- argc = parse_options_subcommand(argc, argv, options, script_subcommands, script_usage,
+ argc = parse_options_subcommand(argc, argv, options, NULL, script_usage,
PARSE_OPT_STOP_AT_NON_OPTION);
if (symbol_conf.guestmount ||
@@ -4315,21 +4309,7 @@ int cmd_script(int argc, const char **argv)
if (symbol__validate_sym_arguments())
return -1;
- if (argc > 1 && strlen(argv[0]) > 2 && strstarts("record", argv[0])) {
- rec_script_path = get_script_path(argv[1], RECORD_SUFFIX);
- if (!rec_script_path)
- return cmd_record(argc, argv);
- }
- if (argc > 1 && strlen(argv[0]) > 2 && strstarts("report", argv[0])) {
- rep_script_path = get_script_path(argv[1], REPORT_SUFFIX);
- if (!rep_script_path) {
- fprintf(stderr,
- "Please specify a valid report script"
- "(see 'perf script -l' for listing)\n");
- return -1;
- }
- }
if (reltime && deltatime) {
fprintf(stderr,
@@ -4345,149 +4325,107 @@ int cmd_script(int argc, const char **argv)
/* make sure PERF_EXEC_PATH is set for scripts */
set_argv_exec_path(get_argv_exec_path());
- if (argc && !script_name && !rec_script_path && !rep_script_path) {
- int live_pipe[2];
- int rep_args;
- pid_t pid;
-
- rec_script_path = get_script_path(argv[0], RECORD_SUFFIX);
- rep_script_path = get_script_path(argv[0], REPORT_SUFFIX);
-
- if (!rec_script_path && !rep_script_path) {
- script_name = find_script(argv[0]);
- if (script_name) {
- argc -= 1;
- argv += 1;
- goto script_found;
- }
- usage_with_options_msg(script_usage, options,
- "Couldn't find script `%s'\n\n See perf"
- " script -l for available scripts.\n", argv[0]);
+ if (argc && !script_name) {
+ if (!strcmp(argv[0], "record") || !strcmp(argv[0], "report")) {
+ fprintf(stderr,
+ "The '%s' subcommand is no longer supported.\n",
+ argv[0]);
+ fprintf(stderr,
+ "Run 'perf script <script> [args]' directly.\n");
+ return -1;
}
-
- if (is_top_script(argv[0])) {
- rep_args = argc - 1;
- } else {
- int rec_args;
-
- rep_args = has_required_arg(rep_script_path);
- rec_args = (argc - 1) - rep_args;
- if (rec_args < 0) {
- usage_with_options_msg(script_usage, options,
- "`%s' script requires options."
- "\n\n See perf script -l for available "
- "scripts and options.\n", argv[0]);
- }
+ script_name = find_script(argv[0]);
+ if (script_name) {
+ argc -= 1;
+ argv += 1;
+ goto script_found;
}
+ usage_with_options_msg(script_usage, options,
+ "Couldn't find script `%s'\n\n"
+ " See perf script -l for available scripts.\n", argv[0]);
+ }
+script_found:
- if (pipe(live_pipe) < 0) {
- perror("failed to create pipe");
- return -1;
- }
+ if (script_name) {
+ pid_t pid;
+ bool is_python = script_name && str_ends_with(script_name, ".py");
+
+ if (cpu_list || symbol_conf.pid_list_str || symbol_conf.tid_list_str ||
+ script.time_str || dlfilter_file || dlargc > 0) {
+ pr_warning("Warning: Filtering options (-c, -p, -t, --time, --dlfilter, --dlarg) are ignored for standalone scripts.\n");
+ }
pid = fork();
if (pid < 0) {
- perror("failed to fork");
- return -1;
+ err = -errno;
+ pr_err("failed to fork\n");
+ goto out;
}
-
- if (!pid) {
+ if (pid == 0) { /* child */
+ __argv = calloc(argc + 6, sizeof(const char *));
j = 0;
+ if (!__argv)
+ _exit(-ENOMEM);
- dup2(live_pipe[1], 1);
- close(live_pipe[0]);
- if (is_top_script(argv[0])) {
- system_wide = true;
- } else if (!system_wide) {
- if (have_cmd(argc - rep_args, &argv[rep_args]) != 0) {
- err = -1;
- goto out;
- }
- }
- __argv = calloc(argc + 6, sizeof(const char *));
- if (!__argv) {
- pr_err("malloc failed\n");
- err = -ENOMEM;
- goto out;
+ if (is_python) {
+ __argv[j++] = "python3";
+ __argv[j++] = "--";
+ __argv[j++] = script_name;
+ } else {
+ __argv[j++] = script_name;
}
+ if (input_name) {
+ __argv[j++] = "-i";
+ __argv[j++] = input_name;
+ } else {
+ struct stat st;
- __argv[j++] = "/bin/sh";
- __argv[j++] = rec_script_path;
- if (system_wide)
- __argv[j++] = "-a";
- __argv[j++] = "-q";
- __argv[j++] = "-o";
- __argv[j++] = "-";
- for (i = rep_args + 1; i < argc; i++)
+ if (fstat(STDIN_FILENO, &st) == 0 && S_ISFIFO(st.st_mode)) {
+ __argv[j++] = "-i";
+ __argv[j++] = "-";
+ }
+ }
+ for (i = 0; i < argc; i++)
__argv[j++] = argv[i];
__argv[j++] = NULL;
- execvp("/bin/sh", (char **)__argv);
- free(__argv);
- exit(-1);
- }
-
- dup2(live_pipe[0], 0);
- close(live_pipe[1]);
+ if (symbol_conf.vmlinux_name)
+ setenv("PERF_SYMBOL_VMLINUX", symbol_conf.vmlinux_name, 1);
+ if (symbol_conf.kallsyms_name)
+ setenv("PERF_SYMBOL_KALLSYMS", symbol_conf.kallsyms_name, 1);
+ if (symbol_conf.symfs && symbol_conf.symfs[0])
+ setenv("PERF_SYMBOL_SYMFS", symbol_conf.symfs, 1);
- __argv = calloc(argc + 4, sizeof(const char *));
- if (!__argv) {
- pr_err("malloc failed\n");
- err = -ENOMEM;
- goto out;
- }
+ if (is_python)
+ execvp("python3", (char **)__argv);
+ else
+ execvp(script_name, (char **)__argv);
+ {
+ int err_code = errno;
- j = 0;
- __argv[j++] = "/bin/sh";
- __argv[j++] = rep_script_path;
- for (i = 1; i < rep_args + 1; i++)
- __argv[j++] = argv[i];
- __argv[j++] = "-i";
- __argv[j++] = "-";
- __argv[j++] = NULL;
-
- execvp("/bin/sh", (char **)__argv);
- free(__argv);
- exit(-1);
- }
-script_found:
- if (rec_script_path)
- script_path = rec_script_path;
- if (rep_script_path)
- script_path = rep_script_path;
-
- if (script_path) {
- j = 0;
-
- if (!rec_script_path)
- system_wide = false;
- else if (!system_wide) {
- if (have_cmd(argc - 1, &argv[1]) != 0) {
- err = -1;
+ pr_err("failed to execute script '%s': %s\n",
+ script_name, strerror(err_code));
+ _exit(err_code);
+ }
+ } else { /* parent */
+ int status = 0;
+
+ while (waitpid(pid, &status, 0) != pid) {
+ if (errno == EINTR)
+ continue;
+ pr_err("failed to wait for script '%s': %s\n",
+ script_name, strerror(errno));
+ err = -errno;
goto out;
}
- }
-
- __argv = calloc(argc + 2, sizeof(const char *));
- if (!__argv) {
- pr_err("malloc failed\n");
- err = -ENOMEM;
+ if (WIFEXITED(status))
+ err = WEXITSTATUS(status);
+ else
+ err = -1;
goto out;
}
-
- __argv[j++] = "/bin/sh";
- __argv[j++] = script_path;
- if (system_wide)
- __argv[j++] = "-a";
- for (i = 2; i < argc; i++)
- __argv[j++] = argv[i];
- __argv[j++] = NULL;
-
- execvp("/bin/sh", (char **)__argv);
- free(__argv);
- exit(-1);
}
if (dlfilter_file) {
@@ -4579,77 +4517,12 @@ int cmd_script(int argc, const char **argv)
goto out_delete;
}
#endif
- if (generate_script_lang) {
- struct stat perf_stat;
- int input;
- char *filename = strdup("perf-script");
-
- if (output_set_by_user()) {
- fprintf(stderr,
- "custom fields not supported for generated scripts");
- err = -EINVAL;
- goto out_delete;
- }
-
- input = open(data.path, O_RDONLY); /* input_name */
- if (input < 0) {
- err = -errno;
- perror("failed to open file");
- goto out_delete;
- }
-
- err = fstat(input, &perf_stat);
- if (err < 0) {
- perror("failed to stat file");
- goto out_delete;
- }
-
- if (!perf_stat.st_size) {
- fprintf(stderr, "zero-sized file, nothing to do!\n");
- goto out_delete;
- }
-
- scripting_ops = script_spec__lookup(generate_script_lang);
- if (!scripting_ops && ends_with(generate_script_lang, ".py")) {
- scripting_ops = script_spec__lookup("python");
- free(filename);
- filename = strdup(generate_script_lang);
- filename[strlen(filename) - 3] = '\0';
- } else if (!scripting_ops && ends_with(generate_script_lang, ".pl")) {
- scripting_ops = script_spec__lookup("perl");
- free(filename);
- filename = strdup(generate_script_lang);
- filename[strlen(filename) - 3] = '\0';
- }
- if (!scripting_ops) {
- fprintf(stderr, "invalid language specifier '%s'\n", generate_script_lang);
- err = -ENOENT;
- goto out_delete;
- }
- if (!filename) {
- err = -ENOMEM;
- goto out_delete;
- }
-#ifdef HAVE_LIBTRACEEVENT
- err = scripting_ops->generate_script(session->tevent.pevent, filename);
-#else
- err = scripting_ops->generate_script(NULL, filename);
-#endif
- free(filename);
- goto out_delete;
- }
err = dlfilter__start(dlfilter, session);
if (err)
goto out_delete;
- if (script_name) {
- err = scripting_ops->start_script(script_name, argc, argv, session);
- if (err)
- goto out_delete;
- pr_debug("perf script started with script %s\n\n", script_name);
- script_started = true;
- }
+
err = perf_session__check_output_opt(session);
@@ -4679,7 +4552,6 @@ int cmd_script(int argc, const char **argv)
err = __cmd_script(&script);
- flush_scripting();
if (verbose > 2 || debug_kmaps)
perf_session__dump_kmaps(session);
@@ -4695,10 +4567,8 @@ int cmd_script(int argc, const char **argv)
perf_session__delete(session);
perf_script__exit(&script);
- if (script_started)
- cleanup_scripting();
+out:
dlfilter__cleanup(dlfilter);
free_dlarg();
-out:
return err;
}
diff --git a/tools/perf/scripts/Build b/tools/perf/scripts/Build
deleted file mode 100644
index fbeab8fff88b..000000000000
--- a/tools/perf/scripts/Build
+++ /dev/null
@@ -1,3 +0,0 @@
-ifeq ($(CONFIG_LIBTRACEEVENT),y)
- perf-util-$(CONFIG_LIBPERL) += perl/Perf-Trace-Util/
-endif
diff --git a/tools/perf/scripts/perl/Perf-Trace-Util/Build b/tools/perf/scripts/perl/Perf-Trace-Util/Build
deleted file mode 100644
index 01a1a0ed51ae..000000000000
--- a/tools/perf/scripts/perl/Perf-Trace-Util/Build
+++ /dev/null
@@ -1,9 +0,0 @@
-perf-util-y += Context.o
-
-CFLAGS_Context.o += $(PERL_EMBED_CCOPTS) -Wno-redundant-decls -Wno-strict-prototypes -Wno-bad-function-cast -Wno-declaration-after-statement -Wno-switch-enum
-CFLAGS_Context.o += -Wno-unused-parameter -Wno-nested-externs -Wno-undef
-CFLAGS_Context.o += -Wno-switch-default -Wno-shadow -Wno-thread-safety-analysis
-
-ifeq ($(CC_NO_CLANG), 1)
- CFLAGS_Context.o += -Wno-unused-command-line-argument
-endif
diff --git a/tools/perf/scripts/perl/Perf-Trace-Util/Context.c b/tools/perf/scripts/perl/Perf-Trace-Util/Context.c
deleted file mode 100644
index 25c47d23a130..000000000000
--- a/tools/perf/scripts/perl/Perf-Trace-Util/Context.c
+++ /dev/null
@@ -1,122 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * This file was generated automatically by ExtUtils::ParseXS version 2.18_02 from the
- * contents of Context.xs. Do not edit this file, edit Context.xs instead.
- *
- * ANY CHANGES MADE HERE WILL BE LOST!
- */
-#include <stdbool.h>
-#ifndef HAS_BOOL
-# define HAS_BOOL 1
-#endif
-#line 1 "Context.xs"
-/*
- * Context.xs. XS interfaces for perf script.
- *
- * Copyright (C) 2009 Tom Zanussi <tzanussi@gmail.com>
- */
-
-#include "EXTERN.h"
-#include "perl.h"
-#include "XSUB.h"
-#include "../../../util/trace-event.h"
-
-#ifndef PERL_UNUSED_VAR
-# define PERL_UNUSED_VAR(var) if (0) var = var
-#endif
-
-#line 42 "Context.c"
-
-XS(XS_Perf__Trace__Context_common_pc); /* prototype to pass -Wmissing-prototypes */
-XS(XS_Perf__Trace__Context_common_pc)
-{
-#ifdef dVAR
- dVAR; dXSARGS;
-#else
- dXSARGS;
-#endif
- if (items != 1)
- Perl_croak(aTHX_ "Usage: %s(%s)", "Perf::Trace::Context::common_pc", "context");
- PERL_UNUSED_VAR(cv); /* -W */
- {
- struct scripting_context * context = INT2PTR(struct scripting_context *,SvIV(ST(0)));
- int RETVAL;
- dXSTARG;
-
- RETVAL = common_pc(context);
- XSprePUSH; PUSHi((IV)RETVAL);
- }
- XSRETURN(1);
-}
-
-
-XS(XS_Perf__Trace__Context_common_flags); /* prototype to pass -Wmissing-prototypes */
-XS(XS_Perf__Trace__Context_common_flags)
-{
-#ifdef dVAR
- dVAR; dXSARGS;
-#else
- dXSARGS;
-#endif
- if (items != 1)
- Perl_croak(aTHX_ "Usage: %s(%s)", "Perf::Trace::Context::common_flags", "context");
- PERL_UNUSED_VAR(cv); /* -W */
- {
- struct scripting_context * context = INT2PTR(struct scripting_context *,SvIV(ST(0)));
- int RETVAL;
- dXSTARG;
-
- RETVAL = common_flags(context);
- XSprePUSH; PUSHi((IV)RETVAL);
- }
- XSRETURN(1);
-}
-
-
-XS(XS_Perf__Trace__Context_common_lock_depth); /* prototype to pass -Wmissing-prototypes */
-XS(XS_Perf__Trace__Context_common_lock_depth)
-{
-#ifdef dVAR
- dVAR; dXSARGS;
-#else
- dXSARGS;
-#endif
- if (items != 1)
- Perl_croak(aTHX_ "Usage: %s(%s)", "Perf::Trace::Context::common_lock_depth", "context");
- PERL_UNUSED_VAR(cv); /* -W */
- {
- struct scripting_context * context = INT2PTR(struct scripting_context *,SvIV(ST(0)));
- int RETVAL;
- dXSTARG;
-
- RETVAL = common_lock_depth(context);
- XSprePUSH; PUSHi((IV)RETVAL);
- }
- XSRETURN(1);
-}
-
-#ifdef __cplusplus
-extern "C"
-#endif
-XS(boot_Perf__Trace__Context); /* prototype to pass -Wmissing-prototypes */
-XS(boot_Perf__Trace__Context)
-{
-#ifdef dVAR
- dVAR; dXSARGS;
-#else
- dXSARGS;
-#endif
- const char* file = __FILE__;
-
- PERL_UNUSED_VAR(cv); /* -W */
- PERL_UNUSED_VAR(items); /* -W */
- XS_VERSION_BOOTCHECK ;
-
- newXSproto("Perf::Trace::Context::common_pc", XS_Perf__Trace__Context_common_pc, file, "$");
- newXSproto("Perf::Trace::Context::common_flags", XS_Perf__Trace__Context_common_flags, file, "$");
- newXSproto("Perf::Trace::Context::common_lock_depth", XS_Perf__Trace__Context_common_lock_depth, file, "$");
- if (PL_unitcheckav)
- call_list(PL_scopestack_ix, PL_unitcheckav);
- XSRETURN_YES;
-}
-
diff --git a/tools/perf/scripts/perl/Perf-Trace-Util/Context.xs b/tools/perf/scripts/perl/Perf-Trace-Util/Context.xs
deleted file mode 100644
index 8c7ea42444d1..000000000000
--- a/tools/perf/scripts/perl/Perf-Trace-Util/Context.xs
+++ /dev/null
@@ -1,42 +0,0 @@
-/*
- * Context.xs. XS interfaces for perf script.
- *
- * Copyright (C) 2009 Tom Zanussi <tzanussi@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#include "EXTERN.h"
-#include "perl.h"
-#include "XSUB.h"
-#include "../../../perf.h"
-#include "../../../util/trace-event.h"
-
-MODULE = Perf::Trace::Context PACKAGE = Perf::Trace::Context
-PROTOTYPES: ENABLE
-
-int
-common_pc(context)
- struct scripting_context * context
-
-int
-common_flags(context)
- struct scripting_context * context
-
-int
-common_lock_depth(context)
- struct scripting_context * context
-
diff --git a/tools/perf/scripts/perl/Perf-Trace-Util/Makefile.PL b/tools/perf/scripts/perl/Perf-Trace-Util/Makefile.PL
deleted file mode 100644
index e8994332d7dc..000000000000
--- a/tools/perf/scripts/perl/Perf-Trace-Util/Makefile.PL
+++ /dev/null
@@ -1,18 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-use 5.010000;
-use ExtUtils::MakeMaker;
-# See lib/ExtUtils/MakeMaker.pm for details of how to influence
-# the contents of the Makefile that is written.
-WriteMakefile(
- NAME => 'Perf::Trace::Context',
- VERSION_FROM => 'lib/Perf/Trace/Context.pm', # finds $VERSION
- PREREQ_PM => {}, # e.g., Module::Name => 1.1
- ($] >= 5.005 ? ## Add these new keywords supported since 5.005
- (ABSTRACT_FROM => 'lib/Perf/Trace/Context.pm', # retrieve abstract from module
- AUTHOR => 'Tom Zanussi <tzanussi@gmail.com>') : ()),
- LIBS => [''], # e.g., '-lm'
- DEFINE => '-I ../..', # e.g., '-DHAVE_SOMETHING'
- INC => '-I.', # e.g., '-I. -I/usr/include/other'
- # Un-comment this if you add C files to link with later:
- OBJECT => 'Context.o', # link all the C files too
-);
diff --git a/tools/perf/scripts/perl/Perf-Trace-Util/README b/tools/perf/scripts/perl/Perf-Trace-Util/README
deleted file mode 100644
index 2f0c7f3043ee..000000000000
--- a/tools/perf/scripts/perl/Perf-Trace-Util/README
+++ /dev/null
@@ -1,59 +0,0 @@
-Perf-Trace-Util version 0.01
-============================
-
-This module contains utility functions for use with perf script.
-
-Core.pm and Util.pm are pure Perl modules; Core.pm contains routines
-that the core perf support for Perl calls on and should always be
-'used', while Util.pm contains useful but optional utility functions
-that scripts may want to use. Context.pm contains the Perl->C
-interface that allows scripts to access data in the embedding perf
-executable; scripts wishing to do that should 'use Context.pm'.
-
-The Perl->C perf interface is completely driven by Context.xs. If you
-want to add new Perl functions that end up accessing C data in the
-perf executable, you add desciptions of the new functions here.
-scripting_context is a pointer to the perf data in the perf executable
-that you want to access - it's passed as the second parameter,
-$context, to all handler functions.
-
-After you do that:
-
- perl Makefile.PL # to create a Makefile for the next step
- make # to create Context.c
-
- edit Context.c to add const to the char* file = __FILE__ line in
- XS(boot_Perf__Trace__Context) to silence a warning/error.
-
- You can delete the Makefile, object files and anything else that was
- generated e.g. blib and shared library, etc, except for of course
- Context.c
-
- You should then be able to run the normal perf make as usual.
-
-INSTALLATION
-
-Building perf with perf script Perl scripting should install this
-module in the right place.
-
-You should make sure libperl and ExtUtils/Embed.pm are installed first
-e.g. apt-get install libperl-dev or yum install perl-ExtUtils-Embed.
-
-DEPENDENCIES
-
-This module requires these other modules and libraries:
-
- None
-
-COPYRIGHT AND LICENCE
-
-Copyright (C) 2009 by Tom Zanussi <tzanussi@gmail.com>
-
-This library is free software; you can redistribute it and/or modify
-it under the same terms as Perl itself, either Perl version 5.10.0 or,
-at your option, any later version of Perl 5 you may have available.
-
-Alternatively, this software may be distributed under the terms of the
-GNU General Public License ("GPL") version 2 as published by the Free
-Software Foundation.
-
diff --git a/tools/perf/scripts/perl/Perf-Trace-Util/lib/Perf/Trace/Context.pm b/tools/perf/scripts/perl/Perf-Trace-Util/lib/Perf/Trace/Context.pm
deleted file mode 100644
index 4e2f6039ac92..000000000000
--- a/tools/perf/scripts/perl/Perf-Trace-Util/lib/Perf/Trace/Context.pm
+++ /dev/null
@@ -1,55 +0,0 @@
-package Perf::Trace::Context;
-
-use 5.010000;
-use strict;
-use warnings;
-
-require Exporter;
-
-our @ISA = qw(Exporter);
-
-our %EXPORT_TAGS = ( 'all' => [ qw(
-) ] );
-
-our @EXPORT_OK = ( @{ $EXPORT_TAGS{'all'} } );
-
-our @EXPORT = qw(
- common_pc common_flags common_lock_depth
-);
-
-our $VERSION = '0.01';
-
-require XSLoader;
-XSLoader::load('Perf::Trace::Context', $VERSION);
-
-1;
-__END__
-=head1 NAME
-
-Perf::Trace::Context - Perl extension for accessing functions in perf.
-
-=head1 SYNOPSIS
-
- use Perf::Trace::Context;
-
-=head1 SEE ALSO
-
-Perf (script) documentation
-
-=head1 AUTHOR
-
-Tom Zanussi, E<lt>tzanussi@gmail.com<gt>
-
-=head1 COPYRIGHT AND LICENSE
-
-Copyright (C) 2009 by Tom Zanussi
-
-This library is free software; you can redistribute it and/or modify
-it under the same terms as Perl itself, either Perl version 5.10.0 or,
-at your option, any later version of Perl 5 you may have available.
-
-Alternatively, this software may be distributed under the terms of the
-GNU General Public License ("GPL") version 2 as published by the Free
-Software Foundation.
-
-=cut
diff --git a/tools/perf/scripts/perl/Perf-Trace-Util/lib/Perf/Trace/Core.pm b/tools/perf/scripts/perl/Perf-Trace-Util/lib/Perf/Trace/Core.pm
deleted file mode 100644
index 9158458d3eeb..000000000000
--- a/tools/perf/scripts/perl/Perf-Trace-Util/lib/Perf/Trace/Core.pm
+++ /dev/null
@@ -1,192 +0,0 @@
-package Perf::Trace::Core;
-
-use 5.010000;
-use strict;
-use warnings;
-
-require Exporter;
-
-our @ISA = qw(Exporter);
-
-our %EXPORT_TAGS = ( 'all' => [ qw(
-) ] );
-
-our @EXPORT_OK = ( @{ $EXPORT_TAGS{'all'} } );
-
-our @EXPORT = qw(
-define_flag_field define_flag_value flag_str dump_flag_fields
-define_symbolic_field define_symbolic_value symbol_str dump_symbolic_fields
-trace_flag_str
-);
-
-our $VERSION = '0.01';
-
-my %trace_flags = (0x00 => "NONE",
- 0x01 => "IRQS_OFF",
- 0x02 => "IRQS_NOSUPPORT",
- 0x04 => "NEED_RESCHED",
- 0x08 => "HARDIRQ",
- 0x10 => "SOFTIRQ");
-
-sub trace_flag_str
-{
- my ($value) = @_;
-
- my $string;
-
- my $print_delim = 0;
-
- foreach my $idx (sort {$a <=> $b} keys %trace_flags) {
- if (!$value && !$idx) {
- $string .= "NONE";
- last;
- }
-
- if ($idx && ($value & $idx) == $idx) {
- if ($print_delim) {
- $string .= " | ";
- }
- $string .= "$trace_flags{$idx}";
- $print_delim = 1;
- $value &= ~$idx;
- }
- }
-
- return $string;
-}
-
-my %flag_fields;
-my %symbolic_fields;
-
-sub flag_str
-{
- my ($event_name, $field_name, $value) = @_;
-
- my $string;
-
- if ($flag_fields{$event_name}{$field_name}) {
- my $print_delim = 0;
- foreach my $idx (sort {$a <=> $b} keys %{$flag_fields{$event_name}{$field_name}{"values"}}) {
- if (!$value && !$idx) {
- $string .= "$flag_fields{$event_name}{$field_name}{'values'}{$idx}";
- last;
- }
- if ($idx && ($value & $idx) == $idx) {
- if ($print_delim && $flag_fields{$event_name}{$field_name}{'delim'}) {
- $string .= " $flag_fields{$event_name}{$field_name}{'delim'} ";
- }
- $string .= "$flag_fields{$event_name}{$field_name}{'values'}{$idx}";
- $print_delim = 1;
- $value &= ~$idx;
- }
- }
- }
-
- return $string;
-}
-
-sub define_flag_field
-{
- my ($event_name, $field_name, $delim) = @_;
-
- $flag_fields{$event_name}{$field_name}{"delim"} = $delim;
-}
-
-sub define_flag_value
-{
- my ($event_name, $field_name, $value, $field_str) = @_;
-
- $flag_fields{$event_name}{$field_name}{"values"}{$value} = $field_str;
-}
-
-sub dump_flag_fields
-{
- for my $event (keys %flag_fields) {
- print "event $event:\n";
- for my $field (keys %{$flag_fields{$event}}) {
- print " field: $field:\n";
- print " delim: $flag_fields{$event}{$field}{'delim'}\n";
- foreach my $idx (sort {$a <=> $b} keys %{$flag_fields{$event}{$field}{"values"}}) {
- print " value $idx: $flag_fields{$event}{$field}{'values'}{$idx}\n";
- }
- }
- }
-}
-
-sub symbol_str
-{
- my ($event_name, $field_name, $value) = @_;
-
- if ($symbolic_fields{$event_name}{$field_name}) {
- foreach my $idx (sort {$a <=> $b} keys %{$symbolic_fields{$event_name}{$field_name}{"values"}}) {
- if (!$value && !$idx) {
- return "$symbolic_fields{$event_name}{$field_name}{'values'}{$idx}";
- last;
- }
- if ($value == $idx) {
- return "$symbolic_fields{$event_name}{$field_name}{'values'}{$idx}";
- }
- }
- }
-
- return undef;
-}
-
-sub define_symbolic_field
-{
- my ($event_name, $field_name) = @_;
-
- # nothing to do, really
-}
-
-sub define_symbolic_value
-{
- my ($event_name, $field_name, $value, $field_str) = @_;
-
- $symbolic_fields{$event_name}{$field_name}{"values"}{$value} = $field_str;
-}
-
-sub dump_symbolic_fields
-{
- for my $event (keys %symbolic_fields) {
- print "event $event:\n";
- for my $field (keys %{$symbolic_fields{$event}}) {
- print " field: $field:\n";
- foreach my $idx (sort {$a <=> $b} keys %{$symbolic_fields{$event}{$field}{"values"}}) {
- print " value $idx: $symbolic_fields{$event}{$field}{'values'}{$idx}\n";
- }
- }
- }
-}
-
-1;
-__END__
-=head1 NAME
-
-Perf::Trace::Core - Perl extension for perf script
-
-=head1 SYNOPSIS
-
- use Perf::Trace::Core
-
-=head1 SEE ALSO
-
-Perf (script) documentation
-
-=head1 AUTHOR
-
-Tom Zanussi, E<lt>tzanussi@gmail.com<gt>
-
-=head1 COPYRIGHT AND LICENSE
-
-Copyright (C) 2009 by Tom Zanussi
-
-This library is free software; you can redistribute it and/or modify
-it under the same terms as Perl itself, either Perl version 5.10.0 or,
-at your option, any later version of Perl 5 you may have available.
-
-Alternatively, this software may be distributed under the terms of the
-GNU General Public License ("GPL") version 2 as published by the Free
-Software Foundation.
-
-=cut
diff --git a/tools/perf/scripts/perl/Perf-Trace-Util/lib/Perf/Trace/Util.pm b/tools/perf/scripts/perl/Perf-Trace-Util/lib/Perf/Trace/Util.pm
deleted file mode 100644
index 053500114625..000000000000
--- a/tools/perf/scripts/perl/Perf-Trace-Util/lib/Perf/Trace/Util.pm
+++ /dev/null
@@ -1,94 +0,0 @@
-package Perf::Trace::Util;
-
-use 5.010000;
-use strict;
-use warnings;
-
-require Exporter;
-
-our @ISA = qw(Exporter);
-
-our %EXPORT_TAGS = ( 'all' => [ qw(
-) ] );
-
-our @EXPORT_OK = ( @{ $EXPORT_TAGS{'all'} } );
-
-our @EXPORT = qw(
-avg nsecs nsecs_secs nsecs_nsecs nsecs_usecs print_nsecs
-clear_term
-);
-
-our $VERSION = '0.01';
-
-sub avg
-{
- my ($total, $n) = @_;
-
- return $total / $n;
-}
-
-my $NSECS_PER_SEC = 1000000000;
-
-sub nsecs
-{
- my ($secs, $nsecs) = @_;
-
- return $secs * $NSECS_PER_SEC + $nsecs;
-}
-
-sub nsecs_secs {
- my ($nsecs) = @_;
-
- return $nsecs / $NSECS_PER_SEC;
-}
-
-sub nsecs_nsecs {
- my ($nsecs) = @_;
-
- return $nsecs % $NSECS_PER_SEC;
-}
-
-sub nsecs_str {
- my ($nsecs) = @_;
-
- my $str = sprintf("%5u.%09u", nsecs_secs($nsecs), nsecs_nsecs($nsecs));
-
- return $str;
-}
-
-sub clear_term
-{
- print "\x1b[H\x1b[2J";
-}
-
-1;
-__END__
-=head1 NAME
-
-Perf::Trace::Util - Perl extension for perf script
-
-=head1 SYNOPSIS
-
- use Perf::Trace::Util;
-
-=head1 SEE ALSO
-
-Perf (script) documentation
-
-=head1 AUTHOR
-
-Tom Zanussi, E<lt>tzanussi@gmail.com<gt>
-
-=head1 COPYRIGHT AND LICENSE
-
-Copyright (C) 2009 by Tom Zanussi
-
-This library is free software; you can redistribute it and/or modify
-it under the same terms as Perl itself, either Perl version 5.10.0 or,
-at your option, any later version of Perl 5 you may have available.
-
-Alternatively, this software may be distributed under the terms of the
-GNU General Public License ("GPL") version 2 as published by the Free
-Software Foundation.
-
-=cut
diff --git a/tools/perf/scripts/perl/Perf-Trace-Util/typemap b/tools/perf/scripts/perl/Perf-Trace-Util/typemap
deleted file mode 100644
index 840836804aa7..000000000000
--- a/tools/perf/scripts/perl/Perf-Trace-Util/typemap
+++ /dev/null
@@ -1 +0,0 @@
-struct scripting_context * T_PTR
diff --git a/tools/perf/scripts/perl/bin/check-perf-trace-record b/tools/perf/scripts/perl/bin/check-perf-trace-record
deleted file mode 100644
index 423ad6aed056..000000000000
--- a/tools/perf/scripts/perl/bin/check-perf-trace-record
+++ /dev/null
@@ -1,2 +0,0 @@
-#!/bin/bash
-perf record -a -e kmem:kmalloc -e irq:softirq_entry -e kmem:kfree
diff --git a/tools/perf/scripts/perl/bin/failed-syscalls-record b/tools/perf/scripts/perl/bin/failed-syscalls-record
deleted file mode 100644
index 74685f318379..000000000000
--- a/tools/perf/scripts/perl/bin/failed-syscalls-record
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-(perf record -e raw_syscalls:sys_exit $@ || \
- perf record -e syscalls:sys_exit $@) 2> /dev/null
diff --git a/tools/perf/scripts/perl/bin/failed-syscalls-report b/tools/perf/scripts/perl/bin/failed-syscalls-report
deleted file mode 100644
index 9f83cc1ad8ba..000000000000
--- a/tools/perf/scripts/perl/bin/failed-syscalls-report
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/bash
-# description: system-wide failed syscalls
-# args: [comm]
-if [ $# -gt 0 ] ; then
- if ! expr match "$1" "-" > /dev/null ; then
- comm=$1
- shift
- fi
-fi
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/perl/failed-syscalls.pl $comm
diff --git a/tools/perf/scripts/perl/bin/rw-by-file-record b/tools/perf/scripts/perl/bin/rw-by-file-record
deleted file mode 100644
index 33efc8673aae..000000000000
--- a/tools/perf/scripts/perl/bin/rw-by-file-record
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-perf record -e syscalls:sys_enter_read -e syscalls:sys_enter_write $@
-
diff --git a/tools/perf/scripts/perl/bin/rw-by-file-report b/tools/perf/scripts/perl/bin/rw-by-file-report
deleted file mode 100644
index 77200b3f3100..000000000000
--- a/tools/perf/scripts/perl/bin/rw-by-file-report
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/bash
-# description: r/w activity for a program, by file
-# args: <comm>
-if [ $# -lt 1 ] ; then
- echo "usage: rw-by-file <comm>"
- exit
-fi
-comm=$1
-shift
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/perl/rw-by-file.pl $comm
diff --git a/tools/perf/scripts/perl/bin/rw-by-pid-record b/tools/perf/scripts/perl/bin/rw-by-pid-record
deleted file mode 100644
index 7cb9db230448..000000000000
--- a/tools/perf/scripts/perl/bin/rw-by-pid-record
+++ /dev/null
@@ -1,2 +0,0 @@
-#!/bin/bash
-perf record -e syscalls:sys_enter_read -e syscalls:sys_exit_read -e syscalls:sys_enter_write -e syscalls:sys_exit_write $@
diff --git a/tools/perf/scripts/perl/bin/rw-by-pid-report b/tools/perf/scripts/perl/bin/rw-by-pid-report
deleted file mode 100644
index a27b9f311f95..000000000000
--- a/tools/perf/scripts/perl/bin/rw-by-pid-report
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-# description: system-wide r/w activity
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/perl/rw-by-pid.pl
diff --git a/tools/perf/scripts/perl/bin/rwtop-record b/tools/perf/scripts/perl/bin/rwtop-record
deleted file mode 100644
index 7cb9db230448..000000000000
--- a/tools/perf/scripts/perl/bin/rwtop-record
+++ /dev/null
@@ -1,2 +0,0 @@
-#!/bin/bash
-perf record -e syscalls:sys_enter_read -e syscalls:sys_exit_read -e syscalls:sys_enter_write -e syscalls:sys_exit_write $@
diff --git a/tools/perf/scripts/perl/bin/rwtop-report b/tools/perf/scripts/perl/bin/rwtop-report
deleted file mode 100644
index 83e11ec2e190..000000000000
--- a/tools/perf/scripts/perl/bin/rwtop-report
+++ /dev/null
@@ -1,20 +0,0 @@
-#!/bin/bash
-# description: system-wide r/w top
-# args: [interval]
-n_args=0
-for i in "$@"
-do
- if expr match "$i" "-" > /dev/null ; then
- break
- fi
- n_args=$(( $n_args + 1 ))
-done
-if [ "$n_args" -gt 1 ] ; then
- echo "usage: rwtop-report [interval]"
- exit
-fi
-if [ "$n_args" -gt 0 ] ; then
- interval=$1
- shift
-fi
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/perl/rwtop.pl $interval
diff --git a/tools/perf/scripts/perl/bin/wakeup-latency-record b/tools/perf/scripts/perl/bin/wakeup-latency-record
deleted file mode 100644
index 464251a1bd7e..000000000000
--- a/tools/perf/scripts/perl/bin/wakeup-latency-record
+++ /dev/null
@@ -1,6 +0,0 @@
-#!/bin/bash
-perf record -e sched:sched_switch -e sched:sched_wakeup $@
-
-
-
-
diff --git a/tools/perf/scripts/perl/bin/wakeup-latency-report b/tools/perf/scripts/perl/bin/wakeup-latency-report
deleted file mode 100644
index 889e8130cca5..000000000000
--- a/tools/perf/scripts/perl/bin/wakeup-latency-report
+++ /dev/null
@@ -1,3 +0,0 @@
-#!/bin/bash
-# description: system-wide min/max/avg wakeup latency
-perf script $@ -s "$PERF_EXEC_PATH"/scripts/perl/wakeup-latency.pl
diff --git a/tools/perf/scripts/perl/check-perf-trace.pl b/tools/perf/scripts/perl/check-perf-trace.pl
deleted file mode 100644
index d307ce8fd6ed..000000000000
--- a/tools/perf/scripts/perl/check-perf-trace.pl
+++ /dev/null
@@ -1,106 +0,0 @@
-# perf script event handlers, generated by perf script -g perl
-# (c) 2009, Tom Zanussi <tzanussi@gmail.com>
-# Licensed under the terms of the GNU GPL License version 2
-
-# This script tests basic functionality such as flag and symbol
-# strings, common_xxx() calls back into perf, begin, end, unhandled
-# events, etc. Basically, if this script runs successfully and
-# displays expected results, perl scripting support should be ok.
-
-use lib "$ENV{'PERF_EXEC_PATH'}/scripts/perl/Perf-Trace-Util/lib";
-use lib "./Perf-Trace-Util/lib";
-use Perf::Trace::Core;
-use Perf::Trace::Context;
-use Perf::Trace::Util;
-
-sub trace_begin
-{
- print "trace_begin\n";
-}
-
-sub trace_end
-{
- print "trace_end\n";
-
- print_unhandled();
-}
-
-sub irq::softirq_entry
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain,
- $vec) = @_;
-
- print_header($event_name, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm);
-
- print_uncommon($context);
-
- printf("vec=%s\n",
- symbol_str("irq::softirq_entry", "vec", $vec));
-}
-
-sub kmem::kmalloc
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain,
- $call_site, $ptr, $bytes_req, $bytes_alloc,
- $gfp_flags) = @_;
-
- print_header($event_name, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm);
-
- print_uncommon($context);
-
- printf("call_site=%p, ptr=%p, bytes_req=%u, bytes_alloc=%u, ".
- "gfp_flags=%s\n",
- $call_site, $ptr, $bytes_req, $bytes_alloc,
-
- flag_str("kmem::kmalloc", "gfp_flags", $gfp_flags));
-}
-
-# print trace fields not included in handler args
-sub print_uncommon
-{
- my ($context) = @_;
-
- printf("common_preempt_count=%d, common_flags=%s, common_lock_depth=%d, ",
- common_pc($context), trace_flag_str(common_flags($context)),
- common_lock_depth($context));
-
-}
-
-my %unhandled;
-
-sub print_unhandled
-{
- if ((scalar keys %unhandled) == 0) {
- return;
- }
-
- print "\nunhandled events:\n\n";
-
- printf("%-40s %10s\n", "event", "count");
- printf("%-40s %10s\n", "----------------------------------------",
- "-----------");
-
- foreach my $event_name (keys %unhandled) {
- printf("%-40s %10d\n", $event_name, $unhandled{$event_name});
- }
-}
-
-sub trace_unhandled
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain) = @_;
-
- $unhandled{$event_name}++;
-}
-
-sub print_header
-{
- my ($event_name, $cpu, $secs, $nsecs, $pid, $comm) = @_;
-
- printf("%-20s %5u %05u.%09u %8u %-20s ",
- $event_name, $cpu, $secs, $nsecs, $pid, $comm);
-}
diff --git a/tools/perf/scripts/perl/failed-syscalls.pl b/tools/perf/scripts/perl/failed-syscalls.pl
deleted file mode 100644
index 05954a8f363a..000000000000
--- a/tools/perf/scripts/perl/failed-syscalls.pl
+++ /dev/null
@@ -1,47 +0,0 @@
-# failed system call counts
-# (c) 2010, Tom Zanussi <tzanussi@gmail.com>
-# Licensed under the terms of the GNU GPL License version 2
-#
-# Displays system-wide failed system call totals
-# If a [comm] arg is specified, only syscalls called by [comm] are displayed.
-
-use lib "$ENV{'PERF_EXEC_PATH'}/scripts/perl/Perf-Trace-Util/lib";
-use lib "./Perf-Trace-Util/lib";
-use Perf::Trace::Core;
-use Perf::Trace::Context;
-use Perf::Trace::Util;
-
-my $for_comm = shift;
-
-my %failed_syscalls;
-
-sub raw_syscalls::sys_exit
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain,
- $id, $ret) = @_;
-
- if ($ret < 0) {
- $failed_syscalls{$common_comm}++;
- }
-}
-
-sub syscalls::sys_exit
-{
- raw_syscalls::sys_exit(@_)
-}
-
-sub trace_end
-{
- printf("\nfailed syscalls by comm:\n\n");
-
- printf("%-20s %10s\n", "comm", "# errors");
- printf("%-20s %6s %10s\n", "--------------------", "----------");
-
- foreach my $comm (sort {$failed_syscalls{$b} <=> $failed_syscalls{$a}}
- keys %failed_syscalls) {
- next if ($for_comm && $comm ne $for_comm);
-
- printf("%-20s %10s\n", $comm, $failed_syscalls{$comm});
- }
-}
diff --git a/tools/perf/scripts/perl/rw-by-file.pl b/tools/perf/scripts/perl/rw-by-file.pl
deleted file mode 100644
index 92a750b8552b..000000000000
--- a/tools/perf/scripts/perl/rw-by-file.pl
+++ /dev/null
@@ -1,106 +0,0 @@
-#!/usr/bin/perl -w
-# SPDX-License-Identifier: GPL-2.0-only
-# (c) 2009, Tom Zanussi <tzanussi@gmail.com>
-
-# Display r/w activity for files read/written to for a given program
-
-# The common_* event handler fields are the most useful fields common to
-# all events. They don't necessarily correspond to the 'common_*' fields
-# in the status files. Those fields not available as handler params can
-# be retrieved via script functions of the form get_common_*().
-
-use 5.010000;
-use strict;
-use warnings;
-
-use lib "$ENV{'PERF_EXEC_PATH'}/scripts/perl/Perf-Trace-Util/lib";
-use lib "./Perf-Trace-Util/lib";
-use Perf::Trace::Core;
-use Perf::Trace::Util;
-
-my $usage = "perf script -s rw-by-file.pl <comm>\n";
-
-my $for_comm = shift or die $usage;
-
-my %reads;
-my %writes;
-
-sub syscalls::sys_enter_read
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain, $nr, $fd, $buf, $count) = @_;
-
- if ($common_comm eq $for_comm) {
- $reads{$fd}{bytes_requested} += $count;
- $reads{$fd}{total_reads}++;
- }
-}
-
-sub syscalls::sys_enter_write
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain, $nr, $fd, $buf, $count) = @_;
-
- if ($common_comm eq $for_comm) {
- $writes{$fd}{bytes_written} += $count;
- $writes{$fd}{total_writes}++;
- }
-}
-
-sub trace_end
-{
- printf("file read counts for $for_comm:\n\n");
-
- printf("%6s %10s %10s\n", "fd", "# reads", "bytes_requested");
- printf("%6s %10s %10s\n", "------", "----------", "-----------");
-
- foreach my $fd (sort {$reads{$b}{bytes_requested} <=>
- $reads{$a}{bytes_requested}} keys %reads) {
- my $total_reads = $reads{$fd}{total_reads};
- my $bytes_requested = $reads{$fd}{bytes_requested};
- printf("%6u %10u %10u\n", $fd, $total_reads, $bytes_requested);
- }
-
- printf("\nfile write counts for $for_comm:\n\n");
-
- printf("%6s %10s %10s\n", "fd", "# writes", "bytes_written");
- printf("%6s %10s %10s\n", "------", "----------", "-----------");
-
- foreach my $fd (sort {$writes{$b}{bytes_written} <=>
- $writes{$a}{bytes_written}} keys %writes) {
- my $total_writes = $writes{$fd}{total_writes};
- my $bytes_written = $writes{$fd}{bytes_written};
- printf("%6u %10u %10u\n", $fd, $total_writes, $bytes_written);
- }
-
- print_unhandled();
-}
-
-my %unhandled;
-
-sub print_unhandled
-{
- if ((scalar keys %unhandled) == 0) {
- return;
- }
-
- print "\nunhandled events:\n\n";
-
- printf("%-40s %10s\n", "event", "count");
- printf("%-40s %10s\n", "----------------------------------------",
- "-----------");
-
- foreach my $event_name (keys %unhandled) {
- printf("%-40s %10d\n", $event_name, $unhandled{$event_name});
- }
-}
-
-sub trace_unhandled
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain) = @_;
-
- $unhandled{$event_name}++;
-}
-
-
diff --git a/tools/perf/scripts/perl/rw-by-pid.pl b/tools/perf/scripts/perl/rw-by-pid.pl
deleted file mode 100644
index d789fe39caab..000000000000
--- a/tools/perf/scripts/perl/rw-by-pid.pl
+++ /dev/null
@@ -1,184 +0,0 @@
-#!/usr/bin/perl -w
-# SPDX-License-Identifier: GPL-2.0-only
-# (c) 2009, Tom Zanussi <tzanussi@gmail.com>
-
-# Display r/w activity for all processes
-
-# The common_* event handler fields are the most useful fields common to
-# all events. They don't necessarily correspond to the 'common_*' fields
-# in the status files. Those fields not available as handler params can
-# be retrieved via script functions of the form get_common_*().
-
-use 5.010000;
-use strict;
-use warnings;
-
-use lib "$ENV{'PERF_EXEC_PATH'}/scripts/perl/Perf-Trace-Util/lib";
-use lib "./Perf-Trace-Util/lib";
-use Perf::Trace::Core;
-use Perf::Trace::Util;
-
-my %reads;
-my %writes;
-
-sub syscalls::sys_exit_read
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain,
- $nr, $ret) = @_;
-
- if ($ret > 0) {
- $reads{$common_pid}{bytes_read} += $ret;
- } else {
- if (!defined ($reads{$common_pid}{bytes_read})) {
- $reads{$common_pid}{bytes_read} = 0;
- }
- $reads{$common_pid}{errors}{$ret}++;
- }
-}
-
-sub syscalls::sys_enter_read
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain,
- $nr, $fd, $buf, $count) = @_;
-
- $reads{$common_pid}{bytes_requested} += $count;
- $reads{$common_pid}{total_reads}++;
- $reads{$common_pid}{comm} = $common_comm;
-}
-
-sub syscalls::sys_exit_write
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain,
- $nr, $ret) = @_;
-
- if ($ret <= 0) {
- $writes{$common_pid}{errors}{$ret}++;
- }
-}
-
-sub syscalls::sys_enter_write
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain,
- $nr, $fd, $buf, $count) = @_;
-
- $writes{$common_pid}{bytes_written} += $count;
- $writes{$common_pid}{total_writes}++;
- $writes{$common_pid}{comm} = $common_comm;
-}
-
-sub trace_end
-{
- printf("read counts by pid:\n\n");
-
- printf("%6s %20s %10s %10s %10s\n", "pid", "comm",
- "# reads", "bytes_requested", "bytes_read");
- printf("%6s %-20s %10s %10s %10s\n", "------", "--------------------",
- "-----------", "----------", "----------");
-
- foreach my $pid (sort { ($reads{$b}{bytes_read} || 0) <=>
- ($reads{$a}{bytes_read} || 0) } keys %reads) {
- my $comm = $reads{$pid}{comm} || "";
- my $total_reads = $reads{$pid}{total_reads} || 0;
- my $bytes_requested = $reads{$pid}{bytes_requested} || 0;
- my $bytes_read = $reads{$pid}{bytes_read} || 0;
-
- printf("%6s %-20s %10s %10s %10s\n", $pid, $comm,
- $total_reads, $bytes_requested, $bytes_read);
- }
-
- printf("\nfailed reads by pid:\n\n");
-
- printf("%6s %20s %6s %10s\n", "pid", "comm", "error #", "# errors");
- printf("%6s %20s %6s %10s\n", "------", "--------------------",
- "------", "----------");
-
- my @errcounts = ();
-
- foreach my $pid (keys %reads) {
- foreach my $error (keys %{$reads{$pid}{errors}}) {
- my $comm = $reads{$pid}{comm} || "";
- my $errcount = $reads{$pid}{errors}{$error} || 0;
- push @errcounts, [$pid, $comm, $error, $errcount];
- }
- }
-
- @errcounts = sort { $b->[3] <=> $a->[3] } @errcounts;
-
- for my $i (0 .. $#errcounts) {
- printf("%6d %-20s %6d %10s\n", $errcounts[$i][0],
- $errcounts[$i][1], $errcounts[$i][2], $errcounts[$i][3]);
- }
-
- printf("\nwrite counts by pid:\n\n");
-
- printf("%6s %20s %10s %10s\n", "pid", "comm",
- "# writes", "bytes_written");
- printf("%6s %-20s %10s %10s\n", "------", "--------------------",
- "-----------", "----------");
-
- foreach my $pid (sort { ($writes{$b}{bytes_written} || 0) <=>
- ($writes{$a}{bytes_written} || 0)} keys %writes) {
- my $comm = $writes{$pid}{comm} || "";
- my $total_writes = $writes{$pid}{total_writes} || 0;
- my $bytes_written = $writes{$pid}{bytes_written} || 0;
-
- printf("%6s %-20s %10s %10s\n", $pid, $comm,
- $total_writes, $bytes_written);
- }
-
- printf("\nfailed writes by pid:\n\n");
-
- printf("%6s %20s %6s %10s\n", "pid", "comm", "error #", "# errors");
- printf("%6s %20s %6s %10s\n", "------", "--------------------",
- "------", "----------");
-
- @errcounts = ();
-
- foreach my $pid (keys %writes) {
- foreach my $error (keys %{$writes{$pid}{errors}}) {
- my $comm = $writes{$pid}{comm} || "";
- my $errcount = $writes{$pid}{errors}{$error} || 0;
- push @errcounts, [$pid, $comm, $error, $errcount];
- }
- }
-
- @errcounts = sort { $b->[3] <=> $a->[3] } @errcounts;
-
- for my $i (0 .. $#errcounts) {
- printf("%6d %-20s %6d %10s\n", $errcounts[$i][0],
- $errcounts[$i][1], $errcounts[$i][2], $errcounts[$i][3]);
- }
-
- print_unhandled();
-}
-
-my %unhandled;
-
-sub print_unhandled
-{
- if ((scalar keys %unhandled) == 0) {
- return;
- }
-
- print "\nunhandled events:\n\n";
-
- printf("%-40s %10s\n", "event", "count");
- printf("%-40s %10s\n", "----------------------------------------",
- "-----------");
-
- foreach my $event_name (keys %unhandled) {
- printf("%-40s %10d\n", $event_name, $unhandled{$event_name});
- }
-}
-
-sub trace_unhandled
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain) = @_;
-
- $unhandled{$event_name}++;
-}
diff --git a/tools/perf/scripts/perl/rwtop.pl b/tools/perf/scripts/perl/rwtop.pl
deleted file mode 100644
index eba4df67af6b..000000000000
--- a/tools/perf/scripts/perl/rwtop.pl
+++ /dev/null
@@ -1,203 +0,0 @@
-#!/usr/bin/perl -w
-# SPDX-License-Identifier: GPL-2.0-only
-# (c) 2010, Tom Zanussi <tzanussi@gmail.com>
-
-# read/write top
-#
-# Periodically displays system-wide r/w call activity, broken down by
-# pid. If an [interval] arg is specified, the display will be
-# refreshed every [interval] seconds. The default interval is 3
-# seconds.
-
-use 5.010000;
-use strict;
-use warnings;
-
-use lib "$ENV{'PERF_EXEC_PATH'}/scripts/perl/Perf-Trace-Util/lib";
-use lib "./Perf-Trace-Util/lib";
-use Perf::Trace::Core;
-use Perf::Trace::Util;
-use POSIX qw/SIGALRM SA_RESTART/;
-
-my $default_interval = 3;
-my $nlines = 20;
-my $print_thread;
-my $print_pending = 0;
-
-my %reads;
-my %writes;
-
-my $interval = shift;
-if (!$interval) {
- $interval = $default_interval;
-}
-
-sub syscalls::sys_exit_read
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain,
- $nr, $ret) = @_;
-
- print_check();
-
- if ($ret > 0) {
- $reads{$common_pid}{bytes_read} += $ret;
- } else {
- if (!defined ($reads{$common_pid}{bytes_read})) {
- $reads{$common_pid}{bytes_read} = 0;
- }
- $reads{$common_pid}{errors}{$ret}++;
- }
-}
-
-sub syscalls::sys_enter_read
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain,
- $nr, $fd, $buf, $count) = @_;
-
- print_check();
-
- $reads{$common_pid}{bytes_requested} += $count;
- $reads{$common_pid}{total_reads}++;
- $reads{$common_pid}{comm} = $common_comm;
-}
-
-sub syscalls::sys_exit_write
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain,
- $nr, $ret) = @_;
-
- print_check();
-
- if ($ret <= 0) {
- $writes{$common_pid}{errors}{$ret}++;
- }
-}
-
-sub syscalls::sys_enter_write
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain,
- $nr, $fd, $buf, $count) = @_;
-
- print_check();
-
- $writes{$common_pid}{bytes_written} += $count;
- $writes{$common_pid}{total_writes}++;
- $writes{$common_pid}{comm} = $common_comm;
-}
-
-sub trace_begin
-{
- my $sa = POSIX::SigAction->new(\&set_print_pending);
- $sa->flags(SA_RESTART);
- $sa->safe(1);
- POSIX::sigaction(SIGALRM, $sa) or die "Can't set SIGALRM handler: $!\n";
- alarm 1;
-}
-
-sub trace_end
-{
- print_unhandled();
- print_totals();
-}
-
-sub print_check()
-{
- if ($print_pending == 1) {
- $print_pending = 0;
- print_totals();
- }
-}
-
-sub set_print_pending()
-{
- $print_pending = 1;
- alarm $interval;
-}
-
-sub print_totals
-{
- my $count;
-
- $count = 0;
-
- clear_term();
-
- printf("\nread counts by pid:\n\n");
-
- printf("%6s %20s %10s %10s %10s\n", "pid", "comm",
- "# reads", "bytes_req", "bytes_read");
- printf("%6s %-20s %10s %10s %10s\n", "------", "--------------------",
- "----------", "----------", "----------");
-
- foreach my $pid (sort { ($reads{$b}{bytes_read} || 0) <=>
- ($reads{$a}{bytes_read} || 0) } keys %reads) {
- my $comm = $reads{$pid}{comm} || "";
- my $total_reads = $reads{$pid}{total_reads} || 0;
- my $bytes_requested = $reads{$pid}{bytes_requested} || 0;
- my $bytes_read = $reads{$pid}{bytes_read} || 0;
-
- printf("%6s %-20s %10s %10s %10s\n", $pid, $comm,
- $total_reads, $bytes_requested, $bytes_read);
-
- if (++$count == $nlines) {
- last;
- }
- }
-
- $count = 0;
-
- printf("\nwrite counts by pid:\n\n");
-
- printf("%6s %20s %10s %13s\n", "pid", "comm",
- "# writes", "bytes_written");
- printf("%6s %-20s %10s %13s\n", "------", "--------------------",
- "----------", "-------------");
-
- foreach my $pid (sort { ($writes{$b}{bytes_written} || 0) <=>
- ($writes{$a}{bytes_written} || 0)} keys %writes) {
- my $comm = $writes{$pid}{comm} || "";
- my $total_writes = $writes{$pid}{total_writes} || 0;
- my $bytes_written = $writes{$pid}{bytes_written} || 0;
-
- printf("%6s %-20s %10s %13s\n", $pid, $comm,
- $total_writes, $bytes_written);
-
- if (++$count == $nlines) {
- last;
- }
- }
-
- %reads = ();
- %writes = ();
-}
-
-my %unhandled;
-
-sub print_unhandled
-{
- if ((scalar keys %unhandled) == 0) {
- return;
- }
-
- print "\nunhandled events:\n\n";
-
- printf("%-40s %10s\n", "event", "count");
- printf("%-40s %10s\n", "----------------------------------------",
- "-----------");
-
- foreach my $event_name (keys %unhandled) {
- printf("%-40s %10d\n", $event_name, $unhandled{$event_name});
- }
-}
-
-sub trace_unhandled
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain) = @_;
-
- $unhandled{$event_name}++;
-}
diff --git a/tools/perf/scripts/perl/wakeup-latency.pl b/tools/perf/scripts/perl/wakeup-latency.pl
deleted file mode 100644
index 53444ff4ec7f..000000000000
--- a/tools/perf/scripts/perl/wakeup-latency.pl
+++ /dev/null
@@ -1,107 +0,0 @@
-#!/usr/bin/perl -w
-# SPDX-License-Identifier: GPL-2.0-only
-# (c) 2009, Tom Zanussi <tzanussi@gmail.com>
-
-# Display avg/min/max wakeup latency
-
-# The common_* event handler fields are the most useful fields common to
-# all events. They don't necessarily correspond to the 'common_*' fields
-# in the status files. Those fields not available as handler params can
-# be retrieved via script functions of the form get_common_*().
-
-use 5.010000;
-use strict;
-use warnings;
-
-use lib "$ENV{'PERF_EXEC_PATH'}/scripts/perl/Perf-Trace-Util/lib";
-use lib "./Perf-Trace-Util/lib";
-use Perf::Trace::Core;
-use Perf::Trace::Util;
-
-my %last_wakeup;
-
-my $max_wakeup_latency;
-my $min_wakeup_latency;
-my $total_wakeup_latency = 0;
-my $total_wakeups = 0;
-
-sub sched::sched_switch
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain,
- $prev_comm, $prev_pid, $prev_prio, $prev_state, $next_comm, $next_pid,
- $next_prio) = @_;
-
- my $wakeup_ts = $last_wakeup{$common_cpu}{ts};
- if ($wakeup_ts) {
- my $switch_ts = nsecs($common_secs, $common_nsecs);
- my $wakeup_latency = $switch_ts - $wakeup_ts;
- if ($wakeup_latency > $max_wakeup_latency) {
- $max_wakeup_latency = $wakeup_latency;
- }
- if ($wakeup_latency < $min_wakeup_latency) {
- $min_wakeup_latency = $wakeup_latency;
- }
- $total_wakeup_latency += $wakeup_latency;
- $total_wakeups++;
- }
- $last_wakeup{$common_cpu}{ts} = 0;
-}
-
-sub sched::sched_wakeup
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain,
- $comm, $pid, $prio, $success, $target_cpu) = @_;
-
- $last_wakeup{$target_cpu}{ts} = nsecs($common_secs, $common_nsecs);
-}
-
-sub trace_begin
-{
- $min_wakeup_latency = 1000000000;
- $max_wakeup_latency = 0;
-}
-
-sub trace_end
-{
- printf("wakeup_latency stats:\n\n");
- print "total_wakeups: $total_wakeups\n";
- if ($total_wakeups) {
- printf("avg_wakeup_latency (ns): %u\n",
- avg($total_wakeup_latency, $total_wakeups));
- } else {
- printf("avg_wakeup_latency (ns): N/A\n");
- }
- printf("min_wakeup_latency (ns): %u\n", $min_wakeup_latency);
- printf("max_wakeup_latency (ns): %u\n", $max_wakeup_latency);
-
- print_unhandled();
-}
-
-my %unhandled;
-
-sub print_unhandled
-{
- if ((scalar keys %unhandled) == 0) {
- return;
- }
-
- print "\nunhandled events:\n\n";
-
- printf("%-40s %10s\n", "event", "count");
- printf("%-40s %10s\n", "----------------------------------------",
- "-----------");
-
- foreach my $event_name (keys %unhandled) {
- printf("%-40s %10d\n", $event_name, $unhandled{$event_name});
- }
-}
-
-sub trace_unhandled
-{
- my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
- $common_pid, $common_comm, $common_callchain) = @_;
-
- $unhandled{$event_name}++;
-}
diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index b5eaf326573c..8fa2c4d58b8b 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -76,7 +76,6 @@ make_no_jevents := NO_JEVENTS=1
make_jevents_all := JEVENTS_ARCH=all
make_no_bpf_skel := BUILD_BPF_SKEL=0
make_gen_vmlinux_h := GEN_VMLINUX_H=1
-make_libperl := LIBPERL=1
make_no_python_module := NO_PYTHON_MODULE=1
make_no_scripts := NO_PYTHON_MODULE=1
make_no_slang := NO_SLANG=1
@@ -149,7 +148,6 @@ run += make_no_jevents
run += make_jevents_all
run += make_no_bpf_skel
run += make_gen_vmlinux_h
-run += make_libperl
run += make_no_python_module
run += make_no_scripts
run += make_no_slang
diff --git a/tools/perf/tests/shell/script_perl.sh b/tools/perf/tests/shell/script_perl.sh
deleted file mode 100755
index b6d65b6fbda1..000000000000
--- a/tools/perf/tests/shell/script_perl.sh
+++ /dev/null
@@ -1,102 +0,0 @@
-#!/bin/bash
-# perf script perl tests
-# SPDX-License-Identifier: GPL-2.0
-
-set -e
-
-# set PERF_EXEC_PATH to find scripts in the source directory
-perfdir=$(dirname "$0")/../..
-if [ -e "$perfdir/scripts/perl/Perf-Trace-Util" ]; then
- export PERF_EXEC_PATH=$perfdir
-fi
-
-
-perfdata=$(mktemp /tmp/__perf_test_script_perl.perf.data.XXXXX)
-generated_script=$(mktemp /tmp/__perf_test_script.XXXXX.pl)
-
-cleanup() {
- rm -f "${perfdata}"
- rm -f "${generated_script}"
- trap - EXIT TERM INT
-}
-
-trap_cleanup() {
- echo "Unexpected signal in ${FUNCNAME[1]}"
- cleanup
- exit 1
-}
-trap trap_cleanup TERM INT
-trap cleanup EXIT
-
-check_perl_support() {
- if perf check feature -q libperl; then
- return 0
- fi
- echo "perf script perl test [Skipped: no libperl support]"
- return 2
-}
-
-test_script() {
- local event_name=$1
- local expected_output=$2
- local record_opts=$3
-
- echo "Testing event: $event_name"
-
- # Try to record. If this fails, it might be permissions or lack of support.
- # We return 2 to indicate "skip this event" rather than "fail test".
- if ! perf record -o "${perfdata}" -e "$event_name" $record_opts -- perf test -w thloop > /dev/null 2>&1; then
- echo "perf script perl test [Skipped: failed to record $event_name]"
- return 2
- fi
-
- echo "Generating perl script..."
- if ! perf script -i "${perfdata}" -g "${generated_script}"; then
- echo "perf script perl test [Failed: script generation for $event_name]"
- return 1
- fi
-
- if [ ! -f "${generated_script}" ]; then
- echo "perf script perl test [Failed: script not generated for $event_name]"
- return 1
- fi
-
- echo "Executing perl script..."
- output=$(perf script -i "${perfdata}" -s "${generated_script}" 2>&1)
-
- if echo "$output" | grep -q "$expected_output"; then
- echo "perf script perl test [Success: $event_name triggered $expected_output]"
- return 0
- else
- echo "perf script perl test [Failed: $event_name did not trigger $expected_output]"
- echo "Output was:"
- echo "$output" | head -n 20
- return 1
- fi
-}
-
-check_perl_support || exit 2
-
-# Try tracepoint first
-test_script "sched:sched_switch" "sched::sched_switch" "-c 1" && res=0 || res=$?
-
-if [ $res -eq 0 ]; then
- exit 0
-elif [ $res -eq 1 ]; then
- exit 1
-fi
-
-# If tracepoint skipped (res=2), try task-clock
-# For generic events like task-clock, the generated script uses process_event()
-# which dumps data using Data::Dumper. We check for "$VAR1" which is standard Dumper output.
-test_script "task-clock" "\$VAR1" "-c 100" && res=0 || res=$?
-
-if [ $res -eq 0 ]; then
- exit 0
-elif [ $res -eq 1 ]; then
- exit 1
-fi
-
-# If both skipped
-echo "perf script perl test [Skipped: Could not record tracepoint or task-clock]"
-exit 2
diff --git a/tools/perf/ui/browsers/scripts.c b/tools/perf/ui/browsers/scripts.c
index 7bf967e6cd19..7f554d930319 100644
--- a/tools/perf/ui/browsers/scripts.c
+++ b/tools/perf/ui/browsers/scripts.c
@@ -86,75 +86,6 @@ static int scripts_config(const char *var, const char *value, void *data)
return 0;
}
-/*
- * Some scripts specify the required events in their "xxx-record" file,
- * this function will check if the events in perf.data match those
- * mentioned in the "xxx-record".
- *
- * Fixme: All existing "xxx-record" are all in good formats "-e event ",
- * which is covered well now. And new parsing code should be added to
- * cover the future complex formats like event groups etc.
- */
-static int check_ev_match(int dir_fd, const char *scriptname, struct perf_session *session)
-{
- char line[BUFSIZ];
- FILE *fp;
-
- {
- char filename[NAME_MAX + 5];
- int fd;
-
- scnprintf(filename, sizeof(filename), "bin/%s-record", scriptname);
- fd = openat(dir_fd, filename, O_RDONLY);
- if (fd == -1)
- return -1;
- fp = fdopen(fd, "r");
- if (!fp)
- return -1;
- }
-
- while (fgets(line, sizeof(line), fp)) {
- char *p = skip_spaces(line);
-
- if (*p == '#')
- continue;
-
- while (strlen(p)) {
- int match, len;
- struct evsel *pos;
- char evname[128];
-
- p = strstr(p, "-e");
- if (!p)
- break;
-
- p += 2;
- p = skip_spaces(p);
- len = strcspn(p, " \t");
- if (!len)
- break;
-
- snprintf(evname, sizeof(evname), "%.*s", (int)len, p);
-
- match = 0;
- evlist__for_each_entry(session->evlist, pos) {
- if (evsel__name_is(pos, evname)) {
- match = 1;
- break;
- }
- }
-
- if (!match) {
- fclose(fp);
- return -1;
- }
- }
- }
-
- fclose(fp);
- return 0;
-}
-
/*
* Return -1 if none is found, otherwise the actual scripts number.
*
@@ -162,95 +93,18 @@ static int check_ev_match(int dir_fd, const char *scriptname, struct perf_sessio
* will list all statically runnable scripts, select one, execute it and
* show the output in a perf browser.
*/
-static int find_scripts(char **scripts_array, char **scripts_path_array, int num,
- int pathlen)
+static int find_scripts(char **scripts_array __maybe_unused,
+ char **scripts_path_array __maybe_unused, int num __maybe_unused,
+ int pathlen __maybe_unused)
{
- int namelen;
- struct dirent *script_dirent, *lang_dirent;
- int scripts_dir_fd, lang_dir_fd;
- DIR *scripts_dir, *lang_dir;
- struct perf_session *session;
- struct perf_data data = {
- .path = input_name,
- .mode = PERF_DATA_MODE_READ,
- };
- char *temp;
int i = 0;
- const char *exec_path = get_argv_exec_path();
-
- session = perf_session__new(&data, NULL);
- if (IS_ERR(session))
- return PTR_ERR(session);
-
- {
- char scripts_path[PATH_MAX];
-
- snprintf(scripts_path, sizeof(scripts_path), "%s/scripts", exec_path);
- scripts_dir_fd = open(scripts_path, O_DIRECTORY);
- }
- if (scripts_dir_fd != -1) {
- scripts_dir = fdopendir(scripts_dir_fd);
- if (scripts_dir) {
- while ((lang_dirent = readdir(scripts_dir)) != NULL) {
- if (lang_dirent->d_type != DT_DIR &&
- (lang_dirent->d_type == DT_UNKNOWN &&
- !is_directory_at(scripts_dir_fd, lang_dirent->d_name)))
- continue;
- if (!strcmp(lang_dirent->d_name, ".") ||
- !strcmp(lang_dirent->d_name, ".."))
- continue;
-
- if (strstr(lang_dirent->d_name, "python"))
- continue;
-
- lang_dir_fd = openat(scripts_dir_fd, lang_dirent->d_name,
- O_DIRECTORY);
- if (lang_dir_fd == -1)
- continue;
- lang_dir = fdopendir(lang_dir_fd);
- if (!lang_dir) {
- close(lang_dir_fd);
- continue;
- }
- while ((script_dirent = readdir(lang_dir)) != NULL) {
- if (script_dirent->d_type == DT_DIR)
- continue;
- if (script_dirent->d_type == DT_UNKNOWN &&
- is_directory_at(lang_dir_fd, script_dirent->d_name))
- continue;
- /* Skip those real time scripts: xxxtop.p[yl] */
- if (strstr(script_dirent->d_name, "top."))
- continue;
- if (i >= num)
- break;
- scnprintf(scripts_path_array[i], pathlen,
- "%s/scripts/%s/%s", exec_path,
- lang_dirent->d_name,
- script_dirent->d_name);
- temp = strrchr(script_dirent->d_name, '.');
- namelen = temp ? (int)(temp - script_dirent->d_name)
- : (int)strlen(script_dirent->d_name);
-
- if (namelen >= SCRIPT_NAMELEN)
- namelen = SCRIPT_NAMELEN - 1;
- snprintf(scripts_array[i], namelen + 1, "%s",
- script_dirent->d_name);
-
- if (check_ev_match(lang_dir_fd, scripts_array[i], session))
- continue;
-
- i++;
- }
- closedir(lang_dir);
- }
- closedir(scripts_dir);
- } else {
- close(scripts_dir_fd);
- }
- }
#ifdef HAVE_PYTHON_MODULE_SUPPORT
{
+ int namelen;
+ struct dirent *script_dirent;
+ char *temp;
+ const char *exec_path = get_argv_exec_path();
char py_scripts_path[PATH_MAX];
int py_scripts_dir_fd;
DIR *py_scripts_dir;
@@ -299,7 +153,6 @@ static int find_scripts(char **scripts_array, char **scripts_path_array, int num
}
}
#endif
- perf_session__delete(session);
return i;
}
diff --git a/tools/perf/util/Build b/tools/perf/util/Build
index 3ea888f4210a..f3edc3bbaede 100644
--- a/tools/perf/util/Build
+++ b/tools/perf/util/Build
@@ -95,7 +95,6 @@ perf-util-y += tool_pmu.o
perf-util-y += tp_pmu.o
perf-util-y += svghelper.o
perf-util-y += trace-event-info.o
-perf-util-y += trace-event-scripting.o
perf-util-$(CONFIG_LIBTRACEEVENT) += trace-event.o
perf-util-$(CONFIG_LIBTRACEEVENT) += trace-event-parse.o
perf-util-$(CONFIG_LIBTRACEEVENT) += trace-event-read.o
@@ -236,7 +235,6 @@ endif
perf-util-y += data-convert-json.o
-perf-util-y += scripting-engines/
perf-util-$(CONFIG_ZLIB) += zlib.o
perf-util-$(CONFIG_LZMA) += lzma.o
diff --git a/tools/perf/util/scripting-engines/Build b/tools/perf/util/scripting-engines/Build
deleted file mode 100644
index 3f1dc10526f8..000000000000
--- a/tools/perf/util/scripting-engines/Build
+++ /dev/null
@@ -1,5 +0,0 @@
-ifeq ($(CONFIG_LIBTRACEEVENT),y)
- perf-util-$(CONFIG_LIBPERL) += trace-event-perl.o
-endif
-
-CFLAGS_trace-event-perl.o += $(PERL_EMBED_CCOPTS) -Wno-redundant-decls -Wno-strict-prototypes -Wno-unused-parameter -Wno-shadow -Wno-nested-externs -Wno-undef -Wno-switch-default -Wno-bad-function-cast -Wno-declaration-after-statement -Wno-switch-enum -Wno-thread-safety-analysis
diff --git a/tools/perf/util/scripting-engines/trace-event-perl.c b/tools/perf/util/scripting-engines/trace-event-perl.c
deleted file mode 100644
index 410dc4cd0600..000000000000
--- a/tools/perf/util/scripting-engines/trace-event-perl.c
+++ /dev/null
@@ -1,770 +0,0 @@
-/*
- * trace-event-perl. Feed perf script events to an embedded Perl interpreter.
- *
- * Copyright (C) 2009 Tom Zanussi <tzanussi@gmail.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
- *
- */
-
-#include <inttypes.h>
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <ctype.h>
-#include <errno.h>
-#include <linux/bitmap.h>
-#include <linux/time64.h>
-#include <event-parse.h>
-
-#include <stdbool.h>
-/* perl needs the following define, right after including stdbool.h */
-#define HAS_BOOL
-#include <EXTERN.h>
-#include <perl.h>
-
-#include "../callchain.h"
-#include "../dso.h"
-#include "../machine.h"
-#include "../map.h"
-#include "../symbol.h"
-#include "../thread.h"
-#include "../event.h"
-#include "../trace-event.h"
-#include "../evsel.h"
-#include "../debug.h"
-
-void boot_Perf__Trace__Context(pTHX_ CV *cv);
-void boot_DynaLoader(pTHX_ CV *cv);
-typedef PerlInterpreter * INTERP;
-
-void xs_init(pTHX);
-
-void xs_init(pTHX)
-{
- const char *file = __FILE__;
- dXSUB_SYS;
-
- newXS("Perf::Trace::Context::bootstrap", boot_Perf__Trace__Context,
- file);
- newXS("DynaLoader::boot_DynaLoader", boot_DynaLoader, file);
-}
-
-INTERP my_perl;
-
-#define TRACE_EVENT_TYPE_MAX \
- ((1 << (sizeof(unsigned short) * 8)) - 1)
-
-extern struct scripting_context *scripting_context;
-
-static char *cur_field_name;
-static int zero_flag_atom;
-
-static void define_symbolic_value(const char *ev_name,
- const char *field_name,
- const char *field_value,
- const char *field_str)
-{
- unsigned long long value;
- dSP;
-
- value = eval_flag(field_value);
-
- ENTER;
- SAVETMPS;
- PUSHMARK(SP);
-
- XPUSHs(sv_2mortal(newSVpv(ev_name, 0)));
- XPUSHs(sv_2mortal(newSVpv(field_name, 0)));
- XPUSHs(sv_2mortal(newSVuv(value)));
- XPUSHs(sv_2mortal(newSVpv(field_str, 0)));
-
- PUTBACK;
- if (get_cv("main::define_symbolic_value", 0))
- call_pv("main::define_symbolic_value", G_SCALAR);
- SPAGAIN;
- PUTBACK;
- FREETMPS;
- LEAVE;
-}
-
-static void define_symbolic_values(struct tep_print_flag_sym *field,
- const char *ev_name,
- const char *field_name)
-{
- define_symbolic_value(ev_name, field_name, field->value, field->str);
- if (field->next)
- define_symbolic_values(field->next, ev_name, field_name);
-}
-
-static void define_symbolic_field(const char *ev_name,
- const char *field_name)
-{
- dSP;
-
- ENTER;
- SAVETMPS;
- PUSHMARK(SP);
-
- XPUSHs(sv_2mortal(newSVpv(ev_name, 0)));
- XPUSHs(sv_2mortal(newSVpv(field_name, 0)));
-
- PUTBACK;
- if (get_cv("main::define_symbolic_field", 0))
- call_pv("main::define_symbolic_field", G_SCALAR);
- SPAGAIN;
- PUTBACK;
- FREETMPS;
- LEAVE;
-}
-
-static void define_flag_value(const char *ev_name,
- const char *field_name,
- const char *field_value,
- const char *field_str)
-{
- unsigned long long value;
- dSP;
-
- value = eval_flag(field_value);
-
- ENTER;
- SAVETMPS;
- PUSHMARK(SP);
-
- XPUSHs(sv_2mortal(newSVpv(ev_name, 0)));
- XPUSHs(sv_2mortal(newSVpv(field_name, 0)));
- XPUSHs(sv_2mortal(newSVuv(value)));
- XPUSHs(sv_2mortal(newSVpv(field_str, 0)));
-
- PUTBACK;
- if (get_cv("main::define_flag_value", 0))
- call_pv("main::define_flag_value", G_SCALAR);
- SPAGAIN;
- PUTBACK;
- FREETMPS;
- LEAVE;
-}
-
-static void define_flag_values(struct tep_print_flag_sym *field,
- const char *ev_name,
- const char *field_name)
-{
- define_flag_value(ev_name, field_name, field->value, field->str);
- if (field->next)
- define_flag_values(field->next, ev_name, field_name);
-}
-
-static void define_flag_field(const char *ev_name,
- const char *field_name,
- const char *delim)
-{
- dSP;
-
- ENTER;
- SAVETMPS;
- PUSHMARK(SP);
-
- XPUSHs(sv_2mortal(newSVpv(ev_name, 0)));
- XPUSHs(sv_2mortal(newSVpv(field_name, 0)));
- XPUSHs(sv_2mortal(newSVpv(delim, 0)));
-
- PUTBACK;
- if (get_cv("main::define_flag_field", 0))
- call_pv("main::define_flag_field", G_SCALAR);
- SPAGAIN;
- PUTBACK;
- FREETMPS;
- LEAVE;
-}
-
-static void define_event_symbols(struct tep_event *event,
- const char *ev_name,
- struct tep_print_arg *args)
-{
- if (args == NULL)
- return;
-
- switch (args->type) {
- case TEP_PRINT_NULL:
- break;
- case TEP_PRINT_ATOM:
- define_flag_value(ev_name, cur_field_name, "0",
- args->atom.atom);
- zero_flag_atom = 0;
- break;
- case TEP_PRINT_FIELD:
- free(cur_field_name);
- cur_field_name = strdup(args->field.name);
- break;
- case TEP_PRINT_FLAGS:
- define_event_symbols(event, ev_name, args->flags.field);
- define_flag_field(ev_name, cur_field_name, args->flags.delim);
- define_flag_values(args->flags.flags, ev_name, cur_field_name);
- break;
- case TEP_PRINT_SYMBOL:
- define_event_symbols(event, ev_name, args->symbol.field);
- define_symbolic_field(ev_name, cur_field_name);
- define_symbolic_values(args->symbol.symbols, ev_name,
- cur_field_name);
- break;
- case TEP_PRINT_HEX:
- case TEP_PRINT_HEX_STR:
- define_event_symbols(event, ev_name, args->hex.field);
- define_event_symbols(event, ev_name, args->hex.size);
- break;
- case TEP_PRINT_INT_ARRAY:
- define_event_symbols(event, ev_name, args->int_array.field);
- define_event_symbols(event, ev_name, args->int_array.count);
- define_event_symbols(event, ev_name, args->int_array.el_size);
- break;
- case TEP_PRINT_BSTRING:
- case TEP_PRINT_DYNAMIC_ARRAY:
- case TEP_PRINT_DYNAMIC_ARRAY_LEN:
- case TEP_PRINT_STRING:
- case TEP_PRINT_BITMASK:
- break;
- case TEP_PRINT_TYPE:
- define_event_symbols(event, ev_name, args->typecast.item);
- break;
- case TEP_PRINT_OP:
- if (strcmp(args->op.op, ":") == 0)
- zero_flag_atom = 1;
- define_event_symbols(event, ev_name, args->op.left);
- define_event_symbols(event, ev_name, args->op.right);
- break;
- case TEP_PRINT_FUNC:
- default:
- pr_err("Unsupported print arg type\n");
- /* we should warn... */
- return;
- }
-
- if (args->next)
- define_event_symbols(event, ev_name, args->next);
-}
-
-static SV *perl_process_callchain(struct perf_sample *sample,
- struct addr_location *al)
-{
- struct callchain_cursor *cursor;
- AV *list;
-
- list = newAV();
- if (!list)
- goto exit;
-
- if (!symbol_conf.use_callchain || !sample->callchain)
- goto exit;
-
- cursor = get_tls_callchain_cursor();
-
- if (thread__resolve_callchain(al->thread, cursor,
- sample, NULL, NULL, scripting_max_stack) != 0) {
- pr_err("Failed to resolve callchain. Skipping\n");
- goto exit;
- }
- callchain_cursor_commit(cursor);
-
-
- while (1) {
- HV *elem;
- struct callchain_cursor_node *node;
- node = callchain_cursor_current(cursor);
- if (!node)
- break;
-
- elem = newHV();
- if (!elem)
- goto exit;
-
- if (!hv_stores(elem, "ip", newSVuv(node->ip))) {
- hv_undef(elem);
- goto exit;
- }
-
- if (node->ms.sym) {
- HV *sym = newHV();
- if (!sym) {
- hv_undef(elem);
- goto exit;
- }
- if (!hv_stores(sym, "start", newSVuv(node->ms.sym->start)) ||
- !hv_stores(sym, "end", newSVuv(node->ms.sym->end)) ||
- !hv_stores(sym, "binding", newSVuv(symbol__binding(node->ms.sym))) ||
- !hv_stores(sym, "name", newSVpvn(node->ms.sym->name,
- node->ms.sym->namelen)) ||
- !hv_stores(elem, "sym", newRV_noinc((SV*)sym))) {
- hv_undef(sym);
- hv_undef(elem);
- goto exit;
- }
- }
-
- if (node->ms.map) {
- struct map *map = node->ms.map;
- struct dso *dso = map ? map__dso(map) : NULL;
- const char *dsoname = "[unknown]";
-
- if (dso) {
- if (symbol_conf.show_kernel_path && dso__long_name(dso))
- dsoname = dso__long_name(dso);
- else
- dsoname = dso__name(dso);
- }
- if (!hv_stores(elem, "dso", newSVpv(dsoname,0))) {
- hv_undef(elem);
- goto exit;
- }
- }
-
- callchain_cursor_advance(cursor);
- av_push(list, newRV_noinc((SV*)elem));
- }
-
-exit:
- return newRV_noinc((SV*)list);
-}
-
-static void perl_process_tracepoint(struct perf_sample *sample,
- struct addr_location *al)
-{
- struct thread *thread = al->thread;
- struct tep_event *event;
- struct tep_format_field *field;
- static char handler[256];
- unsigned long long val;
- unsigned long s, ns;
- int pid;
- int cpu = sample->cpu;
- void *data = sample->raw_data;
- unsigned long long nsecs = sample->time;
- const char *comm = thread__comm_str(thread);
- DECLARE_BITMAP(events_defined, TRACE_EVENT_TYPE_MAX);
- struct evsel *evsel = sample->evsel;
-
- bitmap_zero(events_defined, TRACE_EVENT_TYPE_MAX);
- dSP;
-
- if (evsel->core.attr.type != PERF_TYPE_TRACEPOINT)
- return;
-
- event = evsel__tp_format(evsel);
- if (!event) {
- pr_debug("ug! no event found for type %" PRIu64, (u64)evsel->core.attr.config);
- return;
- }
-
- pid = raw_field_value(event, "common_pid", data);
-
- sprintf(handler, "%s::%s", event->system, event->name);
-
- if (!__test_and_set_bit(event->id, events_defined))
- define_event_symbols(event, handler, event->print_fmt.args);
-
- s = nsecs / NSEC_PER_SEC;
- ns = nsecs - s * NSEC_PER_SEC;
-
- ENTER;
- SAVETMPS;
- PUSHMARK(SP);
-
- XPUSHs(sv_2mortal(newSVpv(handler, 0)));
- XPUSHs(sv_2mortal(newSViv(PTR2IV(scripting_context))));
- XPUSHs(sv_2mortal(newSVuv(cpu)));
- XPUSHs(sv_2mortal(newSVuv(s)));
- XPUSHs(sv_2mortal(newSVuv(ns)));
- XPUSHs(sv_2mortal(newSViv(pid)));
- XPUSHs(sv_2mortal(newSVpv(comm, 0)));
- XPUSHs(sv_2mortal(perl_process_callchain(sample, al)));
-
- /* common fields other than pid can be accessed via xsub fns */
-
- for (field = event->format.fields; field; field = field->next) {
- if (field->flags & TEP_FIELD_IS_STRING) {
- int offset;
- if (field->flags & TEP_FIELD_IS_DYNAMIC) {
- offset = *(int *)(data + field->offset);
- offset &= 0xffff;
- if (tep_field_is_relative(field->flags))
- offset += field->offset + field->size;
- } else
- offset = field->offset;
- XPUSHs(sv_2mortal(newSVpv((char *)data + offset, 0)));
- } else { /* FIELD_IS_NUMERIC */
- val = read_size(event, data + field->offset,
- field->size);
- if (field->flags & TEP_FIELD_IS_SIGNED) {
- XPUSHs(sv_2mortal(newSViv(val)));
- } else {
- XPUSHs(sv_2mortal(newSVuv(val)));
- }
- }
- }
-
- PUTBACK;
-
- if (get_cv(handler, 0))
- call_pv(handler, G_SCALAR);
- else if (get_cv("main::trace_unhandled", 0)) {
- XPUSHs(sv_2mortal(newSVpv(handler, 0)));
- XPUSHs(sv_2mortal(newSViv(PTR2IV(scripting_context))));
- XPUSHs(sv_2mortal(newSVuv(cpu)));
- XPUSHs(sv_2mortal(newSVuv(nsecs)));
- XPUSHs(sv_2mortal(newSViv(pid)));
- XPUSHs(sv_2mortal(newSVpv(comm, 0)));
- XPUSHs(sv_2mortal(perl_process_callchain(sample, al)));
- call_pv("main::trace_unhandled", G_SCALAR);
- }
- SPAGAIN;
- PUTBACK;
- FREETMPS;
- LEAVE;
-}
-
-static void perl_process_event_generic(union perf_event *event, struct perf_sample *sample)
-{
- dSP;
-
- if (!get_cv("process_event", 0))
- return;
-
- ENTER;
- SAVETMPS;
- PUSHMARK(SP);
- XPUSHs(sv_2mortal(newSVpvn((const char *)event, event->header.size)));
- XPUSHs(sv_2mortal(newSVpvn((const char *)&sample->evsel->core.attr,
- sizeof(sample->evsel->core.attr))));
- XPUSHs(sv_2mortal(newSVpvn((const char *)sample, sizeof(*sample))));
- XPUSHs(sv_2mortal(newSVpvn((const char *)sample->raw_data, sample->raw_size)));
- PUTBACK;
- call_pv("process_event", G_SCALAR);
- SPAGAIN;
- PUTBACK;
- FREETMPS;
- LEAVE;
-}
-
-static void perl_process_event(union perf_event *event,
- struct perf_sample *sample,
- struct addr_location *al,
- struct addr_location *addr_al)
-{
- scripting_context__update(scripting_context, event, sample, al, addr_al);
- perl_process_tracepoint(sample, al);
- perl_process_event_generic(event, sample);
-}
-
-static void run_start_sub(void)
-{
- dSP; /* access to Perl stack */
- PUSHMARK(SP);
-
- if (get_cv("main::trace_begin", 0))
- call_pv("main::trace_begin", G_DISCARD | G_NOARGS);
-}
-
-/*
- * Start trace script
- */
-static int perl_start_script(const char *script, int argc, const char **argv,
- struct perf_session *session)
-{
- const char **command_line;
- int i, err = 0;
-
- scripting_context->session = session;
-
- command_line = malloc((argc + 2) * sizeof(const char *));
- if (!command_line)
- return -ENOMEM;
-
- command_line[0] = "";
- command_line[1] = script;
- for (i = 2; i < argc + 2; i++)
- command_line[i] = argv[i - 2];
-
- my_perl = perl_alloc();
- perl_construct(my_perl);
-
- if (perl_parse(my_perl, xs_init, argc + 2, (char **)command_line,
- (char **)NULL)) {
- err = -1;
- goto error;
- }
-
- if (perl_run(my_perl)) {
- err = -1;
- goto error;
- }
-
- if (SvTRUE(ERRSV)) {
- err = -1;
- goto error;
- }
-
- run_start_sub();
-
- free(command_line);
- return 0;
-error:
- perl_free(my_perl);
- free(command_line);
-
- return err;
-}
-
-static int perl_flush_script(void)
-{
- return 0;
-}
-
-/*
- * Stop trace script
- */
-static int perl_stop_script(void)
-{
- dSP; /* access to Perl stack */
- PUSHMARK(SP);
-
- if (get_cv("main::trace_end", 0))
- call_pv("main::trace_end", G_DISCARD | G_NOARGS);
-
- perl_destruct(my_perl);
- perl_free(my_perl);
-
- return 0;
-}
-
-static int perl_generate_script(struct tep_handle *pevent, const char *outfile)
-{
- int i, not_first, count, nr_events;
- struct tep_event **all_events;
- struct tep_event *event = NULL;
- struct tep_format_field *f;
- char fname[PATH_MAX];
- FILE *ofp;
-
- sprintf(fname, "%s.pl", outfile);
- ofp = fopen(fname, "w");
- if (ofp == NULL) {
- fprintf(stderr, "couldn't open %s\n", fname);
- return -1;
- }
-
- fprintf(ofp, "# perf script event handlers, "
- "generated by perf script -g perl\n");
-
- fprintf(ofp, "# Licensed under the terms of the GNU GPL"
- " License version 2\n\n");
-
- fprintf(ofp, "# The common_* event handler fields are the most useful "
- "fields common to\n");
-
- fprintf(ofp, "# all events. They don't necessarily correspond to "
- "the 'common_*' fields\n");
-
- fprintf(ofp, "# in the format files. Those fields not available as "
- "handler params can\n");
-
- fprintf(ofp, "# be retrieved using Perl functions of the form "
- "common_*($context).\n");
-
- fprintf(ofp, "# See Context.pm for the list of available "
- "functions.\n\n");
-
- fprintf(ofp, "use lib \"$ENV{'PERF_EXEC_PATH'}/scripts/perl/"
- "Perf-Trace-Util/lib\";\n");
-
- fprintf(ofp, "use lib \"./Perf-Trace-Util/lib\";\n");
- fprintf(ofp, "use Perf::Trace::Core;\n");
- fprintf(ofp, "use Perf::Trace::Context;\n");
- fprintf(ofp, "use Perf::Trace::Util;\n\n");
-
- fprintf(ofp, "sub trace_begin\n{\n\t# optional\n}\n\n");
- fprintf(ofp, "sub trace_end\n{\n\t# optional\n}\n");
-
-
- fprintf(ofp, "\n\
-sub print_backtrace\n\
-{\n\
- my $callchain = shift;\n\
- for my $node (@$callchain)\n\
- {\n\
- if(exists $node->{sym})\n\
- {\n\
- printf( \"\\t[\\%%x] \\%%s\\n\", $node->{ip}, $node->{sym}{name});\n\
- }\n\
- else\n\
- {\n\
- printf( \"\\t[\\%%x]\\n\", $node{ip});\n\
- }\n\
- }\n\
-}\n\n\
-");
-
- nr_events = tep_get_events_count(pevent);
- all_events = tep_list_events(pevent, TEP_EVENT_SORT_ID);
-
- for (i = 0; all_events && i < nr_events; i++) {
- event = all_events[i];
- fprintf(ofp, "sub %s::%s\n{\n", event->system, event->name);
- fprintf(ofp, "\tmy (");
-
- fprintf(ofp, "$event_name, ");
- fprintf(ofp, "$context, ");
- fprintf(ofp, "$common_cpu, ");
- fprintf(ofp, "$common_secs, ");
- fprintf(ofp, "$common_nsecs,\n");
- fprintf(ofp, "\t $common_pid, ");
- fprintf(ofp, "$common_comm, ");
- fprintf(ofp, "$common_callchain,\n\t ");
-
- not_first = 0;
- count = 0;
-
- for (f = event->format.fields; f; f = f->next) {
- if (not_first++)
- fprintf(ofp, ", ");
- if (++count % 5 == 0)
- fprintf(ofp, "\n\t ");
-
- fprintf(ofp, "$%s", f->name);
- }
- fprintf(ofp, ") = @_;\n\n");
-
- fprintf(ofp, "\tprint_header($event_name, $common_cpu, "
- "$common_secs, $common_nsecs,\n\t "
- "$common_pid, $common_comm, $common_callchain);\n\n");
-
- fprintf(ofp, "\tprintf(\"");
-
- not_first = 0;
- count = 0;
-
- for (f = event->format.fields; f; f = f->next) {
- if (not_first++)
- fprintf(ofp, ", ");
- if (count && count % 4 == 0) {
- fprintf(ofp, "\".\n\t \"");
- }
- count++;
-
- fprintf(ofp, "%s=", f->name);
- if (f->flags & TEP_FIELD_IS_STRING ||
- f->flags & TEP_FIELD_IS_FLAG ||
- f->flags & TEP_FIELD_IS_SYMBOLIC)
- fprintf(ofp, "%%s");
- else if (f->flags & TEP_FIELD_IS_SIGNED)
- fprintf(ofp, "%%d");
- else
- fprintf(ofp, "%%u");
- }
-
- fprintf(ofp, "\\n\",\n\t ");
-
- not_first = 0;
- count = 0;
-
- for (f = event->format.fields; f; f = f->next) {
- if (not_first++)
- fprintf(ofp, ", ");
-
- if (++count % 5 == 0)
- fprintf(ofp, "\n\t ");
-
- if (f->flags & TEP_FIELD_IS_FLAG) {
- if ((count - 1) % 5 != 0) {
- fprintf(ofp, "\n\t ");
- count = 4;
- }
- fprintf(ofp, "flag_str(\"");
- fprintf(ofp, "%s::%s\", ", event->system,
- event->name);
- fprintf(ofp, "\"%s\", $%s)", f->name,
- f->name);
- } else if (f->flags & TEP_FIELD_IS_SYMBOLIC) {
- if ((count - 1) % 5 != 0) {
- fprintf(ofp, "\n\t ");
- count = 4;
- }
- fprintf(ofp, "symbol_str(\"");
- fprintf(ofp, "%s::%s\", ", event->system,
- event->name);
- fprintf(ofp, "\"%s\", $%s)", f->name,
- f->name);
- } else
- fprintf(ofp, "$%s", f->name);
- }
-
- fprintf(ofp, ");\n\n");
-
- fprintf(ofp, "\tprint_backtrace($common_callchain);\n");
-
- fprintf(ofp, "}\n\n");
- }
-
- fprintf(ofp, "sub trace_unhandled\n{\n\tmy ($event_name, $context, "
- "$common_cpu, $common_secs, $common_nsecs,\n\t "
- "$common_pid, $common_comm, $common_callchain) = @_;\n\n");
-
- fprintf(ofp, "\tprint_header($event_name, $common_cpu, "
- "$common_secs, $common_nsecs,\n\t $common_pid, "
- "$common_comm, $common_callchain);\n");
- fprintf(ofp, "\tprint_backtrace($common_callchain);\n");
- fprintf(ofp, "}\n\n");
-
- fprintf(ofp, "sub print_header\n{\n"
- "\tmy ($event_name, $cpu, $secs, $nsecs, $pid, $comm) = @_;\n\n"
- "\tprintf(\"%%-20s %%5u %%05u.%%09u %%8u %%-20s \",\n\t "
- "$event_name, $cpu, $secs, $nsecs, $pid, $comm);\n}\n");
-
- fprintf(ofp,
- "\n# Packed byte string args of process_event():\n"
- "#\n"
- "# $event:\tunion perf_event\tutil/event.h\n"
- "# $attr:\tstruct perf_event_attr\tlinux/perf_event.h\n"
- "# $sample:\tstruct perf_sample\tutil/event.h\n"
- "# $raw_data:\tperf_sample->raw_data\tutil/event.h\n"
- "\n"
- "sub process_event\n"
- "{\n"
- "\tmy ($event, $attr, $sample, $raw_data) = @_;\n"
- "\n"
- "\tmy @event\t= unpack(\"LSS\", $event);\n"
- "\tmy @attr\t= unpack(\"LLQQQQQLLQQ\", $attr);\n"
- "\tmy @sample\t= unpack(\"QLLQQQQQLL\", $sample);\n"
- "\tmy @raw_data\t= unpack(\"C*\", $raw_data);\n"
- "\n"
- "\tuse Data::Dumper;\n"
- "\tprint Dumper \\@event, \\@attr, \\@sample, \\@raw_data;\n"
- "}\n");
-
- fclose(ofp);
-
- fprintf(stderr, "generated Perl script: %s\n", fname);
-
- return 0;
-}
-
-struct scripting_ops perl_scripting_ops = {
- .name = "Perl",
- .dirname = "perl",
- .start_script = perl_start_script,
- .flush_script = perl_flush_script,
- .stop_script = perl_stop_script,
- .process_event = perl_process_event,
- .generate_script = perl_generate_script,
-};
diff --git a/tools/perf/util/trace-event-parse.c b/tools/perf/util/trace-event-parse.c
index 9c015fc2bcfb..374cf82fd86e 100644
--- a/tools/perf/util/trace-event-parse.c
+++ b/tools/perf/util/trace-event-parse.c
@@ -14,71 +14,6 @@
#include <linux/kernel.h>
#include <event-parse.h>
-static int get_common_field(struct scripting_context *context,
- int *offset, int *size, const char *type)
-{
- struct tep_handle *pevent = context->pevent;
- struct tep_event *event;
- struct tep_format_field *field;
-
- if (!*size) {
-
- event = tep_get_first_event(pevent);
- if (!event)
- return 0;
-
- field = tep_find_common_field(event, type);
- if (!field)
- return 0;
- *offset = field->offset;
- *size = field->size;
- }
-
- return tep_read_number(pevent, context->event_data + *offset, *size);
-}
-
-int common_lock_depth(struct scripting_context *context)
-{
- static int offset;
- static int size;
- int ret;
-
- ret = get_common_field(context, &size, &offset,
- "common_lock_depth");
- if (ret < 0)
- return -1;
-
- return ret;
-}
-
-int common_flags(struct scripting_context *context)
-{
- static int offset;
- static int size;
- int ret;
-
- ret = get_common_field(context, &size, &offset,
- "common_flags");
- if (ret < 0)
- return -1;
-
- return ret;
-}
-
-int common_pc(struct scripting_context *context)
-{
- static int offset;
- static int size;
- int ret;
-
- ret = get_common_field(context, &size, &offset,
- "common_preempt_count");
- if (ret < 0)
- return -1;
-
- return ret;
-}
-
unsigned long long
raw_field_value(struct tep_event *event, const char *name, void *data)
{
diff --git a/tools/perf/util/trace-event-scripting.c b/tools/perf/util/trace-event-scripting.c
deleted file mode 100644
index c78b317978dd..000000000000
--- a/tools/perf/util/trace-event-scripting.c
+++ /dev/null
@@ -1,398 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * trace-event-scripting. Scripting engine common and initialization code.
- *
- * Copyright (C) 2009-2010 Tom Zanussi <tzanussi@gmail.com>
- */
-
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-#include <errno.h>
-#ifdef HAVE_LIBTRACEEVENT
-#include <event-parse.h>
-#endif
-
-#include "debug.h"
-#include "event.h"
-#include "trace-event.h"
-#include "evsel.h"
-#include <linux/perf_event.h>
-#include <linux/zalloc.h>
-#include "util/sample.h"
-
-unsigned int scripting_max_stack = PERF_MAX_STACK_DEPTH;
-
-struct scripting_context *scripting_context;
-
-struct script_spec {
- struct list_head node;
- struct scripting_ops *ops;
- char spec[];
-};
-
-static LIST_HEAD(script_specs);
-
-static struct script_spec *script_spec__new(const char *spec,
- struct scripting_ops *ops)
-{
- struct script_spec *s = malloc(sizeof(*s) + strlen(spec) + 1);
-
- if (s != NULL) {
- strcpy(s->spec, spec);
- s->ops = ops;
- }
-
- return s;
-}
-
-static void script_spec__add(struct script_spec *s)
-{
- list_add_tail(&s->node, &script_specs);
-}
-
-static struct script_spec *script_spec__find(const char *spec)
-{
- struct script_spec *s;
-
- list_for_each_entry(s, &script_specs, node)
- if (strcasecmp(s->spec, spec) == 0)
- return s;
- return NULL;
-}
-
-static int script_spec_register(const char *spec, struct scripting_ops *ops)
-{
- struct script_spec *s;
-
- s = script_spec__find(spec);
- if (s)
- return -1;
-
- s = script_spec__new(spec, ops);
- if (!s)
- return -1;
-
- script_spec__add(s);
- return 0;
-}
-
-struct scripting_ops *script_spec__lookup(const char *spec)
-{
- struct script_spec *s = script_spec__find(spec);
-
- if (!s)
- return NULL;
-
- return s->ops;
-}
-
-int script_spec__for_each(int (*cb)(struct scripting_ops *ops, const char *spec))
-{
- struct script_spec *s;
- int ret = 0;
-
- list_for_each_entry(s, &script_specs, node) {
- ret = cb(s->ops, s->spec);
- if (ret)
- break;
- }
- return ret;
-}
-
-void scripting_context__update(struct scripting_context *c,
- union perf_event *event,
- struct perf_sample *sample,
- struct addr_location *al,
- struct addr_location *addr_al)
-{
-#ifdef HAVE_LIBTRACEEVENT
- const struct tep_event *tp_format = evsel__tp_format(sample->evsel);
-
- c->pevent = tp_format ? tp_format->tep : NULL;
-#else
- c->pevent = NULL;
-#endif
- c->event_data = sample->raw_data;
- c->event = event;
- c->sample = sample;
- c->al = al;
- c->addr_al = addr_al;
-}
-
-static int flush_script_unsupported(void)
-{
- return 0;
-}
-
-static int stop_script_unsupported(void)
-{
- return 0;
-}
-
-static void process_event_unsupported(union perf_event *event __maybe_unused,
- struct perf_sample *sample __maybe_unused,
- struct addr_location *al __maybe_unused,
- struct addr_location *addr_al __maybe_unused)
-{
-}
-
-static void print_python_unsupported_msg(void)
-{
- fprintf(stderr, "Python scripting not supported."
- " Install libpython and rebuild perf to enable it.\n"
- "For example:\n # apt-get install python-dev (ubuntu)"
- "\n # yum install python-devel (Fedora)"
- "\n etc.\n");
-}
-
-static int python_start_script_unsupported(const char *script __maybe_unused,
- int argc __maybe_unused,
- const char **argv __maybe_unused,
- struct perf_session *session __maybe_unused)
-{
- print_python_unsupported_msg();
-
- return -1;
-}
-
-static int python_generate_script_unsupported(struct tep_handle *pevent
- __maybe_unused,
- const char *outfile
- __maybe_unused)
-{
- print_python_unsupported_msg();
-
- return -1;
-}
-
-struct scripting_ops python_scripting_unsupported_ops = {
- .name = "Python",
- .dirname = "python",
- .start_script = python_start_script_unsupported,
- .flush_script = flush_script_unsupported,
- .stop_script = stop_script_unsupported,
- .process_event = process_event_unsupported,
- .generate_script = python_generate_script_unsupported,
-};
-
-static void register_python_scripting(struct scripting_ops *scripting_ops)
-{
- if (scripting_context == NULL)
- scripting_context = malloc(sizeof(*scripting_context));
-
- if (scripting_context == NULL ||
- script_spec_register("Python", scripting_ops) ||
- script_spec_register("py", scripting_ops)) {
- pr_err("Error registering Python script extension: disabling it\n");
- zfree(&scripting_context);
- }
-}
-
-void setup_python_scripting(void)
-{
- register_python_scripting(&python_scripting_unsupported_ops);
-}
-
-#ifdef HAVE_LIBTRACEEVENT
-static void print_perl_unsupported_msg(void)
-{
- fprintf(stderr, "Perl scripting not supported."
- " Install libperl and rebuild perf to enable it.\n"
- "For example:\n # apt-get install libperl-dev (ubuntu)"
- "\n # yum install 'perl(ExtUtils::Embed)' (Fedora)"
- "\n etc.\n");
-}
-
-static int perl_start_script_unsupported(const char *script __maybe_unused,
- int argc __maybe_unused,
- const char **argv __maybe_unused,
- struct perf_session *session __maybe_unused)
-{
- print_perl_unsupported_msg();
-
- return -1;
-}
-
-static int perl_generate_script_unsupported(struct tep_handle *pevent
- __maybe_unused,
- const char *outfile __maybe_unused)
-{
- print_perl_unsupported_msg();
-
- return -1;
-}
-
-struct scripting_ops perl_scripting_unsupported_ops = {
- .name = "Perl",
- .dirname = "perl",
- .start_script = perl_start_script_unsupported,
- .flush_script = flush_script_unsupported,
- .stop_script = stop_script_unsupported,
- .process_event = process_event_unsupported,
- .generate_script = perl_generate_script_unsupported,
-};
-
-static void register_perl_scripting(struct scripting_ops *scripting_ops)
-{
- if (scripting_context == NULL)
- scripting_context = malloc(sizeof(*scripting_context));
-
- if (scripting_context == NULL ||
- script_spec_register("Perl", scripting_ops) ||
- script_spec_register("pl", scripting_ops)) {
- pr_err("Error registering Perl script extension: disabling it\n");
- zfree(&scripting_context);
- }
-}
-
-#ifndef HAVE_LIBPERL_SUPPORT
-void setup_perl_scripting(void)
-{
- register_perl_scripting(&perl_scripting_unsupported_ops);
-}
-#else
-extern struct scripting_ops perl_scripting_ops;
-
-void setup_perl_scripting(void)
-{
- register_perl_scripting(&perl_scripting_ops);
-}
-#endif
-#endif
-
-static const struct {
- u32 flags;
- const char *name;
-} sample_flags[] = {
- {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL, "call"},
- {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN, "return"},
- {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CONDITIONAL, "jcc"},
- {PERF_IP_FLAG_BRANCH, "jmp"},
- {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_INTERRUPT, "int"},
- {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN | PERF_IP_FLAG_INTERRUPT, "iret"},
- {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_SYSCALLRET, "syscall"},
- {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_RETURN | PERF_IP_FLAG_SYSCALLRET, "sysret"},
- {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_ASYNC, "async"},
- {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_ASYNC | PERF_IP_FLAG_INTERRUPT,
- "hw int"},
- {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TX_ABORT, "tx abrt"},
- {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TRACE_BEGIN, "tr strt"},
- {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_TRACE_END, "tr end"},
- {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_VMENTRY, "vmentry"},
- {PERF_IP_FLAG_BRANCH | PERF_IP_FLAG_CALL | PERF_IP_FLAG_VMEXIT, "vmexit"},
- {0, NULL}
-};
-
-static const struct {
- u32 flags;
- const char *name;
-} branch_events[] = {
- {PERF_IP_FLAG_BRANCH_MISS, "miss"},
- {PERF_IP_FLAG_NOT_TAKEN, "not_taken"},
- {0, NULL}
-};
-
-static int sample_flags_to_name(u32 flags, char *str, size_t size)
-{
- int i;
- const char *prefix;
- int pos = 0, ret, ev_idx = 0;
- u32 xf = flags & PERF_ADDITIONAL_STATE_MASK;
- u32 types, events;
- char xs[16] = { 0 };
-
- /* Clear additional state bits */
- flags &= ~PERF_ADDITIONAL_STATE_MASK;
-
- if (flags & PERF_IP_FLAG_TRACE_BEGIN)
- prefix = "tr strt ";
- else if (flags & PERF_IP_FLAG_TRACE_END)
- prefix = "tr end ";
- else
- prefix = "";
-
- ret = snprintf(str + pos, size - pos, "%s", prefix);
- if (ret < 0)
- return ret;
- pos += ret;
-
- flags &= ~(PERF_IP_FLAG_TRACE_BEGIN | PERF_IP_FLAG_TRACE_END);
-
- types = flags & ~PERF_IP_FLAG_BRANCH_EVENT_MASK;
- for (i = 0; sample_flags[i].name; i++) {
- if (sample_flags[i].flags != types)
- continue;
-
- ret = snprintf(str + pos, size - pos, "%s", sample_flags[i].name);
- if (ret < 0)
- return ret;
- pos += ret;
- break;
- }
-
- events = flags & PERF_IP_FLAG_BRANCH_EVENT_MASK;
- for (i = 0; branch_events[i].name; i++) {
- if (!(branch_events[i].flags & events))
- continue;
-
- ret = snprintf(str + pos, size - pos, !ev_idx ? "/%s" : ",%s",
- branch_events[i].name);
- if (ret < 0)
- return ret;
- pos += ret;
- ev_idx++;
- }
-
- /* Add an end character '/' for events */
- if (ev_idx) {
- ret = snprintf(str + pos, size - pos, "/");
- if (ret < 0)
- return ret;
- pos += ret;
- }
-
- if (!xf)
- return pos;
-
- snprintf(xs, sizeof(xs), "(%s%s%s)",
- flags & PERF_IP_FLAG_IN_TX ? "x" : "",
- flags & PERF_IP_FLAG_INTR_DISABLE ? "D" : "",
- flags & PERF_IP_FLAG_INTR_TOGGLE ? "t" : "");
-
- /* Right align the string if its length is less than the limit */
- if ((pos + strlen(xs)) < SAMPLE_FLAGS_STR_ALIGNED_SIZE)
- ret = snprintf(str + pos, size - pos, "%*s",
- (int)(SAMPLE_FLAGS_STR_ALIGNED_SIZE - ret), xs);
- else
- ret = snprintf(str + pos, size - pos, " %s", xs);
- if (ret < 0)
- return ret;
-
- return pos + ret;
-}
-
-int perf_sample__sprintf_flags(u32 flags, char *str, size_t sz)
-{
- const char *chars = PERF_IP_FLAG_CHARS;
- const size_t n = strlen(PERF_IP_FLAG_CHARS);
- size_t i, pos = 0;
- int ret;
-
- ret = sample_flags_to_name(flags, str, sz);
- if (ret > 0)
- return ret;
-
- for (i = 0; i < n; i++, flags >>= 1) {
- if ((flags & 1) && pos < sz)
- str[pos++] = chars[i];
- }
- for (; i < 32; i++, flags >>= 1) {
- if ((flags & 1) && pos < sz)
- str[pos++] = '?';
- }
- if (pos < sz)
- str[pos] = 0;
-
- return pos;
-}
diff --git a/tools/perf/util/trace-event.h b/tools/perf/util/trace-event.h
index 720121c74f1d..19f22ac1faf3 100644
--- a/tools/perf/util/trace-event.h
+++ b/tools/perf/util/trace-event.h
@@ -7,15 +7,9 @@
#include <sys/types.h>
#include <linux/types.h>
-struct evlist;
struct machine;
-struct perf_sample;
-union perf_event;
-struct perf_tool;
-struct thread;
-struct tep_plugin_list;
-struct evsel;
struct tep_format_field;
+struct tep_plugin_list;
struct trace_event {
struct tep_handle *pevent;
@@ -79,70 +73,6 @@ struct tracing_data *tracing_data_get(struct list_head *pattrs,
int fd, bool temp);
int tracing_data_put(struct tracing_data *tdata);
-
-struct addr_location;
-
-struct perf_session;
-struct perf_stat_config;
-
-struct scripting_ops {
- const char *name;
- const char *dirname; /* For script path .../scripts/<dirname>/... */
- int (*start_script)(const char *script, int argc, const char **argv,
- struct perf_session *session);
- int (*flush_script) (void);
- int (*stop_script) (void);
- void (*process_event) (union perf_event *event,
- struct perf_sample *sample,
- struct addr_location *al,
- struct addr_location *addr_al);
- void (*process_switch)(union perf_event *event,
- struct perf_sample *sample,
- struct machine *machine);
- void (*process_auxtrace_error)(struct perf_session *session,
- union perf_event *event);
- void (*process_stat)(struct perf_stat_config *config,
- struct evsel *evsel, u64 tstamp);
- void (*process_stat_interval)(u64 tstamp);
- void (*process_throttle)(union perf_event *event,
- struct perf_sample *sample,
- struct machine *machine);
- int (*generate_script) (struct tep_handle *pevent, const char *outfile);
-};
-
-extern unsigned int scripting_max_stack;
-
-struct scripting_ops *script_spec__lookup(const char *spec);
-int script_spec__for_each(int (*cb)(struct scripting_ops *ops, const char *spec));
-
-void setup_perl_scripting(void);
-void setup_python_scripting(void);
-
-struct scripting_context {
- struct tep_handle *pevent;
- void *event_data;
- union perf_event *event;
- struct perf_sample *sample;
- struct addr_location *al;
- struct addr_location *addr_al;
- struct perf_session *session;
-};
-
-void scripting_context__update(struct scripting_context *scripting_context,
- union perf_event *event,
- struct perf_sample *sample,
- struct addr_location *al,
- struct addr_location *addr_al);
-
-int common_pc(struct scripting_context *context);
-int common_flags(struct scripting_context *context);
-int common_lock_depth(struct scripting_context *context);
-
-#define SAMPLE_FLAGS_BUF_SIZE 64
-#define SAMPLE_FLAGS_STR_ALIGNED_SIZE 21
-
-int perf_sample__sprintf_flags(u32 flags, char *str, size_t sz);
-
#if defined(LIBTRACEEVENT_VERSION) && LIBTRACEEVENT_VERSION >= MAKE_LIBTRACEEVENT_VERSION(1, 5, 0)
#include <event-parse.h>
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread* [PATCH v1 49/49] perf Documentation: Update for standalone Python scripts
2026-09-20 5:20 [PATCH v1 00/49] perf: Complete transition to standalone Python scripts Ian Rogers
` (47 preceding siblings ...)
2026-09-20 5:21 ` [PATCH v1 48/49] perf script: Support standalone scripts and remove embedded scripting Ian Rogers
@ 2026-09-20 5:21 ` Ian Rogers
48 siblings, 0 replies; 50+ messages in thread
From: Ian Rogers @ 2026-09-20 5:21 UTC (permalink / raw)
To: irogers, acme, adrian.hunter, alice.mei.rogers, james.clark,
linux-perf-users, namhyung
Cc: dapeng1.mi, leo.yan, linux-kernel, mingo, peterz, tmricht
Update perf-script documentation to reflect standalone Python script
execution and the removal of embedded Python and Perl scripting:
- Remove documentation for the removed -g and -s options and legacy
record/report script wrapper modes in perf-script.txt.
- Remove references to perf-script-perl and delete obsolete
perf-script-perl.txt.
- Rewrite perf-script-python.txt to document writing and running
standalone Python scripts using the perf module.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
---
tools/perf/Documentation/db-export.txt | 16 +-
tools/perf/Documentation/perf-script-perl.txt | 216 ------
.../perf/Documentation/perf-script-python.txt | 718 +++---------------
tools/perf/Documentation/perf-script.txt | 80 +-
tools/perf/Documentation/tips.txt | 1 -
5 files changed, 106 insertions(+), 925 deletions(-)
delete mode 100644 tools/perf/Documentation/perf-script-perl.txt
diff --git a/tools/perf/Documentation/db-export.txt b/tools/perf/Documentation/db-export.txt
index 20024e1f9164..a736be3b6e84 100644
--- a/tools/perf/Documentation/db-export.txt
+++ b/tools/perf/Documentation/db-export.txt
@@ -1,11 +1,11 @@
Database Export
===============
-perf tool's python scripting engine:
+perf tool's python module:
- tools/perf/util/scripting-engines/trace-event-python.c
+ tools/perf/util/python.c
-supports scripts:
+supports standalone scripts:
tools/perf/python/export-to-sqlite.py
tools/perf/python/export-to-postgresql.py
@@ -31,11 +31,5 @@ backward compatibility by testing for the presence of new tables and columns
before using them. e.g. function IsSelectable() in exported-sql-viewer.py
4. The export scripts themselves maintain forward compatibility (i.e. an existing
-script will continue to work with new versions of perf) by accepting a variable
-number of arguments (e.g. def call_return_table(*x)) i.e. perf can pass more
-arguments which old scripts will ignore.
-
-5. The scripting engine tests for the existence of script handler functions
-before calling them. The scripting engine can also test for the support of new
-or optional features by checking for the existence and value of script global
-variables.
+script will continue to work with new versions of perf) by querying event and
+sample attributes dynamically from the perf Python module.
diff --git a/tools/perf/Documentation/perf-script-perl.txt b/tools/perf/Documentation/perf-script-perl.txt
deleted file mode 100644
index 5b479f5e62ff..000000000000
--- a/tools/perf/Documentation/perf-script-perl.txt
+++ /dev/null
@@ -1,216 +0,0 @@
-perf-script-perl(1)
-===================
-
-NAME
-----
-perf-script-perl - Process trace data with a Perl script
-
-SYNOPSIS
---------
-[verse]
-'perf script' [-s [Perl]:script[.pl] ]
-
-DESCRIPTION
------------
-
-This perf script option is used to process perf script data using perf's
-built-in Perl interpreter. It reads and processes the input file and
-displays the results of the trace analysis implemented in the given
-Perl script, if any.
-
-STARTER SCRIPTS
----------------
-
-You can avoid reading the rest of this document by running 'perf script
--g perl' in the same directory as an existing perf.data trace file.
-That will generate a starter script containing a handler for each of
-the event types in the trace file; it simply prints every available
-field for each event in the trace file.
-
-You can also look at the existing scripts in
-~/libexec/perf-core/scripts/perl for typical examples showing how to
-do basic things like aggregate event data, print results, etc. Also,
-the check-perf-script.pl script, while not interesting for its results,
-attempts to exercise all of the main scripting features.
-
-EVENT HANDLERS
---------------
-
-When perf script is invoked using a trace script, a user-defined
-'handler function' is called for each event in the trace. If there's
-no handler function defined for a given event type, the event is
-ignored (or passed to a 'trace_unhandled' function, see below) and the
-next event is processed.
-
-Most of the event's field values are passed as arguments to the
-handler function; some of the less common ones aren't - those are
-available as calls back into the perf executable (see below).
-
-As an example, the following perf record command can be used to record
-all sched_wakeup events in the system:
-
- # perf record -a -e sched:sched_wakeup
-
-Traces meant to be processed using a script should be recorded with
-the above option: -a to enable system-wide collection.
-
-The format file for the sched_wakeup event defines the following fields
-(see /sys/kernel/tracing/events/sched/sched_wakeup/format):
-
-----
- format:
- field:unsigned short common_type;
- field:unsigned char common_flags;
- field:unsigned char common_preempt_count;
- field:int common_pid;
-
- field:char comm[TASK_COMM_LEN];
- field:pid_t pid;
- field:int prio;
- field:int success;
- field:int target_cpu;
-----
-
-The handler function for this event would be defined as:
-
-----
-sub sched::sched_wakeup
-{
- my ($event_name, $context, $common_cpu, $common_secs,
- $common_nsecs, $common_pid, $common_comm,
- $comm, $pid, $prio, $success, $target_cpu) = @_;
-}
-----
-
-The handler function takes the form subsystem::event_name.
-
-The $common_* arguments in the handler's argument list are the set of
-arguments passed to all event handlers; some of the fields correspond
-to the common_* fields in the format file, but some are synthesized,
-and some of the common_* fields aren't common enough to to be passed
-to every event as arguments but are available as library functions.
-
-Here's a brief description of each of the invariant event args:
-
- $event_name the name of the event as text
- $context an opaque 'cookie' used in calls back into perf
- $common_cpu the cpu the event occurred on
- $common_secs the secs portion of the event timestamp
- $common_nsecs the nsecs portion of the event timestamp
- $common_pid the pid of the current task
- $common_comm the name of the current process
-
-All of the remaining fields in the event's format file have
-counterparts as handler function arguments of the same name, as can be
-seen in the example above.
-
-The above provides the basics needed to directly access every field of
-every event in a trace, which covers 90% of what you need to know to
-write a useful trace script. The sections below cover the rest.
-
-SCRIPT LAYOUT
--------------
-
-Every perf script Perl script should start by setting up a Perl module
-search path and 'use'ing a few support modules (see module
-descriptions below):
-
-----
- use lib "$ENV{'PERF_EXEC_PATH'}/scripts/perl/Perf-Trace-Util/lib";
- use lib "./Perf-Trace-Util/lib";
- use Perf::Trace::Core;
- use Perf::Trace::Context;
- use Perf::Trace::Util;
-----
-
-The rest of the script can contain handler functions and support
-functions in any order.
-
-Aside from the event handler functions discussed above, every script
-can implement a set of optional functions:
-
-*trace_begin*, if defined, is called before any event is processed and
-gives scripts a chance to do setup tasks:
-
-----
- sub trace_begin
- {
- }
-----
-
-*trace_end*, if defined, is called after all events have been
- processed and gives scripts a chance to do end-of-script tasks, such
- as display results:
-
-----
-sub trace_end
-{
-}
-----
-
-*trace_unhandled*, if defined, is called after for any event that
- doesn't have a handler explicitly defined for it. The standard set
- of common arguments are passed into it:
-
-----
-sub trace_unhandled
-{
- my ($event_name, $context, $common_cpu, $common_secs,
- $common_nsecs, $common_pid, $common_comm) = @_;
-}
-----
-
-The remaining sections provide descriptions of each of the available
-built-in perf script Perl modules and their associated functions.
-
-AVAILABLE MODULES AND FUNCTIONS
--------------------------------
-
-The following sections describe the functions and variables available
-via the various Perf::Trace::* Perl modules. To use the functions and
-variables from the given module, add the corresponding 'use
-Perf::Trace::XXX' line to your perf script script.
-
-Perf::Trace::Core Module
-~~~~~~~~~~~~~~~~~~~~~~~~
-
-These functions provide some essential functions to user scripts.
-
-The *flag_str* and *symbol_str* functions provide human-readable
-strings for flag and symbolic fields. These correspond to the strings
-and values parsed from the 'print fmt' fields of the event format
-files:
-
- flag_str($event_name, $field_name, $field_value) - returns the string representation corresponding to $field_value for the flag field $field_name of event $event_name
- symbol_str($event_name, $field_name, $field_value) - returns the string representation corresponding to $field_value for the symbolic field $field_name of event $event_name
-
-Perf::Trace::Context Module
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Some of the 'common' fields in the event format file aren't all that
-common, but need to be made accessible to user scripts nonetheless.
-
-Perf::Trace::Context defines a set of functions that can be used to
-access this data in the context of the current event. Each of these
-functions expects a $context variable, which is the same as the
-$context variable passed into every event handler as the second
-argument.
-
- common_pc($context) - returns common_preempt count for the current event
- common_flags($context) - returns common_flags for the current event
- common_lock_depth($context) - returns common_lock_depth for the current event
-
-Perf::Trace::Util Module
-~~~~~~~~~~~~~~~~~~~~~~~~
-
-Various utility functions for use with perf script:
-
- nsecs($secs, $nsecs) - returns total nsecs given secs/nsecs pair
- nsecs_secs($nsecs) - returns whole secs portion given nsecs
- nsecs_nsecs($nsecs) - returns nsecs remainder given nsecs
- nsecs_str($nsecs) - returns printable string in the form secs.nsecs
- avg($total, $n) - returns average given a sum and a total number of values
-
-SEE ALSO
---------
-linkperf:perf-script[1]
diff --git a/tools/perf/Documentation/perf-script-python.txt b/tools/perf/Documentation/perf-script-python.txt
index 27a1cac6fe76..72003a82f3d4 100644
--- a/tools/perf/Documentation/perf-script-python.txt
+++ b/tools/perf/Documentation/perf-script-python.txt
@@ -3,676 +3,148 @@ perf-script-python(1)
NAME
----
-perf-script-python - Process trace data with a Python script
+perf-script-python - Process trace data with a Python script using perf module
SYNOPSIS
--------
[verse]
-'perf script' [-s [Python]:script[.py] ]
+'perf script' <script>.py
DESCRIPTION
-----------
-This perf script option is used to process perf script data using perf's
-built-in Python interpreter. It reads and processes the input file and
-displays the results of the trace analysis implemented in the given
-Python script, if any.
+This document describes how to use the `perf` Python module to process
+trace data recorded by `perf record`.
+
+With the removal of embedded Python interpreter from `perf`, scripts
+are now run as standalone Python programs that import the `perf`
+module to access trace data. Symbol configuration options (`--vmlinux`,
+`--kallsyms`, and `--symfs`) passed to `perf script` are forwarded via
+`PERF_SYMBOL_*` environment variables to `perf.session`, and can also
+be passed directly to `perf.session`. Trace filtering command-line
+options (such as `-c`, `-p`, `-t`, `--time`, and `--dlfilter`) passed
+to `perf script` are not applied to standalone scripts.
A QUICK EXAMPLE
---------------
-This section shows the process, start to finish, of creating a working
-Python script that aggregates and extracts useful information from a
-raw perf script stream. You can avoid reading the rest of this
-document if an example is enough for you; the rest of the document
-provides more details on each step and lists the library functions
-available to script writers.
-
-This example actually details the steps that were used to create the
-'syscall-counts' script you see when you list the available perf script
-scripts via 'perf script -l'. As such, this script also shows how to
-integrate your script into the list of general-purpose 'perf script'
-scripts listed by that command.
-
-The syscall-counts script is a simple script, but demonstrates all the
-basic ideas necessary to create a useful script. Here's an example
-of its output (syscall names are not yet supported, they will appear
-as numbers):
-
-----
-syscall events:
-
-event count
----------------------------------------- -----------
-sys_write 455067
-sys_getdents 4072
-sys_close 3037
-sys_swapoff 1769
-sys_read 923
-sys_sched_setparam 826
-sys_open 331
-sys_newfstat 326
-sys_mmap 217
-sys_munmap 216
-sys_futex 141
-sys_select 102
-sys_poll 84
-sys_setitimer 12
-sys_writev 8
-15 8
-sys_lseek 7
-sys_rt_sigprocmask 6
-sys_wait4 3
-sys_ioctl 3
-sys_set_robust_list 1
-sys_exit 1
-56 1
-sys_access 1
-----
-
-Basically our task is to keep a per-syscall tally that gets updated
-every time a system call occurs in the system. Our script will do
-that, but first we need to record the data that will be processed by
-that script. Theoretically, there are a couple of ways we could do
-that:
-
-- we could enable every event under the tracing/events/syscalls
- directory, but this is over 600 syscalls, well beyond the number
- allowable by perf. These individual syscall events will however be
- useful if we want to later use the guidance we get from the
- general-purpose scripts to drill down and get more detail about
- individual syscalls of interest.
-
-- we can enable the sys_enter and/or sys_exit syscalls found under
- tracing/events/raw_syscalls. These are called for all syscalls; the
- 'id' field can be used to distinguish between individual syscall
- numbers.
-
-For this script, we only need to know that a syscall was entered; we
-don't care how it exited, so we'll use 'perf record' to record only
-the sys_enter events:
-
-----
-# perf record -a -e raw_syscalls:sys_enter
-
-^C[ perf record: Woken up 1 times to write data ]
-[ perf record: Captured and wrote 56.545 MB perf.data (~2470503 samples) ]
-----
-
-The options basically say to collect data for every syscall event
-system-wide and multiplex the per-cpu output into a single stream.
-That single stream will be recorded in a file in the current directory
-called perf.data.
-
-Once we have a perf.data file containing our data, we can use the -g
-'perf script' option to generate a Python script that will contain a
-callback handler for each event type found in the perf.data trace
-stream (for more details, see the STARTER SCRIPTS section).
-
-----
-# perf script -g python
-generated Python script: perf-script.py
-
-The output file created also in the current directory is named
-perf-script.py. Here's the file in its entirety:
-
-# perf script event handlers, generated by perf script -g python
-# Licensed under the terms of the GNU GPL License version 2
-
-# The common_* event handler fields are the most useful fields common to
-# all events. They don't necessarily correspond to the 'common_*' fields
-# in the format files. Those fields not available as handler params can
-# be retrieved using Python functions of the form common_*(context).
-# See the perf-script-python Documentation for the list of available functions.
-
-import os
-import sys
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import *
-from Core import *
-
-def trace_begin():
- print "in trace_begin"
-
-def trace_end():
- print "in trace_end"
-
-def raw_syscalls__sys_enter(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- id, args):
- print_header(event_name, common_cpu, common_secs, common_nsecs,
- common_pid, common_comm)
-
- print "id=%d, args=%s\n" % \
- (id, args),
-
-def trace_unhandled(event_name, context, event_fields_dict):
- print ' '.join(['%s=%s'%(k,str(v))for k,v in sorted(event_fields_dict.items())])
-
-def print_header(event_name, cpu, secs, nsecs, pid, comm):
- print "%-20s %5u %05u.%09u %8u %-20s " % \
- (event_name, cpu, secs, nsecs, pid, comm),
-----
-
-At the top is a comment block followed by some import statements and a
-path append which every perf script script should include.
-
-Following that are a couple generated functions, trace_begin() and
-trace_end(), which are called at the beginning and the end of the
-script respectively (for more details, see the SCRIPT_LAYOUT section
-below).
-
-Following those are the 'event handler' functions generated one for
-every event in the 'perf record' output. The handler functions take
-the form subsystem\__event_name, and contain named parameters, one for
-each field in the event; in this case, there's only one event,
-raw_syscalls__sys_enter(). (see the EVENT HANDLERS section below for
-more info on event handlers).
-
-The final couple of functions are, like the begin and end functions,
-generated for every script. The first, trace_unhandled(), is called
-every time the script finds an event in the perf.data file that
-doesn't correspond to any event handler in the script. This could
-mean either that the record step recorded event types that it wasn't
-really interested in, or the script was run against a trace file that
-doesn't correspond to the script.
-
-The script generated by -g option simply prints a line for each
-event found in the trace stream i.e. it basically just dumps the event
-and its parameter values to stdout. The print_header() function is
-simply a utility function used for that purpose. Let's rename the
-script and run it to see the default output:
-
-----
-# mv perf-script.py syscall-counts.py
-# perf script -s syscall-counts.py
-
-raw_syscalls__sys_enter 1 00840.847582083 7506 perf id=1, args=
-raw_syscalls__sys_enter 1 00840.847595764 7506 perf id=1, args=
-raw_syscalls__sys_enter 1 00840.847620860 7506 perf id=1, args=
-raw_syscalls__sys_enter 1 00840.847710478 6533 npviewer.bin id=78, args=
-raw_syscalls__sys_enter 1 00840.847719204 6533 npviewer.bin id=142, args=
-raw_syscalls__sys_enter 1 00840.847755445 6533 npviewer.bin id=3, args=
-raw_syscalls__sys_enter 1 00840.847775601 6533 npviewer.bin id=3, args=
-raw_syscalls__sys_enter 1 00840.847781820 6533 npviewer.bin id=3, args=
-.
-.
-.
-----
-
-Of course, for this script, we're not interested in printing every
-trace event, but rather aggregating it in a useful way. So we'll get
-rid of everything to do with printing as well as the trace_begin() and
-trace_unhandled() functions, which we won't be using. That leaves us
-with this minimalistic skeleton:
-
-----
-import os
-import sys
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import *
-from Core import *
-
-def trace_end():
- print "in trace_end"
-
-def raw_syscalls__sys_enter(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- id, args):
-----
-
-In trace_end(), we'll simply print the results, but first we need to
-generate some results to print. To do that we need to have our
-sys_enter() handler do the necessary tallying until all events have
-been counted. A hash table indexed by syscall id is a good way to
-store that information; every time the sys_enter() handler is called,
-we simply increment a count associated with that hash entry indexed by
-that syscall id:
-
-----
- syscalls = autodict()
-
- try:
- syscalls[id] += 1
- except TypeError:
- syscalls[id] = 1
-----
-
-The syscalls 'autodict' object is a special kind of Python dictionary
-(implemented in Core.py) that implements Perl's 'autovivifying' hashes
-in Python i.e. with autovivifying hashes, you can assign nested hash
-values without having to go to the trouble of creating intermediate
-levels if they don't exist e.g syscalls[comm][pid][id] = 1 will create
-the intermediate hash levels and finally assign the value 1 to the
-hash entry for 'id' (because the value being assigned isn't a hash
-object itself, the initial value is assigned in the TypeError
-exception. Well, there may be a better way to do this in Python but
-that's what works for now).
-
-Putting that code into the raw_syscalls__sys_enter() handler, we
-effectively end up with a single-level dictionary keyed on syscall id
-and having the counts we've tallied as values.
-
-The print_syscall_totals() function iterates over the entries in the
-dictionary and displays a line for each entry containing the syscall
-name (the dictionary keys contain the syscall ids, which are passed to
-the Util function syscall_name(), which translates the raw syscall
-numbers to the corresponding syscall name strings). The output is
-displayed after all the events in the trace have been processed, by
-calling the print_syscall_totals() function from the trace_end()
-handler called at the end of script processing.
-
-The final script producing the output shown above is shown in its
-entirety below (syscall_name() helper is not yet available, you can
-only deal with id's for now):
-
-----
-import os
-import sys
-
-sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
-from perf_trace_context import *
-from Core import *
-from Util import *
-
-syscalls = autodict()
-
-def trace_end():
- print_syscall_totals()
-
-def raw_syscalls__sys_enter(event_name, context, common_cpu,
- common_secs, common_nsecs, common_pid, common_comm,
- id, args):
- try:
- syscalls[id] += 1
- except TypeError:
- syscalls[id] = 1
-
-def print_syscall_totals():
- if for_comm is not None:
- print "\nsyscall events for %s:\n\n" % (for_comm),
- else:
- print "\nsyscall events:\n\n",
-
- print "%-40s %10s\n" % ("event", "count"),
- print "%-40s %10s\n" % ("----------------------------------------", \
- "-----------"),
-
- for id, val in sorted(syscalls.iteritems(), key = lambda(k, v): (v, k), \
- reverse = True):
- print "%-40s %10d\n" % (syscall_name(id), val),
-----
-
-The script can be run just as before:
-
- # perf script -s syscall-counts.py
-
-So those are the essential steps in writing and running a script. The
-process can be generalized to any tracepoint or set of tracepoints
-you're interested in - basically find the tracepoint(s) you're
-interested in by looking at the list of available events shown by
-'perf list' and/or look in /sys/kernel/tracing/events/ for
-detailed event and field info, record the corresponding trace data
-using 'perf record', passing it the list of interesting events,
-generate a skeleton script using 'perf script -g python' and modify the
-code to aggregate and display it for your particular needs.
-
-After you've done that you may end up with a general-purpose script
-that you want to keep around and have available for future use. By
-writing a couple of very simple shell scripts and putting them in the
-right place, you can have your script listed alongside the other
-scripts listed by the 'perf script -l' command e.g.:
-
-----
-# perf script -l
-List of available trace scripts:
- wakeup-latency system-wide min/max/avg wakeup latency
- rw-by-file <comm> r/w activity for a program, by file
- rw-by-pid system-wide r/w activity
-----
-
-A nice side effect of doing this is that you also then capture the
-probably lengthy 'perf record' command needed to record the events for
-the script.
-
-To have the script appear as a 'built-in' script, you write two simple
-scripts, one for recording and one for 'reporting'.
-
-The 'record' script is a shell script with the same base name as your
-script, but with -record appended. The shell script should be put
-into the perf/scripts/python/bin directory in the kernel source tree.
-In that script, you write the 'perf record' command-line needed for
-your script:
-
-----
-# cat kernel-source/tools/perf/scripts/python/bin/syscall-counts-record
-
-#!/bin/bash
-perf record -a -e raw_syscalls:sys_enter
-----
-
-The 'report' script is also a shell script with the same base name as
-your script, but with -report appended. It should also be located in
-the perf/scripts/python/bin directory. In that script, you write the
-'perf script -s' command-line needed for running your script:
+This section shows how to create a simple Python script that reads a
+`perf.data` file and prints event information.
-----
-# cat kernel-source/tools/perf/scripts/python/bin/syscall-counts-report
+Create a file named `print_events.py` with the following content:
-#!/bin/bash
-# description: system-wide syscall counts
-perf script -s ~/libexec/perf-core/scripts/python/syscall-counts.py
-----
+ #!/usr/bin/env python3
+ import perf
-Note that the location of the Python script given in the shell script
-is in the libexec/perf-core/scripts/python directory - this is where
-the script will be copied by 'make install' when you install perf.
-For the installation to install your script there, your script needs
-to be located in the perf/scripts/python directory in the kernel
-source tree:
+ def process_event(sample):
+ print(f"Event: {sample.evsel} on CPU {sample.sample_cpu} at {sample.sample_time}")
-----
-# ls -al kernel-source/tools/perf/scripts/python
-total 32
-drwxr-xr-x 4 trz trz 4096 2010-01-26 22:30 .
-drwxr-xr-x 4 trz trz 4096 2010-01-26 22:29 ..
-drwxr-xr-x 2 trz trz 4096 2010-01-26 22:29 bin
--rw-r--r-- 1 trz trz 2548 2010-01-26 22:29 check-perf-script.py
-drwxr-xr-x 3 trz trz 4096 2010-01-26 22:49 Perf-Trace-Util
--rw-r--r-- 1 trz trz 1462 2010-01-26 22:30 syscall-counts.py
-----
+ # Open the session with perf.data file
+ session = perf.session(perf.data("perf.data"), sample=process_event)
-Once you've done that (don't forget to do a new 'make install',
-otherwise your script won't show up at run-time), 'perf script -l'
-should show a new entry for your script:
+ # Process all events
+ session.process_events()
-----
-# perf script -l
-List of available trace scripts:
- wakeup-latency system-wide min/max/avg wakeup latency
- rw-by-file <comm> r/w activity for a program, by file
- rw-by-pid system-wide r/w activity
- syscall-counts system-wide syscall counts
-----
+Make the script executable:
+ $ chmod +x print_events.py
-You can now perform the record step via 'perf script record':
+Record some data:
+ $ perf record -a sleep 1
- # perf script record syscall-counts
+Run the script:
+ $ perf script print_events.py
-and display the output using 'perf script report':
+Or run it directly with Python, ensuring `perf.so` is in your `PYTHONPATH`:
+ $ PYTHONPATH=/path/to/perf/python python3 print_events.py
- # perf script report syscall-counts
-
-STARTER SCRIPTS
+THE PERF MODULE
---------------
-You can quickly get started writing a script for a particular set of
-trace data by generating a skeleton script using 'perf script -g
-python' in the same directory as an existing perf.data trace file.
-That will generate a starter script containing a handler for each of
-the event types in the trace file; it simply prints every available
-field for each event in the trace file.
-
-You can also look at the existing scripts in
-~/libexec/perf-core/scripts/python for typical examples showing how to
-do basic things like aggregate event data, print results, etc. Also,
-the check-perf-script.py script, while not interesting for its results,
-attempts to exercise all of the main scripting features.
-
-EVENT HANDLERS
---------------
-
-When perf script is invoked using a trace script, a user-defined
-'handler function' is called for each event in the trace. If there's
-no handler function defined for a given event type, the event is
-ignored (or passed to a 'trace_unhandled' function, see below) and the
-next event is processed.
-
-Most of the event's field values are passed as arguments to the
-handler function; some of the less common ones aren't - those are
-available as calls back into the perf executable (see below).
-
-As an example, the following perf record command can be used to record
-all sched_wakeup events in the system:
-
- # perf record -a -e sched:sched_wakeup
-
-Traces meant to be processed using a script should be recorded with
-the above option: -a to enable system-wide collection.
-
-The format file for the sched_wakeup event defines the following fields
-(see /sys/kernel/tracing/events/sched/sched_wakeup/format):
-
-----
- format:
- field:unsigned short common_type;
- field:unsigned char common_flags;
- field:unsigned char common_preempt_count;
- field:int common_pid;
-
- field:char comm[TASK_COMM_LEN];
- field:pid_t pid;
- field:int prio;
- field:int success;
- field:int target_cpu;
-----
-
-The handler function for this event would be defined as:
-
-----
-def sched__sched_wakeup(event_name, context, common_cpu, common_secs,
- common_nsecs, common_pid, common_comm,
- comm, pid, prio, success, target_cpu):
- pass
-----
-
-The handler function takes the form subsystem__event_name.
-
-The common_* arguments in the handler's argument list are the set of
-arguments passed to all event handlers; some of the fields correspond
-to the common_* fields in the format file, but some are synthesized,
-and some of the common_* fields aren't common enough to to be passed
-to every event as arguments but are available as library functions.
-
-Here's a brief description of each of the invariant event args:
-
- event_name the name of the event as text
- context an opaque 'cookie' used in calls back into perf
- common_cpu the cpu the event occurred on
- common_secs the secs portion of the event timestamp
- common_nsecs the nsecs portion of the event timestamp
- common_pid the pid of the current task
- common_comm the name of the current process
-
-All of the remaining fields in the event's format file have
-counterparts as handler function arguments of the same name, as can be
-seen in the example above.
-
-The above provides the basics needed to directly access every field of
-every event in a trace, which covers 90% of what you need to know to
-write a useful trace script. The sections below cover the rest.
-
-SCRIPT LAYOUT
--------------
-
-Every perf script Python script should start by setting up a Python
-module search path and 'import'ing a few support modules (see module
-descriptions below):
-
-----
- import os
- import sys
-
- sys.path.append(os.environ['PERF_EXEC_PATH'] + \
- '/scripts/python/Perf-Trace-Util/lib/Perf/Trace')
-
- from perf_trace_context import *
- from Core import *
-----
-
-The rest of the script can contain handler functions and support
-functions in any order.
-
-Aside from the event handler functions discussed above, every script
-can implement a set of optional functions:
-
-*trace_begin*, if defined, is called before any event is processed and
-gives scripts a chance to do setup tasks:
-
-----
-def trace_begin():
- pass
-----
-
-*trace_end*, if defined, is called after all events have been
- processed and gives scripts a chance to do end-of-script tasks, such
- as display results:
-
-----
-def trace_end():
- pass
-----
-
-*trace_unhandled*, if defined, is called after for any event that
- doesn't have a handler explicitly defined for it. The standard set
- of common arguments are passed into it:
-
-----
-def trace_unhandled(event_name, context, event_fields_dict):
- pass
-----
-
-*process_event*, if defined, is called for any non-tracepoint event
-
-----
-def process_event(param_dict):
- pass
-----
-
-*context_switch*, if defined, is called for any context switch
-
-----
-def context_switch(ts, cpu, pid, tid, np_pid, np_tid, machine_pid, out, out_preempt, *x):
- pass
-----
-
-*auxtrace_error*, if defined, is called for any AUX area tracing error
-
-----
-def auxtrace_error(typ, code, cpu, pid, tid, ip, ts, msg, cpumode, *x):
- pass
-----
-
-The remaining sections provide descriptions of each of the available
-built-in perf script Python modules and their associated functions.
-
-AVAILABLE MODULES AND FUNCTIONS
--------------------------------
-
-The following sections describe the functions and variables available
-via the various perf script Python modules. To use the functions and
-variables from the given module, add the corresponding 'from XXXX
-import' line to your perf script script.
-
-Core.py Module
-~~~~~~~~~~~~~~
+The `perf` module provides several classes and functions to interact
+with trace data.
-These functions provide some essential functions to user scripts.
+### Module Functions
-The *flag_str* and *symbol_str* functions provide human-readable
-strings for flag and symbolic fields. These correspond to the strings
-and values parsed from the 'print fmt' fields of the event format
-files:
+- `config_get(name)`: Get a perf config value.
+- `metrics()`: Returns a list of metrics represented as string values in dictionaries.
+- `tracepoint(subsystem, name)`: Get tracepoint config.
+- `parse_events(string)`: Parse a string of events and return an `evlist`.
+- `parse_metrics(string, pmu=None)`: Parse a string of metrics or metric groups and return an `evlist`.
+- `pmus()`: Returns a sequence of PMUs.
+- `syscall_name(num, *, elf_machine=None)`: Turns a syscall number to a string.
+- `syscall_id(name, *, elf_machine=None)`: Turns a syscall name to a number.
- flag_str(event_name, field_name, field_value) - returns the string representation corresponding to field_value for the flag field field_name of event event_name
- symbol_str(event_name, field_name, field_value) - returns the string representation corresponding to field_value for the symbolic field field_name of event event_name
+### `perf.pmu`
-The *autodict* function returns a special kind of Python
-dictionary that implements Perl's 'autovivifying' hashes in Python
-i.e. with autovivifying hashes, you can assign nested hash values
-without having to go to the trouble of creating intermediate levels if
-they don't exist.
+Represents a Performance Monitoring Unit.
- autodict() - returns an autovivifying dictionary instance
+- `events()`: Returns a sequence of events encoded as dictionaries.
+- `name()`: Name of the PMU including suffixes.
+### `perf.evlist`
-perf_trace_context Module
-~~~~~~~~~~~~~~~~~~~~~~~~~
+Represents a list of event selectors.
-Some of the 'common' fields in the event format file aren't all that
-common, but need to be made accessible to user scripts nonetheless.
+- `all_cpus()`: CPU map union of all evsel CPU maps.
+- `metrics()`: List of metric names within the evlist.
+- `compute_metric(name, cpu, thread)`: Compute metric for given name, cpu and thread.
+- `mmap()`: mmap the file descriptor table.
+- `open()`: open the file descriptors.
+- `close()`: close the file descriptors.
+- `poll()`: poll the file descriptor table.
+- `get_pollfd()`: get the poll file descriptor table.
+- `add(evsel)`: adds an event selector to the list.
+- `read_on_cpu(cpu)`: reads an event.
+- `config()`: Apply default record options to the evlist.
+- `disable()`: Disable the evsels in the evlist.
+- `enable()`: Enable the evsels in the evlist.
-perf_trace_context defines a set of functions that can be used to
-access this data in the context of the current event. Each of these
-functions expects a context variable, which is the same as the
-context variable passed into every tracepoint event handler as the second
-argument. For non-tracepoint events, the context variable is also present
-as perf_trace_context.perf_script_context .
+### `perf.evsel`
- common_pc(context) - returns common_preempt count for the current event
- common_flags(context) - returns common_flags for the current event
- common_lock_depth(context) - returns common_lock_depth for the current event
- perf_sample_insn(context) - returns the machine code instruction
- perf_set_itrace_options(context, itrace_options) - set --itrace options if they have not been set already
- perf_sample_srcline(context) - returns source_file_name, line_number
- perf_sample_srccode(context) - returns source_file_name, line_number, source_line
- perf_config_get(config_name) - returns the value of the named config item, or None if unset
+Represents an event selector.
-Util.py Module
-~~~~~~~~~~~~~~
+- `open()`: open the event selector file descriptor table.
+- `cpus()`: CPUs the event is to be used with.
+- `threads()`: threads the event is to be used with.
+- `read(cpu, thread)`: read counters. Returns a count object with `val`, `ena`, and `run` attributes.
-Various utility functions for use with perf script:
+### `perf.session`
- nsecs(secs, nsecs) - returns total nsecs given secs/nsecs pair
- nsecs_secs(nsecs) - returns whole secs portion given nsecs
- nsecs_nsecs(nsecs) - returns nsecs remainder given nsecs
- nsecs_str(nsecs) - returns printable string in the form secs.nsecs
- avg(total, n) - returns average given a sum and a total number of values
+Manages a trace session.
-SUPPORTED FIELDS
-----------------
+- `__init__(data, sample=None, stat=None, context_switch=None, call_return=None, itrace=None, vmlinux=None, kallsyms=None, symfs=None)`: Creates a new session. `data` is a `perf.data` object. `sample` is a callback function called for each sample event. `vmlinux`, `kallsyms`, and `symfs` configure symbol resolution paths.
+- `process_events()`: Reads the trace data and calls the sample callback for each event.
+- `find_thread(pid, tid=-1)`: Returns the thread associated with a PID/TID.
-Currently supported fields:
+### `perf.data`
-ev_name, comm, id, stream_id, pid, tid, cpu, ip, time, period, phys_addr,
-addr, symbol, symoff, dso, time_enabled, time_running, values, callchain,
-brstack, brstacksym, datasrc, datasrc_decode, iregs, uregs,
-weight, transaction, raw_buf, attr, cpumode.
+Represents a trace file.
-Fields that may also be present:
+- `__init__(path=None, fd=-1)`: Opens a trace file.
- flags - sample flags
- flags_disp - sample flags display
- insn_cnt - instruction count for determining instructions-per-cycle (IPC)
- cyc_cnt - cycle count for determining IPC
- addr_correlates_sym - addr can correlate to a symbol
- addr_dso - addr dso
- addr_symbol - addr symbol
- addr_symoff - addr symbol offset
+### Sample Object
-Some fields have sub items:
+Passed to the callback function in `perf.session`.
-brstack:
- from, to, from_dsoname, to_dsoname, mispred,
- predicted, in_tx, abort, cycles.
+- `evsel`: The event selector (name of the event).
+- `sample_cpu`: The CPU on which the event occurred.
+- `sample_time`: The timestamp of the event.
+- `sample_pid`: The PID of the process.
+- `sample_tid`: The TID of the thread.
+- `raw_buf`: Raw buffer containing event specific data.
-brstacksym:
- items: from, to, pred, in_tx, abort (converted string)
+COUNTER AND METRIC APIS
+-----------------------
-For example,
-We can use this code to print brstack "from", "to", "cycles".
+The following APIs are used in `tools/perf/python/ilist.py` for
+interactive listing and reading of counters and metrics:
-if 'brstack' in dict:
- for entry in dict['brstack']:
- print "from %s, to %s, cycles %s" % (entry["from"], entry["to"], entry["cycles"])
+- `perf.pmus()`: Used to get all available PMUs.
+- `pmu.events()`: Used to get all events for a specific PMU.
+- `perf.metrics()`: Used to get all available metrics.
+- `perf.parse_metrics(metric_name, pmu)`: Used to parse a metric and get an `evlist`.
+- `evlist.compute_metric(metric_name, cpu, thread)`: Used to compute a metric value for a specific CPU and thread.
+- `evsel.read(cpu, thread)`: Used to read raw counter values.
SEE ALSO
--------
diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt
index 200ea25891d8..59d9dc059723 100644
--- a/tools/perf/Documentation/perf-script.txt
+++ b/tools/perf/Documentation/perf-script.txt
@@ -9,10 +9,7 @@ SYNOPSIS
--------
[verse]
'perf script' [<options>]
-'perf script' [<options>] record <script> [<record-options>] <command>
-'perf script' [<options>] report <script> [script-args]
-'perf script' [<options>] <script> <required-script-args> [<record-options>] <command>
-'perf script' [<options>] <top-script> [script-args]
+'perf script' [<options>] <script> [script-args]
DESCRIPTION
-----------
@@ -23,58 +20,12 @@ There are several variants of perf script:
'perf script' to see a detailed trace of the workload that was
recorded.
- You can also run a set of pre-canned scripts that aggregate and
- summarize the raw trace data in various ways (the list of scripts is
- available via 'perf script -l'). The following variants allow you to
- record and run those scripts:
-
- 'perf script record <script> <command>' to record the events required
- for 'perf script report'. <script> is the name displayed in the
- output of 'perf script --list' i.e. the actual script name minus any
- language extension. If <command> is not specified, the events are
- recorded using the -a (system-wide) 'perf record' option.
-
- 'perf script report <script> [args]' to run and display the results
- of <script>. <script> is the name displayed in the output of 'perf
- script --list' i.e. the actual script name minus any language
- extension. The perf.data output from a previous run of 'perf script
- record <script>' is used and should be present for this command to
- succeed. [args] refers to the (mainly optional) args expected by
- the script.
-
- 'perf script <script> <required-script-args> <command>' to both
- record the events required for <script> and to run the <script>
- using 'live-mode' i.e. without writing anything to disk. <script>
- is the name displayed in the output of 'perf script --list' i.e. the
- actual script name minus any language extension. If <command> is
- not specified, the events are recorded using the -a (system-wide)
- 'perf record' option. If <script> has any required args, they
- should be specified before <command>. This mode doesn't allow for
- optional script args to be specified; if optional script args are
- desired, they can be specified using separate 'perf script record'
- and 'perf script report' commands, with the stdout of the record step
- piped to the stdin of the report script, using the '-o -' and '-i -'
- options of the corresponding commands.
-
- 'perf script <top-script>' to both record the events required for
- <top-script> and to run the <top-script> using 'live-mode'
- i.e. without writing anything to disk. <top-script> is the name
- displayed in the output of 'perf script --list' i.e. the actual
- script name minus any language extension; a <top-script> is defined
- as any script name ending with the string 'top'.
-
- [<record-options>] can be passed to the record steps of 'perf script
- record' and 'live-mode' variants; this isn't possible however for
- <top-script> 'live-mode' or 'perf script report' variants.
-
- See the 'SEE ALSO' section for links to language-specific
- information on how to write and run your own trace scripts.
+ You can also run standalone scripts that aggregate and summarize the
+ raw trace data in various ways (the list of scripts is available via
+ 'perf script -l').
OPTIONS
-------
-<command>...::
- Any command you can specify in a shell.
-
-D::
--dump-raw-trace=::
Display verbose dump of the trace data.
@@ -90,18 +41,7 @@ OPTIONS
--list=::
Display a list of available trace scripts.
--s ['lang']::
---script=::
- Process trace data with the given script ([lang]:script[.ext]).
- If the string 'lang' is specified in place of a script name, a
- list of supported languages will be displayed instead.
--g::
---gen-script=::
- Generate a starter script. If a language is given then the
- script is named perf-script.[ext] according to the
- language. If a file path is given then python is used for
- files ending '.py' and perl used for files ending '.pl'.
--dlfilter=<file>::
Filter sample events using the given shared object file.
@@ -115,12 +55,6 @@ OPTIONS
Display a list of available dlfilters. Use with option -v (must come
before option --list-dlfilters) to show long descriptions.
--a::
- Force system-wide collection. Scripts run without a <command>
- normally use -a by default, while scripts run with a <command>
- normally don't - this option allows the latter to be run in
- system-wide mode.
-
-i::
--input=::
Input file name. (default: perf.data unless stdin is a fifo)
@@ -342,7 +276,6 @@ OPTIONS
Display extended information about the perf.data file. This adds
information which may be very large and thus may clutter the display.
It currently includes: cpu and numa topology of the host system.
- It can only be used with the perf script report mode.
--show-kernel-path::
Try to resolve the path of [kernel.kallsyms]
@@ -543,6 +476,5 @@ include::guest-files.txt[]
SEE ALSO
--------
-linkperf:perf-record[1], linkperf:perf-script-perl[1],
-linkperf:perf-script-python[1], linkperf:perf-intel-pt[1],
-linkperf:perf-dlfilter[1]
+linkperf:perf-record[1], linkperf:perf-script-python[1],
+linkperf:perf-intel-pt[1], linkperf:perf-dlfilter[1]
diff --git a/tools/perf/Documentation/tips.txt b/tools/perf/Documentation/tips.txt
index eb6d56d4d8ba..28953179f59d 100644
--- a/tools/perf/Documentation/tips.txt
+++ b/tools/perf/Documentation/tips.txt
@@ -4,7 +4,6 @@ Compare performance results with: perf diff [<old file> <new file>]
Boolean options have negative forms, e.g.: perf report --no-children
To not accumulate CPU time of children symbols add --no-children
Customize output of perf script with: perf script -F event,ip,sym
-Generate a script for your data: perf script -g <lang>
Save output of perf stat using: perf stat record <target workload>
Create an archive with symtabs to analyse on other machine: perf archive
Search options using a keyword: perf report -h <keyword>
--
2.55.0.1082.g2b9226bbc0-goog
^ permalink raw reply [flat|nested] 50+ messages in thread