mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Ian Rogers <irogers@google.com>
To: "Namhyung Kim" <namhyung@kernel.org>,
	"Peter Zijlstra" <peterz@infradead.org>,
	"Ingo Molnar" <mingo@redhat.com>,
	"Arnaldo Carvalho de Melo" <acme@kernel.org>,
	"Mark Rutland" <mark.rutland@arm.com>,
	"Alexander Shishkin" <alexander.shishkin@linux.intel.com>,
	"Jiri Olsa" <jolsa@kernel.org>,
	"Adrian Hunter" <adrian.hunter@intel.com>,
	"Kan Liang" <kan.liang@linux.intel.com>,
	"Sam James" <sam@gentoo.org>,
	"Jesper Juhl" <jesperjuhl76@gmail.com>,
	"James Clark" <james.clark@linaro.org>,
	"Zhongqiu Han" <quic_zhonhan@quicinc.com>,
	"Yicong Yang" <yangyicong@hisilicon.com>,
	"Thomas Richter" <tmricht@linux.ibm.com>,
	"Michael Petlan" <mpetlan@redhat.com>,
	"Anne Macedo" <retpolanne@posteo.net>,
	"Dominique Martinet" <asmadeus@codewreck.org>,
	"Jean-Philippe Romain" <jean-philippe.romain@foss.st.com>,
	"Junhao He" <hejunhao3@huawei.com>,
	linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org,
	"Krzysztof Łopatowski" <krzysztof.m.lopatowski@gmail.com>
Cc: Ian Rogers <irogers@google.com>
Subject: [PATCH v3 0/8] Add io_dir to avoid memory overhead from opendir
Date: Fri, 21 Feb 2025 22:10:05 -0800	[thread overview]
Message-ID: <20250222061015.303622-1-irogers@google.com> (raw)

glibc's opendir allocates a minimum of 32kb, when called recursively
for a directory tree the memory consumption can add up - nearly 300kb
during perf start-up when processing modules. Add a stack allocated
variant of readdir sized a little more than 1kb

v3: Rebase on top of Krzysztof Łopatowski's work. Add additional
    defines for SYS_getdents64 on all other architectures if its
    definition is missing. Add a patch to further reduce the
    stack/memory usage in machine__set_modules_path_dir by appending
    to a buffer rather than creating a copy.
v2: Remove the feature test and always use a perf supplied getdents64
    to workaround an Alpine Linux issue in v1:
    https://lore.kernel.org/lkml/20231207050433.1426834-1-irogers@google.com/
    As suggested by Krzysztof Łopatowski
    <krzysztof.m.lopatowski@gmail.com> who also pointed to the perf
    trace performance improvements in start-up time eliminating stat
    calls can achieve:
    https://lore.kernel.org/lkml/20250206113314.335376-2-krzysztof.m.lopatowski@gmail.com/
    Convert parse-events and hwmon_pmu to use io_dir.
v1: This was previously part of the memory saving change set:
    https://lore.kernel.org/lkml/20231127220902.1315692-1-irogers@google.com/
    It is separated here and a feature check and syscall workaround
    for missing getdents64 added.

Ian Rogers (8):
  tools lib api: Add io_dir an allocation free readdir alternative
  perf maps: Switch modules tree walk to io_dir__readdir
  perf pmu: Switch to io_dir__readdir
  perf header: Switch mem topology to io_dir__readdir
  perf events: Remove scandir in thread synthesis
  perf parse-events: Switch tracepoints to io_dir__readdir
  perf hwmon_pmu: Switch event discovery to io_dir__readdir
  perf machine: Reuse module path buffer

 tools/lib/api/Makefile             |   2 +-
 tools/lib/api/io_dir.h             | 104 +++++++++++++++++++++++++++++
 tools/perf/util/header.c           |  31 ++++-----
 tools/perf/util/hwmon_pmu.c        |  42 +++++-------
 tools/perf/util/machine.c          |  57 ++++++++--------
 tools/perf/util/parse-events.c     |  32 +++++----
 tools/perf/util/pmu.c              |  46 ++++++-------
 tools/perf/util/pmus.c             |  30 +++------
 tools/perf/util/synthetic-events.c |  22 +++---
 9 files changed, 229 insertions(+), 137 deletions(-)
 create mode 100644 tools/lib/api/io_dir.h

-- 
2.48.1.658.g4767266eb4-goog


             reply	other threads:[~2025-02-22  6:10 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-02-22  6:10 Ian Rogers [this message]
2025-02-22  6:10 ` [PATCH v3 1/8] tools lib api: Add io_dir an allocation free readdir alternative Ian Rogers
2025-02-22  6:10 ` [PATCH v3 2/8] perf maps: Switch modules tree walk to io_dir__readdir Ian Rogers
2025-02-22  6:10 ` [PATCH v3 3/8] perf pmu: Switch " Ian Rogers
2025-02-22  6:10 ` [PATCH v3 4/8] perf header: Switch mem topology " Ian Rogers
2025-02-22  6:10 ` [PATCH v3 5/8] perf events: Remove scandir in thread synthesis Ian Rogers
2025-02-22  6:10 ` [PATCH v3 6/8] perf parse-events: Switch tracepoints to io_dir__readdir Ian Rogers
2025-02-22  6:10 ` [PATCH v3 7/8] perf hwmon_pmu: Switch event discovery " Ian Rogers
2025-02-22  6:10 ` [PATCH v3 8/8] perf machine: Reuse module path buffer Ian Rogers
2025-02-25  0:28 ` [PATCH v3 0/8] Add io_dir to avoid memory overhead from opendir Namhyung Kim
2025-02-25  0:30   ` Namhyung Kim
2025-02-25  1:25     ` Ian Rogers
2025-02-25 21:58 ` Namhyung Kim

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250222061015.303622-1-irogers@google.com \
    --to=irogers@google.com \
    --cc=acme@kernel.org \
    --cc=adrian.hunter@intel.com \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=asmadeus@codewreck.org \
    --cc=hejunhao3@huawei.com \
    --cc=james.clark@linaro.org \
    --cc=jean-philippe.romain@foss.st.com \
    --cc=jesperjuhl76@gmail.com \
    --cc=jolsa@kernel.org \
    --cc=kan.liang@linux.intel.com \
    --cc=krzysztof.m.lopatowski@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mingo@redhat.com \
    --cc=mpetlan@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=peterz@infradead.org \
    --cc=quic_zhonhan@quicinc.com \
    --cc=retpolanne@posteo.net \
    --cc=sam@gentoo.org \
    --cc=tmricht@linux.ibm.com \
    --cc=yangyicong@hisilicon.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®