[RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[SJ Park]

The "beyond page table accessed-bit DAMON extension" project [1]
milestone 2 is for building the basic infrastructure for continued
extension.  After starting it, I studied perf event little and found a
few demons in details.  I therefore spent time on high level design.  As
it started looks like making sense to me, I'd like to share the humble
design first and get feedback from more eyeballs.

Add a blueprint of the milestone 2 as a high level design document.
Describe the API and ABI of data attribute report infrastructure, and an
example use case of it for per-CPUs write-only monitoring.  The API and
ABI will later be extended for more data attributes reporters utilizing
AMD IBS, Intel PEBS, Arm SPEs like h/w features and any possible
DAMON-external reporter kernel components.

Request for comments
====================

This is an RFC for high level design discussion, so no implementation is
provided.  Comments I want to receive include but not limited to below.

Does the idea for perf CPU cycle sampling based access monitoring makes
sense?

Does the design provides clear idea about how it can be extended for the
future extensions that already scoped for milestone 3 (AMD IBS, Intel
PEBS, Arm SPE)?

For the foreseeable extensions (AMD IBS, Intel PEBS, Arm SPE), would the
required change minimum and maintainable?

Next steps
==========

I will start implementing the infrastructure and perf CPU cycle sampling
based access monitoring as described in this patch.  It will go through
the normal development process.  Delivery of it will be the completion
of the milestone 2.  As set on the initial roadmap [1], ETA of the
delivery is LSFMMBPF'27.  No delay is expected so far.

[1] https://lore.kernel.org/all/20260525225208.1179-1-sj@kernel.org/

Signed-off-by: SJ Park <sj@kernel.org>
---
 Documentation/mm/damon/design.rst | 44 +++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst
index a465e34bb1d9c..1e699e3f8a496 100644
--- a/Documentation/mm/damon/design.rst
+++ b/Documentation/mm/damon/design.rst
@@ -433,6 +433,50 @@ monitoring.
 Refer to the :ref:`admin guide <damon_usage_sysfs_probes>` to know how users
 can use the mode.
 
+Data Attributes Reporting
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Infrastructure for receiving attribute information from DAMON-external kernel
+components.
+
+Because DAMON is not hooked in every place but runs in its own worker thread,
+it cannot retrieve some types of data attribute information by itself.  Many of
+those are available to DAMON-external kernel components such as perf event
+callbacks and page fault handler.  For such cases, DAMON provides attributes
+reporting infrastructure.  Using its API, any kernel components can report
+their finding to DAMON.
+
+The API for doing the report is a function, damon_report_attrs().  The function
+receives the attribute information and saves it in DAMON's internal buffer.
+The function is safe to be called in any context including NMI.  The internal
+buffer has a fixed size and keeps only latest reports of the size.  Because one
+buffer size doesn't fit all, multiple report buffers of different size and
+rules can be generated and shared by multiple reporters.  DAMON provides API
+for such management of report buffers.  damon_report_attrs() can specify into
+which buffer the report should be saved.  DAMON user interface (ABI) for doing
+the buffer setup, and enable/disable reports of supported types is also
+supported.
+
+The data attribute probe supports the reports.  The probes can be configured to
+hit for a given memory region depending on the existence of specific types of
+reports that associated with the memory region in given report buffers.
+
+For example, per-CPUs write-only monitoring can be configured using this
+infrastructure like below.
+
+The user sets a report buffer and enable CPU instruction sampling report for
+the buffer.  DAMON will enable perf event counter for CPU cycles sampling.  The
+sampling callback will save the sampling time CPU instruction and the id of the
+CPU that was executing it, into the DAMON report buffer, using
+damon_report_attrs().  DAMON can automatically adjust the perf event sampling
+frequency to minimize old report drops due to the buffer size limit.
+
+The user further sets a DAMON probe.  The probe hits for a given DAMON region
+if a store instruction for a memory in the region that was executed from a
+given CPUs group exists in the report buffer.  As a result, the probe hit
+counts represent relative frequency of writes from a given CPUs group to a
+given memory region.
+
 Dynamic Target Space Updates Handling
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

base-commit: 366c6418699768dba5826d412700a29d562ca039
-- 
2.47.3

Re: [RFC PATCH v0] Docs/mm/damon/design: document data attributes report infra design (beyond page table accessed-bit milestone 2)

[Ravi Jonnalagadda]

Hi SJ,

Thank you for sharing the design early.  Short version: the design makes
sense to me, with one main suggestion.  The first report type should use
the data address the PMU captures at sample time, rather than decode the
instruction later in kdamond.  The measurements behind it are in my
reply to your answer to sashiko.

On Sat, 17 Oct 2026 12:57:45 -0700 SJ Park <sj@kernel.org> wrote:

> Does the idea for perf CPU cycle sampling based access monitoring makes
> sense?

On AMD, yes, with precise_ip (cpu-cycles:p routes to IBS Op): its
memory-op samples carry a data address and load/store type.  On Intel,
even precise cycles samples (cycles:pp) carry no data address; PEBS
memory events do.  Decode from saved registers worked for one of two
test loops on Intel and for neither on AMD (details in the other reply),
hence the suggestion above.

> Does the design provides clear idea about how it can be extended for the
> future extensions that already scoped for milestone 3 (AMD IBS, Intel
> PEBS, Arm SPE)?

Yes, if the report has an address field with a validity bit and carries
the producing report-type (attr) id.  Then IBS, PEBS, Arm SPE (through
an adapter, since SPE uses the perf AUX buffer) and device-side
monitors all produce the same report, into the same queues and probes.

> For the foreseeable extensions (AMD IBS, Intel PEBS, Arm SPE), would the
> required change minimum and maintainable?

Yes, if staged.  The sample-filter matcher and its sysfs already exist
per context; attaching them per probe is the main change.  For example:

  M2.1  in-kernel only: damon_report_attrs(), the queue, region
        lookup, the perf producer; kunit, no new sysfs
  M2.2  your per-CPU write-only example, set up from user space:
        attr_reports files, per-filter report_queue,
        write/cpumask/threads probe filters
  M2.3  counters, a silence check (tracepoint/debugfs first), tuning
  M3    more producers: Arm SPE, device monitors

Implementation notes:

1. One sampler per CPU per attr, owned by attr_reports (as in your
   damon/next usage doc) and shared by all contexts.  IBS has one unit
   per CPU, so perf time-slices multiple events: on EPYC 9555P one
   ibs_op event got 67,140 samples; two on the same CPU got
   33,547 + 33,535.  On Intel the event also needs wakeup_events = 1,
   or an in-kernel handler sees only the last record of each
   large-PEBS batch.

2. A lockless queue, as your NOTE in damon_report_access() anticipates.
   Today's buffer is documented "May sleep" and uses mutex_trylock(),
   which interrupt context cannot use; its inclusive jiffy staleness
   check can also count one report in up to three intervals (HZ=250,
   5 ms interval).  A simple design: a per-CPU circular array with a
   write counter and a per-slot seqcount_t (using
   raw_write_seqcount_begin/end() for NMI safety), overwriting the
   oldest slot.  Readers never consume; each context keeps its
   last-seen write counter per CPU.  A nested writer (e.g. an NMI over
   a task-context write) drops its report and counts it.

3. Report fields: paddr, vaddr, size, time, cpu, tid, tgid, the
   producer's attr id, and flags (is_write, plus a validity bit per
   address, since 0 is a valid paddr).  Store global ids; translate
   namespaces at the filter.

4. Binary search over a sorted region snapshot (your todo in
   kdamond_apply_access_report()) beats a linear walk from about 2
   reports per interval for 1,000 regions (a model, to be measured).

5. Samplers can stop silently: in a report on the v4 thread [1], IBS
   Op samples stopped after about 10 minutes on a Ryzen 5 5600H, but
   not with the NMI watchdog enabled.  So count dropped reports and
   check for silence against an independent reference.  For tuning,
   step sample_period rather than use frequency mode, and do not raise
   perf_event_max_sample_rate: on an early DAMON PEBS prototype (Xeon
   6960P), perf cut it from 100000 to 10000 in about 20 seconds.

Three decisions would unblock M2.1:

  - The first report type: the PMU-captured address (suggested), or
    decode from saved registers in kdamond (your current plan).
  - Whether the report carries the producer's attr id (suggested: yes).
  - The queue: the per-CPU seqcount design in note 2 (suggested), or
    ftrace's ring buffer with one iterator per reader.

One question on the ABI: the usage document in your damon/next tree adds
a report_queue file under each probe filter.  Is it meant as "this
filter tests reports in queue N"?

Happy to write, review or test any piece: the queue with kunit, the
AMD/Intel perf producer, a decode-accuracy study on real workloads, or
runs on Xeon 6960P and EPYC 9555P.  Several exist in RFC v4 [2].

Thanks,
Ravi

[1] https://lore.kernel.org/r/20261017211613.16456-2-aethernet65535@gmail.com
[2] https://lore.kernel.org/r/20261005-damon-perf-rfc-v3-send-2026-10-03-v4-0-b03452e137f3@gmail.com