From: kan.liang@linux.intel.com
To: peterz@infradead.org, mingo@redhat.com, namhyung@kernel.org,
irogers@google.com, mark.rutland@arm.com,
linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org
Cc: eranian@google.com, ctshao@google.com, tmricht@linux.ibm.com,
Kan Liang <kan.liang@linux.intel.com>, Leo Yan <leo.yan@arm.com>,
Aishwarya TCV <aishwarya.tcv@arm.com>,
Alexei Starovoitov <alexei.starovoitov@gmail.com>,
Venkat Rao Bagalkote <venkat88@linux.ibm.com>
Subject: [PATCH V3] perf: Fix the throttle error of some clock events
Date: Wed, 4 Jun 2025 10:15:54 -0700 [thread overview]
Message-ID: <20250604171554.3909897-1-kan.liang@linux.intel.com> (raw)
From: Kan Liang <kan.liang@linux.intel.com>
Both ARM and IBM CI reports RCU stall, which can be reproduced by the
below perf command.
perf record -a -e cpu-clock -- sleep 2
The issue is introduced by the generic throttle patch set, which
unconditionally invoke the event_stop() when throttle is triggered.
The cpu-clock and task-clock are two special SW events, which rely on
the hrtimer. The throttle is invoked in the hrtimer handler. The
event_stop()->hrtimer_cancel() waits for the handler to finish, which is
a deadlock. Instead of invoking the stop(), the HRTIMER_NORESTART should
be used to stop the timer.
There may be two ways to fix it.
- Introduce a PMU flag to track the case. Avoid the event_stop in
perf_event_throttle() if the flag is detected.
It has been implemented in the
https://lore.kernel.org/lkml/20250528175832.2999139-1-kan.liang@linux.intel.com/
The new flag was thought to be an overkill for the issue.
- Add a check in the event_stop. Return immediately if the throttle is
invoked in the hrtimer handler. Rely on the existing HRTIMER_NORESTART
method to stop the timer.
The latter is implemented here.
Move event->hw.interrupts = MAX_INTERRUPTS before the stop(). It makes
the order the same as perf_event_unthrottle(). Except the patch, no one
checks the hw.interrupts in the stop(). There is no impact from the
order change.
Reported-by: Leo Yan <leo.yan@arm.com>
Reported-by: Aishwarya TCV <aishwarya.tcv@arm.com>
Closes: https://lore.kernel.org/lkml/20250527161656.GJ2566836@e132581.arm.com/
Reported-by: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Closes: https://lore.kernel.org/lkml/djxlh5fx326gcenwrr52ry3pk4wxmugu4jccdjysza7tlc5fef@ktp4rffawgcw/
Reported-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com>
Closes: https://lore.kernel.org/lkml/8e8f51d8-af64-4d9e-934b-c0ee9f131293@linux.ibm.com/
Signed-off-by: Kan Liang <kan.liang@linux.intel.com>
---
Changes since V2:
- Apply a different way to fix the issue.
Remove all Tested-by since a different way is applied
- Update the change log
- Add more Reported-by
kernel/events/core.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/kernel/events/core.c b/kernel/events/core.c
index f34c99f8ce8f..46441c23475d 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -2656,8 +2656,8 @@ static void perf_event_unthrottle(struct perf_event *event, bool start)
static void perf_event_throttle(struct perf_event *event)
{
- event->pmu->stop(event, 0);
event->hw.interrupts = MAX_INTERRUPTS;
+ event->pmu->stop(event, 0);
if (event == event->group_leader)
perf_log_throttle(event, 0);
}
@@ -11749,7 +11749,12 @@ static void perf_swevent_cancel_hrtimer(struct perf_event *event)
{
struct hw_perf_event *hwc = &event->hw;
- if (is_sampling_event(event)) {
+ /*
+ * The throttle can be triggered in the hrtimer handler.
+ * The HRTIMER_NORESTART should be used to stop the timer,
+ * rather than hrtimer_cancel(). See perf_swevent_hrtimer()
+ */
+ if (is_sampling_event(event) && (hwc->interrupts != MAX_INTERRUPTS)) {
ktime_t remaining = hrtimer_get_remaining(&hwc->hrtimer);
local64_set(&hwc->period_left, ktime_to_ns(remaining));
--
2.38.1
next reply other threads:[~2025-06-04 17:16 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-06-04 17:15 kan.liang [this message]
2025-06-04 20:08 ` Ian Rogers
2025-06-04 23:21 ` Alexei Starovoitov
2025-06-05 13:46 ` Liang, Kan
2025-06-05 16:58 ` Liang, Kan
2025-06-05 18:45 ` Alexei Starovoitov
2025-06-05 20:24 ` Liang, Kan
2025-06-05 20:46 ` Alexei Starovoitov
2025-06-05 23:50 ` Liang, Kan
2025-06-06 0:39 ` Alexei Starovoitov
2025-06-06 13:05 ` Liang, Kan
2025-06-06 17:42 ` Alexei Starovoitov
2025-06-06 18:38 ` Liang, Kan
2025-06-06 18:40 ` Alexei Starovoitov
2025-06-05 6:39 ` Ingo Molnar
2025-06-05 13:51 ` Liang, Kan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250604171554.3909897-1-kan.liang@linux.intel.com \
--to=kan.liang@linux.intel.com \
--cc=aishwarya.tcv@arm.com \
--cc=alexei.starovoitov@gmail.com \
--cc=ctshao@google.com \
--cc=eranian@google.com \
--cc=irogers@google.com \
--cc=leo.yan@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=tmricht@linux.ibm.com \
--cc=venkat88@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®