[PATCH 0/5] locking/rtmutex: priority inheritance fixes and tests

[PATCH 0/5] locking/rtmutex: priority inheritance fixes and tests

[Mathieu Desnoyers]

Hi Peter, Thomas,

Three fixes to rt_mutex priority inheritance on v7.4-rc1, and two tests
that catch them. They came out of v4 of "sched/deadline: bandwidth
accounting fixes and selftests", posted alongside, which applies on top
of this series: its patch 8 takes a deadline boost from the waiter's
rt_mutex node, which patch 3 here keeps current.

The three fixes, by patch number:

 1  the top waiter spins on a running owner past its own wakeups:
    behind the owner of a PI futex spinning in userspace, a 1ms timed
    lock returns up to 1.4s late and a signal waits as long, and with
    PREEMPT_RT two ww_mutex contexts in an ABBA spin on each other. The
    spin stops when the waiter is woken.
 2  with PREEMPT_RT, the chain walk reads the ww_mutex context that
    rt_mutex_init_waiter() leaves uninitialized, the debug poison on the
    waiters of PI futexes, and reports neither a deadlock on PI futexes
    nor a chain deeper than max_lock_depth. The context is initialized
    with the rest of the waiter.
 3  a waiter whose priority changes while another is ahead of it enters
    its owner's pi_waiters with stale sort keys once promoted, and the
    owner inherits from the wrong waiter. The keys are copied as it
    enters.

Patch 4 adds futex_lock_pi_inherit to the futex selftests. It checks the
priority an owner runs at as its waiters block, time out, get a signal,
change their priority or are promoted to the top of their lock, along
chains of owners; that FUTEX_LOCK_PI refuses a lock cycle, and a chain
deeper than max_lock_depth, with EDEADLK; and how late a timeout or a
signal ends a wait behind an owner that runs. Patch 5 adds to
test-ww_mutex an ABBA with both owners running. v7.4-rc1 fails the cases
of each fix, and the series passes them, with and without PREEMPT_RT,
along with the locking API selftests, the ww_mutex selftest,
locktorture's rtmutex_lock and the futex selftests. Each message has its
numbers.

These patches, their messages and the tests were written by an LLM
coding assistant under my direction and review. Patch 1 came out of a
stress test of the deadline boost, patch 3 out of a review of the
deadline fix by a second model, and patch 2 out of futex_lock_pi_inherit
on its first run with PREEMPT_RT. Each patch went through a local run of
Sashiko AI before posting, which changed the two tests only.

Based on v7.4-rc1, 6628e7d4c6b2.

Mathieu Desnoyers (5):
  locking/rtmutex: Stop spinning on the owner when the waiter is woken
  locking/rtmutex: Initialize the ww_mutex context of a waiter
  locking/rtmutex: Clone a waiter's sort keys as it enters pi_waiters
  selftests/futex: Test priority inheritance on PI futexes
  locking/ww_mutex/test: Time an ABBA with both owners running

 kernel/locking/rtmutex.c                      |  15 +-
 kernel/locking/rtmutex_common.h               |   1 +
 kernel/locking/test-ww_mutex.c                | 213 +++++
 .../selftests/futex/functional/.gitignore     |   1 +
 .../selftests/futex/functional/Makefile       |   3 +-
 .../futex/functional/futex_lock_pi_inherit.c  | 744 ++++++++++++++++++
 .../testing/selftests/futex/functional/run.sh |   3 +
 7 files changed, 976 insertions(+), 4 deletions(-)
 create mode 100644 tools/testing/selftests/futex/functional/futex_lock_pi_inherit.c


base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
-- 
2.47.3

[PATCH 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken

[Mathieu Desnoyers]

The top waiter of an rt_mutex spins while the owner runs,
rtmutex_spin_on_owner(), and does not stop when it is woken: its timeout
and a signal find it running, set it TASK_RUNNING, and the spin goes on
until the owner leaves its CPU or the waiter's own CPU has to
reschedule. A PI futex owner is a user task that may run as long as it
likes. On v7.4-rc1, in 8-CPU VMs, behind an owner spinning in userspace,
a 1ms pthread_mutex_timedlock() returns ETIMEDOUT 126ms to 631ms late on
average, 1.4s at most, a SIGUSR1 is handled up to 1.9s late, and a
waiter killed with SIGKILL is reaped 0.5s late on average. With
PREEMPT_RT the wound of a ww_mutex is missed the same way: two contexts
in an ABBA on two CPUs at SCHED_FIFO spin on each other, and the older
one gets its second lock after 96ms on average, 0.95s at most.

Stop spinning once the waiter is woken, where the sleep the spin stands
in for would end: rt_mutex_slowlock_block() then sees the timeout, the
signal or the wound, as after schedule(). An rtlock waiter is woken out
of TASK_RTLOCK_WAIT by the lock handover alone, and spins as before.

With this patch the 1ms waits end after 1.01ms on average, a signal is
handled within 0.3ms (1.4ms with PREEMPT_RT), and the ww_mutex ABBA
resolves in 12us to 33us; a contended PI mutex keeps its throughput.
The futex_lock_pi_inherit and test-ww_mutex cases added later in this
series catch both.

Fixes: 992caf7f1724 ("locking/rtmutex: Add adaptive spinwait mechanism")
Cc: stable@vger.kernel.org
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
 kernel/locking/rtmutex.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c
index 5a9534c715b8..323973be2074 100644
--- a/kernel/locking/rtmutex.c
+++ b/kernel/locking/rtmutex.c
@@ -1516,9 +1516,13 @@ static bool rtmutex_spin_on_owner(struct rt_mutex_base *lock,
 		 *  - current is not longer the top waiter
 		 *  - current is requested to reschedule (redundant
 		 *    for CONFIG_PREEMPT_RCU=y)
+		 *  - current is woken, by its timeout, a signal or
+		 *    whatever else would end the sleep the spin
+		 *    stands in for
 		 *  - the VCPU on which owner runs is preempted
 		 */
 		if (!owner_on_cpu(owner) || need_resched() ||
+		    task_is_running(current) ||
 		    !data_race(rt_mutex_waiter_is_top_waiter(lock, waiter))) {
 			res = false;
 			break;
-- 
2.47.3

[PATCH 2/5] locking/rtmutex: Initialize the ww_mutex context of a waiter

[Mathieu Desnoyers]

With PREEMPT_RT no deadlock on PI futexes is reported: two threads that
each block on a PI futex the other holds sleep for good, where
FUTEX_LOCK_PI returns EDEADLK without PREEMPT_RT, and a chain deeper
than max_lock_depth is admitted.

rt_mutex_init_waiter() leaves waiter->ww_ctx as it finds it, and only
__rt_mutex_slowlock_locked() sets it. The waiters of PI futexes and of
rtlocks keep the 0x11 poison of CONFIG_DEBUG_RT_MUTEXES, or what the
stack held unless the compiler zeroes it. Since
commit 6467822b8cc9 ("locking/rtmutex: Prevent spurious EDEADLK return
caused by ww_mutexes"), the PREEMPT_RT chain walk takes a waiter with a
context for a ww_mutex one: it turns deadlock detection off and drops
the -EDEADLK of a cycle. The ww_mutex build of the walk would also
dereference the poison, in __waiter_less(), on two such waiters of equal
priority on a lock it goes through.

Clear the context with the rest of the waiter. In 8-CPU VMs with
PREEMPT_RT and CONFIG_DEBUG_RT_MUTEXES, v7.4-rc1 lets the ABBA time out
and admits the deep chain, in 2 VMs of 2; with this patch both get
EDEADLK. The futex_lock_pi_inherit selftest added later in this series
has the two cases, deadlock and max_lock_depth, and found this.

Fixes: 6467822b8cc9 ("locking/rtmutex: Prevent spurious EDEADLK return caused by ww_mutexes")
Cc: stable@vger.kernel.org
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
 kernel/locking/rtmutex_common.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/kernel/locking/rtmutex_common.h b/kernel/locking/rtmutex_common.h
index c38b7bdea7b3..1f56d40c3e66 100644
--- a/kernel/locking/rtmutex_common.h
+++ b/kernel/locking/rtmutex_common.h
@@ -218,6 +218,7 @@ static inline void rt_mutex_init_waiter(struct rt_mutex_waiter *waiter)
 	RB_CLEAR_NODE(&waiter->tree.entry);
 	waiter->wake_state = TASK_NORMAL;
 	waiter->task = NULL;
+	waiter->ww_ctx = NULL;
 }
 
 static inline void rt_mutex_init_rtlock_waiter(struct rt_mutex_waiter *waiter)
-- 
2.47.3

[PATCH 3/5] locking/rtmutex: Clone a waiter's sort keys as it enters pi_waiters

[Mathieu Desnoyers]

An rt_mutex owner can inherit the priority of the wrong waiter. A waiter
has two copies of its sort keys: the lock tree's, updated when its
priority changes, and the pi_tree's, which orders its owner's
pi_waiters, cloned from the first when the waiter blocks and by the
chain walk while it is the top waiter of its lock. A waiter whose
priority changes while another is ahead of it, and that becomes the top
waiter later, as the one ahead times out (remove_waiter()) or the lock
changes hands (try_to_take_rt_mutex()), enters pi_waiters with keys it
no longer has. On v7.4-rc1, an owner with a SCHED_FIFO 50 waiter on one
mutex, and on another a SCHED_FIFO 70 waiter set to SCHED_FIFO 10 behind
one of 80 that times out, runs at SCHED_FIFO 10 while the SCHED_FIFO 50
task waits on it, in 3 runs of 3.

Clone the keys in rt_mutex_enqueue_pi(), under the wait_lock that the
lock tree copy is updated under and the owner's pi_lock that the
pi_tree copy is read under, and drop the clones its callers did. The
owner then runs at SCHED_FIFO 50, in 3 runs of 3. The
futex_lock_pi_inherit selftest added later in this series has the case,
promoted_waiter.

Found reviewing "sched/deadline: Take a deadline boost from the
waiter's rt_mutex node", posted in a series on top of this one, which
takes a deadline boost from these keys.

Fixes: f7853c342418 ("locking/rtmutex: Fix task->pi_waiters integrity")
Cc: stable@vger.kernel.org
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
 kernel/locking/rtmutex.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c
index 323973be2074..e89f0f0e604b 100644
--- a/kernel/locking/rtmutex.c
+++ b/kernel/locking/rtmutex.c
@@ -512,11 +512,19 @@ static __always_inline bool __pi_waiter_less(struct rb_node *a, const struct rb_
 	return rt_waiter_node_less(__node_2_rt_node(a), __node_2_rt_node(b));
 }
 
+/*
+ * Enqueue @waiter in the pi_waiters of @task, its lock's owner, with the
+ * sort keys of its lock tree copy, current under the wait_lock held here:
+ * the chain walk updates the tree copy of a waiter that is not the top
+ * waiter of its lock, and does not clone it, so a copy cloned before would
+ * enter @task's pi_waiters with the keys of a priority it no longer has.
+ */
 static __always_inline void
 rt_mutex_enqueue_pi(struct task_struct *task, struct rt_mutex_waiter *waiter)
 {
 	lockdep_assert_held(&task->pi_lock);
 
+	waiter_clone_prio(waiter, task);
 	rb_add_cached(&waiter->pi_tree.entry, &task->pi_waiters, __pi_waiter_less);
 }
 
@@ -1005,7 +1013,6 @@ static int __sched rt_mutex_adjust_prio_chain(struct task_struct *task,
 		 * and adjust the priority of the owner.
 		 */
 		rt_mutex_dequeue_pi(task, prerequeue_top_waiter);
-		waiter_clone_prio(waiter, task);
 		rt_mutex_enqueue_pi(task, waiter);
 		rt_mutex_adjust_prio(lock, task);
 
@@ -1022,7 +1029,6 @@ static int __sched rt_mutex_adjust_prio_chain(struct task_struct *task,
 		 */
 		rt_mutex_dequeue_pi(task, waiter);
 		waiter = rt_mutex_top_waiter(lock);
-		waiter_clone_prio(waiter, task);
 		rt_mutex_enqueue_pi(task, waiter);
 		rt_mutex_adjust_prio(lock, task);
 	} else {
@@ -1243,7 +1249,6 @@ static int __sched task_blocks_on_rt_mutex(struct rt_mutex_base *lock,
 	waiter->task = task;
 	waiter->lock = lock;
 	waiter_update_prio(waiter, task);
-	waiter_clone_prio(waiter, task);
 
 	/* Get the top priority waiter on the lock */
 	if (rt_mutex_has_waiters(lock))
-- 
2.47.3

[PATCH 4/5] selftests/futex: Test priority inheritance on PI futexes

[Mathieu Desnoyers]

The PI futex selftests check what FUTEX_LOCK_PI returns, behind owners
asleep: none checks the priority an owner runs at, nor waits behind an
owner that runs. Add futex_lock_pi_inherit. Its nine cases read from
/proc the priority each owner runs at as waiters block, time out, get a
signal or change their priority, along chains and as a waiter is
promoted to the top of its lock; check that a lock cycle, and a chain
one deeper than max_lock_depth, are refused with EDEADLK; and time a 1ms
timeout and a signal behind an owner spinning in userspace, 5ms late at
most on average. The priority cases need CAP_SYS_NICE and skip without
it.

On v7.4-rc1, in 8-CPU VMs, it fails promoted_waiter,
timeout_owner_running and signal_owner_running, in 2 VMs of 2; with
PREEMPT_RT and CONFIG_DEBUG_RT_MUTEXES, deadlock and max_lock_depth too,
and signal_owner_running in 1 VM of 2, the signals of the other handled
1.4ms late on average. Each case passes from the fix it covers, earlier
in this series, on, and the series passes the nine, with and without
PREEMPT_RT.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
 .../selftests/futex/functional/.gitignore     |   1 +
 .../selftests/futex/functional/Makefile       |   3 +-
 .../futex/functional/futex_lock_pi_inherit.c  | 744 ++++++++++++++++++
 .../testing/selftests/futex/functional/run.sh |   3 +
 4 files changed, 750 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/futex/functional/futex_lock_pi_inherit.c

diff --git a/tools/testing/selftests/futex/functional/.gitignore b/tools/testing/selftests/futex/functional/.gitignore
index 7c39d10b38e4..fb5bfd1387da 100644
--- a/tools/testing/selftests/futex/functional/.gitignore
+++ b/tools/testing/selftests/futex/functional/.gitignore
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
 futex_lock_pi_exiting
+futex_lock_pi_inherit
 futex_numa_mpol
 futex_priv_hash
 futex_requeue
diff --git a/tools/testing/selftests/futex/functional/Makefile b/tools/testing/selftests/futex/functional/Makefile
index a03bd5acba50..f32a05fc46d6 100644
--- a/tools/testing/selftests/futex/functional/Makefile
+++ b/tools/testing/selftests/futex/functional/Makefile
@@ -28,7 +28,8 @@ TEST_GEN_PROGS := \
 	futex_waitv \
 	futex_numa \
 	robust_list \
-	futex_lock_pi_exiting
+	futex_lock_pi_exiting \
+	futex_lock_pi_inherit
 
 TEST_PROGS := run.sh
 
diff --git a/tools/testing/selftests/futex/functional/futex_lock_pi_inherit.c b/tools/testing/selftests/futex/functional/futex_lock_pi_inherit.c
new file mode 100644
index 000000000000..4f2a597980f5
--- /dev/null
+++ b/tools/testing/selftests/futex/functional/futex_lock_pi_inherit.c
@@ -0,0 +1,744 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * futex_lock_pi_inherit.c
+ *
+ * Priority inheritance on PI futexes as the scheduler applies it, the
+ * deadlocks FUTEX_LOCK_PI refuses, and waits on a PI futex whose owner
+ * keeps running.
+ *
+ * The other PI futex tests check what FUTEX_LOCK_PI returns. These check
+ * the priority each owner runs at, read from /proc (prio - 100: 20 for
+ * SCHED_OTHER at nice 0, -1 - n for SCHED_FIFO n), after a waiter blocks,
+ * times out, gets a signal or changes its priority, along a chain of owners
+ * blocked in turn, and when a waiter is promoted to the top of its lock;
+ * that a lock cycle, and a chain deeper than max_lock_depth, are refused
+ * with EDEADLK; and, behind an owner spinning in userspace, which has its
+ * top waiter spin in the kernel, that the waiter's timeout, or a signal to
+ * it, still ends its wait in time.
+ *
+ * The priority tests need CAP_SYS_NICE for SCHED_FIFO and skip without it;
+ * the depth test builds a chain one deeper than max_lock_depth, 1026
+ * threads by default; the running-owner tests need two CPUs, three for the
+ * signal.
+ */
+
+#define _GNU_SOURCE
+
+#include <errno.h>
+#include <fcntl.h>
+#include <pthread.h>
+#include <sched.h>
+#include <signal.h>
+#include <stdatomic.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <unistd.h>
+#include <sys/syscall.h>
+
+#include "futextest.h"
+#include "kselftest_harness.h"
+
+#define NSEC_PER_USEC	1000ULL
+#define NSEC_PER_MSEC	1000000ULL
+#define NSEC_PER_SEC	1000000000ULL
+
+#define PRIO_OTHER	20
+#define PRIO_FIFO(n)	(-1 - (n))
+
+static pid_t sys_gettid(void)
+{
+	return syscall(SYS_gettid);
+}
+
+static uint64_t now_ns(void)
+{
+	struct timespec ts;
+
+	clock_gettime(CLOCK_MONOTONIC, &ts);
+	return ts.tv_sec * NSEC_PER_SEC + ts.tv_nsec;
+}
+
+/* An absolute CLOCK_REALTIME timeout @ns from now, as FUTEX_LOCK_PI takes. */
+static struct timespec abs_timeout(uint64_t ns)
+{
+	struct timespec ts;
+
+	clock_gettime(CLOCK_REALTIME, &ts);
+	ts.tv_sec += ns / NSEC_PER_SEC;
+	ts.tv_nsec += ns % NSEC_PER_SEC;
+	if (ts.tv_nsec >= (long)NSEC_PER_SEC) {
+		ts.tv_nsec -= NSEC_PER_SEC;
+		ts.tv_sec++;
+	}
+	return ts;
+}
+
+/* Field @field (1-based, as in proc(5)) of the stat line @buf, or -1000. */
+static int stat_field(char *buf, int field, char *state)
+{
+	char *p;
+	int i;
+
+	p = strrchr(buf, ')');
+	if (!p || !p[1] || !p[2])
+		return -1000;
+	if (state)
+		*state = p[2];
+	for (i = 2; p && i < field; i++)
+		p = strchr(p + 1, ' ');
+	return p ? atoi(p + 1) : -1000;
+}
+
+/* Field @field of /proc/self/task/@tid/stat, or -1000. */
+static int task_stat(pid_t tid, int field, char *state)
+{
+	char path[64], buf[1024];
+	ssize_t n;
+	int fd;
+
+	snprintf(path, sizeof(path), "/proc/self/task/%d/stat", tid);
+	fd = open(path, O_RDONLY);
+	if (fd < 0)
+		return -1000;
+	n = read(fd, buf, sizeof(buf) - 1);
+	close(fd);
+	if (n <= 0)
+		return -1000;
+	buf[n] = '\0';
+	return stat_field(buf, field, state);
+}
+
+/* The priority @tid runs at. */
+static int task_prio(pid_t tid)
+{
+	return task_stat(tid, 18, NULL);
+}
+
+/* Wait up to 2s for @tid to run at @prio; the last priority read. */
+static int wait_prio(pid_t tid, int prio)
+{
+	uint64_t end = now_ns() + 2 * NSEC_PER_SEC;
+	int p;
+
+	while ((p = task_prio(tid)) != prio && now_ns() < end)
+		usleep(1000);
+	return p;
+}
+
+static int set_fifo(pid_t tid, int prio)
+{
+	struct sched_param sp = { .sched_priority = prio };
+
+	return prio ? sched_setscheduler(tid, SCHED_FIFO, &sp) :
+		      sched_setscheduler(tid, SCHED_OTHER, &sp);
+}
+
+/* SCHED_FIFO up to 80, the highest priority the tests use. */
+static bool can_fifo(void)
+{
+	if (set_fifo(0, 80))
+		return false;
+	set_fifo(0, 0);
+	return true;
+}
+
+/*
+ * A thread of the tests: it takes the locks of @hold, then blocks on @wait
+ * (none: it sleeps until released), with a timeout of @timeout_ns if not 0.
+ * @blocking is set just before it blocks, @tid once it runs.
+ */
+struct pi_thread {
+	pthread_t thread;
+	futex_t *hold[2];
+	futex_t *wait;
+	uint64_t timeout_ns;
+	int fifo;
+	atomic_int tid;
+	atomic_int blocking;
+	atomic_int done;
+	int ret;		/* 0 or the errno of the lock of @wait */
+	int err;		/* 0 or the errno of the first call that failed */
+};
+
+static futex_t release_word;
+
+static void release_all(void)
+{
+	release_word = 1;
+	futex_wake(&release_word, INT32_MAX, FUTEX_PRIVATE_FLAG);
+}
+
+/* Record the first call of @t that failed, with errno @err. */
+static void thread_fail(struct pi_thread *t, int err)
+{
+	if (!t->err)
+		t->err = err ? err : EINVAL;
+}
+
+static void *pi_thread_fn(void *arg)
+{
+	struct pi_thread *t = arg;
+	int held = 0;
+
+	atomic_store(&t->tid, sys_gettid());
+	if (t->fifo && set_fifo(0, t->fifo)) {
+		thread_fail(t, errno);
+		goto out;
+	}
+	for (; held < 2 && t->hold[held]; held++) {
+		if (futex_lock_pi(t->hold[held], NULL, 0, FUTEX_PRIVATE_FLAG)) {
+			thread_fail(t, errno);
+			goto unlock;
+		}
+	}
+	atomic_store_explicit(&t->blocking, 1, memory_order_release);
+	if (t->wait) {
+		struct timespec ts = abs_timeout(t->timeout_ns);
+
+		t->ret = futex_lock_pi(t->wait, t->timeout_ns ? &ts : NULL, 0,
+				       FUTEX_PRIVATE_FLAG) ? errno : 0;
+		if (!t->ret && futex_unlock_pi(t->wait, FUTEX_PRIVATE_FLAG))
+			thread_fail(t, errno);
+	} else {
+		while (!release_word) {
+			if (futex_wait(&release_word, 0, NULL, FUTEX_PRIVATE_FLAG) &&
+			    errno != EAGAIN && errno != EINTR) {
+				thread_fail(t, errno);
+				break;
+			}
+		}
+	}
+unlock:
+	while (held--)
+		if (futex_unlock_pi(t->hold[held], FUTEX_PRIVATE_FLAG))
+			thread_fail(t, errno);
+out:
+	atomic_store_explicit(&t->done, 1, memory_order_release);
+	return NULL;
+}
+
+/*
+ * Start @t and wait until it is asleep where it blocks; false if it could
+ * not be created, or did not block within 2s (it is then still to join).
+ */
+static bool start(struct pi_thread *t)
+{
+	uint64_t end = now_ns() + 2 * NSEC_PER_SEC;
+	pthread_attr_t attr;
+	char state = 0;
+	int err;
+
+	pthread_attr_init(&attr);
+	pthread_attr_setstacksize(&attr, 64 * 1024);
+	err = pthread_create(&t->thread, &attr, pi_thread_fn, t);
+	pthread_attr_destroy(&attr);
+	if (err) {
+		t->thread = pthread_self();
+		t->err = err;
+		atomic_store(&t->done, 1);
+		ksft_print_msg("pthread_create: %s\n", strerror(err));
+		return false;
+	}
+	while (now_ns() < end) {
+		if (atomic_load_explicit(&t->done, memory_order_acquire)) {
+			ksft_print_msg("a thread failed before blocking: %s\n",
+				       strerror(t->err ? t->err : t->ret));
+			return false;
+		}
+		if (atomic_load_explicit(&t->blocking, memory_order_acquire) &&
+		    task_stat(atomic_load(&t->tid), 3, &state) != -1000 && state == 'S')
+			return true;
+		usleep(1000);
+	}
+	ksft_print_msg("a thread did not block within 2s\n");
+	return false;
+}
+
+static void pi_thread_init(struct pi_thread *t, futex_t *hold0, futex_t *hold1,
+			   futex_t *wait, int fifo, uint64_t timeout_ns)
+{
+	memset(t, 0, sizeof(*t));
+	t->hold[0] = hold0;
+	t->hold[1] = hold1;
+	t->wait = wait;
+	t->fifo = fifo;
+	t->timeout_ns = timeout_ns;
+}
+
+/* Join @t; the error of pthread_join(), or of the first call of @t that failed, or 0. */
+static int join(struct pi_thread *t)
+{
+	int err = 0;
+
+	if (!pthread_equal(t->thread, pthread_self()))
+		err = pthread_join(t->thread, NULL);
+	return err ? err : t->err;
+}
+
+/* A SCHED_FIFO 50 waiter that times out boosts the owner, then no longer. */
+TEST(boost_timeout)
+{
+	futex_t l = FUTEX_INITIALIZER;
+	struct pi_thread o, w;
+
+	if (!can_fifo())
+		SKIP(return, "SCHED_FIFO not permitted: %s", strerror(errno));
+	release_word = 0;
+	pi_thread_init(&o, &l, NULL, NULL, 0, 0);
+	ASSERT_TRUE(start(&o));
+	pi_thread_init(&w, NULL, NULL, &l, 50, NSEC_PER_SEC);
+	ASSERT_TRUE(start(&w));
+	EXPECT_EQ(wait_prio(o.tid, PRIO_FIFO(50)), PRIO_FIFO(50));
+	EXPECT_EQ(join(&w), 0);
+	EXPECT_EQ(w.ret, ETIMEDOUT);
+	EXPECT_EQ(wait_prio(o.tid, PRIO_OTHER), PRIO_OTHER);
+	release_all();
+	EXPECT_EQ(join(&o), 0);
+}
+
+/* The owner follows a blocked waiter as it changes its priority. */
+TEST(waiter_priority_change)
+{
+	futex_t l = FUTEX_INITIALIZER;
+	struct pi_thread o, w;
+
+	if (!can_fifo())
+		SKIP(return, "SCHED_FIFO not permitted: %s", strerror(errno));
+	release_word = 0;
+	pi_thread_init(&o, &l, NULL, NULL, 0, 0);
+	ASSERT_TRUE(start(&o));
+	pi_thread_init(&w, NULL, NULL, &l, 50, 0);
+	ASSERT_TRUE(start(&w));
+	EXPECT_EQ(wait_prio(o.tid, PRIO_FIFO(50)), PRIO_FIFO(50));
+	ASSERT_EQ(set_fifo(w.tid, 70), 0);
+	EXPECT_EQ(task_prio(o.tid), PRIO_FIFO(70));
+	ASSERT_EQ(set_fifo(w.tid, 30), 0);
+	EXPECT_EQ(task_prio(o.tid), PRIO_FIFO(30));
+	ASSERT_EQ(set_fifo(w.tid, 0), 0);
+	EXPECT_EQ(task_prio(o.tid), PRIO_OTHER);
+	release_all();
+	EXPECT_EQ(join(&o), 0);
+	EXPECT_EQ(join(&w), 0);
+	EXPECT_EQ(w.ret, 0);
+}
+
+/*
+ * A waiter whose priority changes while another waiter is ahead of it on
+ * its lock, and that becomes the top waiter of that lock when the one
+ * ahead times out, is the one the owner inherits from, at its new
+ * priority: the owner holds L1, with W1 (SCHED_FIFO 80, timed) and behind
+ * it W2 (SCHED_FIFO 70, set to 10), and L2, with W3 (SCHED_FIFO 50). Once
+ * W1 times out, the owner runs at 50.
+ */
+TEST(promoted_waiter)
+{
+	futex_t l1 = FUTEX_INITIALIZER, l2 = FUTEX_INITIALIZER;
+	struct pi_thread o, w1, w2, w3;
+
+	if (!can_fifo())
+		SKIP(return, "SCHED_FIFO not permitted: %s", strerror(errno));
+	release_word = 0;
+	pi_thread_init(&o, &l1, &l2, NULL, 0, 0);
+	ASSERT_TRUE(start(&o));
+	pi_thread_init(&w3, NULL, NULL, &l2, 50, 0);
+	ASSERT_TRUE(start(&w3));
+	pi_thread_init(&w1, NULL, NULL, &l1, 80, NSEC_PER_SEC);
+	ASSERT_TRUE(start(&w1));
+	pi_thread_init(&w2, NULL, NULL, &l1, 70, 0);
+	ASSERT_TRUE(start(&w2));
+	EXPECT_EQ(wait_prio(o.tid, PRIO_FIFO(80)), PRIO_FIFO(80));
+	ASSERT_EQ(set_fifo(w2.tid, 10), 0);
+	EXPECT_EQ(task_prio(o.tid), PRIO_FIFO(80));
+	EXPECT_EQ(join(&w1), 0);
+	EXPECT_EQ(w1.ret, ETIMEDOUT);
+	EXPECT_EQ(wait_prio(o.tid, PRIO_FIFO(50)), PRIO_FIFO(50))
+		TH_LOG("the owner inherits from the wrong waiter");
+	release_all();
+	EXPECT_EQ(join(&o), 0);
+	EXPECT_EQ(join(&w2), 0);
+	EXPECT_EQ(join(&w3), 0);
+}
+
+/* T1's stat as the waiter's signal handler reads it, with open() and read() alone. */
+static char chain_path[64], chain_buf[1024];
+static _Atomic int chain_read = -1;
+
+static void chain_handler(int sig)
+{
+	int fd = open(chain_path, O_RDONLY);
+	ssize_t n = fd < 0 ? 0 : read(fd, chain_buf, sizeof(chain_buf) - 1);
+
+	if (fd >= 0)
+		close(fd);
+	atomic_store_explicit(&chain_read, n > 0 ? n : 0, memory_order_release);
+}
+
+/*
+ * A chain: T1 holds L1; T2 holds L2 and blocks on L1; W (SCHED_FIFO 60)
+ * blocks on L2. T2 and T1 run at 60 while W waits, and at their own
+ * priority again once it times out, or while it handles a signal.
+ */
+static void chain(struct __test_metadata *_metadata, bool timeout)
+{
+	futex_t l1 = FUTEX_INITIALIZER, l2 = FUTEX_INITIALIZER;
+	struct pi_thread t1, t2, w;
+	int n;
+
+	release_word = 0;
+	pi_thread_init(&t1, &l1, NULL, NULL, 0, 0);
+	ASSERT_TRUE(start(&t1));
+	pi_thread_init(&t2, &l2, NULL, &l1, 0, 0);
+	ASSERT_TRUE(start(&t2));
+	pi_thread_init(&w, NULL, NULL, &l2, 60, timeout ? NSEC_PER_SEC : 0);
+	ASSERT_TRUE(start(&w));
+	EXPECT_EQ(wait_prio(t1.tid, PRIO_FIFO(60)), PRIO_FIFO(60));
+	EXPECT_EQ(task_prio(t2.tid), PRIO_FIFO(60));
+	if (timeout) {
+		EXPECT_EQ(join(&w), 0);
+		EXPECT_EQ(w.ret, ETIMEDOUT);
+		EXPECT_EQ(wait_prio(t1.tid, PRIO_OTHER), PRIO_OTHER);
+		EXPECT_EQ(task_prio(t2.tid), PRIO_OTHER);
+	} else {
+		/* T2's own priority above the waiter's, then below it again. */
+		ASSERT_EQ(set_fifo(t2.tid, 70), 0);
+		EXPECT_EQ(task_prio(t1.tid), PRIO_FIFO(70));
+		ASSERT_EQ(set_fifo(t2.tid, 0), 0);
+		EXPECT_EQ(task_prio(t1.tid), PRIO_FIFO(60));
+		/* W out of the chain in its signal handler, back in after it. */
+		snprintf(chain_path, sizeof(chain_path), "/proc/self/task/%d/stat", t1.tid);
+		struct sigaction sa = { .sa_handler = chain_handler };
+		uint64_t end = now_ns() + 2 * NSEC_PER_SEC;
+
+		atomic_store(&chain_read, -1);
+		ASSERT_EQ(sigaction(SIGUSR1, &sa, NULL), 0);
+		ASSERT_EQ(pthread_kill(w.thread, SIGUSR1), 0);
+		while ((n = atomic_load_explicit(&chain_read, memory_order_acquire)) < 0 &&
+		       now_ns() < end)
+			usleep(1000);
+		ASSERT_GE(n, 0) TH_LOG("the signal was not handled within 2s");
+		chain_buf[n] = '\0';
+		EXPECT_EQ(stat_field(chain_buf, 18, NULL), PRIO_OTHER);
+		EXPECT_EQ(wait_prio(t1.tid, PRIO_FIFO(60)), PRIO_FIFO(60));
+		release_all();
+		EXPECT_EQ(join(&w), 0);
+		EXPECT_EQ(w.ret, 0);
+	}
+	release_all();
+	EXPECT_EQ(join(&t1), 0);
+	EXPECT_EQ(join(&t2), 0);
+}
+
+TEST(chain_timeout)
+{
+	if (!can_fifo())
+		SKIP(return, "SCHED_FIFO not permitted: %s", strerror(errno));
+	chain(_metadata, true);
+}
+
+TEST(chain_priority_change_and_signal)
+{
+	if (!can_fifo())
+		SKIP(return, "SCHED_FIFO not permitted: %s", strerror(errno));
+	chain(_metadata, false);
+}
+
+/* Two owners each blocking on the lock the other holds: EDEADLK. */
+TEST(deadlock)
+{
+	futex_t l1 = FUTEX_INITIALIZER, l2 = FUTEX_INITIALIZER;
+	struct pi_thread a;
+	int ret;
+
+	struct timespec ts;
+
+	release_word = 0;
+	pi_thread_init(&a, &l1, NULL, &l2, 0, 2 * NSEC_PER_SEC);
+	ASSERT_EQ(futex_lock_pi(&l2, NULL, 0, FUTEX_PRIVATE_FLAG), 0);
+	ASSERT_TRUE(start(&a));
+	ts = abs_timeout(NSEC_PER_SEC);
+	ret = futex_lock_pi(&l1, &ts, 0, FUTEX_PRIVATE_FLAG) ? errno : 0;
+	if (!ret)
+		EXPECT_EQ(futex_unlock_pi(&l1, FUTEX_PRIVATE_FLAG), 0);
+	EXPECT_EQ(futex_unlock_pi(&l2, FUTEX_PRIVATE_FLAG), 0);
+	EXPECT_EQ(join(&a), 0);
+	EXPECT_EQ(ret, EDEADLK)
+		TH_LOG("the second of an ABBA got %s, the first %s",
+		       strerror(ret), strerror(a.ret));
+	EXPECT_EQ(a.ret, 0);
+}
+
+/*
+ * A chain deeper than max_lock_depth: t[0] holds l[0], and t[i] holds l[i]
+ * and blocks on l[i - 1]. The depth of a chain walk counts the owners it
+ * finds blocked: t[i] blocking walks i - 1 of them, and is admitted up to
+ * t[depth + 1]; this thread, blocking on l[depth + 1], would walk depth + 1,
+ * and is refused with EDEADLK.
+ */
+#define MAX_DEPTH	4096
+static struct pi_thread depth_t[MAX_DEPTH + 2];
+static futex_t depth_l[MAX_DEPTH + 2];
+
+TEST(max_lock_depth)
+{
+	int depth = 0, i, n, blocked, ret = -1;
+	bool created = true;
+	struct timespec ts;
+	char buf[32] = "";
+	ssize_t len;
+	int fd;
+
+	fd = open("/proc/sys/kernel/max_lock_depth", O_RDONLY);
+	len = fd < 0 ? -1 : read(fd, buf, sizeof(buf) - 1);
+	if (fd >= 0)
+		close(fd);
+	if (len > 0)
+		depth = atoi(buf);
+	if (depth <= 0 || depth > MAX_DEPTH)
+		SKIP(return, "max_lock_depth %s not in 1 to %d", buf, MAX_DEPTH);
+	release_word = 0;
+	for (i = 0; i < depth + 2; i++) {
+		depth_l[i] = FUTEX_INITIALIZER;
+		pi_thread_init(&depth_t[i], &depth_l[i], NULL, i ? &depth_l[i - 1] : NULL, 0, 0);
+		if (!start(&depth_t[i]))
+			break;
+	}
+	blocked = n = i;
+	/* A thread that was created but did not block is still to join. */
+	if (i < depth + 2) {
+		if (pthread_equal(depth_t[i].thread, pthread_self()))
+			created = false;
+		else
+			n++;
+	}
+	if (blocked == depth + 2) {
+		ts = abs_timeout(NSEC_PER_SEC);
+		ret = futex_lock_pi(&depth_l[blocked - 1], &ts, 0, FUTEX_PRIVATE_FLAG) ? errno : 0;
+		if (!ret)
+			EXPECT_EQ(futex_unlock_pi(&depth_l[blocked - 1], FUTEX_PRIVATE_FLAG), 0);
+	}
+	release_all();
+	for (i = 0; i < n; i++)
+		EXPECT_EQ(join(&depth_t[i]), 0);
+	if (!created)
+		SKIP(return, "%d threads of the chain could not be created", depth + 2);
+	ASSERT_EQ(blocked, depth + 2)
+		TH_LOG("thread %d of the chain did not block", blocked);
+	EXPECT_EQ(ret, EDEADLK);
+	for (i = 1; i < blocked; i++)
+		EXPECT_EQ(depth_t[i].ret, 0);
+}
+
+/*
+ * The owner of a PI futex spinning in userspace on a CPU, its waiter on
+ * another: the waiter spins in the kernel while the owner runs, and its
+ * timeout, or a signal to it, must still end the wait in time.
+ */
+static atomic_int owner_stop, owner_holds, owner_err;
+static futex_t running_lock;
+
+/* Pin the calling thread on @cpu; 0 or an errno. */
+static int pin(int cpu)
+{
+	cpu_set_t set;
+
+	CPU_ZERO(&set);
+	CPU_SET(cpu, &set);
+	return sched_setaffinity(0, sizeof(set), &set) ? errno : 0;
+}
+
+static void *running_owner(void *arg)
+{
+	int err = pin((long)arg);
+
+	if (!err && futex_lock_pi(&running_lock, NULL, 0, FUTEX_PRIVATE_FLAG))
+		err = errno;
+	if (err) {
+		atomic_store_explicit(&owner_err, err, memory_order_release);
+		return NULL;
+	}
+	atomic_store_explicit(&owner_holds, 1, memory_order_release);
+	while (!atomic_load_explicit(&owner_stop, memory_order_acquire))
+		;
+	if (futex_unlock_pi(&running_lock, FUTEX_PRIVATE_FLAG))
+		atomic_store_explicit(&owner_err, errno, memory_order_release);
+	return NULL;
+}
+
+/* @n CPUs this thread may run on, in @cpu, or false. */
+static bool pick_cpus(int *cpu, int n)
+{
+	cpu_set_t set;
+	int c, i = 0;
+
+	if (sched_getaffinity(0, sizeof(set), &set))
+		return false;
+	for (c = 0; c < CPU_SETSIZE && i < n; c++)
+		if (CPU_ISSET(c, &set))
+			cpu[i++] = c;
+	return i == n;
+}
+
+/* The owner, spinning on @cpu with running_lock held, in @o; false on a failure. */
+static bool start_running_owner(int cpu, pthread_t *o)
+{
+	uint64_t end = now_ns() + 2 * NSEC_PER_SEC;
+	int err;
+
+	running_lock = FUTEX_INITIALIZER;
+	atomic_store(&owner_stop, 0);
+	atomic_store(&owner_holds, 0);
+	atomic_store(&owner_err, 0);
+	err = pthread_create(o, NULL, running_owner, (void *)(long)cpu);
+	if (err) {
+		ksft_print_msg("pthread_create: %s\n", strerror(err));
+		return false;
+	}
+	while (!atomic_load_explicit(&owner_holds, memory_order_acquire) &&
+	       !atomic_load_explicit(&owner_err, memory_order_acquire) && now_ns() < end)
+		usleep(100);
+	if (atomic_load_explicit(&owner_holds, memory_order_acquire))
+		return true;
+	atomic_store_explicit(&owner_stop, 1, memory_order_release);
+	err = pthread_join(*o, NULL);
+	if (err)
+		ksft_print_msg("pthread_join: %s\n", strerror(err));
+	err = atomic_load(&owner_err);
+	ksft_print_msg("the owner did not take the lock: %s\n", err ? strerror(err) : "timeout");
+	return false;
+}
+
+/* Stop the owner; the error of pthread_join(), or of its first call that failed, or 0. */
+static int stop_running_owner(pthread_t o)
+{
+	int err;
+
+	atomic_store_explicit(&owner_stop, 1, memory_order_release);
+	err = pthread_join(o, NULL);
+	return err ? err : atomic_load(&owner_err);
+}
+
+/*
+ * 20 waits with a timeout of 1ms behind a running owner: 5ms late at most
+ * on average. A kernel that ends the spin of the waiter on its wakeup is
+ * tens of microseconds late; one that does not is late until something
+ * else reschedules one of the two CPUs, milliseconds to seconds.
+ */
+TEST(timeout_owner_running)
+{
+	uint64_t late = 0, t;
+	int cpu[2], i, ret;
+	pthread_t o;
+
+	if (!pick_cpus(cpu, 2))
+		SKIP(return, "two CPUs needed");
+	ASSERT_EQ(pin(cpu[1]), 0);
+	ASSERT_TRUE(start_running_owner(cpu[0], &o));
+	for (i = 0; i < 20; i++) {
+		struct timespec ts = abs_timeout(NSEC_PER_MSEC);
+
+		t = now_ns();
+		ret = futex_lock_pi(&running_lock, &ts, 0, FUTEX_PRIVATE_FLAG) ? errno : 0;
+		t = now_ns() - t;
+		EXPECT_EQ(ret, ETIMEDOUT);
+		late += t > NSEC_PER_MSEC ? t - NSEC_PER_MSEC : 0;
+	}
+	EXPECT_EQ(stop_running_owner(o), 0);
+	TH_LOG("a timeout of 1ms ends %llu us late on average",
+	       (unsigned long long)(late / 20 / NSEC_PER_USEC));
+	EXPECT_LT(late / 20, 5 * NSEC_PER_MSEC);
+}
+
+static _Atomic uint64_t handled_ns;
+static atomic_int waiter_tid, waiter_ret;
+
+static void running_handler(int sig)
+{
+	atomic_store(&handled_ns, now_ns());
+}
+
+static void *running_waiter(void *arg)
+{
+	int ret = pin((long)arg);
+
+	atomic_store(&waiter_tid, sys_gettid());
+	if (!ret)
+		ret = futex_lock_pi(&running_lock, NULL, 0, FUTEX_PRIVATE_FLAG) ? errno : 0;
+	if (!ret && futex_unlock_pi(&running_lock, FUTEX_PRIVATE_FLAG))
+		ret = errno;
+	atomic_store(&waiter_ret, ret);
+	return NULL;
+}
+
+/*
+ * Wait up to 2s for the waiter to be asleep on running_lock, as the futex
+ * word and its state say.
+ */
+static bool waiter_blocked(void)
+{
+	uint64_t end = now_ns() + 2 * NSEC_PER_SEC;
+	char state = 0;
+	pid_t tid;
+
+	while (now_ns() < end) {
+		tid = atomic_load(&waiter_tid);
+		if (tid && (running_lock & FUTEX_WAITERS) &&
+		    task_stat(tid, 3, &state) != -1000 && state == 'S')
+			return true;
+		usleep(100);
+	}
+	return false;
+}
+
+/*
+ * 10 signals to a waiter behind a running owner, sent from a third CPU
+ * once the waiter is asleep on the lock: handled within 5ms on average,
+ * as the timeouts above. A signal not handled within 2s counts for 2s.
+ */
+TEST(signal_owner_running)
+{
+	struct sigaction sa = { .sa_handler = running_handler };
+	uint64_t total = 0, t, end;
+	int cpu[3], i, err, n = 0;
+	bool blocked = true;
+	pthread_t o, w;
+
+	if (!pick_cpus(cpu, 3))
+		SKIP(return, "three CPUs needed");
+	ASSERT_EQ(sigaction(SIGUSR1, &sa, NULL), 0);
+	ASSERT_EQ(pin(cpu[2]), 0);
+	ASSERT_TRUE(start_running_owner(cpu[0], &o));
+	atomic_store(&waiter_tid, 0);
+	atomic_store(&waiter_ret, -1);
+	err = pthread_create(&w, NULL, running_waiter, (void *)(long)cpu[1]);
+	if (err) {
+		stop_running_owner(o);
+		ASSERT_EQ(err, 0) TH_LOG("pthread_create: %s", strerror(err));
+	}
+	for (i = 0; i < 10; i++) {
+		if (!waiter_blocked()) {
+			blocked = false;
+			break;
+		}
+		atomic_store(&handled_ns, 0);
+		t = now_ns();
+		EXPECT_EQ(pthread_kill(w, SIGUSR1), 0);
+		end = t + 2 * NSEC_PER_SEC;
+		while (!atomic_load(&handled_ns) && now_ns() < end)
+			usleep(50);
+		total += (atomic_load(&handled_ns) ? atomic_load(&handled_ns) : end) - t;
+		n++;
+	}
+	EXPECT_EQ(stop_running_owner(o), 0);
+	EXPECT_EQ(pthread_join(w, NULL), 0);
+	EXPECT_EQ(atomic_load(&waiter_ret), 0);
+	ASSERT_TRUE(blocked) TH_LOG("the waiter did not block on the lock");
+	TH_LOG("a signal is handled after %llu us on average",
+	       (unsigned long long)(total / n / NSEC_PER_USEC));
+	EXPECT_LT(total / n, 5 * NSEC_PER_MSEC);
+}
+
+TEST_HARNESS_MAIN
diff --git a/tools/testing/selftests/futex/functional/run.sh b/tools/testing/selftests/futex/functional/run.sh
index d1a681b798bd..ed37522af5c3 100755
--- a/tools/testing/selftests/futex/functional/run.sh
+++ b/tools/testing/selftests/futex/functional/run.sh
@@ -54,3 +54,6 @@ echo
 
 echo
 ./futex_lock_pi_exiting
+
+echo
+./futex_lock_pi_inherit
-- 
2.47.3

[PATCH 5/5] locking/ww_mutex/test: Time an ABBA with both owners running

[Mathieu Desnoyers]

The ABBA cases of test-ww_mutex let neither context run while the other
blocks on it. With PREEMPT_RT a ww_mutex waiter spins on a running
owner, and the wound that resolves an ABBA reaches it through a wakeup.
Add a wound-wait case where both contexts run, each on a CPU of its own
at SCHED_FIFO, the younger one blocked on the older one's lock when the
older one blocks on the younger one's: the older context must get its
second lock within 10ms on average over 10 rounds.

On v7.4-rc1 with PREEMPT_RT the case fails every time, 96ms on average,
0.95s at most, in 2 VMs of 2; with "locking/rtmutex: Stop spinning on
the owner when the waiter is woken", earlier in this series, it passes.
Without PREEMPT_RT it passes on both.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
---
 kernel/locking/test-ww_mutex.c | 213 +++++++++++++++++++++++++++++++++
 1 file changed, 213 insertions(+)

diff --git a/kernel/locking/test-ww_mutex.c b/kernel/locking/test-ww_mutex.c
index 838d631544ed..1bf3a91adfec 100644
--- a/kernel/locking/test-ww_mutex.c
+++ b/kernel/locking/test-ww_mutex.c
@@ -6,10 +6,12 @@
 #include <linux/kernel.h>
 
 #include <linux/completion.h>
+#include <linux/cpu.h>
 #include <linux/delay.h>
 #include <linux/kthread.h>
 #include <linux/module.h>
 #include <linux/prandom.h>
+#include <linux/sched/task.h>
 #include <linux/slab.h>
 #include <linux/ww_mutex.h>
 
@@ -279,6 +281,213 @@ static int test_abba(struct ww_class *class, bool trylock, bool resolve)
 	return ret;
 }
 
+/*
+ * ABBA with both contexts running, each on a CPU of its own at SCHED_FIFO:
+ * the younger one, holding B, blocks on A, which the older one holds while
+ * it stays on its CPU; the older one then blocks on B, which wounds the
+ * younger one, and stays on its CPU too. A waiter that spins while its
+ * owner runs sees the wound only if the wakeup that tells it ends the
+ * spin: until something else gets to run on one of the two CPUs, the two
+ * spin on each other. The older context must get B within 10ms on
+ * average, over 10 rounds; it takes microseconds. Wound-wait only: with
+ * wait-die, the younger context backs out as it blocks on the older one.
+ */
+struct test_abba_spin {
+	struct ww_class *class;
+	struct ww_mutex a_mutex, b_mutex;
+	struct completion start, done[2];
+	struct task_struct *young;
+	atomic_t stage;
+	u64 wait_ns;
+	int err[2];
+};
+
+static bool abba_spin_until(struct test_abba_spin *s, int stage)
+{
+	u64 end = ktime_get_ns() + 5ULL * NSEC_PER_SEC;
+
+	while (atomic_read(&s->stage) < stage) {
+		if (ktime_get_ns() > end)
+			return false;
+		cpu_relax();
+	}
+	return true;
+}
+
+/* Spin until the younger context waits on a lock, its state no longer TASK_RUNNING. */
+static bool abba_spin_until_waiting(struct test_abba_spin *s)
+{
+	u64 end = ktime_get_ns() + 5ULL * NSEC_PER_SEC;
+
+	while (task_is_running(s->young)) {
+		if (ktime_get_ns() > end)
+			return false;
+		cpu_relax();
+	}
+	return true;
+}
+
+static int abba_spin_old(void *arg)
+{
+	struct test_abba_spin *s = arg;
+	struct ww_acquire_ctx ctx;
+	u64 t, end;
+	int err = -ETIMEDOUT;
+
+	sched_set_fifo_low(current);
+	wait_for_completion(&s->start);
+	ww_acquire_init_noinject(&ctx, s->class);
+	atomic_set(&s->stage, 1);
+	if (!abba_spin_until(s, 2))
+		goto fini;
+	err = ww_mutex_lock(&s->a_mutex, &ctx);
+	if (err)
+		goto fini;
+	atomic_set(&s->stage, 3);
+	err = -ETIMEDOUT;
+	if (!abba_spin_until(s, 4) || !abba_spin_until_waiting(s))
+		goto unlock;
+	/* The younger one is in its wait on A: let it reach the spin. */
+	end = ktime_get_ns() + NSEC_PER_MSEC;
+	while (ktime_get_ns() < end)
+		cpu_relax();
+	t = ktime_get_ns();
+	err = ww_mutex_lock(&s->b_mutex, &ctx);
+	s->wait_ns = ktime_get_ns() - t;
+	if (!err)
+		ww_mutex_unlock(&s->b_mutex);
+unlock:
+	ww_mutex_unlock(&s->a_mutex);
+fini:
+	ww_acquire_fini(&ctx);
+	s->err[0] = err;
+	complete(&s->done[0]);
+	return 0;
+}
+
+static int abba_spin_young(void *arg)
+{
+	struct test_abba_spin *s = arg;
+	struct ww_acquire_ctx ctx;
+	int err = -ETIMEDOUT;
+
+	sched_set_fifo_low(current);
+	wait_for_completion(&s->start);
+	if (!abba_spin_until(s, 1))
+		goto out;
+	ww_acquire_init_noinject(&ctx, s->class);
+	err = ww_mutex_lock(&s->b_mutex, &ctx);
+	if (err)
+		goto fini;
+	atomic_set(&s->stage, 2);
+	if (!abba_spin_until(s, 3)) {
+		err = -ETIMEDOUT;
+		ww_mutex_unlock(&s->b_mutex);
+		goto fini;
+	}
+	atomic_set(&s->stage, 4);
+	err = ww_mutex_lock(&s->a_mutex, &ctx);
+	if (err == -EDEADLK) {
+		/* Wounded: back out of B, wait for A, then take B again. */
+		ww_mutex_unlock(&s->b_mutex);
+		ww_mutex_lock_slow(&s->a_mutex, &ctx);
+		err = ww_mutex_lock(&s->b_mutex, &ctx);
+		if (err) {
+			ww_mutex_unlock(&s->a_mutex);
+			goto fini;
+		}
+	} else if (err) {
+		ww_mutex_unlock(&s->b_mutex);
+		goto fini;
+	}
+	ww_mutex_unlock(&s->b_mutex);
+	ww_mutex_unlock(&s->a_mutex);
+fini:
+	ww_acquire_fini(&ctx);
+out:
+	s->err[1] = err;
+	complete(&s->done[1]);
+	return 0;
+}
+
+static int test_abba_spin(struct ww_class *class)
+{
+	int (*fn[2])(void *) = { abba_spin_old, abba_spin_young };
+	unsigned int c, cpu[2] = { nr_cpu_ids, nr_cpu_ids };
+	u64 total = 0, longest = 0;
+	struct test_abba_spin s;
+	int i, round, ret = 0;
+
+	if (class->is_wait_die)
+		return 0;
+	cpus_read_lock();
+	/* The last two online CPUs, the ones boot and housekeeping use least. */
+	for_each_online_cpu(c) {
+		cpu[0] = cpu[1];
+		cpu[1] = c;
+	}
+	for (round = 0; cpu[0] < nr_cpu_ids && round < 10 && !ret; round++) {
+		struct task_struct *t[2];
+
+		ww_mutex_init(&s.a_mutex, class);
+		ww_mutex_init(&s.b_mutex, class);
+		init_completion(&s.start);
+		atomic_set(&s.stage, 0);
+		s.class = class;
+		s.wait_ns = 0;
+		for (i = 0; i < 2; i++) {
+			init_completion(&s.done[i]);
+			s.err[i] = -EINVAL;
+		}
+		for (i = 0; i < 2; i++) {
+			t[i] = kthread_create_on_cpu(fn[i], &s, cpu[i], "ww_abba_spin/%u");
+			if (IS_ERR(t[i])) {
+				ret = PTR_ERR(t[i]);
+				if (i)
+					kthread_stop(t[0]);
+				goto unlock;
+			}
+		}
+		s.young = t[1];
+		/* Held, for kthread_stop() to reap a thread that already returned. */
+		for (i = 0; i < 2; i++) {
+			get_task_struct(t[i]);
+			wake_up_process(t[i]);
+		}
+		/* Asleep until both are woken: neither spins ahead of the other. */
+		complete_all(&s.start);
+		for (i = 0; i < 2; i++) {
+			int err;
+
+			wait_for_completion(&s.done[i]);
+			err = kthread_stop(t[i]);
+			put_task_struct(t[i]);
+			if (err && !ret) {
+				pr_err("%s: kthread_stop: %d\n", __func__, err);
+				ret = err;
+			}
+		}
+		if (s.err[0] || s.err[1]) {
+			pr_err("%s: ABBA with both owners running not resolved, err %d and %d\n",
+			       __func__, s.err[0], s.err[1]);
+			ret = -EINVAL;
+		}
+		total += s.wait_ns;
+		longest = max(longest, s.wait_ns);
+	}
+	if (cpu[0] >= nr_cpu_ids)
+		pr_info("%s: skipped, two online CPUs needed\n", __func__);
+	if (!ret && round && total > 10 * NSEC_PER_MSEC * round) {
+		pr_err("%s: ABBA with both owners running resolved in %llu us on average, %llu us at most\n",
+		       __func__, div_u64(total, round * NSEC_PER_USEC),
+		       div_u64(longest, NSEC_PER_USEC));
+		ret = -EINVAL;
+	}
+unlock:
+	cpus_read_unlock();
+	return ret;
+}
+
 struct test_cycle {
 	struct work_struct work;
 	struct ww_class *class;
@@ -664,6 +873,10 @@ static int run_tests(struct ww_class *class)
 			return ret;
 	}
 
+	ret = test_abba_spin(class);
+	if (ret)
+		return ret;
+
 	ret = test_cycle(class, ncpus);
 	if (ret)
 		return ret;
-- 
2.47.3

Re: [PATCH 5/5] locking/ww_mutex/test: Time an ABBA with both owners running

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Low] locking/ww_mutex/test: missing stage ordering in
  test_abba_spin()
--

commit a75841edff4ce8ecc91baad12d6e7f770d7d72f7
Author: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

locking/ww_mutex/test: Time an ABBA with both owners running

This commit adds a wound-wait test case to test-ww_mutex where two
SCHED_FIFO threads on separate CPUs create an ABBA contention while both
owners remain running.

> diff --git a/kernel/locking/test-ww_mutex.c b/kernel/locking/test-ww_mutex.c
> index 838d631544ed2..1bf3a91adfec5 100644
> --- a/kernel/locking/test-ww_mutex.c
> +++ b/kernel/locking/test-ww_mutex.c
[ ... ]
> @@ -279,6 +281,213 @@ static int test_abba(struct ww_class *class, bool trylock, bool resolve)
>  	return ret;
>  }
>  
[ ... ]
> +static bool abba_spin_until(struct test_abba_spin *s, int stage)
> +{
> +	u64 end = ktime_get_ns() + 5ULL * NSEC_PER_SEC;
> +
> +	while (atomic_read(&s->stage) < stage) {
> +		if (ktime_get_ns() > end)
> +			return false;
> +		cpu_relax();
> +	}
> +	return true;
> +}
[ ... ]
> +static int abba_spin_old(void *arg)
> +{
> +	struct test_abba_spin *s = arg;
> +	struct ww_acquire_ctx ctx;
> +	u64 t, end;
> +	int err = -ETIMEDOUT;
> +
> +	sched_set_fifo_low(current);
> +	wait_for_completion(&s->start);
> +	ww_acquire_init_noinject(&ctx, s->class);
> +	atomic_set(&s->stage, 1);
[ ... ]
> +static int abba_spin_young(void *arg)
> +{
> +	struct test_abba_spin *s = arg;
> +	struct ww_acquire_ctx ctx;
> +	int err = -ETIMEDOUT;
> +
> +	sched_set_fifo_low(current);
> +	wait_for_completion(&s->start);
> +	if (!abba_spin_until(s, 1))
> +		goto out;
> +	ww_acquire_init_noinject(&ctx, s->class);

[Severity: Low]
Can abba_spin_young() obtain a smaller ctx.stamp than abba_spin_old() on
weakly-ordered architectures?

ww_acquire_init() allocates ctx->stamp with
atomic_long_inc_return_relaxed(&ww_class->stamp), and s->stage is updated
and polled with relaxed atomic_set() and atomic_read():

CPU0 (abba_spin_old)              CPU1 (abba_spin_young)
ww_acquire_init_noinject()
  atomic_long_inc_return_relaxed(&s->class->stamp) [reordered]
atomic_set(&s->stage, 1)
                                  abba_spin_until(s, 1)
                                    atomic_read(&s->stage) == 1
                                  ww_acquire_init_noinject()
                                    atomic_long_inc_return_relaxed(...)
  [stamp increment executes here]  <-- young gets the older stamp

If abba_spin_young() receives the smaller stamp, it wounds
abba_spin_old() when blocking on s->a_mutex, causing abba_spin_old()'s
ww_mutex_lock(&s->b_mutex, &ctx) to fail with -EDEADLK.  Using
atomic_set_release() and atomic_read_acquire() on s->stage would order
the stamp allocations.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261106235931.1478800-1-mathieu.desnoyers@efficios.com?part=5

Re: [PATCH 5/5] locking/ww_mutex/test: Time an ABBA with both owners running

[Mathieu Desnoyers]

On 2026-11-06 19:11, sashiko-bot@kernel.org wrote:
> Can abba_spin_young() obtain a smaller ctx.stamp than abba_spin_old() on
> weakly-ordered architectures?
[...]
> Using atomic_set_release() and atomic_read_acquire() on s->stage would
> order the stamp allocations.

Right. ww_acquire_init() takes the stamp with a relaxed increment, and
nothing orders it before the relaxed store of stage 1: on a
weakly-ordered CPU the younger thread can take the older stamp, wound
the other one, and the test fails with -EDEADLK where nothing is wrong.
x86 does not reorder them. v2 makes each stage store a release and the
poll an acquire.

Thanks,

Mathieu



-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com

Re: [PATCH 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken

[Steven Rostedt]

On Fri,  6 Nov 2026 18:59:27 -0500
Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote:

> The top waiter of an rt_mutex spins while the owner runs,
> rtmutex_spin_on_owner(), and does not stop when it is woken: its timeout
> and a signal find it running, set it TASK_RUNNING, and the spin goes on
> until the owner leaves its CPU or the waiter's own CPU has to
> reschedule. A PI futex owner is a user task that may run as long as it
> likes. On v7.4-rc1, in 8-CPU VMs, behind an owner spinning in userspace,
> a 1ms pthread_mutex_timedlock() returns ETIMEDOUT 126ms to 631ms late on
> average, 1.4s at most, a SIGUSR1 is handled up to 1.9s late, and a
> waiter killed with SIGKILL is reaped 0.5s late on average. With
> PREEMPT_RT the wound of a ww_mutex is missed the same way: two contexts
> in an ABBA on two CPUs at SCHED_FIFO spin on each other, and the older
> one gets its second lock after 96ms on average, 0.95s at most.
> 

The above is very verbose. Are you starting to use AI? ;-)

> Stop spinning once the waiter is woken, where the sleep the spin stands
> in for would end: rt_mutex_slowlock_block() then sees the timeout, the
> signal or the wound, as after schedule(). An rtlock waiter is woken out
> of TASK_RTLOCK_WAIT by the lock handover alone, and spins as before.

OK, because its state is set to TASK_RUNNING the schedule_rtlock() will
simply return.

> 
> With this patch the 1ms waits end after 1.01ms on average, a signal is
> handled within 0.3ms (1.4ms with PREEMPT_RT), and the ww_mutex ABBA
> resolves in 12us to 33us; a contended PI mutex keeps its throughput.
> The futex_lock_pi_inherit and test-ww_mutex cases added later in this
> series catch both.
> 
> Fixes: 992caf7f1724 ("locking/rtmutex: Add adaptive spinwait mechanism")
> Cc: stable@vger.kernel.org
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> ---
>  kernel/locking/rtmutex.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c
> index 5a9534c715b8..323973be2074 100644
> --- a/kernel/locking/rtmutex.c
> +++ b/kernel/locking/rtmutex.c
> @@ -1516,9 +1516,13 @@ static bool rtmutex_spin_on_owner(struct rt_mutex_base *lock,
>  		 *  - current is not longer the top waiter
>  		 *  - current is requested to reschedule (redundant
>  		 *    for CONFIG_PREEMPT_RCU=y)
> +		 *  - current is woken, by its timeout, a signal or
> +		 *    whatever else would end the sleep the spin
> +		 *    stands in for
>  		 *  - the VCPU on which owner runs is preempted
>  		 */
>  		if (!owner_on_cpu(owner) || need_resched() ||
> +		    task_is_running(current) ||

So the task enters here with its state non-running, even for
rt_spin_locks, and you break out of the spin if something woke it up.
Breaking out of the spin with the state as TASK_RUNNING just makes it
reanalyze the situation and for rt_spin_locks it will put itself back
to RTLOCK_WAIT and spin again, if the owner hasn't left the CPU.

Makes sense.

Reviewed-by: Steven Rostedt <rostedt@goodmis.org>

-- Steve


>  		    !data_race(rt_mutex_waiter_is_top_waiter(lock, waiter))) {
>  			res = false;
>  			break;

Re: [PATCH 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken

[Peter Zijlstra]

On Sat, Nov 07, 2026 at 12:29:27AM -0500, Steven Rostedt wrote:
> On Fri,  6 Nov 2026 18:59:27 -0500
> Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote:
> 
> > The top waiter of an rt_mutex spins while the owner runs,
> > rtmutex_spin_on_owner(), and does not stop when it is woken: its timeout
> > and a signal find it running, set it TASK_RUNNING, and the spin goes on
> > until the owner leaves its CPU or the waiter's own CPU has to
> > reschedule. A PI futex owner is a user task that may run as long as it
> > likes. On v7.4-rc1, in 8-CPU VMs, behind an owner spinning in userspace,
> > a 1ms pthread_mutex_timedlock() returns ETIMEDOUT 126ms to 631ms late on
> > average, 1.4s at most, a SIGUSR1 is handled up to 1.9s late, and a
> > waiter killed with SIGKILL is reaped 0.5s late on average. With
> > PREEMPT_RT the wound of a ww_mutex is missed the same way: two contexts
> > in an ABBA on two CPUs at SCHED_FIFO spin on each other, and the older
> > one gets its second lock after 96ms on average, 0.95s at most.
> > 
> 
> The above is very verbose. Are you starting to use AI? ;-)

He definitely is. I can't read the above, its word soup. Part of using
AI is making sure the Changelog is actually readable. This is not.

Re: [PATCH 3/5] locking/rtmutex: Clone a waiter's sort keys as it enters pi_waiters

[Peter Zijlstra]

On Fri, Nov 06, 2026 at 06:59:29PM -0500, Mathieu Desnoyers wrote:

snip unreadable changelog

> Fixes: f7853c342418 ("locking/rtmutex: Fix task->pi_waiters integrity")
> Cc: stable@vger.kernel.org
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> ---
>  kernel/locking/rtmutex.c | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c
> index 323973be2074..e89f0f0e604b 100644
> --- a/kernel/locking/rtmutex.c
> +++ b/kernel/locking/rtmutex.c
> @@ -512,11 +512,19 @@ static __always_inline bool __pi_waiter_less(struct rb_node *a, const struct rb_
>  	return rt_waiter_node_less(__node_2_rt_node(a), __node_2_rt_node(b));
>  }
>  
> +/*
> + * Enqueue @waiter in the pi_waiters of @task, its lock's owner, with the
> + * sort keys of its lock tree copy, current under the wait_lock held here:
> + * the chain walk updates the tree copy of a waiter that is not the top
> + * waiter of its lock, and does not clone it, so a copy cloned before would
> + * enter @task's pi_waiters with the keys of a priority it no longer has.
> + */

Oh gawd, Mathieu, can you please tell your AI thing to never write a
Changelog or comment? This is again unreadable soup.

>  static __always_inline void
>  rt_mutex_enqueue_pi(struct task_struct *task, struct rt_mutex_waiter *waiter)
>  {
>  	lockdep_assert_held(&task->pi_lock);
>  
> +	waiter_clone_prio(waiter, task);
>  	rb_add_cached(&waiter->pi_tree.entry, &task->pi_waiters, __pi_waiter_less);
>  }
>  
> @@ -1005,7 +1013,6 @@ static int __sched rt_mutex_adjust_prio_chain(struct task_struct *task,
>  		 * and adjust the priority of the owner.
>  		 */
>  		rt_mutex_dequeue_pi(task, prerequeue_top_waiter);
> -		waiter_clone_prio(waiter, task);
>  		rt_mutex_enqueue_pi(task, waiter);
>  		rt_mutex_adjust_prio(lock, task);
>  
> @@ -1022,7 +1029,6 @@ static int __sched rt_mutex_adjust_prio_chain(struct task_struct *task,
>  		 */
>  		rt_mutex_dequeue_pi(task, waiter);
>  		waiter = rt_mutex_top_waiter(lock);
> -		waiter_clone_prio(waiter, task);
>  		rt_mutex_enqueue_pi(task, waiter);
>  		rt_mutex_adjust_prio(lock, task);
>  	} else {
> @@ -1243,7 +1249,6 @@ static int __sched task_blocks_on_rt_mutex(struct rt_mutex_base *lock,
>  	waiter->task = task;
>  	waiter->lock = lock;
>  	waiter_update_prio(waiter, task);
> -	waiter_clone_prio(waiter, task);

Reverse engineering the problem leads me to think this is the problem
spot. Because this is the one site that is not immediately followed by
rt_mutex_enqueue_pi().

I'm thinking there's race due to dropping pi_lock between this copy and
the subsequent enqueue?

>  
>  	/* Get the top priority waiter on the lock */
>  	if (rt_mutex_has_waiters(lock))
> -- 
> 2.47.3
> 

Re: [PATCH 1/5] locking/rtmutex: Stop spinning on the owner when the waiter is woken

[Mathieu Desnoyers]

On 2026-11-07 05:43, Peter Zijlstra wrote:
> On Sat, Nov 07, 2026 at 12:29:27AM -0500, Steven Rostedt wrote:
>> On Fri,  6 Nov 2026 18:59:27 -0500
>> Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote:
>>
>>> The top waiter of an rt_mutex spins while the owner runs,
>>> rtmutex_spin_on_owner(), and does not stop when it is woken: its timeout
>>> and a signal find it running, set it TASK_RUNNING, and the spin goes on
>>> until the owner leaves its CPU or the waiter's own CPU has to
>>> reschedule. A PI futex owner is a user task that may run as long as it
>>> likes. On v7.4-rc1, in 8-CPU VMs, behind an owner spinning in userspace,
>>> a 1ms pthread_mutex_timedlock() returns ETIMEDOUT 126ms to 631ms late on
>>> average, 1.4s at most, a SIGUSR1 is handled up to 1.9s late, and a
>>> waiter killed with SIGKILL is reaped 0.5s late on average. With
>>> PREEMPT_RT the wound of a ww_mutex is missed the same way: two contexts
>>> in an ABBA on two CPUs at SCHED_FIFO spin on each other, and the older
>>> one gets its second lock after 96ms on average, 0.95s at most.
>>>
>>
>> The above is very verbose. Are you starting to use AI? ;-)

I definitely am, as stated in the cover letter. :-)

> 
> He definitely is. I can't read the above, its word soup. Part of using
> AI is making sure the Changelog is actually readable. This is not.

Feedback on the commit message wording is welcome, I'll try to improve
things for next iterations. Is the following easier to parse ? I fed
Peter's and my own criticism to Claude and got it to rewrite the entire
thing (and keep those learnings for next time):

locking/rtmutex: Stop spinning on the owner when the waiter is woken

The top waiter of an rt_mutex sets its sleep state. Then it spins in
rtmutex_spin_on_owner() as long as the owner is running. A wakeup sets
the waiter back to TASK_RUNNING. The spin loop does not check for this.
So a timeout or a signal is only handled when the owner blocks, or when
the waiter has to reschedule:

   CPU 0 (waiter)                            CPU 1 (owner)

                                             pthread_mutex_lock()
                                               takes the PI futex
                                             spins in userspace
   pthread_mutex_timedlock(1ms)
     futex_lock_pi()
       rt_mutex_wait_proxy_lock()
         set_current_state(TASK_INTERRUPTIBLE)
         rt_mutex_slowlock_block()
           rtmutex_spin_on_owner()
             <timeout> hrtimer_wakeup()
               wake_up_process(waiter)
                 __state = TASK_RUNNING
             owner_on_cpu(owner): true
             need_resched(): false
             keeps spinning                  still spins in userspace

The owner of a PI futex is a user task. It can run for a long time.
With an owner spinning in userspace, a 1ms pthread_mutex_timedlock()
returns up to 1.4s late. A signal to the waiter is delayed in the same
way.

With PREEMPT_RT, a ww_mutex wound is missed too. Two contexts take two
ww_mutexes in opposite order, on two CPUs. They spin on each other for
up to a second.

Fix this by also stopping the spin when the waiter is TASK_RUNNING. The
slowpath then handles the timeout, the signal or the wound, as it does
after schedule().

rtlock waiters are not affected. Only the lock handoff wakes them from
TASK_RTLOCK_WAIT.

With this patch, the 1ms timed lock returns after about 1ms. The
ww_mutex ABBA resolves in microseconds. PI mutex throughput is
unchanged. The futex and ww_mutex selftests added later in this series
cover both cases.

Fixes: 992caf7f1724 ("locking/rtmutex: Add adaptive spinwait mechanism")
Cc: stable@vger.kernel.org
Reviewed-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com