[PATCH 0/4] sparc32: SMP fixes, futexes on SMP, and a cmpxchg system call

[PATCH 0/4] sparc32: SMP fixes, futexes on SMP, and a cmpxchg system call

[Ramin Moussavi]

This series makes NPTL usable on sparc32 SMP kernels. It started with
the uClibc-ng test suite on qemu's SS-20 with -smp 4: the kernel went to
error state under console load, sometimes hung at boot, and had no
futexes at all.

Patches 1 and 2 are fixes and do not depend on the rest:

  1. The per-cpu timer interrupts on sun4m/sun4d and the LEON IPI leave
     their handler with traps enabled at the PIL of the interrupted
     code while the trap frame is still on the stack, so interrupts can
     nest without bound until the CPU goes to error state. The generic
     path was fixed in 2.1.129; these three exits never got the change.

  2. A secondary CPU gets 10000 rounds of udelay(200) to call in. Under
     qemu the delay loop calibration depends on host load, the wait can
     shrink to a few milliseconds, and a CPU that calls in late blocks
     RCU and hangs the boot. Wait by jiffies instead.

Both are marked for stable but have no Fixes: tag. The sun4m and sun4d
code they change is already in 2.6.12-rc2, the start of the git history;
the LEON variants were copied from it in 2011.

Patches 3 and 4 add what user space needs on SMP:

  3. Futex atomics for sparc32, so FUTEX no longer depends on !SMP.
     casa is probed at boot (LEON); without it the word is updated
     under the hashed spinlock the kernel already uses for its atomics.

  4. sparc_atomic_cmpxchg_32, a compare-and-exchange system call for
     user space built for v8, which has no CAS of its own. The ldstub
     emulation C libraries use instead is not async-signal-safe and is
     invisible to the futex code. It takes the same path as the futex
     atomics, like atomic_cmpxchg_32 does on m68k. The uClibc-ng side
     that uses it is ready and waits for this.

Tested on next-20261014 under qemu 11.1.0: the uClibc-ng suite on SS-20
with -smp 4 and UP and on leon3_generic with -smp 4, plus a test
program for the system call; the details are in each patch. sun4d was
only built, qemu has no sun4d machine.

Testing under qemu
------------------

Three qemu problems get in the way of reproducing this, none of them
caused by the series:

- leon3_generic does not boot an image passed with -kernel since qemu
  9.0: the synthetic bootloader no longer sets %sp (92688a91c358 took
  it out of the reset handler). Gaisler's own prebuilt images fail the
  same way.

- leon3_generic does not bring up a second CPU under Linux. The IRQMP
  model never sets the broadcast-available bit, and Linux stops with a
  BUG in leon_smp.c; with that bit set, the secondary CPU dies on the
  swapa with ASI 0x01 (forced cache miss) in leon_cpu_pre_online(),
  which qemu does not decode.

- On sun4m, qemu's ESCC drives only the interrupt line of the channel
  it was called for; the board ORs both lines, and a stale line keeps
  level 12 asserted. Boots then hang at random with an RCU stall about
  20 seconds in (4 of 40 boots with -smp 4 here). That looks like a
  problem with this series but is not.

The LEON SMP results were taken with these fixed in qemu; the fixes are
going to be reported to qemu.

These patches were developed with the help of Claude (Anthropic), as
the Assisted-by tags say. I have reviewed them and tested them as
described.

Ramin Moussavi (4):
  sparc32: keep PIL raised until the IRQ frame is gone
  sparc32: wait for secondary CPUs by time, not by loop count
  sparc32: implement futex atomics, with casa detected at run time
  sparc32: add the sparc_atomic_cmpxchg_32 system call

 arch/sparc/include/asm/cmpxchg_32.h    |  4 ++
 arch/sparc/include/asm/futex_32.h      | 72 +++++++++++++++++++++++++-
 arch/sparc/kernel/entry.S              |  9 ++--
 arch/sparc/kernel/leon_smp.c           |  8 ++-
 arch/sparc/kernel/sun4d_smp.c          |  8 ++-
 arch/sparc/kernel/sun4m_smp.c          |  8 ++-
 arch/sparc/kernel/sys_sparc_32.c       | 35 +++++++++++++
 arch/sparc/kernel/syscalls/syscall.tbl |  1 +
 arch/sparc/kernel/systbls.h            |  1 +
 arch/sparc/kernel/traps_32.c           | 13 ++++-
 arch/sparc/lib/Makefile                |  3 +-
 arch/sparc/lib/atomic32.c              | 53 +++++++++++++++++++
 arch/sparc/lib/casa_32.S               | 30 +++++++++++
 init/Kconfig                           |  1 -
 14 files changed, 224 insertions(+), 22 deletions(-)
 create mode 100644 arch/sparc/lib/casa_32.S


base-commit: d419d682257d813f6cb7a28e42b35d0edf4ad038
-- 
2.53.0

[PATCH 1/4] sparc32: keep PIL raised until the IRQ frame is gone

[Ramin Moussavi]

The per-cpu timer interrupts on sun4m and sun4d SMP, and the LEON SMP
IPI, leave their handler with

	wr	%l0, PSR_ET, %psr
	WRITE_PAUSE
	RESTORE_ALL

%l0 is the PSR at trap time, so this turns traps back on at the PIL of
the interrupted code, usually 0, while the trap frame is still on the
stack. Any interrupt that is pending at that point is taken on top of
the frame, and if it leaves through the same path it opens the same
window one level deeper. A source that keeps firing makes the nesting
unbounded: the kernel stack runs into its thread_info, w_saved and the
task pointer get overwritten, and the next window trap stores through
them with traps disabled. The CPU goes to error state.

The generic path does not have this problem. real_irq_continue keeps
PIL at 15 after handler_irq and lets ret_trap_entry restore the old PIL
together with the rett. It was fixed that way in Linux 2.1.129
(entry.S 1.153, November 1998):

	-	wr	%l0, PSR_ET, %psr
	+	or	%l0, PSR_PIL, %g2	! restore PIL after handler_irq
	+	wr	%g2, PSR_ET, %psr	! keep ET up

smp_ticker had been added in 2.1.36 with a copy of the old exit, and
kept it through the rename to smp4m_ticker and the addition of
smp4d_ticker in 2.1.97. smpleon_ipi copied the pattern in 2011. Do the
same as real_irq_continue in all three.

On hardware the window is a few instructions long and no interrupt
source refills it that fast. qemu does: its ESCC finishes a transmit
instantly, and since 8.2 it takes a pending interrupt right after rett
(930f1865cc65 "target/sparc: Clear may_lookup for npc == DYNAMIC_PC").
A burst of console output on an SS-20 with -smp 4 is enough. A trap
ring buffer in trap_setup showed the level 12 serial interrupt taken
over and over at smp4m_ticker+0x34, timer ticks nesting further down,
and the frames sinking below the thread_info.

Tested on qemu 11.1.0, SS-20, -smp 4, with a test that writes a few
hundred lines to the serial console from a multithreaded process: the
kernel went to error state in 4 of 4 boots before, and booted cleanly
8 of 8 times after. Two full runs of the uClibc-ng test suite on top
then completed without a crash. LEON SMP was tested on qemu's
leon3_generic with -smp 4, where a full run of the suite completed
without a crash as well. sun4d gets the same one line change but was
not run; qemu has no sun4d machine.

Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Ramin Moussavi <ramin.moussavi@yacoub.de>
---
 arch/sparc/kernel/entry.S | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/arch/sparc/kernel/entry.S b/arch/sparc/kernel/entry.S
index ea51ef52c952..87d03a58446e 100644
--- a/arch/sparc/kernel/entry.S
+++ b/arch/sparc/kernel/entry.S
@@ -252,7 +252,8 @@ smp4m_ticker:
 	WRITE_PAUSE
 	call	smp4m_percpu_timer_interrupt
 	 add	%sp, STACKFRAME_SZ, %o0
-	wr	%l0, PSR_ET, %psr
+	or	%l0, PSR_PIL, %g2	! keep PIL up until the frame is gone
+	wr	%g2, PSR_ET, %psr
 	WRITE_PAUSE
 	RESTORE_ALL
 
@@ -353,7 +354,8 @@ smp4d_ticker:
 	WRITE_PAUSE
 	call	smp4d_percpu_timer_interrupt
 	 add	%sp, STACKFRAME_SZ, %o0
-	wr	%l0, PSR_ET, %psr
+	or	%l0, PSR_PIL, %g2	! keep PIL up until the frame is gone
+	wr	%g2, PSR_ET, %psr
 	WRITE_PAUSE
 	RESTORE_ALL
 
@@ -407,7 +409,8 @@ smpleon_ipi:
 	WRITE_PAUSE
 	call	leonsmp_ipi_interrupt
 	 add	%sp, STACKFRAME_SZ, %o1 ! pt_regs
-	wr	%l0, PSR_ET, %psr
+	or	%l0, PSR_PIL, %g2	! keep PIL up until the frame is gone
+	wr	%g2, PSR_ET, %psr
 	WRITE_PAUSE
 	RESTORE_ALL
 

base-commit: d419d682257d813f6cb7a28e42b35d0edf4ad038
-- 
2.53.0

[PATCH 2/4] sparc32: wait for secondary CPUs by time, not by loop count

[Ramin Moussavi]

smp4m_boot_one_cpu(), smp4d_boot_one_cpu() and leon_boot_one_cpu() give
a starting CPU 10000 rounds of udelay(200) to show up in cpu_callin_map,
which is meant to be two seconds. But udelay() is only as long as
loops_per_jiffy says. Under qemu the delay loop calibration varies from
boot to boot depending on host load -- on an emulated SPARCstation 20
between 4 and 671 BogoMIPS for the same kernel and command line -- and
with a low value the wait shrinks to a few milliseconds. The boot CPU
then reports

  Processor 1 is stuck.

although CPU 1 is just about to call in.

That is worse than losing a CPU. The late CPU still runs
sparc_start_secondary(), calls notify_cpu_starting(), which makes it
known to RCU, and then spins in its pre-online loop waiting for
smp_commenced_mask, which is never set for it. As it never comes
online, the grace period waits for it for good:

  rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
  rcu: Offline CPU 1 blocking current GP.

and the boot hangs.

Wait two seconds by jiffies instead. The timer is already running at
this point; calibrate_delay() has used it.

Booting with lpj=1000 forces the short wait. On qemu's SS-20 with
-smp 4, the unpatched kernel declared all three secondary CPUs stuck
within 4 ms of starting each and hung in 4 of 4 boots; patched, it
brought up all four CPUs in 4 of 4. On leon3_generic, with two fixes
to qemu's LEON SMP support that are yet to be sent to qemu-devel, it
is the same: unpatched, all three secondary CPUs were declared stuck
in the same millisecond they were started and the boot hung with the
RCU stall above, in 2 of 2 boots; patched, all four CPUs came up in
2 of 2. sun4d is build-tested only, qemu has no sun4d machine.

Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Ramin Moussavi <ramin.moussavi@yacoub.de>
---
 arch/sparc/kernel/leon_smp.c  | 8 +++-----
 arch/sparc/kernel/sun4d_smp.c | 8 +++-----
 arch/sparc/kernel/sun4m_smp.c | 8 +++-----
 3 files changed, 9 insertions(+), 15 deletions(-)

diff --git a/arch/sparc/kernel/leon_smp.c b/arch/sparc/kernel/leon_smp.c
index 1ee393abc463..21e7b91cb9ca 100644
--- a/arch/sparc/kernel/leon_smp.c
+++ b/arch/sparc/kernel/leon_smp.c
@@ -180,7 +180,7 @@ void __init leon_boot_cpus(void)
 
 int leon_boot_one_cpu(int i, struct task_struct *idle)
 {
-	int timeout;
+	unsigned long timeout;
 
 	current_set[i] = task_thread_info(idle);
 
@@ -205,11 +205,9 @@ int leon_boot_one_cpu(int i, struct task_struct *idle)
 	LEON_BYPASS_STORE_PA(&(leon3_irqctrl_regs->mpstatus), 1 << i);
 
 	/* wheee... it's going... */
-	for (timeout = 0; timeout < 10000; timeout++) {
-		if (cpu_callin_map[i])
-			break;
+	timeout = jiffies + 2 * HZ;
+	while (!cpu_callin_map[i] && time_before(jiffies, timeout))
 		udelay(200);
-	}
 	printk(KERN_INFO "Started CPU %d\n", (unsigned int)i);
 
 	if (!(cpu_callin_map[i])) {
diff --git a/arch/sparc/kernel/sun4d_smp.c b/arch/sparc/kernel/sun4d_smp.c
index 9a62a5cf3337..9355171ff792 100644
--- a/arch/sparc/kernel/sun4d_smp.c
+++ b/arch/sparc/kernel/sun4d_smp.c
@@ -122,7 +122,7 @@ void __init smp4d_boot_cpus(void)
 int smp4d_boot_one_cpu(int i, struct task_struct *idle)
 {
 	unsigned long *entry = &sun4d_cpu_startup;
-	int timeout;
+	unsigned long timeout;
 	int cpu_node;
 
 	cpu_find_by_instance(i, &cpu_node, NULL);
@@ -145,11 +145,9 @@ int smp4d_boot_one_cpu(int i, struct task_struct *idle)
 	printk(KERN_INFO "prom_startcpu returned :)\n");
 
 	/* wheee... it's going... */
-	for (timeout = 0; timeout < 10000; timeout++) {
-		if (cpu_callin_map[i])
-			break;
+	timeout = jiffies + 2 * HZ;
+	while (!cpu_callin_map[i] && time_before(jiffies, timeout))
 		udelay(200);
-	}
 
 	if (!(cpu_callin_map[i])) {
 		printk(KERN_ERR "Processor %d is stuck.\n", i);
diff --git a/arch/sparc/kernel/sun4m_smp.c b/arch/sparc/kernel/sun4m_smp.c
index 056df034e79e..4b3fc081f103 100644
--- a/arch/sparc/kernel/sun4m_smp.c
+++ b/arch/sparc/kernel/sun4m_smp.c
@@ -79,7 +79,7 @@ void __init smp4m_boot_cpus(void)
 int smp4m_boot_one_cpu(int i, struct task_struct *idle)
 {
 	unsigned long *entry = &sun4m_cpu_startup;
-	int timeout;
+	unsigned long timeout;
 	int cpu_node;
 
 	cpu_find_by_mid(i, &cpu_node);
@@ -103,11 +103,9 @@ int smp4m_boot_one_cpu(int i, struct task_struct *idle)
 	prom_startcpu(cpu_node, &smp_penguin_ctable, 0, (char *)entry);
 
 	/* wheee... it's going... */
-	for (timeout = 0; timeout < 10000; timeout++) {
-		if (cpu_callin_map[i])
-			break;
+	timeout = jiffies + 2 * HZ;
+	while (!cpu_callin_map[i] && time_before(jiffies, timeout))
 		udelay(200);
-	}
 
 	if (!(cpu_callin_map[i])) {
 		printk(KERN_ERR "Processor %d is stuck.\n", i);
-- 
2.53.0

[PATCH 3/4] sparc32: implement futex atomics, with casa detected at run time

[Ramin Moussavi]

sparc32 has no futex_atomic_cmpxchg_inatomic() or
arch_futex_atomic_op_inuser() of its own. The generic versions only
work on UP, so FUTEX depends on !(SPARC32 && SMP) and an SMP kernel
has no futexes at all; NPTL cannot run on it.

Provide both. CPUs with casa use it on the user word. That is optional
on LEON and absent from every other sparc32, so instead of a config
option, probe for it once at boot: execute casa under an exception
table entry, and let do_illegal_instruction() fix up a kernel mode
trap that has one. One kernel then runs on v8 and on LEON. casa sits
in a file of its own that is assembled for LEON; the rest of the kernel
stays at -Av8.

Without casa, the word is updated under the hashed spinlock the kernel
already uses for its own atomics. The hash only takes address bits
within the page offset, so every mapping of a shared page ends up on
the same lock, which process-shared futexes need. Page faults are
disabled while it is held; the futex core faults the page in and
retries.

The FUTEX_OP_* operations are a compare-and-exchange loop, so the
instruction or the lock appears in one place only. The structure
follows the LEON futex support Andreas Larsson keeps in Gaisler's
patch set, which chooses casa at configuration time.

Tested on qemu 11.1.0:
  - SS-20, -smp 4: futexes come up; with the next patch and a C
    library that uses it, the uClibc-ng NPTL tests pass
  - SS-20, UP: unchanged results
  - leon3_generic: "sparc32: casa present", futex path through casa
  - SS-20 with the probe forced on: "casa absent", the illegal
    instruction trap is fixed up and boot continues

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Ramin Moussavi <ramin.moussavi@yacoub.de>
---
 arch/sparc/include/asm/cmpxchg_32.h |  4 ++
 arch/sparc/include/asm/futex_32.h   | 72 ++++++++++++++++++++++++++++-
 arch/sparc/kernel/traps_32.c        | 13 +++++-
 arch/sparc/lib/Makefile             |  3 +-
 arch/sparc/lib/atomic32.c           | 53 +++++++++++++++++++++
 arch/sparc/lib/casa_32.S            | 30 ++++++++++++
 init/Kconfig                        |  1 -
 7 files changed, 172 insertions(+), 4 deletions(-)
 create mode 100644 arch/sparc/lib/casa_32.S

diff --git a/arch/sparc/include/asm/cmpxchg_32.h b/arch/sparc/include/asm/cmpxchg_32.h
index 8c1a3ca34eeb..c08c14b62ab5 100644
--- a/arch/sparc/include/asm/cmpxchg_32.h
+++ b/arch/sparc/include/asm/cmpxchg_32.h
@@ -64,6 +64,10 @@ __cmpxchg(volatile void *ptr, unsigned long old, unsigned long new_, int size)
 u64 __cmpxchg_u64(volatile u64 *ptr, u64 old, u64 new);
 #define arch_cmpxchg64(ptr, old, new)	__cmpxchg_u64(ptr, old, new)
 
+int sparc32_user_cmpxchg_u32(u32 __user *uaddr, u32 old, u32 new, u32 *prev);
+int __sparc32_casa(u32 *addr, u32 old, u32 new, u32 *prev);
+extern bool sparc32_has_cas;
+
 #include <asm-generic/cmpxchg-local.h>
 
 /*
diff --git a/arch/sparc/include/asm/futex_32.h b/arch/sparc/include/asm/futex_32.h
index 6a332a9f099c..fe2386058a1f 100644
--- a/arch/sparc/include/asm/futex_32.h
+++ b/arch/sparc/include/asm/futex_32.h
@@ -1,6 +1,76 @@
 #ifndef _ASM_FUTEX_H
 #define _ASM_FUTEX_H
 
-#include <asm-generic/futex.h>
+#include <linux/futex.h>
+#include <linux/preempt.h>
+#include <linux/uaccess.h>
+#include <asm/cmpxchg.h>
+#include <asm/errno.h>
+
+/*
+ * Based on the LEON futex support by Andreas Larsson, with casa chosen at
+ * run time instead of at configuration time.  Without casa the word is
+ * updated under the lock the kernel's own atomics use, which also works
+ * on SMP.
+ */
+static inline int
+futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr,
+			      u32 oldval, u32 newval)
+{
+	int ret;
+
+	if (unlikely(!access_ok(uaddr, sizeof(u32))))
+		return -EFAULT;
+
+	if (!sparc32_has_cas)
+		return sparc32_user_cmpxchg_u32(uaddr, oldval, newval, uval);
+
+	preempt_disable();
+	ret = __sparc32_casa((u32 __force *)uaddr, oldval, newval, uval);
+	preempt_enable();
+	return ret;
+}
+
+static inline int
+arch_futex_atomic_op_inuser(int op, u32 oparg, int *oval, u32 __user *uaddr)
+{
+	u32 old, new, cur;
+	int ret;
+
+	if (unlikely(get_user(old, uaddr)))
+		return -EFAULT;
+
+	for (;;) {
+		switch (op) {
+		case FUTEX_OP_SET:
+			new = oparg;
+			break;
+		case FUTEX_OP_ADD:
+			new = old + oparg;
+			break;
+		case FUTEX_OP_OR:
+			new = old | oparg;
+			break;
+		case FUTEX_OP_ANDN:
+			new = old & ~oparg;
+			break;
+		case FUTEX_OP_XOR:
+			new = old ^ oparg;
+			break;
+		default:
+			return -ENOSYS;
+		}
+
+		ret = futex_atomic_cmpxchg_inatomic(&cur, uaddr, old, new);
+		if (ret)
+			return ret;
+		if (cur == old)
+			break;
+		old = cur;
+	}
+
+	*oval = old;
+	return 0;
+}
 
 #endif
diff --git a/arch/sparc/kernel/traps_32.c b/arch/sparc/kernel/traps_32.c
index bb149f6cc34b..9d77ff2fd6fe 100644
--- a/arch/sparc/kernel/traps_32.c
+++ b/arch/sparc/kernel/traps_32.c
@@ -19,6 +19,7 @@
 #include <linux/smp.h>
 #include <linux/kdebug.h>
 #include <linux/export.h>
+#include <linux/extable.h>
 #include <linux/pgtable.h>
 
 #include <asm/delay.h>
@@ -108,8 +109,18 @@ void do_hw_interrupt(struct pt_regs *regs, unsigned long type)
 void do_illegal_instruction(struct pt_regs *regs, unsigned long pc, unsigned long npc,
 			    unsigned long psr)
 {
-	if(psr & PSR_PS)
+	if (psr & PSR_PS) {
+		const struct exception_table_entry *entry;
+
+		/* The casa probe at boot lands here on CPUs without it. */
+		entry = search_exception_tables(regs->pc);
+		if (entry) {
+			regs->pc = entry->fixup;
+			regs->npc = regs->pc + 4;
+			return;
+		}
 		die_if_kernel("Kernel illegal instruction", regs);
+	}
 #ifdef TRAP_DEBUG
 	printk("Ill instr. at pc=%08lx instruction is %08lx\n",
 	       regs->pc, *(unsigned long *)regs->pc);
diff --git a/arch/sparc/lib/Makefile b/arch/sparc/lib/Makefile
index dd10cdd6f062..284236024e6a 100644
--- a/arch/sparc/lib/Makefile
+++ b/arch/sparc/lib/Makefile
@@ -51,5 +51,6 @@ lib-$(CONFIG_SPARC64) += copy_in_user.o memmove.o
 lib-$(CONFIG_SPARC64) += mcount.o ipcsum.o hweight.o ffs.o
 
 obj-$(CONFIG_SPARC64) += iomap.o
-obj-$(CONFIG_SPARC32) += atomic32.o
+obj-$(CONFIG_SPARC32) += atomic32.o casa_32.o
+AFLAGS_casa_32.o += -Wa,-Aleon
 obj-$(CONFIG_SPARC64) += PeeCeeI.o
diff --git a/arch/sparc/lib/atomic32.c b/arch/sparc/lib/atomic32.c
index 8ae880ebf07a..8f4b6079bf1c 100644
--- a/arch/sparc/lib/atomic32.c
+++ b/arch/sparc/lib/atomic32.c
@@ -11,6 +11,10 @@
 #include <linux/atomic.h>
 #include <linux/spinlock.h>
 #include <linux/module.h>
+#include <linux/init.h>
+#include <linux/printk.h>
+#include <linux/uaccess.h>
+#include <asm/cpu_type.h>
 
 #ifdef CONFIG_SMP
 #define ATOMIC_HASH_SIZE	4
@@ -195,3 +199,52 @@ unsigned long __xchg_u32(volatile u32 *ptr, u32 new)
 	return (unsigned long)prev;
 }
 EXPORT_SYMBOL(__xchg_u32);
+
+/*
+ * Compare-and-exchange on a user word for CPUs without casa, under the
+ * lock that covers the same word in the kernel's own atomics.  The hash
+ * only uses bits within the page offset, so every mapping of a shared
+ * page takes the same lock.
+ *
+ * Page faults cannot be served here.  Returns -EFAULT if the word is not
+ * mapped writable; the caller faults it in and calls again.
+ */
+int sparc32_user_cmpxchg_u32(u32 __user *uaddr, u32 old, u32 new, u32 *prev)
+{
+	unsigned long flags;
+	int ret = 0;
+	u32 val;
+
+	spin_lock_irqsave(ATOMIC_HASH(uaddr), flags);
+	pagefault_disable();
+	if (get_user(val, uaddr))
+		ret = -EFAULT;
+	else if (val == old && put_user(new, uaddr))
+		ret = -EFAULT;
+	pagefault_enable();
+	spin_unlock_irqrestore(ATOMIC_HASH(uaddr), flags);
+
+	*prev = val;
+	return ret;
+}
+
+/*
+ * casa is optional on LEON and absent from every other sparc32.  Probe for
+ * it rather than trusting the configuration, so that one kernel runs on
+ * both.  User words must then always go through casa: userspace built for
+ * LEON uses it directly, and a lock would not exclude that.
+ */
+bool sparc32_has_cas __ro_after_init;
+
+static int __init sparc32_probe_cas(void)
+{
+	u32 word = 0, prev;
+
+	if (sparc_cpu_model != sparc_leon)
+		return 0;
+
+	sparc32_has_cas = !__sparc32_casa(&word, 0, 1, &prev) && word == 1;
+	pr_info("sparc32: casa %s\n", sparc32_has_cas ? "present" : "absent");
+	return 0;
+}
+early_initcall(sparc32_probe_cas);
diff --git a/arch/sparc/lib/casa_32.S b/arch/sparc/lib/casa_32.S
new file mode 100644
index 000000000000..eab7a5969e3f
--- /dev/null
+++ b/arch/sparc/lib/casa_32.S
@@ -0,0 +1,30 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * casa for the LEON CPUs that implement it.  This file alone is assembled
+ * for LEON, the rest of the kernel stays at v8.  Callers check
+ * sparc32_has_cas first, except the boot probe, which relies on the
+ * illegal instruction trap being fixed up through the exception table.
+ */
+
+#include <linux/linkage.h>
+#include <asm/errno.h>
+
+/*
+ * int __sparc32_casa(u32 *addr, u32 old, u32 new, u32 *prev)
+ *
+ * Returns 0 and the previous value in *prev, or -EFAULT if the access
+ * faulted or the CPU has no casa.
+ */
+ENTRY(__sparc32_casa)
+1:	casa	[%o0] 0xb, %o1, %o2
+	st	%o2, [%o3]
+	retl
+	 clr	%o0
+2:	retl
+	 mov	-EFAULT, %o0
+ENDPROC(__sparc32_casa)
+
+	.section __ex_table, "a"
+	.align	4
+	.word	1b, 2b
+	.previous
diff --git a/init/Kconfig b/init/Kconfig
index 54f78d87344a..8fd656bbc4e0 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1682,7 +1682,6 @@ config BASE_SMALL
 
 config FUTEX
 	bool "Enable futex support" if EXPERT
-	depends on !(SPARC32 && SMP)
 	default y
 	imply RT_MUTEXES
 	help
-- 
2.53.0

[PATCH 4/4] sparc32: add the sparc_atomic_cmpxchg_32 system call

[Ramin Moussavi]

User space built for v8 has no compare-and-swap. C libraries emulate it
with ldstub locks of their own, and those have two problems the kernel
cannot see. They are not async-signal-safe: a handler that reaches an
atomic while the code it interrupted holds the lock spins forever. And
the kernel does not know them: FUTEX_WAKE_OP and robust futexes write
the same words without taking those locks.

Let user space ask the kernel instead. The call goes through
futex_atomic_cmpxchg_inatomic(), so it agrees with whatever the futex
code does to the same word, casa on LEON or the hashed lock otherwise,
and a single system call cannot be interrupted by a signal halfway.
m68k solved the same problem the same way with atomic_cmpxchg_32.

The previous value goes back in %o1 and the return value is 0 or an
error, so that every 32-bit value can be told apart from an error; the
carry flag decides. A misaligned word gives -EINVAL. On a fault the
page is faulted in for writing and the operation retried, so a
copy-on-write page breaks first and a read-only one gives -EFAULT.

The 64-bit kernel leaves the compat slot empty: a 32-bit program on a
v9 CPU gets -ENOSYS and keeps its own emulation, which works there.

Tested on qemu 11.1.0 (SS-20 UP, SS-20 -smp 4, leon3_generic) with a
test program: hit and miss, a word of 0xffffffff, a misaligned word,
NULL, a read-only page, copy-on-write after fork(), and two processes
counting to 40000 on a shared page. A kernel without the call gives
-ENOSYS. A uClibc-ng that probes for the call at startup and uses it
for all its atomics passes its full test suite on UP and on -smp 4,
and a loop of 3000 runs of tst-mallocfork, which forks from a signal
handler, no longer hangs.

Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Ramin Moussavi <ramin.moussavi@yacoub.de>
---
 arch/sparc/kernel/sys_sparc_32.c       | 35 ++++++++++++++++++++++++++
 arch/sparc/kernel/syscalls/syscall.tbl |  1 +
 arch/sparc/kernel/systbls.h            |  1 +
 3 files changed, 37 insertions(+)

diff --git a/arch/sparc/kernel/sys_sparc_32.c b/arch/sparc/kernel/sys_sparc_32.c
index fb31bc0c5b48..d065f0a41b04 100644
--- a/arch/sparc/kernel/sys_sparc_32.c
+++ b/arch/sparc/kernel/sys_sparc_32.c
@@ -26,6 +26,7 @@
 #include <linux/hugetlb.h>
 
 #include <linux/uaccess.h>
+#include <asm/futex.h>
 #include <asm/unistd.h>
 
 #include "systbls.h"
@@ -95,6 +96,40 @@ SYSCALL_DEFINE0(sparc_pipe)
 	return error;
 }
 
+/*
+ * Compare-and-exchange for userspace that cannot rely on casa.  It goes
+ * through the futex primitive, so it agrees with whatever the kernel does
+ * to the same word.  The previous value goes back in %o1, so that every
+ * 32-bit value can be told apart from an error.
+ */
+SYSCALL_DEFINE3(sparc_atomic_cmpxchg_32, u32 __user *, uaddr,
+		u32, old, u32, new)
+{
+	struct mm_struct *mm = current->mm;
+	u32 prev;
+	int ret;
+
+	if ((unsigned long)uaddr & 3)
+		return -EINVAL;
+
+	for (;;) {
+		pagefault_disable();
+		ret = futex_atomic_cmpxchg_inatomic(&prev, uaddr, old, new);
+		pagefault_enable();
+		if (ret != -EFAULT)
+			break;
+		mmap_read_lock(mm);
+		ret = fixup_user_fault(mm, (unsigned long)uaddr,
+				       FAULT_FLAG_WRITE, NULL);
+		mmap_read_unlock(mm);
+		if (ret < 0)
+			return -EFAULT;
+	}
+
+	current_pt_regs()->u_regs[UREG_I1] = prev;
+	return 0;
+}
+
 int sparc_mmap_check(unsigned long addr, unsigned long len)
 {
 	/* See asm-sparc/uaccess.h */
diff --git a/arch/sparc/kernel/syscalls/syscall.tbl b/arch/sparc/kernel/syscalls/syscall.tbl
index 5b9fe0e8140f..939e0130b7e2 100644
--- a/arch/sparc/kernel/syscalls/syscall.tbl
+++ b/arch/sparc/kernel/syscalls/syscall.tbl
@@ -518,3 +518,4 @@
 470	common	listns				sys_listns
 471	common	rseq_slice_yield		sys_rseq_slice_yield
 472	common	fchroot				sys_fchroot
+473	32	sparc_atomic_cmpxchg_32		sys_sparc_atomic_cmpxchg_32	sys_ni_syscall
diff --git a/arch/sparc/kernel/systbls.h b/arch/sparc/kernel/systbls.h
index bf014267d619..25e1ceefe323 100644
--- a/arch/sparc/kernel/systbls.h
+++ b/arch/sparc/kernel/systbls.h
@@ -11,6 +11,7 @@
 
 asmlinkage long sys_getpagesize(void);
 asmlinkage long sys_sparc_pipe(void);
+asmlinkage long sys_sparc_atomic_cmpxchg_32(u32 __user *uaddr, u32 old, u32 new);
 asmlinkage long sys_nis_syscall(void);
 asmlinkage long sys_getdomainname(char __user *name, int len);
 void do_rt_sigreturn(struct pt_regs *regs);
-- 
2.53.0

Re: [PATCH 2/4] sparc32: wait for secondary CPUs by time, not by loop count

[Arnd Bergmann]

On Thu, Oct 15, 2026, at 10:55, Ramin Moussavi wrote:
> smp4m_boot_one_cpu(), smp4d_boot_one_cpu() and leon_boot_one_cpu() give
> a starting CPU 10000 rounds of udelay(200) to show up in cpu_callin_map,
> which is meant to be two seconds. But udelay() is only as long as
> loops_per_jiffy says. Under qemu the delay loop calibration varies from
> boot to boot depending on host load -- on an emulated SPARCstation 20
> between 4 and 671 BogoMIPS for the same kernel and command line -- and
> with a low value the wait shrinks to a few milliseconds. The boot CPU
> then reports
>

The patch is not wrong, but having a broken udelay() is still problematic
in other places, so it may be better to fix that instead.

On most modern targets, we tend to just use the cycle counter if the
frequency is known and avoid the delay loop calibration entirely.
Any reason that sparc32 still needs the calibration?

     Arnd

Re: [PATCH 4/4] sparc32: add the sparc_atomic_cmpxchg_32 system call

[Arnd Bergmann]

On Thu, Oct 15, 2026, at 10:55, Ramin Moussavi wrote:
> User space built for v8 has no compare-and-swap. C libraries emulate it
> with ldstub locks of their own, and those have two problems the kernel
> cannot see. They are not async-signal-safe: a handler that reaches an
> atomic while the code it interrupted holds the lock spins forever. And
> the kernel does not know them: FUTEX_WAKE_OP and robust futexes write
> the same words without taking those locks.

As far as I know, the baseline in sparc32 glibc has been leon3 or
sparcv9 for a long time, so CASA should always be available.

Any remaining sun4m users are on an ancient userspace with a
pre-futex glibc.

> Let user space ask the kernel instead. The call goes through
> futex_atomic_cmpxchg_inatomic(), so it agrees with whatever the futex
> code does to the same word, casa on LEON or the hashed lock otherwise,
> and a single system call cannot be interrupted by a signal halfway.
> m68k solved the same problem the same way with atomic_cmpxchg_32.

This sounds like actively regressing performance on LEON compared
to just using CASA and futex. What are you actually trying to
achieve?

> diff --git a/arch/sparc/kernel/syscalls/syscall.tbl 
> b/arch/sparc/kernel/syscalls/syscall.tbl
> index 5b9fe0e8140f..939e0130b7e2 100644
> --- a/arch/sparc/kernel/syscalls/syscall.tbl
> +++ b/arch/sparc/kernel/syscalls/syscall.tbl
> @@ -518,3 +518,4 @@
>  470	common	listns				sys_listns
>  471	common	rseq_slice_yield		sys_rseq_slice_yield
>  472	common	fchroot				sys_fchroot
> +473	32	sparc_atomic_cmpxchg_32		sys_sparc_atomic_cmpxchg_32	sys_ni_syscall

Wrong syscall number.

     Arnd

Re: [PATCH 3/4] sparc32: implement futex atomics, with casa detected at run time

[Arnd Bergmann]

On Thu, Oct 15, 2026, at 10:55, Ramin Moussavi wrote:
> sparc32 has no futex_atomic_cmpxchg_inatomic() or
> arch_futex_atomic_op_inuser() of its own. The generic versions only
> work on UP, so FUTEX depends on !(SPARC32 && SMP) and an SMP kernel
> has no futexes at all; NPTL cannot run on it.

I believe there are existing patches for LEON with CASA that
should just be merged, I would not worry about supporting broken
CPUs that never had this.

      Arnd

Re: [PATCH 3/4] sparc32: implement futex atomics, with casa detected at run time

[John Paul Adrian Glaubitz]

On Thu, 2026-10-15 at 11:23 +0200, Arnd Bergmann wrote:
> On Thu, Oct 15, 2026, at 10:55, Ramin Moussavi wrote:
> > sparc32 has no futex_atomic_cmpxchg_inatomic() or
> > arch_futex_atomic_op_inuser() of its own. The generic versions only
> > work on UP, so FUTEX depends on !(SPARC32 && SMP) and an SMP kernel
> > has no futexes at all; NPTL cannot run on it.
> 
> I believe there are existing patches for LEON with CASA that
> should just be merged, I would not worry about supporting broken
> CPUs that never had this.

I think you mean the series by Imre Kaloz [1]? If yes, he actually
implements CASA emulation unless I'm missing something.

Adrian

> [1] https://lore.kernel.org/sparclinux/20260927005821.8369-1-kaloz@kernel.org/

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: [PATCH 4/4] sparc32: add the sparc_atomic_cmpxchg_32 system call

[John Paul Adrian Glaubitz]

On Thu, 2026-10-15 at 11:21 +0200, Arnd Bergmann wrote:
> On Thu, Oct 15, 2026, at 10:55, Ramin Moussavi wrote:
> > User space built for v8 has no compare-and-swap. C libraries emulate it
> > with ldstub locks of their own, and those have two problems the kernel
> > cannot see. They are not async-signal-safe: a handler that reaches an
> > atomic while the code it interrupted holds the lock spins forever. And
> > the kernel does not know them: FUTEX_WAKE_OP and robust futexes write
> > the same words without taking those locks.
> 
> As far as I know, the baseline in sparc32 glibc has been leon3 or
> sparcv9 for a long time, so CASA should always be available.
> 
> Any remaining sun4m users are on an ancient userspace with a
> pre-futex glibc.

There are several contributors that are trying to revive Linux on pre-Leon
32-bit SPARC machines. Recently, Magnus Lindholm made some contributions
to be able to boot a modern kernel on his old SuperSPARC workstation [1].

The changes have already been merged into Andreas' tree for linux-next [2].

Adrian

> [1] https://lore.kernel.org/all/20261007174128.959307-1-linmag7@gmail.com/
> [2] https://git.kernel.org/pub/scm/linux/kernel/git/alarsson/linux-sparc.git/log/?h=for-next

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: [PATCH 0/4] sparc32: SMP fixes, futexes on SMP, and a cmpxchg system call

[Magnus Lindholm]

Hi Ramin,

On Thu, Oct 15, 2026 at 11:07 AM Ramin Moussavi
<ramin.moussavi@yacoub.de> wrote:
>
> This series makes NPTL usable on sparc32 SMP kernels. It started with
> the uClibc-ng test suite on qemu's SS-20 with -smp 4: the kernel went to
> error state under console load, sometimes hung at boot, and had no
> futexes at all.
>
> Patches 1 and 2 are fixes and do not depend on the rest:
>
>   1. The per-cpu timer interrupts on sun4m/sun4d and the LEON IPI leave
>      their handler with traps enabled at the PIL of the interrupted
>      code while the trap frame is still on the stack, so interrupts can
>      nest without bound until the CPU goes to error state. The generic
>      path was fixed in 2.1.129; these three exits never got the change.
>
>   2. A secondary CPU gets 10000 rounds of udelay(200) to call in. Under
>      qemu the delay loop calibration depends on host load, the wait can
>      shrink to a few milliseconds, and a CPU that calls in late blocks
>      RCU and hangs the boot. Wait by jiffies instead.
>
> Both are marked for stable but have no Fixes: tag. The sun4m and sun4d
> code they change is already in 2.6.12-rc2, the start of the git history;
> the LEON variants were copied from it in 2011.
>
> Patches 3 and 4 add what user space needs on SMP:
>
>   3. Futex atomics for sparc32, so FUTEX no longer depends on !SMP.
>      casa is probed at boot (LEON); without it the word is updated
>      under the hashed spinlock the kernel already uses for its atomics.
>
>   4. sparc_atomic_cmpxchg_32, a compare-and-exchange system call for
>      user space built for v8, which has no CAS of its own. The ldstub
>      emulation C libraries use instead is not async-signal-safe and is
>      invisible to the futex code. It takes the same path as the futex
>      atomics, like atomic_cmpxchg_32 does on m68k. The uClibc-ng side
>      that uses it is ready and waits for this.
>
> Tested on next-20261014 under qemu 11.1.0: the uClibc-ng suite on SS-20
> with -smp 4 and UP and on leon3_generic with -smp 4, plus a test
> program for the system call; the details are in each patch. sun4d was
> only built, qemu has no sun4d machine.
>
> Testing under qemu
> ------------------
>
> Three qemu problems get in the way of reproducing this, none of them
> caused by the series:
>
> - leon3_generic does not boot an image passed with -kernel since qemu
>   9.0: the synthetic bootloader no longer sets %sp (92688a91c358 took
>   it out of the reset handler). Gaisler's own prebuilt images fail the
>   same way.
>
> - leon3_generic does not bring up a second CPU under Linux. The IRQMP
>   model never sets the broadcast-available bit, and Linux stops with a
>   BUG in leon_smp.c; with that bit set, the secondary CPU dies on the
>   swapa with ASI 0x01 (forced cache miss) in leon_cpu_pre_online(),
>   which qemu does not decode.
>
> - On sun4m, qemu's ESCC drives only the interrupt line of the channel
>   it was called for; the board ORs both lines, and a stale line keeps
>   level 12 asserted. Boots then hang at random with an RCU stall about
>   20 seconds in (4 of 40 boots with -smp 4 here). That looks like a
>   problem with this series but is not.
>
> The LEON SMP results were taken with these fixed in qemu; the fixes are
> going to be reported to qemu.
>
> These patches were developed with the help of Claude (Anthropic), as
> the Assisted-by tags say. I have reviewed them and tested them as
> described.
>

Thanks for the series.

For the futex/CAS patches, have you seen Imre Kaloz’s earlier series,
“[PATCH 0/3] sparc32: SMP futexes, casa and idiv emulation”?
https://lore.kernel.org/sparclinux/20260927005821.8369-1-kaloz@kernel.org/

There seems to be substantial overlap. His approach emulates CASA directly,
allowing binaries already using that instruction to work without a new
syscall ABI.
That seems to require fewer userspace changes for those binaries.
Could these efforts be coordinated?

Regards,
Magnus


> Ramin Moussavi (4):
>   sparc32: keep PIL raised until the IRQ frame is gone
>   sparc32: wait for secondary CPUs by time, not by loop count
>   sparc32: implement futex atomics, with casa detected at run time
>   sparc32: add the sparc_atomic_cmpxchg_32 system call
>
>  arch/sparc/include/asm/cmpxchg_32.h    |  4 ++
>  arch/sparc/include/asm/futex_32.h      | 72 +++++++++++++++++++++++++-
>  arch/sparc/kernel/entry.S              |  9 ++--
>  arch/sparc/kernel/leon_smp.c           |  8 ++-
>  arch/sparc/kernel/sun4d_smp.c          |  8 ++-
>  arch/sparc/kernel/sun4m_smp.c          |  8 ++-
>  arch/sparc/kernel/sys_sparc_32.c       | 35 +++++++++++++
>  arch/sparc/kernel/syscalls/syscall.tbl |  1 +
>  arch/sparc/kernel/systbls.h            |  1 +
>  arch/sparc/kernel/traps_32.c           | 13 ++++-
>  arch/sparc/lib/Makefile                |  3 +-
>  arch/sparc/lib/atomic32.c              | 53 +++++++++++++++++++
>  arch/sparc/lib/casa_32.S               | 30 +++++++++++
>  init/Kconfig                           |  1 -
>  14 files changed, 224 insertions(+), 22 deletions(-)
>  create mode 100644 arch/sparc/lib/casa_32.S
>
>
> base-commit: d419d682257d813f6cb7a28e42b35d0edf4ad038
> --
> 2.53.0
>
>

Re: [PATCH 3/4] sparc32: implement futex atomics, with casa detected at run time

[Arnd Bergmann]

On Thu, Oct 15, 2026, at 11:28, John Paul Adrian Glaubitz wrote:
> On Thu, 2026-10-15 at 11:23 +0200, Arnd Bergmann wrote:
>> On Thu, Oct 15, 2026, at 10:55, Ramin Moussavi wrote:
>> > sparc32 has no futex_atomic_cmpxchg_inatomic() or
>> > arch_futex_atomic_op_inuser() of its own. The generic versions only
>> > work on UP, so FUTEX depends on !(SPARC32 && SMP) and an SMP kernel
>> > has no futexes at all; NPTL cannot run on it.
>> 
>> I believe there are existing patches for LEON with CASA that
>> should just be merged, I would not worry about supporting broken
>> CPUs that never had this.
>
> I think you mean the series by Imre Kaloz [1]? If yes, he actually
> implements CASA emulation unless I'm missing something.

No, I had not seen that one, I was actually surprised that Andreas
never merged it as we had discussed this years ago when I looked
at the Leon downstream kernel. I see it is still part of his
downstream patches at [1], at least the versions from 4.9 to
the latest (6.13).

        Arnd

[1] https://download.gaisler.com/anonftp/linux/linux-2.6/kernel/

Re: [PATCH 1/4] sparc32: keep PIL raised until the IRQ frame is gone

[Magnus Lindholm]

Hi Ramin,

On Thu, Oct 15, 2026 at 10:57 AM Ramin Moussavi
<ramin.moussavi@yacoub.de> wrote:
>
> The per-cpu timer interrupts on sun4m and sun4d SMP, and the LEON SMP
> IPI, leave their handler with
>
>         wr      %l0, PSR_ET, %psr
>         WRITE_PAUSE
>         RESTORE_ALL
>
> %l0 is the PSR at trap time, so this turns traps back on at the PIL of
> the interrupted code, usually 0, while the trap frame is still on the
> stack. Any interrupt that is pending at that point is taken on top of
> the frame, and if it leaves through the same path it opens the same
> window one level deeper. A source that keeps firing makes the nesting
> unbounded: the kernel stack runs into its thread_info, w_saved and the
> task pointer get overwritten, and the next window trap stores through
> them with traps disabled. The CPU goes to error state.
>
> The generic path does not have this problem. real_irq_continue keeps
> PIL at 15 after handler_irq and lets ret_trap_entry restore the old PIL
> together with the rett. It was fixed that way in Linux 2.1.129
> (entry.S 1.153, November 1998):
>
>         -       wr      %l0, PSR_ET, %psr
>         +       or      %l0, PSR_PIL, %g2       ! restore PIL after handler_irq
>         +       wr      %g2, PSR_ET, %psr       ! keep ET up
>
> smp_ticker had been added in 2.1.36 with a copy of the old exit, and
> kept it through the rename to smp4m_ticker and the addition of
> smp4d_ticker in 2.1.97. smpleon_ipi copied the pattern in 2011. Do the
> same as real_irq_continue in all three.
>
> On hardware the window is a few instructions long and no interrupt
> source refills it that fast. qemu does: its ESCC finishes a transmit
> instantly, and since 8.2 it takes a pending interrupt right after rett
> (930f1865cc65 "target/sparc: Clear may_lookup for npc == DYNAMIC_PC").
> A burst of console output on an SS-20 with -smp 4 is enough. A trap
> ring buffer in trap_setup showed the level 12 serial interrupt taken
> over and over at smp4m_ticker+0x34, timer ticks nesting further down,
> and the frames sinking below the thread_info.
>
> Tested on qemu 11.1.0, SS-20, -smp 4, with a test that writes a few
> hundred lines to the serial console from a multithreaded process: the
> kernel went to error state in 4 of 4 boots before, and booted cleanly
> 8 of 8 times after. Two full runs of the uClibc-ng test suite on top
> then completed without a crash. LEON SMP was tested on qemu's
> leon3_generic with -smp 4, where a full run of the suite completed
> without a crash as well. sun4d gets the same one line change but was
> not run; qemu has no sun4d machine.
>
> Cc: stable@vger.kernel.org
> Assisted-by: Claude:claude-opus-5-5
> Signed-off-by: Ramin Moussavi <ramin.moussavi@yacoub.de>
> ---
>  arch/sparc/kernel/entry.S | 9 ++++++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
>
> diff --git a/arch/sparc/kernel/entry.S b/arch/sparc/kernel/entry.S
> index ea51ef52c952..87d03a58446e 100644
> --- a/arch/sparc/kernel/entry.S
> +++ b/arch/sparc/kernel/entry.S
> @@ -252,7 +252,8 @@ smp4m_ticker:
>         WRITE_PAUSE
>         call    smp4m_percpu_timer_interrupt
>          add    %sp, STACKFRAME_SZ, %o0
> -       wr      %l0, PSR_ET, %psr
> +       or      %l0, PSR_PIL, %g2       ! keep PIL up until the frame is gone
> +       wr      %g2, PSR_ET, %psr
>         WRITE_PAUSE
>         RESTORE_ALL
>
> @@ -353,7 +354,8 @@ smp4d_ticker:
>         WRITE_PAUSE
>         call    smp4d_percpu_timer_interrupt
>          add    %sp, STACKFRAME_SZ, %o0
> -       wr      %l0, PSR_ET, %psr
> +       or      %l0, PSR_PIL, %g2       ! keep PIL up until the frame is gone
> +       wr      %g2, PSR_ET, %psr
>         WRITE_PAUSE
>         RESTORE_ALL
>
> @@ -407,7 +409,8 @@ smpleon_ipi:
>         WRITE_PAUSE
>         call    leonsmp_ipi_interrupt
>          add    %sp, STACKFRAME_SZ, %o1 ! pt_regs
> -       wr      %l0, PSR_ET, %psr
> +       or      %l0, PSR_PIL, %g2       ! keep PIL up until the frame is gone
> +       wr      %g2, PSR_ET, %psr
>         WRITE_PAUSE
>         RESTORE_ALL
>
>
> base-commit: d419d682257d813f6cb7a28e42b35d0edf4ad038
> --
> 2.53.0
>
>

I plan to test this on my SMP SS-20 with HyperSPARC CPUs. I have seen
intermittent stalls and register-window-related failures, so this looks worth
investigating, although I cannot yet say whether they share this cause.

Regards,
Magnus

Re: [PATCH 1/4] sparc32: keep PIL raised until the IRQ frame is gone

[Andreas Larsson]

On 2026-10-15 10:55, Ramin Moussavi wrote:
> The per-cpu timer interrupts on sun4m and sun4d SMP, and the LEON SMP
> IPI, leave their handler with
> 
> 	wr	%l0, PSR_ET, %psr
> 	WRITE_PAUSE
> 	RESTORE_ALL
> 
> %l0 is the PSR at trap time, so this turns traps back on at the PIL of
> the interrupted code, usually 0, while the trap frame is still on the
> stack. Any interrupt that is pending at that point is taken on top of
> the frame, and if it leaves through the same path it opens the same
> window one level deeper. A source that keeps firing makes the nesting
> unbounded: the kernel stack runs into its thread_info, w_saved and the
> task pointer get overwritten, and the next window trap stores through
> them with traps disabled. The CPU goes to error state.
> 
> The generic path does not have this problem. real_irq_continue keeps
> PIL at 15 after handler_irq and lets ret_trap_entry restore the old PIL
> together with the rett. It was fixed that way in Linux 2.1.129
> (entry.S 1.153, November 1998):
> 
> 	-	wr	%l0, PSR_ET, %psr
> 	+	or	%l0, PSR_PIL, %g2	! restore PIL after handler_irq
> 	+	wr	%g2, PSR_ET, %psr	! keep ET up
> 
> smp_ticker had been added in 2.1.36 with a copy of the old exit, and
> kept it through the rename to smp4m_ticker and the addition of
> smp4d_ticker in 2.1.97. smpleon_ipi copied the pattern in 2011. Do the
> same as real_irq_continue in all three.
> 
> On hardware the window is a few instructions long and no interrupt
> source refills it that fast. qemu does: its ESCC finishes a transmit
> instantly, and since 8.2 it takes a pending interrupt right after rett
> (930f1865cc65 "target/sparc: Clear may_lookup for npc == DYNAMIC_PC").
> A burst of console output on an SS-20 with -smp 4 is enough. A trap
> ring buffer in trap_setup showed the level 12 serial interrupt taken
> over and over at smp4m_ticker+0x34, timer ticks nesting further down,
> and the frames sinking below the thread_info.
> 
> Tested on qemu 11.1.0, SS-20, -smp 4, with a test that writes a few
> hundred lines to the serial console from a multithreaded process: the
> kernel went to error state in 4 of 4 boots before, and booted cleanly
> 8 of 8 times after. Two full runs of the uClibc-ng test suite on top
> then completed without a crash. LEON SMP was tested on qemu's
> leon3_generic with -smp 4, where a full run of the suite completed
> without a crash as well. sun4d gets the same one line change but was
> not run; qemu has no sun4d machine.
> 
> Cc: stable@vger.kernel.org
> Assisted-by: Claude:claude-opus-5-5
> Signed-off-by: Ramin Moussavi <ramin.moussavi@yacoub.de>
> ---
>  arch/sparc/kernel/entry.S | 9 ++++++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/sparc/kernel/entry.S b/arch/sparc/kernel/entry.S
> index ea51ef52c952..87d03a58446e 100644
> --- a/arch/sparc/kernel/entry.S
> +++ b/arch/sparc/kernel/entry.S
> @@ -252,7 +252,8 @@ smp4m_ticker:
>  	WRITE_PAUSE
>  	call	smp4m_percpu_timer_interrupt
>  	 add	%sp, STACKFRAME_SZ, %o0
> -	wr	%l0, PSR_ET, %psr
> +	or	%l0, PSR_PIL, %g2	! keep PIL up until the frame is gone
> +	wr	%g2, PSR_ET, %psr
>  	WRITE_PAUSE
>  	RESTORE_ALL
>  
> @@ -353,7 +354,8 @@ smp4d_ticker:
>  	WRITE_PAUSE
>  	call	smp4d_percpu_timer_interrupt
>  	 add	%sp, STACKFRAME_SZ, %o0
> -	wr	%l0, PSR_ET, %psr
> +	or	%l0, PSR_PIL, %g2	! keep PIL up until the frame is gone
> +	wr	%g2, PSR_ET, %psr
>  	WRITE_PAUSE
>  	RESTORE_ALL
>  
> @@ -407,7 +409,8 @@ smpleon_ipi:
>  	WRITE_PAUSE
>  	call	leonsmp_ipi_interrupt
>  	 add	%sp, STACKFRAME_SZ, %o1 ! pt_regs
> -	wr	%l0, PSR_ET, %psr
> +	or	%l0, PSR_PIL, %g2	! keep PIL up until the frame is gone
> +	wr	%g2, PSR_ET, %psr
>  	WRITE_PAUSE
>  	RESTORE_ALL
>  
> 
> base-commit: d419d682257d813f6cb7a28e42b35d0edf4ad038

Hi Ramin,

Thanks for the patch! This patch however has the exact same change as 

https://lore.kernel.org/sparclinux/20260927005658.8242-1-kaloz@kernel.org/#r

that is not yet picked up but not forgotten.


Could you add a Tested-by: in that thread instead and describe on what
you have tested there. Feel free to link to this thread as a lore link
in your answer as your two writeups complement each other nicely.

Thanks,
Andreas

Re: [PATCH 2/4] sparc32: wait for secondary CPUs by time, not by loop count

[Ramin Moussavi]

On Thu, Oct 15, 2026, at 11:09, Arnd Bergmann wrote:
> The patch is not wrong, but having a broken udelay() is still problematic
> in other places, so it may be better to fix that instead.
>
> On most modern targets, we tend to just use the cycle counter if the
> frequency is known and avoid the delay loop calibration entirely.
> Any reason that sparc32 still needs the calibration?

It could probably be solved with the hardware timers, the ones
timer_cs already uses.

I agree that this would be the better fix, and I could try a udelay()
based on a hardware timer. But I only have qemu here, no real sparc32
hardware, so I would rather not change udelay() for real machines on
my own. If someone with hardware can test it, I would be glad to do it
as a separate series.

Until then I would keep this patch: it is small, does not depend on
udelay() being right, and fixes the boot hang for stable on qemu.

Regards,
Ramin

Re: [PATCH 0/4] sparc32: SMP fixes, futexes on SMP, and a cmpxchg system call

[Ramin Moussavi]

Hi Magnus,

On Thu, Oct 15, 2026 at 11:36 AM Magnus Lindholm wrote:
> For the futex/CAS patches, have you seen Imre Kaloz's earlier series,
> "[PATCH 0/3] sparc32: SMP futexes, casa and idiv emulation"?

thanks to you and Adrian for the pointer -- I had not seen Imre's
series, so we worked on the same problem in parallel. I have replied
in Imre's thread; the open point for me is how a C library can find
out whether the kernel emulates casa:
https://lore.kernel.org/sparclinux/20261015141508.3335785-1-ramin.moussavi@yacoub.de/

Patch 1 is the same as Imre's PIL fix, I have sent my Tested-by there.
Patch 2 does not depend on the rest.

Regards,
Ramin