[PATCH v2 0/2] sparc32: SuperSPARC SMP synchronization fixes

[PATCH v2 0/2] sparc32: SuperSPARC SMP synchronization fixes

[Magnus Lindholm]

Fill two gaps in the SuperSPARC/Viking SMP synchronization paths.

This series is based on the three sparc32 relocatable-kernel fixes which
honour and derive phys_base and advertise the relocatable image. It does
not include those prerequisite patches.

These patches can be found here:
Link: https://lore.kernel.org/sparclinux/20260816075141.3489194-1-linmag7@gmail.com/T/#t

The SuperSPARC Family User's Manual requires software to keep at most one
Demap operation in progress across the system. It also says that an MBus
system must ask every processor which can retain a stale translation to
perform its own local Demap [1, sections 8.5.3 and 9.8.2]. The Sun-4M
architecture specification likewise describes SRMMU flushing as local to
a module [2, section 7.1.4]. Patch 1 serializes each complete shootdown on
sun4m and invokes remote CPUs one at a time. sun4d already serializes its
Viking Demap operations.

SuperSPARC maintains I-cache coherence by snooping, so no remote processor
needs to be told about modified instructions. FLUSH is still required on the
processor that wrote them: the store buffer is not snooped, so FLUSH is what
pushes them into the coherent hierarchy [1, sections 7.4 and 10.2.5; 3,
section A.8.2]. Patch 2 supplies that local FLUSH on two Viking paths that
performed none at all, viking_flush_sig_insns() was an empty stub, and
flush_icache_range() was defined as do { } while (0).

Tested on a dual-CPU sun4m SPARCstation 20, TI SuperSPARC-II with
Viking/MXCC:

  - boot to multi-user with both CPUs online;
  - 200,000 concurrent mprotect iterations over a shared address space
    (patch 1);
  - 40,000 executable-code rewrites, each replacement executed on the
    writing CPU and on the other CPU, with each CPU taking a turn as the
    writer (patch 2).

An earlier revision measured a stale instruction on the first rewrite with
no flush at all, but that was on a kernel predating this series, where the
whole Viking icache path was inert. On a patched kernel the same control is
intermittent, so it is not offered as evidence for anything here; the
argument for the shape of patch 2 is in its changelog.

Changes in v2:

  - patch 2 no longer cross-calls remote CPUs. Hardware snooping maintains
    instruction-cache coherence on the other processors; both FLUSH
    operations are now local. Callers modifying live text must arrange
    safe execution during the update. A later cross-call cannot prevent an
    old instruction from executing before it arrives, and cannot by itself
    make an otherwise unsafe concurrent text modification safe.
  - patch 2 no longer routes Viking sig_insns through the SMP wrapper, so
    that path retains the local-only behaviour mainline already had.
  - patch 1 is unchanged. It is a TLB shootdown: MBus cache coherence does not
    make a peer discard a virtual TLB entry, so each processor must still
    demap its own.

[1] SuperSPARC Family STP1020 & STP1090 Series User's Manual,
    Revision 1.0, April 1994.
[2] Sun-4M System Architecture, Specification 950-1373-01,
    Revision 50, July 19, 1991.
[3] SuperSPARC II Addendum, Revision 1.3, December 1994.

Magnus Lindholm (2):
  sparc32: serialize SuperSPARC demap operations
  sparc32: synchronize SuperSPARC instruction updates

 arch/sparc/include/asm/cacheflush_32.h |   2 +-
 arch/sparc/mm/srmmu.c                  |  98 +++++++++++++++++++++++++
 arch/sparc/mm/viking.S                 |   5 +
 3 files changed, 104 insertions(+), 1 deletion(-)

-- 
2.43.0

[PATCH v2 1/2] sparc32: serialize SuperSPARC demap operations

[Magnus Lindholm]

SuperSPARC requires software to guarantee that only one Demap operation
is in progress across the system. On MBus, each processor which may
retain a stale translation must perform its own local Demap.

A lock around each local operation is not safe on sun4m. Cross-call
callbacks run at interrupt level 15, so they can interrupt a CPU even
while spin_lock_irqsave() protects its local Demap. If the callback then
takes the same lock, it deadlocks against the interrupted owner.

Serialize each complete shootdown at the initiating CPU instead. Invoke
remote CPUs one at a time with lock-free callbacks, perform the
initiator's local Demap last, and release the lock only after the entire
operation is complete.

sun4m_cross_call() waits for the target CPU to complete its callback.
Singleton cross-calls therefore serialize the actual Demap operations,
not merely their dispatch.

sun4d already serializes Viking TLB flushes around its broadcast Demap
operations, so this wrapper is needed only for sun4m.

This is independent of MBus cache coherence. MBus Level 2 and the MXCC
coherence protocol describe physical cache-block transactions; they do
not make an MBus peer discard a virtual TLB entry.

This follows SuperSPARC Family User's Manual sections 8.5.3 and 9.8.2;
Sun-4M System Architecture section 7.1.4; MBus Interface Specification,
Level 2 Overview; and MXCC Addendum section B.2, Multiprocessor Cache
Coherence Support.

Signed-off-by: Magnus Lindholm <linmag7@gmail.com>
---
 arch/sparc/mm/srmmu.c | 88 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)

diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c
index 444295c20b94..ea0cc3683ad2 100644
--- a/arch/sparc/mm/srmmu.c
+++ b/arch/sparc/mm/srmmu.c
@@ -1590,6 +1590,33 @@ static void smp_flush_tlb_all(void)
 	local_ops->tlb_all();
 }
 
+/* Serialize complete sun4m Viking shootdowns; remote callbacks must not lock. */
+static DEFINE_SPINLOCK(viking_tlb_lock);
+
+static void smp_viking_flush_tlb_xcall(void *func, unsigned long arg1,
+					  unsigned long arg2,
+					  unsigned long arg3)
+{
+	int cpu;
+
+	for_each_online_cpu(cpu) {
+		if (cpu == smp_processor_id())
+			continue;
+		sparc32_ipi_ops->cross_call(func, *cpumask_of(cpu),
+					    arg1, arg2, arg3, 0);
+	}
+}
+
+static void smp_viking_flush_tlb_all(void)
+{
+	unsigned long flags;
+
+	spin_lock_irqsave(&viking_tlb_lock, flags);
+	smp_viking_flush_tlb_xcall(local_ops->tlb_all, 0, 0, 0);
+	local_ops->tlb_all();
+	spin_unlock_irqrestore(&viking_tlb_lock, flags);
+}
+
 static bool any_other_mm_cpus(struct mm_struct *mm)
 {
 	return cpumask_any_but(mm_cpumask(mm), smp_processor_id()) < nr_cpu_ids;
@@ -1617,6 +1644,25 @@ static void smp_flush_tlb_mm(struct mm_struct *mm)
 	}
 }
 
+static void smp_viking_flush_tlb_mm(struct mm_struct *mm)
+{
+	unsigned long flags;
+
+	if (mm->context != NO_CONTEXT) {
+		spin_lock_irqsave(&viking_tlb_lock, flags);
+		if (any_other_mm_cpus(mm)) {
+			smp_viking_flush_tlb_xcall(local_ops->tlb_mm,
+						     (unsigned long)mm, 0, 0);
+			if (atomic_read(&mm->mm_users) == 1 &&
+			    current->active_mm == mm)
+				cpumask_copy(mm_cpumask(mm),
+					     cpumask_of(smp_processor_id()));
+		}
+		local_ops->tlb_mm(mm);
+		spin_unlock_irqrestore(&viking_tlb_lock, flags);
+	}
+}
+
 static void smp_flush_cache_range(struct vm_area_struct *vma,
 				  unsigned long start,
 				  unsigned long end)
@@ -1645,6 +1691,24 @@ static void smp_flush_tlb_range(struct vm_area_struct *vma,
 	}
 }
 
+static void smp_viking_flush_tlb_range(struct vm_area_struct *vma,
+				       unsigned long start,
+				       unsigned long end)
+{
+	struct mm_struct *mm = vma->vm_mm;
+	unsigned long flags;
+
+	if (mm->context != NO_CONTEXT) {
+		spin_lock_irqsave(&viking_tlb_lock, flags);
+		if (any_other_mm_cpus(mm))
+			smp_viking_flush_tlb_xcall(local_ops->tlb_range,
+						     (unsigned long)vma,
+						     start, end);
+		local_ops->tlb_range(vma, start, end);
+		spin_unlock_irqrestore(&viking_tlb_lock, flags);
+	}
+}
+
 static void smp_flush_cache_page(struct vm_area_struct *vma, unsigned long page)
 {
 	struct mm_struct *mm = vma->vm_mm;
@@ -1667,6 +1731,23 @@ static void smp_flush_tlb_page(struct vm_area_struct *vma, unsigned long page)
 	}
 }
 
+static void smp_viking_flush_tlb_page(struct vm_area_struct *vma,
+				      unsigned long page)
+{
+	struct mm_struct *mm = vma->vm_mm;
+	unsigned long flags;
+
+	if (mm->context != NO_CONTEXT) {
+		spin_lock_irqsave(&viking_tlb_lock, flags);
+		if (any_other_mm_cpus(mm))
+			smp_viking_flush_tlb_xcall(local_ops->tlb_page,
+						     (unsigned long)vma,
+						     page, 0);
+		local_ops->tlb_page(vma, page);
+		spin_unlock_irqrestore(&viking_tlb_lock, flags);
+	}
+}
+
 static void smp_flush_page_to_ram(unsigned long page)
 {
 	/* Current theory is that those who call this are the one's
@@ -1721,6 +1802,13 @@ void __init load_mmu(void)
 	}
 
 	if (poke_srmmu == poke_viking) {
+		if (sparc_cpu_model == sun4m) {
+			smp_cachetlb_ops.tlb_all = smp_viking_flush_tlb_all;
+			smp_cachetlb_ops.tlb_mm = smp_viking_flush_tlb_mm;
+			smp_cachetlb_ops.tlb_range = smp_viking_flush_tlb_range;
+			smp_cachetlb_ops.tlb_page = smp_viking_flush_tlb_page;
+		}
+
 		/* Avoid unnecessary cross calls. */
 		smp_cachetlb_ops.cache_all = local_ops->cache_all;
 		smp_cachetlb_ops.cache_mm = local_ops->cache_mm;
-- 
2.43.0

[PATCH v2 2/2] sparc32: synchronize SuperSPARC instruction updates

[Magnus Lindholm]

SuperSPARC keeps its instruction cache coherent by snooping bus
transactions, so no remote processor has to be told about modified
instructions. A FLUSH is still required on the processor that wrote them:
the store buffer is not snooped, so FLUSH is what pushes the new
instructions into the coherent hierarchy, and it clears that processor's
own pipeline and instruction buffer.

Two Viking paths performed no flush at all.

viking_flush_sig_insns() was an empty stub, so it provided no explicit
instruction-update synchronization after the kernel wrote a signal
trampoline. Implement it.

flush_icache_range() was defined as do { } while (0) on sparc32, so it
provided no explicit instruction-update synchronization after the module
loader or a kernel text-modification path wrote executable code. Implement
it for Viking.

Both operations remain local. Hardware snooping maintains instruction-cache
coherence on the other processors, so no remote FLUSH is needed for that
purpose. Retain the existing Viking bypass of the SMP sig_insns wrapper.

Safe execution during a text update is the caller's responsibility. For
example, the module loader calls flush_module_icache() before
complete_formation() and do_init_module(). KGDB normally requests a CPU
roundup before modifying text, but that mechanism has exceptions and a
timeout; it is not an unconditional guarantee that every other CPU is
parked. A later cross-call cannot prevent an old instruction from executing
before it arrives, and cannot by itself make an otherwise unsafe concurrent
text modification safe.

The manual is explicit that FLUSH is not scoped to the address given to it:
"No cached information is explicitly flushed by the instruction ... FLUSH
operations simply cause an exact synchronization of all pending activity"
(section 7.4). One FLUSH therefore covers however many words were written
before it, which is why the two-instruction trampoline needs only one.

This follows SuperSPARC Family User's Manual sections 7.4, Flush (IFLUSH),
and 10.2.5, Instruction Cache Consistency, and SuperSPARC II Addendum
section A.8.2, Store Buffer & Snoops.

Signed-off-by: Magnus Lindholm <linmag7@gmail.com>
---
 arch/sparc/include/asm/cacheflush_32.h |  2 +-
 arch/sparc/mm/srmmu.c                  | 10 ++++++++++
 arch/sparc/mm/viking.S                 |  5 +++++
 3 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/arch/sparc/include/asm/cacheflush_32.h b/arch/sparc/include/asm/cacheflush_32.h
index 9fee0ccfccb8..4249663efacc 100644
--- a/arch/sparc/include/asm/cacheflush_32.h
+++ b/arch/sparc/include/asm/cacheflush_32.h
@@ -15,7 +15,7 @@
 	sparc32_cachetlb_ops->cache_range(vma, start, end)
 #define flush_cache_page(vma,addr,pfn) \
 	sparc32_cachetlb_ops->cache_page(vma, addr)
-#define flush_icache_range(start, end)		do { } while (0)
+void flush_icache_range(unsigned long start, unsigned long end);
 
 #define copy_to_user_page(vma, page, vaddr, dst, src, len) \
 	do {							\
diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c
index ea0cc3683ad2..093c543b734e 100644
--- a/arch/sparc/mm/srmmu.c
+++ b/arch/sparc/mm/srmmu.c
@@ -1784,6 +1784,16 @@ static struct sparc32_cachetlb_ops smp_cachetlb_ops __ro_after_init = {
 };
 #endif
 
+void flush_icache_range(unsigned long start, unsigned long end)
+{
+	if (start >= end || poke_srmmu != poke_viking)
+		return;
+
+	/* One local FLUSH synchronizes all preceding instruction stores. */
+	__asm__ __volatile__("flush %0" : : "r" (start) : "memory");
+}
+EXPORT_SYMBOL(flush_icache_range);
+
 /* Load up routines and constants for sun4m and sun4d mmu */
 void __init load_mmu(void)
 {
diff --git a/arch/sparc/mm/viking.S b/arch/sparc/mm/viking.S
index 8b4e251bbba2..92c80426bca6 100644
--- a/arch/sparc/mm/viking.S
+++ b/arch/sparc/mm/viking.S
@@ -201,7 +201,12 @@ viking_flush_tlb_page:
 
 viking_flush_page_to_ram:
 viking_flush_page_for_dma:
+	retl
+	 nop
+
 viking_flush_sig_insns:
+	/* FLUSH is not address scoped here, so one covers both words. */
+	flush	%o1
 	retl
 	 nop
 
-- 
2.43.0