[PATCH] MIPS: mm: tlb-r4k: Uniquify TLB entries on init

[PATCH] MIPS: mm: tlb-r4k: Uniquify TLB entries on init

[Jiaxun Yang]

Hardware or bootloader will initialize TLB entries to any value, which
may collide with kernel's UNIQUE_ENTRYHI value. On MIPS microAptiv/M5150
family of cores this will trigger machine check exception and cause boot
failure. On M5150 simulation this could happen 7 times out of 1000 boots.

Replace local_flush_tlb_all() with r4k_tlb_uniquify() which probes each
TLB ENTRIHI unique value for collisions before it's written and overwrites
any conflicting entries with safe values before initializing the kernel's
UNIQUE_ENTRYHI pattern.

Cc: stable@kernel.org
Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
 arch/mips/mm/tlb-r4k.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 77 insertions(+), 1 deletion(-)

diff --git a/arch/mips/mm/tlb-r4k.c b/arch/mips/mm/tlb-r4k.c
index 76f3b9c0a9f0ce60c42e4a9ea8025e1283678bd1..6467d74d2949e98bcc35ab7c368b3c2ea342a6d3 100644
--- a/arch/mips/mm/tlb-r4k.c
+++ b/arch/mips/mm/tlb-r4k.c
@@ -508,6 +508,82 @@ static int __init set_ntlb(char *str)
 
 __setup("ntlb=", set_ntlb);
 
+/*
+ * This is used to set an absolutely safe entryhi value that will not
+ * collide with any existing TLB entries as well as other UNIQUE_ENTRYHI
+ * values.
+ */
+static unsigned long r4k_safe_entryhi(void)
+{
+	int entry = current_cpu_data.tlbsize;
+	int old_index;
+
+	old_index = read_c0_index();
+	while (entry >= 0) {
+		int idx;
+		unsigned long entryhi = UNIQUE_ENTRYHI(entry);
+
+		write_c0_entryhi(entryhi);
+		mtc0_tlbw_hazard();
+		tlb_probe();
+		tlb_probe_hazard();
+		idx = read_c0_index();
+		if (idx < 0) {
+			/* Unused value found */
+			write_c0_index(old_index);
+			mtc0_tlbw_hazard();
+			return entryhi;
+		}
+		entry++;
+	}
+
+	panic("No safe TLB EntryHi value found");
+	return 0;
+}
+
+/*
+ * Initialize all TLB entries with unique values.
+ */
+static void r4k_tlb_uniquify(void)
+{
+	int entry;
+
+	entry = num_wired_entries();
+
+	htw_stop();
+	write_c0_entrylo0(0);
+	write_c0_entrylo1(0);
+	while (entry < current_cpu_data.tlbsize) {
+		unsigned long entryhi;
+		int collision_idx;
+
+		entryhi = UNIQUE_ENTRYHI(entry);
+		write_c0_entryhi(entryhi);
+		mtc0_tlbw_hazard();
+		tlb_probe();
+		tlb_probe_hazard();
+
+		/* Check for possible collision */
+		collision_idx = read_c0_index();
+		if (collision_idx >= 0 && collision_idx != entry) {
+			/* Override collision entry with a safe value */
+			r4k_safe_entryhi();
+			mtc0_tlbw_hazard();
+			tlb_write_indexed();
+			tlbw_use_hazard();
+			/* Recover correputed entryhi */
+			write_c0_entryhi(entryhi);
+		}
+
+		write_c0_index(entry);
+		mtc0_tlbw_hazard();
+		tlb_write_indexed();
+		entry++;
+	}
+	tlbw_use_hazard();
+	htw_start();
+}
+
 /*
  * Configure TLB (for init or after a CPU has been powered off).
  */
@@ -547,7 +623,7 @@ static void r4k_tlb_configure(void)
 	temp_tlb_entry = current_cpu_data.tlbsize - 1;
 
 	/* From this point on the ARC firmware is dead.	 */
-	local_flush_tlb_all();
+	r4k_tlb_uniquify();
 
 	/* Did I tell you that ARC SUCKS?  */
 }

---
base-commit: 911483b25612c8bc32a706ba940738cc43299496
change-id: 20250605-tlb-fix-578bac7be546

Best regards,
-- 
Jiaxun Yang <jiaxun.yang@flygoat.com>

Re: [PATCH] MIPS: mm: tlb-r4k: Uniquify TLB entries on init

[Jiaxun Yang]

在2025年6月5日周四 上午11:02，Jiaxun Yang写道：
> Hardware or bootloader will initialize TLB entries to any value, which
> may collide with kernel's UNIQUE_ENTRYHI value. On MIPS microAptiv/M5150
> family of cores this will trigger machine check exception and cause boot
> failure. On M5150 simulation this could happen 7 times out of 1000 boots.
>
> Replace local_flush_tlb_all() with r4k_tlb_uniquify() which probes each
> TLB ENTRIHI unique value for collisions before it's written and overwrites
> any conflicting entries with safe values before initializing the kernel's
> UNIQUE_ENTRYHI pattern.

I just realised it might be easier to cycling ASID instead of address in
Entry Hi.

Will come try a v2.

Thanks
Jiaxun

[...]

-- 
- Jiaxun

Re: [PATCH] MIPS: mm: tlb-r4k: Uniquify TLB entries on init

[Maciej W. Rozycki]

On Thu, 5 Jun 2025, Jiaxun Yang wrote:

> +static unsigned long r4k_safe_entryhi(void)
> +{
> +	int entry = current_cpu_data.tlbsize;
> +	int old_index;
> +
> +	old_index = read_c0_index();
> +	while (entry >= 0) {
[...]
> +		entry++;
> +	}

 Hmm, how is it supposed to work: you start from say 48 and then iterate 
until 0x80000000 before giving up?  Also a signed overflow condition is UB 
pre-C23, so the compiler may well optimise the loop control away.

 How did you verify this code?

  Maciej

Re: [PATCH] MIPS: mm: tlb-r4k: Uniquify TLB entries on init

[Jiaxun Yang]

在2025年6月7日周六 上午3:48，Maciej W. Rozycki写道：
> On Thu, 5 Jun 2025, Jiaxun Yang wrote:
>
>> +static unsigned long r4k_safe_entryhi(void)
>> +{
>> +	int entry = current_cpu_data.tlbsize;
>> +	int old_index;
>> +
>> +	old_index = read_c0_index();
>> +	while (entry >= 0) {
> [...]
>> +		entry++;
>> +	}
>
>  Hmm, how is it supposed to work: you start from say 48 and then iterate 
> until 0x80000000 before giving up?  Also a signed overflow condition is UB 
> pre-C23, so the compiler may well optimise the loop control away.

Ah sorry I didn't realise it's a UB. However, this loop is likely to be
terminated very early as it should be fairly easily to find a free EntryHi
value with TLBP.

>
>  How did you verify this code?

Boot tested on Boston as well as M5150 simulation. I tried to manually setup a
TLB entry collision situation with Boot-MIPS.

>
>   Maciej

Thanks

-- 
- Jiaxun