From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail.zytor.com (terminus.zytor.com [198.137.202.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 978291D61A3 for ; Thu, 12 Feb 2026 16:54:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.136 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770915296; cv=none; b=J7nOij9BbfuHaHMeETP2I04PBmYm03vgTawtRX6fss2gqJL9IOkSbNoJ71xohMBcjLoRr4gDymGqIN1tP4NIh5HoWCKPCqVH3nHT6GxX0IWTsIRSjxzv3nh/u+vnspaRrLGLwaw5DOGYFQzXJTBuB8UHCNDgfYy2PmyatbyHnXg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770915296; c=relaxed/simple; bh=6dbL0iKlYHLblOb9tU4GIonWVFcZG4oqC0lpbULIczc=; h=Date:From:To:CC:Subject:In-Reply-To:References:Message-ID: MIME-Version:Content-Type; b=saOcHeoJt2/JljQMvc7tCbUWJCvvnDDzA65KrZ8tq+q7CQDxI3g1ltCT2wlqrkZ6/fbCj0kFzFaKdtTOqWIEpjufvFYZqTh5E0oO4VrQ1myVchhIjJpQp8PQRm41L29N7xDE0d2K83fshRR4mb7FD74iW1slh38oHgLcLgWRO0M= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com; spf=pass smtp.mailfrom=zytor.com; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b=BtHXVfpl; arc=none smtp.client-ip=198.137.202.136 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=zytor.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zytor.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=zytor.com header.i=@zytor.com header.b="BtHXVfpl" Received: from ehlo.thunderbird.net (c-76-133-66-138.hsd1.ca.comcast.net [76.133.66.138]) (authenticated bits=0) by mail.zytor.com (8.18.1/8.17.1) with ESMTPSA id 61CGsJjc340967 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Thu, 12 Feb 2026 08:54:20 -0800 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.zytor.com 61CGsJjc340967 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zytor.com; s=2026012301; t=1770915260; bh=9bvI3nyUIyt4YgOtKrxq/u6wKH6pfJ4HvaYDacjR/RQ=; h=Date:From:To:CC:Subject:In-Reply-To:References:From; b=BtHXVfpl1RVYgDwTIb1LcWGWMC36mBUMJ1YEhLeUJInXODbgxQGXmiTdCMDE+mXqW XV/dkSeu4Ij4KdZ9DwfpV9+4S5e3Nw3N4jChbE8CoPMXaBrTBdETLxNNycPf1Aem58 bX8QDOpquVsLKepjKA6rqHI8jGOUbI9TfTerTzf9rULHnQii/aTdZP4o72eA6nK07w hBrksiq9c9+MHHt3PXDP5+LergvVroYhrP6DR+DoYibEcX1tsNvYTVmB+y82FwhOxk QmqG0VJTnq84NuMowU16G/L+akmpFVZ8bs6+rMMSqwhVNB+yM0hf7DvwcwNLVKcdwm KanTS/7EwhLQg== Date: Thu, 12 Feb 2026 08:54:15 -0800 From: "H. Peter Anvin" To: =?ISO-8859-1?Q?J=FCrgen_Gro=DF?= , linux-kernel@vger.kernel.org, x86@kernel.org CC: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen Subject: =?US-ASCII?Q?Re=3A_=5BPATCH_v2_0/4=5D_x86/mtrr=3A_Allow_MTR?= =?US-ASCII?Q?R_updates_on_multiple_CPUs_in_parallel?= User-Agent: K-9 Mail for Android In-Reply-To: <6352ffa5-d044-4a20-b836-739968ad378f@suse.com> References: <20260130113625.599305-1-jgross@suse.com> <35527cf2-322f-410b-9942-5cba0956b237@suse.com> <50f830e0-f69a-4f8c-a041-6ab6a8ee1cd8@suse.com> <04FA5DE3-CD35-4D22-8300-8ADADF8E35C9@zytor.com> <6352ffa5-d044-4a20-b836-739968ad378f@suse.com> Message-ID: <38FC68A0-6651-4ECA-9DAA-C4148F65AEDB@zytor.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On February 12, 2026 8:24:18 AM PST, "J=C3=BCrgen Gro=C3=9F" wrote: >On 10=2E02=2E26 08:28, H=2E Peter Anvin wrote: >> On February 9, 2026 10:51:04 PM PST, "J=C3=BCrgen Gro=C3=9F" wrote: >>> On 09=2E02=2E26 19:37, H=2E Peter Anvin wrote: >>>> On February 9, 2026 1:12:59 AM PST, Juergen Gross = wrote: >>>>> Ping? >>>>>=20 >>>>> I'd really like to have this in 7=2E0, as it is fixing a real issue = on >>>>> some machines =2E=2E=2E >>>>>=20 >>>>>=20 >>>>> Juergen >>>>>=20 >>>>> On 30=2E01=2E26 12:36, Juergen Gross wrote: >>>>>> Today MTRR updates are serialized to not happen on multiple CPUs at= the >>>>>> same time, as the related coding is using global variables=2E >>>>>>=20 >>>>>> On huge machines with lots of CPUs this can result in problems, as = such >>>>>> updates are happening through stop_machine(), which will call the M= TRR >>>>>> update function with interrupts off on all CPUs at the same time=2E= The >>>>>> interrupts will be switched on only after the last CPU has finished >>>>>> the MTRR update=2E As the update is required to run in uncached mod= e, it >>>>>> can take easily several milliseconds on each CPU, resulting in the >>>>>> whole process to need several seconds=2E This in turn can cause the >>>>>> watchdog to trigger and to recognize a hard system lockup=2E >>>>>>=20 >>>>>> This series is changing the behavior by allowing the MTRR update to >>>>>> happen on all CPUs in parallel=2E >>>>>>=20 >>>>>> Changes in V2: >>>>>> - fix a function comment header in patch 2 >>>>>>=20 >>>>>> Juergen Gross (4): >>>>>> x86/mtrr: Move cache_enable() and cache_disable() to mtrr/gene= ric=2Ec >>>>>> x86/mtrr: Introduce MTRR work state structure >>>>>> x86/mtrr: Add a prepare_set hook to mtrr_ops >>>>>> x86/mtrr: Drop cache_disable_lock >>>>>>=20 >>>>>> arch/x86/include/asm/cacheinfo=2Eh | 2 - >>>>>> arch/x86/include/asm/mtrr=2Eh | 2 - >>>>>> arch/x86/kernel/cpu/cacheinfo=2Ec | 80 +---------------- >>>>>> arch/x86/kernel/cpu/mtrr/generic=2Ec | 139 ++++++++++++++++++++= ++++----- >>>>>> arch/x86/kernel/cpu/mtrr/mtrr=2Ec | 3 + >>>>>> arch/x86/kernel/cpu/mtrr/mtrr=2Eh | 2 + >>>>>> 6 files changed, 122 insertions(+), 106 deletions(-) >>>>>>=20 >>>>>=20 >>>>=20 >>>> First of all, what machines are even needing MTRR updates these days? >>>=20 >>> I'm not aware this machine really needed an update=2E >>>=20 >>>> This isn't a rhetorical question=2E It is important to understand wha= t the underlying problem is=2E >>>=20 >>> It just took several seconds for all CPUs to check if there is an upda= te >>> needed=2E It might be an issue with firmware, topology, whatever=2E It= happened >>> in a test doing 300 cold boots in a row after roughly 70 loop iteratio= ns, >>> always on one of the last CPUs=2E >>>=20 >>> The issue shows that there IS a potential problem with doing the MTRR >>> update one CPU after the other, instead just doing it in parallel (whi= ch >>> is the "official" recommendation anyway)=2E See the comment in >>> cache_disable()=2E And it isn't as if the fix would be very complicate= d=2E >>>=20 >>>=20 >>> Juergen >>=20 >> You are assuming that it won't break any fragile systems=2E I'm much mo= re concerned about why this is happening at all=2E > >I'm having a hard time seeing why my series would break fragile systems= =2E >Its not as if I would change anything regarding the handling on each >cpu=2E > >My main suspect why this is happening is the topology of the system >(8 socket NUMA machine), causing the uncached memory accesses to have a >rather high latency (multiple hops for accessing some memory), causing >each cpu to need some time for checking all MTRRs=2E > > >Juergen Please stop avoiding the issue, which is WHY is this happening AT ALL on a= recent production system=2E The fastest way to do anything is to not do it at all=2E=20 What do the logs look like, with sufficient verbosity, for one thing?