Re: [PATCH] x86: Drop KGDB support

[Steve Wahl <steve.wahl@hpe.com>]

On Fri, Oct 30, 2026 at 10:31:46PM +0000, David Laight wrote:
> On Fri, 30 Oct 2026 15:56:48 -0500
> Steve Wahl <steve.wahl@hpe.com> wrote:
> 
> > On Fri, Oct 30, 2026 at 04:28:41PM +0000, David Laight wrote:
> > > On Thu, 29 Oct 2026 10:32:58 -0500
> > > Steve Wahl <steve.wahl@hpe.com> wrote:
> > >   
> > > > On Thu, Oct 29, 2026 at 10:51:21AM +0000, David Laight wrote:  
> > > > > On Wed, 28 Oct 2026 14:58:12 -0500
> > > > > Steve Wahl <steve.wahl@hpe.com> wrote:
> > > > >     
> > > > > > On Wed, Oct 28, 2026 at 08:10:54AM -0700, Dave Hansen wrote:    
> > > > > ...    
> > > > > > 
> > > > > > For an example, I recently used this method to discover that at
> > > > > > least one version of Intel's MPI library likes to read up
> > > > > > /proc/kallsyms from each of the processes it starts, and somewhere
> > > > > > north of 1000's of processors the atomic increment on opening
> > > > > > kallsyms, implemented with a compare-exchange, becomes a hot lock
> > > > > > and initialization that takes little time on a smaller number of
> > > > > > processors, suddenly balloons to half an hour.    
> > > > > 
> > > > > I'm guessing that was pre-ARMv8.1 ??    
> > > > 
> > > > x86, not arm.  And it was on a distro kernel, I didn't check the
> > > > current community kernel because it was a situation where it was
> > > > enough to understand what's going on in the distro kernel, and avoid
> > > > the behavior in the application.   
> > > 
> > > x86 shouldn't have an exponential issue with cas.
> > > Something else must have been going on.  
> > 
> > Well, the large majority of some 2000+ cpus were all stuck in
> > use_pde() in inode.c, which is
> > 
> > 	return likely(atomic_inc_unless_negative(&pde->in_use));
> 
> There will be one of those per read system call.
> If the code is using a sane buffer length that isn't really that many at all
> (unless you are running an 'allyesconfig' kernel).
> Provided the library isn't doing something really silly like 1 byte reads!

I don't have the backtraces saved.  For whatever reason, I'm left with
the impression that the processes I examined closely were all in the
open call, not the read call.  But you're correct, it looks like reads
would sit on this too.  That makes even more sense that it got stuck
as long as it did.

> It is possible that /proc/kallsyms could be marked PROC_ENTRY_PERMANENT
> and so avoid that atomic operation.

Maybe.  That seems like a specific solution to a general problem.
Even if /proc/kallsyms is fixed, there will be others that are not.

And the right place to fix this is much in question.  Is there a good
reason to read /proc/kallsyms 1000s of times?  I doubt it.  We
switched to a different MPI library that does not have this problem.

(I did not take this to the community for a number of reasons.  I
didn't have time to make sure I could reproduce with the latest
version of the library, with the community kernel instead of the
distro kernel. I don't have a good explanation of why the library
would want to do it in the first place.  We have a workaround for the
time being.)

My point was more that they're taking away the tool that I used to
figure this out.

> > 
> > atomic_inc_unless_negative() has alternates, but I think there's no
> > arch version for x86, so comes down to this:
> > 
> > static __always_inline bool
> > raw_atomic_inc_unless_negative(atomic_t *v)
> > {
> > 	int c = raw_atomic_read(v);
> > 
> > 	do {
> > 		if (unlikely(c < 0))
> > 			return false;
> > 	} while (!raw_atomic_try_cmpxchg(v, &c, c + 1));
> > 
> > 	return true;
> > }
> > 
> > If you manage to synchronize thousands of processors among many NUMA
> > nodes all trying to do that same thing, how can that not create a
> > horrible cache coherency protocol traffic jam?
> 
> On x86, in order for the cmpxchg to fail it must have succeeded for
> a different cpu (as far as I understand things...).
> So yes, you get a lot of cache coherency traffic but progress should
> be made - probably with a maximum total rate of increments that you
> hit well before 1000 cpu hit the code path.
> 
> It might just be that with the large NUMA memory system the required number
> of cache line bounces (one for each successful CAS) just takes that length
> of time - but that is linear in the number of applications/cpus. 

I think I disagree with the "one bounce for each successful CAS"
assertion; I think it's one bounce for each CAS, successful or not.

If there's any mechanism like a short term hold on the cache line
before it will relinquish ownership after a failure, perhaps in this
case the extra "unless negative" instructions are making the loop too
long?  I don't know.

Thanks for your insight,

--> Steve Wahl

> OTOH an LL/SC loop loses the cache when another cpu does the LL (aka read)
> bit. So when heavily contended no cpu manages to do a successful write
> and overall absolutely no progress is made.
> The more instructions between the LL and SC the worse it gets.
> 
> David
> 
> > 
> > Thanks for your thoughts on this; if our hardware is supposed to be
> > doing something to alleviate this and it's not, I / we want to know!
> > 
> > --> Steve  
> > 
> > > David
> > >   
> > > >   
> > > > > The LL/SC retry loops for atomics are completely broken for contended
> > > > > operations.
> > > > > The kernel needs to patch them to use the LSE (single instruction
> > > > > atomics) for ARMv8.1 which guarantee that some progress is made.
> > > > > A CAS loop can still get starved on a single CPU, but some CPU
> > > > > do proceed.
> > > > > You really don't want to use LL/LC on a system with hraair CPU.
> > > > > 
> > > > > One CPU (not arm) that implements LL/SC locks the cache line for a
> > > > > few clocks after LL to give a reasonable chance of the associated SC
> > > > > actually succeeding before the cache line is 'stolen' by another CPU
> > > > > doing an LL.    
> > > > 
> > > > Thanks,
> > > >   
> > > > --> Steve    
> > > >   
> > >   
> > 
> 

-- 
Steve Wahl, Hewlett Packard Enterprise