Re: [PATCH] mm: mmap: map MAP_STACK to VM_NOHUGEPAGE only if THP is enabled

[Lorenzo Stoakes <lorenzo.stoakes@oracle.com>]

On Fri, May 02, 2025 at 03:16:52PM +0100, Matthew Wilcox wrote:
> On Fri, May 02, 2025 at 02:12:16PM +0100, Lorenzo Stoakes wrote:
> > On Fri, May 02, 2025 at 02:03:48PM +0100, Matthew Wilcox wrote:
> > > On Fri, May 02, 2025 at 11:31:41AM +0200, Ignacio Moreno Gonzalez via B4 Relay wrote:
> > > > From: Ignacio Moreno Gonzalez <Ignacio.MorenoGonzalez@kuka.com>
> > > >
> > > > commit c4608d1bf7c6 ("mm: mmap: map MAP_STACK to VM_NOHUGEPAGE") maps
> > > > the mmap option MAP_STACK to VM_NOHUGEPAGE. This is also done if
> > > > CONFIG_TRANSPARENT_HUGETABLES is not defined. But in that case, the
> > > > VM_NOHUGEPAGE does not make sense. For instance, when calling madvise()
> > > > with MADV_NOHUGEPAGE, an error is always returned.
> > >
> > > Isn't that the real problem though?
> >
> > Hmm, but wouldn't we want users who are trying to set MADV_[NO]HUGEPAGE to
> > be made aware that it isn't going to do anything?
>
> ... I thought the patch was clear.  Only setting NOHUGEPAGE becomes a
> no-op.  Setting HUGEPAGE remains EINVAL.

Sorry I"m an idiot, glossed over the 'if (advice == MADV_NOHUGEPAGE)'
bit...

>
> > And wouldn't changing this be a possibly 'breaking userspace' thing if
> > somebody somewhere relies on this?
>
> I don't see what userspace could rely on it returning EINVAL, since it
> won't on a kernel which has THP enabled.

I mean for the purposes of detecting THP being disabled, but I mean it's
far-fetched.

>
> > Also this will make this inconsistent with e.g. MADV_COLLAPSE also?
>
> Not sure how ...
>

Yup this was based on my misreading...

But I'm still not a fan of us setting the VM_NOHUGEPAGE flag like this when
it's meaningless and this is (probably) the only way you can do it on a
non-THP kernel.

It's user-visible in /proc/$pid/smaps (I mean, that's the whole problem
here right?) and impacts merging (though MAP_STACK you'd think it'd
probably not be a big deal for...)

Or ifdef the VM_[NO]HUGEPAGE on CONFIG_TRANSPARENT_HUGEPAGE?

So maybe, let's do both things?