[PATCH v3] mm/memory: use pmdp_get_lockless() for the pmd read in do_fault_around()

[PATCH v3] mm/memory: use pmdp_get_lockless() for the pmd read in do_fault_around()

[Hui Zhu]

From: Hui Zhu <zhuhui@kylinos.cn>

KCSAN reports a data-race in do_read_fault / pmd_install; the same code
is in mainline:

  write (marked) to 0xffff8d52a75fb810 of 8 bytes by task 27487 on cpu 5:
   native_set_pmd arch/x86/include/asm/pgtable_64.h:83 [inline]
   pmd_populate arch/x86/include/asm/pgalloc.h:84 [inline]
   pmd_install+0xe0/0x120 mm/memory.c:444
   finish_fault+0x5bc/0xb00 mm/memory.c:5204
   do_cow_fault mm/memory.c:5432 [inline]
   do_fault+0x5d0/0x6e0 mm/memory.c:5526
   handle_mm_fault+0x211/0x4c0 mm/memory.c:6218
   __mm_populate+0x144/0x270 mm/gup.c:2207
   __do_sys_mlockall+0x2cd/0x380 mm/mlock.c:764

  read to 0xffff8d52a75fb810 of 8 bytes by task 27527 on cpu 9:
   do_fault_around mm/memory.c:5341 [inline]
   do_read_fault+0x162/0x670 mm/memory.c:5381
   do_fault+0x31a/0x6e0 mm/memory.c:5524
   handle_mm_fault+0x211/0x4c0 mm/memory.c:6218
   faultin_page_range+0x227/0x530 mm/gup.c:2156
   __x64_sys_madvise+0x67/0x80 mm/madvise.c:1479

  value changed: 0x0000000000000000 -> 0x000000001386c067

Both paths only hold mmap_lock for read, so do_fault_around() tests
vmf->pmd while pmd_install() populates it under the pmd lock.  The race is
benign: a stale pmd_none() only wastes the prealloc_pte, which do_fault()
frees again, and a pmd seen as populated only makes ->map_pages() find no
PTE and do_read_fault() fall back to __do_fault().

The load still has to be lockless-safe.  pmdp_get() is a plain READ_ONCE()
and can tear where pmd_t is wider than a word, as on i386-PAE where
pmd_install() publishes two words with a single cmpxchg8b.  Use
pmdp_get_lockless(), the accessor for pmd reads without the pmd lock and
what __pte_offset_map() already uses on this very pmd a few instructions
later.  It is pmdp_get() everywhere else, so the report is silenced the
same way.

No functional change on architectures that can load a pmd in one go.

Signed-off-by: Hui Zhu <zhuhui@kylinos.cn>
---
Changelog:
v3:
Shorten the comment to one line (David Hildenbrand).
v2:
Use pmdp_get_lockless() instead of pmdp_get() (David Hildenbrand).

 mm/memory.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/mm/memory.c b/mm/memory.c
index 0fb676dd2bd63..f25c2ce0ef153 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -6017,7 +6017,8 @@ static vm_fault_t do_fault_around(struct vm_fault *vmf)
 	to_pte = min3(from_pte + nr_pages, (pgoff_t)PTRS_PER_PTE,
 		      pte_off + vma_pages(vmf->vma) - vma_off) - 1;
 
-	if (pmd_none(*vmf->pmd)) {
+	/* pmd_install() rechecks pmd_none() under the pmd lock. */
+	if (pmd_none(pmdp_get_lockless(vmf->pmd))) {
 		vmf->prealloc_pte = pte_alloc_one(vmf->vma->vm_mm);
 		if (!vmf->prealloc_pte)
 			return VM_FAULT_OOM;
-- 
2.43.0

Re: [PATCH v3] mm/memory: use pmdp_get_lockless() for the pmd read in do_fault_around()

[David Hildenbrand (Arm)]

On 11/3/26 02:47, Hui Zhu wrote:
> From: Hui Zhu <zhuhui@kylinos.cn>
> 
> KCSAN reports a data-race in do_read_fault / pmd_install; the same code
> is in mainline:
> 
>   write (marked) to 0xffff8d52a75fb810 of 8 bytes by task 27487 on cpu 5:
>    native_set_pmd arch/x86/include/asm/pgtable_64.h:83 [inline]
>    pmd_populate arch/x86/include/asm/pgalloc.h:84 [inline]
>    pmd_install+0xe0/0x120 mm/memory.c:444
>    finish_fault+0x5bc/0xb00 mm/memory.c:5204
>    do_cow_fault mm/memory.c:5432 [inline]
>    do_fault+0x5d0/0x6e0 mm/memory.c:5526
>    handle_mm_fault+0x211/0x4c0 mm/memory.c:6218
>    __mm_populate+0x144/0x270 mm/gup.c:2207
>    __do_sys_mlockall+0x2cd/0x380 mm/mlock.c:764
> 
>   read to 0xffff8d52a75fb810 of 8 bytes by task 27527 on cpu 9:
>    do_fault_around mm/memory.c:5341 [inline]
>    do_read_fault+0x162/0x670 mm/memory.c:5381
>    do_fault+0x31a/0x6e0 mm/memory.c:5524
>    handle_mm_fault+0x211/0x4c0 mm/memory.c:6218
>    faultin_page_range+0x227/0x530 mm/gup.c:2156
>    __x64_sys_madvise+0x67/0x80 mm/madvise.c:1479
> 
>   value changed: 0x0000000000000000 -> 0x000000001386c067
> 
> Both paths only hold mmap_lock for read, so do_fault_around() tests
> vmf->pmd while pmd_install() populates it under the pmd lock.  The race is
> benign: a stale pmd_none() only wastes the prealloc_pte, which do_fault()
> frees again, and a pmd seen as populated only makes ->map_pages() find no
> PTE and do_read_fault() fall back to __do_fault().
> 
> The load still has to be lockless-safe.  pmdp_get() is a plain READ_ONCE()
> and can tear where pmd_t is wider than a word, as on i386-PAE where
> pmd_install() publishes two words with a single cmpxchg8b.  Use
> pmdp_get_lockless(), the accessor for pmd reads without the pmd lock and
> what __pte_offset_map() already uses on this very pmd a few instructions
> later.  It is pmdp_get() everywhere else, so the report is silenced the
> same way.
> 
> No functional change on architectures that can load a pmd in one go.
> 
> Signed-off-by: Hui Zhu <zhuhui@kylinos.cn>
> ---
> Changelog:
> v3:
> Shorten the comment to one line (David Hildenbrand).
> v2:
> Use pmdp_get_lockless() instead of pmdp_get() (David Hildenbrand).
> 
>  mm/memory.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/memory.c b/mm/memory.c
> index 0fb676dd2bd63..f25c2ce0ef153 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -6017,7 +6017,8 @@ static vm_fault_t do_fault_around(struct vm_fault *vmf)
>  	to_pte = min3(from_pte + nr_pages, (pgoff_t)PTRS_PER_PTE,
>  		      pte_off + vma_pages(vmf->vma) - vma_off) - 1;
>  
> -	if (pmd_none(*vmf->pmd)) {
> +	/* pmd_install() rechecks pmd_none() under the pmd lock. */
> +	if (pmd_none(pmdp_get_lockless(vmf->pmd))) {
>  		vmf->prealloc_pte = pte_alloc_one(vmf->vma->vm_mm);
>  		if (!vmf->prealloc_pte)
>  			return VM_FAULT_OOM;

I'll wait for for review and then pick it up.

-- 
Cheers,

David

Re: [PATCH v3] mm/memory: use pmdp_get_lockless() for the pmd read in do_fault_around()

[Lorenzo Stoakes (ARM)]

On Tue, Nov 03, 2026 at 09:47:49AM +0800, Hui Zhu wrote:
> From: Hui Zhu <zhuhui@kylinos.cn>
>
> KCSAN reports a data-race in do_read_fault / pmd_install; the same code
> is in mainline:
>
>   write (marked) to 0xffff8d52a75fb810 of 8 bytes by task 27487 on cpu 5:
>    native_set_pmd arch/x86/include/asm/pgtable_64.h:83 [inline]
>    pmd_populate arch/x86/include/asm/pgalloc.h:84 [inline]
>    pmd_install+0xe0/0x120 mm/memory.c:444
>    finish_fault+0x5bc/0xb00 mm/memory.c:5204
>    do_cow_fault mm/memory.c:5432 [inline]
>    do_fault+0x5d0/0x6e0 mm/memory.c:5526
>    handle_mm_fault+0x211/0x4c0 mm/memory.c:6218
>    __mm_populate+0x144/0x270 mm/gup.c:2207
>    __do_sys_mlockall+0x2cd/0x380 mm/mlock.c:764
>
>   read to 0xffff8d52a75fb810 of 8 bytes by task 27527 on cpu 9:
>    do_fault_around mm/memory.c:5341 [inline]
>    do_read_fault+0x162/0x670 mm/memory.c:5381
>    do_fault+0x31a/0x6e0 mm/memory.c:5524
>    handle_mm_fault+0x211/0x4c0 mm/memory.c:6218
>    faultin_page_range+0x227/0x530 mm/gup.c:2156
>    __x64_sys_madvise+0x67/0x80 mm/madvise.c:1479
>
>   value changed: 0x0000000000000000 -> 0x000000001386c067
>
> Both paths only hold mmap_lock for read, so do_fault_around() tests
> vmf->pmd while pmd_install() populates it under the pmd lock.  The race is
> benign: a stale pmd_none() only wastes the prealloc_pte, which do_fault()
> frees again, and a pmd seen as populated only makes ->map_pages() find no
> PTE and do_read_fault() fall back to __do_fault().
>
> The load still has to be lockless-safe.  pmdp_get() is a plain READ_ONCE()
> and can tear where pmd_t is wider than a word, as on i386-PAE where

Isn't i386 no longer supported? Are there any other arches we are worried
about?

It seems now after Arnd's removals this has to be manually selected by the
user.

Other than that we are worried about mips32 with 64 bit addresses and sh
with x2tlb. More museum-piece stuff...

I hate that we're impacted by this.

Anyway you should probably call out the relevant arches here.

> pmd_install() publishes two words with a single cmpxchg8b.  Use
> pmdp_get_lockless(), the accessor for pmd reads without the pmd lock and
> what __pte_offset_map() already uses on this very pmd a few instructions
> later.  It is pmdp_get() everywhere else, so the report is silenced the
> same way.
>
> No functional change on architectures that can load a pmd in one go.
>
> Signed-off-by: Hui Zhu <zhuhui@kylinos.cn>


> ---
> Changelog:
> v3:
> Shorten the comment to one line (David Hildenbrand).
> v2:
> Use pmdp_get_lockless() instead of pmdp_get() (David Hildenbrand).

Please add links to previous discussion, it's difficult to track stuff
otherwise.

>
>  mm/memory.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/mm/memory.c b/mm/memory.c
> index 0fb676dd2bd63..f25c2ce0ef153 100644
> --- a/mm/memory.c
> +++ b/mm/memory.c
> @@ -6017,7 +6017,8 @@ static vm_fault_t do_fault_around(struct vm_fault *vmf)
>  	to_pte = min3(from_pte + nr_pages, (pgoff_t)PTRS_PER_PTE,
>  		      pte_off + vma_pages(vmf->vma) - vma_off) - 1;
>
> -	if (pmd_none(*vmf->pmd)) {

This pattern is definitely bad I think, we have it in many other places so
should probably convert those to pmdp_get() at least?

> +	/* pmd_install() rechecks pmd_none() under the pmd lock. */
> +	if (pmd_none(pmdp_get_lockless(vmf->pmd))) {

It definitely seems right to use at least pmdp_get() here, but I wonder
about the need for pmdp_get_lockless() here.

pmd_none() only checks the lower word, so a torn 64-bit word on 32-bit
(i.e. the PAE style cases we are worried about) makes no difference:

From x86 pmd_none():

	/* Only check low word on 32-bit platforms, since it might be
	   out of sync with upper half. */

And anyway I guess there is no valid PMD in such systems that would have
the lower word cleared and upper word not cleared.

And we have other instances of this pattern all over the place, when you
read the PMD you don't hold the PMD lock at all so 'lockless' is kinda
questionable here.

I mean yes it's lockless but no more than other cases like this.

So either I think use pmdp_get() or we need to go convert everywhere else
and be strict on pmdp_get() only being used with the PMD lock held.

>  		vmf->prealloc_pte = pte_alloc_one(vmf->vma->vm_mm);
>  		if (!vmf->prealloc_pte)
>  			return VM_FAULT_OOM;
> --
> 2.43.0
>

--
Cheers, Lorenzo

Re: [PATCH v3] mm/memory: use pmdp_get_lockless() for the pmd read in do_fault_around()

[David Hildenbrand (Arm)]

>> +++ b/mm/memory.c
>> @@ -6017,7 +6017,8 @@ static vm_fault_t do_fault_around(struct vm_fault *vmf)
>>  	to_pte = min3(from_pte + nr_pages, (pgoff_t)PTRS_PER_PTE,
>>  		      pte_off + vma_pages(vmf->vma) - vma_off) - 1;
>>
>> -	if (pmd_none(*vmf->pmd)) {
> 
> This pattern is definitely bad I think, we have it in many other places so
> should probably convert those to pmdp_get() at least?

Usama is working on this as part of separating HW and SW PMDs (he only sent out
the PTE part so far).

> 
>> +	/* pmd_install() rechecks pmd_none() under the pmd lock. */
>> +	if (pmd_none(pmdp_get_lockless(vmf->pmd))) {
> 
> It definitely seems right to use at least pmdp_get() here, but I wonder
> about the need for pmdp_get_lockless() here.

Whenever we're not holding the page table lock we should be using
pmdp_get_lockless().

Even if they are semantically equivalent for most cases, it serves as a good
reminder that there are no guarantees.

And that you have to look again under PT lock.

> 
> pmd_none() only checks the lower word, so a torn 64-bit word on 32-bit
> (i.e. the PAE style cases we are worried about) makes no difference:
> 
> From x86 pmd_none():
> 
> 	/* Only check low word on 32-bit platforms, since it might be
> 	   out of sync with upper half. */
> 
> And anyway I guess there is no valid PMD in such systems that would have
> the lower word cleared and upper word not cleared.
> 

I don't understand the discussion really.

No PT lock -> use the lockless variant.

Until we can remove the lockless variant.

Are we consistent with that? Likely not.

But that's all independent of this patch here.

-- 
Cheers,

David

Re: [PATCH v3] mm/memory: use pmdp_get_lockless() for the pmd read in do_fault_around()

[Lorenzo Stoakes (ARM)]

On Tue, Nov 03, 2026 at 05:15:27PM +0100, David Hildenbrand (Arm) wrote:
>
> >> +++ b/mm/memory.c
> >> @@ -6017,7 +6017,8 @@ static vm_fault_t do_fault_around(struct vm_fault *vmf)
> >>  	to_pte = min3(from_pte + nr_pages, (pgoff_t)PTRS_PER_PTE,
> >>  		      pte_off + vma_pages(vmf->vma) - vma_off) - 1;
> >>
> >> -	if (pmd_none(*vmf->pmd)) {
> >
> > This pattern is definitely bad I think, we have it in many other places so
> > should probably convert those to pmdp_get() at least?
>
> Usama is working on this as part of separating HW and SW PMDs (he only sent out
> the PTE part so far).
>
> >
> >> +	/* pmd_install() rechecks pmd_none() under the pmd lock. */
> >> +	if (pmd_none(pmdp_get_lockless(vmf->pmd))) {
> >
> > It definitely seems right to use at least pmdp_get() here, but I wonder
> > about the need for pmdp_get_lockless() here.
>
> Whenever we're not holding the page table lock we should be using
> pmdp_get_lockless().
>
> Even if they are semantically equivalent for most cases, it serves as a good
> reminder that there are no guarantees.
>
> And that you have to look again under PT lock.
>
> >
> > pmd_none() only checks the lower word, so a torn 64-bit word on 32-bit
> > (i.e. the PAE style cases we are worried about) makes no difference:
> >
> > From x86 pmd_none():
> >
> > 	/* Only check low word on 32-bit platforms, since it might be
> > 	   out of sync with upper half. */
> >
> > And anyway I guess there is no valid PMD in such systems that would have
> > the lower word cleared and upper word not cleared.
> >
>
> I don't understand the discussion really.
>
> No PT lock -> use the lockless variant.
>
> Until we can remove the lockless variant.
>
> Are we consistent with that? Likely not.
>
> But that's all independent of this patch here.

It turns out all of this is a complicated mess, I think I've nerd-sniped myself
into sending a series at some point about this...

It turns out we're as wildly inconsistent on pmdp_get_lockless() as we are on
pmdp_get(), so TL;DR my main concern is moot so:

Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>

But for the non-TL;DR... (optional reading, will probably send something):

* Agreed that the semantics matter - and the 'use pXXp_get_lockless() if no
  locks, otherwise pXXp_get()' is preferable.

* But it turns out it's all a giant mess (of course)... for
  CONFIG_GUP_GET_PXX_LOW_HIGH arches you really actually need to also do:

pmdp_get_lockless_start();
pmd = pmdp_get_lockless(...);
pmdp_get_lockless_end();

And that's because, as Hugh points out in commit 146b42e07494 ("mm/pgtable: add
PAE safety to __pte_offset_map()")), you need IRQs disabled for these arches for
it to be safe.

The explanation is quite good:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=146b42e07494e45f7c7bcf2cbf7afd1424afd78e

And then of course there's also pmdp_get_lockless_sync() used by THP collapse
which is some kind of 'true' lockless usage that needs to synchronise with TLB
invalidation against GUP-fast and ugh.

* So I think what matters is whether you actually NEED the value un-torn or
  whether it's an optimistic check that you can live with (rechecked under lock
  as needed).

* So I wonder if we shouldn't just have e.g. pmdp_get_lockless_optimistic() or
  something like that which is just a pmdp_get() in all cases with correct kdoc
  description etc.

* Then we can convert ALL lockless users to using a lockless form without having
  to do the IRQ stuff. Including a ton of the existing lockless users.

* I am not sure why I did this in bullets. :)

--
Cheers, Lorenzo

Re: [PATCH v3] mm/memory: use pmdp_get_lockless() for the pmd read in do_fault_around()

[David Hildenbrand (Arm)]

On 11/3/26 19:24, Lorenzo Stoakes (ARM) wrote:
> On Tue, Nov 03, 2026 at 05:15:27PM +0100, David Hildenbrand (Arm) wrote:
>>
>>>
>>> This pattern is definitely bad I think, we have it in many other places so
>>> should probably convert those to pmdp_get() at least?
>>
>> Usama is working on this as part of separating HW and SW PMDs (he only sent out
>> the PTE part so far).
>>
>>>
>>>
>>> It definitely seems right to use at least pmdp_get() here, but I wonder
>>> about the need for pmdp_get_lockless() here.
>>
>> Whenever we're not holding the page table lock we should be using
>> pmdp_get_lockless().
>>
>> Even if they are semantically equivalent for most cases, it serves as a good
>> reminder that there are no guarantees.
>>
>> And that you have to look again under PT lock.
>>
>>>
>>> pmd_none() only checks the lower word, so a torn 64-bit word on 32-bit
>>> (i.e. the PAE style cases we are worried about) makes no difference:
>>>
>>> From x86 pmd_none():
>>>
>>> 	/* Only check low word on 32-bit platforms, since it might be
>>> 	   out of sync with upper half. */
>>>
>>> And anyway I guess there is no valid PMD in such systems that would have
>>> the lower word cleared and upper word not cleared.
>>>
>>
>> I don't understand the discussion really.
>>
>> No PT lock -> use the lockless variant.
>>
>> Until we can remove the lockless variant.
>>
>> Are we consistent with that? Likely not.
>>
>> But that's all independent of this patch here.
> 
> It turns out all of this is a complicated mess, I think I've nerd-sniped myself
> into sending a series at some point about this...
> 
> It turns out we're as wildly inconsistent on pmdp_get_lockless() as we are on
> pmdp_get(), so TL;DR my main concern is moot so:

Yeah, I know that we use it in some core places already properly, like in
GUP-fast, __handle_mm_fault() or our complicated __pte_offset_map().

> 
> Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
> 
> But for the non-TL;DR... (optional reading, will probably send something):
> 
> * Agreed that the semantics matter - and the 'use pXXp_get_lockless() if no
>   locks, otherwise pXXp_get()' is preferable.
> 

Ack.

> * But it turns out it's all a giant mess (of course)... for
>   CONFIG_GUP_GET_PXX_LOW_HIGH arches you really actually need to also do:
> 
> pmdp_get_lockless_start();
> pmd = pmdp_get_lockless(...);
> pmdp_get_lockless_end();
> 
> And that's because, as Hugh points out in commit 146b42e07494 ("mm/pgtable: add
> PAE safety to __pte_offset_map()")), you need IRQs disabled for these arches for
> it to be safe.

Yes, there is this pmdp_get_lockless_start() magic ....

> 
> The explanation is quite good:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=146b42e07494e45f7c7bcf2cbf7afd1424afd78e
> 
> And then of course there's also pmdp_get_lockless_sync() used by THP collapse
> which is some kind of 'true' lockless usage that needs to synchronise with TLB
> invalidation against GUP-fast and ugh.
> 
> * So I think what matters is whether you actually NEED the value un-torn or
>   whether it's an optimistic check that you can live with (rechecked under lock
>   as needed).
> 
> * So I wonder if we shouldn't just have e.g. pmdp_get_lockless_optimistic() or
>   something like that which is just a pmdp_get() in all cases with correct kdoc
>   description etc.
> 
> * Then we can convert ALL lockless users to using a lockless form without having
>   to do the IRQ stuff. Including a ton of the existing lockless users.
> 
> * I am not sure why I did this in bullets. :)

It's really a mess. I wonder if we can remove that somehow completely ...

arch/mips/Kconfig:      select GUP_GET_PXX_LOW_HIGH if CPU_MIPS32 &&
PHYS_ADDR_T_64BIT
arch/sh/Kconfig:        select GUP_GET_PXX_LOW_HIGH if X2TLB
arch/x86/Kconfig:       select GUP_GET_PXX_LOW_HIGH             if X86_PAE

-- 
Cheers,

David

Re: [PATCH v3] mm/memory: use pmdp_get_lockless() for the pmd read in do_fault_around()

[Lorenzo Stoakes (ARM)]

On Tue, Nov 03, 2026 at 08:51:38PM +0100, David Hildenbrand (Arm) wrote:
> On 11/3/26 19:24, Lorenzo Stoakes (ARM) wrote:
> > It turns out we're as wildly inconsistent on pmdp_get_lockless() as we are on
> > pmdp_get(), so TL;DR my main concern is moot so:
>
> Yeah, I know that we use it in some core places already properly, like in
> GUP-fast, __handle_mm_fault() or our complicated __pte_offset_map().

Yup, but also inconsistently, basically mostly interchangeably with
pmdp_get(). Rarely is this value actually used.

> > * But it turns out it's all a giant mess (of course)... for
> >   CONFIG_GUP_GET_PXX_LOW_HIGH arches you really actually need to also do:
> >
> > pmdp_get_lockless_start();
> > pmd = pmdp_get_lockless(...);
> > pmdp_get_lockless_end();
> >
> > And that's because, as Hugh points out in commit 146b42e07494 ("mm/pgtable: add
> > PAE safety to __pte_offset_map()")), you need IRQs disabled for these arches for
> > it to be safe.
>
> Yes, there is this pmdp_get_lockless_start() magic ....

One of those things that, the more you look into it, the more stuff you find...

> It's really a mess. I wonder if we can remove that somehow completely ...
>
> arch/mips/Kconfig:      select GUP_GET_PXX_LOW_HIGH if CPU_MIPS32 &&
> PHYS_ADDR_T_64BIT
> arch/sh/Kconfig:        select GUP_GET_PXX_LOW_HIGH if X2TLB
> arch/x86/Kconfig:       select GUP_GET_PXX_LOW_HIGH             if X86_PAE

Yeah that'd be ideal.

But actually could we not just simplify and allow these arches to be a little
bit slower (who cares for museum pieces)?

The only user of the start/end magic is __pte_offset_map(). And it does:

	irqflags = pmdp_get_lockless_start();
	pmdval = pmdp_get_lockless(pmd);
	pmdp_get_lockless_end(irqflags);

So that could just get folded into pmdp_get_lockless() for these arches.

That would mean the places that don't require IRQs to be disabled (optimstic
checks) would now be needlessly enabling/disabling, but hey - use a system that
doesn't live in a museum :)

That way we could safely convert every non-PMD lock holding caller to
pmdp_get_lockless() without needing another pmdp_get*() variant + fix confusion
about whether the start/end magic is required.

And this would work even in places where the IRQs are already disabled as it's
fine with nesting.

So you'd just update thusly:

static inline pmd_t pmdp_get_lockless(pmd_t *pmdp)
{
	pmd_t pmd;
+	unsigned long flags;

+	local_irq_save(flags);
	do {
		pmd.pmd_low = pmdp->pmd_low;
		smp_rmb();
		pmd.pmd_high = pmdp->pmd_high;
		smp_rmb();
	} while (unlikely(pmd.pmd_low != pmdp->pmd_low));
-
+	local_irq_restore(flags);
	return pmd;
}

...

pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, pmd_t *pmdvalp)
{
-	unsigned long irqflags;
	pmd_t pmdval;

	rcu_read_lock();
-	irqflags = pmdp_get_lockless_start();
	pmdval = pmdp_get_lockless(pmd);
-	pmdp_get_lockless_end(irqflags);

	...
}

What do you think?

--
Cheers, Lorenzo