Re: [PATCH v3 14/16] mm: support GUP-triggered unsharing of anonymous pages

[David Hildenbrand <david@redhat.com>]

On 14.04.22 19:15, Vlastimil Babka wrote:
> On 3/29/22 18:04, David Hildenbrand wrote:
>> Whenever GUP currently ends up taking a R/O pin on an anonymous page that
>> might be shared -- mapped R/O and !PageAnonExclusive() -- any write fault
>> on the page table entry will end up replacing the mapped anonymous page
>> due to COW, resulting in the GUP pin no longer being consistent with the
>> page actually mapped into the page table.
>>
>> The possible ways to deal with this situation are:
>>  (1) Ignore and pin -- what we do right now.
>>  (2) Fail to pin -- which would be rather surprising to callers and
>>      could break user space.
>>  (3) Trigger unsharing and pin the now exclusive page -- reliable R/O
>>      pins.
>>
>> We want to implement 3) because it provides the clearest semantics and
>> allows for checking in unpin_user_pages() and friends for possible BUGs:
>> when trying to unpin a page that's no longer exclusive, clearly
>> something went very wrong and might result in memory corruptions that
>> might be hard to debug. So we better have a nice way to spot such
>> issues.
>>
>> To implement 3), we need a way for GUP to trigger unsharing:
>> FAULT_FLAG_UNSHARE. FAULT_FLAG_UNSHARE is only applicable to R/O mapped
>> anonymous pages and resembles COW logic during a write fault. However, in
>> contrast to a write fault, GUP-triggered unsharing will, for example, still
>> maintain the write protection.
>>
>> Let's implement FAULT_FLAG_UNSHARE by hooking into the existing write fault
>> handlers for all applicable anonymous page types: ordinary pages, THP and
>> hugetlb.
>>
>> * If FAULT_FLAG_UNSHARE finds a R/O-mapped anonymous page that has been
>>   marked exclusive in the meantime by someone else, there is nothing to do.
>> * If FAULT_FLAG_UNSHARE finds a R/O-mapped anonymous page that's not
>>   marked exclusive, it will try detecting if the process is the exclusive
>>   owner. If exclusive, it can be set exclusive similar to reuse logic
>>   during write faults via page_move_anon_rmap() and there is nothing
>>   else to do; otherwise, we either have to copy and map a fresh,
>>   anonymous exclusive page R/O (ordinary pages, hugetlb), or split the
>>   THP.
>>
>> This commit is heavily based on patches by Andrea.
>>
>> Co-developed-by: Andrea Arcangeli <aarcange@redhat.com>
>> Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
>> Signed-off-by: David Hildenbrand <david@redhat.com>
> 
> Acked-by: Vlastimil Babka <vbabka@suse.cz>
> 
> Modulo a nit and suspected logical bug below.

Thanks!

>> @@ -4515,8 +4550,11 @@ static inline vm_fault_t create_huge_pmd(struct vm_fault *vmf)
>>  /* `inline' is required to avoid gcc 4.1.2 build error */
>>  static inline vm_fault_t wp_huge_pmd(struct vm_fault *vmf)
>>  {
>> +	const bool unshare = vmf->flags & FAULT_FLAG_UNSHARE;
>> +
>>  	if (vma_is_anonymous(vmf->vma)) {
>> -		if (userfaultfd_huge_pmd_wp(vmf->vma, vmf->orig_pmd))
>> +		if (unlikely(unshare) &&
> 
> Is this condition flipped, should it be "likely(!unshare)"? As the similar
> code in do_wp_page() does.

Good catch, this should affect uffd-wp on THP -- it wouldn't trigger as expected. Thanks a lot for finding that!

> 
>> +		    userfaultfd_huge_pmd_wp(vmf->vma, vmf->orig_pmd))
>>  			return handle_userfault(vmf, VM_UFFD_WP);
>>  		return do_huge_pmd_wp_page(vmf);
>>  	}
>> @@ -4651,10 +4689,11 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf)
>>  		update_mmu_tlb(vmf->vma, vmf->address, vmf->pte);
>>  		goto unlock;
>>  	}
>> -	if (vmf->flags & FAULT_FLAG_WRITE) {
>> +	if (vmf->flags & (FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE)) {
>>  		if (!pte_write(entry))
>>  			return do_wp_page(vmf);
>> -		entry = pte_mkdirty(entry);
>> +		else if (likely(vmf->flags & FAULT_FLAG_WRITE))
>> +			entry = pte_mkdirty(entry);
>>  	}
>>  	entry = pte_mkyoung(entry);
>>  	if (ptep_set_access_flags(vmf->vma, vmf->address, vmf->pte, entry,
> 


So the following on top, right?


diff --git a/mm/memory.c b/mm/memory.c
index 8b3cb73f5e44..4584c7e87a70 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -3137,7 +3137,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
                        free_swap_cache(old_page);
                put_page(old_page);
        }
-       return page_copied && !unshare ? VM_FAULT_WRITE : 0;
+       return (page_copied && !unshare) ? VM_FAULT_WRITE : 0;
 oom_free_new:
        put_page(new_page);
 oom:
@@ -4604,7 +4604,7 @@ static inline vm_fault_t wp_huge_pmd(struct vm_fault *vmf)
        const bool unshare = vmf->flags & FAULT_FLAG_UNSHARE;
 
        if (vma_is_anonymous(vmf->vma)) {
-               if (unlikely(unshare) &&
+               if (likely(!unshare) &&
                    userfaultfd_huge_pmd_wp(vmf->vma, vmf->orig_pmd))
                        return handle_userfault(vmf, VM_UFFD_WP);
                return do_huge_pmd_wp_page(vmf);


-- 
Thanks,

David / dhildenb