From: Ryan Roberts <ryan.roberts@arm.com>
To: David Hildenbrand <david@redhat.com>, linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>,
"Matthew Wilcox (Oracle)" <willy@infradead.org>,
Hugh Dickins <hughd@google.com>,
Yin Fengwei <fengwei.yin@intel.com>,
Mike Kravetz <mike.kravetz@oracle.com>,
Muchun Song <muchun.song@linux.dev>, Peter Xu <peterx@redhat.com>
Subject: Re: [PATCH RFC 34/39] mm/rmap: introduce folio_try_dup_anon_rmap_[pte|ptes|pmd]()
Date: Tue, 5 Dec 2023 13:12:56 +0000 [thread overview]
Message-ID: <b7ef017b-f651-40f3-a2bd-70ebe9411dc1@arm.com> (raw)
In-Reply-To: <20231204142146.91437-35-david@redhat.com>
On 04/12/2023 14:21, David Hildenbrand wrote:
> The last user of page_needs_cow_for_dma() and __page_dup_rmap() are gone,
> remove them.
>
> Add folio_try_dup_anon_rmap_ptes() right away, we want to perform rmap
> baching during fork() soon.
>
> Signed-off-by: David Hildenbrand <david@redhat.com>
> ---
> include/linux/mm.h | 6 --
> include/linux/rmap.h | 145 +++++++++++++++++++++++++++++--------------
> 2 files changed, 100 insertions(+), 51 deletions(-)
>
> diff --git a/include/linux/mm.h b/include/linux/mm.h
> index 24c1c7c5a99c0..f7565b35ae931 100644
> --- a/include/linux/mm.h
> +++ b/include/linux/mm.h
> @@ -1964,12 +1964,6 @@ static inline bool folio_needs_cow_for_dma(struct vm_area_struct *vma,
> return folio_maybe_dma_pinned(folio);
> }
>
> -static inline bool page_needs_cow_for_dma(struct vm_area_struct *vma,
> - struct page *page)
> -{
> - return folio_needs_cow_for_dma(vma, page_folio(page));
> -}
> -
> /**
> * is_zero_page - Query if a page is a zero page
> * @page: The page to query
> diff --git a/include/linux/rmap.h b/include/linux/rmap.h
> index 21d72cc602adc..84439f7720c62 100644
> --- a/include/linux/rmap.h
> +++ b/include/linux/rmap.h
> @@ -354,68 +354,123 @@ static inline void folio_dup_file_rmap_pmd(struct folio *folio,
> #endif
> }
>
> -static inline void __page_dup_rmap(struct page *page, bool compound)
> +static inline int __folio_try_dup_anon_rmap(struct folio *folio,
__always_inline?
> + struct page *page, unsigned int nr_pages,
> + struct vm_area_struct *src_vma, enum rmap_mode mode)
> {
> - VM_WARN_ON(folio_test_hugetlb(page_folio(page)));
> + int i;
>
> - if (compound) {
> - struct folio *folio = (struct folio *)page;
> + VM_WARN_ON_FOLIO(!folio_test_anon(folio), folio);
>
> - VM_BUG_ON_PAGE(compound && !PageHead(page), page);
> - atomic_inc(&folio->_entire_mapcount);
> - } else {
> - atomic_inc(&page->_mapcount);
> + /*
> + * No need to check+clear for already shared PTEs/PMDs of the folio.
> + * This includes PTE mappings of (order-0) KSM folios.
> + */
> + if (likely(mode == RMAP_MODE_PTE)) {
Presumbly if __always_inline then the compiler will remove this if/else and just
keep the part indicated by mode? In which case "likely" is pretty useless? Same
for all similar sites in the other patches.
> + for (i = 0; i < nr_pages; i++) {
> + if (PageAnonExclusive(page + i))
> + goto clear;
> + }
> + } else if (mode == RMAP_MODE_PMD) {
> + if (PageAnonExclusive(page))
> + goto clear;
> }
> + goto dup;
> +
> +clear:
> + /*
> + * If this folio may have been pinned by the parent process,
> + * don't allow to duplicate the mappings but instead require to e.g.,
> + * copy the subpage immediately for the child so that we'll always
> + * guarantee the pinned folio won't be randomly replaced in the
> + * future on write faults.
> + */
> + if (likely(!folio_is_device_private(folio) &&
> + unlikely(folio_needs_cow_for_dma(src_vma, folio))))
> + return -EBUSY;
> +
> + if (likely(mode == RMAP_MODE_PTE)) {
> + for (i = 0; i < nr_pages; i++)
Do you really need to reset i=0 here? You have already checked that lower pages
are shared in the above loop, so can't you just start from the first exclusive
page here?
> + ClearPageAnonExclusive(page + i);
> + } else if (mode == RMAP_MODE_PMD) {
> + ClearPageAnonExclusive(page);
> + }
> +
> +dup:
> + __folio_dup_rmap(folio, page, nr_pages, mode);
> + return 0;
> }
>
> /**
> - * page_try_dup_anon_rmap - try duplicating a mapping of an already mapped
> - * anonymous page
> - * @page: the page to duplicate the mapping for
> - * @compound: the page is mapped as compound or as a small page
> - * @vma: the source vma
> + * folio_try_dup_anon_rmap_ptes - try duplicating PTE mappings of a page range
> + * of a folio
> + * @folio: The folio to duplicate the mappings of
> + * @page: The first page to duplicate the mappings of
> + * @nr_pages: The number of pages of which the mapping will be duplicated
> + * @src_vma: The vm area from which the mappings are duplicated
> *
> - * The caller needs to hold the PT lock and the vma->vma_mm->write_protect_seq.
> + * The page range of the folio is defined by [page, page + nr_pages)
> *
> - * Duplicating the mapping can only fail if the page may be pinned; device
> - * private pages cannot get pinned and consequently this function cannot fail.
> + * The caller needs to hold the page table lock and the
> + * vma->vma_mm->write_protect_seq.
> + *
> + * Duplicating the mappings can only fail if the folio may be pinned; device
> + * private folios cannot get pinned and consequently this function cannot fail.
> + *
> + * If duplicating the mappings succeeded, the duplicated PTEs have to be R/O in
> + * the parent and the child. They must *not* be writable after this call.
> + *
> + * Returns 0 if duplicating the mappings succeeded. Returns -EBUSY otherwise.
> + */
> +static inline int folio_try_dup_anon_rmap_ptes(struct folio *folio,
> + struct page *page, unsigned int nr_pages,
> + struct vm_area_struct *src_vma)
> +{
> + return __folio_try_dup_anon_rmap(folio, page, nr_pages, src_vma,
> + RMAP_MODE_PTE);
> +}
> +#define folio_try_dup_anon_rmap_pte(folio, page, vma) \
> + folio_try_dup_anon_rmap_ptes(folio, page, 1, vma)
> +
> +/**
> + * folio_try_dup_anon_rmap_pmd - try duplicating a PMD mapping of a page range
> + * of a folio
> + * @folio: The folio to duplicate the mapping of
> + * @page: The first page to duplicate the mapping of
> + * @src_vma: The vm area from which the mapping is duplicated
> + *
> + * The page range of the folio is defined by [page, page + HPAGE_PMD_NR)
> + *
> + * The caller needs to hold the page table lock and the
> + * vma->vma_mm->write_protect_seq.
> *
> - * If duplicating the mapping succeeds, the page has to be mapped R/O into
> - * the parent and the child. It must *not* get mapped writable after this call.
> + * Duplicating the mapping can only fail if the folio may be pinned; device
> + * private folios cannot get pinned and consequently this function cannot fail.
> + *
> + * If duplicating the mapping succeeds, the duplicated PMD has to be R/O in
> + * the parent and the child. They must *not* be writable after this call.
> *
> * Returns 0 if duplicating the mapping succeeded. Returns -EBUSY otherwise.
> */
> +static inline int folio_try_dup_anon_rmap_pmd(struct folio *folio,
> + struct page *page, struct vm_area_struct *src_vma)
> +{
> +#ifdef CONFIG_TRANSPARENT_HUGEPAGE
> + return __folio_try_dup_anon_rmap(folio, page, HPAGE_PMD_NR, src_vma,
> + RMAP_MODE_PMD);
> +#else
> + WARN_ON_ONCE(true);
> +#endif
> +}
> +
> static inline int page_try_dup_anon_rmap(struct page *page, bool compound,
> struct vm_area_struct *vma)
> {
> - VM_BUG_ON_PAGE(!PageAnon(page), page);
> -
> - /*
> - * No need to check+clear for already shared pages, including KSM
> - * pages.
> - */
> - if (!PageAnonExclusive(page))
> - goto dup;
> -
> - /*
> - * If this page may have been pinned by the parent process,
> - * don't allow to duplicate the mapping but instead require to e.g.,
> - * copy the page immediately for the child so that we'll always
> - * guarantee the pinned page won't be randomly replaced in the
> - * future on write faults.
> - */
> - if (likely(!is_device_private_page(page) &&
> - unlikely(page_needs_cow_for_dma(vma, page))))
> - return -EBUSY;
> + struct folio *folio = page_folio(page);
>
> - ClearPageAnonExclusive(page);
> - /*
> - * It's okay to share the anon page between both processes, mapping
> - * the page R/O into both processes.
> - */
> -dup:
> - __page_dup_rmap(page, compound);
> - return 0;
> + if (likely(!compound))
> + return folio_try_dup_anon_rmap_pte(folio, page, vma);
> + return folio_try_dup_anon_rmap_pmd(folio, page, vma);
> }
>
> /**
next prev parent reply other threads:[~2023-12-05 13:13 UTC|newest]
Thread overview: 82+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-04 14:21 [PATCH RFC 00/39] mm/rmap: interface overhaul David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 01/39] mm/rmap: rename hugepage_add* to hugetlb_add* David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 02/39] mm/rmap: introduce and use hugetlb_remove_rmap() David Hildenbrand
2023-12-06 1:22 ` Yin Fengwei
2023-12-06 12:11 ` David Hildenbrand
2023-12-07 0:56 ` Yin Fengwei
2023-12-04 14:21 ` [PATCH RFC 03/39] mm/rmap: introduce and use hugetlb_add_file_rmap() David Hildenbrand
2023-12-06 1:22 ` Yin Fengwei
2023-12-04 14:21 ` [PATCH RFC 04/39] mm/rmap: introduce and use hugetlb_try_dup_anon_rmap() David Hildenbrand
2023-12-06 1:22 ` Yin Fengwei
2023-12-04 14:21 ` [PATCH RFC 05/39] mm/rmap: introduce and use hugetlb_try_share_anon_rmap() David Hildenbrand
2023-12-06 1:23 ` Yin Fengwei
2023-12-04 14:21 ` [PATCH RFC 06/39] mm/rmap: add hugetlb sanity checks David Hildenbrand
2023-12-06 1:23 ` Yin Fengwei
2023-12-04 14:21 ` [PATCH RFC 07/39] mm/rmap: convert folio_add_file_rmap_range() into folio_add_file_rmap_[pte|ptes|pmd]() David Hildenbrand
2023-12-05 12:04 ` Ryan Roberts
2023-12-05 12:25 ` David Hildenbrand
2023-12-06 1:30 ` Yin Fengwei
2023-12-06 9:17 ` David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 08/39] mm/memory: page_add_file_rmap() -> folio_add_file_rmap_[pte|pmd]() David Hildenbrand
2023-12-08 1:40 ` Yin, Fengwei
2023-12-04 14:21 ` [PATCH RFC 09/39] mm/huge_memory: page_add_file_rmap() -> folio_add_file_rmap_pmd() David Hildenbrand
2023-12-08 1:41 ` Yin, Fengwei
2023-12-04 14:21 ` [PATCH RFC 10/39] mm/migrate: page_add_file_rmap() -> folio_add_file_rmap_pte() David Hildenbrand
2023-12-08 1:42 ` Yin, Fengwei
2023-12-04 14:21 ` [PATCH RFC 11/39] mm/userfaultfd: " David Hildenbrand
2023-12-08 1:42 ` Yin, Fengwei
2023-12-04 14:21 ` [PATCH RFC 12/39] mm/rmap: remove page_add_file_rmap() David Hildenbrand
2023-12-08 1:42 ` Yin, Fengwei
2023-12-04 14:21 ` [PATCH RFC 13/39] mm/rmap: factor out adding folio mappings into __folio_add_rmap() David Hildenbrand
2023-12-08 1:44 ` Yin, Fengwei
2023-12-04 14:21 ` [PATCH RFC 14/39] mm/rmap: introduce folio_add_anon_rmap_[pte|ptes|pmd]() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 15/39] mm/huge_memory: batch rmap operations in __split_huge_pmd_locked() David Hildenbrand
2023-12-05 12:22 ` Ryan Roberts
2023-12-05 12:26 ` David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 16/39] mm/huge_memory: page_add_anon_rmap() -> folio_add_anon_rmap_pmd() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 17/39] mm/migrate: page_add_anon_rmap() -> folio_add_anon_rmap_pte() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 18/39] mm/ksm: " David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 19/39] mm/swapfile: " David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 20/39] mm/memory: " David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 21/39] mm/rmap: remove page_add_anon_rmap() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 22/39] mm/rmap: remove RMAP_COMPOUND David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 23/39] mm/rmap: introduce folio_remove_rmap_[pte|ptes|pmd]() David Hildenbrand
2023-12-05 12:52 ` Ryan Roberts
2023-12-05 13:09 ` David Hildenbrand
2023-12-05 13:37 ` Ryan Roberts
2023-12-04 14:21 ` [PATCH RFC 24/39] kernel/events/uprobes: page_remove_rmap() -> folio_remove_rmap_pte() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 25/39] mm/huge_memory: page_remove_rmap() -> folio_remove_rmap_pmd() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 26/39] mm/khugepaged: page_remove_rmap() -> folio_remove_rmap_pte() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 27/39] mm/ksm: " David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 28/39] mm/memory: " David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 29/39] mm/migrate_device: " David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 30/39] mm/rmap: " David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 31/39] Documentation: stop referring to page_remove_rmap() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 32/39] mm/rmap: remove page_remove_rmap() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 33/39] mm/rmap: convert page_dup_file_rmap() to folio_dup_file_rmap_[pte|ptes|pmd]() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 34/39] mm/rmap: introduce folio_try_dup_anon_rmap_[pte|ptes|pmd]() David Hildenbrand
2023-12-04 17:59 ` David Hildenbrand
2023-12-05 13:11 ` David Hildenbrand
2023-12-05 13:12 ` Ryan Roberts [this message]
2023-12-05 13:17 ` David Hildenbrand
2023-12-05 13:18 ` David Hildenbrand
2023-12-05 13:40 ` Ryan Roberts
2023-12-05 13:50 ` David Hildenbrand
2023-12-05 14:02 ` Ryan Roberts
2023-12-05 14:12 ` David Hildenbrand
2023-12-05 13:32 ` David Hildenbrand
2023-12-05 13:40 ` Ryan Roberts
2023-12-04 14:21 ` [PATCH RFC 35/39] mm/huge_memory: page_try_dup_anon_rmap() -> folio_try_dup_anon_rmap_pmd() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 36/39] mm/memory: page_try_dup_anon_rmap() -> folio_try_dup_anon_rmap_pte() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 37/39] mm/rmap: remove page_try_dup_anon_rmap() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 38/39] mm: convert page_try_share_anon_rmap() to folio_try_share_anon_rmap_[pte|pmd]() David Hildenbrand
2023-12-04 14:21 ` [PATCH RFC 39/39] mm/rmap: rename COMPOUND_MAPPED to ENTIRELY_MAPPED David Hildenbrand
2023-12-04 19:53 ` [PATCH RFC 00/39] mm/rmap: interface overhaul Ryan Roberts
2023-12-05 9:56 ` David Hildenbrand
2023-12-05 13:31 ` Ryan Roberts
2023-12-05 13:39 ` David Hildenbrand
2023-12-05 13:49 ` Ryan Roberts
2023-12-05 13:55 ` David Hildenbrand
2023-12-08 11:24 ` David Hildenbrand
2023-12-08 11:38 ` Ryan Roberts
2023-12-05 12:29 ` Ryan Roberts
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b7ef017b-f651-40f3-a2bd-70ebe9411dc1@arm.com \
--to=ryan.roberts@arm.com \
--cc=akpm@linux-foundation.org \
--cc=david@redhat.com \
--cc=fengwei.yin@intel.com \
--cc=hughd@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mike.kravetz@oracle.com \
--cc=muchun.song@linux.dev \
--cc=peterx@redhat.com \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®