From: Yunsheng Lin <linyunsheng@huawei.com>
To: <davem@davemloft.net>, <kuba@kernel.org>, <pabeni@redhat.com>
Cc: <netdev@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
Yunsheng Lin <linyunsheng@huawei.com>,
Lorenzo Bianconi <lorenzo@kernel.org>,
Alexander Duyck <alexander.duyck@gmail.com>,
Jesper Dangaard Brouer <hawk@kernel.org>,
Ilias Apalodimas <ilias.apalodimas@linaro.org>,
Eric Dumazet <edumazet@google.com>
Subject: [RFC 1/3] page_pool: unify frag page and non-frag page handling
Date: Tue, 16 May 2023 20:47:59 +0800 [thread overview]
Message-ID: <20230516124801.2465-2-linyunsheng@huawei.com> (raw)
In-Reply-To: <20230516124801.2465-1-linyunsheng@huawei.com>
Currently page_pool_dev_alloc_pages() can not be called
when PP_FLAG_PAGE_FRAG is set, because it does not use
the frag reference counting.
As we are already doing a optimization by not updating
page->pp_frag_count in page_pool_defrag_page() for the
last frag user, and non-frag page only have one user,
so we utilize that to unify frag page and non-frag page
handling, so that page_pool_dev_alloc_pages() can also
be called with PP_FLAG_PAGE_FRAG set.
Signed-off-by: Yunsheng Lin <linyunsheng@huawei.com>
CC: Lorenzo Bianconi <lorenzo@kernel.org>
CC: Alexander Duyck <alexander.duyck@gmail.com>
---
include/net/page_pool.h | 24 ++++++++++++++++++------
net/core/page_pool.c | 3 ++-
2 files changed, 20 insertions(+), 7 deletions(-)
diff --git a/include/net/page_pool.h b/include/net/page_pool.h
index c8ec2f34722b..14ac46297ae4 100644
--- a/include/net/page_pool.h
+++ b/include/net/page_pool.h
@@ -50,6 +50,9 @@
PP_FLAG_DMA_SYNC_DEV |\
PP_FLAG_PAGE_FRAG)
+#define PAGE_POOL_DMA_USE_PP_FRAG_COUNT \
+ (sizeof(dma_addr_t) > sizeof(unsigned long))
+
/*
* Fast allocation side cache array/stack
*
@@ -295,7 +298,8 @@ void page_pool_put_defragged_page(struct page_pool *pool, struct page *page,
*/
static inline void page_pool_fragment_page(struct page *page, long nr)
{
- atomic_long_set(&page->pp_frag_count, nr);
+ if (!PAGE_POOL_DMA_USE_PP_FRAG_COUNT)
+ atomic_long_set(&page->pp_frag_count, nr);
}
static inline long page_pool_defrag_page(struct page *page, long nr)
@@ -316,14 +320,25 @@ static inline long page_pool_defrag_page(struct page *page, long nr)
ret = atomic_long_sub_return(nr, &page->pp_frag_count);
WARN_ON(ret < 0);
+
+ /* Reset it to 1 to allow only one user in case the page is
+ * recycled and allocated as non-frag page if it is the last
+ * user, this should be the rare case as it only happen when
+ * two users call page_pool_defrag_page() currently.
+ */
+ if (!ret)
+ atomic_long_set(&page->pp_frag_count, 1);
+
return ret;
}
static inline bool page_pool_is_last_frag(struct page_pool *pool,
struct page *page)
{
- /* If fragments aren't enabled or count is 0 we were the last user */
- return !(pool->p.flags & PP_FLAG_PAGE_FRAG) ||
+ /* When dma_addr_upper is overlapped with pp_frag_count
+ * or we were the last page frag user.
+ */
+ return PAGE_POOL_DMA_USE_PP_FRAG_COUNT ||
(page_pool_defrag_page(page, 1) == 0);
}
@@ -357,9 +372,6 @@ static inline void page_pool_recycle_direct(struct page_pool *pool,
page_pool_put_full_page(pool, page, true);
}
-#define PAGE_POOL_DMA_USE_PP_FRAG_COUNT \
- (sizeof(dma_addr_t) > sizeof(unsigned long))
-
static inline dma_addr_t page_pool_get_dma_addr(struct page *page)
{
dma_addr_t ret = page->dma_addr;
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index e212e9d7edcb..5d93c5dc0549 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -334,6 +334,7 @@ static void page_pool_set_pp_info(struct page_pool *pool,
{
page->pp = pool;
page->pp_magic |= PP_SIGNATURE;
+ page_pool_fragment_page(page, 1);
if (pool->p.init_callback)
pool->p.init_callback(page, pool->p.init_arg);
}
@@ -698,7 +699,7 @@ struct page *page_pool_alloc_frag(struct page_pool *pool,
unsigned int max_size = PAGE_SIZE << pool->p.order;
struct page *page = pool->frag_page;
- if (WARN_ON(!(pool->p.flags & PP_FLAG_PAGE_FRAG) ||
+ if (WARN_ON(PAGE_POOL_DMA_USE_PP_FRAG_COUNT ||
size > max_size))
return NULL;
--
2.33.0
next prev parent reply other threads:[~2023-05-16 12:50 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-16 12:47 [RFC 0/3] support non-frag page for page_pool_alloc_frag() Yunsheng Lin
2023-05-16 12:47 ` Yunsheng Lin [this message]
2023-05-16 12:48 ` [RFC 2/3] page_pool: " Yunsheng Lin
2023-05-16 12:48 ` [RFC 3/3] page_pool: introduce 'struct page_pool_frag' Yunsheng Lin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230516124801.2465-2-linyunsheng@huawei.com \
--to=linyunsheng@huawei.com \
--cc=alexander.duyck@gmail.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=hawk@kernel.org \
--cc=ilias.apalodimas@linaro.org \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lorenzo@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®