From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=74.125.229.6 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794354733; b=rETriq0xYVUDkTQ0FeB43g5XJgyv3ZWl2XO3kQkxWhGHTmnIWP35+qDQqZWtn/Ow4Aim TM/uLeGMIIrC9q11caJfuykKbcX17G74oTAdnTDBDg8l/kdr7w8JnTeGRPtVQiF62ccYC M2GMLbhyPqIBpYwuF1yKMreHXCl+tiZ2i4Z05MC7FhawW4OXUtqoSMPhqztMgLtaMXGPE 2Sw/xzP1b8x/t7kGsxVO7xDt9DY3vKirOhNiOojrEH5NCNS0stVKs0mFgqEeBWopjvzp8 b/2mzpe9XYXkfeOzqknTaV9BqVl//eykqiIryD5bd4c15ISC7iLa0Voah8Y4XriVwzQ== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794354733; h=DMARC-Filter:Received:DKIM-Signature:X-Google-DKIM-Signature: X-Forwarded-Encrypted:X-Gm-Message-State:X-Gm-Gg:X-Received:Received: Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:Content-Transfer-Encoding: In-Reply-To; bh=8piulZmzPz+GLbFi3OeZnbxDdpLELCU5e/qg3zDhY0A=; b=xgdSCoOTqR1LTi3LiPpco+UYFutHXTvaXwGB9NUO14otr3+14PozVBUllcThj4NNyNlZ Qaw+8l8wK9El+qndUq/OU00HtfbzP+q5gtG9sOFqM5swnSovptQttHJ6M6ZiCpyjfHhZ2 2nTN4OJU0sMX1/inVocsJRcygaS9TZDBLGeSU/ulhMQXxlBbUebFQfDBaT1LUzJWpb4de mv53QLkZV1tsk3RzjFGfe83r9vPgX6Fvbx4ZWa3j1ByatVFVZm7JZhP55O+6PYiVSDKQC fB64qB3mGopTuCEkt/2ukZHGHj3oqD3VlxlyHo+9t06BIjv97lfR3aVgFMDL+Y71RLg== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20260707 header.b=A27f8qsu; arc=none smtp.remote-ip=74.125.229.6 Received: from mail-dy2-f6.google.com (mail-dy2-f6.google.com [74.125.229.6]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1184E60A2616 for ; Tue, 10 Nov 2026 23:52:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.2 smtp.subspace.kernel.org 1184E60A2616 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20260707 header.b=A27f8qsu Received: by mail-dy2-f6.google.com with SMTP id 5a478bee46e88-35adc681e1aso3510878eec.0 for ; Tue, 10 Nov 2026 15:52:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1794354731; x=1794959531; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :content-type:mime-version:references:message-id:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to:content-type; bh=8piulZmzPz+GLbFi3OeZnbxDdpLELCU5e/qg3zDhY0A=; b=A27f8qsupZ1PMhEWOW8cQqik8tTxnKeYVbMsiyIbFt3VSqrjhgBaYEj4ZQIkcjp+Gy XR+VwGJsIvjRt1dev3wxh9EflmRpvxOa0/iSnoz8HG4eY0IG3BZJOCxFMYMZta+zGJUZ rRtN8unllRBEgkuV6XWvxHX6uMYwPc18TTAbs5Kfi9eifXAnzm15DnAOkWds6uStvKgS /8AfO8HY0/P2taavxRSDjWqx+YasNKdJlkKa/qfyN5Qn7R4dI3MG28gViCRVE8ZtPaOl Me6lmYvr5/yml8Togyo+cECqfNEWxTDuy+iVjZDZ9VJQi7J29E6FeVMl03Ot0P4qbyMk dpTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1794354731; x=1794959531; h=in-reply-to:content-transfer-encoding:content-disposition :content-type:mime-version:references:message-id:subject:cc:to:from :date:x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to:content-type; bh=8piulZmzPz+GLbFi3OeZnbxDdpLELCU5e/qg3zDhY0A=; b=cYiJYaHJDe1iUhgJRKav6aPXSuC+ALUtbPvYuXGeyQdxNmB5lRIpKOxeqJMpexu0qJ fO6McKtin9i7UhHiwJGhckEy0aaIfBkEGFSHBxgYSbhVMLxZWYFHAra6JQxKqDUZAIsP kZze3I9HNb908gOx4erJdjBRt8uTI3ondybDhQdTtHYXkUD7/IC3sQrgrYocQcSP/LPs pucCHvMn2ZibiUc8OjxpuJxFAMNXbl6XNMggNCk9sRDagk+MusX1CcFn2vyhlL54HChA x6O5/vKks3hwT7ga8FqyIXFkVWpongisnEiRFB262Bx8RVBFcLUr+ulTsY2lFQa3rXsh pnHw== X-Forwarded-Encrypted: i=1; AL+hErEL4ACVjJQnskVcstqakDGFZb8nPH5/8dtgXca3n1mkzgLAmsWNoRI+27lj+/KIzCMFzL83iO3uymtmYwk=@vger.kernel.org X-Gm-Message-State: APuU4Z/5yOy518R+9hDsoF4KMv5e9eH4y0oElMcZ0erwTAMoLMYflJHs PuPLBDQNTzGsQADfim9atG929NAsF5VuJPcu24ZaNRy3/L/XA5AnU3KF X-Gm-Gg: AYBFou2RemR2BhZaOkH7uLe/LynzBGB9+XQ7WXexmk5QO0E/3Rm5+qRfDoGZ2y9vta5 soqQwETuqYr4ML4G32wKs+o3gKguRdyPBhquCow8xb+b2S3jAqBK/tGiSQ/vaq6guHgbyUD/Z9n yxom++9aTCGh8x2Zz4uqUUhOPgcF+ao91rGZ3btEJipe84bg9ZymbM7pJjV+liS74OWnVwjbuJv S9uRZ8oh4A7ZfGSMFHIJGC9ePVBXXbpJxAnfif2zZMNLPGeksFNMU99eK7wp7kkbhY8fH2QcNRw bZF6igzgJ6x85loos52XmnglTuECVRVb+IKtkTa0UPQdHYGeeG9+0F9PXQr2631TiHhB9KOgPlL R+60G+uTADU1VY/mFtC/E2vaBzFyJb57WL+2qzMPdh0lBneN8IKL/WywooCqo/DG4Oy5hwoGiZV QhN/JU93GuomkQu83uSycAehRmsMc/ovCxh9ZfdH1PLryhwjaBYM9VhpY6aaxOErGb0ucjJiIHA l4PPEictFsnvGS07tH+gOCh3fNnM3dp/O/MDzzK4GqA0Ca6QDROvbGr6phfpWW4nNgqtLSissaH 0acmP+3r9V2hvpvvcDfdo9w= X-Received: by 2002:a05:7300:cc9c:b0:36c:b8b1:a44f with SMTP id 5a478bee46e88-37686466019mr1359845eec.31.1794354731058; Tue, 10 Nov 2026 15:52:11 -0800 (PST) Received: from localhost ([2a03:2880:9ff:6a::]) by smtp.gmail.com with ESMTPSA id 5a478bee46e88-375218c863csm9704178eec.28.2026.11.10.15.52.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Nov 2026 15:52:10 -0800 (PST) Date: Tue, 10 Nov 2026 15:02:03 -0800 From: Stanislav Fomichev To: =?utf-8?B?QmrDtnJuIFTDtnBlbA==?= Cc: Magnus Karlsson , Maciej Fijalkowski , Stanislav Fomichev , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman , Jonathan Corbet , Shuah Khan , Randy Dunlap , Alexander Duyck , kernel-team@meta.com, Andrew Lunn , Jesper Dangaard Brouer , Ilias Apalodimas , Alexei Starovoitov , Daniel Borkmann , John Fastabend , Pavel Begunkov , Jens Axboe , Andrii Nakryiko , Eduard Zingerman , Kumar Kartikeya Dwivedi , Martin KaFai Lau , Song Liu , Yonghong Song , Jiri Olsa , Emil Tsalapatis , Ihor Solodrai , netdev@vger.kernel.org, bpf@vger.kernel.org, io-uring@vger.kernel.org, Mina Almasry , "Mike Marciniszyn (Meta)" , Weiming Shi , Nikolay Aleksandrov , David Wei , Alexander Lobakin , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Stuart Clark Subject: Re: [PATCH net-next v1 04/11] xsk: Allocate UMEM chunks through page_pool Message-ID: References: <20261110102039.1002605-1-bjorn@kernel.org> <20261110102039.1002605-5-bjorn@kernel.org> X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20261110102039.1002605-5-bjorn@kernel.org> On 11/10, Björn Töpel wrote: > AF_XDP zero copy needs a second RX allocator in every driver. A > page_pool driver can take UMEM chunks from a memory provider instead, > and keep its refill, recycling and DMA sync code. > > Each aligned chunk becomes a net_iov, whose address netmem_address() > finds without calling the provider. Userspace may post a chunk that > the kernel still owns, so the provider tracks the owner of each chunk > and drops such FILL entries. Chunks that a page pool releases stay in > the kernel for reuse. Nothing installs the provider yet. > > Signed-off-by: Björn Töpel > --- > include/net/netmem.h | 25 ++++- > include/net/xdp.h | 2 +- > net/xdp/Kconfig | 1 + > net/xdp/Makefile | 2 +- > net/xdp/xsk.h | 6 ++ > net/xdp/xsk_pp.c | 246 +++++++++++++++++++++++++++++++++++++++++++ > 6 files changed, 279 insertions(+), 3 deletions(-) > create mode 100644 net/xdp/xsk_pp.c > > diff --git a/include/net/netmem.h b/include/net/netmem.h > index b986ad616334..a3b67ef32b21 100644 > --- a/include/net/netmem.h > +++ b/include/net/netmem.h > @@ -69,6 +69,7 @@ DECLARE_STATIC_KEY_FALSE(page_pool_restricted); > enum net_iov_type { > NET_IOV_DMABUF, > NET_IOV_IOURING, > + NET_IOV_XSK, > }; > > /* A memory descriptor representing abstract networking I/O vectors. > @@ -107,6 +108,10 @@ struct net_iov_area { > size_t num_niovs; > /* Each net_iov covers 1 << niov_shift bytes. */ > unsigned int niov_shift; > + /* CPU address of the first net_iov, or NULL if the CPU cannot read > + * the area. > + */ > + void *vaddr; > }; > > static inline struct net_iov_area *net_iov_owner(const struct net_iov *niov) > @@ -119,6 +124,18 @@ static inline unsigned int net_iov_idx(const struct net_iov *niov) > return niov - net_iov_owner(niov)->niovs; > } > > +static inline void *net_iov_address(const struct net_iov *niov) > +{ > + const struct net_iov_area *area = net_iov_owner(niov); > + unsigned long off; > + > + if (!area->vaddr) > + return NULL; > + > + off = (unsigned long)net_iov_idx(niov) << area->niov_shift; > + return area->vaddr + off; > +} > + > /* Initialize a niov: stamp the owning area, the memory provider type. > */ > static inline void net_iov_init(struct net_iov *niov, > @@ -345,10 +362,16 @@ static inline void *__netmem_address(netmem_ref netmem) > return page_address(__netmem_to_page(netmem)); > } > > +/** > + * netmem_address - get the CPU address of the memory behind @netmem > + * @netmem: netmem to get the address of > + * > + * Return: the address, or NULL if the CPU cannot read @netmem. > + */ > static inline void *netmem_address(netmem_ref netmem) > { > if (netmem_is_net_iov(netmem)) > - return NULL; > + return net_iov_address(netmem_to_net_iov(netmem)); > > return __netmem_address(netmem); > } > diff --git a/include/net/xdp.h b/include/net/xdp.h > index 1105bb9e3292..5ab7802ac4e1 100644 > --- a/include/net/xdp.h > +++ b/include/net/xdp.h > @@ -368,7 +368,7 @@ static inline bool xdp_buff_add_frag(struct xdp_buff *xdp, netmem_ref netmem, > > if (unlikely(netmem_is_pfmemalloc(netmem))) > xdp_buff_set_frag_pfmemalloc(xdp); > - if (unlikely(netmem_is_net_iov(netmem))) > + if (unlikely(netmem_is_net_iov(netmem) && !netmem_address(netmem))) > xdp_buff_set_frag_unreadable(xdp); > > return true; > diff --git a/net/xdp/Kconfig b/net/xdp/Kconfig > index 71af2febe72a..c9c68d3b3712 100644 > --- a/net/xdp/Kconfig > +++ b/net/xdp/Kconfig > @@ -2,6 +2,7 @@ > config XDP_SOCKETS > bool "XDP sockets" > depends on BPF_SYSCALL > + select PAGE_POOL > default n > help > XDP sockets allows a channel between XDP programs and > diff --git a/net/xdp/Makefile b/net/xdp/Makefile > index 30cdc4315f42..6e9d60f9f7f1 100644 > --- a/net/xdp/Makefile > +++ b/net/xdp/Makefile > @@ -1,4 +1,4 @@ > # SPDX-License-Identifier: GPL-2.0-only > obj-$(CONFIG_XDP_SOCKETS) += xsk.o xdp_umem.o xsk_queue.o xskmap.o > -obj-$(CONFIG_XDP_SOCKETS) += xsk_buff_pool.o > +obj-$(CONFIG_XDP_SOCKETS) += xsk_buff_pool.o xsk_pp.o > obj-$(CONFIG_XDP_SOCKETS_DIAG) += xsk_diag.o > diff --git a/net/xdp/xsk.h b/net/xdp/xsk.h > index 7c811b5cce76..4dab7b9a1487 100644 > --- a/net/xdp/xsk.h > +++ b/net/xdp/xsk.h > @@ -34,6 +34,12 @@ static inline struct xdp_sock *xdp_sk(struct sock *sk) > > void xsk_map_try_sock_delete(struct xsk_map *map, struct xdp_sock *xs, > struct xdp_sock __rcu **map_entry); > +struct xsk_pp; > + > +extern const struct memory_provider_ops xsk_pp_ops; > + > +struct xsk_pp *xsk_pp_create(struct xsk_buff_pool *pool); > +void xsk_pp_put(struct xsk_pp *pp); > void xsk_clear_pool_at_qid(struct net_device *dev, u16 queue_id); > int xsk_reg_pool_at_qid(struct net_device *dev, struct xsk_buff_pool *pool, > u16 queue_id); > diff --git a/net/xdp/xsk_pp.c b/net/xdp/xsk_pp.c > new file mode 100644 > index 000000000000..fdd55265a6cd > --- /dev/null > +++ b/net/xdp/xsk_pp.c > @@ -0,0 +1,246 @@ > +// SPDX-License-Identifier: GPL-2.0 > +/* AF_XDP UMEM as a page pool memory provider > + * > + * Each aligned UMEM chunk is a readable net_iov with one owner at a time: > + * > + * userspace --FILL--> page pool --release--> reuse --alloc--> page pool > + * ^ | > + * +-----RX ring-------+ > + */ > + > +#include > +#include > +#include > + > +#include "xsk.h" > +#include "xsk_queue.h" > + > +enum xsk_pp_chunk { > + XSK_PP_USER, /* owned by userspace */ > + XSK_PP_POOL, /* owned by a page pool */ > + XSK_PP_REUSE, /* on the reuse stack */ Can you explain more about that reuse/reuse_cnt? When is the chunk reused? How is it different from FILL->RX path?