mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: "Björn Töpel" <bjorn@kernel.org>
To: Mina Almasry <almasrymina@google.com>
Cc: Magnus Karlsson <magnus.karlsson@intel.com>,
	Maciej Fijalkowski <maciej.fijalkowski@intel.com>,
	Stanislav Fomichev <sdf@fomichev.me>,
	"David S. Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@kernel.org>,
	Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
	Simon Horman <horms@kernel.org>, Jonathan Corbet <corbet@lwn.net>,
	Shuah Khan <skhan@linuxfoundation.org>,
	Randy Dunlap <rdunlap@infradead.org>,
	Alexander Duyck <alexanderduyck@fb.com>,
	kernel-team@meta.com, Andrew Lunn <andrew+netdev@lunn.ch>,
	Jesper Dangaard Brouer <hawk@kernel.org>,
	Ilias Apalodimas <ilias.apalodimas@linaro.org>,
	Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	John Fastabend <john.fastabend@gmail.com>,
	Pavel Begunkov <asml.silence@gmail.com>,
	Jens Axboe <axboe@kernel.dk>, Andrii Nakryiko <andrii@kernel.org>,
	Eduard Zingerman <eddyz87@gmail.com>,
	Kumar Kartikeya Dwivedi <memxor@gmail.com>,
	Martin KaFai Lau <martin.lau@linux.dev>,
	Song Liu <song@kernel.org>,
	Yonghong Song <yonghong.song@linux.dev>,
	Jiri Olsa <jolsa@kernel.org>,
	Emil Tsalapatis <emil@etsalapatis.com>,
	Ihor Solodrai <ihor.solodrai@linux.dev>,
	netdev@vger.kernel.org, bpf@vger.kernel.org,
	io-uring@vger.kernel.org,
	"Mike Marciniszyn (Meta)" <mike.marciniszyn@gmail.com>,
	Weiming Shi <bestswngs@gmail.com>,
	Nikolay Aleksandrov <razor@blackwall.org>,
	David Wei <dw@davidwei.uk>,
	Alexander Lobakin <aleksander.lobakin@intel.com>,
	linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
	Stuart Clark <stuclar@meta.com>
Subject: Re: [PATCH net-next v1 02/11] net: Add memory provider capabilities
Date: Tue, 17 Nov 2026 18:06:52 +0100	[thread overview]
Message-ID: <87zev71j6b.fsf@all.your.base.are.belong.to.us> (raw)
In-Reply-To: <CAHS8izNVDU8+JzeuwDBmF24C-XqkO+ox01qnTvK665Y8pyWNLA@mail.gmail.com>

Mina Almasry <almasrymina@google.com> writes:

> On Sun, Nov 15, 2026 at 10:47 AM Björn Töpel <bjorn@kernel.org> wrote:
>>
>> Mina Almasry <almasrymina@google.com> writes:
>>
>> > On Tue, Nov 10, 2026 at 2:21 AM Björn Töpel <bjorn@kernel.org> wrote:
>> >>
>> >> The core treats every memory provider as unreadable: installing one
>> >> needs header split and no XDP program, and any page pool that allows
>> >> unreadable netmem takes it. An AF_XDP UMEM provider is readable, needs
>> >> XDP, and hands out whole buffers.
>> >>
>> >> Describe providers by MP_CAP_READABLE and MP_CAP_FRAG; devmem and
>> >> io_uring keep their behavior with MP_CAP_FRAG. A page pool takes a
>> >> readable provider only with the new PP_FLAG_MEM_PROVIDER:
>> >
>> > Why do we need this? I thought whatever creates the pool actually
>> > shouldn't get to pick whether the pool picks up the memory provider. I
>> > thought if core has configured the rxq for a memory provider then the
>> > pool is forced to take it.
>>
>> Thanks for looking at series, Mina. ...this needed some thinking.
>>
>> It isn't forced today either: only pools with
>> PP_FLAG_ALLOW_UNREADABLE_NETMEM take the queue's provider. Drivers set
>> it on the pool that holds payloads, and keep headers in host memory: in
>> a separate pool (fbnic, bnxt), or outside the page pool (gve, mlx5).
>> That works because every provider so far backs payloads only.
>>
>> UMEM doesn't. The socket gets whole packets, so UMEM has to back the
>> header buffers, and with multi-buffer the payloads too. The flag was my
>> way to get it there, but you're right that it's the wrong knob: it lets
>> the driver opt in, where it should only say what each pool holds.
>>
>> How about this instead. Both sides use the rx_roles bits from patch 5,
>> HEAD and PAYLOAD:
>>
>>  - The provider says which buffers it backs, next to rx_page_size in
>>    pp_memory_provider_params: devmem and io_uring zcrx PAYLOAD, AF_XDP
>>    HEAD, or HEAD|PAYLOAD with multi-buffer.
>>  - The driver says what each pool holds, in rx_roles. A pool without
>>    roles but with PP_FLAG_ALLOW_UNREADABLE_NETMEM holds PAYLOAD, so
>>    existing drivers don't change.
>>  - page_pool_create() gives the provider to a pool whose roles are all
>>    backed; the others get host pages. No opt-in.
>>  - After ndo_queue_mem_alloc(), the core checks that the new pools that
>>    took the provider cover everything it backs, or fails the restart.
>>
>> With fbnic's header and payload pools:
>>
>>   devmem:                header pool host, payload pool devmem
>>   AF_XDP single-buffer:  header pool UMEM, payload pool host
>>   AF_XDP multi-buffer:   one shared pool, UMEM
>>
>> A shared pool on a devmem queue doesn't take devmem, since devmem
>> doesn't back HEAD, so the check fails and the driver has to split.
>> That's the "headers in host memory" rule, without the driver asking
>> which provider it is.
>>
>> Wdyt?
>>
>
> Hmm, I think maybe the exsiting knobs PP_FLAG_ALLOW_UNREADABLE +
> CAP_READABLE covers your needs.
>
> The pp says whether the memory it provides is readable or not via
> CAP_READABLE (or !CAP_UNREADABLE).
>
> The driver says whether it will allow unreadable memory for this pool.
> For header pools, it doesn't specfiy PP_FLAG_ALLOW_UNREADABLE and for
> paylaod pools it specifies ALLOW_UNREADABLE.
>
> The change I think you need to make is: today we disallow any
> providers if the caller did not specify ALLOW_UNREADABLE (because we
> assume mp's are unreadable). After your series we only need to
> disallow memory providers if the mp is unreadable and the caller said
> it's not fine with unreadable.

I'm all in on making v1 simpler -- I'll try this path! So, an unreadable
provider still goes only to pools that set ALLOW_UNREADABLE. A readable
provider goes to all RX pools of the queue. The roles and
PP_FLAG_MEM_PROVIDER are gone.

One addition: a driver must opt in with a readable_mp flag in its
queue ops. Without it, its pools don't look for a provider. The reason
is that today only ALLOW_UNREADABLE pools must set queue_idx and be
created under the instance lock. Other pools may not, so they could
pick up the wrong queue's provider. Maybe there's a better way?

Thanks!


Björn

  reply	other threads:[~2026-11-17 17:06 UTC|newest]

Thread overview: 86+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-10 10:20 [PATCH net-next v1 00/11] xsk: Zero copy through page-pool memory providers Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 01/11] net: Get skb fragment addresses through netmem Björn Töpel
2026-11-12  8:32   ` saeed bishara
2026-11-12 13:57     ` Björn Töpel
2026-11-13  5:17   ` Mina Almasry
2026-11-13  9:08     ` Björn Töpel
2026-11-15 21:13   ` Jakub Kicinski
2026-11-16 14:31     ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 02/11] net: Add memory provider capabilities Björn Töpel
2026-11-10 22:19   ` Stanislav Fomichev
2026-11-11 12:41     ` Björn Töpel
2026-11-11 23:01       ` Stanislav Fomichev
2026-11-12 14:01         ` Björn Töpel
2026-11-13  0:21           ` Stanislav Fomichev
2026-11-15 18:47             ` Björn Töpel
2026-11-11 22:41   ` netdev-bot+sashiko
2026-11-13  5:30   ` Mina Almasry
2026-11-15 18:46     ` Björn Töpel
2026-11-16 21:01       ` Mina Almasry
2026-11-17 17:06         ` Björn Töpel [this message]
2026-11-15 21:18   ` Jakub Kicinski
2026-11-16 14:20     ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 03/11] xdp: Copy buffers with restricted lifetime Björn Töpel
2026-11-10 22:32   ` Stanislav Fomichev
2026-11-11 13:27     ` Björn Töpel
2026-11-11 23:05       ` Stanislav Fomichev
2026-11-11 22:41   ` netdev-bot+sashiko
2026-11-15 19:02     ` Björn Töpel
2026-11-13 18:54   ` Mina Almasry
2026-11-15 18:52     ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 04/11] xsk: Allocate UMEM chunks through page_pool Björn Töpel
2026-11-10 15:08   ` Magnus Karlsson
2026-11-11 13:51     ` Björn Töpel
2026-11-10 23:02   ` Stanislav Fomichev
2026-11-11 13:59     ` Björn Töpel
2026-11-11 23:09       ` Stanislav Fomichev
2026-11-11 22:41   ` netdev-bot+sashiko
2026-11-13 19:13   ` Mina Almasry
2026-11-15 18:55     ` Björn Töpel
2026-11-16 21:21       ` Mina Almasry
2026-11-17 17:15         ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 05/11] xsk: Install UMEM page pool providers in the core Björn Töpel
2026-11-10 23:50   ` Stanislav Fomichev
2026-11-11 14:01     ` Björn Töpel
2026-11-11  8:59   ` Magnus Karlsson
2026-11-11 19:26     ` Björn Töpel
2026-11-12  8:08       ` Magnus Karlsson
2026-11-11 22:41   ` netdev-bot+sashiko
2026-11-15 19:05     ` Björn Töpel
2026-11-12 11:53   ` Maciej Fijalkowski
2026-11-12 14:48     ` Björn Töpel
2026-11-12 15:43       ` Maciej Fijalkowski
2026-11-12 19:51         ` Björn Töpel
2026-11-13 19:17   ` Mina Almasry
2026-11-15 18:58     ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 06/11] page_pool: Coordinate refill with memory providers Björn Töpel
2026-11-10 23:51   ` Stanislav Fomichev
2026-11-11 19:37     ` Björn Töpel
2026-11-11 23:13       ` Stanislav Fomichev
2026-11-12 14:53         ` Björn Töpel
2026-11-11 22:41   ` netdev-bot+sashiko
2026-11-15 19:17     ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 07/11] xsk: Deliver provider chunks without copying Björn Töpel
2026-11-11 22:41   ` netdev-bot+sashiko
2026-11-15 19:23     ` Björn Töpel
2026-11-11 23:20   ` Stanislav Fomichev
2026-11-12 15:10     ` Björn Töpel
2026-11-12 12:17   ` Maciej Fijalkowski
2026-11-12 19:57     ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 08/11] eth: fbnic: Support XDP redirect Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 09/11] eth: fbnic: Use netmem for RX headers and XDP_TX Björn Töpel
2026-11-13 19:27   ` Mina Almasry
2026-11-10 10:20 ` [PATCH net-next v1 10/11] eth: fbnic: Support RX buffers it cannot split Björn Töpel
2026-11-11 21:45   ` Dimitri Daskalakis
2026-11-11 22:41   ` netdev-bot+sashiko
2026-11-15 19:12     ` Björn Töpel
2026-11-10 10:20 ` [PATCH net-next v1 11/11] eth: fbnic: Support AF_XDP page pool zero copy Björn Töpel
2026-11-11 22:41   ` netdev-bot+sashiko
2026-11-15 19:25     ` Björn Töpel
2026-11-12 11:37   ` Maciej Fijalkowski
2026-11-12 20:03     ` Björn Töpel
2026-11-13 11:40       ` Björn Töpel
2026-11-13 16:28         ` Maciej Fijalkowski
2026-11-15 18:59           ` Björn Töpel
2026-11-16 18:48   ` Dimitri Daskalakis
2026-11-17 17:14     ` Björn Töpel

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87zev71j6b.fsf@all.your.base.are.belong.to.us \
    --to=bjorn@kernel.org \
    --cc=aleksander.lobakin@intel.com \
    --cc=alexanderduyck@fb.com \
    --cc=almasrymina@google.com \
    --cc=andrew+netdev@lunn.ch \
    --cc=andrii@kernel.org \
    --cc=asml.silence@gmail.com \
    --cc=ast@kernel.org \
    --cc=axboe@kernel.dk \
    --cc=bestswngs@gmail.com \
    --cc=bpf@vger.kernel.org \
    --cc=corbet@lwn.net \
    --cc=daniel@iogearbox.net \
    --cc=davem@davemloft.net \
    --cc=dw@davidwei.uk \
    --cc=eddyz87@gmail.com \
    --cc=edumazet@kernel.org \
    --cc=emil@etsalapatis.com \
    --cc=hawk@kernel.org \
    --cc=horms@kernel.org \
    --cc=ihor.solodrai@linux.dev \
    --cc=ilias.apalodimas@linaro.org \
    --cc=io-uring@vger.kernel.org \
    --cc=john.fastabend@gmail.com \
    --cc=jolsa@kernel.org \
    --cc=kernel-team@meta.com \
    --cc=kuba@kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maciej.fijalkowski@intel.com \
    --cc=magnus.karlsson@intel.com \
    --cc=martin.lau@linux.dev \
    --cc=memxor@gmail.com \
    --cc=mike.marciniszyn@gmail.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=razor@blackwall.org \
    --cc=rdunlap@infradead.org \
    --cc=sdf@fomichev.me \
    --cc=skhan@linuxfoundation.org \
    --cc=song@kernel.org \
    --cc=stuclar@meta.com \
    --cc=yonghong.song@linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®