Re: [PATCH net-next v1 04/11] xsk: Allocate UMEM chunks through page_pool

["Björn Töpel" <bjorn@kernel.org>]

Stanislav Fomichev <sdf.kernel@gmail.com> writes:

> On 11/10, Björn Töpel wrote:

>> diff --git a/net/xdp/xsk_pp.c b/net/xdp/xsk_pp.c
>> new file mode 100644
>> index 000000000000..fdd55265a6cd
>> --- /dev/null
>> +++ b/net/xdp/xsk_pp.c
>> @@ -0,0 +1,246 @@
>> +// SPDX-License-Identifier: GPL-2.0
>> +/* AF_XDP UMEM as a page pool memory provider
>> + *
>> + * Each aligned UMEM chunk is a readable net_iov with one owner at a time:
>> + *
>> + *   userspace --FILL--> page pool --release--> reuse --alloc--> page pool
>> + *       ^                   |
>> + *       +-----RX ring-------+
>> + */
>> +
>> +#include <net/page_pool/helpers.h>
>> +#include <net/page_pool/memory_provider.h>
>> +#include <net/xdp_sock_drv.h>
>> +
>> +#include "xsk.h"
>> +#include "xsk_queue.h"
>> +
>> +enum xsk_pp_chunk {
>> +	XSK_PP_USER,	/* owned by userspace */
>> +	XSK_PP_POOL,	/* owned by a page pool */
>> +	XSK_PP_REUSE,	/* on the reuse stack */
>
> Can you explain more about that reuse/reuse_cnt? When is the chunk reused?
> How is it different from FILL->RX path?


            FILL             RX ring
  USER ----------> POOL ---------------> USER
                   |  ^
   release_netmem  |  |  next refill,
                   v  |  before FILL
                  REUSE

POOL is a chunk in the page pool or on a driver ring. XDP_DROP, copied
XDP_PASS and XDP_TX recycle it inside the page pool, so it stays POOL.
FILL->RX is the only way back to userspace: an XSK redirect puts the
chunk on the RX ring, and the flush (patch 7) marks it USER. Userspace
posts it to FILL again.

REUSE is a chunk the page pool lets go of without a packet: when the
pool is destroyed (queue restart, unbind), or its ring is full on a
recycle. Userspace already gave it up, and the kernel can't post to
FILL, so the next refill takes it before FILL. Same job as free_list in
xsk_buff_pool. A FILL entry for a chunk the kernel holds is dropped.

I'll put that in the comment at the top of xsk_pp.c.


Björn