Re: [PATCH net v2 1/2] eth: fbnic: validate the Rx completion buffer index

Re: [PATCH net v2 1/2] eth: fbnic: validate the Rx completion buffer index

[netdev-bot+sashiko]

Thank you for your contribution! Sashiko AI review found 3 potential
issue(s) to consider.

Critical: 0 · High: 2 (2) · Medium: 1 · Low: 0

If you believe none of the issues are legitimate, please reply to each
issue of Medium severity or higher and explain why the AI is wrong.
If any of them are legitimate and you plan to respin, please let patchwork
know by including "pw-bot: cr" as a separate line at the end of your reply
(one such reply per series is enough).

- [High] The new checks only test that the device-supplied buffer ID or
  head fits inside rx_buf[] (`<= size_mask`).
- [High] A META completion with no HDR_AL completion before it still
  builds an skb on a NULL buffer.
- [Medium] fbnic_clean_rcq() sets head0/head1 from the raw descriptor
  (lines 1318 and 1323) before fbnic_pkt_prepare()/fbnic_add_rx_frag()…

Pre-existing issues:
- [High] This problem wasn't introduced by this patch, but
  fbnic_clean_twq1() has the same 'masked running index never reaches an…
- [High] This problem wasn't introduced by this patch, but the sibling
  fields of the same untrusted Rx completion descriptor are still…

--

I found five issues in this patch:

- Three are regressions or incomplete fixes, rated High, High and Medium.
- Two are High-severity problems that already existed before the patch and are related to it.

The main gap is that the new checks only test whether an index fits the rx_buf[] array. They don't test whether the slot was actually posted, so empty in-range slots can still reach a NULL dereference. A bare metadata completion can also still build an skb on a NULL buffer.

eth: fbnic: validate the Rx completion buffer index

This patch checks the device-supplied buffer ID in Rx address/length
completions against the BDQ size_mask in fbnic_pkt_prepare() and
fbnic_add_rx_frag(). It also bounds the reported head in
fbnic_clean_bdq(). Finally, it marks the packet as failed when a payload
completion arrives without a header completion before it.

> diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
> index 10caacffee0f0..b4ca9f7e8bb2e 100644
> --- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
> +++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
> @@ -882,6 +882,13 @@ static void fbnic_clean_bdq(struct fbnic_ring *ring, unsigned int hw_head,
>  {
>  	unsigned int head = ring->head;
>  
> +	/* A head outside the ring cannot name a buffer we posted, and the
> +	 * loop below masks head to the ring so it would never reach it.
> +	 * Leave the ring alone and wait for a head we can act on.
> +	 */
> +	if (unlikely(hw_head > ring->size_mask))
> +		return;

[Severity: Medium]
Can this early return throw away valid progress from earlier in the same
poll?

fbnic_clean_rcq() records head0/head1 straight from the descriptor. It
does this before fbnic_pkt_prepare() or fbnic_add_rx_frag() has a chance
to reject the ID:

fbnic_clean_rcq() {
    ...
		case FBNIC_RCD_TYPE_HDR_AL:
			head0 = FIELD_GET(FBNIC_RCD_AL_BUFF_PAGE_MASK, rcd);
			fbnic_pkt_prepare(nv, rcd, pkt, qt);
    ...
	if (head0 >= 0)
		fbnic_clean_bdq(&qt->sub0, head0, budget);
	fbnic_fill_bdq(&qt->sub0);
    ...
}

If the last HDR_AL or PAY_AL of a poll carries a bad ID, it overwrites
any valid head recorded earlier in that poll. fbnic_clean_bdq() then
reclaims nothing.

On a full ring whose ring->head does not advance, fbnic_desc_unused()
returns 0, so fbnic_fill_bdq() posts nothing.

Suppose the device has already used up its posted buffers when it emits
the bad ID, or it keeps emitting bad IDs. Would a valid head ever arrive?
If not, the Rx queue could stall until reset, and the only sign would be
the dropped counter.

Would validating the ID in fbnic_clean_rcq() before assigning
head0/head1 keep the earlier progress? That would also match the
per-packet drop the patch uses elsewhere.

[Severity: High]
This isn't a bug introduced by this patch, but fbnic_clean_twq1() looks
like it has the same masked-index loop problem the commit message
describes for fbnic_clean_bdq():

fbnic_clean_tcq() {
    ...
		case FBNIC_TCD_TYPE_0:
			if (tcd & FBNIC_TCD_TWQ1)
				head1 = FIELD_GET(FBNIC_TCD_TYPE0_HEAD1_MASK,
						  tcd);
    ...
}

fbnic_clean_twq1() {
    ...
	while (hw_head != head) {
		...
		page_pool_put_page(pp_page_to_nmdesc(page)->pp, page, -1,
				   pp_allow_direct);
next_desc:
		head++;
		head &= ring->size_mask;
	}
    ...
}

FBNIC_TCD_TYPE0_HEAD1_MASK is DESC_GENMASK(31, 16), so head1 can be as
large as 65535. The default Tx ring has 1024 entries.

If HEAD1 is above sub1.size_mask, does this loop ever end in NAPI
context? Neither tx_buf[] nor the descriptors are cleared. Each pass
would therefore call page_pool_put_page() again on the same XDP_TX pages.

fbnic_clean_twq0() does not seem to have this problem, because it
computes clean_desc as (hw_head - head) & size_mask.

> +
>  	if (head == hw_head)
>  		return;
>  

[ ... ]

> @@ -985,14 +992,24 @@ static void fbnic_pkt_prepare(struct fbnic_napi_vector *nv, u64 rcd,
>  {
>  	unsigned int hdr_pg_idx = FIELD_GET(FBNIC_RCD_AL_BUFF_PAGE_MASK, rcd);
>  	unsigned int hdr_pg_off = FIELD_GET(FBNIC_RCD_AL_BUFF_OFF_MASK, rcd);
> -	struct page *page = fbnic_page_pool_get_head(qt, hdr_pg_idx);
>  	unsigned int len = FIELD_GET(FBNIC_RCD_AL_BUFF_LEN_MASK, rcd);
>  	unsigned int frame_sz, hdr_pg_start, hdr_pg_end, headroom;
>  	unsigned char *hdr_start;
> +	struct page *page;
>  
>  	/* data_hard_start should always be NULL when this is called */
>  	WARN_ON_ONCE(pkt->buff.data_hard_start);
>  
> +	/* A buffer ID the device never got from us cannot name a buffer we
> +	 * own, so drop the packet instead of running off the end of rx_buf[].
> +	 */
> +	if (unlikely(hdr_pg_idx > qt->sub0.size_mask)) {
> +		pkt->add_frag_failed = true;
> +		return;
> +	}
> +
> +	page = fbnic_page_pool_get_head(qt, hdr_pg_idx);

[Severity: High]
Does this check cover the "never posted" case that the comment and the
commit message describe? It only checks the rx_buf[] array bound. It
does not check that the slot is in the posted window between ring->head
and ring->tail.

Slots outside that window have netmem == 0, for three reasons:

- fbnic_page_pool_drain() clears netmem on every slot it consumes, and
  rx_buf[] starts out zeroed.
- fbnic_fill_bdq() never fills slot (head - 1), because
  fbnic_desc_unused() returns (head - tail - 1) & size_mask.
- fbnic_fill_bdq() also stops early when page_pool_dev_alloc_netmems()
  fails.

With hdr_pg_idx == (sub0.head - 1) & size_mask, the new check passes.
fbnic_page_pool_get_head() then returns netmem_to_page(0), which is NULL.
Would page_pool_get_dma_addr(page) and page_address(page) then
dereference NULL?

fbnic_add_rx_frag() has the same gap for sub1. There, netmem 0 is passed
to page_pool_dma_sync_netmem_for_cpu() and xdp_buff_add_frag().

The new fbnic_clean_bdq() guard has a similar limit. Take a ring where
fbnic_fill_bdq() stopped early, say size_mask = 255, head = 10 and
tail = 200. An in-range hw_head of 250 passes the new check. The loop
drains slots 10..199, reaches slot 200, whose netmem is 0, and calls
page_pool_unref_netmem() on it:

fbnic_page_pool_drain() {
	...
	netmem_ref netmem = rx_buf->netmem;

	if (!page_pool_unref_netmem(netmem, rx_buf->pagecnt_bias))
	...
}

Wouldn't that reproduce the "null-ptr-deref in fbnic_clean_bdq" splat
quoted in the commit message? It would also move ring->head past
ring->tail.

Would checking against the posted window, for example
((idx - ring->head) & size_mask) < fbnic_desc_used(ring), cover both
the buffer ID and the hw_head cases?

>  	/* Short-cut the end calculation if we know page is fully consumed */
>  	hdr_pg_end = FIELD_GET(FBNIC_RCD_AL_PAGE_FIN, rcd) ?
>  		     FBNIC_BD_FRAG_SIZE : fbnic_hdr_pg_end(hdr_pg_off, len);

[Severity: High]
This is a pre-existing issue, but the other fields of the same untrusted
completion descriptor are still unchecked.

FBNIC_RCD_AL_BUFF_OFF_MASK is 12 bits and FBNIC_RCD_AL_BUFF_LEN_MASK is
13 bits. A buffer fragment is FBNIC_BD_FRAG_SIZE, which is 4096 bytes.

fbnic_hdr_pg_start() does unsigned arithmetic:

fbnic_hdr_pg_start() {
	...
	return pg_off - FBNIC_RX_HROOM;
}

Suppose the offset is below FBNIC_RX_HROOM and is not the special-cased
ALIGN(FBNIC_RX_HROOM, 128). Does hdr_pg_start then wrap to about 4G?
That value feeds dma_sync_single_range_for_cpu() and
hdr_start = page_address(page) + hdr_pg_start.

An oversized len would push data_end past the page through
xdp_prepare_buff(). It would also enlarge frame_sz, so the shared info
written by __build_skb_around() or xdp_buff_add_frag() would land outside
the fragment.

fbnic_add_rx_frag() has a similar gap: truesize and len are not clamped
to FBNIC_BD_FRAG_SIZE - pg_off before the DMA sync and
xdp_buff_add_frag().

[ ... ]

> @@ -1026,10 +1043,27 @@ static void fbnic_add_rx_frag(struct fbnic_napi_vector *nv, u64 rcd,
>  	unsigned int pg_idx = FIELD_GET(FBNIC_RCD_AL_BUFF_PAGE_MASK, rcd);
>  	unsigned int pg_off = FIELD_GET(FBNIC_RCD_AL_BUFF_OFF_MASK, rcd);
>  	unsigned int len = FIELD_GET(FBNIC_RCD_AL_BUFF_LEN_MASK, rcd);
> -	netmem_ref netmem = fbnic_page_pool_get_data(qt, pg_idx);
>  	unsigned int truesize;
> +	netmem_ref netmem;
>  	bool added;
>  
> +	/* No header completion has set buff up, either because one was
> +	 * rejected or because the device sent none, so there is nothing to
> +	 * add a fragment to.  Mark the packet so the metadata descriptor
> +	 * drops it instead of building an skb on an empty buffer.
> +	 */
> +	if (unlikely(!pkt->buff.data_hard_start)) {
> +		pkt->add_frag_failed = true;
> +		return;
> +	}

[Severity: High]
This only runs when a PAY_AL completion arrives. What happens when the
device sends a META completion with no HDR_AL before it?

After the previous packet's META, fbnic_clean_rcq() sets
pkt->buff.data_hard_start to NULL. add_frag_failed keeps the false value
left by the last successful fbnic_pkt_prepare(). After fbnic_nv_flush(),
the whole pkt is zeroed. Either way, a bare META still passes the good
packet test:

fbnic_clean_rcq() {
    ...
		case FBNIC_RCD_TYPE_META:
			if (likely(!fbnic_rcd_metadata_err(rcd) &&
				   !pkt->add_frag_failed)) {
				pkt_bytes = xdp_get_buff_len(&pkt->buff);
				skb = fbnic_run_xdp(nv, pkt);
			}
    ...
}

With no XDP program attached, this goes
fbnic_build_skb()->xdp_build_skb_from_buff()->napi_build_skb() with a
NULL data_hard_start and a stale frame_sz. Isn't that the same
"null-ptr-deref in __build_skb_around" the commit message says it fixes?

With an XDP program attached, would the program run on the previous
packet's data/data_end? That buffer has already been handed to the stack
or recycled to the page_pool.

Should the META case check pkt->buff.data_hard_start itself?

> +
> +	if (unlikely(pg_idx > qt->sub1.size_mask)) {
> +		pkt->add_frag_failed = true;
> +		return;
> +	}
> +
> +	netmem = fbnic_page_pool_get_data(qt, pg_idx);

[ ... ]

-- 
Sashiko AI review · https://netdev-ai.bots.linux.dev/sashiko/#/patchset/cover.1791659860.git.yhlee%40isslab.korea.ac.kr

[PATCH net v2 1/2] eth: fbnic: validate the Rx completion buffer index

[Yehyeong Lee]

The buffer ID in an address/length Rx completion descriptor is a 16 bit
field, FBNIC_RCD_AL_BUFF_PAGE_MASK being DESC_GENMASK(15, 0) once the
fragment bits are removed, so the device can name any index from 0 to
65535.  fbnic_pkt_prepare() and fbnic_add_rx_frag() feed that value
straight into fbnic_page_pool_get_head() and fbnic_page_pool_get_data(),
which use it to subscript qt->sub0.rx_buf[] and qt->sub1.rx_buf[], and
fbnic_clean_rcq() keeps the same value in head0/head1 and hands it to
fbnic_clean_bdq() as the position the device has consumed up to.

Those arrays only cover the buffers the driver posted.  They are
allocated with size_mask + 1 entries, and size_mask comes from
hpq_size / FBNIC_BD_FRAG_COUNT, so at the default FBNIC_HPQ_SIZE_DEFAULT
of 256 each array holds 256 entries of 16 bytes, that is 4096 bytes.  A
buffer ID of 65535 reaches 1048560 bytes past the start of the
allocation.

The fill side already masks every index it computes against
bdq->size_mask, and so does all of the ring head and tail arithmetic.
The completion side does not.  A descriptor naming a buffer that was
never posted makes fbnic_page_pool_get_head() decrement pagecnt_bias
through a pointer outside the array and hand the out-of-bounds
netmem_ref back as a struct page; page_pool_get_dma_addr() and
page_address() then dereference it and the skb is built on whatever it
points at.

Injecting an out-of-range header completion reports under KASAN:

  BUG: KASAN: slab-out-of-bounds in fbnic_page_pool_get_head
  Read of size 8 at addr ffff88800c719008
  The buggy address is located 8 bytes to the right of
   allocated 4096-byte region [ffff88800c718000, ffff88800c719000)

Check the ID against size_mask before it is used and drop the packet if
it is out of range.  Masking it instead would keep the access in bounds
but would silently attribute the completion to an unrelated buffer that
the driver does own, which is worse than a drop.  Set add_frag_failed so
the frame takes the existing error path.

fbnic_clean_bdq() needs the same value bounded, for a different reason.
It walks from ring->head to the head the device reported, masking its
own running index against size_mask on every step:

	do {
		fbnic_page_pool_drain(ring, head, napi_budget);

		head++;
		head &= ring->size_mask;
	} while (head != hw_head);

A hw_head above size_mask is therefore never reached.  The loop drains
every buffer on the ring, wraps, and comes back to a slot whose netmem
fbnic_page_pool_drain() has already cleared, so page_pool_unref_netmem()
runs on a NULL netmem:

  BUG: KASAN: null-ptr-deref in fbnic_clean_bdq
  Read of size 8 at addr 0000000000000028

Return without touching the ring in that case.  head0 and head1 are
recomputed on every poll, so the first head the driver can act on lets
the clean-up catch up from ring->head and no buffer is leaked.  Masking
the head instead would release buffers the device never said it was done
with.

Finally, fbnic_add_rx_frag() has to mark the packet when it bails out on
a buffer that was never set up.  That bail is reached both when a header
completion was rejected and when the device sends a payload completion
with no header completion ahead of it, and in the second case
add_frag_failed is still clear, so the metadata descriptor treats the
packet as good and builds an skb on an empty buffer:

  BUG: KASAN: null-ptr-deref in __build_skb_around
  Write of size 32 at addr 0000000000000ec0

Set add_frag_failed there so the metadata case drops the frame instead.
On the normal path data_hard_start is always set, so the bail is not
taken and the buffer accounting is untouched.

Fixes: a29b8eb6e533 ("eth: fbnic: Add basic Rx handling")
Cc: stable@vger.kernel.org
Signed-off-by: Yehyeong Lee <yhlee@isslab.korea.ac.kr>
---
 drivers/net/ethernet/meta/fbnic/fbnic_txrx.c | 38 ++++++++++++++++++--
 1 file changed, 36 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
index 10caacffee0f0..b4ca9f7e8bb2e 100644
--- a/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
+++ b/drivers/net/ethernet/meta/fbnic/fbnic_txrx.c
@@ -882,6 +882,13 @@ static void fbnic_clean_bdq(struct fbnic_ring *ring, unsigned int hw_head,
 {
 	unsigned int head = ring->head;
 
+	/* A head outside the ring cannot name a buffer we posted, and the
+	 * loop below masks head to the ring so it would never reach it.
+	 * Leave the ring alone and wait for a head we can act on.
+	 */
+	if (unlikely(hw_head > ring->size_mask))
+		return;
+
 	if (head == hw_head)
 		return;
 
@@ -985,14 +992,24 @@ static void fbnic_pkt_prepare(struct fbnic_napi_vector *nv, u64 rcd,
 {
 	unsigned int hdr_pg_idx = FIELD_GET(FBNIC_RCD_AL_BUFF_PAGE_MASK, rcd);
 	unsigned int hdr_pg_off = FIELD_GET(FBNIC_RCD_AL_BUFF_OFF_MASK, rcd);
-	struct page *page = fbnic_page_pool_get_head(qt, hdr_pg_idx);
 	unsigned int len = FIELD_GET(FBNIC_RCD_AL_BUFF_LEN_MASK, rcd);
 	unsigned int frame_sz, hdr_pg_start, hdr_pg_end, headroom;
 	unsigned char *hdr_start;
+	struct page *page;
 
 	/* data_hard_start should always be NULL when this is called */
 	WARN_ON_ONCE(pkt->buff.data_hard_start);
 
+	/* A buffer ID the device never got from us cannot name a buffer we
+	 * own, so drop the packet instead of running off the end of rx_buf[].
+	 */
+	if (unlikely(hdr_pg_idx > qt->sub0.size_mask)) {
+		pkt->add_frag_failed = true;
+		return;
+	}
+
+	page = fbnic_page_pool_get_head(qt, hdr_pg_idx);
+
 	/* Short-cut the end calculation if we know page is fully consumed */
 	hdr_pg_end = FIELD_GET(FBNIC_RCD_AL_PAGE_FIN, rcd) ?
 		     FBNIC_BD_FRAG_SIZE : fbnic_hdr_pg_end(hdr_pg_off, len);
@@ -1026,10 +1043,27 @@ static void fbnic_add_rx_frag(struct fbnic_napi_vector *nv, u64 rcd,
 	unsigned int pg_idx = FIELD_GET(FBNIC_RCD_AL_BUFF_PAGE_MASK, rcd);
 	unsigned int pg_off = FIELD_GET(FBNIC_RCD_AL_BUFF_OFF_MASK, rcd);
 	unsigned int len = FIELD_GET(FBNIC_RCD_AL_BUFF_LEN_MASK, rcd);
-	netmem_ref netmem = fbnic_page_pool_get_data(qt, pg_idx);
 	unsigned int truesize;
+	netmem_ref netmem;
 	bool added;
 
+	/* No header completion has set buff up, either because one was
+	 * rejected or because the device sent none, so there is nothing to
+	 * add a fragment to.  Mark the packet so the metadata descriptor
+	 * drops it instead of building an skb on an empty buffer.
+	 */
+	if (unlikely(!pkt->buff.data_hard_start)) {
+		pkt->add_frag_failed = true;
+		return;
+	}
+
+	if (unlikely(pg_idx > qt->sub1.size_mask)) {
+		pkt->add_frag_failed = true;
+		return;
+	}
+
+	netmem = fbnic_page_pool_get_data(qt, pg_idx);
+
 	truesize = FIELD_GET(FBNIC_RCD_AL_PAGE_FIN, rcd) ?
 		   FBNIC_BD_FRAG_SIZE - pg_off : ALIGN(len, 128);
 
-- 
2.43.0