From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D009437C106 for ; Thu, 26 Mar 2026 17:02:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774544564; cv=none; b=CXiJ569oI8dPW1UjxIt4FAPtmzsfLJtJR3WGL3mtK2vYay95ckaq8Nv0Qs/1FGl14WH/bkfC77kJyaj0G+ukcvpb54xisFcp1yP+/grMrsnXWYN3rQljagwUVLfOtvBDhXWSBDuzkclxZfKs5Mb2yUqKyjDiDUoKkrNqGJkhLYA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774544564; c=relaxed/simple; bh=6A8JoCjSKsdZBMWA9MOK2LfEQrLWoM6rgQMbwDedt+w=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=jXsCURrde9k9RypM38F4sv9xtXBCElqxUZ484B9cwaK1w6OCtXNsNvNM73IVZ1X5GxEA1IH417oy5YFDJmMMmzLOgUn7xtesu+OmGZ6wrNKoV7JnGA9vlbMGBBcxrB2yWddxaQSOyhrbMHUcfAaK9ntTzcKPYrnMhGPeri7dP08= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=dama.to; spf=none smtp.mailfrom=dama.to; dkim=pass (2048-bit key) header.d=dama-to.20230601.gappssmtp.com header.i=@dama-to.20230601.gappssmtp.com header.b=t1bWYVf4; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=dama.to Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=dama.to Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=dama-to.20230601.gappssmtp.com header.i=@dama-to.20230601.gappssmtp.com header.b="t1bWYVf4" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2ab232cc803so5568925ad.3 for ; Thu, 26 Mar 2026 10:02:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dama-to.20230601.gappssmtp.com; s=20230601; t=1774544562; x=1775149362; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:from:to:cc :subject:date:message-id:reply-to; bh=Oth5mhTJ39EvtSrKINf1neXZ7BWti1esZSUQXviFbSk=; b=t1bWYVf4mCjHqcEpeQrhNw3XS7A23/t5af1GvJMdTR90sKyFWxrtR0IeLRU0uwtoa0 w58Soy5ZcRIvz4LcoTZvqJALypQ3medd74HJqGBaeOFHlO0LfTkVTuA5OCmQ22fIW1ud 2p0FF3/nPAbR9/Y0QCevXfG3yBgpQDIXtH5MVb+slpesDJ9THhe859xXmTH91OPKMEo5 mucfoDjVzPkTIVwGpFH+EMzZ/dWe3rBlaTswIL+MKBmdwiJPOUjHBSKouAaaPzXpAYKt 0/11J8IthTTe0h/u+OV9LIbGIsUTuN4v+izTiRfE1K+cPBzZ4+7YZD+mwlbm+CTPORc1 Jhkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774544562; x=1775149362; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Oth5mhTJ39EvtSrKINf1neXZ7BWti1esZSUQXviFbSk=; b=R5aXhwV4dUw3nEACwTK0F+T7ImSqH549VWSLBzA+RrNkphox6mYZfAC5qlEr/l+0oO 4fzLtdcngr3raWz6yw3Q1J/MNSm5SSbzMrVqxU6otogwOF3gk3WYAdWashrSPBDJrfwq IWN0cvd/BGTUML6VbVJY+eG9GTsHB98aFnsSA1Rg/z9YgNAj47Zb92cj3Akcf8Tc/k1G +CosvKHocT0OfACPCkNvgBwh0R+VI+Ap+JDZPoLq7qjFIISEXrWxbW4pTqVOjKvZZM59 xNTjPhOPJzUpcZmH1SHOjDpd5oR1lwMn3Zw/SF/yJk+JNGkxnfbw1HFQAqLu2CvIEjGg FKkQ== X-Forwarded-Encrypted: i=1; AJvYcCXL7KT/WX5PgfKw+ve6HP6XhWygCBcbvKJKPepeBlkuPgOKEwYq3N1ZzCGQqoY4Y7Gxqy6Bn2+fKdHjH5U=@vger.kernel.org X-Gm-Message-State: AOJu0YwBxLzwE2lbXGwqZGSEX5zmjYT1cTzIphV3GRjqC7Ib+0EDFvF+ 4ac/00W6nRcnPWmEgP0uFCBQ18SKK4AmP/CtlEDe1MgtMuipqTdrHER7I/IgxL2qufQ= X-Gm-Gg: ATEYQzx8RdB6twYpsoSiVraUf+yYfSXkaa8pWl4YuKTLVy2fiu5ZCTHvoYNAuEoYtls FkdEWg5kAkdR/uwuSH33XkXWy5EOE6BC9rmpZhFXqfBkuFgl2yZczcdbjyvEB8Y2qgs08XsX3Uk MpTc6KgPlRy7/OIjf6iUVwTLYv5FjcekflSaqrw3Tb5R4RtyVMTC6q4LGTWjmip50vPkIdEc9sx nEGcWaoOniQ9l/NN44oHUG+oE5Tv22ebfOII4m38CuzbZCR/JfunLXEco5MjL9dLQKClTIFRNPg 6Q38Yq00YZbEXTqBmuydyxpglThWaflOzm+KHFokq/kF39gmDWLei5180e/Z6KlzSn0PcfeM/Xa cfKfHiugRZlmBO5raQ0e7bfAJ5/hq9IGXWBLTCIStAqgkgY/L4f9JCZujrP+kly27IBqfsTVrX8 iYFkjt X-Received: by 2002:a17:903:1a86:b0:2ae:7f28:124b with SMTP id d9443c01a7336-2b0b0a153ffmr82939605ad.22.1774544562088; Thu, 26 Mar 2026 10:02:42 -0700 (PDT) Received: from localhost ([2a03:2880:2ff:50::]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2b0bc76b8c3sm47872255ad.14.2026.03.26.10.02.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Mar 2026 10:02:41 -0700 (PDT) Date: Thu, 26 Mar 2026 10:02:39 -0700 From: Joe Damato To: Paolo Abeni Cc: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , horms@kernel.org, linux-kernel@vger.kernel.org, leon@kernel.org Subject: Re: [net-next v5 09/12] net: bnxt: Add SW GSO completion and teardown support Message-ID: Mail-Followup-To: Joe Damato , Paolo Abeni , netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , horms@kernel.org, linux-kernel@vger.kernel.org, leon@kernel.org References: <20260323183844.3146982-1-joe@dama.to> <20260323183844.3146982-10-joe@dama.to> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Thu, Mar 26, 2026 at 01:39:17PM +0100, Paolo Abeni wrote: > On 3/23/26 7:38 PM, Joe Damato wrote: > > Update __bnxt_tx_int and bnxt_free_one_tx_ring_skbs to handle SW GSO > > segments: > > > > - MID segments: adjust tx_pkts/tx_bytes accounting and skip skb free > > (the skb is shared across all segments and freed only once) > > > > - LAST segments: if the DMA IOVA path was used, use dma_iova_destroy to > > tear down the contiguous mapping. On the fallback path, payload DMA > > unmapping is handled by the existing per-BD dma_unmap_len walk. > > > > Both MID and LAST completions advance tx_inline_cons to release the > > segment's inline header slot back to the ring. > > > > is_sw_gso is initialized to zero, so the new code paths are not run. > > > > Suggested-by: Jakub Kicinski > > Reviewed-by: Pavan Chebbi > > Signed-off-by: Joe Damato > > --- > > v5: > > - Added Pavan's Reviewed-by. No functional changes. > > > > v3: > > - completion paths updated to use DMA IOVA APIs to teardown mappings. > > > > rfcv2: > > - Update the shared header buffer consumer on TX completion. > > > > drivers/net/ethernet/broadcom/bnxt/bnxt.c | 82 +++++++++++++++++-- > > .../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 19 ++++- > > 2 files changed, 91 insertions(+), 10 deletions(-) > > > > diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c > > index 2759a4e2b148..40a16f96feba 100644 > > --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c > > +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c > > @@ -74,6 +74,8 @@ > > #include "bnxt_debugfs.h" > > #include "bnxt_coredump.h" > > #include "bnxt_hwmon.h" > > +#include "bnxt_gso.h" > > +#include > > > > #define BNXT_TX_TIMEOUT (5 * HZ) > > #define BNXT_DEF_MSG_ENABLE (NETIF_MSG_DRV | NETIF_MSG_HW | \ > > @@ -817,12 +819,13 @@ static bool __bnxt_tx_int(struct bnxt *bp, struct bnxt_tx_ring_info *txr, > > bool rc = false; > > > > while (RING_TX(bp, cons) != hw_cons) { > > - struct bnxt_sw_tx_bd *tx_buf; > > + struct bnxt_sw_tx_bd *tx_buf, *head_buf; > > struct sk_buff *skb; > > bool is_ts_pkt; > > int j, last; > > > > tx_buf = &txr->tx_buf_ring[RING_TX(bp, cons)]; > > + head_buf = tx_buf; > > skb = tx_buf->skb; > > > > if (unlikely(!skb)) { > > @@ -869,6 +872,23 @@ static bool __bnxt_tx_int(struct bnxt *bp, struct bnxt_tx_ring_info *txr, > > DMA_TO_DEVICE, 0); > > } > > } > > + > > + if (unlikely(head_buf->is_sw_gso)) { > > + txr->tx_inline_cons++; > > + if (head_buf->is_sw_gso == BNXT_SW_GSO_LAST) { > > + if (dma_use_iova(&head_buf->iova_state)) > > I'm likely lost, but AFAICS the previous patch/bnxt_sw_udp_gso_xmit() > initialize head_buf->iova_state only when > `dma_use_iova(&head_buf->iova_state) == true`. I.e. in fallback scenario > the previous iova_state is maintained. Note that calling dma_iova_try_alloc zeroes the state before returning whether the IOVA DMA API can be used or not and I call that uncoditionally (see below). > Additionally AFAICS dma_iova_destroy does not clear `head_buf->iova_state`. That's my understanding, too, that dma_iova_destroy doesn't clear the state. > It looks like that 2 consecutive skb hitting the same slot use a > different dma mapping strategy (fallback vs iova) bat things will > happen?!? should the previous patch always initializing > head_buf->iova_state? AFAICT, to switch the IOMMU domain would require unbind the device, changing the IOMMU type, and re-binding the device... which would destroy all the rings in the process and thus this wouldn't happen. The only way I could potentially imagine this happening would be in extreme IOVA pressure (maybe?): - packet A in slot N, dma_iova_try_alloc suceeds -> head_buf->iova_state copied - completion the packet occurs, dma_iova_destroy is called, head_buf->iova_state is not cleared - packet B in slot N, dma_iova_try_alloc fails due to IOVA pressure... head_buf->iova_state is stale I'm pretty skeptical that this is a realistic case, TBH. That said and since it seems my v5 got CR, I can send a v6 with this slight change to address the case you've mentioned above. I'll send in a couple hours unless I hear otherwise: diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c index 9c30ee063ef5..7c198847a771 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_gso.c @@ -142,8 +142,12 @@ netdev_tx_t bnxt_sw_udp_gso_xmit(struct bnxt *bp, tx_buf->is_sw_gso = last ? BNXT_SW_GSO_LAST : BNXT_SW_GSO_MID; - /* Store IOVA state on the last segment for completion */ - if (last && tso_dma_map_use_iova(&map)) { + /* Store IOVA state on the last segment for completion. + * Always copy so that a stale iova_state from a prior + * occupant of this ring slot cannot be misread by + * dma_use_iova() in the completion path. + */ + if (last) { tx_buf->iova_state = map.iova_state; tx_buf->iova_total_len = map.total_len; }