[PATCH v2 net-next 0/2] Enable Big TCP for MANA devices

[PATCH v2 net-next 0/2] Enable Big TCP for MANA devices

[Shradha Gupta]

Allow the max gso/gro aggregated pkt size to go up to GSO_MAX_SIZE for
MANA NIC. On Azure, this not possible without allowing the same for
netvsc NIC (as the NICs are bonded together).
Therefore, we use netif_set_tso_max_size() to set max aggregated pkt size
to VF's tso_max_size for netvsc too, when the data path is switched over
to the VF

The first patch allows MANA to configure aggregated pkt size of up-to
GSO_MAX_SIZE

The second patch enables the same on the netvsc NIC, if the data path
for the bonded NIC is switched to the VF

 Changes in v2
 * Instead of using 'tcp segment' throughout the patch used the words
   'aggregated pkt size'

Shradha Gupta (2):
  net: mana: Allow tso_max_size to go up-to GSO_MAX_SIZE in MANA
  hv_netvsc: Use VF's tso_max_size value when data path is VF

 drivers/net/ethernet/microsoft/mana/mana_en.c |  2 ++
 drivers/net/hyperv/hyperv_net.h               |  2 ++
 drivers/net/hyperv/netvsc_drv.c               | 15 +++++++++++++++
 drivers/net/hyperv/rndis_filter.c             | 13 +++++++------
 4 files changed, 26 insertions(+), 6 deletions(-)

-- 
2.34.1

[PATCH v2 net-next 1/2] net: mana: Allow tso_max_size to go up-to GSO_MAX_SIZE

[Shradha Gupta]

Allow the max aggregated pkt size to go up-to GSO_MAX_SIZE for MANA NIC.
This patch only increases the max allowable gso/gro pkt size for MANA
devices and does not change the defaults.
Following are the perf benefits by increasing the pkt aggregate size from
legacy gso_max_size value(64K) to newer one(up-to 511K)

for i in {1..10}; do netperf -t TCP_RR  -H 10.0.0.5 -p50000 -- -r80000,80000
-O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done

min	p90	p99	Throughput		gso_max_size
93	171	194	6594.25
97	154	180	7183.74
95	165	189	6927.86
96	165	188	6976.04
93	154	185	7338.05			64K
93	168	189	6938.03
94	169	189	6784.93
92	166	189	7117.56
94	179	191	6678.44
95	157	183	7277.81

min	p90	p99	Throughput
93	134	146	8448.75
95	134	140	8396.54
94	137	148	8204.12
94	137	148	8244.41
94	128	139	8666.52			80K
94	141	153	8116.86
94	138	149	8163.92
92	135	142	8362.72
92	134	142	8497.57
93	136	148	8393.23

Tested on azure env with Accelerated Networking enabled and disabled.

Signed-off-by: Shradha Gupta <shradhagupta@linux.microsoft.com>
---
 Changes in v2
 * Instead of using 'tcp segment' throughout the patch used used more accurate
   term 'aggregated pkt size'
---
 drivers/net/ethernet/microsoft/mana/mana_en.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index aa1e47233fe5..da630cb37cfb 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -2873,6 +2873,8 @@ static int mana_probe_port(struct mana_context *ac, int port_idx,
 	ndev->dev_port = port_idx;
 	SET_NETDEV_DEV(ndev, gc->dev);
 
+	netif_set_tso_max_size(ndev, GSO_MAX_SIZE);
+
 	netif_carrier_off(ndev);
 
 	netdev_rss_key_fill(apc->hashkey, MANA_HASH_KEY_SIZE);
-- 
2.34.1

[PATCH v2 net-next 2/2] hv_netvsc: Use VF's tso_max_size value when data path is VF

[Shradha Gupta]

On Azure, increasing VF's gso/gro packet size to up-to GSO_MAX_SIZE
is not possible without allowing the same for netvsc NIC
(as the NICs are bonded together). For bonded NICs, the min of the max
aggregated pkt size of the members is propagated in the stack.

Therefore, we use netif_set_tso_max_size() to set max aggregated pkt size
to VF's packet size for netvsc too, when the data path is switched over
to the VF
Tested on azure env with Accelerated Networking enabled and disabled.

Signed-off-by: Shradha Gupta <shradhagupta@linux.microsoft.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
---
 Changes in v2
 * Instead of using 'tcp segment' throughout the patch used used more accurate
   term 'aggregated pkt size'
---
 drivers/net/hyperv/hyperv_net.h   |  2 ++
 drivers/net/hyperv/netvsc_drv.c   | 15 +++++++++++++++
 drivers/net/hyperv/rndis_filter.c | 13 +++++++------
 3 files changed, 24 insertions(+), 6 deletions(-)

diff --git a/drivers/net/hyperv/hyperv_net.h b/drivers/net/hyperv/hyperv_net.h
index e690b95b1bbb..def41067ea3f 100644
--- a/drivers/net/hyperv/hyperv_net.h
+++ b/drivers/net/hyperv/hyperv_net.h
@@ -1166,6 +1166,8 @@ struct netvsc_device {
 	u32 max_chn;
 	u32 num_chn;
 
+	u32 netvsc_gso_max_size;
+
 	atomic_t open_chn;
 	struct work_struct subchan_work;
 	wait_queue_head_t subchan_open;
diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
index d6c4abfc3a28..cbb517aa59cf 100644
--- a/drivers/net/hyperv/netvsc_drv.c
+++ b/drivers/net/hyperv/netvsc_drv.c
@@ -2461,6 +2461,21 @@ static int netvsc_vf_changed(struct net_device *vf_netdev, unsigned long event)
 	} else {
 		netdev_info(ndev, "Data path switched %s VF: %s\n",
 			    vf_is_up ? "to" : "from", vf_netdev->name);
+
+		/* In Azure, when accelerated networking in enabled, other NICs
+		 * like MANA, MLX, are configured as a bonded nic with
+		 * Netvsc(failover) NIC. For bonded NICs, the min of the max
+		 * pkt aggregate size of the members is propagated in the stack.
+		 * In order to allow these NICs (MANA/MLX) to use up to
+		 * GSO_MAX_SIZE gso packet size, we need to allow Netvsc NIC to
+		 * also support this in the guest.
+		 * This value is only increased for netvsc NIC when datapath is
+		 * switched over to the VF
+		 */
+		if (vf_is_up)
+			netif_set_tso_max_size(ndev, vf_netdev->tso_max_size);
+		else
+			netif_set_tso_max_size(ndev, netvsc_dev->netvsc_gso_max_size);
 	}
 
 	return NOTIFY_OK;
diff --git a/drivers/net/hyperv/rndis_filter.c b/drivers/net/hyperv/rndis_filter.c
index c0ceeef4fcd8..82747dfacd70 100644
--- a/drivers/net/hyperv/rndis_filter.c
+++ b/drivers/net/hyperv/rndis_filter.c
@@ -1356,9 +1356,10 @@ static int rndis_netdev_set_hwcaps(struct rndis_device *rndis_device,
 	struct net_device_context *net_device_ctx = netdev_priv(net);
 	struct ndis_offload hwcaps;
 	struct ndis_offload_params offloads;
-	unsigned int gso_max_size = GSO_LEGACY_MAX_SIZE;
 	int ret;
 
+	nvdev->netvsc_gso_max_size = GSO_LEGACY_MAX_SIZE;
+
 	/* Find HW offload capabilities */
 	ret = rndis_query_hwcaps(rndis_device, nvdev, &hwcaps);
 	if (ret != 0)
@@ -1390,8 +1391,8 @@ static int rndis_netdev_set_hwcaps(struct rndis_device *rndis_device,
 			offloads.lso_v2_ipv4 = NDIS_OFFLOAD_PARAMETERS_LSOV2_ENABLED;
 			net->hw_features |= NETIF_F_TSO;
 
-			if (hwcaps.lsov2.ip4_maxsz < gso_max_size)
-				gso_max_size = hwcaps.lsov2.ip4_maxsz;
+			if (hwcaps.lsov2.ip4_maxsz < nvdev->netvsc_gso_max_size)
+				nvdev->netvsc_gso_max_size = hwcaps.lsov2.ip4_maxsz;
 		}
 
 		if (hwcaps.csum.ip4_txcsum & NDIS_TXCSUM_CAP_UDP4) {
@@ -1411,8 +1412,8 @@ static int rndis_netdev_set_hwcaps(struct rndis_device *rndis_device,
 			offloads.lso_v2_ipv6 = NDIS_OFFLOAD_PARAMETERS_LSOV2_ENABLED;
 			net->hw_features |= NETIF_F_TSO6;
 
-			if (hwcaps.lsov2.ip6_maxsz < gso_max_size)
-				gso_max_size = hwcaps.lsov2.ip6_maxsz;
+			if (hwcaps.lsov2.ip6_maxsz < nvdev->netvsc_gso_max_size)
+				nvdev->netvsc_gso_max_size = hwcaps.lsov2.ip6_maxsz;
 		}
 
 		if (hwcaps.csum.ip6_txcsum & NDIS_TXCSUM_CAP_UDP6) {
@@ -1438,7 +1439,7 @@ static int rndis_netdev_set_hwcaps(struct rndis_device *rndis_device,
 	 */
 	net->features &= ~NETVSC_SUPPORTED_HW_FEATURES | net->hw_features;
 
-	netif_set_tso_max_size(net, gso_max_size);
+	netif_set_tso_max_size(net, nvdev->netvsc_gso_max_size);
 
 	ret = rndis_filter_set_offload_params(net, nvdev, &offloads);
 
-- 
2.34.1

RE: [PATCH v2 net-next 1/2] net: mana: Allow tso_max_size to go up-to GSO_MAX_SIZE

[Haiyang Zhang]

> -----Original Message-----
> From: Shradha Gupta <shradhagupta@linux.microsoft.com>
> Sent: Sunday, February 9, 2025 11:40 PM
> To: linux-hyperv@vger.kernel.org; netdev@vger.kernel.org; linux-
> kernel@vger.kernel.org
> Cc: Shradha Gupta <shradhagupta@linux.microsoft.com>; KY Srinivasan
> <kys@microsoft.com>; Haiyang Zhang <haiyangz@microsoft.com>; Wei Liu
> <wei.liu@kernel.org>; Dexuan Cui <decui@microsoft.com>; Andrew Lunn
> <andrew+netdev@lunn.ch>; David S. Miller <davem@davemloft.net>; Eric
> Dumazet <edumazet@google.com>; Jakub Kicinski <kuba@kernel.org>; Paolo
> Abeni <pabeni@redhat.com>; Long Li <longli@microsoft.com>; Konstantin
> Taranov <kotaranov@microsoft.com>; Souradeep Chakrabarti
> <schakrabarti@linux.microsoft.com>; Erick Archer
> <erick.archer@outlook.com>; Shradha Gupta <shradhagupta@microsoft.com>
> Subject: [PATCH v2 net-next 1/2] net: mana: Allow tso_max_size to go up-
> to GSO_MAX_SIZE
> 
> Allow the max aggregated pkt size to go up-to GSO_MAX_SIZE for MANA NIC.
> This patch only increases the max allowable gso/gro pkt size for MANA
> devices and does not change the defaults.
> Following are the perf benefits by increasing the pkt aggregate size from
> legacy gso_max_size value(64K) to newer one(up-to 511K)
> 
> for i in {1..10}; do netperf -t TCP_RR  -H 10.0.0.5 -p50000 -- -
> r80000,80000
> -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done
> 
> min	p90	p99	Throughput		gso_max_size
> 93	171	194	6594.25
> 97	154	180	7183.74
> 95	165	189	6927.86
> 96	165	188	6976.04
> 93	154	185	7338.05			64K
> 93	168	189	6938.03
> 94	169	189	6784.93
> 92	166	189	7117.56
> 94	179	191	6678.44
> 95	157	183	7277.81
> 
> min	p90	p99	Throughput
> 93	134	146	8448.75
> 95	134	140	8396.54
> 94	137	148	8204.12
> 94	137	148	8244.41
> 94	128	139	8666.52			80K
> 94	141	153	8116.86
> 94	138	149	8163.92
> 92	135	142	8362.72
> 92	134	142	8497.57
> 93	136	148	8393.23
> 
> Tested on azure env with Accelerated Networking enabled and disabled.
> 
> Signed-off-by: Shradha Gupta <shradhagupta@linux.microsoft.com>

Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>

Re: [PATCH v2 net-next 1/2] net: mana: Allow tso_max_size to go up-to GSO_MAX_SIZE

[Eric Dumazet]

On Mon, Feb 10, 2025 at 5:40 AM Shradha Gupta
<shradhagupta@linux.microsoft.com> wrote:
>
> Allow the max aggregated pkt size to go up-to GSO_MAX_SIZE for MANA NIC.
> This patch only increases the max allowable gso/gro pkt size for MANA
> devices and does not change the defaults.
> Following are the perf benefits by increasing the pkt aggregate size from
> legacy gso_max_size value(64K) to newer one(up-to 511K)
>
> for i in {1..10}; do netperf -t TCP_RR  -H 10.0.0.5 -p50000 -- -r80000,80000
> -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done

Was this tested with IPv6 ?

Re: [PATCH v2 net-next 1/2] net: mana: Allow tso_max_size to go up-to GSO_MAX_SIZE

[Shradha Gupta]

On Mon, Feb 10, 2025 at 04:55:54PM +0100, Eric Dumazet wrote:
> On Mon, Feb 10, 2025 at 5:40???AM Shradha Gupta
> <shradhagupta@linux.microsoft.com> wrote:
> >
> > Allow the max aggregated pkt size to go up-to GSO_MAX_SIZE for MANA NIC.
> > This patch only increases the max allowable gso/gro pkt size for MANA
> > devices and does not change the defaults.
> > Following are the perf benefits by increasing the pkt aggregate size from
> > legacy gso_max_size value(64K) to newer one(up-to 511K)
> >
> > for i in {1..10}; do netperf -t TCP_RR  -H 10.0.0.5 -p50000 -- -r80000,80000
> > -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done
> 
> Was this tested with IPv6 ?

Hi Eric,
yes, sanity and functional testing where performed(manually) and passed on azure
VMs with IPv6.

Re: [PATCH v2 net-next 1/2] net: mana: Allow tso_max_size to go up-to GSO_MAX_SIZE

[Shradha Gupta]

On Mon, Feb 10, 2025 at 09:57:53AM -0800, Shradha Gupta wrote:
> On Mon, Feb 10, 2025 at 04:55:54PM +0100, Eric Dumazet wrote:
> > On Mon, Feb 10, 2025 at 5:40???AM Shradha Gupta
> > <shradhagupta@linux.microsoft.com> wrote:
> > >
> > > Allow the max aggregated pkt size to go up-to GSO_MAX_SIZE for MANA NIC.
> > > This patch only increases the max allowable gso/gro pkt size for MANA
> > > devices and does not change the defaults.
> > > Following are the perf benefits by increasing the pkt aggregate size from
> > > legacy gso_max_size value(64K) to newer one(up-to 511K)
> > >
> > > for i in {1..10}; do netperf -t TCP_RR  -H 10.0.0.5 -p50000 -- -r80000,80000
> > > -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done
> > 
> > Was this tested with IPv6 ?
> 
> Hi Eric,
> yes, sanity and functional testing where performed(manually) and passed on azure
> VMs with IPv6.
Forgot to mention that the tests were performed on both IPv4 and IPv6
and these numbers are from IPv4 testing

regards,
Shradha.

Re: [PATCH v2 net-next 1/2] net: mana: Allow tso_max_size to go up-to GSO_MAX_SIZE

[Eric Dumazet]

On Mon, Feb 10, 2025 at 6:59 PM Shradha Gupta
<shradhagupta@linux.microsoft.com> wrote:
>
> On Mon, Feb 10, 2025 at 09:57:53AM -0800, Shradha Gupta wrote:
> > On Mon, Feb 10, 2025 at 04:55:54PM +0100, Eric Dumazet wrote:
> > > On Mon, Feb 10, 2025 at 5:40???AM Shradha Gupta
> > > <shradhagupta@linux.microsoft.com> wrote:
> > > >
> > > > Allow the max aggregated pkt size to go up-to GSO_MAX_SIZE for MANA NIC.
> > > > This patch only increases the max allowable gso/gro pkt size for MANA
> > > > devices and does not change the defaults.
> > > > Following are the perf benefits by increasing the pkt aggregate size from
> > > > legacy gso_max_size value(64K) to newer one(up-to 511K)
> > > >
> > > > for i in {1..10}; do netperf -t TCP_RR  -H 10.0.0.5 -p50000 -- -r80000,80000
> > > > -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done
> > >
> > > Was this tested with IPv6 ?
> >
> > Hi Eric,
> > yes, sanity and functional testing where performed(manually) and passed on azure
> > VMs with IPv6.
> Forgot to mention that the tests were performed on both IPv4 and IPv6
> and these numbers are from IPv4 testing

Where is the IPv6 jumbo header removed ?

Re: [PATCH v2 net-next 1/2] net: mana: Allow tso_max_size to go up-to GSO_MAX_SIZE

[Shradha Gupta]

On Mon, Feb 10, 2025 at 07:02:04PM +0100, Eric Dumazet wrote:
> On Mon, Feb 10, 2025 at 6:59???PM Shradha Gupta
> <shradhagupta@linux.microsoft.com> wrote:
> >
> > On Mon, Feb 10, 2025 at 09:57:53AM -0800, Shradha Gupta wrote:
> > > On Mon, Feb 10, 2025 at 04:55:54PM +0100, Eric Dumazet wrote:
> > > > On Mon, Feb 10, 2025 at 5:40???AM Shradha Gupta
> > > > <shradhagupta@linux.microsoft.com> wrote:
> > > > >
> > > > > Allow the max aggregated pkt size to go up-to GSO_MAX_SIZE for MANA NIC.
> > > > > This patch only increases the max allowable gso/gro pkt size for MANA
> > > > > devices and does not change the defaults.
> > > > > Following are the perf benefits by increasing the pkt aggregate size from
> > > > > legacy gso_max_size value(64K) to newer one(up-to 511K)
> > > > >
> > > > > for i in {1..10}; do netperf -t TCP_RR  -H 10.0.0.5 -p50000 -- -r80000,80000
> > > > > -O MIN_LATENCY,P90_LATENCY,P99_LATENCY,THROUGHPUT|tail -1; done
> > > >
> > > > Was this tested with IPv6 ?
> > >
> > > Hi Eric,
> > > yes, sanity and functional testing where performed(manually) and passed on azure
> > > VMs with IPv6.
> > Forgot to mention that the tests were performed on both IPv4 and IPv6
> > and these numbers are from IPv4 testing
> 
> Where is the IPv6 jumbo header removed ?
I think this is missed in this patchset. In our IPv6 tests, patch sanity was
performed without changing the default values.
I will add this support, thorughly test IPv6 and send out another
version with complete IPv6 support and numbers.

Thanks for the pointers Eric.

Regards,
Shradha Gupta.