From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CF4D383C86 for ; Tue, 18 Aug 2026 09:42:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.42 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787046135; cv=none; b=lS3gjeAsLiQBjLjNRF4AWb+PvLvdxf6/MZYRC3humtaQDsBnItwtlOa98cEEKQJwK5bcwfubTqvRNS0z22ZYx0SLHM+/m4YJ7aSJ8+gVSAQiH8u2rFRhAIBdJ2Gsu5YCIPrGW18o00UXkfpbRD3tltvTObbUqZ/0jZsMpSMn9ZY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787046135; c=relaxed/simple; bh=kkmFd5j2SrN9y5eoZVrCZOE5zgAjz5TProZMHrKJEaM=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=hEjE4PFZxIYN5Z9p5Kg0pzJCT8PB9/7wyJ9E1Tg224MnEnHeZaKR5cyjgFVm6XLg+J1HVlBmkqI97hssfHPLHJZFlRJpZuqfccq1iMRQTaNisaNOcx8lKSmuBRlILJXt5/LpAQMUFy6Y90ie2nY3cITK9jGOVrORM9LyhJWApx0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=blackwall.org; spf=none smtp.mailfrom=blackwall.org; dkim=pass (2048-bit key) header.d=blackwall.org header.i=@blackwall.org header.b=oOMmTKyF; arc=none smtp.client-ip=209.85.128.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=blackwall.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=blackwall.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=blackwall.org header.i=@blackwall.org header.b="oOMmTKyF" Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-4998e0916faso24675815e9.2 for ; Tue, 18 Aug 2026 02:42:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=blackwall.org; s=google; t=1787046132; x=1787650932; darn=vger.kernel.org; h=content-transfer-encoding:content-type:in-reply-to:from:references :cc:to:content-language:subject:user-agent:mime-version:date :message-id:from:to:cc:subject:date:message-id:reply-to:content-type; bh=A26iULHivlajh+3MA1uO0Cksntrf1DSGsgfSv8mZ+Uo=; b=oOMmTKyFW6Mq2yaH6dRb8Bn4Hb6TA0fmqq/YLEhwPyXYDWMLmUHvVxwRQfEhl6YxWV eI0rf2GPgRQIS0Tb12HHRUY43XPrJa51V530ZRtiOxxPoAid5y0AY8BmaOE7DZccl2Au NXafgr6CLeL2zNE7PUcF3GTQRVpPStMDTPBZLBLZp9kno/o132qxNj/eq8k6fFP770Lo mRWJpTOX2q8jLEnVvdWQrM45J15eN6gApgwM8cfKX0faqNs2x1KVD6CdQEAniwjturZR e5yxhYHYSsxRiOe30PDjAae7hV3/9AA2In74h96h0Bp808wDcDBnxF5IehbwNIvBNSGQ cLzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1787046132; x=1787650932; h=content-transfer-encoding:content-type:in-reply-to:from:references :cc:to:content-language:subject:user-agent:mime-version:date :message-id:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=A26iULHivlajh+3MA1uO0Cksntrf1DSGsgfSv8mZ+Uo=; b=JWVEUA2Vm5PRccA2yRZhmWBnhIlz48w4BTE1CagLSZJN/dmwjr5C78UldotOWmxqo9 0AVrRHIG19E/eP2+iWCqWA1+pTvitaGjbjWJDoVJH4V65NMH8sKzoZp37/lkPe9zFj3U rjqHhiMTcxKmB3K2iJkJ8xzy4qWnUsGxQJboyZKNHEWOXbnt1Etqqv7cqqEYMB5xGE5z oBsKHdB3D/TPW6gZ0bn967jpR4ErgktOaAbgyGpXMKPuC5gXoxg/3RRfCy3Xh9rzeTK6 wegZBDtEnFQcZ/jNxikPMR2+0Zw3JK91al3L3S9N7QGYe34x7KK9BOcLm8xKXjB0/ra+ xiEA== X-Forwarded-Encrypted: i=1; AHgh+Rrpqm8aynD8Dr96IBMmg1mW+guc3oSUfhoJeyOcbzjHqWa88BeMBthMpE6ygJCyiCTykB6s4BfhA63eAtQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yz83bCTftwDkcV1jq/BUOvbBG8S1G1zzKDvaxZWG23Si0/FywoD gtBWlOAZuspbCLRJp5VvZqX/spQn/qQ++tt9d1cE0lRtJr7PHYyweF0+YOSvP3DVTHg= X-Gm-Gg: AR+sD11SSRs7fltOGH+kc5qk4dnyTT9Jrnlal+S+CZezAvoTKqKidrd+CdxLbabrVrP ch7u66z+aisdS+O+xwDiY1Y5OdNtYEOqAcT/ecWPpAl4RMKaRa16oz1a6AeJcOnqwwal9/EKUq/ cT0RcZzdnE4N0nIf0viaope+HIBqJ0/HmDhbNyIOvm7j0kePrlqNMUG0CyjotTa6b0SdoIeHcV+ iGPx0aRTDMXkZ/BZ5CUZxAqtlnzW1yNqWf1vtA0gGkopgIfZiYaEXmFp2Er1q5dc2YtbMl5qSfP Dy2J2uxyl1kpNarRk06mLWwdHJ7n98n7CNrsB+Lhg7SWtnyDrNj4C6pn0A5JNeASQpJLxQUZLer vip5hz0mMQ6UIyh975u7sVJRiUrbHOy1x1l6p0uxwQHpgwUT/AgFWoH0jkiPV4lTL8rD0qDuxqE vgde7wCxR2zU2zCuv9G81BK2dgmHBSK3DkuQeRGKiLNaQWA63Q1htTSpVdKLHnx1vaqoxToWeCQ G4DjQzso6Qrajnamu0= X-Received: by 2002:a05:600c:3485:b0:499:726a:a017 with SMTP id 5b1f17b1804b1-49987938e19mr523413195e9.1.1787046132061; Tue, 18 Aug 2026 02:42:12 -0700 (PDT) Received: from [192.168.0.161] (78-154-15-182.ip.btc-net.bg. [78.154.15.182]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4999d10b7f6sm114382215e9.12.2026.08.18.02.42.11 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 18 Aug 2026 02:42:11 -0700 (PDT) Message-ID: Date: Tue, 18 Aug 2026 12:42:10 +0300 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH net v3 1/2] bonding: convert unbalanced_load to per-cpu state Content-Language: en-US, bg To: Hangbin Liu , Jay Vosburgh , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Simon Horman Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Hangbin Liu References: <20260818-bond_overflow-v3-0-e05d4dbc2fd8@kylinos.cn> <20260818-bond_overflow-v3-1-e05d4dbc2fd8@kylinos.cn> From: Nikolay Aleksandrov In-Reply-To: <20260818-bond_overflow-v3-1-e05d4dbc2fd8@kylinos.cn> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 18/08/2026 11:47, Hangbin Liu wrote: > From: Hangbin Liu > > A later patch widens the bonding TLB tx counters from u32 to u64. The > unbalanced_load counter sits in the transmit hot path, and cross-CPU > synchronization of a u64 would introduce measurable overhead. Convert > unbalanced_load to a per-cpu counter first so that the subsequent > widening only touches per-cpu data local to each CPU. > > Introduce struct unbalanced_load_stats to hold the per-cpu counter, > and move the aggregation into a helper, reset_unbalanced_load(), which > sums and clears all per-cpu instances. > > Signed-off-by: Hangbin Liu > --- > drivers/net/bonding/bond_alb.c | 25 ++++++++++++++++++------- > drivers/net/bonding/bond_main.c | 9 +++++++++ > include/net/bond_alb.h | 6 +++++- > 3 files changed, 32 insertions(+), 8 deletions(-) > > diff --git a/drivers/net/bonding/bond_alb.c b/drivers/net/bonding/bond_alb.c > index 839f7482dc18..d54d834cf72b 100644 > --- a/drivers/net/bonding/bond_alb.c > +++ b/drivers/net/bonding/bond_alb.c > @@ -1345,7 +1345,7 @@ static netdev_tx_t bond_do_alb_xmit(struct sk_buff *skb, struct bonding *bond, > /* unbalanced or unassigned, send through primary */ > tx_slave = rcu_dereference(bond->curr_active_slave); > if (bond->params.tlb_dynamic_lb) > - bond_info->unbalanced_load += skb->len; > + this_cpu_add(bond_info->unbalanced_load->tx_bytes, skb->len); > } > > if (tx_slave && bond_slave_can_tx(tx_slave)) { > @@ -1529,6 +1529,21 @@ netdev_tx_t bond_alb_xmit(struct sk_buff *skb, struct net_device *bond_dev) > return bond_do_alb_xmit(skb, bond, tx_slave); > } > > +static u32 reset_unbalanced_load(struct alb_bond_info *bond_info) > +{ > + struct unbalanced_load_stats *p; > + u32 total_bytes = 0; > + int i; > + > + for_each_possible_cpu(i) { > + p = per_cpu_ptr(bond_info->unbalanced_load, i); > + total_bytes += READ_ONCE(p->tx_bytes); > + WRITE_ONCE(p->tx_bytes, 0); > + } > + > + return total_bytes / BOND_TLB_REBALANCE_INTERVAL; > +} > + > void bond_alb_monitor(struct work_struct *work) > { > struct bonding *bond = container_of(work, struct bonding, > @@ -1570,12 +1585,8 @@ void bond_alb_monitor(struct work_struct *work) > if (atomic_read(&bond_info->tx_rebalance_counter) >= BOND_TLB_REBALANCE_TICKS) { > bond_for_each_slave_rcu(bond, slave, iter) { > tlb_clear_slave(bond, slave, 1); > - if (slave == rcu_access_pointer(bond->curr_active_slave)) { > - SLAVE_TLB_INFO(slave).load = > - bond_info->unbalanced_load / > - BOND_TLB_REBALANCE_INTERVAL; > - bond_info->unbalanced_load = 0; > - } > + if (slave == rcu_access_pointer(bond->curr_active_slave)) > + SLAVE_TLB_INFO(slave).load = reset_unbalanced_load(bond_info); > } > atomic_set(&bond_info->tx_rebalance_counter, 0); > } > diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c > index 522eab060f9e..9fb44e0031c8 100644 > --- a/drivers/net/bonding/bond_main.c > +++ b/drivers/net/bonding/bond_main.c > @@ -5995,6 +5995,7 @@ static void bond_destructor(struct net_device *bond_dev) > destroy_workqueue(bond->wq); > > free_percpu(bond->rr_tx_counter); > + free_percpu(bond->alb_info.unbalanced_load); > } > > void bond_setup(struct net_device *bond_dev) > @@ -6494,6 +6495,10 @@ static int bond_init(struct net_device *bond_dev) > if (!bond->wq) > return -ENOMEM; > > + bond->alb_info.unbalanced_load = alloc_percpu(struct unbalanced_load_stats); > + if (!bond->alb_info.unbalanced_load) > + goto wq_out; > + The bond doesn't generally depend on this struct and shouldn't fail to init if it fails to allocate and the user hasn't chosen alb mode. Also here is too early to tell, the netlink mode hasn't been applied yet. Maybe the allocation could be moved to bond_alb_initialize and the freeing to bond_alb_deinitialize? > bond->notifier_ctx = false; > > spin_lock_init(&bond->stats_lock); > @@ -6511,6 +6516,10 @@ static int bond_init(struct net_device *bond_dev) > eth_hw_addr_random(bond_dev); > > return 0; > + > +wq_out: > + destroy_workqueue(bond->wq); > + return -ENOMEM; > } > > unsigned int bond_get_num_tx_queues(void) > diff --git a/include/net/bond_alb.h b/include/net/bond_alb.h > index e5945427f38d..3fabf4714dec 100644 > --- a/include/net/bond_alb.h > +++ b/include/net/bond_alb.h > @@ -123,9 +123,13 @@ struct tlb_slave_info { > */ > }; > > +struct unbalanced_load_stats { > + u32 tx_bytes; > +}; > + > struct alb_bond_info { > struct tlb_client_info *tx_hashtbl; /* Dynamically allocated */ > - u32 unbalanced_load; > + struct unbalanced_load_stats __percpu *unbalanced_load; > atomic_t tx_rebalance_counter; > int lp_counter; > /* -------- rlb parameters -------- */ >