From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mta1.migadu.com (out-159.mta1.migadu.com [95.215.58.159]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D73BF52D2CB for ; Tue, 13 Oct 2026 18:02:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.159 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791914580; cv=none; b=NX+ijfSsl2gho1exAgBaw9VUaU62BAoIQGcBLbyHVt3purGTqboydRg61ppiRFLqJ7GkLIFzkhr4Uo3y2yhkG2OmRzxwkbbYeUXwXPjHH+0IW1ES+dGbjjkr1KcITHrGPt/b/30MgTB5hI8a5Nex649CCLXt3yvrYQ1/uYswT8M= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1791914580; c=relaxed/simple; bh=X9UuJE4zt+PGi6cCdtN3z6IZ43msDY7aWNMD3kqrheo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nKGvJlmQJ0mHW2ZJO6uHfhy9TapA4wNr9HhAmuPYywPbaL5D3VmAwVf1uF8tRHguSRZB99+iEQmvJb1Bt6zmvqgWYNHqvMcQbTpYGBugp297jpYTyuLRUOKq0L3Q2G6Bg6VG7Fj+PEeW0ccBZOL/2tsL/nfynT21vdecic5JmY8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=KbkhwFXC; arc=none smtp.client-ip=95.215.58.159 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="KbkhwFXC" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=X9UuJE4zt+PGi6cCdtN3z6IZ43msDY7aWNMD3kqrheo=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1791914572; v=1; x=1792519372; b=KbkhwFXCbraG/lF0ulu0iqKaNtsLpV8aIu5a7cmtFXhUXT9B7G74StFTnDL4tc2FmBiBB2tI d4vzrZWF5PgIXk3MgzUzHxmRYG38e6CKDN6wMRjX1VZheeVuSg8xndTN4TeVI34aWuo0sjbfjUb eODdYYTRt2ftS4fldCS0leLw= X-Envelope-To: linux-kernel@vger.kernel.org Received: by mta12.migadu.com with ESMTPS id 719607f313bf44dc; Tue, 13 Oct 2026 18:02:52 +0000 X-Mizu-Trace-ID: 719607f313bf44dc X-Migadu-Flow: FLOW_OUT From: wen.yang@linux.dev To: Sebastian Andrzej Siewior , Clark Williams , Steven Rostedt Cc: Eric Dumazet , Jesper Dangaard Brouer , Jakub Kicinski , Paolo Abeni , linux-rt-devel@list.kernel.org, linux-kernel@vger.kernel.org, Wen Yang Subject: [PATCH 13/15] net: Use backlog-NAPI to clean up the defer_list. Date: Wed, 14 Oct 2026 02:02:29 +0800 Message-Id: <2f607e54ce19721f169a8c2fc4bd433aeab2c7a7.1791912669.git.wen.yang@linux.dev> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: Sebastian Andrzej Siewior commit 80d2eefcb4c84aa9018b2a997ab3a4c567bc821a upstream. The defer_list is a per-CPU list which is used to free skbs outside of the socket lock and on the CPU on which they have been allocated. The list is processed during NAPI callbacks so ideally the list is cleaned up. Should the amount of skbs on the list exceed a certain water mark then the softirq is triggered remotely on the target CPU by invoking a remote function call. The raise of the softirqs via a remote function call leads to waking the ksoftirqd on PREEMPT_RT which is undesired. The backlog-NAPI threads already provide the infrastructure which can be utilized to perform the cleanup of the defer_list. The NAPI state is updated with the input_pkt_queue.lock acquired. It order not to break the state, it is needed to also wake the backlog-NAPI thread with the lock held. This requires to acquire the use the lock in rps_lock_irq*() if the backlog-NAPI threads are used even with RPS disabled. Move the logic of remotely starting softirqs to clean up the defer_list into kick_defer_list_purge(). Make sure a lock is held in rps_lock_irq*() if backlog-NAPI threads are used. Schedule backlog-NAPI for defer_list cleanup if backlog-NAPI is available. Acked-by: Jakub Kicinski Signed-off-by: Sebastian Andrzej Siewior Signed-off-by: Paolo Abeni Signed-off-by: Wen Yang --- include/linux/netdevice.h | 1 + net/core/dev.c | 25 +++++++++++++++++++++---- net/core/skbuff.c | 4 ++-- 3 files changed, 24 insertions(+), 6 deletions(-) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index b399773d123b..d7361bc7635e 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -3245,6 +3245,7 @@ static inline void dev_xmit_recursion_dec(void) __this_cpu_dec(softnet_data.xmit.recursion); } +void kick_defer_list_purge(struct softnet_data *sd, unsigned int cpu); void __netif_schedule(struct Qdisc *q); void netif_schedule_queue(struct netdev_queue *txq); diff --git a/net/core/dev.c b/net/core/dev.c index 0b2cf2f0f5a4..251985e85954 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -220,7 +220,7 @@ static inline struct hlist_head *dev_index_hash(struct net *net, int ifindex) static inline void rps_lock_irqsave(struct softnet_data *sd, unsigned long *flags) { - if (IS_ENABLED(CONFIG_RPS)) + if (IS_ENABLED(CONFIG_RPS) || use_backlog_threads()) spin_lock_irqsave(&sd->input_pkt_queue.lock, *flags); else if (!IS_ENABLED(CONFIG_PREEMPT_RT)) local_irq_save(*flags); @@ -228,7 +228,7 @@ static inline void rps_lock_irqsave(struct softnet_data *sd, static inline void rps_lock_irq_disable(struct softnet_data *sd) { - if (IS_ENABLED(CONFIG_RPS)) + if (IS_ENABLED(CONFIG_RPS) || use_backlog_threads()) spin_lock_irq(&sd->input_pkt_queue.lock); else if (!IS_ENABLED(CONFIG_PREEMPT_RT)) local_irq_disable(); @@ -237,7 +237,7 @@ static inline void rps_lock_irq_disable(struct softnet_data *sd) static inline void rps_unlock_irq_restore(struct softnet_data *sd, unsigned long *flags) { - if (IS_ENABLED(CONFIG_RPS)) + if (IS_ENABLED(CONFIG_RPS) || use_backlog_threads()) spin_unlock_irqrestore(&sd->input_pkt_queue.lock, *flags); else if (!IS_ENABLED(CONFIG_PREEMPT_RT)) local_irq_restore(*flags); @@ -245,7 +245,7 @@ static inline void rps_unlock_irq_restore(struct softnet_data *sd, static inline void rps_unlock_irq_enable(struct softnet_data *sd) { - if (IS_ENABLED(CONFIG_RPS)) + if (IS_ENABLED(CONFIG_RPS) || use_backlog_threads()) spin_unlock_irq(&sd->input_pkt_queue.lock); else if (!IS_ENABLED(CONFIG_PREEMPT_RT)) local_irq_enable(); @@ -4749,6 +4749,23 @@ static void napi_schedule_rps(struct softnet_data *sd) __napi_schedule_irqoff(&mysd->backlog); } +void kick_defer_list_purge(struct softnet_data *sd, unsigned int cpu) +{ + unsigned long flags; + + if (use_backlog_threads()) { + rps_lock_irqsave(sd, &flags); + + if (!__test_and_set_bit(NAPI_STATE_SCHED, &sd->backlog.state)) + __napi_schedule_irqoff(&sd->backlog); + + rps_unlock_irq_restore(sd, &flags); + + } else if (!cmpxchg(&sd->defer_ipi_scheduled, 0, 1)) { + smp_call_function_single_async(cpu, &sd->defer_csd); + } +} + #ifdef CONFIG_NET_FLOW_LIMIT int netdev_flow_limit_table_len __read_mostly = (1 << 12); #endif diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 6111c1bba704..717f23563ea9 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -6748,6 +6748,6 @@ nodefer: __kfree_skb(skb); /* Make sure to trigger NET_RX_SOFTIRQ on the remote CPU * if we are unlucky enough (this seems very unlikely). */ - if (unlikely(kick) && !cmpxchg(&sd->defer_ipi_scheduled, 0, 1)) - smp_call_function_single_async(cpu, &sd->defer_csd); + if (unlikely(kick)) + kick_defer_list_purge(sd, cpu); } -- 2.25.1