From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932996AbcI3L4A (ORCPT ); Fri, 30 Sep 2016 07:56:00 -0400 Received: from terminus.zytor.com ([198.137.202.10]:44562 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932577AbcI3Lzv (ORCPT ); Fri, 30 Sep 2016 07:55:51 -0400 Date: Fri, 30 Sep 2016 04:55:01 -0700 From: tip-bot for Eric Dumazet Message-ID: Cc: tglx@linutronix.de, linux-kernel@vger.kernel.org, hannes@stressinduktion.org, edumazet@google.com, hpa@zytor.com, riel@redhat.com, jbrouer@redhat.com, hannes@redhat.com, peterz@infradead.org, davem@davemloft.net, corbet@lwn.net, pabeni@redhat.com, mingo@kernel.org, torvalds@linux-foundation.org Reply-To: edumazet@google.com, tglx@linutronix.de, hannes@stressinduktion.org, linux-kernel@vger.kernel.org, davem@davemloft.net, pabeni@redhat.com, corbet@lwn.net, peterz@infradead.org, torvalds@linux-foundation.org, mingo@kernel.org, hpa@zytor.com, hannes@redhat.com, riel@redhat.com, jbrouer@redhat.com In-Reply-To: <1472665349.14381.356.camel@edumazet-glaptop3.roam.corp.google.com> References: <1472665349.14381.356.camel@edumazet-glaptop3.roam.corp.google.com> To: linux-tip-commits@vger.kernel.org Subject: [tip:irq/core] softirq: Let ksoftirqd do its job Git-Commit-ID: 4cd13c21b207e80ddb1144c576500098f2d5f882 X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Commit-ID: 4cd13c21b207e80ddb1144c576500098f2d5f882 Gitweb: http://git.kernel.org/tip/4cd13c21b207e80ddb1144c576500098f2d5f882 Author: Eric Dumazet AuthorDate: Wed, 31 Aug 2016 10:42:29 -0700 Committer: Ingo Molnar CommitDate: Fri, 30 Sep 2016 10:43:36 +0200 softirq: Let ksoftirqd do its job A while back, Paolo and Hannes sent an RFC patch adding threaded-able napi poll loop support : (https://patchwork.ozlabs.org/patch/620657/) The problem seems to be that softirqs are very aggressive and are often handled by the current process, even if we are under stress and that ksoftirqd was scheduled, so that innocent threads would have more chance to make progress. This patch makes sure that if ksoftirq is running, we let it perform the softirq work. Jonathan Corbet summarized the issue in https://lwn.net/Articles/687617/ Tested: - NIC receiving traffic handled by CPU 0 - UDP receiver running on CPU 0, using a single UDP socket. - Incoming flood of UDP packets targeting the UDP socket. Before the patch, the UDP receiver could almost never get CPU cycles and could only receive ~2,000 packets per second. After the patch, CPU cycles are split 50/50 between user application and ksoftirqd/0, and we can effectively read ~900,000 packets per second, a huge improvement in DOS situation. (Note that more packets are now dropped by the NIC itself, since the BH handlers get less CPU cycles to drain RX ring buffer) Since the load runs in well identified threads context, an admin can more easily tune process scheduling parameters if needed. Reported-by: Paolo Abeni Reported-by: Hannes Frederic Sowa Signed-off-by: Eric Dumazet Signed-off-by: Peter Zijlstra (Intel) Cc: David Miller Cc: Hannes Frederic Sowa Cc: Jesper Dangaard Brouer Cc: Jonathan Corbet Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Rik van Riel Cc: Thomas Gleixner Link: http://lkml.kernel.org/r/1472665349.14381.356.camel@edumazet-glaptop3.roam.corp.google.com Signed-off-by: Ingo Molnar --- kernel/softirq.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/kernel/softirq.c b/kernel/softirq.c index 17caf4b..8ed90e3 100644 --- a/kernel/softirq.c +++ b/kernel/softirq.c @@ -78,6 +78,17 @@ static void wakeup_softirqd(void) } /* + * If ksoftirqd is scheduled, we do not want to process pending softirqs + * right now. Let ksoftirqd handle this at its own rate, to get fairness. + */ +static bool ksoftirqd_running(void) +{ + struct task_struct *tsk = __this_cpu_read(ksoftirqd); + + return tsk && (tsk->state == TASK_RUNNING); +} + +/* * preempt_count and SOFTIRQ_OFFSET usage: * - preempt_count is changed by SOFTIRQ_OFFSET on entering or leaving * softirq processing. @@ -313,7 +324,7 @@ asmlinkage __visible void do_softirq(void) pending = local_softirq_pending(); - if (pending) + if (pending && !ksoftirqd_running()) do_softirq_own_stack(); local_irq_restore(flags); @@ -340,6 +351,9 @@ void irq_enter(void) static inline void invoke_softirq(void) { + if (ksoftirqd_running()) + return; + if (!force_irqthreads) { #ifdef CONFIG_HAVE_IRQ_EXIT_ON_IRQ_STACK /*