From: John Garry <john.garry@huawei.com>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Question on threaded handlers for managed interrupts
Date: Thu, 22 Apr 2021 17:10:49 +0100 [thread overview]
Message-ID: <b8c4be8c-1d67-c16c-570e-d3c883c77ea2@huawei.com> (raw)
Hi Thomas,
I am finding that I can pretty easily trigger a system hang for certain
scenarios with my storage controller.
So I'm getting something like this when running moderately heavy data
throughput:
Starting 6 processes
[70.656622] sched: RT throttling activatedB/s][r=356k,w=0 IOPS][eta
01h:14m:43s]
[ 207.632161] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:ta
01h:12m:26s]
[ 207.638261] rcu: 0-...!: (1 GPs behind)
idle=312/1/0x4000000000000000 softirq=508/512 fqs=0
[ 207.646777] rcu: 1-...!: (1 GPs behind) idle=694/0/0x0
It ends pretty badly - see [0].
The multi-queue storage controller (see [1] for memory refresh, but note
that I can also trigger on PCI device host controller as well) is using
managed interrupts and threaded handlers. Since the threaded handler
uses SCHED_FIFO, aren't we always vulnerable to this situation with the
managed interrupt and threaded handler combo? Would the advice be to
just use irq polling here?
I unsuccessfully tried to trigger the same on NVMe PCI - however I have
only 1x card, so hardly overloading the system.
Thanks,
John
[0]
https://lore.kernel.org/rcu/412926e8-d3e1-3071-8cb9-098a7f49b64c@huawei.com/T/#mbd60463c543e04f87090d89301e1a5f10de958dd
[1]
https://lore.kernel.org/linux-scsi/1606905417-183214-1-git-send-email-john.garry@huawei.com/#t
next reply other threads:[~2021-04-22 16:13 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-22 16:10 John Garry [this message]
2021-04-23 10:50 ` Thomas Gleixner
2021-04-23 12:02 ` John Garry
2021-04-23 13:01 ` Thomas Gleixner
2021-04-23 15:00 ` John Garry
2021-05-21 12:46 ` John Garry
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b8c4be8c-1d67-c16c-570e-d3c883c77ea2@huawei.com \
--to=john.garry@huawei.com \
--cc=linux-kernel@vger.kernel.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®