From: ebiederm@xmission.com (Eric W. Biederman)
To: Xianpeng Zhao <xpzhao@aerohive.com>
Cc: "linux-kernel\@vger.kernel.org" <linux-kernel@vger.kernel.org>,
<netdev@vger.kernel.org>
Subject: Re: [linux-kernel] dead loop for rtnl_trylock
Date: Mon, 22 Feb 2016 03:39:53 -0600 [thread overview]
Message-ID: <87y4adnkwm.fsf@x220.int.ebiederm.org> (raw)
In-Reply-To: <CY1PR11MB0491C022AD738CCC818BE74AAAAD0@CY1PR11MB0491.namprd11.prod.outlook.com> (Xianpeng Zhao's message of "Tue, 16 Feb 2016 10:31:53 +0000")
Copied netdev as that is the more appropriate mailling list for
questions like this.
Xianpeng Zhao <xpzhao@aerohive.com> writes:
> Hi Group,
>
> I have find a problem in my system, I found there have a chance that cause the system enter dead loop when try to get the rtnl lock in the sysctl function in net/ipv6/addrconf.c
>
> The situation should like this, there are 2 processes may need get the rtnl lock, we call them process A and process B, A have high priority than B.
> B need get the rtnl lock to do something, when B schedule out without release the lock; At this time, the A start to run "echo 1 > /proc/sys/net/ipv6/conf/<ifname>/disable_ipv6", the echo process will run to this code:
>
> if (!rtnl_trylock())
>
> return restart_syscall();
>
> Because the rtnl lock was hold by process B, so here the try will be failure, and run the restart_syscall to let the sys_write do again, even try many times, because the B have very lower priority, the lock was hard to be released, so the echo process created by A will enter a loop of restart system call.
>
> In my case it is the wireless_nlevent_process in process kworker taken the rtnl lock, and another higher priority process need use echo to disable IPv6 met this problem.
>
> I am not very sure, but I think it is better to let the process A sleep a while instead of try it again and again without any delay.
>
> Expects, what's your opinions?
That the entire situation is a mess. From what little I have seen it is
a very rare condition. Does this reproduce easily in your environment?
If we are going the delay route we probably want to put the delay in
restart_syscall or in a wrapper around restart_syscall that we use
for the rtnl_trylock failure case.
On first blush I would suggest the logic for sleeping should be:
if (need_reschedule())
schedule();
That will limit the spinning to a single time slice which is definitely
preferrable.
Ugh. But we already cross the kernel/userspace boundary that already
does that.
If you are encountering a deadlock it is very much because you have been
playing very ugly priority games. At which point my sympathies but this
feels like a case of "Docter it hurts when I do this. Then don't do that."
> @@ -5304,8 +5308,10 @@ static int addrconf_disable_ipv6(struct ctl_table *table, int *p, int newf)
>
> struct net *net;
>
> int old;
>
>
>
> - if (!rtnl_trylock())
>
> + if (!rtnl_trylock()){
>
> + schedule_timeout_uninterruptible(HZ/4);
>
> return restart_syscall();
>
> + }
>
>
>
> net = (struct net *)table->extra2;
>
> old = *p;
Eric
prev parent reply other threads:[~2016-02-22 9:50 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-02-16 10:31 Xianpeng Zhao
2016-02-22 9:39 ` Eric W. Biederman [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87y4adnkwm.fsf@x220.int.ebiederm.org \
--to=ebiederm@xmission.com \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=xpzhao@aerohive.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®