mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Arjan van de Ven <arjan@infradead.org>
To: Roland Dreier <rdreier@cisco.com>
Cc: Zach Brown <zach.brown@oracle.com>,
	openib-general@openib.org,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Ingo Molnar <mingo@elte.hu>
Subject: Re: [openib-general] ipoib lockdep warning
Date: Wed, 12 Jul 2006 08:49:38 +0200	[thread overview]
Message-ID: <1152686978.3217.14.camel@laptopd505.fenrus.org> (raw)
In-Reply-To: <ada7j2jzh5o.fsf@cisco.com>

On Tue, 2006-07-11 at 19:33 -0700, Roland Dreier wrote:
> OK, the patch to lib/idr.c below is at least one way to fix this.
> However, with that applied I get the lockdep warning below, which
> seems to be a false positive -- I'm not sure what the right fix is,
> but the blame really seems to fall on udp_ioctl for poking into the
> sk_buff_head lock itself.

this does not have to be a false positive!
It is not legal to take ANY non-hardirq safe lock after having taken a
lock that's used in hardirq context.
(having said that the skb_queue_tail lock needs a  special treatment for
some real false positives; Linus merged that already)

> And here's the warning I get, which appears to be a false positive:
> 
> ======================================================
> [ INFO: hard-safe -> hard-unsafe lock order detected ]
> ------------------------------------------------------
> swapper/0 [HC0[0]:SC1[2]:HE0:SE0] is trying to acquire:
>  (&skb_queue_lock_key){-+..}, at: [<ffffffff8038683e>] skb_queue_tail+0x1d/0x47
> 
> and this task is already holding:
>  (&priv->lock){.+..}, at: [<ffffffff881efd5b>] ipoib_mcast_send+0x29/0x413 [ib_ipoib]
> which would create a new lock dependency:
>  (&priv->lock){.+..} -> (&skb_queue_lock_key){-+..}
> 

Lets call priv->lock lock A, and the skb_queue_lock lock B to keep this
short


CPU 0			CPU 1
soft irq context	user or softirq context

take lock B
(anywhere in net stack)
			take lock A 
			(ipoib_mcast_send)

interrupt happens

take lock A (spins)
in ISR somewhere

			take lock B (spins)
			(in skb_queue_tail() called from
			ipoib_mcast_send)




Now this assumes your queue is shared with the networking stack,
so that the first "take lock B" on CPU 0 can happen without disabling interrupts.
(as several of the SKB functions do that get used by the networking stack, but not
 the simple ones used by drivers for private queues).
If this queue used in ipoib_mcast_send() is a private queue you probably want to 
apply the patch below (already in/on its way to mainline)

Index: linux-2.6.18-rc1/include/linux/skbuff.h
===================================================================
--- linux-2.6.18-rc1.orig/include/linux/skbuff.h
+++ linux-2.6.18-rc1/include/linux/skbuff.h
@@ -606,10 +606,17 @@ static inline __u32 skb_queue_len(const 
 
 extern struct lock_class_key skb_queue_lock_key;
 
+/*
+ * This function creates a split out lock class for each invocation;
+ * this is needed for now since a whole lot of users of the skb-queue
+ * infrastructure in drivers have different locking usage (in hardirq)
+ * than the networking core (in softirq only). In the long run either the
+ * network layer or drivers should need annotation to consolidate the
+ * main types of usage into 3 classes.
+ */
 static inline void skb_queue_head_init(struct sk_buff_head *list)
 {
 	spin_lock_init(&list->lock);
-	lockdep_set_class(&list->lock, &skb_queue_lock_key);
 	list->prev = list->next = (struct sk_buff *)list;
 	list->qlen = 0;
 }



  reply	other threads:[~2006-07-12  6:49 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2006-07-11 20:10 Zach Brown
2006-07-11 21:16 ` Michael S. Tsirkin
2006-07-11 21:40   ` Sean Hefty
2006-07-11 21:50     ` [openib-general] " Zach Brown
2006-07-11 22:54 ` Roland Dreier
2006-07-11 23:27   ` Zach Brown
2006-07-11 23:43     ` Roland Dreier
2006-07-11 23:53       ` Zach Brown
2006-07-12  0:06         ` Roland Dreier
2006-07-12  9:38       ` Ingo Molnar
2006-07-12 11:09         ` Michael S. Tsirkin
2006-07-12 16:31           ` [openib-general] " Sean Hefty
2006-07-12 18:56           ` Roland Dreier
2006-07-13 15:55           ` [PATCH] IB/core: use correct gfp_mask in sa_query Michael S. Tsirkin
2006-07-12 19:06         ` [openib-general] ipoib lockdep warning Roland Dreier
2006-07-12 20:45         ` [PATCH] Convert idr's internal locking to _irqsave variant Roland Dreier
2006-07-12 21:14           ` Ingo Molnar
2006-07-13  1:30           ` Andrew Morton
2006-07-13 15:42             ` Roland Dreier
2006-07-13 20:54               ` Andrew Morton
2006-07-13 21:03                 ` Roland Dreier
2006-07-13 21:05                   ` Arjan van de Ven
2006-07-14  0:18                     ` Roland Dreier
2006-07-14  6:20                       ` Arjan van de Ven
2006-07-13 21:43                   ` Andrew Morton
2006-07-14  1:08                     ` Roland Dreier
2006-07-14  1:18                       ` Andrew Morton
2006-07-14  1:30                         ` Andrew Morton
2006-07-17 15:57                           ` Roland Dreier
2006-07-12  2:33 ` [openib-general] ipoib lockdep warning Roland Dreier
2006-07-12  6:49   ` Arjan van de Ven [this message]
2006-07-12 19:01     ` Roland Dreier

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1152686978.3217.14.camel@laptopd505.fenrus.org \
    --to=arjan@infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@elte.hu \
    --cc=openib-general@openib.org \
    --cc=rdreier@cisco.com \
    --cc=zach.brown@oracle.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®