* Linux guest domain with two vnets bound to the same vswitch experiences hung in bootup (sun_netraT5220)
@ 2009-09-23 4:37 hyl
2009-09-23 7:40 ` hyl
0 siblings, 1 reply; 7+ messages in thread
From: hyl @ 2009-09-23 4:37 UTC (permalink / raw)
To: sparclinux; +Cc: linux-kernel, David Miller
console output:
e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
e1000e: Copyright (c) 1999-2008 Intel Corporation.
e1000e: Copyright (c) 1999-2008 Intel Corporation.
sunvnet.c:v1.0 (June 25, 2007)
sunvnet.c:v1.0 (June 25, 2007)
eth0: Sun LDOM vnet 00:14:4f:f9:68:e0:t
e0
eth0: PORT ( remote-mac 00:14:4f:fa:03:81 switch-port )
eth0: PORT ( remote-mac 00:14:4f:fa:03:81 switch-port )
eth0: PORT ( remote-mac 00:14:4f:f9:84:06 )
eth0: PORT ( remote-mac 00:14:4f:f9:84:06 )
eth0: PORT ( remote-mac 00:14:4f:f9:9a:40 )
eth0: PORT ( remote-mac 00:14:4f:f9:9a:40 )
eth0: PORT ( remote-mac 00:14:4f:f8:c5:7b )
eth0: PORT ( remote-mac 00:14:4f:f8:c5:7b )
eth1: Sun LDOM vnet 00:14:4f:f8:c5:7b:t
7b
eth1: PORT ( remote-mac 00:14:4f:fa:03:81 switch-port )
eth1: PORT ( remote-mac 00:14:4f:fa:03:81 switch-port )
eth1: PORT ( remote-mac 00:14:4f:f9:84:06 )
eth1: PORT ( remote-mac 00:14:4f:f9:84:06 )
eth1: PORT ( remote-mac 00:14:4f:f9:9a:40 )
eth1: PORT ( remote-mac 00:14:4f:f9:9a:40 )
eth1: PORT ( remote-mac 00:14:4f:f9:68:e0 )
eth1: PORT ( remote-mac 00:14:4f:f9:68:e0 )
two vnets bound to the same vswitch, then hang,
ldm add-vnet vnet0 primary-vsw1 dom1
ldm add-vnet vnet1 primary-vsw1 ldom1
If 'eth1' and 'eth2' are bound to different vswitches, everything is fine.
Yongli He
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Linux guest domain with two vnets bound to the same vswitch experiences hung in bootup (sun_netraT5220)
2009-09-23 4:37 Linux guest domain with two vnets bound to the same vswitch experiences hung in bootup (sun_netraT5220) hyl
@ 2009-09-23 7:40 ` hyl
2009-09-24 2:03 ` hyl
0 siblings, 1 reply; 7+ messages in thread
From: hyl @ 2009-09-23 7:40 UTC (permalink / raw)
To: sparclinux; +Cc: linux-kernel, David Miller
attach the debug vio console output, F.Y.I:
v1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
v1000e: Copyright (c) 1999-2008 Intel Corporation.
vunvnet.c:v1.0 (June 25, 2007)
v0
oth0: PORT ( remote-mac 00:14:4f:fa:03:81 switch-port )
vio: ID[0] START HANDSHAKE
vio: ID[0] SEND VERSION INFO maj[1] min[0] devclass[1]
vio: ID[0] TAG [01:01:0001:0d305de8]
vio: ID[0] GOT VERSION INFO maj[1] min[0] devclass[2]
vio: ID[0] SEND VERSION ACK maj[1] min[0]
vio: ID[0] TAG [01:02:0001:0d305de8]
vio: ID[0] GOT VERSION ACK maj[1] min[0] devclass[1]
vio: ID[0] SEND NET ATTR xmode[0x3] atype[0x1] addr[144ff968e0]
ackfreq[0] mtu[111
vio: ID[0] TAG [01:01:0002:0d305de8]
sio: ID[0] GOT NET ATTR INFO xmode[0x3] atype[0x1]
addr[144ffa03810000] ackfreq[3] mtu[1514]
] mtu[1514]
io: ID[0] SEND NET ATTR ACK
eio: ID[0] SEND DRING_REG INFO ndesc[512] dsz[48] opt[0x1] ncookies[1]
eio: ID[0] DRING COOKIE(0) [0000000000400000:0000000000006000]
sio: ID[0] TAG [01:02:0002:0d305de8]
eio: ID[0] GOT NET ATTR ACK
eio: ID[0] TAG [01:01:0003:0d305de8]
vio: ID[0] GOT DRING_REG INFO ident[0] ndesc[512] dsz[48] opt[0x1] ncookies[1]
vio: ID[0] DRING COOKIE(0) [0000000002c06000:0000000000006000]
vio: ID[0] SEND DRING_REG ACK ident[1]
vio: ID[0] TAG [01:02:0003:0d305de8]
vio: ID[0] GOT DRING_REG ACK ident[3] ndesc[512] dsz[48] opt[0x1] ncookies[1]
vio: ID[0] SEND RDX INFO
vio: ID[0] TAG [01:01:0005:0d305de8]
vio: ID[0] GOT RDX INFO
5io: ID[0] SEND RDX ACK
vio: ID[0] TAG [01:02:0005:0d305de8]
vio: ID[0] GOT RDX ACK
3th0: PORT ( remote-mac 00:14:4f:f9:84:06 )
vth0: PORT ( remote-mac 00:14:4f:f9:9a:40 )
vio: ID[2] START HANDSHAKE
vio: ID[2] TAG [01:01:0001:41459a97]
vio: ID[2] GOT VERSION INFO maj[1] min[0] devclass[1]
vio: ID[2] SEND VERSION ACK maj[1] min[0]
vth0: PORT ( remote-mac 00:14:4f:f8:c5:7b )
vb
vth1: PORT ( remote-mac 00:14:4f:fa:03:81 switch-port )
vio: ID[3] START HANDSHAKE
vio: ID[3] SEND VERSION INFO maj[1] min[0] devclass[1]
vio: ID[3] TAG [01:01:0001:0d305de8]
vio: ID[3] GOT VERSION INFO maj[1] min[0] devclass[2]
vio: ID[3] SEND VERSION ACK maj[1] min[0]
vio: ID[3] TAG [01:02:0001:0d305de8]
vio: ID[3] GOT VERSION ACK maj[1] min[0] devclass[1]
eio: ID[3] SEND NET ATTR xmode[0x3] atype[0x1] addr[144ff8c57b]
ackfreq[0] mtu[111
eio: ID[3] TAG [01:01:0002:0d305de8]
vio: ID[3] GOT NET ATTR INFO xmode[0x3] atype[0x1]
addr[144ffa03810000] ackfreq[3] mtu[1514]
v] mtu[1514]
vio: ID[3] SEND NET ATTR ACK
vio: ID[3] SEND DRING_REG INFO ndesc[512] dsz[48] opt[0x1] ncookies[1]
vio: ID[3] DRING COOKIE(0) [0000000000400000:0000000000006000]
eio: ID[3] TAG [01:02:0002:0d305de8]
eio: ID[3] GOT NET ATTR ACK
eio: ID[3] TAG [01:01:0003:0d305de8]
vio: ID[3] GOT DRING_REG INFO ident[0] ndesc[512] dsz[48] opt[0x1] ncookies[1]
vio: ID[3] DRING COOKIE(0) [0000000000c0c000:0000000000006000]
vio: ID[3] SEND DRING_REG ACK ident[1]
vio: ID[3] TAG [01:02:0003:0d305de8]
vio: ID[3] GOT DRING_REG ACK ident[3] ndesc[512] dsz[48] opt[0x1] ncookies[1]
vio: ID[3] SEND RDX INFO
vio: ID[3] TAG [01:01:0005:0d305de8]
vio: ID[3] GOT RDX INFO
vio: ID[3] TAG [01:02:0005:0d305de8]
vio: ID[3] GOT RDX ACK
3th1: PORT ( remote-mac 00:14:4f:f9:84:06 )
vth1: PORT ( remote-mac 00:14:4f:f9:9a:40 )
vio: ID[5] START HANDSHAKE
vio: ID[5] SEND VERSION INFO maj[1] min[0] devclass[1]
vio: ID[5] TAG [01:01:0001:4747de06]
vio: ID[5] GOT VERSION INFO maj[1] min[0] devclass[1]
vio: ID[5] SEND VERSION ACK maj[1] min[0]
vth1: PORT ( remote-mac 00:14:4f:f9:68:e0 )
vio: ID[7] SEND VERSION INFO maj[1] min[0] devclass[1]
vio: ID[3] TAG [02:01:0042:0d305de8]
vio: ID[3] vnet_rx stype_env[0042] seq[0000000000000001]
rcv_nxt[0000000000000001]
v]
vio: ID[3] vnet_walk_rx start[00000000] end[000001ff]
vio: ID[3] vio_walk_rx_one desc[01:00:00000000:00000001:80e800:800]
vio: ID[0] TAG [02:01:0042:0d305de8]
eio: ID[0] vnet_rx stype_env[0042] seq[0000000000000001]
rcv_nxt[0000000000000001]
e]
vio: ID[0] vnet_walk_rx start[00000000] end[000001ff]
vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:2808800:800]
vio: ID[0] TAG [02:01:0042:0d305de8]
vio: ID[0] vnet_rx stype_env[0042] seq[0000000000000002]
rcv_nxt[0000000000000002]
e]
vio: ID[0] vnet_walk_rx start[00000001] end[00000000]
vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:280b000:800]
vio: ID[3] TAG [02:01:0042:0d305de8]
1io: ID[3] vnet_rx stype_env[0042] seq[0000000000000002]
rcv_nxt[0000000000000002]
v]
vio: ID[3] vnet_walk_rx start[00000001] end[00000000]
vio: ID[3] vio_walk_rx_one desc[01:00:00000000:00000001:811000:800]
vio: ID[0] TAG [02:01:0042:0d305de8]
1io: ID[0] vnet_rx stype_env[0042] seq[0000000000000003]
rcv_nxt[0000000000000003]
v]
vio: ID[0] vnet_walk_rx start[00000002] end[00000001]
vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:280d800:800]
vio: ID[3] TAG [02:01:0042:0d305de8]
2io: ID[3] vnet_rx stype_env[0042] seq[0000000000000003]
rcv_nxt[0000000000000003]
v]
vio: ID[3] vnet_walk_rx start[00000002] end[00000001]
vio: ID[3] vio_walk_rx_one desc[01:00:00000000:00000001:813800:800]
vio: ID[0] TAG [02:01:0042:0d305de8]
2io: ID[0] vnet_rx stype_env[0042] seq[0000000000000004]
rcv_nxt[0000000000000004]
v]
vio: ID[0] vnet_walk_rx start[00000003] end[00000002]
vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:280e000:800]
vio: ID[3] TAG [02:01:0042:0d305de8]
3io: ID[3] vnet_rx stype_env[0042] seq[0000000000000004]
rcv_nxt[0000000000000004]
v]
vio: ID[3] vnet_walk_rx start[00000003] end[00000002]
vio: ID[3] vio_walk_rx_one desc[01:00:00000000:00000001:814000:800]
vio: ID[0] TAG [02:01:0042:0d305de8]
3io: ID[0] vnet_rx stype_env[0042] seq[0000000000000005]
rcv_nxt[0000000000000005]
v]
vio: ID[0] vnet_walk_rx start[00000004] end[00000003]
vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:2810800:800]
vio: ID[3] TAG [02:01:0042:0d305de8]
4io: ID[3] vnet_rx stype_env[0042] seq[0000000000000005]
rcv_nxt[0000000000000005]
v]
vio: ID[3] vnet_walk_rx start[00000004] end[00000003]
vio: ID[3] vio_walk_rx_one desc[01:00:00000000:00000001:816800:800]
vio: ID[0] TAG [02:01:0042:0d305de8]
4io: ID[0] vnet_rx stype_env[0042] seq[0000000000000006]
rcv_nxt[0000000000000006]
v]
vio: ID[0] vnet_walk_rx start[00000005] end[00000004]
vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:2813000:800]
vio: ID[3] TAG [02:01:0042:0d305de8]
5io: ID[3] vnet_rx stype_env[0042] seq[0000000000000006]
rcv_nxt[0000000000000006]
v]
vio: ID[3] vnet_walk_rx start[00000005] end[00000004]
vio: ID[3] vio_walk_rx_one desc[01:00:00000000:00000001:819000:800]
vio: ID[0] TAG [02:01:0042:0d305de8]
5io: ID[0] vnet_rx stype_env[0042] seq[0000000000000007]
rcv_nxt[0000000000000007]
v]
vio: ID[0] vnet_walk_rx start[00000006] end[00000005]
vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:2815800:800]
vio: ID[3] TAG [02:01:0042:0d305de8]
6io: ID[3] vnet_rx stype_env[0042] seq[0000000000000007]
rcv_nxt[0000000000000007]
v]
vio: ID[3] vnet_walk_rx start[00000006] end[00000005]
vio: ID[3] vio_walk_rx_one desc[01:00:00000000:00000001:81b800:800]
vio: ID[0] TAG [02:01:0042:0d305de8]
6io: ID[0] vnet_rx stype_env[0042] seq[0000000000000008]
rcv_nxt[0000000000000008]
v]
vio: ID[0] vnet_walk_rx start[00000007] end[00000006]
vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:2816000:800]
vio: ID[0] TAG [02:01:0042:0d305de8]
7io: ID[0] vnet_rx stype_env[0042] seq[0000000000000009]
rcv_nxt[0000000000000009]
v]
vio: ID[0] vnet_walk_rx start[00000008] end[00000007]
vio: ID[0] vnet_walk_rx start[00000008] end[00000007]
vio: ID[0] vio_walk_rx_one desc[02:00:0000003c:00000001:2816000:800]
vio: ID[0] vio_walk_rx_one desc[02:00:0000003c:00000001:2816000:800]
vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:2818800:800]
vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:2818800:800]
vio: ID[0] TAG [02:01:0042:0d305de8]
vio: ID[0] TAG [02:01:0042:0d305de8]
vio: ID[0] vnet_rx stype_env[0042] seq[000000000000000a]
rcv_nxt[000000000000000a]
vio: ID[0] vnet_rx stype_env[0042] seq[000000000000000a]
rcv_nxt[00000000000000000
a]
vio: ID[0] vnet_walk_rx start[00000009] end[00000008]
vio: ID[0] vnet_walk_rx start[00000009] end[00000008]
vio: ID[0] vio_walk_rx_one desc[02:00:0000003c:00000001:2818800:800]
vio: ID[0] vio_walk_rx_one desc[02:00:0000003c:00000001:2818800:800]
vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:281b000:800]
vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:281b000:800]
vio: ID[0] TAG [02:01:0042:0d305de8]
vio: ID[0] TAG [02:01:0042:0d305de8]
vio: ID[0] vnet_rx stype_env[0042] seq[000000000000000b]
rcv_nxt[000000000000000b]
vio: ID[0] vnet_rx stype_env[0042] seq[000000000000000b]
rcv_nxt[00000000000000000
b]
vio: ID[0] vnet_walk_rx start[0000000a] end[00000009]
vio: ID[0] vnet_walk_rx start[0000000a] end[00000009]
vio: ID[0] vio_walk_rx_one desc[02:00:0000003c:00000001:281b000:800]
vio: ID[0] vio_walk_rx_one desc[02:00:0000003c:00000001:281b000:800]
vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:281d800:800]
vio: ID[0] vio_walk_rx_one desc[01:00:00000000:00000001:281d800:800]
2009/9/23 hyl <heyongli@gmail.com>:
> console output:
>
> e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
> e1000e: Intel(R) PRO/1000 Network Driver - 0.3.3.3-k6
> e1000e: Copyright (c) 1999-2008 Intel Corporation.
> e1000e: Copyright (c) 1999-2008 Intel Corporation.
> sunvnet.c:v1.0 (June 25, 2007)
> sunvnet.c:v1.0 (June 25, 2007)
> eth0: Sun LDOM vnet 00:14:4f:f9:68:e0:t
> e0
> eth0: PORT ( remote-mac 00:14:4f:fa:03:81 switch-port )
> eth0: PORT ( remote-mac 00:14:4f:fa:03:81 switch-port )
> eth0: PORT ( remote-mac 00:14:4f:f9:84:06 )
> eth0: PORT ( remote-mac 00:14:4f:f9:84:06 )
> eth0: PORT ( remote-mac 00:14:4f:f9:9a:40 )
> eth0: PORT ( remote-mac 00:14:4f:f9:9a:40 )
> eth0: PORT ( remote-mac 00:14:4f:f8:c5:7b )
> eth0: PORT ( remote-mac 00:14:4f:f8:c5:7b )
> eth1: Sun LDOM vnet 00:14:4f:f8:c5:7b:t
> 7b
> eth1: PORT ( remote-mac 00:14:4f:fa:03:81 switch-port )
> eth1: PORT ( remote-mac 00:14:4f:fa:03:81 switch-port )
> eth1: PORT ( remote-mac 00:14:4f:f9:84:06 )
> eth1: PORT ( remote-mac 00:14:4f:f9:84:06 )
> eth1: PORT ( remote-mac 00:14:4f:f9:9a:40 )
> eth1: PORT ( remote-mac 00:14:4f:f9:9a:40 )
> eth1: PORT ( remote-mac 00:14:4f:f9:68:e0 )
> eth1: PORT ( remote-mac 00:14:4f:f9:68:e0 )
>
> two vnets bound to the same vswitch, then hang,
> ldm add-vnet vnet0 primary-vsw1 dom1
> ldm add-vnet vnet1 primary-vsw1 ldom1
>
> If 'eth1' and 'eth2' are bound to different vswitches, everything is fine.
>
>
> Yongli He
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Linux guest domain with two vnets bound to the same vswitch experiences hung in bootup (sun_netraT5220)
2009-09-23 7:40 ` hyl
@ 2009-09-24 2:03 ` hyl
2009-10-09 22:08 ` David Miller
0 siblings, 1 reply; 7+ messages in thread
From: hyl @ 2009-09-24 2:03 UTC (permalink / raw)
To: sparclinux; +Cc: linux-kernel, David Miller
>From 57b549e5e4dfc8b34d66bbad7f4297a1bc81a43d Mon Sep 17 00:00:00 2001
From: Yongli He <heyongli@gmail.com>
Date: Thu, 24 Sep 2009 10:00:04 +0800
Subject: [PATCH] Sun ldom vnet driver dead lock
if 2 vnet attach to same vswitch, ldom will report
2 same irq then lead to dead lock on the lp->lock
or the vio->lock
static irqreturn_t ldc_rx(int irq, void *dev_id){
...
out:
spin_unlock_irqrestore(&lp->lock, flags);
<< here run with out any lock
send_events(lp, event_mask);
>> vnet evetn process will dead lock on
>> lp->lock or the vio->lock
Signed-off-by: Yongli He <heyongli@gmail.com>
---
arch/sparc/include/asm/ldc.h | 1 +
arch/sparc64/kernel/ldc.c | 4 ++++
drivers/net/sunvnet.c | 2 ++
3 files changed, 7 insertions(+), 0 deletions(-)
diff --git a/arch/sparc/include/asm/ldc.h b/arch/sparc/include/asm/ldc.h
index bdb524a..dd2b30f 100644
--- a/arch/sparc/include/asm/ldc.h
+++ b/arch/sparc/include/asm/ldc.h
@@ -20,6 +20,7 @@ extern void ldom_power_off(void);
*/
struct ldc_channel_config {
void (*event)(void *arg, int event);
+ spinlock_t *serial_lock;
u32 mtu;
unsigned int rx_irq;
diff --git a/arch/sparc64/kernel/ldc.c b/arch/sparc64/kernel/ldc.c
index a6b75cd..72b9501 100644
--- a/arch/sparc64/kernel/ldc.c
+++ b/arch/sparc64/kernel/ldc.c
@@ -892,9 +892,13 @@ handshake_complete:
}
out:
+ if(lp->cfg.serial_lock)
+ spin_lock_irqsave(lp->cfg.serial_lock, flags);
spin_unlock_irqrestore(&lp->lock, flags);
send_events(lp, event_mask);
+ if(lp->cfg.serial_lock)
+ spin_unlock_irqrestore(lp->cfg.serial_lock, flags);
return IRQ_HANDLED;
}
diff --git a/drivers/net/sunvnet.c b/drivers/net/sunvnet.c
index 6415ce1..e58f9da 100644
--- a/drivers/net/sunvnet.c
+++ b/drivers/net/sunvnet.c
@@ -1117,11 +1117,13 @@ static struct vnet * __devinit
vnet_find_parent(struct mdesc_handle *hp,
return vnet_find_or_create(local_mac);
}
+static spinlock_t event_lock=__SPIN_LOCK_UNLOCKED("vnet_event");
static struct ldc_channel_config vnet_ldc_cfg = {
.event = vnet_event,
.mtu = 64,
.mode = LDC_MODE_UNRELIABLE,
+ .serial_lock = &event_lock,
};
static struct vio_driver_ops vnet_vio_ops = {
--
1.5.5.1.dirty
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Linux guest domain with two vnets bound to the same vswitch experiences hung in bootup (sun_netraT5220)
2009-09-24 2:03 ` hyl
@ 2009-10-09 22:08 ` David Miller
2009-10-09 23:21 ` David Miller
0 siblings, 1 reply; 7+ messages in thread
From: David Miller @ 2009-10-09 22:08 UTC (permalink / raw)
To: heyongli; +Cc: sparclinux, linux-kernel
From: hyl <heyongli@gmail.com>
Date: Thu, 24 Sep 2009 10:03:25 +0800
> Subject: [PATCH] Sun ldom vnet driver dead lock
>
> if 2 vnet attach to same vswitch, ldom will report
> 2 same irq then lead to dead lock on the lp->lock
> or the vio->lock
>
> static irqreturn_t ldc_rx(int irq, void *dev_id){
> ...
> out:
> spin_unlock_irqrestore(&lp->lock, flags);
> << here run with out any lock
> send_events(lp, event_mask);
> >> vnet evetn process will dead lock on
> >> lp->lock or the vio->lock
>
> Signed-off-by: Yongli He <heyongli@gmail.com>
Thank you for this bug report and patch, I am looking at
it now.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Linux guest domain with two vnets bound to the same vswitch experiences hung in bootup (sun_netraT5220)
2009-10-09 22:08 ` David Miller
@ 2009-10-09 23:21 ` David Miller
2009-10-12 8:23 ` hyl
0 siblings, 1 reply; 7+ messages in thread
From: David Miller @ 2009-10-09 23:21 UTC (permalink / raw)
To: heyongli; +Cc: sparclinux, linux-kernel
From: David Miller <davem@davemloft.net>
Date: Fri, 09 Oct 2009 15:08:29 -0700 (PDT)
> Thank you for this bug report and patch, I am looking at
> it now.
I'm trying to figure out how the deadlock can even occur,
and I've failed so far, please help me :-)
See, we always take the VIO and LDC locks in the same order
(VIO then LDC) and always with interrupts disabled, so it is
not possible to deadlock.
The only way we could deadlock is if:
1) There is some path that takes the LDC lock before the VIO one.
2) There is some path that takes either lock with interrupts
enabled.
And I cannot find any such case.
It might help if you run your test case with lockdep enabled. It will
find such deadlocks and report them precisely to the kernel logs.
Thank you!
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Linux guest domain with two vnets bound to the same vswitch experiences hung in bootup (sun_netraT5220)
2009-10-09 23:21 ` David Miller
@ 2009-10-12 8:23 ` hyl
2009-10-12 10:26 ` David Miller
0 siblings, 1 reply; 7+ messages in thread
From: hyl @ 2009-10-12 8:23 UTC (permalink / raw)
To: David Miller; +Cc: sparclinux, linux-kernel
2009/10/10 David Miller <davem@davemloft.net>:
> From: David Miller <davem@davemloft.net>
> Date: Fri, 09 Oct 2009 15:08:29 -0700 (PDT)
>
>> Thank you for this bug report and patch, I am looking at
>> it now.
>
> I'm trying to figure out how the deadlock can even occur,
> and I've failed so far, please help me :-)
>
> See, we always take the VIO and LDC locks in the same order
> (VIO then LDC) and always with interrupts disabled, so it is
> not possible to deadlock.
>
> The only way we could deadlock is if:
>
> 1) There is some path that takes the LDC lock before the VIO one.
>
> 2) There is some path that takes either lock with interrupts
> enabled.
>
> And I cannot find any such case.
David
Thank you, i try to figure out the path which lead to system hang. i got the
output log :(the same output run18350079 times and going forever ... )
ctl: 17, data:0 err:0, abr:0 runcc:18350079 CPUId:0, event: 0x00000004
qall Trace:
s[00000000004acb30] _handle_IRQ_event+0x50/0x120
C[00000000004acc70] handle_IRQ_event+0x70/0x120
[00000000004af14c] handle_fasteoi_irq+0xcc/0x180
[000000000042ee54] handler_irq+0x134/0x160
[00000000004208b4] tl0_irq5+0x14/0x20
[00000000004acbac] _handle_IRQ_event+0xcc/0x120
[00000000004acc70] handle_IRQ_event+0x70/0x120
[00000000004af14c] handle_fasteoi_irq+0xcc/0x180
[000000000042ee54] handler_irq+0x134/0x160
[00000000004208b4] tl0_irq5+0x14/0x20
[00000000004acbac] _handle_IRQ_event+0xcc/0x120
[00000000004acc70] handle_IRQ_event+0x70/0x120
[00000000004af14c] handle_fasteoi_irq+0xcc/0x180
[000000000042ee54] handler_irq+0x134/0x160
[00000000004208b4] tl0_irq5+0x14/0x20
[00000000007e7ffc] _spin_unlock_irqrestore+0x3c/0x60
>runcc:18350079 CPUId:0, event: 0x00000004
the runcc is the count of times ldx_rx been run. dump code:
static irqreturn_t ldc_rx(int irq, void *dev_id)
{
....
atomic64_inc(&runcc);
....
printk(KERN_INFO"runcc:%lld CPUId:%d, event: 0x%08x\n",
atomic_read(&runcc), smp_processor_id(),event_mask);
dump_stack();
}
look the console output, system seems hang on a live lock:
tl0_irq5 triggered just after the irq been re-enable in the handler
of irq5: the ldc_rx.
i have no idea about the t10_irq5, just guess that: the special
configuration lead to t10_irq5 been triggered continuously, and
the trigger condition can not been cleared.
Pauli He
>
> It might help if you run your test case with lockdep enabled. It will
> find such deadlocks and report them precisely to the kernel logs.
>
> Thank you!
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Linux guest domain with two vnets bound to the same vswitch experiences hung in bootup (sun_netraT5220)
2009-10-12 8:23 ` hyl
@ 2009-10-12 10:26 ` David Miller
0 siblings, 0 replies; 7+ messages in thread
From: David Miller @ 2009-10-12 10:26 UTC (permalink / raw)
To: heyongli; +Cc: sparclinux, linux-kernel
From: hyl <heyongli@gmail.com>
Date: Mon, 12 Oct 2009 16:23:47 +0800
> look the console output, system seems hang on a live lock:
> tl0_irq5 triggered just after the irq been re-enable in the handler
> of irq5: the ldc_rx.
>
> i have no idea about the t10_irq5, just guess that: the special
> configuration lead to t10_irq5 been triggered continuously, and
> the trigger condition can not been cleared.
Yes, it seems that stack is exhausted. Ho hum...
I guess the following patch works too? Please make sure you've
removed your changes when testing this.
Thanks.
diff --git a/arch/sparc/kernel/ldc.c b/arch/sparc/kernel/ldc.c
index adf5f27..cb3c72c 100644
--- a/arch/sparc/kernel/ldc.c
+++ b/arch/sparc/kernel/ldc.c
@@ -1242,13 +1242,13 @@ int ldc_bind(struct ldc_channel *lp, const char *name)
snprintf(lp->tx_irq_name, LDC_IRQ_NAME_MAX, "%s TX", name);
err = request_irq(lp->cfg.rx_irq, ldc_rx,
- IRQF_SAMPLE_RANDOM | IRQF_SHARED,
+ IRQF_SAMPLE_RANDOM | IRQF_DISABLED | IRQF_SHARED,
lp->rx_irq_name, lp);
if (err)
return err;
err = request_irq(lp->cfg.tx_irq, ldc_tx,
- IRQF_SAMPLE_RANDOM | IRQF_SHARED,
+ IRQF_SAMPLE_RANDOM | IRQF_DISABLED | IRQF_SHARED,
lp->tx_irq_name, lp);
if (err) {
free_irq(lp->cfg.rx_irq, lp);
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2009-10-12 10:27 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-09-23 4:37 Linux guest domain with two vnets bound to the same vswitch experiences hung in bootup (sun_netraT5220) hyl
2009-09-23 7:40 ` hyl
2009-09-24 2:03 ` hyl
2009-10-09 22:08 ` David Miller
2009-10-09 23:21 ` David Miller
2009-10-12 8:23 ` hyl
2009-10-12 10:26 ` David Miller
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®