nfs mount fail

nfs mount fail

[Yinghai Lu]

mpk12-3214-189-144:~ # cat x
mount -t nfs 10.6.75.100:/data/shared/pxeboot /x

mpk12-3214-189-144:~ # sh x

[  278.734149] ------------[ cut here ]------------
[  278.739620] kernel BUG at mm/slub.c:2969!
[  278.743056] invalid opcode: 0000 [#1] SMP
[  278.750365] last sysfs file: /sys/devices/virtual/net/sit0/type
[  278.755898] CPU 0
[  278.758637] Modules linked in:
[  278.770030] Pid: 2594, comm: rpciod/0 Not tainted
2.6.32-rc5-tip-01483-ga166936-dirty #651
[  278.788458] RIP: 0010:[<ffffffff81174437>]  [<ffffffff81174437>]
kfree+0x96/0x14a
[  278.797545] RSP: 0018:ffff88087b9fdc50  EFLAGS: 00010246
[  278.810035] RAX: 0a00000000000000 RBX: 0000000000037803 RCX: ffff880854828018
[  278.815372] RDX: ffffea0000000000 RSI: 0000000000000378 RDI: ffff881800706475
[  278.832711] RBP: ffff88087b9fdc90 R08: ffff88084d37ce80 R09: 00000000e3115a3e
[  278.848764] R10: 00000000ffffe0be R11: 0000000000000000 R12: ffffea009c02da70
[  278.852385] R13: ffff881800706475 R14: ffff88084439d680 R15: 0000000000000001
[  278.872750] FS:  0000000000000000(0000) GS:ffff88003fc00000(0000)
knlGS:0000000000000000
[  278.889379] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
[  278.895226] CR2: 00007fcb2369c000 CR3: 0000000848df0000 CR4: 00000000000026f0
[  278.911157] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  278.916692] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  278.934540] Process rpciod/0 (pid: 2594, threadinfo
ffff88087b9fc000, task ffff88087bdc2370)
[  278.950913] Stack:
[  278.952455]  ffff88087913e934 ffff88087747e048 00000000e3115a3e
0000000000037803
[  278.969335] <0> ffff880854828000 ffff88084d37ce80 ffff88084439d680
0000000000000001
[  278.976115] <0> ffff88087b9fdcc0 ffffffff81dae6d8 ffffffff81dbb5d5
00000000e3115a3e
[  278.995869] Call Trace:
[  278.997442]  [<ffffffff81dae6d8>] xs_set_port+0x73/0xa3
[  279.011219]  [<ffffffff81dbb5d5>] ? rpcb_dec_getport+0x0/0xc2
[  279.016461]  [<ffffffff81dbb4ac>] rpcb_getport_done+0x80/0xdb
[  279.030585]  [<ffffffff81db1cbf>] rpc_exit_task+0x3b/0x7f
[  279.033790]  [<ffffffff81db24a9>] __rpc_execute+0x97/0x251
[  279.051120]  [<ffffffff81db26e2>] rpc_async_schedule+0x28/0x3e
[  279.057205]  [<ffffffff810bbe9c>] worker_thread+0x1fa/0x317
[  279.071181]  [<ffffffff810bbe42>] ? worker_thread+0x1a0/0x317
[  279.075343]  [<ffffffff81db26ba>] ? rpc_async_schedule+0x0/0x3e
[  279.092033]  [<ffffffff810c26c8>] ? autoremove_wake_function+0x0/0x63
[  279.100028]  [<ffffffff810bbca2>] ? worker_thread+0x0/0x317
[  279.112094]  [<ffffffff810c2277>] kthread+0x9d/0xa5
[  279.116720]  [<ffffffff8104205a>] child_rip+0xa/0x20
[  279.133457]  [<ffffffff810419fc>] ? restore_args+0x0/0x30
[  279.137271]  [<ffffffff810c21da>] ? kthread+0x0/0xa5
[  279.149933]  [<ffffffff81042050>] ? child_rip+0x0/0x20
[  279.154086] Code: 00 00 00 00 00 ea ff ff 48 6b c0 68 4c 8d 24 10
66 41 83 3c 24 00 79 05 4d 8b 64 24 10 49 8b 04 24 84 c0 78 17 66 a9
00 c0 75 04 <0f> 0b eb fe 4c 89 e7 e8 87 b8 fc ff e9 80 00 00 00 48 8b
45 08
[  279.198800] RIP  [<ffffffff81174437>] kfree+0x96/0x14a
[  279.204063]  RSP <ffff88087b9fdc50>
[  279.210307] ---[ end trace 52f0b2ff8e4dec91 ]---

Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
mpk12-3214-189-144 kernel: [  278.734149] ------------[ cut here ]------------

Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
mpk12-3214-189-144 kernel: [  278.743056] invalid opcode: 0000 [#1] SMP

Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
mpk12-3214-189-144 kernel: [  278.750365] last sysfs file:
/sys/devices/virtual/net/sit0/type

Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
mpk12-3214-189-144 kernel: [  278.950913] Stack:

Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
mpk12-3214-189-144 kernel: [  278.995869] Call Trace:

Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
mpk12-3214-189-144 kernel: [  279.154086] Code: 00 00 00 00 00 ea ff
ff 48 6b c0 68 4c 8d 24 10 66 41 83 3c 24 00 79 05 4d 8b 64 24 10 49
8b 04 24 84 c0 78 17 66 a9 00 c0 75 04 <0f> 0b eb fe 4c 89 e7 e8 87 b8
fc ff e9 80 00 00 00 48 8b 45 08

mount.nfs: mount system call failed

Re: nfs mount fail

[Yinghai Lu]

On Sat, Oct 17, 2009 at 6:25 PM, Yinghai Lu <yhlu.kernel@gmail.com> wrote:
> mpk12-3214-189-144:~ # cat x
> mount -t nfs 10.6.75.100:/data/shared/pxeboot /x
>
> mpk12-3214-189-144:~ # sh x
>
> [  278.734149] ------------[ cut here ]------------
> [  278.739620] kernel BUG at mm/slub.c:2969!
> [  278.743056] invalid opcode: 0000 [#1] SMP
> [  278.750365] last sysfs file: /sys/devices/virtual/net/sit0/type
> [  278.755898] CPU 0
> [  278.758637] Modules linked in:
> [  278.770030] Pid: 2594, comm: rpciod/0 Not tainted
> 2.6.32-rc5-tip-01483-ga166936-dirty #651
> [  278.788458] RIP: 0010:[<ffffffff81174437>]  [<ffffffff81174437>]
> kfree+0x96/0x14a
> [  278.797545] RSP: 0018:ffff88087b9fdc50  EFLAGS: 00010246
> [  278.810035] RAX: 0a00000000000000 RBX: 0000000000037803 RCX: ffff880854828018
> [  278.815372] RDX: ffffea0000000000 RSI: 0000000000000378 RDI: ffff881800706475
> [  278.832711] RBP: ffff88087b9fdc90 R08: ffff88084d37ce80 R09: 00000000e3115a3e
> [  278.848764] R10: 00000000ffffe0be R11: 0000000000000000 R12: ffffea009c02da70
> [  278.852385] R13: ffff881800706475 R14: ffff88084439d680 R15: 0000000000000001
> [  278.872750] FS:  0000000000000000(0000) GS:ffff88003fc00000(0000)
> knlGS:0000000000000000
> [  278.889379] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
> [  278.895226] CR2: 00007fcb2369c000 CR3: 0000000848df0000 CR4: 00000000000026f0
> [  278.911157] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [  278.916692] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [  278.934540] Process rpciod/0 (pid: 2594, threadinfo
> ffff88087b9fc000, task ffff88087bdc2370)
> [  278.950913] Stack:
> [  278.952455]  ffff88087913e934 ffff88087747e048 00000000e3115a3e
> 0000000000037803
> [  278.969335] <0> ffff880854828000 ffff88084d37ce80 ffff88084439d680
> 0000000000000001
> [  278.976115] <0> ffff88087b9fdcc0 ffffffff81dae6d8 ffffffff81dbb5d5
> 00000000e3115a3e
> [  278.995869] Call Trace:
> [  278.997442]  [<ffffffff81dae6d8>] xs_set_port+0x73/0xa3
> [  279.011219]  [<ffffffff81dbb5d5>] ? rpcb_dec_getport+0x0/0xc2
> [  279.016461]  [<ffffffff81dbb4ac>] rpcb_getport_done+0x80/0xdb
> [  279.030585]  [<ffffffff81db1cbf>] rpc_exit_task+0x3b/0x7f
> [  279.033790]  [<ffffffff81db24a9>] __rpc_execute+0x97/0x251
> [  279.051120]  [<ffffffff81db26e2>] rpc_async_schedule+0x28/0x3e
> [  279.057205]  [<ffffffff810bbe9c>] worker_thread+0x1fa/0x317
> [  279.071181]  [<ffffffff810bbe42>] ? worker_thread+0x1a0/0x317
> [  279.075343]  [<ffffffff81db26ba>] ? rpc_async_schedule+0x0/0x3e
> [  279.092033]  [<ffffffff810c26c8>] ? autoremove_wake_function+0x0/0x63
> [  279.100028]  [<ffffffff810bbca2>] ? worker_thread+0x0/0x317
> [  279.112094]  [<ffffffff810c2277>] kthread+0x9d/0xa5
> [  279.116720]  [<ffffffff8104205a>] child_rip+0xa/0x20
> [  279.133457]  [<ffffffff810419fc>] ? restore_args+0x0/0x30
> [  279.137271]  [<ffffffff810c21da>] ? kthread+0x0/0xa5
> [  279.149933]  [<ffffffff81042050>] ? child_rip+0x0/0x20
> [  279.154086] Code: 00 00 00 00 00 ea ff ff 48 6b c0 68 4c 8d 24 10
> 66 41 83 3c 24 00 79 05 4d 8b 64 24 10 49 8b 04 24 84 c0 78 17 66 a9
> 00 c0 75 04 <0f> 0b eb fe 4c 89 e7 e8 87 b8 fc ff e9 80 00 00 00 48 8b
> 45 08
> [  279.198800] RIP  [<ffffffff81174437>] kfree+0x96/0x14a
> [  279.204063]  RSP <ffff88087b9fdc50>
> [  279.210307] ---[ end trace 52f0b2ff8e4dec91 ]---
>
> Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
> mpk12-3214-189-144 kernel: [  278.734149] ------------[ cut here ]------------
>
> Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
> mpk12-3214-189-144 kernel: [  278.743056] invalid opcode: 0000 [#1] SMP
>
> Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
> mpk12-3214-189-144 kernel: [  278.750365] last sysfs file:
> /sys/devices/virtual/net/sit0/type
>
> Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
> mpk12-3214-189-144 kernel: [  278.950913] Stack:
>
> Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
> mpk12-3214-189-144 kernel: [  278.995869] Call Trace:
>
> Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
> mpk12-3214-189-144 kernel: [  279.154086] Code: 00 00 00 00 00 ea ff
> ff 48 6b c0 68 4c 8d 24 10 66 41 83 3c 24 00 79 05 4d 8b 64 24 10 49
> 8b 04 24 84 c0 78 17 66 a9 00 c0 75 04 <0f> 0b eb fe 4c 89 e7 e8 87 b8
> fc ff e9 80 00 00 00 48 8b 45 08
>
> mount.nfs: mount system call failed
>

Linus's tree is ok...

Linux mpk12-3214-189-241 2.6.32-rc5-00011-g2fdc246-dirty #653 SMP Sat
Oct 17 20:49:44 PDT 2009 x86_64 x86_64 x86_64 GNU/Linux

YH

Re: nfs mount fail

[Pekka Enberg]

Hi,

On Sun, Oct 18, 2009 at 4:25 AM, Yinghai Lu <yhlu.kernel@gmail.com> wrote:
> mpk12-3214-189-144:~ # cat x
> mount -t nfs 10.6.75.100:/data/shared/pxeboot /x
>
> mpk12-3214-189-144:~ # sh x
>
> [  278.734149] ------------[ cut here ]------------
> [  278.739620] kernel BUG at mm/slub.c:2969!

So this means that someone is passing a pointer to kfree() that did
not come from kmalloc(). Which tree are you testing?

> [  278.743056] invalid opcode: 0000 [#1] SMP
> [  278.750365] last sysfs file: /sys/devices/virtual/net/sit0/type
> [  278.755898] CPU 0
> [  278.758637] Modules linked in:
> [  278.770030] Pid: 2594, comm: rpciod/0 Not tainted
> 2.6.32-rc5-tip-01483-ga166936-dirty #651
> [  278.788458] RIP: 0010:[<ffffffff81174437>]  [<ffffffff81174437>]
> kfree+0x96/0x14a
> [  278.797545] RSP: 0018:ffff88087b9fdc50  EFLAGS: 00010246
> [  278.810035] RAX: 0a00000000000000 RBX: 0000000000037803 RCX: ffff880854828018
> [  278.815372] RDX: ffffea0000000000 RSI: 0000000000000378 RDI: ffff881800706475
> [  278.832711] RBP: ffff88087b9fdc90 R08: ffff88084d37ce80 R09: 00000000e3115a3e
> [  278.848764] R10: 00000000ffffe0be R11: 0000000000000000 R12: ffffea009c02da70
> [  278.852385] R13: ffff881800706475 R14: ffff88084439d680 R15: 0000000000000001
> [  278.872750] FS:  0000000000000000(0000) GS:ffff88003fc00000(0000)
> knlGS:0000000000000000
> [  278.889379] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
> [  278.895226] CR2: 00007fcb2369c000 CR3: 0000000848df0000 CR4: 00000000000026f0
> [  278.911157] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [  278.916692] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [  278.934540] Process rpciod/0 (pid: 2594, threadinfo
> ffff88087b9fc000, task ffff88087bdc2370)
> [  278.950913] Stack:
> [  278.952455]  ffff88087913e934 ffff88087747e048 00000000e3115a3e
> 0000000000037803
> [  278.969335] <0> ffff880854828000 ffff88084d37ce80 ffff88084439d680
> 0000000000000001
> [  278.976115] <0> ffff88087b9fdcc0 ffffffff81dae6d8 ffffffff81dbb5d5
> 00000000e3115a3e
> [  278.995869] Call Trace:
> [  278.997442]  [<ffffffff81dae6d8>] xs_set_port+0x73/0xa3
> [  279.011219]  [<ffffffff81dbb5d5>] ? rpcb_dec_getport+0x0/0xc2
> [  279.016461]  [<ffffffff81dbb4ac>] rpcb_getport_done+0x80/0xdb
> [  279.030585]  [<ffffffff81db1cbf>] rpc_exit_task+0x3b/0x7f
> [  279.033790]  [<ffffffff81db24a9>] __rpc_execute+0x97/0x251
> [  279.051120]  [<ffffffff81db26e2>] rpc_async_schedule+0x28/0x3e
> [  279.057205]  [<ffffffff810bbe9c>] worker_thread+0x1fa/0x317
> [  279.071181]  [<ffffffff810bbe42>] ? worker_thread+0x1a0/0x317
> [  279.075343]  [<ffffffff81db26ba>] ? rpc_async_schedule+0x0/0x3e
> [  279.092033]  [<ffffffff810c26c8>] ? autoremove_wake_function+0x0/0x63
> [  279.100028]  [<ffffffff810bbca2>] ? worker_thread+0x0/0x317
> [  279.112094]  [<ffffffff810c2277>] kthread+0x9d/0xa5
> [  279.116720]  [<ffffffff8104205a>] child_rip+0xa/0x20
> [  279.133457]  [<ffffffff810419fc>] ? restore_args+0x0/0x30
> [  279.137271]  [<ffffffff810c21da>] ? kthread+0x0/0xa5
> [  279.149933]  [<ffffffff81042050>] ? child_rip+0x0/0x20
> [  279.154086] Code: 00 00 00 00 00 ea ff ff 48 6b c0 68 4c 8d 24 10
> 66 41 83 3c 24 00 79 05 4d 8b 64 24 10 49 8b 04 24 84 c0 78 17 66 a9
> 00 c0 75 04 <0f> 0b eb fe 4c 89 e7 e8 87 b8 fc ff e9 80 00 00 00 48 8b
> 45 08
> [  279.198800] RIP  [<ffffffff81174437>] kfree+0x96/0x14a
> [  279.204063]  RSP <ffff88087b9fdc50>
> [  279.210307] ---[ end trace 52f0b2ff8e4dec91 ]---
>
> Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
> mpk12-3214-189-144 kernel: [  278.734149] ------------[ cut here ]------------
>
> Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
> mpk12-3214-189-144 kernel: [  278.743056] invalid opcode: 0000 [#1] SMP
>
> Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
> mpk12-3214-189-144 kernel: [  278.750365] last sysfs file:
> /sys/devices/virtual/net/sit0/type
>
> Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
> mpk12-3214-189-144 kernel: [  278.950913] Stack:
>
> Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
> mpk12-3214-189-144 kernel: [  278.995869] Call Trace:
>
> Message from syslogd@mpk12-3214-189-144 at Sat Oct 17 18:23:40 2009 ...
> mpk12-3214-189-144 kernel: [  279.154086] Code: 00 00 00 00 00 ea ff
> ff 48 6b c0 68 4c 8d 24 10 66 41 83 3c 24 00 79 05 4d 8b 64 24 10 49
> 8b 04 24 84 c0 78 17 66 a9 00 c0 75 04 <0f> 0b eb fe 4c 89 e7 e8 87 b8
> fc ff e9 80 00 00 00 48 8b 45 08
>
> mount.nfs: mount system call failed
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: nfs mount fail

[Arjan van de Ven]

On Mon, 19 Oct 2009 05:35:20 +0300
Pekka Enberg <penberg@cs.helsinki.fi> wrote:

> >
> > [  278.734149] ------------[ cut here ]------------
> > [  278.739620] kernel BUG at mm/slub.c:2969!
> 
> So this means that someone is passing a pointer to kfree() that did
> not come from kmalloc(). Which tree are you testing?

> > [  278.770030] Pid: 2594, comm: rpciod/0 Not tainted
> > 2.6.32-rc5-tip-01483-ga166936-dirty #651

some evil person decided to put the exact kernel tree/version in the
oops output ;-)


--
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: nfs mount fail

[Pekka Enberg]

On Mon, 2009-10-19 at 11:43 +0900, Arjan van de Ven wrote:
> On Mon, 19 Oct 2009 05:35:20 +0300
> Pekka Enberg <penberg@cs.helsinki.fi> wrote:
> 
> > >
> > > [  278.734149] ------------[ cut here ]------------
> > > [  278.739620] kernel BUG at mm/slub.c:2969!
> > 
> > So this means that someone is passing a pointer to kfree() that did
> > not come from kmalloc(). Which tree are you testing?
> 
> > > [  278.770030] Pid: 2594, comm: rpciod/0 Not tainted
> > > 2.6.32-rc5-tip-01483-ga166936-dirty #651
> 
> some evil person decided to put the exact kernel tree/version in the
> oops output ;-)

Oh, right. Ingo, Yinghai says Linus' tree is fine so any ideas why this
shows up in -tip? Also it seems we've had a similar bug before:

  http://lkml.org/lkml/2009/4/2/698

Hmmh?

			Pekka

Re: nfs mount fail

[Yinghai Lu]

On Sun, Oct 18, 2009 at 7:58 PM, Pekka Enberg <penberg@cs.helsinki.fi> wrote:
> On Mon, 2009-10-19 at 11:43 +0900, Arjan van de Ven wrote:
>> On Mon, 19 Oct 2009 05:35:20 +0300
>> Pekka Enberg <penberg@cs.helsinki.fi> wrote:
>>
>> > >
>> > > [  278.734149] ------------[ cut here ]------------
>> > > [  278.739620] kernel BUG at mm/slub.c:2969!
>> >
>> > So this means that someone is passing a pointer to kfree() that did
>> > not come from kmalloc(). Which tree are you testing?
>>
>> > > [  278.770030] Pid: 2594, comm: rpciod/0 Not tainted
>> > > 2.6.32-rc5-tip-01483-ga166936-dirty #651
>>
>> some evil person decided to put the exact kernel tree/version in the
>> oops output ;-)
>
> Oh, right. Ingo, Yinghai says Linus' tree is fine so any ideas why this
> shows up in -tip? Also it seems we've had a similar bug before:
>
>  http://lkml.org/lkml/2009/4/2/698
>
> Hmmh?

yes. something miss merged again...

need change some lines.

---
 fs/nfs/super.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

Index: linux-2.6/fs/nfs/super.c
===================================================================
--- linux-2.6.orig/fs/nfs/super.c
+++ linux-2.6/fs/nfs/super.c
@@ -1231,7 +1231,6 @@ static int nfs_parse_mount_options(char
 				goto out_nomem;
 			token = match_token(string,
 					    nfs_xprt_protocol_tokens, args);
-			kfree(string);

 			switch (token) {
 			case Opt_xprt_udp:
@@ -1254,6 +1253,7 @@ static int nfs_parse_mount_options(char
 			default:
 				dfprintk(MOUNT, "NFS:   unrecognized "
 						"transport protocol\n");
+				kfree(string);
 				return 0;
 			}
 			break;
@@ -1264,6 +1264,8 @@ static int nfs_parse_mount_options(char
 			token = match_token(string,
 					    nfs_xprt_protocol_tokens, args);

+			kfree(string);
+
 			switch (token) {
 			case Opt_xprt_udp:
 				mnt->mount_server.protocol = XPRT_TRANSPORT_UDP;

Re: nfs mount fail

[Trond Myklebust]

On Sun, 2009-10-18 at 22:52 -0700, Yinghai Lu wrote:
> On Sun, Oct 18, 2009 at 7:58 PM, Pekka Enberg <penberg@cs.helsinki.fi> wrote:
> > On Mon, 2009-10-19 at 11:43 +0900, Arjan van de Ven wrote:
> >> On Mon, 19 Oct 2009 05:35:20 +0300
> >> Pekka Enberg <penberg@cs.helsinki.fi> wrote:
> >>
> >> > >
> >> > > [  278.734149] ------------[ cut here ]------------
> >> > > [  278.739620] kernel BUG at mm/slub.c:2969!
> >> >
> >> > So this means that someone is passing a pointer to kfree() that did
> >> > not come from kmalloc(). Which tree are you testing?
> >>
> >> > > [  278.770030] Pid: 2594, comm: rpciod/0 Not tainted
> >> > > 2.6.32-rc5-tip-01483-ga166936-dirty #651
> >>
> >> some evil person decided to put the exact kernel tree/version in the
> >> oops output ;-)
> >
> > Oh, right. Ingo, Yinghai says Linus' tree is fine so any ideas why this
> > shows up in -tip? Also it seems we've had a similar bug before:
> >
> >  http://lkml.org/lkml/2009/4/2/698
> >
> > Hmmh?
> 
> yes. something miss merged again...
> 
> need change some lines.
> 
> ---
>  fs/nfs/super.c |    4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> Index: linux-2.6/fs/nfs/super.c
> ===================================================================
> --- linux-2.6.orig/fs/nfs/super.c
> +++ linux-2.6/fs/nfs/super.c
> @@ -1231,7 +1231,6 @@ static int nfs_parse_mount_options(char
>  				goto out_nomem;
>  			token = match_token(string,
>  					    nfs_xprt_protocol_tokens, args);
> -			kfree(string);
> 
>  			switch (token) {
>  			case Opt_xprt_udp:
> @@ -1254,6 +1253,7 @@ static int nfs_parse_mount_options(char
>  			default:
>  				dfprintk(MOUNT, "NFS:   unrecognized "
>  						"transport protocol\n");
> +				kfree(string);

This doesn't match mainline either. To do so, the above kfree() has to
be at the end of the "Opt_xprt_rdma:" case...

>  				return 0;
>  			}
>  			break;
> @@ -1264,6 +1264,8 @@ static int nfs_parse_mount_options(char
>  			token = match_token(string,
>  					    nfs_xprt_protocol_tokens, args);
> 
> +			kfree(string);
> +
>  			switch (token) {
>  			case Opt_xprt_udp:
>  				mnt->mount_server.protocol = XPRT_TRANSPORT_UDP;
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: nfs mount fail

[Ingo Molnar]

* Trond Myklebust <trond.myklebust@fys.uio.no> wrote:

> > yes. something miss merged again...
> > 
> > need change some lines.
> 
> This doesn't match mainline either. To do so, the above kfree() has to 
> be at the end of the "Opt_xprt_rdma:" case...

it's from a test patch in tip:out-of-tree:

  d40bc6b: <not-for-merge> nfs: fix nfs_parse_mount_options() double kfree()

(attached below)

that fix is wrong apparently - is there a correct fix upstream perhaps?

	Ingo


------------->
>From d40bc6bd8d2353700dea06d398d91a3f54887da6 Mon Sep 17 00:00:00 2001
From: Ingo Molnar <mingo@elte.hu>
Date: Fri, 3 Apr 2009 09:06:15 +0200
Subject: [PATCH] <not-for-merge> nfs: fix nfs_parse_mount_options() double kfree()
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit

Impact: fix crash

Yinghai Lu reported the following crash:

> mpk12-3214-189-158:~ # sh x
> [   63.198629] ------------[ cut here ]------------
> [   63.202589] kernel BUG at mm/slub.c:2753!
> [   63.202589] invalid opcode: 0000 [#1] SMP
> [   63.202589] last sysfs file: /sys/devices/virtual/net/sit0/type
> [   63.202589] CPU 0
> [   63.202589] Modules linked in:
> [   63.202589] Pid: 10027, comm: mount.nfs Not tainted 2.6.29-07100-g833bb30 #21 Sun Fire X4440
> [   63.202589] RIP: 0010:[<ffffffff802e0015>]  [<ffffffff802e0015>] kfree+0x5a/0xcd
> [   63.202589] RSP: 0018:ffff882042ceb9f8  EFLAGS: 00010246
> [   63.202589] RAX: 0200000000000000 RBX: 0000000000000005 RCX: ffffffff80a7dc1f
> [   63.202589] RDX: ffffe20000000000 RSI: ffffc2000000f470 RDI: ffffe2001c018950
> [   63.202589] RBP: ffff882042ceba18 R08: 0000000000000000 R09: ffffffff811019c0
> [   63.202589] R10: 000000004262ce02 R11: ffff882042ceba18 R12: ffff880800706475
> [   63.202589] R13: ffff882042886000 R14: ffff882042cebbd8 R15: ffff882042cebbf0
> [   63.202589] FS:  00007fac729ed6f0(0000) GS:ffffc20000000000(0000) knlGS:0000000000000000
> [   63.202589] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [   63.202589] CR2: 00007fac72c12000 CR3: 0000001841cbb000 CR4: 00000000000006e0
> [   63.202589] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [   63.202589] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [   63.202589] Process mount.nfs (pid: 10027, threadinfo ffff882042cea000, task ffff8820434dc290)
> [   63.202589] Stack:
> [   63.202589]  ffff882042ceba18 000000004262ce02 0000000000000005 ffff882042886028
> [   63.202589]  ffff882042ceba58 ffffffff80a7dc1f 000000004262ce02 ffff882042886000
> [   63.202589]  000000004262ce02 ffff882042886000 ffffffff80a7b4a6 ffff882042c9ee18
> [   63.202589] Call Trace:
> [   63.202589]  [<ffffffff80a7dc1f>] xs_destroy+0x67/0xac
> [   63.202589]  [<ffffffff80a7b4a6>] ? xprt_destroy+0x0/0xa7
> [   63.202589]  [<ffffffff80a7b532>] xprt_destroy+0x8c/0xa7
> [   63.202589]  [<ffffffff80a823b2>] ? put_rpccred+0x112/0x131
> [   63.202589]  [<ffffffff8051cdd5>] kref_put+0x65/0x87
> [   63.202589]  [<ffffffff80a7a9a9>] ? rpc_free_client+0x0/0xf9
> [   63.202589]  [<ffffffff80a7b490>] xprt_put+0x23/0x39
> [   63.202589]  [<ffffffff80a7aa7a>] rpc_free_client+0xd1/0xf9
> [   63.202589]  [<ffffffff80a83345>] ? unx_destroy+0x3c/0x57
> [   63.202589]  [<ffffffff8051cdd5>] kref_put+0x65/0x87
> [   63.202589]  [<ffffffff80a7aaa2>] ? rpc_free_auth+0x0/0x69
> [   63.202589]  [<ffffffff80a7aaf0>] rpc_free_auth+0x4e/0x69
> [   63.202589]  [<ffffffff8025b827>] ? __wake_up+0x52/0x75
> [   63.202589]  [<ffffffff8051cdd5>] kref_put+0x65/0x87
> [   63.202589]  [<ffffffff80a7a98e>] rpc_release_client+0x64/0x7f
> [   63.202589]  [<ffffffff80a8061c>] ? rpc_put_task+0xb0/0xcb
> [   63.202589]  [<ffffffff80a7abe0>] rpc_shutdown_client+0xd5/0xf8
> [   63.202589]  [<ffffffff80a7a893>] ? rpc_call_sync+0x63/0x80
> [   63.202589]  [<ffffffff803fc4ab>] nfs_mount+0x11f/0x1bf
> [   63.202589]  [<ffffffff803f3036>] nfs_get_sb+0x4ac/0x82a
> [   63.202589]  [<ffffffff802e8f24>] vfs_kern_mount+0x61/0xbf
> [   63.202589]  [<ffffffff802fea1d>] ? get_fs_type+0x58/0xc5
> [   63.202589]  [<ffffffff802e9015>] do_kern_mount+0x56/0x108
> [   63.202589]  [<ffffffff80302195>] do_mount+0x729/0x788
> [   63.202589]  [<ffffffff80300025>] ? copy_mount_options+0xdf/0x155
> [   63.202589]  [<ffffffff8030228c>] sys_mount+0x98/0xf8
> [   63.202589]  [<ffffffff80230d6b>] system_call_fastpath+0x16/0x1b
> [   63.202589] Code: 0c 48 ba 00 00 00 00 00 e2 ff ff 48 6b c0 38 48 8d 3c 10 48 8b 07 f6 c4 40 74 04 48 8b 7f 10 48 8b 07 84 c0 78 10 f6 c4 60 75 04 <0f> 0b eb fe e8 90 75 fd ff eb 4c 48 8b 4d 08 4c 8b 4f 10 9c 5b
> [   63.202589] RIP  [<ffffffff802e0015>] kfree+0x5a/0xcd
> [   63.202589]  RSP <ffff882042ceb9f8>
> [   63.524555] ---[ end trace cd0d38e02ad11d61 ]---

Pekka observed that a bogus pointer was passed to kfree().

This commit:

  a67d18f: NFS: load the rpc/rdma transport module automatically

Moved a kfree() of the options strings in nfs_parse_mount_options()
inadvertently and introduced a double kfree(). Fix it.

Reported-by: Yinghai Lu <yinghai@kernel.org>
Analyzed-by: Pekka Enberg <penberg@cs.helsinki.fi>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
 fs/nfs/super.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index a2c18ac..482a2c3 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -1231,6 +1231,7 @@ static int nfs_parse_mount_options(char *raw,
 				goto out_nomem;
 			token = match_token(string,
 					    nfs_xprt_protocol_tokens, args);
+			kfree(string);
 
 			switch (token) {
 			case Opt_xprt_udp:
@@ -1262,7 +1263,6 @@ static int nfs_parse_mount_options(char *raw,
 				goto out_nomem;
 			token = match_token(string,
 					    nfs_xprt_protocol_tokens, args);
-			kfree(string);
 
 			switch (token) {
 			case Opt_xprt_udp:

Re: nfs mount fail

[Pekka Enberg]

Hi Ingo,

On Mon, 2009-10-19 at 08:54 +0200, Ingo Molnar wrote:
> * Trond Myklebust <trond.myklebust@fys.uio.no> wrote:
> 
> > > yes. something miss merged again...
> > > 
> > > need change some lines.
> > 
> > This doesn't match mainline either. To do so, the above kfree() has to 
> > be at the end of the "Opt_xprt_rdma:" case...
> 
> it's from a test patch in tip:out-of-tree:
> 
>   d40bc6b: <not-for-merge> nfs: fix nfs_parse_mount_options() double kfree()
> 
> (attached below)
> 
> that fix is wrong apparently - is there a correct fix upstream perhaps?

AFAICT, yes. See commit d508afb437daee7cf07da085b635c44a4ebf9b38 ("NFS:
Fix a double free in nfs_parse_mount_options()") from Trond.

			Pekka

> 
> 	Ingo
> 
> 
> ------------->
> From d40bc6bd8d2353700dea06d398d91a3f54887da6 Mon Sep 17 00:00:00 2001
> From: Ingo Molnar <mingo@elte.hu>
> Date: Fri, 3 Apr 2009 09:06:15 +0200
> Subject: [PATCH] <not-for-merge> nfs: fix nfs_parse_mount_options() double kfree()
> MIME-Version: 1.0
> Content-Type: text/plain; charset=utf-8
> Content-Transfer-Encoding: 8bit
> 
> Impact: fix crash
> 
> Yinghai Lu reported the following crash:
> 
> > mpk12-3214-189-158:~ # sh x
> > [   63.198629] ------------[ cut here ]------------
> > [   63.202589] kernel BUG at mm/slub.c:2753!
> > [   63.202589] invalid opcode: 0000 [#1] SMP
> > [   63.202589] last sysfs file: /sys/devices/virtual/net/sit0/type
> > [   63.202589] CPU 0
> > [   63.202589] Modules linked in:
> > [   63.202589] Pid: 10027, comm: mount.nfs Not tainted 2.6.29-07100-g833bb30 #21 Sun Fire X4440
> > [   63.202589] RIP: 0010:[<ffffffff802e0015>]  [<ffffffff802e0015>] kfree+0x5a/0xcd
> > [   63.202589] RSP: 0018:ffff882042ceb9f8  EFLAGS: 00010246
> > [   63.202589] RAX: 0200000000000000 RBX: 0000000000000005 RCX: ffffffff80a7dc1f
> > [   63.202589] RDX: ffffe20000000000 RSI: ffffc2000000f470 RDI: ffffe2001c018950
> > [   63.202589] RBP: ffff882042ceba18 R08: 0000000000000000 R09: ffffffff811019c0
> > [   63.202589] R10: 000000004262ce02 R11: ffff882042ceba18 R12: ffff880800706475
> > [   63.202589] R13: ffff882042886000 R14: ffff882042cebbd8 R15: ffff882042cebbf0
> > [   63.202589] FS:  00007fac729ed6f0(0000) GS:ffffc20000000000(0000) knlGS:0000000000000000
> > [   63.202589] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> > [   63.202589] CR2: 00007fac72c12000 CR3: 0000001841cbb000 CR4: 00000000000006e0
> > [   63.202589] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > [   63.202589] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> > [   63.202589] Process mount.nfs (pid: 10027, threadinfo ffff882042cea000, task ffff8820434dc290)
> > [   63.202589] Stack:
> > [   63.202589]  ffff882042ceba18 000000004262ce02 0000000000000005 ffff882042886028
> > [   63.202589]  ffff882042ceba58 ffffffff80a7dc1f 000000004262ce02 ffff882042886000
> > [   63.202589]  000000004262ce02 ffff882042886000 ffffffff80a7b4a6 ffff882042c9ee18
> > [   63.202589] Call Trace:
> > [   63.202589]  [<ffffffff80a7dc1f>] xs_destroy+0x67/0xac
> > [   63.202589]  [<ffffffff80a7b4a6>] ? xprt_destroy+0x0/0xa7
> > [   63.202589]  [<ffffffff80a7b532>] xprt_destroy+0x8c/0xa7
> > [   63.202589]  [<ffffffff80a823b2>] ? put_rpccred+0x112/0x131
> > [   63.202589]  [<ffffffff8051cdd5>] kref_put+0x65/0x87
> > [   63.202589]  [<ffffffff80a7a9a9>] ? rpc_free_client+0x0/0xf9
> > [   63.202589]  [<ffffffff80a7b490>] xprt_put+0x23/0x39
> > [   63.202589]  [<ffffffff80a7aa7a>] rpc_free_client+0xd1/0xf9
> > [   63.202589]  [<ffffffff80a83345>] ? unx_destroy+0x3c/0x57
> > [   63.202589]  [<ffffffff8051cdd5>] kref_put+0x65/0x87
> > [   63.202589]  [<ffffffff80a7aaa2>] ? rpc_free_auth+0x0/0x69
> > [   63.202589]  [<ffffffff80a7aaf0>] rpc_free_auth+0x4e/0x69
> > [   63.202589]  [<ffffffff8025b827>] ? __wake_up+0x52/0x75
> > [   63.202589]  [<ffffffff8051cdd5>] kref_put+0x65/0x87
> > [   63.202589]  [<ffffffff80a7a98e>] rpc_release_client+0x64/0x7f
> > [   63.202589]  [<ffffffff80a8061c>] ? rpc_put_task+0xb0/0xcb
> > [   63.202589]  [<ffffffff80a7abe0>] rpc_shutdown_client+0xd5/0xf8
> > [   63.202589]  [<ffffffff80a7a893>] ? rpc_call_sync+0x63/0x80
> > [   63.202589]  [<ffffffff803fc4ab>] nfs_mount+0x11f/0x1bf
> > [   63.202589]  [<ffffffff803f3036>] nfs_get_sb+0x4ac/0x82a
> > [   63.202589]  [<ffffffff802e8f24>] vfs_kern_mount+0x61/0xbf
> > [   63.202589]  [<ffffffff802fea1d>] ? get_fs_type+0x58/0xc5
> > [   63.202589]  [<ffffffff802e9015>] do_kern_mount+0x56/0x108
> > [   63.202589]  [<ffffffff80302195>] do_mount+0x729/0x788
> > [   63.202589]  [<ffffffff80300025>] ? copy_mount_options+0xdf/0x155
> > [   63.202589]  [<ffffffff8030228c>] sys_mount+0x98/0xf8
> > [   63.202589]  [<ffffffff80230d6b>] system_call_fastpath+0x16/0x1b
> > [   63.202589] Code: 0c 48 ba 00 00 00 00 00 e2 ff ff 48 6b c0 38 48 8d 3c 10 48 8b 07 f6 c4 40 74 04 48 8b 7f 10 48 8b 07 84 c0 78 10 f6 c4 60 75 04 <0f> 0b eb fe e8 90 75 fd ff eb 4c 48 8b 4d 08 4c 8b 4f 10 9c 5b
> > [   63.202589] RIP  [<ffffffff802e0015>] kfree+0x5a/0xcd
> > [   63.202589]  RSP <ffff882042ceb9f8>
> > [   63.524555] ---[ end trace cd0d38e02ad11d61 ]---
> 
> Pekka observed that a bogus pointer was passed to kfree().
> 
> This commit:
> 
>   a67d18f: NFS: load the rpc/rdma transport module automatically
> 
> Moved a kfree() of the options strings in nfs_parse_mount_options()
> inadvertently and introduced a double kfree(). Fix it.
> 
> Reported-by: Yinghai Lu <yinghai@kernel.org>
> Analyzed-by: Pekka Enberg <penberg@cs.helsinki.fi>
> Signed-off-by: Ingo Molnar <mingo@elte.hu>
> ---
>  fs/nfs/super.c |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
> 
> diff --git a/fs/nfs/super.c b/fs/nfs/super.c
> index a2c18ac..482a2c3 100644
> --- a/fs/nfs/super.c
> +++ b/fs/nfs/super.c
> @@ -1231,6 +1231,7 @@ static int nfs_parse_mount_options(char *raw,
>  				goto out_nomem;
>  			token = match_token(string,
>  					    nfs_xprt_protocol_tokens, args);
> +			kfree(string);
>  
>  			switch (token) {
>  			case Opt_xprt_udp:
> @@ -1262,7 +1263,6 @@ static int nfs_parse_mount_options(char *raw,
>  				goto out_nomem;
>  			token = match_token(string,
>  					    nfs_xprt_protocol_tokens, args);
> -			kfree(string);
>  
>  			switch (token) {
>  			case Opt_xprt_udp:
> 

[re-send PATCH] nfs: Fix nfs_parse_mount_options() double kfree()

[Ingo Molnar]

* Ingo Molnar <mingo@elte.hu> wrote:

> * Trond Myklebust <trond.myklebust@fys.uio.no> wrote:
> 
> > > yes. something miss merged again...
> > > 
> > > need change some lines.
> > 
> > This doesn't match mainline either. To do so, the above kfree() has to 
> > be at the end of the "Opt_xprt_rdma:" case...
> 
> it's from a test patch in tip:out-of-tree:
> 
>   d40bc6b: <not-for-merge> nfs: fix nfs_parse_mount_options() double kfree()
> 
> (attached below)
> 
> that fix is wrong apparently - is there a correct fix upstream perhaps?

Looks like this might be related to a long-standing NFS bug not fixed 
upstream yet. You were Cc:-ed to the original thread and to the original 
patch, see this thread on lkml:

	nfs mount fail on linus 20090402 git

This newer "nfs mount fail" thread got started because the surrounding 
code changed and the pending fix got mismerged. The merged up (and 
fixed) version is attached below - what remains of it by today is a 
memory leak fix.

The original NFS bug was apparently fixed upstream without crediting 
Yinghai and Pekka for finding it, and without a reply to the "nfs mount 
fail on linus 20090402 git" thread, so the patch stayed pending.

	Ingo

-------------->
>From 6914a677dc32b99b5be4e76b84b465fa3e417b94 Mon Sep 17 00:00:00 2001
From: Ingo Molnar <mingo@elte.hu>
Date: Fri, 3 Apr 2009 09:06:15 +0200
Subject: [PATCH] <not-for-merge> nfs: fix nfs_parse_mount_options() double kfree()
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit

Impact: fix crash

Yinghai Lu reported the following crash:

> mpk12-3214-189-158:~ # sh x
> [   63.198629] ------------[ cut here ]------------
> [   63.202589] kernel BUG at mm/slub.c:2753!
> [   63.202589] invalid opcode: 0000 [#1] SMP
> [   63.202589] last sysfs file: /sys/devices/virtual/net/sit0/type
> [   63.202589] CPU 0
> [   63.202589] Modules linked in:
> [   63.202589] Pid: 10027, comm: mount.nfs Not tainted 2.6.29-07100-g833bb30 #21 Sun Fire X4440
> [   63.202589] RIP: 0010:[<ffffffff802e0015>]  [<ffffffff802e0015>] kfree+0x5a/0xcd
> [   63.202589] RSP: 0018:ffff882042ceb9f8  EFLAGS: 00010246
> [   63.202589] RAX: 0200000000000000 RBX: 0000000000000005 RCX: ffffffff80a7dc1f
> [   63.202589] RDX: ffffe20000000000 RSI: ffffc2000000f470 RDI: ffffe2001c018950
> [   63.202589] RBP: ffff882042ceba18 R08: 0000000000000000 R09: ffffffff811019c0
> [   63.202589] R10: 000000004262ce02 R11: ffff882042ceba18 R12: ffff880800706475
> [   63.202589] R13: ffff882042886000 R14: ffff882042cebbd8 R15: ffff882042cebbf0
> [   63.202589] FS:  00007fac729ed6f0(0000) GS:ffffc20000000000(0000) knlGS:0000000000000000
> [   63.202589] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [   63.202589] CR2: 00007fac72c12000 CR3: 0000001841cbb000 CR4: 00000000000006e0
> [   63.202589] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [   63.202589] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [   63.202589] Process mount.nfs (pid: 10027, threadinfo ffff882042cea000, task ffff8820434dc290)
> [   63.202589] Stack:
> [   63.202589]  ffff882042ceba18 000000004262ce02 0000000000000005 ffff882042886028
> [   63.202589]  ffff882042ceba58 ffffffff80a7dc1f 000000004262ce02 ffff882042886000
> [   63.202589]  000000004262ce02 ffff882042886000 ffffffff80a7b4a6 ffff882042c9ee18
> [   63.202589] Call Trace:
> [   63.202589]  [<ffffffff80a7dc1f>] xs_destroy+0x67/0xac
> [   63.202589]  [<ffffffff80a7b4a6>] ? xprt_destroy+0x0/0xa7
> [   63.202589]  [<ffffffff80a7b532>] xprt_destroy+0x8c/0xa7
> [   63.202589]  [<ffffffff80a823b2>] ? put_rpccred+0x112/0x131
> [   63.202589]  [<ffffffff8051cdd5>] kref_put+0x65/0x87
> [   63.202589]  [<ffffffff80a7a9a9>] ? rpc_free_client+0x0/0xf9
> [   63.202589]  [<ffffffff80a7b490>] xprt_put+0x23/0x39
> [   63.202589]  [<ffffffff80a7aa7a>] rpc_free_client+0xd1/0xf9
> [   63.202589]  [<ffffffff80a83345>] ? unx_destroy+0x3c/0x57
> [   63.202589]  [<ffffffff8051cdd5>] kref_put+0x65/0x87
> [   63.202589]  [<ffffffff80a7aaa2>] ? rpc_free_auth+0x0/0x69
> [   63.202589]  [<ffffffff80a7aaf0>] rpc_free_auth+0x4e/0x69
> [   63.202589]  [<ffffffff8025b827>] ? __wake_up+0x52/0x75
> [   63.202589]  [<ffffffff8051cdd5>] kref_put+0x65/0x87
> [   63.202589]  [<ffffffff80a7a98e>] rpc_release_client+0x64/0x7f
> [   63.202589]  [<ffffffff80a8061c>] ? rpc_put_task+0xb0/0xcb
> [   63.202589]  [<ffffffff80a7abe0>] rpc_shutdown_client+0xd5/0xf8
> [   63.202589]  [<ffffffff80a7a893>] ? rpc_call_sync+0x63/0x80
> [   63.202589]  [<ffffffff803fc4ab>] nfs_mount+0x11f/0x1bf
> [   63.202589]  [<ffffffff803f3036>] nfs_get_sb+0x4ac/0x82a
> [   63.202589]  [<ffffffff802e8f24>] vfs_kern_mount+0x61/0xbf
> [   63.202589]  [<ffffffff802fea1d>] ? get_fs_type+0x58/0xc5
> [   63.202589]  [<ffffffff802e9015>] do_kern_mount+0x56/0x108
> [   63.202589]  [<ffffffff80302195>] do_mount+0x729/0x788
> [   63.202589]  [<ffffffff80300025>] ? copy_mount_options+0xdf/0x155
> [   63.202589]  [<ffffffff8030228c>] sys_mount+0x98/0xf8
> [   63.202589]  [<ffffffff80230d6b>] system_call_fastpath+0x16/0x1b
> [   63.202589] Code: 0c 48 ba 00 00 00 00 00 e2 ff ff 48 6b c0 38 48 8d 3c 10 48 8b 07 f6 c4 40 74 04 48 8b 7f 10 48 8b 07 84 c0 78 10 f6 c4 60 75 04 <0f> 0b eb fe e8 90 75 fd ff eb 4c 48 8b 4d 08 4c 8b 4f 10 9c 5b
> [   63.202589] RIP  [<ffffffff802e0015>] kfree+0x5a/0xcd
> [   63.202589]  RSP <ffff882042ceb9f8>
> [   63.524555] ---[ end trace cd0d38e02ad11d61 ]---

Pekka observed that a bogus pointer was passed to kfree().

This commit:

  a67d18f: NFS: load the rpc/rdma transport module automatically

Moved a kfree() of the options strings in nfs_parse_mount_options()
inadvertently and introduced a double kfree(). Fix it.

Reported-by: Yinghai Lu <yinghai@kernel.org>
Analyzed-by: Pekka Enberg <penberg@cs.helsinki.fi>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
 fs/nfs/super.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index a2c18ac..1fcb375 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -1253,6 +1253,7 @@ static int nfs_parse_mount_options(char *raw,
 			default:
 				dfprintk(MOUNT, "NFS:   unrecognized "
 						"transport protocol\n");
+				kfree(string);
 				return 0;
 			}
 			break;

Re: nfs mount fail

[Trond Myklebust]

On Mon, 2009-10-19 at 08:54 +0200, Ingo Molnar wrote:
> * Trond Myklebust <trond.myklebust@fys.uio.no> wrote:
> 
> > > yes. something miss merged again...
> > > 
> > > need change some lines.
> > 
> > This doesn't match mainline either. To do so, the above kfree() has to 
> > be at the end of the "Opt_xprt_rdma:" case...
> 
> it's from a test patch in tip:out-of-tree:
> 
>   d40bc6b: <not-for-merge> nfs: fix nfs_parse_mount_options() double kfree()
> 
> (attached below)
> 
> that fix is wrong apparently - is there a correct fix upstream perhaps?
<snip>
> This commit:
> 
>   a67d18f: NFS: load the rpc/rdma transport module automatically
> 
> Moved a kfree() of the options strings in nfs_parse_mount_options()
> inadvertently and introduced a double kfree(). Fix it.
> 
> Reported-by: Yinghai Lu <yinghai@kernel.org>
> Analyzed-by: Pekka Enberg <penberg@cs.helsinki.fi>
> Signed-off-by: Ingo Molnar <mingo@elte.hu>
> ---
>  fs/nfs/super.c |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
> 
> diff --git a/fs/nfs/super.c b/fs/nfs/super.c
> index a2c18ac..482a2c3 100644
> --- a/fs/nfs/super.c
> +++ b/fs/nfs/super.c
> @@ -1231,6 +1231,7 @@ static int nfs_parse_mount_options(char *raw,
>  				goto out_nomem;
>  			token = match_token(string,
>  					    nfs_xprt_protocol_tokens, args);
> +			kfree(string);
>  
>  			switch (token) {
>  			case Opt_xprt_udp:
> @@ -1262,7 +1263,6 @@ static int nfs_parse_mount_options(char *raw,
>  				goto out_nomem;
>  			token = match_token(string,
>  					    nfs_xprt_protocol_tokens, args);
> -			kfree(string);
>  
>  			switch (token) {
>  			case Opt_xprt_udp:

This patch appears to be reverting the correct fix from commit d508afb
in mainline
        http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git&a=commitdiff&h=d508afb437daee7cf07da085b635c44a4ebf9b38

Cheers
  Trond

Re: nfs mount fail

[Ingo Molnar]

* Pekka Enberg <penberg@cs.helsinki.fi> wrote:

> Hi Ingo,
> 
> On Mon, 2009-10-19 at 08:54 +0200, Ingo Molnar wrote:
> > * Trond Myklebust <trond.myklebust@fys.uio.no> wrote:
> > 
> > > > yes. something miss merged again...
> > > > 
> > > > need change some lines.
> > > 
> > > This doesn't match mainline either. To do so, the above kfree() has to 
> > > be at the end of the "Opt_xprt_rdma:" case...
> > 
> > it's from a test patch in tip:out-of-tree:
> > 
> >   d40bc6b: <not-for-merge> nfs: fix nfs_parse_mount_options() double kfree()
> > 
> > (attached below)
> > 
> > that fix is wrong apparently - is there a correct fix upstream perhaps?
> 
> AFAICT, yes. See commit d508afb437daee7cf07da085b635c44a4ebf9b38 ("NFS:
> Fix a double free in nfs_parse_mount_options()") from Trond.

sigh. That was written 3 days after your and Yinghai's patch was sent - 
there's no Reported-by credit, there was no Cc: back to the original 
thread - plus there was a memory leak was left in there as well. Trond?

	Ingo

Re: [re-send PATCH] nfs: Fix nfs_parse_mount_options() double kfree()

[Ingo Molnar]

btw., the subject of this patch should probably be changed to:

  nfs: Fix memory leak in nfs_parse_mount_options()

As half of the original fix went upstream via d508afb4 already.

	Ingo

Re: nfs mount fail

[Yinghai Lu]

On Mon, Oct 19, 2009 at 12:08 AM, Ingo Molnar <mingo@elte.hu> wrote:
>
> * Pekka Enberg <penberg@cs.helsinki.fi> wrote:
>
>> Hi Ingo,
>>
>> On Mon, 2009-10-19 at 08:54 +0200, Ingo Molnar wrote:
>> > * Trond Myklebust <trond.myklebust@fys.uio.no> wrote:
>> >
>> > > > yes. something miss merged again...
>> > > >
>> > > > need change some lines.
>> > >
>> > > This doesn't match mainline either. To do so, the above kfree() has to
>> > > be at the end of the "Opt_xprt_rdma:" case...
>> >
>> > it's from a test patch in tip:out-of-tree:
>> >
>> >   d40bc6b: <not-for-merge> nfs: fix nfs_parse_mount_options() double kfree()
>> >
>> > (attached below)
>> >
>> > that fix is wrong apparently - is there a correct fix upstream perhaps?
>>
>> AFAICT, yes. See commit d508afb437daee7cf07da085b635c44a4ebf9b38 ("NFS:
>> Fix a double free in nfs_parse_mount_options()") from Trond.
>
> sigh. That was written 3 days after your and Yinghai's patch was sent -
> there's no Reported-by credit, there was no Cc: back to the original
> thread - plus there was a memory leak was left in there as well. Trond?
>
sth like this to fix possible mem leaking...

diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index a2c18ac..90be551 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -1253,6 +1253,7 @@ static int nfs_parse_mount_options(char *raw,
 			default:
 				dfprintk(MOUNT, "NFS:   unrecognized "
 						"transport protocol\n");
+				kfree(string);
 				return 0;
 			}
 			break;

Re: nfs mount fail

[Trond Myklebust]

On Mon, 2009-10-19 at 00:31 -0700, Yinghai Lu wrote:
> On Mon, Oct 19, 2009 at 12:08 AM, Ingo Molnar <mingo@elte.hu> wrote:
> >
> > * Pekka Enberg <penberg@cs.helsinki.fi> wrote:
> >
> >> Hi Ingo,
> >>
> >> On Mon, 2009-10-19 at 08:54 +0200, Ingo Molnar wrote:
> >> > * Trond Myklebust <trond.myklebust@fys.uio.no> wrote:
> >> >
> >> > > > yes. something miss merged again...
> >> > > >
> >> > > > need change some lines.
> >> > >
> >> > > This doesn't match mainline either. To do so, the above kfree() has to
> >> > > be at the end of the "Opt_xprt_rdma:" case...
> >> >
> >> > it's from a test patch in tip:out-of-tree:
> >> >
> >> >   d40bc6b: <not-for-merge> nfs: fix nfs_parse_mount_options() double kfree()
> >> >
> >> > (attached below)
> >> >
> >> > that fix is wrong apparently - is there a correct fix upstream perhaps?
> >>
> >> AFAICT, yes. See commit d508afb437daee7cf07da085b635c44a4ebf9b38 ("NFS:
> >> Fix a double free in nfs_parse_mount_options()") from Trond.
> >
> > sigh. That was written 3 days after your and Yinghai's patch was sent -
> > there's no Reported-by credit, there was no Cc: back to the original
> > thread - plus there was a memory leak was left in there as well. Trond?

Not sure what happened there, but I do know that evolution-2.26 manages
to lose my exchange based email every now and again. It is possible that
was why I lost track of the original patch...

> sth like this to fix possible mem leaking...
> 
> diff --git a/fs/nfs/super.c b/fs/nfs/super.c
> index a2c18ac..90be551 100644
> --- a/fs/nfs/super.c
> +++ b/fs/nfs/super.c
> @@ -1253,6 +1253,7 @@ static int nfs_parse_mount_options(char *raw,
>  			default:
>  				dfprintk(MOUNT, "NFS:   unrecognized "
>  						"transport protocol\n");
> +				kfree(string);
>  				return 0;
>  			}
>  			break;

There is a possible clean up there too. We can move the other kfree()
calls out of the inner switch statement, and coalesce them all into a
single call.

Trond

Re: nfs mount fail

[Ingo Molnar]

* Trond Myklebust <trond.myklebust@fys.uio.no> wrote:

> > --- a/fs/nfs/super.c
> > +++ b/fs/nfs/super.c
> > @@ -1253,6 +1253,7 @@ static int nfs_parse_mount_options(char *raw,
> >  			default:
> >  				dfprintk(MOUNT, "NFS:   unrecognized "
> >  						"transport protocol\n");
> > +				kfree(string);
> >  				return 0;
> >  			}
> >  			break;
> 
> There is a possible clean up there too. We can move the other kfree() 
> calls out of the inner switch statement, and coalesce them all into a 
> single call.

Correct - separately from the leak fix. (which potentially wants to go 
to -stable as well)

Plus it's not just the kfree() calls that can be refactored but also the 
~25 match_strdup() call sites. Most of these repetitive sequences:

                case Opt_retrans:
                        string = match_strdup(args);
                        if (string == NULL)
                                goto out_nomem;
                        rc = strict_strtoul(string, 10, &option);
                        kfree(string);
                        if (rc != 0 || option == 0)
                                goto out_invalid_value;
                        mnt->retrans = option;
                        break;

could be pushed into a helper function, along the lines of:

                case Opt_retrans:
			if (parse_opt(args, &mnt->retrans) < 0 || mnt->retrans == 0)
                                goto out_invalid_value;
                        break;

where the non-repetitive value checks can be done after a generic 
parse_opt(). (or something like that)

That makes it more readable as well, as the switch statement will only 
list true per option properties, with minimal repetitive patterns.

	Ingo

Re: nfs mount fail

[Ingo Molnar]

* Ingo Molnar <mingo@elte.hu> wrote:

> 
> * Trond Myklebust <trond.myklebust@fys.uio.no> wrote:
> 
> > > --- a/fs/nfs/super.c
> > > +++ b/fs/nfs/super.c
> > > @@ -1253,6 +1253,7 @@ static int nfs_parse_mount_options(char *raw,
> > >  			default:
> > >  				dfprintk(MOUNT, "NFS:   unrecognized "
> > >  						"transport protocol\n");
> > > +				kfree(string);
> > >  				return 0;
> > >  			}
> > >  			break;
> > 
> > There is a possible clean up there too. We can move the other kfree() 
> > calls out of the inner switch statement, and coalesce them all into a 
> > single call.
> 
> Correct - separately from the leak fix. (which potentially wants to go 
> to -stable as well)

Not necessarily -stable material though - this is a really light memory 
leak and only on a rare failure path, i doubt anyone noticed in 
practice.

So d508afb fixed all that needed fixing and there's nothing serious 
pending here. I've reverted all pending bits in tip:out-of-tree, so it's 
pure -git now. I rarely have to carry any NFS fixes in out-of-tree, this 
was an odd-one-out exception that fell through the cracks.

	Ingo

Re: nfs mount fail

[Trond Myklebust]

On Mon, 2009-10-19 at 10:23 +0200, Ingo Molnar wrote:
> * Ingo Molnar <mingo@elte.hu> wrote:
> 
> > 
> > * Trond Myklebust <trond.myklebust@fys.uio.no> wrote:
> > 
> > > > --- a/fs/nfs/super.c
> > > > +++ b/fs/nfs/super.c
> > > > @@ -1253,6 +1253,7 @@ static int nfs_parse_mount_options(char *raw,
> > > >  			default:
> > > >  				dfprintk(MOUNT, "NFS:   unrecognized "
> > > >  						"transport protocol\n");
> > > > +				kfree(string);
> > > >  				return 0;
> > > >  			}
> > > >  			break;
> > > 
> > > There is a possible clean up there too. We can move the other kfree() 
> > > calls out of the inner switch statement, and coalesce them all into a 
> > > single call.
> > 
> > Correct - separately from the leak fix. (which potentially wants to go 
> > to -stable as well)
> 
> Not necessarily -stable material though - this is a really light memory 
> leak and only on a rare failure path, i doubt anyone noticed in 
> practice.
> 
> So d508afb fixed all that needed fixing and there's nothing serious 
> pending here. I've reverted all pending bits in tip:out-of-tree, so it's 
> pure -git now. I rarely have to carry any NFS fixes in out-of-tree, this 
> was an odd-one-out exception that fell through the cracks.
> 
> 	Ingo

OK. Could you resend the remaining leak fix, with the fixed up changelog
and attributions so I can pass it on to Linus?

Trond

[PATCH] nfs: Fix nfs_parse_mount_options() kfree() leak

[Ingo Molnar]

* Trond Myklebust <trond.myklebust@fys.uio.no> wrote:

> > So d508afb fixed all that needed fixing and there's nothing serious 
> > pending here. I've reverted all pending bits in tip:out-of-tree, so 
> > it's pure -git now. I rarely have to carry any NFS fixes in 
> > out-of-tree, this was an odd-one-out exception that fell through the 
> > cracks.
> > 
> > 	Ingo
> 
> OK. Could you resend the remaining leak fix, with the fixed up 
> changelog and attributions so I can pass it on to Linus?

Sure - find it below.

Thanks,

	Ingo

-------------------------------------------->
>From 42497c08be18c388f696a692ca5f8c3ba48cb183 Mon Sep 17 00:00:00 2001
From: Yinghai Lu <yinghai@kernel.org>
Date: Fri, 3 Apr 2009 09:06:15 +0200
Subject: [PATCH] nfs: Fix nfs_parse_mount_options() kfree() leak

Fix a (small) memory leak in one of the error paths of
the NFS mount options parsing code.

Reported-by: Yinghai Lu <yinghai@kernel.org>
Reported-by: Pekka Enberg <penberg@cs.helsinki.fi>
Signed-off-by: Ingo Molnar <mingo@elte.hu>
---
 fs/nfs/super.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/super.c b/fs/nfs/super.c
index a2c18ac..90be551 100644
--- a/fs/nfs/super.c
+++ b/fs/nfs/super.c
@@ -1253,6 +1253,7 @@ static int nfs_parse_mount_options(char *raw,
 			default:
 				dfprintk(MOUNT, "NFS:   unrecognized "
 						"transport protocol\n");
+				kfree(string);
 				return 0;
 			}
 			break;