From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S262453AbUEONdu (ORCPT ); Sat, 15 May 2004 09:33:50 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S262459AbUEONdu (ORCPT ); Sat, 15 May 2004 09:33:50 -0400 Received: from linuxhacker.ru ([217.76.32.60]:55765 "EHLO shrek.linuxhacker.ru") by vger.kernel.org with ESMTP id S262453AbUEONdp (ORCPT ); Sat, 15 May 2004 09:33:45 -0400 Date: Sat, 15 May 2004 16:21:49 +0300 From: Oleg Drokin To: trond.myklebust@fys.uio.no, linux-kernel@vger.kernel.org Subject: NFS & long symlinks = stack overflow Message-ID: <20040515132149.GA14880@linuxhacker.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Hello! For some time already I am investigating problems where fsstress run on NFS where NFSD is run on ext3 crashes in something like a hour. (but does not crash on reiserfs). On x86 crash looks like stack overflow (deref of pointer with last 3 digits being zero in kmap directly after get_current() call on 2.4.2x). Finally I was able to reduce a test case to two simple operations that reproduce the problem 100% reliably: [root@ranma root]# mount ranma:/testing /mnt -t nfs [root@ranma root]# cd /mnt [root@ranma mnt]# perl -e 'symlink("a"x4095, "f")'; ls -la f Segmentation fault (btw if you try to pass in something like 4090 worth of symbols, then subsequent ls won't crash, but last few symbols of link content will be corrupted) The crash happens on both 2.4 and 2.6. Below are backtraces for both (pretty much identical), but I think it is clear that stack was overflowed in some other place then the crash itself. Also fsstress crashes over time if run in UML (and again if you run nfsd from reiserfs, it does not crash, so I think this confirms the problem is related, as reiserfs imposes more strict limit on symlink length), but the crash is usually because some slab structures are corrupted and I do not yet have two commands that will reproduce the problem in uml instantly. 2.6.6-bk1 trace: Unable to handle kernel paging request at virtual address fffd1000 printing eip: f8ac41c0 *pde = 003eb067 *pte = 00000000 Oops: 0002 [#1] SMP DEBUG_PAGEALLOC CPU: 1 EIP: 0060:[] Not tainted EFLAGS: 00010202 (2.6.6) EIP is at nfs3_xdr_readlinkres+0x66/0xb1 [nfs] eax: fffd0000 ebx: 00000074 ecx: 00000ffc edx: 00000ffc esi: f56630fc edi: f56630bc ebp: f5067c88 esp: f5067c74 ds: 007b es: 007b ss: 0068 Process fsstress (pid: 2016, threadinfo=f5066000 task=f5a3fa60) Stack: 00000000 f5067e54 f8a404d4 f5067d40 f56630bc f5067cb0 f8a25b7b c1816060 f4fc8a60 00000282 c1816d20 f8ac415a f5067d40 f5663120 f5663160 f5067cdc f8a1f1d0 d91e9474 f5067e54 00000002 c036a780 f8ac415a f56630bc f5067d40 Call Trace: [] rpcauth_unwrap_resp+0x5b/0x87 [sunrpc] [] nfs3_xdr_readlinkres+0x0/0xb1 [nfs] [] call_decode+0x117/0x22e [sunrpc] [] nfs3_xdr_readlinkres+0x0/0xb1 [nfs] [] __rpc_execute+0x386/0x3fd [sunrpc] [] default_wake_function+0x0/0xc [] rpc_call_sync+0x62/0xa1 [sunrpc] [] rpc_run_timer+0x0/0xcf [sunrpc] [] nfs3_rpc_wrapper+0x2d/0x6e [nfs] [] nfs3_proc_readlink+0xa9/0xed [nfs] [] add_to_page_cache+0x5a/0x132 [] nfs_symlink_filler+0x62/0x101 [nfs] [] add_to_page_cache_lru+0x34/0x36 [] read_cache_page+0x60/0x20e [] nfs_symlink_filler+0x0/0x101 [nfs] [] nfs_getlink+0x86/0xe8 [nfs] [] nfs_readlink+0x27/0x81 [nfs] [] __user_walk+0x4b/0x4d [] sys_readlink+0x79/0x7d [] filldir64+0x0/0xe3 [] sys_close+0x7d/0xea [] sysenter_past_esp+0x52/0x71 2.4.27-pre2 trace: Unable to handle kernel paging request at virtual address ff8e8000 c01a2c64 *pde = 00005063 Oops: 0002 CPU: 0 EIP: 0010:[] Not tainted Using defaults from ksymoops -t elf32-i386 -a i386 EFLAGS: 00010206 eax: 00000ffc ebx: 00000074 ecx: ff8e7000 edx: 00000ffc esi: f5b220ac edi: 00000000 ebp: f73f5d08 esp: f73f5cf4 ds: 0018 es: 0018 ss: 0018 Process ls (pid: 1094, stackpage=f73f5000) Stack: f76a627c f73f5e84 f5b22074 f5b220cc f5b22108 f73f5d30 c02a3025 f5b22074 f76a6274 f73f5e84 c01a2bd0 f5b22074 f73f4000 f73f5d8c ffffe000 f73f5d6c c02a7ab5 f73f5d8c f73f5d8c f73f5d5c f73f5df4 00000000 f73f4000 00000000 Call Trace: [] [] [] [] [] [] [] [] [] [] [] [] [] [] Code: c6 44 08 04 00 8b 46 10 8b 08 b8 00 e0 ff ff 21 e0 8b 50 30 >>EIP; c01a2c64 <===== >>eax; 00000ffc Before first symbol >>ecx; ff8e7000 >>edx; 00000ffc Before first symbol >>esi; f5b220ac <_end+3574d200/384e51b4> >>ebp; f73f5d08 <_end+37020e5c/384e51b4> >>esp; f73f5cf4 <_end+37020e48/384e51b4> Trace; c02a3025 Trace; c01a2bd0 Trace; c02a7ab5 <__rpc_execute+f5/3c0> Trace; c02a2413 Trace; c02a6ac0 Trace; c019fd86 Trace; c01a030e Trace; c019c66c Trace; c0135937 Trace; c019c719 Trace; c019c600 Trace; c019c7b5 Trace; c015226c Trace; c01076fb Code; c01a2c64 00000000 <_EIP>: Code; c01a2c64 <===== 0: c6 44 08 04 00 movb $0x0,0x4(%eax,%ecx,1) <===== Code; c01a2c69 5: 8b 46 10 mov 0x10(%esi),%eax Code; c01a2c6c 8: 8b 08 mov (%eax),%ecx Code; c01a2c6e a: b8 00 e0 ff ff mov $0xffffe000,%eax Code; c01a2c73 f: 21 e0 and %esp,%eax Code; c01a2c75 11: 8b 50 30 mov 0x30(%eax),%edx Bye, Oleg