From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753453AbbGaOec (ORCPT ); Fri, 31 Jul 2015 10:34:32 -0400 Received: from out02.mta.xmission.com ([166.70.13.232]:41826 "EHLO out02.mta.xmission.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750804AbbGaOe3 (ORCPT ); Fri, 31 Jul 2015 10:34:29 -0400 From: ebiederm@xmission.com (Eric W. Biederman) To: Sven Geggus Cc: linux-kernel@vger.kernel.org, trond.myklebust@primarydata.com, linux-nfs@vger.kernel.org References: <20150731114230.GA31037@geggus.net> Date: Fri, 31 Jul 2015 09:27:51 -0500 In-Reply-To: <20150731114230.GA31037@geggus.net> (Sven Geggus's message of "Fri, 31 Jul 2015 13:42:30 +0200") Message-ID: <87r3noieqg.fsf@x220.int.ebiederm.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-XM-AID: U2FsdGVkX1+riYlAy0dXkW+3OqD1CzLs24qJJBF7xiI= X-SA-Exim-Connect-IP: 97.119.22.40 X-SA-Exim-Mail-From: ebiederm@xmission.com X-Spam-Report: * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP * 0.0 TVD_RCVD_IP Message was received from an IP address * 1.2 LotsOfNums_01 BODY: Lots of long strings of numbers * 0.0 T_TM2_M_HEADER_IN_MSG BODY: No description available. * 0.8 BAYES_50 BODY: Bayes spam probability is 40 to 60% * [score: 0.5000] * -0.0 DCC_CHECK_NEGATIVE Not listed in DCC * [sa06 1397; Body=1 Fuz1=1 Fuz2=1] * 0.5 XM_Body_Dirty_Words Contains a dirty word X-Spam-DCC: XMission; sa06 1397; Body=1 Fuz1=1 Fuz2=1 X-Spam-Combo: *;Sven Geggus X-Spam-Relay-Country: X-Spam-Timing: total 337 ms - load_scoreonly_sql: 0.09 (0.0%), signal_user_changed: 5 (1.6%), b_tie_ro: 3.8 (1.1%), parse: 1.09 (0.3%), extract_message_metadata: 14 (4.3%), get_uri_detail_list: 1.87 (0.6%), tests_pri_-1000: 5 (1.6%), tests_pri_-950: 1.45 (0.4%), tests_pri_-900: 1.13 (0.3%), tests_pri_-400: 25 (7.3%), check_bayes: 23 (6.9%), b_tokenize: 7 (2.0%), b_tok_get_all: 8 (2.4%), b_comp_prob: 2.8 (0.8%), b_tok_touch_all: 3.1 (0.9%), b_finish: 0.94 (0.3%), tests_pri_0: 276 (81.9%), tests_pri_500: 4.1 (1.2%), rewrite_mail: 0.00 (0.0%) Subject: Re: nfs-root: destructive call to __detach_mounts /dev X-Spam-Flag: No X-SA-Exim-Version: 4.2.1 (built Wed, 24 Sep 2014 11:00:52 -0600) X-SA-Exim-Scanned: Yes (on in01.mta.xmission.com) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org I have added the linux-nfs list to hopefully add a wider interested audience. Sven Geggus writes: > Hello, > > I have a couple of machines running Debian GNU/Linux 8 using an NFS-ro-mounted > root filesystem. > > The systems seem to get bitten by VFS changes in some (unfortunaly > somewhat difficult to reproduce) circumstances. > > The effect is, that /dev (dectmpfs) gets unmounted for some strange reason. > > I added the following debug code to fs/namespace.c > > printk(KERN_DEBUG "%s: %s\n", __func__, dentry->d_name.name); > dump_stack(); > > And this is what I get if one of those events happen: The only way to get to d_invalidate in lookup_fast is if d_revalidate fails. Which yields two functions to look at: nfs_lookup_revalidate and nfs4_lookup_revalidate If what is being revalidated is a mount point nfs4_lookup_revalidate calls nfs_lookup_revalidate. So nfs_lookup_revalidate is the only interesting function. The last round of this was with readdir having problems and invalidating the dcache. So now apparently we are down to something weird happening in revalidate. I don't understand the what nfs_lookup_revalidate is doing particularly well. However it appears that if you enable the kconfig SUNRPC_DEBUG option there will be more relevant information. I am not certain what more to print out that SUNRPC_DEBUG won't, but I have verified that SUNRPC_DEBUG will report when a failure path is hit in nfs_lookup_revalidate and which failure path was hit (out_bad or out_error). Which should be sufficient to start narrowing down what is happening even further. I hope that helps a little bit in tracking down what is happening. Eric > __detach_mounts: dev > CPU: 1 PID: 5551 Comm: modtrack Not tainted 4.1.3-debug-00287-g0fe8050-dirty #1 > Hardware name: Dell Inc. Precision WorkStation T3400 /0TP412, BIOS A03 01/31/2008 > ffff880127183c18 ffff880127183bc8 ffffffff815605f8 ffff88012bc4fcb0 > ffff880126c16198 ffff880127183bf8 ffffffff81106cf7 000000000000000f > ffff880126c16198 ffff880127183c18 ffff880127183d00 ffff880127183c48 > Call Trace: > [] dump_stack+0x4c/0x6e > [] __detach_mounts+0x2b/0x12c > [] d_invalidate+0x9a/0xc8 > [] lookup_fast+0x1f5/0x26f > [] do_last.isra.43+0xd6/0x9fb > [] path_openat+0x1d1/0x53e > [] ? user_path_at_empty+0x63/0x93 > [] do_filp_open+0x35/0x85 > [] ? find_next_zero_bit+0x17/0x1d > [] ? __alloc_fd+0xdd/0xef > [] do_sys_open+0x146/0x1d5 > [] SyS_openat+0xf/0x11 > [] system_call_fastpath+0x12/0x6a > > Regards > > Sven > > P.S.: Kernel is vanilla 4.1.3 with aufs patches, but aufs is not related in > this problem.