RE: mounting cifs filesystems not possible

RE: mounting cifs filesystems not possible

[Gerhard Hintermayer]

I discovered that this is a slow_path issue, which is tainting the kernel,
I already filed this as a gentoo bug, but since this is kernel related, I
think here's the right place.  The server I'm trying to mount is a Windows
2003 Server, I  can inquire more if neccessary.
PS: please CC me, I not on the list anymore (too much traffic, just how I
was warned)

Gerhard

Re: mounting cifs filesystems not possible

[Gerhard Hintermayer]

Well this is indeed strange. First I thought the problem was solved
because I had an old version of mount installed, but the successful
mount with latest version of mount.cifs was only by chance. I get 1
successful mount out of ~ 30 tries (each time unloading the cifs
module). And looking at the source in export.c I discovered, that
reexporting cifs- mount points is still not supported :-( : This seems
to be very tricky. If you need more information to track the tainting
problem down, pls give me a note.


PS: please CC gerhard.hintermayer@gmail.com, I'm not on the list

On Thu, Jul 23, 2009 at 1:13 AM, Andrew Morton<akpm@linux-foundation.org> wrote:
> On Fri, 17 Jul 2009 10:22:42 +0200 (CEST)
> "Gerhard Hintermayer" <g.hintermayer@inode.at> wrote:
>
>> I discovered that this is a slow_path issue, which is tainting the kernel,
>> I already filed this as a gentoo bug, but since this is kernel related, I
>> think here's the right place.  The server I'm trying to mount is a Windows
>> 2003 Server, I  can inquire more if neccessary.
>> PS: please CC me, I not on the list anymore (too much traffic, just how I
>> was warned)
>>
>
> please do cc the CIFS maintainer and list on future issues:
>
> COMMON INTERNET FILE SYSTEM (CIFS)
> P:      Steve French
> M:      sfrench@samba.org
> L:      linux-cifs-client@lists.samba.org
> L:      samba-technical@lists.samba.org
> W:      http://linux-cifs.samba.org/
> T:      git git://git.kernel.org/pub/scm/linux/kernel/git/sfrench/cifs-2.6.git
> S:      Supported
> F:      Documentation/filesystems/cifs.txt
> F:      fs/cifs/
>
> Thanks.
>

Re: mounting cifs filesystems not possible

[Steve French]

On Thu, Jul 23, 2009 at 12:26 AM, Gerhard
Hintermayer<gerhard.hintermayer@gmail.com> wrote:
> Well this is indeed strange. First I thought the problem was solved
> because I had an old version of mount installed, but the successful
> mount with latest version of mount.cifs was only by chance. I get 1
> successful mount out of ~ 30 tries (each time unloading the cifs
> module). And looking at the source in export.c I discovered, that
> reexporting cifs- mount points is still not supported :-( : This seems
> to be very tricky. If you need more information to track the tainting
> problem down, pls give me a note.

What is kernel version?
What is symptom?

Have you opened a bug against the project bugzilla (bugzilla.samba.org)?

The dmesg (Linux message log) will show certain serious errors, but
enabling additional cifs debugging flags can increase the debug
output. Setting /proc/fs/cifs/DebugData to 1 will cause additional
cifs informational messages to be logged, and setting
it to 3 (“echo 3 > /proc/fs/cifs/cifsFYI”) will also log return codes
from most cifs entry points to dmesg.
One of the most powerful tools for analyzing networking or network
filesystem problems is “Wireshark."
http://wiki.samba.org/index.php/Capture_Packets shows instructions how
to do wireshark captures.

-- 
Thanks,

Steve

Re: mounting cifs filesystems not possible

[Gerhard Hintermayer]

See my original post to LKML at http://lkml.org/lkml/2009/7/15/217.
You can see kernel version, and dmesg output (including cifsFYI set to
7) there.
I did file a bug report to gentoo bug list and lkml. Shall I also do
so for bugzilla.samba.org ? I think it's more kernel related.

The DebugData of the share I'm trying to mount (fortunaltely I got a
successful mount after several tries):

brklev2b ~ # cat /proc/fs/cifs/DebugData
Display Internal CIFS Data Structures for Debugging
---------------------------------------------------
CIFS Version 1.57
Active VFS Requests: 0
Servers:
1) Name: 10.5.12.105  Domain: EUROPE Uses: 1 OS: Windows Server 2003
R2 3790 Service Pack 1
        NOS: Windows Server 2003 R2 5.2 Capability: 0x1f3fd
        SMB session status: 1   TCP status: 1
        Local Users To Server: 1 SecMode: 0x3 Req On Wire: 0
        Shares:
        1) \\BRKVN05\AtlasPM Mounts: 1 Type: NTFS DevInfo: 0x20
Attributes: 0x700ff
PathComponentMax: 255 Status: 0x1 type: DISK    DISCONNECTED

        MIDs:


Gerhard

On Thu, Jul 23, 2009 at 8:10 AM, Steve French<smfrench@gmail.com> wrote:
> On Thu, Jul 23, 2009 at 12:26 AM, Gerhard
> Hintermayer<gerhard.hintermayer@gmail.com> wrote:
>> Well this is indeed strange. First I thought the problem was solved
>> because I had an old version of mount installed, but the successful
>> mount with latest version of mount.cifs was only by chance. I get 1
>> successful mount out of ~ 30 tries (each time unloading the cifs
>> module). And looking at the source in export.c I discovered, that
>> reexporting cifs- mount points is still not supported :-( : This seems
>> to be very tricky. If you need more information to track the tainting
>> problem down, pls give me a note.
>
> What is kernel version?
> What is symptom?
>
> Have you opened a bug against the project bugzilla (bugzilla.samba.org)?
>
> The dmesg (Linux message log) will show certain serious errors, but
> enabling additional cifs debugging flags can increase the debug
> output. Setting /proc/fs/cifs/DebugData to 1 will cause additional
> cifs informational messages to be logged, and setting
> it to 3 (“echo 3 > /proc/fs/cifs/cifsFYI”) will also log return codes
> from most cifs entry points to dmesg.
> One of the most powerful tools for analyzing networking or network
> filesystem problems is “Wireshark."
> http://wiki.samba.org/index.php/Capture_Packets shows instructions how
> to do wireshark captures.
>
> --
> Thanks,
>
> Steve
>

Re: mounting cifs filesystems not possible

[Steve French]

On Thu, Jul 23, 2009 at 1:24 AM, Gerhard
Hintermayer<gerhard.hintermayer@gmail.com> wrote:
> See my original post to LKML at http://lkml.org/lkml/2009/7/15/217.
> You can see kernel version, and dmesg output (including cifsFYI set to
> 7) there.
> I did file a bug report to gentoo bug list and lkml. Shall I also do
> so for bugzilla.samba.org ? I think it's more kernel related.
>
> The DebugData of the share I'm trying to mount (fortunaltely I got a

> mount -t cifs -o user=aprol,password=*****,ip=10.5.12.105 //BRKVN05/AtlasPM /mnt/AtlasPM

The mount syntax you listed in your post is a little strange with the
mount options preceding rather than following the UNC name and device,
ie rather than the usual:
              mount.cifs {//unc-name} {mount-point} [-o options]

It looks like the first 3/4 of mount succeeded (setting up a session,
and connecting to the share) to the point of doing the lookup of the
root directory of the share (which I don't see in your trace).
Typically there have been the following reasons for that to fail:
1) the user has no permission on the root of the share
or
2) the root of the share is a dfs referral to another server (this
should work now, but was probably not supported a few kernel releases
ago)
or
3) the server reports that the root directory is not a directory (ie
is a file for example)

It may be that an earlier mount failed perhaps oopsing in the vfs, and
that could have left an existing cifs session and tree connection
which made it seem like a later mount worked (since it found an
existing session) but that seems unlikely.

It would be helpful to see if the client did a network request (in
this case SMB transact2 Query Path Info) on the root directory of the
share, so if you have a wireshark trace that would be useful.
-- 
Thanks,

Steve

Re: mounting cifs filesystems not possible

[Gerhard Hintermayer]

Sorry, wireshark not installed.
The strange thing is, that sometimes mounting succeeds. So it
should'nt be an issue with the root share (unless the server doesn't
always answer in the same way)
Here's a dmesg output with a successful mount. (had to try for several
times, unloading/loading of module cifs is not necessary)
The only diffs which might be the cause are different flags (64 vs 0
when unable to mount) in line 1 and a different Xid in line 2.

 fs/cifs/cifsfs.c: Devname: //BRKVN05/AtlasPM flags: 64
 fs/cifs/connect.c: CIFS VFS: in cifs_mount as Xid: 97 with uid: 0
 fs/cifs/connect.c: Username: aprol
 fs/cifs/connect.c: UNC: \\BRKVN05\AtlasPM ip: 10.5.12.105
 fs/cifs/connect.c: Socket created
 fs/cifs/connect.c: sndbuf 16384 rcvbuf 87380 rcvtimeo 0x2bc
 fs/cifs/connect.c: Existing smb sess not found
 fs/cifs/cifssmb.c: secFlags 0x7
 fs/cifs/transport.c: For smb_command 114
 fs/cifs/transport.c: Sending smb:  total_len 51
 fs/cifs/connect.c: Demultiplex PID: 31969
 fs/cifs/connect.c: rfc1002 length 0x6f
 fs/cifs/cifssmb.c: Dialect: 0
 fs/cifs/cifssmb.c: negprot rc 0
 fs/cifs/connect.c: Security Mode: 0x3 Capabilities: 0x1f3fd TimeAdjust: -7200
 fs/cifs/sess.c: sess setup type 2
 fs/cifs/transport.c: For smb_command 115
 fs/cifs/transport.c: Sending smb:  total_len 242
 fs/cifs/connect.c: rfc1002 length 0xc7
 fs/cifs/misc.c: Null buffer passed to cifs_small_buf_release
 fs/cifs/sess.c: ssetup rc from sendrecv2 is 0
 fs/cifs/sess.c: UID = 20482
 fs/cifs/sess.c: bleft 153
 fs/cifs/sess.c: words left: 0
 fs/cifs/sess.c: ssetup freeing small buf f031ae40
 fs/cifs/connect.c: CIFS Session Established successfully
 fs/cifs/connect.c: file mode: 0x5f7  dir mode: 0x1ff
 fs/cifs/transport.c: For smb_command 117
 fs/cifs/transport.c: Sending smb:  total_len 90
 fs/cifs/connect.c: rfc1002 length 0x42
 fs/cifs/connect.c: disk share connection
 fs/cifs/connect.c: Tcon flags: 0x1
 fs/cifs/connect.c: CIFS Tcon rc = 0
 fs/cifs/cifssmb.c: In QFSDeviceInfo
 fs/cifs/transport.c: For smb_command 50
 fs/cifs/transport.c: Sending smb:  total_len 72
 fs/cifs/connect.c: rfc1002 length 0x44
 fs/cifs/cifssmb.c: In QFSAttributeInfo
 fs/cifs/transport.c: For smb_command 50
 fs/cifs/transport.c: Sending smb:  total_len 72
 fs/cifs/connect.c: rfc1002 length 0x50
 fs/cifs/connect.c: CIFS VFS: leaving cifs_mount (xid = 97) rc = 0
 fs/cifs/inode.c: CIFS VFS: in cifs_iget as Xid: 98 with uid: 0
 fs/cifs/inode.c: Getting info on
 fs/cifs/transport.c: For smb_command 50
 fs/cifs/transport.c: Sending smb:  total_len 78
 fs/cifs/connect.c: rfc1002 length 0xb8
 fs/cifs/inode.c: Old time 0
 fs/cifs/inode.c: New time 67264957

Gerhard

On Thu, Jul 23, 2009 at 9:44 AM, Steve French<smfrench@gmail.com> wrote:
> On Thu, Jul 23, 2009 at 1:24 AM, Gerhard
> Hintermayer<gerhard.hintermayer@gmail.com> wrote:
>> See my original post to LKML at http://lkml.org/lkml/2009/7/15/217.
>> You can see kernel version, and dmesg output (including cifsFYI set to
>> 7) there.
>> I did file a bug report to gentoo bug list and lkml. Shall I also do
>> so for bugzilla.samba.org ? I think it's more kernel related.
>>
>> The DebugData of the share I'm trying to mount (fortunaltely I got a
>
>> mount -t cifs -o user=aprol,password=*****,ip=10.5.12.105 //BRKVN05/AtlasPM /mnt/AtlasPM
>
> The mount syntax you listed in your post is a little strange with the
> mount options preceding rather than following the UNC name and device,
> ie rather than the usual:
>              mount.cifs {//unc-name} {mount-point} [-o options]
>
> It looks like the first 3/4 of mount succeeded (setting up a session,
> and connecting to the share) to the point of doing the lookup of the
> root directory of the share (which I don't see in your trace).
> Typically there have been the following reasons for that to fail:
> 1) the user has no permission on the root of the share
> or
> 2) the root of the share is a dfs referral to another server (this
> should work now, but was probably not supported a few kernel releases
> ago)
> or
> 3) the server reports that the root directory is not a directory (ie
> is a file for example)
>
> It may be that an earlier mount failed perhaps oopsing in the vfs, and
> that could have left an existing cifs session and tree connection
> which made it seem like a later mount worked (since it found an
> existing session) but that seems unlikely.
>
> It would be helpful to see if the client did a network request (in
> this case SMB transact2 Query Path Info) on the root directory of the
> share, so if you have a wireshark trace that would be useful.
> --
> Thanks,
>
> Steve
>

Re: mounting cifs filesystems not possible

[Steve French]

On Thu, Jul 23, 2009 at 3:08 AM, Gerhard
Hintermayer<gerhard.hintermayer@gmail.com> wrote:
> Sorry, wireshark not installed.
> The strange thing is, that sometimes mounting succeeds. So it
> should'nt be an issue with the root share (unless the server doesn't
> always answer in the same way)

I am looking for the equivalent lines in dmesg in the failing trace to
those that
follow (in your success trace) the line:

fs/cifs/inode.c: CIFS VFS: in cifs_iget

If we do the QFSAttributeInfo (which is optional), the next (and
final) place to fail
is getting information on the root inode


-- 
Thanks,

Steve

mounting cifs filesystems not possible

[Gerhard Hintermayer]

while trying to mount a  cifs filesystem on a gentoo system:

Linux brklev2b 2.6.29-gentoo-r5 #1 SMP Wed Jul 15 14:46:51 CEST 2009 i686
Intel(R) Xeon(R) CPU 5130 @ 2.00GHz GenuineIntel GNU/Linux

via mount command:

mount -t cifs -o user=aprol,password=*****,
ip=10.5.12.105 //BRKVN05/AtlasPM /mnt/AtlasPM

I get the error : "mount: Not a directory" even though the mountpoint
exists and is a valid directory

Dmesg says:

fs/cifs/cifsfs.c: Devname: //BRKVN05/AtlasPM flags: 0
 fs/cifs/connect.c: CIFS VFS: in cifs_mount as Xid: 3 with uid: 0
 fs/cifs/connect.c: Username: aprol
 fs/cifs/connect.c: UNC: \\BRKVN05\AtlasPM ip: 10.5.12.105
 fs/cifs/connect.c: Socket created
 fs/cifs/connect.c: sndbuf 16384 rcvbuf 87380 rcvtimeo 0x2bc
 fs/cifs/connect.c: Existing smb sess not found
 fs/cifs/cifssmb.c: secFlags 0x7
 fs/cifs/transport.c: For smb_command 114
 fs/cifs/transport.c: Sending smb:  total_len 51
 fs/cifs/connect.c: Demultiplex PID: 5083
 fs/cifs/connect.c: rfc1002 length 0x6f
 fs/cifs/cifssmb.c: Dialect: 0
 fs/cifs/cifssmb.c: negprot rc 0
 fs/cifs/connect.c: Security Mode: 0x3 Capabilities: 0x1f3fd TimeAdjust:
-7200
 fs/cifs/sess.c: sess setup type 2
 fs/cifs/transport.c: For smb_command 115
 fs/cifs/transport.c: Sending smb:  total_len 242
 fs/cifs/connect.c: rfc1002 length 0xc7
 fs/cifs/misc.c: Null buffer passed to cifs_small_buf_release
 fs/cifs/sess.c: ssetup rc from sendrecv2 is 0
 fs/cifs/sess.c: UID = 14336
 fs/cifs/sess.c: bleft 153
 fs/cifs/sess.c: words left: 0
 fs/cifs/sess.c: ssetup freeing small buf edca9740
 fs/cifs/connect.c: CIFS Session Established successfully
 fs/cifs/connect.c: file mode: 0x5f7  dir mode: 0x1ff
 fs/cifs/transport.c: For smb_command 117
 fs/cifs/transport.c: Sending smb:  total_len 90
 fs/cifs/connect.c: rfc1002 length 0x42
 fs/cifs/connect.c: disk share connection
 fs/cifs/connect.c: Tcon flags: 0x1
 fs/cifs/connect.c: CIFS Tcon rc = 0
 fs/cifs/cifssmb.c: In QFSDeviceInfo
 fs/cifs/transport.c: For smb_command 50
 fs/cifs/transport.c: Sending smb:  total_len 72
 fs/cifs/connect.c: rfc1002 length 0x44
 fs/cifs/cifssmb.c: In QFSAttributeInfo
 fs/cifs/transport.c: For smb_command 50
 fs/cifs/transport.c: Sending smb:  total_len 72
 fs/cifs/connect.c: rfc1002 length 0x50
 fs/cifs/connect.c: CIFS VFS: leaving cifs_mount (xid = 3) rc = 0
------------[ cut here ]------------
WARNING: at fs/inode.c:362 invalidate_inodes+0xa6/0xe1()
Hardware name: PowerEdge 2950
Modules linked in: cifs nfsd exportfs ncpfs bonding floppy e1000e bnx2
thermal button processor thermal_sys e1000 nfs lockd sunrpc usbhid sg
scsi_wait_scan [last unloaded: cifs]
Pid: 5082, comm: mount Tainted: G        W  2.6.29-gentoo-r5 #1
Call Trace:
 [<c011d251>] warn_slowpath+0x71/0xa8
 [<c01158f6>] ? __wake_up+0x31/0x3b
 [<f84bc7b0>] ? cifs_init_once+0xe/0x16 [cifs]
 [<c01587e3>] ? cache_alloc_refill+0x62/0x434
 [<c0170283>] ? generic_sync_sb_inodes+0x395/0x3a5
 [<c01301de>] ? lowest_in_progress+0x43/0x4b
 [<c016982a>] invalidate_inodes+0xa6/0xe1
 [<c015c96b>] generic_shutdown_super+0x3c/0xda
 [<c015ca4c>] kill_anon_super+0x12/0x31
 [<c015cabc>] deactivate_super+0x35/0x47
 [<c016c05e>] mntput_no_expire+0xc6/0xe6
 [<c016c91d>] do_add_mount+0xd0/0xd8
 [<c016d511>] do_mount+0x631/0x654
 [<c0142138>] ? __get_free_pages+0x1e/0x24
 [<c016bc53>] ? copy_mount_options+0x2a/0xf9
 [<c016d59e>] sys_mount+0x6a/0xa8
 [<c0102cc5>] sysenter_do_call+0x12/0x25
---[ end trace ea473e21e738f567 ]---
 fs/cifs/cifsfs.c: In cifs_put_super
 fs/cifs/connect.c: CIFS VFS: in cifs_put_tcon as Xid: 4 with uid: 0
 fs/cifs/cifssmb.c: In tree disconnect
 fs/cifs/transport.c: For smb_command 113
 fs/cifs/transport.c: Sending smb:  total_len 39
 fs/cifs/connect.c: rfc1002 length 0x27
 fs/cifs/connect.c: CIFS VFS: in cifs_put_smb_ses as Xid: 5 with uid: 0
 fs/cifs/cifssmb.c: In SMBLogoff for session disconnect
 fs/cifs/transport.c: For smb_command 116
 fs/cifs/transport.c: Sending smb:  total_len 43
 fs/cifs/connect.c: rfc1002 length 0x2b

BTW -CIFS_EXPERIMENTAL is not turned on. Tried to remove that as a first
step, but I need reexporting the directory, so will have to activate it
anyhow.

any ideas what might go wrong. (mounting under an older kernel 2.6.19
worked like a charm)
thanks

Gerhard