ocfs2 problem with nfs v2

ocfs2 problem with nfs v2

[Henrik Carlqvist]

I am trying to setup a redundant NFS server using ocfs2 on a shared disk
between two servers. So far I have only configured ocfs2 on one server,
but I have already seen a problem.

The NFS server works fine for Solaris and Linux NFS v3 clients, but old
linux boxes using NFS v2 have problems with the server.

The NFS clients which are able to reproduce the problem have Slackware 4
installed, this is what things look like on a problematic NFS v2 client:

$ uname -sr
Linux 2.2.6
$ ls ~henhi
/bin/ls: /home/aphus/22/henhi: Operation not supported on transport
endpoint 
$ cat /proc/mounts | grep aphus/22
svinapa:/export/aphus22 /home/aphus/22 nfs rw,intr,addr=svinapa 0 0

At the same time that the NFS client says "Operation not supported..." I
get the following in the logs on the NFS server:
Sep 13 15:50:20 kattapa kernel: (5385,0):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding

The NFS server has Red Hat Enterprise Linux ES release 4 installed with
kernel 2.6.9-42.ELsmp and ocfs2-2.6.9-42.ELsmp-1.2.3-1.i686.rpm was
downloaded from Oracles web.

Red Hat 4 and Slackware 4 might sound as releases that should match each
other, but Slackware 4 was released 1999. As the old 2.2.6 kernel does not
have support for NFS v3 but only NFS v2 my guess is that this might
trigger the bug. I have tried to increase rsize and wsize to 8192, but it
didn't help.

Does anybody know what causes "fh buffer is too small for encoding"?

regards Henrik

Re: ocfs2 problem with nfs v2

[Henrik Carlqvist]

Henrik Carlqvist <hc8@uthyres.com> wrote:
> As the old 2.2.6 kernel does not have support for NFS v3 but only NFS v2
> my guess is that this might trigger the bug.

Today I used a Slackware 9.1 machine to mount the NFS share with NFS v2.
Using NFS v3 in Slackware 9.1 the server works fine, but with NFS v2 I was
able to repeat the bug.

This is what it looks like on the NFS client:

# uname -sr
Linux 2.4.22
# mount -o nfsvers=2 ekorrapa:/san/old /mnt/hd/ 
# cat /proc/mounts | grep mnt/hd
ekorrapa:/san/old /mnt/hd nfs
rw,v2,rsize=8192,wsize=8192,hard,udp,lock,addr=ekorrapa 0 0 
# ls /mnt/hd/
1   11  13  15  17  19  20  3  5  7  9           mail 
10  12  14  16  18  2   21  4  6  8  lost+found
# ls -al /mnt/hd/ 
ls:/mnt/hd/lost+found: Input/output error 
ls: /mnt/hd/1: Input/output error
ls: /mnt/hd/2: Input/output error
ls: /mnt/hd/3: Input/output error
ls: /mnt/hd/4: Input/output error
ls: /mnt/hd/5: Input/output error
ls: /mnt/hd/6: Input/output error
ls: /mnt/hd/7: Input/output error
ls: /mnt/hd/8: Input/output error
ls: /mnt/hd/9: Input/output error
ls: /mnt/hd/10: Input/output error
ls: /mnt/hd/11: Input/output error
ls: /mnt/hd/12: Input/output error
ls: /mnt/hd/13: Input/output error
ls: /mnt/hd/14: Input/output error
ls: /mnt/hd/15: Input/output error
ls: /mnt/hd/16: Input/output error
ls: /mnt/hd/17: Input/output error
ls: /mnt/hd/18: Input/output error
ls: /mnt/hd/19: Input/output error
ls: /mnt/hd/20: Input/output error
ls: /mnt/hd/21: Input/output error
ls: /mnt/hd/mail: Input/output error
total 3
drwxr-xr-x   25 root     root         2048 Sep 13 10:21 . 
drwxr-xr-x    5 root     root          120 Mar 16  2002 ..
 
>From the clients point of view it looks a little bit different compared
with a 2.2.6 kernel. "ls" works, but "ls -l" does not work. With kernel
2.2.6 not even "ls" did work. I also get a slightly different error with
kernel 2.4.22 on the client, "Input/output error" instead of "Operation
not supported on transport endpoint".

However, on the NFS server things look exactly the same as before:

# uname -sr
Linux 2.6.9-42.ELsmp
# cat /proc/mounts | grep old
/dev/emcpowerb1 /san/old ocfs2 rw,noatime 0 0 
# fgrep ocfs2 /var/log/messages
Sep 14 10:53:23 kattapa kernel: (5380,0):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding 
Sep 14 10:53:31 kattapa kernel: (5390,2):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding 
Sep 14 10:53:45 kattapa kernel: (5390,1):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding 
Sep 14 10:53:47 kattapa kernel: (5390,1):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding 
Sep 14 10:53:59 kattapa kernel: (5390,1):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding 
Sep 14 10:54:02 kattapa kernel: (5390,3):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding 
Sep 14 13:21:41 kattapa kernel: (5383,0):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding 
Sep 14 13:35:17 kattapa kernel: (5380,3):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding 
Sep 14 13:35:17 kattapa kernel: (5380,0):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding
Sep 14 13:35:17 kattapa kernel: (5380,0):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding 
Sep 14 13:35:17 kattapa kernel: (5380,3):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding 
Sep 14 13:35:19 kattapa kernel: (5380,0):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding 
Sep 14 13:35:19 kattapa kernel: (5380,1):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding 
Sep 14 13:35:19 kattapa kernel: (5381,0):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding 
Sep 14 13:37:24 kattapa kernel: (5380,0):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding
Sep 14 13:40:38 kattapa kernel: (5380,2):ocfs2_encode_fh:155 ERROR: fh
buffer is too small for encoding

The good thing with this is that the bug is possible to reproduce without
having access to an NFS client running an ancient Linux distribution.
By doing:
mount -o nfsvers=2 server:/nfs/export /mount/point
It is possible to repeat this bug.

regards Henrik

Re: ocfs2 problem with nfs v2

[Henrik Carlqvist]

Henrik Carlqvist <hc8@uthyres.com> wrote:
> Using NFS v3 in Slackware 9.1 the server works fine, but with NFS v2 I
> was able to repeat the bug.
> 
> This is what it looks like on the NFS client:

> # mount -o nfsvers=2 ekorrapa:/san/old /mnt/hd/ 

> # ls -al /mnt/hd/ 
> ls:/mnt/hd/lost+found: Input/output error 
> ls: /mnt/hd/1: Input/output error
> ls: /mnt/hd/2: Input/output error

> However, on the NFS server things look exactly the same as before:
> Sep 14 10:53:23 kattapa kernel: (5380,0):ocfs2_encode_fh:155 ERROR: fh
> buffer is too small for encoding 

This message is posted only to make this thread useful by others which
encounter the same problem. The problem got into bugzilla at
http://oss.oracle.com/bugzilla/show_bug.cgi?id=777 but unfortunately
because of limitations in NFS v2 it would be very hard to make ocfs2 to
work with the default settings of an NFS v2 server.

One solution to the problem is to export with the no_subtree_check
option. This solution has some mild security implications so reading
and understanding the manpage of exportfs should be done before trying
this. With the no_subtree_check option NFS v2 clients are able to use
mounts directly from the NFS server with ocfs2.

Another workaround is to configure an "NFS proxy" which does NFS v3 mounts
from the server with ocfs2 and then reexports those mounts with NFS v2. To
be able to reexport an NFS mounted directory a user space NFS server is
needed. I have tried the old and obsolete server from
http://sourceforge.net/projects/unfs which was able to do NFS v2 reexports
of NFS v3 mounted directories from the NFS v3 server with ocfs2.

regards Henrik