Performance problems with NFS under 2.4.20

Performance problems with NFS under 2.4.20

[Lars Magne Ingebrigtsen]

Upgrading from 2.2.20, I'm seeing vastly increased network traffic,
and after poking around a bit, I find that all calls to open() on
files on NFS-mounted partitions generates one UDP packet.  Switching
on NFS debugging, and then saying

$ cat file
$ cat file

shows me this:

Jan 13 16:27:23 litos kernel: NFS: refresh_inode(b/876609548 ct=1 info=0x2)
Jan 13 16:27:23 litos kernel: nfs: read(//file, 4096@0)
Jan 13 16:27:23 litos kernel: nfs: read(//file, 4096@17)
Jan 13 16:27:23 litos kernel: nfs: flush(b/876609548)
Jan 13 16:27:23 litos kernel: NFS: dentry_delete(//file, 0)
Jan 13 16:27:24 litos kernel: NFS: refresh_inode(b/876609548 ct=1 info=0x2)
Jan 13 16:27:24 litos kernel: nfs: read(//file, 4096@0)
Jan 13 16:27:24 litos kernel: nfs: read(//file, 4096@17)
Jan 13 16:27:24 litos kernel: nfs: flush(b/876609548)
Jan 13 16:27:24 litos kernel: NFS: dentry_delete(//file, 0)

The partition is mounted with just

$ mount server:/db /db

Adding a "-o actimeo=100" makes no difference.

Is this supposed 1) to be this way, or 2) a bug, or 3) a
misconfiguration on my part?

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Performance problems with NFS under 2.4.20

[Trond Myklebust]

>>>>> " " == Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

     > Upgrading from 2.2.20, I'm seeing vastly increased network
     > traffic, and after poking around a bit, I find that all calls
     > to open() on files on NFS-mounted partitions generates one UDP
     > packet.  Switching on NFS debugging, and then saying

     > $ cat file $ cat file

     > shows me this:

     > Jan 13 16:27:23 litos kernel: NFS: refresh_inode(b/876609548
     > ct=1 info=0x2) Jan 13 16:27:23 litos kernel: nfs: read(//file,
     > 4096@0) Jan 13 16:27:23 litos kernel: nfs: read(//file,
     > 4096@17) Jan 13 16:27:23 litos kernel: nfs: flush(b/876609548)
     > Jan 13 16:27:23 litos kernel: NFS: dentry_delete(//file, 0) Jan
     > 13 16:27:24 litos kernel: NFS: refresh_inode(b/876609548 ct=1
     > info=0x2) Jan 13 16:27:24 litos kernel: nfs: read(//file,
     > 4096@0) Jan 13 16:27:24 litos kernel: nfs: read(//file,
     > 4096@17) Jan 13 16:27:24 litos kernel: nfs: flush(b/876609548)
     > Jan 13 16:27:24 litos kernel: NFS: dentry_delete(//file, 0)

     > The partition is mounted with just

     > $ mount server:/db /db

     > Adding a "-o actimeo=100" makes no difference.

     > Is this supposed 1) to be this way, or 2) a bug, or 3) a
     > misconfiguration on my part?

That is quite deliberate.

open() is supposed to generate an RPC call in order to ensure that
cached attributes (and hence cached data) are still valid (this is
part of what is known as NFS 'close-to-open' cache consistency).

If you are certain that you will never access the same file/directory
from 2 different machines, you can try to mount with the 'nocto' mount
option.

Cheers,
  Trond

Re: Performance problems with NFS under 2.4.20

[Lars Magne Ingebrigtsen]

Trond Myklebust <trond.myklebust@fys.uio.no> writes:

> That is quite deliberate.
>
> open() is supposed to generate an RPC call in order to ensure that
> cached attributes (and hence cached data) are still valid (this is
> part of what is known as NFS 'close-to-open' cache consistency).

Ah, right.  

> If you are certain that you will never access the same file/directory
> from 2 different machines, you can try to mount with the 'nocto' mount
> option.

Thanks; "notco" fixes the problem.  

I have several machines that reads the same files/directories, but
only one machine that writes to the directories.  Will that be OK?

(The reason I noticed this at all is that out PHP-based web servers
started generating much internal network traffic after the upgrade.
The PHP directories are NFS-mounted, and due to the number of PHP
library files opened by each web access, the NFS traffic was about 10
times as high as the HTTP traffic.  :-/)

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Performance problems with NFS under 2.4.20

[Trond Myklebust]

>>>>> " " == Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

     > I have several machines that reads the same files/directories,
     > but only one machine that writes to the directories.  Will that
     > be OK?

If you require your data cache to be guaranteed to be consistent you
still have to use some form of locking to ensure that nobody tries to
read a file that is in the process of being updated on the server.
If so, close-to-open helps by ensuring that you can safely rely on
more lightweight locking protocols such as that provided by the
"lockfile" utility (a.k.a. dotlocking) instead of the full NFS Lock
Manager.

Cheers,
  Trond