[PATCH 0/6] nfsd: add new export options for controlling read/write iomode

[PATCH 0/6] nfsd: add new export options for controlling read/write iomode

[Jeff Layton]

When we originally added support for the dontcache and direct iomodes to
nfsd, we added global debugfs switches to control them. That was fine for
experimentation but the goal was always to move these controls to
something more permanent.

This patch series adds a new set of riomode= and wiomode= export
options. With this, we can set the read and write iomode on a per-export
basis to either buffered, direct or dontcache.

The debugfs knobs stick around and govern the default iomode when it's
not set on the export. Eventually these controls should also be moved to
something more permanent (or maybe just be removed), but I've left that
as a future change.

The last patch adds new documentation that attempts to give the admin
some guidance on choosing r/wiomode settings.

A companion nfs-utils patchset will follow that adds support for these
options to exportfs/mountd, and adds a new script to help users choose
iomode settings.

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
Jeff Layton (6):
      NFSD: Add per-export read/write I/O mode settings
      NFSD: Apply per-export I/O modes to READ and WRITE
      NFSD: Document per-export riomode= and wiomode= options
      NFSD: Keep per-export statistics across export cache updates
      NFSD: Count the storage reads of each export's READs
      NFSD: Document how to choose an IO mode

 Documentation/filesystems/nfs/nfsd-io-modes.rst | 194 +++++++++++++++++
 Documentation/netlink/specs/nfsd.yaml           |  24 +++
 fs/nfsd/debugfs.c                               |   6 +-
 fs/nfsd/export.c                                | 268 +++++++++++++++++++++---
 fs/nfsd/export.h                                |  19 ++
 fs/nfsd/netlink.c                               |   4 +-
 fs/nfsd/netlink.h                               |   2 +-
 fs/nfsd/netns.h                                 |   6 +
 fs/nfsd/nfs4xdr.c                               |   6 +-
 fs/nfsd/nfsd.h                                  |  17 +-
 fs/nfsd/stats.h                                 |  33 +++
 fs/nfsd/vfs.c                                   |  59 +++++-
 fs/nfsd/vfs.h                                   |   2 +
 include/uapi/linux/nfsd_netlink.h               |  12 ++
 14 files changed, 599 insertions(+), 53 deletions(-)
---
base-commit: fa65fd6d503d4e965ae27b1a75969e097569390a
change-id: 20261013-export-iomode-1b56e9d89821

Best regards,
-- 
Jeff Layton <jlayton@kernel.org>

[PATCH 1/6] NFSD: Add per-export read/write I/O mode settings

[Jeff Layton]

Add ex_riomode/ex_wiomode to struct svc_export along with a
kernel-internal NFSD_IO_DEFAULT value meaning "follow the global debugfs
knob".

Add an io-mode enum to the nfsd netlink spec so the mode values are
uAPI (NFSD_IO_MODE_*), and define NFSD_IO_BUFFERED/DONTCACHE/DIRECT in
terms of it. The generated NLA_POLICY_MAX policy range-checks the new
read-iomode/write-iomode attributes.

Parse those attributes, copy the settings on cache update, and report
them in nfsd.export/content. The new options are netlink-only.

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 Documentation/netlink/specs/nfsd.yaml | 24 ++++++++++++++++++++++++
 fs/nfsd/export.c                      | 28 ++++++++++++++++++++++++++--
 fs/nfsd/export.h                      |  2 ++
 fs/nfsd/netlink.c                     |  4 +++-
 fs/nfsd/netlink.h                     |  2 +-
 fs/nfsd/nfsd.h                        | 17 +++++++++++++----
 include/uapi/linux/nfsd_netlink.h     | 12 ++++++++++++
 7 files changed, 81 insertions(+), 8 deletions(-)

diff --git a/Documentation/netlink/specs/nfsd.yaml b/Documentation/netlink/specs/nfsd.yaml
index a4991206b45e..eb42ba65619d 100644
--- a/Documentation/netlink/specs/nfsd.yaml
+++ b/Documentation/netlink/specs/nfsd.yaml
@@ -56,6 +56,16 @@ definitions:
     name: nfsclnt-idmax
     value: 1024
     header: linux/nfsd/export.h
+  -
+    type: enum
+    name: io-mode
+    doc: >-
+      NFS READ/WRITE I/O mode. These values match the global io_cache_read
+      and io_cache_write debugfs settings.
+    entries:
+      - buffered
+      - dontcache
+      - direct
 
 attribute-sets:
   -
@@ -302,6 +312,20 @@ attribute-sets:
       -
         name: fsid
         type: s32
+      -
+        name: read-iomode
+        type: u32
+        enum: io-mode
+        doc: >-
+          Per-export NFS READ I/O mode. When absent, the export follows
+          the global io_cache_read setting.
+      -
+        name: write-iomode
+        type: u32
+        enum: io-mode
+        doc: >-
+          Per-export NFS WRITE I/O mode. When absent, the export follows
+          the global io_cache_write setting.
   -
     name: svc-export-reqs
     attributes:
diff --git a/fs/nfsd/export.c b/fs/nfsd/export.c
index 265ea8fd31c6..c71350965789 100644
--- a/fs/nfsd/export.c
+++ b/fs/nfsd/export.c
@@ -838,14 +838,14 @@ static int check_export(const struct path *path, int *flags,
 static int nfsd_nl_parse_one_export(struct cache_detail *cd,
 				    struct nlattr *attr)
 {
-	struct nlattr *tb[NFSD_A_SVC_EXPORT_FSID + 1];
+	struct nlattr *tb[NFSD_A_SVC_EXPORT_WRITE_IOMODE + 1];
 	struct auth_domain *dom = NULL;
 	struct svc_export exp = {}, *expp;
 	struct nlattr *secinfo_attr;
 	struct timespec64 boot;
 	int err, rem;
 
-	err = nla_parse_nested(tb, NFSD_A_SVC_EXPORT_FSID, attr,
+	err = nla_parse_nested(tb, NFSD_A_SVC_EXPORT_WRITE_IOMODE, attr,
 			       nfsd_svc_export_nl_policy, NULL);
 	if (err)
 		return err;
@@ -873,6 +873,8 @@ static int nfsd_nl_parse_one_export(struct cache_detail *cd,
 	exp.cd = cd;
 	exp.ex_devid_map = NULL;
 	exp.ex_xprtsec_modes = NFSEXP_XPRTSEC_ALL;
+	exp.ex_riomode = NFSD_IO_DEFAULT;
+	exp.ex_wiomode = NFSD_IO_DEFAULT;
 
 	/* expiry (required, wallclock seconds) */
 	if (!tb[NFSD_A_SVC_EXPORT_EXPIRY]) {
@@ -986,6 +988,12 @@ static int nfsd_nl_parse_one_export(struct cache_detail *cd,
 			}
 		}
 
+		/* read-iomode / write-iomode (optional u32) */
+		if (tb[NFSD_A_SVC_EXPORT_READ_IOMODE])
+			exp.ex_riomode = nla_get_u32(tb[NFSD_A_SVC_EXPORT_READ_IOMODE]);
+		if (tb[NFSD_A_SVC_EXPORT_WRITE_IOMODE])
+			exp.ex_wiomode = nla_get_u32(tb[NFSD_A_SVC_EXPORT_WRITE_IOMODE]);
+
 		err = check_export(&exp.ex_path, &exp.ex_flags,
 				   exp.ex_uuid);
 		if (err)
@@ -1342,6 +1350,8 @@ static int svc_export_parse(struct cache_detail *cd, char *mesg, int mlen)
 	exp.cd = cd;
 	exp.ex_devid_map = NULL;
 	exp.ex_xprtsec_modes = NFSEXP_XPRTSEC_ALL;
+	exp.ex_riomode = NFSD_IO_DEFAULT;
+	exp.ex_wiomode = NFSD_IO_DEFAULT;
 
 	/* expiry */
 	err = get_expiry(&mesg, &exp.h.expiry_time);
@@ -1462,6 +1472,12 @@ static int is_export_stats_file(struct seq_file *m)
 	return !strcmp(m->file->f_path.dentry->d_name.name, "export_stats");
 }
 
+static const char * const nfsd_iomode_names[] = {
+	[NFSD_IO_BUFFERED]	= "buffered",
+	[NFSD_IO_DONTCACHE]	= "dontcache",
+	[NFSD_IO_DIRECT]	= "direct",
+};
+
 static int svc_export_show(struct seq_file *m,
 			   struct cache_detail *cd,
 			   struct cache_head *h)
@@ -1498,6 +1514,12 @@ static int svc_export_show(struct seq_file *m,
 	    !test_bit(CACHE_NEGATIVE, &h->flags)) {
 		exp_flags(m, exp->ex_flags, exp->ex_fsid,
 			  exp->ex_anon_uid, exp->ex_anon_gid, &exp->ex_fslocs);
+		if (exp->ex_riomode != NFSD_IO_DEFAULT)
+			seq_printf(m, ",riomode=%s",
+				   nfsd_iomode_names[exp->ex_riomode]);
+		if (exp->ex_wiomode != NFSD_IO_DEFAULT)
+			seq_printf(m, ",wiomode=%s",
+				   nfsd_iomode_names[exp->ex_wiomode]);
 		if (exp->ex_uuid) {
 			int i;
 			seq_puts(m, ",uuid=");
@@ -1564,6 +1586,8 @@ static void export_update(struct cache_head *cnew, struct cache_head *citem)
 		new->ex_flavors[i] = item->ex_flavors[i];
 	}
 	new->ex_xprtsec_modes = item->ex_xprtsec_modes;
+	new->ex_riomode = item->ex_riomode;
+	new->ex_wiomode = item->ex_wiomode;
 }
 
 static struct cache_head *svc_export_alloc(void)
diff --git a/fs/nfsd/export.h b/fs/nfsd/export.h
index 117fb28db1e0..0e832990a12c 100644
--- a/fs/nfsd/export.h
+++ b/fs/nfsd/export.h
@@ -77,6 +77,8 @@ struct svc_export {
 	struct cache_detail	*cd;
 	struct rcu_head		ex_rcu;
 	unsigned long		ex_xprtsec_modes;
+	s8			ex_riomode;
+	s8			ex_wiomode;
 	struct export_stats	*ex_stats;
 };
 
diff --git a/fs/nfsd/netlink.c b/fs/nfsd/netlink.c
index 4ea6a7c1ce7f..5c35764e62d1 100644
--- a/fs/nfsd/netlink.c
+++ b/fs/nfsd/netlink.c
@@ -42,7 +42,7 @@ const struct nla_policy nfsd_sock_nl_policy[NFSD_A_SOCK_TRANSPORT_NAME + 1] = {
 	[NFSD_A_SOCK_TRANSPORT_NAME] = { .type = NLA_NUL_STRING, },
 };
 
-const struct nla_policy nfsd_svc_export_nl_policy[NFSD_A_SVC_EXPORT_FSID + 1] = {
+const struct nla_policy nfsd_svc_export_nl_policy[NFSD_A_SVC_EXPORT_WRITE_IOMODE + 1] = {
 	[NFSD_A_SVC_EXPORT_SEQNO] = { .type = NLA_U64, },
 	[NFSD_A_SVC_EXPORT_CLIENT] = { .type = NLA_NUL_STRING, },
 	[NFSD_A_SVC_EXPORT_PATH] = { .type = NLA_NUL_STRING, },
@@ -56,6 +56,8 @@ const struct nla_policy nfsd_svc_export_nl_policy[NFSD_A_SVC_EXPORT_FSID + 1] =
 	[NFSD_A_SVC_EXPORT_XPRTSEC] = NLA_POLICY_MASK(NLA_U32, 0x7),
 	[NFSD_A_SVC_EXPORT_FLAGS] = NLA_POLICY_MASK(NLA_U32, 0x3ffff),
 	[NFSD_A_SVC_EXPORT_FSID] = { .type = NLA_S32, },
+	[NFSD_A_SVC_EXPORT_READ_IOMODE] = NLA_POLICY_MAX(NLA_U32, 2),
+	[NFSD_A_SVC_EXPORT_WRITE_IOMODE] = NLA_POLICY_MAX(NLA_U32, 2),
 };
 
 const struct nla_policy nfsd_version_nl_policy[NFSD_A_VERSION_ENABLED + 1] = {
diff --git a/fs/nfsd/netlink.h b/fs/nfsd/netlink.h
index 57fc86c63b1d..db00e06de15c 100644
--- a/fs/nfsd/netlink.h
+++ b/fs/nfsd/netlink.h
@@ -19,7 +19,7 @@ extern const struct nla_policy nfsd_expkey_nl_policy[NFSD_A_EXPKEY_PATH + 1];
 extern const struct nla_policy nfsd_fslocation_nl_policy[NFSD_A_FSLOCATION_PATH + 1];
 extern const struct nla_policy nfsd_fslocations_nl_policy[NFSD_A_FSLOCATIONS_LOCATION + 1];
 extern const struct nla_policy nfsd_sock_nl_policy[NFSD_A_SOCK_TRANSPORT_NAME + 1];
-extern const struct nla_policy nfsd_svc_export_nl_policy[NFSD_A_SVC_EXPORT_FSID + 1];
+extern const struct nla_policy nfsd_svc_export_nl_policy[NFSD_A_SVC_EXPORT_WRITE_IOMODE + 1];
 extern const struct nla_policy nfsd_version_nl_policy[NFSD_A_VERSION_ENABLED + 1];
 
 int nfsd_nl_rpc_status_get_dumpit(struct sk_buff *skb,
diff --git a/fs/nfsd/nfsd.h b/fs/nfsd/nfsd.h
index b520b51692b0..8b4bed37b46b 100644
--- a/fs/nfsd/nfsd.h
+++ b/fs/nfsd/nfsd.h
@@ -16,6 +16,7 @@
 #include <linux/sunrpc/svc_xprt.h>
 
 #include <uapi/linux/nfsd/debug.h>
+#include <uapi/linux/nfsd_netlink.h>
 
 #undef ifdebug
 #ifdef CONFIG_SUNRPC_DEBUG
@@ -134,10 +135,18 @@ extern bool nfsd_disable_splice_read __read_mostly;
 extern bool nfsd_delegts_enabled __read_mostly;
 
 enum {
-	/* Any new NFSD_IO enum value must be added at the end */
-	NFSD_IO_BUFFERED,
-	NFSD_IO_DONTCACHE,
-	NFSD_IO_DIRECT,
+	/* Values are uAPI: per-export modes are sent over netlink */
+	NFSD_IO_BUFFERED	= NFSD_IO_MODE_BUFFERED,
+	NFSD_IO_DONTCACHE	= NFSD_IO_MODE_DONTCACHE,
+	NFSD_IO_DIRECT		= NFSD_IO_MODE_DIRECT,
+	/*
+	 * Kernel-internal only: a per-export mode of NFSD_IO_DEFAULT
+	 * means "follow the global nfsd_io_cache_read/write knob".
+	 * It is never sent across the kernel-userspace interface.
+	 * It is -1 so that it stays distinct from all present and
+	 * future mode values.
+	 */
+	NFSD_IO_DEFAULT = -1,
 };
 
 extern u64 nfsd_io_cache_read __read_mostly;
diff --git a/include/uapi/linux/nfsd_netlink.h b/include/uapi/linux/nfsd_netlink.h
index dcd2126d848b..6628ed7c0922 100644
--- a/include/uapi/linux/nfsd_netlink.h
+++ b/include/uapi/linux/nfsd_netlink.h
@@ -59,6 +59,16 @@ enum nfsd_rpcbind_flags {
 	NFSD_RPCBIND_FLAGS_NO_UDP = 1,
 };
 
+/*
+ * NFS READ/WRITE I/O mode. These values match the global io_cache_read and
+ * io_cache_write debugfs settings.
+ */
+enum nfsd_io_mode {
+	NFSD_IO_MODE_BUFFERED,
+	NFSD_IO_MODE_DONTCACHE,
+	NFSD_IO_MODE_DIRECT,
+};
+
 enum {
 	NFSD_A_CACHE_NOTIFY_CACHE_TYPE = 1,
 
@@ -185,6 +195,8 @@ enum {
 	NFSD_A_SVC_EXPORT_XPRTSEC,
 	NFSD_A_SVC_EXPORT_FLAGS,
 	NFSD_A_SVC_EXPORT_FSID,
+	NFSD_A_SVC_EXPORT_READ_IOMODE,
+	NFSD_A_SVC_EXPORT_WRITE_IOMODE,
 
 	__NFSD_A_SVC_EXPORT_MAX,
 	NFSD_A_SVC_EXPORT_MAX = (__NFSD_A_SVC_EXPORT_MAX - 1)

-- 
2.56.0

[PATCH 2/6] NFSD: Apply per-export I/O modes to READ and WRITE

[Jeff Layton]

Resolve each export's effective I/O mode (explicit setting, else the
global io_cache_read/write knob) in nfsd_iter_read and nfsd_vfs_write,
and skip splice for reads on exports whose effective read mode is not
buffered. Update the debugfs knob comments to match.

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 fs/nfsd/debugfs.c |  6 ++++--
 fs/nfsd/nfs4xdr.c |  6 ++++--
 fs/nfsd/vfs.c     | 39 ++++++++++++++++++++++++++++++++++++---
 fs/nfsd/vfs.h     |  2 ++
 4 files changed, 46 insertions(+), 7 deletions(-)

diff --git a/fs/nfsd/debugfs.c b/fs/nfsd/debugfs.c
index 4af1e6b8b212..a10159db3b60 100644
--- a/fs/nfsd/debugfs.c
+++ b/fs/nfsd/debugfs.c
@@ -47,7 +47,8 @@ DEFINE_DEBUGFS_ATTRIBUTE(nfsd_dsr_fops, nfsd_dsr_get, nfsd_dsr_set, "%llu\n");
  *   %2: NFS READ will use direct IO
  *
  * This setting takes immediate effect for all NFS versions,
- * all exports, and in all NFSD net namespaces.
+ * all exports without an explicit riomode= setting, and in all
+ * NFSD net namespaces.
  */
 
 static int nfsd_io_cache_read_get(void *data, u64 *val)
@@ -92,7 +93,8 @@ DEFINE_DEBUGFS_ATTRIBUTE(nfsd_io_cache_read_fops, nfsd_io_cache_read_get,
  *   %1: NFS WRITE will use dontcache (buffered IO w/ dropbehind)
  *
  * This setting takes immediate effect for all NFS versions,
- * all exports, and in all NFSD net namespaces.
+ * all exports without an explicit wiomode= setting, and in all
+ * NFSD net namespaces.
  */
 
 static int nfsd_io_cache_write_get(void *data, u64 *val)
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 4d983444604a..9215feeeb0ba 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -5366,7 +5366,8 @@ nfsd4_encode_read(struct nfsd4_compoundres *resp, __be32 nfserr,
 	maxcount = min_t(unsigned long, read->rd_length,
 			 (xdr->buf->buflen - xdr->buf->len));
 
-	if (file->f_op->splice_read && splice_ok)
+	if (file->f_op->splice_read && splice_ok &&
+	    nfsd_read_iomode(read->rd_fhp->fh_export) == NFSD_IO_BUFFERED)
 		nfserr = nfsd4_encode_splice_read(resp, read, file, maxcount);
 	else
 		nfserr = nfsd4_encode_readv(resp, read, maxcount);
@@ -6264,7 +6265,8 @@ nfsd4_encode_read_plus_data(struct nfsd4_compoundres *resp,
 	maxcount = min_t(unsigned long, read->rd_length,
 			 (xdr->buf->buflen - xdr->buf->len));
 
-	if (file->f_op->splice_read && splice_ok)
+	if (file->f_op->splice_read && splice_ok &&
+	    nfsd_read_iomode(read->rd_fhp->fh_export) == NFSD_IO_BUFFERED)
 		nfserr = nfsd4_encode_splice_read(resp, read, file, maxcount);
 	else
 		nfserr = nfsd4_encode_readv(resp, read, maxcount);
diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c
index 7f9ec6c4af6d..15613b6df290 100644
--- a/fs/nfsd/vfs.c
+++ b/fs/nfsd/vfs.c
@@ -1266,6 +1266,38 @@ static bool nfsd_direct_read_ok(struct svc_rqst *rqstp, struct svc_fh *fhp,
 	return true;
 }
 
+/**
+ * nfsd_read_iomode - effective NFS READ I/O mode for an export
+ * @exp: export being accessed
+ *
+ * An export without an explicit riomode= setting follows the global
+ * nfsd_io_cache_read knob.
+ *
+ * Returns one of NFSD_IO_BUFFERED, NFSD_IO_DONTCACHE, or NFSD_IO_DIRECT.
+ */
+int nfsd_read_iomode(struct svc_export *exp)
+{
+	if (exp->ex_riomode == NFSD_IO_DEFAULT)
+		return nfsd_io_cache_read;
+	return exp->ex_riomode;
+}
+
+/**
+ * nfsd_write_iomode - effective NFS WRITE I/O mode for an export
+ * @exp: export being accessed
+ *
+ * An export without an explicit wiomode= setting follows the global
+ * nfsd_io_cache_write knob.
+ *
+ * Returns one of NFSD_IO_BUFFERED, NFSD_IO_DONTCACHE, or NFSD_IO_DIRECT.
+ */
+int nfsd_write_iomode(struct svc_export *exp)
+{
+	if (exp->ex_wiomode == NFSD_IO_DEFAULT)
+		return nfsd_io_cache_write;
+	return exp->ex_wiomode;
+}
+
 /**
  * nfsd_iter_read - Perform a VFS read using an iterator
  * @rqstp: RPC transaction context
@@ -1300,7 +1332,7 @@ __be32 nfsd_iter_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
 
 	init_sync_kiocb(&kiocb, file);
 
-	switch (nfsd_io_cache_read) {
+	switch (nfsd_read_iomode(fhp->fh_export)) {
 	case NFSD_IO_BUFFERED:
 		break;
 	case NFSD_IO_DIRECT:
@@ -1746,7 +1778,7 @@ nfsd_vfs_write(struct svc_rqst *rqstp, struct svc_fh *fhp,
 	if (verf)
 		nfsd_copy_write_verifier(verf, nn);
 
-	switch (nfsd_io_cache_write) {
+	switch (nfsd_write_iomode(exp)) {
 	case NFSD_IO_DIRECT:
 		host_err = nfsd_direct_write(rqstp, fhp, nf, nvecs,
 					     cnt, &kiocb);
@@ -1853,7 +1885,8 @@ __be32 nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
 		return err;
 
 	file = nf->nf_file;
-	if (file->f_op->splice_read && nfsd_read_splice_ok(rqstp))
+	if (file->f_op->splice_read && nfsd_read_splice_ok(rqstp) &&
+	    nfsd_read_iomode(fhp->fh_export) == NFSD_IO_BUFFERED)
 		err = nfsd_splice_read(rqstp, fhp, file, offset, count, eof);
 	else
 		err = nfsd_iter_read(rqstp, fhp, nf, offset, count, eof);
diff --git a/fs/nfsd/vfs.h b/fs/nfsd/vfs.h
index 1f05c83a3ea6..e95611b79397 100644
--- a/fs/nfsd/vfs.h
+++ b/fs/nfsd/vfs.h
@@ -184,6 +184,8 @@ __be32		nfsd_iter_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
 				struct nfsd_file *nf, loff_t offset,
 				unsigned long *count, u32 *eof);
 bool		nfsd_read_splice_ok(struct svc_rqst *rqstp);
+int		nfsd_read_iomode(struct svc_export *exp);
+int		nfsd_write_iomode(struct svc_export *exp);
 __be32		nfsd_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
 				loff_t offset, unsigned long *count,
 				u32 *eof);

-- 
2.56.0

[PATCH 3/6] NFSD: Document per-export riomode= and wiomode= options

[Jeff Layton]

Describe the new export options and their precedence over the global
debugfs knobs.

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 Documentation/filesystems/nfs/nfsd-io-modes.rst | 26 +++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/Documentation/filesystems/nfs/nfsd-io-modes.rst b/Documentation/filesystems/nfs/nfsd-io-modes.rst
index f06fa1b9d406..6b29275e9128 100644
--- a/Documentation/filesystems/nfs/nfsd-io-modes.rst
+++ b/Documentation/filesystems/nfs/nfsd-io-modes.rst
@@ -41,6 +41,32 @@ corresponding IO operation's debugfs interface, e.g.::
 If you experiment with NFSD's IO modes on a recent kernel and have
 interesting results, please report them to linux-nfs@vger.kernel.org
 
+NFSD PER-EXPORT IO MODES
+========================
+
+In addition to the global debugfs knobs above, the IO mode may be set
+independently for each export via the ``riomode=`` and ``wiomode=``
+export options, e.g.::
+
+  /srv/data  client(ro,riomode=direct,wiomode=dontcache)
+
+Each option accepts one of ``buffered``, ``dontcache``, or ``direct``,
+with the same meanings as the corresponding debugfs values.
+
+An export without an explicit ``riomode=`` (or ``wiomode=``) setting
+follows the global ``io_cache_read`` (or ``io_cache_write``) knob, so
+the effective default for both reads and writes remains buffered IO.
+An explicit per-export setting pins that export's IO mode regardless
+of the global knob.
+
+As with the global setting, a read IO mode of ``dontcache`` or
+``direct`` disables the use of splice for reads via that export.
+
+The per-export settings are reported in
+/proc/net/rpc/nfsd.export/content, e.g.::
+
+  /srv/data  client(...,riomode=direct,wiomode=dontcache)
+
 NFSD DONTCACHE
 ==============
 

-- 
2.56.0

[PATCH 4/6] NFSD: Keep per-export statistics across export cache updates

[Jeff Layton]

The counters in /proc/fs/nfsd/export_stats live in the export cache
entry. mountd replaces an entry when it refreshes it, and a cache
flush (exportfs -r) or an nfsd restart removes all of them, so the
counters of an export restart at zero at unpredictable times. A tool
that samples the counters over an interval then loses the traffic of
the replaced entry.

Keep the statistics in a per-net table keyed on the export root and
the client name. The root is identified by the st_dev and inode
number that getattr reports, plus the inode generation. btrfs
subvolumes share i_sb->s_dev and a snapshot root has the inode number
and generation of its source, but getattr gives each subvolume its
own st_dev.

Every export cache entry for the same root and client takes a
reference to the shared export_stats in svc_export_init() and drops
it when the entry is freed. Keying on the inode instead of holding
ex_path keeps unused statistics from pinning the exported mount.

Statistics that no entry has used for an hour no longer match a
lookup, since the dev_t of a vanished filesystem can be reused by
another one. A delayed work item, queued when statistics become idle,
frees them; net shutdown frees the rest.

A replaced cache entry stays hashed until cache_clean() and would show
the same shared counters as its replacement, so export_stats skips it.

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 fs/nfsd/export.c | 236 +++++++++++++++++++++++++++++++++++++++++++++++--------
 fs/nfsd/export.h |  16 ++++
 fs/nfsd/netns.h  |   6 ++
 3 files changed, 226 insertions(+), 32 deletions(-)

diff --git a/fs/nfsd/export.c b/fs/nfsd/export.c
index c71350965789..640973d02b6c 100644
--- a/fs/nfsd/export.c
+++ b/fs/nfsd/export.c
@@ -594,28 +594,198 @@ static void nfsd4_fslocs_free(struct nfsd4_fs_locations *fsloc)
 	fsloc->locations = NULL;
 }
 
-static int export_stats_init(struct export_stats *stats)
+/* Drop export_stats that no export cache entry has used for this long */
+#define EXPORT_STATS_IDLE_SECS	3600
+
+static u64 export_stats_key(dev_t dev, u64 ino)
+{
+	return ((u64)dev << 32) ^ ino;
+}
+
+static bool export_stats_expired(const struct export_stats *stats,
+				 time64_t now)
+{
+	return !stats->es_users &&
+	       now - stats->es_idle_since > EXPORT_STATS_IDLE_SECS;
+}
+
+/*
+ * Expired statistics never match: once the filesystem is gone, its
+ * dev_t can be reused by a different one.
+ */
+static struct export_stats *export_stats_find(struct nfsd_net *nn, dev_t dev,
+					      u64 ino, u32 gen,
+					      const char *client, time64_t now)
+{
+	struct export_stats *stats;
+
+	hash_for_each_possible(nn->export_stats_hash, stats, es_node,
+			       export_stats_key(dev, ino))
+		if (stats->es_dev == dev && stats->es_ino == ino &&
+		    stats->es_gen == gen && !strcmp(stats->es_client, client) &&
+		    !export_stats_expired(stats, now))
+			return stats;
+	return NULL;
+}
+
+static void export_stats_free(struct export_stats *stats)
 {
-	stats->start_time = ktime_get_seconds();
-	return percpu_counter_init_many(stats->counter, 0, GFP_KERNEL,
-					EXP_STATS_COUNTERS_NUM);
+	percpu_counter_destroy_many(stats->counter, EXP_STATS_COUNTERS_NUM);
+	kfree(stats->es_client);
+	kfree(stats);
 }
 
-static void export_stats_reset(struct export_stats *stats)
+static void export_stats_queue_reap(struct nfsd_net *nn)
 {
+	queue_delayed_work(system_dfl_wq, &nn->export_stats_reaper,
+			   (EXPORT_STATS_IDLE_SECS + 1) * HZ);
+}
+
+/*
+ * Free expired statistics. Re-arm while idle ones remain, because
+ * export_stats_put() does not queue the work while it is pending.
+ */
+static void export_stats_reap(struct work_struct *work)
+{
+	struct nfsd_net *nn = container_of(to_delayed_work(work),
+					   struct nfsd_net,
+					   export_stats_reaper);
+	time64_t now = ktime_get_seconds();
+	struct export_stats *stats;
+	struct hlist_node *tmp;
+	HLIST_HEAD(expired);
+	bool idle = false;
+	int bkt;
+
+	spin_lock_bh(&nn->export_stats_lock);
+	hash_for_each_safe(nn->export_stats_hash, bkt, tmp, stats, es_node) {
+		if (export_stats_expired(stats, now)) {
+			hash_del(&stats->es_node);
+			hlist_add_head(&stats->es_node, &expired);
+		} else if (!stats->es_users) {
+			idle = true;
+		}
+	}
+	if (idle)
+		export_stats_queue_reap(nn);
+	spin_unlock_bh(&nn->export_stats_lock);
+
+	hlist_for_each_entry_safe(stats, tmp, &expired, es_node) {
+		hlist_del(&stats->es_node);
+		export_stats_free(stats);
+	}
+}
+
+/*
+ * Return the statistics of the export root and client of @exp, with a
+ * reference. Keying them on the root inode rather than on ex_path keeps
+ * an unused export_stats from pinning the exported mount. Returns NULL
+ * if getattr or memory allocation fails.
+ */
+static struct export_stats *export_stats_get(struct svc_export *exp)
+{
+	struct nfsd_net *nn = net_generic(exp->cd->net, nfsd_net_id);
+	struct inode *inode = d_inode(exp->ex_path.dentry);
+	const char *client = exp->ex_client->name;
+	time64_t now = ktime_get_seconds();
+	u32 gen = inode->i_generation;
+	struct export_stats *stats, *new;
+	struct kstat stat;
+	dev_t dev;
+	u64 ino;
+
+	/*
+	 * btrfs subvolumes share i_sb->s_dev, and a snapshot's root has the
+	 * inode number and generation of its source. getattr reports a
+	 * separate st_dev for each subvolume. The key is never shown to
+	 * anyone, so skip the LSM check.
+	 */
+	if (vfs_getattr_nosec(&exp->ex_path, &stat, STATX_INO,
+			      AT_STATX_DONT_SYNC))
+		return NULL;
+	dev = stat.dev;
+	ino = stat.ino;
+
+	spin_lock_bh(&nn->export_stats_lock);
+	stats = export_stats_find(nn, dev, ino, gen, client, now);
+	if (stats)
+		stats->es_users++;
+	spin_unlock_bh(&nn->export_stats_lock);
+	if (stats)
+		return stats;
+
+	new = kzalloc_obj(*new);
+	if (!new)
+		return NULL;
+	new->es_client = kstrdup(client, GFP_KERNEL);
+	if (!new->es_client ||
+	    percpu_counter_init_many(new->counter, 0, GFP_KERNEL,
+				     EXP_STATS_COUNTERS_NUM)) {
+		kfree(new->es_client);
+		kfree(new);
+		return NULL;
+	}
+	new->es_nn = nn;
+	new->es_users = 1;
+	new->es_dev = dev;
+	new->es_ino = ino;
+	new->es_gen = gen;
+	new->start_time = now;
+
+	spin_lock_bh(&nn->export_stats_lock);
+	stats = export_stats_find(nn, dev, ino, gen, client, now);
 	if (stats) {
-		int i;
+		stats->es_users++;
+	} else {
+		hash_add(nn->export_stats_hash, &new->es_node,
+			 export_stats_key(dev, ino));
+		stats = new;
+		new = NULL;
+	}
+	spin_unlock_bh(&nn->export_stats_lock);
 
-		for (i = 0; i < EXP_STATS_COUNTERS_NUM; i++)
-			percpu_counter_set(&stats->counter[i], 0);
+	if (new)
+		export_stats_free(new);
+	return stats;
+}
+
+/* Called from an RCU callback */
+static void export_stats_put(struct export_stats *stats)
+{
+	struct nfsd_net *nn;
+
+	if (!stats)
+		return;
+	nn = stats->es_nn;
+	spin_lock_bh(&nn->export_stats_lock);
+	if (!--stats->es_users) {
+		stats->es_idle_since = ktime_get_seconds();
+		export_stats_queue_reap(nn);
 	}
+	spin_unlock_bh(&nn->export_stats_lock);
 }
 
-static void export_stats_destroy(struct export_stats *stats)
+static s64 export_stats_sum(struct export_stats *stats, int counter)
 {
-	if (stats)
-		percpu_counter_destroy_many(stats->counter,
-					    EXP_STATS_COUNTERS_NUM);
+	return stats ? percpu_counter_sum_positive(&stats->counter[counter]) : 0;
+}
+
+static void export_stats_shutdown(struct nfsd_net *nn)
+{
+	struct export_stats *stats;
+	struct hlist_node *tmp;
+	int bkt;
+
+	/* Wait for svc_export_release() to drop its references */
+	rcu_barrier();
+	/* Only now, as those last puts can queue the reaper */
+	cancel_delayed_work_sync(&nn->export_stats_reaper);
+	hash_for_each_safe(nn->export_stats_hash, bkt, tmp, stats, es_node) {
+		hash_del(&stats->es_node);
+		if (WARN_ON_ONCE(stats->es_users))
+			continue;
+		export_stats_free(stats);
+	}
 }
 
 static void svc_export_release(struct rcu_head *rcu_head)
@@ -624,8 +794,7 @@ static void svc_export_release(struct rcu_head *rcu_head)
 			ex_rcu);
 
 	nfsd4_fslocs_free(&exp->ex_fslocs);
-	export_stats_destroy(exp->ex_stats);
-	kfree(exp->ex_stats);
+	export_stats_put(exp->ex_stats);
 	kfree(exp->ex_uuid);
 	kfree(exp);
 }
@@ -1497,15 +1666,15 @@ static int svc_export_show(struct seq_file *m,
 	seq_putc(m, '\t');
 	seq_escape(m, exp->ex_client->name, " \t\n\\");
 	if (export_stats) {
-		struct percpu_counter *counter = exp->ex_stats->counter;
+		struct export_stats *stats = exp->ex_stats;
 
-		seq_printf(m, "\t%lld\n", exp->ex_stats->start_time);
+		seq_printf(m, "\t%lld\n", stats ? stats->start_time : 0);
 		seq_printf(m, "\tfh_stale: %lld\n",
-			   percpu_counter_sum_positive(&counter[EXP_STATS_FH_STALE]));
+			   export_stats_sum(stats, EXP_STATS_FH_STALE));
 		seq_printf(m, "\tio_read: %lld\n",
-			   percpu_counter_sum_positive(&counter[EXP_STATS_IO_READ]));
+			   export_stats_sum(stats, EXP_STATS_IO_READ));
 		seq_printf(m, "\tio_write: %lld\n",
-			   percpu_counter_sum_positive(&counter[EXP_STATS_IO_WRITE]));
+			   export_stats_sum(stats, EXP_STATS_IO_WRITE));
 		seq_putc(m, '\n');
 		return 0;
 	}
@@ -1557,7 +1726,7 @@ static void svc_export_init(struct cache_head *cnew, struct cache_head *citem)
 	new->ex_layout_types = 0;
 	new->ex_uuid = NULL;
 	new->cd = item->cd;
-	export_stats_reset(new->ex_stats);
+	new->ex_stats = export_stats_get(new);
 }
 
 static void export_update(struct cache_head *cnew, struct cache_head *citem)
@@ -1596,18 +1765,8 @@ static struct cache_head *svc_export_alloc(void)
 	if (!i)
 		return NULL;
 
-	i->ex_stats = kmalloc_obj(*(i->ex_stats));
-	if (!i->ex_stats) {
-		kfree(i);
-		return NULL;
-	}
-
-	if (export_stats_init(i->ex_stats)) {
-		kfree(i->ex_stats);
-		kfree(i);
-		return NULL;
-	}
-
+	/* svc_export_init() attaches the shared statistics */
+	i->ex_stats = NULL;
 	return &i->h;
 }
 
@@ -2190,6 +2349,14 @@ static int e_show(struct seq_file *m, void *p)
 
 	if (cache_check_rcu(cd, &exp->h, NULL))
 		return 0;
+	/*
+	 * sunrpc_cache_update() leaves a replaced entry hashed, with an
+	 * expiry_time of zero, until cache_clean() removes it. It shares
+	 * export_stats with its replacement, so showing it would count the
+	 * same traffic twice.
+	 */
+	if (export_stats && !exp->h.expiry_time)
+		return 0;
 
 	return svc_export_show(m, cd, cp);
 }
@@ -2212,6 +2379,10 @@ nfsd_export_init(struct net *net)
 
 	dprintk("nfsd: initializing export module (net: %x).\n", net->ns.inum);
 
+	spin_lock_init(&nn->export_stats_lock);
+	hash_init(nn->export_stats_hash);
+	INIT_DELAYED_WORK(&nn->export_stats_reaper, export_stats_reap);
+
 	nn->svc_export_cache = cache_create_net(&svc_export_cache_template, net);
 	if (IS_ERR(nn->svc_export_cache))
 		return PTR_ERR(nn->svc_export_cache);
@@ -2264,6 +2435,7 @@ nfsd_export_shutdown(struct net *net)
 	cache_unregister_net(nn->svc_export_cache, net);
 	cache_destroy_net(nn->svc_expkey_cache, net);
 	cache_destroy_net(nn->svc_export_cache, net);
+	export_stats_shutdown(nn);
 	svcauth_unix_purge(net);
 
 	dprintk("nfsd: export shutdown complete (net: %x).\n", net->ns.inum);
diff --git a/fs/nfsd/export.h b/fs/nfsd/export.h
index 0e832990a12c..70842782629f 100644
--- a/fs/nfsd/export.h
+++ b/fs/nfsd/export.h
@@ -11,6 +11,7 @@
 #include <linux/nfs4.h>
 
 struct knfsd_fh;
+struct nfsd_net;
 struct svc_fh;
 struct svc_rqst;
 
@@ -55,7 +56,22 @@ enum {
 	EXP_STATS_COUNTERS_NUM
 };
 
+/*
+ * Export cache entries are replaced when mountd refreshes them and are
+ * removed by a cache flush. The statistics of an export outlive its
+ * cache entries: every entry for the same export root and client shares
+ * one export_stats, which stays in nn->export_stats_hash for a time
+ * after its last entry is gone.
+ */
 struct export_stats {
+	struct hlist_node	es_node;
+	struct nfsd_net		*es_nn;
+	unsigned int		es_users;	/* under nn->export_stats_lock */
+	time64_t		es_idle_since;
+	dev_t			es_dev;
+	u64			es_ino;
+	u32			es_gen;
+	char			*es_client;
 	time64_t		start_time;
 	struct percpu_counter	counter[EXP_STATS_COUNTERS_NUM];
 };
diff --git a/fs/nfsd/netns.h b/fs/nfsd/netns.h
index fa3545963902..66e8cfff238b 100644
--- a/fs/nfsd/netns.h
+++ b/fs/nfsd/netns.h
@@ -11,6 +11,7 @@
 #include <net/net_namespace.h>
 #include <net/netns/generic.h>
 #include <linux/filelock.h>
+#include <linux/hashtable.h>
 #include <linux/nfs4.h>
 #include <linux/percpu_counter.h>
 #include <linux/percpu-refcount.h>
@@ -77,6 +78,11 @@ struct nfsd_net {
 	struct cache_detail *svc_expkey_cache;
 	struct cache_detail *svc_export_cache;
 
+	/* per-export statistics, shared by export cache entries */
+	spinlock_t export_stats_lock;
+	DECLARE_HASHTABLE(export_stats_hash, 6);
+	struct delayed_work export_stats_reaper;
+
 	struct cache_detail *idtoname_cache;
 	struct cache_detail *nametoid_cache;
 

-- 
2.56.0

[PATCH 5/6] NFSD: Count the storage reads of each export's READs

[Jeff Layton]

The page cache miss ratio of an export is the best indicator of the IO
mode that suits it, but NFSD reports only the bytes that READ returned.
Comparing that with the reads of the block device is approximate: the
device can hold other exports or local data, the count includes every
user of the device, and btrfs can span devices.

submit_bio() charges read bytes to the submitting task. Take a snapshot
of current->ioac.read_bytes when a READ starts, and add the difference
to a new io_read_storage counter of the export when it finishes. This
includes the readahead that the READ started and the metadata that it
needed, and works the same for buffered, dontcache, direct and splice
reads. export_stats shows the counter only with
CONFIG_TASK_IO_ACCOUNTING, so a missing counter means it is not
available.

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 fs/nfsd/export.c |  4 ++++
 fs/nfsd/export.h |  1 +
 fs/nfsd/stats.h  | 33 +++++++++++++++++++++++++++++++++
 fs/nfsd/vfs.c    | 20 ++++++++++++++------
 4 files changed, 52 insertions(+), 6 deletions(-)

diff --git a/fs/nfsd/export.c b/fs/nfsd/export.c
index 640973d02b6c..ac2d71a9c562 100644
--- a/fs/nfsd/export.c
+++ b/fs/nfsd/export.c
@@ -1675,6 +1675,10 @@ static int svc_export_show(struct seq_file *m,
 			   export_stats_sum(stats, EXP_STATS_IO_READ));
 		seq_printf(m, "\tio_write: %lld\n",
 			   export_stats_sum(stats, EXP_STATS_IO_WRITE));
+		if (IS_ENABLED(CONFIG_TASK_IO_ACCOUNTING))
+			seq_printf(m, "\tio_read_storage: %lld\n",
+				   export_stats_sum(stats,
+						    EXP_STATS_IO_READ_STORAGE));
 		seq_putc(m, '\n');
 		return 0;
 	}
diff --git a/fs/nfsd/export.h b/fs/nfsd/export.h
index 70842782629f..8ecd43894bec 100644
--- a/fs/nfsd/export.h
+++ b/fs/nfsd/export.h
@@ -53,6 +53,7 @@ enum {
 	EXP_STATS_FH_STALE,
 	EXP_STATS_IO_READ,
 	EXP_STATS_IO_WRITE,
+	EXP_STATS_IO_READ_STORAGE,
 	EXP_STATS_COUNTERS_NUM
 };
 
diff --git a/fs/nfsd/stats.h b/fs/nfsd/stats.h
index aabfbb1a9c71..d88d2229a4c7 100644
--- a/fs/nfsd/stats.h
+++ b/fs/nfsd/stats.h
@@ -9,6 +9,7 @@
 
 #include <uapi/linux/nfsd/stats.h>
 #include <linux/percpu_counter.h>
+#include <linux/sched.h>
 
 #include "export.h"
 #include "netns.h"
@@ -97,6 +98,38 @@ static inline void nfsd_stats_io_write_add(struct nfsd_net *nn,
 					 amount);
 }
 
+/**
+ * nfsd_storage_read_bytes - bytes the current task has read from storage
+ *
+ * submit_bio() charges read bytes to the submitting task, so the change
+ * across a READ is what that READ, including the readahead it starts,
+ * read from storage instead of the page cache.
+ *
+ * Return: the task's storage read byte count, or zero without
+ * CONFIG_TASK_IO_ACCOUNTING.
+ */
+static inline u64 nfsd_storage_read_bytes(void)
+{
+#ifdef CONFIG_TASK_IO_ACCOUNTING
+	return current->ioac.read_bytes;
+#else
+	return 0;
+#endif
+}
+
+/**
+ * nfsd_stats_io_read_storage_add - Count storage bytes read for an NFS READ
+ * @exp: target export
+ * @amount: byte count
+ */
+static inline void nfsd_stats_io_read_storage_add(struct svc_export *exp,
+						  s64 amount)
+{
+	if (exp && exp->ex_stats && amount > 0)
+		percpu_counter_add_local(&exp->ex_stats->counter[EXP_STATS_IO_READ_STORAGE],
+					 amount);
+}
+
 static inline void nfsd_stats_payload_misses_inc(struct nfsd_net *nn)
 {
 	percpu_counter_inc(&nn->counter[NFSD_STATS_PAYLOAD_MISSES]);
diff --git a/fs/nfsd/vfs.c b/fs/nfsd/vfs.c
index 15613b6df290..50ef96292003 100644
--- a/fs/nfsd/vfs.c
+++ b/fs/nfsd/vfs.c
@@ -1113,8 +1113,11 @@ static u32 nfsd_eof_on_read(struct file *file, loff_t offset, ssize_t len,
 
 static __be32 nfsd_finish_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
 			       struct file *file, loff_t offset,
-			       unsigned long *count, u32 *eof, ssize_t host_err)
+			       unsigned long *count, u32 *eof, ssize_t host_err,
+			       u64 storage_start)
 {
+	nfsd_stats_io_read_storage_add(fhp->fh_export,
+				       nfsd_storage_read_bytes() - storage_start);
 	if (host_err >= 0) {
 		struct nfsd_net *nn = net_generic(SVC_NET(rqstp), nfsd_net_id);
 
@@ -1145,12 +1148,14 @@ __be32 nfsd_splice_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
 			struct file *file, loff_t offset, unsigned long *count,
 			u32 *eof)
 {
+	u64 storage_start = nfsd_storage_read_bytes();
 	ssize_t host_err;
 
 	trace_nfsd_read_splice(rqstp, fhp, offset, *count);
 	host_err = vfs_splice_to_actor(file, offset, *count,
 				       nfsd_direct_splice_actor, rqstp);
-	return nfsd_finish_read(rqstp, fhp, file, offset, count, eof, host_err);
+	return nfsd_finish_read(rqstp, fhp, file, offset, count, eof, host_err,
+				storage_start);
 }
 
 /*
@@ -1168,7 +1173,7 @@ __be32 nfsd_splice_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
 static noinline_for_stack __be32
 nfsd_direct_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
 		 struct nfsd_file *nf, loff_t offset, unsigned long *count,
-		 struct page **page, u32 *eof)
+		 struct page **page, u32 *eof, u64 storage_start)
 {
 	u64 dio_start, dio_end;
 	unsigned long v, total;
@@ -1226,7 +1231,7 @@ nfsd_direct_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
 	}
 
 	return nfsd_finish_read(rqstp, fhp, nf->nf_file, offset, count,
-				eof, host_err);
+				eof, host_err, storage_start);
 }
 
 static bool nfsd_direct_read_ok(struct svc_rqst *rqstp, struct svc_fh *fhp,
@@ -1324,6 +1329,7 @@ __be32 nfsd_iter_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
 	struct page **page = buf->pages + (buf->page_len >> PAGE_SHIFT);
 	unsigned int base = buf->page_len & ~PAGE_MASK;
 	struct file *file = nf->nf_file;
+	u64 storage_start = nfsd_storage_read_bytes();
 	unsigned long v, total;
 	struct iov_iter iter;
 	struct kiocb kiocb;
@@ -1338,7 +1344,8 @@ __be32 nfsd_iter_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
 	case NFSD_IO_DIRECT:
 		if (nfsd_direct_read_ok(rqstp, fhp, nf, offset, *count))
 			return nfsd_direct_read(rqstp, fhp, nf, offset,
-						count, page, eof);
+						count, page, eof,
+						storage_start);
 		fallthrough;
 	case NFSD_IO_DONTCACHE:
 		if (file->f_op->fop_flags & FOP_DONTCACHE)
@@ -1367,7 +1374,8 @@ __be32 nfsd_iter_read(struct svc_rqst *rqstp, struct svc_fh *fhp,
 				  kiocb.ki_flags);
 	iov_iter_bvec(&iter, ITER_DEST, rqstp->rq_bvec, v, *count - total);
 	host_err = vfs_iocb_iter_read(file, &kiocb, &iter);
-	return nfsd_finish_read(rqstp, fhp, file, offset, count, eof, host_err);
+	return nfsd_finish_read(rqstp, fhp, file, offset, count, eof, host_err,
+				storage_start);
 }
 
 /*

-- 
2.56.0

[PATCH 6/6] NFSD: Document how to choose an IO mode

[Jeff Layton]

Admins have no guidance on when to pick buffered, dontcache, or direct
IO for an export. Add a section that:

- explains why a low server page cache hit rate is normal (clients
  cache their own re-reads)
- lists the workload conditions that favor each IO mode
- describes existing metrics to measure: per-export miss ratio from
  export_stats vs. block device reads, workingset refault counters,
  cachestat(2), reclaim cost (PSI, allocstall, compact_stall), and
  client mountstats byte counters
- notes that refault counters do not see dontcache or direct IO, so
  measure while the export uses buffered IO

Signed-off-by: Jeff Layton <jlayton@kernel.org>
---
 Documentation/filesystems/nfs/nfsd-io-modes.rst | 168 ++++++++++++++++++++++++
 1 file changed, 168 insertions(+)

diff --git a/Documentation/filesystems/nfs/nfsd-io-modes.rst b/Documentation/filesystems/nfs/nfsd-io-modes.rst
index 6b29275e9128..400d5e462943 100644
--- a/Documentation/filesystems/nfs/nfsd-io-modes.rst
+++ b/Documentation/filesystems/nfs/nfsd-io-modes.rst
@@ -298,3 +298,171 @@ Tracing:
     segment that it could not invalidate. Only a filesystem that uses
     iomap for direct IO emits it, and other fallbacks, such as ext4's
     own, are not traced.
+
+CHOOSING AN NFSD IO MODE
+========================
+
+The correct IO mode for an export depends on the workload of its
+clients. The most important question is this: does NFSD read the same
+data again before the page cache evicts it?
+
+NFS clients keep their own page cache. When a client reads the same
+data again, its own cache usually supplies the data. The READ request
+does not get to the server. Thus, the server page cache helps mostly
+in these conditions:
+
+- Different clients read the same data.
+- A client reads data that a different client wrote recently.
+- A client reads data again after its own page cache evicted the data.
+
+A low server page cache hit rate is thus normal. It does not by itself
+show that buffered IO is the wrong choice.
+
+Use buffered IO for an export when:
+
+- The working set of the export fits in server memory.
+- Many clients read the same files.
+- Clients send small or misaligned WRITEs. The page cache collects
+  these WRITEs and writes them to storage in larger units.
+
+Use dontcache IO for an export when:
+
+- Clients read or write most data only one time.
+- The working set of the export is much larger than server memory.
+- The data of the export causes the page cache to evict data of other
+  exports.
+
+If the underlying filesystem does not support dontcache, NFSD uses
+buffered IO instead. See NFSD DONTCACHE above.
+
+Use direct IO for an export when the conditions for dontcache are true,
+and also:
+
+- Clients send large, aligned READs and WRITEs.
+- Page reclaim uses a large quantity of CPU time, or makes IO stop.
+
+Change the IO mode of one export at a time. Compare the metrics below
+before and after each change.
+
+Metrics to monitor
+------------------
+
+The metrics that follow can help you choose an IO mode. Measure them
+while the export uses buffered IO.
+
+Miss ratio for each export
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+``/proc/fs/nfsd/export_stats`` shows the number of bytes that NFSD read
+(``io_read``) and wrote (``io_write``) for each export and client. If the
+kernel has ``CONFIG_TASK_IO_ACCOUNTING``, it also shows the number of
+bytes that storage supplied for these READs (``io_read_storage``). This
+value includes the readahead that the READs started, and the filesystem
+metadata that they needed.
+
+The kernel adds an export to ``export_stats`` when a client first uses
+it. The counters of an export continue when mountd refreshes the export
+or when ``exportfs -r`` flushes the export cache. They restart if the
+export stays out of the export cache for an hour. The start time shows
+when the counters started.
+
+Measure ``io_read`` and ``io_read_storage`` over the same interval. Then
+divide the increase of ``io_read_storage`` by the increase of
+``io_read``. This is the miss ratio of the export.
+
+- A ratio near 1 shows that storage supplies almost all data that NFSD
+  reads. The page cache gives little benefit. Dontcache or direct IO
+  can be a better choice.
+- A ratio much smaller than 1 shows that the page cache supplies a
+  large part of the data. Buffered IO is probably the correct choice.
+- A ratio larger than 1 shows that readahead reads data that clients
+  do not request, for example with small random READs.
+
+Without ``io_read_storage``, compare ``io_read`` with the number of bytes
+that the block device read. Field 3 of ``/sys/block/<dev>/stat`` shows
+the device reads in 512-byte sectors. You can also use iostat(1). This
+ratio is for all exports on the device. Local applications also cause
+device reads. Thus, this ratio is only an approximate value.
+
+Reads of recently evicted data
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When page reclaim evicts a file page, the kernel records the eviction.
+If a task reads that page again soon, the kernel counts a refault.
+``/proc/vmstat`` shows these counters:
+
+- ``workingset_refault_file``: the number of file pages that the kernel
+  read again soon after their eviction.
+- ``workingset_activate_file``: the number of refaulted file pages that
+  the kernel identified as part of the working set.
+- ``pgsteal_kswapd`` and ``pgsteal_direct``: the number of pages that
+  page reclaim freed.
+
+Use these counters as follows:
+
+- The ``pgsteal`` counters increase quickly, but
+  ``workingset_refault_file`` stays near zero. This shows that clients
+  read most data only one time. To find the export that causes this,
+  use the miss ratio above or cachestat(2).
+- ``workingset_refault_file`` increases quickly. This shows that clients
+  read the data again, but the page cache is too small to keep it.
+  Set dontcache or direct IO on other exports that read data only one
+  time. This gives more page cache to this workload.
+
+These counters are for the full system, not for each export. Dontcache
+IO drops pages without a record of the eviction. Direct IO does not use
+the page cache. Thus, these counters do not include exports that use
+dontcache or direct IO.
+
+Page cache state of a file
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The cachestat(2) system call shows the page cache state of a range in a
+file. It is available in Linux 6.5 and later. Its output includes:
+
+- ``nr_cache``: the number of pages in the page cache.
+- ``nr_dirty`` and ``nr_writeback``: the number of pages that wait for
+  writeback, or that are in writeback.
+- ``nr_evicted``: the number of pages that the kernel evicted.
+- ``nr_recently_evicted``: the number of evicted pages that would cause
+  a refault if a task read them now.
+
+Use cachestat(2) on the frequently used files of an export. This gives
+data for each export. Monitor ``nr_recently_evicted`` on the files of
+one export while a different export is busy. If the value increases,
+the busy export causes the page cache to evict useful data. Set
+dontcache or direct IO on the busy export.
+
+fincore(1) shows how much of a file is in the page cache.
+
+Cost of page reclaim
+~~~~~~~~~~~~~~~~~~~~
+
+Direct IO helps most when page reclaim uses much CPU time or makes IO
+stop. Monitor these metrics:
+
+- ``/proc/pressure/memory``: the "some" and "full" lines show the
+  percentage of time that tasks waited for memory.
+- ``allocstall_*`` and ``pgscan_direct`` in ``/proc/vmstat``: direct
+  reclaim activity.
+- ``compact_stall`` in ``/proc/vmstat``: memory compaction stalls.
+- The CPU time of the kswapd and kcompactd kernel threads.
+- ``Dirty`` and ``Writeback`` in ``/proc/meminfo``, for WRITE workloads.
+
+High values can occur while NFSD moves large quantities of data that
+clients use only one time. In this condition, direct IO can decrease
+the CPU load and make IO performance more constant.
+
+Client page cache
+~~~~~~~~~~~~~~~~~
+
+On a Linux NFS client, the ``bytes:`` line in ``/proc/self/mountstats``
+shows the bytes that applications read (field 1). It also shows the
+bytes that the client read from the server (field 5). mountstats(8) and
+nfsiostat(8) also show these values.
+
+Field 5 can be much smaller than field 1. This shows that the client
+page cache supplies most reads. In this condition, the server page
+cache keeps copies of data that the clients already have. If other
+clients do not read the same data, dontcache IO on the server can be a
+better choice.

-- 
2.56.0

Re: [PATCH 2/6] NFSD: Apply per-export I/O modes to READ and WRITE

[Chuck Lever]

On 10/27/26 12:45 PM, Jeff Layton wrote:
> Resolve each export's effective I/O mode (explicit setting, else the
> global io_cache_read/write knob) in nfsd_iter_read and nfsd_vfs_write,
> and skip splice for reads on exports whose effective read mode is not
> buffered. Update the debugfs knob comments to match.

[ ... ]

> -	if (file->f_op->splice_read && nfsd_read_splice_ok(rqstp))
> +	if (file->f_op->splice_read && nfsd_read_splice_ok(rqstp) &&
> +	    nfsd_read_iomode(fhp->fh_export) == NFSD_IO_BUFFERED)
>  		err = nfsd_splice_read(rqstp, fhp, file, offset, count, eof);

nfsd_io_cache_read_set() still sets nfsd_disable_splice_read whenever
the global knob is switched to dontcache or direct, so
nfsd_read_splice_ok() returns false before this per-export test is
reached. An export pinned to riomode=buffered on such a server never
gets splice; it falls through to nfsd_iter_read() and does vector
buffered reads. The same applies to the splice_ok decisions in
nfsd4_encode_read() and nfsd4_encode_read_plus_data().

Patch 3/6 says an explicit per-export setting pins that export's
IO mode "regardless of the global knob", and that isn't quite true
now for the buffered case.

I need to think carefully about the interaction between the "disable
splice" setting, which is global, and the per-export I/O setting.

Separately, on the names of the export options: "riomode" and
"wiomode" are hard to read, and "mode" might suggest "permission bits"
to an administrator. How about read_io= and write_io= instead? They
follow the underscore style of the other multi-word export options
(no_subtree_check, root_squash, security_label) and read naturally:

  /srv/data  client(rw,read_io=direct,write_io=dontcache)

The values buffered, dontcache, and direct then carry the meaning
without "cache" or "mode" in the name.

The name appears in the exports syntax, the nfsd.export/content
output, the netlink YAML spec, and the forthcoming nfs-utils series,
so it's worth settling before v2 and before the nfs-utils side gets
merged.


-- 
Chuck Lever (Come to NFS bake-a-thon! https://nfsv4bat.org)

Re: [PATCH 2/6] NFSD: Apply per-export I/O modes to READ and WRITE

[Jeff Layton]

On Tue, 2026-10-27 at 13:54 -0400, Chuck Lever wrote:
> On 10/27/26 12:45 PM, Jeff Layton wrote:
> > Resolve each export's effective I/O mode (explicit setting, else the
> > global io_cache_read/write knob) in nfsd_iter_read and nfsd_vfs_write,
> > and skip splice for reads on exports whose effective read mode is not
> > buffered. Update the debugfs knob comments to match.
> 
> [ ... ]
> 
> > -	if (file->f_op->splice_read && nfsd_read_splice_ok(rqstp))
> > +	if (file->f_op->splice_read && nfsd_read_splice_ok(rqstp) &&
> > +	    nfsd_read_iomode(fhp->fh_export) == NFSD_IO_BUFFERED)
> >  		err = nfsd_splice_read(rqstp, fhp, file, offset, count, eof);
> 
> nfsd_io_cache_read_set() still sets nfsd_disable_splice_read whenever
> the global knob is switched to dontcache or direct, so
> nfsd_read_splice_ok() returns false before this per-export test is
> reached. An export pinned to riomode=buffered on such a server never
> gets splice; it falls through to nfsd_iter_read() and does vector
> buffered reads. The same applies to the splice_ok decisions in
> nfsd4_encode_read() and nfsd4_encode_read_plus_data().
> 
> Patch 3/6 says an explicit per-export setting pins that export's
> IO mode "regardless of the global knob", and that isn't quite true
> now for the buffered case.
> 
> I need to think carefully about the interaction between the "disable
> splice" setting, which is global, and the per-export I/O setting.
> 

Maybe disable_splice should be an export option of its own? It seems
like that should be doable.

> Separately, on the names of the export options: "riomode" and
> "wiomode" are hard to read, and "mode" might suggest "permission bits"
> to an administrator. How about read_io= and write_io= instead? They
> follow the underscore style of the other multi-word export options
> (no_subtree_check, root_squash, security_label) and read naturally:
> 
>   /srv/data  client(rw,read_io=direct,write_io=dontcache)
> 
> The values buffered, dontcache, and direct then carry the meaning
> without "cache" or "mode" in the name.
> 
> The name appears in the exports syntax, the nfsd.export/content
> output, the netlink YAML spec, and the forthcoming nfs-utils series,
> so it's worth settling before v2 and before the nfs-utils side gets
> merged.
> 

Ok, I'll look at renaming them. Thanks for the initial review!
-- 
Jeff Layton <jlayton@kernel.org>

Re: [PATCH 2/6] NFSD: Apply per-export I/O modes to READ and WRITE

[Chuck Lever]

On Tue, Oct 27, 2026, at 1:59 PM, Jeff Layton wrote:
> On Tue, 2026-10-27 at 13:54 -0400, Chuck Lever wrote:
>> On 10/27/26 12:45 PM, Jeff Layton wrote:
>> > Resolve each export's effective I/O mode (explicit setting, else the
>> > global io_cache_read/write knob) in nfsd_iter_read and nfsd_vfs_write,
>> > and skip splice for reads on exports whose effective read mode is not
>> > buffered. Update the debugfs knob comments to match.
>> 
>> [ ... ]
>> 
>> > -	if (file->f_op->splice_read && nfsd_read_splice_ok(rqstp))
>> > +	if (file->f_op->splice_read && nfsd_read_splice_ok(rqstp) &&
>> > +	    nfsd_read_iomode(fhp->fh_export) == NFSD_IO_BUFFERED)
>> >  		err = nfsd_splice_read(rqstp, fhp, file, offset, count, eof);
>> 
>> nfsd_io_cache_read_set() still sets nfsd_disable_splice_read whenever
>> the global knob is switched to dontcache or direct, so
>> nfsd_read_splice_ok() returns false before this per-export test is
>> reached. An export pinned to riomode=buffered on such a server never
>> gets splice; it falls through to nfsd_iter_read() and does vector
>> buffered reads. The same applies to the splice_ok decisions in
>> nfsd4_encode_read() and nfsd4_encode_read_plus_data().
>> 
>> Patch 3/6 says an explicit per-export setting pins that export's
>> IO mode "regardless of the global knob", and that isn't quite true
>> now for the buffered case.
>> 
>> I need to think carefully about the interaction between the "disable
>> splice" setting, which is global, and the per-export I/O setting.
>> 
>
> Maybe disable_splice should be an export option of its own? It seems
> like that should be doable.

That would be easy to implement, but I wonder if it's something we want
to carry as a permanent extension of the export options API.


-- 
Chuck Lever (Come to NFS bake-a-thon! https://nfsv4bat.org)