mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Jeffle Xu <jefflexu@linux.alibaba.com>
To: dhowells@redhat.com, linux-cachefs@redhat.com, xiang@kernel.org,
	chao@kernel.org, linux-erofs@lists.ozlabs.org
Cc: torvalds@linux-foundation.org, gregkh@linuxfoundation.org,
	willy@infradead.org, linux-fsdevel@vger.kernel.org,
	joseph.qi@linux.alibaba.com, bo.liu@linux.alibaba.com,
	tao.peng@linux.alibaba.com, gerry@linux.alibaba.com,
	eguan@linux.alibaba.com, linux-kernel@vger.kernel.org,
	luodaowen.backend@bytedance.com, tianzichen@kuaishou.com,
	fannaihao@baidu.com
Subject: [PATCH v6 11/22] erofs: register global fscache volume
Date: Fri, 25 Mar 2022 20:22:12 +0800	[thread overview]
Message-ID: <20220325122223.102958-12-jefflexu@linux.alibaba.com> (raw)
In-Reply-To: <20220325122223.102958-1-jefflexu@linux.alibaba.com>

All erofs instances will share one global fscache volume.

In this using scenario, one erofs instance could be mounted from one (or
multiple) blob files instead of blkdev. The number of blob files that
each erofs instance could correspond to is limited. For example, when
used for container image distribution, one erofs instance used for
container image for node.js will correspond to ~20 blob files in total.
Thus in densely employed environment, there could be as many as hundreds
of containers and thus thousands of fscache cookies under one fscache
volume.

Then as for cachefiles backend, the hash table managing all cookies
under one volume contains 32K slots. Thus the hashing functionality shall
scale well in this case. Besides, cachefiles backend will scatter
backing files under 256 fan sub-directoris, and thus the scalability of
looking up backing files shall also not be an issue.

Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com>
---
 fs/erofs/Kconfig    | 10 ++++++++++
 fs/erofs/Makefile   |  1 +
 fs/erofs/fscache.c  | 22 ++++++++++++++++++++++
 fs/erofs/internal.h |  9 +++++++++
 fs/erofs/super.c    |  7 +++++++
 5 files changed, 49 insertions(+)
 create mode 100644 fs/erofs/fscache.c

diff --git a/fs/erofs/Kconfig b/fs/erofs/Kconfig
index f57255ab88ed..a6f5d61bdc49 100644
--- a/fs/erofs/Kconfig
+++ b/fs/erofs/Kconfig
@@ -98,3 +98,13 @@ config EROFS_FS_ZIP_LZMA
 	  systems will be readable without selecting this option.
 
 	  If unsure, say N.
+
+config EROFS_FS_ONDEMAND
+	bool "EROFS fscache-based ondeand-read"
+	depends on CACHEFILES_ONDEMAND && (EROFS_FS=m && FSCACHE && CACHEFILES || EROFS_FS=y && FSCACHE=y && CACHEFILES=y)
+	default n
+	help
+	  EROFS is mounted from data blob files and implement on-demand read
+	  semantics through fscache.
+
+	  If unsure, say N.
diff --git a/fs/erofs/Makefile b/fs/erofs/Makefile
index 8a3317e38e5a..99bbc597a3e9 100644
--- a/fs/erofs/Makefile
+++ b/fs/erofs/Makefile
@@ -5,3 +5,4 @@ erofs-objs := super.o inode.o data.o namei.o dir.o utils.o pcpubuf.o sysfs.o
 erofs-$(CONFIG_EROFS_FS_XATTR) += xattr.o
 erofs-$(CONFIG_EROFS_FS_ZIP) += decompressor.o zmap.o zdata.o
 erofs-$(CONFIG_EROFS_FS_ZIP_LZMA) += decompressor_lzma.o
+erofs-$(CONFIG_EROFS_FS_ONDEMAND) += fscache.o
diff --git a/fs/erofs/fscache.c b/fs/erofs/fscache.c
new file mode 100644
index 000000000000..08cf570a0810
--- /dev/null
+++ b/fs/erofs/fscache.c
@@ -0,0 +1,22 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2022, Alibaba Cloud
+ */
+#include <linux/fscache.h>
+#include "internal.h"
+
+static struct fscache_volume *volume;
+
+int __init erofs_init_fscache(void)
+{
+	volume = fscache_acquire_volume("erofs", NULL, NULL, 0);
+	if (!volume)
+		return -EINVAL;
+
+	return 0;
+}
+
+void erofs_exit_fscache(void)
+{
+	fscache_relinquish_volume(volume, NULL, false);
+}
diff --git a/fs/erofs/internal.h b/fs/erofs/internal.h
index 1486e2573667..45b8b0dd8a27 100644
--- a/fs/erofs/internal.h
+++ b/fs/erofs/internal.h
@@ -616,6 +616,15 @@ static inline int z_erofs_load_lzma_config(struct super_block *sb,
 }
 #endif	/* !CONFIG_EROFS_FS_ZIP */
 
+/* fscache.c */
+#ifdef CONFIG_EROFS_FS_ONDEMAND
+int erofs_init_fscache(void);
+void erofs_exit_fscache(void);
+#else
+static inline int erofs_init_fscache(void) { return 0; }
+static inline void erofs_exit_fscache(void) {}
+#endif
+
 #define EFSCORRUPTED    EUCLEAN         /* Filesystem is corrupted */
 
 #endif	/* __EROFS_INTERNAL_H */
diff --git a/fs/erofs/super.c b/fs/erofs/super.c
index 12755217631f..798f0c379e35 100644
--- a/fs/erofs/super.c
+++ b/fs/erofs/super.c
@@ -814,6 +814,10 @@ static int __init erofs_module_init(void)
 	if (err)
 		goto sysfs_err;
 
+	err = erofs_init_fscache();
+	if (err)
+		goto fscache_err;
+
 	err = register_filesystem(&erofs_fs_type);
 	if (err)
 		goto fs_err;
@@ -821,6 +825,8 @@ static int __init erofs_module_init(void)
 	return 0;
 
 fs_err:
+	erofs_exit_fscache();
+fscache_err:
 	erofs_exit_sysfs();
 sysfs_err:
 	z_erofs_exit_zip_subsystem();
@@ -841,6 +847,7 @@ static void __exit erofs_module_exit(void)
 	/* Ensure all RCU free inodes / pclusters are safe to be destroyed. */
 	rcu_barrier();
 
+	erofs_exit_fscache();
 	erofs_exit_sysfs();
 	z_erofs_exit_zip_subsystem();
 	z_erofs_lzma_exit();
-- 
2.27.0


  parent reply	other threads:[~2022-03-25 12:23 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-25 12:22 [PATCH v6 00/22] fscache,erofs: fscache-based on-demand read semantics Jeffle Xu
2022-03-25 12:22 ` [PATCH v6 01/22] fscache: export fscache_end_operation() Jeffle Xu
2022-03-25 12:22 ` [PATCH v6 02/22] cachefiles: extract write routine Jeffle Xu
2022-03-25 12:22 ` [PATCH v6 03/22] cachefiles: notify user daemon with anon_fd when looking up cookie Jeffle Xu
2022-03-25 20:52   ` kernel test robot
2022-03-25 23:25   ` kernel test robot
2022-03-29  6:14   ` [Linux-cachefs] " JeffleXu
2022-03-25 12:22 ` [PATCH v6 04/22] cachefiles: notify user daemon when withdrawing cookie Jeffle Xu
2022-03-25 12:22 ` [PATCH v6 05/22] cachefiles: implement on-demand read Jeffle Xu
2022-03-25 12:22 ` [PATCH v6 06/22] cachefiles: enable on-demand read mode Jeffle Xu
2022-03-25 12:22 ` [PATCH v6 07/22] cachefiles: document " Jeffle Xu
2022-03-25 12:22 ` [PATCH v6 08/22] erofs: use meta buffers for erofs_read_superblock() Jeffle Xu
2022-03-25 12:22 ` [PATCH v6 09/22] erofs: make erofs_map_blocks() generally available Jeffle Xu
2022-03-25 12:22 ` [PATCH v6 10/22] erofs: add mode checking helper Jeffle Xu
2022-03-28  2:42   ` Gao Xiang
2022-03-28  2:46     ` JeffleXu
2022-03-25 12:22 ` Jeffle Xu [this message]
2022-03-25 12:22 ` [PATCH v6 12/22] erofs: add cookie context helper functions Jeffle Xu
2022-03-25 13:41   ` Gao Xiang
2022-03-28  3:11     ` JeffleXu
2022-03-25 12:22 ` [PATCH v6 13/22] erofs: add anonymous inode managing page cache of blob file Jeffle Xu
2022-03-25 13:46   ` [Linux-cachefs] " Gao Xiang
2022-03-25 12:22 ` [PATCH v6 14/22] erofs: add erofs_fscache_read_folios() helper Jeffle Xu
2022-03-28  2:49   ` [Linux-cachefs] " Gao Xiang
2022-03-25 12:22 ` [PATCH v6 15/22] erofs: register cookie context for bootstrap blob Jeffle Xu
2022-03-28  3:04   ` Gao Xiang
2022-03-28  3:15     ` JeffleXu
2022-03-25 12:22 ` [PATCH v6 16/22] erofs: implement fscache-based metadata read Jeffle Xu
2022-03-25 12:22 ` [PATCH v6 17/22] erofs: implement fscache-based data read for non-inline layout Jeffle Xu
2022-03-28  3:14   ` Gao Xiang
2022-03-25 12:22 ` [PATCH v6 18/22] erofs: implement fscache-based data read for inline layout Jeffle Xu
2022-03-25 12:22 ` [PATCH v6 19/22] erofs: register cookie context for data blobs Jeffle Xu
2022-03-28  9:48   ` Gao Xiang
2022-03-25 12:22 ` [PATCH v6 20/22] erofs: implement fscache-based data read " Jeffle Xu
2022-03-28  9:50   ` Gao Xiang
2022-03-25 12:22 ` [PATCH v6 21/22] erofs: implement fscache-based data readahead Jeffle Xu
2022-03-28 10:55   ` Gao Xiang
2022-03-25 12:22 ` [PATCH v6 22/22] erofs: add 'tag' mount option Jeffle Xu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220325122223.102958-12-jefflexu@linux.alibaba.com \
    --to=jefflexu@linux.alibaba.com \
    --cc=bo.liu@linux.alibaba.com \
    --cc=chao@kernel.org \
    --cc=dhowells@redhat.com \
    --cc=eguan@linux.alibaba.com \
    --cc=fannaihao@baidu.com \
    --cc=gerry@linux.alibaba.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=joseph.qi@linux.alibaba.com \
    --cc=linux-cachefs@redhat.com \
    --cc=linux-erofs@lists.ozlabs.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=luodaowen.backend@bytedance.com \
    --cc=tao.peng@linux.alibaba.com \
    --cc=tianzichen@kuaishou.com \
    --cc=torvalds@linux-foundation.org \
    --cc=willy@infradead.org \
    --cc=xiang@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®