From: Jeffle Xu <jefflexu@linux.alibaba.com>
To: dhowells@redhat.com, linux-cachefs@redhat.com, xiang@kernel.org,
chao@kernel.org, linux-erofs@lists.ozlabs.org
Cc: torvalds@linux-foundation.org, gregkh@linuxfoundation.org,
willy@infradead.org, linux-fsdevel@vger.kernel.org,
joseph.qi@linux.alibaba.com, bo.liu@linux.alibaba.com,
tao.peng@linux.alibaba.com, gerry@linux.alibaba.com,
eguan@linux.alibaba.com, linux-kernel@vger.kernel.org,
luodaowen.backend@bytedance.com
Subject: [PATCH v5 11/22] erofs: register global fscache volume
Date: Wed, 16 Mar 2022 21:17:12 +0800 [thread overview]
Message-ID: <20220316131723.111553-12-jefflexu@linux.alibaba.com> (raw)
In-Reply-To: <20220316131723.111553-1-jefflexu@linux.alibaba.com>
All erofs instances will share one global fscache volume.
In this using scenario, one erofs instance could be mounted from one (or
multiple) blob files instead of blkdev. The number of blob files that
each erofs instance could correspond to is limited, since these blob
files are quite large in size. For example, when used for container
image distribution, one erofs instance used for container image for
node.js will correspond to ~20 blob files in total. Thus in densely
employed environment, there could be as many as hundreds of containers
and thus thousands of fscache cookies under one fscache volume.
Then as for cachefiles backend, the hash table managing all cookies
under one volume contains 32K slots. Thus the hashing functionality shall
scale well in this case. Besides, cachefiles backend will scatter
backing files under 256 fan sub-directoris, and thus the scalability of
looking up backing files shall also not be an issue.
Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com>
---
fs/erofs/Makefile | 3 ++-
fs/erofs/fscache.c | 21 +++++++++++++++++++++
fs/erofs/internal.h | 5 +++++
fs/erofs/super.c | 7 +++++++
4 files changed, 35 insertions(+), 1 deletion(-)
create mode 100644 fs/erofs/fscache.c
diff --git a/fs/erofs/Makefile b/fs/erofs/Makefile
index 8a3317e38e5a..21999e8a4728 100644
--- a/fs/erofs/Makefile
+++ b/fs/erofs/Makefile
@@ -1,7 +1,8 @@
# SPDX-License-Identifier: GPL-2.0-only
obj-$(CONFIG_EROFS_FS) += erofs.o
-erofs-objs := super.o inode.o data.o namei.o dir.o utils.o pcpubuf.o sysfs.o
+erofs-objs := super.o inode.o data.o namei.o dir.o utils.o pcpubuf.o sysfs.o \
+ fscache.o
erofs-$(CONFIG_EROFS_FS_XATTR) += xattr.o
erofs-$(CONFIG_EROFS_FS_ZIP) += decompressor.o zmap.o zdata.o
erofs-$(CONFIG_EROFS_FS_ZIP_LZMA) += decompressor_lzma.o
diff --git a/fs/erofs/fscache.c b/fs/erofs/fscache.c
new file mode 100644
index 000000000000..9c32f42e1056
--- /dev/null
+++ b/fs/erofs/fscache.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2021, Alibaba Cloud
+ */
+#include "internal.h"
+
+static struct fscache_volume *volume;
+
+int __init erofs_init_fscache(void)
+{
+ volume = fscache_acquire_volume("erofs", NULL, NULL, 0);
+ if (!volume)
+ return -EINVAL;
+
+ return 0;
+}
+
+void erofs_exit_fscache(void)
+{
+ fscache_relinquish_volume(volume, NULL, false);
+}
diff --git a/fs/erofs/internal.h b/fs/erofs/internal.h
index f66af9ebda43..51fe5c2a419d 100644
--- a/fs/erofs/internal.h
+++ b/fs/erofs/internal.h
@@ -17,6 +17,7 @@
#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/iomap.h>
+#include <linux/fscache.h>
#include "erofs_fs.h"
/* redefine pr_fmt "erofs: " */
@@ -616,6 +617,10 @@ static inline int z_erofs_load_lzma_config(struct super_block *sb,
}
#endif /* !CONFIG_EROFS_FS_ZIP */
+/* fscache.c */
+int erofs_init_fscache(void);
+void erofs_exit_fscache(void);
+
#define EFSCORRUPTED EUCLEAN /* Filesystem is corrupted */
#endif /* __EROFS_INTERNAL_H */
diff --git a/fs/erofs/super.c b/fs/erofs/super.c
index 12755217631f..798f0c379e35 100644
--- a/fs/erofs/super.c
+++ b/fs/erofs/super.c
@@ -814,6 +814,10 @@ static int __init erofs_module_init(void)
if (err)
goto sysfs_err;
+ err = erofs_init_fscache();
+ if (err)
+ goto fscache_err;
+
err = register_filesystem(&erofs_fs_type);
if (err)
goto fs_err;
@@ -821,6 +825,8 @@ static int __init erofs_module_init(void)
return 0;
fs_err:
+ erofs_exit_fscache();
+fscache_err:
erofs_exit_sysfs();
sysfs_err:
z_erofs_exit_zip_subsystem();
@@ -841,6 +847,7 @@ static void __exit erofs_module_exit(void)
/* Ensure all RCU free inodes / pclusters are safe to be destroyed. */
rcu_barrier();
+ erofs_exit_fscache();
erofs_exit_sysfs();
z_erofs_exit_zip_subsystem();
z_erofs_lzma_exit();
--
2.27.0
next prev parent reply other threads:[~2022-03-16 13:18 UTC|newest]
Thread overview: 51+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-16 13:17 [PATCH v5 00/22] fscache,erofs: fscache-based on-demand read semantics Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 01/22] fscache: export fscache_end_operation() Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 02/22] cachefiles: extract write routine Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 03/22] cachefiles: introduce on-demand read mode Jeffle Xu
2022-03-21 13:40 ` Matthew Wilcox
2022-03-21 14:08 ` JeffleXu
2022-03-21 14:26 ` Matthew Wilcox
2022-03-21 15:18 ` JeffleXu
2022-03-21 15:21 ` Matthew Wilcox
2022-03-21 15:30 ` David Howells
2022-03-22 17:04 ` Matthew Wilcox
2022-03-23 5:32 ` JeffleXu
2022-03-21 14:14 ` David Howells
2022-03-21 14:20 ` [Linux-cachefs] " Gao Xiang
2022-03-16 13:17 ` [PATCH v5 04/22] cachefiles: notify user daemon with anon_fd when looking up cookie Jeffle Xu
2022-03-16 19:37 ` kernel test robot
2022-03-16 13:17 ` [PATCH v5 05/22] cachefiles: notify user daemon when withdrawing cookie Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 06/22] cachefiles: implement on-demand read Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 07/22] cachefiles: document on-demand read mode Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 08/22] erofs: use meta buffers for erofs_read_superblock() Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 09/22] erofs: make erofs_map_blocks() generally available Jeffle Xu
2022-03-17 5:35 ` [Linux-cachefs] " Gao Xiang
2022-03-16 13:17 ` [PATCH v5 10/22] erofs: add mode checking helper Jeffle Xu
2022-03-17 5:36 ` [Linux-cachefs] " Gao Xiang
2022-03-18 5:26 ` JeffleXu
2022-03-16 13:17 ` Jeffle Xu [this message]
2022-03-16 21:52 ` [PATCH v5 11/22] erofs: register global fscache volume kernel test robot
2022-03-17 1:49 ` kernel test robot
2022-03-16 13:17 ` [PATCH v5 12/22] erofs: add cookie context helper functions Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 13/22] erofs: add anonymous inode managing page cache of blob file Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 14/22] erofs: add erofs_fscache_read_pages() helper Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 15/22] erofs: register cookie context for bootstrap blob Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 16/22] erofs: implement fscache-based metadata read Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 17/22] erofs: implement fscache-based data read for non-inline layout Jeffle Xu
2022-03-17 6:18 ` [Linux-cachefs] " Gao Xiang
2022-03-18 5:29 ` JeffleXu
2022-03-16 13:17 ` [PATCH v5 18/22] erofs: implement fscache-based data read for inline layout Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 19/22] erofs: register cookie context for data blobs Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 20/22] erofs: implement fscache-based data read " Jeffle Xu
2022-03-16 13:17 ` [PATCH v5 21/22] erofs: implement fscache-based data readahead Jeffle Xu
2022-03-17 5:22 ` Gao Xiang
2022-03-18 5:41 ` JeffleXu
2022-03-16 13:17 ` [PATCH v5 22/22] erofs: add 'uuid' mount option Jeffle Xu
2022-03-21 13:34 ` [PATCH v5 03/22] cachefiles: introduce on-demand read mode David Howells
2022-03-21 14:16 ` JeffleXu
2022-03-21 14:01 ` [PATCH v5 04/22] cachefiles: notify user daemon with anon_fd when looking up cookie David Howells
2022-03-21 14:43 ` JeffleXu
2022-03-21 14:05 ` [PATCH v5 06/22] cachefiles: implement on-demand read David Howells
2022-03-21 14:51 ` JeffleXu
2022-03-21 14:20 ` [PATCH v5 05/22] cachefiles: notify user daemon when withdrawing cookie David Howells
2022-03-21 14:31 ` JeffleXu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220316131723.111553-12-jefflexu@linux.alibaba.com \
--to=jefflexu@linux.alibaba.com \
--cc=bo.liu@linux.alibaba.com \
--cc=chao@kernel.org \
--cc=dhowells@redhat.com \
--cc=eguan@linux.alibaba.com \
--cc=gerry@linux.alibaba.com \
--cc=gregkh@linuxfoundation.org \
--cc=joseph.qi@linux.alibaba.com \
--cc=linux-cachefs@redhat.com \
--cc=linux-erofs@lists.ozlabs.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=luodaowen.backend@bytedance.com \
--cc=tao.peng@linux.alibaba.com \
--cc=torvalds@linux-foundation.org \
--cc=willy@infradead.org \
--cc=xiang@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®