[PATCH 0/4] squashfs: reduce memory usage and update docs

[PATCH 0/4] squashfs: reduce memory usage and update docs

[Phillip Lougher]

Hi,

This patch-set reduces the amount of memory that Squashfs uses
when CONFIG_FILE_DIRECT is configured, and updates various
out of date information in the documentation and Kconfig.

Phillip Lougher (4):
      squashfs: make squashfs_cache_init() return ERR_PTR(-ENOMEM)
      squashfs: don't allocate read_page cache if SQUASHFS_FILE_DIRECT configured
      Documentation: update the Squashfs filesystem documentation
      squashfs: update Kconfig information

 Documentation/filesystems/squashfs.rst | 14 ++++++--------
 fs/squashfs/Kconfig                    |  6 +++---
 fs/squashfs/cache.c                    | 10 +++++++---
 fs/squashfs/squashfs.h                 |  6 ++++++
 fs/squashfs/super.c                    | 19 +++++++++++--------
 5 files changed, 33 insertions(+), 22 deletions(-)

 Thanks

 Phillip

[PATCH 1/4] squashfs: make squashfs_cache_init() return ERR_PTR(-ENOMEM)

[Phillip Lougher]

Make squashfs_cache_init() return an ERR_PTR(-ENOMEM) on failure rather
than NULL.

This tidies up some calling code, but, it also allows NULL to be returned
as a valid result when a cache hasn't be allocated.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
---
 fs/squashfs/cache.c | 10 +++++++---
 fs/squashfs/super.c | 17 ++++++++++-------
 2 files changed, 17 insertions(+), 10 deletions(-)

diff --git a/fs/squashfs/cache.c b/fs/squashfs/cache.c
index 5062326d0efb..4db0d2b0aab8 100644
--- a/fs/squashfs/cache.c
+++ b/fs/squashfs/cache.c
@@ -224,11 +224,15 @@ struct squashfs_cache *squashfs_cache_init(char *name, int entries,
 	int block_size)
 {
 	int i, j;
-	struct squashfs_cache *cache = kzalloc(sizeof(*cache), GFP_KERNEL);
+	struct squashfs_cache *cache;
 
+	if (entries == 0)
+		return NULL;
+
+	cache = kzalloc(sizeof(*cache), GFP_KERNEL);
 	if (cache == NULL) {
 		ERROR("Failed to allocate %s cache\n", name);
-		return NULL;
+		return ERR_PTR(-ENOMEM);
 	}
 
 	cache->entry = kcalloc(entries, sizeof(*(cache->entry)), GFP_KERNEL);
@@ -281,7 +285,7 @@ struct squashfs_cache *squashfs_cache_init(char *name, int entries,
 
 cleanup:
 	squashfs_cache_delete(cache);
-	return NULL;
+	return ERR_PTR(-ENOMEM);
 }
 
 
diff --git a/fs/squashfs/super.c b/fs/squashfs/super.c
index 269c6d61bc29..fedae8dbc5de 100644
--- a/fs/squashfs/super.c
+++ b/fs/squashfs/super.c
@@ -314,26 +314,29 @@ static int squashfs_fill_super(struct super_block *sb, struct fs_context *fc)
 	sb->s_flags |= SB_RDONLY;
 	sb->s_op = &squashfs_super_ops;
 
-	err = -ENOMEM;
-
 	msblk->block_cache = squashfs_cache_init("metadata",
 			SQUASHFS_CACHED_BLKS, SQUASHFS_METADATA_SIZE);
-	if (msblk->block_cache == NULL)
+	if (IS_ERR(msblk->block_cache)) {
+		err = PTR_ERR(msblk->block_cache);
 		goto failed_mount;
+	}
 
 	/* Allocate read_page block */
 	msblk->read_page = squashfs_cache_init("data",
 		msblk->max_thread_num, msblk->block_size);
-	if (msblk->read_page == NULL) {
+	if (IS_ERR(msblk->read_page)) {
 		errorf(fc, "Failed to allocate read_page block");
+		err = PTR_ERR(msblk->read_page);
 		goto failed_mount;
 	}
 
 	if (msblk->devblksize == PAGE_SIZE) {
 		struct inode *cache = new_inode(sb);
 
-		if (cache == NULL)
+		if (cache == NULL) {
+			err = -ENOMEM;
 			goto failed_mount;
+		}
 
 		set_nlink(cache, 1);
 		cache->i_size = OFFSET_MAX;
@@ -406,8 +409,8 @@ static int squashfs_fill_super(struct super_block *sb, struct fs_context *fc)
 
 	msblk->fragment_cache = squashfs_cache_init("fragment",
 		min(SQUASHFS_CACHED_FRAGMENTS, fragments), msblk->block_size);
-	if (msblk->fragment_cache == NULL) {
-		err = -ENOMEM;
+	if (IS_ERR(msblk->fragment_cache)) {
+		err = PTR_ERR(msblk->fragment_cache);
 		goto failed_mount;
 	}
 
-- 
2.39.5

[PATCH 2/4] squashfs: don't allocate read_page cache if SQUASHFS_FILE_DIRECT configured

[Phillip Lougher]

If Squashfs has been configured to directly read datablocks into the
page cache (SQUASHFS_FILE_DIRECT), then the read_page cache is unnecessary.

This improvement is due to the following two commits, which added the
ability to read datablocks into the page cache when pages were
missing, enabling the fallback which used an intermediate buffer
to be removed.

commit f268eedddf359 ("squashfs: extend "page actor" to handle missing pages")
commit 1bb1a07afad97 ("squashfs: don't use intermediate buffer if pages missing")

This reduces the amount of memory used when mounting a filesystem by
block_size * maximum number of threads.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
---
 fs/squashfs/squashfs.h | 6 ++++++
 fs/squashfs/super.c    | 2 +-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/fs/squashfs/squashfs.h b/fs/squashfs/squashfs.h
index 37f3518a804a..218868b20f16 100644
--- a/fs/squashfs/squashfs.h
+++ b/fs/squashfs/squashfs.h
@@ -14,6 +14,12 @@
 
 #define WARNING(s, args...)	pr_warn("SQUASHFS: "s, ## args)
 
+#ifdef CONFIG_SQUASHFS_FILE_CACHE
+#define SQUASHFS_READ_PAGES msblk->max_thread_num
+#else
+#define SQUASHFS_READ_PAGES 0
+#endif
+
 /* block.c */
 extern int squashfs_read_data(struct super_block *, u64, int, u64 *,
 				struct squashfs_page_actor *);
diff --git a/fs/squashfs/super.c b/fs/squashfs/super.c
index fedae8dbc5de..67c55fe32ce8 100644
--- a/fs/squashfs/super.c
+++ b/fs/squashfs/super.c
@@ -323,7 +323,7 @@ static int squashfs_fill_super(struct super_block *sb, struct fs_context *fc)
 
 	/* Allocate read_page block */
 	msblk->read_page = squashfs_cache_init("data",
-		msblk->max_thread_num, msblk->block_size);
+		SQUASHFS_READ_PAGES, msblk->block_size);
 	if (IS_ERR(msblk->read_page)) {
 		errorf(fc, "Failed to allocate read_page block");
 		err = PTR_ERR(msblk->read_page);
-- 
2.39.5

[PATCH 3/4] Documentation: update the Squashfs filesystem documentation

[Phillip Lougher]

This patch updates the following which are out of date.

- Zstd has been added to the compression algorithms supported.
- The filesystem mailing list (for the kernel code) is changed to
  linux-fsdevel rather than the now very little used Sourceforge
  mailing list.
- The Squashfs website has been changed to the Squashfs-tools github
  repository.
- The fact that Squashfs-tools is likely packaged by the linux
  distribution is mentioned.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
---
 Documentation/filesystems/squashfs.rst | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

diff --git a/Documentation/filesystems/squashfs.rst b/Documentation/filesystems/squashfs.rst
index 4af8d6207509..45653b3228f9 100644
--- a/Documentation/filesystems/squashfs.rst
+++ b/Documentation/filesystems/squashfs.rst
@@ -6,7 +6,7 @@ Squashfs 4.0 Filesystem
 
 Squashfs is a compressed read-only filesystem for Linux.
 
-It uses zlib, lz4, lzo, or xz compression to compress files, inodes and
+It uses zlib, lz4, lzo, xz or zstd compression to compress files, inodes and
 directories.  Inodes in the system are very small and all blocks are packed to
 minimise data overhead. Block sizes greater than 4K are supported up to a
 maximum of 1Mbytes (default block size 128K).
@@ -16,8 +16,8 @@ use (i.e. in cases where a .tar.gz file may be used), and in constrained
 block device/memory systems (e.g. embedded systems) where low overhead is
 needed.
 
-Mailing list: squashfs-devel@lists.sourceforge.net
-Web site: www.squashfs.org
+Mailing list (kernel code): linux-fsdevel@vger.kernel.org
+Web site: github.com/plougher/squashfs-tools
 
 1. Filesystem Features
 ----------------------
@@ -58,11 +58,9 @@ inodes have different sizes).
 
 As squashfs is a read-only filesystem, the mksquashfs program must be used to
 create populated squashfs filesystems.  This and other squashfs utilities
-can be obtained from http://www.squashfs.org.  Usage instructions can be
-obtained from this site also.
-
-The squashfs-tools development tree is now located on kernel.org
-	git://git.kernel.org/pub/scm/fs/squashfs/squashfs-tools.git
+are very likely packaged by your linux distribution (called squashfs-tools).
+The source code can be obtained from github.com/plougher/squashfs-tools.
+Usage instructions can also be obtained from this site.
 
 2.1 Mount options
 -----------------
-- 
2.39.5

[PATCH 4/4] squashfs: update Kconfig information

[Phillip Lougher]

Update the compression algorithms supported, and
the Squashfs website location.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
---
 fs/squashfs/Kconfig | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/fs/squashfs/Kconfig b/fs/squashfs/Kconfig
index 60fc98bdf421..b1091e70434a 100644
--- a/fs/squashfs/Kconfig
+++ b/fs/squashfs/Kconfig
@@ -5,8 +5,8 @@ config SQUASHFS
 	help
 	  Saying Y here includes support for SquashFS 4.0 (a Compressed
 	  Read-Only File System).  Squashfs is a highly compressed read-only
-	  filesystem for Linux.  It uses zlib, lzo or xz compression to
-	  compress both files, inodes and directories.  Inodes in the system
+	  filesystem for Linux.  It uses zlib, lz4, lzo, xz or zstd compression
+	  to compress both files, inodes and directories.  Inodes in the system
 	  are very small and all blocks are packed to minimise data overhead.
 	  Block sizes greater than 4K are supported up to a maximum of 1 Mbytes
 	  (default block size 128K).  SquashFS 4.0 supports 64 bit filesystems
@@ -16,7 +16,7 @@ config SQUASHFS
 	  Squashfs is intended for general read-only filesystem use, for
 	  archival use (i.e. in cases where a .tar.gz file may be used), and in
 	  embedded systems where low overhead is needed.  Further information
-	  and tools are available from http://squashfs.sourceforge.net.
+	  and tools are available from github.com/plougher/squashfs-tools.
 
 	  If you want to compile this as a module ( = code which can be
 	  inserted in and removed from the running kernel whenever you want),
-- 
2.39.5