From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out30-99.freemail.mail.aliyun.com (out30-99.freemail.mail.aliyun.com [115.124.30.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3646185E65 for ; Mon, 2 Sep 2024 11:06:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.99 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725275193; cv=none; b=leX+iGArsRdli/Pwn/RFXymQk/OJ5QU2hLnU33Jb+nGYe0WlerAbMQ2R5nXDzyGPkkzPWL6TqROQgYNGGMGPtrnRkFxEXuM1uRUIepvWM5ZfNiEAMk3eE1VJDaxGtvAcAHOvvO00QGS75tEZnhPWzZMT1ySKWr6HlkhjZFmctII= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725275193; c=relaxed/simple; bh=A02DBi6wFd+WuUtyVc1xVQEelVSyLrudwFVIu3RYhF4=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=fUNz57HHvzuA1bucx7Y4ln81z2FrEIgGV3BGM3fPVWQ530VGHoKzzhrjnlgaQMU25lc4BRGrRzqWp3M88ghpF/4W1CbYmVLkGLnHhm5FaFqFUkKw8eEtRgxo32qE3mAmTmFiP7L1OUANxtdSLVWqcfDIT7Z1ihDpN8kiHncKeZs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=UbWscjbQ; arc=none smtp.client-ip=115.124.30.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="UbWscjbQ" DKIM-Signature:v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1725275187; h=From:To:Subject:Date:Message-ID:MIME-Version; bh=54uiycrpKCuIzqYqG1SzcrlXmsXcCqDK/gsnbUk+/kc=; b=UbWscjbQhWpuCOUxo2zVKDylny9RfBI8w2SzbhXR1z6Ng2KoKAt6AvuRUvaTZi8Y9GhCBgZuilSINFgf2dRsJh+qvLh1Nc5i2MEAM0j5x30z6GV/OBPoQwzTXUPxTmXJRJW1AmX9WCGVO+JyAmer0hmEGPjOV4bvx6ZmInu+f1I= Received: from localhost(mailfrom:hongzhen@linux.alibaba.com fp:SMTPD_---0WE7YYnS_1725275186) by smtp.aliyun-inc.com; Mon, 02 Sep 2024 19:06:27 +0800 From: Hongzhen Luo To: linux-erofs@lists.ozlabs.org Cc: lihongbo22@huawei.com, linux-kernel@vger.kernel.org, Hongzhen Luo Subject: [PATCH RFC v4 0/4] erofs: introduce page cache share feature Date: Mon, 2 Sep 2024 19:06:16 +0800 Message-ID: <20240902110620.2202586-1-hongzhen@linux.alibaba.com> X-Mailer: git-send-email 2.43.5 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit v4: Changes since v3: - Implemented .fadvise v3: https://lore.kernel.org/all/20240828111959.3677011-1-hongzhen@linux.alibaba.com/ v2: https://lore.kernel.org/all/20240731080704.678259-1-hongzhen@linux.alibaba.com/ v1: https://lore.kernel.org/all/20240722065355.1396365-1-hongzhen@linux.alibaba.com/ [Background] ================ Currently, reading files with different paths (or names) but the same content will consume multiple copies of the page cache, even if the content of these page caches is the same. For example, reading identical files (e.g., *.so files) from two different minor versions of container images will cost multiple copies of the same page cache, since different containers have different mount points. Therefore, sharing the page cache for files with the same content can save memory. [Implementation] ================ This introduces the page cache share feature in erofs. During the mkfs phase, the file content is hashed and the hash value is stored in the `trusted.erofs.fingerprint` extended attribute. Inodes of files with the same `trusted.erofs.fingerprint` are mapped to the same anonymous inode (indicated by the `ano_inode` field). When a read request occurs, the anonymous inode serves as a "container" whose page cache is shared. The actual operations involving the iomap are carried out by the original inode which is mapped to the anonymous inode. [Effect] ================ I conducted experiments on two aspects across two different minor versions of container images: 1. reading all files in two different minor versions of container images 2. run workloads or use the default entrypoint within the containers^[1] Below is the memory usage for reading all files in two different minor versions of container images: +-------------------+------------------+-------------+---------------+ | Image | Page Cache Share | Memory (MB) | Memory | | | | | Reduction (%) | +-------------------+------------------+-------------+---------------+ | | No | 241 | - | | redis +------------------+-------------+---------------+ | 7.2.4 & 7.2.5 | Yes | 163 | 33% | +-------------------+------------------+-------------+---------------+ | | No | 872 | - | | postgres +------------------+-------------+---------------+ | 16.1 & 16.2 | Yes | 630 | 28% | +-------------------+------------------+-------------+---------------+ | | No | 2771 | - | | tensorflow +------------------+-------------+---------------+ | 1.11.0 & 2.11.1 | Yes | 2340 | 16% | +-------------------+------------------+-------------+---------------+ | | No | 926 | - | | mysql +------------------+-------------+---------------+ | 8.0.11 & 8.0.12 | Yes | 735 | 21% | +-------------------+------------------+-------------+---------------+ | | No | 390 | - | | nginx +------------------+-------------+---------------+ | 7.2.4 & 7.2.5 | Yes | 219 | 44% | +-------------------+------------------+-------------+---------------+ | tomcat | No | 924 | - | | 10.1.25 & 10.1.26 +------------------+-------------+---------------+ | | Yes | 474 | 49% | +-------------------+------------------+-------------+---------------+ Additionally, the table below shows the runtime memory usage of the container: +-------------------+------------------+-------------+---------------+ | Image | Page Cache Share | Memory (MB) | Memory | | | | | Reduction (%) | +-------------------+------------------+-------------+---------------+ | | No | 35 | - | | redis +------------------+-------------+---------------+ | 7.2.4 & 7.2.5 | Yes | 28 | 20% | +-------------------+------------------+-------------+---------------+ | | No | 149 | - | | postgres +------------------+-------------+---------------+ | 16.1 & 16.2 | Yes | 95 | 37% | +-------------------+------------------+-------------+---------------+ | | No | 1028 | - | | tensorflow +------------------+-------------+---------------+ | 1.11.0 & 2.11.1 | Yes | 930 | 10% | +-------------------+------------------+-------------+---------------+ | | No | 155 | - | | mysql +------------------+-------------+---------------+ | 8.0.11 & 8.0.12 | Yes | 132 | 15% | +-------------------+------------------+-------------+---------------+ | | No | 25 | - | | nginx +------------------+-------------+---------------+ | 7.2.4 & 7.2.5 | Yes | 20 | 20% | +-------------------+------------------+-------------+---------------+ | tomcat | No | 186 | - | | 10.1.25 & 10.1.26 +------------------+-------------+---------------+ | | Yes | 98 | 48% | +-------------------+------------------+-------------+---------------+ It can be observed that when reading all the files in the image, the reduced memory usage varies from 16% to 49%, depending on the specific image. Additionally, the container's runtime memory usage reduction ranges from 10% to 48%. [1] Below are the workload for these images: - redis: redis-benchmark - postgres: sysbench - tensorflow: app.py of tensorflow.python.platform - mysql: sysbench - nginx: wrk - tomcat: default entrypoint Hongzhen Luo (4): erofs: move `struct erofs_anon_fs_type` to super.c erofs: introduce page cache share feature erofs: apply the page cache share feature erofs: introduce .fadvise for page cache share fs/erofs/Kconfig | 10 ++ fs/erofs/Makefile | 1 + fs/erofs/data.c | 68 ++++++++++- fs/erofs/fscache.c | 13 -- fs/erofs/inode.c | 12 ++ fs/erofs/internal.h | 11 ++ fs/erofs/pagecache_share.c | 239 +++++++++++++++++++++++++++++++++++++ fs/erofs/pagecache_share.h | 21 ++++ fs/erofs/super.c | 59 +++++++++ fs/erofs/zdata.c | 32 +++++ 10 files changed, 452 insertions(+), 14 deletions(-) create mode 100644 fs/erofs/pagecache_share.c create mode 100644 fs/erofs/pagecache_share.h -- 2.43.5