From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752345AbbERGYZ (ORCPT ); Mon, 18 May 2015 02:24:25 -0400 Received: from mail.kernel.org ([198.145.29.136]:34747 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751102AbbERGYP (ORCPT ); Mon, 18 May 2015 02:24:15 -0400 Date: Sun, 17 May 2015 23:24:12 -0700 From: Jaegeuk Kim To: Tom Marshall Cc: Dave Chinner , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net Subject: Re: [PATCH 03/18] f2fs crypto: declare some definitions for f2fs encryption feature Message-ID: <20150518062412.GA21419@jaegeuk-mac02.hsd1.ca.comcast.net> References: <1431145253-2019-1-git-send-email-jaegeuk@kernel.org> <1431145253-2019-3-git-send-email-jaegeuk@kernel.org> <20150513020208.GK15721@dastard> <20150513064802.GA48682@jaegeuk-mac02.hsd1.ca.comcast.net> <20150514003721.GN15721@dastard> <20150514015643.GA70216@jaegeuk-mac02> <5554D264.7070105@cyngn.com> <20150516011424.GB10530@jaegeuk-mac02.mot.com> <20150516044704.GA27800@eden.sea.cyngn.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150516044704.GA27800@eden.sea.cyngn.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, May 15, 2015 at 09:47:05PM -0700, Tom Marshall wrote: > On Fri, May 15, 2015 at 06:14:24PM -0700, Jaegeuk Kim wrote: > > On Thu, May 14, 2015 at 09:50:44AM -0700, Tom Marshall wrote: > > > Please keep in mind that I'm also working on transparent > > > compression. I'm watching this thread closely so that I can > > > implement a compression library alongside the crypto library. If > > > there is any interest or benefit, I would be glad to work together > > > so that the two can be done cooperatively at the same time. > > > > I can't imagine quickly how compression code can be shared with crypto. > > The basic approach for compression would be that X pages can be compressed into > > small number of pages, Y, which can be a X to Y mapping. > > But, this per-file encryption supports only 1 to 1 4KB mapping, so that it could > > be quite a simple implementation. > > No, I don't intend to share actual code with crypto -- at least not much. > I'm more interested in looking at how the crypto layer is implemented to > give me clues about how to implement a compression layer. Ok, I see. Currently, I've been writing up fs/crypto having shared crypto codes between ext4 and f2fs; I refactored existing codes a little bit though. I'm approaching to introduce a fscypt_operations for each filesystems like this. struct fscrypt_operations { int (*get_context)(struct inode *, void *, size_t, void *); int (*set_context)(struct inode *, const void *, size_t, int, void *); int (*prepare_new_context)(struct inode *); bool (*is_encrypted)(struct inode *); bool (*empty_dir)(struct inode *); unsigned (*max_namelen)(struct inode *); }; And, the following two basic functions will be used by filesystems: fscrypt_encrypt_page() and fscrypt_decrypt_page(). > > > Could you elaborate on your approach or design? Or, codes? > > Whatever, IMO, it needs to implement it by any filesystem first. > > I don't really have any working code yet. I will probably get to that in > the coming few weeks. Right now I'm still working with the ugly VFS > stacking implementation that I posted initially. > > The thing that I have done is dismissed the standard compression framing > formats. > > zlib (and gzip) are designed for streaming and it is quite difficult to > implement random access on it. See the example code in the zlib source, > zran.c. It's not really tenable because 32kb of prior data is required to > be kept as priming information. Even doing fully encapsulated blocks with > Z_FINISH, there is still no way to skip over data without decompressing it > first to build an index. > > lz4 is somewhat better in that blocks are self contained. But block lengths > must be read sequentially. This means that reading an arbitrary position in > a file requires a proportional number of reads to find the desired block. > > So, I am working with a simple framing format that I threw together. The > header has a compression method (zlib or lz4), block size, original input > size, and a block map. Thank you for sharing the approach, and it makes sense to improve random read performance. Thanks,