From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932663AbbCYQF2 (ORCPT ); Wed, 25 Mar 2015 12:05:28 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53465 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932576AbbCYQFX (ORCPT ); Wed, 25 Mar 2015 12:05:23 -0400 From: Jeff Moyer To: Jens Axboe Cc: , , , Subject: Re: [PATCH RFC v2] Support for write stream IDs References: <1427296070-8472-1-git-send-email-axboe@fb.com> X-PGP-KeyID: 1F78E1B4 X-PGP-CertKey: F6FE 280D 8293 F72C 65FD 5A58 1FF8 A7CA 1F78 E1B4 X-PCLoadLetter: What the f**k does that mean? Date: Wed, 25 Mar 2015 12:05:21 -0400 In-Reply-To: <1427296070-8472-1-git-send-email-axboe@fb.com> (Jens Axboe's message of "Wed, 25 Mar 2015 09:07:43 -0600") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Jens Axboe writes: > One of the things that exacerbates write amplification on flash > based devices is that fact that data with different lifetimes get > grouped together on media. Currently we have no interface that > applications can use to separate different types of writes. This > patch set adds support for that. > > The kernel has no knowledge of what stream ID is what. The idea is > that writes with identical stream IDs have similar life times, not > that stream ID 'X' has a shorter lifetime than stream ID 'X+1'. And presumably the device also has no knowledge of what stream ID is what, right? > There are basically two interfaces that could be used for this. One > is fcntl, the other is fadvise. This patchset uses fadvise, with a > new POSIX_FADV_STREAMID hint. The 'offset' field is used to pass > the relevant stream ID. Switching to fcntl (with a SET/GET_STREAMID) > would be trivial. > > The patchset wires up the block parts, adds buffered and O_DIRECT > support, and modifies btrfs/xfs too. It should be trivial to extend > this to all other file systems, I just used xfs and btrfs for testing. > > No block drivers are wired up yet. Patches are against current -git. This proposal leaves lot to the reviewer's imagination. Is there any research in this area you can point to? At a high level, are you sure you've got the right interface? I would think data lifetime would be tied to the file. If that's the case, it might be possible to not export this to userspace at all, and simply make it work under the covers. After all, what prevents multiple applications from using the same stream id at the same time? Cheers, Jeff