From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6843EC433E0 for ; Tue, 16 Feb 2021 11:05:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3515D64DF0 for ; Tue, 16 Feb 2021 11:05:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230239AbhBPLFc (ORCPT ); Tue, 16 Feb 2021 06:05:32 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:53290 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230233AbhBPLDH (ORCPT ); Tue, 16 Feb 2021 06:03:07 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1613473282; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UjPevUc/f1h27YC/iWCBse0mv72F3ucIKOdIwEX5LG8=; b=AvkgOPp+DBPc0FrKWJwDubOtuTW7N7QZR8zo7kgmXnXHIMctpATd5nCqJxHvmK79+qP+s2 P2dvCOwOuspzHa8D9x+3Z2BShEWiivzSmv4nj2i+hOXHyXL4DuFXTaDgfZ1qTrAhE669IH yYZhIxt6TEZ6qc+Gby+0O1WX0NUh1rg= Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-239-hWK5CwlLNYeDwW4p53IWdw-1; Tue, 16 Feb 2021 06:01:21 -0500 X-MC-Unique: hWK5CwlLNYeDwW4p53IWdw-1 Received: by mail-qk1-f197.google.com with SMTP id f140so7529462qke.0 for ; Tue, 16 Feb 2021 03:01:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=UjPevUc/f1h27YC/iWCBse0mv72F3ucIKOdIwEX5LG8=; b=CPWjA2JyZBVme+34WPADdUqPbN1Xpmymfcih5vjqQoQacsqjnAnNRBGbzHSWHd4+41 NYuq8ReXxqNbMZdCGMYAJM/7TcNYRiL8liFrJ2BU69wHb1DisbenQ6nNNd7rPb2g5uys tnEga5Y5clmWanZPBWnFeXclcNjWloh7HBApA48rfpY12yQGwrwgRHQWZ6W0IZYYN8qM nbLQCHaTCGPWwknmBeKvC7ow8duQxrZm/BHucNMRNYZpZP8p0oZTDsp8fPwy17t5FrxC 79NGT4KMjTrGcs9sfTQThU+twmcIDw73WpgsYY4aE0PF6Q7KWfikeQVB3Ac3K1Y6c0VG 71IQ== X-Gm-Message-State: AOAM532s/Q6beCExhC+6f8c6X0tdd0qZ491mgQzQQ/HuduCEE6dyyvRF v+LfwQU5bu+RixpWz42wf5Qqoncbce43vncOxbKCzwqPkmgzo7LecsuSbF7EVLn7FtNIUCMejYl e3VirSsBbwGnlzc6fN7vLUkbl X-Received: by 2002:a05:6214:292:: with SMTP id l18mr19206676qvv.5.1613473277870; Tue, 16 Feb 2021 03:01:17 -0800 (PST) X-Google-Smtp-Source: ABdhPJwfxkMdLP8GICFrYv5qcHZzcwXchEHaGAgSDG1h4XsMOzzlGSXOw3kqQRk/cyROnYM/B9O9Vw== X-Received: by 2002:a05:6214:292:: with SMTP id l18mr19206653qvv.5.1613473277670; Tue, 16 Feb 2021 03:01:17 -0800 (PST) Received: from tleilax.poochiereds.net (68-20-15-154.lightspeed.rlghnc.sbcglobal.net. [68.20.15.154]) by smtp.gmail.com with ESMTPSA id n5sm13100571qtd.5.2021.02.16.03.01.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Feb 2021 03:01:17 -0800 (PST) Message-ID: <752a8c91b7a418fa52cb8a8f28cb30155a574904.camel@redhat.com> Subject: Re: [PATCH 00/33] Network fs helper library & fscache kiocb API [ver #3] From: Jeff Layton To: Steve French Cc: David Howells , Trond Myklebust , Anna Schumaker , Steve French , Dominique Martinet , CIFS , ceph-devel@vger.kernel.org, Matthew Wilcox , linux-cachefs@redhat.com, Alexander Viro , linux-mm , linux-afs@lists.infradead.org, v9fs-developer@lists.sourceforge.net, Christoph Hellwig , linux-fsdevel , linux-nfs , Linus Torvalds , David Wysochanski , LKML Date: Tue, 16 Feb 2021 06:01:16 -0500 In-Reply-To: References: <161340385320.1303470.2392622971006879777.stgit@warthog.procyon.org.uk> <9e49f96cd80eaf9c8ed267a7fbbcb4c6467ee790.camel@redhat.com> Content-Type: text/plain; charset="ISO-8859-15" User-Agent: Evolution 3.38.4 (3.38.4-1.fc33) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 2021-02-15 at 18:40 -0600, Steve French wrote: > Jeff, > What are the performance differences you are seeing (positive or > negative) with ceph and netfs, especially with simple examples like > file copy or grep of large files? > > It could be good if netfs simplifies the problem experienced by > network filesystems on Linux with readahead on large sequential reads > - where we don't get as much parallelism due to only having one > readahead request at a time (thus in many cases there is 'dead time' > on either the network or the file server while waiting for the next > readpages request to be issued). This can be a significant > performance problem for current readpages when network latency is long > (or e.g. in cases when network encryption is enabled, and hardware > offload not available so time consuming on the server or client to > encrypt the packet). > > Do you see netfs much faster than currentreadpages for ceph? > > Have you been able to get much benefit from throttling readahead with > ceph from the current netfs approach for clamping i/o? > I haven't seen big performance differences at all with this set. It's pretty much a wash, and it doesn't seem to change how the I/Os are ultimately driven on the wire. For instance, the clamp_length op basically just mirrors what ceph does today -- it ensures that the length of the I/O can't go past the end of the current object. The main benefits are that we get a large swath of readpage, readpages amd write_begin code out of ceph altogether. All of the netfs's need to gather and vet pages for I/O, etc. Most of that doesn't have anything to do with the filesystem itself. By offloading that into the netfs lib, most of that is taken care of for us and we don't need to bother with doing that ourselves. -- Jeff Layton