From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BDFD1C001DC for ; Tue, 11 Jul 2023 18:52:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231127AbjGKSwo (ORCPT ); Tue, 11 Jul 2023 14:52:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44970 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230248AbjGKSwl (ORCPT ); Tue, 11 Jul 2023 14:52:41 -0400 Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [IPv6:2607:f8b0:4864:20::733]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C59CE1BB for ; Tue, 11 Jul 2023 11:52:39 -0700 (PDT) Received: by mail-qk1-x733.google.com with SMTP id af79cd13be357-7659dc74da1so577919285a.3 for ; Tue, 11 Jul 2023 11:52:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1689101559; x=1691693559; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=csq2hP6IGrsJizVBuDWOWzciwuvySb83mfb1bAUQeAw=; b=O6yj1kfBmt/f46fGl5YF3ZVJn3iClEXfkHrq1NBwzA3zeFlPKxh61DLstld0JhkzXq h9GCIk46LJKP5FEMhlmgxGdR6DZHqGE8zkEPe0m8HwO3oYCvuIT+h216JZzEKzRT45VH fihme1JeEVJ5o7rLj+sp0HM+VyiPgOa+4UWUCJ8nczCJMAvSl0mDXfZE65/pQiDQgR46 KXQ8MO1UkrrB/BNF3dM4Tr7JLFvYzyjqwWXAqiIjG0pmZzaxZyvvqaOWcuYKyh4LOvP4 IvsRAbNbBcQCuYjr3QJF3eqkPZPCTeLSQBqYAlk4Rf6atATMuVoe1YiRvI7PbrsyIUPo WEyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689101559; x=1691693559; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=csq2hP6IGrsJizVBuDWOWzciwuvySb83mfb1bAUQeAw=; b=GHlfxRJPa9MBahjBuB1aZJGm2oCw8j8TsNcm6xjHZcbJvPH3S4yzotXqBgzSmP+BLf 29mqTYl+rqmua+S02Q/keStLV/klsb0NzC/BVKBrFM+rrQfXT5bup6LSs3MP2SFsoB2o Ksuy6wIexJJwrrIn1GP6vOXcxO+5AaewEd4+qIvGw+3G7Ve/M94ZlDYeysoLfqRmInea 1NtYpmkpLj3ge8ZiyJf/TXVkmNJRyiDEm6jEYbaX2SmuMFRHkJ5h6mBwRQIzLC78pQol igKJBJnajHw0JFm12u3YTqj64Rb4BQ4rDfHD0eQVznZl9YJSnisRDwNk5ADUfm55ra+q 4X/Q== X-Gm-Message-State: ABy/qLYYliFXnOD/6Rfa67SeoUNK7i9AyN4xKPsoz2Pv8Yk5dcE2qsU9 IWNYEePaqDWkjb5XPdggw2nYuw== X-Google-Smtp-Source: APBJJlGz3e2wm1ItETv6Tpn24AKM42/GznMyImhnBUCl/FD/+Lt6lzms0CFX41BO0AXGLY19Tbl9Zw== X-Received: by 2002:a05:620a:2904:b0:75d:5640:22e7 with SMTP id m4-20020a05620a290400b0075d564022e7mr22796538qkp.55.1689101558954; Tue, 11 Jul 2023 11:52:38 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-25-194.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.25.194]) by smtp.gmail.com with ESMTPSA id j3-20020a05620a146300b00765ab6d3e81sm1269520qkl.122.2023.07.11.11.52.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jul 2023 11:52:38 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.95) (envelope-from ) id 1qJITV-000JMj-F8; Tue, 11 Jul 2023 15:52:37 -0300 Date: Tue, 11 Jul 2023 15:52:37 -0300 From: Jason Gunthorpe To: Jakub Kicinski Cc: Christoph Hellwig , Mina Almasry , John Hubbard , Dan Williams , David Ahern , Jesper Dangaard Brouer , brouer@redhat.com, Alexander Duyck , Yunsheng Lin , davem@davemloft.net, pabeni@redhat.com, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Lorenzo Bianconi , Yisen Zhuang , Salil Mehta , Eric Dumazet , Sunil Goutham , Geetha sowjanya , Subbaraya Sundeep , hariprasad , Saeed Mahameed , Leon Romanovsky , Felix Fietkau , Ryder Lee , Shayne Chen , Sean Wang , Kalle Valo , Matthias Brugger , AngeloGioacchino Del Regno , Jesper Dangaard Brouer , Ilias Apalodimas , linux-rdma@vger.kernel.org, linux-wireless@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org, Jonathan Lemon Subject: Re: Memory providers multiplexing (Was: [PATCH net-next v4 4/5] page_pool: remove PP_FLAG_PAGE_FRAG flag) Message-ID: References: <20230711042708.GA18658@lst.de> <20230710215906.49514550@kernel.org> <20230711050445.GA19323@lst.de> <20230711090047.37d7fe06@kernel.org> <20230711100636.63b0a88a@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230711100636.63b0a88a@kernel.org> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jul 11, 2023 at 10:06:36AM -0700, Jakub Kicinski wrote: > Now we're getting into our favorite argument and completely > sidetracking the conversation, aren't we? :) And as usual > our ability to present facts is limited by various NDAs.. Yes, well, maybe I should stop taking the bait everytime you write "proprietary" :) > > We also have the roce support in the switch from all major > > switch vendors. > > By which you mean all major switch vendors should support basic RoCE > requirements. But most vendors will try to put special features into > their switches trying to make the full NIC + switch solution as sticky > as possible. Yep. At the high end open standards based ethernet has also notably "failed" as well. Every switch vendor now offers their own proprietary ecosystem on a whole bunch of different axis. They all present "ethernet" toward the host but the host often needs to work in a special way to really take full advantage of the proprietary fabric behaviors. > Last I checked every generation of HW from even a single vendor came out > with a new congestion control algorithm and add-ons. Probably, but I don't really view this as an IB or roce issue. Back in the day, there was "data center ethernet" which was a standardization effort to try and tame some of these problems. roce was imagined as an important workload over DCE, but the effort was ethernet focused and generic. Sadly DCE and successor standard based congestion mangement approaches did not work, or were "standardized" in a way that had a big hole that needed to be filled with proprietary algorithms. Eventualy the interest in standardization seems to have waned and several of the big network operators seem to be valuing their unique congestion management as a proprietary element. From a vendor perspective this is has turned into an interop train wreck. Sigh. roce is just highly sensitive to loss - which is managed in ethernet through congestion management. This is why you see roce and congestion management so tightly linked, and perhaps in some deployments becomes the motivating reason to look at congestion management. However, TCP under congestion management is also very interesting and is a motivation to deploy congestion management in its own right in some cases. Jason