From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF99CC64EB1 for ; Thu, 6 Dec 2018 23:49:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AB59420868 for ; Thu, 6 Dec 2018 23:49:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AB59420868 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=deltatee.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726314AbeLFXtS (ORCPT ); Thu, 6 Dec 2018 18:49:18 -0500 Received: from ale.deltatee.com ([207.54.116.67]:47318 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726219AbeLFXtS (ORCPT ); Thu, 6 Dec 2018 18:49:18 -0500 Received: from guinness.priv.deltatee.com ([172.16.1.162]) by ale.deltatee.com with esmtp (Exim 4.89) (envelope-from ) id 1gV3O8-0006ni-SI; Thu, 06 Dec 2018 16:49:01 -0700 To: Dave Hansen , Jerome Glisse Cc: linux-mm@kvack.org, Andrew Morton , linux-kernel@vger.kernel.org, "Rafael J . Wysocki" , Matthew Wilcox , Ross Zwisler , Keith Busch , Dan Williams , Haggai Eran , Balbir Singh , "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Felix Kuehling , Philip Yang , =?UTF-8?Q?Christian_K=c3=b6nig?= , Paul Blinzer , John Hubbard , Ralph Campbell , Michal Hocko , Jonathan Cameron , Mark Hairgrove , Vivek Kini , Mel Gorman , Dave Airlie , Ben Skeggs , Andrea Arcangeli , Rik van Riel , Ben Woodard , linux-acpi@vger.kernel.org References: <20181205001544.GR2937@redhat.com> <42006749-7912-1e97-8ccd-945e82cebdde@intel.com> <20181205021334.GB3045@redhat.com> <20181205175357.GG3536@redhat.com> <20181206192050.GC3544@redhat.com> <20181206223935.GG3544@redhat.com> <935fc14d-91f2-bc2a-f8b5-665e4145e148@deltatee.com> <5e6c87d5-e4ef-12e7-32bf-c163f7ff58d7@intel.com> From: Logan Gunthorpe Message-ID: Date: Thu, 6 Dec 2018 16:48:57 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.3.0 MIME-Version: 1.0 In-Reply-To: <5e6c87d5-e4ef-12e7-32bf-c163f7ff58d7@intel.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-CA Content-Transfer-Encoding: 7bit X-SA-Exim-Connect-IP: 172.16.1.162 X-SA-Exim-Rcpt-To: linux-acpi@vger.kernel.org, woodard@redhat.com, riel@surriel.com, aarcange@redhat.com, bskeggs@redhat.com, airlied@redhat.com, mgorman@techsingularity.net, vkini@nvidia.com, mhairgrove@nvidia.com, jonathan.cameron@huawei.com, mhocko@kernel.org, rcampbell@nvidia.com, jhubbard@nvidia.com, Paul.Blinzer@amd.com, christian.koenig@amd.com, Philip.Yang@amd.com, felix.kuehling@amd.com, benh@kernel.crashing.org, aneesh.kumar@linux.ibm.com, bsingharora@gmail.com, haggaie@mellanox.com, dan.j.williams@intel.com, keith.busch@intel.com, ross.zwisler@linux.intel.com, willy@infradead.org, rafael@kernel.org, linux-kernel@vger.kernel.org, akpm@linux-foundation.org, linux-mm@kvack.org, jglisse@redhat.com, dave.hansen@intel.com X-SA-Exim-Mail-From: logang@deltatee.com Subject: Re: [RFC PATCH 00/14] Heterogeneous Memory System (HMS) and hbind() X-SA-Exim-Version: 4.2.1 (built Tue, 02 Aug 2016 21:08:31 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2018-12-06 4:38 p.m., Dave Hansen wrote: > On 12/6/18 3:28 PM, Logan Gunthorpe wrote: >> I didn't think this was meant to describe actual real world performance >> between all of the links. If that's the case all of this seems like a >> pipe dream to me. > > The HMAT discussions (that I was a part of at least) settled on just > trying to describe what we called "sticker speed". Nobody had an > expectation that you *really* had to measure everything. > > The best we can do for any of these approaches is approximate things. Yes, though there's a lot of caveats in this assumption alone. Specifically with PCI: the bus may run at however many GB/s but P2P through a CPU's root complexes can slow down significantly (like down to MB/s). I've seen similar things across QPI: I can sometimes do P2P from PCI->QPI->PCI but the performance doesn't even come close to the sticker speed of any of those buses. I'm not sure how anyone is going to deal with those issues, but it does firmly place us in world view #2 instead of #1. But, yes, I agree exposing information like in #2 full out to userspace, especially through sysfs, seems like a nightmare and I don't see anything in HMS to help with that. Providing an API to ask for memory (or another resource) that's accessible by a set of initiators and with a set of requirements for capabilities seems more manageable. Logan