From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 426F3C282CD for ; Wed, 30 Jan 2019 04:18:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 03E7F20820 for ; Wed, 30 Jan 2019 04:18:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="p6RgcDkI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730146AbfA3ESx (ORCPT ); Tue, 29 Jan 2019 23:18:53 -0500 Received: from mail-eopbgr140048.outbound.protection.outlook.com ([40.107.14.48]:64975 "EHLO EUR01-VE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727672AbfA3ESx (ORCPT ); Tue, 29 Jan 2019 23:18:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=odFD/pFRFfFYR8wF/TesPm0cK31nWbm6naTZkKG5Tkw=; b=p6RgcDkIdbBvDffPNVCjkEQweUEYjT3fOX36rD1t5vMdJIESNDty+5NRcaw/AewW1d8PF66Yo5Yct1pqGYqKxJiFZQfRydymYxllxIcsMR6o+qxyy04O2CjQu4miQAirtrYJhoiGDES2YLb7TGg3Xs+lr6bNMm/+bH4oSB4Liz0= Received: from DBBPR05MB6426.eurprd05.prod.outlook.com (20.179.42.80) by DBBPR05MB6345.eurprd05.prod.outlook.com (20.179.41.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1580.16; Wed, 30 Jan 2019 04:18:48 +0000 Received: from DBBPR05MB6426.eurprd05.prod.outlook.com ([fe80::24c2:321d:8b27:ae59]) by DBBPR05MB6426.eurprd05.prod.outlook.com ([fe80::24c2:321d:8b27:ae59%5]) with mapi id 15.20.1580.017; Wed, 30 Jan 2019 04:18:48 +0000 From: Jason Gunthorpe To: Logan Gunthorpe CC: Jerome Glisse , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Greg Kroah-Hartman , "Rafael J . Wysocki" , Bjorn Helgaas , Christian Koenig , Felix Kuehling , "linux-pci@vger.kernel.org" , "dri-devel@lists.freedesktop.org" , Christoph Hellwig , Marek Szyprowski , Robin Murphy , Joerg Roedel , "iommu@lists.linux-foundation.org" Subject: Re: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma Thread-Topic: [RFC PATCH 3/5] mm/vma: add support for peer to peer to device vma Thread-Index: AQHUt/rA/dLikqWEmEaIytHIBNLPlqXGkyOAgAAJwICAAAX+AIAAEreAgAAFCQCAAAk3gIAABX0AgAATFYCAAA25AIAAGRqAgAAykIA= Date: Wed, 30 Jan 2019 04:18:48 +0000 Message-ID: <20190130041841.GB30598@mellanox.com> References: <20190129191120.GE3176@redhat.com> <20190129193250.GK10108@mellanox.com> <99c228c6-ef96-7594-cb43-78931966c75d@deltatee.com> <20190129205749.GN3176@redhat.com> <2b704e96-9c7c-3024-b87f-364b9ba22208@deltatee.com> <20190129215028.GQ3176@redhat.com> <20190129234752.GR3176@redhat.com> <655a335c-ab91-d1fc-1ed3-b5f0d37c6226@deltatee.com> In-Reply-To: <655a335c-ab91-d1fc-1ed3-b5f0d37c6226@deltatee.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: MWHPR14CA0010.namprd14.prod.outlook.com (2603:10b6:300:ae::20) To DBBPR05MB6426.eurprd05.prod.outlook.com (2603:10a6:10:c9::16) authentication-results: spf=none (sender IP is ) smtp.mailfrom=jgg@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [174.3.196.123] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;DBBPR05MB6345;6:H4A4wUJn0ItXsLQN1LgE4YdE70nyc4iTGrlE2qbmv590oIG7lqiM4MlLXBseI6Cb5faHzW216O9vJfWGxiunoOqiY/4dZ7JRCIxcusRm2wxGmmui0I+nKAJWdH7whmwT2HGuiHVh/Hk3CJ1al3ehB+Ngj0HfE+vr7Ywbvews+9XphXGFfJU+eYgNW0s+71Oc672OlQbM3+NoPbbIshawVsnjs5m2NYL+c+Ti3jb5cB6hupAOoET6XQk1zaDzdj+HoxQ4RNVPepaNKcYe95HGg2dDU9tTOs9bKU2kE2GHDrqR9KS0veoPiyQ14/JfLzAJR+VVk7PFLSjMaEnm2Wis1S/zgz7yUshobhKkGll8pzxu7RZOqQcj+h3uOpnf4/eK8irgjcm9CnUPPEnq4+Na6eYuqcKWxrMiXt2RvylRmZwy5QpdZbSnZUSOl8F9LxO1Hye4nhnFIHDdOnSTpRfp7g==;5:EJRY6VOIFbo621ZwP8pChZ1luqiGy6EBgNQqNZhsnuyOBoXvy9X5rVxeWO/eVWYJq/OmvOtq0NAJdWu0XPH/OFxxP8yXn/drjvky28ZG+K1lNH56i345p4UVn9CMmsrz8Vq7BBF2m01tu5aynB+U0tJlI/rvrKociAys+iL7rsrjo3vVnyTwGDoZo4PrPmROonv9hy0YPNUkkC1Mqg4XEw==;7:XWp6E+PllBrWgYWUNR2gsmBynyaeFVr1UhbEFtJ/QJdCqfz++qIqK3LaL0gYcwIGC185OcyHKplvmM+Vcg36U/I2L96jr7yLltJO4DnC8eYgQqOsXPYdPiUFp3fBxOogbNnTB/YU0njrG9K1k3r8UA== x-ms-office365-filtering-correlation-id: 4b51e9cd-5390-4465-8cca-08d6866a0842 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600110)(711020)(4605077)(4618075)(2017052603328)(7153060)(7193020);SRVR:DBBPR05MB6345; x-ms-traffictypediagnostic: DBBPR05MB6345: x-microsoft-antispam-prvs: x-forefront-prvs: 0933E9FD8D x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(39860400002)(366004)(346002)(396003)(376002)(136003)(189003)(199004)(4326008)(76176011)(93886005)(6246003)(25786009)(33656002)(81166006)(8676002)(8936002)(99286004)(81156014)(52116002)(7416002)(6116002)(316002)(14454004)(54906003)(2906002)(36756003)(3846002)(478600001)(6916009)(53936002)(97736004)(66066001)(105586002)(102836004)(6512007)(86362001)(217873002)(386003)(229853002)(11346002)(6506007)(1076003)(256004)(305945005)(6486002)(26005)(476003)(4744005)(446003)(68736007)(71200400001)(71190400001)(106356001)(2616005)(6436002)(486006)(7736002)(186003);DIR:OUT;SFP:1101;SCL:1;SRVR:DBBPR05MB6345;H:DBBPR05MB6426.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: b8GsDOImAOCI4A5iDMbH5ipBxDYzgCyqT37NgpLWL0UmrTUhLFH9cf/LLzWXIWEHtu94jPnEaqiEBaq7yLgrwOWehxWAK7Xnqad9lh1nFrZecn4h0/eeNCFYDPVumqc+81MqxY2aucrdSso0MDgwDcQQuRlYr664uNa9xLGXtXCnJbQPx9oZaRxq2zpGC4bAmET53lerI6CR/E1Wmu2nWfHDeIQhHQ+/rLGtjJ+TkzRMHZeZpoWj8dGrmy2pIOzpviicNDGIuhAP72cJeGgBXZDmR6COmi5ZeGmpp+3d+r7mplCW7G74x5OijDaxg28gBXreLC8EGu0P6PxdVzvCEPEMiTxBQwa3L2I9hBdSWbgMfbPBnFlHgEpbMn94gNai2vCqdSsLQFRF4Gi7qkAri6fhW77QZg7/s5LHQoPbxag= Content-Type: text/plain; charset="us-ascii" Content-ID: <82BFF7F760AB5D47935C40953858BB8C@eurprd05.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4b51e9cd-5390-4465-8cca-08d6866a0842 X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Jan 2019 04:18:48.2718 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR05MB6345 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jan 29, 2019 at 06:17:43PM -0700, Logan Gunthorpe wrote: > This isn't answering my question at all... I specifically asked what is > backing the VMA when we are *not* using HMM. At least for RDMA what backs the VMA today is non-struct-page BAR memory filled in with io_remap_pfn. And we want to expose this for P2P DMA. None of the HMM stuff applies here and the p2p_map/unmap are a nice simple approach that covers all the needs RDMA has, at least. Every attempt to give BAR memory to struct page has run into major trouble, IMHO, so I like that this approach avoids that. And if you don't have struct page then the only kernel object left to hang meta data off is the VMA itself. It seems very similar to the existing P2P work between in-kernel consumers, just that VMA is now mediating a general user space driven discovery process instead of being hard wired into a driver. Jason