From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753332AbcLHUib convert rfc822-to-8bit (ORCPT ); Thu, 8 Dec 2016 15:38:31 -0500 Received: from mx4-phx2.redhat.com ([209.132.183.25]:45389 "EHLO mx4-phx2.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752035AbcLHUia (ORCPT ); Thu, 8 Dec 2016 15:38:30 -0500 Date: Thu, 8 Dec 2016 15:37:23 -0500 (EST) From: Jerome Glisse To: Dave Hansen Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, John Hubbard , Dan Williams , Ross Zwisler Message-ID: <2093258630.3273244.1481229443563.JavaMail.zimbra@redhat.com> In-Reply-To: <7df66ace-ef29-c76b-d61c-88263a61c6d0@intel.com> References: <1481215184-18551-1-git-send-email-jglisse@redhat.com> <1481215184-18551-6-git-send-email-jglisse@redhat.com> <152004793.3187283.1481215199204.JavaMail.zimbra@redhat.com> <7df66ace-ef29-c76b-d61c-88263a61c6d0@intel.com> Subject: Re: [HMM v14 05/16] mm/ZONE_DEVICE/unaddressable: add support for un-addressable device memory MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8BIT X-Originating-IP: [10.19.40.126] X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF34 (Linux)/8.0.6_GA_5922) Thread-Topic: mm/ZONE_DEVICE/unaddressable: add support for un-addressable device memory Thread-Index: SQwQYnGkExIQI/uDkAETD8uusOy9OQ== Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > On 12/08/2016 08:39 AM, Jerome Glisse wrote: > >> On 12/08/2016 08:39 AM, Jérôme Glisse wrote: > >>> > > Architecture that wish to support un-addressable device memory should > >>> > > make > >>> > > sure to never populate the kernel linar mapping for the physical > >>> > > range. > >> > > >> > Does the platform somehow provide a range of physical addresses for this > >> > unaddressable area? How do we know no memory will be hot-added in a > >> > range we're using for unaddressable device memory, for instance? > > That's what one of the big issue. No platform does not reserve any range so > > there is a possibility that some memory get hotpluged and assign this > > range. > > > > I pushed the range decision to higher level (ie it is the device driver > > that > > pick one) so right now for device driver using HMM (NVidia close driver as > > we don't have nouveau ready for that yet) it goes from the highest physical > > address and scan down until finding an empty range big enough. > > I don't think you should be stealing physical address space for things > that don't and can't have physical addresses. Delegating this to > individual device drivers and hoping that they all get it right seems > like a recipe for disaster. Well i expected device driver to use hmm_devmem_add() which does not take physical address but use the above logic to pick one. > > Maybe worth adding to the changelog: > > This feature potentially breaks memory hotplug unless every > driver using it magically predicts the future addresses of > where memory will be hotplugged. I will add debug printk to memory hotplug in case it fails because of some un-addressable resource. If you really dislike memory hotplug being broken then i can go down the way of allowing to hotplug memory above the max physical memory limit. This require more changes but i believe this is doable for some of the memory model (sparsemem and sparsemem extreme). > > BTW, how many more of these "big issues" does this set have? I didn't > see any mention of this in the changelogs. I am not sure what to say here. If you don't use HMM ie no device that hotplug it. Then there is no chance of having issue. If you have a device that use it then someone might try to do something stupid (try to kmap and access such un-addressable page for instance). So i am not sure where to draw the line. Cheers, Jérôme