Re: [Xen-devel] [RFC KERNEL PATCH 0/2] Add Dom0 NVDIMM support for Xen

[Haozhong Zhang <haozhong.zhang@intel.com>]

+Dan Williams

I accidentally dropped him in my last reply. Add him back.

On 10/13/16 16:53 +0800, Haozhong Zhang wrote:
>On 10/13/16 02:34 -0600, Jan Beulich wrote:
>>>>>On 12.10.16 at 18:19, <dan.j.williams@intel.com> wrote:
>>>On Wed, Oct 12, 2016 at 9:01 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>>>>On 12.10.16 at 17:42, <dan.j.williams@intel.com> wrote:
>>>>>On Wed, Oct 12, 2016 at 8:39 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>>>>>>On 12.10.16 at 16:58, <haozhong.zhang@intel.com> wrote:
>>>>>>>On 10/12/16 05:32 -0600, Jan Beulich wrote:
>>>>>>>>>>>On 12.10.16 at 12:33, <haozhong.zhang@intel.com> wrote:
>>>>>>>>>The layout is shown as the following diagram.
>>>>>>>>>
>>>>>>>>>+---------------+-----------+-------+----------+--------------+
>>>>>>>>>| whatever used | Partition | Super | Reserved | /dev/pmem0p1 |
>>>>>>>>>|  by kernel    |   Table   | Block | for Xen  |              |
>>>>>>>>>+---------------+-----------+-------+----------+--------------+
>>>>>>>>>                \_____________________ _______________________/
>>>>>>>>>                                  V
>>>>>>>>>                             /dev/pmem0
>>>>>>>>
>>>>>>>>I have to admit that I dislike this, for not being OS-agnostic.
>>>>>>>>Neither should there be any Xen-specific region, nor should the
>>>>>>>>"whatever used by kernel" one be restricted to just Linux. What
>>>>>>>>I could see is an OS-reserved area ahead of the partition table,
>>>>>>>>the exact usage of which depends on which OS is currently
>>>>>>>>running (and in the Xen case this might be both Xen _and_ the
>>>>>>>>Dom0 kernel, arbitrated by a tbd protocol). After all, when
>>>>>>>>running under Xen, the Dom0 may not have a need for as much
>>>>>>>>control data as it has when running on bare hardware, for it
>>>>>>>>controlling less (if any) of the actual memory ranges when Xen
>>>>>>>>is present.
>>>>>>>>
>>>>>>>
>>>>>>>Isn't this OS-reserved area still not OS-agnostic, as it requires OS
>>>>>>>to know where the reserved area is?  Or do you mean it's not if it's
>>>>>>>defined by a protocol that is accepted by all OSes?
>>>>>>
>>>>>>The latter - we clearly won't get away without some agreement on
>>>>>>where to retrieve position and size of this area. I was simply
>>>>>>assuming that such a protocol already exists.
>>>>>>
>>>>>
>>>>>No, we should not mix the struct page reservation that the Dom0 kernel
>>>>>may actively use with the Xen reservation that the Dom0 kernel does
>>>>>not consume.  Explain again what is wrong with the partition approach?
>>>>
>>>>Not sure what was unclear in my previous reply. I don't think there
>>>>should be apriori knowledge of whether Xen is (going to be) used on
>>>>a system, and even if it gets used, but just occasionally, it would
>>>>(apart from the abstract considerations already given) be a waste
>>>>of resources to set something aside that could be used for other
>>>>purposes while Xen is not running. Static partitioning should only be
>>>>needed for persistent data.
>>>
>>>The reservation needs to be persistent / static even if the data is
>>>volatile, as is the case with struct page, because we can't have the
>>>size of the device change depending on use.  So, from the aspect of
>>>wasting space while Xen is not in use, both partitions and the
>>>intrinsic reservation approach suffer the same problem. Setting that
>>>aside I don't want to mix 2 different use cases into the same
>>>reservation.
>>
>>Then you didn't understand what I've said: I certainly didn't mean
>>the reservation to vary from a device perspective. However, when
>>Xen is in use I don't see why part of that static reservation couldn't
>>be used by Xen, and another part by the Dom0 kernel. The kernel
>>obviously would need to ask the hypervisor how much of the space
>>is left, and where that area starts.
>>
>
>I think Dan means that there should be a clear separation between
>reservations for different usages (kernel/xen/...). The libnvdimm
>driver is for the linux kernel and only needs to maintain the
>reservation for kernel functionality. For others including xen/dm/...,
>if they want reservation for their own purpose, they should maintain
>their own reservations out of libnvdimm driver and avoid bothering the
>libnvdimm driver (e.g. add specific handling in libnvdimm driver).
>
>IIUC, one existing example is device-mapper device (dm) which needs to
>reserve on-device area for its own meta-data. Its choice is to store
>the meta-data on the block device (/dev/pmemN) provided by the
>libnvdimm driver.
>
>I think we can do the similar for Xen, like to lay another pseudo
>device on /dev/pmem and do the reservation, like 2. in my previous
>reply.
>
>Thanks,
>Haozhong
>
>>>The kernel needs to know about the struct page reservation because it
>>>needs to manage the lifetime of page references vs the lifetime of the
>>>device.  It does not have the same relationship with a Xen reservation
>>>which is why I'm proposing they be managed separately.
>>
>>I don't think I understand the difference you try to point out here.
>>Linux'es struct page and Xen's struct page_info serve the same
>>fundamental purpose.
>>
>>Jan
>>
>_______________________________________________
>Linux-nvdimm mailing list
>Linux-nvdimm@lists.01.org
>https://lists.01.org/mailman/listinfo/linux-nvdimm