Re: [RFC PATCH 0/2] arm64: memory-hotplug: Add Memory Hotplug support

[Scott Branden <scott.branden@broadcom.com>]

Hi Xishi,

I followed you suggestions and found pfn_valid is always true.  Answers 
to your questions inline.

I could keep debugging this but hope Marcin sends out some code - I'm 
quite willing to test and help clean up the patchset.

On 16-12-01 07:11 PM, Xishi Qiu wrote:
> On 2016/12/2 10:38, Scott Branden wrote:
>
>> Hi Xishi,
>>
>> Thanks for the reply - please see comments below.
>>
>> On 16-12-01 05:49 PM, Xishi Qiu wrote:
>>> On 2016/12/2 8:19, Scott Branden wrote:
>>>
>>>> This patchset is sent for comment to add memory hotplug support for ARM64
>>>> based platforms.  It follows hotplug code added for other architectures
>>>> in the linux kernel.
>>>>
>>>> I tried testing the memory hotplug feature following documentation from
>>>> Documentation/memory-hotplug.txt.  I don't think it is working as expected
>>>> - see below:
>>>>
>>>> To add memory to the system I did the following:
>>>> echo 0x400000000 > /sys/devices/system/memory/probe
>>>>
>>>> The memory is displayed as system ram:
>>>> cat /proc/iomem:
>>>> 74000000-77ffffff : System RAM
>>>>   74080000-748dffff : Kernel code
>>>>   74950000-749d2fff : Kernel data
>>>> 400000000-43fffffff : System RAM
>>>>
>>>> But does not seem to be added to the kernel memory.
>>>> /proc/meminfo did not change.
>>>>
>>>> What else needs to be done so the memory is added to the kernel memory
>>>> pool for normal allocation?
>>>>
>>>
>>> Hi Scott,
>>>
>>> Do you mean it still don't support hod-add after apply this patchset?
>>
>> After applying the patch it appears to partially support hot-add. Please let me know if you think it is working as expected?
>>
>> The memory probe functions in that the memory is registered with the system and shows up in /proc/iomem.  But, the memory is not available in /proc/meminfo.  Do you think something else needs to be adjusted for ARM64 to hotadd the memory
>>
>> I just found another clue:
>> under /sys/devices/system/memory I only see one memory entry (before or after I try to hotadd additional memory).
>>
>> /sys/devices/system/memory # ls
>> auto_online_blocks  memory0            uevent
>> block_size_bytes    probe
>>
>> In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30 to 28 and recompile I get the following:
>> /sys/devices/system/memory # ls
>> auto_online_blocks  memory7            uevent
>> block_size_bytes    probe
>>
>>
>> In arch/arm64/include/asm/sparsemem.h if I change SECTION_SIZE_BITS from 30 to 27 and recompile I get the following:
>> /sys/devices/system/memory # ls
>> auto_online_blocks  memory14            uevent
>> block_size_bytes    probe
>>
>> If looks to me like something is not working properly in the ARM64 implementation.  I should expect to see multiple memoryX entries under /sys/devices/system/memory?
>>
>
> Hi Scott,
>
> 1. Do you enable the following configs?
> CONFIG_SPARSEMEM
> MEMORY_HOTPLUG
> CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE
Yes, these configs are enabled
>
> 2. I find you missed create mapping in arch_add_memory(), and x86 has it.
Could you please explain this further?  The patch I submitted hass 
arch_add_memory identical to the ia64 implementation.
>
> 3. We will add memblock first, so pfn_valid() maybe always return true(in the
> following function), and this will lead __add_section() failed. Please check
> it.
You are correct - pfn_valid always returns true.  The function is in 
arch/arm64/mm/init.c and different than the one you indicated below:

#ifdef CONFIG_HAVE_ARCH_PFN_VALID
int pfn_valid(unsigned long pfn)
{
	return memblock_is_map_memory(pfn << PAGE_SHIFT);
}
EXPORT_SYMBOL(pfn_valid);
#endif

>
> int pfn_valid(unsigned long pfn)
> {
> 	return (pfn & PFN_MASK) == pfn && memblock_is_memory(pfn << PAGE_SHIFT);
> }
>
> add_memory
>   add_memory_resource
>     memblock_add_node
>       arch_add_memory
>         __add_pages
>           __add_section
>             pfn_valid
>
> Thanks,
> Xishi Qiu
>
>>
>>
>>>
>>> Thanks,
>>> Xishi Qiu
>>>
>>>> Scott Branden (2):
>>>>   arm64: memory-hotplug: Add MEMORY_HOTPLUG, MEMORY_HOTREMOVE,
>>>>     MEMORY_PROBE
>>>>   arm64: defconfig: enable MEMORY_HOTPLUG config options
>>>>
>>>>  arch/arm64/Kconfig           | 10 ++++++++++
>>>>  arch/arm64/configs/defconfig |  3 +++
>>>>  arch/arm64/mm/init.c         | 42 ++++++++++++++++++++++++++++++++++++++++++
>>>>  3 files changed, 55 insertions(+)
>>>>
>>>
>>>
>>>
>>
>> .
>>
>
>
>