Re: [PATCH] arm64: limit MAX_PHYSMEM_BITS based on vmemmap

[Anshuman Khandual <anshuman.khandual@arm.com>]

Hello Scott,

On 7/4/24 02:37, D Scott Phillips wrote:
> Prior to the memory map adjustments in v6.9-rc1, the amdgpu driver could
> trip over the warning of:
> 
> `WARN_ON((start < VMEMMAP_START) || (end > VMEMMAP_END));`

Could you please provide the mainline commit ID for the mentioned memory
adjustment changes.

> 
> in vmemmap_populate()[1]. After the adjustments, it becomes a translation
> fault and panic.

Probably caused by accessing memory which does not exist on that address.

> 
> The cause is that the amdgpu driver allocates some unused space from
> iomem_resource and claims it as MEMORY_DEVICE_PRIVATE and
> devm_memremap_pages() it. An address above those backed by the vmemmap is
> used.
> 
> Adjust MAX_PHYSMEM_BITS so that addresses not backed by the vmemmap will
> not be chosen as device private addresses.
> 
> [1]: Call trace:
>       vmemmap_populate+0x30/0x48
>       __populate_section_memmap+0x40/0x90
>       sparse_add_section+0xfc/0x3e8
>       __add_pages+0xb4/0x168
>       pagemap_range+0x300/0x410
>       memremap_pages+0x184/0x2d8
>       devm_memremap_pages+0x30/0x90
>       kgd2kfd_init_zone_device+0xe0/0x1f0 [amdgpu]
>       amdgpu_device_ip_init+0x674/0x888 [amdgpu]
>       amdgpu_device_init+0x7bc/0xed8 [amdgpu]
>       amdgpu_driver_load_kms+0x28/0x1c0 [amdgpu]
>       amdgpu_pci_probe+0x194/0x580 [amdgpu]
>       local_pci_probe+0x48/0xb8
>       work_for_cpu_fn+0x24/0x40
>       process_one_work+0x170/0x3e0
>       worker_thread+0x2ac/0x3e0
>       kthread+0xf4/0x108
>       ret_from_fork+0x10/0x20
> 
> Signed-off-by: D Scott Phillips <scott@os.amperecomputing.com>
> ---
>  arch/arm64/include/asm/sparsemem.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h
> index 8a8acc220371c..8387301f2e206 100644
> --- a/arch/arm64/include/asm/sparsemem.h
> +++ b/arch/arm64/include/asm/sparsemem.h
> @@ -5,7 +5,7 @@
>  #ifndef __ASM_SPARSEMEM_H
>  #define __ASM_SPARSEMEM_H
>  
> -#define MAX_PHYSMEM_BITS	CONFIG_ARM64_PA_BITS
> +#define MAX_PHYSMEM_BITS	ilog2(VMEMMAP_RANGE)

Just wondering if there is another method, which avoids selecting physical
memory ranges not backed with vmemmap. Also will reducing MAX_PHYSMEM_BITS
below ARM64_PA_BITS have other side effects ? Do other platforms have this
exact same co-relation between MAX_PHYSMEM_BITS and vmemmap range ?

>  
>  /*
>   * Section size must be at least 512MB for 64K base