Re: [PATCH] mm/debug_vm_pgtable: Fix corrupted PG_arch_1 by set_pmd_at()

[Gavin Shan <gshan@redhat.com>]

Hi Anshuman,

On 7/5/21 1:59 PM, Anshuman Khandual wrote:
> On 7/2/21 4:02 PM, Gavin Shan wrote:
>> There are two addresses selected: random virtual address and physical
>> address corresponding to kernel symbol @start_kernel. During the PMD
>> tests in pmd_advanced_tests(), the physical address is aligned down
>> to the starting address of the huge page, whose size is 512MB on ARM64
>> when we have 64KB base page size. After that, set_pmd_at() is called
>> to populate the PMD entry. PG_arch_1, PG_dcache_clean on ARM64, is
>> set to the page flags. Unforunately, the page, corresponding to the
>> starting address of the huge page could be owned by buddy. It means
>> PG_arch_1 can be unconditionally set to page owned by buddy.
>>
>> Afterwards, the page with PG_arch_1 set is fetched from buddy's free
>> area list, but fails the checking. It leads to the following warning
>> on ARM64:
>>
>>     BUG: Bad page state in process memhog  pfn:08000
>>     page:0000000015c0a628 refcount:0 mapcount:0 \
>>          mapping:0000000000000000 index:0x1 pfn:0x8000
>>     flags: 0x7ffff8000000800(arch_1|node=0|zone=0|lastcpupid=0xfffff)
>>     raw: 07ffff8000000800 dead000000000100 dead000000000122 0000000000000000
>>     raw: 0000000000000001 0000000000000000 00000000ffffffff 0000000000000000
>>     page dumped because: PAGE_FLAGS_CHECK_AT_PREP flag(s) set
> 
> Does this problem happen right after the boot ? OR you ran some tests
> and workloads to trigger this ? IIRC never seen this before on arm64.
> Does this happen on other archs too ?
> 

The page flag (PG_arch_1) is corrupted during boot on ARM64 where
64KB base page size is selected, but the failing page check happens
when the page is pulled from buddy's free area list by "memhog".
I don't think other platform has same issue.

>>
>> This fixes the issue by calling flush_dcache_page() after each call
>> to set_{pud, pmd, pte}_at() because PG_arch_1 isn't needed in any case.
> 
> This (arm64 specific solution) might cause some side effects on other
> platforms ? The solution here needs to be generic enough. I will take
> a look into this patch but probably later this week or next week.
> 

Apart from the overhead of flushing the dcache introduced by flush_dcache_page().
I don't think there is any side-effect. By the way, I'm working on a series
to fix this issue and another issue. I will post the series for review pretty
soon and it's going to fix the following issues:

(1) Current code is organized in relaxed fashion. All information are maintained
     in variables in debug_vm_pgtable(). The variables are passed to test functions.
     It make the code hard to be maintained in long term. So I will introduce a
     dedicated data struct (struct vm_pgtable_debug), as place holder for various
     information.

(2) With the data struct, I'm able to allocate page, to be used by set_{pud, pmd, pte}_at()
     because the target page is accessed on ARM64. The PG_arch_1 flag is set to
     the page and the corresponding iCache is flush if execution permission is given.
     There are two issues if the page used by set_{pud, pmd, pte}_at() wasn't allocated
     from buddy: (a) the PG_arch_1 flag corruption as this patch tries to fix; (b) kernel
     crash because of invalid page fault on accessing the target page. The page isn't
     mapped if CONFIG_DEBUG_PAGEALLOC is enabled.

     start_kernel
     mm_init
     mem_init
     memblock_free_all
     free_low_memory_core_early
     __free_memory_core
     __free_pages_memory
     memblock_free_pages
     __free_pages_core
     __free_pages_ok
     free_pages_prepare
     debug_pagealloc_unmap_pages           # The page is unmapped here

Thanks,
Gavin