Re: Fwd: Memory leakage when using read_swap_cache_async() ?

[Hugh Dickins <hughd@google.com>]

On Mon, 12 Oct 2015, Viacheslav Fedorov wrote:

> Hi ladies and gentlemen,
> 
> Please help - I am stuck.
> Looks like memory pages go unaccounted for, somewhere in the system,
> when I am prefetching pages from swap file. Using kernel 3.8.0 (Ubuntu
> distribution).
> 
> So what I do is I have two simple applications that allocate 1GB
> memory each and then proceed to write arbitrary data to the allocated
> memory, with page granularity.
> The system has a total of 1.3GB RAM to simulate memory pressure etc
> and encourage swapping.
> 
> To test the basic functionality of my approach, I included a simple
> "next-page" prefetch code into the mm/memory.c, just after the call to
> do_swap_page (http://lxr.free-electrons.com/source/mm/memory.c?v=3.8#L3637)
> What it should do is prefetch 10 pages following the one that has just
> been swapped in on-demand. I don't really care which ones, what
> matters is want to try something different than reading sequentially
> from the swap file (read-ahead).
> See the code below.
> 
> The problem is, if I run the test apps for the sufficient amount of
> time, the "used" memory increases to the point where the applications
> are killed by an OOM-killer.
> After that, the available system memory is about half of what it
> should have been (e.g. 600MB/1.3GB). Same thing happens to memory
> usage if I stop the applications before the OOM-killer is triggered,
> I am also attaching the /proc/meminfo contents for comparison on a
> freshly-booted system vs. one where the two apps have been running for
> 200 seconds before quitting, with and without the prefetch code in
> kernel.
> Note how Active and Inactive show a lot of memory being "used" for something.
> The printouts were made when the system is completely idle with
> nothing running etc.
> 
> 
> I also tried prefetching from within the swapin_readahead() function
> (http://lxr.free-electrons.com/source/mm/swap_state.c?v=3.8#L373) with
> the same result - memory gets "used up".
> 
> 
> I would greatly appreciate any help!
> Pretty much ran out of ideas as to what could be causing this.
> Thanks a lot,
> Slava
> 
> ============== code fragment used for prefetching ============
> 
> (( at the beginning of handle_pte_fault() function...
>         int rett, ii;
>         pte_t *ptee;
>         swp_entry_t sentry;
>         struct page *page;
> ))
> 
>                 rett = do_swap_page(mm, vma, address,
>                                         pte, pmd, flags, entry);
> 
> ////////////////////////////////////////////////////////////////
>                 /// test for memory leaks////////////////////
>                    for(ii=1;ii<10;ii++){
>                         ptee = pte_offset_map(pmd, address + (ii*2)<<12);
>                         if(pte_present(*ptee) || pte_none(*ptee) ||
> pte_file(*ptee)) return rett; // no such page in page table
>                         sentry = pte_to_swp_entry(*ptee);
>                         if (non_swap_entry(sentry)){
>                         //      printk(KERN_ALERT " non swap entry\n");
>                                 return rett;
>                         }
>                         delayacct_set_flag(DELAYACCT_PF_SWAPIN);
>                         page = lookup_swap_cache(sentry);

That gets a reference to page.

>                         if (!page) {
>                                 page = read_swap_cache_async(sentry,
> GFP_HIGHUSER_MOVABLE, vma, address);
>                                 if(page) page_cache_release(page);
>                         }
>                         lru_add_drain();
> //                      if(page) unlock_page(page);

But you don't appear to release that reference.

>                         delayacct_clear_flag(DELAYACCT_PF_SWAPIN);
>                     }
>    //                     printk(KERN_ALERT "swapped pages\n");
>                 ////////////////////////////////////////////
> //////////////////////////////////////////
> 
> return rett;
> 
> ===================== end code fragment =====================