From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752894AbbJMAkk (ORCPT ); Mon, 12 Oct 2015 20:40:40 -0400 Received: from mail-pa0-f48.google.com ([209.85.220.48]:33030 "EHLO mail-pa0-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752003AbbJMAkh (ORCPT ); Mon, 12 Oct 2015 20:40:37 -0400 Date: Mon, 12 Oct 2015 17:40:29 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Viacheslav Fedorov cc: linux-kernel@vger.kernel.org Subject: Re: Fwd: Memory leakage when using read_swap_cache_async() ? In-Reply-To: Message-ID: References: User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 12 Oct 2015, Viacheslav Fedorov wrote: > Hi ladies and gentlemen, > > Please help - I am stuck. > Looks like memory pages go unaccounted for, somewhere in the system, > when I am prefetching pages from swap file. Using kernel 3.8.0 (Ubuntu > distribution). > > So what I do is I have two simple applications that allocate 1GB > memory each and then proceed to write arbitrary data to the allocated > memory, with page granularity. > The system has a total of 1.3GB RAM to simulate memory pressure etc > and encourage swapping. > > To test the basic functionality of my approach, I included a simple > "next-page" prefetch code into the mm/memory.c, just after the call to > do_swap_page (http://lxr.free-electrons.com/source/mm/memory.c?v=3.8#L3637) > What it should do is prefetch 10 pages following the one that has just > been swapped in on-demand. I don't really care which ones, what > matters is want to try something different than reading sequentially > from the swap file (read-ahead). > See the code below. > > The problem is, if I run the test apps for the sufficient amount of > time, the "used" memory increases to the point where the applications > are killed by an OOM-killer. > After that, the available system memory is about half of what it > should have been (e.g. 600MB/1.3GB). Same thing happens to memory > usage if I stop the applications before the OOM-killer is triggered, > I am also attaching the /proc/meminfo contents for comparison on a > freshly-booted system vs. one where the two apps have been running for > 200 seconds before quitting, with and without the prefetch code in > kernel. > Note how Active and Inactive show a lot of memory being "used" for something. > The printouts were made when the system is completely idle with > nothing running etc. > > > I also tried prefetching from within the swapin_readahead() function > (http://lxr.free-electrons.com/source/mm/swap_state.c?v=3.8#L373) with > the same result - memory gets "used up". > > > I would greatly appreciate any help! > Pretty much ran out of ideas as to what could be causing this. > Thanks a lot, > Slava > > ============== code fragment used for prefetching ============ > > (( at the beginning of handle_pte_fault() function... > int rett, ii; > pte_t *ptee; > swp_entry_t sentry; > struct page *page; > )) > > rett = do_swap_page(mm, vma, address, > pte, pmd, flags, entry); > > //////////////////////////////////////////////////////////////// > /// test for memory leaks//////////////////// > for(ii=1;ii<10;ii++){ > ptee = pte_offset_map(pmd, address + (ii*2)<<12); > if(pte_present(*ptee) || pte_none(*ptee) || > pte_file(*ptee)) return rett; // no such page in page table > sentry = pte_to_swp_entry(*ptee); > if (non_swap_entry(sentry)){ > // printk(KERN_ALERT " non swap entry\n"); > return rett; > } > delayacct_set_flag(DELAYACCT_PF_SWAPIN); > page = lookup_swap_cache(sentry); That gets a reference to page. > if (!page) { > page = read_swap_cache_async(sentry, > GFP_HIGHUSER_MOVABLE, vma, address); > if(page) page_cache_release(page); > } > lru_add_drain(); > // if(page) unlock_page(page); But you don't appear to release that reference. > delayacct_clear_flag(DELAYACCT_PF_SWAPIN); > } > // printk(KERN_ALERT "swapped pages\n"); > //////////////////////////////////////////// > ////////////////////////////////////////// > > return rett; > > ===================== end code fragment =====================