* Fwd: Memory leakage when using read_swap_cache_async() ? [not found] <CAFCpxERnGypH6ZaVzY3KszN4rRq_3oeUcugCDG3Vs0gZooHP7A@mail.gmail.com> @ 2015-10-12 23:43 ` Viacheslav Fedorov 2015-10-13 0:40 ` Hugh Dickins 0 siblings, 1 reply; 3+ messages in thread From: Viacheslav Fedorov @ 2015-10-12 23:43 UTC (permalink / raw) To: linux-kernel [-- Attachment #1: Type: text/plain, Size: 3739 bytes --] Hi ladies and gentlemen, Please help - I am stuck. Looks like memory pages go unaccounted for, somewhere in the system, when I am prefetching pages from swap file. Using kernel 3.8.0 (Ubuntu distribution). So what I do is I have two simple applications that allocate 1GB memory each and then proceed to write arbitrary data to the allocated memory, with page granularity. The system has a total of 1.3GB RAM to simulate memory pressure etc and encourage swapping. To test the basic functionality of my approach, I included a simple "next-page" prefetch code into the mm/memory.c, just after the call to do_swap_page (http://lxr.free-electrons.com/source/mm/memory.c?v=3.8#L3637) What it should do is prefetch 10 pages following the one that has just been swapped in on-demand. I don't really care which ones, what matters is want to try something different than reading sequentially from the swap file (read-ahead). See the code below. The problem is, if I run the test apps for the sufficient amount of time, the "used" memory increases to the point where the applications are killed by an OOM-killer. After that, the available system memory is about half of what it should have been (e.g. 600MB/1.3GB). Same thing happens to memory usage if I stop the applications before the OOM-killer is triggered, I am also attaching the /proc/meminfo contents for comparison on a freshly-booted system vs. one where the two apps have been running for 200 seconds before quitting, with and without the prefetch code in kernel. Note how Active and Inactive show a lot of memory being "used" for something. The printouts were made when the system is completely idle with nothing running etc. I also tried prefetching from within the swapin_readahead() function (http://lxr.free-electrons.com/source/mm/swap_state.c?v=3.8#L373) with the same result - memory gets "used up". I would greatly appreciate any help! Pretty much ran out of ideas as to what could be causing this. Thanks a lot, Slava ============== code fragment used for prefetching ============ (( at the beginning of handle_pte_fault() function... int rett, ii; pte_t *ptee; swp_entry_t sentry; struct page *page; )) rett = do_swap_page(mm, vma, address, pte, pmd, flags, entry); //////////////////////////////////////////////////////////////// /// test for memory leaks//////////////////// for(ii=1;ii<10;ii++){ ptee = pte_offset_map(pmd, address + (ii*2)<<12); if(pte_present(*ptee) || pte_none(*ptee) || pte_file(*ptee)) return rett; // no such page in page table sentry = pte_to_swp_entry(*ptee); if (non_swap_entry(sentry)){ // printk(KERN_ALERT " non swap entry\n"); return rett; } delayacct_set_flag(DELAYACCT_PF_SWAPIN); page = lookup_swap_cache(sentry); if (!page) { page = read_swap_cache_async(sentry, GFP_HIGHUSER_MOVABLE, vma, address); if(page) page_cache_release(page); } lru_add_drain(); // if(page) unlock_page(page); delayacct_clear_flag(DELAYACCT_PF_SWAPIN); } // printk(KERN_ALERT "swapped pages\n"); //////////////////////////////////////////// ////////////////////////////////////////// return rett; ===================== end code fragment ===================== [-- Attachment #2: meminfo_comparisons.txt --] [-- Type: text/plain, Size: 3712 bytes --] prefetch: 200s run: MemTotal: 1270536 kB MemFree: 668748 kB Buffers: 5304 kB Cached: 33248 kB SwapCached: 120424 kB Active: 336016 kB Inactive: 221792 kB Active(anon): 328280 kB Inactive(anon): 190984 kB Active(file): 7736 kB Inactive(file): 30808 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 4191228 kB SwapFree: 4052040 kB Dirty: 0 kB Writeback: 0 kB AnonPages: 6008 kB Mapped: 4596 kB Shmem: 8 kB Slab: 23656 kB SReclaimable: 10304 kB SUnreclaim: 13352 kB KernelStack: 800 kB PageTables: 2008 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 4826496 kB Committed_AS: 72232 kB VmallocTotal: 34359738367 kB VmallocUsed: 267280 kB VmallocChunk: 34359469179 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 34816 kB DirectMap2M: 1378304 kB fresh boot: MemTotal: 1270536 kB MemFree: 1118524 kB Buffers: 13236 kB Cached: 79364 kB SwapCached: 0 kB Active: 37896 kB Inactive: 71008 kB Active(anon): 16336 kB Inactive(anon): 252 kB Active(file): 21560 kB Inactive(file): 70756 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 4191228 kB SwapFree: 4191228 kB Dirty: 24 kB Writeback: 0 kB AnonPages: 16212 kB Mapped: 7320 kB Shmem: 280 kB Slab: 23496 kB SReclaimable: 11060 kB SUnreclaim: 12436 kB KernelStack: 856 kB PageTables: 1656 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 4826496 kB Committed_AS: 60072 kB VmallocTotal: 34359738367 kB VmallocUsed: 267280 kB VmallocChunk: 34359470199 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 36864 kB DirectMap2M: 1376256 kB no prefetch: 200s run: MemTotal: 1270536 kB MemFree: 1219856 kB Buffers: 344 kB Cached: 4348 kB SwapCached: 1512 kB Active: 1768 kB Inactive: 6284 kB Active(anon): 172 kB Inactive(anon): 3124 kB Active(file): 1596 kB Inactive(file): 3160 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 4191228 kB SwapFree: 4168800 kB Dirty: 0 kB Writeback: 0 kB AnonPages: 3340 kB Mapped: 1948 kB Shmem: 0 kB Slab: 22416 kB SReclaimable: 9580 kB SUnreclaim: 12836 kB KernelStack: 768 kB PageTables: 1972 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 4826496 kB Committed_AS: 71036 kB VmallocTotal: 34359738367 kB VmallocUsed: 267280 kB VmallocChunk: 34359470199 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 36864 kB DirectMap2M: 1376256 kB [-- Attachment #3: mbench.c --] [-- Type: text/x-csrc, Size: 251 bytes --] #include <stdio.h> #include <stdlib.h> int main(){ unsigned long i=2; char *mem = (char*)malloc(1024*1024*1424); // 1 gig if(mem) while(1){ mem[i] = rand(); i+=4096; i = i%(1024*1024*1024); } return 0; } ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Fwd: Memory leakage when using read_swap_cache_async() ? 2015-10-12 23:43 ` Fwd: Memory leakage when using read_swap_cache_async() ? Viacheslav Fedorov @ 2015-10-13 0:40 ` Hugh Dickins 2015-10-13 20:17 ` Viacheslav Fedorov 0 siblings, 1 reply; 3+ messages in thread From: Hugh Dickins @ 2015-10-13 0:40 UTC (permalink / raw) To: Viacheslav Fedorov; +Cc: linux-kernel On Mon, 12 Oct 2015, Viacheslav Fedorov wrote: > Hi ladies and gentlemen, > > Please help - I am stuck. > Looks like memory pages go unaccounted for, somewhere in the system, > when I am prefetching pages from swap file. Using kernel 3.8.0 (Ubuntu > distribution). > > So what I do is I have two simple applications that allocate 1GB > memory each and then proceed to write arbitrary data to the allocated > memory, with page granularity. > The system has a total of 1.3GB RAM to simulate memory pressure etc > and encourage swapping. > > To test the basic functionality of my approach, I included a simple > "next-page" prefetch code into the mm/memory.c, just after the call to > do_swap_page (http://lxr.free-electrons.com/source/mm/memory.c?v=3.8#L3637) > What it should do is prefetch 10 pages following the one that has just > been swapped in on-demand. I don't really care which ones, what > matters is want to try something different than reading sequentially > from the swap file (read-ahead). > See the code below. > > The problem is, if I run the test apps for the sufficient amount of > time, the "used" memory increases to the point where the applications > are killed by an OOM-killer. > After that, the available system memory is about half of what it > should have been (e.g. 600MB/1.3GB). Same thing happens to memory > usage if I stop the applications before the OOM-killer is triggered, > I am also attaching the /proc/meminfo contents for comparison on a > freshly-booted system vs. one where the two apps have been running for > 200 seconds before quitting, with and without the prefetch code in > kernel. > Note how Active and Inactive show a lot of memory being "used" for something. > The printouts were made when the system is completely idle with > nothing running etc. > > > I also tried prefetching from within the swapin_readahead() function > (http://lxr.free-electrons.com/source/mm/swap_state.c?v=3.8#L373) with > the same result - memory gets "used up". > > > I would greatly appreciate any help! > Pretty much ran out of ideas as to what could be causing this. > Thanks a lot, > Slava > > ============== code fragment used for prefetching ============ > > (( at the beginning of handle_pte_fault() function... > int rett, ii; > pte_t *ptee; > swp_entry_t sentry; > struct page *page; > )) > > rett = do_swap_page(mm, vma, address, > pte, pmd, flags, entry); > > //////////////////////////////////////////////////////////////// > /// test for memory leaks//////////////////// > for(ii=1;ii<10;ii++){ > ptee = pte_offset_map(pmd, address + (ii*2)<<12); > if(pte_present(*ptee) || pte_none(*ptee) || > pte_file(*ptee)) return rett; // no such page in page table > sentry = pte_to_swp_entry(*ptee); > if (non_swap_entry(sentry)){ > // printk(KERN_ALERT " non swap entry\n"); > return rett; > } > delayacct_set_flag(DELAYACCT_PF_SWAPIN); > page = lookup_swap_cache(sentry); That gets a reference to page. > if (!page) { > page = read_swap_cache_async(sentry, > GFP_HIGHUSER_MOVABLE, vma, address); > if(page) page_cache_release(page); > } > lru_add_drain(); > // if(page) unlock_page(page); But you don't appear to release that reference. > delayacct_clear_flag(DELAYACCT_PF_SWAPIN); > } > // printk(KERN_ALERT "swapped pages\n"); > //////////////////////////////////////////// > ////////////////////////////////////////// > > return rett; > > ===================== end code fragment ===================== ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Fwd: Memory leakage when using read_swap_cache_async() ? 2015-10-13 0:40 ` Hugh Dickins @ 2015-10-13 20:17 ` Viacheslav Fedorov 0 siblings, 0 replies; 3+ messages in thread From: Viacheslav Fedorov @ 2015-10-13 20:17 UTC (permalink / raw) To: Hugh Dickins; +Cc: linux-kernel Wow! Thank you so much! I literally spent a week trying to figure out the reason for the behavior. This solves the issue I was having at this point. Thanks again! Slava On Mon, Oct 12, 2015 at 7:40 PM, Hugh Dickins <hughd@google.com> wrote: > On Mon, 12 Oct 2015, Viacheslav Fedorov wrote: > >> Hi ladies and gentlemen, >> >> Please help - I am stuck. >> Looks like memory pages go unaccounted for, somewhere in the system, >> when I am prefetching pages from swap file. Using kernel 3.8.0 (Ubuntu >> distribution). >> >> So what I do is I have two simple applications that allocate 1GB >> memory each and then proceed to write arbitrary data to the allocated >> memory, with page granularity. >> The system has a total of 1.3GB RAM to simulate memory pressure etc >> and encourage swapping. >> >> To test the basic functionality of my approach, I included a simple >> "next-page" prefetch code into the mm/memory.c, just after the call to >> do_swap_page (http://lxr.free-electrons.com/source/mm/memory.c?v=3.8#L3637) >> What it should do is prefetch 10 pages following the one that has just >> been swapped in on-demand. I don't really care which ones, what >> matters is want to try something different than reading sequentially >> from the swap file (read-ahead). >> See the code below. >> >> The problem is, if I run the test apps for the sufficient amount of >> time, the "used" memory increases to the point where the applications >> are killed by an OOM-killer. >> After that, the available system memory is about half of what it >> should have been (e.g. 600MB/1.3GB). Same thing happens to memory >> usage if I stop the applications before the OOM-killer is triggered, >> I am also attaching the /proc/meminfo contents for comparison on a >> freshly-booted system vs. one where the two apps have been running for >> 200 seconds before quitting, with and without the prefetch code in >> kernel. >> Note how Active and Inactive show a lot of memory being "used" for something. >> The printouts were made when the system is completely idle with >> nothing running etc. >> >> >> I also tried prefetching from within the swapin_readahead() function >> (http://lxr.free-electrons.com/source/mm/swap_state.c?v=3.8#L373) with >> the same result - memory gets "used up". >> >> >> I would greatly appreciate any help! >> Pretty much ran out of ideas as to what could be causing this. >> Thanks a lot, >> Slava >> >> ============== code fragment used for prefetching ============ >> >> (( at the beginning of handle_pte_fault() function... >> int rett, ii; >> pte_t *ptee; >> swp_entry_t sentry; >> struct page *page; >> )) >> >> rett = do_swap_page(mm, vma, address, >> pte, pmd, flags, entry); >> >> //////////////////////////////////////////////////////////////// >> /// test for memory leaks//////////////////// >> for(ii=1;ii<10;ii++){ >> ptee = pte_offset_map(pmd, address + (ii*2)<<12); >> if(pte_present(*ptee) || pte_none(*ptee) || >> pte_file(*ptee)) return rett; // no such page in page table >> sentry = pte_to_swp_entry(*ptee); >> if (non_swap_entry(sentry)){ >> // printk(KERN_ALERT " non swap entry\n"); >> return rett; >> } >> delayacct_set_flag(DELAYACCT_PF_SWAPIN); >> page = lookup_swap_cache(sentry); > > That gets a reference to page. > >> if (!page) { >> page = read_swap_cache_async(sentry, >> GFP_HIGHUSER_MOVABLE, vma, address); >> if(page) page_cache_release(page); >> } >> lru_add_drain(); >> // if(page) unlock_page(page); > > But you don't appear to release that reference. > >> delayacct_clear_flag(DELAYACCT_PF_SWAPIN); >> } >> // printk(KERN_ALERT "swapped pages\n"); >> //////////////////////////////////////////// >> ////////////////////////////////////////// >> >> return rett; >> >> ===================== end code fragment ===================== ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2015-10-13 20:17 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <CAFCpxERnGypH6ZaVzY3KszN4rRq_3oeUcugCDG3Vs0gZooHP7A@mail.gmail.com>
2015-10-12 23:43 ` Fwd: Memory leakage when using read_swap_cache_async() ? Viacheslav Fedorov
2015-10-13 0:40 ` Hugh Dickins
2015-10-13 20:17 ` Viacheslav Fedorov
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®