From: Trevor Cordes <trevor@tecnopolis.ca>
To: Michal Hocko <mhocko@kernel.org>
Cc: Mel Gorman <mgorman@techsingularity.net>,
linux-kernel@vger.kernel.org,
Joonsoo Kim <iamjoonsoo.kim@lge.com>,
Minchan Kim <minchan@kernel.org>, Rik van Riel <riel@surriel.com>,
Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Subject: Re: mm, vmscan: commit makes PAE kernel crash nightly (bisected)
Date: Sun, 15 Jan 2017 00:27:52 -0600 [thread overview]
Message-ID: <20170115002752.4559f668@pog.tecnopolis.ca> (raw)
In-Reply-To: <20170112093613.GF2264@dhcp22.suse.cz>
[-- Attachment #1: Type: text/plain, Size: 1437 bytes --]
On 2017-01-12 Michal Hocko wrote:
> On Wed 11-01-17 16:52:32, Trevor Cordes wrote:
> [...]
> > I'm not sure how I can tell if my bug is because of memcgs so here
> > is a full first oom example (attached).
>
> 4.7 kernel doesn't contain 71c799f4982d ("mm: add per-zone lru list
> stat") so the OOM report will not tell us whether the Normal zone
> doesn't age active lists, unfortunatelly.
I compiled the patch Mel provided into the stock F23 kernel
4.8.13-100.fc23.i686+PAE and it ran for 2 nights. It didn't oom the
first night, but did the second night. So the bug persists even with
that patch. However, it does *seem* a bit "better" since it took 2
nights (usually takes only one, but maybe 10% of the time it does take
two) before oom'ing, *and* it allowed my reboot script to reboot it
cleanly when it saw the oom (which happens only 25% of the time).
I'm attaching the 4.8.13 oom message which should have the memcg info
(71c799f4982d) you are asking for above? Hopefully?
> You can easily check whether this is memcg related by trying to run
> the same workload with cgroup_disable=memory kernel command line
> parameter. This will put all the memcg specifics out of the way.
I will try booting now into cgroup_disable=memory to see if that helps
at all. I'll reply back in 48 hours, or when it oom's, whichever comes
first.
Also, should I bother trying the latest git HEAD to see if that solves
anything? Thanks!
[-- Attachment #2: oom2 --]
[-- Type: application/octet-stream, Size: 23540 bytes --]
Jan 14 03:14:40 firewallfsi kernel: [167409.074463] nmbd invoked oom-killer: gfp_mask=0x27000c0(GFP_KERNEL_ACCOUNT|__GFP_NOTRACK), order=1, oom_score_adj=0
Jan 14 03:14:40 firewallfsi kernel: [167409.074467] nmbd cpuset=/ mems_allowed=0
Jan 14 03:14:40 firewallfsi kernel: [167409.074472] CPU: 4 PID: 1519 Comm: nmbd Not tainted 4.8.13-101.fc23.i686+PAE #1
Jan 14 03:14:40 firewallfsi kernel: [167409.074473] Hardware name: Intel Corporation S1200BTL/S1200BTL, BIOS S1200BT.86B.02.00.0035.030220120927 03/02/2012
Jan 14 03:14:40 firewallfsi kernel: [167409.074478] cd640967 aaaf870b 00000286 efe9fd48 ccf5e9c7 efe9fe74 ed07d8c0 efe9fd90
Jan 14 03:14:40 firewallfsi kernel: [167409.074481] ccde0274 cd559508 ed84cb74 027000c0 efe9fe80 00000001 00000000 efe9fd70
Jan 14 03:14:40 firewallfsi kernel: [167409.074483] cd35cdad efe9fd90 ccf6472f 00000000 f101f0c0 e1284400 ed07d8c0 0000000a
Jan 14 03:14:40 firewallfsi kernel: [167409.074484] Call Trace:
Jan 14 03:14:40 firewallfsi kernel: [167409.074492] [<ccf5e9c7>] dump_stack+0x58/0x81
Jan 14 03:14:40 firewallfsi kernel: [167409.074497] [<ccde0274>] dump_header+0x4a/0x18f
Jan 14 03:14:40 firewallfsi kernel: [167409.074506] [<cd35cdad>] ? _raw_spin_unlock_irqrestore+0xd/0x10
Jan 14 03:14:40 firewallfsi kernel: [167409.074508] [<ccf6472f>] ? ___ratelimit+0x9f/0x100
Jan 14 03:14:40 firewallfsi kernel: [167409.074513] [<ccd7914a>] oom_kill_process+0x1ea/0x3b0
Jan 14 03:14:40 firewallfsi kernel: [167409.074515] [<ccc7687a>] ? has_capability_noaudit+0x1a/0x30
Jan 14 03:14:40 firewallfsi kernel: [167409.074517] [<ccd788eb>] ? oom_badness.part.12+0xcb/0x140
Jan 14 03:14:40 firewallfsi kernel: [167409.074522] [<ccd79569>] out_of_memory+0x1f9/0x230
Jan 14 03:14:40 firewallfsi kernel: [167409.074525] [<ccd7df15>] __alloc_pages_nodemask+0xba5/0xbc0
Jan 14 03:14:40 firewallfsi kernel: [167409.074530] [<ccc6a578>] copy_process.part.40+0x108/0x1490
Jan 14 03:14:40 firewallfsi kernel: [167409.074531] [<ccc6bac4>] _do_fork+0xd4/0x370
Jan 14 03:14:40 firewallfsi kernel: [167409.074539] [<ccd11fae>] ? __audit_syscall_exit+0x1ce/0x260
Jan 14 03:14:40 firewallfsi kernel: [167409.074541] [<ccc6be4c>] SyS_clone+0x2c/0x30
Jan 14 03:14:40 firewallfsi kernel: [167409.074548] [<ccc0378d>] do_fast_syscall_32+0x8d/0x140
Jan 14 03:14:40 firewallfsi kernel: [167409.074555] [<cd35d2b2>] sysenter_past_esp+0x47/0x75
Jan 14 03:14:40 firewallfsi kernel: [167409.074556] Mem-Info:
Jan 14 03:14:40 firewallfsi kernel: [167409.074566] active_anon:162369 inactive_anon:31581 isolated_anon:0
Jan 14 03:14:40 firewallfsi kernel: [167409.074566] active_file:152473 inactive_file:600139 isolated_file:0
Jan 14 03:14:40 firewallfsi kernel: [167409.074566] unevictable:0 dirty:0 writeback:0 unstable:0
Jan 14 03:14:40 firewallfsi kernel: [167409.074566] slab_reclaimable:180758 slab_unreclaimable:12487
Jan 14 03:14:40 firewallfsi kernel: [167409.074566] mapped:25326 shmem:1271 pagetables:1698 bounce:0
Jan 14 03:14:40 firewallfsi kernel: [167409.074566] free:63080 free_pcp:60 free_cma:0
Jan 14 03:14:40 firewallfsi kernel: [167409.074569] Node 0 active_anon:649476kB inactive_anon:126324kB active_file:609892kB inactive_file:2400556kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:101304kB dirty:0kB writeback:0kB shmem:5084kB writeback_tmp:0kB unstable:0kB pages_scanned:24907078 all_unreclaimable? yes
Jan 14 03:14:40 firewallfsi kernel: [167409.074572] DMA free:3200kB min:68kB low:84kB high:100kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15976kB managed:15900kB mlocked:0kB slab_reclaimable:11964kB slab_unreclaimable:452kB kernel_stack:56kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Jan 14 03:14:40 firewallfsi kernel: [167409.074573] lowmem_reserve[]: 0 783 4740 4740
Jan 14 03:14:40 firewallfsi kernel: [167409.074576] Normal free:3484kB min:3544kB low:4428kB high:5312kB active_anon:0kB inactive_anon:0kB active_file:3412kB inactive_file:1560kB unevictable:0kB writepending:0kB present:892920kB managed:815216kB mlocked:0kB slab_reclaimable:711068kB slab_unreclaimable:49496kB kernel_stack:2904kB pagetables:0kB bounce:0kB free_pcp:240kB local_pcp:120kB free_cma:0kB
Jan 14 03:14:40 firewallfsi kernel: [167409.074577] lowmem_reserve[]: 0 0 31652 31652
Jan 14 03:14:40 firewallfsi kernel: [167409.074581] HighMem free:245636kB min:512kB low:4988kB high:9464kB active_anon:649476kB inactive_anon:126324kB active_file:606332kB inactive_file:2399136kB unevictable:0kB writepending:0kB present:4051548kB managed:4051548kB mlocked:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:6792kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Jan 14 03:14:40 firewallfsi kernel: [167409.074582] lowmem_reserve[]: 0 0 0 0
Jan 14 03:14:40 firewallfsi kernel: [167409.074590] DMA: 26*4kB (UME) 19*8kB (UME) 20*16kB (UME) 10*32kB (UE) 8*64kB (UM) 10*128kB (UME) 2*256kB (M) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3200kB
Jan 14 03:14:40 firewallfsi kernel: [167409.074598] Normal: 508*4kB (UMEH) 7*8kB (H) 3*16kB (H) 1*32kB (H) 1*64kB (H) 0*128kB 1*256kB (H) 0*512kB 1*1024kB (H) 0*2048kB 0*4096kB = 3512kB
Jan 14 03:14:40 firewallfsi kernel: [167409.074609] HighMem: 317*4kB (UM) 150*8kB (UM) 96*16kB (UM) 43*32kB (UM) 296*64kB (UM) 55*128kB (M) 131*256kB (UM) 77*512kB (UM) 30*1024kB (M) 16*2048kB (M) 19*4096kB (M) = 245636kB
Jan 14 03:14:40 firewallfsi kernel: [167409.074614] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Jan 14 03:14:40 firewallfsi kernel: [167409.074614] 753924 total pagecache pages
Jan 14 03:14:40 firewallfsi kernel: [167409.074615] 12 pages in swap cache
Jan 14 03:14:40 firewallfsi kernel: [167409.074616] Swap cache stats: add 177, delete 165, find 0/0
Jan 14 03:14:40 firewallfsi kernel: [167409.074617] Free swap = 33783864kB
Jan 14 03:14:40 firewallfsi kernel: [167409.074617] Total swap = 33784572kB
Jan 14 03:14:40 firewallfsi kernel: [167409.074619] 1240111 pages RAM
Jan 14 03:14:40 firewallfsi kernel: [167409.074620] 1012887 pages HighMem/MovableOnly
Jan 14 03:14:40 firewallfsi kernel: [167409.074621] 19445 pages reserved
Jan 14 03:14:40 firewallfsi kernel: [167409.074622] 0 pages hwpoisoned
Jan 14 03:14:40 firewallfsi kernel: [167409.074625] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
Jan 14 03:14:40 firewallfsi kernel: [167409.074650] [ 598] 0 598 5325 3580 14 3 0 0 systemd-journal
Jan 14 03:14:40 firewallfsi kernel: [167409.074652] [ 637] 0 637 3585 1121 8 3 0 -1000 systemd-udevd
Jan 14 03:14:40 firewallfsi kernel: [167409.074655] [ 747] 0 747 808 481 6 3 0 0 mdadm
Jan 14 03:14:40 firewallfsi kernel: [167409.074659] [ 748] 81 748 1700 1020 7 3 0 -900 dbus-daemon
Jan 14 03:14:40 firewallfsi kernel: [167409.074662] [ 772] 0 772 1704 626 6 3 0 0 sh
Jan 14 03:14:40 firewallfsi kernel: [167409.074664] [ 774] 0 774 1033 637 5 3 0 0 irqbalance
Jan 14 03:14:40 firewallfsi kernel: [167409.074666] [ 776] 0 776 988 733 5 3 0 0 systemd-logind
Jan 14 03:14:40 firewallfsi kernel: [167409.074672] [ 779] 0 779 1704 663 7 3 0 0 sh
Jan 14 03:14:40 firewallfsi kernel: [167409.074673] [ 790] 0 790 1472 794 6 3 0 0 smartd
Jan 14 03:14:40 firewallfsi kernel: [167409.074678] [ 792] 0 792 1704 594 7 3 0 0 sh
Jan 14 03:14:40 firewallfsi kernel: [167409.074679] [ 800] 0 800 1704 623 6 3 0 0 sh
Jan 14 03:14:40 firewallfsi kernel: [167409.074680] [ 805] 0 805 1704 607 7 3 0 0 sh
Jan 14 03:14:40 firewallfsi kernel: [167409.074685] [ 806] 0 806 1704 602 7 3 0 0 sh
Jan 14 03:14:40 firewallfsi kernel: [167409.074686] [ 813] 0 813 8633 1106 11 3 0 0 rsyslogd
Jan 14 03:14:40 firewallfsi kernel: [167409.074688] [ 814] 288 814 19546 2254 18 3 0 0 milter-greylist
Jan 14 03:14:40 firewallfsi kernel: [167409.074690] [ 816] 0 816 1736 694 6 3 0 0 tickle-pog
Jan 14 03:14:40 firewallfsi kernel: [167409.074695] [ 817] 0 817 3165 2103 9 3 0 0 mailwarnings
Jan 14 03:14:40 firewallfsi kernel: [167409.074698] [ 820] 0 820 1819 809 6 3 0 0 watch-services
Jan 14 03:14:40 firewallfsi kernel: [167409.074704] [ 821] 0 821 3264 2173 10 3 0 0 restarter
Jan 14 03:14:40 firewallfsi kernel: [167409.074706] [ 824] 0 824 2707 1661 9 3 0 0 udp-sgr
Jan 14 03:14:40 firewallfsi kernel: [167409.074708] [ 825] 0 825 3313 2229 11 3 0 0 watch-ip
Jan 14 03:14:40 firewallfsi kernel: [167409.074713] [ 852] 0 852 583 440 5 3 0 0 acpid
Jan 14 03:14:40 firewallfsi kernel: [167409.074719] [ 853] 0 853 1704 604 6 3 0 0 sh
Jan 14 03:14:40 firewallfsi kernel: [167409.074725] [ 854] 0 854 3234 2142 9 3 0 0 dynamic-ip-upda
Jan 14 03:14:40 firewallfsi kernel: [167409.074731] [ 857] 0 857 1704 636 5 3 0 0 sh
Jan 14 03:14:40 firewallfsi kernel: [167409.074735] [ 858] 0 858 3695 1973 11 3 0 0 fetchmail
Jan 14 03:14:40 firewallfsi kernel: [167409.074737] [ 865] 0 865 7287 1135 11 3 0 0 apcupsd
Jan 14 03:14:40 firewallfsi kernel: [167409.074738] [ 907] 0 907 860 519 5 3 0 0 atd
Jan 14 03:14:40 firewallfsi kernel: [167409.074745] [ 980] 0 980 2665 1030 9 3 0 0 saslauthd
Jan 14 03:14:40 firewallfsi kernel: [167409.074750] [ 981] 0 981 2665 1030 9 3 0 0 saslauthd
Jan 14 03:14:40 firewallfsi kernel: [167409.074751] [ 982] 0 982 2499 125 8 3 0 0 saslauthd
Jan 14 03:14:40 firewallfsi kernel: [167409.074753] [ 983] 0 983 2499 125 8 3 0 0 saslauthd
Jan 14 03:14:40 firewallfsi kernel: [167409.074756] [ 984] 0 984 2499 125 8 3 0 0 saslauthd
Jan 14 03:14:40 firewallfsi kernel: [167409.074758] [ 1066] 27 1066 1707 731 7 3 0 0 mysqld_safe
Jan 14 03:14:40 firewallfsi kernel: [167409.074760] [ 1199] 0 1199 1116 532 6 3 0 0 agetty
Jan 14 03:14:40 firewallfsi kernel: [167409.074764] [ 1200] 0 1200 1116 484 6 3 0 0 agetty
Jan 14 03:14:40 firewallfsi kernel: [167409.074765] [ 1201] 0 1201 1116 480 6 3 0 0 agetty
Jan 14 03:14:40 firewallfsi kernel: [167409.074767] [ 1202] 0 1202 1116 499 6 3 0 0 agetty
Jan 14 03:14:40 firewallfsi kernel: [167409.074769] [ 1203] 0 1203 1116 495 6 3 0 0 agetty
Jan 14 03:14:40 firewallfsi kernel: [167409.074777] [ 1204] 0 1204 1116 527 6 3 0 0 agetty
Jan 14 03:14:40 firewallfsi kernel: [167409.074783] [ 1373] 27 1373 126042 14546 65 3 0 0 mysqld
Jan 14 03:14:40 firewallfsi kernel: [167409.074787] [ 1374] 0 1374 2769 737 9 3 0 -1000 sshd
Jan 14 03:14:40 firewallfsi kernel: [167409.074789] [ 1479] 25 1479 82956 48992 116 3 0 0 named
Jan 14 03:14:40 firewallfsi kernel: [167409.074790] [ 1519] 0 1519 7049 2436 18 3 0 0 nmbd
Jan 14 03:14:40 firewallfsi kernel: [167409.074793] [ 1520] 0 1520 6711 2140 17 3 0 0 nmbd
Jan 14 03:14:40 firewallfsi kernel: [167409.074794] [ 1541] 0 1541 12052 6348 28 3 0 0 /usr/sbin/httpd
Jan 14 03:14:40 firewallfsi kernel: [167409.074796] [ 1769] 48 1769 49262 4341 46 3 0 0 /usr/sbin/httpd
Jan 14 03:14:40 firewallfsi kernel: [167409.074798] [ 1770] 48 1770 16216 4312 29 3 0 0 /usr/sbin/httpd
Jan 14 03:14:40 firewallfsi kernel: [167409.074799] [ 1778] 48 1778 16214 4322 29 3 0 0 /usr/sbin/httpd
Jan 14 03:14:40 firewallfsi kernel: [167409.074806] [ 1792] 48 1792 16220 4322 29 3 0 0 /usr/sbin/httpd
Jan 14 03:14:40 firewallfsi kernel: [167409.074810] [ 1804] 0 1804 9561 3578 22 3 0 0 smbd
Jan 14 03:14:40 firewallfsi kernel: [167409.074812] [ 1805] 0 1805 9197 1097 22 3 0 0 smbd
Jan 14 03:14:40 firewallfsi kernel: [167409.074814] [ 1806] 0 1806 9446 1166 22 3 0 0 smbd
Jan 14 03:14:40 firewallfsi kernel: [167409.074818] [ 1819] 0 1819 1704 639 6 3 0 0 sh
Jan 14 03:14:40 firewallfsi kernel: [167409.074819] [ 1820] 0 1820 2740 1707 9 3 0 0 udp-sgs
Jan 14 03:14:40 firewallfsi kernel: [167409.074821] [ 1889] 0 1889 5116 2456 14 3 0 0 dhclient
Jan 14 03:14:40 firewallfsi kernel: [167409.074826] [ 1971] 0 1971 594 401 5 3 0 0 pptpd
Jan 14 03:14:40 firewallfsi kernel: [167409.074828] [ 1978] 0 1978 954 664 5 3 0 0 dovecot
Jan 14 03:14:40 firewallfsi kernel: [167409.074829] [ 1979] 97 1979 904 521 5 3 0 0 anvil
Jan 14 03:14:40 firewallfsi kernel: [167409.074832] [ 1980] 0 1980 937 577 5 3 0 0 log
Jan 14 03:14:40 firewallfsi kernel: [167409.074839] [ 1982] 0 1982 1131 812 5 3 0 0 config
Jan 14 03:14:40 firewallfsi kernel: [167409.074841] [ 1989] 177 1989 6833 4754 16 3 0 0 dhcpd
Jan 14 03:14:40 firewallfsi kernel: [167409.074842] [ 1990] 0 1990 1900 731 8 3 0 0 crond
Jan 14 03:14:40 firewallfsi kernel: [167409.074843] [ 1992] 38 1992 1536 1057 6 3 0 0 ntpd
Jan 14 03:14:40 firewallfsi kernel: [167409.074845] [ 2007] 48 2007 16220 4532 29 3 0 0 /usr/sbin/httpd
Jan 14 03:14:40 firewallfsi kernel: [167409.074851] [ 2028] 303 2028 1402 954 6 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074852] [ 2029] 48 2029 16386 4598 30 3 0 0 /usr/sbin/httpd
Jan 14 03:14:40 firewallfsi kernel: [167409.074854] [ 2156] 301 2156 1607 1122 6 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074855] [ 2157] 0 2157 4181 1974 11 3 0 0 sshd
Jan 14 03:14:40 firewallfsi kernel: [167409.074859] [ 2159] 0 2159 1584 1090 7 3 0 0 systemd
Jan 14 03:14:40 firewallfsi kernel: [167409.074861] [ 2165] 0 2165 6809 395 11 3 0 0 (sd-pam)
Jan 14 03:14:40 firewallfsi kernel: [167409.074863] [ 2188] 0 2188 4215 973 11 3 0 0 sshd
Jan 14 03:14:40 firewallfsi kernel: [167409.074867] [ 2195] 0 2195 1229 973 6 3 0 0 tcsh
Jan 14 03:14:40 firewallfsi kernel: [167409.074868] [ 2350] 300 2350 11578 4448 26 3 0 0 smbd
Jan 14 03:14:40 firewallfsi kernel: [167409.074870] [ 2555] 304 2555 1399 1004 6 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074875] [ 2966] 302 2966 1449 1120 6 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074877] [22531] 301 22531 2007 1275 7 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074878] [22745] 301 22745 1747 1185 8 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074880] [22750] 301 22750 1458 1052 7 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074884] [22753] 301 22753 4216 1688 9 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074885] [25574] 0 25574 4206 1966 11 3 0 0 sshd
Jan 14 03:14:40 firewallfsi kernel: [167409.074887] [25577] 0 25577 4206 1255 11 3 0 0 sshd
Jan 14 03:14:40 firewallfsi kernel: [167409.074893] [25588] 0 25588 1994 1021 8 3 0 0 tcsh
Jan 14 03:14:40 firewallfsi kernel: [167409.074895] [25616] 0 25616 2664 1687 8 3 0 0 ssh
Jan 14 03:14:40 firewallfsi kernel: [167409.074897] [29170] 273 29170 2208 1411 8 3 0 0 imap-login
Jan 14 03:14:40 firewallfsi kernel: [167409.074898] [29174] 301 29174 1684 1186 6 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074900] [29582] 273 29582 2209 1394 8 3 0 0 imap-login
Jan 14 03:14:40 firewallfsi kernel: [167409.074902] [29583] 301 29583 2006 1198 7 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074909] [ 6411] 273 6411 2208 1394 8 3 0 0 imap-login
Jan 14 03:14:40 firewallfsi kernel: [167409.074910] [ 6417] 301 6417 1446 1063 6 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074912] [13238] 273 13238 2208 1447 8 3 0 0 imap-login
Jan 14 03:14:40 firewallfsi kernel: [167409.074917] [13242] 301 13242 2516 1277 7 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074918] [ 5077] 48 5077 16220 4586 29 3 0 0 /usr/sbin/httpd
Jan 14 03:14:40 firewallfsi kernel: [167409.074919] [ 5080] 48 5080 16216 4317 29 3 0 0 /usr/sbin/httpd
Jan 14 03:14:40 firewallfsi kernel: [167409.074923] [ 5082] 48 5082 16220 4576 29 3 0 0 /usr/sbin/httpd
Jan 14 03:14:40 firewallfsi kernel: [167409.074925] [31732] 0 31732 1307 924 6 3 0 0 reboot-when-oom
Jan 14 03:14:40 firewallfsi kernel: [167409.074928] [31733] 0 31733 1307 909 6 3 0 0 watch-ps
Jan 14 03:14:40 firewallfsi kernel: [167409.074930] [ 7082] 273 7082 2208 1411 8 3 0 0 imap-login
Jan 14 03:14:40 firewallfsi kernel: [167409.074931] [ 7086] 301 7086 3832 1390 9 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074933] [29840] 276 29840 116776 103156 222 3 68 0 clamd
Jan 14 03:14:40 firewallfsi kernel: [167409.074936] [29858] 290 29858 15448 1219 13 3 0 0 clamav-milter
Jan 14 03:14:40 firewallfsi kernel: [167409.074937] [29881] 0 29881 3830 1710 11 3 0 0 sendmail
Jan 14 03:14:40 firewallfsi kernel: [167409.074939] [29898] 51 29898 3502 769 10 3 0 0 sendmail
Jan 14 03:14:40 firewallfsi kernel: [167409.074941] [30041] 23 30041 5440 724 14 3 0 0 squid
Jan 14 03:14:40 firewallfsi kernel: [167409.074949] [30043] 23 30043 11724 8388 25 3 0 0 squid
Jan 14 03:14:40 firewallfsi kernel: [167409.074950] [30044] 23 30044 1179 261 6 3 0 0 unlinkd
Jan 14 03:14:40 firewallfsi kernel: [167409.074957] [21973] 273 21973 2209 1417 8 3 0 0 imap-login
Jan 14 03:14:40 firewallfsi kernel: [167409.074959] [21977] 302 21977 1429 1059 6 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074960] [ 3848] 48 3848 16183 3641 29 3 0 0 /usr/sbin/httpd
Jan 14 03:14:40 firewallfsi kernel: [167409.074962] [11059] 273 11059 2209 1442 9 3 0 0 imap-login
Jan 14 03:14:40 firewallfsi kernel: [167409.074968] [11060] 273 11060 2209 1436 8 3 0 0 imap-login
Jan 14 03:14:40 firewallfsi kernel: [167409.074970] [11061] 273 11061 2209 1454 8 3 0 0 imap-login
Jan 14 03:14:40 firewallfsi kernel: [167409.074976] [11062] 273 11062 2209 1422 8 3 0 0 imap-login
Jan 14 03:14:40 firewallfsi kernel: [167409.074978] [11063] 301 11063 1401 978 6 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074981] [11066] 301 11066 1399 948 6 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074985] [11067] 301 11067 1616 1096 6 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074987] [11068] 301 11068 1396 929 6 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.074989] [23879] 0 23879 3839 1924 11 3 0 0 sendmail
Jan 14 03:14:40 firewallfsi kernel: [167409.074996] [25771] 0 25771 1396 158 6 3 0 0 sleep
Jan 14 03:14:40 firewallfsi kernel: [167409.074998] [27161] 273 27161 2209 1418 8 3 0 0 imap-login
Jan 14 03:14:40 firewallfsi kernel: [167409.075000] [27165] 301 27165 1436 1049 6 3 0 0 imap
Jan 14 03:14:40 firewallfsi kernel: [167409.075001] [27432] 0 27432 1654 406 7 3 0 0 anacron
Jan 14 03:14:40 firewallfsi kernel: [167409.075011] [29124] 0 29124 1396 160 7 3 0 0 sleep
Jan 14 03:14:40 firewallfsi kernel: [167409.075014] Out of memory: Kill process 29840 (clamd) score 10 or sacrifice child
Jan 14 03:14:40 firewallfsi kernel: [167409.075026] Killed process 29840 (clamd) total-vm:467104kB, anon-rss:396772kB, file-rss:15852kB, shmem-rss:0kB
next prev parent reply other threads:[~2017-01-15 6:36 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-01-11 10:32 Trevor Cordes
2017-01-11 12:11 ` Mel Gorman
2017-01-11 12:14 ` Mel Gorman
2017-01-11 22:52 ` Trevor Cordes
2017-01-12 9:36 ` Michal Hocko
2017-01-15 6:27 ` Trevor Cordes [this message]
2017-01-16 11:09 ` Mel Gorman
2017-01-17 13:52 ` Michal Hocko
2017-01-17 14:21 ` Mel Gorman
2017-01-17 14:54 ` Michal Hocko
2017-01-18 7:25 ` Trevor Cordes
2017-01-18 17:48 ` Mel Gorman
2017-01-18 18:07 ` Mel Gorman
2017-01-19 9:48 ` Trevor Cordes
2017-01-19 11:37 ` Michal Hocko
2017-01-20 6:35 ` Trevor Cordes
2017-01-20 11:02 ` Mel Gorman
2017-01-20 15:55 ` Mel Gorman
2017-01-23 0:45 ` Trevor Cordes
2017-01-23 10:48 ` Mel Gorman
2017-01-23 11:04 ` Mel Gorman
2017-01-25 9:46 ` Michal Hocko
2017-01-24 12:59 ` Michal Hocko
2017-01-25 10:02 ` Trevor Cordes
2017-01-25 12:04 ` Michal Hocko
2017-01-29 22:50 ` Trevor Cordes
2017-01-30 7:51 ` Michal Hocko
2017-02-01 9:29 ` Trevor Cordes
2017-02-01 10:14 ` Michal Hocko
2017-02-04 0:36 ` Trevor Cordes
2017-02-04 20:05 ` Rik van Riel
2017-02-05 10:03 ` Michal Hocko
2017-02-05 22:53 ` Trevor Cordes
2017-01-30 9:10 ` Mel Gorman
2017-01-24 12:54 ` Michal Hocko
2017-01-26 23:18 ` Trevor Cordes
2017-01-27 7:36 ` Michal Hocko
2017-01-24 12:51 ` Michal Hocko
2017-01-18 6:52 ` Trevor Cordes
2017-01-17 13:45 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170115002752.4559f668@pog.tecnopolis.ca \
--to=trevor@tecnopolis.ca \
--cc=iamjoonsoo.kim@lge.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@techsingularity.net \
--cc=mhocko@kernel.org \
--cc=minchan@kernel.org \
--cc=riel@surriel.com \
--cc=srikar@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®