From: Trevor Cordes <trevor@tecnopolis.ca>
To: Michal Hocko <mhocko@kernel.org>
Cc: Mel Gorman <mgorman@techsingularity.net>,
linux-kernel@vger.kernel.org,
Joonsoo Kim <iamjoonsoo.kim@lge.com>,
Minchan Kim <minchan@kernel.org>, Rik van Riel <riel@surriel.com>,
Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Subject: Re: mm, vmscan: commit makes PAE kernel crash nightly (bisected)
Date: Thu, 26 Jan 2017 17:18:58 -0600 [thread overview]
Message-ID: <20170126171858.5e1b484a@pog.tecnopolis.ca> (raw)
In-Reply-To: <20170124125458.GH6867@dhcp22.suse.cz>
[-- Attachment #1: Type: text/plain, Size: 1717 bytes --]
On 2017-01-24 Michal Hocko wrote:
> On Sun 22-01-17 18:45:59, Trevor Cordes wrote:
> [...]
> > Also, completely separate from your patch I ran mhocko's 4.9 tree
> > with mem=2G to see if lower ram amount would help, but it didn't.
> > Even with 2G the system oom and hung same as usual. So far the
> > only thing that helps at all was the cgroup_disable=memory option,
> > which makes the problem disappear completely for me.
>
> OK, can we reduce the problem space slightly more and could you boot
> with kmem accounting enabled? cgroup.memory=nokmem,nosocket
I ran for 30 hours with cgroup.memory=nokmem,nosocket using vanilla
4.9.0+ and it oom'd during a big rdiff-backup at 9am. My script was
able to reboot it before it hung. Only one oom occurred before the
reboot, which is a bit odd, usually there is 5-50. See attached
messages log (oom6).
So, still, only cgroup_disable=memory mitigates this bug (so far). If
you need me to test cgroup.memory=nokmem,nosocket with your since-4.9
branch specifically, let me know and I'll add it to the to-test list.
On 2017-01-25 Michal Hocko wrote:
> On Wed 25-01-17 04:02:46, Trevor Cordes wrote:
> > OK, I patched & compiled mhocko's git tree from the other day
> > 4.9.0+. (To confirm, weird, but mhocko's git tree I'm using from a
> > couple of weeks ago shows the newest commit (git log) is
> > 69973b830859bc6529a7a0468ba0d80ee5117826 "Linux 4.9"? Let me know
> > if I'm doing something wrong, see below.)
>
> My fault. I should have noted that you should use since-4.9 branch.
OK, I got it now, I'm retesting the runs I did (with/without the
various patches) on your git tree and will re-report the (correct)
results. Will take a few days. Thanks!
[-- Attachment #2: oom6 --]
[-- Type: application/octet-stream, Size: 23915 bytes --]
Jan 26 08:59:46 firewallfsi kernel: [104057.183579] rdiff-backup invoked oom-killer: gfp_mask=0x2420848(GFP_NOFS|__GFP_NOFAIL|__GFP_HARDWALL|__GFP_MOVABLE), nodemask=0, order=0, oom_score_adj=0
Jan 26 08:59:46 firewallfsi kernel: [104057.186272] rdiff-backup cpuset=/ mems_allowed=0
Jan 26 08:59:46 firewallfsi kernel: [104057.187565] CPU: 3 PID: 18243 Comm: rdiff-backup Not tainted 4.9.0+ #2
Jan 26 08:59:46 firewallfsi kernel: [104057.188821] Hardware name: Intel Corporation S1200BTL/S1200BTL, BIOS S1200BT.86B.02.00.0035.030220120927 03/02/2012
Jan 26 08:59:46 firewallfsi kernel: [104057.190083] de14fab0 ceb604e7 de14fbe8 eeb95a00 de14fae0 ce9e17a6 de14fac0 cef63bad
Jan 26 08:59:46 firewallfsi kernel: [104057.191343] de14fae0 ceb6625f de14fae4 ea682c00 f6ba2600 eeb95a00 cf168bce de14fbe8
Jan 26 08:59:46 firewallfsi kernel: [104057.192588] de14fb24 ce97aff7 ce876d8a de14fb10 ce97ac6b 00000006 00000000 0000000c
Jan 26 08:59:46 firewallfsi kernel: [104057.193821] Call Trace:
Jan 26 08:59:46 firewallfsi kernel: [104057.195025] [<ceb604e7>] dump_stack+0x58/0x81
Jan 26 08:59:46 firewallfsi kernel: [104057.196213] [<ce9e17a6>] dump_header+0x64/0x1a6
Jan 26 08:59:46 firewallfsi kernel: [104057.197381] [<cef63bad>] ? _raw_spin_unlock_irqrestore+0xd/0x10
Jan 26 08:59:46 firewallfsi kernel: [104057.198536] [<ceb6625f>] ? ___ratelimit+0x9f/0x100
Jan 26 08:59:46 firewallfsi kernel: [104057.199672] [<ce97aff7>] oom_kill_process+0x207/0x3d0
Jan 26 08:59:46 firewallfsi kernel: [104057.200790] [<ce876d8a>] ? has_capability_noaudit+0x1a/0x30
Jan 26 08:59:46 firewallfsi kernel: [104057.201893] [<ce97ac6b>] ? oom_badness.part.13+0xcb/0x140
Jan 26 08:59:46 firewallfsi kernel: [104057.202979] [<ce97b4d8>] out_of_memory+0xf8/0x2a0
Jan 26 08:59:46 firewallfsi kernel: [104057.204045] [<ce980016>] __alloc_pages_nodemask+0xc46/0xd10
Jan 26 08:59:46 firewallfsi kernel: [104057.205096] [<ce9768c2>] ? find_get_entry+0x22/0x160
Jan 26 08:59:46 firewallfsi kernel: [104057.206131] [<ceb32010>] ? generic_make_request+0xd0/0x1b0
Jan 26 08:59:46 firewallfsi kernel: [104057.207148] [<ce97727e>] pagecache_get_page+0xbe/0x2d0
Jan 26 08:59:46 firewallfsi kernel: [104057.208148] [<cea19044>] __getblk_gfp+0x104/0x360
Jan 26 08:59:46 firewallfsi kernel: [104057.209127] [<cea19a3a>] ? submit_bh_wbc+0x14a/0x1f0
Jan 26 08:59:46 firewallfsi kernel: [104057.210097] [<cea1a6bb>] __breadahead+0x2b/0x70
Jan 26 08:59:46 firewallfsi kernel: [104057.211039] [<cea690ee>] __ext4_get_inode_loc+0x40e/0x440
Jan 26 08:59:46 firewallfsi kernel: [104057.211966] [<ce9feaf9>] ? inode_init_always+0x119/0x1c0
Jan 26 08:59:46 firewallfsi kernel: [104057.212874] [<cea6c07b>] ext4_iget+0x7b/0xaf0
Jan 26 08:59:46 firewallfsi kernel: [104057.213762] [<ce9fd683>] ? __d_alloc+0x23/0x190
Jan 26 08:59:46 firewallfsi kernel: [104057.214632] [<cea6cb1f>] ext4_iget_normal+0x2f/0x40
Jan 26 08:59:46 firewallfsi kernel: [104057.215483] [<cea779c5>] ext4_lookup+0xb5/0x240
Jan 26 08:59:46 firewallfsi kernel: [104057.216315] [<ce9ef257>] ? legitimize_path.isra.33+0x27/0x60
Jan 26 08:59:46 firewallfsi kernel: [104057.217132] [<ce9ef3fe>] ? unlazy_walk+0x16e/0x1a0
Jan 26 08:59:46 firewallfsi kernel: [104057.217929] [<ce9f01cc>] lookup_slow+0x7c/0x130
Jan 26 08:59:46 firewallfsi kernel: [104057.218710] [<ce9f0da4>] walk_component+0x1e4/0x300
Jan 26 08:59:46 firewallfsi kernel: [104057.219475] [<ce9efdde>] ? path_init+0x19e/0x330
Jan 26 08:59:46 firewallfsi kernel: [104057.220221] [<ce9ee69f>] ? terminate_walk+0x8f/0x100
Jan 26 08:59:46 firewallfsi kernel: [104057.220949] [<ce9f1f83>] path_lookupat+0x53/0xe0
Jan 26 08:59:46 firewallfsi kernel: [104057.221657] [<ce9f4127>] filename_lookup+0x97/0x190
Jan 26 08:59:46 firewallfsi kernel: [104057.222348] [<ce9cf1ad>] ? kmem_cache_alloc+0x15d/0x1c0
Jan 26 08:59:46 firewallfsi kernel: [104057.223021] [<ce9f3d6a>] ? getname_flags+0x3a/0x1a0
Jan 26 08:59:46 firewallfsi kernel: [104057.223676] [<ce9f42f6>] user_path_at_empty+0x36/0x40
Jan 26 08:59:46 firewallfsi kernel: [104057.224311] [<ce9e9a10>] vfs_fstatat+0x60/0xb0
Jan 26 08:59:46 firewallfsi kernel: [104057.224928] [<ce9ea40d>] SyS_lstat64+0x2d/0x50
Jan 26 08:59:46 firewallfsi kernel: [104057.225524] [<ce912e8e>] ? __audit_syscall_exit+0x1ce/0x260
Jan 26 08:59:46 firewallfsi kernel: [104057.226105] [<ce8035a6>] ? syscall_slow_exit_work+0xd6/0xe0
Jan 26 08:59:46 firewallfsi kernel: [104057.226667] [<ce80377a>] do_fast_syscall_32+0x8a/0x150
Jan 26 08:59:46 firewallfsi kernel: [104057.227212] [<cef640ca>] sysenter_past_esp+0x47/0x75
Jan 26 08:59:46 firewallfsi kernel: [104057.227745] Mem-Info:
Jan 26 08:59:46 firewallfsi kernel: [104057.228569] active_anon:126164 inactive_anon:65161 isolated_anon:0
Jan 26 08:59:46 firewallfsi kernel: [104057.228569] active_file:193794 inactive_file:602556 isolated_file:0
Jan 26 08:59:46 firewallfsi kernel: [104057.228569] unevictable:0 dirty:198 writeback:0 unstable:0
Jan 26 08:59:46 firewallfsi kernel: [104057.228569] slab_reclaimable:97990 slab_unreclaimable:9475
Jan 26 08:59:46 firewallfsi kernel: [104057.228569] mapped:24098 shmem:324 pagetables:1512 bounce:0
Jan 26 08:59:46 firewallfsi kernel: [104057.228569] free:109080 free_pcp:1076 free_cma:0
Jan 26 08:59:46 firewallfsi kernel: [104057.233530] Node 0 active_anon:504656kB inactive_anon:260644kB active_file:775176kB inactive_file:2410224kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:96412kB dirty:852kB writeback:0kB shmem:1296kB writeback_tmp:0kB unstable:0kB pages_scanned:0 all_unreclaimable? no
Jan 26 08:59:46 firewallfsi kernel: [104057.236152] DMA free:3180kB min:68kB low:84kB high:100kB active_anon:0kB inactive_anon:0kB active_file:5536kB inactive_file:0kB unevictable:0kB writepending:0kB present:15976kB managed:15900kB mlocked:0kB slab_reclaimable:7016kB slab_unreclaimable:24kB kernel_stack:8kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Jan 26 08:59:46 firewallfsi kernel: lowmem_reserve[]: 0 778 4734 4734
Jan 26 08:59:46 firewallfsi kernel: [104057.240078] Normal free:3504kB min:3532kB low:4412kB high:5292kB active_anon:260kB inactive_anon:0kB active_file:348276kB inactive_file:100kB unevictable:0kB writepending:380kB present:892920kB managed:817180kB mlocked:0kB slab_reclaimable:384944kB slab_unreclaimable:37876kB kernel_stack:2552kB pagetables:0kB bounce:0kB free_pcp:2000kB local_pcp:276kB free_cma:0kB
Jan 26 08:59:46 firewallfsi kernel: lowmem_reserve[]: 0 0 31652 31652
Jan 26 08:59:46 firewallfsi kernel: [104057.244247] HighMem free:429636kB min:512kB low:5000kB high:9488kB active_anon:504396kB inactive_anon:260644kB active_file:421364kB inactive_file:2410124kB unevictable:0kB writepending:500kB present:4051548kB managed:4051548kB mlocked:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:6048kB bounce:0kB free_pcp:2300kB local_pcp:308kB free_cma:0kB
Jan 26 08:59:46 firewallfsi kernel: lowmem_reserve[]: 0 0 0 0
Jan 26 08:59:46 firewallfsi kernel: [104057.248837] DMA: 5*4kB (UE) 21*8kB (UME) 17*16kB (UME) 17*32kB (UE) 8*64kB (UME) 3*128kB (UE) 3*256kB (ME) 1*512kB (U) 0*1024kB 0*2048kB 0*4096kB = 3180kB
Jan 26 08:59:46 firewallfsi kernel: Normal: 432*4kB (UMEH) 24*8kB (UH) 13*16kB (H) 11*32kB (H) 8*64kB (H) 4*128kB (H) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3504kB
Jan 26 08:59:46 firewallfsi kernel: HighMem: 7*4kB (U) 205*8kB (UM) 58*16kB (UM) 21*32kB (UM) 12*64kB (UM) 11*128kB (UM) 3*256kB (UM) 31*512kB (M) 4*1024kB (M) 1*2048kB (M) 98*4096kB (M) = 429636kB
Jan 26 08:59:46 firewallfsi kernel: [104057.256399] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Jan 26 08:59:46 firewallfsi kernel: [104057.257716] 796796 total pagecache pages
Jan 26 08:59:46 firewallfsi kernel: [104057.259050] 99 pages in swap cache
Jan 26 08:59:46 firewallfsi kernel: [104057.260364] Swap cache stats: add 1286, delete 1187, find 150/171
Jan 26 08:59:46 firewallfsi kernel: [104057.261639] Free swap = 33779604kB
Jan 26 08:59:46 firewallfsi kernel: [104057.262912] Total swap = 33784572kB
Jan 26 08:59:46 firewallfsi kernel: [104057.264179] 1240111 pages RAM
Jan 26 08:59:46 firewallfsi kernel: [104057.265440] 1012887 pages HighMem/MovableOnly
Jan 26 08:59:46 firewallfsi kernel: [104057.266706] 18954 pages reserved
Jan 26 08:59:46 firewallfsi kernel: [104057.267969] 0 pages hwpoisoned
Jan 26 08:59:46 firewallfsi kernel: [104057.269225] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
Jan 26 08:59:46 firewallfsi kernel: [104057.270519] [ 588] 0 588 3279 1647 10 3 29 0 systemd-journal
Jan 26 08:59:46 firewallfsi kernel: [104057.271779] [ 624] 0 624 3592 832 8 3 344 -1000 systemd-udevd
Jan 26 08:59:46 firewallfsi kernel: [104057.273037] [ 728] 0 728 8633 1065 10 3 0 0 rsyslogd
Jan 26 08:59:46 firewallfsi kernel: [104057.274298] [ 733] 0 733 1704 625 6 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.275559] [ 736] 0 736 1033 628 5 3 0 0 irqbalance
Jan 26 08:59:46 firewallfsi kernel: [104057.276816] [ 737] 288 737 16729 1733 19 3 0 0 milter-greylist
Jan 26 08:59:46 firewallfsi kernel: [104057.278074] [ 739] 0 739 1704 616 6 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.279326] [ 750] 0 750 1704 644 6 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.280573] [ 751] 0 751 1704 676 6 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.281764] [ 767] 0 767 1005 714 6 3 0 0 systemd-logind
Jan 26 08:59:46 firewallfsi kernel: [104057.282949] [ 768] 0 768 1472 995 6 3 0 0 smartd
Jan 26 08:59:46 firewallfsi kernel: [104057.284124] [ 771] 0 771 1704 675 7 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.285288] [ 772] 0 772 1704 657 6 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.286442] [ 777] 0 777 800 486 6 3 0 0 mdadm
Jan 26 08:59:46 firewallfsi kernel: [104057.287590] [ 779] 0 779 1704 627 7 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.288729] [ 790] 81 790 1700 1036 7 3 0 -900 dbus-daemon
Jan 26 08:59:46 firewallfsi kernel: [104057.289848] [ 810] 0 810 3266 2194 10 3 0 0 restarter
Jan 26 08:59:46 firewallfsi kernel: [104057.290942] [ 811] 0 811 3234 2159 9 3 0 0 dynamic-ip-upda
Jan 26 08:59:46 firewallfsi kernel: [104057.291965] [ 812] 0 812 3165 2073 10 3 0 0 mailwarnings
Jan 26 08:59:46 firewallfsi kernel: [104057.292949] [ 814] 0 814 1736 717 6 3 0 0 tickle-pog
Jan 26 08:59:46 firewallfsi kernel: [104057.293907] [ 815] 0 815 1819 804 6 3 0 0 watch-services
Jan 26 08:59:46 firewallfsi kernel: [104057.294839] [ 816] 0 816 2708 1666 9 3 0 0 udp-sgr
Jan 26 08:59:46 firewallfsi kernel: [104057.295754] [ 817] 0 817 3695 1987 10 3 0 0 fetchmail
Jan 26 08:59:46 firewallfsi kernel: [104057.296646] [ 845] 0 845 2499 370 9 3 0 0 saslauthd
Jan 26 08:59:46 firewallfsi kernel: [104057.297512] [ 846] 0 846 2518 912 9 3 0 0 saslauthd
Jan 26 08:59:46 firewallfsi kernel: [104057.298343] [ 847] 0 847 2499 126 9 3 0 0 saslauthd
Jan 26 08:59:46 firewallfsi kernel: [104057.299150] [ 848] 0 848 2518 912 9 3 0 0 saslauthd
Jan 26 08:59:46 firewallfsi kernel: [104057.299942] [ 849] 0 849 2499 126 9 3 0 0 saslauthd
Jan 26 08:59:46 firewallfsi kernel: [104057.300691] [ 888] 0 888 2769 731 9 3 0 -1000 sshd
Jan 26 08:59:46 firewallfsi kernel: [104057.301407] [ 930] 0 930 1704 634 6 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.302105] [ 931] 0 931 3309 2240 10 3 0 0 watch-ip
Jan 26 08:59:46 firewallfsi kernel: [104057.302765] [ 932] 0 932 583 403 5 3 0 0 acpid
Jan 26 08:59:46 firewallfsi kernel: [104057.303380] [ 937] 0 937 7287 1292 11 3 0 0 apcupsd
Jan 26 08:59:46 firewallfsi kernel: [104057.303975] [ 973] 0 973 860 491 5 3 0 0 atd
Jan 26 08:59:46 firewallfsi kernel: [104057.304544] [ 1036] 0 1036 7049 2372 16 3 0 0 nmbd
Jan 26 08:59:46 firewallfsi kernel: [104057.305081] [ 1037] 0 1037 6840 2278 16 3 0 0 nmbd
Jan 26 08:59:46 firewallfsi kernel: [104057.305593] [ 1052] 25 1052 82757 48632 115 3 0 0 named
Jan 26 08:59:46 firewallfsi kernel: [104057.306084] [ 1065] 27 1065 1707 767 6 3 0 0 mysqld_safe
Jan 26 08:59:46 firewallfsi kernel: [104057.306568] [ 1235] 27 1235 126038 14727 64 3 0 0 mysqld
Jan 26 08:59:46 firewallfsi kernel: [104057.307046] [ 1239] 0 1239 12052 6480 27 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.307528] [ 1357] 0 1357 1116 484 6 3 0 0 agetty
Jan 26 08:59:46 firewallfsi kernel: [104057.307996] [ 1358] 0 1358 1116 545 6 3 0 0 agetty
Jan 26 08:59:46 firewallfsi kernel: [104057.308466] [ 1359] 0 1359 1116 546 6 3 0 0 agetty
Jan 26 08:59:46 firewallfsi kernel: [104057.308926] [ 1360] 0 1360 1116 487 5 3 0 0 agetty
Jan 26 08:59:46 firewallfsi kernel: [104057.309382] [ 1361] 0 1361 1116 519 6 3 0 0 agetty
Jan 26 08:59:46 firewallfsi kernel: [104057.309836] [ 1362] 0 1362 1116 477 6 3 0 0 agetty
Jan 26 08:59:46 firewallfsi kernel: [104057.310281] [ 1578] 48 1578 49258 4298 45 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.310724] [ 1579] 48 1579 16216 4543 28 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.311144] [ 1582] 48 1582 16220 4299 28 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.311561] [ 1599] 48 1599 16386 4796 29 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.311966] [ 1604] 48 1604 16386 4558 29 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.312366] [ 1754] 0 1754 1704 632 6 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.312773] [ 1755] 0 1755 2741 1664 9 3 0 0 udp-sgs
Jan 26 08:59:46 firewallfsi kernel: [104057.313167] [ 1759] 0 1759 9561 3551 22 3 0 0 smbd
Jan 26 08:59:46 firewallfsi kernel: [104057.313551] [ 1760] 0 1760 9197 1115 22 3 0 0 smbd
Jan 26 08:59:46 firewallfsi kernel: [104057.313931] [ 1761] 0 1761 9446 1192 22 3 0 0 smbd
Jan 26 08:59:46 firewallfsi kernel: [104057.314311] [ 1850] 0 1850 5116 2511 13 3 0 0 dhclient
Jan 26 08:59:46 firewallfsi kernel: [104057.314698] [ 1927] 0 1927 594 420 5 3 0 0 pptpd
Jan 26 08:59:46 firewallfsi kernel: [104057.315080] [ 1929] 0 1929 954 672 6 3 0 0 dovecot
Jan 26 08:59:46 firewallfsi kernel: [104057.315467] [ 1930] 97 1930 904 548 5 3 0 0 anvil
Jan 26 08:59:46 firewallfsi kernel: [104057.315853] [ 1931] 0 1931 937 581 5 3 0 0 log
Jan 26 08:59:46 firewallfsi kernel: [104057.316238] [ 1938] 0 1938 1899 739 7 3 0 0 crond
Jan 26 08:59:46 firewallfsi kernel: [104057.316629] [ 1939] 177 1939 6832 4716 17 3 0 0 dhcpd
Jan 26 08:59:46 firewallfsi kernel: [104057.317013] [ 1941] 38 1941 1536 1054 6 3 0 0 ntpd
Jan 26 08:59:46 firewallfsi kernel: [104057.317393] [ 1944] 48 1944 16216 4289 28 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.317783] [ 1947] 48 1947 16220 4571 28 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.318161] [ 1948] 48 1948 16220 4500 28 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.318539] [ 1960] 302 1960 1446 1049 7 3 0 0 imap
Jan 26 08:59:46 firewallfsi kernel: [104057.318913] [ 1962] 301 1962 1647 1133 8 3 0 0 imap
Jan 26 08:59:46 firewallfsi kernel: [104057.319282] [ 1965] 303 1965 1394 921 6 3 0 0 imap
Jan 26 08:59:46 firewallfsi kernel: [104057.319652] [ 2090] 0 2090 4181 1978 11 3 5 0 sshd
Jan 26 08:59:46 firewallfsi kernel: [104057.320019] [ 2092] 0 2092 1584 1140 6 3 0 0 systemd
Jan 26 08:59:46 firewallfsi kernel: [104057.320390] [ 2093] 0 2093 6809 333 11 3 62 0 (sd-pam)
Jan 26 08:59:46 firewallfsi kernel: [104057.320771] [ 2096] 0 2096 4214 1103 11 3 1 0 sshd
Jan 26 08:59:46 firewallfsi kernel: [104057.321149] [ 2097] 0 2097 1239 989 6 3 0 0 tcsh
Jan 26 08:59:46 firewallfsi kernel: [104057.321536] [ 2410] 0 2410 1142 800 5 3 0 0 config
Jan 26 08:59:46 firewallfsi kernel: [104057.321916] [ 2619] 304 2619 1399 906 6 3 0 0 imap
Jan 26 08:59:46 firewallfsi kernel: [104057.322295] [ 6076] 0 6076 1307 923 6 3 0 0 reboot-when-oom
Jan 26 08:59:46 firewallfsi kernel: [104057.322685] [10382] 48 10382 16216 4285 28 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.323076] [12318] 273 12318 2208 1470 8 3 0 0 imap-login
Jan 26 08:59:46 firewallfsi kernel: [104057.323475] [12319] 301 12319 1672 1079 7 3 0 0 imap
Jan 26 08:59:46 firewallfsi kernel: [104057.323863] [10772] 273 10772 2209 1439 8 3 0 0 imap-login
Jan 26 08:59:46 firewallfsi kernel: [104057.324243] [10776] 301 10776 1396 1008 6 3 0 0 imap
Jan 26 08:59:46 firewallfsi kernel: [104057.324630] [11099] 0 11099 11587 4409 26 3 0 0 smbd
Jan 26 08:59:46 firewallfsi kernel: [104057.325012] [12143] 0 12143 589 167 5 3 0 0 tail
Jan 26 08:59:46 firewallfsi kernel: [104057.325392] [18786] 276 18786 118337 104981 225 3 0 0 clamd
Jan 26 08:59:46 firewallfsi kernel: [104057.325779] [18801] 290 18801 13143 1160 15 3 0 0 clamav-milter
Jan 26 08:59:46 firewallfsi kernel: [104057.326164] [18822] 0 18822 3830 1760 11 3 0 0 sendmail
Jan 26 08:59:46 firewallfsi kernel: [104057.326550] [18838] 51 18838 3501 781 10 3 0 0 sendmail
Jan 26 08:59:46 firewallfsi kernel: [104057.326935] [18953] 23 18953 5440 723 13 3 0 0 squid
Jan 26 08:59:46 firewallfsi kernel: [104057.327324] [18955] 23 18955 9136 5994 20 3 0 0 squid
Jan 26 08:59:46 firewallfsi kernel: [104057.327720] [18956] 23 18956 1179 411 6 3 0 0 unlinkd
Jan 26 08:59:46 firewallfsi kernel: [104057.328116] [13386] 0 13386 1396 169 6 3 0 0 sleep
Jan 26 08:59:46 firewallfsi kernel: [104057.328520] [13604] 273 13604 2209 1435 8 3 0 0 imap-login
Jan 26 08:59:46 firewallfsi kernel: [104057.328918] [13608] 302 13608 1419 1042 6 3 0 0 imap
Jan 26 08:59:46 firewallfsi kernel: [104057.329316] [18231] 0 18231 4181 1967 11 3 0 0 sshd
Jan 26 08:59:46 firewallfsi kernel: [104057.329725] [18233] 299 18233 1603 1165 7 3 0 0 systemd
Jan 26 08:59:46 firewallfsi kernel: [104057.330148] [18234] 299 18234 6809 364 11 3 32 0 (sd-pam)
Jan 26 08:59:46 firewallfsi kernel: [104057.330564] [18237] 299 18237 4214 1127 11 3 0 0 sshd
Jan 26 08:59:46 firewallfsi kernel: [104057.330988] [18238] 299 18238 879 564 5 3 0 0 bash
Jan 26 08:59:46 firewallfsi kernel: [104057.331394] [18243] 299 18243 7292 6386 18 3 0 0 rdiff-backup
Jan 26 08:59:46 firewallfsi kernel: [104057.331810] [18244] 299 18244 619 220 4 3 0 0 pv
Jan 26 08:59:46 firewallfsi kernel: [104057.332238] [18905] 0 18905 1396 164 6 3 0 0 sleep
Jan 26 08:59:46 firewallfsi kernel: [104057.332661] Out of memory: Kill process 18786 (clamd) score 10 or sacrifice child
Jan 26 08:59:46 firewallfsi kernel: [104057.333107] Killed process 18786 (clamd) total-vm:473348kB, anon-rss:403644kB, file-rss:16280kB, shmem-rss:0kB
Jan 26 08:59:46 firewallfsi kernel: [104057.366106] audit: type=1131 audit(1485442786.940:3251): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=clamd@scan comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Jan 26 08:59:47 firewallfsi kernel: [104057.493007] audit: type=1130 audit(1485442787.067:3252): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=clamd@scan comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Jan 26 08:59:47 firewallfsi kernel: [104057.495502] audit: type=1131 audit(1485442787.070:3253): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=clamd@scan comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Jan 26 08:59:47 firewallfsi kernel: [104057.497128] audit: type=1130 audit(1485442787.071:3254): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=clamd@scan comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
next prev parent reply other threads:[~2017-01-26 23:19 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-01-11 10:32 Trevor Cordes
2017-01-11 12:11 ` Mel Gorman
2017-01-11 12:14 ` Mel Gorman
2017-01-11 22:52 ` Trevor Cordes
2017-01-12 9:36 ` Michal Hocko
2017-01-15 6:27 ` Trevor Cordes
2017-01-16 11:09 ` Mel Gorman
2017-01-17 13:52 ` Michal Hocko
2017-01-17 14:21 ` Mel Gorman
2017-01-17 14:54 ` Michal Hocko
2017-01-18 7:25 ` Trevor Cordes
2017-01-18 17:48 ` Mel Gorman
2017-01-18 18:07 ` Mel Gorman
2017-01-19 9:48 ` Trevor Cordes
2017-01-19 11:37 ` Michal Hocko
2017-01-20 6:35 ` Trevor Cordes
2017-01-20 11:02 ` Mel Gorman
2017-01-20 15:55 ` Mel Gorman
2017-01-23 0:45 ` Trevor Cordes
2017-01-23 10:48 ` Mel Gorman
2017-01-23 11:04 ` Mel Gorman
2017-01-25 9:46 ` Michal Hocko
2017-01-24 12:59 ` Michal Hocko
2017-01-25 10:02 ` Trevor Cordes
2017-01-25 12:04 ` Michal Hocko
2017-01-29 22:50 ` Trevor Cordes
2017-01-30 7:51 ` Michal Hocko
2017-02-01 9:29 ` Trevor Cordes
2017-02-01 10:14 ` Michal Hocko
2017-02-04 0:36 ` Trevor Cordes
2017-02-04 20:05 ` Rik van Riel
2017-02-05 10:03 ` Michal Hocko
2017-02-05 22:53 ` Trevor Cordes
2017-01-30 9:10 ` Mel Gorman
2017-01-24 12:54 ` Michal Hocko
2017-01-26 23:18 ` Trevor Cordes [this message]
2017-01-27 7:36 ` Michal Hocko
2017-01-24 12:51 ` Michal Hocko
2017-01-18 6:52 ` Trevor Cordes
2017-01-17 13:45 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170126171858.5e1b484a@pog.tecnopolis.ca \
--to=trevor@tecnopolis.ca \
--cc=iamjoonsoo.kim@lge.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@techsingularity.net \
--cc=mhocko@kernel.org \
--cc=minchan@kernel.org \
--cc=riel@surriel.com \
--cc=srikar@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Powered by JetHome