From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751643AbdARG5u (ORCPT ); Wed, 18 Jan 2017 01:57:50 -0500 Received: from smtp-out-no.shaw.ca ([64.59.134.9]:38968 "EHLO smtp-out-no.shaw.ca" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751610AbdARG5r (ORCPT ); Wed, 18 Jan 2017 01:57:47 -0500 X-Authority-Analysis: v=2.2 cv=BNTDlBYG c=1 sm=1 tr=0 a=DXb8iGicsy0C2lNtaEhhGg==:117 a=DXb8iGicsy0C2lNtaEhhGg==:17 a=kj9zAlcOel0A:10 a=IgFoBzBjUZAA:10 a=aLeJFe0jOSZBGkJ2lb0A:9 a=CjuIK1q_8ugA:10 Date: Wed, 18 Jan 2017 00:52:42 -0600 From: Trevor Cordes To: Mel Gorman Cc: Michal Hocko , linux-kernel@vger.kernel.org, Joonsoo Kim , Minchan Kim , Rik van Riel , Srikar Dronamraju Subject: Re: mm, vmscan: commit makes PAE kernel crash nightly (bisected) Message-ID: <20170118005242.23379a74@pog.tecnopolis.ca> In-Reply-To: <20170116110934.7zopy3ecg2lfadkd@techsingularity.net> References: <20170111103243.GA27795@pog.tecnopolis.ca> <20170111121146.i53jyerxont7b74i@techsingularity.net> <20170111121428.4sswbkzyqz7vf6so@techsingularity.net> <20170111165232.0b7abe18@pog.tecnopolis.ca> <20170112093613.GF2264@dhcp22.suse.cz> <20170115002752.4559f668@pog.tecnopolis.ca> <20170116110934.7zopy3ecg2lfadkd@techsingularity.net> X-Mailer: Claws Mail 3.14.1 (GTK+ 2.24.31; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-CMAE-Envelope: MS4wfJr3VShhU09o2O8KU+r7/bpFWuOED/+z+1VCgUpFiJmuSywHViOs+N9pxqnUnU2fozZFzc8k8pBXxmEajcehFMKAlnmd0uGbX/+1aKzqoBt7c1E2xtT9 Q0wt1wflbv7iYCoKfNOpQ6lgvBJY7tTGay32Sw2fLLfUJghkOYoOZOkLc1uFWQeRRBm2DtY1Fh35SM4lWL/YJn4rIbeuXC8Rouq1nFuQ3l2SM8RVCarMYqV9 2+AYaYSqvb3mLDuJaoNH1SupYk0UYey5C8CJRrJUYWrEP5lO94HmWl8QAtamPedpcV/u0CKN9ZbHVpn5DFZ1Kw0KpW4nf80ORt/4ad4Fm2ZzWGEmgiNgSuWx 1ELB3P+hj5GK1xLcyXsxadLAIhVHcg== Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2017-01-16 Mel Gorman wrote: > > > You can easily check whether this is memcg related by trying to > > > run the same workload with cgroup_disable=memory kernel command > > > line parameter. This will put all the memcg specifics out of the > > > way. > > > > I will try booting now into cgroup_disable=memory to see if that > > helps at all. I'll reply back in 48 hours, or when it oom's, > > whichever comes first. > > > > Thanks. It has successfully survived 70 hours and 2 3am cycles (when it normally oom's) with your first patch *and* cgroup_disable=memory grafted on Fedora's 4.8.13. Since it has never survived 2 3am cycles, I strongly suspect the cgroup_disable=memory mitigates my bug. > > Also, should I bother trying the latest git HEAD to see if that > > solves anything? Thanks! > > That's worth trying. If that also fails then could you try the > following hack to encourage direct reclaim to reclaim slab when > buffers are over the limit please? > > diff --git a/mm/vmscan.c b/mm/vmscan.c > index 532a2a750952..46aac487b89a 100644 > --- a/mm/vmscan.c > +++ b/mm/vmscan.c > @@ -2684,6 +2684,7 @@ static void shrink_zones(struct zonelist > *zonelist, struct scan_control *sc) continue; > > if (sc->priority != DEF_PRIORITY && > + !buffer_heads_over_limit && > !pgdat_reclaimable(zone->zone_pgdat)) > continue; /* Let kswapd poll > it */ What's the next best step? HEAD? HEAD + the above patch? A new patch? I'll start a HEAD compile until I hear more. I assume I should test without cgroup_disable=memory as that's just a kludge/workaround, right? Also, is there a way to spot the slab pressure you are talking about before oom's occur? slabinfo? I suppose I'd be able to see some counter slowly getting too high or low? Thanks!