Re: oom-killer is crazy? (Was: [PATCH 0/3] uprobes fixes for 3.5)

[Dave Jones <davej@redhat.com>]

On Fri, Jun 08, 2012 at 04:03:28PM +0200, Oleg Nesterov wrote:
 > On 06/07, Oleg Nesterov wrote:
 > >
 > > This doesn't depend on other uprobes patches I sent, and I think
 > > this is 3.5 material.
 > 
 > And during the testing I found another thing which should be fixed
 > in 3.5 imho. I noticed that oom-killer goes crazy. In the simplest
 > case, when there is the single and "obvious" memory hog it kills
 > sshd daemon.
 > 
 > Hmm. oom_badness() does
 > 
 > 	if (has_capability_noaudit(p, CAP_SYS_ADMIN))
 > 		points -= 30 * totalpages / 1000;
 > 
 > very nice, but what if this underflows? points is unsigned long.
 > points += p->signal->oom_score_adj... looks suspicious too.
 > 
 > Looks like we should remove "unsigned" from oom_badness() and
 > its callers? Probably not, it does "return points ? points : 1".

I've been running this from David for a week, but it still isn't right..

	Dave

diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index ed0e196..416637f 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -183,7 +183,7 @@ static bool oom_unkillable_task(struct task_struct *p,
 unsigned long oom_badness(struct task_struct *p, struct mem_cgroup *memcg,
                          const nodemask_t *nodemask, unsigned long totalpages)
 {
-       unsigned long points;
+       long points;
 
        if (oom_unkillable_task(p, memcg, nodemask))
                return 0;
@@ -223,7 +223,7 @@ unsigned long oom_badness(struct task_struct *p, struct mem_cgroup *memcg,
         * Never return 0 for an eligible task regardless of the root bonus and
         * oom_score_adj (oom_score_adj can't be OOM_SCORE_ADJ_MIN here).
         */
-       return points ? points : 1;
+       return points > 0 ? points : 1;
 }
 
 /*