mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: "Serge E. Hallyn" <serge@hallyn.com>
To: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Cc: Christoph Lameter <clameter@sgi.com>,
	"Serge E. Hallyn" <serge@hallyn.com>,
	Dhaval Giani <dhaval@linux.vnet.ibm.com>,
	bob.picco@hp.com, nacc@us.ibm.com,
	kamezawa.hiroyu@jp.fujitsu.com, mel@skynet.ie,
	akpm@linux-foundation.org, Balbir Singh <balbir@in.ibm.com>,
	Srivatsa Vaddagiri <vatsa@linux.vnet.ibm.com>,
	lkml <linux-kernel@vger.kernel.org>,
	ckrm-tech <ckrm-tech@lists.sourceforge.net>
Subject: Re: Regression in 2.6.23-rc2-mm2, mounting cpusets causes a hang
Date: Wed, 15 Aug 2007 09:31:25 -0500	[thread overview]
Message-ID: <20070815143125.GA11582@vino.hallyn.com> (raw)
In-Reply-To: <1187185392.5422.13.camel@localhost>

Quoting Lee Schermerhorn (Lee.Schermerhorn@hp.com):
> On Tue, 2007-08-14 at 14:56 -0700, Christoph Lameter wrote:
> > On Tue, 14 Aug 2007, Lee Schermerhorn wrote:
> > 
> > > > Ok then you did not have a NUMA system configured. So its okay for the 
> > > > dummies to ignore the stuff. CONFIG_NODES_SHIFT is a constant and does not 
> > > > change. The first bit is always set.
> > > 
> > > The first bit [node 0] is only set for the N_ONLINE [and N_POSSIBLE]
> > > mask.  We could add the static init for the other masks, but since
> > > non-numa platforms are going through the __build_all_zonelists, they
> > > might as well set the MEMORY bits explicitly.  Or, maybe you'll
> > > disagree ;-).
> > 
> > The bitmaps can be completely ignored if !NUMA.
> > 
> > In the non NUMA case we define
> > 
> > static inline int node_state(int node, enum node_states state)
> > {
> >         return node == 0;
> > }
> > 
> > So its always true for node 0. The "bit" is set.
> 
> The issue is with the N_*_MEMORY masks.  They don't get initialized
> properly because node_set_state() is a no-op if !NUMA.  So, where we
> look for intersections with or where we AND with the N_*_MEMORY masks we
> get the empty set.
> 
> > 
> > We are trying to get cpusets to work with !NUMA?
> 
> 
> Well, yes.  In Serge's case, he's trying to use cpusets with !NUMA.
> He'll have to comment on the reasons for that.  Looking at all of the

So I can lock a container to a cpu on a non-numa machine.

> #ifdefs and init/Kconfig, CPUSET does not depend on NUMA--only SMP and
> CONTAINERS [altho' methinks CPUSET should select CONTAINERS rather than
> depend on it...].  So, you can use cpusets to partition of cpus in
> non-NUMA configs.
> 
> In the more general case, tho', I'm looking at all uses of the
> node_online_map and for_each_online_node, for instances where they
> should be replaced with one of the *_MEMORY masks.  IMO, generic code
> that is compiled independent of any CONFIG option, like NUMA,  should
> just work, independent of the config.  Currently, as Serge has shown,
> this is not the case.  So, I think we should fix the *_MEMORY maps to be
> correctly populated in both the NUMA and !NUMA cases.  A couple of
> options:
> 
> 1) just use node_set() when populating the masks,
> 
> 2) initialize all masks to include at least cpu/node 0 in the !NUMA
> case.
> 
> Serge chose #1 to fix his problem.  I followed his lead to fix the other
> 2 places where node_set_state() was being used to initialize the NORMAL
> memory node mask and the CPU node mask.  This will add a few unnecessary
> instructions to !NUMA configs, so we could change to #2.
> 
> Thoughts?

Paul, is the mems stuff in cpusets only really useful for NUMA cases?
(I think it is... but am not sure)  If so I suppose one alternative
could be to just disable that when !NUMA.  But disabling cpusets when
!NUMA is completely wrong.

I personally would think that 1) is still the best option.  Otherwise
the action

	echo $SOME_CPU > /cpusets/set1/cpu
	echo $SOME_CPU > /cpusets/set1/mems

works on a numa machine, and is wrong on a non-numa machine.  With
option 1, the second part doesn't actually restrict the memory, but
at least /cpusets/set1/mems exists and $SOME_CPU doesn't have to be 0 to
be valid.

-serge

  reply	other threads:[~2007-08-15 14:31 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-08-12 15:21 Dhaval Giani
2007-08-13 20:12 ` Serge E. Hallyn
2007-08-14 15:03   ` Lee Schermerhorn
2007-08-14 18:03     ` Serge E. Hallyn
2007-08-14 18:13       ` Lee Schermerhorn
2007-08-14 19:23         ` Serge E. Hallyn
2007-08-14 20:49           ` Serge E. Hallyn
2007-08-14 21:07             ` Lee Schermerhorn
2007-08-14 21:28             ` Christoph Lameter
2007-08-14 21:41               ` Lee Schermerhorn
2007-08-14 21:56                 ` Christoph Lameter
2007-08-15 13:43                   ` Lee Schermerhorn
2007-08-15 14:31                     ` Serge E. Hallyn [this message]
2007-08-15 16:23                       ` Lee Schermerhorn
2007-08-15 16:31                         ` Serge E. Hallyn
2007-08-15 16:52                           ` Dhaval Giani
2007-08-15 17:08                             ` Serge E. Hallyn
2007-08-15 18:07                             ` Lee Schermerhorn
2007-08-15 20:39                               ` Christoph Lameter
2007-08-16 13:26                             ` Lee Schermerhorn
2007-08-16 19:14                               ` Christoph Lameter
2007-08-15 20:38                           ` Christoph Lameter
2007-08-15 16:31                       ` Paul Jackson
2007-08-15 16:29                     ` Paul Jackson
2007-08-15 17:12                       ` Serge E. Hallyn
2007-08-15 18:00                         ` Lee Schermerhorn
2007-08-15 18:33                           ` Andrew Morton
2007-08-31 16:54                             ` [ckrm-tech] " Paul Menage
2007-08-16  4:46                       ` Paul Menage
2007-08-15 20:36                     ` Christoph Lameter
2007-08-15 20:48                       ` Lee Schermerhorn
2007-08-16  2:36                         ` [ckrm-tech] " Paul Jackson
2007-08-14 21:37             ` Lee Schermerhorn
2007-08-14 21:39               ` Christoph Lameter
2007-08-14 21:01           ` Lee Schermerhorn

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20070815143125.GA11582@vino.hallyn.com \
    --to=serge@hallyn.com \
    --cc=Lee.Schermerhorn@hp.com \
    --cc=akpm@linux-foundation.org \
    --cc=balbir@in.ibm.com \
    --cc=bob.picco@hp.com \
    --cc=ckrm-tech@lists.sourceforge.net \
    --cc=clameter@sgi.com \
    --cc=dhaval@linux.vnet.ibm.com \
    --cc=kamezawa.hiroyu@jp.fujitsu.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mel@skynet.ie \
    --cc=nacc@us.ibm.com \
    --cc=vatsa@linux.vnet.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®