Re: [RESEND] shm: shm exit scalability fixes

[Jack Miller <millerjo@us.ibm.com>]

On Tue, Jun 17, 2014 at 10:48:32AM -0700, Davidlohr Bueso wrote:
> On Tue, 2014-06-17 at 12:27 -0500, Jack Miller wrote:
> > [ RESEND note: Adding relevant CCs, fixed a couple of typos in commit message,
> > patches unchanged. Original intro follows. ]
> > 
> > All -
> > 
> > This is small set of patches our team has had kicking around for a few versions
> > internally that fixes tasks getting hung on shm_exit when there are many
> > threads hammering it at once.
> > 
> > Anton wrote a simple test to cause the issue:
> > 
> > http://ozlabs.org/~anton/junkcode/bust_shm_exit.c
> 
> I'm actually in the process of adding shm microbenchmarks to perf-bench
> so I might steal this :-)
>

Cool!

> > 
> > Before applying this patchset, this test code will cause either hanging
> > tracebacks or pthread out of memory errors.
> 
> Are you seeing this issue in any real world setups? While the program
> does stress the path you mention quite well, I fear it is very
> unrealistic... how many shared mem segments do real applications
> actually use/create for scaling issues to appear?

We've seen this while running multiple workloads on the same machine. One
workload that used shared memory extensively, and one that created many
shortlived threads. The testcase is just simulating these two workloads
running simultaneously, so I don't think it's too unreasonable to expect it
could happen in the wild.

Even if this is synthetic, the testcase could also be seen as proof of an
unprivileged denial of service as an arbitrary user could run bust_shm_exit
and subsequently start overloading the system.

> 
> I normally wouldn't mind optimizing synthetic cases like this, but a
> quick look at patch 1/3 shows that we're adding an extra overhead (16
> bytes) in the task_struct.

Yeah, that's definitely not to be done lightly, but I think it's worth it to
make the work on exit proportional to the actual task usage instead of the
number of segments in the namespace.

> 
> In any case, I will take a closer look at the set.

Thanks! I'd appreciate any feedback.

- Jack