Re: [RFC -mm 0/6] sysv ipc: scale msgmnb with the number of cpus

[Nick Piggin <nickpiggin@yahoo.com.au>]

On Friday 06 June 2008 16:09, Solofo.Ramangalahy@bull.net wrote:
> The size in bytes of a SysV IPC message queue, msgmnb, is too small
> for large machines, but we don't want to bloat small machines

What's your evidence for this? Can you provide before / after
performance numbers?

Also, when scaling things like this, it is probably more usual
to use a log scale rather than linear, so that's a thought.

>
> Several methods are used already to modify (mainly increase) msgmnb:
> . distribution specific patch
> . system wide sysctl.conf
> . application specific tuning via /proc/sys/kernel/msgmnb
>
> Integrating this series would:
> . reflect hardware and software evolutions and diversity,
> . reduce configuration/tuning for the applications.
>
> Here is the timeline of the evolution of MSG* #defines:
> Year             1994         1999         1999          2008
> Version           1.0       2.3.27       2.3.30        2.6.24
> #define MSGMNI    128          128           16            16
> #define MSGMAX   4056         8192         8192          8192
> #define MSGMNB  16384        16384        16384         16384
>
> This patch series scales msgmnb, with respect to the number of
> cpus/cores for larger machines. For uniprocessor machines the value
> does not increase.
>
> This series is similar to (and depends on) the series which scales
> msgmni, the number of IPC message queue identifiers, to the amount of
> low memory.
> While Nadia's previous series scaled msgmni along the memory axis,
> hence the message pool (msgmni x msgmnb), this series uses a second
> axis: the number of online CPUs.
> As well as covering the (cpu,memory) space of machines size, this
> reflects the parallelism allowed by lockless send/receive for
> in-flight messages in queues (msgmnb / msgmax messages).
>
> The initial scaling is done at initialization of the ipc namespace.
> Furthermore, the value becomes dynamic with respect to cpu hotplug.
>
> The msgmni and msgmnb values become dependent, as the value of msgmni
> is computed with respect to the value of msgmnb.
>
> The series is as follows:
> . patch 1 introduces the scaling function
> . patch 2 deals with cpu hotplug
> . patch 3 allows user space to disable the scaling mechanism
> . patch 4 allows user space to reenable the scaling mechanism
> . patch 5 finer grain disabling/reenabling scaling mechanism
>           (disconnect msgmnb and msgmni)
> . patch 6 adds documentation
>
> ---
>
> The series applies to 2.6.26-rc2-mm1 + patch suppressing KERN_INFO
> messages as discussed at:
> http://article.gmane.org/gmane.linux.kernel/686229
> "[PATCH 1/1] Only output msgmni value at boot time"
> (in mmotm: ipc-only-output-msgmni-value-at-boot-time.patch)
>
> The plan would be to have this ready for the 2.6.27 merge window if
> there are no objections.
>
>  Documentation/sysctl/kernel.txt |   27 ++++++++++++++++++++++
>  include/linux/ipc_namespace.h   |    4 ++-
>  include/linux/msg.h             |    5 ++++
>  ipc/ipc_sysctl.c                |   48
> ++++++++++++++++++++++++++++++---------- ipc/ipcns_notifier.c            | 
>  23 +++++++------------
>  ipc/msg.c                       |   25 +++++++++++++++++---
>  ipc/util.c                      |   28 +++++++++++++++++++++++
>  ipc/util.h                      |    1
>  8 files changed, 131 insertions(+), 30 deletions(-)