Linux scheduler capabilities for batch jobs.

[J Louis <handstogether8@gmail.com>]

Hello All,

I have been writing user land server code and increasingly I find
myself writing resource management code which I think would be better
handled by the scheduler.  The problem can be described as "don't run
this process if the machine is swapping."  I would think that this
would be a common enough need that it was already in the kernel.  I'm
hoping it is and I have simply overlooked it (part of the reason for
this post,) but I've looked around a good bit and most of the
scheduler enhancements have to do with real time and latency, not
batch jobs.

My problem is analogous to a parallel make.  Say I have an 8 CPU
machine, and I run "make -j8".  If the total memory of the 8 jobs
throws the machine into swap, it begins to thrash and runtime is
awful.  I believe this is aggravated by the scheduler trying to be
fair, and keeping all 8 processes running.  If it was possible to tell
the scheduler that it was OK not to be fair when scheduling these
processes, I think the total runtime could be reduced if it put some
of the processes to sleep while others completed.  Is there a way to
tell the scheduler it is allowed to do this?  Should there be?

Thank You,

Hands