From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: linux-kernel@vger.kernel.org
Cc: mingo@kernel.org, laijs@cn.fujitsu.com, dipankar@in.ibm.com,
akpm@linux-foundation.org, mathieu.desnoyers@efficios.com,
josh@joshtriplett.org, niv@us.ibm.com, tglx@linutronix.de,
peterz@infradead.org, rostedt@goodmis.org, dhowells@redhat.com,
edumazet@google.com, darren@dvhart.com, fweisbec@gmail.com,
oleg@redhat.com, sbw@mit.edu,
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Subject: [PATCH tip/core/rcu 12/55] rcutorture: Add comments, especially on bin packing.
Date: Mon, 17 Feb 2014 14:12:16 -0800 [thread overview]
Message-ID: <1392675179-11560-12-git-send-email-paulmck@linux.vnet.ibm.com> (raw)
In-Reply-To: <1392675179-11560-1-git-send-email-paulmck@linux.vnet.ibm.com>
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
tools/testing/selftests/rcutorture/bin/kvm.sh | 33 +++++++++++++++++++++++++--
1 file changed, 31 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index 18649b87ea6c..7ef3b245c778 100644
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -188,6 +188,7 @@ fi
mkdir $resdir/$ds
if test "$dryrun" = ""
then
+ # Be noisy only if running the script.
echo Results directory: $resdir/$ds
echo $scriptname $args
fi
@@ -201,6 +202,7 @@ then
git rev-parse HEAD >> $resdir/$ds/testid.txt
fi
+# Create a file of test-name/#cpus pairs, sorted by decreasing #cpus.
touch $T/cfgcpu
for CF in $configs
do
@@ -214,12 +216,14 @@ do
done
sort -k2nr $T/cfgcpu > $T/cfgcpu.sort
+# Use a greedy bin-packing algorithm, sorting the list accordingly.
awk < $T/cfgcpu.sort > $T/cfgcpu.pack -v ncpus=$cpus '
BEGIN {
njobs = 0;
}
{
+ # Read file of tests and corresponding required numbers of CPUs.
cf[njobs] = $1;
cpus[njobs] = $2;
njobs++;
@@ -229,26 +233,40 @@ END {
alldone = 0;
batch = 0;
nc = -1;
+
+ # Each pass through the following loop creates on test batch
+ # that can be executed concurrently given ncpus. Note that a
+ # given test that requires more than the available CPUs will run in
+ # their own batch. Such tests just have to make do with what
+ # is available.
while (nc != ncpus) {
batch++;
nc = ncpus;
+
+ # Each pass through the following loop considers one
+ # test for inclusion in the current batch.
for (i = 0; i < njobs; i++) {
if (done[i])
- continue;
+ continue; # Already part of a batch.
if (nc >= cpus[i] || nc == ncpus) {
+
+ # This test fits into the current batch.
done[i] = batch;
nc -= cpus[i];
if (nc <= 0)
- break;
+ break; # Too-big test in its own batch.
}
}
}
+
+ # Dump out the tests in batch order.
for (b = 1; b <= batch; b++)
for (i = 0; i < njobs; i++)
if (done[i] == b)
print cf[i], cpus[i];
}'
+# Generate a script to execute the tests in appropriate batches.
awk < $T/cfgcpu.pack \
-v CONFIGDIR="$CONFIGFRAG/$kversion/" \
-v KVM="$KVM" \
@@ -267,6 +285,7 @@ awk < $T/cfgcpu.pack \
i++;
}
+# Dump out the scripting required to run one test batch.
function dump(first, pastlast)
{
print "echo ----start batch----"
@@ -313,23 +332,31 @@ END {
njobs = i;
nc = ncpus;
first = 0;
+
+ # Each pass through the following loop considers one test.
for (i = 0; i < njobs; i++) {
if (ncpus == 0) {
+ # Sequential test specified, each test its own batch.
dump(i, i + 1);
first = i;
} else if (nc < cpus[i] && i != 0) {
+ # Out of CPUs, dump out a batch.
dump(first, i);
first = i;
nc = ncpus;
}
+ # Account for the CPUs needed by the current test.
nc -= cpus[i];
}
+ # Dump the last batch.
if (ncpus != 0)
dump(first, i);
}' > $T/script
if test "$dryrun" = script
then
+ # Dump out the script, but define the environment variables that
+ # it needs to run standalone.
echo CONFIGFRAG="$CONFIGFRAG; export CONFIGFRAG"
echo KVM="$KVM; export KVM"
echo KVPATH="$KVPATH; export KVPATH"
@@ -344,10 +371,12 @@ then
exit 0
elif test "$dryrun" = sched
then
+ # Extract the test run schedule from the script.
egrep 'start batch|Starting build\.' $T/script |
sed -e 's/:.*$//' -e 's/^echo //'
exit 0
else
+ # Not a dryru, so run the script.
sh $T/script
fi
--
1.8.1.5
next prev parent reply other threads:[~2014-02-17 22:23 UTC|newest]
Thread overview: 81+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-02-17 22:12 [PATCH tip/core/rcu 0/55] Torture-test changes for 3.15 Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 01/55] rcutorture: Print grace-period performance statistics Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 02/55] rcutorture: Print script and arguments to standard output Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 03/55] rcutorture: Move common boot flags to kvm-test-1-rcu.sh Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 04/55] rcutorture: Allow kvm-test-1-rcu.sh to pause after build Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 05/55] rcutorture: Move common qemu flags to kvm-test-1.sh Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 06/55] rcutorture: Reduce SRCU-N number of CPUs Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 07/55] rcutorture: Enable concurrent rcutorture runs Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 08/55] rcutorture: Fix results-directory error message Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 09/55] rcutorture: Add dryrun capability Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 10/55] rcutorture: Handle multiple runs of the same test Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 11/55] rcutorture: Do better bin packing Paul E. McKenney
2014-02-17 22:12 ` Paul E. McKenney [this message]
2014-02-17 22:12 ` [PATCH tip/core/rcu 13/55] rcutorture: Flag tests requiring more CPUs than are available Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 14/55] rcutorture: Print results directory when dumping results Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 15/55] rcutorture: Abstract rcu_torture_random() Paul E. McKenney
2014-02-18 0:11 ` Josh Triplett
2014-02-18 0:22 ` Paul E. McKenney
2014-02-18 0:27 ` Josh Triplett
2014-02-18 21:33 ` Paul E. McKenney
2014-02-19 2:52 ` George Spelvin
2014-02-17 22:12 ` [PATCH tip/core/rcu 16/55] rcutorture: Don't create results directory for dryruns Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 17/55] rcutorture: Abstract torture_param() Paul E. McKenney
2014-02-18 0:23 ` Josh Triplett
2014-02-18 21:31 ` Paul E. McKenney
2014-02-18 23:53 ` josh
2014-02-23 16:48 ` Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 18/55] rcutorture: Rename PRINTK to TOROUT Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 19/55] rcutorture: Abstract TOROUT_STRING() and friends Paul E. McKenney
2014-02-18 0:26 ` Josh Triplett
2014-02-18 0:58 ` Paul E. McKenney
2014-02-18 1:23 ` Josh Triplett
2014-02-18 1:43 ` Paul E. McKenney
2014-02-18 3:36 ` Josh Triplett
2014-02-17 22:12 ` [PATCH tip/core/rcu 20/55] rcutorture: Abstract torture_shutdown_absorb() Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 21/55] rcutorture: Export torture_type and verbose module parameters Paul E. McKenney
2014-02-18 0:29 ` Josh Triplett
2014-02-18 1:01 ` Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 22/55] rcutorture: Abstract torture_shuffle() Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 23/55] rcutorture: Abstract torture_onoff() Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 24/55] rcutorture: Abstract torture-test initialization Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 25/55] rcutorture: Abstract torture-test cleanup Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 26/55] rcutorture: Print dates when doing parallel rcutorture runs Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 27/55] rcutorture: Abstract torture_shutdown_notify() Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 28/55] rcutorture: Privatize fullstop Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 29/55] rcutorture: Add diagnostic for unscheduled system shutdown Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 30/55] rcutorture: Abstract stutter_wait() Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 31/55] rcutorture: Apply ACCESS_ONCE() to racy fullstop accesses Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 32/55] rcutorture: Abstract torture_shutdown() Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 33/55] rcutorture: Clean up rcu_torture_init() error checking Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 34/55] rcutorture: Announce task creation Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 35/55] rcutorture: Fix rcutorture shutdown races Paul E. McKenney
2014-02-18 0:32 ` Josh Triplett
2014-02-18 21:35 ` Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 36/55] rcutorture: Fix missing-return bug in rcu_torture_barrier_init() Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 37/55] rcutorture: Abstract torture_create_kthread() Paul E. McKenney
2014-02-18 0:34 ` Josh Triplett
2014-02-18 21:36 ` Paul E. McKenney
2014-02-19 0:02 ` josh
2014-02-17 22:12 ` [PATCH tip/core/rcu 38/55] rcutorture: Abstract torture_stop_kthread() Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 39/55] rcutorture: Stop generic kthreads in torture_cleanup() Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 40/55] rcutorture: Remove redundant module-alias directives Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 41/55] rcutorture: Abstract kvm-recheck.sh Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 42/55] locktorture: Add a lock-torture kernel module Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 43/55] rcutorture: Add an rcu_busted to test the test Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 44/55] rcutorture: Create config files for scripted test-the-test testing Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 45/55] rcutorture: Create CFcommon file for common Kconfig parameters Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 46/55] rcutorture: Remove RCU dependencies from ver_functions.sh API Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 47/55] rcutorture: Rename kvm-test-1-rcu.sh Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 48/55] rcutorture: Introduce "rcu" directory level underneath configs Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 49/55] locktorture: Add vestigial locktorture configuration Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 50/55] rcutorture: Gracefully handle NULL cleanup hooks Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 51/55] locktorture: Add kvm-recheck.sh plug-in for locktorture Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 52/55] rcutorture: Rename TREE_RCU-Kconfig.txt Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 53/55] rcutorture: Place kvm-test-1-run.sh output into res directory Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 54/55] rcutorture: Add a lock_busted to test the test Paul E. McKenney
2014-02-17 22:12 ` [PATCH tip/core/rcu 55/55] rcutorture: Save kvm.sh output to log Paul E. McKenney
2014-02-18 0:41 ` [PATCH tip/core/rcu 0/55] Torture-test changes for 3.15 Josh Triplett
2014-02-18 0:48 ` Josh Triplett
2014-02-18 1:18 ` Paul E. McKenney
2014-02-18 1:18 ` Paul E. McKenney
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1392675179-11560-12-git-send-email-paulmck@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=darren@dvhart.com \
--cc=dhowells@redhat.com \
--cc=dipankar@in.ibm.com \
--cc=edumazet@google.com \
--cc=fweisbec@gmail.com \
--cc=josh@joshtriplett.org \
--cc=laijs@cn.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mingo@kernel.org \
--cc=niv@us.ibm.com \
--cc=oleg@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=sbw@mit.edu \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®