From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751441AbdJEQkV (ORCPT ); Thu, 5 Oct 2017 12:40:21 -0400 Received: from mx0a-00010702.pphosted.com ([148.163.156.75]:53934 "EHLO mx0b-00010702.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751144AbdJEQkS (ORCPT ); Thu, 5 Oct 2017 12:40:18 -0400 Date: Thu, 5 Oct 2017 11:39:33 -0500 From: Julia Cartwright To: Thomas Gleixner CC: Steven Rostedt , Arnaldo Carvalho de Melo , , , , Arnaldo Carvalho de Melo , Clark Williams , Dean Luick , Dennis Dalessandro , Doug Ledford , Kaike Wan , Leon Romanovsky , , Peter Zijlstra , Sebastian Andrzej Siewior , Sebastian Sanchez Subject: Re: [PATCH 1/2] IB/hfi1: Use preempt_{dis,en}able_nort() Message-ID: <20171005163933.GJ647@jcartwri.amer.corp.natinst.com> References: <20171003154920.31566-1-acme@kernel.org> <20171003154920.31566-2-acme@kernel.org> <20171005141744.GC21185@jcartwri.amer.corp.natinst.com> <20171005153759.GG647@jcartwri.amer.corp.natinst.com> <20171005115539.7daafb66@gandalf.local.home> <20171005160535.GI647@jcartwri.amer.corp.natinst.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.0 (2017-09-02) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-10-05_08:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=30 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=30 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1710050229 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Oct 05, 2017 at 06:16:23PM +0200, Thomas Gleixner wrote: > On Thu, 5 Oct 2017, Julia Cartwright wrote: > > > On Thu, Oct 05, 2017 at 11:55:39AM -0400, Steven Rostedt wrote: > > > On Thu, 5 Oct 2017 10:37:59 -0500 > > > Julia Cartwright wrote: > > > > > > > On Thu, Oct 05, 2017 at 05:27:30PM +0200, Thomas Gleixner wrote: > > > > > On Thu, 5 Oct 2017, Julia Cartwright wrote: > > > > > > On Tue, Oct 03, 2017 at 12:49:19PM -0300, Arnaldo Carvalho de Melo wrote: > > > > > > > - preempt_disable(); > > > > > > > + preempt_disable_nort(); > > > > > > > this_cpu_inc(*sc->buffers_allocated); > > > > > > > > > > > > Have you tried this on RT w/ CONFIG_DEBUG_PREEMPT? I believe that the > > > > > > this_cpu_* operations perform a preemption check, which we'd trip. > > > > > > > > > > Good point. Changing this to migrate_disable() would do the trick. > > > > > > > > Wouldn't we still trip the preempt check even with migration disabled? > > > > In another thread I asked the same question: should the preemption > > > > checks here be converted to migration-checks in RT? > > > > > > Is it a "preemption check"? > > > > Sorry if I was unclear, more precisely: the this_cpu_* family of > > accessors, w/ CONFIG_DEBUG_PREEMPT currently spits out a warning when > > the caller is invoked in a context where preemption is enabled. > > > > The check is shared w/ the smp_processor_id() check, as implemented in > > lib/smp_processor_id.c. It effectively boils down to a check of > > preempt_count() and irqs_disabled(). > > Except that on RT that check cares about the migrate disable state. You can > invoke this_cpu_* and smp_processor_id() in preemptible/interruptible > context because of: > > if (cpumask_equal(current->cpus_ptr, cpumask_of(this_cpu))) > goto out; > > That's true even on mainline. > > But you are right that this check needs some update because > migrate_disable() does not immediately update the allowed cpumask IIRC. Actually, I think it does: migrate_disable() -> p = current; ... migrate_disable_update_cpus_allowed(p) -> p->cpus_ptr = cpumask_of(smp_processor_id()) ... Perhaps it's worth a simple comment update, below. Julia -- 8< -- Subject: [PATCH] kernel: sched: document smp_processor_id/this_cpu safety in migration-disabled context On RT, users of smp_processor_id() and this_cpu_*() per-cpu accessors are considered safe if the caller executes with migration disabled. On !RT, preempt_disable() is sufficient to make this guarantee, however on RT, the lesser migrate_disable() is sufficient. It is not entirely obvious which check in check_preemption_disabled() makes this work, so document it. Signed-off-by: Julia Cartwright --- lib/smp_processor_id.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lib/smp_processor_id.c b/lib/smp_processor_id.c index de3b2d925473..c8091d9eb1b4 100644 --- a/lib/smp_processor_id.c +++ b/lib/smp_processor_id.c @@ -20,7 +20,11 @@ notrace static unsigned int check_preemption_disabled(const char *what1, /* * Kernel threads bound to a single CPU can safely use - * smp_processor_id(): + * smp_processor_id(). + * + * In addition, threads which are currently executing within + * a migration disabled region can safely use smp_processor_id() and + * this_cpu accessors. */ if (cpumask_equal(current->cpus_ptr, cpumask_of(this_cpu))) goto out; -- 2.14.1