From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1751518AbdG0Ixm (ORCPT <rfc822;w@1wt.eu>);
        Thu, 27 Jul 2017 04:53:42 -0400
Received: from merlin.infradead.org ([205.233.59.134]:57044 "EHLO
        merlin.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1750878AbdG0Ixj (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 27 Jul 2017 04:53:39 -0400
Date: Thu, 27 Jul 2017 10:53:12 +0200
From: Peter Zijlstra <peterz@infradead.org>
To: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        Ingo Molnar <mingo@kernel.org>, Lai Jiangshan <jiangshanlai@gmail.com>,
        dipankar <dipankar@in.ibm.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Josh Triplett <josh@joshtriplett.org>,
        Thomas Gleixner <tglx@linutronix.de>, rostedt <rostedt@goodmis.org>,
        David Howells <dhowells@redhat.com>,
        Eric Dumazet <edumazet@google.com>, fweisbec <fweisbec@gmail.com>,
        Oleg Nesterov <oleg@redhat.com>, Will Deacon <will.deacon@arm.com>
Subject: Re: [PATCH tip/core/rcu 4/5] sys_membarrier: Add expedited option
Message-ID: <20170727085312.xweucokjghay3jtx@hirez.programming.kicks-ass.net>
References: <20170724215758.GA12075@linux.vnet.ibm.com>
 <20170725193612.GW3730@linux.vnet.ibm.com>
 <20170725202451.GC28975@worktop>
 <20170725211926.GA3730@linux.vnet.ibm.com>
 <20170725215510.GD28975@worktop>
 <1480480872.25829.1501023013862.JavaMail.zimbra@efficios.com>
 <20170726074656.obadfdu6hdlrmy7r@hirez.programming.kicks-ass.net>
 <20170726154229.GO3730@linux.vnet.ibm.com>
 <1415161894.26359.1501092075006.JavaMail.zimbra@efficios.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1415161894.26359.1501092075006.JavaMail.zimbra@efficios.com>
User-Agent: NeoMutt/20170609 (1.8.3)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Jul 26, 2017 at 06:01:15PM +0000, Mathieu Desnoyers wrote:

> Another alternative for a MEMBARRIER_CMD_SHARED_EXPEDITED would be rate-limiting
> per thread. For instance, we could add a new "ulimit" that would bound the
> number of expedited membarrier per thread that can be done per millisecond,
> and switch to synchronize_sched() whenever a thread goes beyond that limit
> for the rest of the time-slot.

You forgot to ask yourself how you could abuse this.. just spawn more
threads.

Per-thread limits are nearly useless, because spawning new threads is
cheap.

> A RT system that really cares about not having userspace sending IPIs
> to all cpus could set the ulimit value to 0, which would always use
> synchronize_sched().
> 
> Thoughts ?

So I really don't like SHARED_EXPEDITED, and your use-cases (from later
emails) makes me think sys_membarrier() should have a pointer argument
to identify the shared mapping.

But even then, iterating the rmap for something that has 1000+ maps
isn't going to be nice or fast, even in kernel space.

Another crazy idea is using madvise() for this. The new MADV_MEMBAR
could revoke PROT_WRITE and PROT_READ for all extant PTEs. Then the
tasks attempting access will fault and the fault handler can figure out
if it still needs to issue a MB or not before reinstating the PTE.

That is fully contained to the tasks actually having that map, and
doesn't perturb anybody else.