From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S964792AbbCFKvb (ORCPT ); Fri, 6 Mar 2015 05:51:31 -0500 Received: from cantor2.suse.de ([195.135.220.15]:54315 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933398AbbCFKvS (ORCPT ); Fri, 6 Mar 2015 05:51:18 -0500 Date: Fri, 6 Mar 2015 11:51:59 +0100 From: Petr Mladek To: Masami Hiramatsu Cc: Josh Poimboeuf , Seth Jennings , Jiri Kosina , Vojtech Pavlik , live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, Rusty Russell , Miroslav Benes , mingo@kernel.org, mathieu.desnoyers@efficios.com, oleg@redhat.com, paulmck@linux.vnet.ibm.com, andi@firstfloor.org, rostedt@goodmis.org, tglx@linutronix.de Subject: Re: Re: [PATCH 2/2] livepatch: fix patched module loading race Message-ID: <20150306105159.GJ15177@pathway.suse.cz> References: <297cb260437910c43bf0ec85f53578f6cfef6d88.1425423640.git.jpoimboe@redhat.com> <20150304131752.GD15177@pathway.suse.cz> <54F7A8D9.7090808@hitachi.com> <20150305141845.GB1870@treble.redhat.com> <54F901CB.7060601@hitachi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54F901CB.7060601@hitachi.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri 2015-03-06 10:24:27, Masami Hiramatsu wrote: > (2015/03/05 23:18), Josh Poimboeuf wrote: > > On Thu, Mar 05, 2015 at 09:52:41AM +0900, Masami Hiramatsu wrote: > >> (2015/03/04 22:17), Petr Mladek wrote: > >>> On Tue 2015-03-03 17:02:22, Josh Poimboeuf wrote: > >>>> It's possible for klp_register_patch() to see a module before the COMING > >>>> notifier is called, or after the GOING notifier is called. > >>>> > >>>> That can cause all kinds of ugly races. As Pter Mladek reported: > >>>> > >>>> "The problem is that we do not keep the klp_mutex lock all the time when > >>>> the module is being added or removed. > >>>> > >>>> First, the module is visible even before ftrace is ready. If we enable a patch > >>>> in this time frame, adding ftrace ops will fail and the patch will get rejected > >>>> just because bad timing. > >>> > >>> Ah, this is not true after all. I did not properly check when > >>> MODULE_STATE_COMING was set. I though that it was before ftrace was > >>> initialized but it was not true. > >>> > >>> > >>>> Second, if we are "lucky" and enable the patch for the coming module when the > >>>> ftrace is ready but before the module notifier has been called. The notifier > >>>> will try to enable the patch as well. It will detect that it is already patched, > >>>> return error, and the module will get rejected just because bad > >>>> timing. The more serious problem is that it will not call the notifier for > >>>> going module, so that the mess will stay there and we wont be able to load > >>>> the module later. > >>> > >>> Ah, the race is there but the effect is not that serious in the > >>> end. It seems that errors from module notifiers are ignored. In fact, > >>> we do not propagate the error from klp_module_notify_coming(). It means > >>> that WARN() from klp_enable_object() will be printed but the module > >>> will be loaded and patched. > >>> > >>> I am sorry, I was confused by kGraft where kgr_module_init() was > >>> called directly from module_load(). The errors were propagated. It > >>> means that kGraft rejects module when the patch cannot be applied. > >>> > >>> Note that the current solution is perfectly fine for the simple > >>> consistency model. > >>> > >>> > >>>> Third, similar problems are there for going module. If a patch is enabled after > >>>> the notifier finishes but before the module is removed from the list of modules, > >>>> the new patch will be applied to the module. The module might disappear at > >>>> anytime when the patch enabling is in progress, so there might be an access out > >>>> of memory. Or the whole patch might be applied and some mess will be left, > >>>> so it will not be possible to load/patch the module again." > >>> > >>> This is true. > >> > >> No, that's not true if you try_get_module() before patching. After the > >> module state goes GOING (more correctly say, after try_release_module_ref() > >> succeeded), all try_get_module() must fail :) > >> So, please make sure to get module when applying patches. > > > > Hi Masami, > > > > As Jikos pointed out elsewhere, try_get_module() won't solve all the > > GOING races. > > > > The module can be in GOING before mod->exit() is called. If we apply a > > patch between GOING getting set and mod->exit(), try_module_get() will > > fail and the module won't be patched. But module code can still run > > before or during mod->exit(), so the unpatched module code might > > interact badly with new patched code elsewhere. > > Hmm, in that case, we'd better have new GONE state for the module. > At least kprobe needs it. What is the exact problem with kprobes, please? Note that the notifiers for MODULE_STATE_GOING are called after mod->exit(). Therefore it is safe to kill kprobes the fast way when using the notifier. Best Regards, Petr