From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C741C04EBF for ; Tue, 4 Dec 2018 09:28:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3467A20851 for ; Tue, 4 Dec 2018 09:28:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3467A20851 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ah.jp.nec.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726051AbeLDJ2O convert rfc822-to-8bit (ORCPT ); Tue, 4 Dec 2018 04:28:14 -0500 Received: from tyo162.gate.nec.co.jp ([114.179.232.162]:43180 "EHLO tyo162.gate.nec.co.jp" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725767AbeLDJ2N (ORCPT ); Tue, 4 Dec 2018 04:28:13 -0500 Received: from mailgate01.nec.co.jp ([114.179.233.122]) by tyo162.gate.nec.co.jp (8.15.1/8.15.1) with ESMTPS id wB49RxoB003372 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 4 Dec 2018 18:27:59 +0900 Received: from mailsv01.nec.co.jp (mailgate-v.nec.co.jp [10.204.236.94]) by mailgate01.nec.co.jp (8.15.1/8.15.1) with ESMTP id wB49RwpE026185; Tue, 4 Dec 2018 18:27:58 +0900 Received: from mail01b.kamome.nec.co.jp (mail01b.kamome.nec.co.jp [10.25.43.2]) by mailsv01.nec.co.jp (8.15.1/8.15.1) with ESMTP id wB49QvFr002275; Tue, 4 Dec 2018 18:27:58 +0900 Received: from bpxc99gp.gisp.nec.co.jp ([10.38.151.147] [10.38.151.147]) by mail03.kamome.nec.co.jp with ESMTP id BT-MMP-320053; Tue, 4 Dec 2018 18:11:06 +0900 Received: from BPXM23GP.gisp.nec.co.jp ([10.38.151.215]) by BPXC19GP.gisp.nec.co.jp ([10.38.151.147]) with mapi id 14.03.0319.002; Tue, 4 Dec 2018 18:11:05 +0900 From: Naoya Horiguchi To: Michal Hocko CC: Oscar Salvador , Andrew Morton , Dan Williams , "Pavel Tatashin" , "linux-mm@kvack.org" , LKML , Stable tree Subject: Re: [RFC PATCH] hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined Thread-Topic: [RFC PATCH] hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined Thread-Index: AQHUiu9tY5+24nB+PkGyMm7atS1YiqVtlyQAgAAYWwCAAAZUgA== Date: Tue, 4 Dec 2018 09:11:05 +0000 Message-ID: <20181204091104.GA3788@hori1.linux.bs1.fc.nec.co.jp> References: <20181203100309.14784-1-mhocko@kernel.org> <20181204072116.GA24446@hori1.linux.bs1.fc.nec.co.jp> <20181204081801.GA1286@dhcp22.suse.cz> In-Reply-To: <20181204081801.GA1286@dhcp22.suse.cz> Accept-Language: en-US, ja-JP Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.51.8.80] Content-Type: text/plain; charset="iso-2022-jp" Content-ID: <868BCEB5723EFF4496D5263F47421292@gisp.nec.co.jp> Content-Transfer-Encoding: 8BIT MIME-Version: 1.0 X-TM-AS-MML: disable Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Dec 04, 2018 at 09:48:26AM +0100, Michal Hocko wrote: > On Tue 04-12-18 07:21:16, Naoya Horiguchi wrote: > > On Mon, Dec 03, 2018 at 11:03:09AM +0100, Michal Hocko wrote: > > > From: Michal Hocko > > > > > > We have received a bug report that an injected MCE about faulty memory > > > prevents memory offline to succeed. The underlying reason is that the > > > HWPoison page has an elevated reference count and the migration keeps > > > failing. There are two problems with that. First of all it is dubious > > > to migrate the poisoned page because we know that accessing that memory > > > is possible to fail. Secondly it doesn't make any sense to migrate a > > > potentially broken content and preserve the memory corruption over to a > > > new location. > > > > > > Oscar has found out that it is the elevated reference count from > > > memory_failure that is confusing the offlining path. HWPoisoned pages > > > are isolated from the LRU list but __offline_pages might still try to > > > migrate them if there is any preceding migrateable pages in the pfn > > > range. Such a migration would fail due to the reference count but > > > the migration code would put it back on the LRU list. This is quite > > > wrong in itself but it would also make scan_movable_pages stumble over > > > it again without any way out. > > > > > > This means that the hotremove with hwpoisoned pages has never really > > > worked (without a luck). HWPoisoning really needs a larger surgery > > > but an immediate and backportable fix is to skip over these pages during > > > offlining. Even if they are still mapped for some reason then > > > try_to_unmap should turn those mappings into hwpoison ptes and cause > > > SIGBUS on access. Nobody should be really touching the content of the > > > page so it should be safe to ignore them even when there is a pending > > > reference count. > > > > > > Debugged-by: Oscar Salvador > > > Cc: stable > > > Signed-off-by: Michal Hocko > > > --- > > > Hi, > > > I am sending this as an RFC now because I am not fully sure I see all > > > the consequences myself yet. This has passed a testing by Oscar but I > > > would highly appreciate a review from Naoya about my assumptions about > > > hwpoisoning. E.g. it is not entirely clear to me whether there is a > > > potential case where the page might be still mapped. > > > > One potential case is ksm page, for which we give up unmapping and leave > > it unmapped. Rather than that I don't have any idea, but any new type of > > page would be potentially categorized to this class. > > Could you be more specific why hwpoison code gives up on ksm pages while > we can safely unmap here? Actually no big reason. Ksm pages never dominate memory, so we simply didn't have strong motivation to save the pages. > [...] > > > > I think this looks OK (no better idea.) > > > > Reviewed-by: Naoya Horiguchi > > Thanks! > > > I wondered why I didn't find this for long, and found that my testing only > > covered the case where PageHWPoison is the first page of memory block. > > scan_movable_pages() considers PageHWPoison as non-movable, so do_migrate_range() > > started with pfn after the PageHWPoison and never tried to migrate it > > (so effectively ignored every PageHWPoison as the above code does.) > > Yeah, it seems that the hotremove worked only by chance in presence of > hwpoison pages so far. The specific usecase which triggered this patch > is a heavily memory utilized system with in memory database IIRC. So it > is quite likely that hwpoison pages are punched to otherwise used > memory. > > Thanks for the review Naoya! Your welcome, and thank you for reporting/fixing the issue. - Naoya