From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D43DCC04ABB for ; Tue, 11 Sep 2018 15:28:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 96FAD20891 for ; Tue, 11 Sep 2018 15:28:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 96FAD20891 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727721AbeIKU15 (ORCPT ); Tue, 11 Sep 2018 16:27:57 -0400 Received: from mailout.easymail.ca ([64.68.200.34]:49748 "EHLO mailout.easymail.ca" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726804AbeIKU15 (ORCPT ); Tue, 11 Sep 2018 16:27:57 -0400 Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 58EE9C05E9; Tue, 11 Sep 2018 15:28:08 +0000 (UTC) Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (emo01-pco.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id V279YwNBSUIZ; Tue, 11 Sep 2018 15:28:08 +0000 (UTC) Received: from [192.168.1.87] (c-24-9-64-241.hsd1.co.comcast.net [24.9.64.241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id D3746C05D3; Tue, 11 Sep 2018 15:27:55 +0000 (UTC) Subject: Re: [PATCH] arm64: add NUMA emulation support To: Michal Hocko Cc: Will Deacon , catalin.marinas@arm.com, sudeep.holla@arm.com, ganapatrao.kulkarni@cavium.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Shuah Khan References: <20180828174011.GE20375@arm.com> <20180829110802.GD10349@dhcp22.suse.cz> <20180905064252.GW14951@dhcp22.suse.cz> <20180907083452.GC19621@dhcp22.suse.cz> <20180910134845.GH10951@dhcp22.suse.cz> <4d7beb50-f778-507a-4c3c-f6de92f8cfb9@kernel.org> <20180911091124.GP10951@dhcp22.suse.cz> From: Shuah Khan Message-ID: <86aad702-9f42-c572-a8f1-74983cd1bf33@kernel.org> Date: Tue, 11 Sep 2018 09:27:49 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <20180911091124.GP10951@dhcp22.suse.cz> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 09/11/2018 03:11 AM, Michal Hocko wrote: > On Mon 10-09-18 20:02:05, Shuah Khan wrote: >> Hi Michal, >> >> On 09/10/2018 07:48 AM, Michal Hocko wrote: >>> On Fri 07-09-18 16:30:59, Shuah Khan wrote: >>>> On 09/07/2018 02:34 AM, Michal Hocko wrote: >>>>> On Thu 06-09-18 15:53:34, Shuah Khan wrote: >> [....] >>>> >>>> In addition to isolation, being able to reserve a block instead is one of the >>>> issues I am looking to address. Unfortunately memory cgroups won't address that >>>> issue. >>> >>> Could you be more specific why you need reservations other than >>> isolation. >>> >> >> Taking automotive as a specific example, there are two classes of applications: >> 1. critical applications that must run >> 2. Infotainment and misc. user-space. >> >> In this case, being able to reserve a block of memory for critical applications >> will ensure the memory is available for them. If a critical application has to >> restart and/or when an on-demand critical application starts, it might not be able >> to allocate memory if it is not reserved. >> >> When a flat system has multiple memory blocks, with NUMA emulation in conjunction with >> cpusets, one or more block can be reserved for critical applications configuring a set >> of cpus and one of more memory nodes for them. >> >> Memory cgroups will not support such reservation. Hope this helps explain the use-case >> I am trying to address with this patch. > > OK, that is more clear. I still believe that you either have to have a > very good control over memory allocations or a good luck to not see > unexpected kernel allocations in your reserved memory which might easily > break guarantees you would like to accomplish. > Thanks. Right. I am with you on the possibility that root cgroup can eat into the reserved memory. However, with this solution I proposed, there is a guarantee that the cpuset cgroup that is configured for non-critical Infotainment and misc. user-space application will not be able to allocate from the reserved memory node. I am hoping the proposed patch will allow critical apps. reserving memory with the exception that root cgroup and kernel can still allocate from it when needed. Perhaps cpuset exclusive logic could be extended to look for non-exclusive memory nodes first if it doesn't already do that. This is inline with the current cpuset approach is that the critical kernel allocations aren't starved to ensure memory reservations. If you don't think this solution isn't ideal/good, do you have other suggestions for solving the problem? If not would it be okay to start with what I proposed and build on top of as needed? thanks, -- Shuah