From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6E4FC433FF for ; Fri, 9 Aug 2019 21:00:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 88E5320B7C for ; Fri, 9 Aug 2019 21:00:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="pcYp9yfI"; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="UefUInbU" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728007AbfHIVAd (ORCPT ); Fri, 9 Aug 2019 17:00:33 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:57842 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726053AbfHIVAc (ORCPT ); Fri, 9 Aug 2019 17:00:32 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x79KxT2t044817; Fri, 9 Aug 2019 21:00:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=corp-2019-08-05; bh=9sLztaDMB8aeSX3WL8ufO6Qlj+pu4SLKowKGxHUAYS8=; b=pcYp9yfI3GuKYjG0G1hZnUZ4mlVzJhMn8SDnFIAzHBL80YDKrjzKOIYK5jCrJnoZCC7T RRO8mCE2iaWu46snuI9l0EoYMXANzjdEMQ57i1CRXiO6XGPmABWxu2DmDbWlcq7LKuyz yNF6hBnc83a66hJimNA6UgiMfpJe0JRVQWDKBFil0LYD8mLzdiOg2n96TIJwiX282nbH c9Pjrfahh0u7DRAhZHcOitDBo33YQph1lvHMNnfypeliX/9mlv6NZyaXOU7uSuk/Yyl5 5vi7kruxTckNH64kZyDfpogI8yEcQJ7JtVKRpFpngUgaLve2gShL6D1Z25RDCpAIQ70n Cw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=9sLztaDMB8aeSX3WL8ufO6Qlj+pu4SLKowKGxHUAYS8=; b=UefUInbUUZVRUhWHdFjDeVmYihsfyeb61d47cAMjPnlxeMdTTxQ76xdZCw2SPU++rVKt bG0sK1hFkwkSCJS/zxMt/pdcl0734OhVrIAp8qyhNAOUQkwXW/GeHY8WNyXFIgRV/pV7 86uwKsYH94fY2QX0WVToNPbbsBB1VL+XOdq9/mjdldNniAKHzPrIzYiec+dKe36ZfSW7 15CKaLG/oZyet44aTb2NcVFdZ7b934CoDlRI8j4hKxohV9Hh6dP84fv3f65XPbtYhZq7 PCFCjFg8zoSvpYzoXE4PerYHA7HB53JMytkkEs8emfM8PAk1NNjTa1dy7pcK1wejnL69 hw== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by userp2130.oracle.com with ESMTP id 2u8hpsa0jb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 09 Aug 2019 21:00:24 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x79KvYJo172016; Fri, 9 Aug 2019 21:00:24 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userp3030.oracle.com with ESMTP id 2u8pj9jvyc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 09 Aug 2019 21:00:24 +0000 Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x79L0NTc027268; Fri, 9 Aug 2019 21:00:23 GMT Received: from [192.168.1.222] (/71.63.128.209) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 09 Aug 2019 14:00:23 -0700 Subject: Re: [RFC PATCH] hugetlbfs: Add hugetlb_cgroup reservation limits To: Mina Almasry Cc: =?UTF-8?Q?Michal_Koutn=c3=bd?= , shuah , David Rientjes , Shakeel Butt , Greg Thelen , akpm@linux-foundation.org, khalid.aziz@oracle.com, open list , linux-mm@kvack.org, linux-kselftest@vger.kernel.org, cgroups@vger.kernel.org References: <20190808194002.226688-1-almasrymina@google.com> <20190809112738.GB13061@blackbody.suse.cz> From: Mike Kravetz Message-ID: Date: Fri, 9 Aug 2019 14:00:21 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9344 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=509 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908090205 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9344 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=533 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908090205 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 8/9/19 1:57 PM, Mina Almasry wrote: > On Fri, Aug 9, 2019 at 1:39 PM Mike Kravetz wrote: >> >> On 8/9/19 11:05 AM, Mina Almasry wrote: >>> On Fri, Aug 9, 2019 at 4:27 AM Michal Koutný wrote: >>>>> Alternatives considered: >>>>> [...] >>>> (I did not try that but) have you considered: >>>> 3) MAP_POPULATE while you're making the reservation, >>> >>> I have tried this, and the behaviour is not great. Basically if >>> userspace mmaps more memory than its cgroup limit allows with >>> MAP_POPULATE, the kernel will reserve the total amount requested by >>> the userspace, it will fault in up to the cgroup limit, and then it >>> will SIGBUS the task when it tries to access the rest of its >>> 'reserved' memory. >>> >>> So for example: >>> - if /proc/sys/vm/nr_hugepages == 10, and >>> - your cgroup limit is 5 pages, and >>> - you mmap(MAP_POPULATE) 7 pages. >>> >>> Then the kernel will reserve 7 pages, and will fault in 5 of those 7 >>> pages, and will SIGBUS you when you try to access the remaining 2 >>> pages. So the problem persists. Folks would still like to know they >>> are crossing the limits on mmap time. >> >> If you got the failure at mmap time in the MAP_POPULATE case would this >> be useful? >> >> Just thinking that would be a relatively simple change. > > Not quite, unfortunately. A subset of the folks that want to use > hugetlb memory, don't want to use MAP_POPULATE (IIRC, something about > mmaping a huge amount of hugetlb memory at their jobs' startup, and > doing that with MAP_POPULATE adds so much to their startup time that > it is prohibitively expensive - but that's just what I vaguely recall > offhand. I can get you the details if you're interested). Yes, MAP_POPULATE can get expensive as you will need to zero all those huge pages. -- Mike Kravetz