From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752635AbdK1OMU (ORCPT ); Tue, 28 Nov 2017 09:12:20 -0500 Received: from mail-wm0-f53.google.com ([74.125.82.53]:40791 "EHLO mail-wm0-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752235AbdK1OMS (ORCPT ); Tue, 28 Nov 2017 09:12:18 -0500 X-Google-Smtp-Source: AGs4zMbdAfl5YCpoUcPlpyDqmv4GM4fV4uJY4/7gMDBVxmPt/SWUZGtYE/TdgJnyJzj/CPzUh09JfA== From: Michal Hocko To: Cc: Mike Kravetz , Naoya Horiguchi , LKML Subject: Re: hugetlb page migration vs. overcommit Date: Tue, 28 Nov 2017 15:12:09 +0100 Message-Id: <20171128141211.11117-1-mhocko@kernel.org> X-Mailer: git-send-email 2.15.0 In-Reply-To: <20171128101907.jtjthykeuefxu7gl@dhcp22.suse.cz> References: <20171128101907.jtjthykeuefxu7gl@dhcp22.suse.cz> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue 28-11-17 11:19:07, Michal Hocko wrote: > On Wed 22-11-17 16:28:32, Michal Hocko wrote: > > Hi, > > is there any reason why we enforce the overcommit limit during hugetlb > > pages migration? It's in alloc_huge_page_node->__alloc_buddy_huge_page > > path. I am wondering whether this is really an intentional behavior. > > The page migration allocates a page just temporarily so we should be > > able to go over the overcommit limit for the migration duration. The > > reason I am asking is that hugetlb pages tend to be utilized usually > > (otherwise the memory would be just wasted and pool shrunk) but then > > the migration simply fails which breaks memory hotplug and other > > migration dependent functionality which is quite suboptimal. You can > > workaround that by increasing the overcommit limit. > > > > Why don't we simply migrate as long as we are able to allocate the > > target hugetlb page? I have a half baked patch to remove this > > restriction, would there be an opposition to do something like that? > > So I finally got to think about this some more and looked at how we > actually account things more thoroughly. And it is, you both of you > expected, quite subtle and not easy to get around. Per NUMA pools make > things quite complicated. Why? Migration can really increase the overall > pool size. Say we are migrating from Node1 to Node2. Node2 doesn't have > any pre-allocated pages but assume that the overcommit allows us to move > on. All good. Except that the original page will return to the pool > because free_huge_page will see Node1 without any surplus pages and > therefore moves back the page to the pool. Node2 will release the > surplus page only after it is freed which can be an unbound amount of > time. > > While we are still effectively under the overcommit limit the semantic > is kind of strange and I am not sure the behavior is really intended. > I see why per node surplus counter is used here. We simply want to > maintain per node counts after regular page free. So I was thinking > to add a temporary/migrate state to the huge page for migration pages > (start with new page, state transfered to the old page on success) and > free such a page to the allocator regardless of the surplus counters. > > This would mean that the page migration might change inter node pool > sizes but I guess that should be acceptable. What do you guys think? > I can send a draft patch if that helps you to understand the idea. This is what I have currently and it seems to work (or at least it doesn't it doesn't blow up immediately). The first patch is a cleanup and patch2 implements the temporary page idea. Does this make any sense to you at all? -- Michal Hocko