From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 93B411A2396 for ; Thu, 19 Dec 2024 08:07:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734595659; cv=none; b=hQt6q+FzNkrNrrEgveL9sb0DTuVmF2m1Z5ZV/86mVbz75ewR+2Uqcjko66vg5RrYwmG0FcTzDcra5e5bM9jOlufr8byyVOXwRDRZi3hNjV12esin+iyJDrJxBr65kPskt/Gd0Qli+cPogB5irR0iOSixJhJ5MWgOrcdozhkkglY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734595659; c=relaxed/simple; bh=7bwgG7LmkZF0s6dcPsiD60HZFHv4AGcdhEoQAv8JJY4=; h=Message-ID:Date:MIME-Version:Subject:From:To:Cc:References: In-Reply-To:Content-Type; b=ePTX4rN6BJv97sZZ5SwMZX2pQBqLtH49HzDYD5a2dbek4HaT0/C43LPCPu8rCzm3ZmYlycAD7J/7V8fcV0lLbLiFbTsisYRAZfrzq3g38VC9zUelB8tCElXDhnUlcQ8b8FfIhciOv0uP6DQn/MrX4X039kJTQDgwKOtec7JsrFE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C48DF1477; Thu, 19 Dec 2024 00:08:03 -0800 (PST) Received: from [10.163.79.180] (unknown [10.163.79.180]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 606033F720; Thu, 19 Dec 2024 00:07:25 -0800 (PST) Message-ID: <33488d3c-a176-4779-a9c6-e9cbdd1afc62@arm.com> Date: Thu, 19 Dec 2024 13:37:21 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH 10/12] khugepaged: Skip PTE range if a larger mTHP is already mapped From: Dev Jain To: John Hubbard , Ryan Roberts , akpm@linux-foundation.org, david@redhat.com, willy@infradead.org, kirill.shutemov@linux.intel.com Cc: anshuman.khandual@arm.com, catalin.marinas@arm.com, cl@gentwo.org, vbabka@suse.cz, mhocko@suse.com, apopple@nvidia.com, dave.hansen@linux.intel.com, will@kernel.org, baohua@kernel.org, jack@suse.cz, srivatsa@csail.mit.edu, haowenchao22@gmail.com, hughd@google.com, aneesh.kumar@kernel.org, yang@os.amperecomputing.com, peterx@redhat.com, ioworker0@gmail.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, jglisse@google.com, surenb@google.com, vishal.moola@gmail.com, zokeefe@google.com, zhengqi.arch@bytedance.com, 21cnbao@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20241216165105.56185-1-dev.jain@arm.com> <20241216165105.56185-11-dev.jain@arm.com> <7cc1840b-6f6c-4f82-86b8-41bb6fbc1b81@arm.com> <00d429c9-6ade-42c9-a1f3-a7519375324f@nvidia.com> Content-Language: en-US In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 19/12/24 1:29 pm, Dev Jain wrote: > > On 19/12/24 9:10 am, John Hubbard wrote: >> On 12/18/24 1:34 AM, Dev Jain wrote: >>> On 18/12/24 1:06 pm, Ryan Roberts wrote: >>>> On 16/12/2024 16:51, Dev Jain wrote: >>>>> We may hit a situation wherein we have a larger folio mapped. It >>>>> is incorrect >>>>> to go ahead with the collapse since some pages will be unmapped, >>>>> leading to >>>>> the entire folio getting unmapped. Therefore, skip the >>>>> corresponding range. >> ... >>>> It would be good if you can spell out the desired policy when >>>> khugepaged hits >>>> partially unmapped large folios and unaligned large folios. I think >>>> the simple >>>> approach is to always collapse them to fully mapped, aligned folios >>>> even if the >>>> resulting order is smaller than the original. But I'm not sure >>>> that's definitely >>>> going to always be the best thing. >>>> >>>> Regardless, I'm struggling to understand the logic in this patch. >>>> Taking the >>>> order of a folio based on having hit one of it's pages says >>>> anything about >>>> whether the whole of that folio is mapped or not or it's alignment. >>>> And it's not >>>> clear to me how we would get to a situation where we are scanning >>>> for a lower >>>> order and find a (fully mapped, aligned) folio of higher order in >>>> the first place. >>>> >>>> Let's assume the desired policy is that khugepaged should always >>>> collapse to >>>> naturally aligned large folios. If there happens to be an existing >>>> aligned >>>> order-4 folio that is fully mapped, we will identify that for >>>> collapse as part >>>> of the scan for order-4. At that point, we should just notice that >>>> it is already >>>> an aligned order-4 folio and bypass collapse. Of course we may have >>>> already >>>> chosen to collapse it into a higher order, but we should definitely >>>> not get to a >>>> lower order before we notice it. >>>> >>>> Hmm... I guess if the sysfs thp settings have been changed then >>>> things could get >>>> spicy... if order-8 was previously enabled and we have an order-8 >>>> folio, then it >>>> get's disabled and khugepaged is scanning for order-4 (which is >>>> still enabled) >>>> then hits the order-8; what's the expected policy? rework into 2 >>>> order-4 folios >>>> or leave it as as single order-8? >>> >>> Exactly, sorry, I should have made it clear in the patch description >>> that I am >>> handling the following scenario: there is a long running system on >>> which we are >>> using order-8 folios, and now we decide to downgrade to order-4. >>> Will it be a >>> good idea to take the pain of splitting order-8 to 16 order-4 >>> folios? This should >>> be a rare situation in the first place, so I have currently decided >>> to ignore the >>> folios set up by the previous sysfs setting and only focus on >>> collapsing fresh memory. >>> >>> Thinking again, a sys-admin deciding to downgrade order of folios, >>> should do that in >>> the hopes of reducing internal fragmentation or increasing swap >>> speed etc, so it makes >>> sense to shatter large folios....maybe we can have a sysfs tunable >>> for this? >> >> Maybe we should not support it (at runtime) at all. We are trying to >> build >> systems that don't require incredibly detailed sysadmin involvement, and >> this level of tweaking qualifies, thoroughly, as "incredibly detailed >> sysadmin micromanagement", imho. > > Ryan pointed out one thing: what about unaligned, or partially mapped > large > folios? For the previous sysfs settings, it may happen that we have an > unaligned > order-8 folio, let us say it got unaligned due to mremap(). Then it is > a good > idea to start from the order-4 aligned page and start collapsing > memory so > that we can take advantage of the contig bit. Otherwise if it is a > fully-mapped > aligned order-8 folio, then we anyways are abusing the contig bit > advantage > so collapsing is pointless. In fact, in the current code, we are collapsing an unaligned PMD-size folio to an aligned PMD-mapped folio; we will not see a block mapping in the PMD, and go ahead with the scan...so the logic should be, skip the scan if the VAs and PAs are aligned. >> >> Apologies for not having gone through the series in detail yet, but this >> point jumped out at me. >> >> thanks, >