From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DD40D70816 for ; Thu, 19 Dec 2024 07:59:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734595171; cv=none; b=fP+zwrTTwLzDhkbzzy1qulk8+Rywrd41WgZi+ciQrfCCR2xw5Egmh9pjY0LG+I+0YbQn2O4kGiPzz+xt2MTpEJ/503A24qStyDqNOdq+XucGAAVlQqqSKph5jxX4YS2fADjnChT5meMfuTH8HJSxIVscrksReTb9dlhC7mDFwoQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734595171; c=relaxed/simple; bh=cVzdfIOuxXc6xAQvsucqfLIQ1jQX7CYQQKVAfNOhXCg=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=OKifLI8S8kVqISQzwGZzII3n3Gu1LN7E+FApPrifJZWIzkQFK/p9zvid/WLjlL2XdtnboaKOucMgG/p3r+1M7AZHJD9r2JTSWc87ZKAS9TEEqnANt+UBX8v10H26wwu10PVeT9yynQ8I1HVji2K3M20a3RftXxAPx1Y/+8V+9Lk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 970A91477; Wed, 18 Dec 2024 23:59:53 -0800 (PST) Received: from [10.163.79.180] (unknown [10.163.79.180]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1CDC43F720; Wed, 18 Dec 2024 23:59:14 -0800 (PST) Message-ID: Date: Thu, 19 Dec 2024 13:29:11 +0530 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH 10/12] khugepaged: Skip PTE range if a larger mTHP is already mapped To: John Hubbard , Ryan Roberts , akpm@linux-foundation.org, david@redhat.com, willy@infradead.org, kirill.shutemov@linux.intel.com Cc: anshuman.khandual@arm.com, catalin.marinas@arm.com, cl@gentwo.org, vbabka@suse.cz, mhocko@suse.com, apopple@nvidia.com, dave.hansen@linux.intel.com, will@kernel.org, baohua@kernel.org, jack@suse.cz, srivatsa@csail.mit.edu, haowenchao22@gmail.com, hughd@google.com, aneesh.kumar@kernel.org, yang@os.amperecomputing.com, peterx@redhat.com, ioworker0@gmail.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, jglisse@google.com, surenb@google.com, vishal.moola@gmail.com, zokeefe@google.com, zhengqi.arch@bytedance.com, 21cnbao@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20241216165105.56185-1-dev.jain@arm.com> <20241216165105.56185-11-dev.jain@arm.com> <7cc1840b-6f6c-4f82-86b8-41bb6fbc1b81@arm.com> <00d429c9-6ade-42c9-a1f3-a7519375324f@nvidia.com> Content-Language: en-US From: Dev Jain In-Reply-To: <00d429c9-6ade-42c9-a1f3-a7519375324f@nvidia.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 19/12/24 9:10 am, John Hubbard wrote: > On 12/18/24 1:34 AM, Dev Jain wrote: >> On 18/12/24 1:06 pm, Ryan Roberts wrote: >>> On 16/12/2024 16:51, Dev Jain wrote: >>>> We may hit a situation wherein we have a larger folio mapped. It is >>>> incorrect >>>> to go ahead with the collapse since some pages will be unmapped, >>>> leading to >>>> the entire folio getting unmapped. Therefore, skip the >>>> corresponding range. > ... >>> It would be good if you can spell out the desired policy when >>> khugepaged hits >>> partially unmapped large folios and unaligned large folios. I think >>> the simple >>> approach is to always collapse them to fully mapped, aligned folios >>> even if the >>> resulting order is smaller than the original. But I'm not sure >>> that's definitely >>> going to always be the best thing. >>> >>> Regardless, I'm struggling to understand the logic in this patch. >>> Taking the >>> order of a folio based on having hit one of it's pages says anything >>> about >>> whether the whole of that folio is mapped or not or it's alignment. >>> And it's not >>> clear to me how we would get to a situation where we are scanning >>> for a lower >>> order and find a (fully mapped, aligned) folio of higher order in >>> the first place. >>> >>> Let's assume the desired policy is that khugepaged should always >>> collapse to >>> naturally aligned large folios. If there happens to be an existing >>> aligned >>> order-4 folio that is fully mapped, we will identify that for >>> collapse as part >>> of the scan for order-4. At that point, we should just notice that >>> it is already >>> an aligned order-4 folio and bypass collapse. Of course we may have >>> already >>> chosen to collapse it into a higher order, but we should definitely >>> not get to a >>> lower order before we notice it. >>> >>> Hmm... I guess if the sysfs thp settings have been changed then >>> things could get >>> spicy... if order-8 was previously enabled and we have an order-8 >>> folio, then it >>> get's disabled and khugepaged is scanning for order-4 (which is >>> still enabled) >>> then hits the order-8; what's the expected policy? rework into 2 >>> order-4 folios >>> or leave it as as single order-8? >> >> Exactly, sorry, I should have made it clear in the patch description >> that I am >> handling the following scenario: there is a long running system on >> which we are >> using order-8 folios, and now we decide to downgrade to order-4. Will >> it be a >> good idea to take the pain of splitting order-8 to 16 order-4 folios? >> This should >> be a rare situation in the first place, so I have currently decided >> to ignore the >> folios set up by the previous sysfs setting and only focus on >> collapsing fresh memory. >> >> Thinking again, a sys-admin deciding to downgrade order of folios, >> should do that in >> the hopes of reducing internal fragmentation or increasing swap speed >> etc, so it makes >> sense to shatter large folios....maybe we can have a sysfs tunable >> for this? > > Maybe we should not support it (at runtime) at all. We are trying to > build > systems that don't require incredibly detailed sysadmin involvement, and > this level of tweaking qualifies, thoroughly, as "incredibly detailed > sysadmin micromanagement", imho. Ryan pointed out one thing: what about unaligned, or partially mapped large folios? For the previous sysfs settings, it may happen that we have an unaligned order-8 folio, let us say it got unaligned due to mremap(). Then it is a good idea to start from the order-4 aligned page and start collapsing memory so that we can take advantage of the contig bit. Otherwise if it is a fully-mapped aligned order-8 folio, then we anyways are abusing the contig bit advantage so collapsing is pointless. > > Apologies for not having gone through the series in detail yet, but this > point jumped out at me. > > thanks,