From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FFE0398FB5 for ; Thu, 4 Dec 2025 19:39:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764877143; cv=none; b=ZiF0P/cyzPbWQl1RUmxOuwK8sUJAYnHcTRDsn52ultLtV9lYrFUqCBMxmM2m68LGGyJLqYbT3Ix7Ne+hIqSWJ/uWMA6Yz8fC538sHSyZ3vB2eFR8iOYXdfBrueIfxtCjqb/Sa5sLm6A/qGQoAh9ktXgaTVVT7ZOkG5H/1wpym1c= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1764877143; c=relaxed/simple; bh=7xoWfZb/rQlOyV51uYCWOVPmwkaZ4pvqIYT0hW3+62M=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=ThpUlrCJSSkI1O5Z5My/tsV3XK6bxP7ALP2swQbaSpIhxJRHNIheXwfF9gWliAlKr8XP1CVoSQyHNwYMP450PMOdh/bC2jUuqqp3/hQtlrv1Wxq+HHo0LADTKU24krhvOyvLUqVSs8MGUydOoX2SKDtzi9qEintFm9VxNgJRxhc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Lxo07IQA; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Lxo07IQA" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9ED09C4CEFB; Thu, 4 Dec 2025 19:38:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1764877143; bh=7xoWfZb/rQlOyV51uYCWOVPmwkaZ4pvqIYT0hW3+62M=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=Lxo07IQAhkWMt2l8zQ9z0zjXSecELZqm/Sl/HVxpGQ3HrR32VZ/HcjuZZIECRJyuc b9AzhJXdV5j/DiFeh1DcrpWMQCqz/W0KmqtebGB5fy6P7btaWJ2f325YRNUM085/yT 8ax/GnH3DxXkdSyLt/Wkxp+BHtMijuaoql6FBscs/4yJUFlyB78XqPA/3MI/alf+Ga zCzXcq75PuK2Yx/PGKMG+wUeST30nFTCs+39VviL+9YHWYnLqEmJgSWHEyOtKHT6X9 rMeg0TNesxFsC8wSY69xa/Dtn+qNfbKDxsspDfh93eWHo51STHZK6sqNvm3w8dX4gG T2SwhXwLTyAhQ== Message-ID: <8a5b59b9-112d-44cf-b81e-6f79f59bb999@kernel.org> Date: Thu, 4 Dec 2025 20:38:56 +0100 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] Revert "mm: fix MAX_FOLIO_ORDER on powerpc configs with hugetlb" To: Shuah Khan , Mike Rapoport Cc: akpm@linux-foundation.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, lorenzo.stoakes@oracle.com, Liam.Howlett@oracle.com, vbabka@suse.cz, surenb@google.com, mhocko@suse.com, masahiroy@kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20251204023358.54107-1-skhan@linuxfoundation.org> <0b007374-1058-487c-8033-4f0d2830dc89@kernel.org> <78af7da4-d213-42c6-8ca6-c2bdca81f233@linuxfoundation.org> From: "David Hildenbrand (Red Hat)" Content-Language: en-US In-Reply-To: <78af7da4-d213-42c6-8ca6-c2bdca81f233@linuxfoundation.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 12/4/25 18:03, Shuah Khan wrote: > On 12/3/25 23:35, Mike Rapoport wrote: >> On Thu, Dec 04, 2025 at 07:17:06AM +0100, David Hildenbrand (Red Hat) wrote: >>> Hi, >>> >>> On 12/4/25 03:33, Shuah Khan wrote: >>>> This reverts commit 39231e8d6ba7f794b566fd91ebd88c0834a23b98. >>> >>> That was supposed to fix powerpc handling though. So I think we have to >>> understand what is happening here. > > This patch changes include/linux/mm.h and mm/Kconfig in addition to > arch/powerpc/Kconfig and arch/powerpc/platforms/Kconfig.cputype > > With this patch HAVE_GIGANTIC_FOLIOS is enabled on x86_64 config > > The following mm/Kconfig isn't arch specific. This makes this > not powerpc specific and this is enabled on x86_64 Yes, and as the patch explains that's expected. See below. > > +# > +# We can end up creating gigantic folio. > +# > +config HAVE_GIGANTIC_FOLIOS > + def_bool (HUGETLB_PAGE && ARCH_HAS_GIGANTIC_PAGE) || \ > + (ZONE_DEVICE && HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) > + > > The following change in include/linux/mm.h is also generic > and applies to x86_64 as well. > > -#if !defined(CONFIG_ARCH_HAS_GIGANTIC_PAGE) > +#if !defined(CONFIG_HAVE_GIGANTIC_FOLIOS) > > Is this not intended on all architectures? All expected. See below. > >>> >>>> >>>> Enabling HAVE_GIGANTIC_FOLIOS broke kernel build and git clone on two >>>> systems. git fetch-pack fails when cloning large repos and make hangs >>>> or errors out of Makefile.build with Error: 139. These failures are >>>> random with git clone failing after fetching 1% of the objects, and >>>> make hangs while compiling random files. >>> >>> On which architecture do we see these issues and with which kernel configs? >>> Can you share one? > > Config attached. Okay, let's walk this through. The config has: CONFIG_HAVE_GIGANTIC_FOLIOS=y CONFIG_HUGETLB_PAGE=y CONFIG_ARCH_HAS_GIGANTIC_PAGE=y CONFIG_ZONE_DEVICE=y CONFIG_SPARSEMEM=y CONFIG_SPARSEMEM_VMEMMAP=y In the old code: #if !defined(CONFIG_ARCH_HAS_GIGANTIC_PAGE) /* * We don't expect any folios that exceed buddy sizes (and consequently * memory sections). */ #define MAX_FOLIO_ORDER MAX_PAGE_ORDER #elif defined(CONFIG_SPARSEMEM) && !defined(CONFIG_SPARSEMEM_VMEMMAP) /* * Only pages within a single memory section are guaranteed to be * contiguous. By limiting folios to a single memory section, all folio * pages are guaranteed to be contiguous. */ #define MAX_FOLIO_ORDER PFN_SECTION_SHIFT #else /* * There is no real limit on the folio size. We limit them to the maximum we * currently expect (e.g., hugetlb, dax). */ #define MAX_FOLIO_ORDER PUD_ORDER #endif We would get MAX_FOLIO_ORDER = PUD_ORDER = 18 In the new code we will get: #if !defined(CONFIG_HAVE_GIGANTIC_FOLIOS) /* * We don't expect any folios that exceed buddy sizes (and consequently * memory sections). */ #define MAX_FOLIO_ORDER MAX_PAGE_ORDER #elif defined(CONFIG_SPARSEMEM) && !defined(CONFIG_SPARSEMEM_VMEMMAP) /* * Only pages within a single memory section are guaranteed to be * contiguous. By limiting folios to a single memory section, all folio * pages are guaranteed to be contiguous. */ #define MAX_FOLIO_ORDER PFN_SECTION_SHIFT #elif defined(CONFIG_HUGETLB_PAGE) /* * There is no real limit on the folio size. We limit them to the maximum we * currently expect (see CONFIG_HAVE_GIGANTIC_FOLIOS): with hugetlb, we expect * no folios larger than 16 GiB on 64bit and 1 GiB on 32bit. */ #define MAX_FOLIO_ORDER get_order(IS_ENABLED(CONFIG_64BIT) ? SZ_16G : SZ_1G) #else /* * Without hugetlb, gigantic folios that are bigger than a single PUD are * currently impossible. */ #define MAX_FOLIO_ORDER PUD_ORDER #endif MAX_FOLIO_ORDER = get_order(SZ_16G) = 22 That's expected and okay (raising the maximum we expect), as we only want to set a rough upper cap on the maximum folio size. As I raised, observe how MAX_FOLIO_ORDER is only used to * trigger warnings if we observe an unexpectedly large folio size. Safety checks. * use it when dumping a folio to detect possible folio corruption on unexpected folio sizes > >>> >>>> >>>> The blow is is one of the git clone failures: >>>> >>>> git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git linux_6.19 >>>> Cloning into 'linux_6.19'... >>>> remote: Enumerating objects: 11173575, done. >>>> remote: Counting objects: 100% (785/785), done. >>>> remote: Compressing objects: 100% (373/373), done. >>>> remote: Total 11173575 (delta 534), reused 505 (delta 411), pack-reused 11172790 (from 1) >>>> Receiving objects: 100% (11173575/11173575), 3.00 GiB | 7.08 MiB/s, done. >>>> Resolving deltas: 100% (9195212/9195212), done. >>>> fatal: did not receive expected object 0002003e951b5057c16de5a39140abcbf6e44e50 >>>> fatal: fetch-pack: invalid index-pack output >>> >>> If I would have to guess, these symptoms match what we saw between commit >>> adfb6609c680 ("mm/huge_memory: initialise the tags of the huge zero folio") >>> and commit 5bebe8de1926 ("mm/huge_memory: Fix initialization of huge zero folio"). >>> >>> 5bebe8de1926 went into v6.18-rc7. >>> >>> Just to be sure, are you sure we were able to reproduce this issue with a >>> v6.18-rc7 or even v6.18 that contains 5bebe8de1926? >>> >>> Bisecting might give you wrong results, as the problems of adfb6609c680 do not >>> reproduce reliably. >> >> I can confirm that bisecting gives odd results between v6.18-rc5 and >> v6.18-rc6. I was seeing failures in some tests, bisected a few times and >> got a bunch of bogus commits including 3470715e5c22 ("MAINTAINERS: update >> David Hildenbrand's email address") :) > > I am sure this patch is the cause oh the problems I have seen on my two > systems. Reverting this commit solved issues since this commit does > impact all architectures enabling HAVE_GIGANTIC_FOLIOS if the conditions > are right. > >> >> And 5bebe8de1926 actually solved the issue for me. > > Were you seeing the problems I reported without 5bebe8de1926? > Is 5bebe8de1926 is 6.18? We were seeing all kinds of different segmentation faults or corruptions. In my case, every-time I tried to login something would segfault. For others, compilers stopped working or they got different random segfaults. Assume you think you have a shared zero page, but every time you reboot it's filled with other garbage data. Not good when your app assumes something contains 0s. > > I can try this commit with 39231e8d6ba7f794b566fd91ebd88c0834a23b98 > and see what happens on my system. Yes, please. I cannot yet make sense of how MAX_FOLIO_ORDER would make any difference. Unless you would actually be seeing one of the WARNINGS that are based on MAX_FOLIO_ORDER / MAX_FOLIO_NR_PAGES. But I guess none showed up in dmesg? -- Cheers David