From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D28E033CEA5 for ; Sat, 7 Feb 2026 20:19:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.45 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770495578; cv=none; b=CjMRC3c9XK5sDBrJdhJlRTQvXxpLovo/RdXWHo785X/T2xBLCbCsmf7CqmOwKHkmPiEgZclxdpNflOTe9UszqvjHHRP7wzvJAU+reMTmib4CIjeYFOFKVptYmyhXoNER+9f4iUj4J4Hszxac8IaYqszfnHMGEFj+ITPvdbAJ87U= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770495578; c=relaxed/simple; bh=YOvcABvi86FBPIGHFRbcrhtefQ+sifLNtLVYHDYqhb0=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=Hqp3QoqHqPUt8VQyrMmqiPpkgzZzncN8OBr1B71BwWnkK0i4+TKL+ULFrjjsxheJx5DaHGmOIHDLxyhFCrNIlRieABJIetbl/yHqXblBIgzVXiT8xHBQ8pC1yenKQjnCrs22UXAT+JLmgpNFgTMyXjNZwQKqLnR1M/vgc2FA4Hg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dEsjbI2/; arc=none smtp.client-ip=209.85.128.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dEsjbI2/" Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-4806cc07ce7so33138455e9.1 for ; Sat, 07 Feb 2026 12:19:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1770495576; x=1771100376; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=LmE23qjtgb2EulsiY+c7A9BQzukrzPbX/0RgHfT6JCg=; b=dEsjbI2/oSVceaO5SgvhoL1QeYtTq4zLx34miW8ywGggSbyosl62zptBT4s1TXAJJA cNB8EYIlvwfXswEUJhN24lkGCyWUbEKi9hSssIrhVfUUvIaeZVcX9XPpL6EiKh4dar6x 6fIyBZIEAfsWnnofEEI+5/S3dqJiG4JOoKfUewj0LqKclvYnc37eap+XDOcP9LF/W0dg zQsYlXLRUSf3OmCxgzBJJAbfPOSm/+V/AT12k6rCSSnl058UurWvia8S2pZyVWUVVFSF FEfCEJjv3NKjtCYv9uElyLNcPr3CeGXpifg46b7EK7UnUm2/pJ8CZENZu3aXnSds568Q 4JPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770495576; x=1771100376; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=LmE23qjtgb2EulsiY+c7A9BQzukrzPbX/0RgHfT6JCg=; b=rLVcD5KoH10xuKDMWs5OIB4TWAQOm1IeeK37Sx/LU+lT/n9iqsrfr7iKAYPxt3sKGn pMcnWiHM695k9kNHRkqwOipv0eu8xKWsplmEt+luBjA0AvbMHFMa4Sswt5YG/ZnqW6js YpmYCuYaUIfHtgl9buqsJ4uEhXoQiFBMso2bEPmMdUNB/jIUqKBSZz1LhneKy6VSDgsw q2seEHNBCg0F9+FlpidMxvF37ZEtYWJIlaf0aWcAtyNGaDuX/qXl2oNm67Pa6B8Yvn3y Pdf2tuXF77LAp2Ep6DncHgqGMXXmFSWZFp9QMUS5hepXu+DHwrIfUzutuPiMTfKigpLY tJeg== X-Forwarded-Encrypted: i=1; AJvYcCUj4YuWIyrEvMi1prWeDkwzU6mVQoZty7kTYmUa19XwxFS7mUtJo/4VKN3WwvDSAeUk2rMeYkNI0Skdlq8=@vger.kernel.org X-Gm-Message-State: AOJu0Yxzj9agfTZpArd9XO7yYsLnafLxamnYlgS671hquHv+Zwg6fzlf m5P8o5E5SnaIMzlNB7sg+pQTC8SxM9Boqbqq+B9LpiEdDUVivY2IAmE8 X-Gm-Gg: AZuq6aL17kN6E7s5UoajUZgFFtMIYLnve0KM/64zM4X1z7OiBkYMRIAZfwU8pHLSwjI TPnJbfl4jmOteApJ+ZScTZK28GgqwrJskA0SUmMJ9Uej1rb16L0jIKBe0QxOI1b0xpYN2Bug2sb g/5Bs8kvG96EBDvz0mUCrd6YVtPtZ8m3AIpRGWY4slIfbPlwktFJBJQwWKrZTxtJqyiCaphS4LR 7a9a6uapI4SzVbLO+RmMeV/n/rAH0sbANWZUleNY4Zgv3Q6VDqLg66vsYv4pGk5RZpG0V1miz6X kkz3FmIRRUFbcX7u/WXgRY4bKhtxvu78BRFinjez8qYJu8iguQXQ//TJmN5LfW65KhmBbUyDJJb 0FU0yxhTM4fDBN/Ob12AVF0O55gOMrHj6ScGO5jIbdMJddwKZVjF049+6C4t9LIPIzuB9XME8sM YkW9zGlDvVmLX0TstAIUnk4tTdZFlQYtWHWJ9b0DjCcuXkYdz0zq9ozLEYHZI8IO/3q1JqCGoig 0BQjCAt0l3QILQ= X-Received: by 2002:a05:600c:4f8a:b0:480:3230:6c9b with SMTP id 5b1f17b1804b1-483201dd004mr94651865e9.7.1770495575929; Sat, 07 Feb 2026 12:19:35 -0800 (PST) Received: from ?IPV6:2a02:6b6f:e752:9400:18cf:c773:ee86:c436? ([2a02:6b6f:e752:9400:18cf:c773:ee86:c436]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-483206cc7d3sm131304355e9.5.2026.02.07.12.19.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 07 Feb 2026 12:19:34 -0800 (PST) Message-ID: Date: Sat, 7 Feb 2026 20:19:34 +0000 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCHv6 07/17] mm: Rework compound_head() for power-of-2 sizeof(struct page) Content-Language: en-GB To: Kiryl Shutsemau , Andrew Morton , Muchun Song , David Hildenbrand , Matthew Wilcox , Frank van der Linden Cc: Oscar Salvador , Mike Rapoport , Vlastimil Babka , Lorenzo Stoakes , Zi Yan , Baoquan He , Michal Hocko , Johannes Weiner , Jonathan Corbet , Huacai Chen , WANG Xuerui , Palmer Dabbelt , Paul Walmsley , Albert Ou , Alexandre Ghiti , kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org References: <20260202155634.650837-1-kas@kernel.org> <20260202155634.650837-8-kas@kernel.org> From: Usama Arif In-Reply-To: <20260202155634.650837-8-kas@kernel.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 02/02/2026 15:56, Kiryl Shutsemau wrote: > For tail pages, the kernel uses the 'compound_info' field to get to the > head page. The bit 0 of the field indicates whether the page is a > tail page, and if set, the remaining bits represent a pointer to the > head page. > > For cases when size of struct page is power-of-2, change the encoding of > compound_info to store a mask that can be applied to the virtual address > of the tail page in order to access the head page. It is possible > because struct page of the head page is naturally aligned with regards > to order of the page. > > The significant impact of this modification is that all tail pages of > the same order will now have identical 'compound_info', regardless of > the compound page they are associated with. This paves the way for > eliminating fake heads. > > The HugeTLB Vmemmap Optimization (HVO) creates fake heads and it is only > applied when the sizeof(struct page) is power-of-2. Having identical > tail pages allows the same page to be mapped into the vmemmap of all > pages, maintaining memory savings without fake heads. > > If sizeof(struct page) is not power-of-2, there is no functional > changes. > > Limit mask usage to HugeTLB vmemmap optimization (HVO) where it makes > a difference. The approach with mask would work in the wider set of > conditions, but it requires validating that struct pages are naturally > aligned for all orders up to the MAX_FOLIO_ORDER, which can be tricky. > > Signed-off-by: Kiryl Shutsemau > Reviewed-by: Muchun Song > Reviewed-by: Zi Yan > --- Acked-by: Usama Arif Small nit below: > include/linux/page-flags.h | 81 ++++++++++++++++++++++++++++++++++---- > mm/slab.h | 16 ++++++-- > mm/util.c | 16 ++++++-- > 3 files changed, 97 insertions(+), 16 deletions(-) > > diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h > index d14a17ffb55b..8f2c7fbc739b 100644 > --- a/include/linux/page-flags.h > +++ b/include/linux/page-flags.h > @@ -198,6 +198,29 @@ enum pageflags { > > #ifndef __GENERATING_BOUNDS_H > > +/* > + * For tail pages, if the size of struct page is power-of-2 ->compound_info > + * encodes the mask that converts the address of the tail page address to > + * the head page address. > + * > + * Otherwise, ->compound_info has direct pointer to head pages. > + */ > +static __always_inline bool compound_info_has_mask(void) > +{ > + /* > + * Limit mask usage to HugeTLB vmemmap optimization (HVO) where it > + * makes a difference. > + * > + * The approach with mask would work in the wider set of conditions, > + * but it requires validating that struct pages are naturally aligned > + * for all orders up to the MAX_FOLIO_ORDER, which can be tricky. > + */ > + if (!IS_ENABLED(CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP)) > + return false; > + > + return is_power_of_2(sizeof(struct page)); > +} > + > #ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP > DECLARE_STATIC_KEY_FALSE(hugetlb_optimize_vmemmap_key); > > @@ -210,6 +233,10 @@ static __always_inline const struct page *page_fixed_fake_head(const struct page > if (!static_branch_unlikely(&hugetlb_optimize_vmemmap_key)) > return page; > > + /* Fake heads only exists if compound_info_has_mask() is true */ > + if (!compound_info_has_mask()) > + return page; > + > /* > * Only addresses aligned with PAGE_SIZE of struct page may be fake head > * struct page. The alignment check aims to avoid access the fields ( > @@ -223,10 +250,14 @@ static __always_inline const struct page *page_fixed_fake_head(const struct page > * because the @page is a compound page composed with at least > * two contiguous pages. > */ > - unsigned long head = READ_ONCE(page[1].compound_info); > + unsigned long info = READ_ONCE(page[1].compound_info); > > - if (likely(head & 1)) > - return (const struct page *)(head - 1); > + /* See set_compound_head() */ > + if (likely(info & 1)) { > + unsigned long p = (unsigned long)page; > + > + return (const struct page *)(p & info); > + } > } > return page; > } > @@ -281,11 +312,26 @@ static __always_inline int page_is_fake_head(const struct page *page) > > static __always_inline unsigned long _compound_head(const struct page *page) > { > - unsigned long head = READ_ONCE(page->compound_info); > + unsigned long info = READ_ONCE(page->compound_info); > > - if (unlikely(head & 1)) > - return head - 1; > - return (unsigned long)page_fixed_fake_head(page); > + /* Bit 0 encodes PageTail() */ > + if (!(info & 1)) > + return (unsigned long)page_fixed_fake_head(page); > + > + /* > + * If compound_info_has_mask() is false, the rest of compound_info is > + * the pointer to the head page. > + */ > + if (!compound_info_has_mask()) > + return info - 1; > + > + /* > + * If compoun_info_has_mask() is true the rest of the info encodes s/compoun_info_has_mask/compound_info_has_mask/