From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-180.mta1.migadu.com (out-180.mta1.migadu.com [95.215.58.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB92321CC5A for ; Fri, 10 Apr 2026 06:06:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.180 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775801180; cv=none; b=WzIU7AqsR28se3qJX7A4ejC9G0IzKlKBGtNhOrnaZIMh2XIb4u7YvbmJmgI2b3TuWCxVeZkz1viOcdJ3WDllSIfsYGnlXeX9t7LmC7oe9W94vEyxVCfBe8urvEgge30/VR4q/o5ARU1m4qCgSsPHKZhPwLB+0opKevZEU1tfAa0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775801180; c=relaxed/simple; bh=LzOTuFWB8qKuZ5wzgKOIO2/PmOeyT4kI652S8QqAipA=; h=Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc: Message-Id:References:To; b=EeAd+H19mOWMNYG3vHdQuEZrecmimsIRa7yBRh0jldsXrJ3MzGYbKEL1sWjgaQjgLe4L1OvmjisOrTlhTXu1He42YHBiIGBi0K3r4Wzgv1xwuXQ3dkoDkW5qrsOLj/JEzWzgJGdx1xb0D8wQ71yi5N+oiiU3pXzRzXKTjsjyW9Y= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=qvcQTxCw; arc=none smtp.client-ip=95.215.58.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="qvcQTxCw" Content-Type: text/plain; charset=utf-8 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1775801175; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cvqbrRNQVl8/GFKr7amk+i6isct8NH7Vse/3vyO7aos=; b=qvcQTxCw49DKU1sE2QQGtNGvvEvnxxtNyuKmTe9H8RW60gY0W7ZBPYolAZKKqKMpWpgM1I J89WangBx876uhnESIdzCqHvArj8Ev3dkdqnHekRlte+Jya7BQ1Vn7wgzeE03VvEfGENwv BMNGoPHa5/k9L0LM9FbRKfp4ogEwJ78= Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3864.500.181\)) Subject: Re: [RFC PATCH] mm/sparse: remove sparse_buffer X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Muchun Song In-Reply-To: Date: Fri, 10 Apr 2026 14:05:01 +0800 Cc: Muchun Song , Andrew Morton , yinghai@kernel.org, Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Suren Baghdasaryan , Michal Hocko , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Message-Id: <45BF3CAC-2E64-4CA7-A7B8-800FC90930D5@linux.dev> References: <20260407083951.2823915-1-songmuchun@bytedance.com> <70EF8E41-31A2-4B43-BABE-7218FD5F7271@linux.dev> To: Mike Rapoport , "David Hildenbrand (Arm)" X-Migadu-Flow: FLOW_OUT > On Apr 10, 2026, at 11:07, Muchun Song wrote: >=20 >=20 >=20 >> On Apr 9, 2026, at 23:10, Mike Rapoport wrote: >>=20 >> Hi, >>=20 >> On Thu, Apr 09, 2026 at 02:29:38PM +0200, David Hildenbrand (Arm) = wrote: >>> On 4/9/26 13:40, Muchun Song wrote: >>>>=20 >>>>=20 >>>>> On Apr 8, 2026, at 21:40, David Hildenbrand (Arm) = wrote: >>>>>=20 >>>>> On 4/7/26 10:39, Muchun Song wrote: >>>>>> The sparse_buffer was originally introduced in commit = 9bdac9142407 >>>>>> ("sparsemem: Put mem map for one node together.") to allocate a >>>>>> contiguous block of memory for all memmaps of a NUMA node. >>>>>>=20 >>>>>> However, the original commit message did not clearly state the = actual >>>>>> benefits or the necessity of keeping all memmap areas strictly >>>>>> contiguous for a given node. >>>>>=20 >>>>> We don't want the memmap to be scattered around, given that it is = one of >>>>> the biggest allocations during boot. >>>>>=20 >>>>> It's related to not turning too many memory blocks/sections >>>>> un-offlinable I think. >>>>>=20 >>>>> I always imagined that memblock would still keep these allocations = close >>>>> to each other. Can you verify if that is indeed true? >>>>=20 >>>> You raised a very interesting point about whether memblock keeps >>>> these allocations close to each other. I've done a thorough test >>>> on a 16GB VM by printing the actual physical allocations. >>=20 >> memblock always allocates in order, so if there are no other memblock >> allocations between the calls to memmap_alloc(), all these = allocations will >> be together and they all will be coalesced to a single region in >> memblock.reserved. >>=20 >>>> I enabled the existing debug logs in arch/x86/mm/init_64.c to >>>> trace the vmemmap_set_pmd allocations. Here is what really happens: >>>>=20 >>>> When using vmemmap_alloc_block without sparse_buffer, the >>>> memblock allocator allocates 2MB chunks. Because memblock >>>> allocates top-down by default, the physical allocations look >>>> like this: >>>>=20 >>>> [ffe6475cc0000000-ffe6475cc01fffff] PMD -> = [ff3cb082bfc00000-ff3cb082bfdfffff] on node 0 >>>> [ffe6475cc0200000-ffe6475cc03fffff] PMD -> = [ff3cb082bfa00000-ff3cb082bfbfffff] on node 0 >>>> [ffe6475cc0400000-ffe6475cc05fffff] PMD -> = [ff3cb082bf800000-ff3cb082bf9fffff] on node 0 >>=20 >> ... >>=20 >>>> Notice that the physical chunks are strictly adjacent to each >>>> other, but in descending order! >>>>=20 >>>> So, they are NOT "scattered around" the whole node randomly. >>>> Instead, they are packed densely back-to-back in a single >>>> contiguous physical range (just mapped top-down in 2MB pieces). >>>>=20 >>>> Because they are packed tightly together within the same >>>> contiguous physical memory range, they will at most consume or >>>> pollute the exact same number of memory blocks as a single >>>> contiguous allocation (like sparse_buffer did). Therefore, this >>>> will NOT turn additional memory blocks/sections into an >>>> "un-offlinable" state. >>>>=20 >>>> It seems we can safely remove the sparse buffer preallocation >>>> mechanism, don't you think? >>>=20 >>> Yes, what I suspected. Is there a performance implication when doing >>> many individual memmap_alloc(), for example, on a larger system with >>> many sections? >>=20 >> memmap_alloc() will be slower than sparse_buffer_alloc(), allocating = from >> memblock is more involved that sparse_buffer_alloc(), but without >> measurements it's hard to tell how much it'll affect overall = sparse_init(). >=20 > I ran a test on a 256GB VM, and the results are as follows: >=20 > With patch: 741,292 ns > Without patch: 199,555 ns >=20 > The performance is approximately 3.7x slower with the patch applied. I also tested 512GB of data, and the results were roughly twice that of 256GB, so for a 1TB machine, the memory allocation time is only a few milliseconds. It seems we don=E2=80=99t need to worry about the 3.7x performance drop. >=20 > Thanks, > Muchun >=20 >>=20 >>> --=20 >>> Cheers, >>>=20 >>> David >>=20 >> --=20 >> Sincerely yours, >> Mike.