From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sg-1-103.ptr.blmpb.com (sg-1-103.ptr.blmpb.com [118.26.132.103]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0E3D3904E7 for ; Tue, 13 Jan 2026 12:42:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=118.26.132.103 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768308144; cv=none; b=IzmVPNPJ8NARV9CCWumTZReX5aDMWTrnEKaeXbFIIFL6qRfZzDNRHavyY8ainvbbcP9pQCc6sGDFuTNz4WtUdO0OThJwOx0c++YRQ/DWMDAX6ftOuAyz1JaFMOSodXtIPhUBhB6s1eAUjgBOXYbD3fgB++y99qa5Kk9NKKvS7Zc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1768308144; c=relaxed/simple; bh=Be/hNou2fPY2m8LmSoR1u6xGyqvUvQIWUm9xPjx/fNs=; h=Cc:From:Message-Id:To:Subject:In-Reply-To:Content-Type:Date: Mime-Version:References; b=uqRjWBpkWClVnOkaxdnpeRpP1egZQwmBdDM1Oo1B+BCjsG4EbgzZgs0LqRrjWty7AQjbFUgzn5bOOmh4xXuRNLgKljkgCp18Nuu15Quou803JFwVa7vzOKpbxPQEyLH2dGiVSzasMy4vjkgNOrF5cEvJp+1uRXO74p+91RFW5x4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com; spf=pass smtp.mailfrom=bytedance.com; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b=eF4Sw4pw; arc=none smtp.client-ip=118.26.132.103 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=bytedance.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bytedance.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bytedance.com header.i=@bytedance.com header.b="eF4Sw4pw" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; s=2212171451; d=bytedance.com; t=1768308124; h=from:subject: mime-version:from:date:message-id:subject:to:cc:reply-to:content-type: mime-version:in-reply-to:message-id; bh=fU1qogXZNH/lRutT1CKQyIeRvV+WY6BgoX9FINuGm/E=; b=eF4Sw4pwXFoBvDZYE6u698oNdgW+NHlQnZxlEQ5BAkv4HGAN6KsjGI4j4lvQWy6BrEA81D wh/OkgQzod7ppcXLS2h4v8vwG03WYziEXNZyArny08Oi9TwCn7h0u8u2atCEGB2HopNRVG 0ifHRs/TDXEFHElSOZROBSttWbaSYS2cpoupdlRQu0cOHWbxfjaNSM48mdbDSARIEu9bz/ YBN3KdaQ2/m5PM0/qXWPtMNoIducszgGo+k8TRymSfqgm7PBt1s7R0QC08rIo0BhtwKRbd KK5euRDVmvJEGJ3FM2MvUWt7a0I3yifEplqk2ru4rWEonTgaud6Q3iwANfaDjA== Cc: , , , , , , , , , , , From: "Li Zhe" X-Mailer: git-send-email 2.45.2 Message-Id: <20260113124147.48460-1-lizhe.67@bytedance.com> X-Original-From: Li Zhe To: Subject: Re: [PATCH v2 0/8] Introduce a huge-page pre-zeroing mechanism In-Reply-To: <7963534f-cce8-4330-8a67-3f31bd6b2166@kernel.org> Content-Type: text/plain; charset=UTF-8 Date: Tue, 13 Jan 2026 20:41:47 +0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 Content-Transfer-Encoding: 7bit References: <7963534f-cce8-4330-8a67-3f31bd6b2166@kernel.org> X-Lms-Return-Path: On Tue, 13 Jan 2026 11:15:29 +0100, david@kernel.org wrote: > On 1/13/26 07:37, Li Zhe wrote: > > On Mon, 12 Jan 2026 20:52:12 +0100, david@kernel.org wrote: > > > >>> As for concern (4), I believe it is orthogonal to this patchset, and > >>> the cover letter already contains a performance comparison that > >>> demonstrates the additional benefit. > >>> > >>>> I did see some comments in [1] about QEMU supporting user-mode > >>>> parallel zero-page operations; I'm just not sure what the current > >>>> state of that support looks like, or what the corresponding benchmark > >>>> numbers are. > >>> > >>> As noted above, QEMU already employs a parallel page-touch mechanism, > >>> yet the elapsed time remains noticeable. I am not deeply familiar with > >>> QEMU; please correct me if I am mistaken. > >> > >> I implemented some part of the parallel preallocation support in QEMU. > >> > >> With QEMU, you can specify the number of threads and even specify the > >> NUMA-placement of these threads. So you can pretty much fine-tune that > >> for an environment. > >> > >> You still pre-zero all hugetlb pages at VM startup time, just in > >> parallel though. So you pay some price at APP startup time. > > > > Hi David, > > > > Thank you for the comprehensive explanation. > > > > You are absolutely correct: QEMU's parallel preallocation is performed > > only during VM start-up. We submitted this patch series mainly > > because we observed that, even with the existing parallel mechanism, > > launching large-size VMs still incurs prohibitive delays. (Bringing up > > a 2 TB VM still requires more than 40 seconds for zeroing) > > > >> If you know that you will run such a VM (or something else) later, you > >> could pre-zero the memory from user space by using a hugetlb-backed file > >> and supplying that to QEMU as memory backend for the VM. Then, you can > >> start your VM without any pre-zeroing. > >> > >> I guess that approach should work universally. Of course, there are > >> limitations, as you would have to know how much memory an app needs, and > >> have a way to supply that memory in form of a file to that app. > > > > Regarding user-space pre-zeroing, I agree that it is feasible once the > > VM's memory footprint is known. We evaluated this approach internally; > > however, in production environments, it is almost impossible to predict > > the exact amount of memory a VM will require. > > Of course, you could preallocate to the expected maximum and then > truncate the file to the size you need :) The solution you described seems similar to delegating hugepage management to a userspace daemon. I haven't explored this approach before, but it appears quite complex. Beyond ensuring secure memory isolation between VMs, we would also need to handle scenarios where the management daemon or the QEMU process crashes, which implies implementing robust recovery and memory reclamation mechanisms. Do you happen to have any documentation or references regarding userspace hugepage management that I could look into? Compared to the userspace approach, I wonder if implementing hugepage pre-zeroing directly within the kernel would be a simpler and more direct way to accelerate VM creation. Thanks, Zhe