From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23420435AAD for ; Mon, 6 Jul 2026 18:17:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=209.85.215.172 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783361864; cv=pass; b=V5JzN82jN/VXmDr6A6feLckaYplRNSNetAjVwZWgpKnmyC00RBjwPdy11AqFKwtKVzKpCB0QjRT82SG8uztsRPC559wnPI0a3fGpnL27t249WKQXtIK2sCKRrsUX9+Xiape1xv4E7lKm7yXiT16U/n/WZCBY2gVxH61FyrLMn8Q= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783361864; c=relaxed/simple; bh=qof4S+F17VBSSA1zaUR6umxTYpxJBd4EYxHTZo5RHIU=; h=From:In-Reply-To:References:MIME-Version:Date:Message-ID:Subject: To:Cc:Content-Type; b=lyUMHq77WO9XW7L334ZxiWNhogpz0FKXl6yZJeUlgILB74RBQy5ZQnpPPBucEijUfrAAik8pATw2SbXB7EeiwDPGdWrUEvyr56YLjmvglTadPR1gJl5q789P9qBCNpCyQyWvnHCexZ1H2jxCI7acI/y0P4W7WmsV5OXIoPMg7gw= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ZJd7ZdwW; arc=pass smtp.client-ip=209.85.215.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ZJd7ZdwW" Received: by mail-pg1-f172.google.com with SMTP id 41be03b00d2f7-c88973b6965so68453a12.1 for ; Mon, 06 Jul 2026 11:17:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1783361862; cv=none; d=google.com; s=arc-20260327; b=TRgvyaSDAQ4L7RiaesCyve+bR0HjgNL0P9yVzDTj/8wctE8MLTtw28lFROUQv29rM0 IaUdpItnoPrsUF1hQT9OkQHhk1wXJPCgv3FZLex/RoXlM6fEXH78V5Ychy4l1sQ07LAE PRP0ksAwnGmJfkcSuvTZ3FM/pZb6RSnFKCtH+cqjkky5BY22ZASckl10NGmzRX7ekn1q CFPa5emnFDKwW9YC5s2T8aSrTQZxzAmayIzKT2J1R4TZlYCgXXxdRkYIWPJRS1d22oG4 3TP+l/xx1NOrdarwslCulW6DkqjTaHw4dX6f5zfoeSaVwCks90f0bh3xbXsCROr4IAO+ Kp0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=cc:to:subject:message-id:date:mime-version:references:in-reply-to :from:dkim-signature; bh=c+wxJ1Yd1HSbDSoQd0ZgV/xzQAnAHKSd8sgBz4R4ZFc=; fh=g8VjL9dFUhCNYe30ZixXPoOQzPULaASc6DV5Jv/BVEg=; b=mRTCUdFB1IFgu76cFjWr8G/dWWC1lvn8tvPNsaDqQBeCZQe7YNCdRHr0HfQHYQkB6k 9eiaCM/hKaF500tFrX0/TPMyJEk2e9/hoQcbmPeG3i8qsVk7sqxWffk+mB6EWwca2dJE USX9h/3Jdxr5wGX/dgLmkeq6foyXSpeVVOwfrImXvdgRI/0FmZjY+DhSO63BZIhTJmp2 SC09QJwGar7zOxrt8V/260TNeGv1GwiXa+o1NIepcwNLZ8rfxgM8A/2WMlHiibNPYYmT Yd2AUKiZ5vtvL0cKSGBi56aGk87WgiWq7MUwPybOxJusg0u1kEJvD/ZkCqTZX38i8Tg3 trTQ==; darn=vger.kernel.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1783361862; x=1783966662; darn=vger.kernel.org; h=cc:to:subject:message-id:date:mime-version:references:in-reply-to :from:from:to:cc:subject:date:message-id:reply-to; bh=c+wxJ1Yd1HSbDSoQd0ZgV/xzQAnAHKSd8sgBz4R4ZFc=; b=ZJd7ZdwWlAbQKSLwjOSWuHPVGVhCVdGI7XLRLLc7OEy6svimB49ddFMt9Yb57Nkn+3 EOS+mec+Nu9yHfb6xFf+bchy0TMR4xL2PNpwj78qsYBye7WfAQl11eitoKlXKEHIZSjm ZVnA0g23QvXquT1+wVXtMfVwWq5OgLLRSVGuWzV5hypKAvFU8pBMAgbWyNCnOR9C0FoZ fGhryIyT2oiUl8Y4xzhL3QzR8r2O4/+Rj7MfV+YnOWQ95RbSWSaCfC/XOm/HauEdxe1h Gzv4XBYIPNYUOqqU77rMMwZwpCMk/tDkvWfmOM6hjq1cEwqaepW3MPjSLymIt3UvGHeH SHdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783361862; x=1783966662; h=cc:to:subject:message-id:date:mime-version:references:in-reply-to :from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=c+wxJ1Yd1HSbDSoQd0ZgV/xzQAnAHKSd8sgBz4R4ZFc=; b=e8bPTdOhvfMcsL7qbESlvNNShGd3mRutUH2e/8qN6WCJon3Sh7LIp2SRbc5HANvFBl Ngxm608gniJgEmeMBSfJK+bBdcHrAPliy2HHt+/vriuFTT4d9zGbiRviFBwmjv4JEkC1 ohzJPWEnkjwsY51tDW0GL6hX+x5IIEH+djovV5M0Ra462aHvybIlcP98vK/jH1jmcC52 qK05BUizKSrT2AAiU9LuxrGrJannMfvP7lT2xiJKWdkfkJE7r+Kx8XLlR2t9ioDoA6PR Yt4P/i33kyraYDxj/97sykhnNElhUbHEqphoNFblti5SxhL2iC7hhv+0dFEuJVCZ6zsG cSIw== X-Forwarded-Encrypted: i=1; AHgh+RqU5d4Htv4kuMCT/TK4dxVN8RoPbNAcAabYPgK4jXUERp5JnBFYvaLTXx/89oPCB3zdws7EyIVKlOkeXac=@vger.kernel.org X-Gm-Message-State: AOJu0YykIjA+xJBrxf7sqjM5TeknSbS8NO85kVTciEeyKApmHiigAUIE 7I/p7fcegtcSmtc9LQyMjRbtfWhedNwzRR1oQO1cR9Eveeja/1quj11GRkKTeyXnJ4hkatI3XSu Cv7CxcVdv9HnaBcw9v+SX4q9RHXiueWn4lH0IK9VP X-Gm-Gg: AfdE7clCZfVzC8Ma3RiNoYhTtKTz00L2tu2WbB4tBJULqsSwOIYx8UnkbEaqxyoHK2p YGXS0DZNUMrKo1Qp5gd9IoHJoYuXeKJn9JkbD2E3y1uJux3sQB1T4mhB77QPX08AIY0bLOrKGo8 saXyntz0H8Ggws8c61gFBBLHobUF2MpVDSTPQdDuLI6BA/LsPBNhL2sOveA1P8p3AcsF4scHzon YGXYfz+cjoOGJkCUG901vYc1v6HGBlz+06BsY3S3j8WLFuaGmApmqZ78ooO5q+s3iDommfQm3ka uU32RoouoPWWysiHVXlA5u2+PWsbMRXWxTJH07Qf0j2JTcLpu/R4vB2q8oUo6HmzaYj36w== X-Received: by 2002:a05:6a21:6906:b0:3bf:79b5:ebdb with SMTP id adf61e73a8af0-3c01caab306mr16907533637.21.1783361861894; Mon, 06 Jul 2026 11:17:41 -0700 (PDT) Received: from 176938342045 named unknown by gmailapi.google.com with HTTPREST; Mon, 6 Jul 2026 11:17:40 -0700 Received: from 176938342045 named unknown by gmailapi.google.com with HTTPREST; Mon, 6 Jul 2026 11:17:40 -0700 From: Ackerley Tng In-Reply-To: <114e2488-97ed-4740-a8e8-1edd991f26c5@arm.com> References: <20260618-gmem-inplace-conversion-v8-0-9d2959357853@google.com> <20260618-gmem-inplace-conversion-v8-13-9d2959357853@google.com> <114e2488-97ed-4740-a8e8-1edd991f26c5@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Mon, 6 Jul 2026 11:17:40 -0700 X-Gm-Features: AVVi8CeLpju1khujHSRbueqIxtdXRIsffiKvTI6S-exNvUblyDgAMTgWeujtT7w Message-ID: Subject: Re: [PATCH v8 13/46] KVM: guest_memfd: Add base support for KVM_SET_MEMORY_ATTRIBUTES2 To: Suzuki K Poulose , aik@amd.com, andrew.jones@linux.dev, binbin.wu@linux.intel.com, brauner@kernel.org, chao.p.peng@linux.intel.com, david@kernel.org, jmattson@google.com, jthoughton@google.com, michael.roth@amd.com, oupton@kernel.org, pankaj.gupta@amd.com, qperret@google.com, rick.p.edgecombe@intel.com, rientjes@google.com, shivankg@amd.com, steven.price@arm.com, tabba@google.com, willy@infradead.org, wyihan@google.com, yan.y.zhao@intel.com, forkloop@google.com, pratyush@kernel.org, aneesh.kumar@kernel.org, liam@infradead.org, Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Jonathan Corbet , Shuah Khan , Shuah Khan , Vishal Annapurve , Andrew Morton , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , Youngjun Park , Qi Zheng , Shakeel Butt , Kiryl Shutsemau , Baoquan He , Jason Gunthorpe , Vlastimil Babka Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-coco@lists.linux.dev Content-Type: text/plain; charset="UTF-8" Suzuki K Poulose writes: > > [...snip...] > >> +static int __kvm_gmem_set_attributes(struct inode *inode, pgoff_t start, >> + size_t nr_pages, uint64_t attrs) >> +{ >> + struct address_space *mapping = inode->i_mapping; >> + struct gmem_inode *gi = GMEM_I(inode); >> + pgoff_t end = start + nr_pages; >> + struct maple_tree *mt; >> + struct ma_state mas; >> + int r; >> + >> + mt = &gi->attributes; >> + >> + filemap_invalidate_lock(mapping); >> + >> + mas_init(&mas, mt, start); >> + r = kvm_gmem_mas_preallocate(&mas, attrs, start, nr_pages); >> + if (r) >> + goto out; >> + >> + /* >> + * From this point on guest_memfd has performed necessary >> + * checks and can proceed to do guest-breaking changes. >> + */ >> + >> + kvm_gmem_invalidate_start(inode, start, end); > > I added support for Arm CCA KVM patches with the inplace conversion and > I am hitting the following issue. > > 1. I am supporting INIT_SHARED + MMAP flags. > 2. VMM creates the Gmem_fd with both the flags above. > 3. Uses the shared gmem-mmap to load the initial payloads (kernel, dtb). > 4. At the VM finalization time, Populate the loaded regions one by one > by > a) copying the images to a temparory buffer - Since CCA can't really > load the contents in-place. Sounds good :). I see that you blocked this in the kernel by returning -EOPNOTSUPP if (!src_page) [0]. > b) Set the "region" to Private in the gmem_fd (via > SET_MEMORY_ATTRIBUTES2) > c) Invoke CCA backend to populate the private memory via > ioctl(KVM_ARM_RMI_POPULATE,..) [0] > This flow sounds right. > [0] > https://lore.kernel.org/all/20260513131757.116630-27-steven.price@arm.com/ > > > 5. Additionally, VMM can mark the entire RAM to be private before the VM > starts running, again via SET_MEMORY_ATTRIBUTES2. On CCA, this > action is measured and doesn't require the Host to "commit" memory to > the VM. > Instead the host can lazily donate memory on a fault. > For both TDX and SNP, the host can also lazily donate memory, guest_memfd supports this. > But step (5) triggers the invalidation of both private and shared > mappings of the gmem area, from the kvm_gmem_invalidate_start() > above. > > This is because, the entire DRAM now has, some portions PRIVATE (the > loaded regions) and the rest are SHARED (from the Gmem_fd creation). > Thus, kvm_gmem_get_invalidate_filter(Dram_start, Dram_end) causes the > invalidation of both "PRIVATE" and "SHARED" regions, which results > in the destruction of the already loaded data and things go south. > This destruction will happen for TDX as well. I think we managed to get around this because we didn't apply conversion on the already-private ranges. IIUC on SNP, zapping pages in the stage 2 page tables doesn't destroy the data, so that's probably why it has been fine for SNP. > When we know that the kvm_gmem_invalidate_xx is triggered by a > conversion, we don't need to invalidate the existing pages that > are in the requested state. i.e., the following patch on top of > this series does the trick for me : > > > diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c > index a97fcac34a0e..62e0427a49f4 100644 > --- a/virt/kvm/guest_memfd.c > +++ b/virt/kvm/guest_memfd.c > @@ -250,16 +250,23 @@ static void __kvm_gmem_invalidate_start(struct > gmem_file *f, pgoff_t start, > KVM_MMU_UNLOCK(kvm); > } > > +static void kvm_gmem_invalidate_start_filter(struct inode *inode, > pgoff_t start, > + pgoff_t end, > + enum kvm_gfn_range_filter > attr_filter) > +{ > + struct gmem_file *f; > + > + kvm_gmem_for_each_file(f, inode) > + __kvm_gmem_invalidate_start(f, start, end, attr_filter); > +} > + > static void kvm_gmem_invalidate_start(struct inode *inode, pgoff_t start, > pgoff_t end) > { > enum kvm_gfn_range_filter attr_filter; > - struct gmem_file *f; > - > attr_filter = kvm_gmem_get_invalidate_filter(inode, start, end); > > - kvm_gmem_for_each_file(f, inode) > - __kvm_gmem_invalidate_start(f, start, end, attr_filter); > + kvm_gmem_invalidate_start_filter(inode, start, end, attr_filter); > } > > static void __kvm_gmem_invalidate_end(struct gmem_file *f, pgoff_t start, > @@ -724,9 +731,14 @@ static int __kvm_gmem_set_attributes(struct inode > *inode, pgoff_t start, > /* > * From this point on guest_memfd has performed necessary > * checks and can proceed to do guest-breaking changes. > + * Also, we don't have to invalidate the regions that > + * may already be in the requested state. Hence, we could > + * explicitly filter the invalidations to the opposite > + * state. > */ > > - kvm_gmem_invalidate_start(inode, start, end); > + kvm_gmem_invalidate_start_filter(inode, start, end, > + to_private ? KVM_FILTER_SHARED : > KVM_FILTER_PRIVATE); > I think this makes sense. Thanks for catching this. > if (!to_private) > kvm_gmem_invalidate(inode, start, end); > > > Thoughts ? > > Suzuki > > >> >> [...snip...] >>