From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=209.85.161.71 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1793069808; b=vO0FJDgaOcLGNaOEwOFYytG5ljuCpydwdDO5LCR6TRVmY/JoYPswGgJTBmsPaubzW0wj OHoDiQp5qWGxCKA+8wbBwqhFv95N3tvM/kNYaLyIuN0Ah7KgDIX0sjad+T1jNXGvMjdGx zuFIsr7LBatCR9+xSlMvLjM7XrBGibrLYMT12tDqkkKUnxbJm1ss4zOE7VYUQfLxbYiAi fXA2l8LvxWq+1YNOtuFAlLq6gJvIUtirDFjUmYEVde/dMFfk6Vcw77P/LhnTOVvoB8j2W i/wR7HQDUZeNjcYLXMRJmzcPrK+yWVkj0baeeU68faR3cbqcBASEgD1TXj/hRcHWO0Q== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1793069808; h=DMARC-Filter:Received:DKIM-Signature:X-Google-DKIM-Signature: X-Forwarded-Encrypted:X-Gm-Message-State:X-Received:Date:In-Reply-To: Mime-Version:References:X-Mailer:Message-ID:Subject:From:To:Cc: Content-Type; bh=U1tHH5Mqf99jycLltqjXTRn5xKYAyrIr2aHOwRziEw8=; b=eMa8bnTDQDB3IKQZ2KAKlqYNjoaei72a4CtqXjC/DJefjxDhOHfxYGBUs2hNV2w68i8C 2r8Qgv9p9M/uY3VguVrWvqWZ62Q4l1Qsj4YfFqFdJhJSwB8GKYqk45mP9widLEUHcyJac T3OE5IhS1KXmdZoyVo6ecAmKHHqu5Xe1aw49TYed17zYztqyYc86B6LvuUdzxYj733S+7 HArY2cprVAexUshGeXKWlNFsCyyt5Ox6EUZSWXyGzU2W4zqX42DeYBiFWd70ibdb1GsL5 +p0Y9/hAdrafH9m8R2eClXFctJ56JYVrywqrvOnApGF2H/IZLJDbYuiSBPc/Okkw4zg== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass header.from=google.com; spf=pass smtp.mailfrom=flex--kevinloughlin.bounces.google.com; dkim=pass header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20251104 header.b=eRnTUYtx; arc=none smtp.remote-ip=209.85.161.71 Received: from mail-oo1-f71.google.com (mail-oo1-f71.google.com [209.85.161.71]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19182606A31F for ; Tue, 27 Oct 2026 02:56:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.2 smtp.subspace.kernel.org 19182606A31F Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kevinloughlin.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key, unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20251104 header.b=eRnTUYtx Received: by mail-oo1-f71.google.com with SMTP id 006d021491bc7-702f912301aso3671304eaf.0 for ; Mon, 26 Oct 2026 19:56:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1793069807; x=1793674607; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=U1tHH5Mqf99jycLltqjXTRn5xKYAyrIr2aHOwRziEw8=; b=eRnTUYtxl2WJIZYJwL4u+CpIv3e0gmonfQevqXojOEcHwoYASWEfe19L3x4iMJ3rqh I8oGhMkA++aQemckBDM/JQIrbEpSAN3QnwouM0VC6TnWfW8CjzIJGJvAv+V4tJATSqia +gHiIqXRbmQBZPjiqqlQVwmNZVyKryXUAy9jBK/JnOM+X+DDeIfIfnUP8zYzGcDQVK52 EEqrOqwQe8bLr3aYoyYL4SI/+IbyW6F9nWEON11RuuwOlzmSVoMY9MFD1y+qHy6nJDo9 jnVSPMi8H/ZvWrlP49Rbn138vbNg2y01j7TsRNDlq0INZit+Cd0qu1QvEFrX5+p94jzy FGgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1793069807; x=1793674607; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=U1tHH5Mqf99jycLltqjXTRn5xKYAyrIr2aHOwRziEw8=; b=AtGIYsLvSpwQqxuhgYuhM3MYeqz9/uH4t/l4iygPtJpXqkUlQgD/BLfFqkr2N/KVNw tESHtPcSD0QxEqN3f4EKvk0OlF14s4s59LV+/x4fxACRz79o8PXhw27hDDJnIVTZfBQZ bUuDpswIdXG+kONjHMFvxSTNIh2Av0DaUB9JXDKKcVVkslU07XnY116fNgEAVriKlDQ2 v0q3zHtonmX0F0R/uArZ7Xwrgu6nQpLeZnMaLHYVgykzIT8tvC10XdT8Cgs9gsb1g2AL 92RX8o+Zc5YdJd2qxbhq44e69pqUztOD44uZwiui/VKprzk6hE8hkCEo+G2POHJrOhm2 RCKA== X-Forwarded-Encrypted: i=1; AKwUvByoWmGfvCIrSC1l+IK2uLYQ//F9HNIVUBIfrDN4qUszWLiRHvJdWMkkc5xTa6qZdcPECVhotAmdibjmWrg=@vger.kernel.org X-Gm-Message-State: AFq9FYJTptSGk8q5CCZOn9kgAQxRDHRi0YanFU/+PmPWOpj19R8QJ+oQ 0peJRouxcq2FShPwPmfFVJWmhzT4LvWGbrKZYJIUrtGb6fol3vh+gQ975IMBDhWsJn5EYc27kcd GJnk8fS21nNvZfZP2equQvuZ/CPTtjUPlPg== X-Received: from ilbeu28-n2.prod.google.com ([2002:a05:6e02:2f1c:20b0:516:ba98:55b6]) (user=kevinloughlin job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6870:3929:b0:494:396:1205 with SMTP id 586e51a60fabf-4a83f49fec6mr6444601fac.14.1793069806655; Mon, 26 Oct 2026 19:56:46 -0700 (PDT) Date: Tue, 27 Oct 2026 02:56:07 +0000 In-Reply-To: <20261027025616.1632690-1-kevinloughlin@google.com> X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list Mime-Version: 1.0 References: <20261027025616.1632690-1-kevinloughlin@google.com> X-Mailer: git-send-email 2.56.0.307.g5fe7bd680c-goog Message-ID: <20261027025616.1632690-5-kevinloughlin@google.com> Subject: [PATCH 04/13] x86/coco/sev: fix zeropad #VC on unaccepted memory From: Kevin Loughlin To: "Michael S. Tsirkin" , David Hildenbrand , Jason Wang , "=?UTF-8?q?Eugenio=20P=C3=A9rez?=" , Xuan Zhuo , Ard Biesheuvel , Kiryl Shutsemau , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Tom Lendacky , Ashish Kalra , Michael Roth , Rick Edgecombe , Andrew Morton , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Alexander Duyck Cc: virtualization@lists.linux.dev, linux-coco@lists.linux.dev, linux-efi@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sidharth Telang , Punit Salian , Peter Gonda , Ackerley Tng , Jacky Li , Steve Rutherford , David Skidmore , Kevin Loughlin Content-Type: text/plain; charset="UTF-8" When dynamic memory unacceptance is active on AMD SEV-SNP, an accepted 4KB page may sit immediately adjacent to a 2MB block that has been unaccepted via PVALIDATE(validate=false). Speculative unaligned loads across the page boundary via load_unaligned_zeropad() (such as in dentry path walk) trigger a hardware #VC exception with error code SVM_EXIT_PAGE_NOT_VALIDATED (0x404) or SVM_EXIT_NPF (0x400) on the unvalidated neighbor page. Define SVM_EXIT_PAGE_NOT_VALIDATED (0x404) and invoke fixup_exception() in vc_raw_handle_exception() when an SVM_EXIT_PAGE_NOT_VALIDATED or SVM_EXIT_NPF #VC exception occurs in kernel mode at an EX_TYPE_ZEROPAD instruction before dispatching to vc_handle_exitcode(), matching the existing EX_TYPE_ZEROPAD handling in the Intel TDX #VE handler (ve_raise_fault()). Signed-off-by: Kevin Loughlin --- arch/x86/coco/sev/vc-handle.c | 29 +++++++++++++++++++++++++++++ arch/x86/include/uapi/asm/svm.h | 2 ++ 2 files changed, 31 insertions(+) diff --git a/arch/x86/coco/sev/vc-handle.c b/arch/x86/coco/sev/vc-handle.c index f25d302efceb..1ed00771d87d 100644 --- a/arch/x86/coco/sev/vc-handle.c +++ b/arch/x86/coco/sev/vc-handle.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include @@ -872,6 +873,29 @@ static __always_inline bool vc_from_invalid_context(struct pt_regs *regs) return is_vc2_stack(sp) && !is_vc2_stack(prev_sp); } +static bool vc_try_fixup_zeropad(struct es_em_ctxt *ctxt, struct pt_regs *regs, + unsigned long error_code) +{ + unsigned long addr, fault_addr; + + if ((error_code != SVM_EXIT_PAGE_NOT_VALIDATED && + error_code != SVM_EXIT_NPF) || + user_mode(regs) || + ex_get_fixup_type(regs->ip) != EX_TYPE_ZEROPAD) + return false; + + addr = (unsigned long)insn_get_addr_ref(&ctxt->insn, regs); + if (addr == ~0ul) + return false; + + /* + * load_unaligned_zeropad() faults when an unaligned word load crosses + * into an unaccepted page; round up to the crossed page boundary. + */ + fault_addr = (addr + sizeof(long) - 1) & PAGE_MASK; + return fixup_exception(regs, X86_TRAP_VC, error_code, fault_addr); +} + static bool vc_raw_handle_exception(struct pt_regs *regs, unsigned long error_code) { struct ghcb_state state; @@ -885,6 +909,11 @@ static bool vc_raw_handle_exception(struct pt_regs *regs, unsigned long error_co vc_ghcb_invalidate(ghcb); result = vc_init_em_ctxt(&ctxt, regs, error_code); + if (result == ES_OK && vc_try_fixup_zeropad(&ctxt, regs, error_code)) { + __sev_put_ghcb(&state); + return true; + } + if (result == ES_OK) result = vc_handle_exitcode(&ctxt, ghcb, error_code); diff --git a/arch/x86/include/uapi/asm/svm.h b/arch/x86/include/uapi/asm/svm.h index e80676185092..38b8e0c2f5e2 100644 --- a/arch/x86/include/uapi/asm/svm.h +++ b/arch/x86/include/uapi/asm/svm.h @@ -101,6 +101,7 @@ #define SVM_EXIT_AVIC_INCOMPLETE_IPI 0x401 #define SVM_EXIT_AVIC_UNACCELERATED_ACCESS 0x402 #define SVM_EXIT_VMGEXIT 0x403 +#define SVM_EXIT_PAGE_NOT_VALIDATED 0x404 #define SVM_EXIT_PML_FULL 0x407 /* SEV-ES software-defined VMGEXIT events */ @@ -237,6 +238,7 @@ { SVM_EXIT_AVIC_INCOMPLETE_IPI, "avic_incomplete_ipi" }, \ { SVM_EXIT_AVIC_UNACCELERATED_ACCESS, "avic_unaccelerated_access" }, \ { SVM_EXIT_VMGEXIT, "vmgexit" }, \ + { SVM_EXIT_PAGE_NOT_VALIDATED, "page_not_validated" }, \ { SVM_EXIT_PML_FULL, "pml_full" }, \ { SVM_VMGEXIT_MMIO_READ, "vmgexit_mmio_read" }, \ { SVM_VMGEXIT_MMIO_WRITE, "vmgexit_mmio_write" }, \ -- 2.56.0.307.g5fe7bd680c-goog