From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F34A3239099 for ; Wed, 25 Feb 2026 01:20:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.74 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771982457; cv=none; b=EQBY3y501ZXYtdcuj3yfeCbinJLAePSVBEP32juHsufDfhZx3Uv/mg5qi93YnJdpLwOYA5PxkMaogFvWttvaC9+h64k2Qqca1qqCErnce5JzsAqsqObJuCrcDXGVQpEls42nHjShQIsWnINCsBRUKgl6pU+OroAaDOWzH3abypY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1771982457; c=relaxed/simple; bh=ueMQ7lNZG1mNPcnmXsV6s6mz7PApwRZnhIghYU/HyV0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XnjNouQmJ0NCC97IblVgblR8yPTHM7Zxl2NvanVL9iCuhQtrRq35JugkZS2zkx2EaDpJc8jESN7FYg6TmE0/Wbsu2LKcNtsYb4myBUgF9U5VNpM3GXQ3Lca/Jm4xWqusLhpF2a9sSO2H8l/JykPhZXr9FzmZZV0Mgt5++1PD3E0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=aPq45hIx; arc=none smtp.client-ip=209.85.216.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="aPq45hIx" Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-358e425c261so1340444a91.3 for ; Tue, 24 Feb 2026 17:20:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1771982455; x=1772587255; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=ufrjZKfuPvWxhnZ9cZ27jbL/GsnZnbCcINCZtRWeedI=; b=aPq45hIxscwrfC7Ky7Z8jMtyJKr5SaJQy3n40ZhvIlYO+rclSRHgHqXGmeVmqDJsF6 ykFP/pOZryLxPVnwzwF83YiwN66B1PlKna93alqio6gPdxL5S/mvqHD7DvMi5wLtIDv3 m5Ui5Bq/Sl1BVvbsiibu7fYlEHhsJYv/trbh99GqBf1SW1W2gzfXlCSDtmQkgeEskpZr dEcU/wuWeNLaOpCZfARW+uXtJiSXtuZ7IUdAk/8Ykym+DBTYJJeVabZnD1jpNgfESzM9 WoLurWulEbw8yiHZmxUL1Or7umWANhtRxkk6RAwT7S2uFm08dxEUwAJBSzBTP80RxGRL GMrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1771982455; x=1772587255; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ufrjZKfuPvWxhnZ9cZ27jbL/GsnZnbCcINCZtRWeedI=; b=uKUCp4PBCP+aebmFokfN952nnG7M45FkxoDYhKM3NgTAbrDb9fGGAl39A3i9HbSnH5 U0hvq95JteNCHi0WtG1Gte776iMJdZdaYygowkrMrnOseRPf74SxNqzEZMNTBBC5d453 K78/1mS+fulKyBUXb3tMN5+3vEDPvgOahG6gm1hATSCud0od6qckSb4v9QSJOMgq6DOl osTnDK+KyB70srvwcE3e544+3OgwzWvQCSloDHVrZBAI37dXI4nfHHaFurD55h6O37lL 0y10YizYrj+av+KUO1/6iP5L+/Jzlnud+ORppauKF8O/7gZVj202oO+QQRi0mh5hYdcS 4O9Q== X-Forwarded-Encrypted: i=1; AJvYcCVaxlQOigYrE6yUnc4PzHVVIDVqLJB/Y6UEmYd24G/SEoEG2qM26ne+gZQSmdpnnds4YMqTCPXd5ed9DxI=@vger.kernel.org X-Gm-Message-State: AOJu0Yxj4h+y9oWPjctvXinCwAZ2PN2MFPytqf+pf6N/C6qvi5A7BvpF MD8S1EtBnwpiiK8AIYMXL8fPqlbAcbXdZYTMeqzG0r7nb0urRk/yQT1RZ/fwBOhIjvEpbfcU/QL up7ffxA== X-Received: from pjyr11.prod.google.com ([2002:a17:90a:e18b:b0:355:76b7:bf7]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90a:d887:b0:359:120f:d3aa with SMTP id 98e67ed59e1d1-359120fd628mr174902a91.14.1771982455286; Tue, 24 Feb 2026 17:20:55 -0800 (PST) Reply-To: Sean Christopherson Date: Tue, 24 Feb 2026 17:20:37 -0800 In-Reply-To: <20260225012049.920665-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260225012049.920665-1-seanjc@google.com> X-Mailer: git-send-email 2.53.0.414.gf7e9f6c205-goog Message-ID: <20260225012049.920665-3-seanjc@google.com> Subject: [PATCH 02/14] KVM: x86: Open code handling of completed MMIO reads in emulator_read_write() From: Sean Christopherson To: Sean Christopherson , Paolo Bonzini , Kiryl Shutsemau Cc: kvm@vger.kernel.org, x86@kernel.org, linux-coco@lists.linux.dev, linux-kernel@vger.kernel.org, Yashu Zhang , Rick Edgecombe , Binbin Wu , Xiaoyao Li , Tom Lendacky , Michael Roth Content-Type: text/plain; charset="UTF-8" Open code the handling of completed MMIO reads instead of using an ops hook, as burying the logic behind a (likely RETPOLINE'd) indirect call, and with an unintuitive name, makes relatively straightforward code hard to comprehend. Opportunistically add comments to explain the dependencies between the emulator's mem_read cache and the MMIO read completion logic, as it's very easy to overlook the cache's role in getting the read data into the correct destination. No functional change intended. Signed-off-by: Sean Christopherson --- arch/x86/kvm/emulate.c | 13 +++++++++++++ arch/x86/kvm/x86.c | 33 ++++++++++++++++----------------- 2 files changed, 29 insertions(+), 17 deletions(-) diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c index c8e292e9a24d..70850e591350 100644 --- a/arch/x86/kvm/emulate.c +++ b/arch/x86/kvm/emulate.c @@ -1297,12 +1297,25 @@ static int read_emulated(struct x86_emulate_ctxt *ctxt, int rc; struct read_cache *mc = &ctxt->mem_read; + /* + * If the read gets a cache hit, simply copy the value from the cache. + * A "hit" here means that there is unused data in the cache, i.e. when + * re-emulating an instruction to complete a userspace exit, KVM relies + * on "no decode" to ensure the instruction is re-emulated in the same + * sequence, so that multiple reads are fulfilled in the correct order. + */ if (mc->pos < mc->end) goto read_cached; if (KVM_EMULATOR_BUG_ON((mc->end + size) >= sizeof(mc->data), ctxt)) return X86EMUL_UNHANDLEABLE; + /* + * Route all reads to the cache. This allows @dest to be an on-stack + * variable without triggering use-after-free if KVM needs to exit to + * userspace to handle an MMIO read (the MMIO fragment will point at + * the current location in the cache). + */ rc = ctxt->ops->read_emulated(ctxt, addr, mc->data + mc->end, size, &ctxt->exception); if (rc != X86EMUL_CONTINUE) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index ff3a6f86973f..8b1f02cc8196 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8109,8 +8109,6 @@ int emulator_write_phys(struct kvm_vcpu *vcpu, gpa_t gpa, } struct read_write_emulator_ops { - int (*read_write_prepare)(struct kvm_vcpu *vcpu, void *val, - int bytes); int (*read_write_emulate)(struct kvm_vcpu *vcpu, gpa_t gpa, void *val, int bytes); int (*read_write_mmio)(struct kvm_vcpu *vcpu, gpa_t gpa, @@ -8120,18 +8118,6 @@ struct read_write_emulator_ops { bool write; }; -static int read_prepare(struct kvm_vcpu *vcpu, void *val, int bytes) -{ - if (vcpu->mmio_read_completed) { - trace_kvm_mmio(KVM_TRACE_MMIO_READ, bytes, - vcpu->mmio_fragments[0].gpa, val); - vcpu->mmio_read_completed = 0; - return 1; - } - - return 0; -} - static int read_emulate(struct kvm_vcpu *vcpu, gpa_t gpa, void *val, int bytes) { @@ -8167,7 +8153,6 @@ static int write_exit_mmio(struct kvm_vcpu *vcpu, gpa_t gpa, } static const struct read_write_emulator_ops read_emultor = { - .read_write_prepare = read_prepare, .read_write_emulate = read_emulate, .read_write_mmio = vcpu_mmio_read, .read_write_exit_mmio = read_exit_mmio, @@ -8250,9 +8235,23 @@ static int emulator_read_write(struct x86_emulate_ctxt *ctxt, if (WARN_ON_ONCE((bytes > 8u || !ops->write) && object_is_on_stack(val))) return X86EMUL_UNHANDLEABLE; - if (ops->read_write_prepare && - ops->read_write_prepare(vcpu, val, bytes)) + /* + * If the read was already completed via a userspace MMIO exit, there's + * nothing left to do except trace the MMIO read. When completing MMIO + * reads, KVM re-emulates the instruction to propagate the value into + * the correct destination, e.g. into the correct register, but the + * value itself has already been copied to the read cache. + * + * Note! This is *tightly* coupled to read_emulated() satisfying reads + * from the emulator's mem_read cache, so that the MMIO fragment data + * is copied to the correct chunk of the correct operand. + */ + if (!ops->write && vcpu->mmio_read_completed) { + trace_kvm_mmio(KVM_TRACE_MMIO_READ, bytes, + vcpu->mmio_fragments[0].gpa, val); + vcpu->mmio_read_completed = 0; return X86EMUL_CONTINUE; + } vcpu->mmio_nr_fragments = 0; -- 2.53.0.414.gf7e9f6c205-goog