From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 663E8C4167B for ; Fri, 1 Dec 2023 18:16:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1379169AbjLASQI (ORCPT ); Fri, 1 Dec 2023 13:16:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48572 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229454AbjLASQD (ORCPT ); Fri, 1 Dec 2023 13:16:03 -0500 Received: from smtp-fw-2101.amazon.com (smtp-fw-2101.amazon.com [72.21.196.25]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02B75F9; Fri, 1 Dec 2023 10:16:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1701454571; x=1732990571; h=mime-version:content-transfer-encoding:date:message-id: cc:from:to:references:in-reply-to:subject; bh=hgs3mkSW/WgX8t1/olpmhoODPmslM5+oH0THeK3eQWI=; b=Ve1IKSeGrYKw/Ba3YeqbRF91K7neoyETvtJoRUFj91T8Ck1RB+qSDNtO VU+mLSGTWt+fCQxBjWSCBEar2vOM4HLYM0lI08upasd2RpOqrHsic3pwP BU9Bqh2kHF65F89zUslqhLDp4GVatK/lPKY8Fr0FQpbFISNCKc85LwAJl I=; X-IronPort-AV: E=Sophos;i="6.04,242,1695686400"; d="scan'208";a="366083187" Subject: Re: [RFC 05/33] KVM: x86: hyper-v: Introduce VTL call/return prologues in hypercall page Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-iad-1e-m6i4x-0aba4706.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-2101.iad2.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Dec 2023 18:16:08 +0000 Received: from smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev (iad7-ws-svc-p70-lb3-vlan2.iad.amazon.com [10.32.235.34]) by email-inbound-relay-iad-1e-m6i4x-0aba4706.us-east-1.amazon.com (Postfix) with ESMTPS id 9851BA6115; Fri, 1 Dec 2023 18:16:04 +0000 (UTC) Received: from EX19MTAEUA002.ant.amazon.com [10.0.43.254:41546] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.14.81:2525] with esmtp (Farcaster) id 7a9e3de9-5dbc-4eed-a35e-b26a36385304; Fri, 1 Dec 2023 18:16:03 +0000 (UTC) X-Farcaster-Flow-ID: 7a9e3de9-5dbc-4eed-a35e-b26a36385304 Received: from EX19D004EUC001.ant.amazon.com (10.252.51.190) by EX19MTAEUA002.ant.amazon.com (10.252.50.126) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 1 Dec 2023 18:16:03 +0000 Received: from localhost (10.13.235.138) by EX19D004EUC001.ant.amazon.com (10.252.51.190) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 1 Dec 2023 18:15:59 +0000 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" Date: Fri, 1 Dec 2023 18:15:55 +0000 Message-ID: CC: Maxim Levitsky , , , , , , , , , , , , , , From: Nicolas Saenz Julienne To: Sean Christopherson X-Mailer: aerc 0.15.2-182-g389d89a9362e-dirty References: <20231108111806.92604-1-nsaenz@amazon.com> <20231108111806.92604-6-nsaenz@amazon.com> In-Reply-To: X-Originating-IP: [10.13.235.138] X-ClientProxiedBy: EX19D033UWC002.ant.amazon.com (10.13.139.196) To EX19D004EUC001.ant.amazon.com (10.252.51.190) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri Dec 1, 2023 at 5:47 PM UTC, Sean Christopherson wrote: > CAUTION: This email originated from outside of the organization. Do not c= lick links or open attachments unless you can confirm the sender and know t= he content is safe. > > > > On Fri, Dec 01, 2023, Nicolas Saenz Julienne wrote: > > On Fri Dec 1, 2023 at 4:32 PM UTC, Sean Christopherson wrote: > > > On Fri, Dec 01, 2023, Nicolas Saenz Julienne wrote: > > > > > To support this I think that we can add a userspace msr filter on= the HV_X64_MSR_HYPERCALL, > > > > > although I am not 100% sure if a userspace msr filter overrides t= he in-kernel msr handling. > > > > > > > > I thought about it at the time. It's not that simple though, we sho= uld > > > > still let KVM set the hypercall bytecode, and other quirks like the= Xen > > > > one. > > > > > > Yeah, that Xen quirk is quite the killer. > > > > > > Can you provide pseudo-assembly for what the final page is supposed t= o look like? > > > I'm struggling mightily to understand what this is actually trying to= do. > > > > I'll make it as simple as possible (diregard 32bit support and that xen > > exists): > > > > vmcall <- Offset 0, regular Hyper-V hypercalls enter here > > ret > > mov rax,rcx <- VTL call hypercall enters here > > I'm missing who/what defines "here" though. What generates the CALL that= points > at this exact offset? If the exact offset is dictated in the TLFS, then = aren't > we screwed with the whole Xen quirk, which inserts 5 bytes before that fi= rst VMCALL? Yes, sorry, I should've included some more context. Here's a rundown (from memory) of how the first VTL call happens: - CPU0 start running at VTL0. - Hyper-V enables VTL1 on the partition. - Hyper-V enabled VTL1 on CPU0, but doesn't yet switch to it. It passes the initial VTL1 CPU state alongside the enablement hypercall arguments. - Hyper-V sets the Hypercall page overlay address through HV_X64_MSR_HYPERCALL. KVM fills it. - Hyper-V gets the VTL-call and VTL-return offset into the hypercall page using the VP Register HvRegisterVsmCodePageOffsets (VP register handling is in user-space). - Hyper-V performs the first VTL-call, and has all it needs to move between VTL0/1. Nicolas