From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47F4EC433DF for ; Sat, 22 Aug 2020 10:32:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1D3962075E for ; Sat, 22 Aug 2020 10:32:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1598092320; bh=ocK8X0Jkb3+E68Wyft6gMd42NDDxBZpEPNLe3L8tiFA=; h=Date:From:To:Cc:Subject:In-Reply-To:References:List-ID:From; b=Wj8/wOsCqNR3uakmYO6pfLi+5IToF0eTHTkdylEXxl6mDiIzCEhPMXr2cJhuTkRmz qvV7C9L2+jbj7Gcmk+FSrCDrpqmWfciNCuHiQnFiQZbNZJEBcDTCt/mIKIJcWw+ns+ 9RKiF6BJQiuxnyUU5z+ZiKxYS19NUbkCl83iSHWI= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727895AbgHVKb7 (ORCPT ); Sat, 22 Aug 2020 06:31:59 -0400 Received: from mail.kernel.org ([198.145.29.99]:41548 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726728AbgHVKb6 (ORCPT ); Sat, 22 Aug 2020 06:31:58 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5377A206BE; Sat, 22 Aug 2020 10:31:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1598092317; bh=ocK8X0Jkb3+E68Wyft6gMd42NDDxBZpEPNLe3L8tiFA=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=lU3SZupPsSEbWsG7BWE6VTh2YeVqi1JjfQ/B7k1YaBj4wCWceoVYs8S7/7FqLnRCe qQTBV2o0mNpFLA69hKO29VM9/VBk/bBM8e9t9c2PqeTatcLsVPuwMIUdUEOPq1Iudh 8SZnFRe+DyqO6vfRuPulTfb9bBOMn0WYzDIg3vOo= Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1k9QoV-0058m9-8B; Sat, 22 Aug 2020 11:31:55 +0100 Date: Sat, 22 Aug 2020 11:31:54 +0100 Message-ID: <87h7svm0o5.wl-maz@kernel.org> From: Marc Zyngier To: Steven Price Cc: Keqian Zhu , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Catalin Marinas , Will Deacon , James Morse , Suzuki K Poulose , wanghaibin.wang@huawei.com Subject: Re: [RFC PATCH 0/5] KVM: arm64: Add pvtime LPT support In-Reply-To: References: <20200817084110.2672-1-zhukeqian1@huawei.com> <8308f52e4c906cad710575724f9e3855@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL/10.8 EasyPG/1.0.0 Emacs/26.3 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: steven.price@arm.com, zhukeqian1@huawei.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, catalin.marinas@arm.com, will@kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, wanghaibin.wang@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Steven, On Wed, 19 Aug 2020 09:54:40 +0100, Steven Price wrote: >=20 > On 18/08/2020 15:41, Marc Zyngier wrote: > > On 2020-08-17 09:41, Keqian Zhu wrote: > >> Hi all, > >>=20 > >> This patch series picks up the LPT pvtime feature originally developed > >> by Steven Price: https://patchwork.kernel.org/cover/10726499/ > >>=20 > >> Backgroud: > >>=20 > >> There is demand for cross-platform migration, which means we have to > >> solve different CPU features and arch counter frequency between hosts. > >> This patch series can solve the latter problem. > >>=20 > >> About LPT: > >>=20 > >> This implements support for Live Physical Time (LPT) which provides the > >> guest with a method to derive a stable counter of time during which the > >> guest is executing even when the guest is being migrated between hosts > >> with different physical counter frequencies. > >>=20 > >> Changes on Steven Price's work: > >> 1. LPT structure: use symmatical semantics of scale multiplier, and use > >> =C2=A0=C2=A0 fraction bits instead of "shift" to make everything clear. > >> 2. Structure allocation: host kernel does not allocates the LPT > >> structure, > >> =C2=A0=C2=A0 instead it is allocated by userspace through VM attribute= s. The > >> save/restore > >> =C2=A0=C2=A0 functionality can be removed. > >> 3. Since LPT structure just need update once for each guest run, > >> add a flag to > >> =C2=A0=C2=A0 indicate the update status. This has two benifits: 1) avo= id > >> multiple update > >> =C2=A0=C2=A0 by each vCPUs. 2) If the update flag is not set, then ret= urn NOT > >> SUPPORT for > >> =C2=A0=C2=A0 coressponding guest HVC call. > >> 4. Add VM device attributes interface for userspace configuration. > >> 5. Add a base LPT read/write layer to reduce code. > >> 6. Support ptimer scaling. > >> 7. Support timer event stream translation. > >>=20 > >> Things need concern: > >> 1. https://developer.arm.com/docs/den0057/a needs update. > >=20 > > LPT was explicitly removed from the spec because it doesn't really > > solve the problem, specially for the firmware: EFI knows > > nothing about this, for example. How is it going to work? > > Also, nobody was ever able to explain how this would work for > > nested virt. > >=20 > > ARMv8.4 and ARMv8.6 have the feature set that is required to solve > > this problem without adding more PV to the kernel. >=20 > Hi Marc, >=20 > These are good points, however we do still have the situation that > CPUs that don't have ARMv8.4/8.6 clearly cannot implement this. I > presume the use-case Keqian is looking at predates the necessary > support in the CPU - Keqian if you can provide more details on the > architecture(s) involved that would be helpful. My take on this is that it is a fictional use case. In my experience, migration happens across *identical* systems, and *any* difference visible to guests will cause things to go wrong. Errata management gets in the way, as usual (name *one* integration that isn't broken one way or another!). Allowing migration across heterogeneous hosts requires a solution to the errata management problem, which everyone (including me) has decided to ignore so far (and I claim that not having a constant timer frequency exposed to guests is an architecture bug). > Nested virt is indeed more of an issue - we did have some ideas around > using SDEI that never made it to the spec. SDEI? Sigh... Why would SDEI be useful for NV and not for !NV? > However I would argue that the most pragmatic approach would be to > not support the combination of nested virt and LPT. Hopefully that > can wait until the counter scaling support is available and not > require PV. And have yet another set of band aids that paper over the fact that we can't get a consistent story on virtualization? No, thank you. NV is (IMHO) much more important than LPT as it has a chance of getting used. LPT is just another tick box, and the fact that ARM is ready to ignore sideline a decent portion of the architecture is a clear sign that it hasn't been thought out. > We are discussing (re-)releasing the spec with the LPT parts added. If > you have fundamental objections then please me know. I do, see above. I'm stating that the use case doesn't really exist given the state of the available HW and the fragmentation of the architecture, and that ignoring the most important innovation in the virtualization architecture since ARMv7 is at best short-sighted. Time scaling is just an instance of the errata management problem, and that is the issue that needs solving. Papering over part of the problem is not helping. M. --=20 Without deviation from the norm, progress is not possible.