From: David Brazdil <dbrazdil@google.com>
To: Marc Zyngier <maz@kernel.org>
Cc: kvmarm@lists.cs.columbia.edu,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org, James Morse <james.morse@arm.com>,
Julien Thierry <julien.thierry.kdev@gmail.com>,
Suzuki K Poulose <suzuki.poulose@arm.com>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>, Dennis Zhou <dennis@kernel.org>,
Tejun Heo <tj@kernel.org>, Christoph Lameter <cl@linux.com>,
Mark Rutland <mark.rutland@arm.com>,
Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
Quentin Perret <qperret@google.com>,
Andrew Scull <ascull@google.com>,
Andrew Walbran <qwandor@google.com>,
kernel-team@android.com
Subject: Re: [PATCH v2 06/24] kvm: arm64: Move hyp-init params to a per-CPU struct
Date: Wed, 25 Nov 2020 10:39:46 +0000 [thread overview]
Message-ID: <20201125103946.2unmclsdfqpr3eyk@google.com> (raw)
In-Reply-To: <87lfes5f54.wl-maz@kernel.org>
On Mon, Nov 23, 2020 at 02:20:07PM +0000, Marc Zyngier wrote:
> On Mon, 16 Nov 2020 20:43:00 +0000,
> David Brazdil <dbrazdil@google.com> wrote:
> >
> > Once we start initializing KVM on newly booted cores before the rest of
> > the kernel, parameters to __do_hyp_init will need to be provided by EL2
> > rather than EL1. At that point it will not be possible to pass its four
> > arguments directly because PSCI_CPU_ON only supports one context
> > argument.
> >
> > Refactor __do_hyp_init to accept its parameters in a struct. This
> > prepares the code for KVM booting cores as well as removes any limits on
> > the number of __do_hyp_init arguments.
> >
> > Signed-off-by: David Brazdil <dbrazdil@google.com>
> > ---
> > arch/arm64/include/asm/kvm_asm.h | 7 +++++++
> > arch/arm64/include/asm/kvm_hyp.h | 4 ++++
> > arch/arm64/kernel/asm-offsets.c | 4 ++++
> > arch/arm64/kvm/arm.c | 26 ++++++++++++++------------
> > arch/arm64/kvm/hyp/nvhe/hyp-init.S | 21 ++++++++++-----------
> > arch/arm64/kvm/hyp/nvhe/hyp-main.c | 2 ++
> > 6 files changed, 41 insertions(+), 23 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> > index 54387ccd1ab2..01904e88cead 100644
> > --- a/arch/arm64/include/asm/kvm_asm.h
> > +++ b/arch/arm64/include/asm/kvm_asm.h
> > @@ -150,6 +150,13 @@ extern void *__vhe_undefined_symbol;
> >
> > #endif
> >
> > +struct kvm_nvhe_init_params {
> > + unsigned long tpidr_el2;
> > + unsigned long vector_hyp_va;
> > + unsigned long stack_hyp_va;
> > + phys_addr_t pgd_pa;
> > +};
> > +
> > /* Translate a kernel address @ptr into its equivalent linear mapping */
> > #define kvm_ksym_ref(ptr) \
> > ({ \
> > diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
> > index 6b664de5ec1f..a3289071f3d8 100644
> > --- a/arch/arm64/include/asm/kvm_hyp.h
> > +++ b/arch/arm64/include/asm/kvm_hyp.h
> > @@ -15,6 +15,10 @@
> > DECLARE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt);
> > DECLARE_PER_CPU(unsigned long, kvm_hyp_vector);
> >
> > +#ifdef __KVM_NVHE_HYPERVISOR__
> > +DECLARE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params);
> > +#endif
>
> I'm not sure we should bother with this #ifdefery. Having the
> declaration present at all times doesn't really hurt, since it is only
> defined in the HYP code. Cutting down on the conditionals would
> certainly help readability.
>
> > +
> > #define read_sysreg_elx(r,nvh,vh) \
> > ({ \
> > u64 reg; \
> > diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> > index 7d32fc959b1a..4435ad8be938 100644
> > --- a/arch/arm64/kernel/asm-offsets.c
> > +++ b/arch/arm64/kernel/asm-offsets.c
> > @@ -110,6 +110,10 @@ int main(void)
> > DEFINE(CPU_APGAKEYLO_EL1, offsetof(struct kvm_cpu_context, sys_regs[APGAKEYLO_EL1]));
> > DEFINE(HOST_CONTEXT_VCPU, offsetof(struct kvm_cpu_context, __hyp_running_vcpu));
> > DEFINE(HOST_DATA_CONTEXT, offsetof(struct kvm_host_data, host_ctxt));
> > + DEFINE(NVHE_INIT_TPIDR_EL2, offsetof(struct kvm_nvhe_init_params, tpidr_el2));
> > + DEFINE(NVHE_INIT_VECTOR_HYP_VA, offsetof(struct kvm_nvhe_init_params, vector_hyp_va));
> > + DEFINE(NVHE_INIT_STACK_HYP_VA, offsetof(struct kvm_nvhe_init_params, stack_hyp_va));
> > + DEFINE(NVHE_INIT_PGD_PA, offsetof(struct kvm_nvhe_init_params, pgd_pa));
> > #endif
> > #ifdef CONFIG_CPU_PM
> > DEFINE(CPU_CTX_SP, offsetof(struct cpu_suspend_ctx, sp));
> > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> > index c0ffb019ca8b..4838556920fb 100644
> > --- a/arch/arm64/kvm/arm.c
> > +++ b/arch/arm64/kvm/arm.c
> > @@ -50,6 +50,7 @@ DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector);
> >
> > static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page);
> > unsigned long kvm_arm_hyp_percpu_base[NR_CPUS];
> > +DECLARE_KVM_NVHE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params);
> >
> > /* The VMID used in the VTTBR */
> > static atomic64_t kvm_vmid_gen = ATOMIC64_INIT(1);
> > @@ -1347,10 +1348,7 @@ static int kvm_map_vectors(void)
> >
> > static void cpu_init_hyp_mode(void)
> > {
> > - phys_addr_t pgd_ptr;
> > - unsigned long hyp_stack_ptr;
> > - unsigned long vector_ptr;
> > - unsigned long tpidr_el2;
> > + struct kvm_nvhe_init_params *params = this_cpu_ptr_nvhe_sym(kvm_init_params);
> > struct arm_smccc_res res;
> >
> > /* Switch from the HYP stub to our own HYP init vector */
> > @@ -1361,13 +1359,18 @@ static void cpu_init_hyp_mode(void)
> > * kernel's mapping to the linear mapping, and store it in tpidr_el2
> > * so that we can use adr_l to access per-cpu variables in EL2.
> > */
> > - tpidr_el2 = (unsigned long)this_cpu_ptr_nvhe_sym(__per_cpu_start) -
> > - (unsigned long)kvm_ksym_ref(CHOOSE_NVHE_SYM(__per_cpu_start));
> > + params->tpidr_el2 = (unsigned long)this_cpu_ptr_nvhe_sym(__per_cpu_start) -
> > + (unsigned long)kvm_ksym_ref(CHOOSE_NVHE_SYM(__per_cpu_start));
> >
> > - pgd_ptr = kvm_mmu_get_httbr();
> > - hyp_stack_ptr = __this_cpu_read(kvm_arm_hyp_stack_page) + PAGE_SIZE;
> > - hyp_stack_ptr = kern_hyp_va(hyp_stack_ptr);
> > - vector_ptr = (unsigned long)kern_hyp_va(kvm_ksym_ref(__kvm_hyp_host_vector));
> > + params->vector_hyp_va = (unsigned long)kern_hyp_va(kvm_ksym_ref(__kvm_hyp_host_vector));
> > + params->stack_hyp_va = kern_hyp_va(__this_cpu_read(kvm_arm_hyp_stack_page) + PAGE_SIZE);
> > + params->pgd_pa = kvm_mmu_get_httbr();
>
> Note to self: rename this to kvm_mmu_get_hyp_pgd() (another AArch32-ism).
>
> > +
> > + /*
> > + * Flush the init params from the data cache because the struct will
> > + * be read while the MMU is off.
> > + */
> > + __flush_dcache_area(params, sizeof(*params));
>
> nit: please use kvm_flush_dcache_to_poc(), as it clearly indicates to
> which point we are flushing.
Will change, but out of curiosity - how is it different? AFAICT, it is just
an alias with a single use in __clean_dcache_guest_page:
#define kvm_flush_dcache_to_poc(a,l) __flush_dcache_area((a), (l))
>
> Thanks,
>
> M.
>
> --
> Without deviation from the norm, progress is not possible.
next prev parent reply other threads:[~2020-11-25 10:40 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-16 20:42 [PATCH v2 00/24] Opt-in always-on nVHE hypervisor David Brazdil
2020-11-16 20:42 ` [PATCH v2 01/24] psci: Support psci_ops.get_version for v0.1 David Brazdil
2020-11-16 20:42 ` [PATCH v2 02/24] psci: Accessor for configured PSCI function IDs David Brazdil
2020-11-23 13:47 ` Marc Zyngier
2020-11-16 20:42 ` [PATCH v2 03/24] arm64: Make cpu_logical_map() take unsigned int David Brazdil
2020-11-16 20:42 ` [PATCH v2 04/24] arm64: Move MAIR_EL1_SET to asm/memory.h David Brazdil
2020-11-23 13:52 ` Marc Zyngier
2020-11-25 10:31 ` David Brazdil
2020-11-25 11:21 ` Marc Zyngier
2020-11-25 13:26 ` David Brazdil
2020-11-25 13:33 ` Marc Zyngier
2020-11-16 20:42 ` [PATCH v2 05/24] kvm: arm64: Initialize MAIR_EL2 using a constant David Brazdil
2020-11-16 20:43 ` [PATCH v2 06/24] kvm: arm64: Move hyp-init params to a per-CPU struct David Brazdil
2020-11-23 14:20 ` Marc Zyngier
2020-11-25 10:39 ` David Brazdil [this message]
2020-11-25 10:49 ` Marc Zyngier
2020-11-16 20:43 ` [PATCH v2 07/24] kvm: arm64: Refactor handle_trap to use a switch David Brazdil
2020-11-23 14:32 ` Marc Zyngier
2020-11-16 20:43 ` [PATCH v2 08/24] kvm: arm64: Add SMC handler in nVHE EL2 David Brazdil
2020-11-23 18:00 ` Marc Zyngier
2020-11-16 20:43 ` [PATCH v2 09/24] kvm: arm64: Add .hyp.data..ro_after_init ELF section David Brazdil
2020-11-16 20:43 ` [PATCH v2 10/24] kvm: arm64: Support per_cpu_ptr in nVHE hyp code David Brazdil
2020-11-16 20:43 ` [PATCH v2 11/24] kvm: arm64: Create nVHE copy of cpu_logical_map David Brazdil
2020-11-16 20:43 ` [PATCH v2 12/24] kvm: arm64: Bootstrap PSCI SMC handler in nVHE EL2 David Brazdil
2020-11-23 17:55 ` Marc Zyngier
2020-11-16 20:43 ` [PATCH v2 13/24] kvm: arm64: Add offset for hyp VA <-> PA conversion David Brazdil
2020-11-16 20:43 ` [PATCH v2 14/24] kvm: arm64: Forward safe PSCI SMCs coming from host David Brazdil
2020-11-16 20:43 ` [PATCH v2 15/24] kvm: arm64: Extract parts of el2_setup into a macro David Brazdil
2020-11-23 15:27 ` Marc Zyngier
2020-11-25 12:57 ` David Brazdil
2020-11-16 20:43 ` [PATCH v2 16/24] kvm: arm64: Extract __do_hyp_init into a helper function David Brazdil
2020-11-16 20:43 ` [PATCH v2 17/24] kvm: arm64: Add CPU entry point in nVHE hyp David Brazdil
2020-11-16 20:43 ` [PATCH v2 18/24] kvm: arm64: Add function to enter host from KVM nVHE hyp code David Brazdil
2020-11-16 20:43 ` [PATCH v2 19/24] kvm: arm64: Intercept host's PSCI_CPU_ON SMCs David Brazdil
2020-11-23 17:04 ` Marc Zyngier
2020-11-16 20:43 ` [PATCH v2 20/24] kvm: arm64: Intercept host's CPU_SUSPEND PSCI SMCs David Brazdil
2020-11-23 17:22 ` Marc Zyngier
2020-11-16 20:43 ` [PATCH v2 21/24] kvm: arm64: Add kvm-arm.protected early kernel parameter David Brazdil
2020-11-23 17:30 ` Marc Zyngier
2020-11-16 20:43 ` [PATCH v2 22/24] kvm: arm64: Keep nVHE EL2 vector installed David Brazdil
2020-11-16 20:43 ` [PATCH v2 23/24] kvm: arm64: Trap host SMCs in protected mode David Brazdil
2020-11-23 17:36 ` Marc Zyngier
2020-11-16 20:43 ` [PATCH v2 24/24] kvm: arm64: Fix EL2 mode availability checks David Brazdil
2020-11-23 13:44 ` [PATCH v2 00/24] Opt-in always-on nVHE hypervisor Marc Zyngier
2020-11-23 18:01 ` Marc Zyngier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201125103946.2unmclsdfqpr3eyk@google.com \
--to=dbrazdil@google.com \
--cc=ascull@google.com \
--cc=catalin.marinas@arm.com \
--cc=cl@linux.com \
--cc=dennis@kernel.org \
--cc=james.morse@arm.com \
--cc=julien.thierry.kdev@gmail.com \
--cc=kernel-team@android.com \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lorenzo.pieralisi@arm.com \
--cc=mark.rutland@arm.com \
--cc=maz@kernel.org \
--cc=qperret@google.com \
--cc=qwandor@google.com \
--cc=suzuki.poulose@arm.com \
--cc=tj@kernel.org \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®