From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E008E339374 for ; Tue, 11 Aug 2026 17:22:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.47 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786468926; cv=none; b=l6dsIIbO0x3714nx4bDiU9A0sGX6WB2cViWspxGpnMcO0QbKEe9YYWdbT1OnrbumXnNQjgIa6393mZG7yRGsGUUel38AQlJtudLeiO3CIdbREzd5uzIdjtAinbkEdWOM29JzJsO8LpUoq4E2ZEFLXsTGgs9PC+Eia2eDsC6f4AM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786468926; c=relaxed/simple; bh=vR6sSLhLogjcl/g3uUxcV5IhWW8sS+lAQxhBsrzvn6s=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=TRmEjkQ06Pswz9joE70oggh1O0eSym3lgjxGm98k96YPNK5OJRJqygkZGvg2eje6D3u5q0Eby/Qj76f9Co/DzX+jvySb6pXdydxPl7q8OU1Su6XYJ0aYZnb2Fxlh+7z232IUvZ5HhL2ZBvhLR+wlvW8PMdlKoSw/7yUXXEMqNuU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=HlIXIuqs; arc=none smtp.client-ip=209.85.128.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="HlIXIuqs" Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-498012a61f6so2175e9.0 for ; Tue, 11 Aug 2026 10:22:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786468923; x=1787073723; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :content-type:mime-version:references:message-id:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to:content-type; bh=XhKC2uH9JNLWHRqJ28vd49ZvOTGeVzPYd+xhSQmP2ZI=; b=HlIXIuqs74ZQvzgrxuUcrZwpYEY6izMa3pTU4C988HUYVK1zqNZwNcZljYDplylW0F vpGB4qwX8TE4RsShsBjnkoBcvivZKm73eD/+qkNpnGSwuSsKTnBqX78Y1yfJ5BYeb6YM MEolfaWV1mGT8/UxHwSR499k4zpmtwt71mYu1b/gT+hJO2IJn+F5tE83V0ggnNZkbcqV 8ZgwDFMM+ngho5OHhnlnbKVKQNC1l7um+Fc30SibXSOV7pxefqRs2IJ28H6MSNxYO01u qHbySsRlivsarZG+4BngeP89IieDNxmSQXNLyv65QDvCcp+ZnSpNBBcDjJCrMv1r63di SdOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786468923; x=1787073723; h=in-reply-to:content-transfer-encoding:content-disposition :content-type:mime-version:references:message-id:subject:cc:to:from :date:x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to:content-type; bh=XhKC2uH9JNLWHRqJ28vd49ZvOTGeVzPYd+xhSQmP2ZI=; b=XR33ljVYkZ5O1JYEKTP9fObbs4qEAJr6VqgtcvsK7cZ9CtgYS2A7jMq6UqfNMH9cSC jmsvk7yVASZRMobNm9w50mixxulkRIYuV4LCJjn1rTCocLw0rtAShbefWaPKy14Ucqae 4RZauS4Kn5z5/kFxnt271L5YHLc2olzxLFcP2N2mIMaNQAZzA/GT3XvvOlXbYYw41a8C K3w1knqWNqFoSB5qiO1/GHfCgXutvf1FZ7dFX6d8MJoQ/JnJStJUH2z0zAuuSIEFbdFK z0DWpOIs7eySA5X5LO/Qu/USaiYmc7BneXJHKdxmLfpp+TMXX8OXb9P19XhtDzlbhqkz vXZQ== X-Forwarded-Encrypted: i=1; AHgh+RqDK9X6R4ddQXa2P8Ewx+6lmNFZguA2uzR339HGekkkKH3mo1AlvICdtEOT1UZqoHHWQWx/8Fa/fyjdBsc=@vger.kernel.org X-Gm-Message-State: AOJu0YzPNU6PXlCTF/JfxfPPqZvSfmPAtWF2E7OnT6abntMmA8FGZoyF B6jBstSeIQkLwlCZhnxg4Gf0a3q/sGQrpEFqz0MHFtrfrPuuKqxCfwD7mDftx/QsXA== X-Gm-Gg: AR+sD10sfCXvnue9L2MkpkwY00EvLHygdDHINIwQZh44iup8boAZsauZu0ZCFsbtlaV d4bT4djLNqGIBlFqQSs4LbDNtnJE/1XgJLilD0Maty/qKvMsJQsNZa8FyVyn0E3mDRXCAG2PV7V KGMJAdo9Gerw9PaYqOlSiPGEH9AUh2moESy7WnaJuYwe/53Og1tlseYp4ghaOOusHJFXt3taHWI 5tBFyMmIXjA3L1/WPHkJJOqBZ3MAbHDBdW7pUYTr8jWneYYwljRu8hmrMKB2vZDuNshFYmONkuc A1GRqBkTnIraA8p9M0I9ihwmtiAgowQHqz1obBnknqSXHod6KZfAZqNHPC+z6m1ylQ9nN9tH1Q8 DHqnLE5r0CcSQ3IhrfrG1cgRA1EY4EsQKsAH7yUOLty6ro8id5opmtJyQ8Q3E3NUNuOv55r9PZX AFSCuDXPDOToPvJ+xoagSBm6BO1xiVacCFGCiOPlrkyz0cxZF7IJY/0OJEahgqT90ud5gwI7R3e IEvB3fq0Yw8JLLUS8odWubITz/mrg== X-Received: by 2002:a05:600c:4e49:b0:495:5515:3c00 with SMTP id 5b1f17b1804b1-4997aeca4f3mr16485e9.9.1786468922574; Tue, 11 Aug 2026 10:22:02 -0700 (PDT) Received: from google.com (250.192.189.35.bc.googleusercontent.com. [35.189.192.250]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4814a7097b8sm5888525f8f.21.2026.08.11.10.22.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Aug 2026 10:22:01 -0700 (PDT) Date: Tue, 11 Aug 2026 17:21:58 +0000 From: Mostafa Saleh To: Marc Zyngier Cc: Oliver Upton , linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, seiden@linux.ibm.com, joey.gouly@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, vdonnefort@google.com, tabba@google.com, sebastianene@google.com, keirf@google.com, yaoyuan@linux.alibaba.com Subject: Re: [PATCH v2 0/3] KVM: arm64: Fixes for timers and pKVM Message-ID: References: <20260808085824.732659-1-smostafa@google.com> <178621460524.381735.15023846552532101058.b4-ty@kernel.org> <86pkzo23u0.wl-maz@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <86pkzo23u0.wl-maz@kernel.org> On Tue, Aug 11, 2026 at 02:18:15PM +0100, Marc Zyngier wrote: > On Tue, 11 Aug 2026 13:17:25 +0100, > Mostafa Saleh wrote: > > > > On Sat, Aug 8, 2026 at 7:44 PM Oliver Upton wrote: > > > > > > On Sat, 08 Aug 2026 08:58:21 +0000, Mostafa Saleh wrote: > > > > What started as a small patch ended up as a 3 patch series thanks > > > > to Sashiko. > > > > > > > > First patch from Marc to consolidate the offset calculation, > > > > follow up patches fix issues with non-protected VM and timer > > > > offset and protected VM running with broken CNTVOFF_EL2. > > > > > > > > [...] > > > > > > Dropped the unintended SOB in patch 3 you mentioned. > > > > > > Applied to next, thanks! > > > > > > [1/3] KVM: arm64: Make timer_get_offset() work in all contexts > > > https://git.kernel.org/kvmarm/kvmarm/c/2858600ecd01 > > > [2/3] KVM: arm64: Fix timer offsets for non-protected VMs > > > https://git.kernel.org/kvmarm/kvmarm/c/47d3eef780e3 > > > [3/3] KVM: arm64: Fix hvhe and broken CNTVOFF_EL2 > > > https://git.kernel.org/kvmarm/kvmarm/c/2e813a6e8ebe > > > > > > > Thanks Oliver! I believe there is one more bug. I'm not sure where the > > bug is or if it relates to the broken timers. > > Before those patches I could not boot a protected VM because of the > > panic, now after booting protected VMs, I sometimes get a system > > reset. > > On this quality HW, this is usually an indication that you are taking > an exception in a tight loop. > I tried to add a poor man exception storm detection in the kernel handlers (el1h_64_sync_handler and __gic_handle_irq), but it didn’t trigger before the reset. My hunch would be that there are some paths in protected VMs that takes too long that the watchdog fires. I saw the resets mostly at either userspace boot or VM teardown, otherwise the VM seems functional. I will collect timestamps from hypervisor entry/exit and check how large are those. > > I confirmed that cntvoff_el2 does not get written to non-zero, I also > > removed the sysreg write completely (rely on xzr value at init) so my > > guess is that the HW might be allergic to more than just non-zero > > values in cntvoff_el2. > > Is that in hVHE mode? Can you trap the access and route it to the > existing handling code? > Yes, only hVHE. Protected nVHE works fine. One interesting observation is that when starting a VM with a single vcpu I don’t see the reset anymore compared to 4 cpus before. Enabling traps for timer unconditionally for protected VMs still has the same issue. > > I do not have issues with non-protected VMs anymore. > > Do these run with an offset or not? Yes, they have the offset set from kvm_timer_vcpu_init() with kvm_phys_timer_read(). Thanks, Mostafa > > M. > > -- > Without deviation from the norm, progress is not possible.