From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=100.103.45.18 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1792842570; b=E4IV/qeenYQ3g8USheF1RNBNpOdJ/2u7D03gaRPFZ54jI0197bhZG1Ci+pakCB+0YvWa DcPPEwhMvYYKSzceHk59mIkGLCstpqMyb3O9Q2gyMeeOUYLYqAFAjbsQiyQ02daL7U4EK ZXPi4o+uJ2rsc78+kJ+/AWd56ezKLKXEfBpUEcT1M7GjSDURSeYbctwM7nT8By70Ef2fF Rg2ozjjY+5xijSRuaSFkcj6hTtnJbLOb97csOE1Yl6Gs2oczwZUtYidlAbW5adniSiVDo hv/kb5SDAcB+gTMcQCQHCtKUhdd3FZo7echjJBJpzy8Cx4di4JqHheMmWpqWkWb+DnQ== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1792842570; h=Received:DKIM-Signature:Received:Date:Message-ID:From:To:Cc:Subject: In-Reply-To:References:User-Agent:MIME-Version:Content-Type: X-SA-Exim-Connect-IP:X-SA-Exim-Rcpt-To:X-SA-Exim-Mail-From: X-SA-Exim-Scanned; bh=FuqXf7oFIzUBEcQbKPY0llvBnP0Pu2NRXKoSWtEKfn0=; b=tWu90DjtMUGIoNeCdMdBdiU4nncw0db0TUBXr+huXA8Oc8VGwefx7YFAr65b17EhBexl /pYWdyUJFoW3wQsN8JpB8+bvZ7HP4/KNuvRO6u4mvOBYkSocISgdDNLWS6diLO4NapkJp 6HjSRei73kpCuOeMJSBBUBuKLHjhXQ0RAzXGoXGidgRQ0LhRabaIDqRKjTCvJ5K8N6zUW zSOF7PT2s69GM2Yn1psRM+DI35KHE8BwabgD2N6nkHBVJCB8Pc+m+4KwuYFvC7P5JqfOx GABRI7ivhypBxn1L5mKW8U6TyBnH7D9Q4gfiC5skrL1QRY9DTzzU5nkxdxxAPQVg1hw== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=WEjZ1CRI; arc=none smtp.remote-ip=100.103.45.18 Received: from smtp.kernel.org (quasi.space.kernel.org [100.103.45.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A0EE606A31A; Sat, 24 Oct 2026 11:49:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key, unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20260515 header.b=WEjZ1CRI Received: by smtp.kernel.org (Postfix) with ESMTPSA id AE9461F01825; Sat, 24 Oct 2026 11:49:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1792842569; bh=FuqXf7oFIzUBEcQbKPY0llvBnP0Pu2NRXKoSWtEKfn0=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=WEjZ1CRIcbCJAp/JIdnJ/lHTbaGXMf7MCXDF9mrTgwfRVdgWDLQeZZ3+EelpxSQX5 SKoLSEERZcVQTB9OD1Vh5eDSMwhPUsrhQPtlWsGhBJ8KDUc9XvShjbHdFA8AC5AZ8N V5SWdX6Gi4m49WsO7Lpey8V4+pLGfNAK7mQNzV/8MCnNUm8YN0W7ze63PhY4AcUCKi 7IrRuKbkGXgShhOliQe5X2QbCNrQ3SCjhjKtAAPNQSnwRdTLx5PpfjgxeBoJ69aXyB UJ/ZCC2KhCQVSKL/Tu/v9MmzMGbv3wnsDmUaLm8qHRfarR72rZxxao2RglWbwtoyJ9 AHufBBq1UK93g== Received: from sofa.misterjones.org ([185.219.108.64] helo=lobster-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xKaFb-00000006kGD-2If9; Sat, 24 Oct 2026 11:49:27 +0000 Date: Sat, 24 Oct 2026 12:52:44 +0100 Message-ID: <87tsmb8gdv.wl-maz@kernel.org> From: Marc Zyngier To: Amir Noohi Cc: Thomas Gleixner , Oliver Upton , Radu Rendec , Igor Velkov , Fuad Tabba , Joey Gouly , Steffen Eiden , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon , Mark Rutland , kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 0/2] genirq, KVM: arm64: Avoid the irq descriptor lock on vCPU load In-Reply-To: <20261024022450.629980-1-amirmnoohi@gmail.com> References: <20261024022450.629980-1-amirmnoohi@gmail.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: amirmnoohi@gmail.com, tglx@kernel.org, oupton@kernel.org, radu@rendec.net, iav@iav.lv, fuad.tabba@linux.dev, joey.gouly@arm.com, seiden@linux.ibm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false On Sat, 24 Oct 2026 03:24:48 +0100, Amir Noohi wrote: > > KVM/arm64 vCPUs slow each other down even when every vCPU has a physical > CPU to itself. On a 32 CPU Ampere Altra (GICv3, VHE) running mainline, > with an in-kernel GICv3 and each vCPU thread pinned to its own otherwise > idle CPU, a vCPU which exits to userspace in a loop manages 320k exits/s > on its own, 120k when 16 vCPUs do the same and 59k with 31. The aggregate > rate stops growing at 8 vCPUs. No two vCPUs share a CPU and nothing is > over-subscribed. > > The cause is the interrupt descriptor lock of the host timer interrupts. > Every vCPU load sets their active state with irq_set_irqchip_state() > (twice on VHE: virtual and physical timer). The timer interrupts are per > CPU interrupts, so all CPUs share one descriptor and its lock, while the > state being written is per CPU (a redistributor register on GICv3, a > system register on GICv5). The lock serializes nothing but CPUs updating > their own state. At 8 vCPUs, 99.9% of all lock contention events during > the test have __irq_get_desc_lock() or irq_set_irqchip_state() on the > stack. Looking at the test case, this really isn't a representative or realistic workload. This is a contrived setup that has trapping to userspace as its only purpose. I'm not saying that this is useless observation. But it is not something that represents something exacerbated by "real" VMs. I'll look at the patches shortly. Thanks, M. -- Jazz isn't dead. It just smells funny.