From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ot1-f52.google.com (mail-ot1-f52.google.com [209.85.210.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C171D3BE64A for ; Wed, 1 Apr 2026 09:48:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=209.85.210.52 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775036915; cv=pass; b=VbWbs2lT7AqxY66q7TaW1XLy+8Hm24TRGDg1dEcKs8f4oE8HJVUP1ll6goP+ZzIWlAUGPzpWYEcB0ImbWP4DW1Ut4jrYx/bXekv4RWJv3URtG+h9EHbjoX7S+9Q6xIN1B/5NUytWIrmmDWSCQBRKjHFnU/t5BMVHeIXYPhgv7s0= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775036915; c=relaxed/simple; bh=GZli/LHTedU6rip3UvZKNcK1kxLRUBMPFr36LKZlEoE=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=VWbfIqGVRCTyOLEq3j57jdzSbD7XnmhkeH2wLzHri2GQb1IXe/J3IqH7/3nVmNPPjcVgAIWhbcnEdfgWKbEe9I7uSQZXuDovIkr27EukS1AsHI+mX1jAoEs+18TJp/xK+b+O17U3kBRXJ/WN4VfZHs9xUDi385d33GUAAej8aPQ= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=nqXNLNT9; arc=pass smtp.client-ip=209.85.210.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nqXNLNT9" Received: by mail-ot1-f52.google.com with SMTP id 46e09a7af769-7d7d4ebccf7so5665187a34.0 for ; Wed, 01 Apr 2026 02:48:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1775036913; cv=none; d=google.com; s=arc-20240605; b=J74eGDZyHcDQOuOCcgpxf09i61Qa/ewrGFKC8XjFA0tqGuUq/7z0q1sITZCACPg6G9 FhtdnlZJIDse9FLD7krB5AB2KHZdNhRdqiKW3IHKSjyYrSRnakNGK1zmmx553egellle p+Kz5721KJ8L9FxyhPQVesnGt5DhMtVNdnugWl9WriOm/uETu7X6WXo5QLQkj+GfyuDz DvFa3O5tkL4YgENDQGZJQWsu5mYPX/GHuQ51fckS2o3mwmUqFp5QtVJPUZ1R1pX9lp+W qdsvsFGglbbpuxJt1vO0pAZfISEPE/lZcVS7xTXay83i/g9BmHGZ+VgvgH6G6+AUGxN9 wb9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=/wztYxSQ8FeDXYv0doyrHu4Krq/mXz4Kvd+nHq+kFLA=; fh=1zXxEr9pHwgraGl/S6BzSKoDnXVEfiPEHQLjMcp/uZE=; b=EnL4VRSfvnYgF8Y1oYkCnNwGvGi70TlRG+wo/SGRvkOw5ar5omTb3m+DJ8EiTgliJj P7M7uEv3GLH6wkSoAHkMSlOocV35ZbZV4YCRHUGKGq8Ga7aeyije3cDe0y6uhS+0kLIp 17oTqRo8OtWHHK+GytLYkGzP9lvRj4Gk9twmSzPKtqmul2QHOmhHIcvjnKeK+Pch7SuF Ves5DmLt1ZVefkYZ+sRRkcUW2AUBHG6BLYUV5fmeRkgHWBL0yyh2Ro35a/m13nAwgKWJ wQaMIoBtn8UC0Ino0iwc+yiZuGLi9b2Ugd3RCeTFvydNFMSAwwYKrJ2qoCvL32MKAReI bPYg==; darn=vger.kernel.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1775036913; x=1775641713; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=/wztYxSQ8FeDXYv0doyrHu4Krq/mXz4Kvd+nHq+kFLA=; b=nqXNLNT91XCinz4H7zKTIuLdYZhHAff5DjZ7pCYlTtUlefcnZEneocjV/U2Ywd+S4D qRZRNYKMwjzyG0QRv5B0RfZHGtlYKR3K0oGy7ZbF9N8JvkzFC34xCN27CRYah3fyQlDc rqiygl5ExCuxuQXIQPaA2477+NYf6kfMWVTffewxWEL3l7QrOy0UodoD+n3eOLoPzUbN s7q9wLtbRwch3Qp4yFvofL5UJJZwCVmVNGweSo05kl4EHoHLQ9G2EVg9c5Ia2Fop844F wj2oywElaST9KkJEorGtndJ2JNgSn2GYGx8lm5JX1B+gxXEqu5v4HZObbaKZ0wjmvd6B MdFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1775036913; x=1775641713; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=/wztYxSQ8FeDXYv0doyrHu4Krq/mXz4Kvd+nHq+kFLA=; b=iTKJfjwKKyQSYAMFsYQ5JKXMC8igU58t6J/X2I0Lro1JLKdO2EIBnzKZwr7LbQl6gG WnoAUNTJIJVghBIK1kRqRo/plUMUGhOMK43Bzb1tcShtorvVWb69wCUyHrflluKv+sNm 4kmS/saYerixKXKuTjUQXpdY7PmqvpRX+rC1VeYy8RPBXEx0U5sXBea0sI5Zhe7eTwZs QELEmQo7KWdSbmvxximcW5kpSETD8RP69CDBrHqFQRBnpQez1GQQK8X23EoBKyrvyUn0 xVT5sl102OiBT1PaEh4dDNuJDOglNHXKjDEBD4pGEkJbgH9k+YlOffh7U3q92YrsZiS7 B9Xw== X-Forwarded-Encrypted: i=1; AJvYcCWPJpEafz56lRLDoUV0V/DH1boxeIEsqKSDynu1zDG7iuAPsG3vnqJqhoK3o94WHaCafjxxJdHYvcpqNIo=@vger.kernel.org X-Gm-Message-State: AOJu0YznsIr6FpU602mgICW+2kqxWkKGQPMepY+7rrUcf/il0i5HoD+t NO2mZX3L5spZ/4bn8rC/o5EcRLYuNJc1+qKm/3zf0uSx53ML4Xq01XGwMAAVvgKxvO6RB8E+TOZ 8ysOCs8UwtCRALgemFCiwf2jtb5Qc2g8= X-Gm-Gg: ATEYQzwGew3e6X68gW6lhR682dbHYX4vjTDZqNVscD5uusirAA6b85ayos5rR1+IXEu OLS37ikXmzqlHAhCBjMdOkp/hfpQ9RA1EWaBGpSLDUd9xivuMyh/99K0j1MA59FSoB+uJsif62F XX+lp6iIhySwMgzH4bxXSk6rS8H0v/zTs5NBCT/5MHFjHJY5Du5VxjZC4wnVCM+CIoXdTb0tKjG p6T77dr5+zFwQ5n0hny/gCfDWxWrjLqH7ZDWtG56AtfAPzlzg3pe0U83q0jTqz7iZOVN1zCsing aSVeGC77 X-Received: by 2002:a05:6820:1794:b0:67e:331c:9ac4 with SMTP id 006d021491bc7-67fabc05a37mr1474937eaf.23.1775036912600; Wed, 01 Apr 2026 02:48:32 -0700 (PDT) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <20251219035334.39790-1-kernellwp@gmail.com> In-Reply-To: From: Wanpeng Li Date: Wed, 1 Apr 2026 17:48:21 +0800 X-Gm-Features: AQROBzCvshFpmBew3lVhJRFCjtN6YdmENqPm4VJGszE0C4DZtJ2ofSW0-dZMU7w Message-ID: Subject: Re: [PATCH v2 0/9] sched/kvm: Semantics-aware vCPU scheduling for oversubscribed KVM To: Sean Christopherson Cc: Peter Zijlstra , Ingo Molnar , Thomas Gleixner , Paolo Bonzini , K Prateek Nayak , Christian Borntraeger , Steven Rostedt , Vincent Guittot , Juri Lelli , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, Wanpeng Li Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Sean=EF=BC=8C On Fri, 13 Mar 2026 at 09:13, Sean Christopherson wrote= : > > On Fri, Dec 19, 2025, Wanpeng Li wrote: > > Part 2: KVM IPI-Aware Directed Yield (patches 6-9) > > > > Enhance kvm_vcpu_on_spin() with lightweight IPI tracking to improve > > directed yield candidate selection. Track sender/receiver relationships > > when IPIs are delivered and use this information to prioritize yield > > targets. > > > > The tracking mechanism: > > > > - Hooks into kvm_irq_delivery_to_apic() to detect unicast fixed IPIs (t= he > > common case for inter-processor synchronization). When exactly one > > destination vCPU receives an IPI, record the sender->receiver relatio= nship > > with a monotonic timestamp. > > > > In high VM density scenarios, software-based IPI tracking through > > interrupt delivery interception becomes particularly valuable. It > > captures precise sender/receiver relationships that can be leveraged > > for intelligent scheduling decisions, providing performance benefits > > that complement or even exceed hardware-accelerated interrupt deliver= y > > in overcommitted environments. > > > > - Uses lockless READ_ONCE/WRITE_ONCE accessors for minimal overhead. Th= e > > per-vCPU ipi_context structure is carefully designed to avoid cache l= ine > > bouncing. > > > > - Implements a short recency window (50ms default) to avoid stale IPI > > information inflating boost priority on throughput-sensitive workload= s. > > Old IPI relationships are naturally aged out. > > > > - Clears IPI context on EOI with two-stage precision: unconditionally c= lear > > the receiver's context (it processed the interrupt), but only clear t= he > > sender's pending flag if the receiver matches and the IPI is recent. = This > > prevents unrelated EOIs from prematurely clearing valid IPI state. > > That all relies on lack of IPI and EOI virtualization, which seems very > counter-productive given the way hardware is headed. I think there is an important distinction here. APICv / posted interrupts accelerate IPI *delivery*, but they do not help with the host-side *scheduling decision* in kvm_vcpu_on_spin(). A posted interrupt can land in a not-yet-scheduled vCPU's PIR, but that vCPU still won't process it until it actually gets CPU time. IPI tracking targets exactly this gap: which vCPU should we yield to right now. In high VM density / overcommitted scenarios, APICv's advantage narrows precisely because the bottleneck shifts from IPI delivery latency to *scheduling latency* =E2=80=94 the target vCPU may have its post= ed interrupt sitting in PIR but cannot process it because it is competing for physical CPU time with many other vCPUs. In that regime, making a better yield-to decision on the host side has a more direct impact on end-to-end IPI response time than faster hardware delivery to a vCPU that isn't running. So I would not characterize IPI tracking as a workaround for lack of hardware virtualization support. It addresses an orthogonal problem =E2=80= =94 host-side scheduling decisions =E2=80=94 that hardware IPI acceleration doe= s not solve. The two are complementary: APICv makes delivery fast when the target is running; IPI-aware directed yield makes scheduling better when the target is not running. > > My reaction to all of this is that in the long run, we'd be far better of= f getting > the guest to "cooperate" in the sense of communicating intent, status, et= c. As I agree that guest cooperation / PV scheduling hints could provide even richer semantic information for these decisions in the long term. The host-side IPI tracking approach has the practical advantage of working with unmodified guests and mixed-OS deployments today, which covers a large fraction of real-world overcommitted environments. These two directions are not mutually exclusive =E2=80=94 PV hints can coex= ist with host-side heuristics, and the IPI tracking infrastructure could serve as a useful fallback or baseline even after PV scheduling is available, for guests that do not support it. Regards, Wanpeng