mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Kunkun Jiang <jiangkunkun@huawei.com>
To: Marc Zyngier <maz@kernel.org>
Cc: Oliver Upton <oliver.upton@linux.dev>,
	James Morse <james.morse@arm.com>,
	Suzuki K Poulose <suzuki.poulose@arm.com>,
	Zenghui Yu <yuzenghui@huawei.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Will Deacon <will@kernel.org>, Gavin Shan <gshan@redhat.com>,
	Jean-Philippe Brucker <jean-philippe@linaro.org>,
	"open list:IRQCHIP DRIVERS" <linux-kernel@vger.kernel.org>,
	<kvmarm@lists.linux.dev>, <wanghaibin.wang@huawei.com>
Subject: Re: [RFC PATCH] KVM: arm/arm64: GICv4: Support shared VLPI
Date: Mon, 6 Nov 2023 23:33:13 +0800	[thread overview]
Message-ID: <12266a84-1148-954c-c6d6-67c8f9cc80b7@huawei.com> (raw)
In-Reply-To: <87msvt6cc7.wl-maz@kernel.org>

Hi Marc,

On 2023/11/4 18:29, Marc Zyngier wrote:
> On Thu, 02 Nov 2023 14:35:07 +0000,
> Kunkun Jiang <jiangkunkun@huawei.com> wrote:
>> In some scenarios, the guest virtio-pci driver will request two MSI-X,
>> one vector for config, one shared for queues. However, the host driver
>> (vDPA or VFIO) will request a vector for each queue.
> Well, VFIO will request *all* available MSI-X. It doesn't know what a
> queue is.
>
>> In the current implementation of GICv4/4.1 direct injection of vLPI,
>> pINTID and vINTID have one-to-one correspondence. Therefore, the
> This matching is a hard requirement that matches the architecture. You
> cannot change it.
>
>> above scenario cannot be handled correctly. The host kernel will
>> execute its_map_vlpi multiple times but only execute its_unmap_vlpi
>> once. This may cause guest hang[1].
> Why does it hang? As far as it is concerned, it has unmapped the
> interrupts it cares about. Where are the calls to its_map_vlpi()
> coming from? It should only occur if the guest actively programs the
> MSI-X registers. What is your VMM? How can I reproduce this issue?
>
>> |	WARN_ON(!(irq->hw && irq->host_irq == virq));
>> |	if (irq->hw) {
>> |		atomic_dec(&irq->target_vcpu->arch.vgic_cpu.vgic_v3.its_vpe.vlpi_count);
>> |		irq->hw = false;
>> |		ret = its_unmap_vlpi(virq);
>> |	}
>>
>> Add a list to struct vgic_irq to record all host irqs mapped to the vlpi.
>> When performing an action on the vlpi, traverse the list and perform this
>> action on all host irqs.
> This makes no sense. You are blindly associating multiple host
> interrupts with a single guest interrupt. This is a blatant violation
> of the architecture. When unmapping a VLPI from a guest, only this one
> should be turned again into an LPI. Not two, not all, just this one.
>
> Maybe you have found an actual issue, but this patch is absolutely
> unacceptable. Please fully describe the problem, provide traces, and
> if possible a reproducer.
>
>> Link: https://lore.kernel.org/all/0d9fdf42-76b1-afc6-85a9-159c5490bbd4@huawei.com/#t
> I tried to parse this, but it hardly makes sense either. You seem to
> imply that the host driver pre-configures the device, which is
> completely wrong. The host driver (VFIO) should simply request all
> possible physical LPIs, and that's all. It is expected that this
> requesting has no other effect on the HW. Also, since your guest
> driver only configures a single vLPI, there should be only a single
> its_map_vlpi() call.
Sorry to replay so late.

The virtio-scsi device has seven vectors (entry0-6): one for config,
six for queues. In Guest, e.g. centos 7.6 4.19, virtio-pci driver
will request only one vLPI, which is shared for queues.
The entry 0 is used for config. It's not relevant to this issue, so
we're not going to discuss it. The virtio-pci driver write entry1-6
massage.data in the msix-table and trap to QEMU for processing. The
massage.data is as follow:
> entry-0 0
> entry-1 1
> entry-2 1
> entry-3 1
> entry-4 1
> entry-5 1
> entry-6 1 

The calling process of kvm is as follows. its_map_vlpi_will be
executed 6 times. Six host irqs are mapped to one vLPI.
> kvm_irqfd_assign
>     irq_bypass_register_consumer
>         ...
>         kvm_arch_irq_bypass_add_producer
>             kvm_vgic_v4_set_forwarding
>                 its_map_vlpi 

When executing the reboot command inside the Guest,
kvm_vgic_v4_unset_forwarding will be execute 6 times. WARN_ON
will also be triggered 6 times. But its_unmap_vlpi will only
be executed the first time.
> kvm_arch_irq_bypass_del_producer
>     kvm_vgic_v4_unset_forwarding
>         WARN_ON(!(irq->hw && irq->host_irq == virq));
>         if (irq->hw) {
>             irq->hw = false;
> its_unmap_vlpi
>         } 

Therefore, only the mapping between the first host irq and
vLPI is unmapped. When the guest reboots into the BIOS phase,
the remaining 5 host irqs may still send interrupts. This
causes the guest to hang.

Looking forward to your reply.

Thanks,
Kunkun Jiang
> So it seems to me that your HW and SW are doing things that are not
> expected at all.
>
> 	M.
>

  reply	other threads:[~2023-11-06 15:35 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-11-02 14:35 Kunkun Jiang
2023-11-04 10:29 ` Marc Zyngier
2023-11-06 15:33   ` Kunkun Jiang [this message]
     [not found]   ` <1fb8353e-e9c4-2570-c2ca-ec537c18ac4d@huawei.com>
2023-11-06 15:33     ` Marc Zyngier
2023-11-08  9:45       ` Kunkun Jiang
2023-12-02 12:20         ` Marc Zyngier
2023-12-04  8:39           ` Jason Wang
2023-12-04  8:47             ` Jason Wang
2023-12-04  9:00               ` Marc Zyngier
2023-12-04  9:06                 ` Jason Wang
2023-12-04  8:52             ` Marc Zyngier

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=12266a84-1148-954c-c6d6-67c8f9cc80b7@huawei.com \
    --to=jiangkunkun@huawei.com \
    --cc=catalin.marinas@arm.com \
    --cc=gshan@redhat.com \
    --cc=james.morse@arm.com \
    --cc=jean-philippe@linaro.org \
    --cc=kvmarm@lists.linux.dev \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maz@kernel.org \
    --cc=oliver.upton@linux.dev \
    --cc=suzuki.poulose@arm.com \
    --cc=wanghaibin.wang@huawei.com \
    --cc=will@kernel.org \
    --cc=yuzenghui@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®