From: Christian Borntraeger <borntraeger@linux.ibm.com>
To: Mark Rutland <mark.rutland@arm.com>
Cc: linux-kernel@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com>,
Michael Ellerman <mpe@ellerman.id.au>,
aleksandar.qemu.devel@gmail.com, alexandru.elisei@arm.com,
anup.patel@wdc.com, aou@eecs.berkeley.edu, atish.patra@wdc.com,
bp@alien8.de, catalin.marinas@arm.com, chenhuacai@kernel.org,
dave.hansen@linux.intel.com, frankja@linux.ibm.com,
frederic@kernel.org, gor@linux.ibm.com, hca@linux.ibm.com,
james.morse@arm.com, jmattson@google.com, joro@8bytes.org,
luto@kernel.org, maz@kernel.org, mingo@redhat.com,
nsaenzju@redhat.com, palmer@dabbelt.com, paulmck@kernel.org,
paul.walmsley@sifive.com, peterz@infradead.org,
seanjc@google.com, suzuki.poulose@arm.com, svens@linux.ibm.com,
tglx@linutronix.de, tsbogend@alpha.franken.de,
vkuznets@redhat.com, wanpengli@tencent.com, will@kernel.org,
Anup Patel <anup@brainfault.org>,
Atish Patra <atishp@atishpatra.org>
Subject: Re: [PATCH v2 0/7] kvm: fix latent guest entry/exit bugs
Date: Wed, 19 Jan 2022 20:30:17 +0100 [thread overview]
Message-ID: <0654e667-1cfa-5147-6661-b3b63288be0b@linux.ibm.com> (raw)
In-Reply-To: <20220119192217.GD43919@C02TD0UTHF1T.local>
Am 19.01.22 um 20:22 schrieb Mark Rutland:
> On Wed, Jan 19, 2022 at 07:25:20PM +0100, Christian Borntraeger wrote:
>> Am 19.01.22 um 11:58 schrieb Mark Rutland:
>>
>>
>> CCing new emails for Anup and Atish so that they are aware of this thread.
>
> Ah; whoops. I'd meant to fix the Ccs on the patches.
>
> Thanks!
>
> [...]
>
>> I just gave this a spin on s390 with debugging on and I got the following:
>>
>> [ 457.151295] ------------[ cut here ]------------
>> [ 457.151311] WARNING: CPU: 14 PID: 0 at kernel/rcu/tree.c:613 rcu_eqs_enter.constprop.0+0xf8/0x118
>
> Hmm, so IIUC that's:
>
> WARN_ON_ONCE(rdp->dynticks_nmi_nesting != DYNTICK_IRQ_NONIDLE);
>
> ... and we're clearly in the idle thread here.
>
> I wonder, is the s390 guest entry/exit *preemptible* ?
Looks like debug_defconfig is indeed using preemption:
CONFIG_PREEMPT_BUILD=y
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
CONFIG_PREEMPT_COUNT=y
CONFIG_PREEMPTION=y
CONFIG_PREEMPT_RCU=y
CONFIG_PREEMPT_NOTIFIERS=y
CONFIG_DEBUG_PREEMPT=y
CONFIG_PREEMPTIRQ_TRACEPOINTS=y
CONFIG_TRACE_PREEMPT_TOGGLE=y
CONFIG_PREEMPT_TRACER=y
# CONFIG_PREEMPTIRQ_DELAY_TEST is not set
>
> If a timer IRQ can preempt in the middle of the EQS, we wouldn't balance
> things before a ctx-switch to the idle thread, which would then be able
> to hit this.
>
> I'll need to go audit the other architectures for similar.
>
> Thanks,
> Mark.
>
>> [ 457.151324] Modules linked in: vhost_vsock vmw_vsock_virtio_transport_common vsock vhost vhost_iotlb xt_CHECKSUM xt_MASQUERADE xt_conntrack ipt_REJECT xt_tcpudp nft_compat nf_nat_tftp nft_objref nf_conntrack_tftp nft_counter kvm nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink sunrpc mlx5_ib ib_uverbs s390_trng ib_core genwqe_card crc_itu_t vfio_ccw mdev vfio_iommu_type1 eadm_sch vfio zcrypt_cex4 sch_fq_codel configfs ip_tables x_tables mlx5_core ghash_s390 prng aes_s390 des_s390 libdes sha3_512_s390 sha3_256_s390 sha512_s390 sha256_s390 sha1_s390 sha_common pkey zcrypt rng_core autofs4
>> [ 457.151422] CPU: 14 PID: 0 Comm: swapper/14 Not tainted 5.16.0-00007-g89e9021389e2 #3
>> [ 457.151428] Hardware name: IBM 3906 M04 704 (LPAR)
>> [ 457.151432] Krnl PSW : 0404d00180000000 00000000a7c0495c (rcu_eqs_enter.constprop.0+0xfc/0x118)
>> [ 457.151440] R:0 T:1 IO:0 EX:0 Key:0 M:1 W:0 P:0 AS:3 CC:1 PM:0 RI:0 EA:3
>> [ 457.151445] Krnl GPRS: ffffffffebd81d31 4000000000000000 0000000000000070 00000000a7fd7024
>> [ 457.151450] 0000000000000000 0000000000000001 0000000000000000 0000000000000000
>> [ 457.151454] 000000000000000e 000000000000000e 00000000a84d3a88 0000001fd8645c00
>> [ 457.151458] 0000000000000000 0000000000000000 00000000a7c04882 0000038000653dc0
>> [ 457.151468] Krnl Code: 00000000a7c0494c: ebaff0a00004 lmg %r10,%r15,160(%r15)
>> 00000000a7c04952: c0f4fffffef7 brcl 15,00000000a7c04740
>> #00000000a7c04958: af000000 mc 0,0
>> >00000000a7c0495c: a7f4ffa3 brc 15,00000000a7c048a2
>> 00000000a7c04960: c0e500003f70 brasl %r14,00000000a7c0c840
>> 00000000a7c04966: a7f4ffcd brc 15,00000000a7c04900
>> 00000000a7c0496a: c0e500003f6b brasl %r14,00000000a7c0c840
>> 00000000a7c04970: a7f4ffde brc 15,00000000a7c0492c
>> [ 457.151527] Call Trace:
>> [ 457.151530] [<00000000a7c0495c>] rcu_eqs_enter.constprop.0+0xfc/0x118
>> [ 457.151536] ([<00000000a7c04882>] rcu_eqs_enter.constprop.0+0x22/0x118)
>> [ 457.151540] [<00000000a7c14cd2>] default_idle_call+0x62/0xd8
>> [ 457.151545] [<00000000a6f816c6>] do_idle+0xf6/0x1b0
>> [ 457.151553] [<00000000a6f81a06>] cpu_startup_entry+0x36/0x40
>> [ 457.151558] [<00000000a7c16abe>] restart_int_handler+0x6e/0x90
>> [ 457.151563] no locks held by swapper/14/0.
>> [ 457.151567] Last Breaking-Event-Address:
>> [ 457.151570] [<00000000a7c0489e>] rcu_eqs_enter.constprop.0+0x3e/0x118
>> [ 457.151574] irq event stamp: 608654
>> [ 457.151578] hardirqs last enabled at (608653): [<00000000a70190d8>] tick_nohz_idle_enter+0xb0/0x130
>> [ 457.151584] hardirqs last disabled at (608654): [<00000000a6f8173e>] do_idle+0x16e/0x1b0
>> [ 457.151589] softirqs last enabled at (608586): [<00000000a7c1861a>] __do_softirq+0x4ba/0x668
>> [ 457.151594] softirqs last disabled at (608581): [<00000000a6f367c6>] __irq_exit_rcu+0x13e/0x170
>> [ 457.151600] ---[ end trace 2ae2154f9724de86 ]---
>>
>> I can not see right now whats wrong, your patches look sane.
next prev parent reply other threads:[~2022-01-19 19:32 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-19 10:58 Mark Rutland
2022-01-19 10:58 ` [PATCH v2 1/7] entry: add arch_in_rcu_eqs() Mark Rutland
2022-01-19 17:35 ` Christian Borntraeger
2022-01-21 17:34 ` Nicolas Saenz Julienne
2022-01-19 10:58 ` [PATCH v2 2/7] kvm: add guest_state_{enter,exit}_irqoff() Mark Rutland
2022-01-20 11:00 ` Paolo Bonzini
2022-01-21 17:35 ` Nicolas Saenz Julienne
2022-01-19 10:58 ` [PATCH v2 3/7] kvm/arm64: rework guest entry logic Mark Rutland
2022-01-21 17:37 ` Nicolas Saenz Julienne
2022-01-19 10:58 ` [PATCH v2 4/7] kvm/mips: " Mark Rutland
2022-01-20 11:10 ` Paolo Bonzini
2022-01-20 13:33 ` Mark Rutland
2022-01-20 16:44 ` Mark Rutland
2022-01-20 16:57 ` Paolo Bonzini
2022-01-20 17:15 ` Mark Rutland
2022-01-20 17:17 ` Sean Christopherson
2022-01-20 17:29 ` Paolo Bonzini
2022-01-21 12:44 ` Mark Rutland
2022-01-19 10:58 ` [PATCH v2 5/7] kvm/riscv: " Mark Rutland
2022-01-20 11:18 ` Paolo Bonzini
2022-01-20 12:56 ` Mark Rutland
2022-01-20 13:13 ` Paolo Bonzini
2022-01-19 10:58 ` [PATCH v2 6/7] kvm/s390: " Mark Rutland
2022-01-19 10:58 ` [PATCH v2 7/7] kvm/x86: " Mark Rutland
2022-01-20 11:20 ` Paolo Bonzini
2022-01-21 17:40 ` Nicolas Saenz Julienne
2022-01-19 18:25 ` [PATCH v2 0/7] kvm: fix latent guest entry/exit bugs Christian Borntraeger
2022-01-19 18:28 ` Christian Borntraeger
2022-01-19 19:22 ` Mark Rutland
2022-01-19 19:30 ` Christian Borntraeger [this message]
2022-01-20 11:57 ` Mark Rutland
2022-01-20 12:02 ` Christian Borntraeger
2022-01-20 11:28 ` Paolo Bonzini
2022-01-20 12:03 ` Mark Rutland
2022-01-20 15:14 ` Christian Borntraeger
2022-01-21 9:53 ` Christian Borntraeger
2022-01-21 14:17 ` Christian Borntraeger
2022-01-21 14:30 ` Mark Rutland
2022-01-21 14:42 ` Christian Borntraeger
2022-01-21 15:29 ` Mark Rutland
2022-01-21 15:40 ` Christian Borntraeger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0654e667-1cfa-5147-6661-b3b63288be0b@linux.ibm.com \
--to=borntraeger@linux.ibm.com \
--cc=aleksandar.qemu.devel@gmail.com \
--cc=alexandru.elisei@arm.com \
--cc=anup.patel@wdc.com \
--cc=anup@brainfault.org \
--cc=aou@eecs.berkeley.edu \
--cc=atish.patra@wdc.com \
--cc=atishp@atishpatra.org \
--cc=bp@alien8.de \
--cc=catalin.marinas@arm.com \
--cc=chenhuacai@kernel.org \
--cc=dave.hansen@linux.intel.com \
--cc=frankja@linux.ibm.com \
--cc=frederic@kernel.org \
--cc=gor@linux.ibm.com \
--cc=hca@linux.ibm.com \
--cc=james.morse@arm.com \
--cc=jmattson@google.com \
--cc=joro@8bytes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mark.rutland@arm.com \
--cc=maz@kernel.org \
--cc=mingo@redhat.com \
--cc=mpe@ellerman.id.au \
--cc=nsaenzju@redhat.com \
--cc=palmer@dabbelt.com \
--cc=paul.walmsley@sifive.com \
--cc=paulmck@kernel.org \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=seanjc@google.com \
--cc=suzuki.poulose@arm.com \
--cc=svens@linux.ibm.com \
--cc=tglx@linutronix.de \
--cc=tsbogend@alpha.franken.de \
--cc=vkuznets@redhat.com \
--cc=wanpengli@tencent.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Powered by JetHome