From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751787AbdLBMQM (ORCPT ); Sat, 2 Dec 2017 07:16:12 -0500 Received: from mout.kundenserver.de ([212.227.126.134]:55208 "EHLO mout.kundenserver.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751649AbdLBMQK (ORCPT ); Sat, 2 Dec 2017 07:16:10 -0500 In-Reply-To: <61be2e2b-9aeb-1a82-d607-a6af00f8c9c6@redhat.com> References: <92c4f997-80db-fabf-98c8-fcb92da064a7@redhat.com> <7bd45f84-d07e-7fca-6ca3-07dededd092d@redhat.com> <29f8e09f-8920-52d0-02f4-c0fb779135ee@redhat.com> <9c912f3b-081c-8b02-17c8-453ebf36f42c@redhat.com> <10fe2b98-1e26-9539-9f49-0d01f8693e04@redhat.com> <6b41b4e5-6c0c-fce6-21fe-02dd8f550095@redhat.com> <634116a6-6338-4249-7d2d-430b654cc99c@redhat.com> <1f789868-7fda-3553-7078-3298873fb355@redhat.com> <918c4152-bcf9-b28c-0f54-f51d07d82bfc@redhat.com> <68b5d4aa-1d48-d9a1-fc47-62ee8d7ad07a@redhat.com> <623df785-b79c-80d1-899f-6fcc10f70e69@redhat.com> <61be2e2b-9aeb-1a82-d607-a6af00f8c9c6@redhat.com> To: linux-kernel@vger.kernel.org From: Harald Moeller Subject: Re: Shutting down a VM with Kernel 4.14 will sometime hang and a reboot is the only way to recover. Message-ID: <1625c485-ef1d-1fd8-0d4e-99fd6bd631ea@hakimo.net> Date: Sat, 2 Dec 2017 13:16:06 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Provags-ID: V03:K0:f67QZydHOLpgCqyEgt8IYaiNgCyVBhoGGaE34CcGhSr9N6SAv+5 ++3yNSkD1qCiGRX/N8XHnGgICALMq5g0LwIViBxGjORSlBe7Kk4viBo00YlnyzhjUCc+gTL /HYPdaO6oaZ3i1n5TwuS68spFzn9X2tWCcLL/FYL3ocKv+ZcHIIwfP+HX97jl6eESgRT7AL Vfu/JUR71cD+Ib2ivrDZg== X-UI-Out-Filterresults: notjunk:1;V01:K0:K2baCQ72v3U=:rsX+r0YEUzsI+y34O/DawE END60WAm69oIpg3wjThvH4T3Tyr4humqhLO6LMcJdrgbMID/VvuJJsV0GnZQ7t3DIoiaQ6154 LISCoBR/Ltwnyb+jbPbTUbF652Rm3YNOJ1zJzUbdDn5cVxVvCm+8MBsuWzeq0ycusnPQGJ5FN VLerwf+rMt/v7Y5/ustW6cm6fWqzn3rbJxY7pI91iiSRtqSbjl3dNDnklds75L9/sKwcX1k/E 4Z5b5PIIl6wAFWz6H5k4h9nv1L781XObEoSP/TY/tAC5z1hgVcQaKl+T/iJEGNem98+cNGOfY A4rV6MmOhCy4CsSpEWcwI+A8oUfpIenuYxn/Em47CR3NXFMstptjxtCiJweNphbXJC4l+uY76 wieuhxJ6LpubsDWpQEd28EKYj46BMWaKLmIf0VGdGMFZZfSqj0joMMwtm+vty329s3sLI18g9 kLfJApcJbK58My53b9661qw0mlPfJFWYRlswN59vKJJbeGtr8KuMilLG06hRlv2TKtjeuYrMh WVLvR6Hj2G1D0f+9QVAaoi+Nk+Kbto8na/kl4Nz5qIXYd7/TceyKeLigqPBNch9WYklZjiJV+ CumKZzZ9rr77RrXnECUfnncNFJUDd79KDWOyqV03MEHsP9Kx+NO8xCbrHLz6I7vqgDsJpK4WX 9fN5LJVYv+dF5MpByhfs7CCv9obMsN/mPxHRXmAzxPzQYc5ptJx5kqz6E/lk7JCvf/k4Uufak awXsf4fHQhzopnUk2bALKRiwgZzHSm79TTW0YZT7WZDDaoB/MrdJbkaWers= Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello, my name is Harry and this is my first post here, hope I'm doing this the right way, sorry if not ... I'm not a subscriber to the full list yet so I understand I shall ask you to be personally CCed. I am following this as I do experience the same (or sort-a same) issue with 4.14.2. My setup is more simple, just an oVirt host shutting down some VMs. Doesn't happen all the time but I'd say around 3 from 10. This is what I see (slightly different from David): Dec 01 16:11:53 oVirtHost01.xyz.net kernel: INFO: task qemu-kvm:1173 blocked for more than 120 seconds. Dec 01 16:11:53 oVirtHost01.xyz.net kernel:       Tainted: G          I     4.14.2-1.el7.hakimo.x86_64 #4 Dec 01 16:11:53 oVirtHost01.xyz.net kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Dec 01 16:11:53 oVirtHost01.xyz.net kernel: qemu-kvm        D 0  1173      1 0x00000084 Dec 01 16:11:53 oVirtHost01.xyz.net kernel: Call Trace: Dec 01 16:11:53 oVirtHost01.xyz.net kernel: __schedule+0x28d/0x880 Dec 01 16:11:53 oVirtHost01.xyz.net kernel:  schedule+0x36/0x80 Dec 01 16:11:53 oVirtHost01.xyz.net kernel: vhost_net_ubuf_put_and_wait+0x61/0x90 [vhost_net] Dec 01 16:11:53 oVirtHost01.xyz.net kernel:  ? remove_wait_queue+0x60/0x60 Dec 01 16:11:53 oVirtHost01.xyz.net kernel: vhost_net_ioctl+0x317/0x8e0 [vhost_net] Dec 01 16:11:53 oVirtHost01.xyz.net kernel: do_vfs_ioctl+0xa7/0x5f0 Dec 01 16:11:53 oVirtHost01.xyz.net kernel:  SyS_ioctl+0x79/0x90 Dec 01 16:11:53 oVirtHost01.xyz.net kernel: do_syscall_64+0x67/0x1b0 Dec 01 16:11:53 oVirtHost01.xyz.net kernel: entry_SYSCALL64_slow_path+0x25/0x25 Dec 01 16:11:53 oVirtHost01.xyz.net kernel: RIP: 0033:0x7fb8862d1107 Dec 01 16:11:53 oVirtHost01.xyz.net kernel: RSP: 002b:00007fff4acd7e58 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 Dec 01 16:11:53 oVirtHost01.xyz.net kernel: RAX: ffffffffffffffda RBX: 000055abaa2d29c0 RCX: 00007fb8862d1107 Dec 01 16:11:53 oVirtHost01.xyz.net kernel: RDX: 00007fff4acd7e60 RSI: 000000004008af30 RDI: 0000000000000028 Dec 01 16:11:53 oVirtHost01.xyz.net kernel: RBP: 00007fff4acd7e60 R08: 000055aba805e10f R09: 00000000ffffffff Dec 01 16:11:53 oVirtHost01.xyz.net kernel: R10: 0000000000000004 R11: 0000000000000246 R12: 000055ababf32510 Dec 01 16:11:53 oVirtHost01.xyz.net kernel: R13: 0000000000000001 R14: 000055ababf32498 R15: 000055abaa2a0b40 This is still happening after reverting the three suggested commits 1f8b977ab32dc5d148f103326e80d9097f1cefb5 ("sock: enable MSG_ZEROCOPY") c1d1b437816f0afa99202be3cb650c9d174667bc ("net: convert (struct ubuf_info)->refcnt to refcount_t") 581fe0ea61584d88072527ae9fb9dcb9d1f2783e {"net: orphan frags on stand-alone ptype in dev_queue_xmit_nit"} Anything I could be helpful with trying to solve this? Any more info I could provide? Harry