From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35C80C77B61 for ; Thu, 27 Apr 2023 08:57:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243225AbjD0I51 (ORCPT ); Thu, 27 Apr 2023 04:57:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56456 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243145AbjD0I5T (ORCPT ); Thu, 27 Apr 2023 04:57:19 -0400 Received: from out30-113.freemail.mail.aliyun.com (out30-113.freemail.mail.aliyun.com [115.124.30.113]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21C374EE4; Thu, 27 Apr 2023 01:57:17 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R951e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045170;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=10;SR=0;TI=SMTPD_---0Vh709jd_1682585833; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0Vh709jd_1682585833) by smtp.aliyun-inc.com; Thu, 27 Apr 2023 16:57:14 +0800 Message-ID: <1682585517.595783-3-xuanzhuo@linux.alibaba.com> Subject: Re: [PATCH] virtio_net: suppress cpu stall when free_unused_bufs Date: Thu, 27 Apr 2023 16:51:57 +0800 From: Xuan Zhuo To: Wenliang Wang Cc: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, jasowang@redhat.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, "Michael S. Tsirkin" References: <20230427043433.2594960-1-wangwenliang.1995@bytedance.com> <1682576442.2203932-1-xuanzhuo@linux.alibaba.com> <252ee222-f918-426e-68ef-b3710a60662e@bytedance.com> <1682579624.5395834-1-xuanzhuo@linux.alibaba.com> <20230427041206-mutt-send-email-mst@kernel.org> <1682583225.3180113-2-xuanzhuo@linux.alibaba.com> <20230427042259-mutt-send-email-mst@kernel.org> In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 27 Apr 2023 16:49:58 +0800, Wenliang Wang wrote: > On 4/27/23 4:23 PM, Michael S. Tsirkin wrote: > > On Thu, Apr 27, 2023 at 04:13:45PM +0800, Xuan Zhuo wrote: > >> On Thu, 27 Apr 2023 04:12:44 -0400, "Michael S. Tsirkin" wrote: > >>> On Thu, Apr 27, 2023 at 03:13:44PM +0800, Xuan Zhuo wrote: > >>>> On Thu, 27 Apr 2023 15:02:26 +0800, Wenliang Wang wrote: > >>>>> > >>>>> > >>>>> On 4/27/23 2:20 PM, Xuan Zhuo wrote: > >>>>>> On Thu, 27 Apr 2023 12:34:33 +0800, Wenliang Wang wrote: > >>>>>>> For multi-queue and large rx-ring-size use case, the following error > >>>>>> > >>>>>> Cound you give we one number for example? > >>>>> > >>>>> 128 queues and 16K queue_size is typical. > >>>>> > >>>>>> > >>>>>>> occurred when free_unused_bufs: > >>>>>>> rcu: INFO: rcu_sched self-detected stall on CPU. > >>>>>>> > >>>>>>> Signed-off-by: Wenliang Wang > >>>>>>> --- > >>>>>>> drivers/net/virtio_net.c | 1 + > >>>>>>> 1 file changed, 1 insertion(+) > >>>>>>> > >>>>>>> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > >>>>>>> index ea1bd4bb326d..21d8382fd2c7 100644 > >>>>>>> --- a/drivers/net/virtio_net.c > >>>>>>> +++ b/drivers/net/virtio_net.c > >>>>>>> @@ -3565,6 +3565,7 @@ static void free_unused_bufs(struct virtnet_info *vi) > >>>>>>> struct virtqueue *vq = vi->rq[i].vq; > >>>>>>> while ((buf = virtqueue_detach_unused_buf(vq)) != NULL) > >>>>>>> virtnet_rq_free_unused_buf(vq, buf); > >>>>>>> + schedule(); > >>>>>> > >>>>>> Just for rq? > >>>>>> > >>>>>> Do we need to do the same thing for sq? > >>>>> Rq buffers are pre-allocated, take seconds to free rq unused buffers. > >>>>> > >>>>> Sq unused buffers are much less, so do the same for sq is optional. > >>>> > >>>> I got. > >>>> > >>>> I think we should look for a way, compatible with the less queues or the smaller > >>>> rings. Calling schedule() directly may be not a good way. > >>>> > >>>> Thanks. > >>> > >>> Why isn't it a good way? > >> > >> For the small ring, I don't think it is a good way, maybe we only deal with one > >> buf, then call schedule(). > >> > >> We can call the schedule() after processing a certain number of buffers, > >> or check need_resched () first. > >> > >> Thanks. > > > > > > Wenliang, does > > if (need_resched()) > > schedule(); > > fix the issue for you? > > > Yeah, it works better. I prefer to use it in combination with a fixed number(such as 256). Every time 256 buffers are processed, check need_resched(). This can accommodate large rings and small rings. Also, it is necessary to add similar logic to sq. Although the possibility is low, it is possible that the same problem will occur. Thanks. > > > >> > >> > >>> > >>>> > >>>>> > >>>>>> > >>>>>> Thanks. > >>>>>> > >>>>>> > >>>>>>> } > >>>>>>> } > >>>>>>> > >>>>>>> -- > >>>>>>> 2.20.1 > >>>>>>> > >>> > >