From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D75B2C7EE22 for ; Wed, 10 May 2023 05:01:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235750AbjEJFBP (ORCPT ); Wed, 10 May 2023 01:01:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54368 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233120AbjEJFBK (ORCPT ); Wed, 10 May 2023 01:01:10 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D553840F6 for ; Tue, 9 May 2023 22:00:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1683694821; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XeGHjCbz8dSGYd3TX1liaACMeHk61OyZyX7cZcDFTH4=; b=PEF9gLl+qc2ztQLWEa4LgsE1N1cQaWQjUVNMisaL2wBttw93J2y0iVVevPnNYzRwdUXWO9 P/Vl1hqlGUiZwGJ0G0J8d+hY+7iLDfndP0ZcCWtPHJTa2J9jhde23XV3EBUKjFqWX3p1Wy uKylZKtRkwRnam2nUnrxW2ioXAoHvV4= Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-113-XPCkM74DPWyNhIWjwRyfgw-1; Wed, 10 May 2023 01:00:20 -0400 X-MC-Unique: XPCkM74DPWyNhIWjwRyfgw-1 Received: by mail-pg1-f200.google.com with SMTP id 41be03b00d2f7-521262a6680so6164561a12.1 for ; Tue, 09 May 2023 22:00:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683694819; x=1686286819; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XeGHjCbz8dSGYd3TX1liaACMeHk61OyZyX7cZcDFTH4=; b=RFQl2hSC0ZuLHPU5osuOaa1sNES+MZnZ48qzcCwdPRbHYVV8HtN1zZAoCvdPhXQ/31 SCyz8K3GFmxOMYxaBb7XPIwMMtJNor/XXWJeKYSMtL5OErk1msmr0SJCkKFOuuQ+1gXe 27AHpGYGF0plNgNTQfcTKiTWNFWo8+v2VMi++ptYABCyIovoAA7W7vlTV0rFmviqnLGD PRoRW4GPjJz0f20D6XCN4zP4hr4CCceIzv7p/uPcYwvblsxKdI+J4TCE21ObThCjoLay BhGfonxSVRKAsTmjk3fQ3ea2I26kRdppvkRpfRoEWgp92qy1FwGgzJo28LtxdnBjdsVN S4uQ== X-Gm-Message-State: AC+VfDwDYvYzsoNJ8A10D0NsewdHc07OG3E23oUj06KLU7J2vQu+b+oq AnZtI0QJ6XcF0OriZGAeuWD6a/MBInKhUR7K/b/MDwz4I52FyddOulgzWLAq8Sf3D887LpDxgLd +JGlRGLV7tkUdJegnaQRyZLlj X-Received: by 2002:a05:6a20:549e:b0:100:4369:164a with SMTP id i30-20020a056a20549e00b001004369164amr14030939pzk.46.1683694819295; Tue, 09 May 2023 22:00:19 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5ebCTud1rBz1QnWQJHfKRmNeCZ/yR49zeE232z167CAyNmOh2ig/ETAvy8858079sOcnIVIQ== X-Received: by 2002:a05:6a20:549e:b0:100:4369:164a with SMTP id i30-20020a056a20549e00b001004369164amr14030897pzk.46.1683694818843; Tue, 09 May 2023 22:00:18 -0700 (PDT) Received: from [10.72.13.243] ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id w12-20020aa7858c000000b0064867dc8719sm188015pfn.118.2023.05.09.22.00.14 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 09 May 2023 22:00:18 -0700 (PDT) Message-ID: Date: Wed, 10 May 2023 13:00:08 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: [PATCH net v3] virtio_net: Fix error unwinding of XDP initialization To: Xuan Zhuo , Feng Liu Cc: "Michael S . Tsirkin" , Simon Horman , Bodong Wang , William Tu , Parav Pandit , virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org References: <20230503003525.48590-1-feliu@nvidia.com> <1683340417.612963-3-xuanzhuo@linux.alibaba.com> <559ad341-2278-5fad-6805-c7f632e9894e@nvidia.com> <1683510351.569717-1-xuanzhuo@linux.alibaba.com> <1683596602.483001-1-xuanzhuo@linux.alibaba.com> Content-Language: en-US From: Jason Wang In-Reply-To: <1683596602.483001-1-xuanzhuo@linux.alibaba.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 在 2023/5/9 09:43, Xuan Zhuo 写道: > On Mon, 8 May 2023 11:00:10 -0400, Feng Liu wrote: >> >> On 2023-05-07 p.m.9:45, Xuan Zhuo wrote: >>> External email: Use caution opening links or attachments >>> >>> >>> On Sat, 6 May 2023 08:08:02 -0400, Feng Liu wrote: >>>> >>>> On 2023-05-05 p.m.10:33, Xuan Zhuo wrote: >>>>> External email: Use caution opening links or attachments >>>>> >>>>> >>>>> On Tue, 2 May 2023 20:35:25 -0400, Feng Liu wrote: >>>>>> When initializing XDP in virtnet_open(), some rq xdp initialization >>>>>> may hit an error causing net device open failed. However, previous >>>>>> rqs have already initialized XDP and enabled NAPI, which is not the >>>>>> expected behavior. Need to roll back the previous rq initialization >>>>>> to avoid leaks in error unwinding of init code. >>>>>> >>>>>> Also extract a helper function of disable queue pairs, and use newly >>>>>> introduced helper function in error unwinding and virtnet_close; >>>>>> >>>>>> Issue: 3383038 >>>>>> Fixes: 754b8a21a96d ("virtio_net: setup xdp_rxq_info") >>>>>> Signed-off-by: Feng Liu >>>>>> Reviewed-by: William Tu >>>>>> Reviewed-by: Parav Pandit >>>>>> Reviewed-by: Simon Horman >>>>>> Acked-by: Michael S. Tsirkin >>>>>> Change-Id: Ib4c6a97cb7b837cfa484c593dd43a435c47ea68f >>>>>> --- >>>>>> drivers/net/virtio_net.c | 30 ++++++++++++++++++++---------- >>>>>> 1 file changed, 20 insertions(+), 10 deletions(-) >>>>>> >>>>>> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c >>>>>> index 8d8038538fc4..3737cf120cb7 100644 >>>>>> --- a/drivers/net/virtio_net.c >>>>>> +++ b/drivers/net/virtio_net.c >>>>>> @@ -1868,6 +1868,13 @@ static int virtnet_poll(struct napi_struct *napi, int budget) >>>>>> return received; >>>>>> } >>>>>> >>>>>> +static void virtnet_disable_qp(struct virtnet_info *vi, int qp_index) >>>>>> +{ >>>>>> + virtnet_napi_tx_disable(&vi->sq[qp_index].napi); >>>>>> + napi_disable(&vi->rq[qp_index].napi); >>>>>> + xdp_rxq_info_unreg(&vi->rq[qp_index].xdp_rxq); >>>>>> +} >>>>>> + >>>>>> static int virtnet_open(struct net_device *dev) >>>>>> { >>>>>> struct virtnet_info *vi = netdev_priv(dev); >>>>>> @@ -1883,20 +1890,26 @@ static int virtnet_open(struct net_device *dev) >>>>>> >>>>>> err = xdp_rxq_info_reg(&vi->rq[i].xdp_rxq, dev, i, vi->rq[i].napi.napi_id); >>>>>> if (err < 0) >>>>>> - return err; >>>>>> + goto err_xdp_info_reg; >>>>>> >>>>>> err = xdp_rxq_info_reg_mem_model(&vi->rq[i].xdp_rxq, >>>>>> MEM_TYPE_PAGE_SHARED, NULL); >>>>>> - if (err < 0) { >>>>>> - xdp_rxq_info_unreg(&vi->rq[i].xdp_rxq); >>>>>> - return err; >>>>>> - } >>>>>> + if (err < 0) >>>>>> + goto err_xdp_reg_mem_model; >>>>>> >>>>>> virtnet_napi_enable(vi->rq[i].vq, &vi->rq[i].napi); >>>>>> virtnet_napi_tx_enable(vi, vi->sq[i].vq, &vi->sq[i].napi); >>>>>> } >>>>>> >>>>>> return 0; >>>>>> + >>>>>> +err_xdp_reg_mem_model: >>>>>> + xdp_rxq_info_unreg(&vi->rq[i].xdp_rxq); >>>>>> +err_xdp_info_reg: >>>>>> + for (i = i - 1; i >= 0; i--) >>>>>> + virtnet_disable_qp(vi, i); >>>>> >>>>> I would to know should we handle for these: >>>>> >>>>> disable_delayed_refill(vi); >>>>> cancel_delayed_work_sync(&vi->refill); >>>>> >>>>> >>>>> Maybe we should call virtnet_close() with "i" directly. >>>>> >>>>> Thanks. >>>>> >>>>> >>>> Can’t use i directly here, because if xdp_rxq_info_reg fails, napi has >>>> not been enabled for current qp yet, I should roll back from the queue >>>> pairs where napi was enabled before(i--), otherwise it will hang at napi >>>> disable api >>> This is not the point, the key is whether we should handle with: >>> >>> disable_delayed_refill(vi); >>> cancel_delayed_work_sync(&vi->refill); >>> >>> Thanks. >>> >>> >> OK, get the point. Thanks for your careful review. And I check the code >> again. >> >> There are two points that I need to explain: >> >> 1. All refill delay work calls(vi->refill, vi->refill_enabled) are based >> on that the virtio interface is successfully opened, such as >> virtnet_receive, virtnet_rx_resize, _virtnet_set_queues, etc. If there >> is an error in the xdp reg here, it will not trigger these subsequent >> functions. There is no need to call disable_delayed_refill() and >> cancel_delayed_work_sync(). > Maybe something is wrong. I think these lines may call delay work. > > static int virtnet_open(struct net_device *dev) > { > struct virtnet_info *vi = netdev_priv(dev); > int i, err; > > enable_delayed_refill(vi); > > for (i = 0; i < vi->max_queue_pairs; i++) { > if (i < vi->curr_queue_pairs) > /* Make sure we have some buffers: if oom use wq. */ > --> if (!try_fill_recv(vi, &vi->rq[i], GFP_KERNEL)) > --> schedule_delayed_work(&vi->refill, 0); > > err = xdp_rxq_info_reg(&vi->rq[i].xdp_rxq, dev, i, vi->rq[i].napi.napi_id); > if (err < 0) > return err; > > err = xdp_rxq_info_reg_mem_model(&vi->rq[i].xdp_rxq, > MEM_TYPE_PAGE_SHARED, NULL); > if (err < 0) { > xdp_rxq_info_unreg(&vi->rq[i].xdp_rxq); > return err; > } > > virtnet_napi_enable(vi->rq[i].vq, &vi->rq[i].napi); > virtnet_napi_tx_enable(vi, vi->sq[i].vq, &vi->sq[i].napi); > } > > return 0; > } > > > And I think, if we virtnet_open() return error, then the status of virtnet > should like the status after virtnet_close(). > > Or someone has other opinion. I agree, we need to disable and sync with the refill work. Thanks > > Thanks. > >> The logic here is different from that of >> virtnet_close. virtnet_close is based on the success of virtnet_open and >> the tx and rx has been carried out normally. For error unwinding, only >> disable qp is needed. Also encapuslated a helper function of disable qp, >> which is used ing error unwinding and virtnet close >> 2. The current error qp, which has not enabled NAPI, can only call xdp >> unreg, and cannot call the interface of disable NAPI, otherwise the >> kernel will be stuck. So for i-- the reason for calling disable qp on >> the previous queue >> >> Thanks >> >>>>>> + >>>>>> + return err; >>>>>> } >>>>>> >>>>>> static int virtnet_poll_tx(struct napi_struct *napi, int budget) >>>>>> @@ -2305,11 +2318,8 @@ static int virtnet_close(struct net_device *dev) >>>>>> /* Make sure refill_work doesn't re-enable napi! */ >>>>>> cancel_delayed_work_sync(&vi->refill); >>>>>> >>>>>> - for (i = 0; i < vi->max_queue_pairs; i++) { >>>>>> - virtnet_napi_tx_disable(&vi->sq[i].napi); >>>>>> - napi_disable(&vi->rq[i].napi); >>>>>> - xdp_rxq_info_unreg(&vi->rq[i].xdp_rxq); >>>>>> - } >>>>>> + for (i = 0; i < vi->max_queue_pairs; i++) >>>>>> + virtnet_disable_qp(vi, i); >>>>>> >>>>>> return 0; >>>>>> } >>>>>> -- >>>>>> 2.37.1 (Apple Git-137.1) >>>>>>