From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754888AbdEKCrz (ORCPT ); Wed, 10 May 2017 22:47:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60574 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753996AbdEKCry (ORCPT ); Wed, 10 May 2017 22:47:54 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com B4E9DC057FAD Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx08.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=jasowang@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com B4E9DC057FAD Subject: Re: [PATCH net-next V4 10/10] vhost_net: try batch dequing from skb array To: "Michael S. Tsirkin" References: <1494387382-19916-1-git-send-email-jasowang@redhat.com> <1494387382-19916-11-git-send-email-jasowang@redhat.com> <20170510152848-mutt-send-email-mst@kernel.org> Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org From: Jason Wang Message-ID: <6ec5ce73-793b-56b8-51a8-fe119581a874@redhat.com> Date: Thu, 11 May 2017 10:47:47 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20170510152848-mutt-send-email-mst@kernel.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Thu, 11 May 2017 02:47:53 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2017年05月10日 20:34, Michael S. Tsirkin wrote: > On Wed, May 10, 2017 at 11:36:22AM +0800, Jason Wang wrote: >> We used to dequeue one skb during recvmsg() from skb_array, this could >> be inefficient because of the bad cache utilization and spinlock >> touching for each packet. This patch tries to batch them by calling >> batch dequeuing helpers explicitly on the exported skb array and pass >> the skb back through msg_control for underlayer socket to finish the >> userspace copying. >> >> Batch dequeuing is also the requirement for more batching improvement >> on rx. >> >> Tests were done by pktgen on tap with XDP1 in guest on top of batch >> zeroing: >> >> rx batch | pps >> >> 256 2.41Mpps (+6.16%) >> 128 2.48Mpps (+8.80%) >> 64 2.38Mpps (+3.96%) <- Default >> 16 2.31Mpps (+1.76%) >> 4 2.31Mpps (+1.76%) >> 1 2.30Mpps (+1.32%) >> 0 2.27Mpps (+7.48%) >> >> Signed-off-by: Jason Wang >> --- >> drivers/vhost/net.c | 117 +++++++++++++++++++++++++++++++++++++++++++++++++--- >> 1 file changed, 111 insertions(+), 6 deletions(-) >> >> diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c >> index 9b51989..fbaecf3 100644 >> --- a/drivers/vhost/net.c >> +++ b/drivers/vhost/net.c >> @@ -28,6 +28,8 @@ >> #include >> #include >> #include >> +#include >> +#include >> >> #include >> >> @@ -85,6 +87,13 @@ struct vhost_net_ubuf_ref { >> struct vhost_virtqueue *vq; >> }; >> >> +#define VHOST_RX_BATCH 64 >> +struct vhost_net_buf { >> + struct sk_buff *queue[VHOST_RX_BATCH]; >> + int tail; >> + int head; >> +}; >> + > Do you strictly need to put this inline? This structure is quite big > already. Do you see a measureabe difference if you make it > > struct sk_buff **queue; > int tail; > int head; > > ? I don't. > > Will also make it easier to play with the size in the future > should someone want to see how does it work e.g. for different > ring sizes. > Ok, will do this in next version Thanks