From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EADC51531C8 for ; Tue, 24 Mar 2026 00:34:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=170.10.133.124 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774312489; cv=pass; b=mEtp09LQLdodx+hiHCUUprFP0/HxoVP25xec7iLvfhWhJfX7rAOLWULtKIHCp/pMBIwODU0hO7M/++mPFUmlmHTu9L4pcj9rVFfB9iFwa1fbNpq8hAN8YiQNlF4kD8PjpzzDHze3CFCxR/vCnYWkSqTMB5aCUVioEmY/Jxqu3vM= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774312489; c=relaxed/simple; bh=NmX6aq1AjJs12Yq+PWPzbIZnkkSYYGFdr1l5n18R484=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=lDvdptv29zNyexQfvEdRGWVXx0Gr5X7oojihKluFZJ2mApSLNmsaN8QeknbUKaGKDVKDz3pn0dvZPNEDi4sDLCYAav8VwifWzSQ8csxXDvU+unCjfbckK9u6jwRIxI4RfXpEFYq4R8w2qaRGExwd9EX03b4JRj2J7oqAY7eRbKs= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=cjTf+Ms9; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=DrNSQsUj; arc=pass smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cjTf+Ms9"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="DrNSQsUj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1774312484; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=saGQT6VbC9RUqD0YOp9yO8+1VRHnufK9WLW/Cpl18Rg=; b=cjTf+Ms9DNQVzc8WNXlhJ5bNIGklxwJIQE4odXiXNHt6v7Cfq7hgIii2juWOpoQjWY0woY 4qEPkRA7oVdahUaXPeiAUh8nCxxIZ0VhdVNErqRhfrpMdryTFSaObGVGsBb9AO0uLFtws0 WHSo8H8zExNm/7KCmOU8dK4wBaoPTgQ= Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-458-77n5iv6IM9-CKuozXRv1RA-1; Mon, 23 Mar 2026 20:34:42 -0400 X-MC-Unique: 77n5iv6IM9-CKuozXRv1RA-1 X-Mimecast-MFC-AGG-ID: 77n5iv6IM9-CKuozXRv1RA_1774312481 Received: by mail-pl1-f199.google.com with SMTP id d9443c01a7336-2b079b4a8c3so5185805ad.3 for ; Mon, 23 Mar 2026 17:34:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1774312481; cv=none; d=google.com; s=arc-20240605; b=IM87YHGvEALGmhl0PF5g3gBk8Vvs3mzvEgvb/duyiahP/Vk5lMcLiAeb8xB70rQ9FG 5Nse9yzyC5yye6yJrz2MPAvGhoh7e8rjZ4DQqJ72KX4NfXAInKfU9N3DaLX4shp248FI AU5TRk9RZgCXuJOHBfyuvtclhIcGcUo4jaQM86NNgohuzxpmiYgKMIPDw/AKu7C/xBaf B6FhizQ/AC8ybNnVoSGCVQvFbrrOWtU5e7DOzapV644MqvTsdkAa0lfa9c2Z6VpBWLRA ArTmPOY57Txyab0fGZTkAo8MOBdEh1IIfoPE1no8XGfLScW76KLlFPbJLVnd43xPqtBL BNMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=saGQT6VbC9RUqD0YOp9yO8+1VRHnufK9WLW/Cpl18Rg=; fh=/Qh4yKyO7CLhsnWnCX/qkDF2FzwCl+Ezjm97rU8mXy0=; b=VIwXCFOBW1lgNO1gD7szz9HTUl5CDiNLmxL/Mn4q43KRe7FtZrpBVC5fK4/WAYg/DW fijIUwbkL/qD13NXkY22M6qF2yQPAXD1MS/mxKd2atcicoSdm+NrBf7/Wbm2V2+q2nb+ 65vOIYgZJMjY2dUZ8WoHqubKQ8puh0+cVZpwqZIdiPr7/ZcOsDFs/KCJ/rfDCtqEk8et wrHzUAGiyKDT2Mqx3PumanCYFBBzLS14zu5ghqVuAChCMzwhgk5XIYGTeweCKHVQ6HHK dJzIf2nwjeq01hPljs87ZNodzgTtancu5B6os59L3Wdgb4xxO5GibEImn681485WjFXI 43MA==; darn=vger.kernel.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1774312481; x=1774917281; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=saGQT6VbC9RUqD0YOp9yO8+1VRHnufK9WLW/Cpl18Rg=; b=DrNSQsUjIgj127OfK5LhRPhFAr2xw7XFfmRt+y3+TTbb6bRpWhYd08bFSxbUeeWNv6 mzaxuJw4s0NKBvX5UV6DRXxnpglrCpOKq4oSyKB+RFHh6B4iU8FPAPnkny1muZpN0jbt 1VGSJmz/s6xJGYUGCOeyuqTgVH2LtrM762UJCx26g3OQyQu2thTxrHXAUon07p+AvnWA BJAbykCFhzhWlGyTnisvIg2JYVxXe0dkZQo4w1LIo0Q63BTYJMMZM8HXwt6ypCyf+77P bV7gL2BQ6v1jXtzxns8IPsZJe+K6AuGoK5bFk0a+KNUrCAzgoti3dxCHn67UZCN7RJhy GMEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774312481; x=1774917281; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=saGQT6VbC9RUqD0YOp9yO8+1VRHnufK9WLW/Cpl18Rg=; b=d+XQ0VRSmHhDHfNg05OOUI7JDhvj4D8+xqu8NjRsuiV2kpOF8ZY1UIvTlBdZQFZ0B0 Ws/mrmKZAj9cMhHaPN9ecTLuLwUfCAdK4xo0nveS80Zi6X7azN0EaVrgQRQdQVE/sZJl lhYL88hmR/mBa0NuE0M014XpmIfOB6/tdWhSAJzKiaa7UMoQh7D2gNZD4vWm62O0cKIB w0n8Uzj4io5CxVpFnmi8zJ6mTymuytCBCbcYuEgAqp+CSONaZnAqFGtjVO3EKTZl8Q2r SCvRBgA6qqenkVhZ5NItZM1FAHUil9krvtIeOqioIRyZzpEYGT7ohZTqRjSbfZfsQrnY etOQ== X-Forwarded-Encrypted: i=1; AJvYcCWl2WI7GWidDwtTOvL8hx7ns6s/Jpeplx2Vcxk6+OGGMIOS+a+sbuD2/TDi6ENA21pL2yA6gCJiQs9/Yvs=@vger.kernel.org X-Gm-Message-State: AOJu0YwGdyxUS/DYkOfCk8TVkQJRrHA2YskYC5X6f9MRTjHQZcbeUB6C wcrpsGuhSGDhWJnNGZL2V83uSv+2hHvNIODPGyfOFKcQSRqDiIi7TNGJNWfy4wE7bS4MDxwsJ+F G9vHOeoRh8CGLJ8FzDaoq+OeAGOO0Du4oLLv4jHnmVGJmfFa4iEsMZDcFAJpl37DSAcwNZooP6W Sd+anwYsDQrLlCD7htHo4B9JAhcDOAVsX/x0o5olZm X-Gm-Gg: ATEYQzz3RDd4gzaJfBR/NNm9+IUOp6mh+NjxkuK9dZSIMR1kvdF0CyQzmPbbe70TsUH yXugZ1oh1jiPoDQ/DCGSj+iZg9WVZcRVrKZH+Bm7LkB9lhxVT6s5pu8v+nKBGTV8jrL7/lzqVf0 EfnzxTnqGdoR0nvlphBEppcs7kkHP2GSvH5/yuE9PoaTTGk8AYQiRorZxy83Y0DBIpI0CmfnqMB PNF X-Received: by 2002:a17:902:d4c5:b0:2ad:dfb8:8ed3 with SMTP id d9443c01a7336-2b0826d7570mr125521025ad.8.1774312481445; Mon, 23 Mar 2026 17:34:41 -0700 (PDT) X-Received: by 2002:a17:902:d4c5:b0:2ad:dfb8:8ed3 with SMTP id d9443c01a7336-2b0826d7570mr125520745ad.8.1774312481005; Mon, 23 Mar 2026 17:34:41 -0700 (PDT) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <20260310183107.2822016-1-vishs@meta.com> <20260323150136.14452-1-oelghoul@linux.ibm.com> <20260323114313-mutt-send-email-mst@kernel.org> <20260323122728-mutt-send-email-mst@kernel.org> In-Reply-To: <20260323122728-mutt-send-email-mst@kernel.org> From: Jason Wang Date: Tue, 24 Mar 2026 08:34:29 +0800 X-Gm-Features: AaiRm52xN2lo_tVFI0DDvUo4ivVlfAirVP3edMWUgEftkLvUyxZ5RHdMesVost0 Message-ID: Subject: Re: [PATCH net-next v11] virtio_net: add page_pool support for buffer allocation To: "Michael S. Tsirkin" Cc: Omar Elghoul , vishs@meta.com, andrew+netdev@lunn.ch, davem@davemloft.net, dw@davidwei.uk, edumazet@google.com, eperezma@redhat.com, ilias.apalodimas@linaro.org, kernel-team@meta.com, kuba@kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, pabeni@redhat.com, technoboy85@gmail.com, virtualization@lists.linux.dev, xuanzhuo@linux.alibaba.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Mar 24, 2026 at 12:58=E2=80=AFAM Michael S. Tsirkin wrote: > > On Mon, Mar 23, 2026 at 11:52:34AM -0400, Michael S. Tsirkin wrote: > > On Mon, Mar 23, 2026 at 11:01:31AM -0400, Omar Elghoul wrote: > > > Hi, > > > > > > I've been testing linux-next (tags later than 03/17) and hit new issu= es in > > > virtio-net on s390x. I bisected the issue, and I found this patch to = be the > > > first buggy commit. > > > > > > The issue seems to only be reproducible when running in Secure Execut= ion. > > > Tested in a KVM guest, the virtio-net performance appears greatly red= uced, > > > and the dmesg output shows many instances of the following error mess= ages. > > > > > > Partial relevant logs > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > [ 49.332028] macvtap0: bad gso: type: 0, size: 0, flags 1 tunnel 0 = tnl csum 0 > > > [ 74.365668] macvtap0: bad gso: type: 2e, size: 27948, flags 0 tunn= el 0 tnl csum 0 > > > [ 403.302168] macvtap0: bad csum: flags: 2, gso_type: 23 rx_tnl_csum= 0 > > > [ 403.302271] macvtap0: bad csum: flags: 2, gso_type: e0 rx_tnl_csum= 0 > > > [ 403.302279] macvtap0: bad csum: flags: 2, gso_type: e1 rx_tnl_csum= 0 > > > [ 403.309492] macvtap0: bad csum: flags: 2, gso_type: 4c rx_tnl_csum= 0 > > > [ 403.317029] macvtap0: bad csum: flags: 2, gso_type: e0 rx_tnl_csum= 0 > > > > > > Steps to reproduce > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > 1. Boot a Linux guest implementing this patch under QEMU/KVM (*) with= SE > > > enabled and a virtio-net-ccw device attached. > > > 2. Run dmesg. The error message is usually already present at boot ti= me, > > > but if not, it can be reproduced by creating any network traffic. > > > > > > (*) This patch was not tested in a non-KVM hypervisor environment. > > > > > > I've further confirmed that reverting this patch onto its parent comm= it > > > resolves the issue. Please let me know if you'd like me to test a fix= or if > > > you would need more information. > > > > > > Thanks in advance. > > > > > > Best, > > > Omar > > > > Well... I am not sure how I missed it. Obvious in hindsight: > > > > static void receive_buf(struct virtnet_info *vi, struct receive_queue *= rq, > > void *buf, unsigned int len, void **ctx, > > unsigned int *xdp_xmit, > > struct virtnet_rq_stats *stats) > > { > > struct net_device *dev =3D vi->dev; > > struct sk_buff *skb; > > u8 flags; > > > > if (unlikely(len < vi->hdr_len + ETH_HLEN)) { > > pr_debug("%s: short packet %i\n", dev->name, len); > > DEV_STATS_INC(dev, rx_length_errors); > > virtnet_rq_free_buf(vi, rq, buf); > > return; > > } > > > > /* About the flags below: > > * 1. Save the flags early, as the XDP program might overwrite = them. > > * These flags ensure packets marked as VIRTIO_NET_HDR_F_DATA_V= ALID > > * stay valid after XDP processing. > > * 2. XDP doesn't work with partially checksummed packets (refe= r to > > * virtnet_xdp_set()), so packets marked as > > * VIRTIO_NET_HDR_F_NEEDS_CSUM get dropped during XDP processin= g. > > */ > > > > if (vi->mergeable_rx_bufs) { > > flags =3D ((struct virtio_net_common_hdr *)buf)->hdr.fl= ags; > > skb =3D receive_mergeable(dev, vi, rq, buf, ctx, len, x= dp_xmit, > > stats); > > } else if (vi->big_packets) { > > void *p =3D page_address((struct page *)buf); > > > > flags =3D ((struct virtio_net_common_hdr *)p)->hdr.flag= s; > > skb =3D receive_big(dev, vi, rq, buf, len, stats); > > } else { > > flags =3D ((struct virtio_net_common_hdr *)buf)->hdr.fl= ags; > > skb =3D receive_small(dev, vi, rq, buf, ctx, len, xdp_x= mit, stats); > > } > > > > > > So we are reading the header, before dma sync, which is within > > receive_mergeable and friends: > > > > static struct sk_buff *receive_mergeable(struct net_device *dev, > > struct virtnet_info *vi, > > struct receive_queue *rq, > > void *buf, > > void *ctx, > > unsigned int len, > > unsigned int *xdp_xmit, > > struct virtnet_rq_stats *stats= ) > > { > > struct virtio_net_hdr_mrg_rxbuf *hdr =3D buf; > > int num_buf =3D virtio16_to_cpu(vi->vdev, hdr->num_buffers); > > struct page *page =3D virt_to_head_page(buf); > > int offset =3D buf - page_address(page); > > struct sk_buff *head_skb, *curr_skb; > > unsigned int truesize =3D mergeable_ctx_to_truesize(ctx); > > unsigned int headroom =3D mergeable_ctx_to_headroom(ctx); > > > > head_skb =3D NULL; > > > > if (rq->use_page_pool_dma) > > page_pool_dma_sync_for_cpu(rq->page_pool, page, offset,= len); > > > > > > > > Just as a test, the below should fix it (compiled only), but the real > > fix is more complex since we need to be careful to avoid expensive sync= ing > > twice. > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > > index 97035b49bae7..57b4f5954bed 100644 > > --- a/drivers/net/virtio_net.c > > +++ b/drivers/net/virtio_net.c > > @@ -931,9 +931,19 @@ static struct sk_buff *page_to_skb(struct virtnet_= info *vi, > > > > static void *virtnet_rq_get_buf(struct receive_queue *rq, u32 *len, vo= id **ctx) > > { > > + void *buf; > > + > > BUG_ON(!rq->page_pool); > > > > - return virtqueue_get_buf_ctx(rq->vq, len, ctx); > > + buf =3D virtqueue_get_buf_ctx(rq->vq, len, ctx); > > + if (buf && rq->use_page_pool_dma && *len) { > > + struct page *page =3D virt_to_head_page(buf); > > + int offset =3D buf - page_address(page); > > + > > + page_pool_dma_sync_for_cpu(rq->page_pool, page, offset, *= len); > > + } > > + > > + return buf; > > } > > > > static void virtnet_rq_unmap_free_buf(struct virtqueue *vq, void *buf) > > > > > > > > > > -- > > MST > > or maybe like this: > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c > index 97035b49bae7..835f52651006 100644 > --- a/drivers/net/virtio_net.c > +++ b/drivers/net/virtio_net.c > @@ -1956,13 +1956,6 @@ static struct sk_buff *receive_small(struct net_de= vice *dev, > */ > buf -=3D VIRTNET_RX_PAD + xdp_headroom; > > - if (rq->use_page_pool_dma) { > - int offset =3D buf - page_address(page) + > - VIRTNET_RX_PAD + xdp_headroom; > - > - page_pool_dma_sync_for_cpu(rq->page_pool, page, offset, l= en); > - } > - > len -=3D vi->hdr_len; > u64_stats_add(&stats->bytes, len); > > @@ -2398,9 +2391,6 @@ static struct sk_buff *receive_mergeable(struct net= _device *dev, > > head_skb =3D NULL; > > - if (rq->use_page_pool_dma) > - page_pool_dma_sync_for_cpu(rq->page_pool, page, offset, l= en); > - > u64_stats_add(&stats->bytes, len - vi->hdr_len); > > if (check_mergeable_len(dev, ctx, len)) > @@ -2563,6 +2553,13 @@ static void receive_buf(struct virtnet_info *vi, s= truct receive_queue *rq, > return; > } > > + if (rq->use_page_pool_dma) { > + struct page *page =3D virt_to_head_page(buf); > + int offset =3D buf - page_address(page); > + > + page_pool_dma_sync_for_cpu(rq->page_pool, page, offset, l= en); > + } > + > /* About the flags below: > * 1. Save the flags early, as the XDP program might overwrite th= em. > * These flags ensure packets marked as VIRTIO_NET_HDR_F_DATA_VAL= ID > This looks better. Acked-by: Jason Wang Thanks